| wallyworld__ | jcastro: that bucket is his control bucket, created on demand at bootstrap time; it's not a public bucket which we could break | 00:45 |
|---|---|---|
| wallyworld__ | maybe ec2 was being particularly slow at that time - sadly stuff can take time to propagate through after an action is performed from what I understand | 00:47 |
| wallyworld__ | juju does wait for ec2 to "catch up"; maybe it didn't wait long enough this time. or maybe there was another error which went unnoticed in the output | 00:48 |
| sinzui | thumper-afk, that mysql on lxc bug is very erratic. I just deployed it twice on lxc successfully. My successful runs are between the hours of 0 an 6 UTC | 00:57 |
| adam_g | hmph | 01:03 |
| adam_g | trying to use 2 different juju environments in one maas (two different maas users/keys) | 01:04 |
| adam_g | http://paste.ubuntu.com/6338725/ | 01:04 |
| adam_g | one fails to start | 01:04 |
| === thumper-afk is now known as thumper | ||
| wallyworld__ | adam_g: 1.16 has a bug in this area - 1.16.2 is being packaged now and will allow maas to better support juju with multiple users. but i'm not sure if the symptoms you are seeing exactly relate to the problem as i've not run into the issue myself | 01:32 |
| bigjools | looks different to me | 01:32 |
| adam_g | wallyworld__, ssory, should have mentioned. im using 1.16.2 | 01:32 |
| thumper | adam_g: that's weird, not seen that before | 01:33 |
| wallyworld__ | adam_g: in that case, my maas foo is not good enough. maybe bigjools can help | 01:33 |
| adam_g | tried re-bootstrapping again, this time with only one environment. was the same thing | 01:33 |
| adam_g | trying again on another cluster | 01:33 |
| sodre | adam_g: what is exactly the issue you're facing? | 01:34 |
| bigjools | it doesn;t look anything to do with maas to me, there's a mongo connection error | 01:35 |
| sinzui | adam_g, tools are in canonistack and aws, your tests might be different now | 01:35 |
| adam_g | sodre, with two separate maas users, each with their own juju environment, only one bootstraps | 01:35 |
| adam_g | 2013-07-15 00:35:38 DEBUG juju.state open.go:88 connection failed, will retry: dial tcp 127.0.0.1:37017: connection refused | 01:35 |
| adam_g | 2013-07-15 00:35:38 ERROR juju.state open.go:93 TLS handshake failed: x509: certificate has expired or is not yet valid | 01:35 |
| sodre | is one of them *not admin* ? | 01:35 |
| adam_g | sodre, neither of them are | 01:36 |
| sodre | one sec. | 01:36 |
| adam_g | hmm. ive gotten further on another maas cluster (with the same versions of both juju and maas) | 01:37 |
| sodre | can you see if the workaround in #1245092 works for you ? | 01:38 |
| wallyworld__ | sodre: you can revert your rev 110 to remove the json fix using a cherry pick - bzr merge -r 110..109 | 01:38 |
| _mup_ | Bug #1245092: environ-provisioner fails to start <admin> <bootstrap> <Go MAAS API Library:Triaged> <https://launchpad.net/bugs/1245092> | 01:38 |
| adam_g | sodre, make both users admin? | 01:39 |
| sodre | wallyworld__: okay I'll do that. but at the moment I can't test with my live deployment without the json fix :( | 01:39 |
| sodre | adam_g: or at least one of them and see if that one bootstraps. | 01:40 |
| wallyworld__ | you can add the json fix when you tst but just not commit it | 01:40 |
| sinzui | We need to support sync or sum checking with gwacl to upload tools | 01:40 |
| adam_g | sodre, fwiw, i was able to bootstrap 2 environments with non-admin users on another cluster | 01:40 |
| wallyworld__ | sinzui: you saying tools upload is unreliable on Azure? | 01:41 |
| sodre | adam_g: okay... then maybe that is a different bug. | 01:41 |
| sinzui | wallyworld__, Iam saying I have been uploading tools for 20 minutes, most of them are already in azure. | 01:42 |
| wallyworld__ | ah right | 01:42 |
| sinzui | bigjools, Do to accidents in team organisation, I can create series in charms. Does your tarmac charm really need to be in saucy? Would you be willing to skip to trusty? | 01:42 |
| wallyworld__ | i noticed yesterday Azure is really, really sloooooooow | 01:43 |
| sinzui | wallyworld__, tools check sums will change we regenerated, so checksum is a requirement | 01:43 |
| wallyworld__ | yeah. we calculate checksums anyway, so makes sense to short circuit upload if possible | 01:44 |
| bigjools | sinzui: it is series-agnostic | 01:46 |
| bigjools | I pushed it to precise for now | 01:46 |
| thumper | axw: morning | 01:50 |
| axw | thumper: ahoy | 01:50 |
| thumper | axw: I'd like to point you at https://bugs.launchpad.net/juju-core/+bug/1246905 | 01:50 |
| _mup_ | Bug #1246905: status for manual provider hangs <manual-provider> <juju-core:Triaged> <https://launchpad.net/bugs/1246905> | 01:50 |
| axw | looking | 01:50 |
| thumper | axw: I've not dug into it any more than my comment says | 01:51 |
| axw | okey dokey. I'll dig in. thanks | 01:51 |
| axw | thumper: if you have a moment later, I replied to the thread about interfaces... keen to hear if you have any suggestions, as I know you went through something similar not too long ago | 01:52 |
| thumper | axw: heh, you mean that world of hurt? | 01:52 |
| axw | can't remember if you had to deal with this particular problem | 01:52 |
| axw | heh :) | 01:52 |
| adam_g | bigjools, 2013-07-15 00:56:15 ERROR juju.state open.go:93 TLS handshake failed: x509: certificate has expired or is not yet valid <- any idea? | 01:55 |
| bigjools | adam_g: none at all, sorry. this looks like a connection problem to mongo, which is nothing to do with maas | 01:58 |
| bigjools | we tested that multi-user worked already so maybe an env issue for you? | 01:58 |
| adam_g | bigjools, yeah, looks lthat way. multi-user stuff is working fine in another environment | 01:59 |
| bigjools | sorry I can't help more :( | 01:59 |
| adam_g | cls | 02:00 |
| adam_g | what is reponsible for /var/lib/juju/server.pem ? | 02:02 |
| * thumper -> school run | 02:02 | |
| wallyworld__ | adam_g: that is generated by cloud init based on the ca-private-key in the environment settings as far as i know | 02:12 |
| wallyworld__ | or ca-cert perhaps | 02:12 |
| sinzui | Damn. I cannot upload public tools to hp | 02:16 |
| sinzui | I can see my test from this afternoon, but now my creds don't work | 02:16 |
| * sinzui uploads tools using the gui | 02:18 | |
| sinzui | Oh something has cursed me. sync-tools fails, swift fails, js errors using Hp's gui | 02:23 |
| * sinzui drinks more | 02:24 | |
| adam_g | wallyworld__, cloud-config contains: http://paste.ubuntu.com/6338969/ | 02:26 |
| adam_g | mongo is running: usr/bin/mongod --auth --dbpath=/var/lib/juju/db --sslOnNormalPorts --sslPEMKeyFile /var/lib/juju/server.pem --sslPEMKeyPassword xxxxxxx --bind_ip 0.0.0.0 --port 37017 --noprealloc --syslog --smallfiles | 02:26 |
| adam_g | but | 02:26 |
| adam_g | Oct 31 20:21:47 ewing mongod.37017[14138]: Thu Oct 31 20:21:47.025 [conn2326] end connection 10.246.8.3:37271 (0 connections now open) | 02:27 |
| adam_g | Oct 31 20:21:47 ewing mongod.37017[14138]: Thu Oct 31 20:21:47.204 [initandlisten] connection accepted from 127.0.0.1:35146 #2327 (1 connection now open) | 02:27 |
| adam_g | Oct 31 20:21:47 ewing mongod.37017[14138]: Thu Oct 31 20:21:47.205 [conn2327] ERROR: error:14094412:SSL routines:SSL3_READ_BYTES:sslv3 alert bad certificate | 02:27 |
| adam_g | (^ from syslog) | 02:27 |
| adam_g | and | 02:27 |
| adam_g | 2013-10-31 20:22:09 ERROR juju.state open.go:93 TLS handshake failed: x509: certificate has expired or is not yet valid | 02:27 |
| adam_g | cloud-init-output | 02:27 |
| wallyworld__ | adam_g: sadly i'm not very familiar with issues pertaining to certificate generation etc. i have no idea how to debug your issue. if you are around in about 4 hours, the folks from europe will come online and will be able to help a lot more | 02:29 |
| wallyworld__ | the mechanics all seem ok, but the content must be wrong and i don't know why | 02:29 |
| sinzui | I think Hp Cloud is in flames | 02:39 |
| sinzui | I cannot eve talk to my deployed env, or create a new env | 02:39 |
| sodre | wallyworld: when you have time, can you help me with the cherry picking. I am getting an error saying revision 110 does not exist | 02:44 |
| wallyworld__ | sodre: are you working from the same branch you pushed to lp? | 02:45 |
| wallyworld__ | 110. By Patrick Carlos 8 hours ago | 02:45 |
| wallyworld__ | Closes #1246827. GOOSE asks for swift.list to be in json format. | 02:45 |
| _mup_ | Bug #1246827: swift List expects JSON output but does not pass format=json in http request <Go OpenStack Exchange:New> <https://launchpad.net/bugs/1246827> | 02:45 |
| sodre | yeah. that is what happened. | 02:46 |
| wallyworld__ | in any case, so long as your current copy of the branch only has the fixes for that branch that's all that matters | 02:46 |
| thumper | wallyworld__: you firefighting? | 02:46 |
| wallyworld__ | thumper: no, just helping on irc | 02:47 |
| wallyworld__ | and doing a review | 02:47 |
| thumper | wallyworld__: work in progress that I'm considering landing ... https://code.launchpad.net/~thumper/juju-core/kvm-container-interface/+merge/193533 | 02:47 |
| thumper | start of the kvm bits | 02:47 |
| sodre | okay, maybe I need to make a new branch and check-pick from there. | 02:47 |
| thumper | work I had in a pipeline | 02:47 |
| wallyworld__ | thumper: that's a lot of green :-) | 02:48 |
| thumper | sure is | 02:48 |
| thumper | and here you just thought that I sat on my thumb all day | 02:48 |
| wallyworld__ | sodre: no need to start again i don't think - the diff looks ok actually, so you must have removed the code | 02:48 |
| wallyworld__ | thumper: well...... | 02:49 |
| wallyworld__ | no comment | 02:49 |
| sodre | yeah, got it :( | 02:50 |
| sodre | :) | 02:50 |
| sodre | the code needs testing with pure Swift like canonistack or hp cloud, just to be safe. | 02:51 |
| wallyworld__ | sodre: i tested with canonistack and it's ok | 02:53 |
| wallyworld__ | after i made a change | 02:53 |
| wallyworld__ | but the unit tests need fixing also | 02:53 |
| sodre | cool. I tested with radosgw and it works as well. | 02:53 |
| wallyworld__ | thumper: just started; so for lxc, there's a golxc lib. no gokvm equivalent? also, container/interface.go contains "kvm" in the docstrings | 02:55 |
| thumper | hmm... yes no gokvm | 02:55 |
| thumper | I though we'd just wrap it in the container/kvm bit | 02:55 |
| thumper | it isn't complicated | 02:56 |
| wallyworld__ | on the surface, it would be good to implement this consistently | 02:56 |
| sodre | walyworld__: I fixed the unit test. and I've added the StatusNoContent to requestdata. let me know if you need anything else. | 02:56 |
| thumper | much simpler interface than lxc | 02:56 |
| wallyworld__ | ok | 02:56 |
| wallyworld__ | sodre: will do, thanks | 02:56 |
| thumper | the container/lxc or container/kvm is the juju interface to the containers | 02:56 |
| thumper | so in that respect, it is consistent | 02:56 |
| wallyworld__ | ok. will read more | 02:56 |
| sinzui | does anyone have an hp cloud envs that they can get the status of now | 02:57 |
| thumper | I'm not going to add a new top level project for the hell of it | 02:57 |
| thumper | not yet, anyway | 02:57 |
| sinzui | oh brilliant, I think someone took away my access to hp while I was deploying | 02:58 |
| thumper | haha | 02:58 |
| sinzui | I am in the console and watching the feature disappear | 02:59 |
| sinzui | CI is DEAD | 02:59 |
| * thumper calls it a day a little earlier than normal | 03:00 | |
| thumper | given the late night antics | 03:00 |
| thumper | have a good weekend everyone | 03:00 |
| wallyworld__ | sodre: because you have this : "-params.Add("format", "json")" in the diff, it looks like someone has already added that change in and you would now be removing it. so add it back toyour branch and re-push | 03:00 |
| sodre | :) | 03:00 |
| wallyworld__ | sinzui: well that suck donkey balls :-( | 03:00 |
| wallyworld__ | sinzui: want me to try and log in? | 03:01 |
| sinzui | yeah. because someone not me may have just broken all juju deploys. that is to say I had to jack hammer the tools into place and I cannot verofy they work | 03:01 |
| sinzui | wallyworld__, If you bootstrap on HP, do you get 1.16.2 | 03:02 |
| wallyworld__ | sinzui: i'll look, just a sec | 03:02 |
| sinzui | oh, fuck I still need to copy those packages to the public PPA | 03:02 |
| wallyworld__ | sodre: you want to remove 50+ fmt.Sprintf("%s", bodydata) | 03:03 |
| sodre | ohhh... sorry | 03:03 |
| wallyworld__ | np :-) | 03:03 |
| sodre | what do I do with bodydata ? | 03:04 |
| wallyworld__ | sodre: well, i this case i think you can ignore it. this is just a test mock and we don't implement container acl, so we just want to return an ok response | 03:05 |
| sodre | alright I'll take away the entire ioutiol.ReadAll | 03:05 |
| wallyworld__ | sinzui: my wallyworld login on hp cloud still have access to the accounts for the public bucket and also another account with compute resources etc | 03:05 |
| wallyworld__ | sodre: what is the content of the body? | 03:06 |
| wallyworld__ | sodre: maybe we want to just assert that looks sensiblw | 03:06 |
| wallyworld__ | sinzui: did you want to use that for anything till your access can be sortedout? | 03:06 |
| sodre | wallyworld__: I think it is just the header with X-Container-Read | 03:07 |
| sinzui | wallyworld__, I want confirmation that a bootstrap gets the tools I uploaded via the gui | 03:07 |
| wallyworld__ | sinzui: ok, checking now | 03:07 |
| sinzui | wallyworld__, have you deployed using the juju-public-tools region? | 03:08 |
| wallyworld__ | sinzui: yes, the required url is baked into juju for 1.16 | 03:08 |
| wallyworld__ | so no need for tools-url or anything else | 03:09 |
| sinzui | hmm. I was told not to do that | 03:09 |
| wallyworld__ | you were told wrong :-) | 03:09 |
| wallyworld__ | well, i believe so | 03:09 |
| sinzui | Hey the console lets me see juju-scale test again and I can see my three machines are still up | 03:09 |
| wallyworld__ | sinzui: was a reason given?the whole idea of hpcloud in 1.16 was to remove the need for public-bucket-url and tools-url ie make it config free. and that's also in the release notes | 03:10 |
| sinzui | wallyworld__, hey I am in, I have access from the command line again | 03:10 |
| wallyworld__ | \o/ | 03:10 |
| sinzui | log says it selected 1.16.2 for upgrade | 03:11 |
| wallyworld__ | great | 03:11 |
| sinzui | so maybe someone was messing with me or HP got paralytic for 2 hours | 03:12 |
| sinzui | \o/ upgrade is perfect | 03:12 |
| wallyworld__ | whoohoo | 03:13 |
| sinzui | I guess in theory, I could run the script to republish the tools. It will tell me if I did something wrong | 03:13 |
| sodre | wallyworld__: I pushed the code to lp:~psodre/goose/radosgw-acl | 03:13 |
| sodre | now I need to fix the swift-jason branch I just messed up. | 03:14 |
| wallyworld__ | sodre: i see what may have happened - i didn't notice the prereq branch. can you please resubmit the mp and leave out the prereq. you can choose to keep history so existing comments are retained | 03:18 |
| wallyworld__ | for some reason, the preview diff is now messed up also. hopefully resubmit will help | 03:19 |
| sodre | okay... | 03:19 |
| wallyworld__ | sodre: in case you don't know, use the Resubmit proposal link | 03:22 |
| sodre | ahhh | 03:23 |
| sodre | I really need to get used to lp | 03:23 |
| wallyworld__ | it's good once you know all the features | 03:23 |
| sodre | sorry, where is the resubmit | 03:23 |
| wallyworld__ | top right | 03:23 |
| sodre | on the bug page ? | 03:24 |
| wallyworld__ | no, the mp page | 03:24 |
| sodre | got it. | 03:24 |
| sodre | so no prereq. | 03:24 |
| wallyworld__ | yep | 03:24 |
| wallyworld__ | that's better | 03:25 |
| wallyworld__ | except the json line is still there | 03:26 |
| sodre | argh. | 03:26 |
| sodre | how do I see that ? | 03:26 |
| wallyworld__ | look t the preview diff | 03:26 |
| sodre | okay okay... | 03:26 |
| wallyworld__ | on the mp page | 03:26 |
| wallyworld__ | i'd say you added it to get stuff working | 03:26 |
| sodre | I can take it out. | 03:27 |
| wallyworld__ | sodre: normally it would be ok to leave in, but because someone else is supposed to have a fix in progress, it's best not to overlap | 03:28 |
| sodre | I am the one assigned the json bug | 03:28 |
| wallyworld__ | oh? | 03:28 |
| sodre | yeah, sinzui set it to me. | 03:28 |
| wallyworld__ | ok. it was discussed yesterday that someone in our team would start on it. but since you have the bug, go for it | 03:29 |
| wallyworld__ | link the bug to your branch | 03:29 |
| wallyworld__ | so now you have 2 bugs linked | 03:29 |
| sodre | I had a branch just for that bug | 03:29 |
| sodre | it is already linked. | 03:29 |
| sodre | maybe the best would be to approve/merge that first. | 03:29 |
| sodre | then merge the radosgw-acl | 03:30 |
| wallyworld__ | sure. that's the more "correct" way | 03:30 |
| wallyworld__ | although for one line... :-) | 03:30 |
| sodre | i know | 03:30 |
| sodre | okay... let me just link the entire branch to both bugs. | 03:30 |
| wallyworld__ | since you have the branch, propose it and i'lllook | 03:30 |
| wallyworld__ | argh. whatever works :-) | 03:31 |
| sodre | alright its proposed and linked to the bug | 03:32 |
| wallyworld__ | sodre: i've approved, thanks. next step is to set the commit message (usually copy the description) and mark as approved. then the landing bot will take over | 03:37 |
| sodre | okay. is that in the Merge Proposal page as well ? | 03:38 |
| sodre | cool | 03:39 |
| wallyworld__ | yep | 03:40 |
| wallyworld__ | sodre: update the descption to reference both bugs also :-) | 03:41 |
| sodre | is it just a matter of writing #XXXX ? | 03:41 |
| sodre | or is there a special markup ? | 03:41 |
| wallyworld__ | If you write bug XXXXX it will be hyperlinked. I think lp:XXXX works also | 03:43 |
| wallyworld__ | it's been a while so i can't recall with 100% certainty | 03:44 |
| sodre | alright. all done. | 03:46 |
| wallyworld__ | right, so bug #XXXworks also :-) | 03:46 |
| wallyworld__ | sodre: i've marked a approved at the top of the mp. it should land in trunk in about 5 minutes | 03:47 |
| sodre | :) that is so coo | 03:47 |
| sodre | cool | 03:47 |
| sodre | once that is done, can I delete the branches ? | 03:48 |
| wallyworld__ | you can delete them off your local disk yes. but not on launchpad | 03:49 |
| wallyworld__ | you can always get them off lp again if needed | 03:49 |
| sodre | okay. | 03:49 |
| wallyworld__ | sodre: merged already :-) | 03:49 |
| wallyworld__ | you fixed two bugs. thank you | 03:50 |
| sodre | \o/ | 03:50 |
| wallyworld__ | now i'll update the dependencies in juju-core so your work gets included in the next release | 03:50 |
| sodre | I see. Do these changes ever make it back to saucy ? | 03:51 |
| wallyworld__ | long story. they will go into juju 1.18 (next release). that will be available via a ppa. i don't believe we are allowed to put 1.18 in saucy backports | 03:52 |
| sodre | I see. I know that we had a release today, 1.16.2 ( or something) is that available through PPA as well ? | 03:53 |
| wallyworld__ | yes | 03:56 |
| wallyworld__ | if not now, then very soon | 03:56 |
| wallyworld__ | release smoke testing happening as we speak | 03:56 |
| sodre | alright, I'll just tell people at work to update their juju version. | 03:57 |
| wallyworld__ | if they want your fix, they will need to grab trunk and compile themselves | 03:58 |
| wallyworld__ | we will be releasing 1.17 as a beta to iron out any issues prior to 1.18 | 03:59 |
| sodre | Got it, compiling juju with go is fairly straightforward as well. so it should not be an issue. | 03:59 |
| wallyworld__ | sodre: just got the release email for 1.16.2. ppa is at https://launchpad.net/~juju/+archive/stable | 04:06 |
| sodre | thanks. I'll add that. I was looking forward to the MAAS/lxc fix | 04:08 |
| rogpeppe1 | mornin' all | 07:41 |
| === rogpeppe1 is now known as rogpeppe | ||
| axw | morning | 08:04 |
| rogpeppe | axw: you might be interested to have a look at some stuff i've been doing to make it easy (hopefully) to write charms in Go: http://godoc.org/launchpad.net/juju-utils/cmd/gocharm | 09:49 |
| axw | rogpeppe: looks cool :) | 10:19 |
| rogpeppe | axw: thanks | 10:20 |
| axw | I like the thought of having go-gettable "charm helpers" | 10:20 |
| axw | makes them feel a bit more first class | 10:20 |
| rogpeppe | axw: did you see launchpad.net/juju-utils/charmbits/httpserver ? | 10:20 |
| axw | rogpeppe: didn't get that far yet, but I did see it in hte imports | 10:20 |
| rogpeppe | axw: i'm trying to come up with a scheme that lets people build charms modularly | 10:21 |
| rogpeppe | axw: it seems to be working ok so far, although i'm slightly concerned that people might find the abstractions hard to deal with | 10:22 |
| axw | rogpeppe: so that charmbits/httpserver package is used in a charm that is also the application? | 10:22 |
| axw | I mean.. service | 10:22 |
| rogpeppe | axw: yes | 10:23 |
| rogpeppe | axw: the idea is that you can write an entirely self-contained charm in Go if you want | 10:23 |
| axw | yup | 10:23 |
| rogpeppe | axw: the awkward piece being you want to be able to tell upstart to run a piece of your charm as a service and you don't really want to make another binary for that | 10:24 |
| axw | yeah I was just thinking about that.. you haven't got to that part yet? I don't see anything to do with running the binary outside of a hook | 10:25 |
| rogpeppe | axw: actually that bit is done | 10:25 |
| axw | ah yeah | 10:26 |
| axw | I see now. | 10:26 |
| rogpeppe | axw: see http://bazaar.launchpad.net/+branch/juju-utils/view/head:/charmbits/httpserver/server.go#L112 | 10:26 |
| rogpeppe | axw: and hook.Context.RegisterCommand | 10:26 |
| rogpeppe | axw: there's a standup in 7 minutes if you fancy coming along BTW | 10:38 |
| axw | rogpeppe: sure, wife's out so I may as well | 10:38 |
| axw | ta | 10:38 |
| fwereade_ | mgzh, meeting | 10:47 |
| mgzh | ta | 10:48 |
| === mgzh is now known as mgz | ||
| rogpeppe | frickin' new hangouts | 11:25 |
| === BradCrittenden is now known as bac | ||
| rogpeppe | natefinch: ping | 13:06 |
| * fwereade_ has to go to the shop while it's open, bbiab | 13:08 | |
| sodre | !on-call | 13:24 |
| rogpeppe | i like the fact that i can reboot and my local juju environment is still running just fine | 13:44 |
| rogpeppe | i like the fact that i can reboot and my local juju environment is still running just fine | 13:44 |
| * rogpeppe gets some lunch | 13:47 | |
| natefinch | rogpeppe: let me know when you're back from lunch' | 13:55 |
| === BradCrittenden is now known as bac | ||
| abentley | sinzui: When mysql dies for you, does mysql/1 go into an error state? | 14:27 |
| sinzui | abentley, install error | 14:28 |
| abentley | sinzui: For me, it just went to 'error', not 'install-error', so I guess it's because Jenkins only has mem=2G. | 14:29 |
| sinzui | oh | 14:30 |
| natefinch | fwereade_: you around? | 14:36 |
| fwereade_ | natefinch, hey dude, sort of | 14:37 |
| fwereade_ | natefinch, explaining juju to anold colleague | 14:37 |
| natefinch | fwereade_: heh... possibly quick question (if not, that's ok). The server API for set was already written, with part of the implementation having this comment: | 14:38 |
| natefinch | / parseSettingsCompatible parses setting strings in a way that is | 14:38 |
| natefinch | / compatible with the behavior before this CL based on the issue | 14:38 |
| natefinch | / http://pad.lv/1194945. Until then setting an option to an empty | 14:38 |
| natefinch | / string caused it to reset to the default value. We now allow | 14:38 |
| natefinch | / empty strings as actual values, but we want to preserve the API | 14:38 |
| natefinch | / behavior. | 14:38 |
| natefinch | fwereade_: this makes the API implementation retain the behavior of setting a value to "" causing the value to get unset. | 14:39 |
| natefinch | fwereade_: The comment makes it seem like that's on purpose... but that just totally breaks the expected functionality of set if we always go through the API, so I'm not really sure what the correct course of action is. | 14:39 |
| fwereade_ | natefinch, I think we need parallel implementations for now, that's just a compatibility thing for the gui | 14:39 |
| natefinch | fwereade_: oh, the GUI, ok | 14:40 |
| fwereade_ | natefinch, we want to express the cli functionality now | 14:40 |
| fwereade_ | natefinch, and *that's* a job for aslightly neater ServiceSet call | 14:40 |
| fwereade_ | natefinch, so we have halfachanceofoneday being able toset a bunch ofservice attributes transactionally | 14:40 |
| fwereade_ | natefinch, eg, set charm and config at the same time | 14:40 |
| natefinch | fwereade_: so we need two - the existing one for the GUI and a new one for the CLI | 14:41 |
| fwereade_ | natefinch, yes please -- ask frankban if heeverstarted on the newserviceset thing | 14:41 |
| natefinch | fwereade_: cool, thanks for clearing that up | 14:42 |
| rogpeppe | natefinch: ping | 15:06 |
| natefinch | rogpeppe: howdy | 15:12 |
| rogpeppe | natefinch: fancy a chat about HA? | 15:12 |
| natefinch | rogpeppe: definitely | 15:12 |
| rogpeppe | natefinch: we could try G+ again - i've rebooted, so it *may* be ok again | 15:13 |
| natefinch | rogpeppe: might as well give it a second change | 15:13 |
| natefinch | chance | 15:13 |
| rogpeppe | natefinch: https://plus.google.com/hangouts/_/calendar/bWFyay5yYW1tLWNocmlzdGVuc2VuQGNhbm9uaWNhbC5jb20.09gvki7lhmlucq76s2d0lns804?authuser=1 | 15:13 |
| fwereade_ | bbl | 16:15 |
| sinzui | rogpeppe, CI is failing for juju-core. One reason relates lxc-provisoner has no credentials for the user attribute. The test is not using lxc though. We are starting a caonistack instance. Also note that juju 1.16.0 client got the 1.16.2 agent. Do you have any insights https://pastebin.canonical.com/99724/ | 16:25 |
| rogpeppe | sinzui: on a call currently - will get back to you | 16:30 |
| rogpeppe | need to reboot again :-( | 16:44 |
| sinzui | rogpeppe, I can elaborate about the error. I don't think the juju version is the issue because we see the same messages when using 1.16.2. The issue may be that the juju client is in a precise lxc bootstrapping precise on canonistack. | 16:50 |
| sinzui | we don't see this issue running saucy clients/client not in lxc | 16:51 |
| rogpeppe | sinzui: looking now | 16:51 |
| rogpeppe | sinzui: what is the CI actually trying to do? | 16:52 |
| rogpeppe | sinzui: that error message is coming from the openstack provider | 16:53 |
| sinzui | rogpeppe, It is verifying I can deploy a simple stack to canonistack | 16:54 |
| sinzui | rogpeppe, jenkins in canonistack created an lxc to build juju-core tip, then from in the lxc, began bootstrapping canonistack, hp, and aws envs | 16:56 |
| rogpeppe | sinzui: so the bootstrap worked ok? | 16:57 |
| sinzui | rogpeppe, no, bootstrap returned an error | 16:58 |
| rogpeppe | sinzui: hmm, weird, then how come the bootstrap node is working ok? | 16:58 |
| rogpeppe | sinzui: what error did bootstrap return? | 16:58 |
| sinzui | abentley, what error did the bootstrap return for CI of canonistack? | 16:59 |
| rogpeppe | sinzui: so the bootstrap returned an error, but it successfully started an instance running the machine agent? from the error returned, that seems quite odd, as the error should have stopped it bootstrapping before it got anywhere near that stage. | 17:01 |
| sinzui | rogpeppe, me network/ssh is busted. I cannot sshuttle this hour. I think I need to reboot to get into my 3rd test of this scenaro | 17:02 |
| abentley | sinzui, rogpeppe: bootstrap didn't error, but after bootstrap, nothing worked. Not "status", not "deploy". It just hangs. | 17:07 |
| abentley | (And occasionally emits ERROR TLS handshake failed: EOF) | 17:07 |
| rogpeppe | abentley: ah, that makes more sense | 17:07 |
| rogpeppe | abentley: could you paste (privately) the contents of the .jenv file for the environment? (elide private keys if you care) | 17:08 |
| abentley | rogpeppe: https://pastebin.canonical.com/99730/ | 17:10 |
| abentley | rogpeppe: I can also add your public ssh key to the bootstrap node so you can log in, if you like. | 17:11 |
| rogpeppe | abentley: that would be useful, thanks | 17:11 |
| sinzui | abentley, rogpeppe my third test did eventually let me in. all is good. I will go downgrade to 1.16.0 again | 17:11 |
| abentley | rogpeppe: You should be able to ssh to ubuntu@10.55.60.105 now. | 17:13 |
| rogpeppe | abentley: thanks. does that still have the problem? | 17:14 |
| abentley | rogpeppe: Yes. | 17:14 |
| abentley | sinzui: I am experiencing this issue with saucy. | 17:17 |
| sinzui | abentley, I am not... | 17:17 |
| sinzui | abentley, I downgraded to 1.16.0 on precise in an lxc. I deployed and got the 1.16.2 agent. I has just started. I am now deploying services | 17:18 |
| abentley | sinzui: This issue is not really related to juju CI. I'm trying to create a jenkins node where I will eventually do CI. I'm doing this because the existing jenkins node has 2G and that may not be enough. | 17:18 |
| abentley | But this is really just juju bootstrap not leading to to a working environment on canonistack. | 17:19 |
| sinzui | abentley, ha ha, i just hit the max security groups/instances again | 17:20 |
| abentley | sinzui: That was fast! | 17:20 |
| sinzui | abentley, I will tear this down. I don't think we need it since I misunderstood the scenario | 17:22 |
| sinzui | abentley, or maybe no...I think missed the step where I go the the lxc. my machine is now angry that I have an out of date package installed | 17:25 |
| sinzui | also. I need to work from OS X for sometime today. Once again I botched the homebrew pull request. I will never use githubs edit feature again. I will work with the real code and real tests | 17:27 |
| sinzui | now apt is angry. I think I need to do one thing at a time to achieve something | 17:29 |
| rogpeppe | abentley: i can't connect to that machine | 17:33 |
| abentley | rogpeppe: Does it refuse to authenticate you or can you not even connect to it? | 17:42 |
| rogpeppe | abentley: connection timed out | 17:42 |
| abentley | rogpeppe: do you have your .ssh/config set up to allow you to ssh into canonistack machines? | 17:43 |
| abentley | rogpeppe: I just sshed in successfully. | 17:43 |
| rogpeppe | abentley: ah, probably not | 17:44 |
| abentley | rogpeppe: the crucial thing is to have chinstrap act as a proxy. | 17:45 |
| rogpeppe | abentley: sample config? | 17:45 |
| abentley | rogpeppe: https://pastebin.canonical.com/99741/ | 17:46 |
| rogpeppe | abentley: bingo! | 17:47 |
| * sinzui interesting. githubs dimensions aren't broken in safari. Just chromium | 18:12 | |
| rogpeppe | abentley: is the environment working now? | 18:13 |
| rogpeppe | abentley: i think i know what your problem was | 18:13 |
| abentley | rogpeppe: Yes, it's working now. | 18:14 |
| rogpeppe | abentley: ha | 18:14 |
| rogpeppe | abentley: so there's an easy fix: | 18:14 |
| rogpeppe | abentley: run juju status | 18:14 |
| abentley | rogpeppe: But juju status doesn't work. | 18:15 |
| abentley | rogpeppe: I does now, but it didn't. | 18:15 |
| abentley | s/I does/It does/ | 18:15 |
| rogpeppe | abentley: really? it worked for me when i copied your jenv file to your instance and ran juju status | 18:15 |
| rogpeppe | abentley: and that caused the secrets to be pushed to the environment which caused everything to start working | 18:16 |
| abentley | <abentley> sinzui, rogpeppe: bootstrap didn't error, but after bootstrap, nothing worked. Not "status", not "deploy". It just hangs. | 18:16 |
| abentley | (And occasionally emits ERROR TLS handshake failed: EOF) | 18:16 |
| rogpeppe | abentley: hmm, i wonder if that's an addressing problem | 18:16 |
| rogpeppe | abentley: where were you trying to run juju status from? | 18:17 |
| abentley | rogpeppe: IME, that's what you get when the bootstrap node isn't responsing to API requests even at the http level. | 18:17 |
| abentley | rogpeppe: I was running status from this laptop I'm typing on. | 18:17 |
| rogpeppe | abentley: the bootstrap node was responding to API requests - i have a feeling your juju client just had no connectivity to the bootstrap node | 18:17 |
| rogpeppe | abentley: but... you say it works now? | 18:18 |
| abentley | rogpeppe: IME when there's no connectivity, nothing happens at all, not " ERROR TLS handshake failed: EOF" | 18:18 |
| abentley | rogpeppe: I did restart sshuttle just now. | 18:19 |
| rogpeppe | abentley: i wonder if was an issue with that | 18:19 |
| abentley | rogpeppe: I guess it's possible. I took the "ERROR TLS handshake failed: EOF" messages as an indication that the sshuttle tunnel was working. | 18:20 |
| rogpeppe | abentley: basically, the API server seemed to be running just fine, which makes me think that the problem was elsewhere. | 18:20 |
| rogpeppe | abentley: if the tunnel was playing up, you might easily get an error like that (which is from the TLS negotiation taking place *within* the ssh tunnel) | 18:21 |
| abentley | rogpeppe: When I restarted, I switch to using 10.55.60.105 itself as the tunnel endpoint. I had previously been using 10.55.32.48. Normally, that's not a problem, but these machines are in different regions. | 18:23 |
| abentley | 10.55.60.105 is lcy01, 10.55.32.48 is lcy02 | 18:24 |
| rogpeppe | abentley: ah, so that was the problem? | 18:24 |
| abentley | rogpeppe: Yes, I had the same symptoms when I switched back to 10.55.32.48 | 18:24 |
| rogpeppe | abentley: great. | 18:24 |
| rogpeppe | abentley: if you reported a bug, i guess we can mark it as invalid... | 18:25 |
| rogpeppe | abentley: we should probably fix things so that the provisioner doesn't constantly fall over while waiting for a valid environment, i guess | 18:25 |
| rogpeppe | abentley: actually, that should be fixed in trunk already | 18:26 |
| abentley | rogpeppe: I didn't file a bug, but the whole episode only increases the importance of bug #1224057 to me. | 18:26 |
| _mup_ | Bug #1224057: Juju needs more than SSH access <addressability> <canonistack> <improvement> <ssh> <juju-core:Triaged> <https://launchpad.net/bugs/1224057> | 18:26 |
| rogpeppe | fwereade_: current status of my HA discussion with nate: https://docs.google.com/a/canonical.com/document/d/1hGa2PNz6JyGFc7FjA-gX2_UnXRUGAZbS8O9TiS51bl8/edit | 18:28 |
| rogpeppe | fwereade_: you'll probably want to bikeshed a bit on names etc | 18:29 |
| abentley | sinzui: Has there been a bug reported that 1.16.0 deploys agent 1.16.2? | 18:32 |
| * sinzui no | 18:32 | |
| * sinzui abentley: I know version mismatch is discussed in bugs, but I don't see a specific bug about the mismatch | 18:32 | |
| rogpeppe | right, that's me done | 18:35 |
| rogpeppe | happy weekends all | 18:35 |
| abentley | rogpeppe: thanks! | 18:41 |
| abentley | sinzui: Bug #1247232 | 18:44 |
| _mup_ | Bug #1247232: Juju 1.16.0 deploys agent version 1.16.2 <juju-core:Triaged> <https://launchpad.net/bugs/1247232> | 18:44 |
| sinzui | thank you abentley | 19:18 |
| hazmat | jam, how do we trigger the support for untrusted certs? | 19:20 |
| jam | hazmat: set "ssl-hostname-verification: false" in the environment config. | 19:22 |
| hazmat | ah.. found it. ssl-hostname-verification | 19:22 |
| hazmat | jam, thanks | 19:22 |
| jam | rogpeppe1: if you're around https://codereview.appspot.com/20940043 | 19:23 |
| jam | It turns out that Uniter didn't share the common code that you fixed to use the cached addresses | 19:23 |
| rogpeppe1 | jam: oops. looking. | 19:52 |
| jam | rogpeppe1: not really your fault, the code *should* have been shared | 19:52 |
| abentley | sinzui: I am seeing a bug deploying wordpress on the local provider (on precise): https://pastebin.canonical.com/99751/ | 19:58 |
| rogpeppe1 | jam: reviewed | 20:00 |
| jam | rogpeppe1: it doesn't include CACert because Uniter doesn't use it | 20:01 |
| jam | Uniter grabs APIAddresses to pass to the hook context | 20:01 |
| jam | but it doesn't give them the cert | 20:01 |
| rogpeppe1 | jam: hmm, maybe it should | 20:01 |
| rogpeppe1 | jam: i think we *should* pass the cert to the hook context (i think there's an existing issue about that) | 20:01 |
| rogpeppe1 | jam: so i think it's worth providing in the API so we can easily do that later | 20:02 |
| rogpeppe1 | jam: but don't bother if it's a hassle | 20:02 |
| jam | rogpeppe1: when it comes to that point, then we can easily move the function from Addresser to APIAddresser, I'd like to keep the change focused | 20:02 |
| rogpeppe1 | jam: ok | 20:02 |
| jam | rogpeppe1: I'm not worried (particularly much) about the memory overhead. Just that every place that used to include Addresser now has to grow 2 lines to do the same thing. | 20:03 |
| rogpeppe1 | jam: there are only two such places AFAICS | 20:04 |
| abentley | sinzui: from cgroup-lite.log: https://pastebin.canonical.com/99753/ | 20:05 |
| rogpeppe1 | jam: i kinda think it would make both of those places more readable (more explicit that they make both state and API addresses available) | 20:05 |
| rogpeppe1 | jam: but i don't mind much - i just had to think quite hard when reading the code in common | 20:05 |
| jam | rogpeppe1: do you think they should be renamed to StateAddresser and APIAddresser then ? | 20:11 |
| abentley | sinzui: The problem appears to be cgroups-lite being upgraded. If I remove the package manually, the installs complete successfuly. | 20:11 |
| rogpeppe1 | jam: that would be inclination, yes | 20:11 |
| rogpeppe1 | s/be/be my/ | 20:11 |
| abentley | sinzui: It affected both mysql and wordpress. | 20:12 |
| abentley | sinzui: I suspect removing apt-get upgrade from the install scripts would also fix it. | 20:12 |
| sinzui | abentley: I am still swimming in the ramifications of this news | 20:14 |
| sinzui | abentley: Does this relate to cloud-archive? | 20:15 |
| abentley | sinzui: I don't know if it's packaged there. | 20:15 |
| abentley | sinzui: The cloud archive is an apt source in the lxc containers. | 20:18 |
| sinzui | I see | 20:19 |
| abentley | sinzui: I do not see groups-lite in http://ubuntu-cloud.archive.canonical.com/ubuntu/pool/main/c/ | 20:19 |
| * sinzui yep, that is where I am | 20:19 | |
| abentley | s/groups-lite/cgroups-lite/ | 20:19 |
| sinzui | I wonder if a dep in the archive uses it though | 20:19 |
| abentley | sinzui: I bet it's installed in the cloud images, but not an up-to-date version. | 20:20 |
| abentley | sinzui: It was attempting to upgrade it, not to install it. | 20:20 |
| === gary_poster is now known as gary_poster|away | ||
Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!