/srv/irclogs.ubuntu.com/2013/10/31/#juju-dev.txt

sinzuihazmat, yes --fancy shows them running00:08
sinzuihazmat, yes pending status, which is all I have ever seen since 1.14.000:08
hazmathmm00:08
hazmatthat's a good start00:09
hazmatsinzui, jorge and marco had this issue, thumper had to do surgery on them.. but tbd if its needed here00:09
* hazmat waits for cloudinit output00:10
sinzuihazmat, cloud-init: http://pastebin.ubuntu.com/6333073/00:10
hazmatcool.. the source of fail00:10
sinzuiwhich agrees with00:10
sinzuicurtis-local-machine-1  RUNNING  -     -     YES00:10
hazmatsinzui, so let's try a clean tear down..00:11
sinzuiI wonder if I need to locate abentley's bug that explained his two step clear lxc cache00:11
hazmatsinzui, issue there is no cloudinit data made it to the container00:11
hazmatsinzui, juju destroy-environment00:11
sinzuidone00:12
hazmatsinzui, anything in ls /etc/lxc/auto ?00:12
sinzuino00:12
hazmatsinzui, how about ls /var/lib/juju/containers/00:13
sinzuihazmat, empty00:13
hazmatsinzui, hmm.. and ls /var/lib/juju/removed-containers/00:13
hazmatsinzui, this is trunk or 1.17 ?00:14
hazmater. 1.1600:14
sinzuihazmat, 1.16.1 release candidate...the one that is supposed to fix lxc00:14
hazmatah.. one of those00:14
sinzuihazmat, Lots in removed containers00:14
sinzuicurtis-local-machine-1  curtis-me-machine-1    curtis-me-machine-1.2  curtis-me-machine-1.400:15
sinzuicurtis-local-machine-2  curtis-me-machine-1.1  curtis-me-machine-1.3  curtis-me-machine-200:15
hazmatsinzui go ahead and kill them.00:15
hazmatand move  your $JUJU_HOME/$local_provider_name out of the way00:15
sinzuihazmat, done00:15
hazmatand that should be pristine for a new bootstrap00:15
hazmatsinzui, if your bootstraping.. with 1.16.1 tools you may need --upload-tools00:15
hazmatif there not published00:15
sinzuiI think that last step is key. I have not done that today00:16
hazmatsinzui, yeah.. --debug on bootstrap and pastebin for posterity if needed00:16
sinzuihazmat, nothing different from my previous tests...so far http://pastebin.ubuntu.com/6333122/00:22
hazmatsinzui, can you set juju set-env  logging-config="<root>=DEBUG;juju.provisioner=TRACE"00:23
hazmatand then deploy something..00:23
hazmatsinzui, btw re logs $JUJU_HOME/$local-env-name/machine-0.log has the logs of interest00:25
sinzuiyeah, that is what I am tailing00:26
hazmatoh.. cool00:26
thumpersinzui: no all machines log for local at this stage00:26
hazmatsinzui, so .. i get roughly the same content you did00:26
hazmat2013-10-31 00:26:17 INFO juju.provisioner provisioner_task.go:367 started machine 1 as instance kapil-local-machine-1 with hardware <nil>00:26
thumperso, what's the summary of the current issues?00:27
sinzuihazmat, 3 running machines without net00:28
hazmatthumper, local provider seems to work on trunk.. sinzui has it failing for him 1.16.100:28
hazmatthumper, somewhat more critical is that it doesn't seem to work on maas as a container at all atm00:28
sinzuihazmat, all three services/machines are pending00:28
hazmatsinzui, you can watch them do upstart and lxc-create via pstree00:29
* thumper sighs...00:29
hazmatthumper, the containers on maas are critical for some ODS stuff00:30
* thumper nods00:30
thumperlet's get that working first then00:30
hazmatthumper, i'm going to give it a go with trunk on the maas i think..00:30
hazmatjust to see if that helps at all. plus to have a source compile i can instrument.. since the logging here seems to be nil00:30
thumperhazmat: do you have any instructions for a virtual maas setup00:31
hazmatsinzui, cool, is cloudinit done from pstree?00:31
thumperso we can test locally?00:31
sinzuiabentley reported this bug. I don't think it is related because I start my lts container every day00:31
sinzuihttps://bugs.launchpad.net/ubuntu/+source/lxc/+bug/123649000:31
_mup_Bug #1236490: Container has no networking <amd64> <apparmor> <apport-bug> <saucy> <linux (Ubuntu):Fix Released> <lxc (Ubuntu):Invalid> <https://launchpad.net/bugs/1236490>00:31
hazmatthumper, you mean did i attend the maas training ;-) ?00:31
hazmatno actually..00:31
* bigjools bans the maas training from existence00:31
thumpersinzui: hangout? I can talk through some things00:31
hazmatsinzui, can you pastebin lxc-ls --fancy00:32
hazmatnot sure what you mean by no networking00:32
sinzuihazmat, yes it is. I can tell when my fan stops actually00:32
hazmatcloudinit is injected here via file00:32
hazmatsinzui, wow.. is that spinning rust?00:32
hazmatsinzui,  or ssd?00:32
sinzuissd00:32
* hazmat runs a dozen containers normally00:32
hazmatonly 2-4 for juju though00:33
hazmati don't even notice them00:33
sinzuiI have my fan tuned to come on often...its a mac and they run hot00:33
hazmatgotcha00:33
sinzuihazmat, no improvement http://pastebin.ubuntu.com/6333175/00:33
hazmatthat's intriguing00:34
hazmatsinzui, can you go to /var/lib/juju/containers00:34
thumperthere should be console.log in there for each machine00:35
hazmatand pastebin the console.log00:35
thumperthat is the output of running the cloudinit00:35
hazmatalthough in this case i think we want the container.log00:35
thumpersinzui: also pastebin ifconfig00:35
hazmatthe netns stuff should have happened pre lxc00:35
hazmater. pre cloudinit00:36
sinzuihazmat, console.log http://pastebin.ubuntu.com/6333189/00:36
sinzuithumper, my ifconfig or one from a container?00:37
thumpersinzui: from the host00:37
hazmatsinzui, can you pastebin the container.log post ifconfig ..00:38
hazmatthat should detail why the no netns00:38
sinzuithumper, ifconfig http://pastebin.ubuntu.com/6333193/00:38
hazmathuh00:38
hazmatyou've got three veth devices there.00:38
hazmatfor your three containers00:38
hazmatsinzui, is dnsmasq running ?00:38
thumpersinzui: how about the lxc.conf from the /var/lib/juju/containers/<firstmachine>/00:39
hazmatyou should see something like... 116       1190  0.0  0.0  28184   804 ?        S    Oct15   0:04 dnsmasq -u lxc-dnsmasq --strict-order --bind-interfaces --pid-file=/var/run/lxc/dnsmasq.pid --conf-file= --listen-address 10.0.3.1 --dhcp-range 10.0.3.2,10.0.3.254 --dhcp-lease-max=253 --dhcp-no-override --except-interface=lo --interface=lxcbr0 --dhcp-leasefile=/var/lib/misc/dnsmasq.lxcbr0.leases --dhcp-authoritative00:39
hazmater.. ignoring the preamble.. that's from ps aux | grep dnsmasq00:39
sinzuihazmat, yes http://pastebin.ubuntu.com/6333198/00:39
hazmatsinzui, the container.log should have the netns come up....00:40
sinzuihazmat, the lxc.conf from -1 http://pastebin.ubuntu.com/6333199/00:40
* hazmat leaves to thumper's capable hands00:41
thumperok, this needs help from an lxc person00:41
thumperas it is outside the remit of juju00:41
thumperthe container didn't get networking00:42
thumperprior to all the juju shit00:42
sinzuithumper, fab, as I think too00:42
thumpersinzui: if you bring up a machine manually, does it get an ip address?00:42
sinzuiwhile abentley's bug is not identical, I will try the work around00:42
sinzuithumper, do you mean bring one down than bring it back up again?00:43
thumpersinzui: what do you get with "ls -l /var/cache/lxc" ?00:43
thumperactually00:43
thumpersinzui: ls -lh /var/cache/lxc/cloud-precise/00:43
sinzuithumper,00:43
sinzuidrwxr-xr-x 2 root root 4096 Jul  1 18:29 cloud-precise00:43
sinzuidrwxr-xr-x 3 root root 4096 Jan  9  2013 precise00:43
sinzuithumper, -rw-r--r-- 1 root root 221M Jun 24 03:07 ubuntu-12.04.2-server-cloudimg-amd64-root.tar.gz00:44
sinzui-rw-r--r-- 1 root root 206M Dec 17  2012 ubuntu-12.04-server-cloudimg-amd64-root.tar.gz00:44
thumperok, that's it00:44
thumperubuntu-12.04-server-cloudimg-amd64-root.tar.gz is too old00:44
thumperand the refresh mechanism isn't working00:44
thumperthis is the problem marco had00:44
thumperand jorge00:44
thumpersmoser is aware00:45
thumpersinzui: delete the file00:45
hazmatsinzui, can you post the console.log00:45
thumperdestroy the environment00:45
hazmater.. container.log00:45
hazmatsinzui, ^00:45
thumperand start again00:45
hazmatbefore destroying00:45
smoserthats a known issue.00:45
smoseryeah, remove that file.00:45
thumperlxc will download a fresh copy of the cloud image00:45
hazmatoh.. that one00:45
thumperand it "should" work00:45
hazmati forget about that.. i ran into that as well00:45
hazmatsmoser, so what causes that?00:46
hazmatsmoser, with the new lxc cloud template / cloud img download there's no dot versioning around point releases..00:46
hazmatbut never really understood why the old container wouldn't netns00:47
thumpersinzui: still there?00:47
smoser"new lxc cloud template" ?00:47
sinzuihazmat, I have a copy of the container log. anything else before I start purging00:47
smoserthere is no such thing as "dot"00:47
smoserever.00:47
smoser12.04.3 is no different than 12.0400:47
smoserthe cloud images do not distinguish them in name. they did at one point and that was a bug.00:48
hazmatsmoser, it is though from a cached file sense00:48
smoserthat is independent of .2 verus .300:48
hazmatpreviously it did.. and it worked better for caching and invalidation purposes when the point release incremented00:48
smoserpreviously it was broken on the server side in naming00:48
smoserwhich broke people who expect to 'wget http://path/to/some/known/current/release/file.img'00:49
smoserhttps://bugs.launchpad.net/ubuntu/+bug/122036600:49
_mup_Bug #1220366: cloud-images have inconsistent filenames in 12.04.3 <bot-comment> <cloud-images> <cloud-images-build> <Ubuntu:Fix Released by utlemming> <https://launchpad.net/bugs/1220366>00:49
sinzuithumper, I am bootstrapping again00:50
hazmatic00:50
hazmatthat should fix sinzui's issue then00:50
smoserthe cloud template needs to be smarter.00:50
smoserbut we really would rather make a lxc wrapper that was smarter.00:50
smoserie, hallyn doesn't think such knowledge should live in lxc itself.00:51
hazmatsmoser, ala the kvm front end.. or ala docker front end?00:52
smoseruvtool-lxc is the plan in my head.00:52
smoserwhich is also ideally where the "--use-the-fastest" cloning argument goes.00:52
sinzuiMy fan has started00:52
smoser:)00:52
hazmatsmoser, the default behavior with -s works for me generally.. ie autodetect btrfs ;-).. but i'm curious about the docker lvm thin provisioning on sparse file dev pool00:53
sinzuiwhoa! there it is. Installed00:54
hazmatsidnei just got the that merged into lxc-clone (re lvm-thin) support00:54
hazmatsinzui, do we have tools published for 1.16.1 or tarball?00:54
smoserhazmat, i think there were issues with that default behavior.00:54
smoserare you sure it even remains ?00:54
smoserand it wouldn't select unionfs00:55
* hazmat checks00:55
sinzuiThank you rocket scientists! hazmat thumper smoser. I really appreciate your patience00:55
thumpersinzui: awesome00:55
thumperthat's one fixed00:55
thumperand that fault should definitely be logged on the local provider troubleshooting page <- jcastro, marcoceppi00:55
thumpernow to look at maas00:55
bigjoolso/00:55
bigjoolsI just got pinged about this00:56
thumperhazmat: what versions were you playing with?00:56
bigjoolsnarrowed it down any yet?00:56
thumperbigjools: not started looking yet00:56
hazmatthumper, 1.16.0 for maas00:56
bigjoolsthumper: I guess I should fire up my maas server? :)00:56
thumperbigjools: gimmie time to shower (haven't since back from the gym yet), and we can have an initial call prior to working out debugging plan00:56
thumperbigjools: ack00:56
sinzuihazmat, yes, my testing tools are in place in aws and hp and canonistack. I place them in the testing/ subdir and point the tools url to them: http://juju-dist.s3.amazonaws.com/00:56
bigjoolsthumper: roger mcdodger00:56
sinzuihazmat, and I will officially make the tarball in about 10 minutes00:57
hazmatsmoser, it requires explicit lxc-clone -s  but yeah.. it works without specing btrfs directly00:57
hazmatdu -hs /var/lib/lxc/container-name -> 7mb00:57
sinzuihazmat, and you can see all the scripts I am using at https://code.launchpad.net/~juju-qa/juju-core/ci-cd-scripts200:58
sinzuiyou can make your own deb without any juju parts installed00:58
wallyworld_bigjools: how long does an juju azure bootstrap instance normally take to spin up?01:10
smoserwallyworld_, ~ 5 minutes i think01:11
bigjoolswallyworld_: from memory not at all quick, I think it was like 5-10 m01:11
wallyworld_ok, ta. just bootstrapped an and waiting for juju status01:11
wallyworld_s/just/have been01:11
marcoceppithumper: can you summarize what should be logged?01:12
thumpermarcoceppi: the symptom was that the machines were pending, and lxc-ls --fancy showed the machines started but with no ipv4 address01:13
thumpermarcoceppi: ls -lh /var/cache/lxc/cloud-precise showed ubuntu-12.04-server-cloudimg-amd64-root.tar.gz to be over 4 months old01:13
thumpermarcoceppi: delete that file, destroy the environment and try again01:13
thumpera new lxc cloud image will be downloaded and all should be good01:14
thumperbigjools: hangout?01:17
thumpermarcoceppi: does that all make sense?01:18
sinzuithumper, marcoceppi It makes sense to me. marcoceppi  can interview me tomorrow or he/nick can invite me to try updating the docs01:19
* thumper nods01:19
thumperthanks sinzui01:19
bigjoolsthumper: calling you....01:19
thumperwhich me?01:20
bigjoolsthe head01:20
bigjoolswrong one?01:20
bigjoolsthis is why I don';t have two g+ profiles01:20
marcoceppithumper: yeah, that's what happened to me01:48
marcoceppialready recorded01:48
wallyworld_bigjools: do you know the magic command to ssh into a bootstrap node on azure? i have the ip address of the machine01:50
bigjoolsjuju ssh 0 :)01:51
wallyworld_bigjools: doesn't work. times out01:51
wallyworld_bootstrap has failed01:51
wallyworld_but i have the ip address of the bootstrap node01:51
bigjoolsummmm in that case you're probably screwed01:51
bigjoolsit will be firewalled01:52
wallyworld_well, when i say bootstrap failed, it completes but juju status fails01:52
bigjoolsok01:52
wallyworld_and i want to find out why01:52
bigjoolscan you telnet port 22?01:52
wallyworld_i'll try01:52
wallyworld_yes01:53
wallyworld_when i try ssh directly, it fails with a permission denied error (publickey)01:53
bigjoolsso it's up but hasn't added the key01:53
wallyworld_yeah, seems so01:53
sinzuiwallyworld_, with azure, I wait 30 minutes for the machine to really be up01:53
bigjoolswhich means bootstrap was not as successful as you might think01:53
wallyworld_sinzui: really? oh joy01:54
bigjools!01:54
wallyworld_wtf takes so long01:54
bigjoolsoh - yes the crazy provisioner01:54
bigjoolsyou can see in the MS dashboard01:55
wallyworld_so before when juju status timed out and i destoyed my env, i should have just  waited01:55
sinzuiwallyworld_, on canonistack, we get TLS timeouts while the bootstrap node is coming up, but azure is silent. I have seen the machine come up in 20 minutes, but 30 is realistic01:55
wallyworld_dashboard? you have a link?01:55
wallyworld_sinzui: ok. i think i've fixed the azure incompatibility but am trying to make sure01:56
bigjoolswallyworld_: well not exactly you would need my password.  let me look at it01:56
wallyworld_ok01:56
wallyworld_bigjools: i bet i can guess it01:56
wallyworld_aussieaussieaussieoioioi01:56
bigjoolswallyworld_: "ianisacocktrumpet"01:56
wallyworld_that would have been my next guess01:57
bigjoolswallyworld_: in fact it was so secure I can't remember it :)01:58
wallyworld_ha ha ha01:58
bigjoolswallyworld_: get your own credentials01:59
wallyworld_bigjools: they're in the mail apparently01:59
wallyworld_i only needed yours cause of this azure cockup and the time pressure to fix it02:00
bigjoolsyeah np02:00
sinzuiI encrypted my creds and put them in the CI machine so that my team could build better testing02:00
wallyworld_i wonder what else besides juju msft broke with their change?02:00
bigjoolswallyworld_: btw baldrick left a hidden present that the mower found02:01
wallyworld_\o/ :-D02:01
bigjoolsyou can imagine the result02:01
wallyworld_did it splat all over you /me asks hopefully?02:01
bigjools>:(02:01
sinzuiYour dog is named baldrick?02:01
wallyworld_yep :-D02:01
wallyworld_he has a cunning plan02:01
* sinzui was just telling his son about Ebenezer black adder 5 minutes ago.02:02
bigjoolswallyworld_: I see two VMs running02:02
* wallyworld_ is finding it hard to type cause he is laughing so hard to bigjools02:02
bigjoolsone of them from juju the other I think for a gwacl run02:02
wallyworld_bigjools: ah the gwacl one needs to be shutdown. i thought i did02:02
* bigjools considers putting baldrick's present in wallyworld_'s mailbox02:03
wallyworld_sinzui: blackadder is one of my favourite tv shows02:03
wallyworld_bigjools: how can you, it's all vapourised all over you, right?02:03
bigjoolsthere's some left02:03
wallyworld_next time get jen to mow the lawn after baldrick has visited :-D02:04
bigjoolswallyworld_: so the cpu graph shows it finishing a heavy load at 11:4502:04
wallyworld_the juju one?02:04
wallyworld_what's the ip address?02:05
bigjools137.135.11.1402:05
wallyworld_yep, that's the one juju status keeps polling mongo for trying to do a status02:06
wallyworld_:-(02:06
wallyworld_i really need to get the logs off that sucker02:06
bigjoolswallyworld_: 17070 and 37017 ports are open02:06
bigjoolsthat's it02:06
wallyworld_37017 is the mgo port02:07
* bigjools wonders if it's the non-ssl mgo02:07
wallyworld_and it says conection refused02:07
wallyworld_ah02:07
wallyworld_that could be it02:07
* bigjools gets lunch while wallyworld_ fixes it02:07
* wallyworld_ doesn't know how to fix that one02:08
bigjoolsdeploy with saucy?02:08
wallyworld_ah ok02:08
wallyworld_good idea02:08
bigjoolsit might have failed to add the cloud archive02:08
wallyworld_yeah, sounds likely02:09
bigjoolsalthough doesn't explain why you can't ssh in02:09
wallyworld_hopefully saucy will work02:09
bigjoolsheard a lot of swearing coming from the living room earlier - jen found jake pulling keys off her laptop again... ROFL02:09
* bigjools lunches02:10
wallyworld_lol02:10
thumpersinzui: still around?02:11
sinzuiI am02:11
thumpersinzui: I have a critical bug02:11
thumpersinzui: for 1.1602:11
thumpersinzui: it is the lxc and maas problem02:12
thumperthat they are using for ods02:12
thumperdemo02:12
thumperalways last minute02:12
thumperit is a regression introduced when the provisioner moved to the api02:12
sinzuiI just cut 1.16.1, but plan a 1.16.2 for azure.02:12
thumperbut no one noticed02:12
sinzuithumper, Do I need to take down 1.16.102:12
thumperno02:12
thumperI don't think so02:12
thumperbecause they can use a custom release02:12
thumperI think02:12
thumperbut we may have a 1.16.2 real soon now02:13
thumperif they do need a release02:13
thumperwhat's the timeframe for 1.16.2?02:13
thumperbigjools: I have a plan02:13
sinzuiI can make it tomorrow immediately after 1.16.1 is needed. I hope for an azure fix of couse02:13
* thumper nods02:13
thumperok02:13
thumperlet me file this bug and get to work02:13
* sinzui creates milestone02:15
wallyworld_sinzui: azure fix will be commited to gwacl today i hope02:17
sinzuiwallyworld_, excellent. Testing will be quick since it is limited to two providers02:17
wallyworld_i'm doing a local test now02:18
sinzuiDid I mention abentley got rudimentary CI running on HP. we can run about 200 upgrade and bootstrap tests a day02:18
wallyworld_ran into the precise mongo thing i think02:18
wallyworld_trying on saucy02:18
thumperwallyworld_: are you chasing the azure issue?02:23
thumperwallyworld_: is there a bug?02:23
wallyworld_yes. the issue is with msft, not us02:24
wallyworld_we need to change gwacl cause they broke the api02:24
wallyworld_i'm testing now02:24
thumper:)02:24
thumperwallyworld_: can you target the bug to 1.16.2?02:24
wallyworld_but azure is sloooooow02:24
wallyworld_thumper: sure. there's 2 of them - a public and a private02:25
* thumper nods02:25
thumperbigjools: lp:1246556 for the juju maas bug02:26
thumperwhich it seems mup can't understand, bug 124655602:26
_mup_Bug #1246556: lxc containers broken with maas <api> <maas-provider> <juju-core:Triaged by thumper> <https://launchpad.net/bugs/1246556>02:26
* thumper moved some code around, all tests still pass02:41
thumperI find this mildly disturbing02:42
wallyworld_bigjools: not sure how many azure instances i have running - i think 2. a bootstrap using saucy still doesn't allow juju status to connect sadly03:06
wallyworld_thumper: i have 2 small azure mp's for the 1.16.2 release if you have a moment sometime https://code.launchpad.net/~wallyworld/gwacl/azure-management-api-change https://code.launchpad.net/~wallyworld/gwacl/fix-request-eof-2/+merge/19334603:07
* thumper is in deep thought03:10
thumpersorry03:10
wallyworld_np03:11
wallyworld_bigjools: what is the price for 2 small gwacl reviews?03:11
wallyworld_needed for juju 1.16.2 release tomorrow03:12
thumperbigjools: bugger...03:28
thumperbigjools: it seems like the quick fix won't work very well at al03:28
thumpermay have to fix it the right way03:28
bigjoolswallyworld_: I can look for you, after all you came bearing gifts yesterday03:29
wallyworld_indeed i did03:29
wallyworld_they are only small03:29
wallyworld_branches03:29
bigjoolsthumper: great architecture FTW03:29
bigjools466 lines03:30
bigjoolsis small in juju world?03:30
wallyworld_bigjools: sadly i have a saucy node running now which also disallows connections to mongo03:30
bigjoolswallyworld_: is the firewall endpoint open?03:30
wallyworld_not sure. what port is that do you know?03:30
wallyworld_i thought you said before 37017 was open?03:31
wallyworld_which is the port the client is sing to contact mongo03:31
bigjoolsok03:32
bigjoolswallyworld_: hang on I need to make call first03:32
wallyworld_ok03:32
bigjoolswallyworld_: first comment - pease remove the factored version numbers03:49
wallyworld_why?03:49
bigjoolsI previously had someone stop doing that - each api request is separately versioned and if you factor it you will introduce subtle  bugs03:50
wallyworld_it kinda sucks having them all copied and pasted in the code03:50
bigjoolsyes - but it needs to be done03:50
wallyworld_ok then03:50
bigjoolsI rooted out at least 3 bugs last time I unfactored it03:50
wallyworld_sigh03:50
bigjoolsI know03:51
wallyworld_how the fook do we keep track of all the apis03:51
wallyworld_sounds like a nightmare03:51
bigjoolswallyworld_: fwiw the version should be associated with the request struct ideally03:52
bigjoolsthey change versions when the format changes03:52
bigjoolsso they are very closely tied03:52
wallyworld_ok. for now though, i'll just go back to how it was03:52
wallyworld_and do a search and replace of string03:53
bigjoolswallyworld_: for example if you update the refactored version you would change all the calls at once, and introduce potentially subtle sematic changes03:53
wallyworld_i thought they would all change together03:53
wallyworld_as a group03:54
bigjoolsIME they dont03:54
wallyworld_:-(03:54
bigjoolsit might say they do....03:54
bigjoolsbut I call BS03:54
wallyworld_fair enough. it is msft after all03:54
bigjoolsyou might want to mention this in the code03:56
wallyworld_ok04:00
thumperarg04:02
thumperstabby stabby04:02
* thumper is trying to unpick something04:02
thumperbut others are sewing things tighter together04:02
bigjoolswallyworld_: I bailed on the other one04:04
wallyworld_bigjools: i wanted to avoid a random custom data so i could hard code the expected response04:04
bigjoolswallyworld_: why?04:04
wallyworld_same reason i like to use strings in tests04:04
bigjoolsthe test demonstrates that it copes with random input04:04
bigjoolsyou should always use random data where the data itself doesn't matter04:05
wallyworld_using the same api call in a test though as the logic sort of negates the test04:05
bigjoolsotherwise it says to the test reader that you're crafting a particular scenario04:05
bigjoolsfeel free to ignore it, you have my +1 already04:05
wallyworld_ok, i'll think about it04:06
wallyworld_thanks04:06
wallyworld_why bail on other one?04:06
bigjoolsbecause it's deep Go innards that I know nothing about04:06
wallyworld_ok04:06
bigjoolsyou're the Go expert :)04:06
wallyworld_i used same technique as in goose etc04:06
wallyworld_you reckon?04:06
bigjoolsself-approve?04:07
wallyworld_might do04:07
bigjoolsdo you need all you instances on my azure account?04:07
bigjoolsyour04:07
wallyworld_bigjools: the saucy one would be good. but i need to be able to ssh in to see the cloud init log04:08
bigjoolsyou need a custom image with the backdoor04:08
wallyworld_ugh ok04:09
bigjoolsyou have to build one and upload it to private storage and tell the api to use it when booting04:09
bigjoolsI forgot how though :(04:09
wallyworld_i never knew how04:09
thumperjam3: ping04:09
wallyworld_in the first place04:09
=== jam3 is now known as jam
bigjoolsthere's a config item for a custom kernel04:10
bigjoolsimage I mean04:10
jamthumper: pong04:10
thumperjam: hangout? I have a critical bug to discuss04:10
wallyworld_bigjools: and you can't ssh in at all and grab the cloud init log?04:10
jamthumper: sure, I have to head out in about 20min04:11
thumperjam: ack04:11
thumperjam: https://plus.google.com/hangouts/_/72cpjufs8mm0i0htkjiuch72s8?hl=en04:11
jamI'm getting "call ended because of server error" give me a sec04:12
* thumper nods04:13
jamthumper: https://plus.google.com/hangouts/_/72cpjqbfsmf42rpnbt2o36cgk004:13
jamsee if this one works04:13
jamIf I follow your link, I got the "new" hangout layout04:13
jamthumper: I saw you show up for 1 second04:14
thumperjam: I have new, you have old04:14
thumperthey don't like each other04:14
thumperserver error04:14
thumperskype?04:14
jamthumper: know of any way to trigger the right one?04:14
thumpermumble04:14
thumperI don't care04:14
jamthumper: I'm on skype, and you're in my friends list04:14
jambut not showing as online04:14
* thumper starts skype04:14
thumperjam: https://bugs.launchpad.net/juju-core/+bug/124655604:16
_mup_Bug #1246556: lxc containers broken with maas <api> <maas-provider> <juju-core:Triaged by thumper> <https://launchpad.net/bugs/1246556>04:16
bigjoolswallyworld_: no, unless cloud-init finishes putting the ssh key in your're buggered04:16
wallyworld_\o/04:16
wallyworld_maybe it should do that first up if it doesn't already04:16
bigjoolswallyworld_: hence you need the backdoor.  Having said that - there is an api field for adding a user and p/w04:16
bigjoolsbut we don't use it as it's mental04:17
wallyworld_got a hint where i look save me searching?04:17
wallyworld_i might do it to debug04:17
bigjoolshang on04:17
bigjoolswallyworld_: in LinuxProvisioningConfiguration you set a Username and Password.  It's in the same place Customdata goes04:18
wallyworld_ok, ta04:18
bigjoolswallyworld_: line 545 in provider/azure/environ.go04:22
wallyworld_great04:22
bigjoolsit was a mandatory field so we randomised it04:22
bigjoolsyay security04:22
wallyworld_lol serious? mandatory?04:22
wallyworld_wtf04:22
bigjoolsthis is msft04:23
* wallyworld_ sighs *again*04:23
thumperjam:  lp:~thumper/juju-core/maas-lxc04:40
=== thumper is now known as thumper-afk
sodrejuju-devs: is anyone still up ?05:40
wallyworld_sodre: some of us are here05:41
sodrehi wallyworld_: I traced the reason for that earlier panic on juju bootstrap05:41
wallyworld_oh cool05:41
wallyworld_admin-secret issue i think?05:41
sodreI've updated the bug report on launchpad. But I was wondering if you could explain to me something related to it.05:42
sodreyeah, it had nothing to do with the admin-secret.05:42
wallyworld_i'll read the bug05:42
sodreThe issue is that jujud tries to get the provider-state through swift without authentication.05:42
wallyworld_it should use creds when it tries to get it05:44
wallyworld_i think from memory05:44
wallyworld_let me check05:44
sodrefrom the client side, (juju) connects to swift using goose.05:45
sodrebut the bootstrap level seems to use (Go 1.1 package http)05:45
jamsodre, wallyworld_: provider-state needs to be in a container that is world readable because all of the clients download stuff from the container via wget/direct http access05:46
jamwhen we set up the bucket ourselves, we set the .r:.rlistings *05:46
sodrefor somereason that is not being set on on my version05:47
jamsodre: you're on Havana ?05:47
sodreyes05:47
wallyworld_jam: not necessarily the conatiner itself but the file05:47
jamwallyworld_: we need the container, we put all the charm data, tool data, etc in there05:47
wallyworld_i think05:47
jamand all needs to be downloadable from plain wget05:47
jamwe don't need it on EC2 because we can generate Signed URLs, but we don't have support for that on Openstack05:48
wallyworld_and doesn't wget work so long as the file itself is readable?05:48
sodreright now, my way around it has been to put tools in the juju-dist and set that world readable.05:48
sodrebut then this provider-state issue came up.05:49
jamwallyworld_: as in, we put a bunch of stuff in the container, and *each* needs to be world readable. It is a fair point that you could just have each file have the ACL, but we need all of them anyway05:50
wallyworld_jam: sure. but now in trunk, public storage instances are gone from the providers05:50
wallyworld_azure, maas never had them anyway05:50
jamwallyworld_: "private" storage *must* be public for Openstack05:50
jamwallyworld_: so if your patch which got rid of the PublicStorage api also removed setting .r:.rlistings for the other Storage, then you broke openstack deployments05:51
wallyworld_i didn't tinker with the other Storage to my kmowledge05:51
jamwallyworld_: we *don't* want to give our Provider credentials to the agents running on non machine-0, so they have to have credentials-free access to download the charm blobs from storage05:52
jamwallyworld_: "rlisting" doesn't exist in the source tree05:53
wallyworld_jam: right. so i just checked, the "private" storage on openstack does have rlistings perms05:53
jamunless that is in goose05:53
wallyworld_jam: it's a const in goose05:53
jamwallyworld_: k05:53
wallyworld_swift.PublicRead05:53
wallyworld_see line 514 of provider.go05:53
wallyworld_in openstack package05:54
jamwallyworld_: yep05:54
jamI found ti05:54
jamsodre: can you "swift stat" your container to see what the ACL is ?05:54
* wallyworld_ has to go get kid from school05:54
sodreI need to bootstrap again, because I've set it by hand.05:55
sodregive me one se.d05:55
sodrealso, I'm on 1.16, not trunk. fyi05:56
sodre.r:*05:58
sodrehumm... strange....05:58
sodrei need a new bucket name... one sec.05:59
sodrejenvs from hell...06:00
sodrethe read acl is empty http://paste.ubuntu.com/6334137/06:03
sodrejam: any ideas if that can be fixed in 1.16.1 ? or 1.17 ?06:21
wallyworld_sodre: did you create that control bucket by hand?07:00
sodreyes07:00
sodreI got it to work as follows07:00
wallyworld_sodre: because juju will not set read acl if it exists already07:01
sodreI first swift post juju-control-bucket --read-acl .r:*07:01
sodreno... no..07:01
sodreif the bucket does not exist, it is still not setting it to .r:*07:01
wallyworld_really? the code in that area hasn't changed. it uses PublicRead = ACL(".r:*,.rlistings")07:02
sodremaybe it is the , .rlistings ?07:02
sodrebecause it is not setting it to read mode right now07:02
sodre... it could be a radosgw issue again..07:03
wallyworld_could a havana thing maybe?07:03
sodrelet me try the following.07:03
sodreI have a bucket right now with read ACL set to empty.07:04
sodreswift post pet.sodre.juju-control-02 --read-acl .rlistings07:05
sodrethe read acl is still empty.07:05
sodreshould it have changed ?07:05
wallyworld_um. not sure. i don't use the swift tool for that sort of thing too often07:05
wallyworld_cause juju does it for me :-)07:05
sodreare you using radosgw as well ?07:06
wallyworld_i've tested juju on hpcloud and our own internal openstacj deployment using folsom and grizzly07:06
sodretwo variables: It could be Havana, or it could be RadosGW07:07
wallyworld_i'm testing now our our openstack07:08
wallyworld_sodre: so a seift post does set the read acl for me07:08
wallyworld_ie it is non empty after a swift post --read-acl ....07:09
sodrecan you paste the readacl line for me.07:09
wallyworld_sure07:09
wallyworld_ Read ACL: .rlistings07:09
sodrehumm..07:10
sodrestrange!07:10
wallyworld_and i think this is on havana07:10
sodrewhat is .rlistings for anyways ?07:11
wallyworld_it allows unauthed clients to list the container07:11
wallyworld_the contents thereof07:11
sodreI see.07:12
sodreIt seems that Rados does not support rlistings07:13
sodreshouldn't a plain .r:* suffice ?07:13
wallyworld_well that sucks if true07:13
wallyworld_don't think so - certain operations do need to list the files in a container07:13
wallyworld_but i can't tell you the specifics of the top of my head07:14
sodrehttp://wiki.ceph.com/01Planning/02Blueprints/Dumpling/Better_Swift_Compatability_for_Radosgw07:14
wallyworld_ah07:14
wallyworld_well, that sorta explains it :-(07:14
wallyworld_it does say .rlisting has something done07:15
sodrethere as a patch in the wild, but it seems they did not want to apply it.07:17
sodreaccording to http://www.mail-archive.com/ceph-devel@vger.kernel.org/msg13829.html07:18
sodreall that is needed is to set a READ permission to  see it via S307:19
sodreI imagine unauthenticated access.07:19
=== mgz is now known as mgzh
sodrewallyworld_: I've updated the bug report one more time.... what is the easiest way for me to modify the juju-core code?07:40
wallyworld_what do you want to change?07:40
sodreI would like to see why it is not honoring the .r:*07:40
sodrewhen working with RadosGW07:40
wallyworld_right now, you need to use bzr to get a copy of the source code off launchpad07:41
wallyworld_go get works too07:41
wallyworld_have you ever done that?07:41
sodreyeah, I did it quickly today. I am not sure how to ``debug'' the code, ie.. gdb did not really help07:42
sodreis there a way to step through, or are debug statements the standard ?07:42
wallyworld_lol07:43
wallyworld_Go has very limited tooling support for debugging07:43
sodrei.c.07:43
wallyworld_in my experience07:43
wallyworld_i think you can use gdb07:43
wallyworld_but i have not done that myself07:43
wallyworld_i'm used to modern IDEs07:44
sodresure I can use that as well, will it work in eclipse ?07:44
wallyworld_perhaps. I use Intellij where there is a decent Go plugin but it does lack debug support sadly07:44
sodrejuju-core will be the first "Go" code I ever read07:44
wallyworld_it's similar syntax to C - not too hard to read07:45
sodrealright... so I don't spend too much time digging... in which file should I be looking for these rlistings ?07:45
wallyworld_for openstack, there's two main files of interest....07:46
wallyworld_provider/openstack/provider.go <--- this one sets up the storage instance for an openstack environment07:47
wallyworld_it creates it with the ACL I pasted above07:47
wallyworld_and provider/openstack/storage.go <--- this one creates the container (if required) and reads/writes/lists files therein07:47
wallyworld_your main interest will be in storage.go07:48
sodregot it07:48
wallyworld_the Storage interface is quite simple - it has List(), Put(), Get() etc methods07:48
wallyworld_i normally just use lots of fmt.Println() statements to debug07:49
wallyworld_thrre's also debug logging07:49
wallyworld_if you run a command like bootstrap with --debug, it will print useful info also07:49
sodreyeah, --debug is my friend :)07:50
sodreit seems the code went to goose07:51
sodrei'll spend some time with it.07:51
sodrethanks a lot man!07:51
wallyworld_np. do ask if you have more questions07:58
wallyworld_yes, goose is the lib used by juju to talk to openstack07:58
sodrethanks. I should probably go to sleep ... its 4am here07:59
sodrethe good thing is that there is a work around07:59
wallyworld_wow. yes. go to bed!07:59
wallyworld_where are you?07:59
sodretz= EST08:00
sodreI live in the DC area08:00
wallyworld_ah ok. i'm in australia08:01
wallyworld_so almost drinking time here :-)08:01
sodrenice :)08:01
wallyworld_thanks for all your hard work!08:02
sodreI'm trying to deploy OpenStack at work08:02
sodreI need to show them a demo ...08:02
sodreso... here is an issue08:03
sodreI am running my compiled version of juju08:03
sodrehow do I bootstrap my system ?08:03
sodreit can't find 1.1708:03
wallyworld_you need to use --upload-tools08:04
wallyworld_if you are running from source08:04
sodretsk tsk tsk... IU need to sleep08:04
sodre:)08:04
sodres/IU/Im08:04
wallyworld_yeah tomorrow!08:04
sodrebtw...08:05
sodre2013-10-31 08:05:44 INFO juju.environs.sync sync.go:103 listing target bucket08:06
sodre2013-10-31 08:05:44 DEBUG juju.environs.tools storage.go:35 reading v1.* tools08:06
sodre2013-10-31 08:05:45 ERROR juju supercommand.go:282 failed to list contents of container: pet.sodre.juju-control-0208:06
sodreissue *listing contents*08:06
wallyworld_right, that is consistent with not having rlisting permissions08:07
wallyworld_juju needs to list the tools available to find the correct ones to use08:07
sodrethis is a bad bug in RadosGW08:08
sodrealright.. ttyl08:08
=== sodre is now known as sodre_zzz
=== allenap_ is now known as allenap
rogpeppemornin' all08:10
rvbaHi wallyworld_, did you figure out what the issue with the Azure provider was?08:11
wallyworld_rvba: hi, yes. msft screwed us over08:11
rvbawallyworld_: nice :/08:11
wallyworld_they changed the api in an incompatible way08:12
rvbaGreat.08:12
wallyworld_tl;dr; we needed to change the apiversion passed to various management calls08:12
rvbaSo much for having API versions and all that jazz.08:12
wallyworld_yeah08:12
wallyworld_and we also needed to tweak the run utility to base64 encode the userdata08:13
rvbaAll right, thanks for the update.08:13
wallyworld_i'm landing some stuff now. thanks for the input yesterday08:16
wallyworld_just had trick or tweaters at my door08:16
wallyworld_treaters lol08:16
mgzmornin'08:57
=== thumper-afk is now known as thumper
* thumper hopes jam can run the meeting tonight as he is hacking now he's back09:33
natefinchmorning all09:37
jamfwereade: welcome back10:03
jamteam meeting: https://plus.google.com/hangouts/_/calendar/bWFyay5yYW1tLWNocmlzdGVuc2VuQGNhbm9uaWNhbC5jb20.09gvki7lhmlucq76s2d0lns80410:05
jamfwereade_: ^^10:05
jamaxw: ^^10:05
fwereade_jam, I am trying10:05
jamsure, we've been having trouble with new vs old G+ interfaces10:05
axwmy machine froze, joining now10:08
fwereade_jam, it's weird, it is literally *just* the hangout that's not working10:09
jamfwereade_: extra joy in that it was working with just you and I until more people joined10:11
fwereade_jam, yeah, indeed10:12
fwereade_jam, there's a crazy storm out here but I doubt its effect is selective enough to be causing this10:12
jamfwereade_: is dimiter around today?10:12
fwereade_jam, he is, but he was out earlier; he expected to make it back for the meeting though10:13
rogpeppedamn, did i miss the team meeting today?10:52
rogpeppejam: sorry about that, i totally forgot10:52
natefinchrogpeppe: yeah but you didn't miss much10:53
rogpeppejam: i'd have joined if you'd pinged me...10:53
rogpeppenatefinch: ok, that's good10:53
natefinchrogpeppe: thumper was just talking about how much he loves having loggo on github10:53
rogpeppenatefinch: ironically?10:53
thumper?!?10:54
rogpeppethumper: you really *do* love having loggo on github?10:54
thumperheh10:54
* thumper goes back to work10:54
natefinchrogpeppe: I wasn't being serious :)10:54
rogpeppenatefinch: it sounded like you weren't - just checking10:55
natefinchrogpeppe: actually I think the only real complaint was having his username in the import url.  He says it feels too unprofessional10:55
rogpeppenatefinch: yeah, that's actually part of why i chose launchpad for some projects10:55
rogpeppenatefinch: the user name doesn't seem like that important a part of the project10:55
natefinchrogpeppe: yeah, I wish there was a different way to do it... you can do something like a project team and have the project under the team.10:57
rogpeppenatefinch: yeah10:57
thumpercan you have a team of one?11:17
sinzuiyes11:17
sinzuiYou can have a team of 0 in Lp11:18
thumpersinzui: I was referring to github11:18
thumperjam: I think I may have a complete fix with tests11:19
natefinchthumper: yes, you can have a team of 1 on github11:19
thumpernatefinch: so I could create an org called loggo?11:19
thumperis it worth it?11:19
sinzuithumper, github is similar to Lp, no one is in the team when it is created, you add members.11:21
natefinchthumper: up to you... you'd have github.com/loggo/loggo11:21
thumpermeh11:21
thumperexcept they call it an organisation11:22
mgzmake the team ~loggo :)11:22
thumperheh11:22
mgzACCEPT THE TILDE11:22
thumperprobably an invalid character11:22
natefinchyeah it's just alphanumeric and dashes11:23
* thumper twiddles fingers while the local lxc downloads and the amazon one spins up11:24
* thumper proposes while he waits11:24
thumperok... local provider still works with my change11:27
* thumper waits for ec211:27
natefinchwhy working so late, thumper?\11:27
thumpernatefinch: critical bug around lxc and maas11:27
thumpernatefinch: they want it for the ODS demo11:27
thumperand sinzui is waiting on it to cut 1.16.211:28
natefinchthumper: ahh, bummer11:28
thumpersinzui: care to land a branch that increments the version number on lp:juju-core/1.1611:28
sinzuithumper, oops, I will do that, thank your for reminding me11:29
natefinchsinzui: please make sure you increment scripts/win-installer/setup.iss as well.  It needs to get updated for the windows installer.11:30
natefinch(it's just a text file, the version number is at the top, you'll see it)11:30
sinzuinatefinch, oh. I've got no notes on that. Have you been doing it each time you run it?11:31
natefinchsinzui: yeah, I've just been modifying it when I create the installer, but obviously that's not a good habit to keep up11:31
natefinchsinzui: I'm considering just making script that'll update all the right spots.11:31
thumper\o/11:36
thumperworks on ec211:36
thumperhave ubuntu deployed into 1/lxc/011:36
thumperjam, fwereade_: https://codereview.appspot.com/20220043/11:42
thumperand dimitern if he wants11:42
thumperthis is to fix lxc on maas for the demo11:42
thumpermakes it so we don't get an environ config for lxc provisioners11:42
dimiternthumper, looking11:43
thumperlocal provider works11:43
thumperand ec2 was able to provision a container11:43
thumperand deploy the ubuntu charm into it11:43
thumperso we know it at least works on something11:44
sinzuiI wish I knew how to make DeployLocalSuite.TestDeployNumUnits pass on my computer11:44
thumperI really want to hand this off as it is almost 1am11:44
dimiternthumper, reviewed11:50
thumperdimitern: ta11:52
thumpersinzui: about to land my 1.16 branch11:53
thumpersinzui: wallyworld_ landed the azure fixes already11:53
sinzuiI saw11:53
thumpersinzui: once mine is in, you should be good for 1.16.211:53
sinzuirock11:53
sinzuiMaybe I wont purse 1.16.1. It has not built yet11:53
thumpersinzui: however I'm about to go to bed11:54
thumperhopefully all lands ok11:54
thumperhow long does it normally take?11:54
thumperand who can see if it is in progress?11:54
sinzuithumper, jamespages has the juju release scripted. He builds the new package and send it off to Ubuntu and the PPA builders in on go.11:58
sinzuiHe emails me when he has run the script11:58
thumpersinzui: I was really referring to the landing bot11:58
sinzui15 minutes11:58
thumperI think I'll wait just to be sure11:58
sinzuinatefinch, Is this right https://codereview.appspot.com/20230043 ?12:00
natefinchsinzui: great.  LGTM'd12:04
sinzuijamespage, are you about?12:42
natefinchfwereade_: do we have a list of the client commands that need to be API-ified?  I figured I'd knock one or two out14:12
fwereade_natefinch, we have a list of the ones that don't, and there are 2 of them14:40
natefinchfwereade_: haha ok14:41
natefinchfwereade_: what uses the API now, so I can see how it should be set up?14:42
natefinchfwereade_: nevermind, looks like add unit does  (I thought I remembered that one did)14:42
fwereade_natefinch, I think "get" does too14:43
natefinchfwereade_: figured I'd start on status, unless you have another suggestion14:44
fwereade_natefinch, yeah, let me chat to you about it after meeting14:44
natefinchfwereade_: sure.  I'm going to be AFK in about a half hour (for about an hour)14:45
jamnatefinch: status is going to be much bigger than some of the other ones, and might involve some checking to make sure we are getting the passwords set on first connect, etc.15:03
jamsomething like "set" would be much easier if you want to get your feet wet15:03
natefinchjam: that's fine, I can do set to start :)15:10
mattywthe env vars that get set in a running hook - like: CHARM_DIR and JUJU_UNIT_NAME - where abouts are they actually set in the code?15:27
fwereade_mattyw, uniter/context.go15:36
mattywfwereade_, cool thanks, think I just found it15:36
rogpeppefwereade_: do you know any way of configuring the local provider so that it doesn't download a whole precise image when a precise charm is started?15:52
rogpeppefwereade_: (at least, i *think* that's what it's taking ages to do - i'm not sure if there's any way of telling at all)15:53
fwereade_rogpeppe, I in't think we grabbed the whole thing every time15:55
rogpeppefwereade_: i'm not sure how to tell what it's doing15:56
rogpeppefwereade_: will we always grab the whole thing in a newly bootstrapped local env?15:56
fwereade_rogpeppe, I believe that is currently the case, but I have not looked at the code in question myself15:57
rogpeppefwereade_: ah, sorry, i thought you were the one that reviewed it15:57
rogpeppefwereade_: so no way to avoid that first download then. hmm.16:00
rogpeppefwereade_: another small question: is there a good place for a charm to stash local state that *doesn't* get managed under git? for example, a unix-domain socket or something that should not be rolled back on failure?16:05
fwereade_rogpeppe, there's no canonical location, no16:06
rogpeppefwereade_: i guess $CHARM_DIR/../charm-state might work ok, although it's not great.16:09
fwereade_rogpeppe, eww? name it after the charm outside the juju data-dir I would think?16:09
rogpeppefwereade_: or just make up a name in /var/lib which includes the env uuid and the unit name16:09
fwereade_rogpeppe, +116:10
mrammhey all: thumper's blog post about his logging library for Go is now up on hacker news: https://news.ycombinator.com/item?id=6643805 please upvote it if you think it is interesting, and feel free to comment on hacker news if you think you have something to add.16:13
natefinchmramm: if I could get to hacker news, I'd upvote it16:36
adam_gwhat is the proper way to upgrade an environment using a custom built juju and jujud binary pair? someone showed me last week but i've forgotten16:41
mgzadam_g: the magic is simply to have the juju you run to do the upgrade have the matched jujud as a sibling in the directory16:47
mgzthis is very magic, but hey go hey go go16:47
adam_gthats what i thought16:47
adam_gadding the --upload-tools flag seems to skip going to s316:48
mrammnatefinch: problems with hackernews? thumper had them yesterday too. :(16:52
natefinchmramm: looking online, it's a DNS propagation issue16:52
sodre_zzz    juju-devs: has anyone seen this type of error with Juju & RadosGW17:06
sodre_zzzcaused by: failed unmarshaling the response body: bootstrap-verify17:06
mgzsodre_zzz: can you manually fetch that object from the provider storage and see what it contains?17:07
=== sodre_zzz is now known as sodre
mgz...what provider even is that?17:07
sodreOpenStack, but Swift is served by ceph-radosgw17:07
mgzso, `swift list` to get CONTAINER, then `swift get CONTAINER bootstrap-verify`17:08
sodrealright... one sec.17:10
hazmatwallyworld_, wasn't there some facility you mentioned about auto upload-tools?17:13
sodreahhh.. the error is different!17:13
sodremgz: it says  juju was unable to list the contents of the container.17:14
mgzsodre: looking in ~/.juju/environments/ENV.jenv where ENV is the name of your environment will also tell you if you find the 'control-bucket' key at the bottom17:14
sodreLet me pastebinit17:14
sodrewasn't there a but where juju is expecting to see a .json object when listing contents in swift?17:15
sodres/but/bug/17:15
mgzsodre: we might not be parsing the error response correctly17:15
mgzso, you're seeing a follow-on error rather than the underlying cause17:15
mgzsodre: can you file a bug against juju-core for this please?17:15
sodrewill do, but can you take a look at the pastebin first ?17:16
mgzI need to transfer back home now, so would like to pick up later17:16
mgzsodre: sure, fast :)17:16
sodrehttp://paste.ubuntu.com/6336731/17:17
sodrethis is with Havana, fyi.17:17
mgzsodre: yeah, looks like we're just expecting json and getting plain text17:19
sodreyeap, the error is in goose17:20
mgzthere's a bug for this (or something very related) against goose already17:20
sodreyeap, that's what I thought.17:20
mgzif you can find that and add more details, that would be ace17:20
sodrei'll search for that.17:20
sodrethanks mgz17:20
mgzI've already got that bug on my list to look at17:20
fwereade_bbl17:32
=== paraglade_ is now known as paraglade
sodrejam: are you around >?17:44
sodreI have quick goose patch/bug17:44
natefinchsodre: it's pretty late for Jam, almost 10pm.  Anything that I could help you with?17:47
sodresure17:53
sodrethe current version of goose/swift.go17:53
sodreit requests a list of entries in a container but forgets to set the format to json17:54
sodrethe patch is very short,  http://paste.ubuntu.com/6336923/17:55
sodrenatefinch, what do you think ?17:55
natefinchsodre: looks good, except you need to gofmt your source.  You have two spaces where the standard format is a tab.17:57
sodregot it, how is that done ? I am new to go17:57
natefinchgo fmt <filename>  it'll rewrite the file17:58
sodrehttp://paste.ubuntu.com/6336944/, where should I send this to?17:59
natefinchso, in theory, you should branch lp:goose, make your change, and then propose the branch for merging to the lp:goose trunk18:00
sodregot it18:00
natefinchsodre: if you have credentials etc on launchpad, it's not too hard18:01
sodreI am new to Opensource collaboration...18:01
sodreI have an account on lp. but I've never contributed anything other than bug reporrts.18:01
sodrecan you quickly walk me through it or point me to docs with the steps.18:02
natefinchsodre: I'm pretty new to the launchpad process too.  Let me see if I can find the steps somewhere18:03
sodreokay.18:03
natefinchsodre: ok, so presuming you started off by doing a bzr branch lp:goose to get your local copy.... you can do this to commit and propose your change (from the root directory of goose on your local drive):18:07
natefinchbzr commit -m 'commit comment'18:07
natefinchbzr push lp:~sodre/goose/<branchname>18:07
natefinchbzr lp-propose lp:goose18:07
sodrecool18:09
sodreI'm filing the bug report as well.18:09
natefinchawesome18:09
natefinchsodre: thanks for the bug and for the fix.  It's really a huge help even for little fixes like this.18:10
natefinchbrb18:10
sodrenp. I am just trying to get OS running here at work.18:10
natefinchTIL: when it looks like everyone else on freenode has quit at the same time... probably it's you that has actually quit18:32
natefinchsodre: I missed anything you might have said after you said you were just trying to get OS to work.18:34
sodreI didn't say anything :)18:35
sodreI just came across the bug because I need to get OpenStack running where I work.18:35
sinzuinatefinch, will you have time today to create a a juju 1.16.2 windows package and file an Rt to get it signed?18:53
natefinchsinzui: sure will.   Hopefully this time the RT will actually be acted upon19:01
sinzuinatefinch, send me the rt number and I will track it. I am happy for a delay since I never know when the builders will complete19:02
natefinchsinzui: ok19:02
natefinchsinzui: you probably will get an email about it, but here's the link: https://rt.admin.canonical.com/Ticket/Display.html?id=6561819:17
sinzuithank you natefinch. I will add that to my task list for 1.16.219:18
natefinchsinzui: no, thank you.  I just want to make sure the windows installer stays available at the same version as the other platforms19:19
sinzuiThis one has to get to the users since they are the group that most likely will use juju on azure.19:20
natefinchmorning thumper19:43
thumpermorning natefinch19:43
* thumper feels a little knackered19:44
natefinchseems like you were just here :)19:44
thumper:)19:44
thumperyeah19:44
thumperI know that feeling19:44
thumperhowever I also have three kids still in bed19:44
thumperinstead of at school19:44
thumperthe event we went to last night was a lot later than I expected19:44
thumperdidn't get home until about 22:3019:44
natefinchthumper: doh19:44
thumperwhich is way late for the youngest two19:44
thumperif I knew it was going to be that late, I would have organised a babysitter19:45
* thumper shrugs19:45
thumperoh well19:45
natefinchhow old are your kids?19:45
thumper8, 10 and 1219:45
thumperit was a "world of wearable arts" show for the eldest19:45
natefinchsounds like fun19:45
thumperit was quite good, but went on and on19:46
thumperyou can really tell which outfits the kids did themselves, and which parents helped on19:46
thumper:)19:46
natefinchhaha yeah....19:46
jamthumper: I had some comments on your proposal, they aren't critical but it might be good to get them in if sinzui is going to be doing a 1.16.2 soon19:47
sinzuijam, the tarball was sent19:47
jamwell then, I guess it doesn't matter :)19:48
sinzuiI might be convinced to make another if there are zero downloads of it19:48
jamsinzui: I'm not a big fan of reusing numbers regardless19:48
thumperjam: I may not worry too much for the 1.16 branch, but may take them into account on the move to trunk19:49
sinzuime neither. This is case where the one packers we are delivering too is traveling19:49
thumpersinzui: how dare he?19:49
thumpertravel when we need him19:49
thumpergeez19:49
sinzuiSPoF19:50
natefinchbusfactor violation19:50
thumpertrue that19:50
jamsinzui: I *thought* that when he set up the super special PPA he did give you access. At least, I listed a short list of names19:51
jamwith you on it19:51
sinzuiI don't see any new PPA on my list...19:52
sinzuioh, let me check teams19:52
natefinchthumper: if you get time today, I have a couple branches that could use some reviewing:  ec2 instance constraints: https://codereview.appspot.com/14523052/   modification to destroy-environment to require environment name: https://codereview.appspot.com/14502070/19:53
jamI don't know if that team actually got set up, but we did discuss it19:53
thumpernatefinch: ok, I'll add to the list :)19:53
sinzuijam, thank you and your elephantine memory. I was added to a  team  8 days ago19:53
* sinzui starts scripting19:54
thumpernatefinch: so this is "juju destroy-environment amazon"19:54
natefinchthumper: correct19:54
thumpercool19:54
thumperjuju destroy-environment production19:54
thumperoops19:54
natefinchthat's basically the only change from current behavior19:54
natefinchI did add an exclamation point to the warning message, too.19:55
thumper:)19:57
thumpernice19:57
natefinchforgot there was a tweak to scanning the input... the code that was there before was actually completely borked. It happened to work, but also returned an error we were ignoring19:57
arosalesbcsaller, do you have some time to join us @ https://plus.google.com/hangouts/_/calendar/YW50b25pby5yb3NhbGVzQGNhbm9uaWNhbC5jb20.tj9cngmc3p25r5nvbdthk5up8c20:07
sodresinzui: have you already fixed 120900320:14
sinzuisodre, no, no work has been done on it yet20:18
sinzuisodre, sorry, work has started on it and iI need to target it20:19
sodreokay. I've placed the patch on a branch for review. It is one-line.20:21
sodreI found the bug through a different route, so I did not notice it was duplicate of your bug until a few minutes ago.20:22
thumpersinzui: good news, adam_g confirmed my work on maas, it makes lxc containers work (FSVO work)20:22
sinzuiI hit the mysql+apparmor bug while testing local provider today. I almost stalled the releases. I am glad I remembered the bug20:23
thumpersinzui: what's that bug?20:31
sinzuithumper, https://bugs.launchpad.net/juju-core/+bug/123699420:34
_mup_Bug #1236994: Mysql doesn't deploy on local provider due to apparmor <juju-core:Incomplete> <mysql (Juju Charms Collection):New> <https://launchpad.net/bugs/1236994>20:34
thumperwha?20:34
thumperit used to work, what broke?20:34
sinzuithumper, I think the issue is intermittent. I think I deployed that last night when I confirmed my local provider worked20:37
jamespagesinzui, I am now - I guess you need me to push 1.16.2 everywhere right?21:05
natefinchI'm outta here. night all21:08
thumpernight natefinch21:08
jamespagesinzui, don't expect armhf for quantal and raring btw21:29
jamespagewe are missing the dpkg magic fix in those series right now21:30
sinzuiquantal is dead to me21:30
sinzuimaybe we should stop releasing quantal packages. I think users have had 6 months to get to raring21:31
jamespagesinzui, not really - raring expires in 2 months21:32
jamespage:-)21:32
jamespage9 month support these days21:32
jamespagesinzui, I'd prefer we keep publishing - I still have to test using juju-core for those series21:32
sinzuiyeah, but I think juju-core has tried to give users an extra 3 months.21:32
jamespagebut just not fuss about armhf21:32
jamespagesinzui, where do you want to pull trusty binaries from? I've not uploaded to ppa for that series?21:33
jamespageI can do - the distro package will supercede anyone actually running on trusty once it lands21:34
thumpersinzui, jamespage: FWIW +1 on not caring about quantal and raring armhf21:35
sinzuijamespage, this is the script we use for releases and CI. We pull from 3 archives, including Ubuntu: http://pastebin.ubuntu.com/6337941/21:36
sinzuijamespage, I think trusty will work. I need to switch to Lp API to get packages from the new archive.21:37
jamespagesinzui, should do21:37
jamespageI'll leave it off the backports like last cycle then21:37
sinzuisorry lines 197  start where I define the archives to search21:37
jamespagesinzui, OK - its nearly baked - everything is built - just waiting for LP to publish to ppa.launchpad.net21:48
sinzuifab. I am rushing to finsh my script while getting children ready for Halloween21:49
jamespagesinzui, oh fun21:49
jamespagesinzui, bear in mind that http://archive.ubuntu.com/ubuntu/pool/universe/j/juju-core/ won't get the armhf bits21:53
sinzuiugh21:54
jamespagesinzui, trying to remember where they appear21:54
jamespageits ports.ubuntu.com or something21:54
jamespagesinzui, http://ports.ubuntu.com/pool/universe/j/juju-core/21:55
sinzuijamespage, will we do this for the final release of trust?21:55
jamespagesinzui, sorry - do what?21:56
* jamespage thinks his brain is not quite working right21:56
sinzuiwill armhf always be in ports for trust?21:57
jamespagesinzui, yes21:58
jamespageI think so21:58
jamespagesinzui, might be easier to just pull everything from the PPA21:58
sinzuiMy mind has left the building. I look forward to a walk in the dark with people dressed weirder that me21:58
sinzuiDo my eyes deceive me, I think everything is int eh staging ppa22:02
jamespagesinzui, yes - its all done22:05
sodresinzui: should I change #1209003 to Fix Commited ? I have placed a pull request already.22:06
_mup_Bug #1209003: juju bootstrap fails with openstack provider (failed unmarshaling the response body) <openstack> <Go OpenStack Exchange:In Progress by psodre> <juju-core:Triaged> <https://launchpad.net/bugs/1209003>22:06
thumperman...22:19
thumperI'm normally a bit drained on Fridays due to the late meetings22:19
thumperbut actually coding after the meetings to fix the critical bug as knocked me out22:20
* thumper turns on the coffee machine22:20
jamespagesinzui, I'm dropping for today22:20
jamespagesinzui, if you need me todo anything tomorrow am my time ping me a mail22:20
sinzuiThank you for your time jamespage22:20
jamespagesinzui, np - ditto22:21
bigjoolsthumper: a good coffee fixes everything22:24
* thumper looks for wallyworld_22:43
wallyworld_yes?22:44
thumperhey22:44
wallyworld_ho22:44
thumpergot time to chat?22:44
wallyworld_ok22:44
thumperhttps://plus.google.com/hangouts/_/72cpimmkjidp1clo6ipgc9gg90?hl=en22:45
adam_gis bootstrapping an environment /w 1.16.2-precise-amd64 expected to pull 1.16.0 tools or 1.16.2 down from S3?22:45
thumperadam_g: I believe it matches on major and minor22:45
thumperadam_g: to be sure --upload-tools22:45
adam_gthumper, seemed to have only matched major. ill roll with --upload-tools for now22:46
=== thumper is now known as thumper-afk
jcastrohttp://askubuntu.com/questions/369263/juju-bootstrap-fails-with-temporary-failure-in-name-resolution-using-amazon-aw23:41
jcastrodid we break a bucket?23:41

Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!