/srv/irclogs.ubuntu.com/2014/03/14/#juju-dev.txt

* Makyo is away: EoD00:17
* thumper has another black eye00:25
thumperI must really get better defence00:25
thumperhi axw00:51
thumperaxw: do you know how I can test upgrading?00:51
thumpernever done it before00:51
axwthumper: test upgrading? you mean the upgrade steps?00:52
axw(hi btw)00:52
thumperaxw: no, looking at bug 1291207 and I want to upload new tools with an incremented version00:52
_mup_Bug #1291207: juju-run symlink is broken after upgrade-juju <run> <upgrade-juju> <juju-core:Triaged by wallyworld> <https://launchpad.net/bugs/1291207>00:52
thumperand test the unpacking process00:52
thumperI think I know where the problem is, but want to confirm00:52
axwthumper: oh, just "juju upgrade-juju --upload-tools"00:53
axwthat's all I did to trigger the bug00:53
axwthen I tried to "juju run" on one of the machines and found it failed00:53
thumperkk00:53
thumperta00:53
axwlooks like another red eye :(01:08
thumperaxw: fix for  bug 1291207, pretty simple https://codereview.appspot.com/7568004301:30
thumperaxw: ta01:30
thumperwwitzel3: working, browsing, or just can't leave well enough alone?01:30
_mup_Bug #1291207: juju-run symlink is broken after upgrade-juju <run> <upgrade-juju> <juju-core:In Progress by thumper> <https://launchpad.net/bugs/1291207>01:30
wallyworld_axw: out of interest, have you talk to william or anyone to get buy in for the need to be able to specify the principal unit assigned to a machine. just curious to ensure there's not a chance stuff might need to be reworked if not considered the direction we need to go01:45
wallyworld_i don't have an opinion myself01:46
axwwallyworld_: it's slightly different from what he and I originally discussed, so I will get his okay before landing anyway01:55
wallyworld_ok01:55
axwwallyworld_: this is the simplest approach and can be backed out relatively easily, but there will be changes needed to support Availability Zones for ec201:56
wallyworld_ok, i'll review it condiitonal on william's +101:56
axwthanks01:57
sinzuiwallyworld_, axw, thumper: did you ever get access to the ppc machines?02:10
thumpersinzui: yes02:10
wallyworld_sinzui: i haven't tried yet02:10
sinzuifab02:10
wallyworld_but i think i'm good to go02:10
axwyup we all have batuan and vms02:10
thumpersinzui: I have a few branches to land before 1.17.502:11
thumpersinzui: what is your eta?02:11
sinzuiokay. I heard you don't need batuan. I don't use it. But I was locked out for a few days. Juju CI has its own gateway, and I can offer it if needed to get to the machines02:11
sinzuithumper, any bug targeted to 1.17.5 is a blocker, so I have no eta02:12
thumpersinzui: hmm...02:12
thumperwell I'm landing a fix for one of the three outstanding02:12
sinzuithumper, I could release tomorrow without rogpeppe 's branch, but that would need to land next week for 1.18.002:12
thumperthe other two are in progress by dimiter and rog02:12
thumpersinzui: ack02:13
sinzuithumper, I think dimiter's bug looks like a requirement. Do you agree?02:13
thumperlet me look02:14
thumperhmm..02:14
thumperI'd say dimiter's is more important but not sure if it should block it or not02:14
sinzuithank you thumper. I will review the state of things when I wake. I do want to release what we have as 1.17.502:15
thumperI vote yes, but check with william and john02:16
thumperwe should go for a majority02:16
thumperif they both say no, then go with no02:16
thumperaxw: how long do you think to write 'juju unset-env' ?02:17
axwthumper: 1/2 day maybe02:17
sinzuiLooks like CI is desperate to make the backup-restore test pass. I'll disable it since it has never passed02:17
* thumper needs coffee02:17
thumpersinzui: or move it to another CI run02:18
thumpermaybe one day it'll pass :-)02:18
thumpersinzui: can I get you to add a particular CI test?02:18
thumpersinzui: after upgrading an environment, run "juju run --machine 1 hostname"02:18
thumperit shouldn't fail :-)02:19
sinzuithumper, the test has never passed since I wrote it. in fact I really cannot restore a machine and we have had too. canonistack lost CI a few weeks ago02:19
sinzuibug https://bugs.launchpad.net/juju-core/+bug/1291022 details that we have not been able to restore on openstack or ec202:20
_mup_Bug #1291022: Cannot restore a state-server on ec2 and openstack <backup-restore> <ec2-provider> <hp-cloud> <regression> <juju-core:Triaged> <https://launchpad.net/bugs/1291022>02:20
thumperhmm...02:20
sinzuiThough I should update the bug with an example of the what we see from start to finish02:21
wallyworld_sinzui: thumper: dimiter's bug is a blocker imho because upgrades break02:21
thumperwallyworld_: ack02:21
sinzuiwallyworld_, yeah. my sense was is that you could upgrade once to 1.17.5, then never upgrade again02:22
wallyworld_yep that' it02:22
wallyworld_i partially fixed it02:22
wallyworld_but not all attributes that needed setting, only datadir02:23
wallyworld_and my fix was not for local, but other providers02:23
wallyworld_dimiter wrote the original code so he is running with it02:23
thumperwallyworld_: https://codereview.appspot.com/7570004302:32
thumperwallyworld_: nm, axw did it02:33
axwwallyworld_: thanks for the reviews, I've sent an email to William to get his OK02:34
axwand will address those comments now02:35
wallyworld_np02:36
davecheneysinzui: what is the current ip of the jenkins server ?02:41
sinzuidavecheney, http://ec2-54-84-137-170.compute-1.amazonaws.com:8080/02:42
sinzuidavecheney, Sorry, I need to send am email about that a juju reports, with a request for a friendly host name too02:43
thumperdavecheney: do you have a doc listing the steps needed to compile juju core on ppc?02:44
davecheneysinzui: no worries02:46
davecheneythanks02:46
davecheneythumper: yes02:46
davecheneyi shared it with you02:47
davecheneythumper: https://docs.google.com/a/canonical.com/document/d/1m9R2n6LPLNLGjdopcNkQYVG8D5V4FTyvc1vvn-9ZifM/edit02:47
axwthumper: should Azure be in the availability mode by default?02:58
* axw thinks the answer is yes02:58
davecheney=== RUN TestDeepEqual03:00
davecheney--- FAIL: TestDeepEqual (0.00 seconds)03:00
davecheney deepequal_test.go:106: deepEqual(map[2:two 1:one], map[2:two 1:one]); unexpected error "mimatch at top level: type mismatch map[uint]string vs map[int]string; obtained map[uint]string{0x2:\"two\", 0x1:\"one\"}; expected map[int]string{2:\"two\", 1:\"one\"}", want "mismatch at top level: type mismatch map\\[uint\\]string vs map\\[int\\]string; obtained map\\[duint\\]string\\{0x1:\"one\", 0x2:\"two\"\\}; expected map\\[int\\]string\\{203:00
davecheney "github.com/juju/testing/checkers"03:00
davecheneyprobably map ordering expectatoin03:01
axwyup03:03
axw0x1/0x2 are swapped03:04
davecheneyhttps://github.com/juju/testing/issues/103:04
axwI thought gc randomised map range03:05
axwdavecheney: does it not?03:05
davecheneyaxw: it does03:05
davecheneybut technically that is an implementatoin detail03:06
axwI guess not random enough03:06
davecheneyright03:06
axwyeah I know03:06
davecheneybut depending on something that is random sounds03:06
davecheney1. hard03:06
davecheney2. wrong03:06
axwyeah I just wondered why it wasn't failing before03:06
davecheneyright now, small maps, less than 64 bytes are not that random03:07
davecheney1.3 will make them more random03:07
axwokey dokey03:07
davecheneyi lost that argument03:07
davecheneynobody should depend on map ordering03:07
davecheneythumper: some good news03:24
davecheneythe go http client _does_ obey no_proxy03:24
davecheneyubuntu@winton-02:~/src$ go run noproxy.go03:24
davecheney&{403 Forbidden 403 HTTP/1.0 1 0 map[Connection:[close] X-Squid-Error:[ERR_ACCESS_DENIED 0] X-Cache-Lookup:[NONE from batuan.canonical.com:3128] X-Cache:[MISS from batuan.canonical.com] Date:[Fri, 14 Mar 2014 03:23:05 GMT] Content-Length:[1173] Content-Type:[text/html] Server:[squid/2.7.STABLE7] Via:[1.0 batuan.canonical.com:3128 (squid/2.7.STABLE7)]] 0xc210427fc0 1173 [] true map[] 0xc2104331a0} <nil>03:24
davecheneyubuntu@winton-02:~/src$ env no_proxy="google.com" !!03:24
davecheneyenv no_proxy="google.com" go run noproxy.go03:24
davecheney^Cexit status 203:24
davecheneyubuntu@winton-02:~/src$ go run noproxy.go03:26
davecheney&{403 Forbidden 403 HTTP/1.0 1 0 map[Connection:[close] X-Squid-Error:[ERR_ACCESS_DENIED 0] X-Cache-Lookup:[NONE from batuan.canonical.com:3128] X-Cache:[MISS from batuan.canonical.com] Date:[Fri, 14 Mar 2014 03:25:57 GMT] Content-Length:[1169] Content-Type:[text/html] Server:[squid/2.7.STABLE7] Via:[1.0 batuan.canonical.com:3128 (squid/2.7.STABLE7)]] 0xc210427fc0 1169 [] true map[] 0xc2104331a0} <nil>03:26
davecheneyubuntu@winton-02:~/src$ env no_proxy="10.0.3.1" go run noproxy.go03:26
davecheney<nil> Get http://10.0.3.1:80/: dial tcp 10.0.3.1:80: connection refused03:26
davecheneyand it looks like it works for ip addrsses03:26
wallyworldthumper: guess what? i updated my packages and now unity crashes, i have no mouse, and wireless networking is broken \o/03:26
davecheneyso i think it might be custom http transports inside jju that are the problem03:26
davecheneywallyworld: did you expect something else ?03:27
wallyworldwell yes :-(03:27
wallyworldwe are pretty close to release03:27
wallyworldi'm not sure what to do except reinstall beta 103:27
wallyworldcause not having a mouse kinda sucks03:27
davecheneywallyworld: fuck, are we at beta 103:28
davecheneythat is terrible03:28
wallyworldand, well, networking is important too03:28
davecheneyi can still hear you03:28
davecheneycan't be that bad03:28
wallyworldi'm on wired03:28
wallyworldso i'm tied to a corner next to my router03:28
wallyworldnot very comfortable03:29
axwwow, I think I might not update for a bit03:31
axwseeing as I have no rj45 and no adapter03:31
wallyworldit could just be my hardware or something03:31
davecheneywallyworld: what wifi ?03:32
davecheneyintel or atheros ?03:32
wallyworldhmmm. can't recall. intel maybe03:32
davecheneyerk03:33
thumperwallyworld: awesome...03:42
wallyworldyeah, tell me about it03:42
thumperwallyworld: I'd say wait a bit, do another apt-get update/dist-upgrade03:42
thumperand reboot03:42
thumperthat's what fixed it for me03:42
thumperjust package skew as I was downloading03:42
wallyworldtried that, may have to wait a bit more03:42
thumperkk03:42
thumperdavecheney: https://github.com/juju/testing/pull/203:59
axwthumper: OT question, have you flown UA? are they ok?04:00
thumperaxw: not that I recall04:00
thumpermay have for internal flights04:00
thumperaxw: is this a long flight?04:00
thumperI have heard they aren't that good for long haul04:01
axwthumper: yeah. it's $500 more for QF :/04:01
davecheneyaxw: DO NOT FLY UA04:01
davecheneyremember how when you were 5 and your parents took you on your first trip04:02
davecheneyUA ARE STILL FLYING THAT VERY PLANE04:02
axwhehe04:02
thumperdavecheney: how do you run go on ppc04:02
thumperI installed gccgo-4.904:02
thumperbut that didn't install a go binary04:02
thumperdavecheney: are you in the github.com/juju team?04:03
davecheneyhttps://docs.google.com/a/canonical.com/document/d/1m9R2n6LPLNLGjdopcNkQYVG8D5V4FTyvc1vvn-9ZifM/edit04:04
davecheney$ sudo apt-get install gccgo-4.9 gccgo-go04:04
davecheneydon't forget the second package04:04
davecheneythumper: NFI04:04
thumperdavecheney: can you comment on the pull request above?04:04
davecheneyyeah, looks like I can04:05
thumperoh yeah, "ssh rock<tab>"04:05
davecheneyWHOOT04:06
thumperdavecheney: I don't see a comment from you...04:07
davecheneygot a little bit closer to deploying ubuntu charm on ppc04:07
davecheneythumper: i didn't write anything04:07
* thumper wants a LGTM04:07
davecheneyoh, lemmie test it04:07
davecheneyhttp://paste.ubuntu.com/7088244/04:10
thumperdavecheney: tested with cgo and gccgo locally (on amd64)04:10
davecheneyinching closer04:10
thumperdavecheney: ah, no aufs on ppc?04:11
thumperdavecheney: export JUJU_TESTING_LXC_FORCE_SLOW=true04:12
davecheney14:58 < davecheney> a send to a nil channel blocks forever04:13
davecheney14:58 < davecheney> a send to a closed channel panics04:13
davecheney14:59 < davecheney> a receive from a nil channel blocks forever04:13
davecheney14:59 < davecheney> a receive from a closed channel returns the zero value immediately04:13
davecheneyoops04:13
davecheneythumper: the trick is i need to export no_proxy=... during bootstrap04:13
davecheneyso those values are passed to the bootstrap agent04:13
davecheneyit took a while to figure out who was making the http call04:13
davecheneyalso, there will be no precise image for ppc6404:14
davecheneyubuntu@winton-02:~/src/launchpad.net/juju-core$ juju deploy cs:trusty/ubuntu04:15
davecheneyERROR charm not found: cs:trusty/ubuntu04:15
davecheneyfuck me04:15
thumperso make one dear liza04:15
davecheneythe whole trick is I cant/shouldnt use a local charm repo04:16
thumperdavecheney: flick the aufs lxc clone bug at hallyn04:16
davecheneythumper: http://paste.ubuntu.com/7088260/04:17
davecheneysetting that setting doesn't appear to set anything04:17
thumperdavecheney: if we need to disable the aufs clone for ppc, we can make it a real setting maybe04:17
thumperdavecheney: you need it all the time04:17
* davecheney strokes non existant beard04:17
* thumper thinks04:17
thumperhmm...04:17
davecheneylemmie ask hallyn04:17
thumperpoo04:17
thumperdavecheney: I'm not propogating that env var into the upstart job04:18
thumperdavecheney: to really test it, we'll need it stored as a setting04:18
davecheneythumper: which package is it in ?04:18
davecheneyi'll hack the codeze04:18
thumperif you want to hack the codeze, just edit your upstart script04:18
thumperfor jujud04:18
davecheneyok04:19
davecheneyimma going to raise a bug04:19
davecheneythat we can throw at ahllyn04:19
thumperah...04:19
thumperlet me think04:19
thumperdavecheney: what shall we call the local provider setting04:19
thumperlxc-no-clone ?04:20
thumperor lxc-clone04:20
thumperdefault to true04:20
thumperallow false04:20
thumperactually, default to true IFF trusty04:20
thumperdavecheney: are you testing that github.com/juju/testing branch?04:23
davecheneythumper: not right now04:24
thumperkk04:24
thumperdavecheney: I'm fetching it...04:26
thumperjust to confirm on ppc, I know it passes with gccgo locally04:26
thumperdavecheney: http://paste.ubuntu.com/7088314/04:35
davecheneylgtm04:36
davecheneyi was getting there04:36
thumperthat's ok04:36
thumpermerged04:36
* thumper updates juju-core dep04:37
davecheneythumper: confirmed working on ppc04:38
davecheneythanks04:38
thumpernp04:39
thumperthat's what I'm here for :-)04:39
davecheneythumper: https://bugs.launchpad.net/juju-core/+bug/129234604:39
_mup_Bug #1292346: juju deploy fails on ppc64el because aufs is not available <ppc64el> <juju-core:Triaged> <https://launchpad.net/bugs/1292346>04:39
davecheneyfails on both precise and trusty04:39
davecheneydeploy 'said images04:39
thumperok04:40
thumperdid you hack up the upstart script so it didn't try to use clone?04:40
thumperto test normal lxc?04:41
davecheneylet me do some hacking04:41
davecheneyi was distracted twisting marcoceppi 's arm to get a trusty ubuntu charm in the store04:41
thumperaxw or davecheney: https://code.launchpad.net/~thumper/juju-core/update-testing/+merge/21096004:42
axwthumper: done04:42
thumperta04:42
marcoceppithumper: I'm about to compile juju to test the local provider, anything I should know before going in to it?04:43
thumpermarcoceppi: depends04:43
thumperwhat filesystem do you have at /var/lib/lxc04:43
marcoceppiext404:43
thumperand I'm assuming you are on trusty04:43
marcoceppisaucy04:44
thumperha04:44
thumperwon't work04:44
thumpertrusty only baby04:44
marcoceppibut I can be on trusty pretty quickly04:44
thumpermarcoceppi: ok, what it will do is the first time you deploy a charm for a series, it will create a template container 'juju-<series>-template" like juju-precise-template04:45
thumperdon't touch it04:45
davecheneythumper: marcoceppi http://paste.ubuntu.com/7088331/04:45
thumperfirst time it starts it, and waits for cloud init to finish and shut it down04:45
davecheneyoooh04:45
davecheneyohh04:45
davecheneymaybe04:45
thumperthen it just clones to create machines04:45
thumperusing aufs04:45
davecheney  "1":04:45
davecheney    agent-state: pending04:45
davecheney    instance-id: ubuntu-local-machine-104:45
davecheney    series: trusty04:45
davecheney    hardware: arch=ppc6404:45
axwooh :)04:46
thumperdavecheney: ok, it is trying :)04:46
davecheneyhttp://paste.ubuntu.com/7088333/04:46
davecheneygetting very close04:46
davecheneythumper: do the agent states move to started in lxc ?04:48
thumperyes04:48
davecheneyok, so it isn't working then ...04:49
thumperwhen they are running and talking to the api server04:49
thumperit will take some time04:49
davecheneynope, no agents, http://paste.ubuntu.com/7088341/04:49
thumperyou can watch progress by looking in /var/lib/juju/containers/ubuntu-local-machine-104:49
thumperand look at the console.log04:49
davecheneyOH NO04:50
davecheneyi know what it is04:50
thumperdavecheney: cloud-init may not have finished04:50
davecheneyit's finsihed04:50
thumperwat?04:50
davecheneyno running processes04:50
davecheneygccgo binaries need a library installed04:50
axwtake a look in /var/log/cloud-init-output.log04:50
axwugh04:50
davecheneylibgo5.so04:50
thumperugh04:50
thumperdavecheney: hmm...04:51
axwenvirons/cloudinit/cloudinit.go is gonna need another AddPackage04:51
thumperdavecheney: checking a few tests, juju-core/container passes on amd64 with both cgo and gccgo04:51
thumperfails on ppc ggcgo04:51
davecheneylet me get somet details04:51
davecheneyis there a simple way to go into an lxc console ?04:52
davecheneylxc-enter or something ?04:52
thumperyes...04:52
thumperssh04:52
axwdavecheney: you should be able to ssh into it04:52
davecheneyoh04:52
davecheneytoo easy04:52
thumper:)04:52
thumperssh into it,04:52
thumperinstall package04:52
thumperreboot container04:52
davecheney$ juju ssh ubuntu/004:52
davecheneyERROR unit "ubuntu/0" has no public address04:52
davecheneybzzzzzzzzzzzzt04:52
axwtry the machine04:52
thumperlxc-ls --fancy04:53
davecheneyyeah, that works04:53
davecheneyactually no04:53
davecheney$ juju ssh 104:53
davecheneyERROR machine "1" has no public address04:53
thumperno unit agent to set it04:53
davecheneyyup04:53
thumperlxc-ls --fancy04:53
davecheneycatch 2204:53
thumperget the ip address04:53
thumperssh ubuntu@10.0.3.x04:53
davecheneylxc-ls --fancy04:53
davecheneyNAME  STATE  IPV4  IPV6  AUTOSTART04:53
davecheney----------------------------------04:53
davecheneyzip04:53
thumperah...04:53
thumperwat04:54
thumpersounds like lxc-create failed04:54
davecheneythumper: what is your ssh key ?04:54
thumperon lp, elwood04:54
axwtry as sudo04:54
axwdavecheney: ^^04:55
thumperaxw: yeah...04:55
thumperotherwise it is just the user space lxc04:55
thumpersorry didn't specify04:55
davecheneyahh, that works04:55
davecheneyFUK<04:55
davecheneykey errors now04:55
davecheneyscrew this04:55
davecheneyi'll just chroot04:55
thumperkey errors?04:55
thumperreally?04:55
thumperthat should have worked04:56
thumperah...04:56
thumperif you don't have a local ssh there04:56
thumperit will be using the one it generated04:56
thumperaxw: how does he specify?04:56
thumper-i04:56
thumper?04:56
axwdavecheney: use "-i ~/.juju/ssh/juju_id_rsa"04:56
axwI think that's what it's called04:56
davecheneyaxw: ahh yeah04:56
axwyep04:56
davecheneyman, all this stuff juju ssh does for you04:56
davecheneymade me soft04:56
axwmaybe we should allow "juju ssh" to take an IP for this scenario04:57
thumperyeah...04:57
axw:q04:57
axwoops04:57
davecheney$ /var/lib/juju/tools/machine-1/jujud04:58
davecheney/var/lib/juju/tools/machine-1/jujud: error while loading shared libraries: libgo.so.5: cannot open shared object file: No such file or directory04:58
* thumper is waiting...04:58
davecheneygot it04:58
davecheneyyup04:58
davecheneythat was what I though04:58
davecheneyaxw: maybe04:58
thumperfuck yeah!04:58
davecheneythe local provider is a bit of a special case04:58
thumpercloser...04:58
thumperaxw: so... how do we conditionally install the libgo package?04:59
axwthumper: probably just test the arch MachineConfig.Tools04:59
thumperhmm...04:59
axwthumper: otherwise I guess we'd need to encode the toolchain in the version string05:01
axwwhich isn't really practical05:01
thumperno...05:01
thumperunless...05:01
thumperwhat about making a meta-package05:01
thumpercalled 'juju-deps'05:02
thumperthen on ppc that includes libgo05:02
thumperand on others it doesn't05:02
axwI suppose so05:02
thumperwe could then just install the meta-package05:02
axwalso arm6405:02
thumperand not worry about specifics05:02
thumperright05:02
thumperpossible at least05:02
thumperdavecheney: what do you think?05:03
thumperworth raising with jamespage?05:03
davecheneythumper: it's sort of worse than that05:03
davecheneythere is no libgo package05:03
davecheneylibgo.so.5 comes from gccgo itself, the compiler package05:04
davecheneywhich pulls in a tonne more05:04
davecheneyhttps://bugs.launchpad.net/juju-core/+bug/129235305:04
thumperouch05:04
_mup_Bug #1292353: juju must install the libgo.so.5 library when required <ppc64el> <juju-core:Triaged> <https://launchpad.net/bugs/1292353>05:04
davecheneyprobably need to include more project here05:04
axwdavecheney: should we just distribute it with jujud?05:04
thumperaxw: we'd need to update the LD_LIBRARY_PATH too right?05:05
davecheneyaxw: IMO that will make a bad problem worse05:05
axwyeah05:05
davecheneylike thumper says05:05
davecheneywe'd get into the LD_LIBRARY_PATH business05:06
axwor rpath shenanigans, but that's not any better05:06
davecheneyyou can do elfedit fuckery05:06
thumperdavecheney: we have to work out something...05:06
thumpernah don't edit things05:06
thumperthat's terrible05:06
davecheneylemmie raise a bug on gccgo05:06
davecheneyprobably not a big job to get it split out into a runtime pkg05:06
davecheneythumper05:06
thumperwe need libgo.so in a different package05:06
davecheneyif I do apt-get install X y z05:06
davecheneyand z doesn't exist05:06
davecheneywhat does apt do ?05:06
thumperI think it fails05:07
thumperbut not entirely sure05:07
davecheneythumper: you can see where I'm going with this ?05:07
thumperno05:07
davecheneyaxw: the good news is we'll get a libgccgo package05:07
davecheneywhich will be a meta for libgccgo-4.905:07
axwcool05:07
davecheneyjust like gccgo suggests gccgo-4.905:07
davecheneyso we don't need to get too worked up on versions05:07
thumperaye05:08
davecheneythumper: I was thinking of always installing libgccgo [sic]05:08
davecheneyandignoring any errror05:08
davecheneyif it's hard to sniff the target series/arch05:08
axwwell it does mean we've got a new binary compatibility issue to deal with05:08
davecheneyaxw: many, many05:08
davecheneyaxw: i did raise this with many people 6 months ago05:08
davecheneywe're backing into a massive testing matrix05:08
axwdavecheney: it's not hard to tell the target arch05:09
axwwe can get that when we generate the cloud-init script05:09
* thumper is EOWing05:13
thumperlater peeps05:13
davecheneyhttps://bugs.launchpad.net/ubuntu/+source/gccgo-4.9/+bug/129235505:15
_mup_Bug #1292355: gccgo needs a runtime package <ppc64el> <gccgo-4.9 (Ubuntu):New> <https://launchpad.net/bugs/1292355>05:15
davecheneyo/ TheMue05:15
davecheneythumper05:15
marcoceppiwelp, compiled core from trunk, but got a weird error05:17
marcoceppi.go/src/launchpad.net/juju-core/provider/azure/environ.go:539: cannot use []gwacl.ConfigurationSet literal (type []gwacl.ConfigurationSet) as type *gwacl.OSVirtualHardDisk in function argument05:17
marcoceppiso, no testing local provider for me, you guys are on your own05:17
axwmarcoceppi: you need to update gwacl05:18
* axw gets the rev05:18
axwmarcoceppi: bzr update -r 23105:19
axw(in launchpad.net/gwacl)05:19
marcoceppiOH haha05:20
marcoceppiI did that in juju-core05:20
marcoceppiit was not happy05:20
axwthat would be quite old :)05:20
marcoceppihaha, yeah05:21
davecheneysuch wrong channel05:22
davecheneyif this works, i'm going to the pub05:23
davecheneyif this doesn't work, i'll probably go to the pub05:24
davecheneywin/win05:24
davecheneyWORKED!05:25
axwwinner :)05:25
davecheneyhttp://paste.ubuntu.com/7088436/05:25
davecheneymarcoceppi: http://paste.ubuntu.com/7088436/05:25
marcoceppiwe have trusty!05:26
davecheneymarcoceppi: a whole one charm05:27
davecheneyi told you we didn't to test it05:27
davecheneythat charm is a charm05:27
davecheney100% success rate05:27
marcoceppithat charm sure is a charm05:27
marcoceppijust like all the other charms05:27
davecheneyit's a perl05:27
marcoceppibut it's defintely more charmy05:27
marcoceppiyay, it still didnt' compile05:29
marcoceppibunch of signal killed errors05:29
marcoceppioh well, I'll just wait for 1.17.505:29
davecheneycompilation is for loosers05:29
davecheneywinners write in binary05:29
marcoceppiI use magnets and write directly to tapes05:30
davecheneyi use a hammer and a punch and write them directly onto roms05:30
* marcoceppi drops the AIX and walks away05:30
* davecheney bows to marcoceppi 05:30
rogpeppemornin' all07:35
wallyworld_axw: i ended up having to reformat / since first install complained packages were corrupt :-( now i need to set up stuff again. sigh07:53
axwoh shit :(07:53
wallyworld_at least i have a clean install now :-)07:53
axwthat is true. I did consider doing it myself, but cbf at the time07:53
wallyworld_and i backed up stuff first from command line07:53
wallyworld_so no data loss, jut time07:54
axwbit delayed, but morning rogpeppe07:54
rogpeppeaxw: hiya07:54
rogpeppeaxw: i've been meaning to create a bug for this, but i should mention it to you: when bootstrap in the local provider fails, the error is often not very informative these days - it's just "rc: 1"07:57
axwrogpeppe: #127971007:57
_mup_Bug #1279710: failed bootstrap discards stderr <bootstrap> <juju-core:Triaged> <https://launchpad.net/bugs/1279710>07:57
axwthat's because local provider now uses the script stuff, like everything else07:57
rogpeppeaxw: ah, good - you're aware of it07:58
axwI am, just don't have a good solution yet07:58
axwI intend to fix it before 2.0 though for sure07:58
rogpeppeaxw: yeah, i didn't see any solution that jumped out at me, though i didn't look for that long07:58
rogpeppeaxw: cool - it made things quite hard to debug at times07:59
axwrogpeppe: what caused local bootstrap to fail anyway?08:04
axwor you don't know? :)08:04
* rogpeppe tries to remember08:04
rogpeppeaxw: sorry, can't remember - some bug in the branch that was being tested, ISTR08:05
axwrogpeppe: no worries08:06
* rogpeppe is an idiot08:16
rogpeppevladk: good morning!08:20
vladkrogpeppe: good morning!08:20
* rogpeppe struggles to work out how to submit a git pull request08:53
vladkrogpeppe: I had questionable experience with github pull request08:57
rogpeppevladk: :-)08:57
rogpeppevladk: so many concepts08:57
vladkyou may PR the whole branch, but it will be merge completely if you add smth after PR08:59
vladkif you create a PR between two commit, it may produce unpredictable merge08:59
vladkand always head of PR should be on master branch09:02
rogpeppevladk: i've been trying to do something like this: http://paste.ubuntu.com/7089049/09:02
vladkI would not use MERGE button on PR and do manual commits09:02
rogpeppevladk: but when i go to create the pull request on github, it says there's no difference between my branch and master09:03
vladkyou should push before09:03
rogpeppevladk: before pushing the changes?09:04
vladkrogpeppe: you create PR on github, so you should push your commits to github before09:05
rogpeppevladk: before creating the pull request?09:05
rogpeppevladk: i did do that, i think09:05
vladkrogpeppe: may I look your repo on github?09:05
rogpeppevladk: (i haven't been able to create a pull request yet)09:06
rogpeppevladk: sure09:06
rogpeppevladk: this is what i'm looking at currently: https://github.com/go-yaml/yaml/compare/master...rename-imports09:06
rogpeppevladk: the  branch i want to merge is rename-imports09:06
rogpeppevladk: and i want to create a pull request into github.com/go-yaml/yaml09:07
vladkrogpeppe: I don't see any changes in remote-imports09:08
rogpeppevladk: ah!09:09
vladktry 'git push origit' w/o branch name09:09
rogpeppevladk: i think my "git branch foo" created a new branch but didn't switch to it09:10
rogpeppevladk: hmm, but i still committed something, so i'd have expected that to push09:11
rogpeppevladk: i'm worried that if i do that, then i'll push directly to the master09:11
vladkrogpeppe: it's quite easy to delete recent commits from github, so try, don't worry09:14
rogpeppevladk: after reading the push docs, i *think* i see what's going on now09:14
vladkrogpeppe: you should 'git checkout' after 'git branch'09:15
vladkrogpeppe: of 'get checkout -b my-branch-name'09:15
rogpeppevladk: yes, that's the conclusion i've come to09:16
rogpeppevladk: and i should do git push origin :remote-branch-name09:16
rogpeppevladk: (i think)09:16
rogpeppevladk: i was missing the colon09:16
rogpeppevladk: which meant that i was naming a local branch09:16
vladkrogpeppe: probably 'git push origin :remote-branch-name' push change in one specific branch, while 'git push origin' push commits for all branch, but I always used 'git push origin'09:18
rogpeppevladk: i eventually succeeded with "git push --set-upstream origin rename-imports"09:21
* fwereade said yesterday that he thought he'd escaped the flu... yeah, not so much09:35
* fwereade will try to be around off and on but his brain isn't really working very well09:35
TheMuefwereade: today is the first day I get better after fighting for three days with the flu too09:39
axwfwereade: hey, I have availability sets working in my sandbox :)09:41
fwereadeaxw, awesome news!09:41
axwadd-machine and placement both disabled09:41
axwwith a mode to make it like the other providers09:41
axwjust need to do the SSH tunnelling business09:42
fwereadeaxw, and auto-load-balancing for everything in azure mode, old-style one-service-per-machine in normal mode?09:42
fwereadeaxw, cool09:42
axwfwereade: yup09:42
fwereadeaxw, fantastic09:43
fwereadeaxw, how did the environ interfaces end up? because the fact that other providers don't zone-balance is now a bug, and we should get some time scheduled to bring them up to speed :)09:43
axwfwereade: I sent a couple of CLs to you to take a look at, to see if the implementation isn't horrible09:44
axwheh09:44
axwfwereade: so the way I've done it at the moment is I tell StartInstance what the principal units are - that's all Azure needs. For ec2/etc., we'll need an additional callback to get a mapping of units->instance.Id for the services of the principals09:45
axwthen it can spread units by hand09:46
fwereadeaxw, I don't quite follow, but I'm not sure my input filters are functioning correctly -- I'll look at the CLs09:46
axwno worries, it can wait till you're feeling better anyway09:46
axwnot going to land this until 1.18 is out09:47
rogpeppelarge but totally trivial review anyone? https://codereview.appspot.com/75970043/10:10
axwlooking10:10
axwrogpeppe: why the gocheck rev bump?10:13
rogpeppeaxw: because i couldn't be bothered to make a new CL for it10:13
axwfair enough :)10:13
axwsorry didn't see the comment in the description10:13
rogpeppeaxw: it fixes a bug in gocheck10:13
axwjust making sure it was intentional10:13
rogpeppeaxw: np10:13
axwrogpeppe: done10:15
rogpeppeaxw: ta10:15
axwit would be nice if we could review gofix modules, rather than the output10:15
* axw puts it on the end of a too long list of things to do one day10:16
rogpeppeaxw: i didn't actually use a gofix module per se10:19
rogpeppeaxw: i used two tools and did one manual change10:19
rogpeppemgz_: does the bot automatically fetch dependencies now?10:20
axwrogpeppe: sure, but you get what I mean - review a small script (or two), rather than a load of files with the same change in each10:20
rogpeppeaxw: yeah, i do get what you mean, but i'm not convinced that one would actually want to trust that the script works without actually looking at the changes it made10:21
axwyeah, some level of trust would be necessary10:22
rogpeppeaxw: when reviewing a change that's made entirely automatically, i'll hope that the command used is in the CL description, and i'll look at a sample of the output and take the rest on trust10:22
rogpeppeaxw: that's how i remember similar changes being reviewed in the go source10:23
rogpeppeaxw: for example when we moved from os.Error to error10:23
axwyeah ok, same outcome and level of trust10:23
axwand no need for sandboxing and all that10:24
rogpeppeaxw: i could have included the sam command that i used to make the edit, but noone apart from me understands structural regexps :-)10:24
* axw is intrigued10:25
axwand yet, the dishes won't wash themselves10:26
rogpeppeaxw: http://plan9.bell-labs.com/magic/man2html/1/sam10:27
axwthanks, I'll take a look a bit later10:27
rogpeppeaxw: you need a dishwasher mate :-)10:27
adeuringcould somebody please have a look here: https://codereview.appspot.com/75980043 ?10:27
axwyeah :(10:27
axwI was always told I am the dish washer10:27
rogpeppeadeuring: looking10:27
voidspaceSo I got Ubuntu installed on my machine last night. And now I have horrible, horrible problems on both machines. :-)10:32
rogpeppeadeuring: LGTM, but i'm surprised that the old code "escaped" a double-underscore to a single one.10:32
voidspaceMy Mac is dying in sympathy10:32
voidspaceSo trying to get *something working10:32
rogpeppevoidspace: good luck!10:32
rogpeppeadeuring: as it looks as if all underscores were deleted10:32
rogpeppeadeuring: which leaves me wondering if there's something we're missing here10:33
adeuringrogpeppe: I do not understand this as well... I'll ask thedac (who reported the bug) for more details.10:33
adeuringI tried to use the double underscores -- no luck at all10:33
rogpeppeadeuring: with the old version?10:34
rogpeppeadeuring: maybe that's just spurious10:34
adeuringrogpeppe: I tried on saucy with the recent source code, and on precise with the deb package from the stable PPA10:34
rogpeppeadeuring: and you didn't see the "escaping" behaviour?10:35
adeuringrogpeppe: no -- double underscores were stripped the same way a single underscores10:35
rogpeppeadeuring: right, that's what i'd expect10:35
rogpeppeadeuring: BTW we're just moving to using gopkg.in/v1/yaml, so you'll need to send a pull request there10:37
adeuringrogpeppe: ah, ok...10:38
rogpeppeadeuring: the actual repo is at github.com/go-yaml/yaml10:39
axwrogpeppe: when you have a moment please, https://codereview.appspot.com/75990043/10:45
rogpeppeaxw: will do10:46
natefinchfwereade: standup10:51
mgz_dammit internet10:55
mgz_gah!11:03
voidspacerogpeppe: I'm going to try and get things working here and I'll ping you when I'm ready to do anything actually useful11:17
voidspacerogpeppe: if that's ok11:17
wwitzel3brb (hopefully), restart11:17
rogpeppevoidspace: sgtm11:17
voidspacerogpeppe: and if you could get the ratelimiter installed for the bot so I can land my branch that would be awesome11:18
rogpeppevoidspace: just doing that11:18
voidspace*great*11:18
natefinchwwitzel3: what hours do you normally work?  I tend to get little done between 7 & 9am because I have my younger daughter sleeping either in my arms or (if I'm lucky) next to me.11:27
wwitzel3natefinch: I usually try to get started at about 5:45 so I can go through email and bootstrap my memory of what I did yesterday before standup.11:28
natefinchwwitzel3: I usually do that 10 minutes before the standup ;)11:29
wwitzel3natefinch: and then I "stop" around 3 .. but all that means is I stop feeling bad about walking away .. I never actually get off my computer :P11:29
natefinchwwitzel3: haha11:29
natefinchthat's cool11:29
wwitzel3natefinch: or it means I'm hear, but I'll just pretend like I don't see someone pinging me :D11:30
wwitzel3here11:30
wwitzel3mixed thoughts, was going to say pretend I don't hear them, then switched ping, and then grammatical error .. dumb11:31
natefinchwwitzel3: ok.  I'll try to free up by 9 this morning so we can pair.11:31
natefinchwwitzel3: haha I've done that11:31
wwitzel3natefinch: sure, I have plenty to keep me busy .. there are some code reviews I'm going to peak at and i have some HR stuff I've been neglecting11:32
natefinchwwitzel3: cool, if you ever find yourself with nothing to do, take a look at the bugs11:33
wwitzel3natefinch: also after your praise of how smooth your upgrade went, I'm running that process now too11:33
natefinchwwitzel3: good luck :)11:34
voidspaceooh, I might have unity back11:36
rogpeppeaxw: reviewed12:11
rogpeppemgz_: how do you get your juju client talking to the gobot juju server? presumably you use chinstrap somehow.12:14
mgz_rogpeppe: yeah, you just want some ssh config like12:14
rogpeppemgz_: how does that help, given that juju doesn't use ssh to talk to the server?12:15
mgz_well, you probably know that bit12:15
mgz_then use sshuttle through either the bootstrap node or chinstrap12:15
* rogpeppe reads sshuttle(8)12:16
wwitzel3natefinch: well, it seemed to go well (the upgrade)12:16
natefinchwwitzel3: sweet.   lots of good stuff in Trusty12:17
mgz_eg: `sshuttle -vr rogpeppe@chinstrap.canonical.com 10.55.61.0/16 >~/sshuttle.log 2>&1 &`12:17
wwitzel3natefinch: it also just seems a bit more performant .. and my fan isn't kicking on as much as I do things12:17
rogpeppemgz_: thanks. i would have taken a while to come up with that12:17
natefinchwwitzel3: yeah, I'd heard some people had fan troubles in Saucy. I still get a fan going in hangouts sometimes, but I think that's just hangouts12:18
rogpeppemgz_: ha, that doesn't work because it tries to run sudo12:19
mgz_wat12:20
* rogpeppe doesn't background it12:20
rogpeppemgz_: ah, i'm rog@chinstrap12:20
mgz_that'll be it12:21
rogpeppemgz_: success, thanks12:21
rogpeppemgz_: why does it do the bzr binds *after* running godeps -u ?12:26
mgz_sound't really matter?12:26
mgz_*shouldn't12:26
mgz_wait, ti does12:26
rogpeppemgz_: don't the binds change the code?12:26
mgz_but that's the right way around12:27
* rogpeppe never used bzr bind12:27
mgz_bind makes that branch a checkout of a remote branch12:27
rogpeppemgz_: so it doesn't actually change any files in the branch?12:27
mgz_if you did that, then updated *that* bound branch, you'd change the rev the remote branch was on12:28
mgz_we don't actually do this12:28
mgz_because godeps doesn't change juju-core, which is what we bind12:28
mgz_but would be a bit of a bear trap12:28
rogpeppemgz_: what are the binds doing?12:29
mgz_the bind is basically so the bot doesn't need to push after committing a successful merge, because that's what tarmac expects12:29
mgz_it means acttions on the local branch are reflected on the remote branch12:29
rogpeppemgz_: i guess i don't understand why we want to perform actions on, goose, say12:30
mgz_the bot also controls goose landings12:30
mgz_but yes, this is now very dodgy12:30
rogpeppemgz_: ah12:30
mgz_I'll need to split out goose to a seperate tree set12:30
rogpeppeand gwacl, presumably12:30
mgz_gwacl already is12:30
rogpeppeso no actual need to bind gwacl?12:31
mgz_no bind is fine, but it's a bind under ~tarmac/gwacl-trees12:31
rogpeppeah, it's in a separate tree, yes12:31
mgz_which is not a shared location with out juju-core ~tarmac/trees location, unlike goos12:31
sinzuidoes anyone know the state of bug 1291400? Is there a branch in review?12:38
_mup_Bug #1291400: migrate 1.16 agent config to 1.18 properly (DataDir, Jobs, LogDir) <regression> <upgrade-juju> <juju-core:In Progress by dimitern> <https://launchpad.net/bugs/1291400>12:38
axwrogpeppe: thanks, will pick it back up on monday12:40
adeuringrogpeppe: a github MP: https://github.com/go-yaml/yaml/pull/312:48
rogpeppeadeuring: LGTM12:50
=== BradCrittenden is now known as bac
sinzuirogpeppe, mgz_ jam, natefinch  : I want to release 1.17.5. There are two bugs not fixed. Is it *bad* if we land those fixes for 1.18.0?13:19
rogpeppesinzui: which are the bugs?13:19
sinzuirogpeppe, bug 1271144 and bug 129140013:19
_mup_Bug #1271144: br0 not brought up by cloud-init script with MAAS provider <cloud-installer> <landscape> <local-provider> <lxc> <maas> <regression> <juju-core:In Progress by rogpeppe> <https://launchpad.net/bugs/1271144>13:19
_mup_Bug #1291400: migrate 1.16 agent config to 1.18 properly (DataDir, Jobs, LogDir) <regression> <upgrade-juju> <juju-core:In Progress by dimitern> <https://launchpad.net/bugs/1291400>13:19
rogpeppesinzui: i'm just about to land the fix to the first one13:20
sinzuiI will wait for that, thanks rogpeppe13:20
sinzuiI may need to beat the azure tests. the cloud has been misbehaving and failing the revision tests13:21
adeuringthedac: while i would cliam that i have a fix for bug 1243827, seen from the bugfix it absolutely unclear how using double underscores could avoid the bug. can you tell me which versions of juju-core and juju-deployer you are using?13:36
_mup_Bug #1243827: juju is stripping underscore from options <canonical-webops> <config> <goyaml:Fix Released by adeuring> <juju-core:Invalid by adeuring> <juju-deployer:Invalid by hazmat> <https://launchpad.net/bugs/1243827>13:36
rogpeppemgz_: another thing i don't quite understand from that tarmac script: where is the actual branch to be tested downloaded?13:43
rogpeppemgz_: does it happen before the config-changed-script is run?13:44
mgz_it's merged into the branch in trees13:44
mgz_it's independant, in a cron job that uses tarmac, and the tarmac config (seperate config item) manages13:45
rogpeppemgz_: so we don't run godeps -u on every branch?13:47
rogpeppemgz_: how are we supposed to add a new dependency then?13:47
mgz_no, because that's actually insufficient, we'd also need to go get13:47
rogpeppemgz_: (given that go get launchpad.net/juju-core won't get the new dependency)13:47
rogpeppemgz_: yeah13:47
mgz_rogpeppe: modify the config to pull that specific dep in advance, trigger the config-changed hook, then the next cron run will have what it needs13:48
rogpeppemgz_: so add an explicit go get to the config?13:48
mgz_ie, add a `bzr branch lp:new-dep trees/launchpad.net/new-dep` line or similar13:48
rogpeppemgz_: then remove it after the relevant branch has merged?13:48
rogpeppemgz_: yeah, same difference13:49
mgz_yup13:49
* rogpeppe tries not to be distracted by adding pull support to godeps13:49
mgz_sinzui: do you need any more help for the release?13:52
mgz_I think we'll have to kick dimiter's bug from 1.17.5 for now13:53
sinzuimgz_, I am fine. Azure is not, but since we know azure is not fine, I wont let that damn rogpeppe's branch when it lands13:53
rogpeppesinzui: it's approved and should land in the next 16 years13:54
rogpeppesinzui: ^h^h^h^h^h^hminutes13:55
sinzuifab. CI is idle and ready to take it13:55
mgz_rogpeppe: text conflict13:55
rogpeppebollocks13:55
rogpeppereapproved13:58
natefinchwwitzel3: sorry for the delay, you want to do some pairing?13:59
* Makyo is back (gone 13:44:06)14:01
natefinchjb.go14:01
natefinchgah, stupid focus follows mouse14:01
wwitzel3_natefinch: hey14:08
mattywnot exactly a juju-core question - but is there a way to use the testing/mgo.go stuff to setup a mongo database with a password?14:17
rogpeppemattyw: using juju-core/state?14:17
rogpeppemattyw: or independent of juju?14:18
mattywrogpeppe, totally independant of juju14:18
rogpeppemattyw: have a look at SetAdminMongoPassword in the state package14:19
rogpeppei just grabbed github.com/go-juju BTW, as a placeholder in case we decide to use gopkg.in at some point in the future14:21
rogpeppesinzui: that branch has merged14:29
sinzuirogpeppe, CI has already built it, is running unit tests and publishing the test tools for the next round of tests. I think I am 1h from starting the release14:30
rogpeppesinzui: brilliant14:30
=== hatch__ is now known as hatch
natefinchrogpeppe, mgz_: any ideas as to why juju would keep telling me it's already bootstrapped amazon when there's definitely no jenv around?14:44
rogpeppenatefinch: what's the output of bootstrap --debug ?14:44
natefinch2014-03-14 14:40:52 DEBUG juju.environs.configstore disk.go:64 Making /home/nate/.juju/environments14:44
natefinch2014-03-14 14:40:52 INFO juju.provider.ec2 ec2.go:207 opening environment "amazon"14:44
natefinch2014-03-14 14:40:53 ERROR juju.cmd supercommand.go:296 environment is already bootstrapped14:44
natefinchnot terribly useful14:45
rogpeppenatefinch: what does your amazon environment entry look like in environments.yaml14:45
rogpeppe?14:45
natefinch  amazon:14:46
natefinch    type: ec214:46
natefinch    admin-secret: 36c59e8a199197b4b3c0950ab844dec414:46
natefinch    # globally unique S3 bucket name14:46
natefinch    control-bucket: juju-c2feecf0b1cd9ef0d4ea241dd90e10c314:46
natefinch    # override if your workstation is running a different series to which you are deploying14:46
natefinch    # default-series: precise14:46
natefinch    bootstrap-timeout: 360014:46
natefinch    # region defaults to us-east-1, override if required14:46
natefinch    # region: us-east-114:46
natefinch    # Usually set via the env variable AWS_ACCESS_KEY_ID, but can be specified here14:46
natefinch    # access-key: <secret>14:46
natefinch    # Usually set via the env variable AWS_SECRET_ACCESS_KEY, but can be specified here14:46
natefinch    # secret-key: <secret>14:46
rogpeppenatefinch: i bet it works if you delete the control-bucket entry14:46
natefinchhmm14:46
rogpeppenatefinch: are you sure you have no currently running instances?14:46
rogpeppenatefinch: you can delete admin-secret too14:47
natefinchrogpeppe: I did manually kill a bootstrap node last night14:47
rogpeppenatefinch: well, that'll be the reason14:47
rogpeppenatefinch: there's more to an environment than just the jenv file14:47
natefinchdestroy-environment --force should put us back in a reasonable position, though, and it doesn't14:47
rogpeppenatefinch: i don't think so14:48
rogpeppenatefinch: in your particular case, perhaps14:48
rogpeppenatefinch: but in general we don't know where the environment is if you've deleted the .jenv file14:48
natefinchthere should never be a time when, after doing destroy-environment --force that juju bootstrap tells me it's still bootstrapped14:48
natefinchthat's what --force is for14:49
natefinchit's the "just fucking do it" flag14:49
natefinchanyway....14:49
rogpeppenatefinch: it's awkward, because in this case only you'd need to ignore the .jenv file and use info from environments.yaml14:51
thedacadeuring: wrt bug#1243827 we were using juju core 1.14.x and juju-deployer 0.2.3+bzr115-0~26~precise1ubuntu1~0 at the time. AFAIK, the problem still remains with 1.16.x and 0.3.0-0ubuntu2. I'll see if I can test that theory today. But the bug has the exact example of the failure.14:51
_mup_Bug #1243827: juju is stripping underscore from options <canonical-webops> <config> <goyaml:Fix Released by adeuring> <juju-core:Invalid by adeuring> <juju-deployer:Invalid by hazmat> <https://launchpad.net/bugs/1243827>14:51
rogpeppenatefinch: well, i suppose if the .jenv file doesn't exist, we should perhaps create a new Environ and then destroy it14:51
natefinchrogpeppe: there's no jenv, that's the thing.  Maybe it's just a matter of the fact that the environments.yaml is an anti-feature14:51
rogpeppenatefinch: there *was* a jenv14:51
natefinchrogpeppe: yeah14:52
rogpeppenatefinch: until you deleted it (probably manually, or perhaps with destroy-environment --force)14:52
rogpeppenatefinch: i think it's that control-bucket is an anti-feature14:52
natefinchrogpeppe: could be yeah14:53
rogpeppenatefinch: without that, you'll get a new env each time14:53
rogpeppenatefinch: you can still leak environments though (and you probably would have in this case)14:53
adeuringthedac: thanks for the version info. and right, the problem still exists in recent versions -- but I did not test juju 1.14.14:53
natefinchrogpeppe: ahh, yeah, good point.14:54
natefinchrogpeppe: taking out the control bucket worked14:55
natefinchrogpeppe: I'll probably have to go clean up the original bucket14:55
sinzuijuju devs, r2422 is blessed. I will release it as 1.17.515:30
rogpeppesmall review anyone? https://codereview.appspot.com/76120043/15:48
rogpeppenatefinch: ^15:48
natefinchrogpeppe: pairing with wayne right now, may get to it in a little bit once we figure out what's wrong with my code15:49
rogpeppenatefinch: ok15:49
lazyPoweranyone have a moment to help a user that's having issues getting the cloudimg deployed with juju? I'm baffled as to why juju isn't fetching and deploying the cloudimg tarball for him.15:59
lazyPowerthis is extending past my knowledge of how juju-core works15:59
=== BradCrittenden is now known as ba
=== ba is now known as bac
sinzuijamespage, 1.17.5 tarball is released. I am going to make the source packages for the ppas now. While you could make the packages with the current SPB, I changed the testing branch to install the bash completions file in the juju tree. https://launchpad.net/juju-core/+milestone/1.17.516:05
jamespage* Juju support for juju-mongodb is temporally removed. The feature16:06
jamespage  will be restored when it is more complete.16:06
sinzuijamespage, ~juju-packaging/+archive/devel has the source package now16:06
* jamespage crys a bit16:06
sinzuijamespage, yeah. I hoped it would be put back before today16:06
* bodie_ offers jamespage a slightly disheveled hankie16:07
natefinchrogpeppe, mgz_: any idea why mongo would be running, but jujud wouldn't be able to connect to it?  I can connect to it using the mongo command line, but jujud just says connection refused16:07
rogpeppenatefinch: what does (ps alxw | grep mongo) tell you?16:08
rogpeppenatefinch: is this in a testing context?16:09
bodie_I'm having a hard time understanding some of the "tribal knowledge" around how to actually run and test things16:09
natefinchrogpeppe: unfortunately the instance got killed, even with a ridiculously long timeout in my environments.yaml16:09
bodie_is there an ideal dev workflow here?16:09
bodie_like, my tests were breaking yesterday and I had no idea why -- I assume juju is running without full green lights?16:09
rogpeppenatefinch: i need more context16:09
bodie_I was going on the assumption that I'd get greens, make changes, see what breaks, fix it, etc16:09
rogpeppelazyPower: i don't think i understand your question16:10
natefinchrogpeppe: ok, I'll retry it after lunch16:10
rogpeppelazyPower: what's "cloudimg" ?16:10
natefinchbodie_: all the tests should pass... there are a few  that are flaky16:10
lazyPowerrogpeppe: there's a user outside the us (not sure if this is a cause) - that wasn't able to bootstrap a local environment. The cloudimg tarball was failing to download. He managed to fetch the tarball manually but i didn't know where to tell him to put it16:11
lazyPowermarcoceppi joined in and clued us in that it lives in /var/cache16:11
rogpeppelazyPower: ah, local environment - i'm afraid i've never looked into it in that much detail so i won't be much help, sorry16:12
jcw4natefinch do folks usuall run go test juju-core/... ?16:12
natefinchrogpeppe: it looks like the bootstrap-timeout in environments.yaml isn't actually working... at least, my instances are still getting killed way before the timeout I set there16:12
natefinchjcw4: yep.   And in fact, code can't land unless that passes16:12
jcw4natefinch interesting so if they fail locally somethings wrong on *our* environment?16:13
natefinchjcw4: however, those tests take a long time, so generally you want to run a subset during iterative development, and only run the full thing when you're pretty sure your code is right16:13
jcw4natefinch I'll grep the docs, but is there a CI link for juju-core?16:13
natefinchjcw4: there are tests under juju-core/replicaset that are unfortunately flaky, if it's those that are failing, it's not just you... it's an ongoing problem16:14
jcw4also providers/azure16:14
jcw4well actually compile failures there16:14
natefinchjcw4: make sure you update your dependencies, they may have changed (like the azure provider, I think some changes landed recently to that)16:16
jcw4natefinch will do, thx16:16
natefinchjcw4: if you do go get launchpad.net/godeps  you'll have a godeps executable, that you can then use from the root juju-core folder  doing godeps -u dependencies.tsv and it'll tell you if anything is out of date16:17
natefinchjcw4: you'll have to update the out of date stuff manually, but at least you'll know what needs doing16:17
jcw4natefinch excellent much better than manually finding outdated stuff16:17
=== Ursinha is now known as Ursinha-afk
=== Ursinha-afk is now known as Ursinha
bodie_ok, I'm getting a bunch of test fails with a freshly checked out copy of everything16:34
bodie_not too sure what's up16:34
bodie_is that normal?16:34
bodie_running go test ./... in $GOPATH/src/juju-core16:35
natefinchbodie_: can you paste the test output to pastebin.ubuntu.com - that would help figure things out17:11
bodie_sure17:12
bodie_it's pretty huge though17:14
bodie_I think the initial database connection is failing, so pretty much everything else barfs too17:14
bodie_but I recompiled Mongo with SSL enabled and all that, and it works fine17:15
natefinchbodie_: yeah, I know the test output is huge...  it's something I've wanted to be able to turn on and off for a while, it just hasn't been at the top of the priority list.17:19
bodie_lol, from man watch17:20
bodie_"you can watch for your administrator to install the latest kernel with watch uname -r (just kidding)"17:21
bodie_oh, the "just kidding" isn't in my version of the manpage17:21
rogpeppenatefinch: that branch i pointed you towards earlier makes it easier to make test output more or less verbose17:28
natefinchrogpeppe: hey, that's cool.17:28
rogpeppenatefinch: i'm still swithering about whether it's better as an environment variable or as a flag17:28
natefinchrogpeppe: yeah, that's tricky. both would be nice, so you could set it in the environment the way you like the default, and then override with the flag as necessary17:29
rogpeppenatefinch: i'm not a big fan of both. i prefer one way for most things.17:30
rogpeppenatefinch: i went with the flag because i think that people might be able to break tests that test log output by setting the level to ERROR17:31
natefinchrogpeppe: if I had to pick one or the other, I guess the environment variable, since then it's sticky, and I'm guessing there are people who are going to want the default to be one thing or the other17:31
natefinchrogpeppe: I think tests that depend on the log output are broken to begin with17:31
rogpeppenatefinch: that's not necessarily true17:31
rogpeppenatefinch: some tests are specifically testing that something is logged17:31
natefinchrogpeppe: I suppose that's valid, though I'd prefer if they were able to hook into the logging system to see that something would have been logged rather than checking the output to see what is actually output17:32
rogpeppenatefinch: they are generally doing that17:33
rogpeppenatefinch: but the logging system is influenced by the global logging config17:33
rogpeppenatefinch: such tests should probably reconfigure the logging system as desired17:33
rogpeppenatefinch: but i didn't want to go through making that change everywhere17:34
natefinchrogpeppe: yeah, that was my thought, though then that kind of defeats the purpose of being able to adjust how much is logged17:34
rogpeppenatefinch: only for those (very few) tests.17:34
bodie_natefinch, here's my test output...17:34
bodie_http://paste.ubuntu.com/7091353/17:34
bodie_but, I totally built mongo with SSL17:35
natefinchrogpeppe: ideally there would be two different loggers, one that respects the log level and does the real logging, and one that ignores it that is used for tests.  But I don't know anything about how loggo works to know if that's feasible17:35
rogpeppenatefinch: there are many different log levels17:35
rogpeppenatefinch: it's hierarchical actually17:35
bodie_oh, now it's over 1300 lines17:36
bodie_ok.17:36
rogpeppebodie_: that's small :-)17:37
rogpeppebodie_: i had a 500k line output from a test yesterday17:37
bodie_lol17:37
bodie_that's silly17:37
bodie_ain't nobody got time fo dat17:37
natefinchbodie_: that's definitely mongo not having ssl, still: mongod: Error parsing command line: unknown option sslOnNormalPorts17:37
rogpeppebodie_: darn straight17:37
natefinchbodie_: as for azure....  what branch are you building?17:37
bodie_seriously, though.  there really ought to be a fakey backend or something17:37
rogpeppebodie_: yeah.17:38
rogpeppebodie_: choosing the right level for it is the difficult thing17:38
bodie_right...17:38
rogpeppebodie_: i've wondered about faking up an in-memory mongo17:38
bodie_env variable to define backend testing level?17:38
bodie_FAKE_BACKEND=true?17:39
rogpeppebodie_: yeah, but someone's got to implement it17:39
bodie_right, heh17:39
natefinchrogpeppe: my point is not about having levels or not, my point is about having different log targets.  So you can log like debug+ to the debug log over there and log info+ to stdout and have a testing output that always logs everything.17:39
rogpeppenatefinch: you can do that too17:39
bodie_this is just the freshly checked out trunk17:40
rogpeppenatefinch: except that the global config chokes log messages at source17:40
natefinchrogpeppe: well, see, that's the problem :)17:40
rogpeppenatefinch: honestly, it's not actually a problem17:40
bodie_grrhh...  I wiped out the old mongo installation though17:41
bodie_I double-checked just now that mongo and mongod are the copies I built yesterday17:41
natefinchbodie_: if you do mongod --help do you see a section of flags labelled SSL?17:41
bodie_ugh17:42
bodie_no17:42
natefinchbodie_: there's your problem17:42
natefinchbrb17:43
bodie_oh I know why17:43
rogpeppenatefinch, mgz_: still looking for a review of https://codereview.appspot.com/76120043 if poss17:49
natefinchrogpeppe: looking17:51
natefinchwwitzel3: you around?17:52
natefinchrogpeppe: lgtm'd18:00
bodie_alright18:00
rogpeppenatefinch: ta18:00
bodie_successfully built mongo with ssl18:00
bodie_the instructions are screwy18:00
natefinchbodie_: I told you18:01
natefinchbodie_:  I have a write up somewhere with actually good instructions, but I can't figure out where they went.18:01
bodie_when you do scons install, you have to use the --ssl flag both times18:01
bodie_simple18:01
bodie_i assumed it was something like ... build locally into a ./build folder with SSL...18:02
bodie_then scons install uses that18:02
bodie_BUT NO18:02
natefinchyeah18:03
sinzuinatefinch, do you have a few minutes to review the branch that updates trunk to 1.17.6?   test-release-hp:18:06
sinzui    # juju --show-log bootstrap -e test-release-hp18:06
sinzui    type: openstack18:06
sinzui    test-mode: true18:06
sinzui    use-floating-ip: false18:06
sinzui    admin-secret: 1717-burgers-hotdogs-potato-salad-baked-beans18:06
sinzui    control-bucket: juju-pavlova-snags--chiko-roll-meat-pie-pasty-sauce-2013-10-1018:06
sinzui    default-series: precise18:06
sinzui    auth-url: https://region-a.geo-1.identity.hpcloudsvc.com:35357/v2.0/18:06
sinzui    authorized-keys: |18:06
sinzui      ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDyEdd8eDyy2WA6Q+W4GV2NqKX+yfkkW5ogSu3/7DjlqjED7dCNkevq2qpdR1AMkbKTouihWdyc8QKl9lzuwn0zoocXJUVgIOPV+KFxSAhj1djGvryHDjYUwdrLYUMu3CUFeIsRao2cn7EgZs0w1Y1quqr9c8cEg7XsAs0ZMN9YksEjG000VupOIZJNtk+5EYJm/6vNFI83IOn7ctWfjXymBuh7XM8d8vszyYDRdeDXY5Q9VLqHOP7/CFteIvcdHnSC1ObQuKzXRWz+m9thgQnRQjvirdwvDUXhjjQk9MNJZj84EukB8HyAVSN863MfuVGoCsNn7iEdtT6W2nKTWyL3 abentley@speedy18:06
sinzui    tools-url: https://region-a.geo-1.objects.hpcloudsvc.com/v1/60502529753910/juju-dist/testing/tools18:06
sinzui    auth-mode: userpass18:06
sinzui    username: "sinzui"18:06
sinzui    password: "MyPineappleCalls4uInTheNight"18:06
sinzui    tenant-name: "juju-scale-test"18:06
sinzui    region: az-3.region-a.geo-118:06
sinzuiwell not18:06
sinzuinot that crack18:06
sinzuinatefinch, https://codereview.appspot.com/7617004318:06
natefinchsinzui: you have the best passwords and secrets I've ever seen :)18:07
sinzuinatefinch, yeah :( I am pretty said to see that go18:07
natefinchsinzui: lgtm.... wish we could do the versions some other way than changing sourcecode, though18:08
=== Ursinha is now known as Ursinha-afk
=== Ursinha-afk is now known as Ursinha
mrammsinzui: our upgrade compatability policy includes 1.18 clients tested against a 1.16 server right?18:16
sinzuiYes mramm18:17
mrammok18:17
mrammthought so, thanks sinzui18:17
bodie_... Panic: not authorized on admin to execute command { listDatabases: 1 } (PC=0x414386)18:20
bodie_that doesn't seem right18:20
bodie_unless tests aren't running as my user?18:20
natefinchbodie_: I haven't seen that one18:20
natefinchbodie_: they run as you18:21
bodie_http://paste.ubuntu.com/7091566/18:21
bodie_O_o18:22
natefinchbodie_: the azure stuff has to be launchpad.net/gwacl being out of sync somehow18:22
natefinchbodie_: not sure about the mongo problem....18:23
bodie_http://paste.ubuntu.com/7091591/18:24
bodie_:S18:24
rogpeppeoh bugger, looks like the last time i claimed aws expenses was exactly a year ago18:24
bodie_I thought we confirmed the azure provider wasn't working right now18:24
natefinchrogpeppe: haha18:24
rogpeppewell, i can only try18:24
=== Ursinha is now known as Ursinha-afk
bodie_so uh... the azure provider is working for other people?18:33
bodie_like, if you go test, you don't get the error with azure?18:33
natefinchbodie_: lemme try18:34
bodie_it's right at the top, if you do get it18:34
=== Ursinha-afk is now known as Ursinha
natefinchbodie_: totally broken, you're right18:38
bodie_ah18:39
bodie_ok18:39
bodie_so just disregard or how do I pass tests in order to submit revisions?18:39
natefinchbodie_: you won't be able to get code landed with those tests broken like that....  O18:40
natefinchbodie_: I'll see if I can fix it18:40
bodie_nice!  can I help anyhow?18:41
bodie_I'm looking for a place to start digging18:41
natefinchbodie_: I'm sure it's just a matter of gwacl getting changed without juju getting corresponding changes in the tests18:41
rogpeppetotal $1759.2618:44
natefinchbodie_: it's a problem with dependencies outside juju-core... I'm sure the gwacl tests pass just fine, it's just that gwacl can't know you also need to fix juju-core (and there's still a chicken and egg problem where you have to check one in first, and that breaks either itself or the other one)18:44
natefinchrogpeppe: for 12 months that's not terrible.  Seems a bit high, though.  Did you have some unusually big months for some reason?  mine are usually $30-50 a month18:45
rogpeppenatefinch: biggest was $323.3918:45
natefinchrogpeppe: that's pretty big18:46
rogpeppenatefinch: i was probably doing a lot of live tests then18:46
natefinchrogpeppe: btw, looks like the azure tests are broken... probably gwacl changes landing without corresponding juju-core tests.  Is that something the guys down under are working on?18:47
bodie_natefinch, I really appreciate all the assistance getting up to speed here.  it's pretty daunting18:48
natefinchbodie_: it really is pretty daunting.  We should write up a getting started document. It was hard for me getting up to speed too (I started ~9 months ago)18:48
natefinchbodie_: yeah, looks like the signature of one of the functions in gwacl changed a couple days ago, and no one noticed it broke the juju-core tests19:02
bodie_ah, good stuff19:14
bodie_I feel useful.. sorta19:14
bodie_is there a specific version of mongo we should be using?19:17
bodie_I just built the latest master, which I think is 2.419:17
bodie_I'm getting an error "not authorized for upsert" which apparently others are also seeing19:17
bodie_if anyone is running 2.4 and not seeing that error, would like to know :)19:18
natefinchbodie_: I'm running 2.4 and not getting that error... I believe we support 2.2 and above.19:21
bodie_oh, i'm on 2.7-pre19:23
bodie_O_o19:23
bodie_I thought I was on the stable...19:24
rogpeppei am done for the day19:24
rogpeppewell past it actually19:25
wwitzel3rogpeppe: have a good weekend :)19:25
rogpeppewwitzel3: and you!19:25
bodie_fyi: https://svn.boost.org/trac/boost/ticket/724219:38
bodie_ugh, maybe this isn't what I need after all19:39
=== hatch__ is now known as hatch
bodie_Panic: not authorized on admin to execute command { listDatabases: 1 } (PC=0x414386)20:34
bodie_any thoughts?20:34
bodie_using Trusty packaged mongo20:34
bodie_this from the test suits20:34
bodie_suite*20:34
natefinchbodie_: the trusty packaged mongo may not have SSL either.  I haven't seen that particular error before, unfortunately20:36
natefinchunfortunately, it's end of day for me here20:36
bodie_alrighty :)20:38
bodie_gustavo said it has ssl and i'm no longer getting the SSL related error... not sure what this one's about, though20:38
natefinchgood luck, and if you don't figure it out, there's always \Mmonday :)20:38
bodie_farewell and have an excellent weekend20:39
natefinchyou too20:39
niemeyerbodie_: This looks like plain lack of login20:39
niemeyerbodie_: and this is not about SSL20:39
niemeyerbodie_: Or at least not in an obvious way20:40
niemeyerbodie_: It wouldn't get to the point of issuing a listDatabase command and getting "auth denied" if the SSL communication was corrupted20:40
niemeyerbodie_: Are you able to run the test in isolation?:20:41
bodie_negative20:42
bodie_PANIC: mgo_test.go:42: mgoSuite.TearDownTest20:42
bodie_  in resetAdminPasswordAndFetchDBNames20:43
bodie_hm, that didn't paste properly20:43
bodie_home/bodie/go/src/launchpad.net/juju-core/testing/mgo.go:36620:43
bodie_  in resetAdminPasswordAndFetchDBNames20:43
bodie_obviously /home20:43
bodie_http://paste.ubuntu.com/7092264/20:44
bodie_I feel like this one should be obvious20:45
bodie_but I'm not quite seeing it20:45
niemeyerbodie_: Hmm21:07
niemeyerbodie_: I'd check how isUnauthorized is implemented21:07
niemeyerbodie_: The error apparently *is* a lack of authorization one21:07
niemeyerbodie_: It may be doing a string check against the error, and that string may have changed across MongoDB releases21:08
=== hatch__ is now known as hatch
bodie_okie doke21:16
bodie_howdy thumper21:16
thumpero/21:17
thumperjust passing through21:17
hazmatthumper, do we have no-proxy support in our proxy bits?23:36
hazmatoh.. that passthrough was a while ago23:38

Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!