/srv/irclogs.ubuntu.com/2013/10/16/#juju-dev.txt

wallyworld_bigjools: wtf. bootstrap on your maas box works now00:16
wallyworld_bigjools: so roger and i were debugging and trying things. then the server got shutdown. now, it seems it has just started working. i tried with and without all the debug logging00:21
wallyworld_maybe the power cycle on the server helped, not sure00:21
bigjoolswallyworld_: dafuq00:22
thumperhaha00:23
wallyworld_bigjools: so looks like we have wasted 2 man days when we should just have listened to Roy ans Moss and "turned it off and on again"00:24
bigjools\o/00:24
bigjoolsI still think it's a bug00:24
wallyworld_well, not much juju can do if maas/apache closes the connection00:25
wallyworld_from underneath it00:25
wallyworld_i guess it could retry00:25
wallyworld_but wtf00:25
wallyworld_that's the only place that needs such logic00:25
bigjoolsnetwork code should never ever assume connections will stay open00:34
wallyworld_that's for the http lib to worry about00:34
bigjoolsyes00:35
bigjoolsI think the Go http lib is a little crazy00:35
bigjoolsexposing that Close setting is one thing, but requiring it before it can cope with the other end closing it is a bug00:35
wallyworld_hard to argue that i think00:35
bigjoolsanyway can someone land this please, I am not in the juju team any more: https://code.launchpad.net/~allenap/juju-core/maas-environment-uuid-use/+merge/19124900:36
wallyworld_i'll add you00:36
bigjoolsplease don;t :)00:36
wallyworld_what's it worth to you00:36
bigjoolscoffee and lunch at the Tavern?00:36
wallyworld_tempting00:36
bigjoolslower latency to my maas server?00:37
wallyworld_who knows00:37
bigjoolsI'm slobbing it on the outdoor sofas today00:37
wallyworld_bigjools: so that branch, does it duplicate tools?00:38
bigjoolsnot started it yet00:38
bigjoolsI do not intend to duplicate them00:38
wallyworld_ah, sorry, i thought the one you wanted landed was it00:38
bigjoolsno, it's gavin's agent_name fixes00:39
wallyworld_i should have read the description00:39
wallyworld_bigjools: do you intend to propose this against 1.16 too?00:39
bigjoolsfailing the tavern, $5 big mac? :)00:39
wallyworld_or just land in trunk?00:39
bigjoolshonestly NFI what's best00:40
bigjoolsI am not familiar with the release plans00:40
wallyworld_thumper: when is 1.18 due out?00:40
* thumper shrugs00:40
bigjoolssomeone said yesterday that there's another release for saucy00:40
wallyworld_there is00:40
wallyworld_1.18 i think00:40
bigjoolsso trunk then00:40
wallyworld_bigjools: so gavin's fix, how critial is it00:41
bigjoolsvery00:41
bigjoolsand the one I am about to do00:41
wallyworld_i wonder if we need a 1.16.1 then00:41
wallyworld_fwereade: any idea on release plans as per the backscroll?00:41
bigjoolsare you going to approve the MP then?00:42
wallyworld_yeah, sorry saw something shiny and got distracted00:44
wallyworld_should be in the bot now00:44
bigjoolsheh00:45
bigjoolsthanks00:45
wallyworld_bigjools: about lunch, is there a place to buy decent coffee beans out your way?00:46
bigjoolswallyworld_: the little bean in Kenmore00:46
wallyworld_that's more out my way :-)00:46
bigjoolsit's on the way :)00:46
bigjoolsthat's the nearest00:47
bigjoolsthough there's a new coffee shop coming apparently \o/00:47
wallyworld_i need coffee. i could drive out to you and get beans also. kill two birds with the one stone00:47
bigjoolspoyfekt00:47
bigjoolsremember that the cafe closed down, you have to go to the smaller place on the other side of the road now00:48
wallyworld_what time?00:48
wallyworld_yes00:48
thumperwallyworld_: I have a feeling that we'll only be able to put 1.16 point releases directly into saucy00:48
bigjoolsanhy time you want00:48
bigjoolshowever00:48
thumperbut continue as normal with trunk00:48
bigjoolsI have a call from 12-100:48
thumperand the ppa00:48
wallyworld_bigjools: i'll leave soon i guess00:49
bigjoolssure00:49
wallyworld_thumper: that sucks00:49
thumperwallyworld_: that's working with distro00:49
wallyworld_i thought 1.18 was going into saucy00:49
bigjoolsthey will only take cherry picks into saucy now00:49
thumperwallyworld_: unlikely at this stage00:49
wallyworld_cause i've done a bunch of stuff in trunk00:49
wallyworld_assuming it would be in saucy00:49
wallyworld_this is very bad00:50
wallyworld_1.16 is not ready00:50
wallyworld_there's still the tools repository to do00:50
wallyworld_and the ongoing maas stuff00:50
wallyworld_and lots of other tooling stuff00:50
wallyworld_if we are forced to cherry pick stuff, it will be like the whole fucking cherry tree00:51
wallyworld_bigjools: leaving now00:58
bigjoolswallyworld_: righto00:58
bigjoolsevery time I leave the juju code base for a while and then come back to work on it, I struggle to get everything compiling.  I presume this is because of mismatched dependencies.  What's the best way of dealing with this?01:10
bigjoolsor, I suspect, branches moving and Go has the bug of using the wrong url for a branch :/01:12
* thumper nods01:12
thumperthat is one01:12
bigjoolsyeah goamz moved it seems01:12
thumperapparently jam had a proposal to get golang to use lp: urls for launchpad01:13
thumperno interest01:13
bigjoolsoh dear01:13
davecheneybigjools: yup, if the owner of goamz has moved01:18
davecheneythe go get'd branch is probably pointing at the wrong place01:19
bigjoolsindeed it was01:19
davecheneybigjools: niemeyer added support for bzr to go get01:20
davecheneyif you can show me what is wrong, i can try to get it fixed01:20
bigjoolsthumper can explain it better than me01:20
bigjoolsbut the upshot is that it needs to pull from lp:project01:20
bigjoolsnot the actual branch url01:20
thumperdavecheney: when the go tool resolves bzr branches to launchpad01:21
thumperit expands the project name into the full http url with unique name01:21
thumperthis is very slow01:21
bigjoolsthe old url is http://bazaar.launchpad.net/~gophers/goamz/trunk/01:21
thumpermost LP users have their lp identiy set in bzr01:21
bigjoolsthe new url is bzr+ssh://bazaar.launchpad.net/+branch/goamz/01:21
thumperwhich means lp: urls resolve to bzr+ssh01:21
bigjoolsthe latter is owner-agnostic01:21
thumperif you don't lp urls resolve to http01:21
thumperso lp: is better01:21
thumperalso01:22
thumperbzr+ssh://bazaar.launchpad.net/+branch/project01:22
thumperalways resolves to the development focus trunk of the project01:22
thumpereven if the owner changes01:22
thumperbut go get will turn "launchpad.net/loggo" into http://bazaar.launchpad.net/~thumper/loggo/trunk01:22
thumperinstead of bzr+ssh://bazaar.launchpad.net/+branch/loggo01:23
thumperif go get passed "lp:loggo" to bzr01:23
thumperbzr translates to the best it knows01:23
thumperwhich is bzr+ssh if it has your id01:23
thumperand http if not01:23
davecheneythumper: i'm pretty sure the choice of http is deliberate01:23
thumperdavecheney: deliberate and stupid01:23
thumperIMO01:23
davecheneyfair01:24
thumperit is a choice made by someone who doesn't understand the bzr tool01:24
thumperand when jam suggested a patch to golang, they ignored it01:24
* davecheney has no comment01:24
thumpereven though he is probably the best person to make such a suggestion01:24
* thumper goes back to reviewing wallyworld's brach01:25
wallyworld\o/01:25
* thumper needs to go pick up the car from the garage01:34
thumperbbs01:34
=== thumper is now known as thumper-afk
=== thumper-afk is now known as thumper
thumperaxw: how are you doin?02:12
axwthumper: heya02:14
axwnot too shabby02:14
axwworking on fixing null provider bugs02:14
axwthe apt repo one's a bit of a pain, need to extract the key from the keyserver... cloud-init would normally take care of that02:15
* thumper nods02:15
thumperthere isn't a handy command we can use?02:15
thumperdoesn't add-apt-repository download the key?02:16
axwthumper: only for ppas02:17
thumperbummer02:17
axwI'm looking at the cloud-archive case02:17
* thumper nods02:17
* thumper goes to pick up the wife02:23
thumpergeez02:23
thumperbroken day02:23
thumperbbs02:23
axwdavecheney: are you aware of any tools for looking for unused functions/vars/types/etc.?02:27
axwor, how can I identify all functions that are only ever used in tests02:28
davecheneyaxw: I think there is a mode for go vet in 1.202:36
davecheneyand kamil kissel has written a tool02:37
axwdavecheney: thanks, I'll take a look02:37
wallyworldthumper: ping02:53
wallyworldthumper: i did some fixes for axw's review in the wrong branch in the pipeline. i'm fixing now so ignore the new diff in your review.02:54
thumperok03:09
wallyworldthumper: what i did do though is reply to your comments on both merge proposals. i'll fix the issues like gc.HasLen etc but there's also a few things i've replied back to03:30
* thumper nods03:32
thumperwallyworld: I feel I may pop down to harvey norman to look at the coffee machine03:32
thumperreally need one that doesn't make me angry03:33
wallyworldyes indeed03:33
wallyworldget the dual boiler!03:33
wallyworldthumper: do all tests really need to extend loggingsuite even if they don't requite the base functionaity03:33
wallyworldseems like a waste03:34
thumperwallyworld: the logging suite captures the logging03:34
thumperwithout it, tests become noisy03:34
thumperif someone decides to add logging somewhere03:34
thumperthat is in the testing path03:34
wallyworldfair point03:34
thumperthat's all it really does03:34
wallyworldi guess we should fix all existing test suites at some point then03:35
* thumper nods03:35
wallyworldthumper: if you still have any spare bandwidth left today, i've done fixes for those 2 mp's04:40
axw_wallyworld: when you have a moment, I've updated https://codereview.appspot.com/14527043/04:48
wallyworldsure04:48
axw_wallyworld: sync no longer resolves metadata, but "juju metadata generate-tools" will still04:48
wallyworldaxw_: i think everything should calc the sha etc, drop the option to allow it not to be done. the sha256 and size is absolutely needed for sync tools04:56
wallyworldand generate metadata is typically run using local files so it can always be done for that as well04:57
axw_wallyworld: it's only for existing tools with no metadata - I thought the conclusion was that it would be okay after 1.16?04:57
wallyworldafter 1.16, there should be no metadata without size/checksum04:58
wallyworldso if this mp is to go into trunk, then drop the resolve option altogether04:59
wallyworldimo04:59
wallyworldalways just do the checksum/size04:59
axw_wallyworld: as in, behave as if the call were specified with fetch/resolve==true all the time?04:59
wallyworldyeah05:00
axw_what's the point if there's no metadata without size/checksum?05:00
wallyworldthe fetch=true tells the command to read the tarball data to do the size/checksum when the metadata is generated05:00
wallyworldand that's what we always want now05:00
wallyworldsince we don't want to produce metadata without size/checksum05:00
wallyworldso fetch=false should be verboten05:01
wallyworldmake sense? or am i missing something?05:02
axw_wallyworld: with the change, metadata is still beign generated with size/hash. It's populated when the tools are copied to storage05:02
axw_wallyworld: the only thing that's affected is tools that are in storage, but either don't have metadata, or have metadata without size or hash05:02
wallyworldso - if i have some tarballs locally, and i just want to generate metadata json, and not copy the tarballs anywhere - that's what the generate-metadata command does - that should always happen with size/hash05:04
wallyworldand even if the tarballs are not local, ie on a cloud, the same applies05:04
wallyworldthe generate-metadata command should always produce json with size/hash now05:04
axw_wallyworld: yes, it does and will continue to do so with this change05:04
axw_generate-tools only05:05
wallyworldas of 1.16, there should be no metadata without size/hash05:05
wallyworldso i'm not sure if your comment above holds?05:05
axw_right05:05
wallyworldthis one i mean05:05
wallyworld[15:02:48] <axw_> wallyworld: the only thing that's affected is tools that are in storage, but either don't have metadata, or have metadata without size or hash05:05
axw_right, so my point is - the change won't break anything :)05:06
wallyworldsure, but why cater for a forbidden scenario05:06
axw_as in, it only affects a scenario that won't occur05:06
wallyworldit just complicates the code base05:06
axw_I'm explicitly not catering for it now05:06
wallyworldbut there's still the fetch option etc05:06
wallyworldthat is no longer needed05:07
wallyworldfetch=true always05:07
axw_sorry, wallyworld are you talking just about the metadata plugin?05:07
axw_as in, get rid of the command line option and have *that* always fetch?05:07
wallyworldi was just looking quite narrowly at the diff in the code review05:07
wallyworldand saw the option to resolve or not still there05:07
axw_wallyworld: yeah, that's *only* in the plugin now.05:08
wallyworldi do think we should always fetch, but we can do that as a separate mp05:08
axw_I can make it always do it05:08
wallyworldsorry, my brain hadn't made the distinction of what was where when reading the diff05:08
axw_can I just confirm that it's okay *not* to resolve metadata for syncing?05:08
wallyworldwe do need to resolve for syncing05:09
axw_when I say resolve metadata, I mean fill in size/hash05:09
wallyworldcause we may have new tools05:09
wallyworldthat need to be copied05:09
axw_wallyworld: heh, I mean for existing tools05:09
axw_sorry05:09
axw_not for newly copied ones05:09
axw_newly copied ones will always get it, there's no option to disable it05:10
wallyworldok, i think it's reasonable, in trunk, to assume existing tools will have size.hash05:10
wallyworldagree?05:10
axw_okay, cool05:10
axw_yes05:10
wallyworldby brain hurts :-)05:10
wallyworldmy05:10
axw_sorry :)05:10
wallyworldnot your fault05:10
axw_wallyworld: and I'll update the generate-tools command to always fetch05:10
wallyworldok, that would be great. i like leaving less legacy / tech-debt :-)05:11
wallyworldthaks :-)05:11
axw_wallyworld: updated05:15
wallyworldlooking05:15
wallyworldaxw_: looks good, land that fucker :-)05:19
axw_sweet, thanks05:19
wallyworldthank you for making it all work :-)05:19
axw_heh nps05:20
bigjoolspls to be reviewerating https://code.launchpad.net/~julian-edwards/juju-core/maas-uuid-file-prefix/+merge/19133605:34
bigjoolssorry no Blofeld05:34
fwereadebigjools, so how's the environment-uuid config field hooked up to the actual environment uuid?05:43
bigjoolsfwereade: don't know the details, allenap did that already05:44
axw_fwereade: the UUID is allocated randomly, at prepare time05:46
axw_so... pointing at the same env requires sharing the UUID05:46
fwereadeaxw_, bigjools: looks like that's not an environment UUID at all, it's just somemade-up shit :/05:50
axw_yeah05:50
* fwereade sighs deeply05:51
fwereadebigjools, your branch looks fine05:51
bigjoolsfwereade: ok thanks05:51
bigjoolsand wow are you working late or in a different TZ?05:52
fwereadebigjools, early05:52
fwereadebigjools, flying to the US later today05:52
bigjoolsfwereade: it calls utils.NewUUID() in gavin's branch05:52
fwereadebigjools, need to go and see laura fora bit though, might not be back05:52
bigjoolsso what is the somemadeup-shit you're talking about?05:53
bigjoolshoho05:53
fwereadebigjools, the problem is the overwhelming bugfuck insanity of naming that thing "environment-uuid" when we already have an "environement uuid" that is not at all the same thing05:53
fwereadebigjools, how to write unmaintainable code vol 1 ch1page 105:53
fwereadebigjools, but I cannot deal with this now,I might be back shortly05:54
bigjoolsit's very easy to criticise05:54
bigjoolsbut at least it got done05:54
bigjoolsso do you guys still need two +1s or can I land on one now?05:56
axw_bigjools: just one05:57
bigjoolsthanks axw_05:57
=== axw_ is now known as axw
bigjoolsaxw: can you approve it please, I am ont in the juju team so I can't do it05:59
axwbigjools: sure05:59
bigjoolsthank you sir05:59
fwereadebigjools, ok, I did not express myself in a helpful way and I apologise for that06:42
fwereadebigjools, but I think it really is a problem that some environments now have two UUIDs and there's no clear distinction between them06:42
fwereadebigjools, would it be possible to do a quick branch that just s/environment-uuid/maas-agent-name/ and eliminates this source of confusion?06:43
bigjoolsfwereade: I'm not sure where Gavin received his advice from, but I believe it was mostly under the direction of someone in the core team and that whoever it was had a plan to resolve this06:43
fwereadebigjools, yeah, I just read the review :(06:44
bigjoolssadly this is what happens when stuff needs to go in quickly before a release06:45
fwereadebigjools, yeah, I would kinda like to figure out how the api-key fiction got created and then propagated so widely in the first place06:47
fwereadebigjools, it never even crossed my mind that it was completely made up06:47
fwereadebigjools, because it's persisted all the way through back from python days06:47
fwereadebigjools, and we never even had a maas environment to check against for such a long time06:48
rogpeppemornin' all07:04
rogpeppefwereade: the environment-uuid thing is all my fault07:04
rogpeppefwereade: i don't really see what harm it can cause tbh07:05
fwereaderogpeppe, heyhey, I saw the review, and I think I see the reasoning... but ISTM that now we have two "environment uuids" for maas environments, and I don't see how we're ever going to be able to pull them back together07:05
rogpeppefwereade: they don't join up07:05
rogpeppefwereade: the environment-uuid in the config doesn't make anywhere else, does it?07:06
fwereaderogpeppe, then why do they have the same name? it looked like it was justified on the strength of being step 1 towards picking one at prepare time rather than bootstrap time07:06
fwereaderogpeppe, which would be great, if we did it07:07
fwereaderogpeppe, but now we have an environment config with one value, used by some parts of the system, and an environment doc with another used by different parts of the system07:07
fwereaderogpeppe, and to imagine that never the twain shall meet strikes me as... optimistic07:07
rogpeppefwereade: well, currently maas has a private attribute called environment-uuid; the environment uuid in state doesn't come from or go into the config07:08
rogpeppefwereade: given that state.Initialize takes an environ config, we can easily change that at a later stage to put the environ-uuid from that into the current uuid doc07:09
rogpeppefwereade: and likewise we can easily change environs.Prepare to create it07:09
rogpeppefwereade: and when we do that, i *think* everything will just work, and the maas environ-uuid will then join up with the state uuid07:10
rogpeppewallyworld_: after sleeping on it, i *think* i know what's going on with the maas EOF bug07:11
fwereaderogpeppe, that's fine for new environments, but existing environments will need to keep both around07:11
rogpeppefwereade: is that a problem?07:12
fwereaderogpeppe, I think so, yes, because there is no longer a singular concept of environment uuid07:12
fwereaderogpeppe, and I don't see how an existing environment can ever be brought in line07:13
rogpeppefwereade: is that a problem?07:13
fwereaderogpeppe, well, yes, because an environment uuid is the only thing we have for globally identifying an environment07:13
fwereaderogpeppe, and the last thing I want is to have to respond to bug reports by saying "ah, yes, it doesn't work because you should have used the *other* environment uuid"07:14
rogpeppefwereade: is it any worse than if maas created a new attribute, for example maas-machine-identifier ?07:14
fwereaderogpeppe, yes, I think it is much worse07:15
fwereaderogpeppe, a new identifier would have been great07:15
fwereaderogpeppe, I thought I even saw you advocating that yesterday morning as I rushed by, and I thought "ah cool, everything's undercontrol"07:15
rogpeppefwereade: i advocated one or the other07:15
rogpeppefwereade: i quite liked the idea of just using environment-uuid, because i *don't* think there's a great problem currently - the maas attribute is not really visible to the user07:16
fwereaderogpeppe, you think nobody looking at the environ config is going to be fooled?07:17
fwereaderogpeppe, the environ config is most certainly visible07:18
fwereaderogpeppe, it's *more* visible to the user than the one in the environ doc07:18
rogpeppefwereade: i actually think that fixing it properly is going to be quite a small change.07:18
fwereaderogpeppe, what do we do about all the environments that have two uuids then?07:19
rogpeppefwereade: we just need to change environs/config to add UUID, change environs.Prepare to create it and change state.Initialize to use it07:19
fwereaderogpeppe, apart from the fact that we have to carry code FOREVER to handle the fact that sometimes they're different07:19
rogpeppefwereade: really?07:19
rogpeppefwereade: what code would we need?07:20
fwereaderogpeppe, code to figure out which one is "meant" at any given time07:20
fwereaderogpeppe, as it is today we will be starting envs with two uuids07:20
fwereaderogpeppe, both of which are exposed to external systems07:20
rogpeppefwereade: the other side of the coin is that in the future, we *would* like maas to use the environ uuid to tag its machines07:21
fwereaderogpeppe, and which we therefore cannot change07:21
rogpeppefwereade: and if we don't make it use environ-uuid, it will forever use some other identifier07:21
rogpeppefwereade: well, some other attribute anyway07:22
rogpeppefwereade: because it could still take its value from environ-uuid07:22
fwereaderogpeppe, yeah, that would be nice, we would be able to derive the differently-named attribute from the real uuid if a legacy one werenot already set07:22
fwereaderogpeppe, bigjools: is there *any* way we can get this fixed without releasing in this state?07:23
rogpeppefwereade: well, it's just a naming issue right?07:23
rogpeppefwereade: so we just need to change the name07:24
fwereaderogpeppe, yeah, but I am out of the loop and have no idea what timelines etc are in play07:24
bigjoolsfwereade: we are at the mercy of the release managers in ubuntu07:25
fwereaderogpeppe, if you can fix it, or ask someone else to, in time to not release with it in place, please please do so... but I have about half an hour to get up, pack, and catch a taxi to the airport07:26
bigjoolsthis is a major flaw in juju and maas and really needs to at least be a zero-day fix07:26
bigjoolsso there is time to change it I think07:26
rogpeppefwereade: ok. how about i just fix it properly? i *think* it's quite a small change, though i may be wrong07:26
fwereaderogpeppe, if you were to use environment-uuid in InitializeState that would be fine with me too07:27
bigjoolsbut one of you needs to do it AFAIC  because my engineers have done enough already07:27
rogpeppefwereade: i'll give it a go07:27
fwereaderogpeppe, can you do that please? and coordinate with jamespage I guess? tyvm07:27
rogpeppefwereade: i know what's going with the MAAS bootstrap EOF bug BTW, i'm pretty sure07:28
rogpeppefwereade: it's a very interesting conjunction of issues07:28
wallyworld_rogpeppe: hi07:40
rogpeppewallyworld_: hiya07:40
wallyworld_sorry, i was out getting my presecription filled before i go away07:41
rogpeppepwd07:41
wallyworld_rogpeppe: a reboot of the server fixed everything07:41
rogpeppewallyworld_: of the MAAS server?07:41
wallyworld_yep07:41
wallyworld_i think juju's http is flawed07:41
rogpeppewallyworld_: i don't believe the problem is fixed07:41
wallyworld_it should cope with disappearing connections07:41
wallyworld_any networking stack needs to be robust07:42
wallyworld_to connections going away07:42
rogpeppewallyworld_: i think the real problem is an underlying problem with the http protocol itself07:42
wallyworld_sure, ut the http lib needs to hide that07:42
rogpeppewallyworld_: i'm not entirely sure whether it's possible07:42
wallyworld_http libs from python et al do07:42
rogpeppewallyworld_: i wonder how they cope with this race:07:43
rogpeppewallyworld_: you use an existing connection and send a request, but the remote end drops the connection before it reads your request07:43
frankbanhi juju devs: is it safe to use ~/.juju/current-environment as a reliable way to retrieve the current default env name? or should we just consider it an internal detail?07:43
rogpeppewallyworld_: then it looks like you're getting EOF in response to your request07:43
bigjoolswhy is a request data object dealing with protocols?07:44
rogpeppebigjools: ?07:44
bigjoolsrequest has a Close on it07:44
bigjoolsseems odd07:44
rogpeppebigjools: it's an http header07:44
wallyworld_frankban: the value in that file can be overridden by JUJU_ENV i think07:44
wallyworld_frankban: so i would not rely on it07:45
rogpeppewallyworld_: when the above scenario happens, should the http client resend the http request on a new connection (possibly duplicating side-effects) or just return the error?07:45
wallyworld_not sure. i'd like to know how other libs handle it07:46
rogpeppewallyworld_: me too07:46
frankbanwallyworld_: sure, I am trying to implement this logic: if JUJU_ENV is set, use it, otherwise, retrieve the default env as set by "juju switch". So my question is: how to reliably grab that value in the second code path?07:46
wallyworld_but i've never seen this sort of behaviour elsewhere07:46
rogpeppewallyworld_: the thing is, it's usually a race with a very narrow window07:47
rogpeppewallyworld_: but in this case, an unfortunate set of circumstances conspire to make it happen every time07:47
bigjoolsrogpeppe: in that case I'd expect the transport to deal with headers that affect its operation07:47
wallyworld_frankban: what if juju switch has not been called yet?07:47
rogpeppebigjools: where should the user be able to tell the http package whether connections should be reused or not?07:48
wallyworld_not on the request object that is for sure :-)07:48
frankbanwallyworld_: it's ok, we tried and failed, and we have no default value.07:48
frankbanwallyworld_: the last chance could be looking for environments.yaml[default] actually07:49
wallyworld_frankban: is this a python script or something?07:49
frankbanwallyworld_: yes it is07:49
rogpeppewallyworld_: the reason (i'm pretty sure, though i haven't had time this morning to verify) why we were seeing the problem every time, is that just before we send the request that fails, we do some very cpu-intensive operations for more than 5 seconds07:50
wallyworld_frankban: so i think the order juju-core checks is: juju_env, juju switch file, env.yaml07:50
wallyworld_frankban: so if you do that, you should be ok07:51
frankbanwallyworld_: so, in order: JUJU_ENV -> juju switch -> environments.yaml[default] -> error "please specify an env name".07:51
wallyworld_frankban: i think so07:51
rogpeppewallyworld_: and that meant that the goroutine that usually sees the remote connection being dropped was not being scheduled in that time07:51
frankbanheh07:51
bigjoolsrogpeppe: I'd have a higher level function on the transport rather than exposing protocol details on a request object07:51
rogpeppebigjools: the transport is actually lower level here, no?07:51
rogpeppebigjools: and most http clients don't see it07:52
bigjoolsrogpeppe: not in that sense, I mean a function on the transport to say whether to do it or not.  manipulating headers is low-level07:52
fwereaderogpeppe, hey, change of heart -- please *don't* use environment-uuid, just change the name to something maas-specific07:52
frankbanwallyworld_: yes my question is about the "juju switch" part: parsing the output seems fragile, and I was wondering if  ~/.juju/current-environment is considered an internal detail. anyway, implementing something like "juju switch --format json" could be a good idea07:52
fwereaderogpeppe, I'm not convinced we have properly thought through the issues witrh setting it early07:52
fwereaderogpeppe, and I don't want maas/juju collisions07:52
rogpeppefwereade: really?07:52
bigjoolsfwereade: I chatted to wallyworld_ about this earlier and we concluded that its akin to a private bucket name07:53
fwereaderogpeppe, really really07:53
rogpeppefwereade: ok - i've almost done it, BTW07:53
wallyworld_frankban: i just checked. the checks are done in the order specified07:53
fwereaderogpeppe, just call it maas-agent-name or something07:53
jamespagefwereade, rogpeppe: what do I need to know about?07:53
fwereaderogpeppe, sorry, but I just want to avoid adding more little threads connecting different bits07:53
fwereaderogpeppe, that at least betrays its actual usage07:54
fwereaderogpeppe, then, as a followup, when we have done early-set-uuid properly07:54
fwereaderogpeppe, we can make maas-agent-name derive therefrom if unset07:54
wallyworld_frankban: the current-environment file just has a single line with the env name. ideally i agree about the output bit you suggest. but i can't see it changing07:54
rogpeppefwereade: actually, we shouldn't allow maas-agent-name to be set explicitly07:54
fwereaderogpeppe, it will already be set explicitly inenvironments we upgrade,ok?07:55
frankbanwallyworld_: ok, so it's ok to use the current-environment file, correct?07:55
rogpeppefwereade: i don't *think* so07:55
rogpeppefwereade: i think the only time it could be explicitly set is if someone specifies it in their environments.yaml07:56
fwereaderogpeppe, it *will* because we will set it *now* and it will need to persist in the env07:56
wallyworld_frankban: well, given there's nothing else, then yes. but i think we need a command to print the current env name to hide the details07:56
fwereaderogpeppe, when we upgrade thosesubsequently we will need to deal with it07:56
fwereaderogpeppe, I don't much care how we react to its resence in Prepare, cutting it off there doesn't seemcrazy07:56
wallyworld_frankban: we can whip something up next week at the sprint perhaps07:56
axwwallyworld_, frankban "juju switch" shows you the current env07:57
rogpeppefwereade: i think that's fine - maas will just always use maas-agent-name07:57
rogpeppefwereade: but at Prepare time, it can derive maas-agent-name from environ-uuid07:57
wallyworld_frankban: oh, we already do it it seems07:57
wallyworld_that i didn;t realise, sorry for the noise07:57
frankbanaxw, wallyworld_: yes "juju switch" is already there, but it seems to me fragile to parse the output, that's why I was suggesting something like "juju switch --format json"07:58
axwfrankban: ah sorry, I missed that07:58
frankbanaxw: the current output is: Current environment: "ec2"07:58
fwereaderogpeppe, agreed Ithink07:59
wallyworld_frankban: almost json :-)07:59
rogpeppefwereade: it does mean that we'll need the maas-agent-name attribute indefinitely in the future, which is what i was hoping to avoid07:59
axwfrankban: agreed, I think we should have a machine readable output mode07:59
wallyworld_remove the space, add {}07:59
fwereaderogpeppe, just keep environment-uuid out of there for now, Ifear the tentacles' reach07:59
wallyworld_axw: i agree too07:59
rogpeppefwereade: but i see your point about having two things called "environment-uuid" being confusing too07:59
frankbanwallyworld_, axw: do you want me to file a bug about "juju switch --format machine-readable"?07:59
* fwereade has to go right now, thanks guys08:00
wallyworld_yes08:00
rogpeppefwereade: safe journeys08:00
axwlater fwereade, see you next week08:00
fwereadecheers08:00
axwfrankban: yes please08:00
* fwereade has to shut down an env beforeheflies actually08:00
rogpeppeaxw: do you know much about the http protocol?08:00
axwrogpeppe: I know enough to be dangerous, but maybe not intimately enough... why?08:01
rogpeppeaxw: just wondering how most http clients deal with the inherent race involved in reusing connections when the client might drop the connection at any moment08:01
TheMuemorning08:02
frankbanaxw, wallyworld_: it seems there is one already: bug 119324408:02
_mup_Bug #1193244: juju env could be friendlier to scripts <improvement> <juju-core:In Progress by themue> <https://launchpad.net/bugs/1193244>08:02
axwcool08:02
axwgood morning TheMue08:02
wallyworld_frankban: we'll fix that for sure08:02
TheMueaxw: oh, morning, came in at the right moment?08:02
axwheh :)08:02
frankbanwallyworld_: great, thanks08:03
wallyworld_frankban: well, TheMue will :-)08:03
frankban:-)08:03
TheMuewallyworld_: yeah08:03
axwrogpeppe: sorry, I know the protocol well enough, but not how most clients work in that regard08:03
TheMuefrankban: seen my last comment there? regarding the way to control the output?08:03
frankbanTheMue: --raw sounds good08:04
rogpeppeaxw: we came across an issue that triggered that race reliably08:04
wallyworld_rogpeppe: isn't it the server that drops the connection rather than the client?08:05
rogpeppeaxw: so we'd see the connection drop at *almost exactly* the same moment we make a request08:05
rogpeppewallyworld_: yes, but the Go client wasn't *seeing* the connection being dropped because it was busy doing other things08:06
wallyworld_rogpeppe: sure, but i was referring to your comment above that the client dropped the connectiopn08:06
wallyworld_just clarifying08:06
rogpeppewallyworld_: ha, yes08:06
rogpeppewallyworld_: i meant server there08:06
wallyworld_:-)08:07
TheMuefrankban: fine, then I will do it that way08:07
frankbanTheMue: thanks!08:08
axwrogpeppe: tbh, this seems like the Go stdlib should handle.08:08
axwit's suggested that you reuse clients, and that it's safe to do so08:09
rogpeppeaxw: agreed. but i'm not quite sure how it can do so reliably08:09
wallyworld_axw: that's what me and bigjools think too08:09
rogpeppeaxw: istm that this is an inherent race in the http protocol08:09
rogpeppeaxw: and i'm not sure how it can be dealt with other than just trying to reduce the window for the race08:10
axwhmmm08:11
dimiternmorning all08:12
rogpeppeaxw: (the window could certainly be smaller in the Go http client)08:12
rogpeppedimitern: yo!08:12
axwmorning dimitern08:12
rogpeppeaxw: actually, on reading of the rfc 2616, it looks like the Go http client is just wrong here08:16
rogpeppe"08:16
rogpeppeClient software SHOULD reopen the08:16
rogpeppe   transport connection and retransmit the aborted sequence of requests08:16
rogpeppe   without user interaction so long as the request sequence is08:16
rogpeppe   idempotent (see section 9.1.2).08:16
rogpeppe"08:16
axwyeah, but "so long as the request sequence is idempotent"08:16
axwI was thinking that, but how do you guarantee idempotency?08:16
axwthat's an application level thing08:17
rogpeppeaxw: yeah, but it defines GET, HEAD, PUT and DELETE as being idempotent08:17
rogpeppeaxw: i guess that means you still have a potential problem for POST08:17
* axw wonders how many applications are not idempotent for those methods ;)08:17
rogpeppeaxw: http://tools.ietf.org/html/rfc2616#section-9.1.208:17
rogpeppeaxw: i wonder that too08:17
axwrogpeppe: anyway, "wrong" is maybe too harsh for not implementing an RFC "SHOULD"08:18
axwbut it would certainly be useful for an option at least08:18
rogpeppeaxw: yeah08:19
TheMuedimitern: morning, how has PyCon been?08:19
dimiternTheMue, ah, it was interesting and not too long :)08:20
TheMuedimitern: I somehow left Python a few years ago, so would almost have to relearn it. ;)08:21
rogpeppeaxw, wallyworld_: there's an outstanding Go issue for this actually: https://code.google.com/p/go/issues/detail?id=467708:22
axwrogpeppe: heh, nice, same conclusion :)08:22
jamespagefwereade, rogpeppe: what do I need to know about? (hint: release of saucy is tomorrow - if I need to upload it needs to be today otherwise I'm in SRU territory)08:22
wallyworld_rogpeppe: fat lot of good that is - it's not going to e fixed08:23
rogpeppejamespage: i need to make a small naming fix to the maas provider08:23
rogpeppewallyworld_: it's not going to be fixed for 1.2, yeah08:23
wallyworld_so go's http is broken and there's nothing we can do about it. oh joy08:23
wallyworld_wtf08:23
rogpeppewallyworld_: we can set the Close field08:23
wallyworld_and remove connection reuse?08:24
rogpeppewallyworld_: or change the transport to allow no idle connections08:24
wallyworld_why oh why does Go get so much so wrong08:24
rogpeppewallyworld_: yeah08:24
wallyworld_version control, proper hhtp stack etc etc08:24
wallyworld_it's not like comp sci is a new field of science08:25
* wallyworld_ is very frustrated08:25
axwI like this comment from the Chromium dev: "* If you pipeline requests and get a transport error, we pray that HEADs and GETs are actually idempotent and retry."08:25
axwwallyworld_: I'd actually prefer that this not be enabled by default, because it's generally not safe to assume GETs are idempotent08:26
bigjoolsdon't fret wallyworld_, you get the pleasure of my company on Saturday for 15 hours08:26
axwopt-in would be good08:26
wallyworld_oh joy08:26
wallyworld_axw: sure, but we don't kow if it's needed or not at any time08:26
bigjoolsGET is supposed to be idempotent, are there really crackful websites that are not?08:27
wallyworld_+1 to that08:27
axwbigjools: supposed to be, sure, but people do all sorts of wrong things in their web applications08:27
bigjoolsthen that's their tough titty08:27
wallyworld_axw: that's no excuse for not assumimg the spec holds08:28
axwnot to mention HTTP servers and proxies that don't follow protocols08:28
bigjoolstools should not behave as stupidly08:28
wallyworld_no excuse08:28
wallyworld_the wrong implementations should get fixed08:28
wallyworld_not the clients08:28
bigjoolsjust look at the havoc IE6 created08:28
wallyworld_or else the problem will never be fixed08:28
wallyworld_ie6 is a great example08:28
wallyworld_axw: rogpeppe: so you'd think a robust http lib would be the cornerstone requirement of any new langage. and yet they say not fixed for 1.2????? what else could be more important08:30
wallyworld_TLS is another example08:30
wallyworld_they refused to accept the need for it - so we are forced to fork08:31
rogpeppewallyworld_: i trust Adam Langley when he says that tls renegotiation is badly broken08:31
wallyworld_well it works for us08:32
wallyworld_or if broken, just fix it already08:32
wallyworld_and give us a feature complete http stack08:32
wallyworld_not like it's important or anything08:32
rogpeppewallyworld_: there's always a tension when trying to write clean software that's dealing with crappy standards08:33
wallyworld_sure, but it's not like Python, C++ etc etc etc etc weren't around to learn from08:33
wallyworld_solved problems08:33
wallyworld_meanwhile Juju breaks and we deal with the fallout08:34
wallyworld_customers don't care the Go is deficient, they blame Juju and us08:34
wallyworld_and we stuff Juju full of all of these hack and workarounds to paper over Go's cracks08:34
jamespagerogpeppe, bug reference?09:10
jamespageI'd like to raise ubuntu tasks now so they get noticed in the right places09:10
rogpeppejamespage: i'll just file a bug for it09:10
rogpeppejamespage: https://bugs.launchpad.net/juju-core/+bug/124042309:13
_mup_Bug #1240423: provider/maas: environment-uuid is the wrong name to use for the machine disambiguation tag <juju-core:New> <https://launchpad.net/bugs/1240423>09:13
jamespagerogpeppe, is that linked to bug 122927510:02
_mup_Bug #1229275: [maas] juju destroy-environment also destroys nodes that are not controlled by juju <maas> <theme-oil> <juju:Triaged> <juju-core:In Progress by thumper> <maas (Ubuntu):Triaged> <https://launchpad.net/bugs/1229275>10:02
rogpeppejamespage: yes10:03
jamespagerogpeppe, so I'm right in saying the juju-core 1.16.0 with MAAS in Saucy will exhibit this problem?10:04
rogpeppejamespage: the above bug? yes, i believe so, unless https://code.launchpad.net/~allenap/juju-core/maas-environment-uuid/+merge/191146 has landed in 1.16 yet10:05
jamespagerogpeppe, so I'm going to need that as a fix for 1.16.0 as well10:19
rogpeppejamespage: yes10:20
jamespage:-)10:20
rogpeppejamespage: i hope to provide one in the next hour or so10:20
jamespagerogpeppe, lovely - thanks!10:21
rogpeppejamespage: i've changed the code - i just need to test in various places10:21
rogpeppeallenap: i'd appreciate a review of this please: https://codereview.appspot.com/14741045/10:45
dimiternrogpeppe, mgz, TheMue, others - standup10:46
rogpeppedimitern, mgz, TheMue: ^ (this urgently needs to go in BTW)10:46
TheMueoh10:47
rogpeppebigjools: ^10:48
natefinchrogpeppe: the standup link seems not to be working?10:48
rogpeppenatefinch: it works for me10:48
rogpeppenatefinch: https://plus.google.com/hangouts/_/calendar/am9obi5tZWluZWxAY2Fub25pY2FsLmNvbQ.mf0d8r5pfb44m16v9b2n5i29ig?authuser=110:48
* TheMue => lunch11:35
rogpeppemgz, natefinch, dimitern, TheMue: PLEASE could someone review this? https://codereview.appspot.com/14741045/12:19
natefinchrogpeppe: it's already open in my browser :)\12:19
rogpeppenatefinch: thanks12:19
mgzrogpeppe: whoops, didn't hit publish12:20
mgzrogpeppe: HIT IT.12:21
mgzwhoops caps, but kinda funny.12:22
natefinchrofl12:22
rogpeppemgz: i thought that was deliberate12:23
natefinchrogpeppe: me too12:23
rogpeppeanyone around here know about the new maas agent_name semantics?12:23
rogpeppemgz: i've just realised that the branch might be wrong12:23
mgzhm, in what way?12:24
natefinchmgz: I couldn't help but hear it like this: http://www.youtube.com/watch?v=erbL_BxITHw12:24
rogpeppemgz: because it still filters by agent_name even when there's no maas-instance-uuid12:24
mgzI didn't closely review the first branch, so may be missing some subtelties12:24
rogpeppemgz: i don't know what the agent_name filtering semantics are though12:24
rogpeppeallenap: ping12:25
mgzthe filtering is all our code, no?12:27
mgzyeah, the filter is just our standard stuff12:27
rogpeppemgz: um, it looks like the filter is passed directly to the maas API GET request12:28
mgzwhich does also raise the back-compat question...12:28
mgzrogpeppe: hm, I see what you mean12:28
rogpeppemgz: so i don't know what will happen if you pass an agent_name="" filter12:29
rogpeppemgz: it *might* match all instances, or it might not.12:29
rvbarogpeppe: it will match all instances12:29
mgzwell, that's not something you've changed in this branch12:29
rogpeppemgz: no it isn't12:29
rogpeppervba: ok, that's good12:29
rvbaIt is good indeed :).12:30
rogpeppervba: and there's no problem having a blank agent name in the acquire params?12:30
rvbarogpeppe: that's fine too (I was sure of it but I still tested it this morning.)12:30
rvbaI mean, what I tested was a version of juju using agent_name with a version of MAAS which doesn't know about it.12:31
rvbanot exactly what you asked12:31
rogpeppervba: yeah, true12:32
rvbaBut using a blank agent name is the same as not providing one.12:33
rogpeppervba: it would be nice to make sure. or, alternatively, a less risky strategy is just to lose the agent_name params if the agent name is blank12:33
rogpeppervba: so you can't filter instances that have blank agent names?12:33
rvbaLike I said, as far as MAAS is concerned, this is the same.12:33
rvbaNo.12:33
rvbaerr12:34
rvbaYes you can do that.  But you have to know that a blank agent name is the default.12:34
rogpeppervba: ah, so using a blank agent name isn't *quite* the same as not providing one, then?12:35
rvbaIt is exactly the same.12:35
rogpeppervba: so how would you distinguish between a) asking for all instances regardless of agent_name and b) asking for any instances which happen to have a blank agent_name ?12:36
rvbarogpeppe: no, I was wrong, when listing instances, it's not the same.12:37
rvbaSorry for the confusion.12:37
rogpeppervba: ok, cool. i actually think that's better for our purposes12:37
rogpeppervba: as it means that existing maas juju deployments won't see new environments12:38
rogpeppervba: as long as they've been upgraded12:38
rvbaRight.12:38
rvbaBut you can have only one of these "old" environments.12:38
TheMuefrankban: ping12:46
rogpeppervba: yeah - that's a current restriction though12:46
frankbanTheMue: pong12:46
rogpeppervba: i'd appreciate it a lot if you could take a glance through this before i merge: https://codereview.appspot.com/1474104512:46
TheMuefrankban: just seen in the channel log that you talked about a json output of env/switch12:47
rogpeppemgz: i've renamed maas-instance-uuid to maas-agent-name throughout as it seemed to make more sense once i read through a bit more of the code12:47
TheMuefrankban: I understood raw as being simply the env name(s)12:47
TheMuefrankban: in case of json I would prefer a different flag than --raw12:48
TheMuefrankban: what do you say?12:48
frankbanTheMue: since "juju switch" will always return just a single string, --raw seems reasonable. I was thinking about --format just for symmetricity with "juju api-endpoints", but I don't think that's required12:52
frankbanTheMue: what is --raw supposed to return if no default env is set? just an empty string?12:53
TheMuefrankban: it is <not specified>, at least today12:54
rvbarogpeppe: looking now12:54
rogpeppervba: thanks12:54
frankbanTheMue: hum... perhaps an empty string is better? I guess spaces and <> are not allowed in env names, so that could be ok, but a user still have to parse the return value, or just compare with "<not specified>".12:57
TheMuefrankban: I prefer the "<not specified>" as it is more explicit than just an empty string12:59
frankbanTheMue: the retcode is in both cases, right?13:00
frankbanTheMue: is 0 I mean13:00
TheMuefrankban: yep13:01
frankbanTheMue: if you are ok with users comparing against "<not specified>" the it's all good. But then we must ensure that string will not change in the future. On the other hand, if --raw is the machine friendly command, I am not sure it has to be explicit for humans, but as said, I am +1 on your plan13:06
natefinchfrankban: when would a script ever need to check the output of juju switch? Couldn't it just always do juju switch <foo> or -e <foo> each time?   Not saying no one will ever do it, it just seems like an unnecessary step13:08
rogpeppehmm, bzr's "merge specific revisions" logic seems not to work very well13:09
rogpeppenatefinch: i agree13:09
abentleyrogpeppe: how do you mean?13:10
rogpeppeabentley: i just had some conflicts with some very weird diffs in13:11
abentleyrogpeppe: Well, if you think it's bzr's fault, give me steps to reproduce, and I can have a look.  I did write most of that code.13:12
frankbannatefinch: assume you have a script that needs to bootstrap an environment. that script can either 1) force the user to pass a "-e" parameter or 2) make that parameter optional. In the second case, we still want to ensure an environment is ready to be bootstrapped, and we might also want to grab the env name. So we check JUJU_ENV, then we ask to juju switch, and finally we look inside environments.yaml[default]13:12
rogpeppeabentley: here's an example: http://paste.ubuntu.com/6245564/13:13
abentleynatefinch: A script would need to check the output of "juju switch" in order to determine what the current environment is.  For example, I have a script that runs "nova" using the credentials of the current juju environment.13:14
rogpeppeabentley: note the line sitting in the middle of nowhere - it actually came from a test function in the merge-source that was almost entirely lost13:14
rogpeppeabentley: ah, sorry, you're talking about juju switch, not bzr :-)13:14
natefinchabentley: the script can tell if *an* environment is chosen, but not necessarily the right now.  Better to just always switch to the right one first13:15
rogpeppeperhaps what we need is a command that prints the current env name13:15
natefinchbrb screaming baby13:15
TheMuefrankban: in case of a set JUJU_ENV the command juju env returns it13:15
rogpeppeso scripts aren't trying to second-guess JUJU_ENV, juju switch, environments.yaml juju logic13:16
abentleynatefinch: The right one is the current one.  The best way to determine that is to ask juju what the current one is.13:16
mattywrogpeppe, is there a canonical example of writing a go client to connect to the api somewhere in the juju source?13:16
rogpeppemattyw: i don't think so.13:16
abentleynatefinch: One way to do that is to run "juju switch" with no env specified.13:17
frankbanTheMue: yes, and also if current-environment is missing, juju switch seems to return the default in envs.yaml.13:17
rogpeppemattyw: there's a command in launchpad.net/juju-utils that does, but it's quite possible it doesn't compile currently13:17
abentleyrogpeppe: But I bet if you look at THIS and OTHER, that line was preserved in both.13:18
* TheMue will come back in a few moments13:18
rogpeppeabentley: OTHER and THIS were both as i expected13:18
rogpeppeabentley: but usually i expect the merge to contain all the lines from both13:19
rogpeppeabentley: in this case almost an entire function had gone missing, leaving me wondering what else might have gone13:20
abentleyrogpeppe: That's not what merge does.  It attempts to apply changes from both, which is both insertion and deletion.13:20
rogpeppeabentley: i don't believe the source branch deleted anything in that place13:20
rogpeppeabentley: note that this isn't a straight branch merge i'm talking about here13:21
rogpeppeabentley: i did: merge -r1984..1985 trunk13:21
abentleyrogpeppe: Okay, and trunk here is juju-core?13:21
rogpeppeabentley: to try to bring some trunk changes into 1.1613:21
rogpeppeabentley: yeah13:21
rogpeppeabentley: you could duplicate the problem yourself easily, if you're interested13:22
abentleyrogpeppe: And you're merging into which branch?13:22
rogpeppeabentley: into lp:juju-core/1.1613:22
natefinchabentley: (sorry, had to step out for a second) I'm confused as to why the script needs to know the current environment name, if it just assumes whatever is current is the right one.13:31
abentleynatefinch: Because it needs to parse environments.yaml and determine what the correct values are for OS_REGION_NAME, OS_USERNAME, OS_PASSWORD, OS_TENANT_NAME and OS_AUTH_URL13:35
abentleynatefinch: That varies depending on whether the current env uses my personal credentials or my team credentials.13:36
natefinchabentley: why is the script parsing environments.yaml?  That's for configuring juju... not configuring the script13:37
natefinchabentley: oh wait, you mean, it's going to set the environment variables based on what environment is set in juju?13:38
abentleynatefinch: Right.13:38
natefinchabentley: that seems like... a bad idea.  If you're going to store the credentials in the script, why not store them in environments.yaml?  Or do we not support those particular values in environments.yaml?13:39
abentleyI'm not storing the credentials in the script, I'm getting  them from environments.yaml.13:40
natefinchabentley: I'm confused again.  If stuff is stored in environments.yaml already, why do you need to set them as environment variables?  Won't juju pick them up from environments.yaml itself?13:40
rogpeppervba, natefinch, TheMue, wallyworld_: this CL merges the maas changes into 1.16 https://codereview.appspot.com/1474604413:41
abentleynatefinch: The script runs "nova", not "juju".13:41
natefinchabentley: ahh, that's what I was misunderstanding.13:42
rogpeppervba, natefinch, TheMue, wallyworld_: i've done it as a single branch to save an hour or so of committing overhead; i hope that's ok13:42
natefinchabentley: I see you already said that, but I guess I missed it.13:42
rogpeppenatefinch: do you have access to a maas environment that you could run up a quick live test of that branch on, by any chance?13:43
natefinchrogpeppe: I wish I did... the virtual maas environment I had set up got nuked somehow13:44
rogpeppenatefinch: hmm13:44
rogpeppeanyone got a maas environment available?13:44
abentleynatefinch: I have another script that uses the current juju environment (by default) to run sshuttle.  Again, it uses "switch" to determine the current environment.13:45
rogpeppeabentley: could you clarify for me why the script needs to know the name of the current environment?13:46
rogpeppeabentley: (i'm not saying you haven't got a good use case, but i'm interested in what it is)13:47
rogpeppeif i wasn't clear, i would really like a review of this CL please - it *needs* to land today. https://codereview.appspot.com/1474604413:53
rogpeppervba, axw, natefinch, TheMue, wallyworld_: ^13:53
rvbarogpeppe: ah ok, I didn't get it had to be reviewed again.13:54
rogpeppervba: well, i think it probably should be, as this is the actual change that's landing on 1.16, and i've had to do some manual merge conflict resolution13:54
rvbaokay13:54
rogpeppervba: thanks13:55
TheMuerogpeppe: just starting review13:56
TheMuerogpeppe: reviewd14:05
rogpeppeTheMue: thanks14:05
TheMueeh, reviewed14:05
rogpeppeTheMue: those are changes that should be made in trunk - i'm not making them independently in this review14:06
* rogpeppe grabs a bit to eat14:06
rogpeppebite14:06
TheMuerogpeppe: ok14:06
abentleyrogpeppe: My use case for the first script is to use nova to manipulate the instances of my current juju environment, especially "nova status" and "nova add-floating-ip".14:13
abentleyrogpeppe: The use case for the second script is to be able to access the current environment using an SSH tunnel, since the bootstrap node has a private IP.  I use the current environment name to determine which "juju-*-machine-0" to ssh into for my tunnel.14:15
rogpeppeabentley: so this is useful only because the openstack provider uses the environment name to name some of its resources, right?14:17
rogpeppeabentley: s/this/switch printing the current environment name/14:18
abentleyrogpeppe: Yes, the second script is useful because the instance name can be predicted from the environment name.  The first script is useful regardless.14:18
rogpeppeabentley: we'll probably change this behaviour in the future, BTW - the environment name doesn't make for a very good unique key.14:19
abentleyrogpeppe: In the context of a given nova account, I think there's some sense in requiring unique environment names.  Things can be very confusing otherwise.14:22
rogpeppeabentley: we are moving towards the idea of using the environ UUID14:22
rogpeppeabentley: which will be generated at bootstrap time14:22
rogpeppeabentley: so if you've destroyed an environment but for some reason some instances are still around, there's then no possibility of crosstalk with a newly bootstrapped environment, even if it happens to have the same name14:23
rogpeppeabentley: is that the kind of thing you think might be confusing?14:24
abentleyrogpeppe: That seems okay, as long as it's not ephemeral.  Writing essential info to *.jenv alone makes sharing accounts painful.14:24
rogpeppeabentley: we already write essential info to .jenv alone14:24
abentleyrogpeppe: I know,  and it's evil.14:24
rogpeppeabentley: to share accounts, you'll need to share the .jenv file14:24
rogpeppeabentley: (but you won't need to share anything else at all)14:25
rogpeppeabentley: i don't see any other way that we can have local state14:25
abentleyrogpeppe: And then someone tears down the environment and re-bootstraps, and everyone's broken.14:25
abentleyrogpeppe: Write it to environments.yaml instead of .jenv.14:25
abentleyrogpeppe: Then everyone with the same config can access the same environment.14:26
rogpeppeabentley: we can't without losing comments etc14:26
abentleyrogpeppe: Losing comments is better than breaking other team members.14:26
abentleyrogpeppe: And perhaps there is a yaml parser out there that doesn't lose comments.  There certainly are for ini files.14:27
rogpeppeabentley: most YAML parser out there can barely parse YAML :-)14:27
rogpeppeparsers14:27
rogpeppeabentley: it's not just comments - there are many ways to format YAML14:28
rogpeppeabentley: i'm afraid the only way to do it is to share the .jenv files (possibly in a shared filesystem)14:28
abentleyrogpeppe: No, that's not acceptable as long as .jenv files are deleted by destroy-environment.14:29
rogpeppeabentley: if you destroy an environment and then bootstrap it again, it's actually not the same environment any more14:29
abentleyrogpeppe: The same people need to be able to access it, though.14:29
abentleyrogpeppe: Why can't you store all the .jenv-unique data in swift/s3?14:30
rogpeppeabentley: because some of that data is used to work out which swift/s3 bucket to talk to14:31
rogpeppeabentley: for instance, control-bucket is now automatically generated14:31
abentleyrogpeppe: but if I specify control-bucket in environments.yaml, it's respected?14:32
rogpeppeabentley: yes, currently14:32
rogpeppeabentley: i agree that this is a very significant change in behaviour BTW, and i understand your point of view14:32
abentleyrogpeppe: It is very discouraging to see this.  It seems like juju is going to get worse and worse from the perspective of teams.14:33
rogpeppeabentley: i think the way of the future is to have a service that stores .jenv files14:33
rogpeppeabentley: (the code is actually written with that in mind)14:34
abentleyrogpeppe: I don't think we need a new service.  Just a swift/s3 bucket that doesn't change.  One per provider would be fine.14:36
rogpeppeabentley: s3 isn't great for shared write access14:37
rogpeppeabentley: i.e. you can't change anything atomically AFAIK14:38
abentleyrogpeppe: Okay, but I don't want to live with a worse-and-worse juju for a long time because the jenv sharing service is more complex than necessary.14:38
abentleyand therefore takes more time to implement.14:38
rogpeppeabentley: essentially all it needs to do is implement the Storage interface defined in environs/configstore/interface.go14:40
rogpeppeabentley: there's some thinking to be done as to how to implement the encryption14:42
rogpeppeabentley: (i.e. do you let the server see the plaintext contents of the .jenv files?)14:43
rogpeppeabentley: but i don't anticipate it being more than a week's worth of work for the server and maybe another week to integrate it into juju-core.14:44
abentleyrogpeppe: Is there a plan to split environments.yaml into multiple files, or is that a misunderstanding of *.jenv files?14:46
rogpeppeabentley: there's a plan to lose environments.yaml entirely14:46
abentleyrogpeppe: What replaces it?14:46
rogpeppeabentley: another config file based around somewhat different abstractions14:47
rogpeppeabentley: fwereade has a better idea than me - he thrashed out some of the details with bdfl14:47
abentleyrogpeppe: For teams, it would be nice if each environment had its own config, because some environments are team environments and some are personal environments.  It's easier to maintain the team environments if they're not intermixed with personal ones.14:48
abentleys/config/config file/14:48
rogpeppeabentley: personally, i'd be happy if the environments.yaml file became simply a URL to the config-file storage server and probably a key for that too14:49
rogpeppeabentley: then we'll be able to operate more coherently in a team environment14:50
abentleyrogpeppe: I'd be unhappy, because then it wouldn't have the openstack credentials, and my first script wouldn't work.14:50
rogpeppeabentley: your script could use the server to fetch the openstack credential14:50
rogpeppes14:50
abentleyrogpeppe: Don't I need credentials in order to use the server?14:50
rogpeppeabentley: not provider-specific credentials14:51
abentleyrogpeppe: Doesn't that make it harder for private clouds?  I would think you'd want it to be a per-provider service at least.14:52
rogpeppeabentley: (if we do it right, a single config storage server could easily serve the needs of thousands of clients - the demands aren't high)14:52
rogpeppeabentley: for private clouds, you would indeed need to run a server somewhere.14:52
rogpeppeabentley: or have a file-based mechanism that you could use too14:54
rogpeppeabentley: i'm thinking off the top of my head here BTW - none of this stuff has been discussed yet in the team14:56
w7zyeay, finally internet back14:59
sinzuirogpeppe, Thank you for fixing the gnuflag branch. I will remove the workarounds from the scripts15:14
abentleyrogpeppe: About the merge, the source deletes 4 lines from environprovider_test.go: http://pastebin.ubuntu.com/6246103/15:14
abentleyrogpeppe: If you look at the diff of provider/maas/environprovider_test.go (with conflicts and everything), you'll see that no function was deleted.  What happened is that trunk altered a function that doesn't exist in 1.16.15:16
abentleyrogpeppe: That's why there's a conflict with just one line showing, because that's the only line of the non-existent function that trunk wanted to change.15:17
rogpeppeabentley: hmm, which function was altered that doesn't exist in 1.16?15:18
abentleyrogpeppe: TestUnknownAttrsContainEnvironmentUUID15:21
rogpeppeabentley: hmm, interesting. i think i must have got the merge command wrong. i thought that "bzr merge -c 1234" was equivalent to "bzr merge -r 1234", but i'm guessing that it's actually equivalent to "bzr merge -r 1233..1234"15:28
* rogpeppe is not worried that more stuff has been lost15:28
rogpeppes/not/now/15:28
abentleyrogpeppe: The latter is correct.15:28
abentleyrogpeppe: You may have actually wanted bzr merge -r 1984..1985?15:29
abentleyrogpeppe: I mean you may have actually wanted bzr merge -r 1983..1985?15:29
rogpeppeabentley: yes15:29
abentleyrogpeppe: It may help to know that the -r parameters affect diff the same way as merge.15:30
rogpeppeabentley: yeah, i'd thought they were slightly different15:30
abentleyrogpeppe: So if you do diff -r 1983..1985, you'll see all of the changes that merge will attempt to integrate.15:30
rogpeppeabentley: ok, it seems we're lucky in this case15:32
rogpeppeabentley: the only changes that were made in 1984 were overwritten by changes made in a later branch that i also merged15:32
* rogpeppe slaps own wrist and considers himself once-bitten15:32
rogpeppeabentley: thanks for solving the mystery for me15:34
abentleyrogpeppe: You're welcome.15:34
rogpeppeif anyone's been following it, i now have a tiny demo program of the net/http problem that has been causing us problems: http://paste.ubuntu.com/6246231/15:48
rogpeppei'm thinking this might be the cause of bugs like this: https://bugs.launchpad.net/juju-core/+bug/122825515:49
_mup_Bug #1228255: Live bootstrap tests fail on canonistack <test-failure> <juju-core:Triaged> <https://launchpad.net/bugs/1228255>15:49
rogpeppesee my comment on https://code.google.com/p/go/issues/detail?id=467716:02
rogpepperebooting16:04
natefinchrogpeppe: interesting about the http thing, though I think your bug report has a typo: "It succeeds when run with GOMAXPROCS>0."16:09
natefinchrogpeppe: pretty sure gomaxproxs is always >0   ;)16:09
TheMuenatefinch: as long as your system doesn't blocks16:12
rogpeppenatefinch: oops16:17
rogpeppenatefinch: correct16:17
rogpeppenatefinch: corrected16:17
TheMueso, have a good night guys17:14
TheMuecu tomorrow17:14
natefinchTheMue: g'night17:14
rogpeppei'm also off now17:19
rogpeppeg'night all17:19
natefinchg'night!17:19
thumperabentley: how's tricks?20:04
abentleythumper: I'm doing fine.  How are you?20:05
thumperabentley: pretty good20:05
thumperabentley: I was wondering about your bzr plugin for lbox20:05
thumperabentley: did you have one?20:05
thumperwell, reitveld20:05
thumperI guess20:05
abentleythumper: Yes, since reitveld was updating launchpad and juju-gui has tarmac, it really just sets the commit message and marks the MP approved.20:07
gary_posterthumper, abentley http://jujugui.wordpress.com/2013/05/24/thanks-to-diogo-matsubara-well-be-migrating-to/20:07
smoserhttps://bugs.launchpad.net/juju-core/+bug/124066720:07
thumperabentley: ah, but not for submitting?20:07
_mup_Bug #1240667: Version of django in cloud-tools conflicts with horizon:grizzly <ubuntu-cloud-archive:Confirmed> <juju-core:Confirmed> <https://launchpad.net/bugs/1240667>20:07
thumperwell, proposing20:07
thumpersmoser: is that for maas?20:08
smoserits really only solvable in juju i think20:08
abentleythumper: No, I didn't do one for proposing.  Is lbox propose giving you grief?20:08
smosernot specific to maas20:08
thumperabentley: regularly20:08
thumpersmoser: but juju doesn't use django20:09
gary_posterthumper, API server is falling over (it closes the socket).  that usually means there's some juju error to look at.  could you direct me towards the probably pertinent logs?20:09
gary_posteroh wait20:09
gary_postermachine 020:09
abentleythumper: Any chance of switching to MPs?20:09
thumpergary_poster: probably the api logs on machine-020:09
thumperabentley: only when we get inline commenting20:09
thumperabentley: I've raised this before20:09
smoserdjango is in the cloud archive for maas, yes. but juju added the cloud archive to the node.20:10
smoserwhich is where the problem came from.20:10
thumperlbox diff generation isn't as good as LPs20:10
thumpersmoser: ah20:10
thumpersmoser: that's annoying20:10
abentleythumper: Wish I had a quick solution for you, but I guess you'll have to roll your own if you want to propose on Reitveld.20:10
thumperI thought you had one that dealt with pipelines20:11
thumperI often forget the -req bit for lbox20:11
smoserthumper, yeah. its a problem there for django, but it could be a problem for anything really.20:11
smoserand coudl expose itself with lxc. or even mongodb.20:11
* thumper nods20:12
thumpersmoser: any plan or ideas?20:12
smosernone that i like.20:12
abentleythumper: lp-propose handles pipelines.  I don't know of a reitveld equivalent.  I can dig up the plugin that generates diffs if you like.20:12
thumperabentley: nah20:12
thumperabentley: probably not going to get time to look at it20:13
smoserwhen does juju need to enable the cloud archive ?20:13
thumpertoo much else on20:13
smoserseem my comments there, am I correct?20:13
* thumper looks at the bug20:13
smoserif so, then the most reasonable solution is only to use cloud archive on bootstrap node or "un-containerized" node (that would then want to create containers with lxc)20:14
thumpersmoser: yeah, your comments are right20:14
thumperI wish people didn't use the packaged django20:15
thumpervirtualenvs are the way to go with python IMO20:16
smoseroh yeah, of course, installing random stuff from untrusted sources on the internet that can change is always the best way to build software.20:18
* smoser realizes that argument might not seem ridiculous here20:18
natefinchsmoser: haha20:19
gary_posterthumper or any sympathetic soul, there is no api log that I see on machine 0.  I see /var/log/juju/machine-0.log.  In it, http://pastebin.ubuntu.com/6247538/ seems to show that the AllWatcher is falling over.  Any thoughts on where to look next?20:20
thumperthe machine-0.log has all the api stuff in it20:20
thumpergary_poster: can you post the error?20:21
thumperpastebin20:21
thumperor something20:21
* thumper sighs20:21
thumperI see it there20:21
* thumper clicks20:21
thumperit is all organge20:21
thumperorange20:21
hazmatthat looks like a client close of connection20:21
gary_posterorgange?:20:21
thumperI'm used to seeing the hyperlinks as blue20:21
thumperdumb client20:21
gary_posterhazmat, nope.20:21
hazmatgary_poster, do you have the corresponding browser trace?20:22
hazmatwebsocket trace from the client that is20:22
gary_posterhazmat, yes.  Connection Close Frame20:22
hazmatgary_poster, do you have a way to reproduce?20:22
thumperhmm...20:22
gary_posterhazmat, yes, but it is expensive.  Deploy this and go to GUI.  https://raw.github.com/paulczar/charm-championship/master/monitoringstack.sh20:23
hazmatah.. that one.20:23
thumpergary_poster: what is the gui asking for?20:32
thumperwhen it is falling over?20:32
gary_posterthumper, that log is as close to a record of that as we have.  in request 5, we ask the all watcher to give us the next output.  this is the first such request, I am pretty sure, so it will be the full system status.  We then ask for various other things, and get successful responses (such as line 3, which is our 16th request to juju in this connection) but then on line 5 (and 4?) of the pastebin, juju says that the20:35
gary_poster AllWatcher Next request has an error...because the "state watcher was stopped".  By whom? I'd love to know.  That appears to be death knell for the whole connection.20:35
gary_posterThe correlation is by the "RequestId": line 5 is a reaction to line 120:36
thumpergary_poster: the whole "all watcher" is a part I've not yet delved into, and I gather a rather complicated beast20:48
gary_posterthumper, ack.  this may be a big deal.  it might explain some other reports I've heard.20:48
thumpergary_poster: probably needs lots of debugging added to it to find out what is going on20:48
* thumper sighs20:48
thumperbig deals always happen at the last moment, no?20:48
gary_posterthumper, ack.  I'm checking with another source to see if they can dupe in another situation20:49
gary_posteryeah :-/20:49
thumpergary_poster: if you can get me something simpler to generate the problem with it'd be appreciated20:49
thumpergary_poster: let me finish off the reviews I'm in the middle of20:50
thumperand then I'll start poking20:50
gary_posterthumper, heh, I'd love to.  I think I need some hint on cause before I can come up with a smaller case.20:50
thumpergary_poster: does that monitoringstack deploy list always cause the problem?20:50
thumperis it really reproducable?20:50
thumperdo I have to open the gui?20:51
thumperor does it just happen?20:51
gary_posterthumper, so far, yes, it is reproducable.  API connection falls over in about 4.35 seconds, from the perspective of the client.20:51
thumpergary_poster: how far through the script does it get before it falls over?20:52
gary_posterthumper, and yes, you go to the environment with the GUI and log in20:52
thumpergary_poster: does the system appear stable prior?20:52
gary_posterthumper, the script sets up an environment.  then you go to the gui, and just look at it, and the connection falls over20:52
* thumper nods20:53
thumperok20:53
gary_posterthumper, other than the API connection, everything seems fine20:53
thumpergary_poster: how long are you around for?20:53
gary_posterthumper, and other than the AllWatcher Next, within that shining 4 seconds of connectivity, other replies in the connecton seem to be behaving fine20:53
gary_posterthumper, my EoD is in 6 minutes, and I should probably go not too soon after that.  last night was already a long one.20:54
thumpergary_poster: ack20:54
thumpergary_poster: do we have a bug yet?20:54
gary_posterthumper, no.  I will file one before I leave, or, if I get confirmation from the reporter, add juju core to an existing gui bug.20:55
gary_poster(add the dupe instructons I have so far)20:55
gary_posterand add20:55
gary_postersuch as they are20:55
thumperok, ta20:56
sinzuithumper, Can you take 10 minutes to advise/comment on this bug https://bugs.launchpad.net/juju-core/+bug/123230421:00
_mup_Bug #1232304: consider tuning git setup for juju-core, and document caveats <canonical-webops> <doc> <feature> <pes> <juju-core:Triaged> <https://launchpad.net/bugs/1232304>21:00
thumpersinzui: I can try21:00
natefinchsinzui, thumper:  any process that relies on storing binary blobs in git is flawed21:04
sinzuinatefinch, agreed21:04
sinzuiI really think there is a process problem in the bug. If we tune git, how much more can we scale before he hit the next problem?21:05
natefinchsinzui: tuning git is not the solution. Not storing blogs in git is the solution.   Is it a matter of charm authors doing something wrong, or a problem of juju doing something wrong?  I don't know the charm upgrade code at all.21:06
natefinchand by "wrong" I can also mean "we haven't given them a better way"21:07
thumpernatefinch: care to comment on that bug?21:08
thumpernatefinch: I know nothing about git21:08
* thumper will try to replicate gary_poster's bug on the local provider to save AWS bills21:09
gary_poster+121:09
* gary_poster needs to try and get local working on this machine. lost > day on itso been putting it off21:09
thumpergary_poster: we could look next week :)21:10
gary_posterthumper, it's my desktop.  laptop was fine last I checked21:10
natefinchthumper: all I know is that when you store binary in git, and then submit a change that changes the binary.... it doesn't store the diff of the two binaries, it just stores the two binaries.  So, if you have a 200 meg zip file and you add one thing to it and re-commit, you now have two 200-meg zip files in the repo.21:10
thumpergary_poster: ah21:10
natefinchthumper: and whenever you get code from git, you get the WHOLE REPO.  There's no getting a specific branch to reduce the amount you have to download.21:11
thumperWTF... my local provider won't bootstrap...21:12
thumperdoes no-one test this shit?21:13
* thumper rages21:13
natefinchthumper: anyway, I'll comment on the bug, but since I don't know the upgrade charm code, I'm not sure where the problem lies21:13
natefinchthumper: I get ERROR juju supercommand.go:282 Get http://10.0.3.1:8040/provider-state: dial tcp 10.0.3.1:8040: connection refused21:15
thumpernatefinch: me too21:15
* thumper sighs heavily21:16
* thumper wonders if 1.16 works21:16
thumperit used to21:16
thumperno.21:16
thumperthat fails too21:16
thumperWTF!!!!21:16
* thumper headdesks21:17
natefinchit's pretty epically bad if local bootstrap doesn't work in 1.1621:17
thumpertrue that21:17
* thumper files critical bug21:17
gary_posterthumper, I did not triage https://bugs.launchpad.net/juju-core/+bug/1240708 as critical yet but I filed it21:19
_mup_Bug #1240708: API server falls over repeatably during AllWatcher Next, killing GUI <juju-core:New> <https://launchpad.net/bugs/1240708>21:19
thumpergary_poster: sorry, but have to fix the local provider first21:19
gary_posterthumper, completely understood21:20
thumpergary_poster: it is broken, AGAIN21:20
gary_poster:-(21:20
thumperyeah21:20
natefinchthumper: I guess we don't have tests that actually try bootstrapping local?  Seems like, of all the providers, that one should be the most thoroughly tested.21:21
thumpernatefinch: it needs root21:21
thumperso we can't do it in unit tests21:21
thumpernatefinch: we plan to have it work in the qa lab21:21
thumperbut that is still being set up21:21
thumperat least I think I know how to get this working21:22
gary_posterthumper I have to run.  I will check back later briefly21:23
* thumper puts on his debugging music nice and loud21:23
thumpergary_poster: ack21:23
natefinchthumper: making people type in a sudo password when running the tests seems like a worthwhile pain in order to get full testing on the local provider21:24
thumpernatefinch: perhaps...21:24
* thumper goes to fix the problem21:24
natefinchthumper: good luck21:24
davecheneyor add themselves to whell21:25
davecheneywheel21:25
davecheneyor sudoers21:25
davecheneyor something to make it automagic21:26
natefinchindeed21:26
davecheneybut is probably going to be a non starter for CI21:27
* natefinch is at EOD21:28
thumperwell that was easy21:28
natefinchthumper: ha! awesome21:28
davecheneywhat is the plan for getting all these post 1.16 fixes into Saucy21:31
thumperdavecheney: 1.16.121:35
thumperI have no other answer at this stage21:35
davecheneyroger21:36
thumperdavecheney: if you insist, we could get roger to do everything21:36
thumper:P21:36
* thumper is frustrated with lbox and how it reuses merged merge proposals21:37
thumperI never want that21:37
thumperdavecheney: you may have re-approved the old one, I'm resubmitting21:38
thumperhttps://codereview.appspot.com/1457304621:38
davecheneythumper: LGTM. Looks like the same21:40
thumperdavecheney: it is :)21:40
thumperdavecheney: it was me leaving a clean trail behind me21:40
thumperdavecheney: once it lands in trunk, I'll submit for 1.1621:41
thumperI made sure I branched off a common ancestry revision21:42
thumperso I don't need to cherry pick21:42
thumperdavecheney: do you handle the packaging for juju?21:45
thumperdavecheney: in main world, we had seb and didier21:45
thumperdavecheney: who we'd pass a patch to in situations like this and they'd rebuild the deb21:45
davecheneythumper: that honor belongs to sinzui21:47
thumpersinzui: ping21:47
sinzuihi thumper21:47
thumpersinzui: got time for a hangout?21:47
sinzuiyes21:47
thumpersinzui: need bandwidth21:47
thumpersinzui: https://plus.google.com/hangouts/_/95c73d1f1d77129b8096dd279bf17d654e856cda?hl=en21:48
wallyworld_sinzui: hi21:48
sinzuihi wallyworld_21:48
wallyworld_you otp?21:48
thumperwallyworld_: he is soon :)21:48
davecheneyplease form an orderly line21:49
wallyworld_ok, sinzui maybe you can ping me when done21:49
sinzuiwallyworld_, your up22:06
wallyworld_sinzui: i was just wondering the plan for getting tools and metadata onto streams.canonical.com now that it has been commissioned22:07
sinzuiI plan to test it for 1.1722:08
sinzui1.18 will make it available to everyone22:09
wallyworld_sinzui: ok. i will need to make changes to juju-core to update the url etc22:09
wallyworld_when were you planning to start?22:09
sinzuiDid you see the bug I updated about that?22:09
wallyworld_ah no22:09
sinzuistreams.canonical.com/juju/tools22:10
wallyworld_i mustn't be subscribed to the bug22:10
wallyworld_yeah, that url looks fine22:10
wallyworld_do you have a bug # handy?22:10
sinzui^ We share the host with cloud images. We made an executive decision that juju-dist was too long22:10
* sinzui looks22:11
wallyworld_ok, it will take a little work to retool everything, but it's only software22:11
sinzuiwallyworld_, https://bugs.launchpad.net/juju-core/+bug/122096522:11
_mup_Bug #1220965: add official tools repository to metadata search <tech-debt> <juju-core:Triaged> <https://launchpad.net/bugs/1220965>22:11
sinzuiwallyworld_, really? you need juju-dist?22:12
wallyworld_wtf, i even reported that bug22:12
sinzuiwallyworld_, I read EVERY bug over the weekend. Even yesterday I was dizzy from too much information22:12
wallyworld_sinzui: actually, no. i was not thinking straight. juju-dist is assumed as the container name for cloud storage (ie private bucket)22:13
wallyworld_i'll need to check my mail filters to see where emails from that bug went22:13
wallyworld_anyways, that url will be fine22:13
wallyworld_i'll update juju-core with the necessary changes22:14
wallyworld_will be done this week i expect22:14
sinzuiwallyworld_, If it is a problem to match the path /juju/tools/, I can discuss the issue with Ben22:14
wallyworld_nah, should be fine22:14
wallyworld_will be fine, i was just caffeine deprived22:14
sinzui:/22:15
sinzuiDrink22:15
wallyworld_will do :-)22:15
wallyworld_sinzui: can you let me know when stuff has been uploaded? how hard is it to just copy across the tools from s3 or wherever?22:16
sinzuiNot sure yet22:16
wallyworld_ok22:16
sinzuiat the moment. I assemble all the tools to call sync-tools and make meta data. Since I have a cache from aws of the historic tools, I can build the tree in about 5 minutes. the release-public-tools script then deploys to all CPCs taking 15 minutes22:18
sinzuiwallyworld_, So the new process is pull/sync from streams.canonical.com, then 15 minute publication to all CPCs22:19
wallyworld_the ticket talks about syncing from sawo or something like that22:19
wallyworld_my question above was more aimed at getting some initial metadata up on streans.c.c so i could test22:20
sinzuithats right. I have no experience uploading to it and know how long from that server to the new server22:20
thumpergui won't install locally22:21
thumpergrr22:21
sinzuithumper, dir permissions?22:21
wallyworld_sinzui: i'll leave you alone in peace to work your magic and you can let me know when there's some news :-)22:21
gary_posterthumper, what is error?22:21
gary_posterthumper, for the API server falling over, more info.  From comment I added in bug: "We verified that the other bug has the same behavior (linked as dupe). Apparently, then, the charms are likely unrelated, because the other scenario is of an openstack bundle. This may simply be scale--and not very much of it."22:22
thumpergary_poster: http://pastebin.ubuntu.com/6248091/22:22
thumpergary_poster: logged in, and all at latest revision22:22
thumpergary_poster: in fact, every charm failed to install22:23
gary_posterthumper, yeah, "apt-get -y install python-apt python-launchpadlib python-tempita" failed, which doesn't seem like a gui issue22:24
thumperyeah...22:24
thumperhang on22:24
* thumper pastebins22:24
thumperhttp://pastebin.ubuntu.com/6248095/22:25
thumperseems to be a cloud-init issue22:25
thumperwhich is why all failed22:25
gary_poster:-( :-( :-(22:25
sidneithumper: yeah, it's all broken :/22:25
thumpersidnei: any idea?22:25
sidneithumper: it's an issue with procps and lxc, we've been tracking it all day22:25
gary_posterI have to run22:25
thumpergary_poster: ack22:26
thumpersidnei: oh, didn't know22:26
sidneithumper: it started happening yesterday because procps landed in precise-updates22:26
thumperah22:26
sidneibug #115764322:26
_mup_Bug #1157643: procps fail to start <failed> <patch> <procps> <start> <procps (Ubuntu):Confirmed> <https://launchpad.net/bugs/1157643>22:26
thumpersidnei: are they going to roll it back?22:26
sidneithumper: nope, that doesn't help unfortunately22:27
thumperwhy?22:27
sidneithumper: the problem is that procps has been broken in lxc since ever, but since it's installed by default and init doesn't block on it failing it went unnoticed22:27
sidneithumper: it only became an issue when calling the postinst script from dpkg22:27
thumperhaha22:27
thumperoops22:27
sidneiso reverting won't help22:28
sidneithumper: there's a patch in the bug, but it needs to go into sru and all that i guess22:29
* thumper nods22:29
sidneiuntil then, no workie lxc :/22:30
thumperoh well, I guess using aws to test this other failure is what we need then22:30
thumperdavecheney: this one is for 1.16 https://codereview.appspot.com/14439067/22:34
* thumper self approves22:36
thumperhuh - rieveld ignores my own LGTMs :)22:37
sidneithumper: there :)22:41
bigjoolso/22:43
thumpero/22:47
thumpergym time22:49
wallyworld_thumper: don't forget my 2 branches :-)22:50
thumperwallyworld_: oh yeah... after gym22:54
wallyworld_thumper: ok :-) also i think we need a 1.18 in saucy22:54
wallyworld_but we can discuss later22:54
thumperwallyworld_: it won't happen dude22:54
thumperyes, lets22:54
wallyworld_well, stuff will be broken22:54
wallyworld_no tools repository22:55
wallyworld_unless we back port lots :-(22:55
wallyworld_i wonder why it's so hard to get own own software into our own distro22:56
davecheneywallyworld_: i couldn't agree more strongly23:00
wallyworld_davecheney: yeah :-( there's a lot going into trunk right now23:00
wallyworld_we either want juju to work or we don't23:01
rogpeppe1ooh, i *so* nearly just tried to buy a camera from this site. then i looked at the terms and conditions... http://www.pcmshop.info/index.php?route=information/information&information_id=523:15
rogpeppe1mark v cheney ftw23:15
davecheneyrogpeppe1: we'll, I didn't want you to have to fall on your sword this sprint23:16
davecheneyi figured i was my time23:16
wallyworld_rogpeppe1: where in goamz did you see req.Close = true? i can't seem to find it23:43
wallyworld_ah never mind, found it23:47

Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!