/srv/irclogs.ubuntu.com/2013/04/15/#juju-dev.txt

davecheneythumper: I think you need to repropose https://codereview.appspot.com/8701043/00:04
davecheneyit talks about a StringSet type00:04
thumperok...00:04
davecheneybut one does not exist in trunk00:04
* thumper proposes00:04
bigjoolsI had no idea you loved me00:04
thumperI also tweaked a few bits in the stringset file, to remove some ", _" bits00:04
thumperwhen iterating over the dict00:05
thumperbigjools: I do love you, but you are married already00:05
bigjoolsand I have baggage00:05
davecheneypick me pick me00:07
davecheneyi only have carry on00:07
thumperdavecheney: updated00:07
davecheneythumper: responded00:22
davecheneyvery nice00:22
davecheneyjust a few thigns00:22
thumperta00:22
thumperI'll look after I've made some lunch :)00:22
thumperthinking bacon, cheese and egg toasty00:22
davecheneykk00:23
davecheneym_3: ping00:27
thumperdavecheney: it isn't possible to provide an interface for a structure that'll support iterating using range is there?00:43
davecheneythumper: no sadly not, user types cannot implement whatever range wants00:44
thumper:(00:44
thumperdavecheney: by using NewStrings in the union, intersection, difference methods, means we can directly poke the implementation map knowing that it has been initialized00:56
thumperdavecheney: otherwise we'd need to use the Add function that has the check every time00:56
davecheneythumper: fair enough, ignore that comment00:57
thumperok00:57
* thumper continues...00:57
=== wedgwood_away is now known as wedgwood
=== wedgwood is now known as wedgwood_away
thumperdavecheney: does the juju team have a pgp key-pair for signing packages?03:27
thumperdavecheney: I'm thinking about the tool signing process03:27
thumperdavecheney: for uploading tools, and for verification03:27
thumperI'm messing around with go.crypto trying to see how it works03:27
thumperin particular, how we could verify keys03:27
davecheneythumper: ~juju does not have keys, to the best of my knowledge03:58
thumperdavecheney: I've just created a 4096 bit RSA gpg key here for juju-dev03:58
thumperdavecheney: for my experimentation03:58
thumperperhaps we could use that...03:58
thumperif this works03:58
davecheneysure04:02
davecheneyit's just entropy04:02
thumperdavecheney: I'm now getting go people following me on twitter...04:02
davecheneythis is a bad thing ?04:02
thumpernot necessarily04:05
thumpermaybe if I tweet enough, we'll get generics :)04:05
thumperperhaps it is time for me to write my inaugral go blog post04:05
davecheneygotta do it once a year, just like changing your jocks04:06
thumper:)04:06
* thumper is looking for examples of someone actually using go.crypto/openpgp04:08
davecheneyjamespage_: ping04:13
davecheneymramm said that you had mongo for quantal available04:14
davecheneycan you hook me up ?04:14
jamespagedavecheney, I've requested the backports for 12.04 and 12.1004:15
jamespageppa:james-page/mongodb2.2 contains backports for 12.04 and 12.1004:15
davecheneyjamespage: fantastic04:15
jamespagesuggest that those packages are copied to some sort of official PPA until the packages land in backports04:15
davecheneyok, checking it out04:16
davecheneywe have ~juju/experimental as the PPA which is injected into cloudinit for P and Q machines when they are bootstrapped04:16
* thumper off to take the kids ice-skating...04:34
thumperdavecheney: just in case you know people working on go.crypto : http://stackoverflow.com/questions/16007695/verifying-a-signature-using-go-crypto-openpgp04:34
davecheneyjamespage: https://launchpad.net/~juju/+archive/experimental/+packages05:08
davecheneyis it possible to bump the build version for quantal ?05:08
davecheneyI need a 0ubuntu205:08
davecheneyunless it is possible to delete from a PPA (which I assume is not possible)05:08
jamwallyworld_: welcome back, I see you escaped the lions, but did you do so all in one piece? :)05:56
wallyworld_jam: thanks. yes, all in one piece. they are quite placid, unless you are a zebra wtc05:57
wallyworld_jam: bloody brilliant. i apt-upgraded this morning and now it appears no usb devices can be seen :-(05:59
jamwallyworld_: is your keyboard and mouse PS2 then?05:59
jamthat is pretty awful05:59
wallyworld_jam: no, i'm referring to my headphones, and also checked with a memory stick05:59
wallyworld_just now, preparing for meeting05:59
jamsure05:59
wallyworld_let me reboot. works for windows :-(06:00
jamtry another port perhaps?06:00
jam:)06:00
wallyworld_done that06:00
jamwallyworld_: saw you for a moment there06:05
wallyworld_yeah, usual mumble shittiness06:05
wallyworld_still trying to make it happy06:05
jamwallyworld_: I'm fine doing google chat for our 1:1's if it works better for you. It is just the standup where we can't all get along.06:06
wallyworld_i'll try once more06:06
mrammdavecheney, thumper: hi06:20
davecheneymramm: could you please expand on your prevoius reply06:20
davecheneyshould I stop or continue ?06:20
mrammkeep working on what you are working on06:20
davecheneymramm: understood06:20
* davecheney goes back to compiling06:21
mrammI'm saying that they will be working on whatever needs working on to get stuff into the release06:21
davecheneyunderstood06:22
wallyworld_jam: mumble died, sec06:36
jamwallyworld_: I heard you for a second06:36
jambut apparently you don't hear me06:36
rogpeppemornin' all06:51
rogpeppefwereade_: pin06:51
rogpeppeg06:51
fwereade_rogpeppe, heyhey06:51
rogpeppefwereade_: fancy a chat?06:51
fwereade_rogpeppe, sure, would you start one please?06:52
rogpeppefwereade_: https://plus.google.com/hangouts/_/a978de2c1b769c0250011c76024919656376b1f8?authuser=0&hl=en-GB06:52
aero1Hi all! I've been working on https://github.com/AeroNotix/hpcloud before learning about juju. Would my help be worthwhile in juju itself or do we follow different goals?07:41
aero1HPCloud has some extensions to OpenStack such as RDMS etc, I have bindings to those.07:41
jamaero1: do you mean 'hpcloud' vs "goose' (the openstack bindings?)08:06
aero1sure08:06
jamjuju is more about being able to deploy ubuntu packages into the cloud.08:06
aero1oh ok08:06
davecheneyUploading mongodb-clients_2.2.4-0ubuntu2_amd64.deb: 2645k/19783k08:06
aero1so, goose.08:06
davecheneyUploading mongodb-clients_2.2.4-0ubuntu2_amd64.deb: 2645k/19783k08:06
davecheneywhoo08:06
davecheneyfinally08:07
davecheneyonly taken all day08:07
jamdavecheney: grats08:07
jamgoose is meant to be reasonably generic, I don't think we have a problem with having some extensions as well.08:07
davecheneyjam: this should fix the problem bootstrapping quantal nodes08:07
jamaero1: but it certainly is focused on being access to any openstack deployment.08:07
* fwereade_ cheers at davecheney08:07
jamdavecheney: so the ppa had non-ssl?08:07
davecheneyjam: nope, bigjools fucked it up :)08:07
davecheneywe had a fight with LP, and lost08:08
bigjoolsbollocks did I08:08
jambigjools: well, as long as we can blame you, it doesn't matter if you did :)08:08
davecheneybigjools: https://bugs.launchpad.net/juju-core/+bug/116819608:08
davecheneythe web doesn't lie08:08
davecheneysomething is broken, you probably did it08:08
bigjoolssure08:08
bigjoolsthe executable had ssl options on it.  NEXT.08:09
davecheneygood, i'm glad we're all clear on that08:09
davecheneybigjools: jam Successfully uploaded packages.08:20
davecheneybut it hasn't shown up on the PPA page yet08:20
davecheneyhow long is the delay ?08:20
jamdavecheney: if those are source packages, you get queued into the builders, and it depends how busy they are.08:20
jamGenerally a couple hours.08:20
davecheneythis was a binary08:20
davecheneyi spent most of the day building it08:21
jamdavecheney: personally, I didn't think you could upload binaries to PPAs. I thought LP had to build it.08:21
davecheneyhttp://paste.ubuntu.com/5709833/08:21
davecheneywhere did it go ?08:21
davecheneyi have to go and make dinner08:24
jamdavecheney: I believe the deb handler will silently ignore files it doesn't like, to avoid being an attack vector.08:24
davecheneyif anyone knows where that package has gone, pleas let me know08:24
davecheneymaybe i nede to push my gpg key to LP08:25
davecheneyis there a command for that ?08:25
jamdavecheney: I'll ask around on Launchpad, but I'm pretty sure the default is that you can only upload source packages to LP08:25
davecheneyi'm following the instructions mims sent me08:25
davecheneywhich produced the current ppa's we have in experimental08:25
jambigjools: can you confirm?08:26
jamdavecheney: I thought there was a <10min time for the archive to poll the new uploads before it will see them.08:26
jamBut that should be reasonably fast.08:26
jamso we might just be waiting there.08:26
jamIt sounds like you should get an email08:27
davecheneyit's awesome that the instructions for doing this on LP are completely out of date08:29
davecheneyi think there was a problem with my gpg key08:30
davecheneyi suspect LP just threw away that upload08:31
davecheneylucky(~/pbuilder/quantal_result) % dput ppa:juju/experimental mongodb_2.2.4-0ubuntu2_amd64.changes08:32
davecheneyPackage has already been uploaded to ppa on ppa.launchpad.net08:32
davecheneyNothing more to do for mongodb_2.2.4-0ubuntu2_amd64.changes08:32
davecheneyyou mean I have to build it again !08:32
davecheneyFUCK08:32
jamdavecheney: I don't see any gpg keys here: https://launchpad.net/~dave-cheney08:39
jamdavecheney: so you probably do need to at least add that bit.08:40
jamIf it is saying "already uploaded" it may or may not still be in the queue of things to finish.08:40
jamdavecheney: if/when you get back, wgrant and stevenk would be interested in talking to you about it in #launchpad-dev08:58
danilosdavecheney, hi, I am still having test failures on raring 64bit with system mongodb: http://paste.ubuntu.com/5709879/ (if I simply change my path to first include the extracted 2.2.0 tarball binaries, then all the tests pass)09:07
=== edamato is now known as ed-afk
TheMuelunchtime10:31
dimiternfwereade_: ping10:49
fwereade_dimitern, pong10:49
dimiternfwereade_: just run the partial implementation of openstack instance selection past you10:50
fwereade_dimitern, cool, g+?10:50
dimiternfwereade_: i'll start one10:50
dimiternfwereade_: https://plus.google.com/hangouts/_/d16686e011ab34034d07f07641d351a34b0c6c9d?authuser=0&hl=en10:51
dimiternfwereade_: https://codereview.appspot.com/875304411:03
rogpeppefwereade_: just looking at tools.ReadList - given that we no longer fetch just the tools with a given major version from the provider, i don't think there's any particular reason, AFAICS, to have majorVersion as an argument any more. i *think* that's the only reason for ErrNoTools to exist.11:04
fwereade_rogpeppe, hmm, I think it still has value, on the basis that you can only ever be legitimately interested in a single major version at a time11:07
fwereade_rogpeppe, otherwise you need to remember to do a filter step everywhere that uses it (and filter needs a new field to do that)11:08
rogpeppefwereade_: i'm not sure - it seems like pre-guessing the uses it might be useful for. we might want to know which major versions are supported in an environment. and in our particular case, we want to know about other major versions so we can know if there are any tools.11:09
rogpeppefwereade_: yeah, filter would need another field11:09
rogpeppefwereade_: but that seems more intuitive, i think11:09
fwereade_rogpeppe, I don't think it's ever legitimate for the CLI to upgrade juju to a non-matching major version11:09
rogpeppefwereade_: that might not be why we're calling ReadList11:10
rogpeppefwereade_: it's a utility function11:10
rogpeppefwereade_: one review, BTW: https://codereview.appspot.com/8727044/11:12
fwereade_rogpeppe, tyvm11:12
fwereade_rogpeppe, feels a bit speculative to me tbh11:12
fwereade_rogpeppe, not saying that use case will never show up, just that it's not here now11:13
rogpeppefwereade_: i like to pare down functions to their minimum if possible. streamlining increases the possibility for elegant generality in the future11:13
rogpeppefwereade_: it feels like we're putting a special case in for this one scenario11:13
fwereade_rogpeppe, well, actually, I'm preserving behaviour that, on consideration, does what we need right now11:14
fwereade_rogpeppe, there was a definite temptation to take that on as well but... we wouldn't use it11:15
rogpeppefwereade_: i'm not suggesting you change the behaviour. just that i know that i added the major-version argument specifically so that we could fetch only tools that matched it. we're no longer doing that, so i think it could go. and that simplies other stuff too, i think. but if you think it's worth it, fine.11:16
fwereade_rogpeppe, I think that as it stands its benefits outweigh its costs... that's not to say that everything I've done over the last few days couldn't be done much better, ofc11:28
fwereade_rogpeppe, believe it or not, I've been trying to keep this stuff focused11:28
fwereade_rogpeppe, I just keep bumping up against weird little interactions11:28
rogpeppefwereade_: i guess i don't see the benefits, but i believe you!11:28
jamdimitern: poke11:32
fwereaderogpeppe, I see what you mean about the name/effect of FindBootstrapTools but I'm not sure about the alternative11:33
rogpeppefwereade: it was just one thought11:33
rogpeppefwereade: the fact that you didn't actually test that side effect is quite indicative to me that the name isn't right11:34
fwereaderogpeppe, yeah, that is pretty embarrassing11:34
rogpeppefwereade: i think that making the side effect the primary purpose feels good to me, but i'm totally unsure about the right name for it11:35
rogpeppefwereade: BTW, what's the reasoning behind the "usefulVersion" name i've seen in a few places?11:36
fwereaderogpeppe, "it means FFS this test depends on something, let's fake it up and move on"11:37
rogpeppefwereade: ah11:37
rogpeppefwereade: perhaps "requiredVersion" might be more helpful?11:37
fwereaderogpeppe, +1, thanks11:38
rogpeppefwereade: and perhaps a comment saying what actually requires it, for future reference11:38
fwereaderogpeppe, yeah, sounds reasonable11:38
* fwereade now has to figure them all out again ;p11:39
* fwereade wonders whether that'll maybe learn him11:39
rogpeppefwereade: :-)11:58
fwereadewtf, the provisioner doesn't remove dead machines?12:02
fwereadegaah12:02
* fwereade knows what he's doing *now* then12:02
* dimitern lunch12:04
jamfwereade: going to lunch with dimitern to drink away your sorrow ? :)12:11
rogpeppefwereade: have you tested any of your recent branches live?12:14
fwereaderogpeppe, yes, they appear to all do as they should12:18
fwereaderogpeppe, I'll be doing a final one after whatever total set of changes land12:18
fwereaderogpeppe, (land in my branches, I mean, not land in trunk)12:19
rogpeppefwereade: cool. i saw the APIInfo being erased in FinishMachineConfig and thought "huh?" but i see now why it's happening12:19
rogpeppefwereade: i've got a suggestion in my review, just coming up12:19
fwereaderogpeppe, yeah, that bit's still a bit squirrely but it's better than before, I'm pretty sure12:19
rogpeppefwereade: i hope you like my suggestion, which hopefully is better still.12:19
fwereaderogpeppe, cool12:20
fwereaderogpeppe, I look forward to it12:20
rogpeppefwereade: i've published my comments so far: https://codereview.appspot.com/8726044/12:28
fwereaderogpeppe, tyvm12:34
fwereaderogpeppe, I felt like changes to MachineConfig itself were out of scope here... the changes I made will, I think, make it easier to fix MachineConfig in the future12:39
fwereaderogpeppe, but there's something smarter struggling to get out, I think, I'm just not quite sure what it is yet12:40
rogpeppefwereade: hmm, i see now that you haven't changed environs/cloudinit at all12:41
fwereaderogpeppe, yeah, I was just drawing duplicated behaviour together12:41
rogpeppefwereade: i really don't like FinishMachineConfig though - it feels highly squirrely12:41
rogpeppefwereade: although...12:42
fwereaderogpeppe, it's a single squirrel instead of a... <looks up collective nouns...> dray, or scurry, thereof12:42
rogpeppefwereade: the basic idea of using lots of info from the config.Config to inform our cloudinit script seems dead right12:42
fwereaderogpeppe, I don;t claim it's anything more that that12:42
fwereaderogpeppe, yeah, I'm confident it's a good direction, but only a first step12:43
rogpeppefwereade: it's just that now for any potential provider-writer, it is totally non-obvious which fields in the MachineConfig should be filled out be whom.12:44
rogpeppes/be whom/by whom/12:44
fwereaderogpeppe, agreed, but MachineConfig has pretty solid validation, so I'm not too botherered there12:44
rogpeppefwereade: that's not the point!12:44
fwereaderogpeppe, it all had to be cargo-culter from ec2 for the existing ones anyway, really12:45
rogpeppefwereade: it still does12:45
rogpeppefwereade: but the amount of code copied will be smaller, which is good12:45
fwereaderogpeppe, precisely my point, no worse than before ;p12:45
rogpeppefwereade: i'm not entirely sure.12:46
rogpeppefwereade: you're probably right.12:46
rogpeppefwereade: but it doesn't feel like a proper simplification12:47
rogpeppefwereade: and there's definitely one to be made.12:47
fwereaderogpeppe, totally agreed12:47
fwereaderogpeppe, I was trying to do that before atlanta but got too tangled12:47
rogpeppefwereade: basically the MachineConfig fields are parameters to cloudinit.Configure and now half of them are redundant12:47
fwereaderogpeppe, well, none are *redundant* yet, but they're plainly due for the chopping block12:48
rogpeppefwereade: they are redundant, i think - it would be wrong if a provider didn't call FinishMachineConfig immediately before Configure.12:49
rogpeppefwereade: here's a suggestion:12:49
rogpeppefwereade: don't remove any fields fron MachineConfig12:49
rogpeppefwereade: but...12:49
rogpeppefwereade: move FinishMachineConfig into cloudinit.Configure12:50
rogpeppefwereade: and mark the fields that it fills out as deprecated in the MachineConfig12:50
rogpeppefwereade: i would be much happier with that12:50
rogpeppefwereade: and the providers would be simpler still.12:50
rogpeppefwereade: and they wouldn't need to change when the fields go12:52
fwereaderogpeppe, I really just want to avoid touching environs/cloudinit in this pass through the code12:53
rogpeppefwereade: ok, then please leave a TODO comment on FinishMachineConfig.12:53
fwereaderogpeppe, they were generated in a disturbing fashion; now they are generated in a slightly less disturbing one; that's all :)12:54
fwereaderogpeppe, sgtm12:54
jamwallyworld: poke13:12
rogpeppefwereade: rest of comments published13:24
rogpeppefwereade: interestingly, sync-tools is probably a place where we *do* want to do a ReadList across major versions. there's no particular reason why we only want to copy tools with a single major version AFAICS.13:31
TheMuefwereade: do i get it right that regarding the units and the relations "only" the relation-errors are interesting?13:32
fwereaderogpeppe, my view is that a given CLI major version can and will, for now, only interact with tools sharing a major version13:33
fwereadeTheMue, we don't really have relation errors13:33
rogpeppefwereade: even to the extent of refusing to copy tools for other major versions? i think that's a bit gratuitous - we see them but we can't touch them.13:34
fwereaderogpeppe, we can't deploy them either, so...13:34
rogpeppefwereade: someone else might be able to though13:34
fwereaderogpeppe, then they can run sync-tools?13:34
rogpeppefwereade: maybe they want to use a public bucket?13:34
fwereaderogpeppe, when I hear a user complaining that they need to run two different major versions of juju out of the same bucket, but *don't* have CLI tools for all the major versions they are running, I will ask the user how they plan to run the alternate major versions without suitable CLI tools13:36
rogpeppefwereade: it means that if you're an admin and want to copy several versions into a public bucket, you have to have a separate juju binary for each major version, and run it that many times.13:36
TheMuefwereade: aha?!? that's the only stuff generated out of the relations and the relation service map in Python I can find. *wonder*13:38
rogpeppefwereade: the point is that you can use sync-tools for many users13:38
fwereadeTheMue, I'm sure we had a chat about relation errors13:38
rogpeppefwereade: if there wasn't a --public flag i'd tend to agree13:38
fwereadeTheMue, does it not check for unit presence in the relation? I'm pretty sure it does13:41
fwereaderogpeppe, I see the public flag as a convenience for users who want their tools shared across a few of their own environments at this stage13:42
fwereaderogpeppe, it's probably worth revisiting when we get another major version though :)13:42
fwereadeTheMue, coming up with a sensible plan (and justification for it) is your main job here I think13:43
TheMuefwereade: _process_unit passes the relations and the service map to _process_unit_relations, and there below unit only relation errors are generated as output13:43
fwereadeTheMue, that's frickin' awesome news13:45
fwereadeTheMue, nothing to do for units13:45
fwereadeTheMue, yay!13:45
TheMuefwereade: afaics yes. i'm checking the tests, but there also only relation-errors are mentioned13:47
fwereadeTheMue, (incidentally, this informs my suspicion of specs -- there's a python doc explaining in detail why they had to make a backward-incompatible change to unit relation status output, describing relation-errors plus a bunch of other things -- and I didn't check the code for that bit)13:47
fwereadeTheMue, it now emerges it isn't there, I agree13:47
TheMuefwereade: *phew* ;)13:48
rogpeppefwereade: you've got another review: https://codereview.appspot.com/8748046/13:48
TheMuefwereade: btw, the change after your last review is proposed again13:48
=== wedgwood_away is now known as wedgwood
rogpeppefwereade: i'm wondering whether to put juju-wait in its own repo, or make a repo for several like-minded tools. juju-utils, perhaps?13:49
dimiternrogpeppe: we already discussed having something like lp:juju-core-tools, there's even a card for that13:52
rogpeppedimitern: ha. too late!13:52
rogpeppedimitern: https://launchpad.net/juju-utils13:52
dimiternrogpeppe: :) the card can be changed as well13:53
fwereaderogpeppe, lovely, thanks13:53
dimiternrogpeppe: it's in Blue backlog btw13:54
rogpeppefwereade, mramm: https://plus.google.com/hangouts/_/539f4239bf2fd8f454b789d64cd7307166bc908314:07
rvbafwereade: here it is: https://code.launchpad.net/~rvb/juju-core/fix-maas-pvd-conf/+merge/15893814:32
rogpeppelunch14:43
fwereadervba, thanks14:49
TheMuefwereade: thx for review. sorting is IMHO only interesting for testing, so you can exactly test against the expected value14:53
fwereadeTheMue, it's the dupes, not the sorting14:54
fwereadeTheMue, I kinda feel that if there's a dupe it's probably meant to be there14:54
fwereadeTheMue, or even if it represents total crack, we should show it14:54
TheMuefwereade: ok, so i'll change it into a sorted slice again and we'll what happes during the audit14:59
TheMuefwereade: ;)14:59
fwereadeTheMue, well, I was suggesting we keep it out of paranoia but flag it with a TODO for later investigation15:00
fwereadervba, hey, I have a theory re mysql/mediawiki: was mediawiki in an error state trying to leave the relation?15:09
rvbafwereade: you mean after I ran "juju destroy mysql"?15:10
fwereadervba, yeah15:10
dimiternTheMue: how about peer relations?15:11
rvbaI don't know but it happens that I have this setup in the lab right now http://paste.ubuntu.com/5710574/ so I can try to destroy mysql and have a look…15:11
TheMuedimitern: just seen your review, thanks, will add.15:14
fwereadervba, I would be interested to see what happens, yeah, because I *do* have a local tweak to the code in mine... that I *think* is actually not necessary15:14
fwereadedimitern, TheMue: good point re peers15:14
fwereadeTheMue, should just need a test; but also, subordinates; but please land this one as-is (just a peer relation test should be fine)15:15
dimiternTheMue: yeah, peer relations are simple enough to make it a separate test case; this one is already complex enough15:17
TheMuefwereade, dimitern: sure, won't make it larger15:18
fwereadervba, ah, hell, it has finally clicked15:19
fwereadervba, you *do* need my code change15:19
rvbafwereade: here are the state I'm in now, with links to all the logs. http://paste.ubuntu.com/5710599/15:19
dimiternfwereade: 3 reviewed, 2 to go15:19
fwereadervba, but even with that, if mediawiki has a hook error on mysql's departure, it won't drop its reference to that relation until it's resolved15:20
fwereadervba, and that will keep the service alive15:20
fwereadervba, nothing to do with your unit situation15:20
dimiternfwereade: if you don't mind, I'll wait for your https://codereview.appspot.com/8726044/ to land first, and then I'll land my openstack constraints branch15:21
fwereadedimitern, +1, tyvm15:21
rvbafwereade: right.15:22
fwereadervba, ok, sorry for crack suggestion just now, but I think I know (1) what's wrong and (2) how to fix it, and will hopefully nail (3) how to test it before too long15:23
rvbadimitern: Thanks for the review of lp:~rvb/juju-core/fix-maas-pvd-conf.  Would you mind landing it for me please?  lbox refuses to cooperate with me.15:25
dimiternrvba: sure, I'll pull your branch and land it for you, if you're fine with that15:25
rvbadimitern: that would be great, thanks!15:26
rvbafwereade: cool… once you have a branch ready, I can help with the real-world testing if you want.15:26
fwereadervba, I've approved https://code.launchpad.net/~rvb/juju-core/fix-maas-pvd-conf/+merge/158938 -- I think it's a trivial15:28
fwereadervba, go ahead and land it :)15:28
dimiternfwereade, rvba: I'll land it for him15:29
dimiternfwereade: (problems with lbox)15:29
dimiternrvba: here it is: https://codereview.appspot.com/877204315:47
fwereadedimitern, ofc, thanks15:47
dimiternrvba: take a look if it's ok, I'll submit it15:47
dimiternfwereade: set the old MP to Rejected because of the above15:47
fwereadedimitern, sgtm, just comment it15:48
dimiternfwereade: yeah, did and linked the new CL in the old MP15:48
fwereadedimitern, <315:48
rvbadimitern: looks good, thanks for doing this.15:48
dimiternrvba: np, lbox is a bitch at first, I know :)15:50
rvbadimitern: if that error rings a bell: http://paste.ubuntu.com/5710470/, please do advise :)15:51
dimiternrvba: rings a bell, but not sure how. I had to patch lbox locally to use it, I can send you a diff to see if it works for you15:52
rvbadimitern: that would be great… I was also thinking about manually hacking lbox's inferPushURL() method…15:53
dimiternrvba: actually, it's not lbox - it's lpad the one that lbox uses to communicate with lp15:58
dimiternrvba: http://paste.ubuntu.com/5710712/15:58
rvbadimitern: ta15:59
dimiternrvba: apply this (well, change dimitern to match your lp id) and see if it helps - you can try a simple branch and "lbox propose -wip"15:59
rvbadimitern: will do, thanks again.16:00
dimiternrvba: just a reminder, after patching, run "cd $GOPATH/src/launchpad.net/lbox && go install"16:02
fwereadedimitern, rvba: https://codereview.appspot.com/874804816:07
fwereadervba, confirmation that it works in your exact scenario would be appreciated, but I'm pretty confident16:08
fwereadebbiab16:08
rvbafwereade: dimitern: I'm testing this in the lab right now…16:10
dimiternfwereade: LGTM16:11
rogpeppefwereade: is there a ticket for the python/go environments.yaml incompatibility issue?16:26
rogpeppefwereade: i'm having difficulty reproducing the problem16:26
rogpeppemgz: ^16:27
dimiterndanilos: I see you have a kanban account already16:29
dimiternrogpeppe: is a defer statement function-scoped or block-scoped?16:31
=== deryck is now known as deryck[lunch]
rvbafwereade: doesn't look like your fix worked: http://paste.ubuntu.com/5710819/16:34
rogpeppedimitern: the former16:34
dimiternrogpeppe: I thought so, although seeing it into an if block within a function raised an eyebrow16:35
rogpeppedimitern: that's just fine16:35
rogpeppedimitern: can be in a for loop too...16:35
dimiternrogpeppe: ah, cool - although potentially messy (for i=1-1000 defer something each time..)16:36
dimiternrvba: I can see the unit is removed, but the service seems still there16:37
rvbadimitern: yep16:38
dimiternrvba: that wasn't the case before, right? the both the unit and the service were in the status output16:38
rvbadimitern: indeed (that was before the fix: http://paste.ubuntu.com/5710599/)16:39
dimiternfwereade , rvba: I think the fix should include removeOps in the amended case, like we're doing a bit further up in the "if s.doc.Life == Dying && s.doc.RelationCount == 0 && s.doc.UnitCount == 1" case16:41
dimiternrvba: removeOps is responsible for removing the service itself; the change is in removeUnitOps, which only removes the unit (except in the former mentioned case)16:42
mgzrogpeppe: there is a ticket, that might have been overly general, then closed when a specific issue was fixed16:45
dimiternrvba: added comment to the proposal16:45
rogpeppemgz: do you know of a specific problem?16:47
rogpeppemgz: as far as i can tell, go juju will parse unknown environment attributes without complaint16:47
rogpeppemgz: but i'm pretty sure that's actually not the case, and that this really is a genuine issue16:48
rogpeppedimitern: you chatted to ian about this - do you know anything about the problem?16:49
dimiternrogpeppe: if should ignore unknown keys (like "placement: local"), but barf on known keys whose type was changed16:49
dimiternrogpeppe: we didn't discuss specifics yet, and from the kanban chart I can see danilos actually picked that one now16:49
rogpeppedimitern: ah, do you mean we want a *single environment entry* to work with both go and python juju?16:50
dimiternrogpeppe: i'm sure that's not what we need - not for a bootstrapped env for sure16:50
dimiternrogpeppe: but for a fresh one, perhaps16:50
rogpeppedimitern: i don't know. seems like a somewhat dubious requirement to me.16:51
rogpeppedimitern: if that's really what the requirement is16:51
dimiternrogpeppe: we'll never make existing py-juju envs work with go-juju, but at least trying to do something with an existing py-env.yaml should fail gracefully, I think16:52
rogpeppedimitern: i'm not sure what you mean by "fail gracefully" there16:52
dimiternrogpeppe: like producing a meaningful error16:52
dimiternrogpeppe: can we detect a py env with certainty from the yaml only?16:53
rogpeppedimitern: only with dodgy heuristics, probably16:54
rogpeppedimitern: basically, i can't see this as a high priority task16:54
dimiternrogpeppe: i think the general idea, after having py/go juju co-installable, is to detect incompatible yaml and report it early16:55
rogpeppedimitern: but i'd really like to talk to someone for whom this is a priority16:55
rogpeppedimitern: we do that16:55
dimiternrogpeppe: how does it look?16:56
rogpeppedimitern: you get an error like:16:56
rogpeppeerror: placement: expected nothing, got "345"16:56
dimiternrogpeppe: and how is this helpful?16:56
dimiternrogpeppe: for the user16:57
rogpeppedimitern: it's detecting incompatible yaml and reporting it early...16:57
dimiternrogpeppe: but not in a meaningful for the user way16:57
rogpeppedimitern: the error message could probably do with some work, it's true16:57
rogpeppedimitern: i'd like us to work on stuff that is important functionally as we've only got hours remaining16:57
dimiternrogpeppe: put yourself in the user's shoes - the error should provide a hint what's actually wrong and how to fix it16:57
rogpeppedimitern: i agree16:58
rogpeppedimitern: but i wouldn't put this at the top of the list of things that need solving *now*16:58
dimiternrogpeppe: +1, not sure how and why this is high prio16:58
rogpeppedimitern: also, i think that the whole idea behind all the JUJU_HOME work was so that we would not have to share the environments.yaml file!16:59
rogpeppedimitern: and that added loads of complexity to the system16:59
rogpeppedimitern: and now we're saying that it's not enough16:59
dimiternrogpeppe: we discussed this on the standup; jam said he suggested having a different name/path for go-env.yaml, but it was rejected17:00
rogpeppedimitern: i also think that's the wrong solution17:00
rogpeppedimitern: a) we *can* currently use the same environments.yaml, AFAIK (but you can't have the same environment entry for both go and py juju)17:01
rogpeppedimitern: b) we have JUJU_HOME17:01
dimiternrogpeppe: agreed, on both points (except for the more helpful error msg)17:02
rogpeppem_3, hazmat: do you know anything more about this issue, by any chance?17:03
rogpeppedimitern: which ticket on the kanban board are you referring to?17:07
dimiternrogpeppe: "environments.yaml should cope with pyjuju and gojuju environments"17:08
dimiternrogpeppe: the description seems to shed some light on the actual issue as well17:08
fwereadervba, working as intended according to that paste17:09
rogpeppedimitern: the problem is that the "best compromise" is exactly what we do now17:09
rogpeppedimitern: i fixed the issue, was writing tests, and then realised that17:09
fwereadervba, the errored mediawiki is keeping a ref to the relation, which is keeping a ref to the service17:09
dimiternfwereade: ah, ok then17:10
rogpeppedimitern: AFAICS the description "juju-core will fail to start if there are any unrecognized keys" is not true17:10
fwereadervba, if you resolve mediawiki/0 it mysql should go away17:10
fwereadegtg again17:10
dimiternrogpeppe: it's still dubious imho17:10
rogpeppedimitern: what's still dubious?17:10
rogpeppedimitern: our error messages?17:10
dimiternrogpeppe: that's for sure, but the "fail to start" as well17:11
rogpeppedimitern: we don't fail to start if there are any unrecognised keys in environments.yaml17:11
dimiternrogpeppe: start=? bootstrap? connect? return an error on any cmd?17:11
rogpeppedimitern: any of the above17:11
dimiternrogpeppe: so it seems we're good then17:12
rogpeppedimitern: when the unrecognised keys aren't part of the chosen environment17:12
rogpeppedimitern: exactly17:12
rogpeppedimitern: but perhaps i'm wrong and there's a subtle issue i've missed17:12
rogpeppedimitern: which is why i want to speak to someone that's experienced the issue17:12
rogpeppedimitern: before dismissing the ticket17:13
dimiternrogpeppe: it's probably worth a @juju-dev post17:13
rogpeppedimitern: yeah17:13
dimiterneod or me17:28
dimiterngood night all!17:28
=== deryck[lunch] is now known as deryck
rogpeppefwereade: does this LGTY in its new location? https://codereview.appspot.com/8565044/18:04
m_3rogpeppe: hey18:04
rogpeppem_3: hiya18:04
m_3did you get the answer you needed about shared envs?18:05
rogpeppem_3: not really18:05
m_3'origin: ppa' is usually the biggest difference atm18:05
rogpeppem_3: do you want to be able to use the same environment entry in both go and py juju?18:05
rogpeppem_3: i'm not entirely sure whether that's a goal worth striving for18:06
m_3rogpeppe: yeah, that'd be the preference, but not too high of a priority18:06
m_3rogpeppe: let's say goju envs and pyju envs in the same file18:07
m_3rogpeppe: but not the same envs18:07
rogpeppem_3: we can do that currently18:07
rogpeppem_3: unless you know something that i don't...18:07
m_3i.e., we don't want to promote people trying to use the same environment18:07
* rogpeppe realises that m_3 knows *loads* of things that he doesn't18:07
rogpeppem_3: yeah, that's what i'm thinking18:08
rogpeppem_3: so by my thinking there's nothing that currently needs to be solved18:08
m_3lemme check to see if I can  use them together18:08
rogpeppem_3: thanks18:08
m_3I've been using dedicated environments.yaml files to day18:08
m_3date18:08
m_3until package changes (with update-alternatives) lands18:09
rogpeppem_3: BTW if you have some moments free, i'd very much appreciate your thoughts on the new juju-wait command, which i *think* is just about as we discussed: https://codereview.appspot.com/8565044/18:09
m_3rogpeppe: looks like they can coexist in the same file18:11
rogpeppem_3: cool18:11
rogpeppem_3: you can try out the juju-wait cmd by doing: go get launchpad.net/~rogpeppe/juju-utils/000-juju-wait/cmd/juju-wait18:11
m_3no need to change anything atm then18:11
rogpeppem_3: great!18:11
rogpeppem_3: although you probably want to delete $GOPATH/src/launchpad.net/~rogpeppe afterwards...18:12
m_3rogpeppe: will that change enough that I should tear the envi down before go getting?18:12
rogpeppem_3: nope18:12
rogpeppem_3: it should just work18:12
m_3ack18:12
m_3I'll try it in scale stuff now then18:12
rogpeppem_3: brill18:12
m_3gotta rebuild tho?18:13
rogpeppem_3: nope18:15
rogpeppem_3: that one command should have made "juju-wait" available18:15
rogpeppem_3: assuming $GOPATH/bin is in your PATH18:15
m_3rocking18:16
rogpeppem_3: although...18:16
rogpeppem_3: it depends what revno of juju-core you're on18:16
rogpeppem_3: and what version you've bootstrapped18:17
rogpeppem_3: if you've bootstrapped 1127 or later, you should be good18:17
m_3yup18:17
rogpeppefwereade: ping18:20
fwereaderogpeppe, pong18:21
m_3sorry.... suuuuper-latent18:21
fwereaderogpeppe, supper is just here18:21
rogpeppefwereade: ok, back later?18:21
fwereaderogpeppe, definitely18:21
rogpeppefwereade: cool, ping us then18:21
* rogpeppe smells curry odours drifting up the stairs18:22
m_3rogpeppe: http://paste.ubuntu.com/5711150/18:35
rogpeppem_3: ah!18:36
rogpeppem_3: trivially fixed18:36
rogpeppem_3: try go get -u launchpad.net/~rogpeppe/juju-utils/000-juju-wait/cmd/juju-wait18:38
rogpeppem_3: and try the command again18:38
rogpeppem_3: i had forgotten to import all the providers18:38
rogpeppem_3: it's an interesting point actually - we probably want to make it easier for commands to import all providers.18:39
m_3rogpeppe: trying18:45
rogpeppem_3: any joy?19:07
m_3rogpeppe: still waiting19:09
m_3rogpeppe: it seems to be waiting on hadoop-slave/9919:09
m_3so now I'm just waiting for provisioning19:09
m_3might have to do something with secgroups19:09
m_3not sure atm19:09
rogpeppem_3: a good first test is to juju-wait for something that you already know is in the state you're waiting for19:10
m_3lemme try it on a lower number :)19:10
rogpeppem_3: you can do it without interrupting the other one19:10
m_3rogpeppe: seems to be beautiful19:11
rogpeppem_3: there's a known provisioning problem at the moment - if the provisioner gets a temporary error, it just marks the machine with the error and never retries19:11
rogpeppem_3: excellent!19:11
fwereaderogpeppe, ping19:15
rogpeppefwereade: pong19:15
fwereaderogpeppe, more-or-less back19:16
rogpeppefwereade: i'm just implementing service config settings in the AllWatcher19:16
rogpeppefwereade: i just wanted to run my plan by you in case it's crack19:16
rogpeppefwereade: i'll watch the settings collection; if i see a service settings change, i look at the current service info that i've got stored; if the settings are for a different url, then i ignore the change, otherwise i set it.19:18
rogpeppefwereade: when i see a new service, i fetch the settings too19:18
rogpeppefwereade: basically a similar approach to the constraints19:18
rogpeppefwereade: does that sound more or less right?19:18
fwereaderogpeppe, hmm,have you considered per-charm settings?19:21
rogpeppefwereade: that's the "if the settings are for a different url" part19:21
fwereaderogpeppe, sorry, I did not read as closely as I should have: yes, that sounds perfectly correct19:22
rogpeppefwereade: great19:22
fwereaderogpeppe, service existence should guarantee settings existence for the its charm url19:23
rogpeppefwereade: i'm not sure19:23
rogpeppefwereade: i don't think there's an ordering guarantee on transaction ops19:23
fwereaderogpeppe, they execute inthe order passed, AIUI19:24
rogpeppefwereade: i'd like to know that for sure. i thought transactions with ops in different collections could execute concurrently.19:25
rogpeppefwereade: i've been assuming there's no such guarantee19:25
rogpeppefwereade: it makes my life easier if there is such a guarantee, but i'd like chapter and verse from the txn package before i change things accordingly.19:26
fwereaderogpeppe, this is one where I fall back to argument from authority rather than experience -- niemeyer said they execute in order, and I believe him19:26
niemeyerfwereade, rogpeppe: you can trust it.. it'd be a very weak system otherwise19:27
niemeyerfwereade, rogpeppe: Silly example: a bank transference might temporarily have more total money than exists19:27
rogpeppeniemeyer: ok. it might be worth adjusting the documentation for txn.Runner.Run then.19:28
rogpeppeniemeyer: it doesn't mention ordering at all19:28
rogpeppeniemeyer: "19:28
rogpeppeOperations across documents are not atomically applied, but are19:28
rogpeppe    guaranteed to be eventually all applied or all aborted19:28
rogpeppe"19:28
rogpeppeniemeyer: i'd read that as "no ordering guaranteed"19:28
rogpeppeniemeyer: BTW in benchmarks, i'm only seeing 20-30 transaction-based operations per second. is there some way we can speed that up, or is that to be expected?19:30
niemeyerrogpeppe: It's significantly faster than that..19:32
niemeyerrogpeppe: IIRC, txn can pump about ~250 operations per second19:32
niemeyerrogpeppe: Of course, that depends on the operations performed19:32
TheRealMueso, time to stop, cu tomorrow19:33
niemeyerrogpeppe: In fact, it's actually better than this19:33
niemeyerrogpeppe: My original tests were showing about 200 *transactions* per second19:33
niemeyerrogpeppe: So that was at least double that in operations, or about 500 operations per second19:34
niemeyerrogpeppe: If transactions increase the number of operations, naturally that will impact the volume of *transactions* you get19:34
m_3hey, can somebody help me turn off secgroup per machine?... just temporarily for scale testing19:34
rogpeppeniemeyer: there's a benchmark in state that i'd be interested in trying to improve19:35
rogpeppeniemeyer: BenchmarkAddUnit19:35
niemeyerrogpeppe: Sure, so.. hmm.. improve it..? :)19:35
rogpeppeniemeyer: i can call AddUnit about 47 times a second19:35
rogpeppeniemeyer: (as of just now)19:35
m_3I'll let dave check this for profiling, but just wanna get past this hurdle19:35
rogpeppeniemeyer: it's true that AddUnit is reasonably complex (it does a fetch, then a transaction), but it's not unrepresentative of the kinds of things we'll want to do19:37
rogpeppeniemeyer: and adding 10000 units is something we do want to do19:37
niemeyerrogpeppe: Sure19:38
niemeyerrogpeppe: +1 :)19:38
* rogpeppe goes for something to eat19:38
rogpeppeback in a bit19:38
fwereadem_3, firewall-mode: global, I *think*19:54
fwereadem_3, you definitely need a fresh environment for that though19:55
fwereadem_3, I don't know how the firewaller will react to a mode switch but it is very unlikely to be pretty19:55
fwereadem_3: yeah, firewall-mode: global should do it19:57
rogpeppeback20:12
m_3fwereade: got it thanks... that got us past the hump20:30
fwereadem_3, sweet20:30
m_3next hump is ram limits on the acct :(20:43
m_3rogpeppe: wait working like a champ20:45
rogpeppem_3: that's very good to know!20:45
rogpeppem_3: will submit soon. then it'll be go get launchpad.net/juju-utils/cmd/juju-wait20:45
rogpeppefwereade: oh darn, i've forgotten the other wrinkle - watching the settings collection doesn't give you values which are defaulted. i guess i'll need to keep all the charm metadata around and merge each time.20:57
thumpermorning21:21
rogpeppethumper: hiya21:26
thumperhi rogpeppe21:27
rogpeppethumper: i'd appreciate a review of this branch, if you have a moment: https://codereview.appspot.com/8761045/22:46
rogpeppem_3: juju-wait has landed22:48
m_3rogpeppe: ack, thanks!22:53
rogpeppem_3: np22:53
rogpeppem_3: is it working?22:54
rogpeppem_3: (i mean juju-core in general, really)22:54
thumperrogpeppe: sure, just looking at one for fwereade_22:57
rogpeppethumper: np22:57
m_3rogpeppe: we're waiting to get past some more acct limits23:05
m_3but great so far23:05
rogpeppem_3: that's good to hear23:06
* thumper goes to make coffee and bagel23:06
m_3rogpeppe: we're gonna do the wed charmschool with juju-core23:06
m_3I think23:06
m_3barring anything we run into tomorrow23:06
rogpeppem_3: that will be interesting23:06
m_3I'll do a dry-run tomorrow to decide23:18
rogpepperight, that's me done and dusted23:25
rogpeppeg'night all23:25
thumpernight23:26
=== wedgwood is now known as wedgwood_away

Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!