/srv/irclogs.ubuntu.com/2013/09/04/#juju-dev.txt

bigjoolsanyone know if the go-bot setup is documented anywhere?00:58
wallyworld_bigjools: there was an email sent to juju-dev01:36
bigjoolswallyworld_: archive-diving I go then, ta01:37
wallyworld_it turned into a thread of about maube 10 emails01:37
wallyworld_it may well be documented somewhere but that's all i know of01:37
wallyworld_the thread is called Landing bot setup steps01:38
bigjoolswallyworld_: when was it sent?01:40
wallyworld_bigjools: july 3001:40
bigjoolsta01:41
bigjoolsaaand found it01:45
* thumper hmmms01:55
* thumper wishes once again that go had generics01:56
thumpercopying boilerplate is error prone and annoying01:56
* thumper wanders off to think a little01:56
axwthumper: are we doing a code review today?03:07
thumperyeah, why not03:14
thumperI figure we all get on a hangout and look through the code03:14
thumperor do you think we should look first, then discuss on the call?03:14
axwI have looked already, so either way03:16
thumperwallyworld_: your opinion^^^03:20
wallyworld_i haven't looked03:20
wallyworld_i guess i have 10 mins03:21
* thumper goes to see what file it is again03:22
axwprovider/state.go03:23
* thumper is in the hangout03:31
bigjoolswallyworld_: how long should I wait for a canonistack instance to bootstrap?  it's been 15 minutes04:04
wallyworld_bigjools: cnonistack is unusable most times for me04:04
wallyworld_too slow04:04
bigjools\o/04:04
wallyworld_the servers are heavily overcommitted i think04:05
bigjoolswallyworld_: I have an instance04:05
bigjoolswhich I can't ssh into.... woo04:06
bigjoolssigh04:06
wallyworld_yay04:06
bigjoolstoday is Fail Central04:07
axwbigjools: I had better luck with lcy02 than 01 - try that if you're not already on it04:40
* thumper has had enough05:16
bigjoolsaxw: got it.  The wiki page I found didn't mention I needed sshuttle ...05:55
axwahh, right05:58
axwI got caught a few times by that05:58
bigjoolshow do I convince juju to destroy a service with a failed install hook?06:21
bigjoolsresolved I guess?06:22
axwbigjools: yep06:23
bigjoolsah it leaves an empy machine - will that get used next deployment?06:23
bigjoolsguess I will find out soon06:23
axwso I'm told, but I haven't witnessed it06:23
axwotherwise you can --to06:24
bradmbigjools: it does use it again, but I'm pretty sure it doesn't clean it up - so its got all the config of the previous service on it06:27
bradmbigjools: which makes it pretty useless in reality, since you can't be sure of the config06:27
bigjoolsbradm: that's probably a bug I guess.  Thanks.06:27
bradmbigjools: you can fake it out by doing a juju deploy ubuntu and then destroy the service06:28
bradmbigjools: we've used that in the past to pre-deploy juju instances06:28
bigjoolsok cheers06:28
=== tasdomas_afk is now known as tasdomas
davecheneybigjools: use resolved to get the machine to a started state (even if this is a lie) then destroy it06:51
davecheneybigjools: juju never reuses machines06:51
davecheneynever06:51
davecheney--to isn't part of that statement06:51
bigjoolsdavecheney: ok thanks - I am recalling what you did in the sprint06:51
davecheney'cos it just takes off all the safety guards06:51
* davecheney goes back to bd06:52
davecheneybed06:52
bigjools...06:52
=== _mup__ is now known as _mup_
bradmit must have been in pyjuju days when it did that, since I can't replicate it now07:01
bradmso thats good07:01
jambradm: bigjools: we are supposed to suppot "Clean & Empty" which means nothing was actually deployed to the machine yet we will use it for a future deploy/add-unit.07:19
jamyou can use "juju add-machine" (I think it works with a provider) to pre alloc machines.07:19
bradmjam: we did use the juju deploy ubuntu ; juju destroy-service ubuntu with pyjuju at a sprint to get the machines deployed to make training easier07:23
bradmjam: its debatable how much it actually helped, but anyway :)07:24
jambradm: I'm pretty sure that doesn't work by default in juju-core, but you should be able to "juju deploy --to X" to force it, and07:24
jamjuju add-machine should let you allocate machines ahead of time07:24
bradmjam: yes, it doesn't seem to work like it did in pyjuju, I tested it on canonistack07:24
bradmjam: honestly, I think its a good thing, I could imagine it biting us as we use juju more07:25
jambradm: right, the default policy is intended to be "if a machine was made dirty by installing something directly, don't default to putting something onto that machine"07:25
bradmjam: totally makes sense.07:26
jamthe problem that  bigjools highlights, is that if you destroy a unit, what is the point of leaving the machine around (by default)07:26
bradmjam: it's really a hack what we were doing07:26
bradmjam: to recover logs, any data that you need07:26
jambradm: sure, but I would say "allow a flag to destroy-unit that keeps the machine alive" not "create a flag to also destroy the machine"07:27
jamso if you need something, you can just remove the unit, but otherwise don't leave unusable machines around07:27
bradmjam: sure, or even a config option that says "keep the units after a service destroy" and default to destroying it, any of those07:28
bradmI'm still getting up to speed at how we're using juju07:28
dimiternmorning07:48
dimiternfwereade_: hey07:48
fwereade_dimitern, heyhey07:48
dimiternfwereade_: I did the changes as discussed (hopefully) - https://codereview.appspot.com/13494043/07:49
fwereade_dimitern, sweet, I'll take a look before I go out... I have more houses to look at07:50
dimiternfwereade_: can you take a closer look please, and I'd appreciate a stepwise testing plan :) (never used the local provider, and not sure how to trigger these hooks - should I write I charm?)07:50
dimiternfwereade_: thanks07:50
fwereade_dimitern, I'd suggest writing two metadata-only charms that just define relations07:51
fwereade_dimitern, that can relate to one another07:51
fwereade_dimitern, deploy one unit of each07:51
fwereade_dimitern, wait until they're started07:51
fwereade_dimitern, then open 2 windows, and debug-hooks each of the units07:52
fwereade_dimitern, then add a relation between them07:52
dimiternfwereade_: ok07:52
fwereade_dimitern, and then you can basically do whatever you need -- it'll just pause and let you run whatever you want for each hook07:53
dimiternfwereade_: hmm07:53
dimiternfwereade_: that's what's unclear - what should I do while paused?07:53
dimiternfwereade_: call relation-set/get someting?07:54
fwereade_dimitern, specifically you need I think to just keep on setting fresh unit data on either side of the relation and checking it arrives properly at the otherside07:54
fwereade_dimitern, yeah exactly07:54
fwereade_dimitern, set on one side07:54
fwereade_dimitern, get on the other07:54
dimiternfwereade_: ok, sounds simple enough07:55
fwereade_dimitern, if you want to be extra clever add a config to the charm07:55
dimiternfwereade_: how?07:55
fwereade_dimitern, config.yaml07:55
dimiternfwereade_: aah :) ok07:55
fwereade_dimitern, just a string setting you can change to trigger a hook and call relation-set in to get out of steady state again07:55
dimiternfwereade_: just a sec07:56
fwereade_dimitern, cath is pointing out that we're going to be late... I will review as soon as I'm back07:56
fwereade_dimitern, sorry :(07:56
dimiternfwereade_: ok07:56
dimiternfwereade_: will try it out07:56
dimiternfwereade_: tyvm07:57
dimiternTheMue: hey08:16
dimiternTheMue: can you point me out to the local provider docs you wrote?08:16
TheMuedimitern: sure, one moment08:17
TheMuedimitern: https://juju.ubuntu.com/docs/config-local.html08:17
dimiternTheMue: cheers08:20
TheMuedimitern: yw08:24
=== liam_ is now known as Guest751
bigjoolsI've got a charm that does an adduser in the installation hook, and it errors out with "Authentication token manipulation error"09:15
TheMueah, great, now shows exactly the behavior i wanted09:50
TheMuedimitern: btw, if you discover errors or missing parts in the doc please let me know09:51
dimiternTheMue: well, it worked, but only as root09:51
dimiternTheMue: I had to copy my env.yaml to /root/.juju/ and also my ssh keys to /root/.ssh09:51
dimiternTheMue: and add juju binaries to root's PATH09:52
dimiternTheMue: sudo juju bootstrap failed with "juju not found"09:52
dimiternTheMue: so I had to do sudo su - and run as root from that point on09:53
TheMuedimitern: strange09:53
TheMuedimitern: i've done it as regular user and only bootstrapped with sudo09:54
TheMuedimitern: it's described in the code that way09:54
dimiternTheMue: perhaps some steps are missing09:54
TheMuedimitern: which release? i took 13.04 on a clean system09:55
dimiternTheMue: sudo x doesn't do a login shell session, so PATH is not set09:55
dimiternTheMue: raring09:55
dimiternTheMue: and my system is mostly clean (reinstalled a month ago, so most of the packages are missing)09:57
dimiternTheMue: didn't install mongodb-server, just lxc - I have mongo installed for juju tests already09:57
TheMuedimitern: i'm using a testing image in a parallel vm for those tests. here i installed juju, mongo and lxc as user (but with sudo)09:58
TheMuedimitern: and then i generated the config which has been in $HOME/.juju (as expected)09:58
TheMuedimitern: after that bootstrapping with sudo (so it uses my env)09:59
TheMuedimitern: et voila09:59
dimiternTheMue: well it didn't work for me09:59
TheMuedimitern: deployed our lovely pair wp/mysql then and it worked09:59
TheMuedimitern: service deployment has been as non-root w/o sudo10:00
TheMuedimitern: because the provisioner already works as root10:01
dimiternTheMue: I suppose if sudo juju bootstrap worked, the rest would've worked as well10:02
dimiternTheMue: but juju could not be found, so I had to do as described above10:03
dimiternTheMue: if I had juju installed from the archive/ppa it should work though, it'll be available for any user10:04
dimiternTheMue: so my point is - if you're running juju from source, the described steps should be different10:04
TheMuedimitern: sounds reasonable, on that system i had no juju source10:08
jamTheMue: thanks for the reviews. One more: https://codereview.appspot.com/13360044/10:53
TheMuejam: looking10:54
jamTheMue: did it help to split this out into concrete steps? I sort of evolved the patch all at once, but I thought it might be easier to review in layers.10:55
TheMuejam: yes, helps a lot. would have been too large then10:56
TheMuejam: but i can't review the first one, get an error10:56
jamTheMue: one of them I submitted with a missing prereq so I deleted and resubmitted it.10:58
TheMuejam: ah, ok10:58
jamTheMue: do you mean https://codereview.appspot.com/13391047/ ? (the httpsuite-ssl change, vs the httpclient-ssl change, vs the final client/Client change)10:59
jamI thought you had reviewed 2 patches ,but it was just that I got your email 2 times (one from Reitveld, once from LP)11:00
TheMuejam: yes, that one. "old chunk mismatch"11:00
jamTheMue: for https://codereview.appspot.com/13391047/ the unified diff looks correct, I don't know what is wrong with the side-by-side diff11:01
jamI'll try to repropose11:02
natefinchmorning all11:02
TheMuejam: thx11:02
TheMuenatefinch: morning11:02
jamTheMue: reproposing seems to have fixed the bug11:03
jammorning natefinch11:03
TheMuejam: ah, yes, looks better now, will look at it in a few moments11:03
TheMuejam: LGTM, wow, all together a nice change11:12
jamTheMue: fwereade_, wallyworld_, dimitern, natefinch, mgz, jam: standup ? https://plus.google.com/hangouts/_/f497381ca4d154890227b3b35a85a985b894b47111:30
jamfwereade_, mgz  &&11:33
jam^^11:33
=== wallyworld_ is now known as wallyworld
=== wallyworld is now known as Guest33423
dimiternfwereade_: can you take a look at this please https://codereview.appspot.com/13523043/11:42
dimiternfwereade_: I'll try the extra live testing steps you mentioned about the previous change11:43
fwereade_dimitern, great, thanks11:43
natefinchjam: if you guys are still reviewing... it bugs me that loadState closes the reader it gets. It seems much clearer to have the callers close their own readers, even if it does mean one duplicated line. I think it's better to have the close method next to the method that creates the Closer13:17
jamnatefinch: I did have that one in my list, but we're wrapping up and it wasn't huge13:17
natefinchjam: yeah, not really a big deal13:18
jamnatefinch: it does eliminate some redundancy, but it makes it a bit harder to audit for the http.Get bug13:18
jam(you must call Close explicitly or http leaks file handles)13:18
* TheMue has to step out, wife awaited me since 313:18
natefinchjam: really, I think it's just a separation of concerns... loadstate doesn't care if the thing is closed, it just wants to be able to read. And if you then have something that wants to use loadState that doesn't have a close method, you have to manuifacture one13:19
jamnatefinch: "is it better that LoadStateFromURL hand off the defer r.Close to the loadState13:20
jamhelper? It eliminates redundancy, but makes it slightly harder. (I wouldn't13:20
jamdirectly guess that "Load*" would close the object it is given. though taking a13:20
jamReadCloser does tell you that.)13:20
jam"13:20
jamwas my notes13:20
natefinchyep13:20
natefinchjam: what's my business unit?  None of these things seem to fit quite right (doing the travel form for October sprint)13:22
jamnatefinch: you're under CDO13:24
jamthat is the group underneath Robbie (Mark Ramm's direct manager)13:24
natefinchjam: ok, cool13:24
dimiternfwereade_: so when you can https://codereview.appspot.com/13523043/13:25
fwereade_I'm even doing it now :)13:25
dimiternfwereade_: sweet!13:26
fwereade_dimitern, reviewed, LGTM with one extra test13:30
dimiternfwereade_: thanks13:31
dimiternfwereade_: am I getting this correctly - once the mysqlUnit has entered scope, leaving scope should generate a change in the watcher, right?13:41
fwereade_dimitern, scope for unit X is unaffected by any change to unit X13:41
dimiternfwereade_: yeah, but unit X in this case is wordpressUnit13:42
fwereade_dimitern, ah ok yes13:42
fwereade_dimitern, mysqlUnit leaving scope should not generate a change, but should do a depart13:42
dimiternfwereade_: isn't that also a change?13:42
dimiternfwereade_: in Departed13:43
dimiternfwereade_: I'm not getting anything though.. strange13:43
fwereade_dimitern, ah yes, got you, I had mismatched levels13:43
fwereade_dimitern, missing SyncBackend somewhere?13:43
dimiternfwereade_: hmm perhaps, will dig into it13:43
fwereade_dimitern, oh, and, I meant to say13:43
fwereade_dimitern, you're written the WatcherC13:44
fwereade_dimitern, would be really nice to use it in the state tests too13:44
fwereade_dimitern, if you swap that in there you'll be able to tell whether the test harness works ;)13:44
dimiternfwereade_: ah, right!13:44
dimiternfwereade_: I'll start there then13:44
dimiternfwereade_: does this look ok to you? http://paste.ubuntu.com/6062708/13:44
sidneihi folks, can i get a pair of reviews? https://codereview.appspot.com/1324204913:45
fwereade_dimitern, hmm, might be worth a repeated sync on ShortWait timeout13:45
dimiternfwereade_: just Sync() ?13:46
fwereade_dimitern, but check how the state tests do it, I might be wrong13:46
fwereade_dimitern, ah, hmm13:46
dimiternfwereade_: do I need the StartSync() then?13:46
fwereade_dimitern, just a sec, I should remind myself13:47
dimiternfwereade_: hmm there's no state.Sync() anymore it seems13:47
dimiternsidnei: now you need only a single LGTM btw13:49
sidneieven better :)13:49
dimiternsidnei: fwereade_ might be the right reviewer for this13:50
fwereade_sidnei, I am now churning through reviews13:50
sidneithanks!13:50
dimiternfwereade_: nevermind, found the issue :)14:04
dimiternfwereade_: one spurious for loop that never ends14:05
teknicowedgwood: you might find something useful in https://code.launchpad.net/~teknico/charm-helpers/lint-fixes/+merge/18185914:07
teknicoif not, no worries :-)14:07
wedgwoodthanks. I normally do reviews on Wednesday afternoons.14:08
wedgwoodteknico: taking a cursory look, I see one place where imports are commented out (rather than removed) (line 36-39) and one place where logic is changed (line 268-276)14:12
teknicowedgwood: the first one is because those imports are example placeholders, IIUC14:14
teknicowedgwood: the second is different logic but same behavior: the flow does not ever exit the for loop prematurely14:15
wedgwoodteknico: ah, right you are. tbh I thought for ... else behaved differently.14:17
teknicowedgwood: that was a possibility I wanted to dig up by changing the code ;-)14:18
wedgwoodit's definitely doesn't do what it seems like it should. Poor choice of keywords.14:18
teknicowedgwood: yes, it needs to be looked at in the right wavelength light to even begin making sense :-)14:21
wedgwoodIn any case, thanks for the cleanups :)14:23
teknicowedgwood: my pleasure, I hope I wasn't too much obsess... I mean, comprehensive about them :-)14:25
wedgwoodteknico: improvements are improvements, I say14:26
teknicoI actually removed a few more hundred lines of diffs that changed double-quote string delimiters into single-quote ones, it was becoming a bit too much. :-) maybe some other time14:27
sidneifwereade_: addressed comments, first time using export_test, so hopefully it's good. :)14:43
fwereade_sidnei, cheers14:43
fwereade_sidnei, LGTM, just pull that duplicated code block out into a func and you're good14:46
sidneifwereade_: a method of DeployerSuite or a standalone func?14:50
dimiternfwereade_: ping15:00
fwereade_sidnei, standalone, I think, unless it's using fields on the type15:00
fwereade_dimitern, pong15:01
dimiternfwereade_: so the txnRevno for settings I get from relUnit.Settings always seems to be -1 than the one I get from the watcher15:02
* fwereade_ raises a somewhat surprised eyebrow15:02
dimiternfwereade_: I'll push the branch as soon as the tests finish15:02
dimiternfwereade_: so originally I had c.Assert(actual.Changed, DeepEquals, expected.Changed) in AssertChange()15:03
dimiternfwereade_: but it seems all but the versions match (not sure if it's always off by one but it seems likely)15:03
dimiternfwereade_: I noticed that in relationunit_test after changing them to use the RelationUnitsWatcherC15:04
dimiternfwereade_: the versions were not checked before, just the len(changed), which seems weird15:04
fwereade_dimitern, ahhhh-ha, that rings a bell, just a mo15:05
dimiternfwereade_: so I resorted to checking actual.Changed, HasLen, len(expected.Changed) and then making sure each key in actual is also in changed15:05
dimiternfwereade_: it seemed weird that on remoteUnit.EnterScope(nil) with a fresh relation the initial event reported txnRevno2 for unit settings (always)15:06
fwereade_dimitern, so, just one thought... what are you basing expected versions on?15:07
fwereade_dimitern, I think you'll find there is a reason15:07
dimiternfwereade_: no idea really - I thought txnRevno should be 1 on a fresh scope15:07
fwereade_dimitern, try setting the unit's public address before entering scope ;p15:08
dimiternfwereade_: hmm, ok15:08
fwereade_dimitern, (entering scope depends on the unit's life, so the txn revno will be 1 + max(scope doc, unit doc))15:09
dimiternfwereade_: you mean instead of EnterScope(nil) to do myRelUnit.EnterScope(map[string]interface{}{"private-address": "blah"})15:10
dimiternfwereade_: in both cases the initial event comes with version 215:10
fwereade_dimitern, no, I mean make an extra change to the unit doc before joining15:10
dimiternfwereade_: aah15:10
fwereade_dimitern, so basically settings versions are volatile15:11
fwereade_dimitern, now I think of it you should be testing that they're increasing per unit, but should not be testing specific values15:11
dimiternfwereade_: I did mysqlUnit.SetPrivateAddress("1.2.3.4") and then myRelUnit, err := rel.Unit(s.mysqlUnit); myRelUnit.EnterScope(nil) - still15:11
dimiternfwereade_: still ver 215:11
fwereade_dimitern, you sure private wasn't already set?15:12
dimiternfwereade_: positive15:12
dimiternfwereade_: I could try some other unit doc field..15:12
fwereade_dimitern, or just set it twice, even, I think15:12
dimiternfwereade_: did SetPrivateAddress() twice - the same ver 215:13
dimiternfwereade_: (with different values for private address)15:14
dimiternfwereade_: I could just ignore the version and make sure the changed field's keys are sane15:14
fwereade_dimitern, hum, I'd expected that it would be the unit doc15:15
fwereade_dimitern, ok, in any given test you *should* ignore the version15:16
fwereade_dimitern, but we should *also* have a test that the version increases with each change of a unit15:16
fwereade_dimitern, the value will not be stable as the codebase changes15:17
fwereade_dimitern, but we depend on its always getting bigger15:17
dimiternfwereade_: when should it increase?15:17
fwereade_dimitern, sorry: the settings revno should increase by at least 1 whenever the settings change15:21
dimiternfwereade_: where should this test reside?15:24
fwereade_dimitern, this may be crazy, but... how about integrating it in the WatcherC?15:24
fwereade_dimitern, the client says what should change15:24
dimiternfwereade_: but doesn't specify a version15:25
fwereade_dimitern, the WatcherC validates the stream of data-per-unit coming out of the watcher by checking strictly increasing versions?15:25
dimiternfwereade_: the watcherC keeps the old txnRevno and compares it's greater15:25
fwereade_dimitern, yeah, exactly15:25
fwereade_dimitern, does that sound crazy?15:25
dimiternfwereade_: sgtm15:25
fwereade_dimitern, cool15:25
dimiternfwereade_: so I need to keep track of each unit that's changed and make the check on each AssertChange15:29
fwereade_dimitern, yeah, just keep a map of unit: last-change and whenever you get a new one check and set15:30
sidneifwereade_: it's not, but neither the 'bundle' function is. i shall convert both then.15:34
fwereade_sidnei, nice, thanks15:34
dimiternfwereade_: worked nicely15:39
fwereade_dimitern, sweet15:39
sidneifwereade_: so, on a similar subject. we need to get this fix into currently-deployed prodstack instances at some point. what's the process for that, once we have a built release (which at this point is 1.15.x?) to upgrade the existing deployments that are on 1.13.3-ish?15:42
natefinchmgz: can you take a look at this real quick? It's been mostly LGTM'd by john, but I added one more tweak to get the stuff working live with azure: https://codereview.appspot.com/13082044/15:42
mgznatefinch: sure15:43
mgzhm, can't persuade rietveld to give me the 1->2 diff...15:44
mgzah, pick a file, then change in the dropdown works15:44
natefinchmgz: yeah, I don't know what's going on there.  The only addition was one line, adding DeploymentName to the struct we pass to GetDeploymentRequest15:44
mgzyeah, but is still blank... I'll branch the thing and look at the actual history15:45
mgzprobably just the merges from trunk upseting it15:48
natefinchmgz: yeah15:49
dimiternfwereade_: would like you to have a final look though https://codereview.appspot.com/1352304315:50
TheMuefwereade_: thx for review16:21
=== tasdomas is now known as tasdomas_afk
=== wedgwood is now known as Guest57531
=== Guest33423 is now known as wallyworld
=== wallyworld is now known as Guest52886

Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!