davecheneyall: master is not passing the tests for me00:00
davecheneyuniter, api client, worker/uniter00:00
davecheneyall fail00:00
perrito666anastasiamac: I actually added anothe rcomment but apparently did not go trough reviewboard00:00
perrito666anastasiamac: try to change the uses of fmt.Errorf for errors.Annotate00:01
anastasiamacperrito666:of course! thnx :-) will do00:02
anastasiamacperrito666: constructive suggestions r an amazing feedback, especially atm ;-)00:03
anastasiamacperrito666: english hard atm too...00:04
perrito666anastasiamac: I am south american I am certainly in no position to criticize anyone's english00:06
anastasiamacperrito666: :-)00:08
thumperdavecheney: master as of 9am this morning passed for me00:22
thumperdavecheney: what is your last commit?00:22
davecheneythumper: i just pulled master00:23
davecheneyi was trying to get my branch up to date and the tests were all failing00:23
davecheneyso i switch to master00:23
davecheneyand they are still failing00:23
thumperdavecheney: let me try00:23
=== ChanServ changed the topic of #juju-dev to: https://juju.ubuntu.com | On-call reviewer: see calendar | Open critical bugs: None
thumperdavecheney: running now00:24
davecheneythumper: eg, http://paste.ubuntu.com/8726080/00:29
thumperworks here00:31
thumpersmells of timing issue00:32
davecheneywell shit00:32
davecheneyok, i'll break out my tools00:32
davecheneyoh hang on00:33
davecheneythese are the actions00:33
davecheneyi raised bugs about this before00:33
thumperI thought this was parallel charm upload?00:34
davecheneythumper: what rev of charm.v2 do you have ?00:35
davecheneyi don't trust godeps any more00:35
thumperit should be charm.v400:35
thumperI think00:35
* thumper looks00:36
waiganimenn0_: added another step for the migrations. state/watcher.go update <coll>Watcher.merge() - change.Id -> localID00:36
thumperdavecheney: yeah, I'm using charm.v400:36
davecheneythumper: can you tell me the exact rev00:36
bigjoolshey axw, can you remind me what causes juju to try and release twice when bootstrap goes awry?00:36
waiganimenn0_: I think that one is a bug that will bite a few more times00:36
davecheneycharm.v4, for all of the promise of gopkg.in00:36
davecheneydoesn't ensure we're running the same code00:37
menn0_waigani: yeah, that's one that I catch while looking for transactions that touch the collection, but it's good to mention it explicitly00:37
thumperdavecheney: 8b3cc836f54c2c78ce73d198100a30bb31d2839200:38
thumperdavecheney: I have godeps working fine00:38
davecheneyscratch that00:38
davecheneythumper: could you run00:39
thumperdavecheney: if you have "export JUJU_MAKE_GODEPS=true" and use "make check"00:39
davecheneythumper: i don't trust godeps00:39
thumperit makes sure your deps are up to date and runs the tests00:39
davecheneyi know how to run the command00:39
waiganimenn0_: yeah, saves some brain power. the change.Id being passed in will always be a DocID but needs to be a localID. I misunderstood that during the transaction sweep.00:39
davecheneyi just don't trust it to work00:39
davecheneythumper: can you please try00:39
thumperit fails for me when I have different origins and upstreams00:39
davecheneyenv GOMAXPROCS=42 go test github.com/juju/juju/api/uniter00:39
thumperso I have to fetch upstream manually00:39
thumperdavecheney: sure00:39
thumperdavecheney: test passed here00:41
menn0_waigani, thumper: i'm currently trying to figure out a regression with the relation/relationscopes branch.00:44
waiganimenn0_: want to pair?00:44
menn0_waigani: not at this stage thanks00:45
=== menn0_ is now known as menn0
menn0waigani: I'm slowly making progress00:45
waiganimenn0: cool. If you need a rubber duck let me know00:46
menn0waigani, thumper: when I destroy a service some relations aren't being deleted00:46
menn0waigani, thumper: i'll get there but it's painful00:46
waiganimenn0: makes sense if it's looking for the wrong id right?00:47
thumpermenn0: hmm... interesting00:47
menn0waigani: yes. i'm sure it's something like that but it's not obvious. it could be a watcher problem.00:47
menn0it only seems to affect subordinate relations AFAICT00:48
waiganimenn0: checked the merge func? ;)00:48
menn0waigani: no but I think I just found it. missed a change in WatchRelations00:49
menn0stupid prefix matching crap00:50
waiganiah right00:50
* thumper heads off to see specialist00:58
menn0waigani: the unit tests didn't notice by fluke01:01
menn0waigani: with an _id like this "60d7b1b0-4fea-452f-8748-70c05cf489ec:wp0:db mysql:server"01:01
menn0waigani: WatchRelations was matching on _ids starting with "mysql:" or containing " mysql:"01:02
axwbigjools: if bootstrap fails after acquiring, it'll release the bootstrap node; juju will then destroy the environment. whether it releases again depends on whether it's possible for MAAS to report that the bootstrap node still has the agent_name we gave it01:02
waiganiah fuck01:02
axwbigjools: now that I write that out in full, it probably never happens01:02
waiganimenn0: is it a regex match?01:03
menn0waigani: it just happened to always match with the "containing" case in the test but the prefix match is no longer correct01:03
bigjoolsaxw: heh, ok :)01:03
menn0waigani: no just a simple bit of string matching01:03
menn0waigani: I'll update the test and then fix01:03
bigjoolsaxw: it's just that we have observed this behaviour but now we can't make it happen again01:03
menn0waigani: it's a bit of a fluke that I found it01:03
menn0waigani: which is worrying01:04
bigjoolsaxw: in here: https://bugs.launchpad.net/maas/+bug/138632701:04
mupBug #1386327: Juju fails to destroy-environment with wipe enabled: Node cannot be released in its current state ('Disk erasing'). <MAAS:Incomplete> <https://launchpad.net/bugs/1386327>01:04
waiganimenn0: might be worth while searching all the transactions01:04
axwbigjools: if the first release failed, then it would attempt again (just the once)01:04
waiganimenn0: or at least adding it to the doc for something to check01:04
bigjoolsaxw: right01:04
menn0waigani: yeah, I'll add it to the doc01:04
waiganiMESS doc that is01:04
waiganimenn0: you drew the short straw on that one01:05
waiganibut good you spotted the problem, we can now keep an eye out for it01:06
axwbigjools: is it possible for release to return before agent_name is removed from a node?01:06
bigjoolsaxw: yes01:06
bigjoolsin the bug, it has to wipe the disk before it returns, so sits (internally) at a different state.  The extra state is swallowed in the API to avoid confusing unaware API clients01:07
waiganimenn0: how did you notice relations were left behind after a service was destroyed?01:07
bigjoolsaxw: so you end up seeing what looks like a node that didn't start releasing, but it really is01:07
menn0waigani: the service wasn't completely being removed01:08
menn0waigani: the units and machines were but the service was being left behind in a Dying state with one relation intact01:09
menn0waigani: I've just updated the manual test instructions to include testing of service removal01:10
waiganimenn0: ah right, fair call01:10
davecheneythumper: solved it01:17
davecheneyout of date names dependency01:17
davecheneygodeps didn't detect it01:17
waiganimenn0: does mediawiki take ages to install for you?01:22
waiganiit's been sitting on pending for well over 5min01:22
menn0waigani: I'm not sure exactly how long but that seems on the long side. also depends on you machine and how busy it is of course.01:23
menn0waigani: or, you've broken something :)01:23
waiganimight be time for a restart01:24
davecheneyok, now I have some other failing tests01:29
perrito666:| gmail now adds a button "view issue" and "view pull request" on the email list for PRs01:43
perrito666davecheney: want me to try?01:43
thumperdavecheney: have you updated godeps recently?01:47
perrito666nite ppl01:50
perrito666thumper: btw, for that heat you wanted, near 30° its 22:51 and has been night for 3hs :p01:51
thumperperrito666: it is about 15 here01:52
waiganiand has been day for more hours01:52
menn0waigani: I'm adding State.strictLocalID as part of this01:54
menn0it's like localID() but errors if the prefix isn't there01:54
waiganimenn0: okay...01:55
waiganimenn0: used when we assume the id passed in is a DocID01:55
menn0waigani: it's useful in cases where ids for multiple envs could be seen but you just want to deal with the ones for the State's environment01:56
menn0waigani: for example in the watchers01:56
waiganimenn0: oh I see01:57
menn0waigani: i'll push this branch soon so you can see01:57
* thumper needs git help02:14
thumpermenn0: around?02:14
menn0thumper: yep02:15
rick_h_thumper: what'd you break?02:15
thumpermy branch02:15
rick_h_stop doing that :P02:15
thumpermy branch was 5 commits behind master02:15
thumperand I did "git reset master" without "git merge master"02:15
thumperand now I have a bunch of changes I didn't make02:15
thumperwas trying the other rebase options02:16
thumperforgot it only really works when you have all of the master revisions02:16
thumperso I have some dangling head somewhere02:16
thumperthat I want back02:16
thumperas the tip of the branch02:16
thumperhow do I find it?02:16
* menn0 googles02:16
rick_h_trying to think if you can get it without reflog'ing it02:17
rick_h_reset's not something I ever do except for git reset --hard HEAD when I want to really kill some work02:17
rick_h_thumper: highly recommend https://github.com/juju/juju-gui/blob/develop/HACKING.rst#syncing-your-feature-branch-with-develop-trunk for the whole 'sync with master to my feature branch' just s/develop/master for your use.02:18
thumperI blame menn002:18
rick_h_good call, find a scape goat02:18
thumperI'm just going to revert everything I know isn't mine02:18
thumperi looks like it will be quicker02:19
menn0thumper: i'm not sure why I get the blame. I don't think I encouraged you to use reset did I?02:19
thumpermenn0: if you had never mentioned reset, I'd still be happily rebasing :P02:19
thumpermenn0: and you fit my need of a scapegoat02:19
menn0thumper: ok, to get back to where you were02:22
menn0thumper: git reflog | head02:22
thumperI'm there already02:22
menn0thumper: ok02:22
thumperfor future me02:22
thumperwhat does that do?02:22
menn0thumper: shows you the history of what you've been doing02:22
thumperok, I see the commit I care about02:22
menn0thumper: to recover you find the hash from when you last committed to the branch02:22
menn0thumper: and then git reset --hard <hash>02:22
menn0thumper: I just tried it out to be sure02:23
thumperis that all?02:23
menn0thumper: yep02:23
rick_h_reflog ftw02:23
thumperok, will remember that for next time I screw up02:23
menn0thumper: the thing is that dangling references will eventually get garbage collected by git02:23
menn0thumper: so you don't want to leave it too long02:23
menn0thumper: after the screwup to recover02:23
menn0thumper: this link is helpful: http://effectif.com/git/recovering-lost-git-commits02:24
* thumper bookmarks02:24
menn0thumper: best quote: "it's like time travel, only cheaper" :)02:24
davecheneythumper: no i have not updated godeps02:42
davecheneyi think the last time I did was when we were on that sprint in NZ02:42
menn0waigani, thumper: finally! http://reviews.vapour.ws/r/284/02:47
menn0waigani: reviewing your charms branch now02:48
waiganimenn0: thanks, I've got minUnits up too02:48
waiganimenn0: let me nut out this problem then I'll get onto your branch02:48
menn0waigani: np02:49
menn0waigani: I think we need to a EnvUUID method to state to avoid repeating "st.EnvironTag().Id()" all over the place02:53
waiganimenn0: agreed02:53
ericsnowmenn0: when you get a chance, I'd love to get your feedback on http://reviews.vapour.ws/r/243.02:57
menn0ericsnow: will do. I had seen that review request - just hadn't gotten there yet.02:58
ericsnowmenn0: no worries, I appreciate your time :)02:58
davecheneycan someone help please03:05
davecheney^ this goes into stress.bash03:05
davecheneycd $GOPATH/src/github.com/juju/juju/apiserver/client03:06
davecheneybash stress.bash03:06
* menn0 will run03:06
davecheneythere is a logical race between uploading the charm and whatever the test is doing03:06
menn0davecheney: it's still running but I see a clientSuite.TestAddCharmConcurrently failure03:10
menn0davecheney: done now03:10
davecheneyfailing hiallriously03:10
menn0davecheney: that's what I see03:10
davecheneyi have anothe rfailure03:10
menn0davecheney: actually, on closer inspection my failure is different03:11
menn0davecheney: http://paste.ubuntu.com/8727496/03:12
mupBug #1386968: apiserver/client: clientSuite.TestAddCharmConcurrently failure do to local race <juju-core:New> <https://launchpad.net/bugs/1386968>03:14
menn0ericsnow: ping03:18
ericsnowmenn0: hey03:19
menn0ericsnow: so doesn't option 2 involve using --oplog03:19
menn0ericsnow: ?03:19
ericsnowmenn0: yeah03:19
menn0ericsnow: but it looks like it's not being used when specific dbs are backed up03:19
ericsnowmenn0: correct03:19
menn0ericsnow: so what's the plan?03:20
ericsnowmenn0: with option 2 we don't dump specific DBs03:20
menn0ericsnow: ok. so why leave the support in there for backing up individual dbs?03:20
ericsnowmenn0: I'm hopeful that we will be able to stick to option 203:20
ericsnowmenn0: in case option 2 doesn't pan out :)03:21
menn0menn0: right. got it.03:21
waiganimenn0: started looking but Bella will be here in 10min, I'll be doing the dad thing03:22
menn0waigani: np03:22
menn0waigani: i have an errand to run shortly myself03:23
waiganimenn0: btw cleanup docs has an id type of bson.ObjectId - which screws up our upgrade step assumtions03:23
waiganimenn0: not a huge problem, but I'll have to do some rejigging tomorrow03:24
* thumper goes to lie down03:24
thumpershoulder is really starting to ache now03:24
menn0waigani: yeah, I've seen that. let's discuss tomorrow. there's a few ways we could tackle that.03:24
thumpergo an injection in it this afternoon03:24
thumperhopefully should settle down in the next two weeks03:24
thumperhere's hoping03:24
waiganimenn0: sounds like a plan03:25
menn0ericsnow: are you sure this branch even compiles?03:26
menn0ericsnow: mongoDumper.Dump looks incomplete with both unused and undeclared variables03:27
menn0ericsnow: and mongoDumper.strip() has no code03:27
ericsnowmenn0: yeah, I created the review request thinking I'd wrapped things up03:27
ericsnowmenn0: I have it fixed up and will be updating momentarily03:27
menn0ericsnow: ok03:28
waiganimenn0: shouldn't we always use strictLocalID unless we have a reason not to?03:33
davecheneytada, data race http://paste.ubuntu.com/8727559/03:36
davecheneyfunc (s *MockCharmStore) WithTestMode(testMode bool) charm.Repository { s.TestMode = testMode return s03:37
davecheneywe have _SO MANY_ of these races03:37
davecheneyoh wow03:40
davecheneythat methods is pants on head retarded03:40
davecheneyit doens't return you a _new_ repository with a test flag set03:40
davecheneyit updates the repo that you passed to it ...03:41
davecheneyoh, and fantastic, the callers of that expect this behavior03:41
davecheneyfml, https://bugs.launchpad.net/juju-core/+bug/1386968/comments/203:43
mupBug #1386968: apiserver/client: clientSuite.TestAddCharmConcurrently failure do to local race <juju-core:Confirmed> <https://launchpad.net/bugs/1386968>03:43
ericsnowmenn0: I've updated that patch04:04
ericsnowmenn0: and am signing off04:04
menn0ericsnow: ok. i'm looking at your other one now04:04
ericsnowmenn0: cool04:05
menn0ericsnow: I'll try to finish reviewing both04:05
ericsnowmenn0: thanks again04:05
menn0ericsnow: np.04:05
wallyworld_axw: i've found a problem with the recent work to validate credentials, and the use of Open vs Prepare - it does break validate-tools04:08
wallyworld_you want to run validate-tools before deploying an environment04:09
wallyworld_to see where the tools will come from04:09
wallyworld_but that's not possible now, since the environment is not prepared, and hence is missing derived attrs04:09
wallyworld_like control-bucket for ec204:09
wallyworld_in any case, control-bucket should now be obsolete for deploying new environments04:12
axwwallyworld_: yuck, that's a problem. I want to be able to change the manual provider so we can specify the bootstrap host in placement directives, but that involves restricting Prepare to bootstrap04:24
davecheneymenn0: if you read this and are not filled with dredd, http://reviews.vapour.ws/r/286/diff/#04:26
davecheneylet me know04:26
wallyworld_axw: yeah. the environment is created in order to get parameters specific to the environment to use to search04:26
menn0davecheney: I have to stop for a bit soon so I won't be able to get to this for a few hours04:27
wallyworld_axw: given the validation commands are now broken, short term to get alpha3 released, we'll need a fix that works with what we have04:27
wallyworld_i have not raised a bug yet04:28
davecheneymenn0: that's ok04:28
davecheneyyou don't need to review it04:28
axwwallyworld_: revert to using Prepare for now I guess, will just need to change the metadata tests to set up a mock ec2 endpoint04:28
wallyworld_axw: will that revert affect the credentials validation04:28
wallyworld_i can change to using Prepare in my current redo of the tools stream stuff04:29
wallyworld_i found this issue by testing that work04:29
axwwallyworld_: if it's not too much hassle. otherwise let me know and I'll fix it04:30
wallyworld_it's no problem04:30
wallyworld_i'm in the area anyway04:30
menn0davecheney: i've just had a quick look and although it's not pretty it does look reasonable04:31
davecheneyit's much better04:32
davecheneythe caller doesn't need to remember to defer this magic cleanup function04:32
davecheneythat's something the test suite should do04:32
davecheneyand more improtantly04:32
davecheneyit highlights just how horrible that test is04:32
davecheneyupdating a shared mutable package level variable04:32
menn0davecheney: you know that CleanupSuite (embedded in BaseSuite) already provides something very similar to what you've done with afterFunc?04:33
menn0davecheney: there's an AddCleanup method04:33
davecheneymenn0: cool04:33
davecheneyi was going to use that04:33
davecheneybut for some reason I thought it was only on the suites that had something to do with mongo04:34
davecheneyi'll fix the patch to use AddCleanup04:34
menn0davecheney: ok04:34
menn0davecheney: I'll be back on later but I need to go know04:34
davecheneythanks for pointing me to the place it lives04:35
wallyworld_axw: sadly, the metadata commands fail with an access credentials check error - i wonder if the verifyCredentials calls could be moved to Open() instead of Prepare(). I wonder if that's too late in the bootstrap process05:05
axwwallyworld_: the bug (raised by mark) requested that we do this ASAP in the bootstrap process - and it is kinda necessary to ensure we get the error message early on05:22
axwwallyworld_: what is going wrong with the metadata commands?05:22
wallyworld_axw: yeah, i understand the context. the metadata commands can be run without env credentials, so that verify check fails05:23
wallyworld_we could add a method to the BootstrapContext05:23
axwwallyworld_: can we do this some other way? without opening an environment? cos really, we're not exactly opening a fully defined environment05:23
wallyworld_right now, an env instance is needed to ask for things from05:24
axwyes... can we change that.05:24
wallyworld_not that easily, but yes05:24
wallyworld_because creating an env manipulates config05:25
wallyworld_i'll have a poke around, but there's potentially a fair bit to unpick05:26
axwyeah I took a look, doesn't look like something we can do easily05:26
wallyworld_axw: could add a method to BootstrapContext - ValidateCredentials() bool05:27
wallyworld_would work for now05:27
wallyworld_so that we can get alpha3 out05:28
wallyworld_i also gotta make sync-tools backwards compatible, bit of a pita05:28
axwwallyworld_: we can make it bootstrap only I suppose, would be nice not to but it'll do for now05:29
wallyworld_yeah, agreed it's not nice :-(05:29
wallyworld_i'll try it after fixing sync tools05:30
wallyworld_lots of touch points there05:30
=== urulama__ is now known as urulama
dimiternmorning all07:24
axwwallyworld_: FYI I've not sent a PR with my metadata changes yet, because I want to explore the usage of it a bit first. I would like to avoid churning packages07:54
wallyworld_axw: +107:54
wallyworld_axw: i'm almost finished the tools rework, including credentials verification fix. keep getting mongo replicaset errors testing sadly07:55
wallyworld_timeouts mainly07:55
axwokey dokey08:04
axwgotta make dinner shortly, ping when it's ready and I'll take a look08:04
dimiternwallyworld_, hey08:07
dimiternwallyworld_, so re that network-bridge-related bugs08:07
dimiternwallyworld_, that guy probably misread or misunderstood what the code he pasted does08:07
wallyworld_i'm not 100% either08:08
dimiternwallyworld_, but seeing lxcbr0 in there *at all* is wrong and should be fixed08:08
wallyworld_dimitern: which bit?08:09
dimiternwallyworld_, in fact I'm thinking of doing that today, as this is likely the 15th bug report related to that piece of code in the maas provider08:09
fwereadeok, uniter tests passing again, popping out to shops, bbs08:09
wallyworld_dimitern: that would be awesome08:10
wallyworld_i don't know enough about networking08:10
dimiternwallyworld_, there is some code in there that aims to "restore" /etc/network/interfaces the way they *should* look like in a stock trusty install, but that's really not the point - we shouldn't be using the bridge at all, but first we should respect the network-bridge setting08:10
dimiternat least08:11
wallyworld_sounds reasonable08:11
dimiternwallyworld_, cool, thanks08:12
dimiternTheMue, morning08:12
wallyworld_dimitern: no, thank you, i have no idea really08:12
dimiternwallyworld_, well, I have to admit the simple solution of just using "network-bridge" in there as a temporary fix (so at least it's configurable) simply didn't occur to me earlier, and we can't just remove the bridge creation until we've implemented the static ip reservation api maas 1.7 gives us; so tl;dr - thanks for pointing that out :)08:14
wallyworld_np, it was an educated guess :-)08:15
mattywmorning all08:51
voidspacemorning all09:01
TheMuemattyw, voidspace: heya09:03
mattywTheMue, morning morning09:03
fwereadevoidspace, would you be free to take a look at a couple of gsamfira's PRs? https://github.com/juju/juju/pull/782 https://github.com/juju/juju/pull/83309:13
fwereadevoidspace, gsamfira: I should be able to get to them later, but I realised I broke something in the uniter and I'm focused on unbreaking it09:13
gsamfirafwereade: heh, no worries09:14
fwereadeand if anyone *else* is free -- jam, dimitern maybe? -- http://reviews.vapour.ws/r/288/diff/# is the code, on which I'd really a appreciate a logic/style pre-review, but I haven't yet fixed all the tests (sigh)09:14
jamfwereade: looking09:15
jamfwereade: is "State().Members" a map and you want the keys to be in the member name slice ?09:17
jam(I wonder if memberName, _ :=  would be an obvious way to write that)09:17
jamat least, I first read it as "Members is a slice" and we just want to copy the slice, but then iteration doesn't work like that :)09:18
jamfwereade: I also thought: "contextRelations := map[int]*ContextRelation{}" was bad form because you end up with a nil map, you actually want make() there, don't you?09:20
jamhmm. play.golang says I'm wrong09:21
jamvar x map[int]string is a nil map, but x := map[int]string{} is an empty map09:21
voidspacejam: have you picked up both PRs or should I look at one?09:24
jamvoidspace: I'm looking at fwereade's not gsamfira's09:25
voidspacejam: ah, cool09:25
voidspacefwereade: sure, I'll look at the gsamfira reviews09:25
fwereadejam, yeah, I've never understood the preference for make()ing maps over just using a literal09:26
jamfwereade: well, because "var foo map" doesn't work09:27
jamfwereade: so it would seem that the way to create a map is with "make"09:27
fwereadejam, sure, but the literal syntax works fine, so I'm not sure why people prefer to avoid it09:29
jamfwereade: in *my* case, not knowing (or expecting) it to wokr09:30
fwereadejam, it's essentially just the same as slices09:30
fwereadejam, var gives nil, type{} gives empty09:31
jamfwereade: except empty slice lets you append as well as a nil slice09:31
jamand call len09:31
fwereadejam, indeed, *that*'s the reason people use var foo string{} :)09:31
jamand iterate09:31
fwereadejam, len and range work on nil maps too09:31
fwereadejam, it's just insertion09:31
jamfwereade: right, but you can't add stuff09:31
jamwhich is why it is odd09:31
jam"the things I want to do with a slice" function the same with nil or empty09:32
jam"the things I want to do with a map" all work the same except for, ya know, adding stuff09:32
fwereadejam, it's always felt a bit warty to me09:32
fwereadejam, defensible ofc but a bit inelegant09:32
jamfwereade: question about "members" cache vs "others" cache09:42
jamwhy do we clean out others here and there, does that effect our transactionality ?09:43
jam(things which we touch are never refreshed09:43
voidspacedimitern: the network id that AllocateAddress will take, is that a "provider specific" id (i.e. one we can pass directly to ec2)09:44
dimiternvoidspace, that's the idea, yes09:44
dimiternvoidspace, for ec2 it probably will be a subnet-id09:44
fwereadejam, so, the idea is that you *can* always get settings for remote units that aren't currently part of the relation09:45
dimiternvoidspace, but it's unused for now - intended to be future-proof :)09:45
fwereadejam, but that we expect that to be "rare"09:45
jamfwereade: I don't disagree on either of those points09:45
fwereadejam, so we persist them only for the lifetime of the context09:45
jamis this a lifetime of data cached in memory issue ?09:46
fwereadejam, it doesn't affect the guarantees we make09:46
fwereadejam, yeah09:46
fwereadejam, it's always been, and always will be, possible to see relation data "from the future"09:46
fwereadejam, but within a given context, once you've asked for some settings, they won't change09:46
jamfwereade: sure, but what causes Prune() to be called09:47
jamas if you do "relation-get NOT_A_MEMBER" and then trigger a Prune and then another "relation-get NOT_A_MEMBER" you will get different data09:47
jam(*can* get different data)09:48
voidspacedimitern: ah, we're designing in imaginary future use cases...09:48
dimiternvoidspace, it's not imaginary, it's in the draft model :)09:48
voidspacedimitern: :-p09:49
jamfwereade: also "relation-get A_MEMBER" and then mark that as no longer a member and "relation-get WAS_A_MEMBER" will give different data09:49
fwereadejam, we Prune to the current membership list at context creation time09:50
fwereadejam, calling prune/remove/invalidate while a context is in play will not have happy consequenes, I agree09:51
jamfwereade: so I do see one possible bug, which is if you do "context.Settings("x/2"), then context.InvalidateMember("x/2"), and then context.Settings("x/2") it won't reread the data09:51
jambecause it will pull it out of "other"09:52
fwereadejam, how would it get into "other" in the first place?09:53
jamfwereade: the first Settings call09:53
fwereadejam, TestInvalidateMemberUncachesMemberSettings09:53
jamfwereade: but it isn't a member09:53
jamuntil you call InvalidateMember09:53
fwereadesurely it will09:53
jamfwereade: I think it is just a "delete self.other[memberName]" in InvalidateMember09:54
fwereadeah yes I see09:54
jamfwereade: RelationCache.Settings unconditionally falls back to other09:54
fwereadejam, thanks, well spotted09:54
jamif settings is nil09:54
fwereadejam, yeah09:54
fwereadejam, I think I'd prefer:09:56
fwereade        if !isMember {09:56
fwereade            settings = cache.others[unitName]09:56
fwereade        }09:56
fwereade        if settings == nil {09:56
fwereade            var err error09:56
fwereade            settings, err = cache.readSettings(unitName)09:56
fwereade            if err != nil {09:56
fwereade                return nil, err09:56
fwereade            }09:56
fwereade        }09:56
jamfwereade: lgtm09:56
fwereadejam, cheers09:56
jamit still seems a bit odd that you might have the data sitting around09:57
jamfwereade: but still test-worthy however it is solved09:57
fwereadejam, agreed09:58
jamfwereade: so going back to understand the lifetime of these objects09:58
fwereadejam, I think the fundamental problem is an assumption that I'll always call Prune before Remove or Invalidate09:59
jamfwereade: I'm more thinking about when does the cache actually get invalidated10:00
voidspacedimitern: is the default-vpc represented as a network interface in ec2?10:00
fwereadejam, expand please?10:00
jamso we have this data which is cached, which if you explicitly call InvalidateMember will refresh the cache10:01
jambut otherwise the cache has unbounded lifetime10:01
jamit is created when you start a context10:01
jam(and pruned at that point)10:01
jamI would have thought a cache would only exist for the lifetime of a contextx10:01
dimiternvoidspace, no, it's an account attribute containing a vpc id in the form "vpc-xxxx" an interface would have id like "eni-xxxx"10:02
fwereadejam, consider charms which real the full state of a given relation to determine their config10:03
fwereadejam, they enter a relation with 10 remote units10:03
voidspacedimitern: ok, so we need to tell ec2 which network interface to assign the ip address to10:03
fwereadejam, even if they don't have a -joined hook implemented, they'll run 10 -changed hooks10:03
voidspacedimitern: do we have to get all of them and work out which subnet the address belongs to and which interface the subnet is on?10:04
fwereadejam, and in the Nth hook they'll grab N sets of settings10:04
fwereadejam, despite N-1 of them being ones they got just a second or so ago10:04
jamfwereade: sure. though when the hooks trigger tomorrow won't they want updated data ?10:05
jamIs this just live until the current set of hooks finish ?10:05
fwereadejam, they *might*, but we're guaranteed to run a -changed hook, at some point, for any that have changed their data10:06
fwereadejam, so we invalidate just before running the -changed hook and call it good10:06
jamfwereade: but if you do so, then isn't that still true for the case of running 10 changed hooks ?10:07
jamanyway, otp now, I'll need to context switch, but I want to discuss it with you just to get an understanding.10:07
fwereadejam, ok, I will quickly dump in here, ping me when you want10:07
fwereade10 units in scope with cached settings10:08
fwereadeall 10 remote units change their settings10:08
fwereadewe trigger a changed for the first one, and invalidate its settings10:08
fwereadethat changed hook asks for all settings, downloads exactly one (the invalidated one)10:09
fwereadechanged for the second one, with invalidated settings10:09
fwereadeagain asks for all settings, only needs to hit the api for one set10:09
fwereadetotal impact: 10 settings gets over the 10 hooks; unit saw a consistent -- if not strictly accurate -- view of the changes, and is now fully up to date10:10
fwereade(and next week, when just one of them changes settings, we again just download one set of changed settings)10:11
fwereadejam, see above10:11
jamfwereade: as long as the jujud daemon wasn't restarted10:12
fwereadejam, so, yes, it would be nice to cache them on disk10:12
fwereadejam, I'm not doing that right now10:12
jamfwereade: well, our API shouldn't be so expensive that that is necessary, should it?10:12
fwereadejam, depends how big the services get10:13
fwereadejam, all data is big data when you have enough of it ;p10:13
fwereadejam, but, yes, I don't see a strong need to cache it on disk10:13
jamfwereade: I'm personally more in favor of less caching until you know that it will help10:14
fwereadejam, I think people have asked for it once or twice10:14
jamas cache invalidation is hard10:14
fwereadejam, agreed :)10:14
fwereadejam, I think this case is well-understood, though10:15
jamand tends to do well on benchmarks but when you have real load hit rates may be low10:15
fwereadejam., not to mention pre-existing ;)10:15
fwereadejam, quite so10:15
jamIIRC some of launchpad spend up when they stopped caching some stuff because the extra cached data had low hit rates and was pushing out good data10:15
jamvery different use acse10:16
jamcase, though.10:16
fwereadejam, however, given that the gimme-all-the-relation-data tool is a pretty frequent request10:16
fwereadejam, and seems to be the first thing people write when doing helpers for a new language10:16
fwereadejam, I'm reasonably confident that it maps well to many use cases10:16
natefinchfwereade: Have you looked at the output variables spec? https://docs.google.com/a/canonical.com/document/d/1Bz8YGJL7OcVgcK-K47MEH7-vlJVPvm34wKUPWO8FQko/edit10:22
natefinchfwereade: I'm supposed to be sizing this for alexis for this cycle, and I want to make sure the spec actually describes the thing we want to build10:23
natefinchfwereade: ahh, hmm, hazmat left a comment on that one about there being a new document from brussels, but I can't seem to find it.  Fun.10:25
fwereadenatefinch, dammit10:27
fwereadenatefinch, nor can I10:28
fwereadenatefinch, niemeyer put it together, would you ask him when he comes online?10:28
natefinchfwereade: sure10:28
wallyworld_axw: finally got it sorted after having dinner, if you have time, or else it can wait till tomorrow https://github.com/juju/juju/pull/988 just the 2nd commit really10:29
natefinchfwereade: also - charm level constraints have container and architecture blacklists..... how would those actually work, from a user's point of view?  we don't generally put things in containers automatically, so the only time we do is when a user specifically tells us to, at which point, do we just warn them that the charm probably won't work there?  That doesn't seem like the best UX, but I'm not sure how else to implem10:30
natefinchent it10:30
wallyworld_fwereade: i'd love 5 mins of your time if you have it10:30
fwereadewallyworld_, sure, start a hangout? natefinch, with you properly in 5?10:31
natefinchfwereade: sure sure10:31
wallyworld_fwereade: yeah, our 1:110:31
fwereadewallyworld_, regular-catchup?10:32
fwereadewallyworld_, I think I'm in there10:33
fwereadenatefinch, fwiw, I would be treating it like series10:33
hazmatnatefinch, re output variables?10:35
hazmatnatefinch, https://docs.google.com/a/canonical.com/document/d/1JcWkE4SNxXuFClZGBcwnU3w13IpRU1yxMhddQG6mKyE/edit#heading=h.swixk07ju5hq10:36
natefinchhazmat: thanks.... google docs couldn't find that one for some reason10:37
hazmatnatefinch, odd.. its setup for find and edit by everyone @ canonical10:37
natefinchhazmat: yeah, I don't know why search wasn't finding it10:37
natefinchhazmat: of course now that I've opened it, search finds it just fine :/10:38
natefinchhazmat: great, this looks much more like what I was expecting than the old spec10:39
fwereadenatefinch, I think there are some things to worry about with that spec -- the concept of a "schema" is so tortured as to be a positive impediment to understanding10:48
fwereadenatefinch, but I will make my comments on the doc (and try to talk to gustavo about it tomorrow)10:48
fwereadenatefinch, anyway, re blacklists10:48
fwereadenatefinch, IMO we shouldn't even try to deploy things in places they declare they won't work10:49
natefinchfwereade: that seems to make sense10:49
fwereadenatefinch, in particular I don't think arch/container blacklists are even constraints10:49
fwereadenatefinch, for constraints in charms, I think there's more wiggle room -- the constraints in charms are *recommendations*, that we will follow if we can but deploy anyway and warn if we can't meet them10:50
fwereadenatefinch, and they are ofc just suggestions, and completely ignored if the service has real constraints10:50
fwereadenatefinch, indicating that the administrator knows he knows better10:50
fwereadenatefinch, that said10:51
fwereadenatefinch, constraints in charms are not currently a thing AFAIK?10:51
natefinchfwereade: my thought was that charm constraints would turn into real constraints during deploy, but get overridden by any other constraints specified by the CLI etc.10:51
natefinchfwereade: correct10:51
natefinchfwereade: it is a thing I am being tasked with writing :)10:51
natefinchfwereade: well, my team :)10:51
hazmatfwereade, unrelated.. but was curious.. what's the status on leader election?10:52
fwereadehazmat, ramping up10:52
fwereadehazmat, I'm churning on the uniter10:52
hazmatfwereade, k, thanks10:52
fwereadehazmat, it's taking a bit of time to actually get us to having a global sense of hook ordering10:53
fwereadehazmat, and katco is about to make a start on the server side10:53
fwereadenatefinch, given that container type and arch are already constraints though10:54
fwereadenatefinch, probably it makes sense to draw a distinction between preferred and required constraints in a charm10:55
natefinchfwereade: *nod* there's RAM and rootdisk charm-level constraints in the spec as well10:55
natefinchfwereade: those are a little easier to grok than the arch and container blacklists, since blacklists are the opposite of what we do with constraints now10:56
fwereadenatefinch, provider-specific constraints are gonna be fun, too10:56
fwereadenatefinch, do you overlap with thumper?10:56
natefinchfwereade: yeah, late in the day for a couple hours10:56
fwereadenatefinch, please talk to him about the upcoming distinctions between provider/account/environment, and think through the implications10:57
fwereadenatefinch, I will hopefully be in a position to pop on late tonight so I will join you if I can10:57
fwereadenatefinch, I would suggest it's something to write into the spec but explicitly carve out as a "phase 2" task10:58
natefinchfwereade: cool.  I was hoping to push off provider-specific constraints until v2, but not sure if everyone will agree with that. :)   there's just instance-type and cloud-blacklist10:58
natefinch(v2 of charm level constraints that is)10:58
fwereadenatefinch, ah, didn't know about cloud-blacklist10:58
fwereadenatefinch, to my mind, that implies you need to think it through properly for v110:59
fwereadenatefinch, the big issue os how we identify clouds10:59
natefinchI can think it through for v1, I just don't want to write it for v1 ;)10:59
natefinchfwereade: yes, since right now openstack is openstack, not HP vs. canonistack10:59
fwereadenatefinch, the core question is "what is a cloud" :) ...and how do we clearly identify them11:00
fwereadenatefinch, exactly11:00
fwereadenatefinch, not to mention that azure != azure-in-china11:00
fwereadenatefinch, and that in some ways that matters (does an account bind to a cloud?)11:00
fwereadenatefinch, and in some ways it doesn't (instance sizes are probably the same in the azure case at least)11:01
fwereadenatefinch, (although, yeah, instance types across openstacks will surely vary)11:01
fwereadenatefinch, I think it's about "provider" as a concept distinct from "provider *type*"11:01
fwereadenatefinch, anyway thumper is at the sharp end re providers/accounts, and your notions of "what is a cloud" will need to align11:02
fwereadenatefinch, is that enough context for you to have a useful chat with him, do you think?11:02
natefinchfwereade: yep, thanks, that's very helpful.11:02
fwereadenatefinch, cool11:03
jamfwereade: so I think I've covered the logic question and aside from that one bug, lgtm11:05
* TheMue has to step out, yesterday mentioned appointment in garage. bbiab11:08
perrito666morning all btw11:16
natefinchmorning perrito66611:17
fwereadejam, lovely, thanks11:22
natefinchThe longer I spend trying to come up with  estimates for work items, the larger the estimates get11:24
perrito666natefinch: it usually is the other way around11:26
natefinchperrito666: no way... the longer I spend on it, the more paranoid I get that I'm not estimating enough time.  I don't think anyone in the history of programming has ever over-estimated the amount of time something will take to implement.11:27
perrito666a truck full of oxygen tanks just took a speedbump at 40-60 ... I am still trying to have my blood unfreeze.11:33
perrito666that is km not miles11:33
perrito666anyone feels like reviewing this overly trivial change? http://reviews.vapour.ws/r/289/diff/#12:15
perrito666brb, mechanic12:16
voidspacefwereade: ping12:26
fwereadevoidspace, pong12:26
voidspacefwereade: are you up to date with the latest strategy for reboot and the clearing of the reboot flag?12:27
voidspacefwereade: the latest strategy is to clear the flag *immediately before* reboot12:27
voidspacefwereade: so there is still a very small window for jujud to die and the reboot not to happen12:28
fwereadevoidspace, I think so, yeah -- so long as we schedule a delayed reboot before clearing the flag, and then do an immediate reboot afterwards to save time, I'm ok with it12:28
voidspacefwereade: cool12:28
voidspacefwereade: some of your comments on the PR, from a while back,  haven't been replied to since discussions12:29
voidspacebut they're now out of date12:29
fwereadevoidspace, it leaves the possibility that someone will cancel the reboot but I can live with that12:29
voidspacefwereade: by "someone", you mean a user logged into a machine?12:29
fwereadevoidspace, yeah12:30
voidspacethey get the mess they asked for then12:30
fwereadevoidspace, or, y'know, just any old manifestation of the universe's desire to teach us humility12:30
voidspaceI have quite a few friends like that12:30
voidspaceor in fact, just juju itself...12:31
=== urulama is now known as urulama___
kwmonroewelp.. that's the berries12:42
jamkwmonroe: that won't be the first password that this irc channel has voided  :)12:54
kwmonroeugh.  just got good at typing it too.12:55
jamkwmonroe: if it had been longer lived, I would just say "you probably needed to rotate your passwords anyway"x12:55
ashipikaHello… could somebody help me out with the lxc provider?13:16
jamashipika: what's up ?13:16
ashipikamy question is: is there a performance benefit to reusing existing containers for new units instead of creating new ones.. especially in the case of lxc-clone-aufs being defined in the environments.yaml13:17
ashipikajam: i'm trying to fix a really annoying bug..13:18
ashipikajam: where there's a leftover container that fails to start in a previous deploy and is not cleaned up during destroy-envrionment13:18
ashipikajam: so when i bootstrap a new environment and try to add a unit, it reuses the leftover container.. and of course, fails to start again13:20
ashipikajam: a workaround i wrote is to retry by starting a new container.. and it works.. but the questions is why reuse containers in the first place..13:20
=== ChanServ changed the topic of #juju-dev to: https://juju.ubuntu.com | On-call reviewer: see calendar | Open critical bugs: 1387172
sinzuinatefinch, :(  the two restore tests are failing. I thought they would be fixed with the other fixes landed yesterday. Can you arrange for someone to look into bug 138717213:31
mupBug #1387172: restore consistently fails <backup-restore> <ci> <regression> <test-failure> <juju-core:Triaged> <https://launchpad.net/bugs/1387172>13:31
=== dimitern is now known as dimitern_afk
perrito666sinzui: looking13:33
bacsinzui, mgz: jenkins-github-lander has been updated.  bin/lander-check-pulls now accepts a --debug flag to be a little less mysterious about what it is doing.13:33
mgzbac: I'll have a look13:35
natefinchsinzui: sure13:37
perrito666natefinch: o/13:37
sinzuiperrito666, sorry, I know restore code didn't change13:38
perrito666sinzui: worry not, I am pretty sure I know what changed13:38
perrito666natefinch: sinzui I am pretty sure this is because of the changes on _id field for machines13:46
marcoceppiwhat's the default scope?13:46
marcoceppion a relation13:46
mattywhas anyone seen/ is fixing this failure? http://paste.ubuntu.com/8733212/. It's very intermittent13:47
mgzmattyw: that one is new to me13:47
mattywmgz, I'm being lucky today, It happened here: http://juju-ci.vapour.ws:8080/job/github-merge-juju/1053/console13:48
mgzI saw another one of yours on that too13:48
mattywmgz, and I got a different failure here http://juju-ci.vapour.ws:8080/job/github-merge-juju/1052/console13:48
mattywmgz, yup, pretty* sure it's not my change doing it13:48
mgzseems worth a bug with intermittent-failure tag13:49
perrito666mattyw: mgz devecheney was complaining of the same last night iirc13:49
perrito666mattyw: updated dependencies later?13:49
mgzperrito666: he got it in a landing job, which should always have deps as specified13:50
mattywperrito666, yep, I've not actually seen the test fail locally though13:50
natefinchperrito666: yeah, that id change seems like it screwed up a bunch of stuff13:50
perrito666natefinch: it did13:50
perrito666I am also not sure if my fix is right, I am for the moment just trying to figure if I got the issue right13:51
natefinchperrito666: this is why you don't encode metadata into an id field13:51
mgzbackup/restore on ci is currently very unhappy13:51
perrito666mgz: I know I know, working on it13:52
mgzperrito666: <313:53
perrito666man, testing old restore requires some patience14:10
katcohey everyone, i'm ocr today, but i've been directed to try and fix a bug before our cut tomorrow, so i'll be working on that. sorry for any inconvenience.14:36
perrito666its ok we can optically recognize characters for you14:39
natefinchfwereade: you have any time to talk about exposing zones to charms? There's a spec, but it's huge and not super-focused14:39
perrito666this is how waiting for the old restore to finish feels :p https://www.youtube.com/watch?v=qLlUgilKqms14:42
perrito666who wants to be the hero to unlock CI with a mighy review? http://reviews.vapour.ws/r/292/14:48
mgzperrito666: done14:50
fwereadenatefinch, probably in an hour?15:03
natefinchfwereade: sure15:03
perrito666sinzui: mgz how far are we from a new run of functional-backup-restore* ?15:22
sinzuiperrito666, 1h15:22
* perrito666 cries15:23
sinzuiperrito666, this is a rebuild of the last rev15:23
perrito666sinzui: tx15:23
voidspaceand back...15:31
TheMuevoidspace: made a knot into your internet wire?15:40
voidspaceTheMue: they actually fixed it!15:40
voidspaceTheMue: I'm back up to 15mbs15:40
voidspaceafter six months...15:40
voidspaceTheMue: only 1mbps upstream, but it's still double yesterday...15:41
TheMuevoidspace: is this the promised bandwidth?15:42
voidspaceTheMue: yep15:44
TheMuevoidspace: that's good. for a longer time I only had about a half of the promised bandwidth, the backend has been too slow for all who ordererd a higher bandwidth15:45
TheMuevoidspace: now I'm at 80 to 90% downstream and 100% upstream most of the times15:46
TheMuevoidspace: means 40 to 45 mbits down and 10 up15:46
voidspaceTheMue: yeah, that's pretty normal to get much less than the "maximum" you pay for15:47
TheMuevoidspace: exactly15:47
voidspaceTheMue: in theory this should be a 12mbit line - it's currently 15mbit and I'm actually getting 1215:47
voidspaceTheMue: 4045mbits15:47
voidspaceTheMue: lucky man :-)15:47
TheMuevoidspace: I am, yeah. but you know, it never can be enough *lol*15:48
voidspacetrue enough15:51
voidspacegsamfira: ping15:52
voidspacenetworkInterface := instancesResp.Reservation.Instances[0].Interfaces[0]15:58
voidspaceheh :-)15:58
gsamfiravoidspace pong15:59
voidspaceactually I want15:59
voidspacenetworkInterfaceId := instancesResp.Reservation.Instances[0].Interfaces[0].Id15:59
natefinchfwereade: in moonstone whenever you're ready: https://plus.google.com/hangouts/_/canonical.com/moonstone?authuser=116:00
perrito666sinzui: mm, something is odd in the test run its failing even before getting to the test per se17:01
sinzuiperrito666, :( Hp might be having issues. Your rev is still queued to be tested17:13
natefinchhazmat: do you have time in the next hour to talk about exposing zones to charms?  It's on my list of stuff to do this cycle and I want to make sure I understand what needs to be implemented17:18
hazmatnatefinch, i have time now for 12m17:18
natefinchhazmat: https://plus.google.com/hangouts/_/canonical.com/moonstone?authuser=117:18
=== dimitern_afk is now known as dimitern
perrito666can I get a second review on http://reviews.vapour.ws/r/289/? its one line18:13
natefinchperrito666: I guess there's no test for that?18:14
natefinchperrito666: maybe there should be?18:14
* natefinch gives more work to Horacio, as usual.18:15
perrito666natefinch: yes, good idea18:15
perrito666natefinch: its my job to do the work you give me :p not that I am going to complain18:15
natefinchgah, why the hell can't people document their friggin' features?18:17
natefinchlatest offender:  zone placements   evidently juju deploy zone=foo is a thing, but you wouldn't know it from juju help deploy18:18
perrito666natefinch: well it wouldnt be fair, if they did I would be the only one with more work18:19
perrito666dont we have tests that check that all flags are documented?18:19
natefinchprobably not.... that's a little hard to do18:19
perrito666natefinch: well you know the flags you just need to call help and check that they at least appear there18:20
natefinchperrito666: well, in this case, zone is a placement directive, which means it's not actually a flag18:22
natefinchit's 'juju deploy mysql zone=us-east-1b'18:23
natefinch(not -zone=us-east-1b)18:24
natefinchperrito666: er, I guess it's only add-machine18:30
perrito666natefinch: tested19:03
mfoordg'night all19:03
natefinchperrito666: looking19:10
natefinchperrito666: reviewed, couple minor things in the test19:12
perrito666natefinch: did you notice that you can actually select blocks of code and comment on that?19:18
sebas5384hi o/19:21
sebas5384i was wondering if I still need swift in openstack in order to bootstrap an openstack environment19:21
natefinchsebas5384: we're working on removing the storage requirement for juju, but I'm not sure it's in stable code yet19:24
perrito666natefinch: fixed19:25
sebas5384natefinch: yeah, think I sow that in a release notes somewhere19:25
natefinchsinzui: the storage requirement for juju was removed recently, but I presume that's in 1.21, which isn't a stable release, right?19:25
sinzuinatefinch, that is right19:27
natefinchsebas5384: so if you want to do some testing with 1.21 alpha, you're welcome to.  We're pushing to get it to stable in the next few weeks19:28
sebas5384ahh great, I'm going to download it to test it :)19:28
sebas5384thanks!! :D19:29
natefinchperrito666: ship it!19:31
perrito666utils dont have a bot right?19:35
* perrito666 feels a bit silly19:35
natefinchI think that's correct19:35
perrito666man do I feel dumb :p19:37
perrito666reminds me of an european guy I found in a public restroom at a party on the weekend he was making all kind of gestures to the water tap until I passed and just turned the handle on and he stood looking at it and said "oh, one of those"19:38
natefinchhaha, I do that19:38
perrito666sinzui: natefinch mgz aaand we have a pass http://juju-ci.vapour.ws:8080/job/functional-ha-backup-restore/963/19:44
natefinchperrito666: yay!19:45
sinzuiyep I was just marking the bug fix released19:45
perrito666sinzui: tx19:45
sinzuithere is something a miss in the unittest runs. both an instance and a real machine disappeared. I am rerunning the tests19:45
perrito666sinzui: I  have my finger ready to trigger a $$merge$$ :p19:49
sinzuiperrito666, go ahead, you are unblocked19:49
perrito666davecheney: could you tell me a bit more about http://reviews.vapour.ws/r/275/#comment1666 ?19:49
=== ChanServ changed the topic of #juju-dev to: https://juju.ubuntu.com | On-call reviewer: see calendar | Open critical bugs: None
davecheneyperrito666: it's better to write code that doesn't need that bit of boiler plate20:33
davecheney'cos if that method gets refactored and that code lost20:33
davecheneyall hell will break loose20:33
natefinchI agree...   foo := foo is cute for language nerds, but it's never a good idea in production code.20:41
thumpermenn0: change pwd code updated21:25
menn0thumper: ok, i'll have a look shortly21:25
perrito666davecheney: I guessed so much, then I git blamed and suddenly I doubted myself21:39
menn0thumper: done21:49
menn0thumper: if you have a chance could you look at http://reviews.vapour.ws/r/284/ pls?21:50
katcois there a way to specify an ec2 availability zone when bootstrapping?21:57
perrito666katco: having ec2 complaining of no more machines?22:00
katcoperrito666: i am =/22:00
perrito666katco: change region, goes faster22:00
katcoperrito666: you sound like you've run into this before :)22:00
katcoperrito666: tyvm, sir. i'm unblocked :)22:01
perrito666katco: backup and restore testing makes me bump into this often :p22:02
katcoperrito666: hehe22:02
wwitzel3menn0: ping22:14
menn0wwitzel3: hi22:14
wwitzel3menn0: I am unable to replicate that bug, can you try destroying your lxc template and see if you run in to the same issue?22:15
menn0wwitzel3: will do22:15
perrito666davecheney: natefinch if I understand correctly that is done because the member from the for declaration is always the same var but the one inside is subject to the scope of that loop right?22:16
wwitzel3menn0: I tried with both trusty and precise and I'm not getting any of those errors, what i think is likely the cause is the lxc-template we generate for machine-0 probably has the old "anything" certs22:16
menn0wwitzel3: that's quite likely. I think I've had these templates for ages.22:17
wwitzel3menn0: I'm testing out ec2 right now just to be sure.22:17
menn0wwitzel3: if that is the problem (I'm bootstrapping again now) how do avoid it biting other people?22:18
menn0wwitzel3: should we have an upgrade step for alpha2 or 3 that deletes the templates?22:18
wwitzel3menn0: good question, lets see if removing the template indeed fixes it for you, and if so, I'll defer to thumper or jam on what they want to do.22:19
menn0wwitzel3: sounds good22:19
wwitzel3menn0: and then what ever they tell me to do, I'll do :)22:19
wwitzel3menn0: we could always go with the olde, "local provider isn't production"22:20
menn0wwitzel3: true22:20
wwitzel3menn0: but that isn't very friendl ;)22:20
menn0wwitzel3: no it's not and deleting the templates probably isn't that hard22:20
menn0wwitzel3: although I guess it's not an upgrade problem actually22:21
menn0wwitzel3: this happens even with a fresh bootstrap22:21
menn0wwitzel3: I wonder if there's a way that Juju could detect that the certs are bad22:21
wwitzel3menn0: maybe, I don't know enough about how that template is generated .. I think we won't that the machine-0 certs are bad until another machine attempts to connect.22:22
menn0waigani: since thumper doesn't appear to be around could you have a look at http://reviews.vapour.ws/r/284/ please?22:27
waiganimenn0: half way through22:27
menn0waigani: ok cheers :)22:27
wwitzel3menn0: ec2 is good22:29
menn0wwitzel3: my bootstrap just finished here and I still see the problem22:30
menn0wwitzel3: :(22:30
menn0wwitzel3: the template was definitely re-created from scratch. I watched Juju do that by monitoring lxc-ls --fancy22:31
davecheneyperrito666: https://golang.org/doc/faq#closures_and_goroutines22:33
perrito666davecheney: tx22:35
davecheneytalks about a different problem, but the underlying cause is the same22:35
davecheneyfor _, x := range A_SLICE_OF_VALUES {22:35
davecheney        go something(x)22:36
davecheneyin this case it was22:36
davecheneyfor _, x := range A_SLICE_OF_VALUES {22:36
davecheneyappend(something, &x)22:36
wwitzel3menn0: hrmm, i've got nothing .. all clean http://paste.ubuntu.com/8738740/ (machine-0), http://paste.ubuntu.com/8738764/ (unit-mysql)22:38
perrito666davecheney: that is why it does x:=x to get a non shared x I assume22:38
wwitzel3menn0: all-machines is aggregating .. machine-1 and machine-2 are good .. not even one occurance of that error you are seeing :/22:38
menn0wwitzel3: well I'm seeing the same as what I reported on the ticket. All machine logs except machine-0 and and all unit logs are reporting the rsyslog worker continually restarting22:39
menn0wwitzel3: so you've tried it with the local provider on your machine?22:39
perrito666and yes, it is waiting for someone to not know this to delete that line thinking its useless22:39
davecheneyperrito666: yup22:40
davecheneybut that nastyness can be avoided by changing the type of the slice being iterated over22:40
wwitzel3menn0: yeah those logs are local provider22:40
perrito666well since I am refactoring, why not22:40
* thumper is back22:40
thumperphysio again22:40
menn0wwitzel3: and you're using current master not 1.20?22:40
* davecheney pops champaign 22:40
* thumper is somewhat broken22:41
perrito666davecheney: or, I could delete that line and let the hell break loose and then go to sleep :p22:41
davecheneyperrito666: whatever works for you22:41
wwitzel3menn0: yep, current master, 4e078f90b7d31b80feac167197679ad67a59650c22:41
menn0wwitzel3: that's what i'm using22:42
menn0wwitzel3: wtf!22:42
menn0wwitzel3: there must be some difference between our machines22:42
wwitzel3yeah :/22:42
menn0wwitzel3: go version? I'm using 1.2.1 linux/amd6422:43
wwitzel3menn0: go version go1.2.1 linux/amd6422:43
wwitzel3menn0: Linux wwitzel3-ThinkPad-W540 3.16.0-23-generic #31-Ubuntu SMP Tue Oct 21 17:56:17 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux22:43
davecheneyperrito666: i thought this was matty's change anyway ?22:44
menn0wwitzel3: remind me how to get that?22:45
perrito666nope, I refactored the tests for peergrouper because it was causing me an import cycle22:45
wwitzel3menn0: uname -a22:46
menn0wwitzel3: Linux bazhou 3.13.0-37-generic #64-Ubuntu SMP Mon Sep 22 21:28:38 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux22:46
waiganimenn0: done http://reviews.vapour.ws/r/284/22:47
menn0wwitzel3: your kernel is a little ahead but I don't see how this would affect this problem22:47
wwitzel3menn0: yeah, me either22:47
menn0wwitzel3: are you on utopic or trusty22:48
menn0wwitzel3: i'm still on trusty?22:48
wwitzel3menn0: utopic, deploying trusty22:48
wwitzel3menn0: but my ec2 was all trusty22:48
menn0wwitzel3: but juju was built on utopic right?22:49
menn0wwitzel3: that might be the difference22:49
menn0wwitzel3: I think we need to try on some other people's machines to narrow this down22:50
anastasiamackatco: morning :-) saw ur email22:51
wwitzel3menn0: maybe .. we need to draw some straws22:51
wwitzel3exactly :)22:51
katcoanastasiamac: good morning :)22:51
anastasiamackatco: makes sense to me but what do i know22:51
menn0thumper: what distro is your machine running?22:51
thumpermenn0: ubuntu :-)22:51
menn0thumper: which series I mean22:51
wwitzel3thumper wins!22:52
menn0can you follow the repro instructions for bug 1387388 with current master please?22:52
mupBug #1387388: rsyslog worker broken due to certificate signing problem <juju-core:New> <https://launchpad.net/bugs/1387388>22:52
menn0thumper: it happens consistently for me but never for wwitzel3 but I'm on trusty and he's building on utopic22:53
menn0thumper: we're trying to establish if that's the difference22:53
* thumper grabs upstream22:55
thumperWARNING unknown config field "tools-url"22:55
thumperI get that six times22:55
thumperjust destroying the local environment22:56
wwitzel3yeah, I got that too22:56
menn0that happens when you mix Juju versions22:56
menn0I think what's happening is that tools-url is no longer a known config option22:56
menn0everything still works but it is annoying22:57
thumperis there an upgrade step that fixes the config?22:57
menn0I don't think that will help22:57
menn0it's the client22:57
thumperno it isn't22:57
menn0perhaps i'm wrong...22:57
thumperwhere is that warning coming from22:58
thumperI bet it is the bootstrap-config stored in the JENV22:58
menn0that sounds right22:58
wwitzel3menn0: oh, the certs are stored in the local.jenv .. so if that was stale, that might also impact the rsyslog clients.22:59
wwitzel3menn0: but we remove that with destroy-environment right?23:00
menn0wwitzel3: the .jenv is certainly supposed to get removed on destroy23:00
wwitzel3menn0: yeah, I thought so23:01
wwitzel3menn0: brb,grabbing some food23:01
thumpermenn0: what should I see?23:05
menn0thumper: check the logs of any of the machines except 0, or any of the units23:06
thumperseems to work here23:06
menn0thumper: do you see the rsyslog worker restarting23:06
thumperalthough I have different logging issues23:06
thumperbut I think that is an older currently running local provider for another user23:06
perrito666oh I see why this change was not made before23:10
anastasiamacthumper: wallyworld assured me that noone in the world has "tools-url" anymore23:10
anastasiamacthumper: and if they do, they will change it :-)23:10
wallyworld_tools-url was deprecated post 1.1623:11
anastasiamacwallyworld_: yes and removed now-ish ;-)23:12
menn0waigani: thanks for the review. looks like you weren't looking at the latest version as the first thing you picked up was fixed in rev 2 (pushed last night)23:12
thumperwallyworld_: so where did it come from?23:13
rick_h_thumper: just looking at the title, admin can see other user's passwords?23:13
thumperrick_h_: not see, but change23:13
rick_h_thumper: ah ok /me goes back to reviews23:13
* wallyworld_ reads backscroll23:13
thumperrick_h_: juju only stores a salted hash of the password23:13
thumperrick_h_: so if you forget or lose it, we have to have a way to generate another23:13
rick_h_thumper: ok cool, sorry. Later in my day so misread that as admin could see it23:14
perrito666davecheney: sorry mate, making the change to a pointer slice goes all the end to replica set public API and replicaset.Config document it is not worth it, Ill make it cleaner so it is obvious to someone that plays go by the ear23:14
thumperrick_h_: np23:14
wallyworld_thumper: i'm guessing you have an environments.yaml that is old, one which had a tools-url config attribute specified perhaps23:14
* thumper just hit two different intermittent test failures23:14
thumperwallyworld_: there is no tools-url in my environments.yaml23:14
thumperI looked23:14
thumperand the environment I created was a few days ago23:15
thumperwallyworld_: it is possible that the tools-url was populated but empty23:15
thumperso it existed as a value, but empty23:15
wallyworld_in that case i'm not sure, will look into it23:15
wallyworld_it should have been totally deleted23:15
wallyworld_from the code23:15
wallyworld_so should not be present, empty or not23:16
anastasiamacwallyworld_,thumper: tools-url is not in current codebase23:16
* anastasiamac thinks23:17
wallyworld_thumper: maybe you have an older jenv file which is being used?23:17
thumperwallyworld_: nope23:17
thumperanastasiamac: yes, but when I created the environment a few days ago, was it there?23:18
wallyworld_it could have been23:18
anastasiamacthumper: can u do a search in ur codebase?...23:18
thumperanastasiamac: I have master :)23:18
thumperI expect it to be the same as your master23:18
thumperdvcs ftw23:18
anastasiamacthumper: my master is the same as urs23:19
thumpernot there23:19
menn0wwitzel3, thumper: I'll do some more experimentation to see if I can figure out the rsyslog issue23:19
thumperwallyworld_: tools-url was in the 1.20 branch23:20
wallyworld_yes, it was processed as a deprecated attribute23:20
menn0thumper, wallyworld_, anastasiamac: I certainly see this warning when I've bootstrapped with 1.20 and then upgrade to 1.2123:20
menn0when I later destroy the environment with the 1.2123:20
wallyworld_what we should do in 1.21 is add an upgrade step to delete it23:20
anastasiamacmenn0: could u paste the warning u get?23:21
wallyworld_i think because it is defined in the config schema, an empty value is inserted23:21
wallyworld_we already have an upgrade step to delete old values23:21
thumperwallyworld_: 1.20 branch, environs/config/config.go:85223:21
waiganimenn0: just the first comment if from rev 1. The rest are rev 2.23:21
wallyworld_we just need to add tools-url to that existing step23:21
thumperwallyworld_: it always added a blank "tools-url" to the config23:21
menn0waigani: yep. got it23:22
wallyworld_right, that explains it23:22
wallyworld_we'll add the upgrade step to delete it23:22
thumperwallyworld_: also, bootstrap-config in the jenv on the user's machine will still have it23:22
thumperwallyworld_: which means they will still get the warning23:22
thumperevery time they use the environment23:22
thumperunless you update the bootstrap config in the jenv file23:22
wallyworld_maybe we should only emit that warning if the value is not ""23:22
* thumper nods23:23
thumperthat would reduce the likelihood of false errors23:23
wallyworld_cause changing jenv = yuk23:23
* wallyworld_ looks at anastasiamac :-)23:23
thumperalexisb: ping23:23
thumperha, it is only 4:23pm for alexisb23:23
thumperso close23:24
thumperalexisb: last part of first iteration of user management has landed for 1.2123:24
thumperit is DONE DONE23:24
wallyworld_about time :-P23:24
thumpertrue that23:24
* anastasiamac looks back at wallyworld23:24
thumpernow... on to this messy things23:24
anastasiamacwallyworld_: m onto it ;-)23:25
rick_h_thumper: and this means the remote auth stuff is there to hook into?23:25
thumperanastasiamac: wallyworld_ is looking at you with the not so subtle suggestion that you do all the work23:25
thumperrick_h_: no23:25
rick_h_thumper: but but but :P23:25
thumperrick_h_: this means that juju has users and environments have users23:25
wallyworld_anastasiamac: awesome - you need to add to the existing upgrade step, and also not emit a warning if an unknown alue is empty23:25
thumperrick_h_: and they can log in23:25
thumperrick_h_: cmars is working on the remote identity stuff23:25
thumperrick_h_: and making good progress23:25
wallyworld_thumper: it's called delegation :-)23:25
rick_h_thumper: so how long before until we need the GUI to be speaking this stuff?23:25
thumperrick_h_: so this remote identity stuff is not blocked on us at all23:26
cmarsrick_h_, we can stub out the GUI until you're ready :)23:26
thumperrick_h_: not sure what you mean there23:26
rick_h_thumper: cmars I mean when will someone bootstrap with these users and want to type that username/password into a GUI?23:26
thumperrick_h_: the GUI doesn't need to do any of this23:26
thumperrick_h_: so, this isn't the boostrap user being different23:27
cmarsrick_h_, ah, that's not remote identity23:27
rick_h_thumper: so if I juju deploy juju-gui into an environment as user XXXX, I don't have the secret, but need too login with user/password?23:27
thumperrick_h_: that will come in 1.2223:27
rick_h_thumper: ah ok23:27
rick_h_thumper: yea, that's whaat I'm tring to track when we need to start updating our login box23:27
rick_h_thumper: ok, but we do need to update the username box23:27
ericsnowmenn0: when you have a sec, take a look at the latest revision revision of those two review requests (243, 268)23:27
thumperrick_h_: you can use "juju api-info user password"23:27
rick_h_because logging in with the secret as 'admin' might not be true23:27
thumperrick_h_: and that will give you the username and password to log into the gui23:27
rick_h_thumper: ok, is this documented so we can poke at it?23:28
thumperrick_h_: I added it to the 1.21 release notes23:28
thumperrick_h_: poke away23:28
thumperrick_h_: it is in 1.21-alpha223:28
menn0ericsnow: I will try. juggling a bunch of stuff right now23:28
rick_h_thumper: ok, so this will be in 1.21 mid-month it sounds like23:28
ericsnowmenn0: no worries :)23:28
thumperrick_h_: ack23:28
rick_h_thumper: ok23:29
thumperrick_h_: but for now, the initial user is still "admin"23:29
thumperrick_h_: and the password is still stored in the same place23:29
rick_h_thumper: right, but they can create more23:29
thumperrick_h_: ack23:29
rick_h_thumper: and those new ones the user will not be able to login to the GUI without updating it23:29
thumperrick_h_: when you add a user, it gives you a jenv file23:29
rick_h_thumper: so sounds like I need to get some work items down23:29
thumperrick_h_: ah... wat?23:30
thumperrick_h_: why would they not be able to log in?23:30
rick_h_thumper: I bootstrap env as admin.23:30
rick_h_thumper: I create rick user with password23:30
thumperrick_h_: call me23:30
rick_h_thumper: k, sec23:30
ericsnowmenn0: can I give you a hand?23:30
alexisbthumper, AWESOME!!!!!!23:31
* thumper runs to let the dog out23:31
rick_h_thumper: https://plus.google.com/hangouts/_/canonical.com/daily-standup?authuser=123:31
* fwereade also cheers at thumper23:32
menn0ericsnow: thanks but I don't think so. I need to get a branch landed and narrow down a rsyslog worker problem that only seems to happen on my machine23:33
ericsnowmenn0: k, let me know if that changes (I should be on a little longer at least)23:33
thumperwallyworld_: I need to put our 1:1 off for 30 minutes23:46
thumperwallyworld_: is that ok with you23:46
thumperjust found something I was supposed to do an hour ago :-(23:46
* thumper waits for 15s for a response23:46
wallyworld_thumper: sure, np23:47
menn0waigani: i've responded to your review23:47
waiganimenn0: done23:57
menn0waigani: thank you23:58
waiganialexisb: ready when you are23:59

Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!