/srv/irclogs.ubuntu.com/2013/06/25/#juju-dev.txt

bigjoolswhy do we have canonical-written code that isn't hosted in LP?00:00
davecheneybigjools: which code ?00:01
bigjoolsmgo00:02
davecheneybigjools: https://launchpad.net/mgo00:03
davecheneyit is00:03
davecheneybut gustavo is using a 'vanity url'00:03
bigjoolsah ok00:03
bigjoolsthanks00:03
niemeyerbigjools: mgo is not Canonical written, by the way00:03
niemeyerbigjools: it is Gustavo written00:03
bigjoolsI'd change that purely from the PoV of getting lTAB to work in the shell :)00:03
bigjoolsniemeyer: under Canonical time?00:04
niemeyerbigjools: under Gustavo time00:04
* davecheney butts out00:04
niemeyerIts creation predates even Canonical's adoption in Go, so there's little to be said00:04
niemeyers/in Go/of Go/00:04
wallyworldthumper: have you ever had a bzr pointless merge error?00:54
thumperyes01:04
thumperwallyworld: I'm off to walk the dog now01:05
thumperbbl01:05
wallyworldthumper: ok, can you ping me when you get back01:05
wallyworldWARNING  Merging https://code.launchpad.net/~wallyworld/goose/null-project-description into https://code.launchpad.net/~go-bot/goose/trunk would be pointless01:05
wallyworldi'm not sure why the merge fails01:05
wallyworldthe mp shows a nice diff etc01:06
thumperwallyworld: interesting, haven't had exactly that before01:29
thumperwallyworld: try merging trunk into null-project-description first01:29
thumperwallyworld: also, try 'bzr missing --mine ' from the null-project01:29
wallyworldthumper: ok. u had done that prior to proposing i'd swear but i'll try again01:29
wallyworldthumper: i've commented on https://codereview.appspot.com/10447045/, perhaps you could do the same01:30
* thumper looks01:30
wallyworldthumper: so, no revs to merge in from trunk, and bzr missing shows what i would expect https://pastebin.canonical.com/93338/01:45
thumperwallyworld: ok, I have no idea01:47
wallyworldnp01:47
wallyworldi'll bug john latert01:47
wallyworldthumper: i'd like another quick chat about constraints when you have a moment01:48
thumperwallyworld: sure, let me go make a drink and I'll be right back01:48
wallyworldkk01:48
thumpergot a hangout?01:53
wallyworldi'll make one01:53
wallyworldhttps://plus.google.com/hangouts/_/8868e66b07fa02bdc903be4601200d470dae9ee301:54
* thumper bbl for more fwereade chats :)04:53
rogpeppemornin' all05:51
jammorning rogpeppe, you seem up early05:56
rogpeppejam: i wanna get stuff done before i go away on thurs05:56
wallyworldjam: i have to take my son to the dr but do you know why i get a bzr pointless merge error trying to get my bracnh into trunk?06:09
wallyworldWARNING  Merging https://code.launchpad.net/~wallyworld/goose/null-project-description into https://code.launchpad.net/~go-bot/goose/trunk would be pointless06:09
wallyworldi get the above in the tarmac log06:09
wallyworldthe mp diff looks fine06:09
wallyworldbzr missing looks fine - shows the 2 revs that i have committed06:09
wallyworldtim doesn't know what's wrong06:10
wallyworldmaybe you do?06:10
wallyworldi'll check in when i get back06:10
jamwallyworld: because goose successfully merged and committed your change, but failed to push it back to Launchpad06:11
jamso its local branch has merged your changes.06:11
jamwallyworld: I just pushed it out. I don't specifically know why it would have gotten into this situation.06:12
rogpeppejam: i have some outstanding reviews, BTW, which it would be great to get moving; in particular fwereade has verbally ok'd this; i need another review and i'd appreciate your input. https://codereview.appspot.com/10259049/06:13
bigjoolshi.  We need to generate an iso9660 from the user data that the Azure provider needs. We can shell out, or invoke some C, or .... anything better you can think of?06:16
jambigjools: the recommendation that seems to come from the lxc work is that you should shell out06:20
bigjoolsjam: ok cheers.06:20
=== tasdomas_afk is now known as tasdomas
rogpeppebigjools: would it be that hard to write a little package that generates an iso9660 image? the format doesn't look that abstruse, at first glance anyway.07:08
bigjoolsrogpeppe: I'd really rather not reinvent the wheel07:09
rogpeppebigjools: i'm slightly concerned about juju acquiring many external dependencies. is there something that can produce an iso9660 image installed by default in ubuntu?07:11
bigjoolsyes07:11
bigjoolswhat's the problem with external dependencies?07:12
rogpeppebigjools: they might not always be available on platforms we want juju to run on.07:14
bigjoolsok07:14
bigjoolsthe one I am looking at is genisoimage07:15
bigjoolsah it gets installed by ubuntu-desktop07:16
bigjoolsrogpeppe: I think we'll shell out for now (we're short on time) but the motivated can easily replace it with a native piece of code07:20
bigjoolsat least, it's good to show the rest of the code works before spending the time writing this07:21
rogpeppebigjools: that seems reasonable. you could make a package for it anyway, designed to be reimplemented at some future point.07:21
bigjoolsrogpeppe: yes, that would be good07:22
bigjoolsgives a nice clean break07:22
TheMuemorning07:26
wallyworldfwereade: would you be free for a quick chat?08:33
fwereadewallyworld, sure, start a hangout, with you in a sec08:37
wallyworldfwereade: https://plus.google.com/hangouts/_/8868e66b07fa02bdc903be4601200d470dae9ee308:39
thumperjam: hello there09:46
thumperjam: landing issue because I tried to add a new dependency09:47
thumperjam: launchpad.net/golxc09:47
thumpermgz: morning09:47
thumperjam: oh, just saw the emails, seems like you are on it09:48
mgzhey thumper09:51
thumpermgz: hey, have you managed to hand off the api stuff yet?09:51
mgzthumper: yeah... +- one annyoing branch that's ready to land09:53
jamthumper: I installed the new dependency, still fai10:04
jamfails10:04
jam# launchpad.net/juju-core/container/lxc container/lxc/instance.go:49: undefined: instance.Metadata10:04
thumperoh ffs10:04
thumperthat;s right10:04
thumperwallyworld: removed it again10:04
thumperI'll have to fix it tomorrow10:04
thumperjam: but thanks10:04
jamthumper: np. Poke me if you need updated golxc/etc.10:05
thumperok, ta10:05
wallyworldthumper: i did tell you :-)10:05
rogpeppemgz: is that branch really ready to land now? i'm blocked on it10:08
mgzrogpeppe: really :)10:09
rogpeppemgz: i don't see anything different from yesterday (assuming we're both talking about https://codereview.appspot.com/10439043)10:11
mgzpushing now-ish10:17
jamrogpeppe: so the reason thumper went with another package name was so that you could do: import . "launchpad.net/juju-core/testing/checkers" is there a reason you prefer not to use '.' ?10:18
jam(the idea is to make it act the same as gocheck checkers)10:18
rogpeppejam: yeah, i really don't think we should do any more importing to .10:19
rogpeppejam: having one package imported from . is bad enough10:19
rogpeppejam: and i don't think the saved typing is a good enough justification10:20
jamrogpeppe: I'd rather be consistent, so we should probably have that discussion on the list. thumper's assertion at least was without 'import .' there isn't really a benefit of 'testing.IsTrue' over Equals, true)10:20
jamrogpeppe: I do think it makes the code 'read' better.10:20
jamc.Assert(err, Satisfies, errors.IsNotFoundError) without the extra "checkers." in there.10:20
rogpeppejam: we're introducing pollution to the local name space. there's a reason Go doesn't do that all the time10:21
rogpeppejam: yes, i understand that point10:21
jamrogpeppe: so at least my argument is: either (a) put them in 'testing' and import it as a module or (b) put them in checkers and import them as '.'10:21
rogpeppejam: i hadn't realised that was the only reason for putting the checkers in a new package10:22
rogpeppejam: in which case i'd put them all in testing10:23
rogpeppejam: there is another possible reason for putting them in a new package, which is that testing has lots of dependencies, where checkers has almost none.10:24
mgzrogpeppe: 's up10:24
rogpeppemgz: ta10:24
mgzI think william might have wanted something different on the inital-event handling, but I didn't understand where he wanted the change, and just landing this seems... like a desired thing10:25
jammgz: for the test, I think if we move NotifyWatcherC somewhere we can use it, then you can just NotifyWatcherC(resource).AssertOneChange()10:29
mgzyup, though I got the impression he wanted the actual logic, not the test, to change? I wasn't completely clear.10:30
rogpeppejam: surely reading from a channel with a timeout is not something we *need* to factor out10:30
rogpeppejam: we do it all over the place10:30
jammgz: "then do some basic verification of the watcher's state with something like NotifyWatcherC"10:31
mgzrogpeppe: the current code, for instance, does not check that there aren't further events10:31
jamrogpeppe: I think NotifyWatcherC is intended to become something more than just reading off a channel.10:31
rogpeppemgz: that's true, but we aren't testing the watcher here10:31
mgzhelpers are as much about making sure everyone gets the code right as saving typing10:31
rogpeppemgz: we're testing that the watcher is there and watching the right thing10:31
rogpeppemgz: your test tests the former but not actually the latter10:31
rogpeppemgz: to be honest, i think the client test should test that and kill two birds with one stone10:32
mgzI agree there's some question over what should be covered in the client tests rather than here10:32
rogpeppemgz: originally i *only* did client tests, reasoning that they cover almost exactly the same ground10:33
rogpeppemgz: but since the advent of bulk-for-everything (and the lack of a client interface to that), server-side-only tests are necessary10:33
jammgz: testing that the side effect happened is very unit-y vs integration-y, and since we don't have a client-side thing yet...10:33
jamrogpeppe: while integration tests can cover everything unit-tests do, they tend to be overbroad and trigger to many failures when something low-level changes, vs a unit test that tends to be more precise. (I'm personally in favor of having both)10:34
rogpeppejam: i don't really see the interposing the API server makes it that much more of an integration test10:35
rogpeppejam: we're still calling all the way down to mongo10:35
jamrogpeppe: more piecs == more integration10:35
jampieces10:35
jamI don't see how that is hard to see.10:36
jamtesting-per-layer is a good thing to do IMO10:36
rogpeppejam: i'm more interested in test coverage10:36
rogpeppejam: and keeping tests from taking hours10:36
rogpeppejam: and i dislike having a lot of seriously overlapping test code10:37
rogpeppejam: because it wastes time and energy10:37
rogpeppejam: i do see your point about the failures being harder to diagnose though10:39
jamrogpeppe: easier to diagnose failures often save *lots* of debugging time. Which in the lifetime of a project can easily dominate the overall cost.10:39
jamrogpeppe: I agree spending huge amounts of runtime testing the same codepath is a bit of a waste.10:40
jamI personally am in favor of layer testing, and a small number of integration tests.10:40
jamSo you don't have to look at all the edge cases at integration time10:40
jamand just cover things that fail because of the combination, and general "does it work" tests.10:41
TheMuefwereade: ping10:41
rogpeppejam: in this case, i tend to see the server type + the rpc package as one "layer"10:41
rogpeppejam: and that nothing other than the rpc package will ever talk to the server types10:42
mgzso, I have a friend who might be interested in helping with the containerisation work10:42
rogpeppejam: and the overhead of talking through that is fairly minimal10:42
mgzthumper, have we got any reasonably seperate parts that someone could have a go at?10:43
jamrogpeppe: except anytime you have an RPC you *really* want to test them in isolation, so you don't run into the "client 1.11 tests all pass, and 1.10 tests all pass, but 1.11 can't talk to 1.10 because we weren't asserting what the conversation actually was"10:43
TheMuejam: thx for review10:44
rogpeppejam: that's an interesting point; i think we need both.10:45
TheMuejam: I came about chmod the dir because it is created in tests externally. this also can happen later10:45
rogpeppejam: i'm not entirely sure of the best way to do the compatibility checks.10:46
TheMuejam: that's why I correct it during the initial writing ;)10:46
rogpeppemgz: you have a review10:48
jamrogpeppe: I agree we need both10:49
rogpeppejam: i think we could have automated tests for rpc message compatibility, but version compatibility involves much more than just the format of the rpc messages10:50
frankbanfwereade, anyone else: could you please review  https://codereview.appspot.com/10497043 ? Thanks!10:53
rogpeppefrankban: looking10:53
frankbanthanks10:53
fwereadefrankban, I'll take a look after lunch10:55
fwereadeTheMue, pong, very quickly10:55
frankbanfwereade: great thank you10:55
fwereademgz, rogpeppe, jam: I had liked rogpeppe's(?) original model in which the Watch call returns only when the guaranteed initial event has been read off the watcher, so the client-side watcher can hand that straight over to the out chan on creation, which seems kinda nice10:57
rogpeppefwereade: ah, good catch10:57
fwereademgz, rogpeppe, jam: ofc this is a degenerate case, there's no actual data to send, but for consistency's sake we should still be consuming the initial event before we hand over over as a resource to be Watch()ed10:58
rogpeppemgz: yeah, it should do that10:58
mgzfwereade: okay, that's the statement from the review I wasn't clear on10:58
fwereademgz, Next()ed, rather10:58
mgzso, the Watch call needs to pull from the channel, then the test needs to assert there are no further events10:58
fwereademgz, yeah, do an AssertNoChange, tweak the machine, do an AssertOneChange10:59
rogpeppemgz: yes to the former; for the latter, i'd change something in the Machine, Sync, and verify you get a change10:59
fwereaderogpeppe, I think the NotifyWatcherC gives quite a good vocabulary for those tests10:59
rogpeppefwereade: i don't think it's necessary to AssertNoChange11:00
rogpeppefwereade: we're not testing the actual watcher here11:00
rogpeppefwereade: just that it exists and is attached to the right thing11:00
TheMuefwereade: oh, a quick pong? so we should talk about auto-sync later11:00
fwereaderogpeppe, how else do we verify the original event was read? we're testing the SUT by reference to the known and tested-elsewhere characteristics of the watcher11:00
rogpeppefwereade: hmm, good point11:01
fwereadeTheMue, ok, can we maybe talk about it just before kanban? or are you blocked? ...I never double-checked your english, sorry11:03
fwereadeTheMue, would you link me that CL quickly please?11:03
mgzfwereade: so, I still don't know *where* exactly you want that initial event pulled off11:04
TheMuefwereade: we can talk before kanban, yes11:04
fwereademgz, the Watch call should create the watcher, read its initial event, register it as a resource and return its resource id11:05
TheMuefwereade: review is https://codereview.appspot.com/10441044/11:05
mgzwhich one though, the MachinerAPI Watch call, or the Machine Watch call, or newEntityWatcher...11:05
fwereademgz, MachinerAPI11:05
mgzokay.11:06
fwereademgz, this is what we're currently implementing, in terms of the watchers that already exist and have this somewhat convenient initial-event model, which we want to take advantage of in the api11:07
mgz...I thinmk I'll do this in a new branch11:07
mgzbecause it really wants the test helpers11:07
fwereademgz, ok, sgtm11:08
rogpeppefwereade: i have about two other branches blocked on mgz's branch BTW, so i'd very much like something to go in11:09
rogpeppefwereade: soon11:09
mgzlanding the current, then doing that change11:09
rogpeppefwereade, frankban: here's an interesting question: if someone has a service with minUnits=5, then destroys a unit, should the new unit be created when the old unit has gone away entirely, or when it starts to die?11:30
rogpeppedavecheney: hiya11:32
rogpeppelunch11:44
frankbanrogpeppe: my understanding is that MinimumUnits expresses the concept of "minimum amount of units that should be alive". If that's correct, it seems sane to me to react right when one unit starts to die.11:56
jammramm: I saw you show up for a second. I just got back from the restroom11:59
rogpeppemgz: i think you need to re-approve your branch12:09
rogpeppemgz: it died with one of them "bad MAC" errors12:09
mgzrogpeppe: do you know about those?12:10
mgzfirst time I've seen it, will resubmitting help?12:10
rogpeppemgz: yeah, it's intermittent12:10
rogpeppemgz: i thought that was the problem that jam fixed12:10
jammgz, rogpeppe: I think this is a different error. This is using system mongo instead of the tarball mongo12:11
jamI'll try to fix it quicky.12:11
jamrogpeppe, mgz: I'm resubmitting now.12:13
mgzjam: thanks!12:14
jammgz: when I update the config that the tarmac charm uses it rewrites crontab, which means I have to manually go fix it up, and I forgot to set the PATH12:23
mgzurgh12:24
=== ChanServ changed the topic of #juju-dev to: https://juju.ubuntu.com | On-call reviewer: rogpeppe | Bugs: 6 Critical, 81 High - https://bugs.launchpad.net/juju-core/
TheMueclear13:09
jammgz, rogpeppe: Looks like my fix worked.13:16
rogpeppejam: cool13:17
jam(uninstalled the system mongo which the charm also installs, fix up the PATH for the test suite.)13:17
jamI'm actually pretty impressed that there is only 1 failing test13:17
jam(well 2, but it looks like 1 fails to tear down so 2 notices the setup isn't clean)13:17
Makyogary_poster, mramm - is today the cross-team meeting today? I have something on my calendar, but it looks old.13:56
mrammMakyo: it's thursdays now13:57
Makyomramm, Thanks.  Same time?13:57
Makyomramm, actually, I won't have the hangout; can you invite me?13:58
gary_posterMakyo, sorry, I might have misunderstood what you said you wanted last Friday. cross-team is half hour after kanban time13:59
=== wedgwood_away is now known as wedgwood
gary_posterMakyo, I'll delete the Tuesday juju core kanban appt for you?13:59
Makyogary_poster, That's what I meant, I just had the old event on my board for today.13:59
Makyogary_poster, I think I beat you to it :)13:59
gary_poster:-) you declined, I deleted14:00
MakyoAh, okay.14:00
=== BradCrittenden is now known as bac
=== tasdomas is now known as tasdomas_afk
rogpeppea branch to review, if anyone fancies it: https://codereview.appspot.com/1049404316:15
rogpeppefrankban: your merge proposal seems to be corrupted; could you re-propose, please? https://codereview.appspot.com/1049704316:18
=== meetingology` is now known as meetingology
frankbanrogpeppe: done16:20
rogpeppefrankban: ta16:21
ackkhi, I'm hitting an error deploying with juju-core on openstack: instances goes into ERROR state with "ProcessExecutionError". I tried to remove the unit and destroy the service, but it all seems stuck16:21
mgz_ackk: that sounds pretty solidly like an issue with nova, not juju16:23
ackkmgz_, can you force the destroy of the service/unit in juju?16:23
mgz_just `juju destroy-environment`, and manually `nova delete` if needed16:23
mgz_you can use terminate-machine for one machine, but starting from scratch seems wiser16:24
ackkmgz_, yeah I tried that, it doesn't work because the machine is still associated to the unit (which has life: dying)16:25
mgz_right, you just need to wipe if things get that screwed16:25
rogpeppefrankban: you've got a review16:25
frankbanrogpeppe: thanks16:26
ackkmgz_, I see. thanks16:26
dpb1To get the unit tests to pass, is there a reference ~/.juju/environments.yaml file that i need to have?16:36
dpb1I'm trying to follow the "Testing" section from the "CONTRIBUTING" file, but what is written there is not working.16:38
rogpeppei have to go a bit early today16:42
rogpeppeg'night all16:42
hazmatfwereade, in your nomenclature LKP = ?16:45
mgz_dpb1: do you mean the live tests? you don't need environments.yaml for the unit tests16:45
dpb1mgz_: I'm running just make check from the juju-core checkout.  I installed mongodb (obvious error) the errors I'm getting now are not as obvious.  I can paste in one at a time, but I'm wondering if there is more than what the CONTRIBUTING file states.16:48
dpb1mgz_: I checked out juju core with go get -v -u launchpad.net/juju-core/...16:49
dpb1mgz_: I just found the README as well.  I'm missing some things, let me check back, thanks16:51
mgz_dpb1: feel free to pastebin anything if you get stuck16:51
dpb1mgz_: for starters, I'm getting this: http://paste.ubuntu.com/5799084/  (cstack2 is an environment from my ~/.juju/environments.yaml file, which is why I asked my original question the way I did).17:05
mgz_that's probably just bad isolation in one test...17:17
mgz_dpb1: try running with JUJU_HOME=/tmp or something17:18
dpb1mgz_: ok, I have tons of failures like these, will try now.  Hope that will be it17:19
dpb1mgz_:  you were close.  apparently some tests don't isolate against JUJU_ENV (which I had set).  All good now17:24
dpb1lbox is throwing an error about not being about diff branches when I submit a proposal: http://paste.ubuntu.com/5799185/  -- what am I missing?17:43
=== tasdomas_afk is now known as tasdomas
andreas__hso I have a unit in pending state, and nova list shows the instance is in ERROR. It never launched. Any idea how to recover without destroying the environment?18:10
andreas__http://pastebin.ubuntu.com/5799285/18:11
andreas__it's related to #118795918:12
_mup_Bug #1187959: juju does not detect instance launch error, waits forever? <juju-core:Triaged> <https://launchpad.net/bugs/1187959>18:12
andreas__ok, a combination of juju destroy-service and destroy-unit allowed me to "terminate" that machine18:14
andreas__...except terminate-machine doesn't work18:20
andreas__does nothing18:20
=== andreas__ is now known as ahasenack
hazmatmaybe i misunderstanding something.. when does a service in state 'dying' get garbage collected?18:48
ahasenackhazmat: I don't think it does19:04
ahasenackhazmat: or I haven't waited long enough19:04
thumpermorning21:12
=== tasdomas is now known as tasdomas_afk
thumperpoke fwereade21:37
thumperhazmat: ping21:38
hazmatthumper, pong21:42
thumperhazmat: what was the dependency tool you found that caused you to drop the requirements.txt work for juju-core?21:43
hazmatthumper, none, the use case for a commit or ci test runner was fulfilled i thought21:44
thumperhmm... not really21:45
thumpernot in a reproducable way21:45
hazmatthumper, why's is that.. it puts all deps at a known revision based on req.txt or gets head?21:53
thumperhead21:53
thumperAFAIK21:53
thumperI thought, and I may be wrong here, that we just have tarmac doing the landings for us21:54
thumpernot that there is any special revno checking there21:54
hazmatthumper, a frontend script/make blows away the tree between runs21:54
hazmatthumper, go get -u should still pull/update to trunk afaik, but blowing away the tree is simple as well21:56
hazmatthumper, there's a bunch of other build tools, one other i might have mentioned is https://github.com/mozilla-services/heka-build21:57
thumperok, I may take a look21:59
hazmatthumper, g+?22:04
thumperhazmat: sure22:04
=== wedgwood is now known as Guest19180
=== Guest19180 is now known as wedgwood
=== wedgwood is now known as wedgwood_away
thumperwallyworld_: you know how at the end of last week I said your watcher was returning dupes...23:38
thumperwallyworld_: well, I was wrong23:38
thumperwallyworld_: my code was bad23:38
wallyworld_\o/23:38
thumperI'm just trying to work out how to write a test for it23:39
wallyworld_my code would never have any bugs :-P23:39
wallyworld_thumper: talked to martin last night. he's across what we need to do. i asked him to send an email to us summarising the steps to address the main use case (deploy into container on new instance) as well as the next use case (mysql and wordpress in separate containers on an instance)23:41
* thumper nods23:41
wallyworld_there's something easy we can do initially. it might get complicated later23:41
wallyworld_but that can wait23:42
wallyworld_lxc.net is an easy thing to get inter-container networking on the same machine23:42
wallyworld_and we can bridge to get the first use case going23:42
wallyworld_he already has some work in progress towards the goals so that's good23:43
thumperwallyworld_: provisioner_test, TestProvisioningDoesNotOccurForContainers23:56
thumperwallyworld_: why do you have cleanup code at the end of the test?23:56
thumperwallyworld_: doesn't the test framework clean that up?23:56
* wallyworld_ looks23:58

Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!