/srv/irclogs.ubuntu.com/2011/09/29/#juju.txt

SpamapSHmm.. weird problem here.. after a successful install hook..00:03
SpamapShttp://paste.ubuntu.com/698811/00:03
jimbakerSpamapS, trying to think where to start there with that (twisted's reactor structure makes such output frequently useless)00:14
jimbakerSpamapS, is it related at all to the bug you just reported (bug 861298)?00:15
_mup_Bug #861298: libasound2 addition to linaro-x11-base seems suspicious <Linaro Seeds:New> < https://launchpad.net/bugs/861298 >00:15
SpamapSjimbaker: I'm betting that the ZK connection failed for some reason... and it just needs to be handled/reconnected00:15
jimbaker(wrong bug # obviously)00:15
SpamapSjimbaker: same env, but I don't think its related no00:15
jimbakerSpamapS, so is the agent still running on that node?00:17
SpamapSjimbaker: oo, darn, I destroyed the env.. didn't check00:18
jimbakerSpamapS, making agents robust against failure with upstart is something we discussed, i assume it's still not there yet00:18
jimbakerSpamapS, no worries, random failures are easy enough to recreate ;)00:19
jimbaker(of the sort seen here, seriously)00:19
SpamapSjimbaker: I'll report it if I see it again00:19
SpamapSthis particular case is a *very* busy nova compute machine00:19
SpamapSso its entirely possible that ZK was overwhelmed or something else weird happened00:20
jimbakerSpamapS, ahh, that will much more likely expose things00:20
jimbakerSpamapS, also i need to review https://code.launchpad.net/~hazmat/txzookeeper/errors-with-path/+merge/77254, this looks good for getting somewhat better info - knowing the actual path is a pretty big signal, especially as we add more nodes00:21
SpamapS+1 for that00:22
SpamapSuseful errors was a theme of an entire month of development we did at my last job00:22
SpamapSBecause we had so many confused 3 hour pager responses where you spent 3 hours just trying to figure out WTF happened00:23
fwereadejimbaker, SpamapS: upstart is rather critical for orchestra too, I think it's likely to be top of my list as soon as I'm not thinking about the charm store00:24
jimbakeri do wonder if niemeyer is trying to intentionally pun on wtf with the waterfall at http://wtf.labix.org/wtf/00:24
jimbakeri'm sure he is00:24
niemeyerjimbaker: Yes :-)00:24
fwereadepotential recursive acronym, too: WTF Test Failures00:26
niemeyerfwereade: Hah, nice00:26
fwereadeniemeyer: if anyone asks, you could say that, and act all mystified -- "does WTF stand for something else as well? goodness me!"00:27
niemeyerfwereade: Totally :-00:27
niemeyer)00:27
jimbakerbiab, i'm going to hang out over at the python in the cloud meetup going on locally in boulder shortly00:29
jimbakersomeone will be demoing some sort of python mgmt tool for running apps in ec200:30
fwereadeniemeyer: ok, I thought I was ready to merge resolve-formula-names (hm, which should be called merge-charm-names)00:44
fwereadeniemeyer: then I thought the breaking UI changes might need some documentations00:44
fwereadeniemeyer: ...then I thought "NO TIME NO TIME"00:45
fwereadeniemeyer: er, opinions?00:45
niemeyerfwereade: Hm00:45
niemeyerfwereade: One thing we definitely have to do is sending a mail to ensemble@ pointing out what is going on, how the interface is changing, and why00:46
fwereadeniemeyer: indeed -- shall I try to get that in before the merge?00:47
niemeyerfwereade: Yeah, it feels like a good idea to point to people in sync with the merge00:49
niemeyerfwereade: So that trunk users will get to know about it00:49
niemeyerfwereade: and PPA users in a bit00:49
fwereadeniemeyer: ok, I'll write something up now00:50
fwereadeniemeyer: still ensemble@lists.ubuntu.com then?00:50
hazmatTalk done!.. wahoo..01:09
hazmatits very easy to slip up charm/formula01:09
niemeyerjimbaker: and the saga continues..01:17
niemeyerhazmat: Woohay01:17
niemeyerhazmat: how was it?01:17
niemeyerfwereade: Sorry01:21
niemeyerfwereade: juju@01:21
niemeyerfwereade: As hazmat just said, easy to slip :)01:22
fwereadeniemeyer: absolutely :)01:22
fwereadeniemeyer: I wondered if you meant it, because I could swear I signed up to juju@ but still seem to be getting all my mails from ensemble@01:22
niemeyerfwereade: Hmm.. there's an alias between them so it continues working01:24
niemeyerfwereade: I don't think we should be getting emails from the old address, though01:24
fwereadeniemeyer: don't worry about it, I'll double-check, maybe aI got confused around the changeover01:25
niemeyerfwereade: What might be going on is that people (like me!) are still using the old address01:28
fwereadeniemeyer: ah, could be01:30
jimbakerniemeyer, the company is opdemand, they are currently demoing wordpress ;)01:33
jimbakerhttp://www.opdemand.com/01:33
jimbakerthey publish information from one tier to another tier01:35
niemeyerjimbaker: pvt01:39
hazmatouch just got dissed hard by the opscode (adam) from stage01:50
hazmatidempotency... see me after class01:50
niemeyerhazmat: Very noble of him :-)02:01
niemeyerhazmat: The best answer to a public insult is making juju rock harder02:06
_mup_juju/env-origin r374 committed by jim.baker@canonical.com02:08
_mup_Fixed review points02:08
_mup_juju/env-origin r375 committed by jim.baker@canonical.com02:15
_mup_Merged trunk02:15
jimbakeroh, i'm glad i just tried running ./test against trunk, the failures are there, not in the merge with env-origin05:16
SpamapSwe should rename the juju binary to just jj05:19
jimbakerSpamapS, please don't give anyone any ideas on that ;)05:20
jimbakerSpamapS, i assume you're seeing the same failure as here: http://wtf.labix.org/366/unittests.out.FAILED05:21
jimbakerlooks like it was introduced in r36505:23
jimbakerbcsaller, have you tried looking at trunk since your merge?05:23
bcsallerjimbaker: yeah, its a known issue, sorry. Still trying to get omega in the proper state to merge, it was missing some tests and we know how testing goes... esp. with this level of system interaction05:28
jimbakerbcsaller, ok, thanks for the update05:28
bcsallerjimbaker: it happened because lxc-lib is a prereq to omega but other thing depending on it got merged first05:28
jimbakerbcsaller, ok, makes sense05:30
jimbakerpresumably one of these things corrects the ppa name too05:30
jimbakeralthough i would expect the bridge to the support of juju-origin in my env-origin branch would take care of that regardless05:31
bcsallerjimbaker: it will, but I think I will do that in a subsequent branch, using the ppa to get started with local dev (and fixing trunk tonight) seems fine05:33
_mup_juju/env-origin r376 committed by jim.baker@canonical.com05:35
_mup_Merged trunk05:35
jimbakerbcsaller, sounds like a good plan to me05:36
bcsallercool05:36
SpamapSjimbaker: I don't like typing juju .. its all right index finger.. jj would be a nice shortened version. :)05:52
SpamapSwe can just symlink it in the package. :)05:52
jimbakerSpamapS, i wonder if my typing here is canonical - i use index for j, middle for u, so it's not as bad in this case05:53
jimbaker(that's probably a misuse of canonical...)05:54
jimbaker(more like standard i guess)05:54
jimbakerfwereade, is local resolution of charms now broken?05:59
jimbakeri noticed the change to the series, but i'm seeing this on the machine agent06:00
jimbakerfwereade, so i'm seeing this, http://paste.ubuntu.com/698912/ - anyway, i guess it will have to wait until tomorrow, just using irc async here :)06:02
jimbakerSpamapS, apparently my typing is nonstandard. darn. http://en.wikipedia.org/wiki/Touch_typing06:07
SpamapSjimbaker: very nonstandard, I don't even know how you reach u with your middle finger06:29
SpamapSfwereade: one thing about this breakage.. now any version before this is completely broken because the PPA package wants local: prefixed06:47
SpamapSCharmStateNotFound: Charm 'local:hadoop-slave-1' was not found06:47
SpamapSthats not something you mentioned in your email. :-/06:48
_mup_juju/env-origin r377 committed by jim.baker@canonical.com07:28
_mup_Remove tabs in rst file07:28
roghurray, all tests pass09:14
fwereaderog: belated /cheer10:36
rog:-)10:36
rogmy computer made the requisite sound effect when you mentioned my name10:37
rogwell ok, it was a sort of bleep, but close enough10:37
fwereaderog: haha, awesome10:37
fwereadeSpamapS: hm, I hadn't considered that side effect10:49
fwereadeSpamapS: I kinda feel that we're still (just barely) in "developing, expect breakage" mode, and so it's (just barely) acceptable, if annoying10:53
fwereadeSpamapS: am I right in thinking that error shows up when you have an env running, you upgrade juju locally, and then try to issue commands to the env running the old version?10:54
fwereadeSpamapS: ...or, yes, also if you try to run any older version against the PPA10:55
fwereadeSpamapS: is my interpretation sane?10:55
_mup_Bug #862415 was filed: Juju bootstrap node disappearance <juju:New> < https://launchpad.net/bugs/862415 >13:59
_mup_Bug #862417 was filed: Cloud Foundry server charm was not found on the instance <cloud> <foundry> <juju:New> < https://launchpad.net/bugs/862417 >14:03
_mup_Bug #862418 was filed: Add a way to show warning/error messages back to the user <juju:New> < https://launchpad.net/bugs/862418 >14:04
_mup_Bug #862417 was filed: Cloud Foundry server charm was not found on the instance <cloud> <foundry> <juju:New> < https://launchpad.net/bugs/862417 >14:04
lynxmanSpamapS: whenever you're awake, let me know if you have the final juju package for oneiric :)14:06
_mup_Bug #862422 was filed: Add a "block" add/remove unit hook <juju:New> < https://launchpad.net/bugs/862422 >14:07
rogi can never remember where to create a new issue for code review...14:19
rogis it standard to always create a new branch when making some changes? if i haven't done that, how can i change the branch name?14:37
fwereaderog: depends: if I'm doing a significant chunk of work I'll start by branching, if I'm just (say) addressing a minor review point in an existing branch I'll continue the existing one14:39
fwereaderog: there's a line between them but I find its location varies with my current levels of optimism, forgetfulness, foolishness14:40
fwereaderog: offhand, I'm not sure how best to retroactively branch something without the process being somewhat manual14:41
fwereaderog: you could just bzr diff --old=[revision you wanted to branch from] > somewhere14:42
fwereaderog: bzr revert -r [that revision again]14:42
fwereaderog: bzr commit14:42
fwereaderog: ...and then branch from there and apply the patch14:42
fwereaderog, and everyone else: if there's a better way I'd love to hear it :)14:43
rogat the moment, i'm trying a branch and then a merge with the changes i made in the old branch14:43
rogdunno if it'll work. we'll see.14:43
fwereaderog: I think I tried something like that once, and it went wrong -- if you then revert the parent branch, and *then* merge from it again, I think it'll try to apply your subsequent unchanges14:44
fwereaderog: ...er, if you see what I mean14:45
rogfwereade: hmm, not sure i do. is that something i shouldn't do, or something i will inevitably do?14:47
rogfwereade: it *seems* to have worked14:49
rogexcept the log history is no longer linear, which might be a problem14:49
fwereaderog: possibly I misunderstood what I did, or what you did14:49
rogi did: cd $HOME/gozk; bzr branch lp:gozk my-upcoming-revision; cd my-upcoming-revision; bzr merge ../zk; bzr commit14:50
rogwhere $HOME/gozk/zk was the place i'd been doing the edits14:50
fwereaderog: ah, yes, that sounds fine14:55
fwereaderog: I'd been imagining a situation where you'd made several commits already to the wrong branch14:55
rogfwereade: i made commits, but hadn't pushed14:56
fwereaderog: still sounds fine :)14:57
rogcool14:57
* fwereade crosses fingers14:57
* rog also crosses his fingers.14:58
hazmatbcsaller, none of the things that had lxc-lib as a pre-requisite got merged first.. they where using what was already committed to trunk15:04
hazmatlxc-lib-clone changed the api15:05
jimbakerhazmat, yes, that's what we are seeing on wtf.labix.org right now15:09
hazmatjimbaker, it should be okay after lxc-omega gets merged, but breaking trunk isn't15:10
jimbakerhazmat, agreed with that for sure15:10
jimbaker(invariably when trunk is broken and i do a merge, i blame my branch first. goose chase, not productive... anyway, it happens)15:11
SpamapSfwereade: yes, all older versions of juju break when they deploy because they pull from the PPA15:17
SpamapSfwereade: breaking stuff is fine, but telling us about it *well before* committing would be quite helpful.15:17
SpamapSwhy are we committing to trunk without running the test suite?15:18
SpamapSEvery time I've merged one of my little fixes in, I merge and then run the test suite. :-P15:19
jimbakerSpamapS, fwereade, agreed. and announce it here would be nice. the other thing that bit me last night15:19
fwereadeSpamapS, jimbaker: I'm sorry, it was entirely wrong to assume anyone else had the faintest idea what I was up to :(15:20
fwereadeit never really crossed my mind that niemeyer and I had talked about it lots, but not with everyone15:20
jimbaker fwereade, no worries, just some suggestions to get everyone in the loop first. this channel has its deficiencies, but i poll it much more frequently than email ;)15:21
jimbaker(much more signal for what i immediately need to know)15:21
fwereadejimbaker: definitely15:22
SpamapSAt this point nobody can test the packages in 11.10 properly.. it only works by using juju-branch: lp:ubuntu/oneiric/juju   but thats not the end of the world15:22
SpamapSThe email as great.. and its the only reason I didn't go "WTF!!" right away.. I knew what was up. Its just that the scope of the breakage may not have been fully explored. Given the velocity you all are maintaining, I'm not surprised (or upset :)15:23
SpamapSs/email as/email was/15:23
fwereadeSpamapS: so, the stuff that's currently in 11.10 is old, and therefore broken; I had the impression we were planning to slip in a last-minute update (er... tonight?) with lxc stuff, and origin stuff ...and this stuff15:24
fwereadeSpamapS: and that implies the breakage is at least somewhat temporary15:25
fwereadeSpamapS: have I misunderstood the plans?15:25
SpamapSwe are planning to update again. The point of updating before then though, was to allow testing of what was done up until now.15:26
fwereadeSpamapS: indeed :(15:27
SpamapSDon't worry, I totally understand the implications and reasons behind this incompatible change. I'm only frustrated that I pushed so hard to get a somewhat new version in that is totally broken. :-/15:27
SpamapS"old" is the revision from Monday morning btw.15:28
fwereadeSpamapS: and now you're stuck only able to test the moving target that is trunk :(15:28
SpamapSNo I can test using juju-branch:15:28
fwereadeSpamapS: heh, "old" is everything before this morning :(15:28
fwereadeSpamapS: ah, good15:28
fwereadeSpamapS: which is to say, "less bad"15:29
fwereadeSpamapS: in future I will definitely make sure people know of breaking changes well in advance (if I can't avoid making them at all)15:29
fwereadeSpamapS: is there anything I can do to mitigate the pain now?15:30
SpamapSno. :-/15:30
SpamapSJust git 'er done. :)15:30
SpamapSseriously, ignore my whining.. you all have important work to do.15:31
fwereadeyeah, but so do you, and I'm sorry to have disrupted it :(15:31
SpamapSIt only means the bugs that I might find now will be found after you're done.. and you'll have more of a time crunch to fix them next week.15:32
SpamapSfwereade: please don't feel guilty in any way. You guys can't stop for every squirrel crossing the road. ;)15:33
fwereadeSpamapS: don't worry, I'm neither sobbing nor rending my clothes15:34
fwereadeSpamapS: still wish I'd done it differently, but we live and learn ;)15:34
jimbakerfwereade, please no self-flagellation, ok? ;)15:35
rogjimbaker: i've just pushed a new gozk merge request. any comments or feedback welcome.15:36
roghttps://code.launchpad.net/~rogpeppe/gozk/update-event-interface/+merge/7756015:36
jimbakerrog, thanks!15:37
robbiewSpamapS: so basically all charms are broken now, right?15:37
* robbiew notes this is why we need juju pulling from the archive at release...not a ppa :/15:37
robbiewI realize we need this change to allow for the store to work...which means **WE** will need to fix everyone's charm before release15:38
robbiewnot fair to expect authors to go back and do that now, imo15:38
robbiewand apparently there's no legacy support :/15:38
SpamapSrobbiew: all charms and all juju deployments from 11.10 are broken, yes.  Tracking in bug 82814715:40
_mup_Bug #828147: Ensemble branch option needs to allow for distro pkg, ppa, and source branch install <cloudfoundry:New> <juju:In Progress by jimbaker> <juju (Ubuntu):Triaged by clint-fewbar> < https://launchpad.net/bugs/828147 >15:40
robbiewawesome15:41
SpamapSrobbiew: there is a workaround, which is to add 'juju-branch: lp:ubuntu/oneiric/juju' to your environment config15:41
jimbakerrobbiew, really hope that branch gets approved soon15:42
jimbakerin terms of env-origin15:42
robbiewjimbaker: that change must get in15:42
robbiewagreed15:42
SpamapSThe alternative is to patch the distro version to default to pulling from the distro. I've tried that in a privately built package and it works fine, but the problem is the deployed provisioning agent then still has the distro version deploying from _PPA ..15:42
SpamapSAnyway, the best thing to do is probably to just test from the PPA.. ignore the distro package.. and try to help these guys finish off the work so we can upload "the real juju" ASAP15:43
robbiewagreed15:44
robbiewa hack in the distro is suboptimal15:44
SpamapSworst case.. if something goes horribly wrong and we can't do that.. we can regress the PPA to the distro version and move daily builds to a different PPA.15:44
robbiewif env-origin works, then we can just use that set to the archive, right?15:44
jimbakerfwereade, re robbiew's point - the backwards breaking change seems to be only to change the directory structure. i understand with the juju rename there's been some drastic changes already. but we could make it easier possibly by assuming oneiric15:45
SpamapSrobbiew: env-origin is intelligent, and chooses distro when your client version came from the distro15:45
robbiewcool15:45
SpamapSIts probably going to break horribly on OS X tho. :P15:45
jimbakerrobbiew, env-origin does the right thing15:45
jimbakerSpamapS, it doesn't know about os x yet15:45
SpamapSjimbaker: it shouldn't really. Serious users will probably be very explicit about their environments.yaml15:46
jimbakerSpamapS, the workaround is to set juju-origin manually15:46
jimbakerSpamapS, agreed15:46
jimbakerSpamapS, it might be something as simple as that for os x/non ubuntu clients, it basically insists that this setting be made15:46
SpamapSActually I think it should just choose distro at that point.15:47
jimbakerSpamapS, also a reasonable choice15:48
SpamapShmm.. something's borked with juju-branch too actually15:48
SpamapSAhh, my AMI's are out of date and couldn't download packages.15:49
fwereadeSpamapS, jimbaker: it seems that it would be better to insist on explicit default-series *everywhere* than to special-case non-ubuntu15:52
jimbakerfwereade, certainly a valid point. being clever can surprise15:53
fwereadeSpamapS: that would mean everybody had to change their environments.yaml, though, which is maybe too much to expect15:53
fwereadeSpamapS, jimbaker: if that would be acceptable, though, it feels like the Right Thing to me15:55
robbiewI guess the big problem we need to resolve is broken existing charms...and getting them fixed asap15:56
robbiewthe change is in...and newer charms will be written accordingly15:56
robbiewfor example...our much talked about cloudfoundry charm is now broken :/15:57
robbiewwe need that working by release15:57
robbiewand so on15:57
fwereaderobbiew, SpamapS: are we saying that *charms* are broken, or that *repositories* are broken?15:57
RoAkSoAxhazmat: ping?15:57
jimbakerfwereade, but as i understand it, the charms are not themselves broken by this change, it's how we refer to them in terms of their repository structure and when deployed?15:57
fwereadejimbaker: that's exactly what I'm fretting about15:58
jimbakerfwereade, exactly15:58
fwereade:)15:58
jimbakerfwereade, i looked at the change in trunk. the sole change to the example charms, which should be reasonable for any other charms, is moving the files around one level15:58
jimbaker(reasonable, as in talking about other charms, that is)15:59
fwereadejimbaker: exactly, I don't believe that any charms are themselves broken15:59
robbiewright....sorry...you are correct15:59
fwereadejimbaker: every repository outside the source tree is, though15:59
jimbakernow there is certainly broken code out there that manages this process15:59
jimbakerso pretty much everyone has their own script to deploy a stack of apps. no one is typing it in again and again16:00
jimbakerthat's broken now16:00
robbiewexactly...nothing we can do about that though...sed/awk to the rescue there16:00
jimbakerbut a trivial fix to add in the right locator info16:00
jimbakerthe only challenge is that it totally sucks when they are deployed16:01
fwereadejimbaker: we discussed defaulting to "local:" when --repository is given, but that causes us more problems down the line16:01
robbiewfwereade: ftr, I'm happy this change has gone in...as we need it for the Charm Collection/store thingy ;)16:01
jimbakererrors are nontransparent, and show up in debug logs16:01
fwereadejimbaker: bad-repo errors, right?16:02
jimbakerfwereade, i forget specifics, i made a paste on this last night16:03
fwereadejimbaker: hopefully "cs:blah/blah not found in repo https://store.juju.ubuntu.com" is reasonably clear?16:03
jimbakerlet me see if i can dig it up16:03
fwereadejimbaker: but I suspect it's not so nice if the repo has the wrog structure16:03
jimbakerfwereade, depends on where it is. if from the command line, cool16:03
jimbakerif it is in the debug-log, not so cool16:04
jimbakerway too buried. yes, i will go there. but even for me it's an extra step to ssh in to find what's happening, and i know what i'm looking for16:04
fwereadehm, yes :(16:05
jimbakerthere may be some gnashing and wailing and rending and tearing at the cheeks16:05
fwereadejimbaker ...but wait, when are we hitting repositories except from the command line?16:06
jimbakerfwereade, need to dig up this error for you...16:07
fwereadejimbaker: thanks16:07
jimbakerfwereade, lost it. i will have to recreate. hopefully not some dream from last night...16:09
fwereadejimbaker: np, although from my perspective I kinda hope it *is* a dream16:11
fwereade;)16:11
SpamapSrobbiew: the cloudfoundry charms should work w/ the PPA16:11
robbiewSpamapS: cool, thx16:12
fwereadejimbaker: you have reminded me of one possibility, can I get a +1 trivial on http://pastebin.ubuntu.com/699194/ please?16:17
jimbakerfwereade, looking at it now16:20
SpamapSHey didn't we introduce a schema version or something into ZK so that clients would detect when they are trying to twiddle an incompatible ZK?16:20
SpamapSSeems like this latest formula change woudl be a good time to bump that.16:21
jimbakerfwereade, so basically this covers a bad path16:21
fwereadejimbaker: yep, that's it16:21
fwereadejimbaker: it should just make the error clearer when people try to deploy from an un-upgraded local repo16:21
jimbakerfwereade, well +1, i assume this will not let it get into the bad state i discussed16:21
jimbakerSpamapS, yeah, we have that bit, it would be nice to use in this case. we have the power...16:22
fwereadejimbaker: it won't start or stop anything working, it'll just complain differently16:22
jimbakerfwereade, and i believe more importantly, complain earlier16:22
jimbakerand closer16:22
fwereadeSpamapS, jimbaker: oh, blast, I thought that was discussed but not implemented16:22
fwereadejimbaker: indeed16:23
jimbakerfwereade, https://code.launchpad.net/~jimbaker/juju/verify-version/+merge/7155916:23
jimbakerfwereade, don't worry, i thought i knew what you were working on too16:24
robbiewlol16:24
fwereadehaha16:24
jimbakermostly right, but a few missing, possibly critical bits ;)16:24
jimbakerfwereade, so if you want to make a change to VERSION = 1 in ensemble.state.topology, that sounds like a trivial we might all approve16:27
fwereadejimbaker: was just about to suggest it :)16:27
jimbakerit complains early and close. almost all stuff goes through the topology16:28
fwereadejimbaker: for form's sake: http://pastebin.ubuntu.com/699200/16:28
jimbakersounds good, the minimum requirements specified in that comment have been satisfied, so +1 !16:29
fwereadejimbaker: cheers :)16:30
rogright, i'm off. it's an unseasonably warm and beautiful evening here. the garden calls. see you tomorrow.16:39
jimbakerrog, enjoy!16:41
_mup_Bug #862415 was filed: Juju bootstrap node disappearance <juju:New> < https://launchpad.net/bugs/862415 >16:52
RoAkSoAxb/win 216:53
Aramhello.17:00
Aramwill niemeyer be here later?17:00
niemeyerHallo!17:01
Aramhi!17:02
niemeyerAram: Hey Aram17:03
niemeyerbcsaller: ping17:03
bcsallerniemeyer: hey17:03
niemeyerbcsaller: Hey!17:03
niemeyerbcsaller: I see the problem not only isn't fixed, but it's getting worse: http://wtf.labix.org/17:04
niemeyerbcsaller: Can you please revert the change, if you can't make the tests work again?17:04
bcsallerI didn't get the final +1, my branch is ready to merge17:04
bcsallershould be in your email and I pinged kapil about it as well17:05
niemeyerbcsaller: It already has my +1 I think17:05
niemeyerbcsaller: Or is it another branch?17:05
bcsalleryou said it needed one more sign off at the end and I made changes for the review and no one ok'd them17:05
niemeyerbcsaller: Yeah, from Kapil specifically17:06
bcsalleryeah17:06
niemeyerbcsaller: Did you talk to him about this already?17:08
bcsallerniemeyer: I pinged him about it this morning but haven't seen him17:08
robbiewhe's at a conference today17:08
bcsallerall the tests are passing, I can go and merge it if your ok with that niemeyer17:09
niemeyerbcsaller: Yeah, I think that's the best thing to do17:09
niemeyerbcsaller: Let's invite him for a post-review17:10
jimbakerniemeyer, the env-origin branch is ready for review17:11
niemeyerjimbaker: Cool, I'll check it out17:12
jimbakerniemeyer, thanks17:12
_mup_Bug #862595 was filed: Provisioning agent and destroy-environment show NoneType not iterable on machine shutdown with Openstack <openstack> <juju:New> < https://launchpad.net/bugs/862595 >17:27
HarryPandais there anything in place to test charms with Jenkins etc.17:27
SpamapSHarryPanda: jamespage has some preliminary work on that.17:30
jimbakerSpamapS, looks like more txaws issues there with 86259517:32
SpamapSjimbaker: yeah.. I think that may actually be causing big problems on the provisioning agent.. it doesn't seem to want to provision new instances after that17:33
bcsallerniemeyer: http://wtf.labix.org/ is happy again17:34
bcsallerniemeyer: well, the non-ftests part anyway17:34
jimbakerSpamapS, the provisioning agent certainly doesn't expect errors. i'm not certain what it should be, but just barfing is not one of them17:34
jimbakerbcsaller, those are because of the respository changes, so that's independent17:35
jimbakerSpamapS, ok, that may be overstating it, but it is vulnerable17:36
SpamapSjimbaker: seems like whats happening here is that while the provider has shutdown the machine and changed ZK to remove it, the error has somehow caused the provider to not thing the machine is gone17:37
SpamapSjimbaker: leading to bug 86192817:37
_mup_Bug #861928: provisioning agent gets confused when machines are terminated <juju:New> < https://launchpad.net/bugs/861928 >17:37
jimbakerSpamapS, the periodic rescan is supposed to sync it with reality for cases like this, as i understand it, but the changes in aws api impl is presumably making it confused17:38
SpamapSjimbaker: probably because the group is still there17:39
jimbakerSpamapS, you mean the security group per machine?17:42
SpamapSjimbaker: hrm, no its not there17:43
SpamapSseems like there's some order of ops that gets confused by that NoneType iterable error.17:44
jimbakerSpamapS, as a general rule, we expect specific errors from txaws - it looks like it's not catching this TypeError, so it bubbles all the way up to the reactor loop17:46
_mup_juju/env-origin r378 committed by jim.baker@canonical.com17:51
_mup_Merged trunk17:51
jimbakerjust had an opportunity to see the incompatible juju protocol version in action17:57
SpamapSjimbaker: hah :)17:58
jimbakerit was expected, fortunately... i think the error message could be more helpful on what to do in this case18:00
SpamapSjimbaker: so this does look like nova-api doesn't return the same expected XML after TerminateInstances18:03
jimbakerSpamapS, at the python meetup last night, i think someone mentioned that txaws is using an older version of the aws api. don't know about the truth of this18:04
jimbakerSpamapS, it may have nothing to do here, but supporting multiple apis is tough18:05
jimbakeras usual, we might want to look at boto18:06
SpamapSwell in this case, comparing to boto .. boto just returns whatever list was responded with18:06
SpamapStxaws asserts that it has to be id=instancesSet18:06
niemeyerjimbaker: ping18:21
jimbakerniemeyer, hi18:21
niemeyerjimbaker: Hi18:21
niemeyerjimbaker: Please re-read points 3, 4 and 518:22
niemeyerjimbaker: and compare to the implementation18:22
jimbakerniemeyer, ok18:22
niemeyerjimbaker: Is there anything missing there?18:23
jimbakerniemeyer, #3 - python-txzookeeper dependency is no longer relevant; #4, it uses the original implementation; #5, there is no attempt to test deb repositories, since they are no longer supported18:25
niemeyerjimbaker: Maybe I'm just missing.. where's that: 3) Otherwise, store the version in a variable, and keep parsing18:25
niemeyerjimbaker: 4) Find the version table, and find the installed version18:25
niemeyer?18:25
jimbakerniemeyer, ok, i'm referring to the review points. you are referring to a list from irc18:26
niemeyerjimbaker: yeah18:26
jimbakerlooking at that now18:26
jimbakerniemeyer, ok, i believe what you are saying here is that 6) refers to the installed version, which was stored earlier in 3), is that correct?18:28
niemeyer1) Grab the installed version from the "Installed:" line18:29
niemeyer2) If this is "(none)" return "branch"18:30
niemeyer3) Otherwise, store the version in a variable, and keep parsing18:30
niemeyer4) Find the version table, and find the installed version18:30
niemeyerjimbaker: Is there any ambiguity here?18:30
jimbakerniemeyer, correct, there is no ambiguity. the version table has more information than i have assumed, and this parse needs to take it in account18:31
niemeyerjimbaker: Thanks.18:31
niemeyerbiab18:35
SpamapSwow.. nova-api's response to TerminateInstances is *completely* different from ec218:49
jimbakeri'm feeling sick right now, i will be back later18:56
SpamapShttp://pastebin.com/jFDQLrcQ18:57
SpamapSthats ec2's response18:57
SpamapShttp://pastebin.com/ZMXwA4MY18:57
SpamapSAnd that is openstack18:57
_mup_juju/ftests r5 committed by gustavo@niemeyer.net19:31
_mup_Removing unused imports.19:31
SpamapSnote that bug 862595 is actually pretty serious.. really confuses the provisioning agent.19:40
_mup_Bug #862595: terminate_instances raises NoneType not iterable on machine shutdown with Openstack <openstack> <juju:Invalid> <txAWS:New> <txaws (Ubuntu):Triaged> < https://launchpad.net/bugs/862595 >19:40
SpamapShazmat: when you're around, if you can ack the merge proposal against txaws .. I will patch it into the Ubuntu txaws19:41
robbiewbcsaller: free for a catch up call?19:50
bcsallerrobbiew: I was just sitting down to lunch, can we push it a little bit?19:54
robbiewbcsaller: absolutely not!19:55
robbiewlol19:55
robbiewsure19:55
bcsallerthanks19:55
bcsallerrobbiew: free when you are20:15
_mup_juju/ftests r6 committed by gustavo@niemeyer.net20:15
_mup_Reverse the order of revisions and link them to Launchpad.20:15
robbiewbcsaller: cool...need 5min20:23
robbiewbcsaller: g+, phone, skype, mumble...pick yer poison :)20:27
bcsallerrobbiew: sent g+ invite20:28
robbiewbcsaller: cool...one sec20:28
_mup_juju/ftests r7 committed by gustavo@niemeyer.net21:05
_mup_Move environments.yaml creation into the ec2 suite, and21:05
_mup_make it use trunk rather than packages, so that the code21:05
_mup_has better chances of being in sync. Hopefully this fix21:05
_mup_the current breakage.21:05
_mup_juju/trunk r370 committed by gustavo@niemeyer.net21:12
_mup_Rename readme.txt to README since we already have one of those. [trivial]21:12
_mup_But really, this is just bumping the revno for an ftest cycle. :-)21:12
* hazmat catches up21:16
niemeyerHah.. missed the URLs21:19
niemeyerhazmat: YO!21:19
hazmatniemeyer, greetings hows your conference?21:20
hazmatSpamapS, which merge proposal?21:20
niemeyerhazmat: Awesome! Delighted to see everyone here, even though has been a bit tough to be in and out, and still need to do my talk for tomorrow21:20
niemeyerhazmat: Fixing the ftests now..21:21
niemeyerhazmat: Hey, I had made a comment on bcsaller's review to invite you for one last look on lxc-omega before the merge since there were a few things since your last review21:21
niemeyerhazmat: But tests were broken in trunk, so we decided to move on with it21:21
hazmatniemeyer, yeah.. saw that.. i'll have a last look at the commit21:22
niemeyerhazmat: If possible, would you mind to do a post-review on this once you have some spare time?21:22
niemeyerhazmat: Cheers!21:22
_mup_juju/ftests r8 committed by gustavo@niemeyer.net21:24
_mup_Forgot to update the ec2 tests to use local: urls (!).21:24
=== robbiew is now known as robbiew-afk
hazmatSpamapS, +1 on the terminate fix21:29
SpamapShazmat: cool thanks!21:35
jamespageSpamapS: zookeeper -> upstart transition - next release right?21:36
jamespageor do we want that now?21:36
SpamapSjamespage: yes next release :)21:36
jamespagecoolio21:36
jamespageshould be pretty easy21:36
SpamapSjamespage: agreed21:36
SpamapSjamespage: we should be able to do it in Debian so we don't have a permanent delta21:36
jamespagewe should be able to - I need todo the tomcat's as well in the same way21:37
hazmat3.4 release of zk is going to require some packaging love for next release, the build system is a bit different afaicr21:37
* jamespage sighs21:38
jamespagebetter take a look a trunk sooner rather than later then...21:38
SpamapSindeed, would be good to get that going into experimental soon21:39
jamespageyeah - that had crossed my mind21:42
jamespagemeh - looks much the same to me21:43
jamespageI'll have a go at an new upstream release for experimental some time in the next few weeks21:43
SpamapSwell anyway, time to run off to the urgent care to take care of this pinkeye.. grubby filthy things those kids are.21:43
jamespagehehe21:43
hazmatbcsaller, i see one bug in the omega merge to trunk21:59
bcsallerhazmat: whats that22:00
hazmatbcsaller, on the merge proposal22:00
hazmatbcsaller, its referencing self.master_template.. which is undefined, so it won't work if you don't already have one22:00
hazmathere's my diff to fix that as well as other style issues, http://paste.ubuntu.com/699361/22:01
bcsallerI'll fix it, thanks, well spotted22:01
bcsallerhazmat: I'll go ahead and merge that as a trivial then I guess22:05
_mup_juju/ftests r9 committed by gustavo@niemeyer.net22:17
_mup_Show what the wordpress title seems to be.22:17
_mup_juju/ftests r10 committed by gustavo@niemeyer.net23:00
_mup_Fixed title extraction.23:00
hazmatniemeyer, jimbaker, bcsaller can i get a +1 on this trivial http://paste.ubuntu.com/699361/23:08
hazmatactuall23:08
hazmatnevermind23:08
hazmati'll incorporate into my next branch23:08
bcsallerhazmat: I think that was already merged as a trivial23:09
hazmatbcsaller, ah.. cool.. wasn't sure since there wasn't any mup noise23:10
hazmatit would be nice if mup could directly monitor the repo23:10
hazmatniemeyer, thanks for the email btw23:20
niemeyerhazmat: No problem23:20
hazmatniemeyer, was talking to some of the heroku guys on how they do multi-tenant with lxc last night, they basically dictate to the process which port to listen on23:21
hazmatniemeyer, this conference is pretty awesome btw for ops stuff,  we should have some more people here23:22
hazmathopefully they'll get videos up in timely fashion this year23:22
niemeyerhazmat: Interesting23:23
_mup_juju/env-origin r379 committed by jim.baker@canonical.com23:42
_mup_Properly parse version table of apt-cache policy23:42
_mup_juju/env-origin r380 committed by jim.baker@canonical.com23:47
_mup_Merged trunk23:47

Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!