[00:11] RoAkSoAx: you've been quiet.. I trust things are going well? [00:14] <_mup_> ensemble/security-connection r282 committed by kapil.thangavelu@canonical.com [00:14] <_mup_> additional tests for security policy connection integration. [00:16] <_mup_> Bug #813856 was filed: Security policies need to be integrated into zookeeper connections < https://launchpad.net/bugs/813856 > [00:18] <_mup_> ensemble/security-connection r283 committed by kapil.thangavelu@canonical.com [00:18] <_mup_> mixin the security policy connection integration into the existing ssh zk client. [00:21] any reason why relation-changed would fire two or more times in a row? is a relation-changed fired for every relation-set on the other side of the relationship? [00:26] adam_g, relation changed gets fired after relation-joined as well [00:26] hazmat: in all cases, on both ends? [00:27] adam_g, yes.. it also gets fired when a hook does a relation-set (although only once for the hook, regardless of how many times it does relation-set in the hook) [00:29] hmm. ok [00:29] we used not to do after joined automatically, but noticed some of the formulas where getting their behavior wrong, as they where waiting on values in relation-changed.. but those values may have already been set by the remote end by the time the relation-joined hook fired locally so the relation-changed wouldn't have ever fired... ideally the hooks should have checked for the values in relation-joined and we could have avoided the spurious relation- [00:29] change, but we figured it would be easier for formula authors this way (fire the relation-changed after a relation-join) [00:30] re relation-set multiple times within a hook, the sets are all buffered to after the hook is executed so there's a single change event for all the related units. [00:38] hallyn, did you really just say "jaunty" ? [01:06] SpampS yeah everything is going well... changes made by william william have improved some things and some others have tp be rewritt [01:06] argh from cell phonr [01:41] smoser: I did indeed [09:13] does anyone recall the details of the test failure in which "" != "" [09:13] I seem to recall it's something to do with the packages we're using === daker_ is now known as daker === _mup__ is now known as _mup_ [09:54] fwereade, it needs an update of txzookeeper [09:54] fwereade, the __repr__ of client event changed [09:54] hazmat: I appear to be up to date [09:55] fwereade, you should check if that's the version/checkout being used [09:55] hazmat: I have my PYTHONPATH set; is there anything else I should do? [09:56] fwereade, you can startup python, import txzookeeper, print txzookeeper it will show the on disk location, verify that version is the one you think and is the latest [09:57] sometimes if your mix packages, or do setup.py install you'll end up with a different copy on your system [09:58] fwereade, you installed txzookeeper from? [09:58] ppa or bzr? [09:58] hazmat: ppa [09:58] should it be in /usr/lib/pymodules/python2.7? [09:58] or does that indicate I have the wrong one again somehow? [09:59] fwereade, what version of the package do you have? [10:00] hazmat: 0.8.0 [10:00] hazmat: is there somewhere I can check what version I *should* have without asking people? [10:02] fwereade, the bzr revision number should be in the package versionthere.. so as far being able to determine this by yourself, the package version bzr revision should match that of txzookeeper trunk (rev 43) [10:02] hazmat: hm, good to know [10:03] hazmat: so my life would probably be easier if I just kept a checkout of txzookeeper and installed that whenever it changed? [10:03] hazmat: or should I actually be watching trunk for many things? [10:04] fwereade, txzookeeper is pretty stable, maybe one change every couple months at this point. i use virtualenvs and run it with python setup.py develop which links the source tree into the python package dir, so updating the source tree is sufficient. [10:05] fwereade, can you confirm the exact package version ? [10:05] s/exact/full [10:06] hazmat: 0.8.0 is what I get from txzookeeper.version [10:06] fwereade, apt-cache show txzookeeper will show the package version [10:07] * fwereade raises eyebrow ("no packages found") [10:08] fwereade, so that would imply it wasn't installed by ppa [10:09] hazmat: hmm [10:09] hazmat: I just re-added the repository and updated again [10:09] or maybe its python-txzookeeper [10:09] hazmat: ooh, that rings a bell [10:09] doesn't look like it from here.. https://launchpad.net/~ensemble/+archive/ppa?field.series_filter=natty but it sounds sensible ;-) [10:09] hazmat: 0.8.0-0ensemble43~natty1 [10:10] hmm.. that's the latest [10:10] hazmat: which looks right based on what you've been saying [10:10] fwereade, which test is it that's failing [10:15] hazmat: sorry, bad time for system to fall over [10:15] hazmat: let me recover my state... [10:15] fwereade, no worries [10:15] hazmat: rerunning suite now, but you can find it by grepping for ClientEvent [10:17] ensemble.state.tests.test_relation.???.test_watch_user_callback_invocation_delays_node_watch [10:18] UnitRelationStateTest [10:18] hazmat: trunk works for you then? [10:23] fwereade, yeah.. works fine for me [10:23] hazmat: bah :) [10:25] hazmat: well, not to worry for now, I'll poke around and see if I can figure something out [10:25] hazmat: thanks for your help [10:26] fwereade, yeah.. i'm not sure what the issue is.. i'm firing up an ensemble instance to verify the unit tests there [11:28] fwereade, just verified the failure using txzk package on ec2 [11:30] hmm [11:32] it looks like i'm pushing and pulling from different branches [11:32] they have the same content though [11:51] hey i am blocked at preseed conf file for my formula, can anyone explain how it works ? [12:13] <_mup_> ensemble/trunk-merge r271 committed by kapil.thangavelu@canonical.com [12:13] <_mup_> merge trunk [13:03] hazmat: hey, soory I missed you [13:04] hazmat: yeah, I verified that from my perspective it really is the tests that are wrong (all the txzookeepers I can find repr with state: connected), and thought I'd wait to hear back [13:13] fwereade, it looks like that test is broken [13:13] fwereade, the repr of the client event now has the connection status [13:13] the test needs an update to reflect that [13:13] hazmat: cool (well, not cool, but at least it's not my own mistake :p) [13:14] hazmat: I already have a local fix, shall I create a bug and propose a merge? [13:14] hazmat: or should we complain to whoever last merged? [13:14] fwereade, its a trivial, typically just paste the diff, get an ack, and commit to trunk [13:15] hazmat: http://paste.ubuntu.com/649098/ look ok to you? [13:16] hmm.. looks like ben merged the failed test in last rev [13:16] fwereade, looks good +1 [13:16] hazmat: cheers [13:16] hazmat: just rerunning the tests for paranoia's sake [13:18] it looks like some other test logic got removed in that merge [13:18] unrelated to this fix [13:47] fwereade: o/ I have a few issues to show you in just a second [13:47] RoAkSoAx: cool, just shout when you need me [13:49] fwereade: will do in just a few secs [13:49] or minutes :) [13:51] fwereade: can I start using trunk to test the orchestra integration instead of the bootstrap branch? [13:51] fwereade: as all your change shave already been merged tright? [13:51] and I think I might use of a couple of bugfixes [13:52] RoAkSoAx: trunk has the big refactoring [13:53] RoAkSoAx: there's some stuff in lp:~fwereade/ensemble/cobbler-connect-production that might be of use to you [13:53] fwereade: so is it safe to use my work with that branch? [13:53] RoAkSoAx: it's a hopefully-useful CobblerConnect implemented with twisted [13:54] RoAkSoAx: and it should be just enough to maunch a machine, without passing anything useful to it whatsoever [13:54] fwereade: right now im getting erros that doesn't seem related to the refactoring [13:54] that's why I'm asking [13:54] RoAkSoAx: I'm going to give it a final once-over and then propose a merge [13:55] RoAkSoAx: like what? [13:55] fwereade: unhasable dict [13:55] error [13:55] when trying to deploy [13:56] RoAkSoAx: sorry, doesn't ring a bell :( [13:56] RoAkSoAx: paste a traceback? [13:57] fwereade: yeah I'm gonna reproduce with a clean zookeeper machine [13:57] I'm deploying it atm [14:07] fwereade: now I can't even bootstrap [14:07] http://pastebin.ubuntu.com/649146/ [14:07] RoAkSoAx: is that on trunk? [14:08] fwereade: bot, ensemble-bootstrap [14:09] RoAkSoAx: what it (probably) means is that OrchestraLaunchMachine is returning a string (name?) instead of a deferred list of OrchestraMachines [14:10] eek [14:10] fwereade: yeah, and the thing is that from yesterday till today nothing changed and yesterday there was no error :) [14:12] RoAkSoAx: wait, I misread [14:13] RoAkSoAx: that's the result of get_zookeeper_machines [14:13] fwereade: heh never mind found the error. the same thing that's giving error when launching machines, is the thing that's giving error when bootstrapping if I comment out the first thing [14:14] fwereade: but yeah, I've bumped into stuff that seems to still be ec2 specific [14:14] let me re-check [14:14] RoAkSoAx: very probable... I did my best, but it's still all a bit new to me ;) what in particular [14:17] fwereade: yeah we are both learning our way here:) [14:17] fwereade, yo i have just pushed the changes https://code.launchpad.net/~daker/ensemble/small-fix/+merge/68038 [14:17] daker: cool, thanks, I'll take a look [14:22] daker: tiny tweaks :) [14:23] RoAkSoAx: I'm just firming up my tests a little, shout if I can help [14:24] fwereade, ah you mean that between the two sub_parser.add_argument there should a break, right ? [14:24] daker: sorry, no [14:25] daker: I just meant that the original source line was over 80 chars [14:25] and ? the help text should be reduced ? [14:26] daker: and so a line break, probably just before the "help=", would make the code nice without needing to touch the text [14:26] daker: the text looks fine to me [14:27] daker: if the source is still over 80 characters even with that break, you can just do something like the following (give me a moment...) [14:28] fwereade, is it good like that http://paste.ubuntu.com/649169/ ? [14:29] daker: that just breaks 80 chars, how about http://paste.ubuntu.com/649177/ [14:30] ah ok i understand now! [14:30] bcsaller, the merge of config-set-lifecycle looks like it had some testing regressions added [14:31] bcsaller, fwereade took care of the failing test, but it looks like (as noted in review) that some other tests also lost some of their logic [14:45] fwereade: what should __check_state from findzookeepers.py return? [14:47] RoAkSoAx: thinking [14:48] RoAkSoAx: it should *eventually* return a list of OrchestraMachines === jamespage1 is now known as jamespage [14:48] RoAkSoAx: (with just one element for now) [14:48] RoAkSoAx: ...ok, I put that badly [14:48] RoAkSoAx: it can return whatever it wants, but the callback should be able to turn *that* into a list of etc [14:49] RoAkSoAx: the callback chain should end up returning a list of machines [14:49] RoAkSoAx: am I making sense? [14:50] fwereade: i think so :) yes! [14:51] RoAkSoAx: cool :) [14:51] RoAkSoAx: as long as we get the expected results/errors from .run(), how you do it is up to you [15:04] jimbaker: hi [15:06] jimbaker: I haven't managed to induce any errors in EC2BootstrapTest, try as I might [15:06] jimbaker: any pointers? [15:06] jimbaker: however, I *have* found a different test that will fail pretty soon if run with -u [15:15] RoAkSoAx: you know the kickstart file on the cobbler server? [15:16] RoAkSoAx: that I had to make a little change to to get past the system install step? [15:17] RoAkSoAx: such a file must have a certain set of properties before we can actually deploy an ensemble machine running it, right? [15:18] RoAkSoAx: ...but those files are not actually part of ensemble [15:19] hazmat: all the tests on trunk are passing for me. I thought I did it properly, though I did have to clear .pyc files before it worked [15:19] hazmat: maybe you can point me at what you're seeing? [15:24] bcsaller: is your txzookeeper up to date? [15:24] fwereade, can you take a look now https://code.launchpad.net/~daker/ensemble/small-fix/+merge/68038 ? [15:24] bcsaller: if you search the source for ClientEvent you'll find the test [15:25] fwereade: ahh, the state stuff in the repr [15:25] bcsaller: yep [15:26] bcsaller: I know there was something funny -- I hit something similar on the sprint -- and niemeyer went off, discussed packaging with someone, and magically fixed it [15:26] bcsaller: ...but I don't know the details [15:27] bcsaller: regardless: the current version does have "state: connected" in the __str__, so I changed the test to expect it [15:27] fwereade: I updated my txzk, I'll give it another pass [15:27] bcsaller: that one's already fixed, I think hazmat was concerned about something else, but I didn't review so I don't know what it is [15:27] fwereade, which test is that? [15:28] bzr qdiff -c -1 on trunk will show the fix from fwereade, the fix that remains is for the tests that have lost logic.. as per bzr qdiff -c -2 [15:28] jimbaker: ensemble.state.tests.test_agent.AgentDomainTest.test_connect_agent [15:28] fwereade, yes that one fails on a regular basis [15:28] jimbaker: ah, cool :) [15:29] jimbaker: well, not cool :p [15:29] fwereade, just wonder why the bootstrap tests fail for me [15:29] most of these bits are missing from test_hook https://pastebin.canonical.com/50069/ [15:29] bcsaller, ^ [15:29] fwereade: yeah, didn't realize there was a txzk version issue there, its working on trunk now. [15:29] hazmat: those bits don't apply, we inlined and removed that api [15:30] jimbaker: how often are you seeing the bootstrap failure? [15:30] bcsaller, the flush logic at the top does [15:30] if i run -u, it will eventually fail [15:30] fwereade, also the tests now fail if you run them disconnected [15:30] seems like less than 10 iterations [15:31] the calls to determine the image id aren't being mocked [15:31] jimbaker: for that specific test method? [15:31] fwereade, when i ran just the bootstrap tests, for example [15:31] hazmat: I have noticed that, I didn't think it was me, but given that I was hitting that code most recently I guess it was [15:32] jimbaker: weord, I've tried with all variants from .providers on up [15:32] jimbaker: quick thought [15:32] fwereade, which is generally how i will isolate: run a section of tests w/ -u (test/test suite/script if possible) [15:33] jimbaker: is your machine under noticeably low or high load? [15:33] fwereade, low load [15:33] jimbaker: bah, so has mine been [15:33] fwereade, however i will retry now, on both my laptop and desktop [15:34] hazmat: shall I pick that issue up? [15:34] fwereade, curiously i was seeing it fail in the same place on my macbook pro yesterday [15:34] jimbaker: I'm sure it's a real problem, I just wish I could figure out how to repro it [15:34] fwereade, if you want/time allows, else just file an issue for it [15:34] fwereade, the cobbler work is higher priority [15:35] SpamapS, you had asked earlier about osx, the 2.7ism that apparently snuck in was around argparse [15:35] hazmat: true [15:35] hazmat: cheers [15:35] bbs, diconnecting to get the error list [15:35] SpamapS, this shouldn't impact actualy client usage however - it's just testing against specific error messages, and these got changed in 2.7 [15:36] probably need to change the tests [15:36] SpamapS: ping [15:36] the actual mac failures were all about different temp file paths [15:37] and also uninteresting [15:40] <_mup_> Bug #814144 was filed: test errors when no network available < https://launchpad.net/bugs/814144 > [16:06] fwereade, given the network dependency in bug 814144 on the same tests mentioned earlier as having looping problems (./test -u ensemble.providers.ec2.tests.test_bootstrap), maybe this theory explains things [16:06] <_mup_> Bug #814144: test errors when no network available < https://launchpad.net/bugs/814144 > [16:07] 1. yesterday i was running the tests at a coffee shop w/ a so-so network, and they were eventually failing, maybe because of additional timing [16:07] 2. today, i'm running them at home w/ a good low-latency network, no failures [16:11] jimbaker: ah-ha! [16:11] jimbaker: very plausible [16:12] jimbaker: consider me on it [16:12] fwereade, sounds good [16:14] fwereade, is my MP good now ? [16:14] daker: so sorry, I think I lost you in a context switch [16:15] no worries https://code.launchpad.net/~daker/ensemble/small-fix/+merge/68038 [16:15] daker: would be good to lose the spaces before the colons, sorry I notied them late [16:16] if confirmed..., if value.strip()... [16:16] but lgtm otherwise [16:16] shang, we met in budapest, right? [16:18] fwereade, i am lost :/ [16:24] daker: sorry again [16:25] if confirmed is False : [16:25] should read [16:25] if confirmed is False : [16:25] if confirmed is False: [16:25] * fwereade winces [16:25] ...and there's another one a couple of lines later [16:26] it's just for consistency's sake ;) [16:29] i'm pretty certain anything like that would be picked up by the pep8 tool [16:29] i recommend always running pep8/pyflakes (reviewers certainly do) [16:32] also, in this particular case, best to say something like: if confirmed: ... [16:33] no need to say is False, certainly the argparse setting ensures it's True or False, and if even if it didn't, i don't see that it would matter if it were 0 or None [16:51] RoAkSoAx: pong, sup? [16:54] SpamapS: howdy!! I was working on the load_state stuff, and doing that, http://pastebin.ubuntu.com/649258/, returns something like: exceptions.AttributeError: 'str' object has no attribute 'get' [16:54] any ideas? [17:03] RoAkSoAx: reading now [17:04] RoAkSoAx, the state being returned by load_state is a string it appears, does the error happen if you remove the bottom line [17:04] RoAkSoAx, you also can't do this interactively since there are deferreds in the mix [17:05] fwereade, sorry again, can you give a final look https://code.launchpad.net/~daker/ensemble/small-fix/+merge/68038 ? [17:05] Its possible the state that was saved wasn't properly serialized [17:05] the reactor needs to be started and the result processed either with a callback or with @inlineCallbacks and yield [17:05] hazmat: yeah I'm working on that ;) [17:08] daker: looks great to me, my approve stands :) [17:10] daker, just added a review comment to it [17:18] hazmat, i hope it's good now ツ [17:19] * daker crosse his fingers [17:28] daker, looks good, except for the missing test [17:30] hazmat: golly, we even have tests for the command line, I didn't even think to demand them... [17:30] hazmat: I think the laxity at my last place has damaged my brain :/ [17:30] hazmat: I used to test *everything* [17:31] hazmat: ...I'm so happy to be here :D [17:31] fwereade, in this case the cli is our primary interface, if we're not testing it.... then bad things ;-) [17:31] hazmat: quite... I just got used to being happy when tests for *anything* showed up ;) [17:32] hazmat: ok, that's not fair, but we didn't test anything like as much as we did at the place before [17:33] anyway, I think I'm degenerating into friday evening rambling, and I should be off [17:33] bye all :) [17:33] Friday? [17:33] lol [17:33] * robbiew notes that fwereade is on holiday tomorrow [17:34] <_mup_> Bug #814194 was filed: Implement support for expose and unexpose hooks < https://launchpad.net/bugs/814194 > [17:34] daker, here's a test for it btw, if you could add this to your branch in ensemble/control/tests/test_shutdown the branch should be good.. https://pastebin.canonical.com/50078/ [17:35] oh good thanls hazmat [17:36] daker, np, thanks for working on it [17:36] hazmat, would be good if you can put it on paste.u.c, i don't have permission access that site [17:37] to* [17:37] daker, http://pastebin.ubuntu.com/649319/ [17:39] hazmat, one last question, how can i run the test ? [17:40] daker, the easiest option assuming you have the zookeeper package installed (and the zk server running) is just trial ensemble/control/tests/test_shutdown.py [17:40] ok [17:48] any thoughts http://paste.ubuntu.com/649326/ ? [17:49] hazmat, ^ [17:49] PYTHONPATH=/home/daker/Projects/small-fix/ trial ensemble/control/tests/test_shutdown.py [17:49] daker, ^ [17:49] trial is from the twisted pkg [17:50] hazmat, is the test includes the -y argument ? [17:50] daker, the test i pasted does [17:51] ok hazmat the test succeed [17:51] daker, the error is because the test is being run directly, which A) isn't going to work as the test runner won't be invoked b) relative imports need a fully qualified reference to work correctly when the script is being run this way the current module doesn't really know its package [17:51] daker, cool, i tested before i pasted, but its good to have confirmation ;-) [17:57] hazmat, done! https://code.launchpad.net/~daker/ensemble/small-fix/+merge/68038 [18:03] Yo all [18:04] niemeyer, hi [18:07] <_mup_> ensemble/security-groups r285 committed by kapil.thangavelu@canonical.com [18:07] <_mup_> security groups === daker is now known as daker_ [19:22] <_mup_> ensemble/security-groups r286 committed by kapil.thangavelu@canonical.com [19:22] <_mup_> group principal membership management and connection attaching. [20:02] <_mup_> Bug #814260 was filed: Ensemble security group principals < https://launchpad.net/bugs/814260 > [20:12] <_mup_> ensemble/security-groups r287 committed by kapil.thangavelu@canonical.com [20:12] <_mup_> add get_token to groupprincipal for parity with normal principal [20:16] <_mup_> ensemble/trunk-merge r272 committed by kapil.thangavelu@canonical.com [20:16] <_mup_> merge trunk [21:06] <_mup_> ensemble/security-otp-principal r289 committed by kapil.thangavelu@canonical.com [21:06] <_mup_> otp principal for passing credentials via insecure channels [21:18] <_mup_> ensemble/robust-hook-exit r281 committed by jim.baker@canonical.com [21:18] <_mup_> Use processExited as the normal end of the process, while making processEnded available for purposes like testing [21:28] <_mup_> ensemble/robust-hook-exit r282 committed by jim.baker@canonical.com [21:28] <_mup_> PEP8 [21:40] <_mup_> ensemble/robust-hook-exit r283 committed by jim.baker@canonical.com [21:40] <_mup_> Doc string [22:02] <_mup_> ensemble/security-otp-principal r290 committed by kapil.thangavelu@canonical.com [22:02] <_mup_> minimal principal interface for otp, using the otp consumes it [22:07] <_mup_> Bug #814320 was filed: Need OTP for passing credentials to an external process < https://launchpad.net/bugs/814320 > [22:09] phew.. i think thats all the core security machinery needed [22:09] sigh.. "Launchpad encountered an internal error during the following operation: generating the diff for a merge proposal. It was logged with id OOPS-2028MPJ3. Sorry for the inconvenience. [22:09] " [22:09] oh well