/srv/irclogs.ubuntu.com/2011/07/21/#ubuntu-ensemble.txt

SpamapSRoAkSoAx: you've been quiet.. I trust things are going well?00:11
_mup_ensemble/security-connection r282 committed by kapil.thangavelu@canonical.com00:14
_mup_additional tests for security policy connection integration.00:14
_mup_Bug #813856 was filed: Security policies need to be integrated into zookeeper connections <Ensemble:In Progress by hazmat> < https://launchpad.net/bugs/813856 >00:16
_mup_ensemble/security-connection r283 committed by kapil.thangavelu@canonical.com00:18
_mup_mixin the security policy connection integration into the existing ssh zk client.00:18
adam_gany reason why relation-changed would fire two or more times in a row? is a relation-changed fired for every relation-set on the other side of the relationship?00:21
hazmatadam_g, relation changed gets fired after relation-joined as well00:26
adam_ghazmat: in all cases, on both ends? 00:26
hazmatadam_g, yes.. it also gets fired when a hook does a relation-set (although only once for the hook, regardless of how many times it does relation-set in the hook)00:27
adam_ghmm. ok00:29
hazmatwe used not to do after joined automatically, but noticed some of the formulas where getting their behavior wrong, as they where waiting on values in relation-changed.. but those values may have already been set by the remote end by the time the relation-joined hook fired locally so the relation-changed wouldn't have ever fired... ideally the hooks should have checked for the values in relation-joined and we could have avoided the spurious relation-00:29
hazmatchange, but we figured it would be easier for formula authors this way (fire the relation-changed after a relation-join)00:29
hazmatre relation-set multiple times within a hook, the sets are all buffered to after the hook is executed so there's a single change event for all the related units.00:30
smoserhallyn, did you really just say "jaunty" ?00:38
RoAkSoAxSpampS yeah everything is going well...  changes made by william william have improved some things and some others have tp be rewritt01:06
RoAkSoAxargh from cell phonr 01:06
hallynsmoser: I did indeed01:41
fwereadedoes anyone recall the details of the test failure in which "<ClientEvent changed at blah state:connected>" != "<ClientEvent changed at blah>"09:13
fwereadeI seem to recall it's something to do with the packages we're using09:13
=== daker_ is now known as daker
=== _mup__ is now known as _mup_
hazmatfwereade, it needs an update of txzookeeper09:54
hazmatfwereade, the __repr__ of client event changed09:54
fwereadehazmat: I appear to be up to date09:54
hazmatfwereade, you should check if that's the version/checkout being used09:55
fwereadehazmat: I have my PYTHONPATH set; is there anything else I should do?09:55
hazmatfwereade, you can startup python, import txzookeeper, print txzookeeper it will show the on disk location, verify that version is the one you think and is the latest09:56
hazmatsometimes if your mix packages, or do setup.py install you'll end up with a different copy on your system09:57
hazmatfwereade, you installed txzookeeper from?09:58
hazmatppa or bzr?09:58
fwereadehazmat: ppa09:58
fwereadeshould it be in /usr/lib/pymodules/python2.7?09:58
fwereadeor does that indicate I have the wrong one again somehow?09:58
hazmatfwereade, what version of the package do you have?09:59
fwereadehazmat: 0.8.010:00
fwereadehazmat: is there somewhere I can check what version I *should* have without asking people?10:00
hazmatfwereade, the bzr revision number should be in  the package versionthere.. so as far being able to determine this by yourself, the package version bzr revision should match that of txzookeeper trunk (rev 43)10:02
fwereadehazmat: hm, good to know10:02
fwereadehazmat: so my life would probably be easier if I just kept a checkout of txzookeeper and installed that whenever it changed?10:03
fwereadehazmat: or should I actually be watching trunk for many things?10:03
hazmatfwereade, txzookeeper is pretty stable, maybe one change every couple months at this point. i use virtualenvs and run it with python setup.py develop which links the source tree into the python package dir, so updating the source tree is sufficient.10:04
hazmatfwereade, can you confirm the exact package version ?10:05
hazmats/exact/full10:05
fwereadehazmat: 0.8.0 is what I get from txzookeeper.version10:06
hazmatfwereade, apt-cache show txzookeeper will show the package version10:06
* fwereade raises eyebrow ("no packages found")10:07
hazmatfwereade, so that would imply it wasn't installed by ppa10:08
fwereadehazmat: hmm10:09
fwereadehazmat: I just re-added the repository and updated again10:09
hazmator maybe its python-txzookeeper10:09
fwereadehazmat: ooh, that rings a bell10:09
hazmatdoesn't look like it from here.. https://launchpad.net/~ensemble/+archive/ppa?field.series_filter=natty but it sounds sensible ;-)10:09
fwereadehazmat: 0.8.0-0ensemble43~natty110:09
hazmathmm.. that's the latest10:10
fwereadehazmat: which looks right based on what you've been saying10:10
hazmatfwereade, which test is it that's failing10:10
fwereadehazmat: sorry, bad time for system to fall over10:15
fwereadehazmat: let me recover my state...10:15
hazmatfwereade, no worries10:15
fwereadehazmat: rerunning suite now, but you can find it by grepping for ClientEvent10:15
fwereadeensemble.state.tests.test_relation.???.test_watch_user_callback_invocation_delays_node_watch10:17
fwereadeUnitRelationStateTest10:18
fwereadehazmat: trunk works for you then?10:18
hazmatfwereade, yeah.. works fine for me10:23
fwereadehazmat: bah :)10:23
fwereadehazmat: well, not to worry for now, I'll poke around and see if I can figure something out10:25
fwereadehazmat: thanks for your help10:25
hazmatfwereade, yeah.. i'm not sure what the issue is.. i'm firing up an ensemble instance to verify the unit tests there10:26
hazmatfwereade, just verified the failure using txzk package on ec211:28
hazmathmm11:30
hazmatit looks like i'm pushing and pulling from different branches11:32
hazmatthey have the same content though11:32
dakerhey i am blocked at preseed conf file for my formula, can anyone explain how it works ?11:51
_mup_ensemble/trunk-merge r271 committed by kapil.thangavelu@canonical.com12:13
_mup_merge trunk12:13
fwereadehazmat: hey, soory I missed you13:03
fwereadehazmat: yeah, I verified that from my perspective it really is the tests that are wrong (all the txzookeepers I can find repr with state: connected), and thought I'd wait to hear back13:04
hazmatfwereade, it looks like that test is broken13:13
hazmatfwereade, the repr of the client event now has the connection status13:13
hazmatthe test needs an update to reflect that13:13
fwereadehazmat: cool (well, not cool, but at least it's not my own mistake :p)13:13
fwereadehazmat: I already have a local fix, shall I create a bug and propose a merge?13:14
fwereadehazmat: or should we complain to whoever last merged?13:14
hazmatfwereade, its a trivial, typically just paste the diff, get an ack, and commit to trunk13:14
fwereadehazmat: http://paste.ubuntu.com/649098/ look ok to you?13:15
hazmathmm.. looks like ben merged the failed test in last rev13:16
hazmatfwereade, looks good +113:16
fwereadehazmat: cheers13:16
fwereadehazmat: just rerunning the tests for paranoia's sake13:16
hazmatit looks like some other test logic got removed in that merge13:18
hazmatunrelated to this fix13:18
RoAkSoAxfwereade: o/ I have a few issues to show you in just a second13:47
fwereadeRoAkSoAx: cool, just shout when you need me13:47
RoAkSoAxfwereade: will do in just a few secs13:49
RoAkSoAxor minutes :)13:49
RoAkSoAxfwereade: can I start using trunk to test the orchestra integration instead of the bootstrap branch?13:51
RoAkSoAxfwereade: as all your change shave already been merged tright?13:51
RoAkSoAxand I think I might use of a couple of bugfixes13:51
fwereadeRoAkSoAx: trunk has the big refactoring13:52
fwereadeRoAkSoAx: there's some stuff in lp:~fwereade/ensemble/cobbler-connect-production that might be of use to you13:53
RoAkSoAxfwereade: so is it safe to use my work with that branch?13:53
fwereadeRoAkSoAx: it's a hopefully-useful CobblerConnect implemented with twisted13:53
fwereadeRoAkSoAx: and it should be just enough to maunch a machine, without passing anything useful to it whatsoever13:54
RoAkSoAxfwereade: right now im getting erros that doesn't seem related to the refactoring13:54
RoAkSoAxthat's why I'm asking13:54
fwereadeRoAkSoAx: I'm going to give it a final once-over and then propose a merge13:54
fwereadeRoAkSoAx: like what?13:55
RoAkSoAxfwereade: unhasable dict13:55
RoAkSoAxerror13:55
RoAkSoAxwhen trying to deploy 13:55
fwereadeRoAkSoAx: sorry, doesn't ring a bell :(13:56
fwereadeRoAkSoAx: paste a traceback?13:56
RoAkSoAxfwereade: yeah I'm gonna reproduce with a clean zookeeper machine13:57
RoAkSoAxI'm deploying it atm13:57
RoAkSoAxfwereade: now I can't even bootstrap14:07
RoAkSoAxhttp://pastebin.ubuntu.com/649146/14:07
fwereadeRoAkSoAx: is that on trunk?14:07
RoAkSoAxfwereade: bot, ensemble-bootstrap14:08
fwereadeRoAkSoAx: what it (probably) means is that OrchestraLaunchMachine is returning a string (name?) instead of a deferred list of OrchestraMachines14:09
Davieyeek14:10
RoAkSoAxfwereade: yeah, and the thing is that from yesterday till today nothing changed and yesterday there was no error :)14:10
fwereadeRoAkSoAx: wait, I misread14:12
fwereadeRoAkSoAx: that's the result of get_zookeeper_machines14:13
RoAkSoAxfwereade: heh never mind found the error. the same thing that's giving error when launching machines, is the thing that's giving error when bootstrapping if I comment out the first thing14:13
RoAkSoAxfwereade: but yeah, I've bumped into stuff that seems to still be ec2 specific14:14
RoAkSoAxlet me re-check14:14
fwereadeRoAkSoAx: very probable... I did my best, but it's still all a bit new to me ;) what in particular14:14
RoAkSoAxfwereade: yeah we are both learning our way here:)14:17
dakerfwereade, yo i have just pushed the changes https://code.launchpad.net/~daker/ensemble/small-fix/+merge/6803814:17
fwereadedaker: cool, thanks, I'll take a look14:17
fwereadedaker: tiny tweaks :)14:22
fwereadeRoAkSoAx: I'm just firming up my tests a little, shout if I can help14:23
dakerfwereade, ah you mean that between the two sub_parser.add_argument there should a break, right ?14:24
fwereadedaker: sorry, no14:24
fwereadedaker: I just meant that the original source line was over 80 chars14:25
dakerand ? the help text should be reduced ?14:25
fwereadedaker: and so a line break, probably just before the "help=", would make the code nice without needing to touch the text14:26
fwereadedaker: the text looks fine to me14:26
fwereadedaker: if the source is still over 80 characters even with that break, you can just do something like the following (give me a moment...)14:27
dakerfwereade, is it good like that http://paste.ubuntu.com/649169/ ?14:28
fwereadedaker: that just breaks 80 chars, how about http://paste.ubuntu.com/649177/14:29
dakerah ok i understand now!14:30
hazmatbcsaller, the merge of config-set-lifecycle looks like it had some testing regressions added14:30
hazmatbcsaller, fwereade took care of the failing test, but it looks like (as noted in review) that some other tests also lost some of their logic14:31
RoAkSoAxfwereade: what should __check_state from findzookeepers.py return?14:45
fwereadeRoAkSoAx: thinking14:47
fwereadeRoAkSoAx: it should *eventually* return a list of OrchestraMachines14:48
=== jamespage1 is now known as jamespage
fwereadeRoAkSoAx: (with just one element for now)14:48
fwereadeRoAkSoAx: ...ok, I put that badly14:48
fwereadeRoAkSoAx: it can return whatever it wants, but the callback should be able to turn *that* into a list of etc14:48
fwereadeRoAkSoAx: the callback chain should end up returning a list of machines14:49
fwereadeRoAkSoAx: am I making sense?14:49
RoAkSoAxfwereade: i think so :) yes!14:50
fwereadeRoAkSoAx: cool :)14:51
fwereadeRoAkSoAx: as long as we get the expected results/errors from .run(), how you do it is up to you14:51
fwereadejimbaker: hi15:04
fwereadejimbaker: I haven't managed to induce any errors in EC2BootstrapTest, try as I might15:06
fwereadejimbaker: any pointers?15:06
fwereadejimbaker: however, I *have* found a different test that will fail pretty soon if run with -u15:06
fwereadeRoAkSoAx: you know the kickstart file on the cobbler server?15:15
fwereadeRoAkSoAx: that I had to make a little change to to get past the system install step?15:16
fwereadeRoAkSoAx: such a file must have a certain set of properties before we can actually deploy an ensemble machine running it, right?15:17
fwereadeRoAkSoAx: ...but those files are not actually part of ensemble15:18
bcsallerhazmat: all the tests on trunk are passing for me. I thought I did it properly, though I did have to clear .pyc files before it worked15:19
bcsallerhazmat: maybe you can point me at what you're seeing?15:19
fwereadebcsaller: is your txzookeeper up to date?15:24
dakerfwereade, can you take a look now https://code.launchpad.net/~daker/ensemble/small-fix/+merge/68038 ?15:24
fwereadebcsaller: if you search the source for ClientEvent you'll find the test15:24
bcsallerfwereade: ahh, the state stuff in the repr15:25
fwereadebcsaller: yep15:25
fwereadebcsaller: I know there was something funny -- I hit something similar on the sprint -- and niemeyer went off, discussed packaging with someone, and magically fixed it15:26
fwereadebcsaller: ...but I don't know the details15:26
fwereadebcsaller: regardless: the current version does have "state: connected" in the __str__, so I changed the test to expect it15:27
bcsallerfwereade: I updated my txzk, I'll give it another pass15:27
fwereadebcsaller: that one's already fixed, I think hazmat was concerned about something else, but I didn't review so I don't know what it is15:27
jimbakerfwereade, which test is that?15:27
hazmatbzr qdiff -c -1 on trunk will show the fix from fwereade, the fix that remains is for the tests that have lost logic.. as per bzr qdiff -c -215:28
fwereadejimbaker: ensemble.state.tests.test_agent.AgentDomainTest.test_connect_agent15:28
jimbakerfwereade, yes that one fails on a regular basis15:28
fwereadejimbaker: ah, cool :)15:28
fwereadejimbaker: well, not cool :p15:29
jimbakerfwereade, just wonder why the bootstrap tests fail for me15:29
hazmatmost of these bits are missing from test_hook https://pastebin.canonical.com/50069/15:29
hazmatbcsaller, ^15:29
bcsallerfwereade: yeah, didn't realize there was a txzk version issue there, its working on trunk now. 15:29
bcsallerhazmat: those bits don't apply, we inlined and removed that api15:29
fwereadejimbaker: how often are you seeing the bootstrap failure?15:30
hazmatbcsaller, the flush logic at the top does15:30
jimbakerif i run -u, it will eventually fail15:30
hazmatfwereade, also the tests now fail if you run them disconnected15:30
jimbakerseems like less than 10 iterations15:30
hazmatthe calls to determine the image id aren't being mocked15:31
fwereadejimbaker: for that specific test method?15:31
jimbakerfwereade, when i ran just the bootstrap tests, for example15:31
fwereadehazmat: I have noticed that, I didn't think it was me, but given that I was hitting that code most recently I guess it was15:31
fwereadejimbaker: weord, I've tried with all variants from .providers on up15:32
fwereadejimbaker: quick thought15:32
jimbakerfwereade, which is generally how i will isolate: run a section of tests w/ -u (test/test suite/script if possible)15:32
fwereadejimbaker: is your machine under noticeably low or high load?15:33
jimbakerfwereade, low load15:33
fwereadejimbaker: bah, so has mine been15:33
jimbakerfwereade, however i will retry now, on both my laptop and desktop15:33
fwereadehazmat: shall I pick that issue up?15:34
jimbakerfwereade, curiously i was seeing it fail in the same place on my macbook pro yesterday15:34
fwereadejimbaker: I'm sure it's a real problem, I just wish I could figure out how to repro it15:34
hazmatfwereade, if you want/time allows, else just file an issue for it15:34
hazmatfwereade, the cobbler work is higher priority15:34
jimbakerSpamapS, you had asked earlier about osx, the 2.7ism that apparently snuck in was around argparse15:35
fwereadehazmat: true15:35
fwereadehazmat: cheers15:35
fwereadebbs, diconnecting to get the error list15:35
jimbakerSpamapS, this shouldn't impact actualy client usage however - it's just testing against specific error messages, and these got changed in 2.715:35
jimbakerprobably need to change the tests15:36
RoAkSoAxSpamapS: ping15:36
jimbakerthe actual mac failures were all about different temp file paths15:36
jimbakerand also uninteresting15:37
_mup_Bug #814144 was filed: test errors when no network available <Ensemble:New> < https://launchpad.net/bugs/814144 >15:40
jimbakerfwereade, given the network dependency in bug 814144 on the same tests mentioned earlier as having looping problems (./test -u ensemble.providers.ec2.tests.test_bootstrap), maybe this theory explains things16:06
_mup_Bug #814144: test errors when no network available <Ensemble:New> < https://launchpad.net/bugs/814144 >16:06
jimbaker1. yesterday i was running the tests at a coffee shop w/ a so-so network, and they were eventually failing, maybe because of additional timing16:07
jimbaker2. today, i'm running them at home w/ a good low-latency network, no failures16:07
fwereadejimbaker: ah-ha!16:11
fwereadejimbaker: very plausible16:11
fwereadejimbaker: consider me on it16:12
jimbakerfwereade, sounds good16:12
dakerfwereade, is my MP good now ?16:14
fwereadedaker: so sorry, I think I lost you in a context switch16:14
dakerno worries https://code.launchpad.net/~daker/ensemble/small-fix/+merge/6803816:15
fwereadedaker: would be good to lose the spaces before the colons, sorry I notied them late16:15
fwereadeif confirmed..., if value.strip()...16:16
fwereadebut lgtm otherwise16:16
jimbakershang, we met in budapest, right?16:16
dakerfwereade, i am lost :/16:18
fwereadedaker: sorry again16:24
fwereadeif confirmed is False :16:25
fwereadeshould read16:25
fwereadeif confirmed is False :16:25
fwereadeif confirmed is False:16:25
* fwereade winces16:25
fwereade...and there's another one a couple of lines later16:25
fwereadeit's just for consistency's sake ;)16:26
jimbakeri'm pretty certain anything like that would be picked up by the pep8 tool16:29
jimbakeri recommend always running pep8/pyflakes (reviewers certainly do)16:29
jimbakeralso, in this particular case, best to say something like: if confirmed: ...16:32
jimbakerno need to say is False, certainly the argparse setting ensures it's True or False, and if even if it didn't, i don't see that it would matter if it were 0 or None16:33
SpamapSRoAkSoAx: pong, sup?16:51
RoAkSoAxSpamapS: howdy!! I was working on the load_state stuff, and doing that, http://pastebin.ubuntu.com/649258/, returns something like: exceptions.AttributeError: 'str' object has no attribute 'get' 16:54
RoAkSoAxany ideas?16:54
SpamapSRoAkSoAx: reading now17:03
hazmatRoAkSoAx, the state being returned by load_state is a string it appears, does the error happen if you remove the bottom line17:04
hazmatRoAkSoAx, you also can't do this interactively since there are deferreds in the mix17:04
dakerfwereade, sorry again, can you give a final look https://code.launchpad.net/~daker/ensemble/small-fix/+merge/68038 ?17:05
SpamapSIts possible the state that was saved wasn't properly serialized17:05
hazmatthe reactor needs to be started and the result processed either with a callback or with @inlineCallbacks and yield17:05
RoAkSoAxhazmat: yeah I'm working on that ;)17:05
fwereadedaker: looks great to me, my approve stands :)17:08
hazmatdaker, just added a review comment to it17:10
dakerhazmat, i hope it's good now ツ17:18
* daker crosse his fingers17:19
hazmatdaker, looks good, except for the missing test17:28
fwereadehazmat: golly, we even have tests for the command line, I didn't even think to demand them...17:30
fwereadehazmat: I think the laxity at my last place has damaged my brain :/17:30
fwereadehazmat: I used to test *everything*17:30
fwereadehazmat: ...I'm so happy to be here :D17:31
hazmatfwereade, in this case the cli is our primary interface, if we're not testing it.... then bad things ;-)17:31
fwereadehazmat: quite... I just got used to being happy when tests for *anything* showed up ;)17:31
fwereadehazmat: ok, that's not fair, but we didn't test anything like as much as we did at the place before17:32
fwereadeanyway, I think I'm degenerating into friday evening rambling, and I should be off17:33
fwereadebye all :)17:33
robbiewFriday?17:33
robbiewlol17:33
* robbiew notes that fwereade is on holiday tomorrow17:33
_mup_Bug #814194 was filed: Implement support for expose and unexpose hooks <Ensemble:New> < https://launchpad.net/bugs/814194 >17:34
hazmatdaker, here's a test for it btw, if you could add this to your branch in ensemble/control/tests/test_shutdown the branch should be good.. https://pastebin.canonical.com/50078/17:34
dakeroh good thanls hazmat 17:35
hazmatdaker, np, thanks for working on it17:36
dakerhazmat, would be good if you can put it on paste.u.c, i don't have permission  access that site17:36
dakerto*17:37
hazmatdaker, http://pastebin.ubuntu.com/649319/17:37
dakerhazmat, one last question, how can i run the test ?17:39
hazmatdaker, the easiest option assuming you have the zookeeper package installed (and the zk server running) is just trial ensemble/control/tests/test_shutdown.py17:40
dakerok17:40
dakerany thoughts http://paste.ubuntu.com/649326/ ?17:48
dakerhazmat, ^17:49
hazmatPYTHONPATH=/home/daker/Projects/small-fix/ trial ensemble/control/tests/test_shutdown.py17:49
hazmatdaker, ^17:49
hazmattrial is from the twisted pkg17:49
dakerhazmat, is the test includes the -y argument ?17:50
hazmatdaker, the test i pasted does17:50
dakerok hazmat the test succeed 17:51
hazmatdaker, the error is because the test is being run directly, which A) isn't going to work as the test runner won't be invoked b) relative imports need a fully qualified reference to work correctly when the script is being run this way the current module doesn't really know its package17:51
hazmatdaker, cool, i tested before i pasted, but its good to have confirmation ;-)17:51
dakerhazmat, done! https://code.launchpad.net/~daker/ensemble/small-fix/+merge/6803817:57
niemeyerYo all18:03
jimbakerniemeyer, hi18:04
_mup_ensemble/security-groups r285 committed by kapil.thangavelu@canonical.com18:07
_mup_security groups18:07
=== daker is now known as daker_
_mup_ensemble/security-groups r286 committed by kapil.thangavelu@canonical.com19:22
_mup_group principal membership management and connection attaching.19:22
_mup_Bug #814260 was filed: Ensemble security group principals <Ensemble:New for hazmat> < https://launchpad.net/bugs/814260 >20:02
_mup_ensemble/security-groups r287 committed by kapil.thangavelu@canonical.com20:12
_mup_add get_token to groupprincipal for parity with normal principal20:12
_mup_ensemble/trunk-merge r272 committed by kapil.thangavelu@canonical.com20:16
_mup_merge trunk20:16
_mup_ensemble/security-otp-principal r289 committed by kapil.thangavelu@canonical.com21:06
_mup_otp principal for passing credentials via insecure channels21:06
_mup_ensemble/robust-hook-exit r281 committed by jim.baker@canonical.com21:18
_mup_Use processExited as the normal end of the process, while making processEnded available for purposes like testing21:18
_mup_ensemble/robust-hook-exit r282 committed by jim.baker@canonical.com21:28
_mup_PEP821:28
_mup_ensemble/robust-hook-exit r283 committed by jim.baker@canonical.com21:40
_mup_Doc string21:40
_mup_ensemble/security-otp-principal r290 committed by kapil.thangavelu@canonical.com22:02
_mup_minimal principal interface for otp, using the otp consumes it22:02
_mup_Bug #814320 was filed: Need OTP for passing credentials to an external process <Ensemble:In Progress by hazmat> < https://launchpad.net/bugs/814320 >22:07
hazmatphew.. i think thats all the core security machinery needed22:09
hazmatsigh.. "Launchpad encountered an internal error during the following operation: generating the diff for a merge proposal.  It was logged with id OOPS-2028MPJ3.  Sorry for the inconvenience.22:09
hazmat"22:09
hazmatoh well22:09

Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!