[10:08] hi gmb [10:20] gmb, the latest drop of juju deprecated some settings in environment.yaml in favor of usng --constraints. it makes the rewriting o the environments file done by your run_lp_tests script unecessary. i've made changes to it and to charmhelpers to account for juju changes but haven't completed the changes or pushed them anywhere yet. [10:21] i'll do that after breakfast. just wanted to give you a heads up so you don't duplicate effort. [10:45] bac, Awesome, I was just looking at that. [10:45] I'll let it lie, then. [10:45] gmb: great. i'll get to it post-oatmeal [12:10] bac benji frankban gmb, call asap [12:10] Booting FF now. [12:10] * bac enhordes [12:33] * frankban lunches [12:33] i have pushed run_lp_tests branch to lp:~yellow/+junk/run_lp_tests [12:33] that allows me to move the card, so the lane is unblocked now [12:34] * gmb moves bug 974586 card, blocks the lane again. [12:34] <_mup_> Bug #974586: bugtracker test logging failures < https://launchpad.net/bugs/974586 > [12:41] * bac should read warthogs more. didn't know poolie was leaving. :( [12:55] * gmb -> lunch [12:56] how does one set up PYTHONPATH so that it'll load the code being tested instead of the system version? i would've expect this approach to work, but the local version (/home/bac/add-charm-helpers/helpers/python) is stuck in the middle and isn't getting loaded. http://paste.ubuntu.com/924798/plain/ [12:57] gary_poster: I'm back for the time being teathered to my phone. I doubt I [12:58] can do video confrencing, but other things seem to be ok. [13:00] so eggs in /usr/local/lib always get inserted before PYTHONPATH? [13:00] cool benji. replying to jml now. his branch looks good [13:01] bac, that's a policy; maybe a deb one [13:01] ergh [13:02] gary_poster: will you point me at his branch? [13:02] gary_poster: do you know a work-around? gmb added this to the makefile but it is ineffective: [13:02] PYTHONPATH=helpers/python python helpers/python/charmhelpers/tests/test_charmhelpers.py [13:03] benji, sorry forgot you did not have IRC history. #launchpad-dev now fwiw. https://code.launchpad.net/~jml/testtools/forward-current-tags/+merge/101538 [13:04] thanks; I'm following along [13:12] bac: if you only need to get rid of the system site packages to test an app locally, I think virtualenv is an easy tool to get that kind of isolation [13:14] frankban: but the package in question is the one being tested. i think requiring people to use virtualenv is a bit heavy handed. i can achieve the same results by hacking sys.path in the test. ugly. [13:16] bac: oh, I thought it was only for your local environment [13:17] bac, sorry, was talking on launchpad-dev. reviewing backlog... [13:17] np [13:18] bac, system Pythons suck for isolation, sorry. sys.path hack might be best bet. -S will leave a clean sys.path initially, but will probably not be what you want [13:19] gary_poster: sys.path hack isn't so bad. mainly i was just surprised to see my PYTHONPATH stuck in the middle. :( [13:19] bac, it's pretty common to make a requirement that virtualenv --whatever-that-option-is be required in these kinds of circumstances I think [13:19] yeah [13:19] reminds me of dark days trying to figure that stuff out :-P [13:26] i've pushed a new branch of charm-tools to lp:~yellow/charm-tools/trunk and have created a merge proposal. [13:26] until it is packaged we should try to work off of the ~yellow version [13:27] gmb, you up for reviewing lpsetup branch from frankban? [13:38] bac, actually, did you see than Sean actually committed code with his fix? [13:39] gary_poster: i did not. [13:39] I'm tempted to use it, rather than ours, and see if it works [13:39] that's good. [13:39] bac, but a review from you would be appreciated by him and me [13:41] gary_poster: ok. i'll add comments to the bug since he didn't use a MP [13:41] bac, makes sense [13:42] i'm a bit partial, but i think my approach was cleaner. :) [13:43] bac, heh, you are probably right :-) [13:46] gary_poster: so he has committed code but not made a release. are you still going to make an interim egg and package it in LP? [13:51] bac, yes, that's what I was thinking. [13:53] gary_poster, frankban: Just finishing lunch etc., but I'll be happy to take that review in the next 20 minutes or so. [13:53] thx gmb [13:53] thanks gmb [13:56] gary_poster: when you have a minute, I'd like to discuss about lxc and dns servers [13:58] ack frankban. should be available in 3 [13:58] thanks gary_poster [14:00] frankban, in https://talkgadget.google.com/hangouts/_/extras/canonical.com/goldenhorde & ready whenever you are [14:10] frankban, Is there a diff for your lpsetup changes somwhere? [14:11] gmb: https://code.launchpad.net/~frankban/lpsetup/create-scripts/+merge/101536 [14:11] Ta [14:25] gmb, frankban: please see my comment on frankban's MP. [14:26] bac, I'd already made the same note :) [14:26] okey dokey [15:03] thanks bac, I've just replied [15:11] frankban: we need to agree as a group whether our related work will use the LP style guide or not. IIRC "launchpad related" projects have continued to use it. from a ease-of-programming perspective that is what i would advocate, meaning i'd prefer to have one set of guidelines and use it across projects where practical. the exception to that rule is for projects that will be community-supported and only have a tangential relationship to LP. [15:14] bac, frankban, given that lpsetup has PEP 8 everywhere, I'd suggest keeping PEP8 for this MP, with bac raising the issue on the dev list. Depending on that resolution, we can have a future style-only branch. [15:14] for now, let's go with consistency within lpsetup [15:15] well, and for the future too; we can switch all at once [15:17] frankban, gary_poster: i hadn't reviewed the whole branch and didn't realize that was the case. our style guide also says "stay consistent with existing code" or something like that. [15:17] right bac, cool. still I agree with your assertion that we should collectively agree [15:18] about this kind of policy [15:18] bac, if we have not already agreed...? [15:18] with open("worms") as can: [15:18] heh [15:18] colour = random.choice(palette) [15:18] paint() [15:19] :-) [15:19] There should be a law against people phoning me whilst I'm reviewing. [15:20] bac, :-) [15:30] frankban, I'm really sorry, I've been looking at this branch for ages and I can't find anything I don't like. [15:30] r=me [15:30] Apologies. [15:31] gmb: what a shame :-) however thanks for trying [15:31] :) [15:35] gary_poster: my connectivity looks like it might be back, but it's intermittent; I'm going to take an early lunch and hope that it's stable by the time I'm done [15:48] benji ack good luck [16:20] gary_poster: I take it that I should re-implement our worker tagging test result on top of lp:~jml/testtools/forward-current-tags. Correct? [16:21] benji, yes, that's my suggestion. Please feel free to argue otherwise, as usual [16:21] gary_poster: does that mean that it's usual for me to argue? ;P [16:22] benji, :-P no [16:22] heh [16:22] that looks like a good approach; I'll see where it takes me [16:25] cool thx [16:55] gary_poster, It's just been pointed out to me that the first Monday of UDS is a UK national holiday. Is it okay with you if I take the coming Monday (16th) in lieu of it, since I'm going to be swimming the nerd sea on that day? [16:56] gmb, sure, though what is the nerd sea? :-) [16:56] gary_poster, The UDS lobby, of course :) [16:56] (I hasten to add, it was Jono Bacon's name for it, not mine :)) [16:57] gary_poster, Thank you. I'll add it in CanonicalAdmin now. [16:57] lol gmb, cool [17:00] Ahaha. [17:00] * gmb accidentally builds LP tree on Precise. [17:00] Hilarity ensues. [17:00] in order to work around an ec2 test problem with lightweight checkouts, I have to specify --pqm-public-location. Because I'm trying to have fun with sed, I end up with the general command... [17:00] ec2 test --pqm-public-location=`bzr info | sed -e '/push branch/!d' -e '/push branch/s/\s*push branch: //'` [17:01] evilness is fun [17:01] so is making a noun into a noun [17:01] Wow. [17:01] evilnessitude might be even better [17:02] gary_poster, That's disturbingly like something I would do :) [17:02] heh [17:02] ok, test started, sed played with... [17:02] time for some lunch [17:03] * gmb knocks off for the evening. Night all. [17:05] night, gmb [17:27] night gmb. approved Monday. [17:50] gary_poster, with an 8 cpu ec2 machine, should we expect a slave to come up within an hour? [17:51] bac, give or take. two hours is usually too long. You should be able to look at the juju log on the slave instance to see the output of setuplxc [18:19] bac, I self-approved this, but in case you are curious: https://code.launchpad.net/~gary/launchpad/trivial/+merge/101619 [18:19] tests have been running for a couple of hours now. [18:20] bac, I'd like to get a parallel test run or two or three to see how we are doing. Did you get one running? [18:21] memcache change looks good. thanks for the thorough documenting [18:21] gary_poster: i just started one a little bit ago and am waiting for the slave to come up [18:22] i think gmb's script is working now it was just too aggressive in only giving the slave 1 hour to come up [18:22] bac, memcache: cool, thanks. waiting for slave: great. Please lemme know when tests have run: I'd like you to collect the output of testr failing after a test run. [18:22] gary_poster: will do [18:23] thanks [18:23] sadly i don't think my run got your memcache fix [18:26] no, it won't [18:57] gary_poster: my attempt failed, at least the automatic part. [18:57] bac, :-/ how so? [18:57] while waiting for the slave to come up, a call to 'juju status' got [18:58] CalledProcessError: Command '['juju', 'status']' returned non-zero exit status 1 [18:58] 2012-04-11 14:47:17,862 INFO Connecting to environment... [18:58] 2012-04-11 14:47:19,807 INFO Connected to environment. [18:58] [('SSL routines', 'SSL3_READ_BYTES', 'ssl handshake failure')] [18:58] 2012-04-11 14:47:22,699 ERROR [('SSL routines', 'SSL3_READ_BYTES', 'ssl handshake failure')] [18:58] this is not uncommon [18:58] but, i can manually wait for the slave to come up and force the build by hand [19:00] it has been up 66 minutes so perhaps it'll be happy soon [19:02] hm [19:02] ok [19:14] ok, setuplxc is done. took about 80 minutes [19:19] gary_poster: you ever see juju status report 'relation-errors'? [19:19] relation-errors: [19:19] buildbot: [19:19] - buildbot-slave [19:20] bac, rarely, yes [19:20] well it is my lucky day [19:20] trying to think of something helpful.... [19:20] bac, usually the logs are the place I go [19:21] yeah, i see some stuff in charm.log [19:22] nothing too informative, though [19:23] bac, also try master's log [19:23] /var/lib/buildbot/masters/master/twistd.log [19:25] hmm, at :06 the bb master received SIGTERM and shut itself down [19:25] i don't think the slave was even up yet [19:27] i was able to restart the master by hand [19:32] and manually started a slave. go juju go. [19:38] gary_poster: my hand starting stuff did work. it looks like the slaves started as the wrong user. [19:38] /usr/bin/xvfb-run: 181: /var/lib/buildbot/slaves/slave/lucid-devel/build/bin/test: Permission denied [19:39] which confuses me as the log shows: [19:39] sudo /usr/local/bin/launchpad-lxc-test -t stories/gpg [19:50] bac, the permission denied with xvfb-run usually indicates a bad command passed to xvfb-run [19:50] caused by crazy bash escaping [19:50] we've had careful escaping to try and prevent this [19:50] gary_poster: such as where? [19:51] so, the trigger for the problem is that we are trying to pass an argument with spaces to xvfb-run's -a argument IIRC [19:51] gary_poster: i'm using stock yellow charms. are you suggesting it is an error in one of the hooks? [19:52] bac, no, the problem is arguably in ssh, but we are not fixing that today. :-P . Today's breakage of the workaround we implemented is in one of these: [19:52] launchpad-lxc-test [19:52] or lxc-start-ephemeral [19:52] I don't think there would be another [19:54] bac, to see a related behavior, change lxc-start-ephemeral to begin with set -ex (IIRC) [19:55] that will spit out the commands lxc-start-ephemeral sends [19:55] bac, could you give me the url of your master [20:00] gary_poster: looking [20:01] gary_poster: ec2-23-21-33-127.compute-1.amazonaws.com:8010 [20:17] bac, ack. Maybe we should pair on getting a test run manually tomorrow. If we run into any problems, you can get a feel for where the bodies are buried [20:17] gary_poster: that's a good idea [20:17] if this instance is no good to you i'll tear it down now [20:21] bac yeah tear down [20:21] sorry for frustrations (and slow turn around; other stuff going on too) [20:22] ok [20:48] gary_poster: sean says he'll push a new py-memcached tonight [20:48] bac, heh, ok cool [20:49] twas the right decision [20:49] cool bac [20:49] besides we haven't seen it yet [20:53] :-) true [20:54] I also haven't seen my testresults yet...