/srv/irclogs.ubuntu.com/2012/05/22/#launchpad-yellow.txt

benjiI think I have termbeamer installable from a PPA now.  Testing would be appreciated.11:23
bacbenji: cool11:43
benjibac: sudo apt-add-repository ppa:benji/termbeamer; sudo apt-get install termbeamer if you want to try it out11:44
bacah, you forgot sudo apt-get update!  :)11:44
gary_posterhey bac, did you get what you needed from that ec2 instance?  I have some worker lists from it if you haven't that I could save.  I want to restart the instance and see if the changes to my charm work11:45
bacgary_poster: please save.  i need more data to chase a theory11:45
gary_postercool11:45
frankbangary_poster: http://ec2-23-21-26-159.compute-1.amazonaws.com:8010/builders/lucid_lp/builds/1/steps/shell_8/logs/summary the first failure seems a new one, the second should be 99672911:46
frankbangary_poster: and thanks for the review11:46
frankbans11:46
gary_posterfrankban, the first one is also known, and Brad has a simple egg I was supposed to test to address it (simply increasing a timeout)11:47
gary_posterThat said, frankban, I hadn't filed a bug because it was tied to some previous work Brad had done11:47
gary_posterthat was probably a mistake11:47
gary_posterso if you file a bug, Brad or I can take it11:47
frankbangary_poster: oh, ok11:48
benjibac: I did forget the update.  Once it's installed try "tb --help"11:48
frankbangary_poster: do you need the worker list?11:48
gary_posterreviews: you're welcome of course--thanks for that work11:48
gary_posterfrankban, nah thanks11:48
frankbangary_poster: https://bugs.launchpad.net/launchpad/+bug/100282011:50
_mup_Bug #1002820: lp.testing.layers.RabbitMQLayer:setUp times out rarely/intermittently in parallel tests <paralleltest> <Launchpad itself:Triaged> < https://launchpad.net/bugs/1002820 >11:50
frankbanadding it to the board11:51
gary_posterperfect thanks frankban11:51
* benji rushes to make coffee before the call.11:51
bacbenji: what happens if i have two-level auth turned on for my gmail account?11:54
bacbenji: and i do.  :)11:54
benjibac: me too, you have to create an application password: support.google.com/accounts/bin/answer.py?answer=18583311:56
bacgreat11:57
bacbenji: hey that appears to work.  i did get a traceback: http://paste.ubuntu.com/1000684/11:59
benjibac: that's caused by not having my fixed python-vte package, which I made a PPA dependency but I guess that doesn't work the way I think it works; add the benji/vte-fixes ppa and do the update/upgrade dance and you should be good to go12:01
benjithe best part of being home is that I can reliably make a decent cup of coffee12:02
gary_posterbenji, which home is this now? :-)12:02
benjigary_poster: heh, VA12:02
gary_posterhow/when did you get there?12:03
gary_posterI guess when is the interesting part, assuming a teleporter wasn't involved12:03
bacbenji: you don't travel with your own stash of coffee?12:03
benjihow: Dodge Caravan, when: Sunday at about 8:30 pm local time12:04
gary_posterah!  I thought you were still in TN yesterday12:04
bacbenji: we now have a clean launch.  just like SpaceX.12:04
benjibac: heh, maybe I should from now on; although my dad has one of those keurig one-cup-at-a-time things, they're pretty good12:05
gary_posterfrankban, bac, you are the designated default maintainers for our buildbot-slave and buildbot-master charms, respectively, because you had the most commits.  Would you object if I try to see if we can make yellow the maintainer for both of those?12:05
bacnope12:06
benjibac: maybe after the call we can do a tb test run12:06
baci'll prepare the iron lung12:07
bacor was that polio?12:07
frankbangary_poster: no objections12:07
bachey you've got a name for your follow on project12:07
benjilol12:08
gary_posterlol12:08
gary_posterthanks bac & frankban12:08
gary_posterbac benji frankban gmb call in 212:08
gary_posterI'll get us a room, as it were12:08
gmbk12:08
benji(I think it was polio.  People with TB just died.)12:09
gary_posterhttps://talkgadget.google.com/hangouts/_/ca56b39f7ec51a3a743513e476a5512a15a0253e12:10
gary_postergmb yoohoo12:11
gmbgary_poster, Firefox in quitting hilarity.12:11
gmbBear with me.12:11
gary_poster:-P12:12
gary_postercool12:12
benjibac: want to try termbeamer?12:26
bacoh sure12:26
baci am brad.crittenden@gmail12:26
bacbenji: what is your xmpp?12:27
benjibac: you've been invited to chat (approving the other person is a one-time first step that we have to do)12:27
benjibenji.york@gmail12:27
bacso i just did --xmpp-receive12:28
bacand nothing is happening12:28
bactb --xmpp-user=brad.crittenden@gmail.com --xmpp-password=XXX --xmpp-receive12:29
bacis that right benji?12:29
benjibac: did you go to gmail and approve my request to chat?12:29
benjibac: yep, it looks good12:29
gmbWups. Better remove that sweary fake test from the LP tree before I commit...12:29
bacbenji: go where?  should i receive an email?12:30
benjibac: the gmail UI should prompt you12:30
bacer, cc.booty2?12:31
bacok, i authed that.  still see nothing12:31
* bac -> tea12:32
benjibac: I invited the wrong you to chat, look for another auth message12:33
gary_posterbac, workers 0-7 of r15280 on parallel buildbot: http://pastebin.ubuntu.com/1000718/ http://pastebin.ubuntu.com/1000719/ http://pastebin.ubuntu.com/1000720/ http://pastebin.ubuntu.com/1000722/ http://pastebin.ubuntu.com/1000724/ http://pastebin.ubuntu.com/1000727/ http://pastebin.ubuntu.com/1000728/ http://pastebin.ubuntu.com/1000729/12:34
gary_posterargh12:35
gary_posterwe don't have 15280 on lpbuildbot >:-(12:35
* gmb lunches12:35
gary_posterI'll make a new set12:38
bacthanks gary_poster12:40
gary_posterwelcome12:40
bacbenji: done12:40
bacbenji: i can see you12:40
benjiyay!12:40
benjibac: watch this...12:41
bacok12:41
bacwhat/12:41
benjidid your window get smaller when I resied mine?12:41
bacno12:41
benjidarn12:41
bacoh, i made my window big12:41
benjibac: oh! I forgot to get you to use the --gui-client switch12:42
benjirestart with that12:42
bacdone12:42
bacyes, smallerized12:43
benji:)12:43
benjicool12:43
bacyes, very neat12:43
baccan i do anything or just watch?12:43
gary_posterwatch the blinken lights12:43
benjibac: right now just watch, the very next thing I want to add is client to server communication12:43
bacthat will be cooler but this is very neat indeed12:44
benjibac: did you see the flashing crosshair?12:44
bacyes12:44
benjithat's the first client-to-server thing I want to add, so we can both point things out12:45
benjibac: do these colors look sane?12:46
bacyes.12:46
bacthe italics and underline2 didn't do anything12:47
benjibac: yeah, most terminals don't support that (including the terminal widget I'm using)12:47
bacbenji: also note i'm running this in an ssh session from X on OS X to precise and it still works.12:48
benjiit does support strikethough, but I don't bother sending that data, strikethrough seems rare12:48
benjivery cool12:48
benjibac: do you want to try being the sender?12:48
bacok, sure12:49
bacbenji: do i use --gui-client?12:49
benjibac: nope12:49
benjithe server is gui-only so there's no switch for that12:49
benjibac: I see you12:50
benjibac: heh; try your console editor of choice12:50
benjiyou and your white background12:50
bachow do i point?12:51
benjicontrol-click12:51
benjibac: looks good12:51
bacoh, you can see the crosshair?12:52
benjiyep12:52
baci don't see anything when i control click12:52
benjiI need to make the server show it too for feedback12:52
* benji adds to the TODO12:52
bacwell that's a smashing success12:53
benjicool12:54
gary_posterbac db-devel r11611.  parallel tests report 16718 tests, lpbuildbot reports 16761 tests (https://lpbuildbot.canonical.com/builders/lucid_db_lp/builds/1979).  parallel buildbot test worker lists follow, 0-7: http://pastebin.ubuntu.com/1000745/ http://pastebin.ubuntu.com/1000746/ http://pastebin.ubuntu.com/1000747/ http://pastebin.ubuntu.com/1000749/ http://pastebin.ubuntu.com/1000750/ http://pastebin.ubuntu.com/1012:55
gary_poster00753/ http://pastebin.ubuntu.com/1000756/ http://pastebin.ubuntu.com/1000757/ .  If I add the line numbers from those eight lists, I get 16718, so it seems to represent what the parallel test buildbot is counting.  Also frankban, those test lists are perfectly round-robin in counts (every list is either 2089 or 2090) which is good...except odd because it was not a first test run. :-/  Anyway, bac, that looks lik12:55
gary_postere something you can use.12:55
gary_posterI'm going to shut down the instance and restart now,12:55
gary_poster.12:55
bacgary_poster: great.  that blows my theory, though.  had parallel shown more tests then i had a possible explanation12:57
gary_poster:-/ k13:03
gary_posteroh, bac, did you see my idea on presenting about parallel testing to that pythoncarolina thing in CH?  Not sure if it fits your timeline13:05
bacgary_poster: no13:15
* gary_poster looks for it13:15
bacgary_poster: found it13:15
bacthat sounds interesting13:16
gary_postercool13:16
bacgary_poster: if i'm not around you can just wing it.  :)13:16
gary_poster:-)13:16
bacgary_poster: so i ran into a guy at RDU wearing an Open Stack t-shirt.  he said "yeah, i work for this company called 'Canonical'"...  odd13:17
gary_posterbac, weird.  https://directory.canonical.com/list/state/NC/ ... bfox or bladernr?13:21
bacgary_poster: no, i know those guys.  some new guy named adam stokes who doesn't have his address in the directory correctly13:22
gary_posterhuh13:22
gary_postercool,  I guess13:22
bacand he pronounced it caneenical13:22
benjiheh13:25
benjithat's the little-used Knights Who Say Caneeenical pronunciation13:26
frankbanwow, DEBUG = True in canonicaladmin: https://directory.canonical.com/NOOOO13:27
benjifrankban: that is hilarious13:27
frankbangary_poster: I've moved the card to done-done, AFAICT testrepository is doing the right thing13:27
bacfrankban: well, at least that is only on the directory not canonicaladmin13:28
frankbanbac: yes13:28
benjihttps://directory.canonical.com/orgchart/ takes a while to respond :)13:29
benjifrankban: we should report that to whomever is responsible (I don't know who that is)13:33
frankbanbenji: indeed (me either)13:36
bacbenji, frankban: probably stuartm13:43
gary_posterfrankban, awesome thanks14:16
bacgary_poster: we have some tests (mainly doctests i think) that get run multiple times.  lpbuildbot is reporting them individually where our parallel test output only lists them once.  summary: http://paste.ubuntu.com/1000909/14:20
gary_posterbac, should we be paranoid that these tests are not actually being run multiple times as they should be?14:20
gary_posterconcerned14:21
bacgary_poster: i'm looking at the data collection code now.  i suspect it is a hash indexed by test name which throws away multiples14:21
gary_posterbac, that would make sense, except that we have seen some duplicates, like that api test from, um, some bug we had14:22
gary_posterbenji, I see you are working on the rabbitmq bug14:22
gary_posterdid you get the egg from bac already?14:23
gary_posterback in a sec14:23
benjigary_poster: I feel like I should know what you're talking about because it was discussed this morning.14:24
bacbenji: last week i produced a rabbit-fixture egg that set absurdly high timeouts14:24
benjiand those timeouts were still being reached?14:24
bacbenji: no, gary never used it14:25
bacbenji: and now i can't find it.  :(14:25
bacbenji: found14:26
benjiwhat is the intended use of that egg?14:27
bacbenji: to run a novel branch of launchpad that uses that egg under parallel testing to see if the timeouts go away14:28
gary_posterbac, benji, this is intermittent.14:28
benjibac: are the timeouts frequent enough that doing that would tell us anything?14:29
gary_postertherefore, I suggest that we may need to actually land use of that egg14:29
bacbenji: it is rabbitfixture 0.3.3-bac-exp and is now in download-cache/dist14:29
gary_posterand see if it goes away.14:29
gary_posterWe've seen it twice in about 30 or 40 runs, I'd guess14:29
benjiMeta: I think there is room for process improvment here.  I feel like these things should have been written down in the bug.14:29
benjiyeah, either reproducing it or dramatically upping the timeouts seem like the only two rational approaches14:30
benjibac: did you try to reproduce the timeout?14:31
gary_posterbenji, process improvement: Gary should have made a bug.  It was kind of a follow-on to existing work, so I didn't. Lesson: always make a bug.  It is not written in the bug because it is a placeholder that I asked Francesco to make. Ideally I would have at least added a "talk to me or Brad" to the bug.14:33
gary_posterWe did discuss it in this morning's meeting, but again, Gary should have written it down.  Lesson: always write stuff down in the proper place.14:34
gary_posterI know those lessons, just didn't do 'em.14:34
gary_posterLesson: kick Gary.14:34
gary_posterbenji, I am pretty sure bac did not try to repro this timeout.  He had reproed an earlier one to determine that there was a bug in the egg.14:35
gary_posterSomeone had already fixed the code14:35
bacand if we are to make an egg that is referenced by launchpad in production then it needs to be a full release of rabbitfixture and the code pushed14:35
gary_posterbut not released it14:35
bacgary_poster: yes, but that 'fix' was not sufficient14:35
bacbenji: no, i have not tried to reproduce with the new experimental egg14:35
gary_postersimply because the timeout was not enough for what we need, as we believe14:36
bacyes.  it would be the proper fix for a non-parallel environment14:36
benjiso I should make a "full release" of rabbitfixture and add that to LP, right?14:36
gary_posterYeah, that's my suggestion.  I'm not sure what Brad means by a full release14:37
gary_posterbac, do you think we should merge this?  I was thinking we ought to have a fork, and maybe file a bug to remove the fork and merge to trunk once we get 40 or so clean runs14:37
bacgary, i mean 0.3.4 instead of 0.3.3.-exp14:37
gary_postergotcha, then I understood...14:38
bacgary_poster: a fork is fine14:38
gary_posterok14:38
gary_posterbenji ^^14:38
bacgary_poster: just don't do it via egg drops14:38
gary_postergotcha bac, agreed14:38
bacper your previous edict14:38
gary_posteri'm all about my edicts14:38
baci prefer hot'n'sour over egg drop14:38
benjirevision: I will fork rabbitfixture, make an egg, put that in the dependencies, and land an LP branch that uses that version14:38
gary_postergo thou! go forth! nay, stop that villainy!14:39
bacbenji: do you want my code?14:39
benjibac: sure (any reason we don't just use your egg as-is?)14:39
gary_posterbecause it is an egg drop14:39
gary_posternot a branch14:39
benjiah14:39
benjiI missed that bit.14:39
frankbanaaargh gary_poster, I've written buildout instead of buildbot in the dependencies dir (slave charm) can I r=you the fix? btw, we should call the next project "buildnot"14:40
benjiwhat PPAs are to debs X are to eggs (we need an X)14:41
bacbenji: see lp:~/rabbitfixture/timeout-exp14:41
benji"buildnot"!14:41
bacbenji: review the changes as you may not like them.  it is a mix of just upping the timeout and trying to parameterize it14:41
bacbuildsnot14:42
benjibac: ok, thanks14:42
benjipfft14:42
gary_posterfrankban, sorry, of course14:54
gary_posterthis error looks really familiar... http://ec2-184-72-171-77.compute-1.amazonaws.com:8010/builders/lucid_lp/builds/0/steps/shell_8/logs/summary14:55
gary_posteranyone recognize it?  Looking for it...14:55
benjigary_poster: that looks similar to one Brad and I fixed14:56
gary_posterbug 99351014:56
_mup_Bug #993510: lp.services.job.tests.test_runner.TestJobRunner.test_runJobHandleErrors_oops_generated_user_notify_fails fails intermittently/rarely in parallel tests <paralleltest> <Launchpad itself:Triaged> < https://launchpad.net/bugs/993510 >14:56
gary_posterbug 99269214:56
_mup_Bug #992692: lp.services.mail.tests.test_incoming.TestIncoming.test_invalid_to_addresses fails intermittenty/rarely in parallel tests <paralleltest> <qa-untestable> <Launchpad itself:Fix Released by benji> < https://launchpad.net/bugs/992692 >14:56
bacgary_poster: do you have the full, unfiltered subunit output for the db-devel r 11611 run?14:57
benjigary_poster: 992692 is the one I was thinking of14:57
gary_posterbac, no, sorry.  If that's necessary then I have another machine running tests now, but you'll have to start the analysis over again14:57
gary_poster:-(14:57
bacgary_poster: it's all scripted14:57
bacso it is no thing14:58
bacalso, to save you work i prefer the full subunit output not the ones divided by worker14:58
gary_posterok cool bac, got it14:58
bacgary_poster: eta for completion?14:58
gary_posterbac, 5 min.15:30
gary_posterWe are getting these somewhat frequently15:31
gary_posterhttp://ec2-184-72-171-77.compute-1.amazonaws.com:8010/builders/lucid_db_lp/builds/0/steps/shell_6/logs/stdio15:31
gary_posterDon't quite understand what is going on15:31
gary_posterI think it happens when we switch between devel and db-devel15:31
gary_posterbut could be wrong15:31
gary_posterbac, maybe you can just download this  http://ec2-184-72-171-77.compute-1.amazonaws.com:8010/builders/lucid_lp/builds/0/steps/shell_8/logs/subunit/text15:33
bacok15:33
frankbangmb, am I right if I say that you can create a co-located branch from a normal one (residing in a shared repo) with "bzr switch -b foo"?16:10
gmbfrankban: I have no idea, I've never tried that :). Sounds sane, though.16:13
frankbangmb: oh, because you started using colo-branches with the plugin I guess16:15
gary_posterfrankban, try it :-)16:15
gmbyep16:15
frankbanok16:15
gary_posterit's kind of sad that I really need to use Google to search Launchpad's buugs16:26
bacyay, i have water again!17:13
gary_posteryay!17:21
gary_posterwater good17:21
bacthere is a big capacitor that is used at start up to give the pump a good jolt.  it was nasty and the wires fell out when touched.17:22
baci heart my redneck plumber17:22
bacgary_poster:  do you know what revno was used for the run referenced above?17:23
gary_posterheh17:23
gary_posteryeah I can get it17:24
gary_posterbac, 15282 (see http://ec2-184-72-171-77.compute-1.amazonaws.com:8010/builders/lucid_lp/builds/0/)17:24
bacgot it17:24
gary_posterfrankban, was colocation as easy as it sounded?17:29
frankbangary_poster: yes17:29
gary_posterfrankban, cool!  I'll have to try.  Sounds like the advantages of lightweight checkouts but simpler to use17:29
frankbangary_poster: yes it's cool, and actually I think each colocated branch is internally a lightweight checkout of the current branch17:32
gary_postermakes sense17:32
bacgary_poster: i think the duplicate tests are not being run the required number of times.  each only shows up once in the unified subunit output from the latest run.  the test data i had from yesterday (for r 15139) shows multiple lines for the repeated tests.18:04
gary_posterbac, oh, so maybe a regression with the newer version of the testr and friends packages.  Thank you for investigating.18:05
bacgary_poster: yeah, i think so.  what was the date of the 15139 run?18:06
bacnm18:06
gary_posterbac, to call this finished, then, we should file a bug.  We can throw it at something like testr and see if it sticks (if Robert will investigate) but +1 if you are willing and interested to take it farther18:06
bacapril 2318:06
gary_posterright, exactly18:07
bacok, i didn't realize it was that old18:07
bacgary_poster: dumb question, how can i do the equivalent of 'bin/test -t faqcollection.txt' with testr?18:25
bacok, so this appears to be what i was after:18:29
bacbin/test --subunit -t faqcollection.txt |testr load18:29
bacduh18:29
gary_postergood to know18:31
gary_posterbad stats today :-/19:09
bacgary_poster: have time for a chat?20:12
gary_posterbac, supposed to be talking to flacoste, but not, so sure20:13
bacyou want to make a hangout20:13
gary_posteryeah ok20:13
bacsince whatever i did yesterday failed20:13
gary_posterbac https://talkgadget.google.com/hangouts/_/49af6cb2474cf14af2cb37afc56f73c7c5fc120a20:14
bacgary_poster: i've been doing some standalone (i.e. not buildbot) tests with testr --parallel20:56
bacif i run "testr run --parallel -- -t faqcollection.txt" then faqcollection.txt appears once in the test partitions but gets run three times as seen in e.g. .testrepository/120:56
baclooking at the code i don't see any use of sets or other attempts to make the test ids unique20:57
gary_posterbac, dunno20:58
gary_posterbac, also, when you run them non-parallel, are they in different layers?20:59
bacso i remain a bit confused.  the subunit output from the test you showed me today only had the suspect tests listed once20:59
bacno20:59
gary_posteror more generally20:59
gary_posterhow are they different?  are we sure we are running the, differently even when we see them multiple times across workers?21:00
bacbut, i think i may have a different version of testrepository installed locally due to the funky ppa naming21:00
gary_posteroh21:00
baclet me try to get the same version and see what happens21:00
gary_postermaybe21:00
gary_posterok21:00
gary_poster"the," was supposed to be "them"21:01
gary_posterbac I have to run21:01
bacok, have fun.  wear a helmet if you go out21:01
gary_posterwill be back later fwiw21:01
gary_posterheh thx21:01
bachail hurts21:01

Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!