gary_posterbac benji frankban gmb call in 212:08
bacgary_poster: i'd like to see the rabbit timeout failure logs if you still have them.12:22
* gmb -> lunch12:35
gary_posterbac, http://pastebin.ubuntu.com/992273/12:41
bacfor me, an interesting aspect of watching _Portlandia_ is imagining our Portland-based colleagues interacting with them.12:41
bacgary_poster: that log is interesting in that it shows it used the new rabbitfixture and the timeout loop worked properly.  given that, perhaps we do need to try upping the timeout, though 15 seconds is pretty long already.12:45
bacgary_poster: and next time it may be worth looking at the rabbit log file mentioned in the exception, if the machine doesn't get torn down12:46
gary_posterbac, ok cool.  For reference, on eight core concurrency, 30 seconds is usually enough to start up all the lxc containers and 60 is always enough; with 32 cores, 240 was the minimum12:46
gary_posterand this failure was on 32 cores12:46
gary_postersorry, not 240, 160 was the minimum12:46
gary_posterso more than 5 *12:47
bacgary_poster: so perhaps we make an experimental version with absurdly long startup timeout and log the actual start up time so we can see how this one service is impacted by number of cores12:49
gary_posterok bac.  sounds like a plan.  I wish getting the logs were easier. :-/12:52
gary_posterfrankban, lpbuildbot approved.  looks good, thank you12:56
frankbanthanks gary_poster12:57
benjigary_poster: https://dev.launchpad.net/yellow/Termbeamer13:00
benjiI did the basics and added a "Motivation" section mostly cribbed from the email I sent about the old way of sharing terminals13:00
gary_posterbenji, looks great13:21
gary_posterfrankban, sorry, late.  joining.13:41
gary_postergmb, you around?14:12
bacwould someone like to review https://code.launchpad.net/~bac/lpbuildbot/cleanup-testr/+merge/106197 -- to add the functionality to delete old testr data ?14:23
gary_postergmb yoohoo14:31
gmbgary_poster: Yep, around.14:31
gmbbenji's ping issue is infefectious.14:32
gary_postercool gmb.  frankban is looking for a next task and I thought pairing with you would be wunderbar14:32
gary_posterwe have our call now14:32
gary_posterwould it be worth talking with him to introduce the problem14:32
gary_posterso he can investigate a bit14:32
gary_posterand then we can call14:32
gary_posterand then you can rejoin?14:32
gary_postergmb, note that frankban will not be here tomorrow14:32
gary_posterbac, if no one has taken it by 11 or 11:30 I'll be able to14:33
benjiand you guys can collaborate remotely via terminal <wink> <wink>, <nudge> <nudge>14:33
gary_poster:-) yeah that too14:33
gmbgary_poster: That works for me; I can pick something else up tomorrow or make what headway I can or... something.14:33
gary_postergmb, cool.  Your manager14:33
gary_poster's pipedream is that you two identify the problem today14:33
gary_posterand then you have a chance to code it up tomorrow :-)14:34
gary_posterand look at that caterpiller smoking the hookah outside!14:34
gary_posterpipedream?  crazy?  you know?  oh ok. :-P14:34
gmbgary_poster: So, should frankban and I talk and then you and I can talk when I'm free, or do you want in on this, since you have more context and I've only given it a tentative prodding?14:35
gary_postereh, I wasn't intending to participate, but sure, I'll join the party14:35
gary_postergmb, frankban https://talkgadget.google.com/hangouts/_/extras/canonical.com/goldenhorde ?14:36
frankbangary_poster: joining14:37
gmbfrankban: So, I'm going to grab coffee and then I'll be right with you.15:07
frankbanok gmb15:07
* benji reboots15:08
gmbfrankban, I'm in https://plus.google.com/hangouts/_/extras/zope.testing.die.in.a.fire whenever you're ready.15:32
frankbangmb: http://pastebin.ubuntu.com/992640/16:28
gmbfrankban, Thanks.16:28
gary_posterbenji, hey.  So, I'm trying to get my head around the implications of your most recent results for bug 99218417:03
_mup_Bug #992184: lib/lp/services/database/doc/textsearching.txt fails intermittently/rarely on parallel tests <paralleltest> <Launchpad itself:Triaged by stub> < https://launchpad.net/bugs/992184 >17:03
gary_poster - previous tests are irrelevant17:04
gary_poster - all code other than the python call to nl_term_candidates is irrelevant?17:04
benjigary_poster: grr/yay! by beep bug is back again, now I can test my latest theory17:04
benjigary_poster: I believe both points are correct.  Another one: there are other calls to nl_term_candidates in that test, but this one is the only one that ever fails17:05
benji(I need to write that in the bug.)17:05
gary_posterbenji, can you dupe in bin/harness alone?17:07
* benji tries to remember what bin/harness is.17:07
gary_posterbenji,it's like bin/py except the db is initialized17:07
gary_posterI think :-P17:08
benjiI'll try to repro with that.17:08
benjigary_poster: not so much, utility lookup error17:15
gary_posterbenji, which utility?17:15
gary_posterit executes zcml before starting AFAICT17:15
benjigary_poster: lp.services.webapp.interfaces.IStoreSelector17:16
gary_posterbenji, so you mean bin/harness didn't even start for you?17:16
gary_poster(it does for me)17:16
gary_poster(but I'm several revisions behind the times I bet)17:16
benjigary_poster: it does, but since it doesn't accept a script on the command line to run I just hacked it to run the code I want and that code generates the error17:17
benjithe docstring suggests that it should work, I'm still looking at it17:17
gary_posteroh ok17:17
benjiI got it to work, now to run it in a loop for a while.17:19
gary_posterbenji, that's really weird that only the running the test in a loop worked17:35
gary_posterI imagine you already investigated the test's setup?17:35
benjigary_poster: nope, that would be a good thing to look at.17:36
benjigrr, no python file mentions the doctest in question; someone must be a smarty pants and loading it by looking at the files in a directory17:37
benjiah, found it17:41
benjithere's no setup, it is run in the DatabaseFunctionalLayer17:41
gary_posterbenji, I think layers define pretest and posttest code17:54
gary_posteror, they can, I should say17:54
gary_posterMy system updated, and wants a restart.  back in a mo.17:55
benjiI'm looking at the various layers triggered by that test now, I'm going to try to see which bits can be removed without making the test fail.  Maybe some will keep it passing but never let it generate the error and we'll have a clue.17:56
gary_posterbenji, no rush, https://talkgadget.google.com/hangouts/_/extras/canonical.com/goldenhordeoneonone is ready when you are18:32
benjigary_poster: sure, one minute18:32
bacgary_poster: ping me when ready19:31
gary_posterack thanks bac almost19:31
* bac -> refill19:32
gary_posterbac, ping (https://talkgadget.google.com/hangouts/_/extras/canonical.com/goldenhordeoneonone)19:45
* benji reboots to test missing beep hypothesis.19:49
gary_posterbac, approved with only the comment we discussed on the call20:08
bacgary_poster: great, thanks20:08
bacgary_poster: i've added rabbitfixture-0.3.3-bac-exp.tar.gz to download-cache with 90 second timeout20:11
gary_posterok bac, I'll give it a whirl tomorrow and report back20:18
bacso apple is going all renewable at the NC data center20:20
bacthey claim a 20 megawatt solar site will produce 42 million kwh20:21
bacand a 5 megawatt fuel cell will produce 40 million kwh20:21
bachow's that work?  do they base the solar output on 6 hr/shine/day?20:21

Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!