StevenKwgrant: http://pastebin.ubuntu.com/1229519/00:01
wgrantStevenK: That looks most excellent. Perhaps try some tests that exercise the DB?00:01
StevenK-t bugs ?00:01
wgrantThough DatabaseLayer setup worked, so it's a good sign00:02
StevenKAnd it can run while I nom00:02
wgrant-t bugs is huge00:02
wgrantTry archiveuploader00:02
wgrantRelatively quick and terrible00:02
=== matsubara is now known as matsubara-afk
StevenKwgrant: Right, that looks pretty excellent too00:37
wgrantSo, jenkins :)00:39
StevenKwgrant: Yes00:41
wgrantStevenK: Should be pretty easy, I guess00:41
wgrantNo need for the complex build rules any more00:41
wgrantJust a .testr.conf, lp-setup-lxc-build, testr run --parallel00:41
wgrantOr so00:41
StevenKDo I need the testr from testing-cabal?00:42
wgrantStevenK: You might need new testrepository, testtools and subunit00:48
wgrantMight as well grab them all from there00:48
StevenKJust installing java and fixing ssh access00:48
StevenKThe Jenkins slave is java, no choice in the matter00:53
wallyworldi know, just trolling00:53
StevenKSlave successfully connected and online00:55
StevenKwgrant: So, .testr.conf ?00:55
StevenKThis is different from the .testr.conf already in the LP tree?00:55
wgrantStevenK: Yeah. Grab it from testr.conf in lp:~launchpad/lpbuildbot/public00:56
wgrantIt has a list command and uses lp-setup-lxc-test rather than bin/test00:56
StevenKSo I should move it into the tree as the first step?00:57
wgrantJust clobber the .testr.conf that's in trunk locally for now00:57
wgrantbuildbot copies it in at the start of each build00:58
StevenKI've added a build step00:59
LPCIBotProject devel build #1091: STILL FAILING in 1 min 3 sec: https://lpci.wedontsleep.org/job/devel/1091/01:01
StevenK sudo lp-setup-lxc-build lptests /home/jenkins/.ssh/id_rsa01:02
StevenKsudo: lp-setup-lxc-build: command not found01:02
wgrantIs it in PATH?01:02
StevenKIt's in /home/jenkins/bin, but it looks like that doesn't work01:03
wgrant~/bin isn't in PATH by default01:03
StevenKWell, yes01:03
LPCIBotProject devel build #1092: STILL FAILING in 8.9 sec: https://lpci.wedontsleep.org/job/devel/1092/01:04
LPCIBotProject devel build #1093: STILL FAILING in 28 sec: https://lpci.wedontsleep.org/job/devel/1093/01:06
StevenKNeed testr init first01:06
wgrantIt nearly worked01:06
StevenKAre you spying? :-)01:06
wgrantHow many threads does the instance have?01:07
* StevenK firewalls wgrant off01:07
StevenK4 processors, apparently01:07
wgrantYou'll want testr run --parallel --concurrency=SOMETHING --subunit --full-results01:07
wgrantAnd you'll need to set a shared TEMP, which probably means it needs to be in the code dir01:07
wgrantbuildbot creates temp/ in the tree at the start of each build01:08
StevenKThis is looking okay01:10
wgrantconcurrency=8 is probably going to die01:10
wgrantBut we'll see01:10
StevenKDie how?01:11
StevenKWe have 34GiB of RAM01:11
wgrantIt'll thrash01:11
wgrantAh, so at least RAM won't be a problem01:11
StevenKOh, yeah, java.01:11
wgrantAlso, your build failed01:11
StevenKSo we have 4GiB of RAM available01:11
wgrantlazr.restful missing from download-cache?01:11
wgrantbuildbot's happy with it, so I assume you just failed to update01:11
LPCIBotProject devel build #1094: STILL FAILING in 2 min 19 sec: https://lpci.wedontsleep.org/job/devel/1094/01:11
wgrantBut it otherwise worked01:12
StevenKI guess I need to run bzr pull in the download-cache01:12
wgrantYou'll want to add bzr up download-cache, and probably update-sourcecode, steps01:12
wgrantBut you can just do a one-off for now, I suppose01:12
StevenKI don't need db setup?01:13
wgrantlp-setup-lxc-build should do that01:13
StevenKI thought so01:13
StevenKHere's a script I prepared earlier01:14
LPCIBotProject devel build #1095: STILL FAILING in 10 sec: https://lpci.wedontsleep.org/job/devel/1095/01:14
wgrantEvidently not :)01:14
wgrantStevenK: scripts/init_testr.py from the lpbuildbot branch may be handy01:14
wgrantStevenK: It preserves .testrepository between runs, which makes balancing the tests between runners more effective01:15
wgrantAnd prevents the issue that killed this build :)01:15
StevenKSo far I'm blowing it away, but that can be fixed01:17
wgrantThis is looking better01:18
StevenKwallyworld: Are you watching too, or just ignoring us?01:19
wgrantDoing proper work, hopefully :P01:19
StevenKThis is proper work!01:20
wallyworldwatching what?01:20
StevenKAlso known as buildbot and buildbot-poll die in a large fire01:20
wgrantStevenK: Check in temp/01:20
wgrantAre there 8 files with lots of tests listed?01:21
wgrantIt's starting up 8 new containers, so it looks good01:21
wgrantI think we're good :)01:21
StevenKWe are01:21
wgrantAnd it's even tagged properly01:22
StevenKjenkins@domU-12-31-39-17-3C-7F:~/launchpad/lp-branches/workspace/devel$ uptime01:22
wgrantI must work that out on prod buildbot with gnuoy next week01:22
StevenK 01:22:17 up  3:08,  1 user,  load average: 6.70, 2.75, 1.1401:22
wallyworldwgrant: i recall reading somewhere why the test count is inflated now. can you remind me?01:22
wgrantwallyworld: The Zope testrunner layer setup/teardowns appear as tests01:23
wgranteg 'test: lp.testing.layers.BaseLayer:tearDown'01:23
StevenKI wonder how it will take with 801:23
wallyworldsurely we can filter that out01:23
wgrantIf you grep them out then the count is accurate01:23
wgrantStevenK: Didn't jenkins previously show live test stats?01:24
StevenKIt will show progress, not stats01:24
StevenKBut it takes like 3 or 4 builds to prime01:25
wgrantDoesn't subunit2junitxml let it show live counts?01:25
wallyworldwgrant: StevenK: if you want a break from jenkins, a quickie.... https://code.launchpad.net/~wallyworld/launchpad/embargoed-sharing-policy-ui-1055617/+merge/12658501:27
StevenK8 threads is probably what, 2 hours?01:27
StevenKwallyworld: It's building01:27
StevenKSo now wgrant and I should do a critical or something01:28
wallyworlddo a review first :-)01:28
wgrantI'm waiting for a local testr run too01:28
wallyworldoh bollocks. legit bb failure01:29
* wallyworld fixes01:29
StevenKLine 53 can be sucked back onto 47?01:29
StevenKWhich probably makes the branch neutral01:29
StevenKjenkins@domU-12-31-39-17-3C-7F:~/launchpad/lp-branches/workspace/devel$ uptime01:30
StevenK 01:30:10 up  3:16,  1 user,  load average: 11.62, 9.77, 5.1501:30
wgrantStevenK: Well, you did ask it to run 8 multiprocess test suites on 4 cores :)01:30
StevenKI should have gone bigger?01:31
wgrantDoesn't matter too much01:31
StevenKPft, we have 20GiB of RAM free01:31
wgrantI'm not sure what yellow used for scalability testing, but we don't really need that here01:32
wgrantWe don't care about scaling, just working01:32
StevenKIt seems to work01:33
wgrantStevenK: You should probably steal the remaining build steps from master.cfg in the lpbuildbot branch01:34
wgrantWith scripts/init_testr and co01:34
StevenKAww, can't we just move on and get this on prase? :-P01:34
wgrantAlso shake Julian until MAAS Tarmac and Jenkins configs fall out!01:34
StevenKI think other projects have used it as well01:35
StevenKopenstack is one I can remember. Until they switched to git, at least01:35
wallyworldStevenK: you ok to +1 that branch you looked at?01:52
StevenKwallyworld: Sorry, done.01:53
wallyworldnp thank you01:53
wallyworldi was doing the testfix anyway till now01:54
lifelesswgrant: pqm 0.6 up on LP, and in trunk.02:24
lifelesswgrant: YMMV, may the farce be with U02:25
wgrantOf course I might just dump PQM and use Tarmac instead :)02:26
lifelessfine by me02:26
lifelesslong as you're not running tests under it02:26
lifelesstarmac isn't even faintly paranoid enough about *that*02:27
StevenKTarmac+Jenkins ?02:27
wgrantRight, Tarmac + Jenkins is the sensible solution02:27
lifelessthough I don't know why you'd have tarmac in the picture for that.02:27
wgrantOther projects (eg. maas) have you submitting to Tarmac, Tarmac asks Jenkins to build it, then Tarmac commits if Jenkins succeeds02:28
lifelessyou may not know this, but PQM can get its queue from LP02:29
lifelessor could before the code team destroy the API :)02:30
lifelessthere is something funky with that layering02:30
lifelessI need to reset my whiteboard to make a case, i think02:30
wallyworlddo we know why TestArchiveWebservice.test_getAllPermissions_constant_query_count sometimes fails?02:33
wgrantwallyworld: It might depend on whether it's the first browser test in that process02:34
wgrantlifeless: Yeah, but Tarmac is at least less crufty and abandoned than PQM is today...02:35
bigjoolstarmac + jenkins is working extremely well for maas so far02:39
StevenKbigjools: Are the configs available so we can duplicate it?02:40
StevenKwallyworld: You fail. Again.02:50
StevenKOh, the failure wasn't your fault.02:51
StevenKCarry on02:51
wgrantThat subunit stream looks a little broken02:54
StevenKwgrant: I wonder if bug 1020439 is limited to those few branches02:54
_mup_Bug #1020439: AttributeError: 'NoneType' object has no attribute 'items'  in +preview-diff <code-review> <oops> <Launchpad itself:Triaged> < https://launchpad.net/bugs/1020439 >02:54
wgrantStevenK: Yes02:54
StevenKOh, they have no merge_diff ?02:54
wgrantSo assign to you and close02:54
wgrantOh oops, wrong bug02:54
* wgrant -> lunch02:55
StevenKHelpful much02:55
wgrantIt may be the same, but I can't remember02:55
wgrantAnd am on my way out02:55
StevenKwgrant: It still happens02:55
StevenKwget a URL from the OOPS gives 0002:55
StevenKThe MP exists, and has a diff02:58
LPCIBotProject devel build #1096: STILL FAILING in 1 hr 46 min: https://lpci.wedontsleep.org/job/devel/1096/03:01
* StevenK prods Jenkins03:02
StevenKHm, it misreports the failure, too :-(03:04
* wallyworld has an appointment with the tax man :-(03:39
StevenKwallyworld: To pay him?03:39
wallyworldno, to figure out some of my shit for last year03:41
nigelbDon't you pay him for that? :)03:56
lifelessWe don't pay him for his shit.03:57
lifelessThat would just be wrong wrong wrong03:57
StevenKwgrant: Halp?04:48
StevenKwgrant: bug 1020439 is triggered when previewdiff.diff.diffstat = None and I'm trying to figure out how to not make lazr.restful eat a bullet04:49
_mup_Bug #1020439: AttributeError: 'NoneType' object has no attribute 'items'  in +preview-diff <code-review> <oops> <Launchpad itself:Triaged> < https://launchpad.net/bugs/1020439 >04:49
lifelessStevenK: does lazr.restful want diff to never be None ?04:52
StevenKlifeless: Where is that defined?04:52
lifelessthe schema for the interface04:53
lifelessI'm wagging here, of course04:53
LPCIBotProject devel build #1097: STILL FAILING in 1 hr 46 min: https://lpci.wedontsleep.org/job/devel/1097/04:53
StevenKJenkins, you make me very sad.04:53
StevenKlifeless: readonly=True, I'm not sure what the other defaults are04:53
lifelessme neither04:54
StevenKThey are not set to required=True04:54
StevenKSo I guess they're False04:54
StevenK_StringException: lost connection during test 'lp/registry/javascript/tests/test_milestone_creation'04:56
lifelessStevenK: so - marshaller04:56
lifelessStevenK: look at the marshallers available, I'll bet that its one with a bug in it04:57
StevenK<class 'lazr.restful.marshallers.DictFieldMarshaller'>04:58
lifelessStevenK: so a few lines up05:00
lifeless        marshaller = getMultiAdapter((field, self.request), IFieldMarshaller)05:00
lifelessthat suggests that that is going off the rails, to me at least05:00
StevenKlifeless: How?05:01
StevenKThe field is declared as an exported dict05:01
StevenKAnd it's attempting to unmarshall the contents05:01
StevenKIt just doesn't deal with None at all05:01
StevenKRight, if value is None: return {} works nicely05:03
StevenK    def _get_diffstat(self):05:03
StevenK        if self._diffstat is None:05:03
StevenK            return None05:03
lifelesshow is that different to05:04
lifelessreturn self._diffstat ?05:04
StevenKBelow that it returns a dict05:04
StevenKFantastic, I don't need to touch lazr.restful05:08
StevenKlifeless: So, can you look at the last Jenkins build?05:12
StevenKlifeless: I have this inkling there is something screwy with the subunit stream05:12
lifelessurl me up05:13
StevenKlifeless: https://lpci.wedontsleep.org/job/devel/lastBuild/05:13
lifelessERROR: cannot verify lpci.wedontsleep.org's certificate, issued by `/C=IL/O=StartCom Ltd./OU=Secure Digital Certificate Signing/CN=StartCom Class 1 Primary Intermediate Server CA':05:18
lifeless  Issued certificate has expired.05:18
StevenKWe should fix that by moving to prase05:21
* StevenK bangs that drum a little harder05:21
lifelessStevenK: ok, so tricks:05:23
lifelesscat consoleText| subunit-stats | less05:23
lifelesspage down till you see05:23
lifelessStarting up the container...05:23
lifelessafter that you should see nothing as subunit will be eating it05:23
lifelesswhen you start seeing something, its because subunit thinks a test prior to it has not finished05:24
lifelessand is in pass-through mode on everything other than a terminal for that test.05:24
lifelesstest: lib/lp/soyuz/stories/ppa/xx-private-ppas.txt05:24
lifelessis the first output05:24
lifelessuse that to seek in the raw stream to debug05:24
lifelessless consoleText05:24
lifeless / lib/lp/soyuz/stories/ppa/xx-private-ppas.txt05:24
lifelessand work back:05:24
lifelesstest: Could not communicate with subprocess05:25
lifelesstime: 2012-09-27 04:39:15.455646Z05:25
lifelesstest: lib/lp/soyuz/stories/ppa/xx-private-ppas.txt05:25
lifeless^ thats invalid yo05:25
StevenKCould not communicate with subprocess is zope.testing garbage, isn't it?05:26
lifelessyes, its odd that it says 'test: Could not communicate with subprocess'05:26
lifelesstheres no immediate evidence of interleaving going on05:27
lifelessit *could* be a race condition with gc or something05:27
lifelessor just bad code for handling that situation05:27
lifelessthere is a reporter thingy in zope.testing05:27
lifelessit has a special class for subunit05:27
lifelessthat class may have a bug05:27
lifelesswhere it reports this situation via the literal 'test: Could not communicate with subprocess\n'05:28
lifelesswhich would be a bug.05:28
StevenKThen why doesn't buildbot see that :-(05:28
lifelessit may not be encountering 'Could not communicate with subprocess'05:28
lifelessor it may be one of the things gary keeps posting a bug link to05:28
lifelessStevenK: s    def error_with_banner(self, message):05:29
lifeless        self._emit_fake_test(message, self.TAG_ERROR_WITH_BANNER)05:30
StevenKThat does seem a little bit like crack, yes.05:31
lifelesswhich calls startTest(test), _emit_tag(tag), addSuccess(test)05:31
lifelessso, that should be ok, but..05:31
lifelessand in fact..05:32
lifelesstest: Could not communicate with subprocess05:32
lifeless is line 10889005:32
lifelesssuccessful: Could not communicate with subprocess05:32
lifelessis 10899605:32
StevenKsubunit doesn't like tests with whitespace?05:33
lifelessso there is an interleaving problem05:33
lifelesssubunit doesn't care about that05:33
lifelessopen consoleText with vim05:33
lifelesssomething wrote, to the same fd, a tonne of other test info between the three lines of that self._emit_fake_test function05:34
StevenKwallyworld, wgrant: https://code.launchpad.net/~stevenk/launchpad/preview-diff-none-type/+merge/12660105:34
StevenKlifeless: Yeah, the line numbers are staggeringly different05:35
lifelessStevenK: part of the issue here is that you're getting the demultiplexed stream out of testr05:36
StevenKtestr run --parallel --subunit --full-results --concurrency=8 | subunit2junitxml -f -o test-results.xml05:36
lifelesstestr is running 8 streams on separate fd's and demultiplexing them.05:37
lifelessits possibly a bug in your testtools version05:37
lifelesswhat testtools and subunit versions are you running ?05:37
StevenKii  python-testtools                0.9.14+bzr267~ppa37~precise1    Extensions to the Python unittest library05:38
StevenKii  subunit                         0.0.8+bzr176~ppa126~precise1    command line tools for processing Subunit streams05:38
StevenKBoth from testing-cabal05:38
lifelessI have to run, cynthia time05:39
lifelessdrop the subunit2junitxml -f -o test-results.xml for now05:39
lifelessand the --full-results and the --subunit05:39
nigelbargh, sorry!05:39
lifelesstestr will write its output to .testrepository, the jenkins fs explorer thingy will let us pull the stream down and see what it gets05:40
lifelessI will return in ~2 hours05:40
StevenKWhich is how long the build will take05:40
wgrantStevenK: Did the build look successful other than the dead runner and corrupt stream?06:18
StevenKwgrant: Mostly06:19
StevenKI'm doing what lifeless' suggested, so we'll see06:19
StevenKBut since testr likes hiding things, we have no output at all06:20
wgrantStevenK: Tail the stream06:20
wgrantIt's in ~/.testrepository06:20
wgrantIt'll be tmpFOO06:20
StevenKwgrant: Can haz review?06:21
wgrantNot ~/, but the tree06:21
StevenKYes, I found it06:22
StevenK-rw------- 1 jenkins jenkins 3.4M Sep 27 06:22 .testrepository/tmpTDk8xw06:22
wgrantStevenK: Nothing else relies on diffstat being None sometimes?06:23
StevenKHm, maybe06:24
StevenKwgrant: Good catch. http://pastebin.ubuntu.com/1229847/06:28
wgrantThis is interesting06:28
wgrantI think our bugtracker may be sentient06:29
wgrantLook at the first result06:29
wgrantFirst thought was "these results are terrible". Open first result, "Cannot search for identifier containing underscores... well I can search for it but the results aren't any good."06:30
StevenKwgrant: I can't see any other code that relies on diffstat being None06:31
wgrantDid you consider making lazr.restful deal with it?06:31
StevenKI did, and then decided it was a case of GIGO06:32
wgrantWell, we allow object references to be nullable06:32
wgrantWhy not dicts?06:32
StevenKLike you say, it could be fixed in lazr.restful's marshaller. But I decided that fixing LP was a little easier06:34
StevenKAnd it didn't involve working out how to run its tests and then spinning a new release, uploading to pypi and download-cache and then making an LP branch anyway06:35
StevenKIf you strongly object then I can do that on Tuesday06:37
wgrantStevenK: It's not that hard; wallyworld did it yesterday06:39
wallyworld__and finished the job today06:40
StevenKI know it's not hard. It's easier to just change LP.06:40
StevenKwallyworld__: I don't think lp:lazr.restful contains your changes07:06
wallyworld__hmm. i'm sure i merged, let me check07:06
StevenKHmm, yeah, revision 20007:07
wallyworld__ah, merged restfulclient07:07
wallyworld__forgot restful07:07
wallyworld__let me do it now07:07
StevenKThe version.txt in the tree and pypi did not match07:08
StevenKAnd I didn't want to unfix your carefully fixed bug07:08
wallyworld__StevenK: done, sorry07:09
wgrantStevenK: Why isn't None None?07:16
wgrantNone shouldn't be {}07:16
wgrantIt should be None07:16
StevenKWell, it's better than what we have currently, which is AttributeError07:16
wgrantExposing an unstable interface in lazr.restful is not better than AttributeError :)07:17
wallyworld__wgrant: StevenK: i had to reboot again after another unity crash and now lp.dev won't load in ff, but loads in chrome. "The connection was interrupted". any idea?07:19
StevenKStop stabbing zope while loading it in Firefox?07:20
wallyworld__i have no knife07:20
wgrantwallyworld__: Tried restarting Apache? That sometimes indicates it's SSL listening on a non-SSL port or vice-versa07:20
wallyworld__wgrant: yeah, tried that07:20
StevenKwgrant: Diff updated07:21
wgrantStevenK: Have you tried this with lazr.restfulclient?07:22
* StevenK de-prams his own toys07:22
StevenKwgrant: No, I haven't07:23
LPCIBotProject devel build #1098: STILL FAILING in 1 hr 45 min: https://lpci.wedontsleep.org/job/devel/1098/07:27
StevenKThat isn't fun.07:28
adeuringgood morning07:52
=== rvba is now known as Guest78352
=== almaisan-away is now known as al-maisan
=== _rvba is now known as rvba
=== al-maisan is now known as almaisan-away
=== gary_poster|away is now known as gary_poster
=== matsubara-afk is now known as matsubara
jtvSay... I don't have a working launchpad setup at the moment.  Would anyone be able to test & land this branch for me?  https://code.launchpad.net/~jtv/launchpad/format-relative-imports/+merge/12487813:17
=== Guest32361 is now known as slank
=== slank is now known as Guest2828
stubjtv: sending via ec213:53
jtvOh, branch.  Thanks!13:53
stubresponsive r us13:53
tumbleweedstub: talking of responsiveness, have a moment to eyeball https://code.launchpad.net/~stefanor/launchpad/edit-packagesets/+merge/124555 ?14:04
stubtumbleweed: done14:12
tumbleweedstub: thanks14:13
=== jcsackett changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: jcsackett | Firefighting: - | Critical bugs: ~300
=== slank is now known as Guest55818
adeuringjcsackett: thanks for your review!16:29
* adeuring nearly forgot the MP16:29
jcsackettadeuring: you're welcome.16:30
=== matsubara is now known as matsubara-lunch
=== matsubara-lunch is now known as matsubara
lifelesssinzui: https://bugs.launchpad.net/launchpad/+bug/736010 is affecting openstack18:28
_mup_Bug #736010: Milestone:+index timeouts <timeout> <Launchpad itself:Triaged> < https://launchpad.net/bugs/736010 >18:28
lifelesssinzui: wondering if one of your timeout demolishing machines could cast an eye over it.18:29
abentleyderyck: I'm getting a test failure on stable.  Both test and code are old.  Can you reproduce?  http://pastebin.ubuntu.com/1230926/18:30
deryckabentley, let me see....18:32
lifelesssinzui: oh, I see ttx is in #launchpad talking abou tit18:35
sinzuilifeless, We have discussed that bug a lot. We think a lot of work is needed t fix this issue. This is a design problem, not a a bad query problem18:42
deryckabentley, I don't get that failure.18:44
lifelesssinzui: Why do you say its a design problem ?18:44
lifelesssinzui: I see 2000ms of time between query 66 and 67 in https://oops.canonical.com/oops/?oopsid=OOPS-c3db3abe60cbad08100ecb38fcac389a#statementlog18:45
abentleyderyck: Wacky.  It looks legit to me, but it also doesn't seem to happen on ec2.18:45
derycksorry to shout, fat fingers :)18:46
deryckI wonder what's different for you that you see it?18:46
sinzuilifeless, the page only works if you can display all milestones and blueprints. It does not scale. We can improve the queries, but it still will not scale for large projects. I think a proper fix is to load just the crucial/first items, then use ajax to complete the listings, then do the analysis.18:47
abentleyderyck: Maybe python version.  Are you on quantal 2.7.3?18:47
lifelesssinzui: I am confused; the OOPS doesn't show queries as a problem at all.18:47
lifelesssinzui: Are we talking about the same issue ?18:47
deryckabentley, ah, no.  I still have upgraded actually. :(18:48
deryckabentley, but I am on 2.7.318:48
sinzuiyes we are, This bug has been around a long time and making queries faster does not fix the issue18:48
lifelesssinzui: ok, so we're slightly in sync :) - I see bad python code / tal code.18:48
sinzuiYou can fix that query, but it want users the believe you have fixed the issue for there hundreds of items, we need to think about the solution differently18:49
lifelesssinzui: I agree that doing latency hiding and incremental loading would be better.18:49
sinzuiLate evaulation that because the page relies on bug and blueprint rendering rules18:49
lifelesssinzui: but is there not a low hanging just-be-more-efficient step to get Ubuntu and openstack unstuck ?18:49
lifelesssinzui: its not late evaluation, only 68 queries on the page before it timed out .18:49
sinzuiI think we just want to pull the data and have the browser do the analysis...such as I do in the burndown charts I wrote18:50
lifelesssinzui: those charts are sweet ;)18:50
sinzuiI don't believe the issue is low hanging. I have landed more than a dozen speed ups in 2 years. so have you and William18:51
sinzuiI asks jcsackett to not leap into that bug given the tremendous failure rate18:51
lifelessloaded for me18:52
jcsackettlifeless, sinzui: timeout fixed.18:52
lifeless69 queries in 6.83 seconds18:52
lifelessjcsackett: how ?18:52
jcsackettas i said, was increasing the wrong hard_timeout rule.18:52
* jcsackett misread the flags.18:52
jcsackettthe flag was for ProjectMilestone; we just now fixed milestone18:52
jcsackettwell "fixed" ;-p18:52
sinzuilifeless, wgrant, jcsackett, and wallyworld__can discuss the effort to make that page properly fast for all project sizes in a few hours18:53
lifelesssinzui: my curiosity is piqued18:54
lifelesssinzui: I agree on the long term18:54
lifelessbut I'm wondering why the short term is so poor18:54
sinzui1 day of effort to fix one bug is high certainty to deliver value18:55
sinzui5 days to fix 5 bugs is less certain, but equal value18:55
sinzui10 days to fix 1 bug is even less certain to deliver less value.18:56
sinzuilifeless, we are a "machine" because we are taking the most certain bugs first. I asked jcsackett to not work on that one yet because there is more risk involved.18:57
* sinzui hoped to fix another escalated bug before having to do feature analysis to fix criticals18:57
lifelesssinzui: all cool18:57
lifelesssinzui: I meant, what about the code makes this one so poor18:58
lifelesssinzui: not about your team or your choices ;)18:58
sinzuiwe often choose to make pages do less, or batch. milestones do need to show all items. The tales calls are an embarrassment. I choose to reuse code to keep presentation consistent, but the adaptions are costly.19:00
sinzuilifeless, the analysis is costly in Python I think. I know I can do it in JS now.19:02
lifelesssinzui: I'm getting a profile19:05
lifelessjust sneaked in: 3378720 function calls (3296251 primitive calls) in 19.845 CPU seconds19:06
lifelesssinzui: there are some low hanging fruit that might make a lot of things better (or might be profile artifacts)19:12
lifeless     5134    0.810    0.000    0.979    0.000 enumcol.py:39(__init__)19:12
lifeless^ 4% of total runtime, I believe wgrant has complained about enumcol performance before19:12
lifelessa few more like that19:13
sinzuithis page might break now that there are private bluerprints too. milestones do the private bugs query then cache the permissions, but I did not see such as change land for blueprints19:14
lifelesssinzui: the ubuntu timing out page is only 140k in size19:19
abentleyjcsackett: I have a sequence of four branches that need review.  Are you up for it?20:24
jcsackettabentley: if said sequence starts with new-tests, i'm already looking at it.20:24
abentleyjcsackett: Excellent.  Continues with https://code.launchpad.net/~abentley/launchpad/storm-sprint-queries/+merge/126770 https://code.launchpad.net/~abentley/launchpad/specification-cleanup/+merge/126789 https://code.launchpad.net/~abentley/launchpad/hide-sprint-blueprints/+merge/12679220:25
jcsackettabentley: cool. i can get 'em all.20:27
sinzuijcsackett, do you have time to review https://code.launchpad.net/~sinzui/launchpad/email-authentication-errors/+merge/12680121:05
jcsackettsinzui: sure, you're in the queue.21:06
jcsackettstill looking through abentley's series.21:06
sinzuiabentley, rick_h_. was the privacy banner or information portlets updated in the last few days21:12
sinzuiJS is broken on pages that show the privacy banne like bugs and archives21:12
abentleysinzui: Yes, rick_h_ changed the banner code yesterday IIRC.21:13
sinzuifab, maybe the diff will help me find the problem21:13
abentleysinzui: (rick_h is away)21:14
sinzuiokay, diff it will be21:14
jcsackettabentley: i'm through your series of branches. r=me on all but one which has a few questions.21:35
abentleyjcsackett: I've replied.21:35
abentleyjcsackett: Thanks!21:35
jcsackettabentley: missed both things you pointed out. given those, r=me on that one as well.21:38
* jcsackett moves on to sinzui's.21:38
=== jcsackett changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: - | Firefighting: - | Critical bugs: ~300
sinzuiwallyworld__, If your waking: read this to learn what I have learned so far https://bugs.launchpad.net/launchpad/+bug/105771421:39
_mup_Bug #1057714: javascript is broken on pages that show the privacy banner <bugs> <javascript> <regression> <Launchpad itself:Triaged> < https://launchpad.net/bugs/1057714 >21:39
wallyworld__sinzui: haven't read that bug but i think it may be a dupe of mine from yesterday, for the mp ypu approved21:40
wallyworld__mine is bug 105724821:40
_mup_Bug #1057248: javascript error on bugtask page <javascript> <regression> <Launchpad itself:In Progress by wallyworld> < https://launchpad.net/bugs/1057248 >21:40
sinzuiwallyworld__, does this look familiar: http://pastebin.ubuntu.com/1231240/21:40
sinzuiThis is the change, but I do not know why info_type is not defined in the privacy module21:41
wallyworld__sinzui: yes. i changed privacy.js and information_type.js21:41
wallyworld__there were vrious issues - 'throw' was misspelt, lots of lint21:41
wallyworld__plus the info_type issue21:42
wallyworld__landing my branch now21:42
lifelesswgrant: enumcol would benefit from a cache I suspect21:42
lifelesswgrant: one of the rare times I'll say that21:42
wallyworld__we can deploy this morning21:42
sinzuiwe can run the yui layer locally, then lp-land21:42
wallyworld__sinzui: yes, doing that now21:42
sinzuisorry wallyworld__I remember the diff, but I did not read all the details int he bug. I suck21:44
jcsackettsinzui: r=me on your branch.21:45
sinzuiwallyworld__, I was rushing to review branches because I had a day trip to look at houses21:45
sinzuithank you jcsackett21:45
cjwatsonwgrant: We've flushed nearly all the queue now following beta-2, and it all seemed pretty snappy; no timeouts in 80+ accepts.  At this point I'm confident with going ahead and removing the queue script.  Is there any remaining review you feel the need to do on https://code.launchpad.net/~cjwatson/launchpad/remove-queue-tool/+merge/114464 ?22:54
wgrantcjwatson: Let me look23:01
wgrantlifeless: Hmm23:03
wgrantlifeless: I might try to profile it a bit harder now that someone other than me thinks it might be a problem.23:03
rick_h_wallyworld__: man, so sorry. Just thought I added unused stuff. Tests passed, etc. I shouldn't have tried to rush that before I left23:05
wgrantcjwatson: r=me. Are you also going to remove the feature flag check?23:06
wallyworld__rick_h_: no problem. shit happens :-)23:07
rick_h_wallyworld__: I owe you one. broke it apart and tried to stage it in nice simple bits that didn't get too involved and boom23:08
wallyworld__rick_h_: we may need to look at the events - seems there are 2 sets (show/hide banner, info type public/private) that do the same thing. may be able to be refactored to just one set23:09
rick_h_wallyworld__: well so I can explain that23:10
wallyworld__i didn't get into it too far23:10
cjwatsonwgrant: That's going through EC2 at the moment (https://code.launchpad.net/~cjwatson/launchpad/pabj-remove-feature-flag/+merge/126090)23:10
rick_h_because the filbug.js uses the privacy banner directly with a custom message, you need to be able to talk to the privacy banner with a custom message23:10
wgrantcjwatson: Ah, great23:10
wgrantI have a fair bit of MP mail; haven't quite caught up this morning yet23:10
cjwatsonYep.  Thanks.  I look forward to my giant LoC credit23:11
rick_h_wallyworld__: but the normal use case is that someone changes the information_type widget, which fires the information_type:changed, and files information_type:is_public/private and the banner will auto show/hide based on that event23:11
cjwatsonAnd I haven't even removed delayed copies yet ...23:11
rick_h_wallyworld__: so there's really two use cases and almost should be a different banner, since privacy and security are sharing code tbh23:11
lifelesswgrant: there is a kcachegrind file for you23:11
wallyworld__rick_h_: ok, thanks for the explanation23:11
wallyworld__my main aim was to add the missing tests and get the fix out23:11
rick_h_wallyworld__: and in the next branch the banners will render themselves vs the html being in the .pt file, so the events will be used as ways to say "need a banner html setup here"23:12
wallyworld__cool :-)23:12
rick_h_wallyworld__: right, understand. and it's not 100% tested because it's part 1 of 4 and in general this was 'new unused' code since nothing is firing the events yet23:12
wallyworld__rick_h_: one thing. i had to move the info_type namespace variable from the top of the module or else it was undefined23:13
wallyworld__i don't know why23:13
rick_h_wallyworld__: so like I said appreciate it. Been offline most of the day on vaca so bummed to log in and see I did wrong for you guys23:13
rick_h_wallyworld__: hmmm, let me look23:13
wgrantlifeless: But it's from qastaging, so it's not completely useful23:14
wgrantNot useless, though23:14
lifelessnot useless23:14
rick_h_wallyworld__: ok that's strange. I'll check it out when I get to the follow up branch when I get back23:15
wgrantrick_h_: Missing context here, but we only recently moved the banners *into* the HTML, to avoid them popping into the page later on once the JS loads.23:15
wallyworld__rick_h_: thanks. i didn't get into the root cause. i'd love to know why23:15
rick_h_wallyworld__: nothing looks odd, but running on 3hrs sleep after driving 600+ miles overnight23:15
wallyworld__rick_h_: where did you drive from/to?23:16
rick_h_wallyworld__: https://maps.google.com/maps?saddr=clarkston,+mi&daddr=Charlottesville,+VA&hl=en&sll=37.6,-95.665&sspn=38.593229,86.396484&geocode=FRAWjAIdYh8H-ymTC47xX5ckiDGKYHxzNWKthw%3BFfpHRAIdeopS-ymPpFDqLYaziTH8dIvDlvCGkA&oq=charlottesvi&mra=ls&t=m&z=623:16
wallyworld__looks like a nice drive23:17
rick_h_wgrant: yea, I'll hopefully be able to take care of that still. The big thing is we need banners on more places and it takes a bunch of code to wire it up it seems. Trying to split it into contained parts23:17
rick_h_wgrant: for instance trying to not have the privacy banner need to know about the information type, LP.cache locations, etc.23:18
rick_h_wgrant: but thanks for the heads up. I didn't realize it was moved due to a flash effect so good to keep an eye on23:18
rick_h_wallyworld__: yea, well with a 2yr old those overnight drives still the best so he sleeps through most of it. Thanksfully we also had the lion king movie soundtrack for backup lol23:19
wallyworld__ah, those were the days. long gone for me now23:19
wallyworld__rick_h_: wgrant still listens to the Lion King for his beddy byes time23:19
lifelesswgrant: the other thing is there is a huge amount of content being re-rendered23:21
lifelesswgrant: a simple static result cache for rendering of person priority importance status might help a great deal.23:22
wgrantWe can possibly just also make that not terrible23:22
lifelessor do both23:22
lifelessevaluating the same template with the same parameters in the same request to the same bytes seems non horrible to me23:23
wgrantSays he who removed template fragment caching :P23:23
wgrant(yes, yes, it was the wrong way to cache, but blah)23:23
lifelessI'm not against caches /per se/23:24
wgrantwallyworld__: qas is updated, if you want to check out the JS stuff23:57
wgrantNot sure why it's [testfix]23:57
wallyworld__finally, been waiting23:57
wallyworld__wgrant: because db bb was borked and i needed to get it landed23:57
wallyworld__we should fix that23:57
wgrantwallyworld__: In that situation you should force the build and wait 5 minutes23:58
wgrant[testfix] skips all QA23:58

Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!