/srv/irclogs.ubuntu.com/2010/11/26/#launchpad-dev.txt

thumperwhere are the soupmatchers?00:32
* thumper pulls again00:33
lifelesslp:00:33
lifeless?00:33
wgrantlifeless: Did your parallel-supporting DB fixture ever land?00:37
lifelessno00:37
wgrant:(00:37
lifelessfeel free to hack on it00:37
wgrantlifeless: Hm, that's a bit of an odd one.00:41
wgrantThanks.00:41
lifelesswgrant: oh, its easy00:41
lifelesswgrant: canonical.lp.dbhost00:41
lifelessetc00:41
wgrantProbably, yes.00:41
lifelessare redudant fugly shit00:41
wgrantIt's just not what I would have expected.00:41
lifelessthe script entry point isn't resetting it quite right00:41
lifelesswith the result that fti.py attempts to work on launchpad_dev00:42
lifelessrather than the supplied -d launchpad_test_template or whatever00:42
wgrantYep.00:42
thumpergah00:59
thumperI have a test that failed on ec2, but passes locally00:59
thumperWTF?00:59
thumperI've done a make schema00:59
thumperbut still weird shit ...00:59
lifelessisolation error?01:00
lifelesswhat test01:00
thumperp.code.browser.tests.test_sourcepackagerecipe.TestSourcePackageRecipeView.test_request_builds_page01:01
thumperwit an l at the start01:01
* thumper fires up make run01:03
pooliethumper: hi?01:04
thumperpoolie: hey01:04
poolieshall we have a chat if you're back?01:07
thumpersure01:08
* thumper shoves it back through ec2 again01:10
ccxCZtoo tired to write the mail now, here is link if anybody wants to play with it http://webprojekty.cz/ccx/trac/wiki/lpbot01:38
thumperccxCZ: a link to what?01:49
ccxCZirc notification bot for launchpad projects01:53
ccxCZanyway gn, I have to go01:53
lifelesswgrant: databasefixture and librarian branches are updated and pushed, if you're interested01:54
=== ccxCZ- is now known as ccxCZ
lifeless=== Top 10 Time Out Counts by Page ID ===02:16
lifeless    Hard / Soft  Page ID02:16
lifeless     134 / 5092  Archive:+index02:16
lifeless      70 /  240  BugTask:+index02:16
lifeless      24 /  249  Distribution:+bugs02:16
lifeless      20 /  118  ProjectGroupSet:CollectionResource:#project_groups02:16
lifeless      11 /  265  POFile:+translate02:16
lifeless      11 /   13  DistroArchSeries:+index02:16
lifeless      11 /    1  ProjectGroup:+milestones02:16
lifeless       7 /   35  Milestone:+index02:16
lifeless       6 /  306  Distribution:+bugtarget-portlet-bugfilters-stats02:16
lifeless       5 /    9  Person:+bugs02:17
wgrantOw :(02:17
wgrantI must get back to Archive:+index.02:17
lifelesswgrant: that would be the bomb02:29
wallyworldthumper: ping03:48
jtvpoolie: a thought… would it save EC2 startup time (or server load) if we could use a virtual-machine image that had some fixed version of our source tree or repo pre-installed, so that it only needs to download the updates?04:05
lifelessjtv: thats what we have, isn't it ?04:07
jtvoh :)04:07
thumperwallyworld: hey04:41
thumperwallyworld: it is time for pizza and movies :)04:41
wallyworldjust wanted to confirm that you want me to look at bug 670452 next04:41
_mup_Bug #670452: Hard to find related branches when composing recipe <recipe> <Launchpad Bazaar Integration:Triaged> <https://launchpad.net/bugs/670452>04:42
thumperwallyworld: aye04:42
wallyworldcool. i'll take a look once i finish seeing if i can improve that oops reporting test04:43
wallyworldhaveagoodweekend04:43
* wgrant theorises that HTML forms were designed as a cruel joke.04:50
jtvwgrant: careful—that line of reasoning leads down a dangerous path to paranoia, misanthropy and most probably, truth.04:56
jtvGeneral observation about software stuff, not a comment about HTML forms specifically.04:57
wgrantHeh.04:58
jtvThink Mahabharata-style stripping of illusions.04:58
jtvI could say something about pills in The Matrix if I could bear the thought of that crappy film.04:59
jtvBesides, I'm regularly informed through emails from helpful strangers that given the choice, there is no good reason to spurn the blue pill.04:59
jtv05:03
jtvYou're still reading the Wikipedia entry on Mahabharata, aren't you?05:03
wgrantI was debating whether or not to look it up.05:03
jtvIf you were debating it on your own, you probably want prozac, not viagra.  My bad.05:04
StevenKjtv: Hey, the first one was awesome. It was just the other two that made me cringe05:04
jtvStevenK: the first pill, the first book of the Mahabharata, or the first wgrant personality in the debate?05:05
StevenKjtv: I'm sorry, the first Matrix movie05:07
jtvAh that one.  Did one thing for me: realize how precious my time was.  Only time I ever walked out of a cinema shortly after the film started.05:08
jtvI've met other people who did that.05:08
wgrantI only saw them a few months ago. I quite liked the first one.05:08
spmjtv: I had a similar experience with my wife, sister and sisters friend at a production of Les Mis. Yay. Play is over. We can go home now. "Um Steve? no. That's just the first half, this is the intermission" AAAAAAAAAAAAAAAAAAAAA05:10
jtvspm: hang on, still laughing05:11
spm:-)05:11
jtvHah.  Whew.  That's better.05:11
spmSo I went home, ladies, enjoy the play. Ring me when you want to be picked up. :-D05:11
spmBored out of my tiny little mind. Some lady was whing about doing it tough. Some dudes with guns were whatever. Bored.05:12
spmBored against the backdrop of the French Revolution. But still bored.05:12
jtvSaw one cartoon about this kind of thing… spectator #1: "man but this is boring.  I should have brought a good book!"05:13
jtvSpectator #2: "that's why I always read the reviews first."  Goes back to reading Tolstoy.05:13
spmbwhahahaa05:13
pooliejtv, hi, it could05:18
jtvpoolie: lifeless noted that we already seem to do this.05:22
pooliecould be good to know what fraction of total time is spent pulling that source05:22
pooliei suspect it's not very high05:22
henningeOne of the current test failures in devel is strange - it reports a "test with error" but the test does not output a traceback or anything ...06:02
henningelooks like this: henning@dustpuppy:$ bin/test -vvcm lp.services.job.tests.test_runner -t TestJobCronScript.test_configures_oops_handler06:03
henningeRunning tests at level 106:03
henningeRunning canonical.testing.layers.LaunchpadZopelessLayer tests:06:03
henninge  Set up canonical.testing.layers.BaseLayer in 0.004 seconds.06:03
henninge  Set up canonical.testing.layers.ZopelessLayer in 9.778 seconds.06:03
henninge  Set up canonical.testing.layers.DatabaseLayer in 0.482 seconds.06:03
henninge  Set up canonical.testing.layers.LibrarianLayer in 10.697 seconds.06:03
henninge  Set up canonical.testing.layers.MemcachedLayer in 0.112 seconds.06:03
henninge  Set up canonical.testing.layers.LaunchpadLayer in 0.000 seconds.06:03
henninge  Set up canonical.testing.layers.LaunchpadScriptLayer in 0.005 seconds.06:03
henninge  Set up canonical.testing.layers.LaunchpadZopelessLayer in 0.000 seconds.06:03
henninge  Running:06:03
henninge lp.services.job.tests.test_runner.TestJobCronScript.test_configures_oops_handler06:03
henninge  Ran 1 tests with 0 failures and 1 errors in 0.848 seconds.06:03
henningeTearing down left over layers:06:03
henninge  Tear down canonical.testing.layers.LaunchpadZopelessLayer in 0.000 seconds.06:03
henninge  Tear down canonical.testing.layers.LaunchpadScriptLayer in 0.002 seconds.06:03
henninge  Tear down canonical.testing.layers.ZopelessLayer ... not supported06:03
henninge  Tear down canonical.testing.layers.LaunchpadLayer in 0.000 seconds.06:03
henninge  Tear down canonical.testing.layers.LibrarianLayer in 0.003 seconds.06:03
henninge  Tear down canonical.testing.layers.MemcachedLayer in 0.101 seconds.06:03
henninge  Tear down canonical.testing.layers.DatabaseLayer in 0.236 seconds.06:03
henninge  Tear down canonical.testing.layers.BaseLayer in 0.001 seconds.06:03
henningeTests with errors:06:03
StevenKhenninge: Pastebin?06:03
henninge   lp.services.job.tests.test_runner.TestJobCronScript.test_configures_oops_handler (subunit.RemotedTestCase)06:03
lifelessis that test a twisted test?06:03
henningeoops06:03
lifelesshenninge: does it have run_tests_with = ?06:04
henningesorry, meant to paste http://paste.ubuntu.com/536572/06:04
henningelifeless: I don't konw what that is but that string does not appear in the file .06:04
henningelooks like this http://paste.ubuntu.com/536573/06:05
lifelesshenninge: what about ZopeTestInSubProcess06:06
henningeactually, I just realized that that I pasted the wrong output but it's similar.06:07
lifelessah, interesting06:08
henningelifeless: no, that neither06:09
lifelessthis might be allenaps change to use unicode in subunit06:09
henningeaha06:10
* henninge tries with that reverted06:12
henningenope,same result :(06:14
lifelesswgrant: we're turning the publicrestrictedlibrarian back on06:19
lifelessbecause it seems to work in tests06:20
wgrantlifeless: Yay.06:20
wgrantBut why? :/06:20
wgrantWas Apache fixed?06:20
lifelesswgrant: can you give it a thorough spin06:20
lifelesswgrant: To quote spm, FIIK.06:20
wgrantHeh.06:20
wgrantSure.06:20
wgrantSay when.06:20
henningeso, r11980 seems to be the culprit for the current test failure06:21
henningeat least for two of them ...06:21
henningethe ones that don't produce output06:21
spmwgrant: no, no changes in apache. I did a moderate amount of local faffing around to determine that the proposed changes shouldn't make a difference (apache ignores those bits), ie it should "Just Work" as is. So we switched it on, tested, off. and it worked. ie FIIK.06:23
lifelesswgrant: oh, it was live at '19:21 < lifeless> wgrant: can you give it a thorough spin06:23
lifeless'06:23
wgrantlifeless: Still broken.06:24
wgrantCan you see https://launchpad.net/~soyuz-team/+archive/ppa/+build/1334368?06:25
wgrantThe build log link there.06:25
wgrantAnd lots of other similar ones.06:25
StevenKThe page you are trying to view cannot be shown because it uses an invalid or unsupported form of compression.06:25
lifelessI'm not in soyuz-team06:26
lifelesswgrant: gimme a url06:26
wgrantlifeless: Neither am I, but I have a sub. I thought you might be a commercial admin.06:26
wgranthttps://i35262589.restricted.launchpadlibrarian.net/35262589/buildlog_ubuntu-karmic-i386.openproj_1.4-2%2Bpx1_FAILEDTOBUILD.txt.gz?token=b8e6f2f070469191af32ce99bb92fae806:26
wgrantStevenK: Right, it's the usual uncompressed 404, served with gzip headers.06:27
wgrantSo there are two levels of WTFery.06:27
StevenKHaha06:27
StevenKHTTP request sent, awaiting response... 404 Not Found06:28
StevenKAh yes06:28
lifelessspm: please turn it off again06:30
spmdone06:31
spmso how did it work for me/us? but not here?06:31
spmOh. its the 404?06:31
wgrantIt works for some files.06:31
lifelessspm: ok, so it *is* the 404s06:35
spmnot the files themselves, the 404 response?06:35
lifelesswgrant: does that same build log show properly now?06:35
StevenKRepeating the same wget still gives me a 40406:36
lifelessof course06:36
lifelessyou need the source url06:36
lifelesson the appservers06:36
lifelessthats what redirects or proxies06:36
StevenKYes, the build log now works06:36
wgrantlifeless: Proxying works fine.06:36
lifelesswgrant: whats the original url ?06:37
StevenKhttps://launchpad.net/~soyuz-team/+archive/ppa/+build/1334368/+files/buildlog_ubuntu-karmic-i386.openproj_1.4-2%2Bpx1_FAILEDTOBUILD.txt.gz ?06:38
lifelessI suspect url mangling06:39
lifelessI need to chat to spm briefly06:39
* spm questions the brief part06:39
StevenKBwahaha06:39
lifelessso I think its the %06:46
lifelesswgrant: whats the debian version string for that, unmangled06:51
StevenK1.4-2+px106:51
lifelessthanks07:02
lifelessso, % urls are mangled by something in the request path07:03
lifelessthey are decoded07:03
lifelessand then passed on decoded?!07:03
* lifeless bets on apache07:03
lifelesswouldn't know a standard if it bit it07:03
StevenKlifeless: I was thinking that07:03
lifelesshttps://bugs.launchpad.net/launchpad-foundations/+bug/68167507:13
_mup_Bug #681675: content coding for build logs is done in apache rather than the librarian <Launchpad Foundations:New> <https://launchpad.net/bugs/681675>07:13
lifelessupdated https://bugs.launchpad.net/launchpad-foundations/+bug/67727007:13
_mup_Bug #677270: apache/squid  breaks restricted librarian on urls with percent encoded characters. <Launchpad Foundations:In Progress by canonical-losas> <https://launchpad.net/bugs/677270>07:13
henningeAnybody have a clue for me? This test is failing *even* when I revert to the revision of the last successful buildbot run.07:14
henningehttp://paste.ubuntu.com/536585/07:15
henningeIt start two jobs and both seem to fail on _pythonpath_. Is that a local error for me?07:15
henningeI am getting this on devel r1197907:16
StevenKSo does buildbot, it seems07:16
henningewhich is said "last known good" revision07:16
StevenKhenninge: If changing devel doesn't fix it, I'd blame an egg change?07:16
henningehm07:16
lifelesshenninge: 11980 is the twisted runner change, I was thinking it was to blame :)07:17
henningebut doesn't version.cfg control which version of an egg a revision is using?07:17
lifelessjml: ^07:17
wgranthenninge: You need to remake.07:17
lifelesshenninge: you have to run bin/buildout again07:17
henningelifeless: it is to blame for the other two failing test07:17
henningewgrant: make clean ; make build? Did that ...07:17
wgranthenninge: Hmm.07:18
* henninge tries bin/buildout07:18
lifelessthere's probably a missing dep on versions.cfg07:18
henningeno change07:19
lifelessbuh07:19
* StevenK stares at Hudson: "took 5 min 5 sec"07:21
StevenKHowever, r11980 is devel #250 on Hudson, and that didn't fail any tests07:21
* henninge tries r11980 locally again07:23
* henninge notices that db-devel passed revision 1000007:28
StevenKorg.apache.commons.jelly.JellyTagException: jar:file:/var/run/hudson/war/WEB-INF/lib/hudson-core-1.377.jar!/hudson/model/AbstractBuild/sidepanel.jelly:34:42:  java.lang.OutOfMemoryError: Java heap space07:28
StevenKWheee!07:28
lifeless\//07:28
StevenKJava, you suck, and I hate you07:28
StevenKI think Hudson is now unhappy, since it now gives 503s07:29
StevenKAnd even /etc/init.d/hudson stop ... doesn't07:31
StevenKHeh07:33
StevenKRight, it does, it just takes 2 minutes? Srsly?07:33
lifelessyou were in swap07:34
StevenKGrah, it is 50 MiB into it07:34
StevenKHow is 1.5GiB not enough for Java?!07:34
lifelesshow much memory did you tell java it can use?07:35
StevenKWhatever the default is?07:35
lifelessyou might have been swapped out even if you weren't exceeding the machine memory in one proc07:35
StevenKIt's still 48MiB into swap with apache and java dead, so it isn't that07:38
StevenKThis machine still has a full Launchpad development environment on it, so that can't be helping07:38
StevenKAnd restarting Hudson means it has lost track of the build it was in the middle of07:41
* StevenK shoots the buildslave in the head07:41
LPCIBotProject parallel-test build (15): STILL FAILING in 34 sec: https://hudson.wedontsleep.org/job/parallel-test/15/07:47
StevenKlifeless: Do you want me a queue another parallel-test build?07:49
lifelessStevenK: ask me on the 3rd07:49
lifelessStevenK: of january07:49
lifelesssorry, the 4th07:50
StevenKBy which it would have landed?07:50
lifelessno, by which I'll be back at work07:51
spmlifeless: "by which I'll be back at work"? you haven't heard? we're replacing you with a perl script. I thought you knew. ??08:00
lifelessspm: never a win08:00
StevenK#!/usr/bin/perl -w08:00
lifelesssee08:00
StevenKwhile (1) { nag(); }08:00
lifelessminus win08:00
spmnah. -w is for wussies08:00
StevenKPerl coder for six years at previous job. Old habits die hard08:01
=== almaisan-away is now known as al-maisan
StevenKAnd there goes X in a screaming heap08:16
adeuringgood morning08:37
jmlhello09:06
jmlwho's fixing the build?09:07
jmlhenninge: hello09:17
jmlhenninge: have you submitted anything to fix the build?09:18
wgrantI'm confused.09:18
wgrantr10005 doesn't look like a testfix to me...09:18
henningejml: I hoped to find out what to do about the third test still failing09:18
henningenot much point in fixing two out of three tests09:19
jmlwgrant: what about it? (I haven't looked, trying to deal with the current failure)09:19
henningewgrant: no but then db-devel wasn't broken09:19
jmlhenninge: agreed. just wanted to check09:19
wgranthenninge: It's labelled as testfix.09:20
jmlhey look09:20
wgrantBut doesn't seem to have a test fix.09:20
jmlit's ZopeTestInSubProcess again09:20
jmlffs09:20
henningewgrant: sorry, jtv is just telling me off about it, too ...09:21
jtv"now now henninge, that's not nice"09:21
jtvthere, see?09:21
wgrantI think it might confuse the QA bot a bit, but I guess we'll see.09:21
henningewe'll see ...09:21
jmlI wonder what happened re upgrading subunit09:22
jmlI see we're at the "someone should do something about this" phase of discussion09:23
jmlok.09:23
wgrantjml: That's surely the best phase.09:23
jmlwgrant: I don't know, I'm a "someone else just did something about it" kind of guy.09:24
jmlhenninge: test_timeout fails for you locally, even if you revert to r11979?09:30
jmlwe should change these to be testtools RunTest thingies anyway09:35
lifelesspresumably they are in sub processes for a good reason.09:37
henningejml: yes, it does.09:38
jmllifeless: I mean, there should be a testtools subprocess RunTest thingy09:38
jmllifeless: rather than using a mixin with a run() method09:38
lifeless+109:38
jmlhenninge: ok. so something is very weird.09:39
jmlbecause a 'known good' build is breaking09:39
jmlI'm going to try to get more info on what the child process is doing09:40
jmlI *really* doubt r11980 has anything to do with these failures, fwiw.09:40
lifelessjml: Its plausible09:41
lifelessjml: if the reactor is in a surprising state09:41
lifelessit forks, it doesn't exec09:41
jmlhow can I get at the stdout & stderr of the child process?09:44
bigjoolshello09:45
* jml cheats09:46
jmllifeless: oh wait, it's not plausible at all09:48
lifelessjml: ok09:48
jmllifeless: because the test fails with r11979 too09:48
jmlalso fails when not run in a subprocess09:50
henningejml: to be clear: I could verify that the other two failing tests (*not* the timeout test) pass on 11979 but fail on 11980. The timeout test failure does not seem to be connected to that revision.09:55
henninges/verify/verify locally/09:56
jmlhenninge: do you have a plausible reason as to why?10:01
henningeno, I don't ... :(10:02
henningeactually ...10:02
jmlallenap: I don't think it's related to the unicode thing10:03
jmlhenninge: I suspect bad local state, actually10:03
StevenKjml: As well as inside the buildbot slaves?10:04
jmlStevenK: yes.10:04
henningejml: I don't deny that. Did you try it on your machine?10:04
henningejml: the only clue is this: http://paste.ubuntu.com/536585/10:04
StevenKI find that a little hard to believe. Can haz proof?10:04
=== jkakar_ is now known as jkakar
henningeThe test starts two jobs wich both fail on a missing _pythonpath_10:04
jmlStevenK: working on it.10:05
henningeBut I don't know how that can happen just for those two and not for everthing else.10:05
jmlthe clue is that reverting to about 20 revisions ago still has the failure10:05
henningegood point10:05
StevenKjml: Hudson might actually be your proof10:06
StevenKjml: Now that I think about it10:06
jmlalso, remember, the Makefile has been changed four times recently (11937, 11957, 11958 and 11979)10:07
jmlStevenK: in that it works?10:08
StevenKjml: Indeed10:08
StevenKjml: But the Hudson slaves are different to the buildbot slaves10:08
jmlnaturally10:08
jmlhow do we clean out the build on buildbot?10:10
StevenKI suspect we run make clean only10:10
jmlhow do we move from suspicion to certainty?10:11
StevenKWe channel mars, or someone who actually does more with buildbot than try and replace it10:12
* jml has to go10:16
bigjoolsrf-get and update-sourcecode doesn't want to fix the lack of a AsynchronousDeferredRunTestForBrokenTwisted module....11:06
allenapjml: I did find an interesting problem. The read side of the pipe in ZTISP was being opened rU. Locally that has caused tests to error, but with no output at all. Changing it to rb means that they succeed. I don't really know why, but perhaps this is related to the unicode thing too. I'm landing a branch now to revert the getreader() thing, and to open both ends of the pipe as binary.11:09
=== matsubara-afk is now known as matsubara
=== al-maisan is now known as almaisan-away
jmlbigjools: that sucks. will try to look into that after the build is fixed12:31
jmlallenap: cool. that fixes all three failing tests?12:31
bigjoolsjml: a make clean fixored it12:32
jmlbigjools: ahh good12:32
bigjoolsnot sure why I needed that though :(12:32
jmlbigjools: me neither. maybe I did something a bit stupid as I developing it.12:36
jmlit's not clear how to evolve Launchpad & an upstream in parallel12:36
jmlhmm12:38
jmlI wonder how I can make a script that downloads an openid-protected page12:38
allenapjml: I think it should fix them, but then I haven't been able to replicate the exact failures. I'm hoping :)12:40
bigjoolsjml: see utilities/pastebin12:41
bigjoolswell, the old one12:41
allenapjml: from faustian_pact import convenient_openid_wrapper12:41
jmlhmm. I don't think I have that one.12:41
bigjoolssure you do, you have bzr repo? :)12:41
jmlbigjools: faustian_pact is in Launchpad's history?12:42
bigjoolsmaybe not that one12:42
bigjoolsI can't see you facing allenap on IRC when you talk12:42
jmlyeah, they probably haven't ported that to kde yet12:43
jmldb_lp broken again12:44
jmlany volunteers?12:45
* bigjools 's kde sarcasm detector app goes off12:48
jmlallenap: test_timeout still fails for me with your branch, but the other two are fine.12:48
jmlhttps://bugs.launchpad.net/launchpad-foundations/+bug/68177013:01
_mup_Bug #681770: Failure in lp.services.job.tests.test_runner.TestTwistedJobRunner.test_timeout <Launchpad Foundations:New> <https://launchpad.net/bugs/681770>13:01
=== jml changed the topic of #launchpad-dev to: Launchpad Development Channel | Week 2 of 10.12 | PQM is open | firefighting: db-devel broken | https:/​/​dev.launchpad.net/ | Get the code: https:/​/​dev.launchpad.net/​Getting
jmljkakar_: hi13:34
jkakar_Hiya jml!13:36
=== jkakar_ is now known as jkakar
jmljkakar: how's things?13:37
jkakarjml: Going really well, thanks.  I'm sprinting in Barcelona, so it's been an exciting week.  Just got back from lunch.13:39
jkakarjml: How about you?13:39
jmljkakar: well. bit of a cold this week but otherwise fine. slowly mastering the art of the automobile13:40
jmljkakar: you coming to London any time soon?13:41
jml(also, yay sprinting)13:44
jkakarjml: Driving, oooh, fun times. :)13:47
jkakarjml: Will be in London in a couple weeks, yep.13:47
jmlall confirmed?13:48
bigjoolsbloody windmill13:57
marsbigjools, ?13:58
bigjoolsit hangs when I run it locally, which means my test run never completes13:58
marsah, not fun.  That is new, I don't think anyone has mentioned that yet13:59
marsdid Deryck kill the windmill suite?13:59
mars(yet?)13:59
bigjoolsnever mind Deryck, I want to kill it :)13:59
marsheh13:59
marsbigjools, I should ask: is this problem new?13:59
jmlspeaking of which...13:59
marsjml, your @skiptest decorator is the big win for us there13:59
jmlthe build is still broken, failing with a windmill test. I suspect a fix like http://pastebin.ubuntu.com/533852/ might be needed14:00
bigjoolsmars: I'm not sure :(14:00
bigjoolsit only happens on one of my boxes14:00
jmlmars: wasn't mine,  I don't think14:00
* jml doesn't actually know anything about windmill, that paste is just what deryck did last time14:01
marsjml, that fix didn't land?14:01
jmlmars: it landed. a different test is failing.14:02
marsjml, if the needed fix is similar to what you posted, then it will have to be applied by inspecting the test for races caused by missing timeouts14:05
jmlmars: make sure you pass that on to whoever sets about actually fixing it.14:06
marssure, I can do that14:06
marsDeryck's windmill fix has not landed14:07
jmlwell, I guess we'll look at the bug filed for that particular breakage and find out why14:08
=== matsubara is now known as matsubara-lunch
=== almaisan-away is now known as al-maisan
flacostejml: is it only you and me today on The Strategerie?14:46
=== jelmer_ is now known as jelmer
jmlflacoste: https://bugs.launchpad.net/launchpad-foundations/+bug/68177014:56
_mup_Bug #681770: Failure in lp.services.job.tests.test_runner.TestTwistedJobRunner.test_timeout <Launchpad Foundations:New> <https://launchpad.net/bugs/681770>14:56
flacostejml: lost connection?14:57
=== matsubara-lunch is now known as matsubara
bigjoolswould it be evil to use a storm validator to auto-update a different field?15:26
=== salgado is now known as salgado-lunch
bigjoolsoh dear, it's not good when the same class name appears twice in a test file is it ...15:45
=== beuno is now known as beuno-lunch
jmlflymake! pyflakes! happiness!16:02
bigjoolspyflakes didn't detect it16:02
bigjoolsI only noticed because on of my new tests wasn't getting run16:03
bigjoolsone*16:03
jmlreally?16:03
jelmerit gives a warning here for one of the two declarations16:03
bigjoolshmm16:03
jmlbigjools: if so, that's a bug I'm going to fix right now.16:03
bigjoolsI wonder if I've got an old vim pyflakes16:04
bigjoolscan someone please look at this, I've got a test failure that I cannot fathom.  It's totally unrelated to my changes yet reverting them makes the test pass.   http://pastebin.ubuntu.com/536736/16:06
jelmerbigjools: It must trigger a database flush or something..16:08
jelmerbigjools: my bet is that the fact you've changed builderok into a property is the culprit16:09
bigjoolsjelmer: indeed - but how is that affecting whether something else is unicode?16:09
bigjoolsoh, I wonder if it hits the DB and gets converted16:10
jelmerbigjools: Yeah, that's what's probably happening.16:10
bigjoolsthat's nasty16:10
bigjoolsthanks jelmer16:10
bigjoolsjml: bin/lint.sh does not show the duplicated clas16:11
bigjoolss16:11
bigjoolssee lib/lp/buildmaster/tests/test_builder.py16:11
jmlbigjools: I don't use or care about lint.sh16:11
bigjoolsTestBuilder is declared twice16:11
bigjoolsyes but vimflakes doesn't see it either - what do you care about?16:12
jmlpyflakes16:12
bigjoolsthat does not see it either16:12
bigjoolspyflakes lib/lp/buildmaster/tests/test_builder.py16:13
bigjoolsnowt16:13
jmlI see.16:13
* jml fixes16:13
bigjoolsjelmer: heh - we both missed the point of the test - the values should be unequal.  The unicode was a red herring.16:16
jelmerargh, I see16:17
jelmerbigjools: it was a database flush that was causing the issues though?16:17
bigjoolsI don't know16:18
bigjoolsfg16:18
jml/home/jml/src/launchpad/devel/lib/lp/buildmaster/tests/test_builder.py:127: redefinition of class 'TestBuilder' from line 8916:27
jml\o/16:27
=== salgado-lunch is now known as salgado
jmllp:~jml/pyflakes/duplicate-class-defs if anyone wants to use it.16:35
jmlhttp://divmod.org/trac/ticket/3034 if you care about upstreaming16:35
* bigjools claps jml16:37
jelmerjml: that was quick!16:38
jmljelmer: pyflakes is really easy to patch16:39
jmlparticularly after the first patch :)16:39
jelmer jml: I tried to merge the latest trunk into the bzr lazy_import branch at one point and gave up.16:40
jmloh yeah. they changed it quite significantly recently to use ast or something16:40
jmlstill structured the same but quite different textually16:41
bigjoolsjelmer: I found the problem16:46
bigjoolsBuilder.selectBy(builderok=True)16:46
jelmerah, a select on a non-storm property?16:47
bigjoolsyeah16:47
jelmerconfusing error then..16:48
bigjoolsthe selectBy doesn't break!16:48
bigjoolsit just returns nothing16:48
=== beuno-lunch is now known as beuno
=== matsubara is now known as matsubara-afk
=== al-maisan is now known as almaisan-away
jmlwho wants a Friday afternoon vaguely on-topic distraction?17:12
jmlI have pretty pictures17:12
jmlhttp://people.canonical.com/~jml/lpstats/good-and-bad-by-time-bar.png17:12
jmlhttp://people.canonical.com/~jml/lpstats/how-many-tests-are-slow.png17:12
* bigjools needs one of those17:13
bigjoolsnot a metric? :)17:13
jmlright17:13
jmlit *could* be a metric, but the numbers don't quite add up and I haven't figured out why17:14
bigjoolskidding - slightly.  What's the colours/numbers measuring?17:14
jmlthe colours are ... bands of test run time17:14
jmle.g. tests that take under a second to run, but over 0.1s are green in both graphs17:15
bigjoolsright, so it's a distribution17:15
jmlhttp://people.canonical.com/~jml/lpstats/good-and-bad-by-time.png and http://people.canonical.com/~jml/lpstats/good-and-bad.png are different versions of the same respective data17:15
jmlbigjools: yes17:15
bigjoolsis it easy to change to a line graph?17:15
jmlbigjools: one distribution is by number of tests within that band17:15
bigjoolsmight be easier to interpret17:15
jmlbigjools: the other is by time taken by tests in that band17:16
jmlbigjools: I think the bar graphs are close to what you mean by line graphs17:16
jmlunless you mean something like http://people.canonical.com/~jml/Tests-finished-each-minute.png17:16
bigjoolsI've only ever seen distribution graphs done as a curvy line, usually looking like a bell :)17:17
jmlthey are frequently done as bars, particularly for logarithmic distributions17:17
jml(i.e. http://people.canonical.com/~jml/lpstats/good-and-bad-by-time.png and http://people.canonical.com/~jml/lpstats/good-and-bad.png)17:18
jmlor maybe I meant e.g.17:18
bigjoolsbut then it's Friday afternoon, I've had no sleep from watching too much cricket, and I need a beer17:18
jmlhah17:18
bigjoolsapart from that I feel quite lucid17:19
jmlanyway, the rough conclusion is that the tests that take over a second make up a small fraction of the tests by number17:19
jml(~2000 / 12000)17:19
bigjoolsI bet we could make those 1 second or under tests faster17:20
jmlbut make up the majority by time17:20
jml(~8000 / 10000)17:20
jmlbigjools: we could, and that would be great, but the graphs basically say that if we _only_ did that, it wouldn't matter much.17:20
bigjoolsmaybe17:22
jmlif we took the test suite as it is now, and ran the fastest tests first, those taking under a second would finish in the first half hour, the remaining time would be spent in the ~15% of the tests that take over a second17:22
bigjoolslet's just delete them all17:22
bigjoolsbtw how do doctests fit in this?17:23
bigjoolsare they one test or lots of tests?17:23
jmleach doctest is counted as one test.17:23
bigjoolsI bet those 15% are largely doctests :)17:24
bigjools(including pagetests)17:24
bigjoolsin fact the latter are slower than an Aussie outfielder17:24
jmldata is gathered from: ./bin/test --subunit | subunit-ls --times --no-passthrough | sed -e 's/ /,/' > test-runtime-data.csv17:24
jmlhah17:24
bigjoolsnice analysis though17:26
bigjoolsditching pagetests gets more weight! :)17:27
jmlbigjools: fwiw, of the ~1500 tests that take over a second, 574 are doctests17:32
bigjoolsoh, wow17:33
jml315 of those are pagetests (rough count)17:33
bigjoolsI expect manty of the others are soyuz uploader tests17:33
jmlI did this: gzip -dc /home/jml/Downloads/testtools-experiment-r11753.subunit.gz | subunit-ls --times --no-passthrough | awk '{ print $2 " " $1 }' | sort -n > test-run-times17:34
jmlthen used emacs to chop out all of the sub 1s tests17:34
jmlthen wc -l test-run-times, grep -c _txt test-run-times and grep -c ' [0-9x][0-9x]' test-run-times17:34
jmlbigjools: slow unit tests, ordered by time: http://pastebin.ubuntu.com/536769/17:35
bigjoolssome obvious suspects17:36
jmlbigjools: by name: http://pastebin.ubuntu.com/536771/17:37
jmlheh17:37
bigjoolstest_all_scripts makes me weep17:38
jmlof course, subunit encodes layer start up as a test :)17:38
bigjoolsit's duplicating tests done elsewhere17:38
jml82.739 gina_txt17:38
jml86.680 soyuz-upload_txt17:38
bigjoolsthe shame17:38
salgadoabentley, any idea what's with the list of 'properties changed' messages in the diff at https://code.launchpad.net/~salgado/launchpad/expose-blueprints/+merge/41898 ?18:07
jmlg'night all.18:08
jmlhave a good weekend18:08
salgadoabentley, looks like a bug in the incremental diff thing?18:09
abentleysalgado: No, they're new to me.18:09
abentleysalgado: Yes, definitely that.18:09
salgadoabentley, can you look into that or do you want me to report a bug and/or complain on the ML?18:13
abentleysalgado: please file a bug.18:14
abentleysalgado: I will investigate, but it needs tracking.18:14
salgadoabentley, https://bugs.launchpad.net/launchpad-code/+bug/68188518:16
_mup_Bug #681885: Spurious 'properties changed' line on incremental diffs <Launchpad Bazaar Integration:New> <https://launchpad.net/bugs/681885>18:16
salgadothanks for investigating18:17
abentleysalgado: So I think there's a bug in bzrlib that makes the execute bit of directories and symlinks handled differently for RevisionTrees and PreviewTrees.  For RevisionTrees, it's False and for PreviewTrees, it's None.18:27
abentleysalgado: This bug will only be exercised if you merge from trunk or your prerequisite branch, but we certainly want you to be able to do that.18:29
salgadooh, right, that makes sense.  the busted diff is the one which includes a merge from trunk18:30
abentleysalgado: I'm going to see if the same thing happens with merge --preview.18:30
abentleyflacoste: I'm confused about the recent changes to the release process, and lifeless's proposal and how they would affect the PQM closure window.18:44
flacosteabentley: let's have a call about it in 30 minutes?18:45
abentleyflacoste: great!18:45
=== matsubara-afk is now known as matsubara
flacosteabentley: i'm on mumble19:23
cr3in launchpad, is there a preference in naming methods when getting latest items, like IPersonPublic.latestKarma compared to IPersonPublic.getLatestMaintainedPackages?19:39
cr3the former could've been named getLatestKarma or the latter named latestMaintainedPackages, so I'm wondering if there's a general preference... and a motivation would also be nice :)19:40
salgadocr3, the former is not an appropriate method name in Launchpad -- methods must be named after verbs19:40
cr3salgado: awesome, I'm pleased so hear that methods should start with active verbs19:42
salgadoin general we have good coding/naming standards in Launchpad.  the only problem is that we don't always follow them19:43
cr3salgado: no worries, this innevitably happens in projects when they become very large even with the best of intentions19:45
cr3salgado: for example, I can imagine having to define my own oauth classes to be used by the python oauth module which has different naming conventions, and then having those same conventions leak into the rest of my code.19:46
cr3in those cases, I usually duplicate method or attribute names at the end of my class definition with a comment like: # Compatibility with oauth module19:47
* lifeless thinks that this is noise and overspecified in launchpad19:49
cr3lifeless: what's noise and overspecified exactly? naming conventions or duplicating names for compatibility purposes?19:51
lifelessboth19:51
cr3lifeless: I'm not sure I follow, what's noisy about naming conventions?19:52
lifelessthe former is more to remember to meet 'ok to land' which adds little if any actual value; the latter is extra code which makes backtraces longer, will confuse folk that actually use the real module when analysing your code19:52
cr3lifeless: I'm not sure I agree but I'll admit that the importance of naming conventions is mostly a matter of taste, so open to pointless debates like vim vs emacs, but I happen to find value in following conventions for consistency purposes which gives me pleasure when using interface that are predictable19:58
lifelessI think there are several reasons I don't find that [very] valuable19:59
lifelessfirstly, its an all or nothing: either you can completely confidently guess at the right thing, or you need to do a search & read docs.20:00
lifelesseven in small programs there is often enough nuance needed that the obvious thing isn't necessarily the right thing.20:01
lifelesssecondly, unless you completely ban all external code (which is a bad idea for so many reasons), you have to work on multiple code bases when fixing and refactoring.20:01
lifelessIts extremely rare to be able to consume a component and *never* find a bug or issue in it that needs fixing.20:02
lifelessI do find value in contracts - e.g. in the context manager contract20:05
lifelessthat can be argued as a consistency thing, but I think its more reuse.20:05
lifelessIts not 'same style', its interface substitutable20:06
cr3lifeless: contract definately has value20:07
flacostemars: is there a reason that we didn't disable domain hashing in our google analytics tracker code?20:12
flacostemars: setAllowHash20:13
flacostemars: if i read http://code.google.com/apis/analytics/docs/tracking/gaTrackingSite.html#domainSubDomains correctly we should be using it20:14
flacosteotherwise, it means that we are overestimating visitors as each visitor to a different subdomain is considered different20:14
marsflacoste, I wasn't aware of that option20:44
flacostemars: ok, i'll land the fix20:44
marsflacoste, thanks.  I haven't seen that option before - IIRC I read the help docs, but not the tracking setup docs20:46
lifelessflacoste: did you see the apache pain ? :)20:54
flacostelifeless: yes, it was a deeper issue than i thought, but i'm glad that we have a fix through mod_rewrite20:55
lifelessflacoste: we do?20:55
lifelessflacoste: we left it unfixed..20:55
flacostelifeless: [B]20:55
flacosteit seems that will work, no?20:55
lifelessflacoste: not in our testing20:55
flacoste[P,B]20:55
lifelessit mangled the /20:55
flacoste!20:56
flacostewow20:56
flacostethat sucks big time20:56
lifelessGET /57533044/foo+bar.txt?token=4820:56
lifelesswrong due to +20:56
flacostewould setting the header in the librarian, bypass this?20:56
flacoste(as we wouldn't have to use <Location)?20:56
lifelessGET /57533044%252ffoo%252bbar%252etxt?token=48820:56
lifelessflacoste: the content encoding header is orthogonal20:57
lifelessflacoste: it corrupts 404s, it doesn't cause them.20:57
lifelessAFAICT20:57
flacostewhy is it working in the other setup?20:57
lifelessflacoste: its not working anywhere is it ?20:57
lifelessflacoste: define 'it' :)20:58
flacostewell, it works when we disable the feature flag20:58
lifelessthe librarian isn't used then20:58
flacostewe are able to retrieve these files now20:58
flacostedoh, of course20:58
lifelesswith the flag off, the appserver serves it using the internal librarian port to obtain the content20:58
flacosteit works because we are serving this content from the app server20:58
flacosteright20:58
flacostemissed that20:58
flacosteso the only fix is either fix apache or change front-end for the restricted librarian?20:59
lifelessof course, with the flag off kees has a script that gets 503 timeouts so often it takes hours to download security debs to check :(20:59
flacostelifeless: we could turn the feature flag on for kees :-)20:59
lifelessflacoste: we could try to reverse engineer which things will be decoded by apache and match those rules in the librarian.20:59
lifelessbut that seems spectacularly fugly.21:00
flacosteyeah21:00
lifelessflacoste: folk are still commenting on this cluster of bugs in apache this year.21:00
lifelesseven though they are marked variously 'wontfix' and 'fix released'21:00
flacostehow did tom feel with switching front-end?21:00
lifelessthere may be some dark art to make it work.21:00
lifelesswe didn't talk in detail about it21:00
lifelessapache is a known quantity, and there is a feeling that it adds some security vs running (whatever) backend service live on the interwebs21:01
elmoI'm not having whatever backend service terminal SSL :-P21:01
elmos/terminal/terminate/21:01
lifelesselmo: I was thinking squid21:01
elmowell21:02
elmoI don't believe this is intractable21:02
lifelesselmo: I wouldn't want the complexity of SSL in the librarian code - shudder.21:02
elmoU1 ran into similar rewrite escaping issues and were able to work around it21:02
lifelesselmo: I thought they did, but I couldn't find the resolution21:02
elmoI'd like to explore fixing or working around apache and at least filing bugs about it's broken behaviour further before we seriously consider abandoning it21:02
lifelesselmo: certainly21:03
lifelesselmo: could you find what U1 did to address this?21:03
elmolifeless: right now?  no, but I'll have a look21:03
lifelessthanks21:03
lifelesselmo: in terms of fixing apache21:04
lifelessI linked to one insightful comment from apache upstream21:04
lifelessapparently it does a escape decode very early in its processing pipeline21:04
lifelessbefore mod rewrite or mod proxy get close to the request21:04
=== matsubara is now known as matsubara-afk
lifelessI guess adding a request variable at that point to save the original value and pass it down might help.21:05
henningeUm ...21:27
henningeI think it is getting too late for me ...21:30
wgrantThe DB config is far more duplicated and evil than I could ever have imagined :/22:40
lifelesswgrant: looking at my branch ?22:42
wgrantlifeless: Yeah, I've fixed it.22:42
lifelesswgrant: I'd like to nuke the canonical.lp.dbhost etc22:42
lifelesswgrant: I just didn't want to make the branch bigger22:42
wgrantlifeless: Yes...22:42
lifelesswgrant: what did I have wrong?22:42
wgrantlifeless: The db_options option parser thing used to set canonical.lp.dbname. But you removed that when you turned it into a function.22:43
wgrantAlso, you missed two references to canonical.lp.dbname.22:43
wgrantlifeless: I've made db_options set lp.dbname_override, which lp.get_dbname() now consults. Any less horrific suggestions?22:44
lifelessseems fine22:49
lifelesswgrant: want me to land it ?23:32
wgrantlifeless: Please do.23:37
wgrantmake check runs OK now, until it hangs as it does on devel.23:37
wgrantSo ec2 should be happy.23:37
lifelesswheres your fix? I'll pull into my branch23:37
wgrantlifeless: lp:~wgrant/launchpad/databasefixture23:37
lifelesshow do you get other people to get emails on ec2land ?23:45
wgrantec2 land, I don't know.23:46
wgrantec2 test takes a -e option.23:46
lifelessah23:46
lifelesswell, I've ec2landed it23:46
lifelesswill forward you the epic failure, should there be one, tonight23:46
wgrantHopefully it won't be necessary!23:46
wgrantThanks.23:46
lifelessalso \o/ the site message is gone23:49
lifelesshopefully we'll see less noddy bugs on lp itself.23:49
wgrantYep.23:49
lifelessi have a reasonable query implementation in my head now23:50
lifelessneed to write it up23:50
lifelessits away23:54

Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!