/srv/irclogs.ubuntu.com/2010/03/22/#launchpad-dev.txt

* mwhudson lunches00:13
* NCommander is amazed at how long the LP test suite takes to run00:48
mwhudsonNCommander: welcome to our world00:49
NCommandermwhudson: how long does a fullrun take?00:51
mwhudsonNCommander: about 4 hours, i think00:51
NCommandermwhudson: (I'm seeing some codehosting failures which I'm concerend about, since I didn't touch that bit of the code)00:51
NCommandermwhudson: this has been going for at least 6-700:51
mwhudsonNCommander: what hardware?00:51
mwhudsonthat's pretty slow though00:51
NCommandermwhudson: 2.2Ghz dual core intel dual core processor00:52
NCommander8 GiB of RAM running on an SSD00:52
NCommanderGot a few weird failures00:52
NCommandermwhudson: I'm running lucid versus karmic though, which may explain some of the failures00:52
mwhudsonah possibly00:53
NCommandermwhudson: I know its possible to run the test suit eon EC2; can you shed some light into that?00:53
mwhudsonNCommander: have you seen https://dev.launchpad.net/EC2Test ?00:54
NCommandermwhudson: how much does EC2 cost for something like this?00:55
mwhudsona few $ i guess00:55
mwhudsonit uses a c1.xlarge instance for 4-5 hours00:55
NCommandermwhudson: might be worth it although I can't expense it as I'm not on the LP team :-/00:56
mwhudsonNCommander: i can run it for you if you like00:56
* mwhudson is currently trying to find stuff to delete so i can upgrade to lucid without scaring myself00:56
NCommandermwhudson: that may be worthwhile, although I'm concerned I may have actually broken part of LP (first db changes I ever made)00:56
mwhudsonNCommander: i don't see why that means you shouldn't run it under ec2 though?00:57
NCommandermwhudson: oh, no, that was just a side comment, not a blocker :-)00:57
mwhudson:)00:57
mwhudsonNCommander: where is the branch on lp?00:57
NCommandermwhudson: its at ~mcasadevall/launchpad/raw_source_changelog, but I'm thinking I should upload this last round of changes as a new branch00:58
NCommanderI rewrote the functionality to use librarian rather than storing in teh DB00:58
mwhudsonNCommander: well, ok, i'm not going anywhere, so if you want me to test a branch, just ping me00:58
* NCommander also feeds a second code review may be a good idea00:58
NCommandermwhudson: give me a sec00:58
NCommandermwhudson: lp:~mcasadevall/launchpad/raw_source_changelog_librarian01:02
mwhudsonNCommander: what email address do you want results sent to?01:03
mwhudson(/msg if you like...)01:03
NCommandermwhudson: mcasadevall@ubuntu.com01:27
NCommandermwhudson: (sorry for the delay in answering)01:28
mwhudsonNCommander: ok, starting up ec2 test01:28
NCommandermwhudson: thanks. I'm thinking I should probably turn my old laptop into running UEC and then run ec2test on it :-)01:33
mwhudsonheh heh01:33
mwhudsonnot if you want results quickly i suspect01:34
NCommandermwhudson: the old laptop is 2.2Ghz 4GiB of RAM01:34
NCommanderits not that slow01:34
mwhudsonoh ok01:35
mwhudsonno :)01:35
NCommandermwhudson: Total: 25575 tests, 16 failures, 19 errors in 510 minutes 32.100 seconds.01:39
NCommanderirony, it just finished01:39
NCommanderI tee'ed the output, but I don't even know if the failures are due to lucid, or due to something in my environment :-/01:40
mwhudsonNCommander: well the results from ec2 will tell you some of that01:42
NCommandermwhudson: thanks :-/01:42
wgrantNCommander: The only known Lucid failure is in apt-ftparchive.02:17
wgrantNCommander: Lucid's adds SHA1 and SHA256 hashes in addition to pre-Lucid's MD5s.02:17
wgrantAny other failures are probably real problems.02:17
NCommanderwgrant: I think my postgresql server dumped out towards the end, I got a bunch of *strange* errors with write errors :-/02:18
wgrantHeh.02:19
wgrantDoes anybody happen to have an idea of where the time is taken in the primary archive publisher?02:19
NCommanderwgrant: well, at least then I know which tests to check02:24
wgrantNCommander: Right.02:25
lifelesswgrant: not an accurate one; its a problem I'd like to look at. If you choose to look at it I'd be very happy to be a sounding board.04:16
wgrantlifeless: From what I can tell, publish-distro.py takes about 20-25 minutes currently.04:21
wgrantI've cut two or so minutes out of that locally by optimising domination.04:21
wgrantAnd I think most of the rest mus tbe in the apt-ftparchive integration. But I'd like to know for sure.04:22
wgrant(apt-ftparchive integration is the one bit I can't sanely simulate locally, since it requires lots and lots of real packages, not just fake contentless ones)04:23
persiawgrant: You could simulate by mirroring karmic and "publishing" bits of current lucid.04:24
wgrantpersia: But that requires a local copy of the archive, which has a sane Internet connection as a prereq.04:25
persiaThe pacific ocean clearly needs a denser net of high-capacity optical fibre04:27
wgrantBut anyway, I have just reduced native index generation time for largely-unchanged Ubuntu-sized binary pockets to 2-3 seconds.04:28
spmor electrical flows that exceed the speed of light04:28
wgrantIt would be very interesting to know how much longer than 20 seconds the apt-ftparchive integration code takes to run.04:29
mwhudsonspm: yeee-haw!04:29
wgrantCan anybody see cron.publish-ftpmaster logs?04:29
persiaIs there some test I can run locally that doesn't require me to setup/install launchpad first?04:29
spmmwhudson: indeed04:29
wgrantpersia: No.04:30
wgrantBut the times are all in the logs that presumably somebody can see.04:30
mwhudsondammit, now i want to watch that movie again, and i only watched it last week or so04:30
wgrantWhich movie?04:30
mwhudsondr strangelove04:31
wgrantAh.04:31
mwhudsonspm posted http://davideubank.files.wordpress.com/2008/09/slim-pickens_riding-the-bomb_enh-lores-720p.jpg with the comment "the life of a losa" or something like that04:31
wgrantAhhh.04:32
spmit' an accurate description.04:32
wgrantspm: Does cron.publish-ftpmaster log anywhere that you can check quickly? I am interested in the time taken by index generation (starting with '* Step C', ending with '* Step D')04:34
spmlooks...04:34
spmurg. we don't rotate the poppy log. /me hads a MUST FIX.04:37
spmwgrant: http://paste.ubuntu.com/399107/04:41
wgrantAha.04:41
wgrantSo we can cut 17 minutes of the publisher run-time to probably well under one.04:42
wgrantThanks spm.04:42
spmheh; if you can cut the time down? we'll thank you! :-)04:42
persiawgrant: This isn't all part of some master plan to release 48 times a day, is it?04:46
* NCommander waves to persia 04:46
spmpersia: ??? there's more minutes than 48 in a single day! :-P04:46
persiaspm: Sure, but one has to run the entire publisher for each release, so it would need lots more optimisation to release every minute.04:48
wgrantpersia: To at least make it possible to drive it that fast if necessary, yes04:48
persiaActually, with the "if necessary" bit, that's a good thing.  I'm just not sure I'd like it that fast by default: sometimes folk catch stuff within a run, which gets hard if it gets smaller.04:49
wgrant(this maaaay not be unrelated to Saturday's events)04:49
persiaheh04:49
NCommanderpersia: well, given a fast enough machine, it should be possible to publish once a minute04:50
wgrantNCommander: s/fast enough machine/less pathetic publishing code/04:51
persiaNCommander: You'd also need a faster network, faster mirrors, etc.04:51
NCommanderpersia: or a longer minute04:51
persiafeh.04:51
NCommanderI mean, if you redefine a minute ...04:51
NCommanderMicrosoft redefined stable after all, it could be done :-)04:51
wgrantMy timeline has a 19 minute gap :(04:52
persiaMy minute is 551,557,906,200 vibrations of cesium-133 at STP.04:52
NCommanderMy minute is 0.06 kiloseconds04:53
NCommanderMy second however is define as 0.0864 standard seconds. What sorta second am I?04:54
persiawgrant: Which bit is missing?04:54
spma 2nd rater ?04:54
spmNCommander: you really left yourself open to that one :-)04:55
NCommanderer04:55
NCommander*0.864 seconds04:55
NCommanderspm: its a decimal second :-)04:55
wgrantpersia: There is a 19 minute hole containing dsync's link-dupes run, cron.germinate and the cocoplum->syncproxy sync. The last probably takes three minutes.04:56
wgrantSo one of the first two is taking an awful long time.04:56
persialink-dupes04:56
wgrantI'd suspect so, yes.04:57
wgrantI'm not quite sure why it's run.04:57
persiaIt takes > 2 hours to run link-dupes against my mirror set.04:57
wgrantDo you know?04:57
persiaMassively saves space.04:57
wgrantEven though we symlink matching files anyway?04:57
persiaErm.04:57
persiaThen no, doesn't do anything at all.04:58
persiaI'd rather see matching files hardlinked, because it mirrors better, but that's just preference.04:58
persiaSo even though it has no effect, it still takes forever to run fdupes.04:58
wgrantI mean, when we need a file published in multiple locations, we symlink the non-master copy as any good archive software should.04:58
wgrantThis is well before the archive is split into archive/ports/security, so I don't see why there would be any dupes.04:59
persiaIf it's pre-mirror-split, there should be no dupes.04:59
wgrantRight.04:59
persiaOr rather, only known multi-publication.04:59
wgrantI'm guessing you use link-dupes because you maintain mirrors of both ports and non-ports (or portions thereof)?05:01
persiaActually, I use my own hacked fdupes-based script, and also mirror debian, but yes.05:02
wgrantSo if there's little point running it, and we fix NMAF to support extra-overrides and migrate the primary archive to use it, *and* my two optimisation branches actually work properly, we can cut more than half an hour off the 40-minute-ish publisher.05:05
persiaWhich could shorten the time-to-fix for something like Saturday's incident down to ~90 minutes with manual publisher runs.05:06
wgrants/90/3005:06
mwhudsonwhat happened on saturday?05:06
persiawgrant: consider build times too.05:06
wgrantUpload CDBS, wait for it to build in a few minutes, publish. Upload two broken packages, built in 10 minutes, publish.05:06
wgrantmwhudson: Lucid broke right beta 1.05:07
persiamwhudson: some updates were published to lucid that broke gnome-panel autostart on login.05:07
wgrantgnome-panel and nautilus broke so they wouldn't autostart.05:07
wgrantLeaving all the willing testers with no desktop at all.05:07
spmwooo05:07
persia(or at least everyone who logged out and logged in after the updates)05:08
mwhudsoni see....05:08
wgrantpersia: Which lots of people did, after they couldn't open folders any more :/05:08
spmspeaking as someone who's 7 mins from downloading for the u/g. is it all fixed on latest updates?05:08
persiaOh, yeah.  non-logout/non-reboot is the significant minority.05:08
mwhudsonjust out of totally idle curiosity, what versions of things contain the fixes?05:08
mwhudsonspm: zing!05:08
persiaIt's fixed.05:08
wgrantmwhudson: -0ubuntu2 of both of those is broken.05:08
wgrant-0ubuntu3 is safe.05:09
wgrantIt's been fixed since Saturday night.05:09
wgrantIt just took a lot longer than it could have if the publisher didn't take an hour.05:09
mwhudsonmmm05:09
mwhudsoni have -0ubuntu1 of those05:09
mwhudsoni guess my mirror aint so fresh05:09
spmhaving my main workstation cactus is a pain; but not unsurvivable. karmic forced me onto my laptop for a week or 2; totally broke my workstation's startup.05:09
persiaGetting the publisher down to ~10 minutes makes it something people will be more willing to babysit too.05:10
wgrantRight.05:10
wgrantPlus it'd be less of a pain around releases, when you see release managers tossing up whether to run it now or not.05:12
persiaAbsolutely.05:13
mwhudsonha, ha05:13
mwhudson255 updates05:13
NCommandermwhudson: thanks for ec2ing my branch. TIme to fix some failures05:55
NCommandermwhudson: you got less failrues and errors than I do it05:55
NCommanderwhat can cause this: ProgrammingError: permission denied for relation sourcepackagerelease05:58
mwhudsonNCommander: have you encountered database/schema/security.cfg yet?05:58
NCommandermwhudson: nope06:00
mwhudsonNCommander: in that case "ProgrammingError: permission denied for relation sourcepackagerelease" means you need to make friends with this particular file06:01
mwhudsonNCommander: different bits of launchpad connect to the database as different database users06:01
mwhudsonand the different users have permissions to access the various tables in different ways06:01
NCommandermwhudson: ugh. I have no idea how to fix this, since there is a SELECT permisison already in place06:03
mwhudsonNCommander: for which user though?06:04
NCommandermwhudson: librariangc. Looking at the code, its trying to DELETE, but I'm kinda confused on how objects leave librarian06:05
NCommandermwhudson: it seems to me its trying to delete rows, but I'm not understanding why librariangc isn't happy06:07
NCommandermwhudson: oh wait ...06:08
NCommandermwhudson: alright, I think I figured out why the librariangc tests are failing :-)06:09
mwhudsonNCommander: grats :)06:12
NCommandermwhudson: I assume I need to re-make schema after editing security.cf06:18
NCommander*cfg06:18
mwhudsonNCommander: i think you can just run python database/schema/security.py launchpad_ftest_template06:19
mwhudsonor something06:19
mwhudsonbut i usually just run make schema, yeah06:19
NCommandermwhudson: thanks, I'm still somewhat new at all this LP hackery. I'll probably ask you to re-run ec2test for me since I got a lot mor efailures on 127.0.0.1 than ec2 did :-/06:19
* NCommander will probably setup EC2 sometime in the future but can't ATM :-/06:20
StevenKNCommander: It's pretty easy, it took me about 10 minutes06:20
NCommanderStevenK: how much did it cost per run?06:21
StevenKRoughly $3.5006:21
StevenKBut they don't charge until the end of the month06:22
NCommanderStevenK: that can add up fast -/06:22
mwhudsoni spent a lot on ec2 in february06:22
mwhudsonluckily i can expense it...06:22
NCommandermwhudson: woo, librarian fixed06:32
* NCommander now works on registery failures06:32
mwhudsonNCommander: i'm going to be going afk soon, but europe should be getting up soon ish :)06:33
NCommandermwhudson: hrm, is it possible for a test to be failing in db-devel?06:47
* NCommander can't see how this failure is possible06:47
NCommanderer06:47
NCommanderor how is impossible06:47
StevenKNCommander: Stuff doesn't land if the tests fail06:48
mwhudsonStevenK: not true any more, sadly06:48
NCommandermwhudson: that was what I was afraid of06:48
StevenKAwww06:48
mwhudsonNCommander: which tests fail for you?06:48
NCommandermwhudson: lp.registry.tests.test_distroseries.TestDistroSeriesPackaging06:48
mwhudsonoh06:48
mwhudsonthat doesn't seem to be failing in buildbot06:49
NCommandermwhudson: NameError: global name 'gpg_key' is not defined06:49
NCommanderLooking at the file in question though, there is only one refernece to gpg_key in the whole thing06:49
mwhudson!06:49
mwhudsonis that in devel or db-devel?06:49
NCommandermwhudson: db-devel, but hold on06:50
* NCommander might have done something stupid06:50
wgrantDamn. Turns out my 20 second index generation run had broken bz2 compression. Fixing that blows it out to more than a minute :(06:51
StevenKwgrant: A minute versus 24, though?06:51
mwhudsonNCommander: i don't see gpg_key in that file06:51
mwhudson(on either branch)06:51
wgrant17-18, but yes.06:51
wgrantThe rest of the 24 is being cut away in other places.06:52
NCommandermwhudson: I dunno, I think I went a little merge happy at some point, and lack of sleep is not helping06:52
mwhudsonlack of sleep often doesn't help with hacking06:53
NCommandermwhudson: no, but my girlfriend went ona  cleaning spree06:57
NCommanderand my bed currently isn't where I left it06:57
mwhudsonNCommander: haha06:58
wgrantAre we week 2 or 3 at the moment?06:58
mwhudsonwgrant: just starting week 306:58
wgrantAha.06:58
* wgrant had better get his last PPA download stats branch in order, then.06:59
* NCommander broke something bad07:00
NCommandervery bad07:00
NCommanderanyone around who can run the ec2test script OR run the test suite locally without getting unrelated failures07:25
* NCommander has it fixed07:25
=== mthaddon` is now known as mthaddon
wgrantbigjools: Morning.08:57
bigjoolsg'day08:57
wgrantWhat are the blockers for NMAFing primary? Just extra overrides?08:58
bigjoolsyep, barring any hidden bugs08:59
bigjoolsand possibly performance08:59
wgrantPerformance is definitely an issue.08:59
wgrantBut with a bit of caching I can publish all indices for a larger-than-Ubuntu archive in 20 seconds.08:59
wgrant(one series thereof, that is, like most normal runs)09:00
bigjoolswgrant: using NMAF?09:02
wgrantbigjools: Yeah.09:02
bigjoolsit would be interesting to do like for like comparisons09:03
mrevellHello09:03
bigjoolshey mrevell09:03
bigjoolswgrant: did you see Bug 543162 BTW?09:03
mupBug #543162: launchpad-buildd exception: Try to register an already registered process. <Launchpad Auto Build System:New> <https://launchpad.net/bugs/543162>09:03
wgrantbigjools: I was going to, but a-f requires real sources and binaries, whereas NMAF doesn't care if they're garbage.09:03
bigjoolswgrant: dogfood FTW :)09:04
wgrantbigjools: I have, yeah. lamont poked me about it at the time to check if there was anything obvious.09:04
bigjoolsit's an odd one - I can't work out if twisted blew up or the slave code09:04
wgrantbigjools: Is dogfood a good performance indicator?09:05
bigjoolswgrant: not against anything else, but relative to itself of course it is09:05
noodles775bigjools: if it's related to tearing down twisted code, that behaviour changed recently.09:05
wgrantI mean, a-f is going to hit the DB in different ways to NMAF.09:05
bigjoolswgrant: indeed09:05
bigjoolsnoodles775: was that in lucid though?09:06
wgrantI tried to use memcache, but it was far too slow :(09:06
noodles775bigjools: nm, not on cesium.09:06
bigjoolsironic :)09:07
wgrantnoodles775: It's the buildds themselves, which are in fact pre-Wellington.09:07
noodles775wgrant: Right (just reading the bug).09:08
wgrantIt's only been seen on armel, but it happened on two of them (at a reasonably inconvenient, almost simultaneous time)09:11
wgrantbigjools: Also, do you know the purpose of the dsync link-dupes call in cron.publish-ftpmaster? There shouldn't be any duplicates...09:13
bigjoolsno idea, it's always been there09:14
wgrant(and it probably takes forever and then a little while longer (or just 15 minutes)09:14
wgrantbigjools: Who's likely to know?09:18
wgrant'cause slow publishing makes me sad and breaks lots of users' systems.09:19
bigjoolswgrant: probably people who have left :/09:22
bigjoolsI can check with IS09:23
wgrantIt's from dak, so elmo probably does.09:23
deryckMorning, all.09:50
=== stub1 is now known as stub
jmlgood morning Launchpadders09:53
wgrantWhat do I need to do to get the PPA download stats script running on production?10:25
bigjoolswgrant: is it a cron job?10:25
bigjoolswhere is it?10:26
wgrantbigjools: It's a cron job. cronscripts/parse-ppa-apache-access-logs.py10:27
bigjoolsok10:27
bigjoolswgrant: we need to run it on dogfood first10:27
wgrantRight.10:27
wgrantOr staging.10:27
wgrantAnd it needs production configs.10:28
bigjoolsnot staging10:28
wgrantIt doesn't need a real archive. Just logs to be put somewhere.10:28
bigjoolsI'll work on the configs and test it on DF10:28
wgrantThanks.10:28
bigjoolshmm busy day10:29
wgrantbigjools: Ew, history deletion.10:34
bigjoolswgrant: ?10:35
wgrantbigjools: Deleting archives and EVERYTHING with them.10:35
bigjoolswgrant: yep10:35
bigjoolslike deleting a branch10:35
wgrantYes, both are repulsive.10:35
bigjoolsdeletion is deletion, not some half-assed attempt10:36
bigjoolsif you want history, keep the f***ing thing10:36
bigjoolsor it can be disabled10:36
wgrantIf you want malicious or misled users to be able to inappropriately destroy things irrevocably, let them actually delete thousands of records in one click.10:36
wgrantIf you want users to be able to remove the only records of their malicious activities on the buildds...10:37
wgrantAlso, how do binary copies work?10:39
wgrantIf I delete an archive that has had binaries copied from it, the builds are more than a little awkward.10:39
bigjoolsthe binaries won't be deleted10:40
wgrantWhat happens to the builds?10:40
wgrantWe cannot delete them.10:40
bigjoolsneither those10:40
wgrantSo we lie about their archive?10:40
bigjoolsno, we change the code to say their original archive was deleted10:41
bigjoolsdid you read the bug?10:41
wgrantIt didn't cover builds in particular. I guess stuff might not break if the archive disappears.10:42
bigjoolsthat's what the test suite is for10:43
=== elmo_ is now known as elmo
elmook, seriously, this sucks11:46
elmoI've turned referer back on, and I still can't add attachments to bugs11:46
wgrantWhat's the error? Still 'Unexpected Form Data' with a 500?11:47
elmooh, lala11:47
elmosorry, never mind11:47
elmoit's my fascist firefox apparmor profile11:48
elmo\o/11:48
wgrantHeh.11:48
kfogeladeuring: PQM is behaving like it has not seen my submissions (http://paste.ubuntu.com/399248/).  Am I missing something?  Can it have built so fast that I missed it? :-)12:02
kfogeladeuring: (oh, I made a cut-and-paste error in that paste -- the submissions actually do match their directories, don't worry)12:03
adeuringkfogel: ;)12:03
kfogeladeuring: https://pqm.launchpad.net/ shows nothing though12:04
kfogeladeuring: bbiab12:05
bigjoolswgrant: ok I have dogfood updated, what configs does that cronscript need? it looks generic12:10
wgrantbigjools:12:11
wgrant[ppa_apache_log_parser]12:11
wgrantlogs_root: /srv/ppa.launchpad.net-logs12:11
wgrantPaths will vary.12:11
bigjoolswgrant: what about where it puts results?12:11
wgrantbigjools: The DB?12:12
bigjoolsah nm the config is ppa specific12:12
bigjoolshow does it know to read that config?12:12
wgrantbigjools: It's used in cronscripts/parse-ppa-apache-access-logs.py12:14
* bigjools slaps face12:14
bigjoolsI was looking at the wrong script, sorry12:14
wgrantHeh.12:14
wgrantI thought something might have been a bit odd.12:14
bigjoolsyeah :)12:14
bigjoolswgrant: so does this script depend on only ppa logs being in the logs_root ?12:22
wgrantbigjools: It'll ignore them if they don't match a PPA binary for any reason.12:22
wgrantSo it shouldn't matter.12:22
bigjoolsmmkay12:23
bigjoolswgrant: well it does if one of the files is not readable :/12:24
wgrantYeah.12:24
wgrantNot my code :P12:24
* bigjools watches the buck flying overhead12:24
wgrantIt will also do strange things if you run both the librarian and PPA log parsers over the same directory, but that's unlikely to happen.12:25
adeuringkfogel: did you perhaps receive a mail from pqm with an error message?12:33
bigjoolswgrant: it doesn't log much does it :)12:33
bigjoolsand 2.1g resident :/12:33
* bigjools gets coffee while waiting for it12:33
maxbCan someone fix the grammar fail in the PPA description?  https://edge.launchpad.net/~launchpad/+archive/ppa12:39
wgrantbigjools: Did you see the warning in the docstring (lovingly copied from the librarian one)12:40
wgrant(I didn't think there'd be many logs on dogfood -- sorry)12:43
bigjoolsmaxb: done12:47
bigjoolsthanks12:47
bigjoolswgrant: I fiddled the log so that it only appears as a symlink in another dir12:48
wgrantm_o_d: Did you run rocketfuel-setup?12:50
m_o_dwgrant: yes12:50
wgrantm_o_d: Did you see it installing lots of packages?12:50
m_o_dwgrant: yes, my error is when i run make schema12:51
wgrantm_o_d: I know. But it's complaining about something that rocketfuel-setup would have instaled.12:51
wgrantDo you have python-tickcount installed?12:51
jmlbigjools, any recommendations for coffee from hasbean?12:51
m_o_dyes, 0.1-0ubuntu10launchpad112:51
wgrantm_o_d: Which version of Ubuntu are you using?12:52
bigjoolsjml: oh yes :)12:52
=== mrevell is now known as mrevell-lunch
m_o_dwgrant: lucid12:53
bigjoolsjml: http://www.hasbean.co.uk/products/Guatemala-Finca-Puerta-Verde-Antigua-2009%252d2010.html12:53
bigjoolsyou had that last week12:53
bigjoolswell, on Friday when the other stuff ran out12:53
wgrantm_o_d: Try to apt-get upgrade. Does that propose anything python-related?12:55
jmlbigjools, thanks.12:55
bigjoolsjml: also check out the Bolivia Machacamarca12:55
bigjoolsjml: that one is simply awesome12:56
wgrantbigjools: Is the script still doing stuff?12:56
bigjoolswgrant: no it finished some time ago, I got sidetracked12:56
bigjoolsbinarypackagereleasedownloadcount has got 4736 rows now12:57
wgrantExcellent.12:57
jmlbigjools, ta. I'll order the Guatemala one and see how I go.12:57
wgrantbigjools: Do you have my latest branch?12:58
bigjoolswgrant: I'm using db-devel12:58
wgrantAh, the webapp is not up.12:58
jmlbigjools, I'll have to do a taste test.12:58
bigjoolsjml: most stuff from central America is great12:58
bigjoolswgrant: noodles775 is playing with iut12:59
kfogeladeuring: yeah -- testfix/rc13:04
kfogeladeuring: :-(13:04
barrybigjools, or maybe anybody else running on lucid+py2.6.  have you seen this error?13:09
barryhttp://paste.ubuntu.com/399288/13:09
bigjoolsbarry: no not seen that13:10
wgrantmaxb: Arrrrgh it's back. ^^13:10
bigjoolsbarry: but then I'm not really running 2.6 :)13:10
wgrantbarry: That one has been plaguing me on and off since I started using Lucid a month in.13:10
wgrantmaxb has seen it too.13:10
barrywgrant, bigjools, maxb arg! :)13:10
maxb*blink*13:10
maxbUserWarning: Unbuilt egg for setuptools [unknown version] (/usr/lib/python2.6/dist-packages)   <--- that's a new one13:11
barrywgrant, maxb, bigjools i really don't want to install python2.5 if i can help it.13:11
bigjoolsbarry: it's part of lp-dev-deps isn't it?13:11
wgrantbarry: You have to. Launchpad doesn't run on 2.6.13:11
barrymaxb: i do have python-setuptools installed, which should provide distribute13:11
bigjoolswot wgrant said13:12
maxbbarry: You want to install python2.5. Unless you want to dedicate your entire time to massive porting project for the forseeable future, you want to install python2.513:12
wgrant(this is a bug that really needs to be fixed soon)13:12
bigjoolsand I thought barry was fixing it actually :)13:12
barrydang.13:12
barryit's gotta run on 2.6 pretty soon, though, right?13:12
kfogeladeuring, wgrant: have a sec to chat about https://bugs.edge.launchpad.net/malone/+bug/78565/comments/16  (ironically, you'll have to navigate from there to the bug for context, becuase the bug isn't fixed yet).13:13
mupBug #78565: no direct link from bug comment page to corresponding bug <breadcrumbs> <confusing-ui> <Launchpad Bugs:Triaged> <https://launchpad.net/bugs/78565>13:13
wgrantkfogel: The breadcrumb infrastructure is buggy, and the bug breadcrumbs are buggy.13:13
kfogelwgrant: because when the final breadcrumb does not match the page being actually displayed, the breadcrumb is still unlinked?13:13
kfogelwgrant: or some other bugginess?13:13
wgrantThat is, the breadcrumb infrastructure should linkify (and probably whinge) if the target of the final crumb is not the current context.13:14
wgrantAnd bug comments should have breadcrumbs of their own.13:14
wgrantAh, that comment.13:14
kfogelwgrant: mmmm.  The latter alone would fix this bug, of course.13:14
wgrantRight.13:14
wgrantI fixed a few Soyuz cases of this a couple of weeks ago.13:14
kfogelwgrant: oh hey, let me find those branches, since I can probably save some time by looking at that diff.13:15
kfogelwgrant: or, do you want to just do this one too?  Either way is fine with me.13:15
wgrantkfogel: devel r1048013:15
kfogelthx13:15
kfogelwgrant: ^^13:15
wgrantI should sleep and do assignments, not add breadcrumbs.13:16
kfogelwgrant: heh, np.13:16
* maxb lunches13:17
* wgrant really should get around to starting the page titles, breadcrumbs and root contexts discussion on the ML at some point.13:17
leonardri had to manually install 4 packages (3 dev packages + python-geoip) to get make to work in lucid. is this a problem? would it help other people if i mentioned these packages on the wiki?13:19
kfogelleonardr: 'make' in general, or just for lp?13:20
leonardrkfogel: just for lp, sorry13:21
leonardri had to install those packages to start doing my work again13:22
maxbleonardr: Well, let's figure out whether they ought to be in lp-(dev-)deps, and add them if so13:36
maxbWhat were they all?13:37
leonardrmaxb: recreating the list13:37
leonardrpython-geoip13:37
leonardrlibapr1-dev13:37
leonardrpython-subversion13:38
leonardrpython-libsvn13:38
leonardrlibgpgme11-dev13:38
leonardrpython-all-dev13:38
leonardri think that's it--i still can't start launchpad but it seems to be a postgres problem13:38
leonardri can build everything13:38
maxbpython-libsvn ?!13:39
maxbleonardr: I conclude that launchpad-dependencies was removed by the upgrade to lucid13:40
leonardrmaxb: ok, i see what happened13:40
jmla thing that stumped me for a while yesterday was forgetting to re-enable the launchpad sources.list lines13:40
leonardri re-enabled the sources.list lines13:41
leonardrand i ran update/upgrade13:41
leonardrbut nothing says you have to have launchpad-dependencies just because it's available13:41
leonardrso it never got installed13:41
maxbSo it wouldn't be a bad idea to remind people on the wiki that the Ubuntu upgrader will disable 3rd-party sources.list entries and potentially remove launchpad-developer-dependencies13:42
bigjoolsthe upgrade has always remove lp-deps, it's a royal PITA13:42
maxbI like to do something slightly cheeky: When the upgrader pops up the box saying "I've disabled 3rd-party sources.list lines", I re-enable them before I click OK :-)13:43
maxb(only for PPAs I *know* to be correctly populated for lucid, obviously)13:44
leonardri knew from last time the repo would be disabled but i didn't make the connection that i would have to reinstall the package13:44
kfogelnoodles775: you're looking into the pqm stoppage, right?13:56
=== matsubara_ is now known as matsubara
=== mrevell-lunch is now known as mrevell
bigjoolskfogel: he's at lunch, but yes he was14:22
kfogelbigjools: thx14:22
=== didrocks` is now known as didrocks
=== dobey_ is now known as dobey
noodles775kfogel: Yes, I've got a branch on ec2 test to land for the failure earlier this morning, but buildbot has completed successfully since then.14:49
kfogelnoodles775: argh14:51
kfogelnoodles775: but the fix in your branch is still needed?14:51
noodles775kfogel: it seems to be timing dependent, so yes.14:51
kfogelnoodles775: well, I guess this just means you can land your branch without 'testfix' :-).14:52
jmlwgrant, still around?15:28
jmlabentley, any movement on https://code.edge.launchpad.net/~abentley/launchpad/multiple-series-recipe/+merge/21257 ?15:29
=== matsubara is now known as matsubara-lunch
intellectronicajml: is there no way we could figure out which packages to chart programatically?15:34
jmlintellectronica, I couldn't think of anything, but I wasn't thinking very hard.15:35
intellectronicajml: my worry is that this will rot, and when we add new packages we'll forget to add them to the list15:35
jmlintellectronica, we could, for example, specially mark the __init__ of packages that we care about15:35
jmlintellectronica, yeah, it's a good worry.15:35
jmlintellectronica, luckily, it's really obvious when you've missed a package that's been added or removed15:36
jmlintellectronica, removed packages show up as disconnected islands15:36
intellectronicajml: r=me on the code. it would be nice to have a solution for this, so it's worth thinking of, and maybe filing a bug, but i can't think of a solution right now either15:37
jmlintellectronica, every single file of an added package appears on the graph unless it's included in the cluster list15:37
intellectronicayeah, i understand15:37
jmlintellectronica, thanks.15:38
jmlintellectronica, do you reckon it's ok to land the code before I update launchpad-developer-dependencies?15:38
intellectronicajml: i think it is. it won't block anyone - the worst that can happen is that people won't be able to generate the graph15:39
jmlintellectronica, cool. I'll land it now then.15:39
=== gary_poster is now known as gary-lunch
=== deryck is now known as deryck[lunch]
abentleyjml, no.  mars says bjorn is still away.15:45
jmlabentley, oh, if it's just waiting on BjornT then it's ok to land and then get a post-merge review, iirc.15:46
jmlabentley, in your last comment you said there was stuff to discuss with the code team.15:46
abentleyjml, I discussed it, we decided to leave it the way it was.  Stub did not require changing it.15:48
jmlabentley, cool.15:48
jmlnoodles775, https://code.edge.launchpad.net/~michael.nelson/launchpad/496862-ppa-installable-binaries/+merge/21623 looks good to land. hint hint.15:48
gmbjml: Quick question: How do you feel about us adding a next_check column to BugWatch? This would allow us to have a garbo-hourly process that decides when a watch should next be checked based on how much it's error in the past; that saves us complex queries in BugTracker.getBugWatchesNeedingUpdate() and means that we keep that logic out of checkwatches.15:54
gmbjml: https://bugs.edge.launchpad.net/malone/+bug/544238 for reference15:54
mupBug #544238: Add a next_check column to BugWatch <story-bug-watch-error-tracking> <story-reliable-bug-syncing> <Launchpad Bugs:In Progress by gmb> <https://launchpad.net/bugs/544238>15:54
jmlgmb, the quick answer is that it sounds similar to the way the branch puller works and how job tables frequently work15:54
gmbRight15:55
jmlgmb, so, I guess, let's do more of that and avoid gratuitous inconsistencies15:55
gmbjml: Cool. We've kinda figured out that the best way to make checkwatches play nice is to take some of its logical toys away, so I think this is a good first step. I'll knock a patch together.15:56
noodles775jml: I'd love to, but I don't know that it's ready. Did you seen the screenshot I put on the bug (linked from the MP)?16:00
jmlnoodles775, no I didn't.16:02
=== salgado is now known as salgado-lunch
gmbjml: I've requested a review from you and Stuart for said DB patch. https://code.edge.launchpad.net/~gmb/launchpad/bugwatch-next_check-bug-544238/+merge/2186916:07
=== deryck[lunch] is now known as deryck
=== matsubara-lunch is now known as matsubara
persia`How painful would it be to convince Soyuz to allow some architecture: all packages to build on non-i386?17:10
bigjoolsvery17:10
bigjoolsI was looking at a bug about that yesterday17:10
bigjoolsan old bug :)17:10
persia`The one that openbios-sparc and openhackware have FTBFS since warty?17:11
persia`So, I know I can construct a package that can trick itself into generating an arch:all binary when told to only build arch:any packages.17:12
persia`Would it be easier just to allow Soyuz to accept the uploads, whilst still keeping the standard sbuild arguments?17:12
bigjoolspersia`: I'm not sure I understand17:14
persia`So, there's two issues at hand.17:15
persia`1) when Soyuz dispatches to sbuild, it only passes -A for i38617:15
persia`2) when Soyuz accepts an upload, it doesn't like extra packages appearing in the .changes file.17:15
bigjoolswhy are they issues?17:16
persia`So, if we manage to force the package to build the arch:all package even without sbuild's hint, it fails to upload.17:16
persia`Oh.  Some stuff that's useful on any architecture is impossible to build on i386.17:17
bigjoolsthat's rather special17:17
bigjoolsand hurts my brain to think about how that can happen17:17
persia`openbios-sparc is the classic example.17:18
persia`It can only be built on sparc, but is required to run sparc emulation for all architectures.17:18
bigjoolson inspection of that name, it doesn't seem arch independent :)17:18
persia`Why not?17:19
persia`It's just a bios blob.17:19
persia`So, as data, it's useful for any data consumer, on any architecture.17:20
persia`And without it, qemu is broken.17:20
bigjoolsso basically it seems that soyuz is simply not designed to deal with this case17:20
persia`Yes.17:20
bigjoolswhich makes me sad17:20
persia`But since there's only a few packages in the archive (3 or 4) that are affected, it's a low priority bug.17:20
persia`If it was easy, and just not done, I'd go find someone to do it.17:20
persia`If it's really hard, then I'm not going to be able to get some random person intersted in fixing it.17:21
bigjoolsI guess it would be nice to have a way of overriding the nominated arch-indep builder in the package17:21
persia`That would be an ideal way to fix it, yes.17:21
bigjoolswhich is a lot easier than making soyuz deal with arch:all on any arch17:21
persia`We could insert X-Build-Affinity: (or similar) in the .dsc17:21
bigjoolsyep17:21
bigjoolswould be pretty easy to do I think, barring any dodgy bits of code I've forgotten about17:22
maxbPart of the problem is that there's no standard way in the deb formats for this sort of thing (leading to minging hacks like Packages-arch-specific)17:22
persia`maxb: hence the X, but that's easy to insert.17:22
bigjoolsp-a-s makes me want to stick needles in my eyes17:23
persia`Why do we have P-a-s again?17:23
persia`Isn't that why we have Architecture: in debian/control ?17:23
maxbBecause Packages / Sources files don't contain enough info to compute whether a given source will build a given binary on all architectures or not17:23
persia`P-a-s just makes pain for porters, in my experience.17:23
persia`maxb: So?  Some stuff FTBFS.17:24
maxbhuh?17:24
persia`And we go fiddle with http://qa.ubuntuwire.com/ftbfs until it builds.17:24
maxbPerhaps I should say that they don't contain enough info to compute whether a given source *ought* to build .....#17:24
persia`maxb: How doesn't the Architecture: field in the .dsc do that?17:26
* maxb gets example17:27
persia`The vast majority of my experience with P-a-s has been complaining that it was too restrictive, so I may be biased on this point :)17:28
maxbSo, take gdb, whose .dsc says:17:28
maxbBinary: gdb, gdb64, gdbserver, libgdb-dev, gdb-source17:28
maxbArchitecture: any17:28
persia`OK.17:28
maxbBut, then you look into debian/control inside the unpacked source, and you find:17:28
maxbPackage: gdb6417:28
maxbArchitecture: i386 powerpc s390 sparc17:28
maxbThat's the information which is lost, and then kludged back in via P-a-s so that quinn-diff can figure out whether binaries are missing because they're supposed to be, or because they haven't built yet17:29
persia`So you don't actually trust the .changes files for this.17:30
=== salgado-lunch is now known as salgado
persia`Which explains why it wasn't possible to trick openbios-sparc to generate arch:all in the arch:any run.17:31
maxbWhat's the .changes got to do with this?17:31
persia`The .changes generated from a build is *supposed* to list all the packages that the build generates.17:32
persia`So, if you trusted the changes, you wouldn't need to run quinn-diff to find out if stuff was missing.17:33
=== gary-lunch is now known as gary_poster
maxbI'm not necessarily sure LP uses quinn-diff at any point (in fact I doubt it), I'm more describing the original purpose of P-a-s in Debian17:34
persia`Which actually makes a lot more sense, since the archive system doesn't have trusted slave buildds.17:35
maxbP-a-s is really all about figuring out whether you need to build something, rather than processing the results of a build17:35
persia`So, in Soyuz, were we only have source-only uploads, I think it's safe to dispatch the source to the set of architectures defined in the .dsc, and accept the results it generates without P-a-s.17:36
persia`Because we know we have to build on at least that set of architectures.17:37
persia`The gdb example wouldn't generate gdb64 on amd64, armel, or ia64, but that ought be fine.17:37
maxbAnother aspect of P-a-s is figuring out whether a source needs a build on a particular architecture at all - i.e. perhaps it will generate zero binary packages on some archs17:39
persia`That should never be the case if the .dsc has "any".17:39
persia`(barring build failures)17:39
maxbI know I've seen real cases of this17:40
maxbI agree P-a-s is a total hack, fwiw, I just don't think we're in a position to get rid of it without a fair bit more work17:41
persia`If you find a case where some architecture listed in the .dsc really produces 0 binaries intentionally, I'd love to know the package.17:43
persia`But for now, I'll leave that alone, and just try to find someone to do X-Build-Affinity: (unless someone gives me a better name)17:44
bigjoolspersia`: X-Arch-Indep-Build-Arch would be a lot more obvious17:50
persia`bigjools: Changed in my notes.  Thanks.17:51
bigjoolsthanks17:51
jmlg'night all17:59
mrevellGoodly evelode everyone18:01
mrevellDamn, where's the close button in Lucid? :)18:01
=== beuno is now known as beuno-lunch
kfogelintellectronica: ec2 builds known to be failing right now?  https://pastebin.canonical.com/29489/18:35
=== beuno-lunch is now known as beuno
=== matsubara is now known as matsubara-afk
thumpermorning20:08
wgrantjml: Hi.20:32
wgrantjml: Ah, I see the email. That branch got stuck waiting for Soyuz to answer, I think.20:34
wgrantCan somebody please land https://code.edge.launchpad.net/~wgrant/launchpad/export-detailed-binary-download-stats/+merge/21828? The ec2test failed with the two buildfarm spurious failures that have been showing up recently.20:35
jmlwgrant: sure, I'll do it.21:05
wgrantjml: Thanks.21:06
bdmurrayIs there anybody would could run a simple production db query for me?21:07
jmlwgrant: can you do me a favour and repeat the URL? I need to use my other computer to land it.21:12
wgrantjml: https://code.edge.launchpad.net/~wgrant/launchpad/export-detailed-binary-download-stats/+merge/21828?21:12
wgrantThis is why we don't use OS X.21:12
jmlwgrant, :)21:13
jmlwgrant, even if I bought a computer with Ubuntu pre-installed, I wouldn't have got Launchpad set up on it today21:14
wgrantkfogel: Which were those failures? TestMinTimeToNextBuilder and buildd-slave.txt?21:16
wgrantelmo: Do you know about the dsync link-dupes call in cron.publish-ftpmaster? Why would there be dupes? The archive is not yet split at that point.21:21
jmlwgrant: it should be running through ec2 now.21:30
wgrantjml: It'll probably just fail spuriously again...21:30
jmlwgrant: perhaps.21:30
kfogelwgrant: https://pastebin.canonical.com/29489/, but now I have a run that's been going okay for a couple of hours at least21:36
wgrantkfogel: ENOTCANONICAL21:36
kfogelwgrant: oops, sorry21:37
kfogelwgrant: one sec21:37
kfogelwgrant: http://paste.ubuntu.com/399555/, and it looked like a file existence error21:38
kfogelwgrant: I'm treating it as spurious21:39
wgrantkfogel: That's not one I've seen before.21:39
wgrantBut yes, I find it unlikely that you broke that.21:39
kfogelwgrant: by the way, https://code.edge.launchpad.net/~kfogel/launchpad/78565-bug-comment-link-to-bug21:40
wgrantkfogel: Ah, good.21:41
wgrantI wish we had consistent rules for page titles, labels, breadcrumbs and root contexts.21:41
kfogelwgrant: the "comment" vs "message" thing really bugs me :-).  And that's not even in the UI.21:42
kfogelwgrant: I think everyone would agree that breadcrumbs should behave this way -- it's a bug in every case where they don't.21:42
wgrantkfogel: Well, Messages are used internally for others things as well.21:42
cody-somervillekfogel, Thats not spurious. That means your sourcecode tree is out of date.21:42
cody-somervilleerr... maybe spoke too soon.21:43
kfogelcody-somerville: oh, really?  I though I did an update this morning...21:43
=== matsubara-afk is now known as matsubara
wgrantThat's the opposite error to what one would normally get.21:43
wgrantPlus ec2test updates download-cache and sourcecode before each build.21:44
kfogelcody-somerville: but in any case, it's definitely up-to-date now, and the ec2 run is working.  So even if you're not right, you're not wrong :-).21:44
cody-somervilleShould file a bug on it. Ya know, the whole stop the line stuff.21:45
cody-somervilleThis is the system speaking to us and we're just not understanding it.21:46
cody-somerville<insert our LEAN type comments here>21:46
=== salgado is now known as salgado-afk
kfogelcody-somerville: I think it may be what noodles775 was already working on though22:04
lifelesswgrant: thats a while :P22:30
wgrantlifeless: Hm?22:33
lifeless15:21 < wgrant> lifeless: From what I can tell, publish-distro.py takes about 20-25 minutes currently.22:33
wgrantlifeless: Yeah, and apt-ftparchive is more than 17 minutes of that.22:34
wgrantFortunately 16 minutes of that can go away.22:34
wgrantpublish-distro.py itself runs for 20 minutes, normally. 2 minutes of that is inefficient domination (which I have down to <2s), 17 minutes of that is inefficient index generation (which I have down to <60s, or <20s if we ignore bzip2 indices, which we sadly can't), and the remaining <60s is other stuff.22:37
maxbdomination?22:45
lifelesswgrant: cool22:49
wgrantmaxb: Domination has two related meanings in Soyuz: 1) Marking packages Superseded and 2) Marking Superseded, Deleted or Obsolete packages for removal when they have no references.23:11
=== matsubara is now known as matsubara-afk
=== bjf is now known as bjf-afk

Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!