wgrant | StevenK: I can has blessing? | 00:00 |
---|---|---|
StevenK | thumper: https://code.launchpad.net/~wgrant/launchpad/bug-707741-addFile-master/+merge/47606 please | 00:00 |
StevenK | wgrant: I was getting there :-) | 00:01 |
wgrant | StevenK: Great, thanks. | 00:01 |
wgrant | So many regressions :( | 00:01 |
lifeless | leonardr: is there some reason we can't have a destructor parameter? That doesn't seem to alter the boolean nature | 00:02 |
wgrant | lifeless: Thanks. | 00:05 |
wgrant | lifeless: Bugs needs a kanban UI. | 00:09 |
wgrant | ... well, this is amusing. | 00:11 |
wgrant | getUnlandedSourceBranchRevisions has a reasonably complex query, and it is completely untested. | 00:11 |
wgrant | I expect that sort of thing from Soyuz, not Code :/ | 00:12 |
lifeless | poolie: https://pastebin.canonical.com/42398/ | 00:24 |
lifeless | -> allergy injection | 00:27 |
* henninge eods | 00:29 | |
wgrant | StevenK: Another regression fix for you, if you have a moment: https://code.launchpad.net/~wgrant/launchpad/bug-707808-unlanded-revisions/+merge/47617 | 00:36 |
StevenK | lifeless, thumper: https://code.launchpad.net/~wgrant/launchpad/bug-707808-unlanded-revisions/+merge/47617 if you please | 00:38 |
thumper | StevenK: ack | 00:58 |
* StevenK kicks the merging code | 00:59 | |
wgrant | thumper: https://code.launchpad.net/~deon-wurley/phpldapadmin/1.2.x <- are Remote branches without URLs legal? | 01:04 |
thumper | wgrant: yes... | 01:04 |
thumper | wgrant: but sad | 01:04 |
wgrant | WTF | 01:04 |
wgrant | But OK. | 01:04 |
thumper | wgrant: not much point really | 01:04 |
Ursinha | wgrant, hi | 01:04 |
wgrant | Ursinha: Hi, 'sup? | 01:05 |
Ursinha | wgrant, so, there are bunches of ApacheLogParserError | 01:05 |
wgrant | Ah, those. | 01:05 |
wgrant | All fixed, but not deployed yet. | 01:05 |
Ursinha | like this: https://lp-oops.canonical.com/oops.py/?oopsid=OOPS-1851PPA1051 | 01:05 |
wgrant | Since germanium isn't in nodowntime. | 01:05 |
Ursinha | hmm, I see. | 01:05 |
Ursinha | wgrant, thanks :) | 01:06 |
wgrant | I hope we can germanium in nodowntime soon :( | 01:07 |
wgrant | +put | 01:07 |
wgrant | thumper: Thanks. | 01:08 |
thumper | np | 01:08 |
wgrant | All major regressions fixed, yay. | 01:08 |
StevenK | Now deploy them? | 01:09 |
wgrant | Maybe in 8 or so hours. | 01:09 |
StevenK | Longer, devel will hit testfix | 01:11 |
wgrant | Heh. | 01:14 |
wgrant | Oh fuck, you're serious. | 01:14 |
wgrant | Gnrwgwnfjew | 01:15 |
thumper | why | 01:15 |
thumper | ? | 01:15 |
wgrant | That spurious librarian thing. | 01:15 |
thumper | oh FFS | 01:15 |
wgrant | We should kill the current build. | 01:15 |
thumper | yes | 01:15 |
thumper | +1 | 01:15 |
wgrant | spm: Can we easily kill a buildbot build? | 01:15 |
spm | yah. which one. | 01:15 |
StevenK | Where easy is er, not | 01:15 |
wgrant | If not, could you do it the hard way? :P | 01:15 |
wgrant | lucid_lp | 01:15 |
spm | where kill == stop. | 01:15 |
StevenK | spm: 556 on lucid_lp | 01:15 |
wgrant | I'm off to lunch now; could someone force a build once it's done? | 01:17 |
StevenK | wgrant: Aye | 01:18 |
StevenK | spm: Tell me when it stops hurting | 01:18 |
spm | so the lsave is completely ded. how do we clean this up so it won't run that revno again? | 01:18 |
StevenK | The revno is fine | 01:19 |
StevenK | The errors are spurious and we don't want to be in testfix | 01:19 |
spm | oh I see. perhaps getting the spurious nature of the errors fixed would be a GoodIdea™? ;-) | 01:19 |
StevenK | Here, have a Hudson | 01:19 |
StevenK | That should help | 01:19 |
spm | :-) | 01:20 |
StevenK | spm: Do you need to restart the master to get the slave back? | 01:20 |
spm | seems to be showing back to me. should be able to force and magic happen | 01:20 |
StevenK | It's buildbot, it isn't close to magic | 01:21 |
StevenK | spm: Done, thanks | 01:21 |
spm | anti-magic? | 01:21 |
StevenK | wgrant: Build forced | 01:21 |
* thumper stabs the librarian | 02:28 | |
thumper | AttributeError: 'NoneType' object has no attribute 'add_section' | 02:28 |
thumper | while trying to set up the librarian in tests | 02:28 |
thumper | anyone got any ideas? | 02:28 |
lifeless | poolie: ok I'm back | 02:40 |
lifeless | thumper: that means the layer which supplies the config to edit hasn't been setup | 02:40 |
thumper | lifeless: which I'm getting why? | 02:41 |
thumper | it has only just started happening | 02:41 |
thumper | after merging dev | 02:41 |
thumper | also | 02:41 |
thumper | bin/kill-test-services is borked | 02:41 |
lifeless | theres a bug on that | 02:41 |
lifeless | with an explanation; fix should be trivial | 02:41 |
poolie | lifeless, i think your mail is fine | 02:42 |
poolie | i replied too | 02:42 |
lifeless | poolie: well, I've just sent my mail | 02:45 |
poolie | interesting new reply from max | 02:46 |
thumper | lifeless: my librarian layer won't setup | 02:46 |
wgrant | thumper: Turn LP_PERSISTENT_TEST_SERVICES off. | 02:47 |
wgrant | It doesn't work any more. | 02:47 |
thumper | ah | 02:47 |
thumper | poo | 02:47 |
lifeless | iz bug | 02:47 |
lifeless | I put work in to make it work | 02:47 |
wgrant | It's been that way for a week or so now. | 02:47 |
lifeless | but its not actually tested | 02:47 |
wgrant | Ah. | 02:47 |
wgrant | I see. | 02:47 |
lifeless | I didn't *intend* to break it | 02:47 |
lifeless | but I couldn't tell that I *had* | 02:47 |
lifeless | also | 02:47 |
lifeless | its useless for parallel testing | 02:47 |
wgrant | I presumed you turned a blind eye to your breakage of it. | 02:48 |
wgrant | It is, yes. But we have no parallel testing yet. | 02:48 |
lifeless | so I'm not very interested in maintaining it | 02:48 |
lifeless | wgrant: we have some, a decent subset of tests work fine in parallel now | 02:48 |
wgrant | Indeed. | 02:48 |
wgrant | It is close. | 02:48 |
wgrant | The main problem now is AppServerLayer. | 02:48 |
lifeless | indeed | 02:48 |
wgrant | Plus some directories that need randomising. | 02:48 |
lifeless | yes | 02:49 |
maxb | lifeless: You binged? | 02:49 |
wgrant | 2.7 is close enough to be done by a maintenance squad, and I think parallel testing is probably almost there too. | 02:49 |
lifeless | maxb: python-subunit 0.0.6 in the lp ppa | 02:50 |
lifeless | maxb: part of fixing pqm to mail on conflicts | 02:50 |
lifeless | maxb: however it looks like the losas can grab from the bzr ppa ok | 02:50 |
lifeless | wgrant: feature work -> feature queue | 02:50 |
lifeless | wgrant: if you're looking for something to do, I see a single bug causing 16K oopses a day. | 02:50 |
lifeless | wgrant: (hint) | 02:51 |
wgrant | Heh, no, I'm not looking for stuff to do :( | 02:51 |
wgrant | That's the codebrowse connection closed one? | 02:51 |
StevenK | lifeless: No fair if it's loggerhead | 02:51 |
lifeless | wgrant: seriously, yes its close, but I guarantee it will have a long tail | 02:51 |
lifeless | StevenK: loggerhead probably has more test coverage the LP proper | 02:51 |
lifeless | wgrant: and the long tail is what makes me want to give it to a feature squad: get it live in buildbot/hudson, working in tarmac, profile it and assess scaling, disk IO utilisation, make recommendations on a new server to run massively parallel tests | 02:52 |
lifeless | wgrant: figure out the damn db leak bug | 02:52 |
wgrant | lifeless: I think that figuring that out is probably best achieved by deleting layers and zope.testrunner. | 02:53 |
lifeless | wgrant: I have handwavy plans for that | 02:53 |
lifeless | wgrant: its also pretty shallow | 02:53 |
lifeless | but, time. | 02:53 |
wgrant | Indeed. | 02:53 |
lifeless | wgrant: focus will get us places! | 02:53 |
StevenK | lifeless: I'd prefer if codebrowse actually looked like the rest of LP | 02:53 |
lifeless | StevenK: so would codebrowse :P | 02:53 |
lifeless | StevenK: it is themeable | 02:54 |
StevenK | But I can recall you telling me that's pointless in Dallas | 02:54 |
lifeless | StevenK: that doesn't sound like me; I think I may have said tha tthat is the least of the issues | 02:54 |
wgrant | StevenK: Everything is pointless in Dallas. | 02:54 |
lifeless | StevenK: which != pointless, but I could well have been confusing the issue | 02:55 |
poolie | i might try just running tip on a big tree | 02:55 |
poolie | and see how it does | 02:55 |
lifeless | huwshimi: hi | 02:56 |
huwshimi | lifeless: Hey there | 02:57 |
lifeless | huwshimi: I dunno if you've seen https://dev.launchpad.net/BugTriage yet ? | 02:57 |
huwshimi | lifeless: Uh, no. Are you referring to the bug I just filed? | 02:58 |
lifeless | huwshimi: yes I am :) | 03:00 |
lifeless | huwshimi: have a read :) | 03:00 |
huwshimi | lifeless: Yeah thanks. Reading it now. | 03:00 |
lifeless | huwshimi: its useful context to fit in with the team; the specific thing that alerted me was creating a medium importance bug (we don't use medium) and then starting work on it (thus ignoring the critical and high bugs) | 03:01 |
lifeless | huwshimi: I think your particular focus means that looking at all critical/high bugs is irrelevant to you | 03:01 |
lifeless | huwshimi: -but- you probably want to be sorting the design related bugs by the impact/importance you think they have | 03:02 |
lifeless | huwshimi: anyhow, no drama; we have a massive learning curve and I primarily wanted to let you know about this part of it :) | 03:03 |
huwshimi | lifeless: Sure thanks for letting me know. I'm about to submit a bunch of bugs so I'm glad you told me now :) | 03:03 |
huwshimi | lifeless: There was some discussion last week about triaging bugs ourselves. Was there a decision made about that? Am I better off not self triaging for now? | 03:14 |
lifeless | huwshimi: self triage is great | 03:18 |
lifeless | follow the guidelines in the wiki page | 03:19 |
lifeless | beyond that, if there are disagreements, english is a wonderful tool for figuring em out ;) | 03:19 |
huwshimi | lifeless: OK thanks for that :) | 03:19 |
* thumper grrs | 03:31 | |
* thumper is slowly getting there | 04:41 | |
StevenK | lifeless: So I don't think I can do the heavy lifting with SQL. If the recipe has no builds, http://pastebin.ubuntu.com/558848/ returns nothing, and so no daily builds get dispatched | 04:52 |
poolie | how do i, through the api, find all bugs assigned to a particular team? | 04:52 |
poolie | s/team/person | 04:52 |
lifeless | StevenK: looking | 04:53 |
lifeless | StevenK: you probably want either a LEFT OUTER JOIN or a UNION, or a COALESCE | 04:53 |
lifeless | StevenK: probably a left outer join | 04:53 |
StevenK | lifeless: Storm doesn't have a LeftOuterJoin? | 04:54 |
thumper | hmm... | 04:54 |
thumper | how do I emit a tag in a page template of a particular type? | 04:54 |
lifeless | StevenK: it does, LeftJoin IIRC | 04:55 |
lifeless | we use it all over | 04:55 |
thumper | view/tag is something like "h1", or "span" | 04:55 |
thumper | and I want to get <h1 id='${view/id}'> in the pt based on view/tag | 04:55 |
* thumper is open to suggestions | 04:55 | |
* StevenK is open to patches, never having done joins through Storm before | 04:55 | |
thumper | I'm thinking I may need an attribute and structure | 04:55 |
lifeless | StevenK: grep for LeftJoin | 04:56 |
lifeless | StevenK: I'm in the middle of a deep n meaningful over loggerhead | 04:56 |
lifeless | its breaking my brain to think SQL at the same time | 04:56 |
StevenK | Mission accomplished | 04:57 |
* StevenK hides | 04:57 | |
StevenK | lifeless: If you want to point what I'm doing wrong in http://pastebin.ubuntu.com/558856/ , that would be awesome. I can't find any usage like this in the tree | 05:19 |
* thumper is almost done refactoring widgets | 05:22 | |
lifeless | StevenK: wtf | 05:25 |
StevenK | Clearly then, the answer is "everything" | 05:26 |
lifeless | StevenK: the pattern is | 05:26 |
lifeless | (IIRC) LeftJoin(lefttable, righttable, condition) | 05:27 |
lifeless | e.g. | 05:27 |
lifeless | SourcePackageRecipe LEFT OUTER JOIN SourcePackageRecipeBuild on SourcePackageRecipeBuild.recipe == SourcePackageRecipe.id | 05:28 |
lifeless | -> | 05:28 |
lifeless | LeftJoin(SourcePackageRecipe, SourcePackageRecipeBuild, SourcePackageRecipeBuild.recipe == SourcePackageRecipe.id) | 05:28 |
lifeless | the result of that is a table itself | 05:28 |
lifeless | so to nest you'd do | 05:28 |
lifeless | LeftJoin(LeftJoin(SourcePackageRecipe, SourcePackageRecipeBuild, SourcePackageRecipeBuild.recipe == SourcePackageRecipe.id), PackageBuild, PackageBuild.id ==SourcePackageRecipeBuild.package_build_id) | 05:29 |
lifeless | but you don't need a nested left join | 05:29 |
lifeless | you only need one outer join - at the point you're willing to have a NULL row | 05:29 |
lifeless | so | 05:29 |
lifeless | Join(LeftJoin(SourcePackageRecipe, SourcePackageRecipeBuild, SourcePackageRecipeBuild.recipe == SourcePackageRecipe.id), PackageBuild, PackageBuild.id ==SourcePackageRecipeBuild.package_build_id) | 05:29 |
lifeless | etc | 05:29 |
lifeless | Join(Join(LeftJoin(SourcePackageRecipe, SourcePackageRecipeBuild, SourcePackageRecipeBuild.recipe == SourcePackageRecipe.id), PackageBuild, PackageBuild.id ==SourcePackageRecipeBuild.package_build_id), BuildFarmJob, BuildFarmJob.id == PackageBuild.build_farm_job_id) | 05:30 |
lifeless | is your using object | 05:30 |
lifeless | then in the where clause you put | 05:30 |
lifeless | Or(BuildFarmJob.id == None, BuildFarmJob.date_created > one_day_ago) | 05:30 |
thumper | w00t w00t | 05:35 |
lifeless | StevenK: does that make sense? | 05:35 |
lifeless | StevenK: use LP_DEBUG_SQL to see the sql being emitted | 05:35 |
lifeless | StevenK: and adjust until you have a query you're happy with | 05:35 |
StevenK | Yes, it made sense | 05:36 |
lifeless | ok | 05:36 |
lifeless | did it help ? | 05:36 |
StevenK | lifeless: Not really, it still doesn't return SPRecipes that haven't built | 05:40 |
StevenK | I suspect SourcePackageRecipeBuild.id == None will help, since they are created for builds, which then creates the PackageBuild and BFJ | 05:41 |
lifeless | StevenK: you probably want to bring back the sprb as well | 05:43 |
lifeless | since that will tell you the when. Or perhaps not. | 05:44 |
* thumper EODs | 06:03 | |
* huwshimi waves goodbye | 06:08 | |
lifeless | StevenK: need more help? I have cycles in ~ 10 | 06:21 |
StevenK | lifeless: Sorry, I was picking up Sarah, and I EOD'd 1.6 hours ago | 06:42 |
lifeless | kk | 06:44 |
lifeless | grab me tomorrow | 06:44 |
lifeless | and we can nut it out | 06:45 |
StevenK | lifeless: Was my plan | 07:12 |
=== almaisan-away is now known as al-maisan | ||
adeuring | good morning | 09:19 |
mrevell | Guten morgen | 09:22 |
=== al-maisan is now known as almaisan-away | ||
=== allenap changed the topic of #launchpad-dev to: Launchpad development https://dev.launchpad.net/ | PQM is open | firefighting: - | On call reviewer: allenap | https://code.launchpad.net/launchpad-project/+activereviews | ||
=== salgado-afk is now known as salgado | ||
gmb | Does anyone know what in the sweet hell this is about?: | 11:35 |
gmb | An error occurred during a connection to launchpad.dev. | 11:35 |
gmb | SSL received a record that exceeded the maximum permissible length. | 11:35 |
gmb | (Error code: ssl_error_rx_record_too_long) | 11:35 |
wgrant | gmb: This is on a fresh LP install? | 11:42 |
gmb | wgrant: No, it's an old one. But I did have to re-run rocketfuel-setup recently to correct a couple of pebkacs. | 11:45 |
Ursinha | good morning launchpad | 11:47 |
* gmb re-does the apache config dance | 11:47 | |
wgrant | gmb: Odd. I normally only see that when it's a fresh install needing an Apache restart, or I've broken the vhost config somehow. | 11:49 |
gmb | wgrant: Yeah. I think I've borked the vhost config in some myserious way (probably because I have a remote access setup and rocketfuel-setup fought with it). | 11:49 |
wgrant | Ah. | 11:49 |
gmb | I'm redoing it now. | 11:49 |
maxb | I find the nicest way to manage a remote access thingy is to let rocketfuel-setup manage the local-launchpad config file, but a2dissite it, copy it, and modify the copy | 11:50 |
=== Ursinha is now known as Ursinha-afk | ||
gmb | maxb: Very wise. I shall do that henceforth. | 11:54 |
gmb | Hoorah. It works. | 11:55 |
gmb | /me lunches | 11:58 |
deryck | Morning, all. | 12:06 |
Ursinha | morning deryck | 12:08 |
=== mrevell is now known as mrevell-lunch | ||
salgado | what's the best thing to use when writing a setup.py these days? setuptools? distribute? | 12:20 |
jelmer | salgado: what sort of features do you need from it? | 12:24 |
jelmer | distutils is widely available and seems to work pretty well for basic python modules | 12:24 |
salgado | jelmer, our needs seem to be rather basic, so maybe distutils will be enough indeed | 12:29 |
jml | salgado: I copy an existing setup.py :) | 12:31 |
salgado | jml, that's always a good strategy | 12:45 |
jml | can someone give me a dumb manager's summary of the codebrowse discussion on the mailing list? | 13:01 |
=== jcsackett changed the topic of #launchpad-dev to: Launchpad development https://dev.launchpad.net/ | PQM is open | firefighting: - | On call reviewer: allenap, jcsackett | https://code.launchpad.net/launchpad-project/+activereviews | ||
=== mrevell-lunch is now known as mrevell | ||
=== almaisan-away is now known as al-maisan | ||
salgado | benji, your last message to that mp I reviewed yesterday had only some quoted text from my previous message | 14:13 |
benji | salgado: hmm, that's odd. I just said "sorry about that, thanks for asking Curtis to look at it" | 14:14 |
salgado | benji, looks like the issue I encountered yesterday; already reported a bug for it | 14:15 |
benji | do you have the bug number at hand? | 14:16 |
abentley | jelmer, ping | 14:18 |
abentley | jelmer, unping | 14:18 |
oyv | hi | 14:18 |
oyv | i'm trying to setup launchpad on a virtual machine, but when i try make run i get an importerror (no module named loom.branch) | 14:19 |
oyv | anybody got any hints? | 14:19 |
oyv | http://pastebin.com/JBn59ekF | 14:20 |
abentley | flacoste, now that domain expertise is spread across squads, maybe it would be good to have a list of domain experts, e.g. on the wiki? | 14:22 |
oyv | (i can import the branch in python shells, so it should be installed and everything) | 14:25 |
jcsackett | oyv: are you following the instructions on the wiki? | 14:28 |
oyv | i use isntructions from here: https://dev.launchpad.net/Getting | 14:29 |
oyv | (and the "running" page) | 14:29 |
jcsackett | oyv, ok. what version of ubuntu are you running? | 14:30 |
oyv | Ubuntu 10.04.1 LTS (lucid lynx) | 14:32 |
jcsackett | oyv: i encountered this at one point; i am trying to remember how i fixed it. | 14:34 |
jcsackett | oyv: i assume you are not actually using bzr loom? it shouldn't fail in that case, i'm just defining what's going on. | 14:38 |
oyv | i don't think i'm using it, not really sure what it is ;) | 14:39 |
jcsackett | it's a plugin for bzr, http://doc.bazaar.canonical.com/plugins/en/loom-plugin.html. many lp developers use it, but last i checked it was not a dependency. | 14:40 |
jcsackett | and if it is, it should have been automatically installed when you hit the update step in the get/run instructions. | 14:41 |
jml | jcsackett: it is a dep | 14:41 |
jcsackett | jml: really? did that change? when i started on lp i was told it wasn't, b/c i was hitting this (or a similar) issue. | 14:41 |
oyv | edb@launchpad:~/launchpad/lp-branches/devel$ bzr plugins | 14:42 |
oyv | <snip> | 14:42 |
oyv | loom 2.1.0 Loom is a bzr plugin which adds new commands to manage a loom of patches. | 14:42 |
* jcsackett supposes he could have been deasily misinformed. | 14:42 | |
jml | jcsackett: since 2008, I think. it's needed to process looms | 14:42 |
oyv | doesn't that mean it's installed? | 14:42 |
benji | either is fine with me; but more importantly it means I need to go pour my coffee right now | 14:43 |
benji | pfft, wrong chan | 14:43 |
jcsackett | jml: dig. clearly i was misinformed. :-P | 14:45 |
jcsackett | oyv: sorry, i'm not sure what might be going on. it does seem that it is installed as a plugin. | 14:51 |
jcsackett | i could be of more help if i had a functioning machine right now, but i'm rebuilding after hardware failure yesterday, so i can't explore much on my end. :-/ | 14:52 |
=== al-maisan is now known as almaisan-away | ||
oyv | ok, thanks anyway :) | 14:53 |
LPCIBot | Project devel build (396): FAILURE in 4 hr 38 min: https://hudson.wedontsleep.org/job/devel/396/ | 14:55 |
LPCIBot | Launchpad Patch Queue Manager: [r=lifeless, | 14:55 |
LPCIBot | stevenk][bug=707741] Fix LibrarianClient.addFile to function under | 14:55 |
LPCIBot | SlaveDatabasePolicy. | 14:55 |
maxb | oyv: I don't suppose you get a traceback related to that ImportError? | 14:55 |
oyv | http://pastebin.com/JBn59ekF | 14:56 |
maxb | When you're running "make run", the copy of bzr-loom that is supposed to be being used is the one found via bzrplugins/loom/ in the Launchpad tree | 14:56 |
salgado | benji, it's bug 708258 | 14:56 |
_mup_ | Bug #708258: Failed to parse merge proposal comment sent via email <code-review> <email> <Launchpad itself:Triaged> < https://launchpad.net/bugs/708258 > | 14:56 |
benji | salgado: thanks | 14:56 |
=== Ursinha is now known as Ursinha-afk | ||
oyv | hmm | 14:58 |
* maxb sobs a bit as launchpad-database-setup tries to configure pg8.2 | 14:59 | |
oyv | there's only one folder in the bzrplugin folder, lpserve | 14:59 |
maxb | Your tree is broken then | 14:59 |
oyv | damnit.. | 14:59 |
oyv | ;) | 14:59 |
oyv | thanks | 14:59 |
flacoste | abentley: that's what the wiki pages related to the Launchpad in 30 minutes presentation was meant to do | 14:59 |
flacoste | abentley: https://dev.launchpad.net/Foundations/ComponentHelp | 15:00 |
flacoste | i think Tim did something similar | 15:00 |
flacoste | and so did Bugs | 15:00 |
flacoste | not sure if Soyuz and Translations put their stuff in that format | 15:01 |
abentley | flacoste, Ah, didn't think of looking there. | 15:02 |
deryck | ah, adeuring, call time. Sorry got distracted running tests and answering emails. | 15:02 |
flacoste | abentley: it would probably make sense to collate all the content in one place, or at least make an easy-to-find index page to it | 15:02 |
adeuring | deryck: ok, no problem | 15:02 |
bigjools | goooooood morning | 15:02 |
abentley | flacoste, +1 | 15:02 |
=== Ursinha-afk is now known as Ursinha | ||
abentley | bigjools, can you confirm that we never send emails for successful binary builds? Because there are some tests of Build.notify with BuildStatus.FULLYBUILT. | 15:07 |
bigjools | abentley: I can confirm that | 15:09 |
bigjools | that test sounds a bit bong | 15:09 |
abentley | bigjools, great. | 15:09 |
deryck | henninge, I need 5 minutes to wrap up some notes, and then we can chat. if that works for you. | 15:11 |
henninge | deryck: that's fine | 15:11 |
abentley | bigjools, since recipe builds want to notify on success, I'm moving the differentiation into BinaryPackageBuild.notify/SourcePackageRecipeBuild.notify, and fixing the tests. | 15:12 |
bigjools | abentley: you *want* them to notify on success or you're removing that? | 15:12 |
abentley | bigjools, I *want* them to notify on success. | 15:12 |
bigjools | ok | 15:13 |
james_w | really? | 15:13 |
bigjools | sounds odd to be | 15:13 |
bigjools | me | 15:13 |
james_w | will the coalesce? | 15:13 |
james_w | they | 15:13 |
abentley | james_w, yes. How else do I know that a recipe build has happened? | 15:13 |
bigjools | prepare for wrath if you do this! | 15:13 |
abentley | bigjools, we've been doing this from the start. | 15:13 |
bigjools | interesting | 15:14 |
james_w | because they happen every day? because Launchpad is reliable and does what I ask? | 15:14 |
bigjools | we really need a better email notification story | 15:14 |
abentley | james_w, no, they only happen when the source changes. | 15:15 |
abentley | james_w, my bzr recipe triggers once or twice a month. | 15:15 |
james_w | are you /really/ going to send people 200 emails every day in the extreme case? | 15:15 |
james_w | how is that useful? | 15:15 |
=== salgado is now known as salgado-lunch | ||
abentley | james_w, it's useful so that people know when there's a new build. | 15:15 |
james_w | but that information isn't very useful at all as you move towards heavy users | 15:16 |
abentley | james_w, if you get it and you don't want it, you can filter it out. If you don't get it and you want it, you have no option. | 15:16 |
james_w | do you really want to discourage heavy use of the service using email volume | 15:16 |
allenap | gmb: Do you have time to do a sanity check on https://code.launchpad.net/~allenap/launchpad/freedesktop-importance-flip-flop-bug-707478/+merge/47667 please? | 15:17 |
gmb | allenap: Sure. | 15:17 |
james_w | client-side filtering has been deemed to be not sufficient in the bugs case | 15:17 |
abentley | james_w, client-side filtering is better than no mail. | 15:19 |
james_w | are you sure your users would agree with that? | 15:19 |
abentley | james_w, I am sure you can find users to disagree with anything. | 15:19 |
james_w | well, let's all go shopping then | 15:20 |
gmb | allenap: Approved. | 15:22 |
abentley | james_w, I am just fixing a bug where the wrong kind of notification is sent, not increasing the number of notifications. | 15:22 |
allenap | gmb: Thank you :) | 15:23 |
james_w | ok, then you are absolved from any responsibility for the system | 15:23 |
abentley | james_w, so I've been talking with jelmer about this, and we both agree that there's a lot of room for improvement in the build notification story. | 15:26 |
abentley | james_w, jelmer would like to see a notification of successful binary builds, grouped so that you only get one email for multiple builds. | 15:27 |
abentley | james_w, and then we would be able to omit the success emails for recipe builds. | 15:28 |
bigjools | all emails that LP sends should be controllable from a single page | 15:29 |
bigjools | per person, I mean | 15:29 |
abentley | bigjools, I don't know what to think about that. It would be a verrrry long page. | 15:30 |
bigjools | potentially but not always | 15:30 |
bigjools | it can be ajaxified but you know what I mean - the subject of being sent machine-generated email is very divisive | 15:31 |
abentley | bigjools, generally, asking users to make choices is bad. The fewer choices, the smoother something feels. This makes me think that the need for such a page indicates a design problem. | 15:35 |
henninge | allenap: thanks for the review! | 15:40 |
bigjools | abentley: that's the Gnone way, which is completely disagree with | 15:40 |
abentley | bigjools, I'm not disputing the actual need, but I do wonder whether we're not thinking big enough. | 15:40 |
bigjools | s/is/I/ | 15:40 |
henninge | adeuring: did you see my review of your branch? | 15:41 |
bigjools | anyway, I'm not bikeshedding over this | 15:41 |
adeuring | henninge: yes, thanks. Actually, I think we should keep the assert() calls in setUp() because it is so esay to cause a mess there, and I am not sure if this would always result in test failures | 15:42 |
henninge | adeuring: well, I think that is true for a lot of places in the code, though. | 15:43 |
henninge | adeuring: but I don't really mind leaving them in there. | 15:44 |
adeuring | henninge: OK; perhaps my concerns are related to my unfamiliarity with translations ;) | 15:44 |
henninge | adeuring: ... I tried not to be so blunt ;-) | 15:45 |
henninge | adeuring: I have done that before, added asserts to make sure I got things right but once it worked, I removed them. | 15:45 |
adeuring | henninge: ;) but nevertheless, if we can screw the setup just by exchanging two factory calls, I think it is worth to check we don't do that... | 15:45 |
henninge | adeuring: that's why I suggested leaving just the one in there. | 15:46 |
henninge | and a comment | 15:46 |
henninge | "# The order of creation is important here to make sure is_current_ubuntu is set to False. | 15:47 |
henninge | self.assertFalse(message.is_current_ubuntu)" | 15:47 |
henninge | adeuring: ^ like that, only use the right variable for 'message'. | 15:48 |
allenap | gmb: Do you remember why sourceforge.net bug watches are disabled? | 16:01 |
gmb | allenap: Because our super-duper sourceforge slurping screenscraping fantastical disaster rather relied on them never, ever changing their templates, and they did. | 16:02 |
gmb | (I might have overstated how good that code actually was) | 16:02 |
allenap | gmb: Oh yes, thanks. I've just had a look at their templates and they're very much simpler now. Might fix that bad boy. | 16:09 |
gmb | allenap: Don't they offer an API now? | 16:09 |
allenap | gmb: Do they? Awesome. | 16:10 |
gmb | ISTR they do. Maybe I saw that on the ForgePlucker mailing list. | 16:10 |
=== salgado-lunch is now known as salgado | ||
abentley | allenap, jcsackett could you please review https://code.launchpad.net/~abentley/launchpad/build-mail3/+merge/47679 ? It's long, but only because of indentation fixes in a doctest. | 16:14 |
jcsackett | abentley: sure. | 16:14 |
abentley | jcsackett, thanks. | 16:15 |
allenap | gmb: I can't find an API, but tell me if you happen upon it. | 16:15 |
gmb | allenap: Yeah, I'm not able to find one either, disappointingly. Must have been a happy dream. | 16:16 |
gmb | I need new dreams. | 16:16 |
allenap | Hehe, you do :) | 16:16 |
=== almaisan-away is now known as al-maisan | ||
abentley | deryck, I feel like I lack the domain knowledge to dive into any of the tasks on the Kanban board. Can you suggest one? | 16:30 |
* deryck is looking | 16:30 | |
deryck | abentley, so I'm sure I lack the domain knowledge too :-) However, the card for bug 696009 seems a nice next step..... | 16:31 |
_mup_ | Bug #696009: Provide ITranslationMessage.shareIfPossible unit tests <tech-debt> <upstream-translations-sharing> <Launchpad itself:Triaged> < https://launchpad.net/bugs/696009 > | 16:31 |
deryck | abentley, it's still in the area of test clean up. so will broaden domain knowledge as cleanup happens. | 16:32 |
abentley | deryck, okay, I'll tak that. | 16:33 |
abentley | s/tak/take. | 16:33 |
deryck | ok, cool | 16:33 |
=== deryck is now known as deryck[lunch] | ||
henninge | adeuring: am I to expect a new revision on your branch? Otherwise I'll start landing mine (which includes yours and abentley's) | 16:35 |
henninge | actually, I'll make sure to merge the latest first | 16:35 |
adeuring | henninge: well, I'd prefer to leave the tests as they are... so, no new revesion ;) | 16:38 |
henninge | adeuring: ok, thanks. ;) | 16:38 |
jcsackett | abentley: comments on your MP. feel free to answer there or here. | 16:46 |
abentley | jcsackett, fire away. | 16:46 |
jcsackett | abentley: as commented on the MP, i'm wondering about the use of self.builds in the from_build method. | 16:47 |
jcsackett | mainly, in the first hit, can that get big enough to cause performance issues? | 16:48 |
abentley | jcsackett, it is always one or zero hits. | 16:48 |
jcsackett | abentley: so does the cache help? the cache is always gone at the end of the transaction, yes? | 16:49 |
gmb | I broke the build; rolling it back now. Sorry folks. | 16:50 |
abentley | jcsackett, the cache will not help with the cases we have at hand, because we only use it once. | 16:50 |
jcsackett | so putting it in cached_property is forward looking? | 16:50 |
abentley | jcsackett, but I felt that it made sense to cache it since one does not expect an attribute to be expensive. | 16:51 |
=== al-maisan is now known as almaisan-away | ||
gmb | Oh, nice. I specified --rollback for lp-land but it doesn't seem to have tagged the commit message thus. | 16:51 |
* gmb files a bug. | 16:51 | |
abentley | jcsackett, you could say putting it in cached_property is forward-looking. | 16:53 |
jcsackett | abentley: so do you see any problem with the use of self.builds being called in there? since no preloading will happen, can hitting self.builds get sufficiently expensive to be a worry? | 16:53 |
jcsackett | i'm not saying it will. i'm not familiar with this part of the codebase, so i'm wondering. | 16:54 |
=== almaisan-away is now known as al-maisan | ||
abentley | jcsackett, I don't know what you're asking. Are you wanting me to run an EXPLAIN on the query? | 16:54 |
jcsackett | abentley: i'm just double checking performance concerns. | 16:55 |
abentley | jcsackett, what kind of answer are you looking for? | 16:55 |
jcsackett | abentley: builds is a SQLMultipleJoin; i'm wondering if when self.build gets called we may return a huge rowset in some cases. | 16:57 |
abentley | jcsackett, as I said, it's always one or zero results. | 16:57 |
=== al-maisan is now known as almaisan-away | ||
jcsackett | abenltey, ah! i thought you meant from_build was called one or zero times. | 16:57 |
abentley | jcsackett, there is a comment in the code saying it shouldn't be a multiple join. | 16:57 |
jcsackett | abentley: yes, i see now. | 16:59 |
jcsackett | apologies for the confusion. | 16:59 |
abentley | jcsackett, cool | 16:59 |
=== Ursinha is now known as Ursinha-lunch | ||
jcsackett | abentley: r=me. i need follow up, as a mentee :-P. bac, can you follow up on https://code.launchpad.net/~abentley/launchpad/build-mail3/+merge/47679? | 17:01 |
=== jcsackett changed the topic of #launchpad-dev to: Launchpad development https://dev.launchpad.net/ | PQM is open | firefighting: - | On call reviewer: allenap, jcsackett* | https://code.launchpad.net/launchpad-project/+activereviews | ||
=== matsubara is now known as matsubara-lunch | ||
abentley | jcsackett, I've added a comment per your suggestion. | 17:14 |
jcsackett | abentley: thank you. :-) | 17:19 |
=== deryck[lunch] is now known as deryck | ||
=== beuno is now known as beuno-lunch | ||
deryck | hurrah! Windmill re-enable branch finally made it through. | 17:43 |
=== matsubara-lunch is now known as matsubara | ||
lifeless | jml: morning | 18:26 |
lifeless | deryck: cool | 18:26 |
jml | lifeless: hi | 18:26 |
lifeless | jml: would you like a catchup call - we seem to be trading 1-liners this week | 18:27 |
jml | lifeless: Yes, I'd like one. Only have 15mins though. | 18:27 |
lifeless | skype? | 18:28 |
jml | lifeless: sure. | 18:28 |
=== beuno-lunch is now known as beuno | ||
thumper | flacoste: ping | 18:46 |
bac | hi abentley | 18:59 |
abentley | bac, hi. | 19:00 |
bac | abentley: i'm trying to mentor jcsackett's review. the diff in the MP is overwhelmed by lint changes. i've tried to get a diff with just the non-lint changes but failed. could you easily whip one up and paste it? | 19:00 |
abentley | bac, not sure. I don't think they were separate commits, or anything. | 19:02 |
abentley | bac, how's this? http://pastebin.ubuntu.com/559153/ | 19:04 |
bac | abentley: 228 is much better than 1654! :) thanks! | 19:05 |
aroman | hello, all! Is there any way of seeing how many people are using a PPA of mine/a package? Or even just seeing how many people have branched from bzr or downloaded a .tar.gz? Thanks! | 19:08 |
lifeless | aroman: there is an LP API that can get you download statistics for a PPA | 19:13 |
flacoste | hi thumper | 19:14 |
aroman | lifeless: ah, but there's no sort of graphical frontend to see that information? | 19:14 |
lifeless | not yet | 19:14 |
thumper | flacoste: can you talk briefly about your review? | 19:14 |
aroman | lifeless: ah, alright, well i'll check that out then. I assume it's python, right? | 19:14 |
lifeless | flacoste: hi; can I get a small timeslice today ? (but not for ~30) | 19:14 |
flacoste | thumper: sure, mumble? | 19:14 |
thumper | ok | 19:14 |
lifeless | aroman: its a RESTful api, but we have a python library you can use | 19:14 |
flacoste | lifeless: how small? | 19:15 |
lifeless | flacoste: 10 min ? | 19:15 |
aroman | lifeless: excellent, thanks! | 19:15 |
lifeless | flacoste: crossing t's dotting i's on the loggerhead thing | 19:15 |
LPCIBot | Yippie, build fixed! | 19:27 |
LPCIBot | Project devel build (397): FIXED in 4 hr 31 min: https://hudson.wedontsleep.org/job/devel/397/ | 19:27 |
LPCIBot | Launchpad Patch Queue Manager: [r=gary][ui=none][bug=704685] BugSubscription.bug_notification_level | 19:27 |
LPCIBot | is now exposed in the devel webservice API. | 19:27 |
gary_poster | yay gmb ! | 19:27 |
gary_poster | oh, boo | 19:28 |
gary_poster | that's hudson | 19:28 |
gary_poster | on buildbot, it is being rolled back :-/ | 19:28 |
lifeless | oh, why? | 19:28 |
gary_poster | https://lpbuildbot.canonical.com/builders/lucid_lp/builds/561/steps/shell_6/logs/summary | 19:29 |
lifeless | ah scaling test failed | 19:30 |
lifeless | I'd like to make those more reliable and isolated | 19:30 |
lifeless | they are just a little flaky atm | 19:30 |
lifeless | (different standalone vs in a larger run) | 19:30 |
gary_poster | yeah | 19:31 |
lifeless | I think they are still a net win | 19:31 |
lifeless | but it can be frustrating getting them going | 19:31 |
lifeless | allenap: around? | 19:38 |
=== Ursinha-lunch is now known as Ursinha | ||
jtv | bigjools: got a minute for a question about FTPArchiveHandler? | 19:58 |
bigjools | jtv: yup | 19:58 |
jtv | great | 19:59 |
jtv | I was just wondering… it uses the LoopTuner all over the place already, but only for gc. | 19:59 |
jtv | Was it originally meant to commit transactions as well? | 19:59 |
bigjools | no, it's writing out files for apt-ftparchive to use | 20:00 |
jtv | Okay, but it's not writing to the DB that I can see and it's a big suspect in the long-transaction crime mystery I'm pursuing. | 20:01 |
jtv | So it'd seem that a few commits would be a relatively harmless way to get around the timeouts. | 20:02 |
jtv | I was thinking perhaps I could force the main body of work on the slave store, and throw in a few commits. | 20:03 |
* bigjools thinks | 20:03 | |
bigjools | that code is probably the bit I understand the least in soyuz | 20:03 |
bigjools | it's never gone wrong since cprov left so I never had to look at it :) | 20:03 |
bigjools | jtv: I'm a bit scared of using the slave store in case of replication delays | 20:05 |
jtv | Scared is sensible. But what's there to replicate? | 20:06 |
bigjools | all the publishing data it relies on, which is fast-changing | 20:06 |
jtv | I see. | 20:07 |
bigjools | jtv: it would be a good start to work out if it really is keeping a transaction open | 20:08 |
bigjools | if it's not writing to the db can we change the $mumble | 20:08 |
jtv | It has to be. It's not committing anywhere, and it's looping over lots of packages AFAICS. | 20:08 |
jtv | The $mumble? | 20:08 |
bigjools | yeah, the thing | 20:09 |
jtv | Oh, the thing. | 20:09 |
jtv | The database policy? | 20:09 |
bigjools | isloation? | 20:09 |
bigjools | isolation ,even | 20:09 |
jtv | Isolation level? | 20:09 |
bigjools | dunno if that would affect it | 20:09 |
jtv | That would affect reads, not too much to do with writes. | 20:09 |
bigjools | can we connection r/o to the master? | 20:09 |
bigjools | sigh | 20:09 |
bigjools | connect* | 20:10 |
jtv | I don't think we can, no. | 20:10 |
jtv | Something we could try though is run a test with a slave-only policy, and see what fails. | 20:10 |
bigjools | ok | 20:11 |
bigjools | I see why you suggested the slave now | 20:11 |
allenap | lifeless: Hi. | 20:11 |
allenap | lifeless: Is this about the BugMessage crack? | 20:11 |
jtv | bigjools: Well it also helps scale-out and locking, of course. | 20:11 |
lifeless | yes | 20:11 |
lifeless | allenap: though I'm not sure its crack L) | 20:12 |
jtv | bigjools: unless there's something that changes the database and then immediately runs the script, we should get about as consistent a view of the database from a slave as we do from the master—just slightly older. | 20:12 |
allenap | I'm reading the code now to try and remind myself of the weirdnesses. | 20:13 |
lifeless | allenap: so | 20:13 |
lifeless | we decorate Message to make it an IndexedMessage but the actual relation is BugMessage which we don't expose at all | 20:13 |
flacoste | thumper: i'm back | 20:13 |
lifeless | thats beside the point though | 20:14 |
jtv | bigjools: "the thing"—https://www.ubersoft.net/comic/hd/2011/01/next-time-try-index-cards | 20:14 |
thumper | flacoste: want to continue? | 20:14 |
flacoste | thumper: sure thing | 20:14 |
lifeless | we can still decorate but get the indices from the db once they are cached | 20:14 |
bigjools | jtv: the thing I am scared of is writing out inconsistent indices in the Ubuntu archive. That would be fairly nasty. | 20:15 |
bigjools | now while I am scared I am not sure how realistic the chances of that are | 20:15 |
jtv | In that case, we could try moving to the slave and _not_ committing. | 20:16 |
jtv | If that works out we'd still get a single huge transaction, but also a guarantee that it takes no write locks. | 20:17 |
allenap | lifeless: Yeah, that sounds sane. | 20:17 |
bigjools | jtv: well publish-distro does do commits | 20:17 |
jtv | Yes, just not in its huge loops. | 20:17 |
bigjools | after a-f runs | 20:17 |
bigjools | it needs to mark stuff as publishe | 20:17 |
bigjools | d | 20:17 |
lifeless | allenap: the goal is to be able to do a range query on BugMessage and then get just the Messages we want | 20:18 |
jtv | bigjools: I guess that happens all the way at the end? | 20:18 |
lifeless | allenap: which will also let us move to ajax population of pages like bug 1 | 20:18 |
_mup_ | Bug #1: Microsoft has a majority market share <iso-testing> <ubuntu> <Clubdistro:Confirmed> <Computer Science Ubuntu:Invalid by compscibuntu-bugs> <EasyPeasy Overview:Invalid by ramvi> <GNOME Screensaver:Won't Fix> <Ichthux:Invalid by raphink> <JAK LINUX:Invalid> <The Linux OS Project:In Progress> <metacity:In Progress> <OpenOffice:In Progress by lh-maviya> <Tabuntu:Invalid by tinarussell> <Tivion:Invalid by shakaran> <Tv-Player:New> <Ubuntu:I | 20:18 |
allenap | lifeless: Judging by Bug._indexed_messages, that should work well. | 20:18 |
lifeless | allenap: indeed; I wrote Bug._indexed_messages in october or so as part of optimising within the prior schema | 20:19 |
allenap | lifeless: Like load-on-scroll? | 20:19 |
allenap | lifeless: Yeah, it looked like one of yours :) | 20:19 |
lifeless | allenap: hah! I'll take that as a complement. | 20:19 |
lifeless | allenap: and yes, load on scroll | 20:19 |
jtv | bigjools: if we did the actual work on the slave but the marking-as-published on the master, we'd risk marking a slightly newer version as published than we actually published. Is that a problem? | 20:19 |
allenap | Awesome. | 20:19 |
lifeless | allenap: should be easy if we can get one message + adjacent actions pretty cheaply. | 20:20 |
bigjools | jtv: there's 4 steps in the publisher: 1) write files to pool, 2) domination, 3) generate files for a-f and run it, 4) write release files | 20:20 |
bigjools | jtv: big problem ,yes | 20:20 |
allenap | lifeless: Adjacent actions? | 20:20 |
jtv | bigjools: though strictly speaking, I would expect that that problem already exists to about the same extent | 20:21 |
lifeless | allenap: look at a BugTask:+index page | 20:21 |
lifeless | allenap: it shows action action message action message etc interleaved | 20:21 |
lifeless | allenap: it pretends there is one sequence | 20:21 |
bigjools | jtv: actually, the publishing record would only get written by this process anyway at this point in its lifecycle | 20:22 |
allenap | lifeless: Ah, yes, I wrote a lot of that ;) I was having an association fail on the word "actions". | 20:22 |
allenap | Well, not a lot, some. | 20:22 |
jtv | bigjools: assuming there's no overlapping runs, I guess it'd only be a problem if there were a few last changes while the script was running, and then nothing before the next run (so it'd think it was already published or something). | 20:22 |
allenap | lifeless: Are you going to work on this, or are you softening me up to do it? | 20:23 |
flacoste | lifeless: i'm free whenever you are | 20:23 |
lifeless | allenap: I'm helping on the oops/performance aspect | 20:23 |
lifeless | allenap: future stuff will be feature cards I suspect, unless you were to have a sudden fit of zomg I want to do this | 20:24 |
lifeless | flacoste: voice? | 20:24 |
flacoste | lifeless: sure | 20:24 |
allenap | lifeless: Okay, so you'll get the index into the database, and the get-adjacent-actions and load-on-scroll stuff is up to others? | 20:24 |
flacoste | lifeless: skype me | 20:25 |
lifeless | allenap: yeah, I don't have long enough timeslices to do larger stuff | 20:25 |
allenap | lifeless: That's cool, I just wanted to be sure what's expected of me. | 20:26 |
lifeless | be excellent | 20:27 |
lifeless | thats it | 20:27 |
allenap | Heh :) | 20:28 |
bigjools | jtv: so it marks everything published before a-f runs | 20:30 |
bigjools | there's the long transaction | 20:30 |
jtv | bigjools: it may help my understanding if I see what that entails… do you happen to know where that is done? | 20:30 |
bigjools | jtv: look at lib/lp/archivepublisher/publishing.py | 20:31 |
* jtv looks at lib/lp/archivepublisher/publishing.py | 20:31 | |
bigjools | and scripts/publish-distro.py which is what calls it | 20:31 |
bigjools | C_doFTPArchive is for Ubuntu, C_writeIndexes is for PPAs | 20:32 |
jtv | Ah, that helps | 20:32 |
jtv | So the marking-as-published… what do I look for? | 20:32 |
bigjools | ArchivePublisherBase.publish() | 20:32 |
bigjools | follow it down to there from distroseries.publish() | 20:33 |
jtv | ah cool | 20:33 |
jtv | But I think the script commits soon after that happens anyway. | 20:33 |
bigjools | the latter being called from A_publish() | 20:33 |
jtv | The problem is in C_doFTPArchive | 20:34 |
jtv | (I think) | 20:34 |
bigjools | yes | 20:34 |
bigjools | each stage is wrapped it try_and_commit() | 20:34 |
bigjools | in* | 20:34 |
jtv | ahh got it: setPublished | 20:37 |
jtv | And we want to publish a state that's no older than that datepublished timestamp, right? | 20:38 |
bigjools | jtv: so my concern is that we only see half of the records that were just set published if C_doFTPArchive is using a different store | 20:40 |
bigjools | or maybe even none | 20:41 |
bigjools | but that's extreme | 20:41 |
jtv | I wonder if we could set "now minus replication lag" as the publication date. | 20:43 |
bac | hi deryck, you still around? | 20:43 |
bigjools | the date is not a concern | 20:43 |
bigjools | it's the inconsistency | 20:44 |
jtv | bigjools: would that change though if we otherwise ran everything on the slave store? | 20:45 |
bigjools | because what can happen is that we write the pool file, miss it in a-f and then write the release file with it in | 20:45 |
bigjools | that would be quite disastrous | 20:45 |
jtv | What's a-f stand for by the way? | 20:45 |
bigjools | Apt FTPArchive | 20:45 |
bigjools | in that case we'd end up with checksums that are incorrect | 20:46 |
jtv | You're saying the whole pool file might be lost? Or an individual package that's in there? | 20:46 |
bigjools | no | 20:46 |
bigjools | a-f writes the indexes - with replication lag it could miss something | 20:47 |
bigjools | then if that something replicates before we get to stage "D" where it writes the Release file, the Release file will be wrong compared to the index | 20:47 |
bigjools | I think | 20:48 |
jtv | Well if it's that complicated I probably can't afford to mess with it anyway. :/ | 20:48 |
bigjools | the publisher is *the* most critical part of soyuz, if it goes wrong we can do a lot of damage | 20:48 |
bigjools | which is why I am rather conservative in this area :) | 20:49 |
jtv | If we could move the whole thing minus something small over to the slave, we'd have a consistent view (just a slightly older one) and no worries. But if there's any sort of read-modify interaction it gets riskier. | 20:49 |
bigjools | yeah | 20:49 |
bigjools | we should go through it sometime and record all the interactions | 20:50 |
jtv | bigjools: would it make sense to do a trial run with a slave-only policy and setPublished disabled? | 20:53 |
bigjools | I dont think we have any tests that do everything at once like that, so we'd need to check the archive state manually | 20:54 |
jtv | Meaning we'd probably miss something? | 20:55 |
bigjools | possibly but if you point an apt client at the archive it would soon belly-ache | 20:57 |
bigjools | we should get wgrant to comment on this | 20:58 |
wallyworld_ | thumper: StevenK: we having standup? | 21:04 |
wallyworld_ | thumper: i'm talking, jsut a sec | 21:05 |
wallyworld_ | thumper: you ok, my sound backend bad | 21:05 |
wallyworld_ | fixing | 21:05 |
thumper | wallyworld_, StevenK: FYI https://code.launchpad.net/~thumper/launchpad/refactor-lazrjs-text-widgets/+merge/47634 | 21:05 |
StevenK | thumper: I can hear you too | 21:05 |
wallyworld_ | thumper: https://code.edge.launchpad.net/~wallyworld/launchpad/recipe-find-related-branches/+merge/47367 | 21:17 |
=== salgado is now known as salgado-afk | ||
huwshimi | Ugh, so my laptop has decided that when I'm logged in I should only be able to type numbers :( | 21:30 |
StevenK | Yet, you're typing not numbers. :-) | 21:30 |
huwshimi | StevenK: I am using my netbook to whinge about it | 21:31 |
StevenK | Heh | 21:31 |
wallyworld_ | thumper: https://pastebin.canonical.com/42451/ | 21:33 |
wgrant | bigjools: Hi | 21:34 |
bigjools | morning wgrant | 21:34 |
wgrant | bigjools: This whole discussion makes me cry. | 21:37 |
wgrant | Why do we want to move it to the slave? | 21:39 |
bigjools | wgrant: no doubt | 21:39 |
benji | huwshimi: numlock? | 21:39 |
huwshimi | benji: Thanks, I just figured that out right then. | 21:39 |
wgrant | We *could* do it, and things would remain consistent, but some things would show as published when they weren't. | 21:39 |
StevenK | thumper: I've been trying! | 21:39 |
bigjools | wgrant: jtv has been investigating but he wants to get rid of a long-running transaction | 21:39 |
wgrant | And the latency would be two hours. | 21:39 |
huwshimi | benji: They could have at least labelled it :) | 21:40 |
bigjools | wgrant: yes that's exactly my concern in addition to incorrect indices vs release files | 21:40 |
wgrant | bigjools: We may have extra stuff on disk, but we may also be removing stuff from disk later that is still referenced by the indices. | 21:41 |
wgrant | So no, we cannot sensibly use the slave. | 21:41 |
wgrant | However, yes, we can remove the long-running transactions by making the publisher not take 300 years. | 21:41 |
wgrant | But that is a fair bit of work. | 21:41 |
lifeless | we could add a ro connection to the master | 21:41 |
bigjools | about 300 years | 21:41 |
wgrant | lifeless: Is it only R/W transactions that are the problem? | 21:42 |
lifeless | wgrant: no | 21:42 |
wgrant | That's what I thought. | 21:42 |
lifeless | wgrant: but they are a bigger problem than r/o transactions | 21:42 |
lifeless | r/o transactions prevent index and row gc (mvcc chaff) | 21:42 |
lifeless | r/w transactions cause related row exclusive locks | 21:43 |
wgrant | Right. | 21:43 |
lifeless | but its the actual *changes* in r/w transactions that matter | 21:43 |
* bigjools gets tests working with a version column of type debversion \\o/ | 21:43 | |
lifeless | length is just a (poor) proxy for predicting problems. | 21:43 |
wgrant | There should be none. But it would be nice to verify that. | 21:43 |
wgrant | bigjools: Yay! | 21:43 |
bigjools | now, to optimise the queries | 21:44 |
StevenK | thumper: So I use Mumble on the old laptop the audio is choppy and I can talk, if I use the new laptop the audio is excellent and I can't talk | 21:44 |
thumper | heh | 21:45 |
bigjools | wgrant: btw, the 2-builds-per-builder happened again today | 21:45 |
wgrant | bigjools: Same builder? | 21:45 |
bigjools | no, after talking to lamont it turns out that it happens when he kills a long-running stuck build | 21:45 |
wgrant | I noticed 6 or so builders earlier disabled with strange messages. | 21:45 |
wgrant | But they were all happy once I re-enabled them. | 21:46 |
bigjools | because we rely on aborting nonvirt builders | 21:46 |
lifeless | flacoste: poolie: draft sent to you | 21:46 |
bigjools | so I am going to change to code disable the builder it we see it aborting | 21:46 |
wgrant | lifeless: Do you know what's going on with sinzui's mailing list QA? | 21:46 |
wgrant | bigjools: Why? | 21:47 |
bigjools | abort does not work on nonvirts | 21:47 |
wgrant | ABORTING will drop to ABORTED, modulo a slave bug which doesn't properly kill sbuild because of a missing trap. | 21:47 |
bigjools | exactly | 21:47 |
bigjools | so until that's fixed... | 21:47 |
lifeless | wgrant: yes | 21:47 |
lifeless | wgrant: can I brain dump and have you chase? | 21:47 |
wgrant | lifeless: That is what I was hoping for. | 21:47 |
lifeless | wgrant: ok so lp prod configs was broken for qastaging mailman | 21:48 |
lifeless | the qa port is 9097 | 21:48 |
wgrant | bigjools: Until that's fixed, buildd-manager will ignore the builder and there is a nagios check to tell lamont to fix it. | 21:48 |
wgrant | Right. | 21:48 |
lifeless | it said 8097 | 21:48 |
lifeless | there is a branch to fix that | 21:48 |
wgrant | I believe that got reviewed and landed. | 21:48 |
lifeless | so if it has | 21:48 |
wgrant | It should be good? | 21:48 |
lifeless | then it should be deployed and the mailman log should be calling into the api with the newer code | 21:49 |
wgrant | It doesn't seem to be landed. I might fix that. | 21:49 |
lifeless | remaining steps | 21:49 |
wgrant | I've only run mailman locally a couple of times, so I'm not 100% on how exactly it works, but I guess I'll work it out. | 21:49 |
wgrant | lifeless: qastaging should automatically update configs with its usual update, right? | 21:51 |
lifeless | wgrant: yes | 21:51 |
lifeless | wgrant: so, 1) get the right port out | 21:51 |
lifeless | 2) make sure its querying members (via the logs) | 21:52 |
lifeless | 3) profit | 21:52 |
wgrant | Great, thanks. | 21:52 |
wgrant | If all goes well, we can deploy in a couple of hours :) | 21:52 |
StevenK | lifeless: So, the query Storm is creating is: SELECT SourcePackageRecipe.build_daily, SourcePackageRecipe.daily_build_archive, SourcePackageRecipe.date_created, SourcePackageRecipe.date_last_modified, SourcePackageRecipe.description, SourcePackageRecipe.id, SourcePackageRecipe.is_stale, SourcePackageRecipe.name, SourcePackageRecipe.owner, SourcePackageRecipe.registrant FROM SourcePackageRecipe LEFT JOIN SourcePackageRecipeBuild ON SourcePac | 21:53 |
StevenK | Hm. That was a bit longer than it looked in the terminal | 21:53 |
wgrant | bigjools: We haven't purged non-main indices for existing PPAs yet, have we? | 21:54 |
lifeless | wgrant: new ppas won't have them at all | 21:54 |
wgrant | I know. | 21:54 |
wgrant | It broke a few things :) | 21:54 |
wgrant | But they're all fixed now. | 21:54 |
wgrant | But we were also going to manually remove the old ones. | 21:55 |
flacoste | lifeless: replied | 21:56 |
lifeless | thanks | 21:57 |
lifeless | poolie: hi | 21:57 |
bigjools | wgrant: no | 21:58 |
bigjools | wgrant: go ahead and clean up, then we can re-enable the daily job | 21:58 |
StevenK | lifeless: O hai. I have added a recipe with no builds and dropped the query and it returns 0 rows | 22:40 |
lifeless | StevenK: you need to paste the query without getting cut off :) | 22:40 |
StevenK | lifeless: http://pastebin.ubuntu.com/559270/ ; I've taken what LP_DEBUG_SQL gave me, replaced the %s's and dropped most of SourcePackageRecipe's columns from the SELECT | 22:42 |
jtv | hi wgrant! | 22:43 |
StevenK | If I drop the extra JOINs, it returns a row | 22:43 |
wgrant | Morning jtv. | 22:44 |
jtv | wgrant: bigjools & I were looking at the long transactions in the archive publisher earlier. | 22:45 |
wgrant | jtv: So I saw. | 22:45 |
jtv | You're everywhere. | 22:45 |
wgrant | We cannot sanely move it onto a slave. | 22:45 |
jtv | I was wondering whether we could move essentially the whole process over to a slave without risking inconsistencies. | 22:45 |
wgrant | We need to make it take less insanely long. | 22:45 |
jtv | Ah. | 22:45 |
jtv | That answers that. | 22:46 |
bigjools | jtv: wgrant made some more scary points that I'd not mentioned in addition to my scary ones | 22:46 |
jtv | Oh good | 22:46 |
jtv | that sounds like fun. | 22:46 |
jtv | Making it take less insanely long looks somewhat doable, but I'll need a way to test. | 22:46 |
jtv | It's FTPArchiveHandler.run that takes the bulk of the time, right? | 22:46 |
wgrant | There are two things that take forever: file list generation, and a-f itself. | 22:47 |
wgrant | The latter can be parallelised. | 22:47 |
lifeless | StevenK: I'm fidddling | 22:47 |
bigjools | wgrant: I'd like to see a-f and NMAF in a race | 22:47 |
lifeless | StevenK: trivially doing left joins all over works, but that can get inefficient | 22:47 |
jtv | wgrant: and the former looks like it may be a typical naïve-fetch-inside-loop pattern. | 22:48 |
bigjools | I think it would be close | 22:48 |
jtv | It's _almost_ a simple prejoin but there's an interaction with slicing to watch out for. | 22:48 |
wgrant | bigjools: a-f is hundreds of times slower. | 22:48 |
wgrant | Er. | 22:48 |
wgrant | NMAF is. | 22:48 |
StevenK | NMAF is *slower*? | 22:48 |
wgrant | jtv: Possibly. But the queries themselves are bad. | 22:48 |
wgrant | StevenK: Yes, it issues many many many more queries. | 22:48 |
jtv | wgrant: the pattern, or some of the individual ones? If you've got a list of known troublemakers, that'd help me see. | 22:49 |
wgrant | jtv: I don't recall exactly. But I believe it's fairly efficient query-count-wise, but the queries are not quick. | 22:50 |
wgrant | It's been more than a year since I seriously tried optimising this phase of the publisher. | 22:50 |
wgrant | I got a *long* way, but it was sort of full of hacks. | 22:50 |
wgrant | Down to 3-4 seconds for each index file, on NMAF, for 1.5x primary's size. | 22:51 |
wgrant | cprov also optimised the a-f file list code a lot just before he left. | 22:51 |
jtv | My may suspect based on following the arrows in the code was FTPArchiveHandler.publishFileLists. | 22:51 |
jtv | That contains some loops over all source/binary packages, I believe. | 22:52 |
wgrant | Yes, but that uses the views. | 22:52 |
bigjools | which are evil | 22:52 |
* jtv crosses himself | 22:52 | |
wgrant | It should not do any queries. | 22:52 |
jtv | Luckily I made merit by visiting the temple: saw a working Difference Engine 2 yesterday. | 22:52 |
* bigjools looks for garlic and silver | 22:52 | |
wgrant | Until right at the end when I do the disabled chekc. | 22:52 |
wgrant | (which is only once per pocket-das) | 22:52 |
jtv | What's a pocket-das? | 22:53 |
wgrant | (PackagePublishingPocket, DistroArchSeries) | 22:53 |
StevenK | hardy-i386-RELEASE | 22:53 |
StevenK | For instance | 22:53 |
wgrant | How many cores does cocoplum have? | 22:54 |
bigjools | StevenK, wgrant: I am putting my debversion stuff on DF, please to not be touching for a bit | 22:54 |
wgrant | bigjools: k | 22:54 |
=== matsubara is now known as matsubara-afk | ||
bigjools | unless you guys are using it in which case I can wait | 22:54 |
StevenK | wgrant: 4 | 22:55 |
wgrant | :( | 22:55 |
lifeless | StevenK: SourcePackageRecipeBuild JOIN PackageBuild ON PackageBuild.id = SourcePackageRecipeBuild.package_build JOIN BuildFarmJob ON BuildFarmJob.id = PackageBuild.build_farm_job right join SourcePackageRecipe ON SourcePackageRecipeBuild.recipe = SourcePackageRecipe.id | 22:55 |
lifeless | StevenK: put that in from 'FROM' ... 'WHERE' | 22:55 |
jtv | wgrant: it looked to me as if, unless we mess with the code's structure which I'm reluctant to do, it should do a bunch of batched prefetch queries in each iteration of its loop-tuner callback. If it's using views, maybe that's worth eliminating as well. | 22:56 |
wgrant | jtv: Because it's using views, it's already entirely prefetched. | 22:56 |
wgrant | All you can do is optimise the view. | 22:56 |
jtv | Actually, prefetching may be what's slowing it down then. | 22:56 |
wgrant | However, file list generation is only a couple of minutes. | 22:57 |
wgrant | Running a-f takes 15 or so. | 22:57 |
StevenK | lifeless: That looks good. How do I tell Storm that? | 22:57 |
lifeless | RightJoin | 22:57 |
StevenK | Instead of left? | 22:57 |
lifeless | StevenK: works like LeftJoin in terms of function calls | 22:57 |
lifeless | StevenK: yes; same parameters, same translation | 22:57 |
lifeless | StevenK: play with it a little to get the invocation you need | 22:58 |
lifeless | I'm just checking the query plan on staging | 22:58 |
lifeless | 96ms | 22:58 |
lifeless | so fast | 22:59 |
lifeless | StevenK: you'll also want DISTINCT, because you only want one row per sourcepackagerecipe | 23:00 |
lifeless | StevenK: as written, on staging, we can get | 23:00 |
lifeless | id | name | owner | 23:00 |
lifeless | ----+-------------+--------- | 23:00 |
lifeless | 15 | awn-testing | 1382524 | 23:00 |
lifeless | (3 rows) | 23:00 |
StevenK | lifeless: I'm still distilling your query into Storm. *Then* I'll worry about distinct | 23:04 |
lifeless | StevenK: I'd do it for you, but known how this works will help you | 23:04 |
jtv | wgrant: drat. So to speed the code up significantly we'd have to run parallel apt-ftparchive instances on separate file lists? | 23:06 |
wgrant | jtv: Yes. We already have separate file lists, so it's fairly easy. | 23:07 |
StevenK | lifeless: Storm has SELECT ... FROM SourcePackageRecipe; whereas your query has SELECT ... FROM SourcePackageRecipeBuild; is that pertinent? | 23:07 |
jtv | wgrant: and then… just concatenate the outputs? | 23:07 |
wgrant | jtv: No. Each file list results in one index. | 23:08 |
wgrant | If you look at the log, you'll see it generates one index for (maverick-release, source, main), another for (maverick-release, i386, universe), etc. | 23:08 |
wgrant | Each of those is big. | 23:09 |
wgrant | And each of those can be done in parallel. | 23:09 |
lifeless | StevenK: yes | 23:09 |
lifeless | StevenK: or probably yes, show me thje FROM..WHERE bit ? | 23:09 |
StevenK | lifeless: http://pastebin.ubuntu.com/559277/ | 23:10 |
jtv | wgrant: ah so parallel runs per architecture, plus one for source, and they should all overlap fairly well. | 23:11 |
wgrant | jtv: Yes. | 23:11 |
jtv | The hard part would probably managing the parallel processes. | 23:11 |
wgrant | Yes. And that's not terribly hard. | 23:11 |
wgrant | In the run I'm looking at, running a-f takes 16 minutes. File list generation takes 1.5. | 23:12 |
StevenK | lifeless: I suspect you got distracted? | 23:30 |
lifeless | StevenK: born distracted | 23:37 |
StevenK | Duh | 23:37 |
lifeless | StevenK: ok | 23:42 |
lifeless | so that thing you pasted means | 23:42 |
lifeless | 'select the things joined together where there are 0 or many sourcepackagerecipes' | 23:43 |
lifeless | what you want is | 23:43 |
lifeless | 'select source package recipes where there are 0 or many (things joined together)' | 23:43 |
lifeless | right joins allow NULL on the left hand side | 23:43 |
poolie | lifeless, spm, as a specific next step on bug 701329, can we get haproxy logs? | 23:43 |
_mup_ | Bug #701329: error: [Errno 32] Broken pipe <oops> <Launchpad itself:Triaged> <loggerhead:Triaged> < https://launchpad.net/bugs/701329 > | 23:43 |
lifeless | that is, they include 'every row on the right hand side' | 23:44 |
lifeless | poolie: can we not just catch the exception ? | 23:44 |
poolie | or, try to titrate them to a level where they tell us about interesting errors without flooding the system | 23:44 |
lifeless | poolie: what are you trying to determine | 23:44 |
poolie | why haproxy closed the connection | 23:44 |
poolie | and from which ip it came | 23:44 |
lifeless | poolie: can you back it out a step higher | 23:45 |
lifeless | I don't understand why this isn't a fairly direct code change | 23:45 |
poolie | so, to close this particular bug, we can, say | 23:47 |
poolie | just make it not oops when the connection is abruptly closed? | 23:47 |
poolie | perhaps it should log a warning-type error | 23:47 |
poolie | that seems pretty easy | 23:47 |
lifeless | none of our other servers log on this | 23:47 |
lifeless | *or* | 23:47 |
poolie | it makes me wonder if we are papering over a problem | 23:47 |
lifeless | none of our other servers are experiencing it | 23:47 |
poolie | so i'd like to see a bit more data about why the connection is being dropped | 23:47 |
poolie | haproxy is the obvious place to look for that | 23:48 |
poolie | and, i think it is a bit weird to have a production service where logging is entirely disabled | 23:48 |
poolie | (perhaps "it's weird" is not a reason to change it but i think it's reasonable to ask) | 23:48 |
lifeless | I thought you were told that it was a volume problem ? | 23:48 |
lifeless | haproxy being a bit of a firehose | 23:49 |
lifeless | anyhow | 23:49 |
lifeless | if you're working on this bug | 23:49 |
lifeless | thats cool | 23:49 |
poolie | lifeless, i'd guess that zope etc don't traceback on epipe, and i agree that in general loggerhead shouldn't | 23:49 |
lifeless | uhm, what can I do to hepl. | 23:49 |
lifeless | firstly, have you looked at some of the OOPSes? | 23:49 |
poolie | yes | 23:49 |
poolie | i'd like to turn logging >=WARNING | 23:49 |
poolie | and see if that is in fact a firehose | 23:49 |
poolie | if it is, that's a bit alarming, and i'd like to just see what it's logging | 23:50 |
poolie | then we can turn it off again | 23:50 |
poolie | is that ok? | 23:50 |
lifeless | I've no objections in principle, though the losas generally make informed decisions about this sort of thing :) | 23:50 |
poolie | spm, do you mind trying that? | 23:51 |
spm | we start with "no" and negotiate from there. | 23:51 |
poolie | sigh | 23:51 |
spm | poolie: nope, worth a shot. :-) | 23:51 |
poolie | thanks | 23:51 |
* StevenK was waiting for "Depends. Do you have cake?" | 23:51 | |
poolie | the haproxy manual says you can set the log leevl | 23:51 |
spm | StevenK: it's poolie. he lives closer than wgrant, and rides a motorbike ie Hells Angel in training. I've learnt to be nice to him. | 23:52 |
lifeless | poolie: so | 23:52 |
lifeless | poolie: looking at https://lp-oops.canonical.com/oops.py/?oopsid=1836CBA1 | 23:52 |
lifeless | poolie: there are several things that make this hard to analyze | 23:52 |
lifeless | poolie: if I was working on it, I would fix those first | 23:53 |
lifeless | poolie: firstly there is no total time recorded | 23:53 |
* StevenK suspects that poolie is shinier, due to lifeless completly context switching and goes afk for ten or so | 23:53 | |
lifeless | poolie: nor any timeline (which we could use to track xmlrpc / bzr call overhead if we wanted to) | 23:53 |
poolie | i've learned to front-load anything that may need SA action | 23:53 |
lifeless | StevenK: sorry, I thought I answered you? | 23:53 |
lifeless | StevenK: @10:44 | 23:54 |
lifeless | poolie: if we had timing data, we could assess whether this is really a timeout or not | 23:54 |
lifeless | poolie: in fact, we could add a thing to say 'if the time is > some N, convert to a timeout' | 23:55 |
poolie | yup | 23:55 |
lifeless | we probably need to expose an API call to ask for the timeout | 23:55 |
poolie | but if haproxy is saying "request blah blah timeout, killed" | 23:55 |
lifeless | e.g. 'what should my timeout be please?' | 23:55 |
poolie | that seems easier | 23:55 |
lifeless | poolie: I worry that we're going to need a lot of correlation to match up all 15K a day | 23:56 |
lifeless | poolie: gathering data at source seems more useful (to me) | 23:56 |
lifeless | poolie: anyhow, thats just what I'd do if I was working on it | 23:56 |
lifeless | however | 23:57 |
lifeless | note that haproxy probably won't tell us | 23:57 |
lifeless | if this is coming in behind haproxy | 23:57 |
lifeless | spm: how often does nagios query ? | 23:57 |
spm | i think it's up to 3 times every 10 minutes | 23:58 |
poolie | lifeless, i don't want to match up all of them, i just want to know if haproxy thinks its client is dropping the connection, or if it is dropping the connection itself | 23:58 |
poolie | or if it's oblivious | 23:58 |
poolie | it shouldn't be too hard to answer | 23:58 |
lifeless | sure | 23:58 |
spm | but probably easier to look in the logs and count | 23:59 |
lifeless | I am open to many ways to figure out whats up | 23:59 |
* thumper is busy writing real documentation | 23:59 |
Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!