[00:00] * cjwatson looks at the whole of r13734 [00:01] In that revision, it had one caller, which was always copying from series.previous_series to series [00:01] I'm right in my presumption that series.previous_series is always in the same distribution, aren't I? [00:01] Right [00:01] Yes [00:01] well [00:01] we hope so [00:02] I did check it was true for Ubuntu [00:02] The code ensures it's always true [00:02] The DB does not [00:02] So I think this was essentially an attempt at generalising to derived distributions [00:02] But that actually what we have now in PackageCopier does a better job [00:02] And the publisher code doesn't care [00:03] Yep [00:03] r=me [00:03] We really need to merge the uploader and copier eventually... [00:03] Yeah, it's a pretty ugly separation [00:04] They're kind of converging by iterative generalisation [00:04] And it's only getting uglier [00:04] Yeah [00:04] But stuff like that duplicated policy code is really hard to fix in the current layout [00:04] And there are subtle differences. [00:05] Largely unavoidable in the current layout, as you say [00:05] Different ancestry calculation between them [00:05] And within the uploader, come to that, but that's a different matter [00:05] Yes... [00:06] cf. my comment in check_copy_permissions :-) [00:06] really we want to rework PackageUpload+PackageCopyJob to work with just a "bunch of packages" [00:06] Then copies and uploads can use the same infrastructure, just from different sources [00:06] That would also probably necessitate making custom uploads not horrible [00:06] Or at least less horrible [00:07] They could do with having something that resembles a database representation [00:07] Myes. [00:07] Rather than just being bolted onto the side of PackageUpload and forgotten about [00:07] cjwatson: long tradition of that :) [00:08] cjwatson: when schema changes were impracticle [00:08] Mm [00:08] At least we don't have sourceless custom uploads any more :) [00:08] Indeed [00:11] Right, thanks, I've sent that off to EC2. I'll QA in the morning if it's got that far [00:11] " 1 → 75 of 296 results " [00:11] ! [00:11] cjwatson: Great [00:37] wgrant: Criticals? [00:38] StevenK: Yeah [00:38] In launchpad, though, not launchpad-project. [00:38] Yeah [00:39] wgrant: Right, I can see the 35 BMPs [00:40] Well, my query doesn't return the BMP ids, it returns the duplicated merge_diffs, but the 35 is the important bit. [00:40] Is it actually 35? [00:40] Right [00:40] SELECT merge_diff FROM branchmergeproposal WHERE merge_diff IS NOT NULL GROUP BY merge_diff HAVING COUNT(*) > 1; [00:40] Right, exactly. [00:40] Incredibly simple [00:41] wgrant: So I'm not sure what we should do, though [00:41] StevenK: Delete 35 MP diffs :) [00:42] wgrant: How will that help? There are two BMPs with the same merge_diff [00:42] Um, yeah, maybe I'm misremembering [00:42] Do we have a bug about this? [00:42] I do not think so [00:42] But I haven't gone diffing [00:43] Er, nice slip. digging [00:44] Ah [00:44] So I guess we want to duplicate the previewdiffs [00:45] How do we do that in 2.5 seconds? :-) [00:47] There's only 35 records... [00:48] wgrant: Sure, but I'm not sure how to pull out only one BMP, duplicate its previewdiff and then set it back. [00:50] StevenK: We'd probably want to duplicate the LibraryFileAlias, Diff, and PreviewDiff, then set BranchMergeProposal.merge_diff [00:51] LFA is involved? [00:52] wgrant: I can't see how LFA is involved [00:52] StevenK: diff.diff_text is an LFA [00:52] Ah [00:52] No two previewdiffs reference one diff, and no two diffs reference one LFA [00:55] wgrant: I'm just not sure how to duplicate rows, making sure that FKs are sane and then setting BMP.merge_diff [00:56] Isn't bug 497772 released now? That was apparently in launchpad-buildd 112, and top-of-changelog is 114 [00:56] <_mup_> Bug #497772: exceptions.AttributeError: 'DebianBuildManager' object has no attribute '_subprocess'

< https://launchpad.net/bugs/497772 > [00:56] StevenK: I'd probably write a query to find a temp table of the newer BMP of each of the 35 pairs, then execute a PL/pgSQL function to duplicate the three rows and switch the FK [00:57] cjwatson: Ah, but what launchpad-buildd is deployed on the builders ... [00:57] 114 [00:57] So yeah, that can be closed [00:57] launchpad-buildd | 113~0.IS.08.04 | hardy-cat-buildd | source, all [00:57] launchpad-buildd | 113~0.IS.08.04 | lucid-cat-buildd | source, all [00:57] But either way ... [00:57] Buildd toolchain package versions: launchpad-buildd_114-0~53~0.IS.08.04 python-lpbuildd_114-0~53~0.IS.08.04 bzr_2.5.1-0ubuntu2. [00:57] Ah, good [00:57] Mysterious [00:58] launchpad-buildd | 114-0~53~0.IS.08.04 | hardy-cat-proposed | source, all [00:58] Ah [00:58] Where did you get a cat madison? [00:58] chinstrap:~cjwatson/madison-lite-cat/ [00:58] madison-lite --config=config there [00:58] Ahh, thanks [00:59] And I run ./make-local-mirror occasionally to refresh the local Packages/Sources [00:59] Actually in fact that's cronned houry [00:59] *hourly [01:00] It is a foul hack but it works [01:03] Is there any way to get the OOPSen in bug 809937? Neither lp-oops.c.c nor oops.c.c seems to have them any more [01:03] <_mup_> Bug #809937: Timeout accepting packages < https://launchpad.net/bugs/809937 > [01:04] I suspect that's a dup of bug 745799, but it would be nice to be able to check [01:04] <_mup_> Bug #745799: DistroSeries:+queue Timeout accepting packages (bug structural subscriptions) < https://launchpad.net/bugs/745799 > [01:04] It's probably a dupe [01:05] We'll know for sure when the +queue POST oopses never appear again :) [01:05] wgrant: So my plan based on your plan is a function that we call once per previewdiff, I'm clear on how to duplicate the data using INSERT INTO, but how do get back the id so I can keep FKs sane? [01:05] But it may be worth duping that [01:07] StevenK: INSERT INTO foo (bar, baz) VALUES (what, ever) RETURNING id INTO some_var; [01:07] RETURNING is normal PostgreSQL; it's how we get the IDs back in Storm [01:07] INTO is PL/pgSQL [01:20] wgrant: UPDATE branchmergeproposal SET ... WHERE merge_diff = pd_id LIMIT 1; doesn't make sense, right? === Ursinha is now known as Ursinha-afk [01:33] StevenK: Indeed not [01:33] StevenK: The function will presumably be called with a BMP ID [01:35] wgrant: I've written it for a previewdiff id [01:35] It even works [01:35] Let me pastebin it up so you can insult me. [01:35] :-P [01:35] wgrant: http://pastebin.ubuntu.com/1225731/ [01:36] StevenK: Right, easy solution is to use the MP ID instead [01:36] Then grab the merge_diff from that === Ursinha-afk is now known as Ursinha [01:37] wgrant: It works this way? :-) [01:37] Oh, I see you do the SELECT first, right [01:38] Is that quick enough on DF? [01:38] If not, just move the SELECT merge_diff bit to before the statement timeout [01:38] So I've tested one run, I'm just about to try it with -f and a ROLLBACK [01:38] Right [01:39] Hah, my end SELECT line is wrong [01:41] wgrant: It times out. :-( [01:41] Right, so swap lines 6 and 8 [01:42] wgrant: It times out at INSERT INTO libraryfilealias [01:42] It already printed SELECT 35 [01:43] Have you explained the INSERT INTO? [01:43] wgrant: Just about to. [01:45] wgrant: 3ms [01:45] Perhaps just a cold cache? Run again? [01:46] Oooh [01:47] It prints duplicate_preview_diff, 35 blank rows and then (35 rows), can make it uh, not do that? [01:47] Into a pager, which results having to hit q [01:49] wgrant: ^ [01:50] Not sure [01:50] But it's because the function is returning null [01:50] wgrant: Actually, I think the SELECT duplicate_preview_diff(merge_diff) ... is doing it [01:51] Exactly. [01:52] You asked it to return the result of that function [01:52] Which is NULL [01:52] for each row [01:52] Ah [01:52] so 35 times [01:52] But it's declared as RETURNS void ? [01:52] Which means it returns null [01:52] out for a short while [01:52] It just can't return anything itself; it always evaluates to null [01:53] wgrant: So what do I instead? [01:53] StevenK: Ignore it, probably [01:54] I'm sure ops can press q [01:55] lifeless: I'd like to chat about some SQL I'd like to run on prod when you're back. [02:01] * bigjools owes wallyworld_ alcohol for adding the commit msg to MP emails [02:01] or caffeine, pick your drug of choice [02:01] * wallyworld_ likes Southern Comfort [02:02] i assume it worked to your satisfaction then [02:02] Is that a euphemism for wgrant? [02:02] oooooh [02:02] haha [02:21] StevenK: https://oops.canonical.com/oops.py/?oopsid=OOPS-4845cff98595cfa456992a710aaff661 is a bit odd [02:21] I suspect it's a regression from your stuff [02:22] Blink [02:22] Oh, we switched to SRF [02:22] Ah [02:22] Yeah [02:23] And when given a name, I bet it uses BPBSet instead of BFJSet [02:23] And the former is probably SQLObject [02:23] Yeah [03:03] StevenK: hi [03:12] lifeless: O HAI. The SQL in question is https://pastebin.canonical.com/75216/ [03:13] sudo execute me some sql ? [03:13] lifeless: The background is that when wgrant and I were migrating staticdiff to previewdiff we didn't check for and discard duplicates, so now we have 35 previewdiffs that are referenced by more than one branchmergeproposal [03:14] StevenK: I see [03:14] StevenK: you have afunction in there; how have you validated it ? [03:14] lifeless: I ran it on DF [03:15] StevenK: so if it did the wrong thing, we have no idea ? :) [03:15] let me give it some eyeballing [03:17] ok, I **think** it will be ok [03:18] Right, it is just duplicating the data and then picking one of the BMPs as a victim [03:19] And we have 2 previewdiffs that are lucky and linked from 3 BMPs [03:19] So we'll have to run it twice [03:23] lifeless: If you'd prefer, I'm happy to perform some validation on DF after lunch? [03:24] that would give me some comfort yes [03:24] its just a little too large to visually confirm [03:25] we need a "pebkac" bug status [03:25] StevenK: like, check it really only updates one of them, not both sides [03:52] lifeless: it does, because the final update will fail because the id column is unique and it matches with the = operator. [03:57] lifeless: http://pastebin.ubuntu.com/1225881/ [04:04] StevenK: Want me to run it? [04:05] stub: I was happy enough to go through webops, but please do. [04:05] stub: You'll need to run it twice -- two lucky previewdiffs are linked to 3 BMPs [04:07] StevenK: Done. 35 rows fixed, then 2 in the second run [04:08] stub: Fantastic, thanks. [04:24] StevenK, stub: Thanks [04:24] The broken MPs are no longer broken. [04:24] It should makes the OOPS report a bit happier too [04:25] StevenK: Were you going to look at the Builder:+history regression, or should I fix it? [04:25] wgrant: Can you remind me of that Builder:+history regression and your thoughts on it? [04:26] StevenK: The OOPS I pointed out this morning. If you look at Builder.getBuildRecords, you'll see it uses a different method if a name filter is specified. [04:26] BinaryPackageBuildSet.getBuildsForBuilder returns an SQLObjectResultSet, which SRF chokes on [04:27] So we just need to Stormify BinaryPackageBuildSet.getBuildsForBuilder [04:27] Probably about 5 minutes of work :) [04:27] Plus DEATH TO SQLOBJCT [04:27] +E [04:28] wgrant: No point for a test, just make it Stormier? [04:28] I think so. [04:28] wgrant: Did you see buildbot is still full of hate and not love? [04:28] Although a test could be done in about 4 lines. [04:28] StevenK: A second spurious rabbitmq failure? Yeah [04:28] A bit odd [04:28] But we'll see if it happens again [04:33] wgrant: It's a bit more than 5 minutes. [04:33] BinaryPackageBuildSet.handleOptionalParamsForBuildQueries() is disgusting [04:34] Oh, I missed that call [04:35] But it's not that bad [04:35] The only issue is that you probably have to port the rest as well [04:35] Yes [04:35] That's only two other methods, luckily [04:40] Didn't we write a Storm archive privacy method? [04:40] * StevenK tries to remember [04:43] wgrant: a little more context when downgrading importance would be lovely [04:44] wgrant: it would avoid questions like 'why isn't it critical', which I might otherwise ask [04:45] lifeless: The only questionable things I've downgraded are the crashes in ec2 demo [04:45] AFAICR [04:45] yah [04:45] just a -little- context, is all I'm asking for [04:46] I decided that nobody cares about ec2 demo apart from jml, and it's never been reliable, so it's probably not worth a critical [04:47] Or two [04:47] sure [04:47] but all folk like me see [04:47] is 'critical->high' [04:47] with no more info at all [04:47] True [04:47] not even the fact its ec2 demo vs ec2 land [04:48] so just saying 'ec2 demo is a best effort subproject - not treating as critical' would be more than enough [04:48] [Bug 993300] Re: Error in apache config during ec2 demo [04:48] <_mup_> Bug #993300: Error in apache config during ec2 demo < https://launchpad.net/bugs/993300 > [04:48] It says "demo" [04:48] but k [05:34] wgrant: Holy crap, I think I win [05:38] StevenK: What have you broken? [05:38] wgrant: 3 files changed, 74 insertions(+), 91 deletions(-) [05:38] It almost works [05:39] :( +74 [05:39] -91 [05:40] Sure [05:40] But normally if I frown at him he gets it even more negative [05:41] It only works sometimes. [05:41] * StevenK tries to work out where builder fits in === jtv1 is now known as jtv [06:10] wgrant: https://code.launchpad.net/~stevenk/launchpad/stormier-getbuildsforbuilder/+merge/126159 [06:12] StevenK: Looking === danilo_ is now known as danilos === almaisan-away is now known as al-maisan [06:46] wgrant: So, I was happy to move that into get_archive_privacy_filter, but BFJ's usage of it wants Or(PackageBuild.id == None, ) [06:47] StevenK: Does it want it? [06:47] or is it a bug? [06:48] I don't think it is [06:49] Why does BFJ's usage of it want that? [06:49] Oh, I misread [06:49] Surely BFJ also wants the admin/anon special cases? [06:49] It has it! [06:51] StevenK: And how does it differ? [06:51] clauses.append(Or(PackageBuild.id == None, Not(Archive._private))) [06:51] That's the user is None case for BFJ [06:51] clauses.append(Not(Archive._private)) [06:52] And the one I added in the branch [06:52] OK, I'm confused [06:52] "I was happy to move that into get_archive_privacy_filter" [06:52] What is "that"? [06:52] I could hack around it by doing if clause: return Or(clause, filter) else return filter [07:03] StevenK: BuildFarmJobSet.getBuildsForBuilder does the same thing, except in the admin case [07:03] And get_archive_privacy_filter should return True for an admin [07:04] So the admin case will be Or(PackageBuild.id == None, True) [07:04] Which should perform fine [07:09] wgrant: http://pastebin.ubuntu.com/1226047/ [07:09] It doesn't fix the LIKE bit yet [07:09] Or my indentation sins [07:11] wgrant: origin = [PackageBuild] is because it's always used whereas the old code pulled it in later [07:11] k [07:12] Hmm, why is where wrapped in brackets with no comma or line wrap [07:12] StevenK: I'm not sure it's worth having that clause arg [07:13] wgrant: You'd rather toss Or(PackageBuild.id == None, ...) into BinaryPackageBuild's callsite too? [07:13] Even though it wasn't there before [07:13] StevenK: SOmetimes people (eg. me, sometimes) wrap 'where=Foo.bar == baz' in parens [07:13] To make it less ambiguous [07:13] Fair enough [07:13] 'cause it really looks like the = should bind tighter there [07:13] But it doesn't [07:13] Trying to work how you'd prefer the filter = [ block to be indented [07:14] Which? [07:14] In get_archive_privacy_filter? [07:14] Yeah [07:14] The second element of the list is indented further than the first [07:15] Oh, duh [07:15] Which doesn't make much sense [07:15] Yeah [07:15] StevenK: So, back to PackageBuild.id == None: no, I would include PackageBuild.id == None in BFJS as we do today [07:15] wgrant: So, PackageBuild.id == None into BinaryPackageBuild's callsite? [07:16] So it becomes Or(PackageBuild.id == None, get_archive_privacy_filter(user)) [07:16] *get_archive_pri ... [07:16] :-) [07:16] Not if it returns an Or, no [07:16] It wasn't [07:16] It wasn't, no [07:16] But it should [07:16] Or() doesn't make sense for the BPB case [07:17] It should return Not(Archive._private), or True, or Or(Not(Archive._private), Archive.ownerID.blah) [07:17] I think [07:17] Why not? [07:17] Oh, hmm [07:18] wgrant: Wait, what about Or(PackageBuild.id == None, Or(...)) ? [07:18] What about that? [07:18] It's arguably slightly ugly, but meh? [07:18] Sorry, is having the or's nested a bad thing? [07:19] Only if you consider nested ORs to be a bad thing. [07:19] And I don't see why you would [07:21] wgrant: http://pastebin.ubuntu.com/1226063/ [07:26] StevenK: Looks reasonable. There's no other admin/anon specialcases in the callsites that you can eliminate? [07:27] Not that I can see. [07:27] :( [07:27] However, I'm going to actually stop work and do the LIKE stuff tomorrow [07:27] :) [07:27] So I'll have a dig to see if that function can be grafted into other sections of the code. [07:28] And it will be glorious, etc, etc [07:33] good morning [07:45] right now, what was I doing this morning? Thats right, I was doing a plugin for bzr [07:45] * lifeless writes some code [07:47] lifeless: The morning may have gotten away from you. [07:47] rather [08:10] wallyworld: I see you're working on the text for the commercial email, I have an action item to add some text to that could you point me in the right direction please? [08:11] czajkowski: sure. the change has landed already and is awaiting deployment. the file i edited is product-commercial-subscription-expired-open-source.txt in email templates [08:12] is this the one you want? [08:12] czajkowski: there's a few other templates in the same directory as well [08:12] wallyworld: not done this before, basically looking for the mail that is sent to a person after they purchase a subscription so I can add the launchpadstatus feed links on identi.ca/twitter and our blog url [08:13] for other commercial emails of various sorts [08:13] nods [08:13] czajkowski: ok, let me find the template [08:17] czajkowski: product-license-other-proprietary.txt is sent when a project is first created with a proprietary licence and gets an initial 30 day subscription, but just right now, i can't find what is sent when a subscription voucher is redeemed === allenap` is now known as allenap [08:18] czajkowski: if you overlap with sinzui, he will likely be able to tell you immediately where to look. if if you have a bit of text from an email, i can search for it [08:21] wallyworld: I dont but will do in a bit thanks for the help, also see pm for other bit === jam1 is now known as jam [09:11] wow [09:11] I can't get used to this fast buildbot ;) [09:12] heh [09:12] I'd say you've been working on launchpad for too long, BUT THAT'S NOT TRUE ;_; [09:13] moving back to a bzr-pqm style landing would be nice, rather than the current complications [09:16] mgz: Purple has been talking about it [09:16] I trust purple. [09:16] Means ec2 can just go and die too [09:16] J [09:16] E [09:16] N [09:16] J [09:16] I [09:16] N [09:16] S [09:16] *FFFAAARRRXXXXX* at the typo. [09:16] ehehehe [09:16] LOL [09:16] lifeless: Sounds good. Get me hardware and I'll break my back to set it up? [09:17] StevenK: we have the hardware now; prae as master and commander, huey and dewie as slaves [09:18] lifeless: I have a plan in the next few days to look at configuring and ec2 instance with LXC as an ssh slave and make sure Jenkins can drive it [09:21] yeah [09:21] We discussed on the call this morning about setting up Jenkins to do pre-merge testing using the existing slave setup [09:28] the server names are getting quite hilarious :P [09:28] nigelb: huey and dewie are not the real names [09:29] And prae is praseodymium [09:29] Oh boy [09:29] I hope the names don't match up to some sort of internal DNS. That's a hard to type name ;) [09:29] we have codenames for codenames around here, [09:30] nigelb: they do. [09:30] lifeless: oh dear [09:31] Awww, leningradskaya no longer resolves [09:31] (yes, real server name) [09:31] nigelb: we just call it praë [09:31] StevenK: *snicker* [09:32] nigelb: So there's this thing, called tab complete? [09:32] Hah. [09:33] ssh running under zsh will tab complete from known_hosts and .ssh/config [09:33] there is a helper [09:33] true true. I use it all the time. [09:33] that knows about all hosts [09:33] for Canonical [09:33] Oooh. [09:33] (internal only) [09:34] I don't know if the engine code for it is open - haven't checked. I don't think its super deep though, you can probably invent the same for whereever you are quite easily. [09:34] bzr tree, metadata populated by sysadmins, pull and run make. [09:35] ah interesting. [09:35] I've not yet worked with organizations that's gotten into > 30 servers. [09:35] Ahh, well .... [09:35] :-) [09:36] YEah, small time :P [09:38] FWIW, in grumpy mode: I find the business of slang names for servers makes things very difficult to follow; there are times when it's taken me months to realise that what I thought were two different servers people were talking about was actually just one. Much as I object to the pervasive coining of new initialisms in Ubuntu-land which also confuse newcomers, so it's probably a lost cause ... [09:44] huey and dewie are sluagh and radande :) [09:44] And prasé is praseodymium === frankban changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: frankban | Firefighting: - | Critical bugs: ∞ === al-maisan is now known as almaisan-away [10:56] morning [11:07] 3 [11:08] 2 [11:08] 1 [11:08] * rick_h_ wonders what happens now [11:08] breakage [11:25] wgrant: lp:~lifeless/+junk/bzr-mkghosts [11:25]