StevenKSomebody should blow sampledata up00:06
StevenKwgrant: https://code.launchpad.net/~stevenk/launchpad/performant-getSourcesForDeletion/+merge/15370401:18
wgrantStevenK: Have you checked that my theory is correct?01:21
StevenKIt sounded plausible to me01:21
wgrantA query to check that scheduleddeletiondate is set iff (published or has published (or unremoved? can't remember) binaries) would give me more confidence01:22
wgrantI guess if you check scheduleddeletiondate on the binaries it might work01:22
wgrantThough it's not going to corrupt data if my assumption is incorrect, we should still try to verify it against existing data.01:25
StevenKwgrant: Query is http://pastebin.ubuntu.com/5624173/, results http://pastebin.ubuntu.com/5624172/01:36
wgrantStevenK: https://pastebin.canonical.com/86983/01:46
StevenKwgrant: http://pastebin.ubuntu.com/5624196/01:48
wgrantI know01:48
wgrantSo it looks like it might be right, but maybe not01:48
wgrantWe should probably investigate those cases to see what might be up01:48
wgrantThey may indicate publisher bugs01:48
StevenKWhich cases?01:48
wgrantThose 2001:49
wgrantIt's also possible that a publisher run will fix those01:49
wgrantBecause this invariant is only eventually consistent01:49
StevenKWell, let's see on prod01:49
StevenKSince qas is a bit odd01:49
wgrantYou also check that the binary is actually published01:50
wgrantRather than condemned01:50
StevenKThat was what the old method did01:50
wgrantAnd I only changed the archive in one part of the query01:50
StevenKRight, so the results are fairly useless01:51
wgrantThat also explains why it ran so quickly01:52
StevenKIf you're running it on sour, I'll kill mine01:52
wgrantI am01:52
StevenKMine has been killed01:53
StevenKwgrant: That query with 32 for both archives is 0 rows02:06
StevenKwgrant: What did that query return for PRIMARY?03:14
wgrantStevenK: 20000 rows03:15
StevenKBloody hell03:16
StevenKI don't think Archive:+delete-packages works for PRIMARY anyway?03:16
wgrantBut the primary archive and PPAs use the same domination code.03:16
wgrantAny problem in the primary archive can apply to another archive.03:17
wgrantSo it is a useful pathological test case.03:17
StevenKRight, so do you want to split up the work, or what is your plan?03:17
wgrantHmm, overrides, maybe03:20
* wgrant gives up03:20
StevenKOverrides will indeed fuck that up03:20
StevenKThinking about it03:20
StevenKNo overrides for PPAs03:20
StevenKwgrant: So I think it's plausible that overrides won't be doing that query any favours.03:32
wgrantRunning it over the first 10k PPAs gives about 500 results03:34
wgrantPossibly explainable by copies03:35
wgrantI guess it'll do03:35
StevenKwgrant: Wait, how do copies explain it?03:36
wgrantStevenK: Undeletions03:36
wgrantIf I delete something, let it be removed, then revive it03:36
StevenKOh, the necromancy style of copies03:37
wgrantThe old pubs may have been removed from disk, despite there now being live binaries03:37
StevenKwgrant: If it will do, can I get a review, then?03:51
wgrantStevenK: r=me with a comment03:55
StevenKwgrant: Your comment assumes I understand the reasons :-P03:56
wgrantStevenK: We only expose deletion of sources in the UI03:57
wgrantBut you may need to delete a superseded source's binaries03:57
wgrantSo we want to show a source if can itself be deleted, or if it has binaries that can be.03:57
wgrantDue to the GPL and sanity, we don't condemn sources until their binaries are gone.03:58
StevenKYeah, I just recalled GPL compliance03:58
StevenK+        # We will return sources that can be deleted, or deleted sources that03:59
StevenK+        # still have published binaries. We can use scheduleddeletiondate03:59
StevenK+        # rather than linking through BPB, BPR and BPPH since we don't condemn04:00
StevenK+        # sources until their binaries are all gone due to GPL compliance.04:00
StevenKwgrant: ^04:00
wgrantSounds reasonable.04:00
StevenKwgrant: How goes your TranslationMessage denorm?04:04
wgrantStevenK: Given that it's a fairly large table, I'm looking for other things I can do at the same time to avoid rewriting it more than once04:12
StevenKwgrant: It requires a full table rewrite?04:13
wgrantStevenK: Denorming a column does, yes.04:20
wgrantWe need to populate it.04:20
wgrantResolving the message sharing performance debacle is probably a much more difficult matter, but we'll see04:24
wgrantI'm reasonably convinced that the feature was primarily designed to make me cry years later.04:24
StevenKGiven TM is a 6GiB table, we should only rewrite it once, yes.04:25
wgrantA statement count of 328 doesn't seem too bad until you realise that the page only shows 10 items and times out before it's done half the queries.04:34
StevenKwgrant: I wonder if we want tmpreaper for ackee and loganberry to combat bug 881255 ?04:36
_mup_Bug #881255: bzr-limbo-?????? and gpg-?????? in /tmp is not being cleaned up and taking up a lot of space <canonical-losa-lp> <Launchpad itself:Triaged> < https://launchpad.net/bugs/881255 >04:36
wgrantStevenK: Combat? Conceal.04:37
StevenKConceal works too04:38
StevenKwgrant: Hmmmm05:15
wgrantThe failure?05:15
StevenKwgrant: Deleting publications does not set scheduleddeletiondate, so the sources turn up in the view again05:15
wgrantStevenK: Bah, yes.05:16
wgrantAlthough that's probably not necessarily a huge problem05:16
wgrantExcept for tests.05:16
StevenKIt will get set by the publisher?05:16
wgrantIt'll show the status correctly.05:16
wgrantIt's done during domination.05:17
StevenKIt will show the status correctly, but there is probably nothing stopping users from selecting a DELETED publication and trying to do so again and then getting a OOPS05:18
wgrantIt shouldn't oops05:18
StevenKHm, indeed05:19
wgrantRecall that we show them when they have published binaries exactly so that an already deleted source can be deleted again05:19
StevenKI thought I saw an assert for !DELETED05:20
StevenKBut I can't see one05:20
StevenKwgrant: I think the easiest thing for the tests is to set scheduleddeletiondate05:20
wgrantDepends on what the test wants to achieve05:21
wgrantBut probably, yes.05:21
StevenKarchive-views.txt is checking a double POST05:21
StevenKWe can probably just destroy the view.has_sources_for_display check05:22
=== almaisan-away is now known as al-maisan
=== al-maisan is now known as almaisan-away
=== wedgwood_away is now known as wedgwood
=== matsubara is now known as matsubara-lunch
=== deryck is now known as deryck[lunch]
=== Ursinha_ is now known as Ursinha
=== matsubara-lunch is now known as matsubara
=== deryck[lunch] is now known as deryck
=== Ursinha is now known as Ursinha-afk
=== BradCrittenden is now known as bac
=== Ursinha-afk is now known as Ursinha
=== Ursinha_ is now known as Ursinha
=== wedgwood is now known as wedgwood_away

Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!