| cjwatson | wgrant: bzr mv lazr.jobrunner-0.11.tar.gz dist/ | 00:08 |
|---|---|---|
| wgrant | cjwatson: blargh, oops | 00:09 |
| wgrant | cjwatson: Fixed, sorry | 00:09 |
| cjwatson | np | 00:10 |
| cjwatson | Huh. I set lp.distributions["ubuntu"].redirect_release_uploads to True, and yet https://launchpad.net/ubuntu/raring/queue?queue_state=1 shows anna having gone to the Release pocket :-( | 00:41 |
| * cjwatson wonders what he got wrong | 00:41 | |
| cjwatson | I don't suppose poppy uses a different tree? | 00:42 |
| cjwatson | Well, except this is process-upload, not poppy as such | 00:43 |
| cjwatson | Unless I was in slightly too much of a rush and beat the rollout to pepo ... | 00:44 |
| cjwatson | That was it. Phew. | 00:45 |
| wgrant | Yeah, appservers are done early on | 00:46 |
| wgrant | pepo not so much | 00:46 |
| spm | cjwatson: I can probably fill you in on a secret. we webops watch for activity from certain users - you're one of the lucky ones - and deliberately break things when you're on, just for our twisted amusement. | 00:47 |
| nigelb | lol | 00:47 |
| spm | Some of the devs have gotten in on this, and do similar, but release NDTs at those times. | 00:47 |
| nigelb | Heh, wgrant "I'm sorry for your loss" | 00:48 |
| spm | cjwatson: http://www.xfos.com.au/tears.jpg so yeah, drink up mon! | 00:48 |
| cjwatson | spm: I already assume computers are malevolent; operators being malevolent is just a logical extension | 00:49 |
| spm | c | 00:49 |
| nigelb | clearly spm's *s* expands to Simon, of BOFH. | 00:49 |
| spm | the p is derived from Rick. | 00:49 |
| spm | applied to, perhaps | 00:50 |
| wgrant | Hm | 02:04 |
| wgrant | Something increased the registry 99% render time from about 1.7s to 2.4s between the 19th and the 21st | 02:05 |
| wgrant | And it remains high | 02:05 |
| wgrant | That coincides well with a large number of private projects query changes | 02:06 |
| wallyworld_ | wgrant: if you have a spare moment today, could you take a look at my preliminary experimentation with setting the correct binary package components? the extra joins in the binary overrides query are needed to get the associated source package release to get the spr component, but performance may be an issue. is what i've done on the right track? | 02:29 |
| wallyworld_ | https://code.launchpad.net/~wallyworld/launchpad/package-defaults-192076/+merge/131509 | 02:29 |
| wgrant | wallyworld_: Hm, doing it in the override queries is an interesting approach | 02:32 |
| * wallyworld_ has no idea about soyuz | 02:33 | |
| wallyworld_ | i just tried to gather the info that was needed | 02:33 |
| wallyworld_ | maybe i can do another query later in the workflow | 02:34 |
| wallyworld_ | i'm also unsure how to do the distro validity check | 02:34 |
| wgrant | wallyworld_: So, the main place this is a problem is in NascentUpload.processUnknownFile | 02:38 |
| * wallyworld_ looks at that method | 02:38 | |
| wgrant | It's a bit sad that you've chosen possibly the worst bit of Soyuz to work on | 02:40 |
| wallyworld_ | i have no idea what's good and what's bad | 02:40 |
| wgrant | The override stuff is particularly hideous, as it follows a design which I intended to be followed up by a major refactoring cleanup pass | 02:40 |
| wallyworld_ | the calculateSourceOverrides() method seems to implement the rules in bug 120052, so i was doing a similar approach for binary packages | 02:42 |
| _mup_ | Bug #120052: Component mapping for new source packages <lp-soyuz> <Launchpad itself:Fix Released by julian-edwards> < https://launchpad.net/bugs/120052 > | 02:42 |
| wallyworld_ | or so it seems to me anyway | 02:43 |
| wgrant | wallyworld_: Right, but you'll see that NascentUpload uses UnknownOverridePolicy.getComponentOverride directly | 02:43 |
| wallyworld_ | does NascentUpload process binary packages? | 02:44 |
| wallyworld_ | since that's what the bug is referring to | 02:44 |
| wgrant | Yes. | 02:44 |
| wgrant | It processes both source and binary uploads | 02:44 |
| wallyworld_ | hmmm. ok. so binary packages are processed there and also in the code i've touched | 02:44 |
| wgrant | The code you've touched is mostly for copies | 02:45 |
| wallyworld_ | copyBinariesTo() | 02:45 |
| wgrant | Which are somewhat interesting, but post-date the bug by some years | 02:46 |
| wgrant | They're not the common case, even today | 02:46 |
| wallyworld_ | so, if NascentUpload is changed, the copy stuff should be changed too i guess | 02:46 |
| wallyworld_ | to be consistent | 02:46 |
| wgrant | Sort of | 02:46 |
| wgrant | The copy overrides stuff is pretty awful | 02:47 |
| wgrant | I don't quite remember how that side of things works with respect to the queue | 02:47 |
| wallyworld_ | so, i assume that NascentUpload has enough info to find the spr associated with a binary, then i can go from there | 02:47 |
| wgrant | Right, when uploading a binary we always have a target build, which has a related SPR | 02:48 |
| wallyworld_ | how do i check if a component is valid for a distroseries? | 02:49 |
| wgrant | That's recorded in the ComponentSelection table | 02:50 |
| wgrant | However | 02:50 |
| wgrant | however... | 02:50 |
| wgrant | Are you using SPR.component? | 02:50 |
| wgrant | I haven't read the MP in detail | 02:50 |
| wallyworld_ | yes | 02:50 |
| wgrant | That's incorrect, sadly. | 02:50 |
| wgrant | You need to find SPPH.component, for some relevant SPPH | 02:50 |
| wgrant | Which is probably BinaryPackageBuild.current_source_publication | 02:51 |
| wallyworld_ | my brain hurts | 02:51 |
| wgrant | SPR.component is set only once, when the SPR is created in the DB for the first time | 02:51 |
| wallyworld_ | ok, i haven't seen any data, i was just guessing based on attribute names | 02:51 |
| wgrant | For lots of Ubuntu packages, they're first imported from Debian | 02:51 |
| wgrant | So they appear first in main in the Debian primary archive, so SPR.component == main | 02:52 |
| wgrant | Then they're copied into Ubuntu, where the SPPH.component is overridden to universe | 02:52 |
| wgrant | The binaries should default not to main, but to universe | 02:52 |
| wgrant | Even though SPR.component still == main | 02:52 |
| wallyworld_ | ok. could i perhaps just add an override mapping from main->universe? | 02:53 |
| wgrant | You may now be getting the rationale behind my campaign to unify uploads and copies | 02:53 |
| wgrant | Nope | 02:53 |
| wgrant | It won't always be universe in Ubuntu | 02:53 |
| wgrant | It could be main | 02:53 |
| wgrant | or restricted | 02:53 |
| wgrant | or multiverse | 02:53 |
| wallyworld_ | sure, but i mean just for the default mapping | 02:54 |
| wgrant | So, the general rule is that if a new binary has no existing ancestral component, it should inherit the component of its build's current SPPH | 02:54 |
| wallyworld_ | ok, so i look in ComponentSelection with spph.component to see if it is valid for the distroseries | 02:56 |
| wallyworld_ | if valid, use that, if not valid, what do i do then? | 02:56 |
| wallyworld_ | just use universe? | 02:56 |
| wgrant | Well | 02:58 |
| wgrant | I'd just always use SPPH.component, probably | 02:58 |
| wgrant | The SPPH is in the relevant distroseries | 02:58 |
| wgrant | If it's not, crashing is OK | 02:58 |
| wallyworld_ | ok, and if there isn't one? | 02:58 |
| wallyworld_ | raise an exception | 02:58 |
| wgrant | That's a more interesting question | 02:59 |
| wgrant | We already have an issue today with builds where there is no current source publication | 02:59 |
| wgrant | Usually because the source was supereseded/deleted before the build finished | 02:59 |
| wgrant | eg. http://launchpadlibrarian.net/120668537/aYDS2Qq6K4jADFvNDYo0RyGBld1.txt | 03:00 |
| wgrant | And I think there's a bug for that | 03:00 |
| wallyworld_ | ok, that can be for some other branch to fix i guess | 03:00 |
| wallyworld_ | so above you said "sort of" when i asked about ensuring both places were consistent - copy and upload | 03:00 |
| wallyworld_ | if i change upload, then copy has to change too i would think | 03:00 |
| wgrant | So, I'd use SPPH.component, assume it's valid, use the current default mechanism if it's not set | 03:00 |
| wgrant | You need to preserve the current default mechanism for sources, anyway | 03:01 |
| wgrant | (processUnknownFile is used for both sources and binaries) | 03:01 |
| wallyworld_ | yes, i wasn't going to touch sources | 03:01 |
| wgrant | I have to hope that StevenK remembers how missing copy overrides work | 03:02 |
| wallyworld_ | i guess target_build = None for sources? | 03:02 |
| wgrant | It's relatively modern and I have repressed that memory | 03:02 |
| wgrant | Um | 03:02 |
| wgrant | That's one way | 03:02 |
| wgrant | You'll need to redesign the way processUnknownFile is called | 03:03 |
| wallyworld_ | missing copy overrides seems to use the archive component or defaults to "universe" | 03:03 |
| wgrant | So it can be given some context | 03:03 |
| wallyworld_ | ok, i haven't looked at the code fully yet | 03:03 |
| wallyworld_ | so, is using the archive component correct then for mising copies? | 03:03 |
| wgrant | Oh, as in archive.default_component? | 03:04 |
| wallyworld_ | yes | 03:04 |
| wgrant | If it's set, sure | 03:04 |
| wgrant | It overrides the traditional default logic | 03:04 |
| wallyworld_ | the code defaults to universe if it is not set | 03:04 |
| wgrant | But only for partner/PPA | 03:05 |
| wgrant | Well | 03:05 |
| wgrant | IIRC it defaults to universe unless [SB]PR.component is contrib or non-free, in which case it defaults to multiverse | 03:05 |
| wgrant | Right? | 03:05 |
| wallyworld_ | that's force sources i think | 03:05 |
| wallyworld_ | for | 03:05 |
| wallyworld_ | not binaries | 03:05 |
| wgrant | Today it's meant to go: ancestry.component, archive.default_component, [SB]PR.component -> universe/multiverse | 03:06 |
| wgrant | The bugfix will be: ancestry.component, (if binary) build.current_source_publication.component, archive.default_component, [SB]PR.component -> universe/multiverse | 03:07 |
| wgrant | I think | 03:07 |
| wallyworld_ | i think it looks in spph first, and then uses a default mapping for unknown | 03:08 |
| wgrant | Right, it will always look up the [SB]PPH ancestry first | 03:08 |
| wallyworld_ | default mapping is contrib/non-free -> multiverse | 03:08 |
| wallyworld_ | everything else -> universe | 03:09 |
| wgrant | Right, that's the last resort | 03:09 |
| wallyworld_ | above, build.current_source_publication.component = spph.component, right? | 03:09 |
| wgrant | current_source_publication is an SPPH, right | 03:10 |
| wallyworld_ | ok, i'll give it a go. i might delay doing the extra query when copying binaries so that only unknown ones are processed that way | 03:11 |
| wallyworld_ | rather than doing the extra joins up front | 03:11 |
| wgrant | Yeah, definitely. | 03:12 |
| wallyworld_ | i wonder how many copied binaries are unknown | 03:12 |
| wgrant | It happens most often with kernel SRUs and security updates | 03:12 |
| wgrant | But it's relatively very rare | 03:12 |
| wallyworld_ | ok. thanks for all the help | 03:12 |
| wgrant | The plan was to rewrite find_and_apply_overrides to basically use lp.soyuz.adapters.overrides for everything, but it never got done | 03:13 |
| wallyworld_ | i haven't come across find_and_apply_overrides yet | 03:13 |
| wallyworld_ | ah, it's in NascentUpload | 03:14 |
| wgrant | Right | 03:14 |
| wgrant | It's what calls processUnknownFile | 03:14 |
| wgrant | It's the original implementation of the override logic | 03:14 |
| wgrant | Copies didn't respect overrides until 18 months ago | 03:15 |
| wallyworld_ | so we have 2 implementations, do they do the same thing? | 03:15 |
| wallyworld_ | ie give the same result? | 03:15 |
| wgrant | Mostly except sort of not | 03:15 |
| wgrant | The idea was that lp.soyuz.adapters.overrides would be the One True Implementation | 03:15 |
| wgrant | But the old one was never destroyed, in true LP style :( | 03:15 |
| wallyworld_ | so maybe i should at least make upload and copy do the same thing then | 03:15 |
| wallyworld_ | and delete the old code | 03:16 |
| wgrant | That sounds slightly too adventurous | 03:16 |
| wgrant | For a first adventure into Soyuz | 03:16 |
| wgrant | Although it would be a good tour, I guess | 03:16 |
| wallyworld_ | isn't it just the find_and_apply_overrides method to be replaced? | 03:16 |
| wallyworld_ | with calls to the adaptor | 03:16 |
| wgrant | mostly | 03:16 |
| wgrant | It's unlikely to be as smooth as we hope | 03:17 |
| wgrant | As we don't have publications here | 03:17 |
| wallyworld_ | is there adequate test coverage? | 03:17 |
| wgrant | Not adequate enough that I'd deploy it without extensive manual QA, but probably enough for development. | 03:18 |
| wallyworld_ | ok, i'll have a play around and see what falls out. but if i do it, i'll have to beg for help with qa | 03:19 |
| wgrant | With the history of this stuff, I would be QAing it personally regardless of who wrote it and had already QAed it, don't worry | 03:19 |
| wgrant | The archiveuploader tests are some of the most archaic we have | 03:20 |
| * StevenK can recall that about his overrides policy code | 03:20 | |
| wallyworld_ | ok, thanks. it may be that i chicken out | 03:20 |
| wallyworld_ | but i'd like to get rid of inconsistent, "duplicate" code | 03:21 |
| wallyworld_ | where possible | 03:21 |
| wgrant | Definitely | 03:21 |
| wgrant | This is an example that really needs it, but it's not the easiest way to start off in Soyuz. | 03:22 |
| wgrant | So we'll see | 03:22 |
| wgrant | StevenK and I probably know both sides of this code reasonably well | 03:23 |
| wgrant | So can unstick you as required | 03:23 |
| wallyworld_ | ok, thanks | 03:23 |
| wallyworld_ | it's hard because i have no domain knowledge and don't know where to find data to help me understand the code :-) | 03:24 |
| wallyworld_ | but, we'll see | 03:24 |
| StevenK | wallyworld_: I think your first step is to print out http://behlerblog.files.wordpress.com/2012/06/frustrated-kit.jpg | 03:24 |
| wallyworld_ | hah, i did that when i first used zope | 03:25 |
| StevenK | Tell me about it, so did I. | 03:25 |
| StevenK | *And* I had to learn Soyuz at the same time | 03:26 |
| wallyworld_ | a good debugger is absolutely mandatory | 03:26 |
| wgrant | But you had Soyuz experience! | 03:26 |
| StevenK | wgrant: Sure, with using it! | 03:26 |
| nigelb | lol | 03:26 |
| StevenK | Tells me absolutely nothing about how the code hangs together. | 03:26 |
| nigelb | That reminds me of my first lp patch. | 03:26 |
| wallyworld_ | StevenK: so, same with women | 03:27 |
| nigelb | I was using grep to find things and I needed like 5 windows for grepping different things. | 03:27 |
| wallyworld_ | nigelb: you need an IDE with code navigation features | 03:27 |
| wallyworld_ | much easier | 03:27 |
| nigelb | I'm guessing you use pycharm? | 03:27 |
| ajmitch | nigelb: you obviously were changing a simple part of LP then | 03:27 |
| wallyworld_ | yup | 03:28 |
| nigelb | ajmitch: Heh, yes. | 03:28 |
| nigelb | I need to get back to LP dev. Been lazy. | 03:28 |
| wallyworld_ | s/lazy/smart | 03:28 |
| nigelb | ^ I was trying to be diplomatic | 03:29 |
| wallyworld_ | i was joking :-) | 03:29 |
| wallyworld_ | lp is nice once you get to know and love it | 03:29 |
| nigelb | s/love/not want to kill yourself/g | 03:29 |
| nigelb | *kill yourself because of | 03:29 |
| wallyworld_ | it's just like any lrge software system with lots of legacy | 03:30 |
| nigelb | That's true. | 03:30 |
| ajmitch | wallyworld_: stockholm syndrome is real | 03:30 |
| nigelb | ajmitch++ | 03:30 |
| wallyworld_ | lol | 03:30 |
| nigelb | hahah | 03:30 |
| wallyworld_ | sadly i have to pause the fun soyuz stuff to deal with some ec2 hate mail | 03:36 |
| nigelb | ajmitch: I'll admit, fixing LP bugs was among the most satisfying bugfixes I've done. | 03:37 |
| nigelb | When I did the mouseover for bug titles thing, a lot of people pinged me say they love it ;) | 03:38 |
| nigelb | (granted it took like 3 attempts at landing to actually land) | 03:38 |
| wallyworld_ | wgrant: have you seen ec2 failure with "ClosedError: Connection is closed" but the tests pass locally? | 03:41 |
| wgrant | wallyworld_: In what sort of test? | 03:42 |
| wallyworld_ | a couple of doc tests | 03:42 |
| wgrant | Hm | 03:42 |
| wallyworld_ | eg xx-bug-also-affects.txt | 03:42 |
| wgrant | If they're the only failures and not very relevant to your changes, just lp-land I guess | 03:42 |
| wgrant | Hm | 03:42 |
| wgrant | This is for the +distrotask change? | 03:43 |
| wallyworld_ | yes | 03:43 |
| wallyworld_ | could be relevant | 03:43 |
| wallyworld_ | but i would expect a local failure also | 03:43 |
| wgrant | Yeah | 03:43 |
| wgrant | Perhaps ec2 again to be safe | 03:43 |
| wallyworld_ | yeah :-( | 03:43 |
| wgrant | Won't be a significant delay, as we are unlikely to deploy again today | 03:43 |
| wallyworld_ | well, here's hoping for 2nd time lucky | 03:43 |
| nigelb | lifeless: Did you see that sentry has a js logger thing? | 05:28 |
| nigelb | Hehe, I was going to setup sentry for python logging when I noticed it :) | 05:28 |
| lifeless | nigelb: yah, did you see the branch for oops that adds that ? | 06:57 |
| lifeless | imbrandon was working on a revised version | 06:58 |
| nigelb | lifeless: hah. no. | 06:58 |
| nigelb | awesome. | 06:58 |
| lifeless | nigelb: https://code.launchpad.net/js-oopsd | 06:58 |
| nigelb | \o/ | 07:02 |
| nigelb | Nice. | 07:02 |
| deryck | StevenK, are you around? | 07:06 |
| adeuring | deryck: https://code.launchpad.net/~adeuring/launchpad/bug-1067736/+merge/131527 | 07:09 |
| deryck | adeuring, got it. | 07:10 |
| rick_h_ | deryck: https://bugs.launchpad.net/launchpad/+bug/1068817 | 07:21 |
| _mup_ | Bug #1068817: Person.assigned_specs is an attractive nuisance <private-projects> <specifications> <Launchpad itself:In Progress by rharding> < https://launchpad.net/bugs/1068817 > | 07:21 |
| StevenK | deryck: I am now, but not really as it were. | 07:36 |
| deryck | StevenK, right. I replied on abentley's MP. We just need to get that landed, and I assumed you were done with it. | 07:36 |
| StevenK | deryck: Link me, sorry? | 07:37 |
| deryck | StevenK, https://code.launchpad.net/~abentley/launchpad/person-assigned-specs-in-progress/+merge/130888 | 07:38 |
| deryck | StevenK, last comment from me. | 07:38 |
| StevenK | deryck: Well, obviously :-) | 07:38 |
| deryck | Just trying to be clear :) | 07:38 |
| StevenK | deryck: I am concerned about the NULL thing, since it may leak data or provide results that are not wanted, so then please do land that branch, but also please investigate to make sure that the NULL doesn't sneak in for the actual code. | 07:40 |
| deryck | StevenK, there is no NULL in the code. that was a manual substitution we did to test the query. | 07:41 |
| deryck | StevenK, the actuall code-generated query uses two values from the enum. | 07:41 |
| StevenK | deryck: Ah. What I do to test SQL like that is to run a single test with LP_DEBUG_SQL=1 to grab the actual SQL, change the id's for say person or what not and then run it | 07:43 |
| abentley | StevenK: We were trying to be clever, and failing. | 07:43 |
| deryck | StevenK, cool, thanks for the tip. | 07:43 |
| deryck | StevenK, so we're cool then? | 07:44 |
| StevenK | deryck: Yes. I'm happy to approve it and you guys can land it if you wish? | 07:45 |
| deryck | StevenK, thanks! | 07:45 |
| StevenK | deryck, abentley: Done. | 07:46 |
| === almaisan-away is now known as al-maisan | ||
| abentley | rick_h_: Could you please review this? https://code.launchpad.net/~abentley/launchpad/milestone-spec-privacy/+merge/131560 | 09:32 |
| === al-maisan is now known as almaisan-away | ||
| === yofel_ is now known as yofel | ||
| adeuring | deryck: https://code.launchpad.net/~adeuring/launchpad/filter-private-product-RosettaApplication/+merge/131613 | 13:17 |
| === almaisan-away is now known as al-maisan | ||
| === al-maisan is now known as almaisan-away | ||
| === matsubara is now known as matsubara-lunch | ||
| === matsubara-lunch is now known as matsubara | ||
| === Ursinha is now known as Ursinha-afk | ||
| === Ursinha-afk is now known as Ursinha | ||
Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!
