[00:04] thumper: mumble? [00:08] sinzui: I think he's away today === salgado is now known as salgado-afk === Ursinha-afk is now known as Ursinha [01:22] wallyworld: I may be misunderstanding here, but picker.js' animate_validation_content and reset_form appear to be global. Is that intentional? [01:24] wgrant: no, they should be global but i though because they were not added to the namespace they wouldn't be exposed?? [01:24] should *not* be [01:26] wallyworld: I'm not really sure how that works. [01:26] As I said, I may be misunderstanding. [01:26] wgrant: i'm not sure either but i thought i was doing it correctly. that's why i've asked for a ui review cause whoever does that typically knows javascript better than me [01:27] * wallyworld sighs. doctests make me sad [01:29] wallyworld: I just tried it myself... it's very pretty. [01:29] Although it has the issue that plagues most LP overlays: it resizes a lot. [01:30] wgrant: it was a bit of a learning curve for me but i enjoyed it. the resizing i didn't think was too obtrusive. i thought it better than leaving the overlay way too big for the content [01:32] wallyworld: Yeah, it's a general problem that we don't really have a solution for. [01:34] wgrant: the animation really helps i think. not sure why but it's easier on the eye when there's resizing involved rather than something just splatting onto the screen [01:34] Yeah. [01:34] Although it's possibly a bit long, it looks great. [01:35] wgrant: i experimented with the timing. it';s easy to change. i didn't think it was too bad myself but i'll happily be guided bu someone else [01:36] if it's too short you don't see the effect [01:48] wallyworld: Re-reviewed. [01:49] wgrant: thanks. looking now [01:52] ok, so bugtag needs to be redesigned [01:55] wgrant: the use of inline styles for the buttons div was copied from an existing usage [01:55] wallyworld: Ugh. OK. [01:55] sinzui: still around ? [01:55] I am [01:55] sinzui: oh hai [01:56] sinzui: there is a refactoring we need to do to fix performance in soyuz/recipes/translation template builds [01:56] sinzui: I don't know if wgrant has mentioned it to you yet (or not) [01:56] wgrant: but we should propogate bad practice so i'll fix it [01:56] *shouldn't* [01:57] wallyworld: Thanks. [01:57] lifeless: I hadn't. [01:57] sinzui: the BFJ/PB tables need to be folded into their adjacent tables (which are all 1:1) to permit context specific scans rather than all-builds-ever-history-scans [01:57] lifeless: I can make those related bug the next ones to be started [01:57] wallyworld: Oh, one other thing... are there spinners on both ends of the picker, if there is latency before it updates? [01:58] sinzui: its going to be a new-schema, migrate-over-a-week-or-so, then start using transition job [01:59] sinzui: I'll get a bug filed with supporting analysis and point you at it. [02:00] wgrant: you mean when the confirmation info is loaded? no. i don't see an obvious place to put a spinner. even though there probably should be one [02:00] wallyworld: Yeah, that was my thought, too... it's not clear where it should be. [02:00] there no need for one when the user confirms the selection cause there's already one there as per a normal save [02:00] Right. [02:01] if it were a desktop app i'd change the cursor [02:03] Ouch. [02:03] 138 / 434 Distribution:EntryResource:searchTasks [02:04] wgrant: btw we repacked the fti index last night [02:04] I saw that. [02:04] And the resulting timeouts :) [02:05] (looks like the lock was held for a little too long) [02:20] sinzui: bug 758258 [02:20] <_mup_> Bug #758258: buildfarmjob schema is inefficient for reporting < https://launchpad.net/bugs/758258 > [02:20] I assigned it to the green squad [02:27] thank you! [02:27] its almost small enough that I tackled it mysself [02:27] but not quite [02:52] lifeless: Could you please mentor https://code.launchpad.net/~wallyworld/launchpad/disabled-project-bugs-shown/+merge/57265? [03:10] wgrant: you missed a bit :) [03:11] Mm, I guess. [03:11] Well. [03:12] Something has completely destroyed my monospace fonts. [03:12] I think I may reboot and see if it is any better. [03:12] brb [03:14] lifeless: the reason for just adding to the doc tests is that all the other testing for getBugTaskAndNominationViews is done there. i new unit tests were written, then the doc test would have to be ported over to keep it all together. i can do that, but it seemed like much extra work for marginal benefit [03:14] wallyworld: I don't buy the keep it together argument [03:15] wallyworld: given the following options: [03:15] - a longer doctest [03:15] - unaltered doctest and some unit tests [03:15] - all unit tests [03:15] clearly the third option is best [03:15] but I argue that the second option is better than the first [03:16] lifeless: i agree that #1 or #3 is preferable but #2 is bad. all tests for a given piece of code should be together so they are easily discoverable etc imho [03:17] you don't want to fragment the tests for a given module/class all over the code base [03:18] depends on what you value [03:18] right now, you can't safely change that function and only run that doctest [03:18] I think the tech debt of increasing the doctest outweighs the tech debt of having the tests in two different places [03:19] but thats up to you, I can only share why I would do it differently. [03:19] lifeless: np. it's a matter of opinion in this case. i'll add a new unit test then [03:20] wallyworld: the other issues are significantly more important :) [03:20] lifeless: with the filter - there was already filtering being done in the list comprehension. i just added an extra condition. [03:21] wallyworld: I think you are reading too shallowly [03:21] wallyworld: Thats a partition, not a filter [03:21] or 'thats a partition special case of multiple filters' [03:22] my thinking was that the bugtasks were already eagerly loaded prior to the method being called. we were already extracting the ones we wanted from that list using a comprehension. why not add another condition to the filter? [03:23] wallyworld: because: [03:23] - things that iterate on the bugtasks will show too many [03:23] - we'll be eager loading too much data [03:24] ah ok. so you want the initial loading of the self.bugtasks to be filtered [03:25] although these com from the context [03:25] right, the context has the wrong bugtasks [03:25] searchTasks knows how to do all the filtering you're doing [03:26] (though, why are you filtering on experimental distroseries?!) [03:26] ok. makes more sense now. i was trying to make the smallest change to fix the reported issue [03:26] wallyworld: I am totally in favour of doing the simplest thing that meets all the criteria [03:27] but in this case it made sense to look a little further afield [03:27] I don't think that this code need sto change at all though, and the filtering rules need to be harmonised - we don't want to duplicate stuff [03:28] note this: [03:28] self.bugtasks = list(self.context.bugtasks) [03:28] yes, agreed. sometimes i'm more wary than needed for fear of breaking things like code i'm not 100% fmilar with [03:29] lifeless: so self.context.bugtasks a resultset then? [03:30] wallyworld: *blink* [03:30] is a [03:30] wallyworld: no, nor should it be [03:32] ok. a collection. which is not loaded until the list() is constructed. but we want to so a search instead [03:32] yup [03:32] self.context is a Bug [03:33] so do a bugtask search with bug=self.context.id [03:33] listify the result [03:33] right. i knew self.context was a bug but didn;t realise it's bugtasks would not already be loaded [03:33] wallyworld: one of them will be [03:33] wallyworld: (the one for the url we're on) [03:33] wallyworld: more /may be/ - but the key thing is the eager loading [03:33] wallyworld: see all the other work the constructor does [03:34] wallyworld: we dont' want to do that on unwanted bugtasks. [03:34] sure. but i thought they would all already be loaded, hence not seeing the issue with the list comprehension. makes more sense now. [03:35] thanks for the input [03:35] wallyworld: no worries [03:35] wallyworld: the trick here when looking at other cases [03:35] wallyworld: is to look at what *else* is done to the same data [03:44] wallyworld: sorry that I wasn't clearer in my review; I guess I had some hidden assumptions aout what you'd looked at [03:45] lifeless: np. i had a wrong assumption about what had already been loaded and that lead me down an incorrect path [04:01] wgrant: I would analyse the current searchTasks timeouts but [04:01] bug 740750 [04:01] <_mup_> Bug #740750: API timeouts broken and returning no useful data... < https://launchpad.net/bugs/740750 > [04:40] lifeless: turns out self.bugtasks = list(self.context.bugtasks) doesn't do any new sql cause all bugtasks are already loaded earlier. so doing a search actually *increases* the sql count. i'm tracking down where the earlier loading of all bugtasks is done [04:41] wallyworld: traversal [04:42] lifeless: yeah, just got a hit on my conditional breakpoint. gotta figure out what to do about it [04:47] wallyworld: I'd ignore it; the traversal load almost certainly doesn't do the right query [04:47] wallyworld: and its not a significant cost [04:48] lifeless: in the traversal code BugTargetTraversalMixin._get_task_for_context() it just loops through bug.bugtasks, hence loading them all [04:48] [compared to loading milestones etc for disabled rows] [04:48] wallyworld: yup [04:50] lifeless: ok, will do the search. bugtaskset.search() only filters on active projects. not distroseries or productseries. i guess that doesn't matter and it's really just the active projects we care about filtering? [04:50] wallyworld: right, I wasn't aware distros could be inactive [04:51] lifeless: (product|distro)series have a read only property "active" [04:51] set to false when the status is experimental etc [04:51] wallyworld: ah, this is different to the pillar 'active' field I think [04:51] wallyworld: the pillar active is set to false when we disable and hide a product/project [04:51] ok. that's me not fully understanding the domain model then [04:52] or me, IMBW [04:52] second opinon time [04:52] wgrant: ^ second opinion plox [04:57] wallyworld: anyhow, if we should filter more, then searchtasks should be filtering more. [04:57] wallyworld: so useing searchtasks is right either way [04:57] jtv: hi [04:57] https://devpad.canonical.com/~lpqateam/qa_reports/deployment-db-stable.html [04:57] hi lifeless [04:58] has a few patches from you needing qa [04:58] lifeless: agreed. i'll just leave it as is for now (filtering only on active projects) since that appears to be the issue at hand [04:59] lifeless: I shall see to it fortwith—think all of it is probably safe-to-deploy regardless of actual Q/A. [04:59] forthwith. [04:59] jtv: cool [04:59] Thank you spell checquer. [04:59] jtv: we have a couple of weaks (sic) :) [04:59] (sick) [05:00] Wee half er Koppel [05:01] lifeless: Hi, sorry, having fglrx issues. [05:01] lifeless, wallyworld: IPillar['active'] is what we want here. [05:01] Product has a DB field, Distribution is always True. [05:01] But it's what we want. [05:01] lifeless: the first one is the only one where deployment has any repercussions at all. [05:02] wgrant: bugtaskset.searchtasks is required to honour this anyway [05:02] wgrant: right? [05:02] lifeless: I believe it is fixed, yes. [05:02] right [05:02] so , searchtasks ftw [05:02] registry will be able to see the disabled tasks (which is appropriate) [05:03] Yeah. [05:03] wgrant: the branch that gets rid of all the cache-flushing and gc'ing in the publisher… you tried it out on dogfood before it landed; did you see enough detail to know that it's safe to deploy? [05:03] * jtv corrects himself [05:03] "that it looks reasonably safe to deploy" [05:03] jtv: Let me just look through the diff again, but it all seemed to work OK. [05:04] I'd like to try fully publishing lucid to get a memory worst-case, but we don't have enough disk. [05:04] So natty will have to do, and it manages that fine. [05:05] Then I think the only reasonable concern is memory usage, which is _probably_ solved and even if it's not, we'd ultimately be better off addressing afresh. [05:06] So… let's say that makes it qa-ok. [05:06] Right. I'm going to get it to publish natty again and watch the peak usage, but I would qa-ok it. [05:06] Thanks! [05:07] The effects won't be as dramatic as they were in the test, since we tested with the replacement publish-ftpmaster script. [05:07] That means that publish-distro will have its memory to itself, and any flushing it fails to do won't affect anything else in the procedure. [05:09] lifeless: you have a clear runway [05:11] \o/ [05:12] we really should finish up the lmirror stuff [05:12] I haven't heard much about that in a while. [05:14] I haven't had time to optimise the parser, which was the only scaling problem I recall in it [05:22] wallyworld: ping [05:22] wallyworld: bug 756878 [05:22] <_mup_> Bug #756878: Old bzr-builder < https://launchpad.net/bugs/756878 > [05:22] wallyworld: why did you close it wont fix ? [05:23] lifeless: there's nothing to fix? nest-part is already supported since 0.4 and the current release is 0.6 [05:23] The bug suggests we've reverted Launchpad's. [05:24] wallyworld: the version in natty has /nothing/ to do with what we're running on the web servers. [05:24] wallyworld: nor with the version we run on the build slaves. [05:24] Or on the buildds themselves, where it matters. [05:24] lifeless: Snap [05:24] wallyworld: both of which matter, because the one on the web server is where our validation code runs [05:24] wallyworld: and the buildds is where the actual builds happen [05:25] lifeless: where do we get the package for the servers you just mentioned? [05:25] maybe i misunderstood the bug, sorry :-( [05:25] The buildds use hardy-cat, the webapp should use sourcecode. [05:25] wallyworld: there are three dependency systems we have [05:25] wallyworld: launchpad-dependencies package [05:26] wallyworld: utilities/sourcedeps.conf [05:26] wallyworld: and versions.cfg [05:26] wgrant: hardy-cat? They're still hardy? [05:26] StevenK: Yes, until dapper dies. [05:26] Ah [05:26] Which is soon, I think [05:26] Yes! [05:27] wallyworld: so in this case, if you check utilities/sourcedeps.conf and look at the branch, you can see there is unused work on the branch [05:27] not to mention it may be arbitrarily out of date with upstream [05:27] looking now [05:29] wallyworld: in general nothing in lp will depend on current distro versions - not because we don't use them - we do, but because we're version locked, in one of those dependency tools, and we have custom archives for things that aren't present on the LTS release of the distro we are running [05:30] lifeless: ok. sorry. sourcedeps.conf does refer to a really ancient rev [05:30] should we be updating that? [05:31] wallyworld: its only one rev back on the branch we run from [05:31] wallyworld: but yes. [05:31] bzr-builder is alittle more complex because you need to upgrade the web ui & the buildds separately [05:31] lifeless: rev 68 was around jan 2010. tip is 119 [05:31] wallyworld: different branches [05:31] wallyworld: you can't compare the revnos [05:32] lifeless: i thought i was looking just at trunk [05:32] wallyworld: sourcedeps.conf lists a branch and a revno [05:33] wallyworld: that branch is the lp branch, not the upstream branch [05:33] revno: 68 [merge] [05:33] committer: Launchpad Patch Queue Manager [05:33] branch nick: trunk [05:33] timestamp: Tue 2010-11-23 18:32:13 +0000 [05:33] we're running a version merged in late november 2010 [05:33] rev 69 [05:33] timestamp: Tue 2010-12-07 21:24:13 +0000 [05:33] message: [05:33] [r=me] Update to lp:bzr-builder tip [05:33] is available [05:33] Isn't there some test harness thing that takes care of running scripts? I thought it was 'runscript' but I can't find it. [05:33] and presumably there is more work in upstream trunk too [05:33] ok. so i'll re-open the bug [05:34] wallyworld: checking the work missing against trunk just requires branching our branch and running bzr missing lp:bzr-builder [05:36] lifeless: ok. should the bug be marked as "high"? or "critical" since it affects a major lp feature? [05:36] I don't know if we ever actually supported nest-part [05:37] I suspect someone got mostly through upgrading but didn't quite finish it (that rev 69 in our branch) [05:37] it may be as simple as s/68/69/ in sourcedeps.conf and send to pqm [05:38] well we can hope [05:39] i'll take a look later [05:42] Can someone explain to me the rules for what types of links/info we display in the sidebar? [05:42] or point me to a doc [05:55] lifeless: the gui does say "Assign", "Choose Again". but the code is generic. the labels to put on the buttons are specified by the caller [05:56] ah, kk. [06:02] Can haz review? [06:05] no [06:05] can't has [06:07] wgrant: want to, and I"ll mentor? [06:07] StevenK: Where is it? [06:07] wgrant: https://code.launchpad.net/~stevenk/launchpad/ids-bugfixes/+merge/57270 [06:07] https://mail.google.com/mail/u/1/?ui=2&shva=1#inbox/12f481611058e601 [06:07] :P [06:08] I was so tempted to answer "On Launchpad. Duh." [06:08] StevenK: IDSJ takes forever... the job runner won't block on that? [06:09] wgrant: IDS or IDSJ? [06:09] Both. [06:09] wgrant: Oh, I should have added to the MP that I've QA'd this on dogfood before submitting. [06:10] wgrant: But run_jobs -v initialisedistroseries is happy enough on DF [06:12] StevenK: Oh, you give run_jobs a job type to run, it doesn't just run all of them? [06:14] wgrant: Exactly -- the run_jobs cronscript looks pretty much identical to the one I removed, it just pulls the source interface from the schema. === Ursinha is now known as Ursinha-afk [06:29] wgrant: The flush did seem to be required, when running the job runner it would OOPS saying the DAS could not be found. [06:32] wgrant: You're right, it seems it just wants ChildSeries.parent_series to get at the parent. I'd *love* to rewrite that horrible query, but it scares me. [06:33] wgrant: That error came from i-f-p. I got Julian'd into keeping the wording when I originally did the work. === Ursinha-afk is now known as Ursinha [07:21] wgrant: I've reverted the Store.find() change, and fixed the query. You feel nervous about the pending builds change? [07:21] StevenK: It has potentially disastrous consequences. [07:22] Copying an incomplete build set is terrible terrible thing to do in the current model. [07:22] In the worst case you can never copy that source again! [07:23] wgrant: You know IDS just uses the package cloner, right? [07:23] Yes. And? [07:24] But it doesn't touch BPB at all [07:24] createMissingBuilds does... [07:26] So, we copy a SPPH for a source, and a BPPH for the completed i386 build, since the amd64 build is NEEDSBUILD. The package cloner calls createMissingBuilds() which creates the missing amd64 build. And this is wrong? [07:26] StevenK: You now have two amd64 NEEDSBUILD builds in the same archive. [07:27] One can upload, the other will fail. [07:27] And you can never reconcile that. [07:27] Even if they are targeted to different distroseries? [07:27] so [07:27] big picture, IIUC [07:27] Same binary name in same archive = oh god what have we done [07:27] is that we need to do this $anytime [07:28] so waiting for quiesced == problem [07:28] lifeless: Sure. [07:28] But we cannot resolve this until we make binary copying not stupid. [07:28] wgrant: Right, so it's okay, IFF it's a different archive? [07:28] is this the same root issue as uploading to both poopies ? [07:28] StevenK: It's not okay, it's just not definitely going to fuck shit up. [07:28] lifeless: Not really. [07:29] lifeless: Not at all, actually. [07:29] wgrant: I asked because bigjools said that the upload issue and in-ui copying were the same problem [07:29] or at least, thats what I thought he said [07:29] lifeless: Yes, but that is a different in-UI copying issue. [07:29] They both have the same file conflict race. [07:30] But the race there is that the conflict will not be detected. [07:30] Here it will be detected, but you are stuck with it forever. [07:31] why would it create a missing build when there is a pending build? [07:32] Because .createMissingBuilds() is dumb. [07:32] ok [07:32] No, because our binary copying model is dumb. [07:32] And impossible to handle sanely. [07:32] so given my (admittedly shallow) understanding, we have more work todo before we can do this safely. [07:32] however [07:32] It's going to be amusing when we have armel PPAs. [07:33] given our isolation levels we're probably not entirely safe even when there are no pending builds [07:33] You will no longer be able to copy between all series. [07:33] [we only require serializable] [07:33] Upload to one series, builds (i386, amd64, armel). Copy to hardy, creates new lpia build. Can no longer copy either, because they don't have a superset of the archive's binaries. [07:34] argh bug 434733 comments [07:34] <_mup_> Bug #434733: marking public bug as duplicate of private bug leads to confusing UI <404>

< https://launchpad.net/bugs/434733 > [07:36] wgrant: Okay, so this obviously requires more thought. If I revert the NEEDSBUILD changes, are you broadly happy with the MP? [07:37] StevenK: I'll look over it again, but that is indeed the only deal- and distro-breaker. [07:46] does https://bugs.launchpad.net/launchpadlib/+bug/758460 ring any bells? [07:46] <_mup_> Bug #758460: ConfigParser.NoOptionError: No option 'consumer_key' in section: '1' < https://launchpad.net/bugs/758460 > === poolie_ is now known as poolie [07:48] wgrant: Changes pushed, diff updated. [07:58] StevenK: You've fixed bug #744849, right? [07:58] <_mup_> Bug #744849: populate-archive.py unusably slow < https://launchpad.net/bugs/744849 > [07:59] ah it's just the same as bug 745801 [07:59] <_mup_> Bug #745801: system-based authorization doesn't store useful credentials in gnome-keyring

< https://launchpad.net/bugs/745801 > [08:00] lifeless: Can bug #637761 be closed? [08:00] <_mup_> Bug #637761: single-thread appserver experiment (rt41361) < https://launchpad.net/bugs/637761 > [08:00] lifeless: Given that it was the greatest success we have had in years? [08:00] in a sense [08:00] though we're not migrated yet [08:00] I don't mind either way [08:01] We've run the experiment. [08:01] => experiment bug is old. [08:04] wgrant: like I say, I don't mind either way [08:04] wgrant: The branch that fixes bug #744849 landed this morning. [08:04] <_mup_> Bug #744849: populate-archive.py unusably slow < https://launchpad.net/bugs/744849 > [08:05] man, conflict wise, its going to be a long month [08:05] Argh structual-subscription.js AGAIN? [08:06] wgrant: In fact, the fix hit stable about 30 minutes ago. [08:07] lifeless: Yeah, it's tempting to restrict features to a single branch :( [08:07] lifeless: Or just roll out more frequently. [08:07] s/roll out/DB-downtime/ [08:08] wgrant: The qa-tagger would have done that in about 30 minutes? :-) [08:08] StevenK: Is it linked? [08:09] [r=lifeless, wgrant][bug=744849][no-qa] ... [08:09] wgrant: well, working on more often [08:10] StevenK: Ah. [08:10] lifeless: :( [08:11] wgrant: bigger branches have worse penalties [08:11] lifeless: One branch meaning either db-devel or devel. [08:11] Not a single feature branch. [08:12] Yes, a single feature branch would be crackful. [08:12] StevenK: pre-2008-Soyuz crackful, yes. [08:12] wgrant: You don't think the commit is needed? [08:12] StevenK: It wasn't before, was it? [08:12] What has changed? [08:12] I wanted to be explicit, I guess [08:12] Oh, did the thing at the end commit itself? [08:13] Whereas setParent does not? [08:13] I guess. [08:13] That's a reasonable justification :) [08:13] So you just withdrew your objection to the commit? [08:13] I asked for a reason. [08:14] You gave one! [08:14] Well, that's sort of an objection [08:14] The flush is due to a number of OOPS I saw on dogfood [08:14] wgrant: oh; hmmm [08:15] lifeless: Otherwise it's going to be a very long month on this side of the world :) [08:15] wgrant: According to Julian, large parts of Soyuz use parent_series as a "Yes, this series is initialised." [08:15] StevenK: Those are bugs. [08:15] Not according to Julian. [08:15] I've had this argument with him, I lost. [08:16] He is going to lose to me. [08:16] :) [08:16] if /any/ part of the system does not honour that, using it as such a flag would be a bug. [08:16] StevenK: Warty exists, so Soyuz does not require it. [08:16] As there must be at least one series with no parent. [08:16] StevenK: Warty even had lots of security uploads. [08:16] Some part of the system cannot honour it. [08:17] And thus using parent_series to pun with 'is initialised' is a bug. [08:17] Precisely. [08:17] It is both a stupid overloading and *impossible*. [08:17] wgrant: Julian also said that if Warty was touched Soyuz would blow up. [08:17] StevenK: add a field [08:17] StevenK: You couldn't +edit warty for a while, no. [08:17] StevenK: But you can do anything else to it. [08:17] lifeless: I like that idea even less. [08:17] StevenK: its entirely appropriate to be explicit about these things. [08:18] The other option is IDSJ.Job.status [08:18] DistroSeries.i_am_not_going_to_break_everything [08:18] StevenK: No. [08:18] StevenK: Hardy doesn't have an IDSJ. [08:18] StevenK: whats your objection [08:18]