[00:22] I can has review? https://code.launchpad.net/~lifeless/lazr.batchnavigator/keyoffset/+merge/57091 [00:43] huwshimi: hi [00:43] lifeless: Morning [00:44] huwshimi: wondering if you could eyeball bug 119939 [00:44] <_mup_> Bug #119939: strike through resolved bugs < https://launchpad.net/bugs/119939 > [00:44] huwshimi: I closed it wontfix because its clearly easy to do if we wanted to, and multiple ui refreshes haven't don it [00:44] but I don't have a UI analysis for why its a bad idea, and the filer is (reasonably) asking why we have chosen not to do this [00:45] huwshimi: if I was wrong to close it and you think it would be a good idea, we can reopen easily. [00:45] lifeless: I might point out that multiple UI refreshes also haven't made the UI not suck. [00:46] wgrant: you might [00:46] wgrant: and thus me talking to huw :P [00:46] it just seemed like a bug that would never get traction - do, or do not. [00:47] mwhudson: perhaps I can rope you in for that review ^? [00:47] lifeless: So, I don't think we should do the strikethrough, but it would be super useful to have some kind of indication of the status of the linked bug (e.g. on tooltip or inline: bug #1234 [won't fix]). [00:47] <_mup_> Bug #1234: Gina is an unmaintainable mess of command line options, environment variables and shell scripts < https://launchpad.net/bugs/1234 > [00:48] huwshimi: do you mean in linkification, or other places? [00:49] fmt: and linkification have different algorithms [not necessarily for good reasons] [00:49] huwshimi: if you look here: [00:49] https://code.launchpad.net/~sinzui/launchpad/person-merge-job-4 [00:49] huwshimi: the linked bugs have a mouseover status - 'critical - in progress' [00:49] lifeless: I think this would be appropriate in bug comments/descriptions etc. [00:50] huwshimi: ok, thats in linkification. The good news is that I have a canned answer for you. The bad news is 'that turns out to be suprisingly expensive performance wise, back burner at best'. [00:51] lifeless: Yeah I suspected. [00:51] its expensive because of bug privacy [00:52] huwshimi: anyhow, if you could comment on that bug [which is specifically about strikethrough] or perhaps alter it to talk about freeform bug linkfication and exposing more metadata, that would be cool. [00:53] lifeless: I think it would be a good thing to do, but if the chances of it ever happening are slim to none then maybe we should kill the bug [00:53] lifeless: OK will do. [00:53] lifeless: maybe you should also make a note that we can not do it yet due to performance [00:53] huwshimi: all things are possible. I have no objection to a bug open saying 'linkified bugs would be more useful if they showed bug metadata such as title, status, milestone etc' [00:54] lifeless: is "linkified" the correct term for this situation? [00:55] huwshimi: hi. do you have any css you want to give me for the confirmation dialog? [00:55] lifeless: As in, will other people know what we're talking about [00:55] wallyworld: Not yet. But I do want to give you some :) [00:55] np. thanks [00:56] wallyworld: I should be able to send you some today [00:56] excellent smithers [00:57] huwshimi: it may not be the official term but its close enough [00:57] lifeless: Sure thanks :) [00:57] huwshimi: we have modelled links like 'duplicate' etc [00:57] huwshimi: and we have freeform stuff we infer by linkification [00:58] modelled stuff already shows the metadata (on mouse over) [01:00] wgrant: have you seen https://bugs.launchpad.net/launchpad/+bug/454307 ? [01:00] <_mup_> Bug #454307: indices/md5sums.gz doesn't match repository < https://launchpad.net/bugs/454307 > [01:00] thumper: Can you QA your branch-by-ID thing? [01:00] wgrant: yeah [01:00] I think [01:00] lifeless: Never seen that before, because it was mistriaged. [01:00] lifeless: what do we have in place for codehosting on qastaging? [01:00] Fixed. [01:00] wgrant: I suspected as much - chalk another one up for silos [01:01] or easier to just use staging? [01:01] thumper: Everything. [01:01] thumper: codebrowse you need to access by port forwarding, but everything else works. [01:01] wgrant: is it a genuine new bug and not a duplicate? [01:01] lifeless: I don't recall anything like it. [01:02] wgrant: win [01:02] ShipIt should be running for the last time right now :) [01:02] Then we can disable and delete it. [01:06] lifeless: Do you think https://qastaging.launchpad.net/ubuntu is OK? The "Latest derivatives" portlet is new, not flagged, probably undesirable, and mostly bad data. [01:07] Bug #747502 [01:07] <_mup_> Bug #747502: Derived distributions must be discoverable from the "parent" distribution

< https://launchpad.net/bugs/747502 > [01:08] wgrant: I would hesitate to ship that [01:08] Shall I flag it? [01:08] wgrant: please [01:09] wgrant: also a gentle reminder to the list 'if changing existing uis with unfinished stuff, feature flag it' [01:09] Sure. [01:28] mwhudson: ping [01:28] lifeless: hi? [01:29] mwhudson: hi; did you see my query re a small lasr.batchnavigator review ? [01:29] oh right, i clicked the link and then forgot about it [01:29] heh :) thanks! [01:33] https://github.com/defunkt/jquery-pjax#readme [01:34] hi huwshimi? [01:41] poolie: Hey [01:44] hey there [01:44] i had a random idea the other day; i'm not saying it's very urgent to actually do, but i just thought i'd ask you [01:44] and that was, perhaps bug heat should just be hidden from display and used only as a sort key [01:44] (and perhaps be in the api) [01:45] that would avoid some number of complaints or bugs about the absolute values and the scaling onto flames being a bit weird [01:45] (eg that they're inconsistent across pillars) [01:46] I think thats a bad idea [01:46] because it hides the machinery [01:46] which makes it less understandable, and thus more frustrating when its wrong [01:46] I think we should fix the machinery instead [01:47] but the machinery is almost entirely hidden today [01:47] it is not understandable at all [01:47] I'm not justifying what we have [01:47] I explaining why I think making it less visible would be a mistake [01:47] Interesting. I don't really know much about bug heat. What do we use it for apart from the default bugs page? [01:48] it's shown as a series of flames in some bug lists, and on bug pages [01:48] we permit a search sorted by heat [01:48] and for sort ordering [01:48] in fact that order is a common default [01:48] and the distro and some other teams use hot bugs as a health check [01:48] poolie: the default sort is importance [01:48] poolie: only the hot bugs widget sorts by heat by default [01:48] really? [01:49] because claims they are hot bugs [01:49] the title may be wrong [01:49] poolie: right thats the widget [01:49] note that it doesn't show page navigation [01:49] it does link to a sort by heat [01:49] ok, and the 'show me more' is by heat [01:49] but if you click on search at the top [01:49] but, you're right, 'open bugs' is by importance [01:49] you get by importance [01:49] etc, yes. [01:50] and, amusingly enough 'high bugs' sorts by importance too [01:50] there is/was a bug open saying 'please show the inputs to the heat on the flames' [01:50] i might have filed it [01:50] when you say people use it as a health check [01:50] do you mean they count how many 4-flame bugs there are? [01:50] or they just look at the open bugs sorted by heat [01:50] One of the complicating factors there is that the algorithm decays, so you can't statically calculate, you have to redo each time. [01:51] poolie: the latter [01:51] well, my idea would keep that [01:51] stub is investigating heat as a possible cause of index bloat [01:51] i think sorting is pretty worthwhile [01:51] i suspect showing it on the bug page is pretty much noise [01:51] if it is, we'll need to revisit the entire implementation [01:52] I suspect we'll get the same sort with a /little/ fuzz without the decay [01:52] (because commenting on a bug removes the decay for a day) [01:52] anyhow, it was just a random thought [01:52] that perhaps making it serve the purpose of providing an ordering, and making it provide an absolute metric, and making it provide a scaled ranking [01:53] may be asking too much [01:54] lifeless: https://code.launchpad.net/~wgrant/launchpad/bug-756983/+merge/57097 [01:55] lifeless, do you think anyone loves, or looks at, the flame icon? [01:55] they may well do [01:55] wgrant: I think bigjools and raphael may be confused by your comment abot desirability [01:55] but not me [01:55] lifeless: Possibly. [01:57] poolie: I think that if we don't expose it, we will have questions like your one about 'sort by date changed' [01:57] poolie: we don't expose date changed in the UI, so you can't decide if its buggy or not and you file a bug. [01:58] lifeless, if you mean the bug i filed, i think that was actually a complaint about it not being in the ui [01:58] gah, in the API [01:58] poolie: the bug you filed was that a particular bug was sorted at the top of 'my bugs sorted by date' [01:59] well, that is just weird [02:00] poolie: bug 752169 [02:00] it seems like there for some reason the value is correct but the sort is wrong [02:00] <_mup_> Bug #752169: wrong ordering for date_last_updated? < https://launchpad.net/bugs/752169 > [02:00] right [02:00] poolie: all I'm saying is that when there is a sort order and the data behind it isn't visible, folk will get confused. [02:00] sure [02:01] i don't think the bug is very analogous [02:01] but i agree that being able to see "why is it doing that" can be good [02:02] i don't think the flames are very helpful in understanding it [02:03] i think it's more likely people will consider the numeric heat is wrong, than that lp will fail to sort integers once they have been calculated [02:03] (which seems to be happening in 752169) [02:03] sure [02:12] lifeless: i don't know if you care, but ListRangeFactory.getSlice will not DTRT in size==0 and forwards=False [02:12] i think [02:13] (as in, i think it will return the whole list) [02:13] slicing is one of the points were one wants a negative zero [02:14] mwhudson: I don't think we need to care; size 0 is utterly bogus anyway. [02:14] of course, some one will trigger a timeout using that someday. [02:17] :( [02:17] lp://qastaging/ urls don't work [02:17] lifeless: it's almost a natural law, any code around slices will get at least one edge case a bit wrong [02:17] spm: what's the status of the staging code update? [02:18] should be able to kick that off shortly actually [02:18] (i spent a long time fixing problems in CPython's code wrt this sort of thing) [02:18] spm: it says it is doing one now [02:18] thumper: technically that's a lie, it's just the existing locks and blocks still in place from the fail on the w/e [02:19] spm: ok, so ETA? [02:19] to kick it off, RSN, completion, no idea. [02:19] ok [02:20] I think the last successful full restore was ages back, and rob's asked for a full in that case, so... 15-20 hours? [02:21] mwhudson: yup, I've at least one patch in CPython for this too - on find() IIRC [02:21] (which takes slice params, but didn't interpret them correctly) [02:21] wallyworld: On the dialogue do you know how to get the green bar on below the title back? [02:25] huwshimi: is that missing? let me look [02:27] huwshimi: so it is. i will figure out how to make that stay. it may be that it's tied to the "steps" node which is hidden when the confirmation is shown [02:28] yeah I wondered that. I had a look at the privacy dialog on the same page and it somehow manages to have it [02:30] win [02:30] DataError: OFFSET must not be negative [02:41] StevenK: good morning [02:41] Morning [02:43] lifeless: anyway to see the post params for the oops? https://lp-oops.canonical.com/oops.py/?oopsid=1131EC415 [02:44] thumper: sure it [02:44] s [02:45] field.actions.* in the request variables [02:45] It's in the URL. [02:45] ah... I see it now [02:45] http://infram.wordpress.com/kadosu-categorized-document-search/specification-\u2026put-of-pluginsspecification-for-filtering-text-output-of-plugins [02:45] is the problem [02:47] erm [02:47] wtf [02:47] # Branch: launchpad [02:47] # Revno: 7688 [02:47] ? [02:48] the revno [02:48] oh [02:48] 2009 oops [02:48] :) [02:48] https://lp-oops.canonical.com/oops/?oopsid=OOPS-1926EE1366 looks more interesting [02:48] :( [02:49] it seems the middle click button on my mouse is dying [02:49] argh [02:49] sqlobject fail fail fail fail fail fail fail fail fail fail fail [02:49] lifeless: Oh? [02:50] partial implementation of slice protocol [02:50] resultset[-5:None] -> boom [03:01] mwhudson: pushing a tweak [03:01] lifeless: whee! [03:02] rule #1 if you make it look like a slice, let it damn well implement slice [03:02] * lifeless grumps [03:02] lifeless: does resultset[-5:] work? [03:02] WTF? we have our own urlparse? [03:03] all it does is encode('ascii') [03:03] thumper: something to do with unicode [03:03] yeah, but I'm still trying to work out why we care [03:03] We are broken and generate Unicode URLs in a lot of places. [03:03] the cache gets polluted with unicode objects if you feed it a unicode url [03:04] or did, maybe that's fixed in 2.6... [03:04] what cache? [03:04] mwhudson: the fail is because it passes the offset down to sql unaltered [03:04] lifeless: lovely [03:04] mwhudson: in principle it could reverse all the sort elements automatically and adjust [03:04] mwhudson: -5:None = reversed[:5] afterall [03:05] lifeless: total_size is replaced with total_size_link in recent webservice versions. [03:05] thumper: urlparse has a cache [03:05] (re bug #757007) [03:05] <_mup_> Bug #757007: total_size element returned in all batch retrievals < https://launchpad.net/bugs/757007 > [03:05] mwhudson: I realise you're asking if __getitem__ in storm handles the second parameter differently; I haven't checked all the way [03:05] wgrant: our tests look for total_size [03:05] wgrant: and its still there [03:05] mwhudson: really? [03:05] WTF for? [03:05] thumper: beats me [03:06] because parsing urls is slow [03:06] lifeless: They must be using an older version. [03:06] I forget the threshold.. possibly 1.0. [03:06] and rather than expose a parser object with state, - win - [03:06] first_version_with_total_size_link = "devel" [03:06] lifeless: yes because parsing the same url over and over again is a common case, obviously [03:06] * mwhudson coughs [03:07] mwhudson: if you use damaged stacks, yes. [03:07] thumper: looks like the bug launchpad is working around is fixed though [03:07] mwhudson: yeah [03:07] ? [03:07] encoding a url to ascii is sensible though [03:08] if it can't be encoded, its *not* a valid url and we should reject it. [03:08] the stdlib url stuff is broken in many more ways than mere caching [03:08] * lifeless wouldn't ever use it by choice [03:08] even more sadly upstream just fundamentally don't understand what urls are. [03:09] See the 50 mail thread or whatever was on the *worse* breaking in 3.x [03:09] lifeless: catching the unicode decode error in validate url seems pretty simple [03:10] thumper: yup, works for me [03:11] lifeless: A friend from uni tried to fix various bits of the 3.x stdlib URL unicode stuff... it was all pretty wrong and upstream was not at all receptive to fixing it in a reasonable way. [03:11] s/pretty/entirely/ :) [03:11] Yeah. [03:11] as your friend probably knows [03:11] I think it's less bad now, though. [03:11] Still awful, though! [03:12] py3 broke email, broke the web stacks. [03:12] The latest WSGI is 3.xable, isn't it? [03:12] I suspect so [03:12] They finally fixed that, IIRC. [03:12] yes [03:12] with class sensitive string manipulation functions [03:13] Yay [03:13] because bytes cannot be permitted to have a find() method, obviously. [03:15] I wish they'd backported more of the bytes API to 2.7 :/ [03:15] Like, say, having a bytes type that indexes like a bytes, rather than making it index like a str. [03:15] other than a misleading type alias? [03:15] Grar. [03:16] Exactly. [03:16] It's an alias to something that is not at all like a bytes! [03:16] gustavo had a pretty solid rant on this [03:16] my opinion is that the bytes type is daft [03:16] I ended up casting lots of stuff to a bytearray to get compatible indexing. [03:17] mwhudson: can you check my incremental commit, see if still happy ? [03:17] lifeless: looking [03:17] With that, at the cost of one more count(*) on Last links & only on Last links [03:17] it fixes the last of the test fallout [03:18] Is r12758 actually QA-able at all? [03:19] It's been top of the deployment report for a few days [03:19] StevenK: thumper was looking at it earlier. [03:19] probably got distwacted [03:19] The HWDB thing needs QA. [03:19] StevenK: if this is the stacking issue, yes it is qa-able [03:20] I'm wanting staging to update to poke properly [03:20] wgrant: revno? [03:20] r12787 [03:20] thumper: thats going to be a day or more [03:20] thumper: use qastaging [03:20] Fixes the top OOPS. [03:20] lifeless: wow, IFiniteSequence? [03:20] but ok [03:20] mwhudson: yes, really - its what the z3batch code does [03:21] lifeless: yes, still looks fine [03:21] mwhudson: so I assume there is an adapter registered for result sets somewhere or other [03:30] is the branch scanner running on qastaging? [03:30] or do we need to ask? [03:31] I'd ask. [03:33] spm: qastaging branch scanner plz [03:36] thumper: it's not setup to run, is running manually atm [03:36] spm: can you please run manually? [03:36] seems to be working even. /shock horr [03:36] spm: I'll be asking again in a few minutes [03:36] or [03:36] oh. I didn;t type that clearly. "is running manually atm" means I've kicked off a manual run already [03:37] 2011-04-11 02:36:50 INFO Adding 479 new revisions. [03:37] 2011-04-11 02:36:56 INFO Deleting 0 branchrevision records. [03:37] 2011-04-11 02:36:56 INFO Inserting 34782 branchrevision records. [03:37] that isn't the one I wanted... [03:37] is it done? [03:37] not yet [03:38] how is qastaging codehosting set up? [03:38] branch scanner is "cronscripts/scan_branches.py" ? [03:38] spm: aye [03:39] spm: when you you get your new au losa? [03:39] thumper: not sure what you mean? it's setup and running on tellurium. lpqastaging.tellurium.canonical.com. [03:39] Less than a month until the new victims. [03:40] spm: as in a completely separate hosting area? [03:40] I'm not sure when they start. [03:40] May 5? [03:40] Something like that. [03:40] spm: ShipIt's finished. Please remove the cronjobs. [03:40] thumper: yes [03:41] thumper: it has its own branches and copies the same set across on restores that stging does [03:41] ok [03:41] spm: please ping when scan complete [03:44] * StevenK grumbles loudly at IDS. [03:45] If you pass in processor families to the package cloner, it will look up every SPPH for the destination distroseries and call .createMissingBuilds() on it. [03:46] Yes? [03:46] thumper: https://pastebin.canonical.com/45894/ [03:46] Which is *slow* [03:46] Yes. [03:46] This makes me sad. [03:47] createMissingBuilds sucks, and needs to be fixed as the next phase for some timeouts. [03:47] spm: very :(( [03:47] spm: that is because there was a scan job for a branch not copied to qastaging [03:48] spm: and the oops config isn't set for the scanner on qastaging [03:48] spm: if I said make all scan jobs as complete, could you do that without sql? [03:49] wgrant: It might be in-scope for us, because calling .createMissingBuilds() on 17,000 SPPHs makes mawson cry. [03:49] StevenK: Sure it's not the populate-archive bug? [03:50] StevenK: It may not be, but getPublishedSources now preloads huge amounts of useless stuff which makes any script using it on thousands of publications completely useless. [03:50] wgrant: lib/lp/soyuz/model/packagecloner.py, _create_missing_builds() [03:51] The package cloner suffers from Underscore Disease. [03:51] Yeah, p-a bug. [03:51] It's still slow, but not that slow. [03:51] Cowboy the patch from https://wiki.canonical.com/IncidentReports/2011-03-29-LP-populate-archive-slow [03:52] spm: Hi, I need a cronjob run on qas. [03:52] 'scripts/process-hwdb-submissions.py -vv', and I'd like the output. [03:53] wgrant: Because eager loading fixes everything. :-( [03:53] StevenK: Yes. [03:55] StevenK: its a key tool [03:55] the problem is that an object model assuming cheap lookups is broken in the real world. [03:55] thumper: "if I said make all scan jobs as complete, could you do that without sql?" Um. I dunno .but we can use sql on qas? [03:55] spm: I'll get back to you shortly [03:56] np [03:56] lifeless: The problem is the eager loading breaks at least parts of Soyuz. [03:56] s/parts/two parts/ [03:56] wgrant: do you expect lots of output, or just a little? ie log or screen gab [03:56] StevenK: thats the symptom [03:56] spm: I'm not entirely sure. Might be best to log. [03:56] okidoki [03:56] StevenK: the problem is that a single api was used when there are different usecases at hand. [03:56] spm: It depends how the DB was when qastaging was restored. [03:57] lifeless: The use-cases were identical. [03:57] lifeless: But now we have a policy of eager loading. [03:57] So they differ. [03:58] wgrant: they weren't really identical, the differentiation was around performance [03:58] wgrant: ftr: ~/tmp/wgrant-process-hwdb-submissions.log [03:59] ahh. cronscripts/process-hwdb-submissions.py would work better [03:59] spm: Er, yes, sorry. [04:00] lifeless: The only things that required the eager loading were the callsites that directly impacted on render times. .getPublishedSources() is called from other places, too. [04:00] wgrant: heh, is cool. it was third time lucky in any event. '/cripts/process...' didn't work either. :-) [04:00] StevenK: it required eager loading because it was making things slower for render times [04:00] StevenK: I agree that other places were negatively impacted [04:01] wgrant: argh. one sec. will fix. FATAL: Ident authentication failed for user "hwdb-submission-processor" [04:01] Yes, hence my "directly impacted on render times" [04:01] * spm pulls out the yak clippers [04:01] StevenK: the function I change directly impacted render times. [04:03] Perhaps we change the internal API of .getPublishedSources() to take a eagerload boolean, and it can return early if it's False (and it defaults to True) [04:03] It's not particularly elegant, but it solves both cases. [04:05] StevenK: sure, thats what we've done elsewhere. [04:05] StevenK: I didn't here because of oversight, not intent [04:05] StevenK: the scripts weren't visible to me when I grepped around quickly, for $whatever reason. [04:07] wgrant: https://pastebin.canonical.com/45895/ [04:07] spm: Thanks. [04:08] * StevenK looks for the p-a bug [04:08] spm: Do you want an RT for killing the shipit cronjobs? Or will the existing shipit cleanup one do? [04:08] the existing covers it [04:09] wgrant: Hm, does the p-a slowness even have a bug? [04:09] StevenK: Bug #744849 [04:09] <_mup_> Bug #744849: populate-archive.py unusably slow < https://launchpad.net/bugs/744849 > [04:10] Bah, that didn't show up when I searched for populate-archive [04:11] wgrant: Thanks, I'll look at it after lunch. [04:23] wallyworld: Did you have any luck with that border? [04:24] huwshimi: yes. all good. i've improved the animations and also provided animation for when the user hits "no". all i need now is some css for the confirmation content and i can submit the mp [04:24] wallyworld: Is all that pushed to your branch? [04:24] huwshimi: yes [04:25] wallyworld: Thanks I'll take a look [04:25] huwshimi: the mp is also written but on hold so you can see that too if you want [04:25] huwshimi: sorry, i wasn't aware you were waiting to hear back from me [04:27] wallyworld: That's ok I just wanted to see how it looked with the bar added [04:30]