[01:24] <StevenK> lifeless: So, sinzui told me about another table that I had forgotten about on the call this morning.
[01:24] <StevenK> launchpad_dogfood=> SELECT sourcepackagename from distributionsourcepackage as dsp join sourcepackagename as spn on dsp.sourcepackagename = spn.id where dsp.distribution = 1 and spn.name like '%linux%';
[01:24] <StevenK> Time: 37.420 ms
[01:24] <lifeless> is that populated now ?
[01:24] <StevenK> Seems to be
[01:24] <StevenK> 30k rows, a whole bunch for Ubuntu and Debian
[01:25] <StevenK> lifeless: Try that query on staging for me?
[01:25] <lifeless> 53ms
[01:25] <lifeless> 136 rows
[01:25] <lifeless> which suggests it includes stale packages.
[01:26] <lifeless> is linux 2.6.8 still valid, for instance ?
[01:26] <lifeless> StevenK: spn was never a problem anyhow ;)
[01:27] <StevenK> It's a published source package, at least
[01:27] <StevenK> lifeless: Well, no, but if we can fetch a good list of spns in 50 ms, it gives us a bit more time for the binary side
[01:29] <wgrant> StevenK: You can't use DSP.
[01:29] <StevenK> :-(
[01:29] <wgrant> StevenK: It's used for storing bug reporting guidelines.
[01:29] <wgrant> That is its only meaning.
[01:29] <wgrant> Some uploads and gina create them.
[01:29] <wgrant> But you cannot derive any meaning from them.
[01:30] <StevenK> Curtis was of the opinion that it was used for upstream linking too
[01:30] <lifeless> it is
[01:30] <lifeless> it could be used, if you extend it to know active/inactive
[01:31] <lifeless> and make sure its kept up to date
[01:31] <StevenK> TBH, if wgrant says the data in it is not trustworthy I'm inclined to not use it
[01:31] <lifeless> StevenK: oh, and for the vocab, don't forget to also look for published source package branches with no SPR
[01:32] <StevenK> lifeless: Right, and that 26 minute query can take even longer :-P
[01:33] <lifeless> StevenK: there is a escalated bug for ensemble requesting this
[01:33] <StevenK> lifeless: Curtis and I have discussed it a few times. I want to get to the vocab working and then work on extending it
[01:33] <lifeless> sure
[01:36] <wgrant> Aha!
[01:36] <wgrant> Unity has taken minimalism a step further.
[01:36] <wgrant> It is now Nautilus.
[01:37] <wgrant> The launcher and dash and indicators are gone... all there is is the Nautilus root window and its menu bar.
[01:50] <wallyworld_> does anyone know how to create a project or distro with an icon for testing purposes?
[01:51] <lifeless> set the branding
[01:51] <lifeless> click on 'edit details' scroll to the bottom then 'edit branding', I think
[01:51] <lifeless> its a very clunky bit of ui
[01:51] <wallyworld_> lifeless: i mean in code
[01:52] <lifeless> not offhand - does the factory have an option ?
[01:52] <wallyworld_> not that i can see :-(
[01:52] <StevenK> factory.makeDistribution() and then you probably have to twiddle something in it
[01:52] <StevenK> Let's see
[01:52] <lifeless> look for tests for branding to see how
[01:53] <wallyworld_> ok, thanks. my other searches have failed to find anything
[01:53] <StevenK> wallyworld_: Create an LFA and then set IDistribution.icon to it
[01:53] <StevenK> You also have IDistribution.logo
[01:53] <wallyworld_> LFA?
[01:53] <StevenK> LibraryFileAlias
[01:54] <wallyworld_> ok, there is a factory or something for that?
[01:54] <StevenK> factory.makeLibraryFileAlias(content=<foo>) ; transaction.commit()
[01:54] <wallyworld_> excellent, thanks
[02:59] <wallyworld_> wgrant: StevenK: want a small review (one line change plus tests)? https://code.launchpad.net/~wallyworld/launchpad/person-affiliation-breakage-823644/+merge/70981
[03:00] <StevenK> wallyworld_: It will cost you. What do you have?
[03:00] <StevenK> Oh, right. You're married with kids, so nothing.
[03:00] <wallyworld_> my eternal gratitude?
[03:00] <wallyworld_> hah
[03:00] <wallyworld_> i keep photos in my wallet where my money used to be :-/
[03:02] <j-johan-edwards> > from lazr.restful.interfaces import IRepresentationCache
[03:02] <j-johan-edwards> ImportError: cannot import name IRepresentationCache
[03:03] <j-johan-edwards> Any idea what might be going on? (I have lucid's python-lazr.restful installed)
[03:04] <StevenK> wallyworld_: r=me with a comment
[03:04] <wallyworld_> StevenK: thanks
[03:06] <wallyworld_> StevenK: with the commit, there's a number of other usages of LFA in tests without a commit. modt of them seem to not have one
[03:07] <StevenK> j-johan-edwards: The version of python-lazr.restful in lucid is about 18 months old
[03:07] <StevenK> wallyworld_: Right, so I'm not saying thou must add thy commit, or no +1 from me
[03:08] <StevenK> wallyworld_: I'm just saying it's something to watch for
[03:08] <wallyworld_> StevenK: np. thanks
[03:08] <j-johan-edwards> StevenK: Ah. The wiki Running page said Lucid was the only support version
[03:08] <StevenK> j-johan-edwards: What are you trying to do with lazr.restful?
[03:10] <j-johan-edwards> Trying to make heads or tails of the source tree. Imported lp.soyuz.model.binarypackagerelease, which asks for that module.
[03:11] <StevenK> j-johan-edwards: I would strongly suggest you use rocketfuel-setup to check out LP's tree
[03:11] <j-johan-edwards> I did. It claimed everything was okay, so I ran `make schema`, which also worked
[03:11] <j-johan-edwards> `make run` also works, oddly
[03:12] <j-johan-edwards> I actually tried building on Natty too, but had a similar import error for lazr.config
[03:12] <StevenK> Oh! Are you running 'python' and trying to import stuff?
[03:13] <j-johan-edwards> Yup
[03:13] <StevenK> Yes, that won't work
[03:13] <StevenK> Use 'bin/py' in the source tree, or 'make iharness'
[03:14] <j-johan-edwards> Thanks a ton! Just tried it again, and that works great.
[03:22] <wgrant> j-johan-edwards: LP uses lots of dependencies that aren't in Lucid. You can find them in eggs/ within the LP branch.
[03:24] <lifeless> nesting project groups would be mega useful.
[03:25] <wgrant> lifeless: No.
[03:25] <wgrant> lifeless: Nesting projects :)
[03:30] <lifeless> wgrant: well, if project groups died, sure
[03:30] <wgrant> They are going to die before they get any significant changes.
[03:57] <LPCIBot> Project devel build #960: FAILURE in 5 hr 42 min: https://lpci.wedontsleep.org/job/devel/960/
[04:18] <lifeless> is it intentional that LP shows a /!\ in the involvement portlet when you select 'N/A' for translations ?
[04:19] <StevenK> It's probably supposed to be an icon
[04:20] <lifeless> well it is an icon
[04:20] <lifeless> I'm just confused
[04:20] <lifeless> if you say 'unknown', a warning with 'lp needs to know' is sensible
[04:20] <lifeless> but its set to N/A
[04:20] <lifeless> shouldn't that just hide the line ?
[04:29] <j-johan-edwards> Hello, sorry to come crying for help again so soon
[04:29] <j-johan-edwards> Every getUtility(Iwhatever) call I make results in a zope.component.interfaces.ComponentLookupError
[04:30] <wgrant> j-johan-edwards: bin/py doesn't set up utilities -- use bin/harness for that.
[04:30] <j-johan-edwards> thanks!
[04:32] <j-johan-edwards> Is there some piece of documentation I'm missing on the wiki? Or are some sections of the Hacking page (eg #Storm) just incomplete?
[04:32]  * j-johan-edwards ignores the possibility he simply fails to grasp the obvious
[04:32] <lifeless> j-johan-edwards: theres a vast learning curve around lp development
[04:32] <lifeless> we want to improve that
[04:32] <lifeless> (by lowering the curve, not documenting the curve :P)
[04:33] <wgrant> j-johan-edwards: Well, Hacking doesn't say it should be run in bin/py.
[04:33] <StevenK> It isn't a curve, it's a bleeding cliff!
[04:34] <j-johan-edwards> haha
[04:35] <lifeless> j-johan-edwards: so what are you hacking on ?
[04:37] <j-johan-edwards> lp:archive-index (I might give up if I can't hack in though)
[04:38] <lifeless> j-johan-edwards: integration with LP ?
[04:38] <j-johan-edwards> lifeless: yeah, for the past few months I've been working on an archive crawler for mvo
[04:39] <j-johan-edwards> but it's becoming obvious it won't scale, so an automated solution is important
[04:39] <lifeless> what sort of data does it extract?
[04:39] <j-johan-edwards> app-install-data-ubuntu, basically
[04:40] <StevenK> Right, so it's as I feared. You'd like an easier way to get at contents rather than parsing Contents-<arch>.gz?
[04:41] <j-johan-edwards> Yup, I saw your populate-bprc branch, which boosted my hopes of making that possible
[04:41]  * StevenK had no idea he had a stalker ... :-P
[04:42] <j-johan-edwards> haha, lifeless actually directed me to you, so you've got two
[04:42] <StevenK> lifeless just stalks all LP devs, so that's nothing new
[04:43] <lifeless> StevenK: and now its my job!
[04:43]  * StevenK waits for "And I'm watching you ..."
[04:44] <stub> StevenK: that will arrive by a postcard slipped under your door.
[04:45] <StevenK> Haha
[04:45] <wgrant> No, he'll find it in his pocket.
[04:45] <lifeless> ... of his pyjamas
[04:45] <StevenK> Along with a video called "The Branch" ?
[04:46] <StevenK> j-johan-edwards: Keep in mind the BPRC work is in my spare time, and I'm not really pushing it with any priority
[04:46] <nigelb> StevenK: "It isn't a curve, it's a bleeding cliff!" +!
[04:46] <nigelb> +1
[04:46] <wgrant> It's not that bad!
[04:47] <wgrant> I submitted my first branch after like a day.
[04:47] <StevenK> wgrant: Yes, but 1) You're insane. 2) You already knew Zope.
[04:47] <wgrant> And it even changed tests.
[04:47] <nigelb> haha
[04:47] <nigelb> I wsa about tos ay that.
[04:47] <nigelb> I wrote a one line patch.
[04:47] <nigelb> Then I realized I had to write 25 lines of tests :)
[04:48] <nigelb> so, for someone starting out, you open the code. and you need to grep to find what you're looking for. Then you a *lot* xml files.
[04:49] <StevenK> My first branch for LP changed 2 lines. Sadly, it then broke 200 tests
[04:49] <nigelb> fun!
[04:49] <nigelb> I redid a branch 2 times I think.
[04:50] <StevenK> nigelb: Why? You can get away with ignoring the ZCML
[04:50] <StevenK> And bzr grep is *love*
[04:50] <nigelb> StevenK: of course you can. but its scary to see a lot of xml :)
[04:50] <StevenK> nigelb: Lots of XML is solved by shovelling more XML at the problem
[04:51] <StevenK> After a while you forget what you were trying to solve.
[04:52] <nigelb> is that like regular expresssions?
[04:52] <nigelb> ;)
[04:52] <StevenK> Oh hey, I wrote one of those yesterday
[04:52]  * ajmitch is suddenly scared
[04:52] <StevenK> And then ran it over devel ...
[04:52] <nigelb> "if you have a problem and you use regular expressions to solve it, then you have 2 problems"
[04:53] <StevenK> % bzr di -r submit: | wc -l
[04:53] <StevenK> Using submit branch /home/steven/launchpad/lp-branches/devel
[04:53] <StevenK> 6501
[04:54] <nigelb> also of note, now I'm more comfortable with lp.
[04:55] <StevenK> nigelb: You need to submit more branches!
[04:55] <nigelb> StevenK: working on one, need more hours in a day.
[04:55] <StevenK> Make a branch for that too?
[04:56] <nigelb> against what project?
[04:56] <StevenK> lp:universe
[04:57] <nigelb> aha
[05:00]  * StevenK ponders how to write a db patch, given this whole FDT thing
[05:00] <nigelb> I'm going through rtfd that jml set up some time back.
[05:01] <nigelb> I wonder if lp will ever get a feature like rtfd/github pages
[05:01] <lifeless> if someone does it well, sure
[05:02] <lifeless> though there is merit in just integrating with rtfd well, particularly if that site is open source (is it?)
[05:02] <nigelb> it is
[05:02] <nigelb> its django based
[05:02] <lifeless> nigelb: and its source is available ?
[05:02] <nigelb> https://github.com/rtfd/readthedocs.org
[05:03] <lifeless> cool
[05:03] <lifeless> now just need to get them hosted on an open site
[05:04] <nigelb> so, out of the box, rtfd already can scan bzr branches.
[05:04] <StevenK> That is one thing that annoys me. How much flak did LP cop for not being open, and you don't see people complaining about github
[05:04] <nigelb> heh
[05:04] <wgrant> Nobody is going to clamour for GitHub to be open, because GitHub is not crap.
[05:04] <lifeless> that seems spurious :)
[05:04] <wgrant> Oh?
[05:05] <wgrant> People wanted LP to be open partly because LP was a piece of shit :)
[05:05] <lifeless> I would expect the reverse correlatio
[05:05] <lifeless> more good -> more desire for it to be open
[05:05] <nigelb> everyone seems to have a github.
[05:05] <wgrant_> jtv: QA!
[05:06] <nigelb> haha
[05:06] <wgrant> That was a real WTF moment for a while there.
[05:06] <jtv> wgrant: FO!
[05:06] <nigelb> lol
[05:06] <nigelb> WIN
[05:06] <nigelb> StevenK: That was awesome.
[05:06] <wgrant> I must need to crank up the protection on wgrant_...
[05:07] <StevenK> Now to look up my Nickserv password again
[05:07] <nigelb> can't protect it for these less that 30 sec bursts.
[05:07] <nigelb> unless you're signed into it.
[05:07] <nigelb> in which case, StevenK can still d wgrant__
[05:07] <nigelb> or wgrant`
[05:08] <StevenK> It was for comedic value only
[05:08] <nigelb> and it was sucessfull run!
[05:08] <wgrant> Yeah, I only have wgrant and wgrant_, and ENFORCE only works after a certain time.
[05:08] <wgrant> Sad.
[05:08] <StevenK> I told jtv he had QA to do and he told me I could have been better at impersonating wgrant.
[05:08] <StevenK> So I was.
[05:23] <nigelb> So, hacking on Lp can still be counted as rest when I'm sick right? :)
[05:23] <wgrant> I hope so, or I'm a bad person.
[05:23] <lifeless> if it counts as rest, thats proof you are sick ?
[05:24] <nigelb> hah
[05:30] <lifeless> pop quiz
[05:31]  * wgrant blames python-oops.
[05:31] <lifeless> Would defining an oops report as a 'dict suitable for bson serialisation' be problematic ?
[05:31] <lifeless> wgrant: :P
[05:31] <wgrant> lifeless: That's what has always made a lot of sense to me.
[05:35] <j-johan-edwards> I just retrieved a record from a table
[05:35] <j-johan-edwards> Most beautiful moment of my life...
[05:37]  * wallyworld__ doing qa, and finds an interesting user on qas:
[05:37] <wallyworld__> Cost Of Viagra - Online Pharmacy - No Prescription Drugs, Health and Beauty, plus more in Launchpad in Launchpad (dalefyoboardinghouse)
[05:37] <wallyworld__> <email address hidden>
[05:37] <wallyworld__> Member since 2009-12-17
[05:37] <wgrant> Hah.
[05:37] <nigelb> lol
[05:38] <wallyworld__> i'm too young for viagra
[05:38] <wgrant> We get a lot of them, sadly.
[05:38] <wallyworld__> just
[05:38] <wallyworld__> do we have permission to deactivate such users ourselves?
[05:39] <wgrant> Administer account
[05:39] <wgrant> Set status to suspended.
[05:39] <wgrant> Declare victory.
[05:39] <wgrant> There are instructions on the wiki.
[05:39] <wgrant> On dealing with spam.
[05:39] <wallyworld__> np. was just wondering out loud if we had permission to do that
[06:02] <lifeless> oh wow
[06:02] <lifeless> this is going to break LP :P
[06:02] <lifeless> we've been parsing iso8601 timestamps as TZ unaware.
[06:02] <wgrant> Python's good at that.
[06:04] <lifeless> ah well, extracted code can be good.
[06:04] <lifeless> LP can deal.
[06:05] <lifeless> woot
[06:05] <nigelb> extracted? abstracted?
[06:06] <lifeless> extracted
[06:06] <nigelb> what does that mean?
[06:09] <lifeless> to take out of
[06:09] <lifeless> well, 'taken out of' 'removed'
[06:10] <nigelb> ah
[06:10] <lifeless> lp:python-oops
[06:19] <lifeless> can has review? https://code.launchpad.net/~lifeless/python-oops/extraction/+merge/70993
[06:21] <StevenK> Suppose
[06:22] <lifeless> original code is in lib/canonical/launchpad/webapp/errorlog (and tests/test_errorlog)
[06:45] <lifeless> StevenK: thanks!
[06:51] <lifeless> stub: how long does the 'ALTER TABLE BugMessage ALTER COLUMN owner SET NOT NULL;' take to apply ?
[06:52] <lifeless> stub: (I'm wondering if an index could do it instead, and be done CONCURRENTLY
[06:58] <nigelb> who's the postgres expert?
[06:58]  * nigelb points them to https://schemaverse.com
[07:01] <lifeless> can has review? https://code.launchpad.net/~lifeless/launchpad/useoops/+merge/70995
[07:01] <lifeless> nigelb: thats evil
[07:04] <nigelb> heh
[07:05] <rvba> lifeless: Hi, I'd be happy to move all the js files used to ~lpqateam but I'm afraid you will have to tell me how to do that (if I can do it myself) because I don't seem to have the right permissions.
[07:06] <lifeless> I can probably do that later tonight; or losa could;
[07:06] <lifeless> or francis or the qa team :)
[07:11] <rvba> lifeless: ok thanks, if you don't have to do it before you EOD, I'll grab a losa later today then.
[07:13] <jtv> Thanks for the review, StevenK
[07:15] <rvba> don't have *time* even
[07:17] <jtv> StevenK: The extra check you're asking for is not under the purview of the method that the test is for; it's DSD.update's job and presumably you tested for it!
[07:17] <jtv> The fact that update() is called on the DSD is also already tested.
[07:39] <adeuring> good morning
[07:39] <stub> lifeless: as long as it takes to do a full table scan of bugmessage
[07:39] <stub> lifeless: You can't duplicate that constraint with an index.
[07:40] <stub> lifeless: it might use the index on the column to avoid the scan, but I wouldn't count on it. Staging timing is what we want to see.
[07:41] <stub> full scan is < 10 secs
[07:51] <lifeless> stub: cool
[07:52] <lifeless> stub: we're going to need to think up ways to handle big tables ;)
[08:06] <stub> lifeless: Apart from bypassing PG's protections and attacking the data dictionary directly, you can't. Theoretically CONCURRENTLY could be added to this stuff, but I doubt anyone will bother.
[08:06] <stub> lifeless: Best way of handling big tables is to not have them :-)
[08:06] <allenap> Thanks for the review StevenK :)
[08:06] <lifeless> heh :P
[08:07] <lifeless> can has review? https://code.launchpad.net/~lifeless/launchpad/useoops/+merge/70995
[08:07] <stub> lifeless: Not totally joking. Archiving old data, partitioning
[08:07] <lifeless> stub: yeah, I know
[08:09] <stub> lifeless: reviewed
[08:09] <lifeless> thanks!
[08:10] <StevenK> allenap: I'm not so happy with JS-only reviews, but it looked pretty good.
[08:11] <allenap> StevenK: Didn't you hear? The SOA is going to be based on node.js.
[08:12] <lifeless> allenap: you can troll so much better if you want to
[08:14] <allenap> StevenK: Didn't you hear? The SOA is going to be based on Drupal.
[08:15] <StevenK> allenap: Are you just working down a list? Is PHP the next item? :-P
[08:15] <wgrant> Drupal *is* PHP.
[08:15] <allenap> StevenK: I thought I'd gone past PHP with Drupal.
[08:18] <lifeless> stub: so poking at the internals and reindexing seems plausible as a future recipe
[08:20] <stub> lifeless: I'd consider not adding the constraint before considering internal hacking. That could screw us, and we not know for quite a while.
[08:20] <lifeless> sure
[08:20] <lifeless> not something to do lightly
[08:22] <stub> Yay standardization. Landscape just got pgbouncer installed on port 5434 - assume 5433 was already taken ;)
[08:23] <lifeless> \o/
[08:25] <wgrant> stub: Your slony1-2 packages didn't burn down your machine?
[08:33] <mrevell> Morning
[08:38] <stub> wgrant: Working fine. And all the staging stuff worked out of the box like you said - thought I would have to rewrite parts of the automation but looks unnecessary!
[08:38] <wgrant> stub: Yeah, I was a bit wary about trying it when I did, thinking I was going down a bit of a rabbit hole... but nope.
[08:38] <wgrant> All worked.
[08:41] <wgrant> Python exceptions are slow :(
[08:43] <jtv> Yes.
[08:46] <wgrant> For dicts, it seems that these relations hold: LBYL with 'in' < get() < catching KeyError
[08:46] <wgrant> I would not have expected get() to be slower than a manual check.
[08:46] <lifeless> mmm
[08:47] <lifeless> 'in' + get will be slower than just get, if you know you want the item
[08:47] <lifeless> in itself is fast, yes
[08:47] <wgrant> Nope.
[08:47] <lifeless> really?
[08:47] <lifeless> !cite
[08:48] <wgrant> Well, if there are lots of misses then checking 'in' first is faster.
[08:48] <wgrant> Saves 50ms in privilege delta generation.
[08:48] <wgrant> Not significant, but rather odd.
[08:52] <lifeless> strange:
[08:52] <lifeless> http://paste.ubuntu.com/662462/
[08:52] <wgrant> Odd.
[08:53] <lifeless> get on hit or miss is ~constant
[08:53] <lifeless> in is also approx constant, and 1/3 a get
[08:54] <jtv> how sensitive are they to what's in the dict?
[08:54] <lifeless> the hash() call on the object is crucial
[08:55] <jtv> This is a pretty big difference already.
[08:55] <lifeless> beyond that, its dict size - and they scale very well
[08:56] <jtv> btree, right?
[08:56] <jtv> Just curious about irregularities w.r.t. how deep in the tree the element is, and how branch prediction factors in.
[08:58] <lifeless> its a hash
[08:58] <jtv> Open or closed?
[08:58] <lifeless> open
[08:59] <lifeless> http://permalink.gmane.org/gmane.comp.python.devel/29960
[09:00] <lifeless> mrevell: hey
[09:00] <lifeless> mrevell: I'd like to catch up with you sometime; when would be good?
[09:01] <jtv> No linear probing?  I would've expected that to work pretty well with good access predictors.
[09:01] <mrevell> lifeless, Howdy. How does after the TL call sound?
[09:02] <lifeless> mrevell: sweet
[09:35] <wgrant> stub: So, I have table privilege resetting down from 5.5s to 60ms.
[09:35] <wgrant> Which should shave 40s off the production outage time.
[09:35] <wgrant> Which is possibly handy.
[09:36] <stub> wgrant: \o/
[09:36] <wgrant> The whole thing still takes 450ms per DB, but the first 400ms is as-yet unoptimised and probably small enough that we don't care too much.
[09:38] <LPCIBot> Project db-devel build #797: STILL FAILING in 5 hr 41 min: https://lpci.wedontsleep.org/job/db-devel/797/
[09:38] <stub> wgrant: Yup. We are in the good enough zone for now. We need to test this on staging first of course - if we land it now on db-devel it should get a run before the first possible fastdowntime deploy.
[09:39] <stub> wgrant: or we can leave it until after and have synced db-devel->devel etc.
[09:39] <rvba> matsubara: Hi Diogo, may I bother you with a small "moving files around" problem? I'd like your help to put a few files in https://devpad.canonical.com/~lpqateam/.
[09:39] <rvba> matsubara: Here is the whole story https://code.launchpad.net/~rvb/launchpad/trace-report/+merge/70839
[09:39] <matsubara> hi rvba. sure thing. let me take a look
[09:42] <matsubara> rvba, I need to figure out how the ppr is deployed. I'll do that today and let you know what's needed
[09:45] <rvba> matsubara: Ok great. (It's just a matter of moving the 3 or 4 Javascript file that are needed by reports like https://devpad.canonical.com/~lpqateam/ppr/lpnet/latest-monthly-pageids.html but I'll obviously have to change the paths in the python code that generates the reports once you've decided where to put them)
[09:46] <rvba> files*
[10:05] <allenap> Okay, who removed check-db-revision.py?!
[10:06] <StevenK> allenap: "Come on, own up! I can wait all night ..."
[10:07] <matsubara> rvba, I put all the js files here: https://devpad.canonical.com/~lpqateam/ppr/js/
[10:07] <matsubara> would that work for you?
[10:07] <rvba> matsubara: absolutely. Thanks a lot. I'll modify the script.
[10:08] <matsubara> rvba, cool. you're welcome. let me know when that lands so I can update the launchpad branch on devpad
[10:10] <rvba> matsubara: I'll make the changes right away and throw that at ec2.
[10:26] <danilos> allenap, re check-db-revision, could it be related to our hot/cold db patches stuff? iow, 2208-*-0 patches are checked, and all the others aren't (at least that's how I understood it)
[10:28] <wgrant> danilos, allenap`: We needed a way to have optional patches.
[10:28] <wgrant> danilos, allenap`: And it was decided to just remove the checks entirely.
[10:28] <allenap> wgrant: The problem is that check_schema in Makefile is broken. I wondered if anyone had noticed. If not, I'll kill it.
[10:30] <wgrant> Hm, why didn't that hit anyone earlier, I wonder.
[10:30] <wgrant> Anyway, yeah, should be killed.
[10:36] <jml> is there a syntax highlighting mode for build failures, or something?
[10:39] <wgrant> jml: For binary package builds? No.
[10:39] <wgrant> Given that they are in whatever format the package chooses, that would be... non-trivial.
[10:39] <jml> well
[10:40] <jml> I guess I don't really know enough about what canonical.buildd does, but I could imagine it inserting some kind of markup to make failures easier to read.
[10:40] <jml> or, you know, *find*
[10:42] <wgrant> jml: We have no control over the output between the lines starting 'dpkg-buildpackage: ' and the '************************************************' right before the 'FAILED [dpkg-buildpackage died]'
[10:43] <wgrant> We like to treat everything between 'RUN: /usr/share/launchpad-buildd/slavebin/sbuild-package' and 'RUN: /usr/share/launchpad-buildd/slavebin/scan-for-processes' as not ours, too, since that would bring us further away from real sbuild, when we hope to use the package soon.
[10:44] <jml> for my build failure, those are lines 524-597 out of 720
[10:44] <jml> which is a pretty small fraction
[10:44]  * jml looks again
[10:45] <jml> and would probably be interesting to call out in some way
[10:45] <jml> starting from sbuild-package expands it quite a lot.
[10:46] <jml> wgrant: are there plans to switch to real sbuild?
[10:46] <wgrant> jml: There's even a branch.
[10:46] <jml> zowie
[10:46] <wgrant> But it's not quite landable.
[10:46] <wgrant> It's been dormant for around a year.
[10:46] <wgrant> May get to it when I'm on maintenance again.
[10:46] <wgrant> But it needs to be done this year some time.
[10:46] <wgrant> Or Ubuntu will explode.
[10:46] <jml> well more advanced than 'stop using commercial admin' plans
[10:46] <jml> ubuntu has already exploded a little
[10:46] <wgrant> Heh.
[10:47] <jml> apt-file doesn't work any more. Debian has its Contents files in different places now.
[10:48] <wgrant> They seem to be in the same place as usual...
[10:48] <wgrant> http://ftp.debian.org/debian/dists/sid/Contents-i386.gz
[10:48] <wgrant> Huh.
[10:49] <wgrant> launchpad_main holds only DELETE on packagebuild.
[10:49] <wgrant> Not SELECT or anything.
[10:50] <jml> wgrant: apt-file is looking for them under the component dirs.
[10:50] <jml> e.g. http://ftp.debian.org/debian/dists/sid/main/
[10:51] <wgrant> jml: They're there too (Ubuntu doesn't have component-specific ones, though)
[10:51] <jml> yeah, so apt-file is broken on Ubuntu.
[10:51] <wgrant> Ohh, I see what you mean.
[10:51] <wgrant> Right.
[10:52] <wgrant> I thought you meant our apt-file was broken for Debian.
[10:52] <wgrant> Right.
[10:52] <wgrant> We'll be writing out Contents files natively soon (StevenK has a branch for big bits of that), so it will be easier.
[10:53] <wgrant> We still use apt-ftparchive to do it, which Debian stopped doing a year or so ago.
[10:53] <jml> what does Debian use now?
[10:54] <wgrant> They write it out directly from dak's DB.
[10:55] <elmo> wait what?
[10:55] <elmo> they put package file listings into the DB?
[10:55] <wgrant> Yes.
[10:55] <elmo> wow, that's insane
[10:55] <wgrant> Howso?
[10:55] <wgrant> a-f is not precisely fast.
[10:55] <elmo> the answer is not to bloat out the DB with something that's you don't use in any sort of relational way
[10:56] <elmo> I mean if it works for Debian, all power to them
[10:56] <elmo> but for us, it would be crazy
[10:56] <wgrant> It should ideally be in some separate database, yes.
[10:56] <elmo> it's a cache, not a DB
[10:57] <wgrant> A DB cache.
[10:57] <elmo> *shrug* sure - putting that into launchpad_prod would be, IMNSHO, worse than silly
[10:57] <elmo> (and that's me trying very hard to be polite)
[10:59] <wgrant> The hottest part of the table would be ~300MB that is accessed hourly, or possibly only once a day.
[10:59] <wgrant> It's not *that* terrible.
[11:00] <elmo> a single Contents file for a single architecture/series is 300Mb
[11:00] <elmo> so I question your maths
[11:00] <wgrant> You have a good point there.
[11:00] <elmo> and quite frankly, even if it only ends up adding tens of gigabytes, it's still completely the wrong thing to do
[11:00] <wgrant> I'd forgotten about the whole multiple archs thing there for a moment.
[11:01] <wgrant> Maybe in a few years we'll have multiple DBs.
[11:01] <elmo> and until then we'll keep adding stuff to the single one we have that we don't need to?
[12:35] <allenap> danilos: Do you fancy reviewing a 1910 line diff? 90% of it is mechanical change that you can skip. https://code.launchpad.net/~allenap/launchpad/lazr-anim-stuff/+merge/71016
[12:36] <danilos> allenap, "fancy" is the wrong word, but sure :)
[12:36] <allenap> danilos: Thanks :)
[12:47] <danilos> allenap, ok, mostly good, I wonder if a check Y.Lang.isValue(to) would work better for the "to" variable (line 248 of the diff)
[12:48] <allenap> danilos: Yeah, that would keep in spirit with the rest of the changes.
[12:48] <danilos> allenap, right, that's all I could find to complain about, so r=me :)
[12:49] <allenap> danilos: Thanks!
[13:00] <deryck> Morning, all.
[13:01] <allenap> Morning deryck :)
[13:03]  * danilos -> food
[13:45] <allenap> danilos: Got time for a *much* shorter JavaScript branch? https://code.launchpad.net/~allenap/launchpad/phantom-overlay-bug-820828/+merge/71044
[13:49] <danilos> allenap, you've got a bunch of conflicts there
[13:51] <danilos> allenap, though, that seems to be from the anim branch
[13:52] <allenap> danilos: Yeah. I'm working on them now.
[13:52] <danilos> allenap, r=me, but I wonder about the var declaration and assignment split-up: is there a reason to prefer one over the other? or is that just your coding style that you hope becomes LP JS style?
[13:54] <allenap> danilos: Ah, that's to silence lint as much as anything. It complains that PersonPicker is not fully defined in PersonPicker.
[13:55] <allenap> I have no preference for that style, but I don't mind it either.
[13:55] <bac> matsubara: great job on the dashboard!
[13:55] <allenap> danilos: Thanks for the review :)
[13:59] <matsubara> thanks bac :-)
[15:07] <gary_poster> abentley, are you up for answering an importd question from spm, or should I try to get an answer from thumper or mwhudson?  https://answers.launchpad.net/launchpad/+question/167471
[15:08] <abentley> gary_poster: best to ask thumper or mwhudson.
[15:08] <gary_poster> abentley, ack will do thanks
[15:09] <gary_poster> bigjools, would you be willing to field https://answers.launchpad.net/launchpad/+question/167457?  If I had to guess, I would guess that this is a "convert question to bug and mark WONTFIX" but it's really well out of my league.
[15:10] <bigjools> gary_poster: I'l answer it.
[15:10] <gary_poster> Thank you bigjools
[15:16] <bigjools> gary_poster: answered.  He probably won't like it... I hope you grok what I said
[15:17] <gary_poster> bigjools, I read it and understood, thank you (other than not being familiar specifically with quilt).
[15:17] <bigjools> gary_poster: I am not that familiar with it either :)
[15:18] <gary_poster> heh ok :-)
[15:26] <LPCIBot> Yippie, build fixed!
[15:26] <LPCIBot> Project devel build #961: FIXED in 5 hr 47 min: https://lpci.wedontsleep.org/job/devel/961/
[15:48] <allenap> abentley: Hi there, got time for a little bug fix in browser code? https://code.launchpad.net/~allenap/launchpad/dsd-batch-position-reset-bug-822781/+merge/71069
[15:48] <abentley> allenap: sure.
[15:48] <allenap> Thanks.
[15:49] <abentley> allenap: what is the full_url for?
[15:51] <abentley> allenap: I think "The forms should post to themselves, including GET params" is internally inconsistent.
[15:54] <allenap> abentley: The full_url is to show that the URL is not just for the action, which it was before. It's just a naming exercise to show that the action_url and next_url are both essentially the URL with query parameters.
[15:54] <allenap> abentley: I'll fix the docstring, oops.
[15:59] <abentley> allenap: It would be nice if  BasicLaunchpadRequest.get_url could produce URLs that included the query string.
[15:59] <abentley> s/get_url/getURL/
[16:00] <abentley> allenap: r=me.
[16:02] <allenap> abentley: I could add a getFullURL or getURLWithQuery method to LaunchpadBrowserRequestMixin, which would then sort out LaunchpadTestRequest at the same time as BasicLaunchpadRequest.
[16:02] <abentley> allenap: Or getURL(include_query=False).  Any of those would be nice.
[16:03] <allenap> abentley: Okay, I'll do that, shouldn't take long... <days later>...
[16:03] <abentley> allenap: -)
[16:23] <sinzui> jcsackett, do you have time to mumble?
[16:24] <jcsackett> sinzui: sure.
[17:03] <allenap> abentley-lunch: I've made that change to getURL. Would you be able to take another look at the mp? It's still reasonably short. The only problem is that I have to go now, but I can check back in ~2 hours.
[17:03] <abentley-lunch> allenap: sure.
[17:03] <allenap> Thanks!
[17:07] <abentley> allenap: is it actually true that the form is POSTed to a URL with GET parameters?
[17:24] <abentley> allenap: it is pretty unusual to post to a URL with a query string, although I'm not clear whether it violates the spec.
[17:26] <henninge> abentley: Hi! Can you please review my branch? https://code.launchpad.net/~henninge/launchpad/bug-823164-remove-translations-by/+merge/70961
[17:27] <abentley> henninge: sure.
[17:27] <henninge> abentley: cool, thanks
[17:41] <abentley> henninge: r=me
[17:44] <henninge> abentley: thank you! ;-)
[18:46] <lifeless> morning
[18:48] <nigelb> Good morning.
[19:13] <allenap> abentley: I don't think there is anything against POSTing to a URL with a query string. I think the earlier docstring was misleading: a query string is part of the URL, but we've become accustomed to thinking of them as GET parameters.
[19:15] <abentley> allenap: I guess that makes sense.
[19:16] <abentley> allenap: thanks for making the change.
[19:17] <allenap> abentley: It was a good idea.
[19:20] <sixstring> I'm digging into the canonical-identity-provider code. I know I'm going to need some help plugging my stuff into it. Is this the right IRC channel for asking for help with canonical-identity-provider?
[19:22] <sixstring> bots?
[19:28] <allenap> sixstring: I don't think many of us hack on the central SSO parts, but I can't find a better place for it. sinzui, do you know any more?
[19:31] <allenap> sixstring: Email https://launchpad.net/~stuartmetcalfe; if anyone will know where the best place to hang out for c-i-p is, it's him.
[19:33] <sixstring> allenap: Thanks. Will do.
[19:35] <allenap> abentley: I forgot to say, thanks for the (double) review.
[19:35] <abentley> allenap: no problem.
[19:39] <sinzui> sixstring, allenap SSO is 100% ISD. We did not write any of the code
[19:39] <sixstring> ISD?
[19:39] <sixstring> Sorry, I don't know that abbrev, sinzui.
[19:40] <sinzui> sixstring, https://launchpad.net/canonical-identity-provider The team that maintain the app
[19:41] <sixstring> OK, sinzui. That makes sense now.
[19:41]  * sixstring is still figuring out how Canonical works.
[19:55] <nigelb> sinzui: hey
[20:00] <sixstring> sinzui, allenap: stuart pointed me to #canonical-isd.
[20:04] <nigelb> sinzui: was my post unclear that sso is a separate service?
[20:05] <sinzui> nigelb, I did not see your post. I know very well it is a separate service
[20:05] <nigelb> hm, wonder who commented then.
[20:05] <sinzui> I advocate turning of login.launchpad.net to stop confusing people about what Lp knows about users
[20:06] <nigelb> well, except there are other comlications.
[20:06] <nigelb> login.u.c is more stricter in config than login.l.c
[20:06] <nigelb> and there's some slight schema change
[20:06] <nigelb> but, yeah. Eventually, we'll all switch.
[20:07] <bdmurray> gary_poster: do you have any plans to follow up with the reporter of bug 823367?
[20:07] <_mup_> Bug #823367: Natty installation failed <installer-crash> <natty> <ubiquity-2.6.10> <ubiquity (Ubuntu):New> < https://launchpad.net/bugs/823367 >
[20:12] <gary_poster> bdmurray, I did not, but I can if desired.  I think I can get her email again
[20:14] <bdmurray> gary_poster: well its a common error if installing behind a proxy server with a specific behavior so working around it may be complicated for her.  I'll just set to invalid rather than finding the master bug.
[20:14] <gary_poster> bdmurray, ok.  If you had a page that might help her, I'd certainly be happy to pass it along.
[20:17] <bdmurray> gary_poster: I don't think we have a really good workaround documented
[20:17] <gary_poster> fair enough bdmurray.  OK thanks for heads up.
[21:09] <LPCIBot> Yippie, build fixed!
[21:09] <LPCIBot> Project db-devel build #798: FIXED in 5 hr 43 min: https://lpci.wedontsleep.org/job/db-devel/798/
[22:54] <wgrant> nigelb: There are no schema differences (they're the same DB), and I believe both are equally unstrict now.
[22:54] <nigelb> oh. WIN.
[22:57] <wgrant> lifeless: Bug #820796 didn't make it to coco, did it?
[22:57] <_mup_> Bug #820796: Lock before publishing <derivation> <qa-ok> <Launchpad itself:Fix Released by jtv> < https://launchpad.net/bugs/820796 >
[22:58] <lifeless> wgrant: blah
[22:58] <lifeless> deploy request is still up for it
[22:58] <wgrant> The logical response is to create a new Fix Release Requested status.'
[22:59] <lifeless> wgrant: !cite
[22:59] <wgrant> :(
[22:59] <lifeless> or we could have different code bases for different services
[23:00] <lifeless> I know thats heretical and all
[23:00] <wgrant> You are a horrible person.
[23:03] <lifeless> I'm shocked!
[23:03] <lifeless> here, let me delete more code from LP for us.
[23:03] <wgrant> You used ... though.
[23:04] <wgrant> That increases your evil rank significantly.
[23:04] <lifeless> where? oh in the bug ?
[23:05] <wgrant> from ...oops import
[23:06] <lifeless> oh yeah
[23:07] <lifeless> well, thats cause of absolute imports needing __future__
[23:07] <wgrant> Yeah.
[23:07] <wgrant> Not sure why implicit relative imports ever existed.
[23:08] <lifeless> python1
[23:13] <wgrant> Before my time.
[23:14] <lifeless> yeah
[23:14] <lifeless> python started without packages :)
[23:16] <sinzui> wallyworld, http://pastebin.ubuntu.com/663028/
[23:17] <wallyworld> sinzui: awesome. thank you!
[23:17] <sinzui> sorry about the surprise in there
[23:17] <wallyworld> yea, np :-(
[23:24] <lifeless> wgrant: I thought bug 824227 wasn't needed with the rendering improvements ?
[23:24] <_mup_> Bug #824227: Prevent changing bugtask pillars to/from a distribution with series tasks <Launchpad itself:In Progress by wgrant> < https://launchpad.net/bugs/824227 >
[23:29] <wgrant> lifeless: For project it's fine... for distributions it renders fine, but everything past that is more complicated.
[23:29] <wgrant> lifeless: If you approve a new distroseries nomination, all distribution/distributionsourcepackage tasks get a new task.
[23:30] <wgrant> If you then create a new distribution/distributionsourcepackage task, it also creates a task for all approved nominations.
[23:30] <wgrant> This gets amusing if you retarget to/from a DSP task, as you'll have partial sets of tasks which are unable to be fixed.
[23:30] <lifeless> those seem like bugs
[23:30] <wgrant> It requires entirely reworking how distroseries nominations work.
[23:31] <lifeless> which reminds me
[23:31] <lifeless> I was going to mail -users and u-devel about that
[23:31] <lifeless> I'd like to drop the table
[23:31] <wgrant> Oh?
[23:31] <lifeless> replace with a 'nominated' status
[23:32] <wgrant> Yeah, probably.
[23:32] <lifeless> use 'invalid' for rejected; the separate issue with making invalid be hidden would take care of hiding rejected ones
[23:32] <wgrant> Particularly since nominations are restricted now, they're pretty pointless.
[23:32] <lifeless> well, they have the same scope as tasks
[23:32] <lifeless> (once all the known bugs are fixed I mean)
[23:33] <lifeless> I also need to write up a perf tuesday this week; I'm thinking concurrency/taskswitching modelling.
[23:33] <wgrant> Oh?
[23:34] <lifeless> which line are you ohing?
[23:34] <wgrant> Where does concurrency/taskswitching come in?
[23:35] <lifeless> couple of places
[23:35] <lifeless> some other canonical projects are having significant perf issues, and running hundreds of concurrent queries on single pg instances
[23:36] <lifeless> not deliberately - starts with thousands of clients, mostly idle, and then stats kicks in when a spike comes through the system
[23:37] <wgrant> Erm.
[23:37] <wgrant> https://code.launchpad.net/~gary/launchpad/bug724025/+merge/71076
[23:38] <wgrant> Doesn't that make large bugs just about uneditable?
[23:38] <wgrant> Rather than simply non-AJAX.
[23:39] <lifeless> heh, yes. So perhaps not good enough