[00:16] <mars> mbarnett, any word on the buildbot stuff?  I'm way past EOD here, but I don't want to leave the whole system offline like this
[00:18] <lifeless> perhaps spm has the torch now
[00:19]  * spm hates at buildbot, reiterates plea from the heart and declares success in getting the PoS back
[00:19] <mars> thanks spm
[00:20] <spm> mars: np; it will need any 'was happening' builds to be stabbed tho. requerued? whatever it's called. forced?
[00:21] <mars> spm, could you please restart the lucid_lp and lucid_db_lp build slaves too?  I don't want to have to return to this tonight
[00:21] <mars> after that I will force everything to rebuild
[00:22] <spm> mars: you can do all of that yourself
[00:22] <spm> the lucid slaves have to be killed as part of the recovery
[00:22] <mars> oh, good
[00:23] <spm> basically you *must* shutdown everything BB related. kill/stab/burn with fire. then bring up the lucid slaves; then the master. repeat on restarting the master till the *(%(%^*&%^&*$%&*$%ing thing works.
[00:23] <mars> spm, so your procedures now include the steps to preven process trash from half-done builds lying around?
[00:23] <mars> ok
[00:23] <mars> so 'yes'
[00:23] <spm> mars: that's the burn with fire part :-)
[00:23] <mars> \o/
[00:23] <lifeless> sinzui: still around?
[00:24] <lifeless> sinzui: how does class="listing sortable" interact with batching?
[00:24] <mars> all builds forced
[00:25] <mwhudson> lifeless: poorly, because sortable just sorts the current batch
[00:25] <mars> spm, lifeless, if the prod_lp slave dies again, we'll have to do a full restart and force all the builders again.  It's the only way.
[00:25] <lifeless> mwhudson: is it tolerable? (for +assignments)
[00:25] <lifeless> mars: thanks
[00:26] <mars> aw FL*(#
[00:26] <spm> mars: orsum
[00:26] <mwhudson> lifeless: no idea really, it's not great
[00:27] <mars> lifeless, spm, so, about that "if prod_lp" dies thing - surprise!  Guess what just happened...
[00:27] <lifeless> mars: raarh
[00:27] <mwhudson> substantiate failed!
[00:27] <mwhudson> quickly too
[00:27] <mars> ugh, all the EC2 builders died
[00:27] <spm> mars: I'll need a few more clues? ;-)
[00:27] <lifeless> hammer
[00:27] <lifeless> tongs
[00:28] <mwhudson> maybe aws is having a bad day
[00:28] <lifeless> apply
[00:28] <mars> spm https://lpbuildbot.canonical.com/waterfall
[00:28] <spm> mars try a second force - I did terminate the (old) ec2 instances that were running. so the master may have belatedly got poor signals?
[00:30] <mars> need to go afk, crying babes in arms, back as soon as possible
[00:33] <mbarnett> mars: sorry for dropping you there
[00:33]  * spm forces a build on prod_lp
[00:42] <wgrant> After the last rollout, we have a few users who need their multiple accounts merged.
[00:42] <wgrant> Except they now only have access to the new account -- the one they don't want to keep.
[00:42] <wgrant> Should I ask them to file a question so an admin can merge them the right way?
[00:43] <poolie> jam: 'from twisted.internet import defer; defer.setDebugging(True)'
[00:46] <lifeless> mwhudson: are there tests of specification template rendering ?
[00:49] <mwhudson> lifeless: mostly only in the page tests
[00:49] <mwhudson> akaik
[00:49] <lifeless> mwhudson: where would they be? the .txt files i found had no browser_ stuff
[00:50] <lifeless> mwhudson: skip that, time I learnt something new
[00:50] <lifeless> mwhudson: whats the preferred way to render a template
[00:50] <lifeless> I want to check that it gets batched in unittest style code.
[00:50] <mwhudson> lifeless: using testbrowser is probably easiest
[00:51] <lifeless> self.getUserBrowser() ?
[00:51] <mwhudson> otherwise, create_initialized_view() and call .render() i think?
[00:51] <mwhudson> yeah
[00:51] <lifeless> ah, .render is probably cleaner
[00:51] <lifeless> less layers for what I'm interested in.
[00:54] <mwhudson> yeah, probably
[00:54] <lifeless> ahhh stories/standalone is where stuff was.
[00:54] <lifeless> nvm I'm deep into doing it right :)
[01:35] <wgrant> spm: Do you have a moment to do a quick account merge? https://answers.edge.launchpad.net/launchpad/+question/125444 -- he's been unable to log in since the rollout last week.,
[01:35] <spm> not really, but will do.
[01:35] <wgrant> Heh, thanks.
[01:36] <spm> ahh. thanks! you've done the research. much appreciated!
[01:37] <spm> is done
[01:37] <wgrant> Yeah, that's why it was really quick.
[01:37] <wgrant> Thanks.
[01:49] <wgrant> abentley: Why do we not permit arbitrary code execution in the tree build step?
[01:50] <wgrant> Some seem to think that it is because of security.
[01:50] <wgrant> But that cannot be the case.
[01:58] <poolie> rockstar: hi?
[02:09] <lifeless> wgrant: why can't it be the case?
[02:10] <wgrant> lifeless: Because the chroot doesn't provide security.
[02:10] <wgrant> So doing something outside the chroot can't be a security vulnerability.
[02:17] <lifeless>  Hard / Soft  Page ID
[02:17] <lifeless>      229 / 1575  ScopedCollection:CollectionResource:#message-page-resource
[02:17] <lifeless>       91 /  255  BugTask:+index
[02:17] <lifeless>       67 /   34  DistributionSourcePackage:+filebug
[02:17] <lifeless>       52 /    7  Distribution:+search
[02:17] <lifeless>       48 /  231  CodeImportSchedulerApplication:CodeImportSchedulerAPI
[02:19] <wgrant> What is message-page-resource?
[02:19] <wgrant> I haven't seen that before.
[02:20] <lifeless> I'm looking ATM
[02:20] <lifeless> https://api.launchpad.net/1.0/bugs/136469/messages
[02:20] <_mup_> Bug #136469: toshiba p100 series dsdt acpi error no sound, works with acpi turned off. <linux (Ubuntu):Fix Released by ubuntu-audio> <linux-source-2.6.22 (Ubuntu):Won't Fix by ubuntu-audio> <https://launchpad.net/bugs/136469>
[02:21] <lifeless> ws.start=75&ws.size=75
[02:21] <lifeless> SQL time: 4344 ms
[02:21] <lifeless> Non-sql time: 10882 ms
[02:21] <lifeless> Total time: 15226 ms
[02:21] <lifeless> Statement Count: 288
[02:21] <wgrant> Somehow I doubt it.
[02:21] <lifeless> ?
[02:22] <rockstar> poolie, hi.
[02:22] <wgrant> That much non-SQL time for so few objects?
[02:22] <poolie> hi rockstar, i was just going to ask if you could pair with jam on getting his bzr-lp forking server finished
[02:23] <lifeless> wgrant: welcome to bad-O, or thread-thrashing, or a few things
[02:23] <poolie> and on reviewing what he's done so far
[02:23] <wgrant> lifeless: I meant that it was probably not bad code, but thread-thrashing, yes.
[02:23] <lifeless> wgrant: its -so- frequent its not going to be a passive victim of worse pages
[02:23] <poolie> i think you may have been there for some discussion of it at the epic
[02:23] <rockstar> poolie, I might not be the best guy to pair with him.  I'm mostly a frontend guy recently.
[02:23] <rockstar> poolie, in fact, I don't know much about the forking stuff at all.
[02:23] <jam> rockstar: the big advantage is that you are in my timezone :)
[02:23] <rockstar> poolie, but if you think I might be of help to him, I'm happy to do it.
[02:23] <rockstar> jam, :)
[02:24] <rockstar> jam, abentley might be better.  He's paired with jml on parts of that code at least.
[02:24] <jam> rockstar: I've got a lot of small questions
[02:24] <poolie> abentley could also be great
[02:24] <rockstar> jam, okay.  Shoot.  I might be able to answer a few of them at least.
[02:26] <lifeless> who knows about how deferred mail works (for bugs, mp's etc etc)
[02:26] <lifeless> questions need this done to them pretty soon
[02:26] <lifeless> but I don't want to do it arbitrarily differently.
[02:27] <wgrant> lifeless: Haven't we established that we really don't want to follow the Bugs method?
[02:27] <lifeless> wgrant: policy of naming, no. Method of deferring - I haven't considered.
[02:27] <lifeless> wgrant: tell me about it.
[02:27] <wgrant> Bugs and Code do it differently.
[02:27] <lifeless> great. Please be informing your humble TA
[02:27]  * lifeless grovels and drools a little
[02:28] <wgrant> Bugs changes create rows in BugNotification, with a BugNotificationRecipient for each (direct or indirect) subscriber.
[02:28] <wgrant> send-bug-notifications.py then processes those every 5 minutes or so, I believe.
[02:29] <wgrant> MPs only started deferred sending recently... and I think they use BranchMergeProposalJob for that, but I haven't looked closely yet.
[02:34] <jam> rockstar: I'm mostly watching a movie with my son right now, but I'll chat with you a bit tomorrow if you're around
[02:34] <rockstar> jam, okay.  Maybe abentley and I can combine our powers.
[03:08] <lifeless> mtaylor: oh hai
[03:11] <lifeless> spm: I can has profile?
[03:11] <spm> damn. just 5 secs too slow on heading too lunch ;-)
[03:11] <lifeless> rotfl
[03:11] <lifeless> har
[03:12] <lifeless> yes? no? maybe?
[03:12] <spm> oh, the yes was iomplied, just logging in
[03:13] <lifeless> if you like, once its enabled, I'll keep poking at it while you go to lunch
[03:13] <spm> sure
[03:13] <spm> is restarting....
[03:13] <lifeless> wgrant: https://api.staging.launchpad.net/1.0/bugs/1/messages?ws.start=150&ws.size=75 - da boom -
[03:14] <_mup_> Bug #1: Microsoft has a majority market share <iso-testing> <ubuntu> <Clubdistro:Confirmed> <Computer Science Ubuntu:Invalid by compscibuntu-bugs> <EasyPeasy Overview:Invalid by ramvi> <GNOME Screensaver:Won't Fix> <Ichthux:Invalid by raphink> <JAK LINUX:Invalid> <The Linux OS Project:In Progress> <OpenOffice:In Progress by lh-maviya> <Tabuntu:Invalid by tinarussell> <Tivion:Invalid by shakaran> <Tv-Player:New> <Ubuntu:In Progress by sabdfl> <
[03:15] <lifeless> spm: whats staging load like ?
[03:15] <lifeless> (can you kick it low before going)
[03:16] <spm> lifeless: 4ish, kickin'
[03:17] <spm> lifeless: and dropping. should be good to go
[03:17] <lifeless> thank thee kindly
[03:18] <lifeless> grah
[03:18] <lifeless> no
[03:18] <lifeless> ++profile doesn't work on the API
[03:18] <lifeless> bug filing time
[03:18] <lifeless> spm: turn it back off thanks, and shoo fo rlunch
[03:18] <lifeless> spm: (when you get back is fine, no rush from my POV)
[03:20] <spm> lifeless: is done
[03:23] <lifeless> wgrant: https://bugs.edge.launchpad.net/malone/+bug/497386 for your skepticsm
[03:23] <_mup_> Bug #497386: ScopedCollection:CollectionResource:#message-page-resource timeouts for bugs with many messages <api> <timeout> <Launchpad Bugs:Triaged> <https://launchpad.net/bugs/497386>
[03:25] <wgrant> lifeless: Am I missing something, or can (1) be done trivially with a DRS?
[03:26] <lifeless> wgrant: with a rank() function probably, as long as someone unwinds the login in indexed_messages appropriately
[03:26] <lifeless> wgrant: (thats yes)
[03:27] <lifeless> possibly better to just store in the table though
[03:34] <lifeless> wgrant: more data in it now
[03:37] <lifeless> hah
[03:37] <lifeless> indexed_messages is whats exported
[03:37] <lifeless> I bet its calculating one of the links in it that is the *actual* bottleneck today
[03:44] <lifeless> hmm, I wonder how to trigger jsonification of an object for a test
[03:44] <lifeless> wgrant: have you perchance poked at that ?
[03:58] <wgrant> lifeless: No, sorry.
[04:05] <james_w> lifeless: jsonified by the lazr.restful machinery?
[04:05] <james_w> There are methods to make webservice requests, but I've never seen a test that jsonifies something in Launchpad
[04:05] <james_w> looking at the lazr.restful doctests might give a hint
[04:07] <lifeless> james_w: yeah
[04:07] <lifeless> wondering how close to the issue I can get my test was all
[04:14] <james_w> lifeless: representation = simplejson.dumps(self, cls=ResourceJSONEncoder)
[04:14] <james_w> something like that might be close, but isn't everything the lazr.restful does
[04:14] <lifeless> yeah
[04:15] <lifeless> I'm just going to test the larger case - the key one - adding messages shouldn't increase queries.
[04:15] <lifeless> once I get out of this ubuntu-bugs quagmire with our latest enthusiastic spammer
[04:15] <james_w> otherwise you need to get an EntryResource for the object, and call _representation on it apparently
[04:16] <james_w> and you need a request to do that
[04:17] <james_w> yeah, the simplejson.dumps should be close, but bypasses all the lazr.restful machinery, so it depends where you want to test
[04:17] <james_w> well, not all, but lots
[04:21] <lifeless> need the machinery :P
[04:22] <james_w> lifeless: you need the cache and things, not just the serialisation?
[04:24] <lifeless> I need to reproduce what the appserver does
[04:24] <lifeless> otherwise the test isn't complete.
[04:24] <lifeless> anyhow, I'm good.
[04:29] <james_w> well, you don't run the test via apache :-P
[04:30] <StevenK> I've added a new model to soyuz in db-devel, and now that I'm getting around to using it, the security proxy is hiding all of it. How can I check/debug/fix that?
[04:32] <lifeless> james_w: yes, but thats outside the scope that can trigger more db calls
[04:32] <lifeless> StevenK: interfaces
[04:32] <StevenK> lifeless: I added that too
[04:33] <lifeless> and enabled it via zcml ?
[04:33] <StevenK> lifeless: I certainly added it to the zcml, I'm not sure if it was right.
[04:33] <StevenK> lifeless: I can dig up the MP if you want to take a squiz
[04:36] <lifeless> StevenK: perhaps you should describe the symptoms more ;)
[04:36] <lifeless> like 'in a test xxytyzz'
[04:38] <StevenK> lifeless: I've added a new method to lp.registry.interfaces.IDistroSeries called deriveDistroSeries(), and in the model code it's calling job = getUtility(IInitialiseDistroSeriesJobSource).create(child), and whenever I try and access any attribute on job, I get denied.
[04:39] <lifeless> is job the right type?
[04:40] <StevenK> If I remove the security proxy to check, yes
[04:41] <lifeless> check the security you've permitted for the attributes is appropriate
[04:41] <lifeless> also getUtility(IInitialiseDistroSeriesJobSource).create(child) seems awfully ugly
[04:44] <StevenK> lifeless: How do I check the security? My zcml-foo is fairly non-existant
[05:19] <poolie> oh lifeless i meant to say: what a nice tuesday mail
[05:19] <poolie> here, have a peanut :)
[05:19] <poolie> (in the sense of gallery, not monkey)
[05:19] <StevenK> Bwaha
[05:23] <StevenK> Is there any easy way to introspect the name of a variable?
[05:27] <poolie> look at it
[05:27] <poolie> where are you starting from?
[05:27] <poolie> StevenK: if you just have an object reference in python you can't directly tell what variable holds it
[05:27] <poolie> however, you can look at locals() and globals() and walk through the stack
[05:29] <lifeless> poolie: thanks
[05:29] <StevenK> poolie: I'm looping over a list of variables and croaking when they're not set, I was wondering if I could get at the variable name easily for a nice error message
[05:29] <poolie> pastebin the code?
[05:29] <lifeless> StevenK: paste the code
[05:29] <lifeless> lol
[05:29] <poolie> :)
[05:29] <poolie> pics or gtfo
[05:30] <lifeless> poolie: was there anything particular about the tuesday may that made it nice?
[05:30] <poolie> i meant actually the mail about it
[05:31] <poolie> and only indirectly the content
[05:31] <poolie> but it sounds like it was fun
[05:31] <poolie> i liked the "today i learned about $x"
[05:31] <poolie> it suggests other people could possibly learn about things too :)
[05:32] <StevenK> http://paste.ubuntu.com/493988/
[05:32] <poolie> oh it's going to say "None can't be unset"? :)
[05:33] <poolie> so you could do 'if not locals().get(param_name)' and list the things as quoted strings
[05:33] <poolie> i don't know if that's exactly idiomatic
[05:33] <poolie> alternatively just 'assert displayname;assert summary;...'
[05:33] <StevenK> poolie: Exactly
[05:34] <poolie> which will give a clear but less pretty error
[05:34] <poolie> (and it wouldn't be allowed in bzr code, which bans the 'assert' statement because it can be turned off)
[05:34] <lifeless> so
[05:34] <lifeless> I'd define a tested helper in lp.services
[05:34] <poolie> also i would do raise DerivationError(....
[05:34] <poolie> not the old syntax
[05:34] <lifeless> and do
[05:35] <lifeless> something useful with it :)
[05:35] <lifeless> the args are available on the stack too ;)
[05:36] <poolie> one could have a decorator
[05:36] <lifeless> I think we may already have one
[05:36] <poolie> @require_nonnull_parameters(['breakfast', 'lunch'])
[05:36] <lifeless> the apis stuff has some declaritive foo here
[05:38] <StevenK> Yes, but the arguments are only required in one circumstance
[05:41] <lifeless> which circumstance is that?
[05:42] <spiv> argvalues = inspect.getargvalues(inspect.currentframe()); for arg in argvalues.args: if argvalues.locals.get(arg) is None: raise ...  # ;)
[05:43] <lifeless> jtv: hey, so hows the template page in edge ?
[05:44] <jtv> lifeless: works fine
[05:44] <lifeless> time to render?
[05:44] <jtv> found it surprisingly fast on staging earlier
[05:44] <StevenK> spiv: Hold on, I'm dabbing the blood from my eyes :-)
[05:45] <jtv> lifeless: pretty decent; still slow but probably from a part of the process that we don't watch as closely such as transfer to the client
[05:46] <jtv> Haven't seen any timeouts at all on edge; I just managed to trigger one on staging though.
[05:46] <jtv> I'm guessing on staging it depends a lot on load.
[05:47] <spiv> StevenK: oh, if you want blood then I'm sure I can find uglier ways to do that... sys._getframe().f_code.co_* springs to mind.
[05:47] <jtv> Unfortunately chromium's "view source" breaks a lot on this page
[05:47] <jtv> spiv: doesn't fit as neatly into the meter as "you got it" though
[05:48] <StevenK> spiv: Bah :-)
[05:48] <jtv> If You Want Blood… sys._getgrame().f_code.co_*
[05:48] <jtv> lifeless: just got a render time of 7 seconds.  There's other things the new code will let us do if it's too much.
[05:51] <jtv> lifeless: in its current basic structure we could bring the critical loop body down to a single "%" operation.
[05:52] <lifeless> jtv: jtv check the render comments to evaluate time, for now :)
[05:52] <lifeless> once we're down to a second of render time we can worry about transfer time
[05:52] <lifeless> jtv: oh, I see chrome fail
[05:55] <jtv> lifeless: just got one rendered in 2.33 seconds, so I think rendering time is close to losing its position as the dominant time sink.
[05:56] <jtv> Funny: looks as if firefox is actually faster than chromium for this page
[06:03] <lifeless> jtv: thats great
[06:06] <jtv> A bit disappointing to see so much variability.  It may be because we've become entirely bottlenecked on appserver CPU.
[06:07] <jtv> At least, I think we have; I'll know more when my oops shows up.
[06:07] <jtv> But as this becomes the norm for "performance-interesting" pages, we may have to look more critically at the GIL.
[06:08] <lifeless> I've filed a bug and RT to experiment on this
[06:09] <jtv> Ah, there's my informational oops for +templates: 3885ms, of which 206ms are spent blocked.
[06:09] <lifeless> how many rows?
[06:10] <jtv> Just over 2K.
[06:11] <jtv> All the queries are at the head and the bottom, with a big chunk of rendering time for the table in the middle.
[06:12] <jtv> The Big Query takes 86ms.
[06:36] <lifeless> so 2K - probably 666ms in storm
[06:36] <lifeless> on an idle machine.
[06:36] <lifeless> more or less depending on row width
[06:36] <StevenK> lifeless: O hai, did you miss my question?
[06:57] <lifeless> question?
[06:57] <StevenK> < StevenK> lifeless: How do I check the security? My zcml-foo is fairly non-existant
[06:58] <lifeless> about the same as mine I suspect
[06:59] <StevenK> lifeless: Bugger :-)
[07:12] <lifeless> stub: how long till 8.4 ?
[07:12] <stub> The packages are on sourcherry so I want to do staging today.
[07:13] <lifeless> awesome
[07:13] <lifeless> I want to use rank() to get message indices
[07:13] <lifeless> should shave a tonne of time off of bug 1.
[07:13] <_mup_> Bug #1: Microsoft has a majority market share <iso-testing> <ubuntu> <Clubdistro:Confirmed> <Computer Science Ubuntu:Invalid by compscibuntu-bugs> <EasyPeasy Overview:Invalid by ramvi> <GNOME Screensaver:Won't Fix> <Ichthux:Invalid by raphink> <JAK LINUX:Invalid> <The Linux OS Project:In Progress> <OpenOffice:In Progress by lh-maviya> <Tabuntu:Invalid by tinarussell> <Tivion:Invalid by shakaran> <Tv-Player:New> <Ubuntu:In Progress by sabdfl> <
[07:13] <stub> lifeless: tsearch2 rank()? Should be in 8.3.
[07:15] <lifeless> stub: window function rank()
[07:15] <lifeless> rank() OVER (..)
[07:15] <lifeless> spm: hi
[07:15] <spm> yo
[07:15] <lifeless> spm: production rollout - buildbot borkage.
[07:15] <lifeless> https://lpbuildbot.canonical.com/builders/prod_lp/builds/98
[07:16] <lifeless> it grabbed rev 9762
[07:16] <lifeless> I see 9765 as tip of production-devel.
[07:16] <lifeless> we want ot remove the cowboy and get stuff out there. Dunno whats going on.
[07:16] <lifeless> However, I has to run - family time.
[07:16] <spm> beyond bb being a pos as in?
[07:17] <lifeless> yes exactly
[07:17] <lifeless> why isn't it getting the tip of production-devel
[07:17] <lifeless> https://lpbuildbot.canonical.com/builders/prod_lp/builds/98/steps/bzr/logs/stdio
[07:17] <lifeless>  argv: ['/usr/bin/bzr', 'checkout', '--revision', '9762', 'bzr+ssh://bazaar.launchpad.net/~launchpad-pqm/launchpad/production-devel', 'build']
[07:17] <lifeless> so its explicitly asking for an old rev
[07:17] <spm> that... is seriously weird
[07:17] <lifeless> which is the nuts.
[07:18] <lifeless> spm: can you take it on, and hand it off to someone else when you EOD?
[07:18] <lifeless> we need a build of 9765 to do all three CPs
[07:18] <spm> probably not. Tom's not around. I'll do what I can, but no promises.
[07:18] <lifeless> and remove the thing, cowboy
[07:18] <lifeless> I'll check in in ~ 4 hours I guess.
[07:18] <spm> Oh I think I may know what did that.
[07:19] <stub> So does everything explode if people push --overwrite a branch that is linked to builds etc.?
[07:19] <spm> I used the 'restart with last failed build' juju in an attempt to encourge the pos to work. that'd do it.
[07:20] <spm> builds forced, see if that grabs the latest
[07:27] <lifeless> bbiab
[08:47] <adeuring> good morning
[08:49] <mrevell> Hey up everyone.
[10:34] <jml> hello all
[10:35] <lifeless> hello
[10:37] <jml> lifeless, you broke prod_lp again :(
[10:38] <lifeless> jml: no, buildbot failed
[10:38] <lifeless> and failed again
[10:38] <lifeless> and failed afte that
[10:38] <jml> :(
[10:38] <lifeless> and after that
[10:38] <lifeless> so the changes, in production-devel, are still in production-devel and not in production-stable
[10:41] <jamesh> you'd get less buildbot failures if you had less tests
[10:42] <lifeless> jamesh: thanks for that.
[11:11] <lifeless> night all
[11:17] <bigjools> lifeless: wait!
[11:17] <bigjools> darn too late probably.  jml, prod_devel is failing still
[11:17] <bigjools> can you help please?
[11:17] <jml> bigjools, I can try. On the phone.
[11:18] <bigjools> me too :)
[11:18] <lifeless> /usr/bin/bzr checkout --revision 9762 bzr+ssh://bazaar.launchpad.net/~launchpad-pqm/launchpad/production-de
[11:18] <lifeless> its grabbing the wrong rev
[11:18] <lifeless> (you're lucky, I am looking up the phone number for tomorows meeting)
[11:19] <lifeless> the branch is fine
[11:19] <lifeless> BB is confused
[11:19] <lifeless> dunno why
[11:19] <lifeless> I see rev 9765 in prod-devel
[11:19] <lifeless> ok, water, phone, phone # all set.
[11:19] <lifeless> _> flip side
[11:23] <jml> I'v queued another build
[11:51] <bigjools> jml: what do you mean?
[11:51] <jml> bigjools, I went to "Force build" and forced a build.
[11:52] <jml> building
[11:52] <jml> 1 pending
[11:52] <bigjools> jml: ok - do you know if that's any different to what happened at 07:43?
[11:52] <bigjools> the overnight build failed too, and then someone kicked it off at that time
[11:52] <bigjools> I don't have much hope that it'll work this time :/
[11:52] <jml> bigjools, no, I don't. but I'm going to try it in lieu of having a better idea.
[11:53] <bigjools> ok
[11:53] <bigjools> I've got a rather urgent CP to get out :(
[12:04] <jml> bigjools, ok. I'm off the phone. You have my full attention.
[12:04] <bigjools> jml: not much we can do until the current run finishes
[12:04] <bigjools> we can see if the next one checks out the right revno
[12:05] <bigjools> if not, then I'm getting on the batphone
[12:05] <jml> I wonder where it gets the revno from
[12:06] <bigjools> why does it even need to know
[12:07] <jml> bigjools, I'm guessing something about race conditions.
[12:07] <jml> bigjools, or perhaps some secondary approval step from the losas?
[12:09] <jml> good grief. every thing is failing for different reasons.
[12:09] <bigjools> ew, it takes 45 minutes from buildbot instantiation until it actually starts running tests
[12:10] <jml> yes.
[12:15] <deryck> Morning, all.
[12:15] <bigjools> hi deryck
[12:28] <bigjools> jml: it got the right revno this time
[12:28] <bigjools> only 4 hours to wait now... :/
[13:20] <jml> bigjools, yeah I see it :)
[13:23] <wgrant> Launchpad's misrepresentation of partner makes me sad :(
[13:23] <wgrant> And makes RMS FUD harder to debate.
[13:25] <jml> wgrant, which aspect in particular
[13:25] <wgrant> jml: It says that Skype and Flash are in Ubuntu.
[13:25] <wgrant> Which makes it difficult to tell RMS that he's wrong when he says they are :P
[13:26] <beuno> oh, everything is difficult with RMS anyway...   :)
[13:26] <wgrant> It is...
[13:28] <jml> wgrant, I can see why that would be frustrating.
[13:29] <jml> wgrant, otoh, I'm increasingly of the opinion that we should focus our efforts toward helping the people who are using Launchpad to try to get things done.
[13:29] <wgrant> Probably, yes.
[14:25]  * gmb -> breaking to escape brain-stall. 
[14:36] <deryck> gmb, when you're back, could you look at bug 638284?
[14:36] <_mup_> Bug #638284: Launchpad's Remote Tracker does not honor Debian Bug Tracker status changes <Launchpad Bugs:New> <https://launchpad.net/bugs/638284>
[14:37] <deryck> I believe this is the debbugs sync disabled due to disk space issue, but would like you to confirm.
[14:57] <gmb> deryck, You're right.
[14:58] <gmb> deryck, Perhaps we should use that bug as a way of keeping users up-to-date with the problem. Thoughts?
[14:58] <deryck> gmb, yes, definitely.  If we don't already have a bug about it, then let's use that one.
[14:58] <gmb> deryck, I just looked and don't see one. I think that's mostly because we've been fielding Questions, IRC pings and emails about it but no bugs until now.
[14:59] <gmb> And I know that the LOSAs have an RT for the problem.
[15:00] <deryck> gmb, ok, cool.  Do you mind updating the bug and doing this?
[15:01] <gmb> deryck, Done.
[15:01] <bac> abentley, adeuring, bac, danilo, sinzui, deryck, EdwinGrubbs, flacoste, gary, gmb, henninge, jelmer, jtv, bigjools, leonard, mars, noodles775: reviewer meeting starting
[15:01] <deryck> awesome!
[15:06] <jml> sinzui, where's the list of bugs remaining for the bridging-the-gap work that registry is finishing off?
[15:07] <sinzui> https://bugs.edge.launchpad.net/launchpad-registry/+bugs?field.tag=bridging-the-gap
[15:10] <jml> sinzui, cunning. thanks.
[15:11] <jml> sinzui, is there a bug filed for the bad tooltip text for the branch configuration link?
[15:11] <sinzui> yes, your incomplete one
[15:12] <sinzui> jml, I cannot reproduce it. The description of what is in play (rendering engine + os) make it unlikely to be an issue with our code
[15:12] <jml> sinzui, I don't mean the corruption thing, I mean the fact that the text says "Set branch for this series"
[15:13] <sinzui> oh, sorry
[15:13] <sinzui> let me look.
[15:14]  * jml is sorely tempted to move LEP mgmt to blueprints.
[15:15] <bigjools> I think that's a good idea
[15:16] <bigjools> it would give a needed thrust to improving blueprints
[15:16] <sinzui> jml, I tried that earlier this year, there is still a lot of blueprint update by hand because the statuses are more than any project needs and notifications are broken
[15:16] <sinzui> You cannot use feedback requests for example...you still need to send the feedback request yourself
[15:17] <deryck> jml, and then we should merge blueprints and bugs. ;)
[15:17] <jml> deryck, *yes*
[15:17] <sinzui> jml: I do not see a bug for the tooltip. The bug is in an Involvement Menu subclass and is trivial
[15:17] <sinzui> deryck, and answers.
[15:17] <jml> sinzui, ok. I'll file it anyway, if you don't mind.
[15:18] <deryck> sinzui, woah there.  Not sure I'm feeling that one. ;) :)
[15:18] <sinzui> please do, my team are asking for some easy work since the app UI is very hard
[15:18] <sinzui> deryck, sure you do, less timeouts converting a bug to question.
[15:18] <deryck> heh
[15:19] <deryck> well that is persuasive
[15:19] <sinzui> No more bug-question links and more portlet to show it
[15:19] <deryck> I just think we want to work at moving the support requests out of bugs, not making it easier to misunderstand the two.
[15:20] <deryck> gmb or allenap, bug 5786 is no longer relevant, correct?  NEEDSINFO is not referring to upstream status, but to an old bugtask state I take it?
[15:20] <_mup_> Bug #5786: NeedsInfo should be moved into a separate bug or bugtask flag <Launchpad Bugs:Triaged> <https://launchpad.net/bugs/5786>
[15:21] <gmb> deryck, I think so, yes. In which case it was superseded by Incomplete aeons ago.
[15:22] <deryck> right
[15:28] <jml> sinzui, we talked about https://dev.launchpad.net/Registry/RegistryKarma a while back. What did we decide?
[15:28] <sinzui> We will do nothing but continue to hope for a replacement for karma
[15:29] <jml> sinzui, excellent. I'll make a note.
[15:30]  * sinzui has talked to 2 users this month to explain karma is not a measure of value or work
[15:32] <jml> sinzui, what about this one? https://dev.launchpad.net/Registry/EssentialSourcePackageInformation
[15:33] <sinzui> That is done!
[15:35] <jml> sinzui, ooh. I'll review then.
[15:36] <jml> sinzui, the LEP says that https://edge.launchpad.net/ubuntu/+source/wsjt should have a link to copyright information. I can't see a link like that.
[15:37] <sinzui> jml, that page should not, the series source package does...they change by series
[15:38] <sinzui> jml: the SP descriptions will be used in the derivative distros diff page. the copyright is being used in register an upstream project
[15:40] <jml> sinzui, sweet!
[15:41] <deryck> adeuring, hi.  I'm linking your current kanban board card against bug 633698.  This bug is good for this issue you're seeing?
[15:41] <_mup_> Bug #633698: missing API oops reports? <Launchpad Bugs:New> <https://launchpad.net/bugs/633698>
[15:42]  * sinzui wonders if it is appropriate to quote his mistypes: "The involvement
[15:42] <sinzui> portlet is controlled by the pillar owners and its links are enabled to
[15:42] <sinzui> attack contributors to the applications the pillar uses."
[15:42] <adeuring> deryck: sure
[15:42] <sinzui> I think I meant to type attract, but maybe subconsciously i did mean attack
[15:43] <jml> heh heh
[15:43] <deryck> A little more clearly described now:  bug 633698
[15:43] <_mup_> Bug #633698: Librarian errors missing API oops reports <librarian> <Launchpad Bugs:In Progress by adeuring> <https://launchpad.net/bugs/633698>
[15:45] <deryck> It seems people are either ignoring results suggested by dupe finder or it's become less useful with the recent preformance work.
[15:54] <jml> bigjools, are package dependencies available over the API?
[15:55] <bigjools> jml: do you mean as properties of source packages?
[15:55] <jml> bigjools, Yes.
[15:55] <bigjools> jml: no, we don't export those
[15:55] <jml> bigjools, I mean, if they're available in another way that would be interesting too.
[15:55] <jml> bigjools, any reason or just round tuits?
[15:56] <bigjools> jml: we don't export source packages at all, just their publications.
[15:56] <jml> deryck, I think it has become less useful.
[15:56] <bigjools> deryck: I hardly ever see people paying attention to it
[15:57] <jml> deryck, but a third hypothesis is more bugs filed via email.
[15:57] <jml> bigjools, is there a reason for that?
[15:57] <bigjools> jml: yes, what is a source package's URL?
[15:57] <jml> bigjools, https://launchpad.net/ubuntu/+source/packagename
[15:57] <bigjools> jml: wrong! that's a distrosourcepackage
[15:58] <deryck> yeah, I can tell the difference in bugs filed via email and against the project group.  I'm seeing it more commonly filed straight against malone, too.
[15:58] <jml> bigjools, ubuntu/lucid/+source/packagename
[15:58] <sinzui> bigjools, jml, I want to toss a small grenade into your conversation. I do not think the DSP/SP pages and API is very good. Our pages have poor rank in Google. packages.ubuntu.com always wins. When a user is searching the Google package information, they are going to the other site, then following that link to Lp :(
[15:58] <jml> sinzui, interesting.
[15:58] <bigjools> jml: unfortunately wrong again.  We need a way to put a sourcepackagerelease in its own URL.  But I don't think that's useful at all though.
[15:59] <bigjools> bear in mind that dependencies can change between versions
[15:59] <jml> bigjools, something is using the word incorrectly here.
[15:59] <jml> bigjools, https://edge.launchpad.net/ubuntu/maverick/+source/wsjt perhaps
[15:59] <bigjools> this is why I added a bunch of source-related properties to publications
[15:59] <jml> bigjools, since it does say "source package"
[16:00] <bigjools> yes, but it's not a source package release
[16:00] <bigjools> it's a source package in ubuntu maverick
[16:00] <jml> oh, you didn't ask for a URL for that :)
[16:00]  * bigjools rolls eyes
[16:00] <sinzui> jml: bigjoolsI do not have any immediate ideas, I think this conversation is registry related too. SPR is a specific detail that users often do not care about, they want the current info for their distros or series. I think we need to re-ask who uses this information and what pages do we provide for them
[16:00] <bigjools> ok ok :)
[16:01] <jml> bigjools, my use case here is wanting to get build dependencies so I can start working on a branch.
[16:01] <bigjools> sinzui: I think probably both are valid, which is another reason why I prefer adding more properties to publishing records
[16:01] <bigjools> because traversing to those from the SPR or the DSP is trivial
[16:02] <jml> bigjools, so I need to be able to get from the branch's project to some kind of package record, and from there to the dependencies
[16:02] <jml> bigjools, is this currently possible over the API?
[16:02] <bigjools> jml: it's not :(
[16:02] <sinzui> I think users arrive at an SPR too early. Searching a series takes me to the SPR. If I bookmark that page, It could be wrong in 24 hours
[16:03] <bigjools> sinzui: we don't have a page for an SPR
[16:03] <jml> bigjools, what would we need to do to make it possible?
[16:03] <sinzui> bigjools, True
[16:03] <bigjools> jml: are you looking for a package in a distroseries in this project?
[16:03] <jml> (I don't really care about the web ui for this, fwiw, although I do acknowledge that it's interesting)
[16:03] <bigjools> or a particular version? or something else?
[16:03] <jml> bigjools, I guess I don't care & am looking for the latest in Ubuntu, since that's likeliest to be close to trunk
[16:04] <bigjools> ok
[16:04] <bigjools> jml: are you looking at build dependencies or installation dependencies?
[16:04] <jml> bigjools, both. but for this use case, let's focus on build.
[16:04] <bigjools> then I'd do this:
[16:04] <bigjools> add the build deps as a method on sourcepackagepublishinghistory
[16:05] <bigjools> then use distrosourcepackage.currentrelease or similar to get the publication
[16:05] <bigjools> but you need to export all that on the API
[16:06] <jml> bigjools, are DSP and SPPH already exported?
[16:06] <bigjools> actually, scratch that latter, use distribution.getCurrentSourceReleases()
[16:06] <bigjools> which takes a string package name
[16:06] <bigjools> SPPH is exported
[16:06] <bigjools> distro is exported
[16:07] <jml> OK, so it's just a simple matter of programming then.
[16:07] <bigjools> yep, hopefully
[16:09] <jml> bigjools, thanks
[16:09] <bigjools> jml: oh actually, feh, getCurrentSourceReleases() returns DSPs which are not exported.  You should use IArchive.getPublishedSources
[16:09] <bigjools> which returns SPPHs
[16:09] <jml> bigjools, ah. fair enough.
[16:10] <bigjools> getting the archive is trivial
[16:10] <bigjools> distro.main_archive for example
[16:10] <jml> bigjools, from time to time the thought occurs to me, perhaps soyuz's apis could be simpler.
[16:10] <bigjools> jml: hellyes.  however, we're constrained by the decision to expose the internal model
[16:11] <bigjools> but if you have any ideas on how to make it better, I am all ears
[16:11] <jml> by "api" I meant internal model, actually.
[16:11] <jml> no ideas right now :)
[16:11] <jml> I'll let you know if any come up :)
[16:11] <bigjools> jml: I get frustrated with the model too.
[16:11] <bigjools> do you remember my presentation at the original epic?
[16:11] <bigjools> and the db diagram?
[16:13] <jml> bigjools, yeah, I do.
[16:14] <bigjools> jml: recipe builds made that twice as bad :)
[16:22] <henninge> danilos: ping
[16:29] <danilos> henninge, hi
[16:29] <danilos> henninge, sorry, was otp
[16:29] <henninge> np
[16:29] <henninge> danilos:  what is the meaning of the "from_upstream" attribute of a queue entry nowadays?
[16:30] <henninge> used to be "is_published"
[16:30] <danilos> henninge, ah, I think it needs to change (I know we changed it already)... it's not really from_upstream, I think it should be "from_package"
[16:31] <danilos> henninge, though, then again, we can keep it as from_upstream to indicate what side  it's coming on
[16:32] <henninge> danilos: to determine if an import is from upstream, the potemplate can be queried for productseries/distroseries.
[16:32] <henninge> that would make from_upstream a computed attribute
[16:32] <danilos> henninge, except in sourcepackage uploads, which are from_upstream, basically
[16:32] <henninge> ???
[16:32] <danilos> henninge, well, translations we import on ubuntu from source packages are upstream translations
[16:33] <danilos> henninge, they are just from tarballs instead of from a VCS
[16:33] <henninge> danilos: I am not sure I understand
[16:33] <henninge> danilos: will they be imported into upstream templates or ubuntu templates?
[16:34] <danilos> henninge, uhm, with the new model, into both
[16:34] <danilos> henninge, but they will be considered is_current_upstream
[16:34] <henninge> danilos: ah!
[16:35] <henninge> danilos: so this is another parameter to the "share with other side" policy
[16:35] <danilos> henninge, well, kind of... the good end result will depend on that policy being right :)
[16:35] <henninge> danilos: "If the import is for a sourcepacakge but is marked as 'from_upstream', share it with upstream"
[16:36] <danilos> henninge, no, "if import is for a sourcepackage but marked as 'from_upstream', consider it an 'upstream' translation"
[16:36] <henninge> but that is more difficult for the importer
[16:36] <danilos> henninge, then, a policy of "if there's an upstream translation and it's better than ubuntu one, use that one for ubuntu as well" will give us the same result we get today
[16:37] <danilos> henninge, how come? it just means go through the same code path as if it's an import for a productseries
[16:37] <henninge> yes but it needs to determine the productseries first.
[16:37] <henninge> from the sourcepackage
[16:37] <henninge> code duplication
[16:39] <danilos> henninge, it should be possible to avoid it by calling setCurrentTranslation(upstream=True) directly
[16:39] <henninge> danilos: My take: when sharing translations with the other side it does not matter on which side the translations are imported
[16:40] <henninge> hm, maybe this is about template creation.
[16:40] <danilos> henninge, well, I think we just need to call setCurrentTranslation with appropriate parameters, we don't need to know the exact template
[16:40] <danilos> henninge, because we've got potmsgset already
[16:41] <henninge> on template import? not necessarily
[16:41] <danilos> henninge, I mean, we've got it on "ubuntu" side, we don't need it on the upstream side as well
[16:42] <danilos> henninge, by "got", I mean "just created or existing"
[16:42] <danilos> henninge, we should not import templates from sourcepackages as "upstream"
[16:43] <jml> bigjools, the stakeholder board says "generalized build histories" is done. is that accurate?
[16:43] <danilos> henninge, basically, what will happen is that we'll internally have potmsgsets with upstream translations, but there may not even be any upstream templates in LP
[16:43] <bigjools> jml: yup
[16:43] <henninge> yes, I am beginning to see that
[16:43] <jml> bigjools, cool! I'll review the LEP for sign-off & get back to you.
[16:44] <bigjools> jml: we're just waiting on jtv to finish off his work on actually using it, and then we can go on a made delete-fest of all the old code
[16:44] <bigjools> s/made/mad/
[16:44] <bigjools> recipes already use it
[16:45] <jml> bigjools, yay deleting code
[16:45] <bigjools> it's a good feeling :)
[16:47] <danilos> bigjools, most of what we could do so far was in ec2 this morning ;)
[16:48] <bigjools> \o/
[16:48] <jml> "could do"?
[16:49] <danilos> jml, if I understood it correctly, some bits need to be moved to using the new model fully (we were blocking that), and we didn't do any of that
[16:50] <jml> danilos, ahh ok.
[16:51] <jml> I have a mild mistrust of anything that claims to be generalized and is only used for one type of thing.
[16:53] <abentley> jml, +1.
[17:07] <abentley> Does anyone know how to configure zope to treat the return value of itertools.groupby() the same way it treats generators?
[17:10] <jml> abentley, I don't follow. Doesn't itertools.groupby() return an iterator?
[17:11] <abentley> jml, yes it does, but the security proxy doesn't treat iterators the way it treats generators.
[17:11] <jml> abentley, I'm surprised, and also desire to be crystal clear on terminology.
[17:12] <jml> abentley, generator here means... ?
[17:12] <abentley> jml, so if I return itertools.groupby(), I get a security proxy violation on __iter__.  If I return (x for x in itertools.groupby), everything is good.
[17:12] <jml> buggeration.
[17:13]  * jml checks a thing
[17:13] <abentley> jml, generators are iterators produced by functions with yield statements or generator expressions.
[17:14] <abentley> iterators is a superset including generators as well as hand-crafted iterator classes.  itertools.groupby appears to return the latter.
[17:14] <jml> abentley, ok. I think I see what's going on.
[17:14] <jml> abentley, groupby returns an object that implements the iterator protocol
[17:15] <abentley> jml, correct.
[17:15] <jml> abentley, I'm guessing that Zope's handling for __iter__ and what-not is done in a type-based fashion.
[17:15] <jml> abentley, and that the highly specific type returned by itertools.groupby() is not included in the list of types for which zope makes this exception.
[17:15] <abentley> jml, me too.  I suppose they may have associated interfaces with the built-in types, though.
[17:16] <jml> abentley, that's my guess
[17:16] <jml> except they've missed the built-in type called itertools.groupby
[17:16] <jml> I guess now it's time to explore zope.security
[17:17] <abentley> jml, well, I was hoping someone would know off the top of their head.  But I don't see gary here today.
[17:25] <jml> abentley, something in zope.security.checkers._defaultCheckers
[17:26] <jml> working up from there
[17:27] <jml> abentley, z.s.checkers.defineChecker(itertools.groupby, z.s.checkers.NamesChecker(['next', '__iter__'])) might do the trick
[17:28] <jml> not sure how to do that in zcml
[17:28] <abentley> jml, thanks, I'll give that a shot.
[17:50] <jml> abentley, I've just done a pass over the SPRB LEP and kicked it into "In progress"
[17:51] <abentley> jml, does that reflect drafting or implementation?
[17:51] <jml> abentley, implementation
[17:52] <jml> abentley, I'm happy with the LEP if you are.
[17:52] <jml> I haven't looked over the UI stuff.
[17:53] <abentley> jml, I think it's okay.  What do you mean by Allow users to use James Westby's imports to provide the debian directory *without the user leaving the web UI*
[17:54] <jml> abentley, you noticed. I mean that if the way to use the lp:ubuntu/foo imports involves switching to a terminal and making a new branch or something similar, then that's not really acceptable
[17:55] <abentley> jml, I am surprised by the idea that using the lp:ubuntu/foo imports might involve switching to a terminal and making a new branch or something similar.
[17:55] <abentley> jml, Wouldn't that defeat the purpose of using recipes?
[17:56] <jml> abentley, pretty much :) perhaps the qualifier is redundant.
[17:56] <abentley> jml, to me it suggests perhaps you want it to automatically recommend a packaging branch or something.
[17:57] <abentley> jml, so that the user doesn't have to leave the recipe creation page to search for the correct branch.
[17:57] <jml> oh, I see.
[17:57] <jml> well that would be nice but certainly not critical.
[17:59] <abentley> jml, what makes mark a stakeholder?
[18:00] <jml> abentley, this is something he cares about a great deal. it's largely his vision.
[18:01] <abentley> jml, does "Promptly provide full information on failed builds to someone capable of debugging the failure" mean it must spam me?  Or the packaging author?  Or the build requester?
[18:02] <abentley> jml, "Builds are performed in a timely manner" is something that the build farm does not guarantee for any builds.  Why do we have this special burden?
[18:02] <jml> abentley, re timely
[18:03] <jml> abentley, it's something that is necessary for me to consider the feature complete. it doesn't mean at all that you will be obliged to do the work to make it so.
[18:04] <jml> abentley, indeed, that work is already been done by team soyuz.
[18:04] <abentley> jml, it seems like a big scope creep to me.
[18:04] <jml> abentley, not to me.
[18:04] <jml> abentley, it's always been _daily_ builds
[18:05] <rockstar> james_w, how do you usually run the bzr-builder tests?
[18:05] <james_w> rockstar: bzr selftest -s bp.builder
[18:05] <abentley> jml, It has not been done.  In fact, I was supposed to work on a new build farm, and AIUI that task is now on hold.
[18:05] <jml> abentley, sorry, I meant "being"
[18:05] <rockstar> james_w, ah, didn't know about -s
[18:07] <abentley> jml, I'm not aware of any Soyuz plans to change the basic scheduling algorithms, but until that's done, we'll have the potential for build starvation.
[18:07] <bigjools> it's an extremely hard problem to solve, but I've not forgotten it
[18:07] <jml> abentley, so we'll have to solve it then.
[18:08]  * jml seizes this opportunity to order a book on queuing theory
[18:08] <abentley> bigjools, but until someone solves it, jml won't consider the daily builds LEP complete.
[18:08] <bigjools> that's crazy
[18:09] <jml> bigjools, what can I say, I'm loco.
[18:09] <bigjools> recipe builds are now scored the same as PPA jobs, they won't starve
[18:10] <abentley> bigjools, if we lose some of our loaded builders and a bunch of higher-scored builds get created, they can.
[18:10] <jml> (£80 is not the price of a book!)
[18:10] <abentley> s/loaded/loaned
[18:11] <bigjools> abentley: in theory yes, but there's only one thing that generates higher-scored builds and they are not frequent, so in practice it does not happen
[18:12] <abentley> bigjools, aren't there a bunch of modifiers that can push up the score?  Even a bunch of builds with a 2010 score can starve the daily builds.
[18:13] <abentley> Or was that 2510?
[18:15] <jml> abentley, wrt "provide information on failed builds", out of those, I mean the build requester
[18:16] <jml> abentley, errors in the system are already (I hope!) handle by OOPSes
[18:16] <jml> handle*d*
[18:17] <jml> abentley, or the recipe author, I guess...
[18:17] <abentley> jml, I believe the build requester is required to be a member of the recipe author.
[18:18] <jml> abentley, that's good. even so, maybe the broader group is more appropriate. I don't know right now.
[18:18] <abentley> jml, we don't distinguish between errors in the build that are system errors vs user errors.
[18:19] <abentley> jml, rather like code imports.
[18:19] <jml> abentley, by design?
[18:20] <abentley> jml, kinda.  Our builds are just like other builds in this regard.
[18:20] <jml> it seems that, for example, any exception that is raised outside of slave is guaranteed to not be a user error.
[18:20] <abentley> jml, sure.
[18:20]  * jml is having a bad day with grammar.
[18:20] <abentley> jml, that's why I said errors in the build.
[18:21] <jml> abentley, what happens with errors in the build now?
[18:21] <abentley> Once the build is completed and the slave is no longer responsible, errors can be handled as user errors or oopses.
[18:22] <abentley> jml, we send an email to the user who requested the build.
[18:22] <jml> abentley, so how do we learn of failures in the system?
[18:23] <abentley> jml, like with code imports, we rely on users to escalate system errors as bugs.
[18:23] <jml> I could have sworn we had oopses for code imports.
[18:24] <abentley> jml, maybe we do.
[18:25] <jml> abentley, ok.
[18:26] <jml> abentley, in any case, I'll leave banging on about architectural transparency to lifeless. I mostly care that users who have a build that screws up find out about it and are given enough data to try to fix it.
[18:26] <abentley> jml, anyhow, how can we in principle know whether an error is due to user error or system error?
[18:26] <jml> abentley, with the puller we have some heuristics. but that approach is probably not relevant here.
[18:29] <abentley> jml, can I substitute "the person who requested the build" for "someone capable of debugging the failure"?
[18:31] <jml> abentley, sure.
[18:34] <abentley> jml, I'm also adding "Renaming branches does not break recipes" under "nice to have".
[18:34] <jml> abentley, good call.
[18:34] <jml> abentley, Launchpad more broadly needs to get its act together wrt renaming stuff.
[18:35] <abentley> jml, (we kind of assumed that was important, so branches are stored as DB ids)
[18:36] <abentley> jml, the relevant iterator from my zope security question is itertools._grouper, and it comes from a C module and is not exposed.
[18:37] <jml> you mean type(itertools.groupby([1, 2, 3]).__iter__()) == itertools._grouper?
[18:39] <jml> abentley, any more questions about the LEP?
[18:39] <abentley> jam, I mean type(list(itertools.groupby([1, 2, 3]).__iter__())[0][1]) == itertools._grouper
[18:39] <jml> ahhh.
[18:40] <jam> abentley: I think you mean jml :)
[18:40] <abentley> jml, should there be any requirements about bzr branch format support?
[18:40] <jml> Python sucks.
[18:40] <abentley> jam, sorry.
[18:40] <jam> np, I just showed up
[18:40] <jml> abentley, I don't care about anything that's not 2a.
[18:40] <jml> abentley, is that what you mean?
[18:40] <abentley> jml, many of our supported distros don't support 2a.
[18:41] <jml> abentley, why does that matter?
[18:42] <jml> abentley, because the people using those distros won't be able to push up compliant branches? or because we run the bzr-builder recipe in an image based on the distro it's targeted for?
[18:42] <abentley> jml, do we want to be able to build bzr recipes for old distros if said distros don't support 2a but the relevant branches are in 2a?
[18:42] <jml> abentley, yes, I think so.
[18:43] <abentley> jml, our initial implementation did not support that, so it's easy to imagine that an implementation would not support that unless we make it a requirement.
[18:44] <jml> abentley, for sure.
[18:44] <jml> "branch formats usable within recipes is independent of branch formats supported by target distro" or something.
[18:46] <abentley> jml, didn't see you writing.  I put "Allow recipes to be built from bzr 2a-format branches for any supported distorseries (whether or not the distroseries itself supports 2a). "
[18:46] <jml> fair enough
[18:47] <jml> abentley, if bzr gets another format between now and the LEP being finished we can adjust accordingly :)
[18:47] <abentley> jml, if you keep the timeliness requirement, it probably will :-P
[18:47] <jml> hah
[18:49] <abentley> jml, anyhow, aside from the timeliness requirement, I'm happy with the LEP as it stands.
[18:50] <jml> abentley, cool.
[18:51] <lifeless> moin
[18:52] <abentley> jml, do we have a graph of how long it takes SRPBs to get dispatched?  I know we have one for build queue size.
[18:52] <abentley> ah, lifeless is around.  Must be lunchtime.
[18:52] <jml> abentley, I don't know.  I don't think so.
[18:53] <abentley> jml, could you set that up, or should I ask thumperfrancis?
[18:53] <jml> abentley, ask flumper.
[18:53] <jml> abentley, I just recalled a use case that isn't very well documented
[18:54] <jml> abentley, one big use case for recipes is having a daily build of trunk without debian/ubuntu patches.
[18:54] <jml> or with as few as possible.
[18:55] <jml> abentley, I'm not sure how this translates into requirements.
[18:55] <abentley> jml, I don't know, either, because some patches may be necessary to build at all.
[18:56] <abentley> One could perhaps fork the packaging branch, remove unwanted changes.
[18:57] <jml> abentley, or make a completely new packaging branch
[18:57] <abentley> Then either use that in the recipe, or use the recipe to merge the packaging branch.
[18:57] <jml> abentley, and express the ubuntu one as a recipe on top of that.
[18:57] <abentley> jml, you can also use nest-part to exclude changes outside the debian directory.
[18:58] <rockstar> mars, ping
[18:58] <jml> abentley, the difference between the "pure" packaging and the ubuntu packaging is primarily of interest for testing upstream code.
[18:58] <abentley> jml, but of course, some patches will be provided within the debian directory.
[18:59] <abentley> jml, it also provides a vector for upstreams who feel that ubuntu's changes are unwelcome to get a pristine version to their users.
[19:00] <jml> abentley, hello.
[19:00] <abentley> jml, hi.
[19:00] <jml> abentley, sorry, mischan
[19:01] <jml> abentley, yes. that's also a good case.
[19:02] <jml> abentley, perhaps this means that we should have some kind of special marking for "pure" packaging...
[19:02] <jml> something to chew on.
[19:02]  * jml otp
[19:02] <abentley> If you look at bzr, the ubuntu version removes configobj from the tree and depends on its package instead.
[19:03] <abentley> a "pristine" bzr would have configobj in the tree, and would therefore have an unnecessary dependency.
[19:03] <abentley> Unless you change the bzr packaging manually.
[19:09] <lifeless> deryck: can we have a brief call after this ?
[19:10] <deryck> lifeless, I have a call with thumper after this, but depending on how long it lasts, after that would be fine.
[19:11] <jkakar> Does the LP schema allow a project to be in more than one project group... I know the UI doesn't allow it... but is it possible?
[19:12] <lifeless> deryck: great
[19:12] <lifeless> jkakar: iirc no there is a unique constraint
[19:13] <jkakar> lifeless: Cool, thanks.  It wonder if enough people would benefit from projects being able to be in more than one group to make it worth changing.
[19:13] <lifeless> jkakar: I want to eliminate pgs as a special case
[19:14] <lifeless> use multiple tagging/categorising systems
[19:14] <lifeless> e.g. tags to to bottom up associations
[19:14] <jkakar> lifeless: That sounds cool.
[19:14] <lifeless> and 'defined groups' to do top down defined structures
[19:14] <jkakar> lifeless: What I want is a way to group projects such that I can have a +activereviews page that shows me reviews for the group.
[19:14] <lifeless> and permit both to exist
[19:14] <jkakar> lifeless: Also, to see bugs in the group.
[19:15] <jkakar> lifeless: Also, to create milestones for the group.
[19:15] <jkakar> So I guess I want the group to be identical to a project, but with less arbitrary limits on how I can form them. :)
[19:15] <lifeless> yes
[19:15] <lifeless> I would like the same too, in the fullness of time.
[19:15] <jkakar> lifeless: Cool!
[19:42]  * jml gone
[19:43] <mars> sinzui, do you think a wiki page would be a better way than the ML to track Maverick upgrade issues?
[19:43] <lifeless> sinzui: hi https://bugs.edge.launchpad.net/launchpad-registry/+bug/638924
[19:43] <_mup_> Bug #638924: Milestone:+index timeouts with many announcements <timeout> <Launchpad Registry:Triaged> <https://launchpad.net/bugs/638924>
[19:44] <lifeless> sinzui: I've just dug a bit deeper into that timeout
[19:44] <lifeless> sinzui: I drew some different conclusions about the issue, and thought perhaps you'd like to spend a little time later discussing it
[19:47] <sinzui> lifeless, I would like to discuss it. I have two more meetings today, but I am free this evening and tomorrow
[19:47] <lifeless> sinzui: ping me anytime
[19:47] <sinzui> fab
[19:47] <lifeless> sinzui: I'd like to get some hints out there for doing the analysis stuff
[19:48] <lifeless> Ursinha: https://devpad.canonical.com/~lpqateam/qa_reports/launchpad-stable-deployment.txt seems to have not updated ?
[19:49] <lifeless> Ursinha: also, as that rev *is* deployed, I think you're pointing it at the wrong branch ?
[19:50] <Ursinha> lifeless, wait, I'm checking
[19:50] <lifeless> (its meant to start at the deployed rev, not randomly far back)
[19:50] <Ursinha> it seems it was updated few minutes ago
[19:50] <mars> lifeless, Ursinha, the ctime on those reports says they are only 3 minutes old
[19:50] <lifeless> so, there is a bug then
[19:50] <lifeless> bug 11518
[19:50] <_mup_> Bug #11518: Linux kernel-image-2.6.9-1-686 synaptic pass through problem hoary <Ubuntu:Invalid by fabbione> <https://launchpad.net/bugs/11518>
[19:50] <lifeless> bah
[19:50] <lifeless> rev 11518 has bug 627741
[19:50] <_mup_> Bug #627741: oops-prune is aborting with a regex mismatch(?) <canonical-losa-lp> <qa-untestable> <Launchpad Foundations:Fix Committed by stub> <https://launchpad.net/bugs/627741>
[19:50] <lifeless> thats qa-untestable
[19:51] <lifeless> thats one issue
[19:51] <lifeless>  the second issue is that that rev is *already deployed*
[19:51] <lifeless> it should be starting much higher up
[19:51] <lifeless> what branches are you giving it ?
[19:57] <Ursinha> lifeless, it seems last deployed revision on lpnet was 11515
[19:58] <Ursinha> script starts processing at 11516
[19:58] <lifeless> hang on though
[19:58] <Ursinha> which is correct
[19:58] <lifeless> 'stable' is deployed to edge
[19:58] <Ursinha> now for the other problem
[19:58] <lifeless> db-stable is deployed to lpnet
[19:58] <lifeless> actually thats a convenient lie but its good enough for now.
[19:59] <lifeless> (because we need something to measure which db commits can be deployed)
[19:59] <lifeless> Ursinha: what branches is it running with please ?
[20:02] <Ursinha> lifeless, stable and db-stable
[20:02] <lifeless> Ursinha: it takes branch pairs IIRC
[20:02] <lifeless> Ursinha: source and target
[20:02] <Ursinha> MergeDeployment is lpnet
[20:03] <Ursinha> devel, stable and db-devel, db-stable
[20:03] <Ursinha> lifeless, merge environment for both stable and db-stable is lpnet
[20:03] <Ursinha> so it looks lpnet for the last merged branch
[20:03] <lifeless> ok, that will make sense once we remove edge
[20:04] <lifeless> for now though, the devel/stable one needs to check against 'stable'
[20:05] <lifeless> Ursinha: ah yes, I see
[20:05] <Ursinha> lifeless, let me see if I got the picture: people land code on devel/db-devel, and it goes do edge/staging, they QA it,  and it supposedly should go live on lpnet
[20:05] <Ursinha> right?
[20:05] <lifeless> and I didn't provide a knob for this.
[20:05] <lifeless> ignore db- for this discussion
[20:06] <lifeless> Ursinha: actually, what I'll do is ask for a CP of everything.
[20:06] <lifeless> that will sort this out.
[20:06] <Ursinha> not sure what you mean
[20:07] <lifeless> but it would be nice to know that everything currently in stable has been QA'd, to pick a rev to CP
[20:07] <lifeless> Ursinha: well a CP is a merge to production-devel
[20:07] <Ursinha> lifeless, I thought this was exactly how the script is working now
[20:08] <lifeless> ahh,, and I'll be able to do that if we figure out this other isue.
[20:08] <Ursinha> what's on stable is what should be QAed
[20:08] <lifeless> Ursinha: ok, yes, I'm being noddy.
[20:08] <lifeless> Ursinha: so lets talk about the other thing :) [sorry!]
[20:08] <Ursinha> hehe
[20:08] <Ursinha> I'm investigating the bug thing
[20:08] <lifeless> ok
[20:17] <mars> rockstar, I have a reply on the way to you, but I need to answer a question about the JS build system first
[20:21] <rockstar> mars, great, thanks.
[20:21] <rockstar> mars, I'd like to deal with this today, as it means that I have a kanban card that's just sitting.
[20:21] <Ursinha> lifeless, ok, so that was a stupid thing: qa-needstesting wasn't considered a deployable status, only qa-ok
[20:21]  * Ursinha goes add tests
[20:22] <lifeless> Ursinha: qa-untestable do you mean ?
[20:22] <Ursinha> lifeless, argh, yes, sorry
[20:22] <lifeless> qa-needstesting isn't deployable ;)
[20:22] <Ursinha> lifeless, qa-untestable
[20:22] <Ursinha> lifeless, https://devpad.canonical.com/~lpqateam/qa_reports/launchpad-stable-deployment.txt
[20:22] <Ursinha> it didn't go too far though
[20:23] <lifeless> thats ok
[20:23] <lifeless> we can look into that in a minute
[20:31] <rockstar> ARGH!  Stabby stabby moin!
[20:47] <rockstar> thumper, when you're around, I'd like to chat about merge queues.
[21:04] <deryck> Later on, everyone....
[21:04] <lifeless> rockstar: we hsould just use em.
[21:05] <lifeless> if bugs turn up, we can fix.
[21:05] <rockstar> lifeless, :)
[21:05] <rockstar> lifeless, there are some things about the current model that I think complicated things greatly.
[21:05] <mars> rockstar, understook
[21:05] <mars> understood even
[21:05] <lifeless> rockstar: not disagreeing
[21:05] <lifeless> rockstar: but its inventory today.
[21:05] <rockstar> mars, wtf keyboard layout are you using that had d and k right to them?  :)
[21:06] <rockstar> lifeless, yeah, inventory that I want to throw out on the streets!  :)
[21:11] <rockstar> lifeless, you will see, though, that we have a whole lane in kanban for finishing up merge queues now.
[21:13] <jam> rockstar: ping about bug #612754
[21:13] <_mup_> Bug #612754: Submit Request Failure: Signature couldn't be verified: (7, 8, u'Bad signature') - with email signed and sent from sup-mail <Launchpad Bazaar Integration:Incomplete> <https://launchpad.net/bugs/612754>
[21:13] <jam> It is blocking me pretty much on every submission
[21:13] <lifeless> rockstar: a lane? wouldn't it just fit under features?
[21:13] <lifeless> rockstar: anyhow, details aside...
[21:14] <lifeless> rockstar: I saw your update; why one queue per project? isn't one queue per series easier conceptually?
[21:22]  * benji goes afk for a bit.
[21:29] <rockstar> lifeless, one queue per branch, but many branches can be in one queue.
[21:29] <rockstar> lifeless, we can't just draw the line at series or anything.  Think about all the branches currently managed by ~launchpad-pqm
[21:29] <rockstar> jam, EVERY submission?
[21:31] <jam> rockstar: any time I send a "merge: approve"
[21:31] <jam> and a lot of bug reports, too, I believe
[21:32] <rockstar> jam, yeah, it would use the same mechanism.  I think might fall under "foundations" domain, but not sure.
[21:32] <rockstar> jam, could you try with a different mail client?
[21:32] <jam> I can obviously go to the website, but the email interface is broken for me
[21:32] <thumper> jam: which email client?
[21:32] <rockstar> jam, mathiaz was having intermittent problems, so I was suspecting that it was a mail client issue.
[21:32] <jam> rockstar: I don't have another mail client integrated with gpg. But even so, I can check the raw texts pass, and I had Vincent do the same thing for me
[21:33] <thumper> rockstar: I'm nomming
[21:33] <thumper> rockstar: and I'd like to grab abentley first
[21:33] <jam> thumper: thunderbird 3 w/ enigmail
[21:33] <rockstar> jam, is Vincent using the same client?
[21:33] <jam> rockstar: vincent uses gnus, or whatever the emacs client is
[21:33] <rockstar> thumper, we have our 1-1 today, right?
[21:33] <rockstar> jam, *sigh*
[21:33] <abentley> thumper, certainly.
[21:33] <thumper> rockstar: I guess
[21:33] <jam> rockstar: you should be able to just try "gpg --verify submit_request_failure.txt" that I attached to the email
[21:34] <rockstar> jam, okay.
[21:36] <jam> rockstar: I just sent another email (which I expect to be blocked). I suppose I can CC/BCC you one of them?
[21:36] <rockstar> jam, absolutely.
[21:37] <jam> (I just BCC'd you on an artificial one, though I think that request is actually bogus, as you can't set WIP from email)
[21:37] <jam> but it should still gpg verify
[21:38] <rockstar> jam, so I've verified that your message was good.
[21:41] <lifeless> rockstar: sure, s/series/target branch/
[21:41] <lifeless> rockstar: what I mean is that we wouldn't want db-devel and devel to both use the same queue - they would starve each other
[21:42] <jam> rockstar: well, at least you and I agree. Now if we can just get LP to feel the same :)
[21:42] <jam> rockstar: I haven't see the messages show up on 'review' but I also haven't gotten the "Submit Request Failure" yet.
[21:48] <rockstar> lifeless, I disagree that they'd starve eachother.  We're essentially doing the same thing now with pqm.
[21:49] <lifeless> rockstar: pqm isn't running the tests
[21:49] <lifeless> rockstar: and pqm did starve each other
[21:49] <lifeless> rockstar: huge issues with that
[21:49] <rockstar> lifeless, tarmac, in this case, wouldn't run the tests either.
[21:50] <lifeless> rockstar: we want to make the lander run the tests again
[21:50] <lifeless> rockstar: thats gary's experiment, landing in batches of 5
[21:50] <rockstar> lifeless, and we can do something like that.  But at that point, we'd have to separate out what branches are on what queue.
[22:07] <lifeless> is it just me
[22:07] <lifeless> or is
[22:07] <lifeless> FAILURE: lp.services.job.tests.test_runner.TestJobRunner.test_oops_messages_used_when_handling (subunit.RemotedTestCase)
[22:07] <lifeless> failing for every branch ?
[22:08] <lifeless> on ec2
[22:10] <jelmer> I had an error like that as well
[22:10] <jelmer> I figured since the error in the oops was soyuz related it was somehow my fault
[22:11] <wallyworld_> lifeless: i got the same error too when trying to land a branch
[22:12] <lifeless> ok
[22:13] <lifeless> its probably not unrelated to my oops work
[22:14] <lifeless> running just that test works
[22:14] <wallyworld_> don't you hate it when that happens :-(
[22:15] <lifeless> so either oops-writing is naffed
[22:15] <lifeless> or get last oops is
[22:17] <lifeless> mars: still here?
[22:18] <wallyworld_> thumper: rockstar: abentley: skype?
[22:18] <rockstar> wallyworld_, sure.
[22:18] <thumper> rockstar, wallyworld_: abentley and I are chatting
[22:18] <thumper> give us a few minutes to finish what we are talking about plz
[22:19] <wallyworld_> thumper: np
[22:19] <mwhudson> morning
[22:19]  * jelmer waves
[22:21] <lifeless> I'm looking in uniquefileallocator if folk are interested
[22:22] <mars> lifeless, yep
[22:22] <mars> lifeless, what's up?
[22:22] <lifeless> mars: how clean is the oops dir in an ec2 test run
[22:22] <lifeless> mars: see the list for the context
[22:23] <mars> lifeless, no idea. someone could run with ec2 test with --postmortem and find out though.
[22:28] <lifeless> we really should change the filenames on disk to be less magical
[22:39] <lifeless> and for fun
[22:39] <lifeless> running the two relevant tests together, here, works.
[22:40] <lifeless> :!bin/test -vvt test_uploadprocessor.TestUploadProcessor.testOopsCreation -t lp.services.job.tests.test_runner.TestJobRunner.test_oops_messages_used_when_handling
[22:41] <lifeless> mars: a premortem for ec2 would be nice
[22:41] <lifeless> mars: e.g. set it all up, but run nothing
[22:43] <james_w> you can do it with "-d" and some stepping
[22:43] <james_w> I thought jml had implemented --dont-test or something
[22:43] <thumper> jelmer: any idea why https://code.edge.launchpad.net/~vcs-imports/mesa/master is failing?
[22:48] <lifeless> hah, I just noticed my lp vm is in sydney time.. that explains some confusion I had
[22:49] <jelmer> thumper: looking
[22:49] <mars> lifeless, jml was implementing that option.  Otherwise you can do it with "ec2 test -p -o '-t asdf'"
[22:50] <lifeless> this really has me flummoxed
[22:50] <jelmer> thumper: Looking at the history, it seems the branch was reimported from scratch recently?
[22:50] <thumper> jelmer: yes, yesterday
[22:50] <lifeless> i have tried deleting some oopses
[22:50] <lifeless> and it correctly ignores the holes and writes to the end and finds them again
[22:51] <jelmer> thumper: So I guess this can't really be a bzr-git bug that caused incorrect repo data.. I'll try locally & file a bzr-git bug.
[22:51] <jelmer> thumper: From just looking at the logs I don't really have an idea what could be going wrong.
[23:17] <lifeless> I've put a branch up that may help
[23:42] <poolie> lifeless: so, about sending output from process-mail.py to a log file
[23:43] <lifeless> theres an option to do that already apparently
[23:43] <lifeless> --log-file or something
[23:43] <poolie> mm, apparently it's not used
[23:43] <lifeless> should just need an RT (and for someone to be monitoring the log)
[23:43] <poolie> they just manually redirect it in the cron script
[23:43] <lifeless> yeah
[23:44] <poolie> i did look at it a few weeks ago and ISTR that --log-file may be broken or something
[23:44] <poolie> imbw
[23:44] <poolie> but i think i said "how does it even work?" and spm said they don't use it :/
[23:44] <lifeless> its a thing that was built and not deployed
[23:44] <lifeless> we should use it
[23:45] <poolie> also, as i recall, someone said "we can't just send it to a file because we need to know about errors"
[23:45] <poolie> maybe this was you?
[23:45] <lifeless> we do need a story around that
[23:45] <poolie> but i don't see how sending it to a file is much worse
[23:45] <lifeless> ideally --log-file would not hide ERROR severity messages
[23:45] <lifeless> if you wanted to pull on this yak I think that would be great.
[23:46] <poolie> 'i am in yaks stepped in so far that should i wade no more, returning were as tedious as to go over'
[23:47] <poolie> so i propose
[23:47] <poolie> - for now, just sending an rt asking them to send it to a file
[23:47] <poolie> if they object, we'll deal with that, but i don't see how it should make things any worse
[23:48] <poolie> - filing a bug about splitting >=error to stderr and everything to a file
[23:48] <poolie> (which will necessitate actually using --log-file not just >&file)
[23:48] <lifeless> seems fine to me if you add in
[23:48] <poolie> and i may work on that one
[23:48] <lifeless>  - email lp-dev letting folk know that errors from it may not show in the -errors-report anymore
[23:48] <poolie> then using the results of #1 to work out what's up with dkim
[23:49] <poolie> so i can say the rt has your approval?
[23:49] <lifeless> sure; cc me?
[23:51] <poolie> for the sake of parallelizing, is the errors mailbox anywhere i could see it?
[23:52] <lifeless> lp-error-reports list should have it, yes
[23:53] <poolie> so i'd need to join that list, disable delivery, then look in the archives?
[23:53] <lifeless> yeah, I think so.
[23:54] <poolie> some devops people would say that sending mail even in the case of errors is not the best idea
[23:54] <poolie> if things are falling over, mailservers may be swamped or unreliabel, etc
[23:55] <lifeless> mmm
[23:55] <lifeless> that applies to all automation
[23:55] <lifeless> email is one of most robust systems we have : more robust that being able to ssh into the problem box.
[23:56] <poolie> it's a cheap but inefficient way to broadcast things
[23:57] <lifeless> sure
[23:57] <lifeless> we don't have better yet.
[23:57] <lifeless> particularly for errors.
[23:59] <poolie> so is it reasonable to send everything, including errors, to a file for now?
[23:59] <lifeless> can we do voice? I'm having deja vu and perhaps more bandwidth will address your questions quicker.