#launchpad-dev 2009-02-14
<Laibsch> Is it just me or is something wrong with the PPA pages?
<Laibsch> Whenever I click on the arrow to get the details for a package I am taken to the same page all over again
<Laibsch> But no details
 * Laibsch suspects some JavaScript problem
<wgrant> Laibsch: See the topic - this isn't for support. It doesn't seem to actually be for anything at all at the moment.
#launchpad-dev 2009-02-15
<Laibsch> wgrant: this is not about support but about a malfunction of LP
<Laibsch> I assume it was some recent code change that triggered this
<Laibsch> Furthermore, I suspect it may have to do with the renaming of PPAs
<ScottK> Laibsch: This still isn't the right channel.  You want #launchpad.
<ScottK> Until Launchpad is open sourced, by definition what you have is a support discussion.
<Laibsch> ScottK: Well, I'm not sure that the people who could actually do the support (and who are responsible for the release of likely broken code) are in that channel. So, what's the use?
<ScottK> Your odds are better there than here.
<Laibsch> "just let us freely introduce regressions and don't bother our peace for sake of channel discipline?"
<Laibsch> "just let us freely introduce regressions and don't bother our peace for sake of channel discipline"?
<ScottK> No, it's not a question of channel discipline, its a question of you asking where your odds are better.
#launchpad-dev 2010-02-15
<thumper> mwhudson: ping
<thumper> bug 298284
<thumper> comments?
<mup> Bug #298284: No headers suitable for filtering branch subscription emails for Gmail users <email> <Launchpad Bazaar Integration:Won't Fix> <https://launchpad.net/bugs/298284>
<thumper> wgrant: since you are around, any comments on that bug?
<wgrant> "fuck stupid mail clients", basically.
<thumper> wgrant: well... around 50% of the launchpad users use gmail
<wgrant> Silly people.
<mwhudson> silly gmail too
<mwhudson> thumper: i can't see how setting list-id as stub suggests would hurt
<lifeless> perhaps you submit bugfixes....
<lifeless> oh right, its proprietary
<lifeless> what you can do is write a imap client to do filtering for you on the gmail boxen
<wgrant> Is there a good reason that revision emails come from noreply@ rather than the committer?
<thumper> wgrant: no
<thumper> wgrant: at least I don't think so
<thumper> wgrant: what if the person doesn't want their email address shown?
<thumper> wgrant: or we don't know who they are...
<thumper> mwhudson: I'm beginning to think we should just do it (add a list id that is)
<wgrant> thumper: If they're committing to a public branch with an email address but don't want it disclosed, they are probably insane.
<thumper> mwhudson: conduct an experiment for code email, and if it ends up working, get other LP email sources to add it too
<thumper> wgrant: heh
<mwhudson> wgrant: some of our users are insane
<mwhudson> thumper: +1
 * thumper is finally through the inbox
 * mwhudson has a desk to put together, will be in and out irc-wise
<thumper> mwhudson: my first balsamiq mockup sent to the list (and you)
<thumper> mwhudson: do you think this is the right approach to starting?
<thumper> (I do)
 * mwhudson looks
 * thumper afk for a bit to get dinner on
<thumper> Ursinha: are you really here
<thumper> ?
 * thumper EODs
<lifeless> ciao
<noodles775> G'day all.
<mrevell> Morning
<wgrant> bigjools: I don't think anybody cares in the slightest about the "maintainer defaults". We have been saying this for years!
<lifeless> wgrant: ?
<wgrant> Bug 521722
<mup> Bug #521722: Display of Component on distro source summary page may be wrong (but we know the right value) <Soyuz:Incomplete> <https://launchpad.net/bugs/521722>
<bigjools> wgrant: fine, we should just remove that column then
<wgrant> bigjools: Why not replace it with the real value?
<stub> bigjools: re: https://code.edge.launchpad.net/~wgrant/launchpad/sprbu-columns-to-sprb/+merge/18995 , do we have a record on who did the upload elsewhere? (SourcePackageRecipeBuildUpload.registrant at the moment)
<bigjools> what real value?
<bigjools> it's got many
<bigjools> which are listed below
<stub> (or wgrant)
<wgrant> stub: SourcePackageRecipeBuild.requester.
<bigjools> what he said
<wgrant> bigjools: The current component in the development series is the current component.
<stub> k
<bigjools> wgrant: I don't see the benefit of repeating it
<wgrant> bigjools: Hm, true, it is just there.
<wgrant> Also, I wonder if we can make https://launchpad.net/debian/+source/dpkg more useful. The table doesn't take Pending SPPHs into account, so it's empty for gina'd archives.
<bunjee> I'm currently downloading the sources of Launchpad through the installer, and this seems to be rather slow, I'm stuck at 33760 or something and it freezes from time to time
<bunjee> Can we download them from another place?
<wgrant> stub: Thanks.
<adiroiban> Ursinha: hi, any idea why for bug 509252 the status was not updated?
<mup> Bug #509252: Remove AdminPoTemplateSubset from security.py <cleanup> <Launchpad Translations:In Progress by adiroiban> <https://launchpad.net/bugs/509252>
<leonardr> allenap, let me know when you have some time to help me with the bug problem i emailed you about
<allenap> leonardr: Very soon, I just saw your message.
<leonardr> i've made a little progress in diagnosis
<allenap> leonardr: Okay, there are some weird things to do with bug comments/messages. I'll get your branches and try to replicate here.
<leonardr> allenap, i can tell you it's not a navigation problem. a stock launchpad branch also turned up with a BugComment object there
<allenap> leonardr: So I should be able to replicate with only your lazr.restful branch?
<leonardr> allenap: my launchpad branch includes changes necessary to integrate the new lazr.restful branch
<leonardr> so you'll need both
<leonardr> allenap: i've confirmed that in a stock launchpad, the same code runs and self.entry.parent returns None
<leonardr> so i think it's something to do with the way the entry adapter was generated
<allenap> leonardr: Okay.
<allenap> leonardr: I think I'm being stupid. How do you normally set up your lazr.restful branch in a Launchpad branch?
<leonardr> allenap, good question
<leonardr> symlink it to a directory in your launchpad root directory
<leonardr> and change buildout.cfg to mention it
<leonardr> develop =
<leonardr>     .
<leonardr>     lazr.restful.dev
<leonardr> then run bin/buildout
<allenap> leonardr: Ah ha, that sounds good. Thanks
<leonardr> allenap: does this make any sense to you? i've got both versions of launchpad running side-by-side
<leonardr> and it looks like the object returned by traversal is a BugComment in both cases
<leonardr> but in stock Launchpad, the object that gets made into an EntryResource is a Message, and in my changed Launchpad, the object that gets put into the EntryResource is the same BugComment that was retrieved from traversal
<leonardr> obviously a Message has a .parent and a BugComment does not
<allenap> leonardr: Cool, that's useful info. I'm still trying to get the branch to run the test :-/ But I know that there's some smoke and mirrors in the bug message/comment area, and I helped to create some of it, so I'm hopeful I'll find the culprit.
<leonardr> cool
<leonardr> let me know if i can help you get it running
<allenap> leonardr: It keeps failing in create-lp-wadl.py with: AttributeError: 'WebServiceTestRequest' object has no attribute 'version'
<leonardr> allenap: that makes me think you're not using the new lazr.restful
<leonardr> allenap: i did make clean and am now getting the same error
<leonardr> this should be easy to fix
<allenap> leonardr: I have to go and feed children, so I'll be back in ~45 minutes, sorry about that.
<leonardr> ok
<leonardr> flacoste, i need some help from you today. i'm trying to integrate the new lazr.restful into launchpad so that we can make launchpadlib use '1.0' before the feature freeze
<leonardr> my gut feeling is that i won't make it, but if i'm going to make it i'll need advice from you since gary is not in today
<james_w> leonardr: is /beta/ going to start changing at that point?
<leonardr> james_w: no, beta will _stop_ changing at that point
<james_w> nice
<james_w> thanks :-)
<leonardr> james_w: the goal is to retire 'beta' at the same time that karmic is retired
<leonardr> if i don't make this goal then we will be retiring 'beta' when lucid is retired
<leonardr> allenap: i've pushed an update that will make the wadl generation work (in doing so i discovered a much bigger problem with the wadl generation, but you don't need to worry about it)
<leonardr> allenap: found it!
<allenap> leonardr: Awesome :)
<leonardr> i need your help figuring out what to do
<leonardr> the problem is bugcomment_to_entry
<allenap> leonardr: Phew, at least I might be able to help eventually.
<leonardr> allenap: basically, in the new system, you can no longer adapt just an object to an IEntry
<leonardr> you need to multi-adapt a 2-tuple (object, version_marker)
<allenap> leonardr: Ah. What does the version_marker signify?
<leonardr> allenap: which version of the web service the user is trying to access
<leonardr> ie. beta, 1.0, etc
<leonardr> i'm going to try just adding the marker interface to the adapter function and ignoring it. since this mechanism doesn't change between versions that _should_ work
<leonardr> allenap: i think it's working
<allenap> leonardr: Cool. I still don't quite get it all, but I'll pull your branch and figure it out. I've not been very useful to you, except perhaps as an over-the-shoulder viewer.
<leonardr> allenap: i've pushed my revision. looking at webservice.zcml that seems to be the only special adapter launchpad has
<leonardr> which would explain why everything else is working
<allenap> leonardr: Ah, I get it now. Awesome.
<flacoste> hi leonardr
<leonardr> hi
<leonardr> flacoste: right now i know of two unresolved issues w/r/t making launchpad multi-version
<leonardr> the cached WADL file and the apidoc derived from that file
<leonardr> right now my code is getting the 'devel' version of the WADL
<leonardr> we need to get separate WADL and generate separate apidoc for each version
<leonardr> but, maybe we don't need to do that right now
<leonardr> flacoste: i'm going to add a launchpad test to make sure that the 1.0 web service is _accessible_. then i can get launchpad/lazr.restful reviewed and landed, and then i can do a new launchpadlib
<leonardr> good plan?
<flacoste> leonardr: i agree, you don't need the separate apidoc right now, nor the separate WADL file
<flacoste> one thing we don't want though
<flacoste> is that people start developping 1.0 launchpadlib application
<flacoste> while we are still changing it
<flacoste> might be better to freeze beta now
<flacoste> cut devel
<flacoste> and release a launchpadlib that talks to devel by default
<flacoste> and discuss with the Ubuntu side that they would be ok with us uploading a version that switch devel to 1.0 in a couple of weeks
<flacoste> before beta
<flacoste> once we have frozen 1.0
<leonardr> flacoste: what changes were you hoping to make in 1.0?
<flacoste> leonardr: remove obsolete mutators
<flacoste> as named operations
<leonardr> ok, that's the only one i was planning to do in the next couple weeks
<flacoste> and maybe a general call to the teams to think about any API they wanted to change
<flacoste> break backward compatibility
<leonardr> flacoste: i don't think i agree about releasing a launchpadlib that talks to devel by default
<leonardr> what is the worst that happens if we release a 1.0 launchpadlib and people start using it?
<leonardr> 1.0 will start out one way and then change
<leonardr> that's the same as if they start using devel
<leonardr> and if something goes wrong and we end up releasing a devel launchpadlib in lucid, we're in big trouble
<flacoste> leonardr: expectations management
<leonardr> right now people don't even notice the version
<flacoste> we'll have to announce that the version change is happening and what it means
<flacoste> people will have to know that they can revert to beta if they want
<flacoste> i think doing devel -> 1.0 is a less confusing story
<leonardr> flacoste: do you know who i should talk to on the ubuntu side?
<flacoste> james_w: do you have a suggestion about the above suggestion?
<leonardr> james_w, maybe?
<flacoste> james_w: s/suggestion/conversation/
<leonardr> flacoste, james_w: sorry, my laptop battery died. what have you been talking about?
<flacoste> leonardr: no news from james_w yes
<flacoste> yet
<leonardr> ok
<james_w> hi
<james_w> sorry, was at lunch
<james_w> you want to know if we would be ok with a devel -> 1.0 default change in a couple of weeks
<james_w> ?
<flacoste> james_w: basically, yes
<james_w> I don't have a problem with it
<james_w> I'm not on the release team though
<flacoste> james_w: has launchpadlib been moved to main? apt still shows it as Ubuntu MOTUY Developers
<james_w> it has
<flacoste> james_w: are you the maintainer?
<james_w> it's in Debian now, so the Debian python team take care of it
<flacoste> james_w: and who is that?
<james_w> there's a whole bunch of them
<james_w> Luca is the one that has done most of the work so far
<flacoste> so I guess we should talk to him
<flacoste> and someone on the release team
<flacoste> pitti or slangasek i assume
<james_w> I can talk to Luca if you get the go ahead
<leonardr> flacoste: can you sort things out with the distro developers while i complete the integration?
<james_w> yeah, #ubuntu-release works
<flacoste> leonardr: i think it's best if you or gary handles it directly, let me know if there is problem
<leonardr> ok
<kfogel> I never modified sendmail.py in my branch.  But when I merged from today's devel, I got a conflict there.  I resolved it in the obvious way (half of it was empty text, and it looked like a bzr artifact), I still have this diff:
<kfogel> http://paste.ubuntu.com/376899/
<kfogel> (I'm sure this is related to r10306 in devel)
<kfogel> the sha part I can explain from my resolution, and can fix.  the set thing mystifies me
<kfogel> oh, duh, nm.  devel vs db-devel issue    sigh
<kfogel> adeuring: got a sec?  I'm beginning to wonder if I'm on the right track here.  http://paste.ubuntu.com/376948/
<adeuring> kfogel: LET ME LOOK...
<kfogel> adeuring: caps lock? :-)
<adeuring> kfogel: yeah... This key should be banned...
<kfogel> :-)
<adeuring> or I should remove it from my keyboard
<thekorn> hi thumper, when you are around, and have a minute, I've a question about bug 520412
<mup> Bug #520412: fix review_types argument of the API's Branch.createMergeProposal method <code-review> <oops> <Launchpad Bazaar Integration:Triaged> <https://launchpad.net/bugs/520412>
<kfogel> adeuring: what I'm trying to do is make a portlet template that can be reused on other pages as well, not just the product-index page
<adeuring> kfogel: is patches-view.test an IProduct ?
<adeuring> erm, patches-view-test
<kfogel> adeuring: yes
<adeuring> odd... I that case, I'm lost. In theory, things look correct, I'd say... But: Did you consider to define the protlet for IHasBugs instead of IProduct?
<adeuring> kfogel: I know, that question is not related to your problem, but anyway...
<kfogel> adeuring: phone, one sec
<adeuring> kfogel: no problem... I'm afraid you need to as somebody else anyway :(
<adeuring> kfogel: Just tired the diff you posted. Looks good; no problems to access http://launchpad.dev/firefox ; the new portlet appears. But there is a little bug in your ZCML data: a duplicate tag </browser:pages>
<adeuring> s(tired/tried/
<kfogel> adeuring: d'oh
<kfogel> adeuring: thank you (off phone now)
<kfogel> adeuring: I wonder why I'm getting that OOPS.
<kfogel> adeuring: you're right that the portlet should be for IHasBugs, I think
<adeuring> kfogel: A guess: Your test object is something else than an IProduct, or
<adeuring> you did not test the diff that you posted.
<adeuring> more likely the latter -- the web server should simply not start wirth that ZCML data ;)
<kfogel> adeuring: aaaaaah, that may be it
<kfogel> adeuring: thx
<kfogel> adeuring: working for me now
<adeuring> kfogel: great!
<jtv> Ursinha: poimport error reporting seems to be broken on staging... do you know anything about it?
<kfogel> adeuring: when you asked "Did you consider to define the portlet for IHasBugs instead of IProduct?", did you mean something like http://paste.ubuntu.com/376995/ ?
<adeuring> kfogel: yes
<kfogel> adeuring: cool, thanks
<kfogel> adeuring: man, I wish we were sitting in the same room :-)
<adeuring> kfogel: shall we organize  a mini sprint ;)?
<kfogel> adeuring: I'm all for it sometime soon, actually.  I have some travel plans solidifying right now, but once I know those, maybe I can come out to Berlin or you to NY in the next month or two...
<adeuring> kfogel: sounds like an interesting idea! But as a nitpick, I don't live in berlin, but in Bielefeld (ca 3 hours by train from berin)
<adeuring> s/berin/Berlin/
<kfogel> adeuring: oh, sorry -- dunno why I thought you lived in Berlin.
<adeuring> kfogel: because both names start with 'B' and sound somewhat foreign ;)?
<kfogel> adeuring: I'm not *that* American, man :-)
 * kfogel skirts around self-examination
<adeuring> kfogel: perhaps is because Bielefeld does not exist: http://www.dw-world.de/dw/article/0,,1400913,00.html
<kfogel> adeuring: I always suspected you might live in a place that does not exist.  Don't think we didn't all notice that you showed up in Alabama with an empty suitcase, or that your ticket back listed a non-existent terminal at Atlanta airport.
<adeuring> kfogel: you didn't noticed my other suitcase!
<kfogel> adeuring: that's because that one didn't exist at all!
<kfogel> .oO (sheesh)
<adeuring> kfogel: bullshit. I travel _always_ with two suitcases!
<adeuring> though one most times invislibke
<adeuring> ...invisible
<kfogel> Now you're talking crazy, clearly.
<adeuring> well, I'm from a town that doesn't exist -- what do you expect ;)?
<jtv> mthaddon, if you're still here: could you (a) stop for the day or (b) give me another import run on staging?  :-)
<jml> g'night folks.
 * jml will be back later for kiwi chat!
<mwhudson> morning jml
<jelmer> hello mwhudson
<mwhudson> jelmer: hi
<kfogel> adeuring: http://paste.ubuntu.com/377042/
<kfogel> adeuring: background: I realize that using the same patch batcher as we use in the main +patches view is not a good long-term solution; I was just trying to get something working, and then to refine to somehow show the "top 5" youngest patches or something in the portlet.
<kfogel> I failed at the "get something working" stage, as you can see :-).
<lifeless> morning morning people
<adeuring> kfogel: that's because lp.registry.browser.product.ProductView does not define batchedPatchTasks . You should use the view class that has this method in the ZCML attribute 'class="..."'
<adeuring> kfogel: and sorry for the late answer. was out to buy some food
<kfogel> adeuring: no problem -- it's already late for you!
<kfogel> adeuring: edits to .zcml files require restart or no?
<kfogel> adeuring: it seems to have been working without restart...
<kfogel> adeuring: ...but I wonder if I'm just getting lucky or something.
<adeuring> kfogel: I think youmust restart the server
 * kfogel winds up the mouse
<beuno> rockstar, thumper, did you guys know that branch icons seem to be missing on edge?
<thumper> beuno: yes, bug has been filed
<thumper> rockstar: ping?
<kfogel> adeuring: it must be late there -- curious, how much longer you plan to be on?
<adeuring> kfogel: perhaps an hour or so
<kfogel> adeuring: *nod*  thx
<rockstar> thumper, hi.
<leonardr> mars, can you help me with a windmill test failure?
<kfogel> adeuring: oh wow, while adding some more test data, I just got this great confirmation box:
<kfogel> This file does not look like a patch. What is a patch?
<kfogel> Is this file a patch:
<kfogel>  yes
<kfogel>  no
<kfogel> adeuring: that is so awesome
<mars> leonardr, sure, I can have a look.  what is the problem?
<leonardr> mars, http://paste.ubuntu.com/377137/
<adeuring> kfogel: thanks :)
<james_w> I just hit the bug that caused me to propose https://code.edge.launchpad.net/~james-w/launchpad/sync-source-negative-versions/+merge/16861 again, and I was a little surprised to realise that it's been approved for 6 weeks and isn't yet merged, let alone in production, could someone move it along please?
<mars> leonardr, did you try running those tests locally?
<mars> leonardr, bin/test --layer=BugsWindmillLayer -t test_security_settings_form_overlay
<leonardr> mars, trying now
<kfogel> james_w: can you not land via PQM?
<james_w> kfogel: I cannot
<poolie> hello james_w, kfogel
<kfogel> james_w: two questions: one, why not?  (we should fix that?)
<james_w> hi poolie
<james_w> kfogel: I'm not on the LP team
<kfogel> two: I notice the repeat of "if dest_version is None" in the diff... looking closely at the redundancy now.
<kfogel> poolie: hey there
<kfogel> james_w: oy vey
<kfogel> james_w: ok
<poolie> hi kfogel
<james_w> I don't know if that's a rule, but it's why I don't have it currently, and I'm not even sure that I would want the ability :-)
<poolie> james_w: i may see about batch-upgrading those branches soon
<leonardr> mars: i get a different error, but it's one that looks like my fault...
<james_w> thanks
<kfogel> james_w: yeah, looking at the diff and at the full file's source... I can't take a break from what I'm working on right now long enough to puzzle out whether that apparently-repeated conditional makes sense, but it might be worth a second look from you?
<leonardr> mars: here's the local errors:
<leonardr> http://paste.ubuntu.com/377148/
<leonardr> The "could not fulfil proxy request" looks like it could have started failing due to multiversion. could that have caused the other errors?
<james_w> kfogel: it does make sense (at least to me) the first case is for the actual catching of the case where the package isn't in Ubuntu, the second is to not crash trying to do "ubuntu" in None.
<james_w> kfogel: it may be possible to restructure everything to avoid it, but there may well be other duplication if that is done
<kfogel> james_w: if (dest_version is None
<kfogel>     or apt_pkg.VersionCompare(dest_version, source_version) < 0):
<kfogel>         if (not Options.force
<kfogel>             and dest_version is not None and dest_version.find("ubuntu") != -1)):
<kfogel> james_w: (off the top of my head, just for clarity)
<kfogel> james_w: but I see what you mean; it's tricky
<kfogel> adeuring: got a sec for quick discussion?  (might be useful to check out lp:~kfogel/launchpad/255868-link-to-patches-view for this)
<adeuring> kfogel: yes
<kfogel> adeuring: basically, I'm trying to figure out a) the best way, visually, to present this "latest patches" list, and b) how to limit it to the five (or whatever) youngest patches.
<adeuring> kfogel: I am really bad for aestheical things ;)
<kfogel> adeuring: no problem, let's limit it to question (b) :-)
<adeuring> ok ;)
<adeuring> kfogel: To limit the result set, just use something like storm_result_set[:5]
<kfogel> adeuring: right now it will just show batch-size number of bugtasks with patches -- could be 50.
<kfogel> adeuring: aaaaaaaah
<kfogel> adeuring: in the .pt file??
<adeuring> kfogel: it is probably a bt nicer to do that in the view class. But it should also be possible in the template
<kfogel> adeuring: well, the view class is BugsPatchesView, the same view we're using for the full +patches view.  Ah, but maybe I should make a new method in the view, instead of using batchedPatchTasks.  Something like latestPatchTasks or something.  Is that the usual route?
<mars> leonardr, got lost in a context switch, looking
<adeuring> kfogel: yes, that's what I meant
<kfogel> adeuring: thanks, all clear now
<adeuring> great
<leonardr> mars: when i run it locally and watch the browser window i see 2 failures
<leonardr> the first one is that we're expected to be at the login page when we're actually at the bug detail page
<leonardr> and the other is an exception generated by the web service, which i believe is caused by the attempted request to /api/beta/api/devel/
<mars> leonardr, have you tried running it stand-alone, and changing the privacy?
<leonardr> mars, trying that now
<mars> leonardr, don't worry about the first error, that is something in windmill we need to silence
<mars> it does not cause a failure, just a lot of noise
<leonardr> so the ERROR is not the same as a bug failure?
<mars> line 9 is just a loud BANG, line 11 is your branch dying
<leonardr> mars: yes, changing the bug privacy settings POSTs to /api/beta/api/devel/bugs/15
<mup> Bug #15: PO file import errors should be more verbose <feature> <Launchpad Translations:Fix Released by carlos> <https://launchpad.net/bugs/15>
<leonardr> that's definitely my fault but i don't know how it happens...
<mars> how what happens?
<leonardr> how the /api/devel/ gets in there
<mars> client.js
<mars> or base-template-macros.pt
<mars> grep for the string '/api' in *.js and/or *.pt
<leonardr> mars: well, i know that if someone was in fact sending a request to /api/devel it would get munged by LP.client.normalize_uri into /api/beta/api/devel
<leonardr> but no one should be doing that yet
<mars> leonardr, are there any in-page <script> nodes that have JS with the old path in there
<mars> leonardr, looking at base-layout-macros.pt, there is a fmt:api_url call
<leonardr> mars: aha!
<leonardr> that will give the url of the latest version
<leonardr> let me try just changing everything to use /api/devel and see how much damage it does
<mars> most of the JS is driven by variables pulled from on-page script nodes.  So a bad webservice URL probably comes onto the page via template somehow.
<leonardr> mars: attempting to use the devel web service gives incredibly weird results, so i'm going to hack fmt:api_url
<leonardr> the weird results:
<mars> leonardr, heh, ok
<leonardr> (Pdb) self.entry.private = True
<leonardr> *** AssertionError: Expected int for Person foreign key reference, got <type 'object'>
<mars> you should be able to capture that with pdb
<mars> leonardr, I need to sign off for tonight.  Please ping me tomorrow if you need more help debugging the templates
<leonardr> mars, sure
<leonardr> i think i will just come back to this tomorrow as well, i've been working all day
<noodles775> Hey thumper, mwhudson, jml: is 22UTC still ok? (ie. in 5mins)
<thumper> noodles775: good for me
<mwhudson> noodles775: sure
<thumper> noodles775: skype?
<noodles775> thumper: yeah, absoludity is my id.
<thumper> noodles775: lets try a conf call
<thumper> noodles775: skype still failing for me on a conf call
<mwhudson> that all sounded fine to me
<thumper> noodles775: can you try hosting?
<mwhudson> i can host
<thumper> either or, I don't care
<noodles775> OK, otherwise voip is fine too.
<mwhudson> or noodles775
 * noodles775 hosts
<mwhudson> ah
<wgrant> win 23
<wgrant> Argh.
<lifeless> lose 45
<mwhudson> noodles775: ...
<wgrant> lifeless: Indeed.
<mwhudson> noodles775: are you trying to host too
<mwhudson> ?
<mwhudson> i've just started a call...
<noodles775> yes, I did.
<noodles775> OK, add me in :)
<mwhudson> jml: poke
<jml> mwhudson, back online
<jml> please invite
<mwhudson> jml: am doing
<noodles775> thumper: https://dev.launchpad.net/BuildBranchToArchiveUI/UseCaseManualBuild
<noodles775> thumper: https://dev.launchpad.net/BuildBranchToArchiveUI/UseCaseComplexRecipe
<mwhudson> skype fail
<mwhudson> or possibly new zealand internet fail
<noodles775> https://edge.launchpad.net/+apidoc/#distribution_source_package
<thumper> :(
<mwhudson> thumper: yay
<mwhudson> skype
<thumper> mwhudson: you dropped too?
<mwhudson> thumper: no, just you
<poolie> mwhudson: you legend (re upgrades)
<jml> noodles775, stupid question, but which mockup are you looking at right now?
<noodles775> https://dev.launchpad.net/BuildBranchToArchiveUI/UseCaseManualBuild
<noodles775> https://dev.launchpad.net/BuildBranchToArchiveUI
 * wgrant loves LP_PERSISTENT_TEST_SERVICES. Sub-20s test runs are actually getting back into the realms of practicality.
<thumper> wgrant: what is that?
<wgrant> thumper: As of a few days ago, setting that will keep librarian and memcached running between test runs.
<thumper> hmm...
<wgrant> thumper: So you save the entire startup time.
<thumper> wgrant: and how do you set it?
<wgrant> export LP_PERSISTENT_TEST_SERVICES=1
<thumper> wgrant: and how do you tell it when you're done?
<wgrant> No idea.
<thumper> wgrant: where is it documented?
<wgrant> thumper: I only know of it from seeing devel r10273 fly past.
<wgrant> Ah, bin/kill-test-services
<poolie> sheesh, you guys need to talk more
<wgrant> Sssh.
<mwhudson> poolie: i can supply stupid shell scripts at little notice!
<thumper> mwhudson: do you have any feel for how long it is taking?
<mwhudson> thumper: i think it's probably about 5% done?
<thumper> mwhudson: through 3k?
<mwhudson> ish
<mwhudson> maybe less
<mwhudson> "a couple of days"
<mwhudson> particular as i'm going to have to kill it and go into town in a minute
<mwhudson> (i don't have an ssh key on devpad)
<thumper> screen FTW
<thumper> I created an SSH key for devpad :)
<mwhudson> doesn't help with agent forwarding
<mwhudson> thumper: well, in that case...
<thumper> mwhudson: how soon are you going?
<thumper> mwhudson: I have a trivial bug fix :)
<mwhudson> thumper: kinda now ish
<thumper> mwhudson: I'll submit it normally and you can look later :)
<mwhudson> thumper: want to run "cp ~mwh/.bazaar/plugins/l.py ~/.bazaar/plugins/"
<wgrant> That's a descriptive name.
<mwhudson> and then "for i in `cat ~mwh/non-2a-branches`; do echo $i; date; bzr upgrade lp:$i ; bzr flibble lp:$i ; done"
<mwhudson> thumper: on devpad?
<thumper> flibble?
<mwhudson> wgrant: the command name is even better!
<mwhudson> yes, flibble
<wgrant> mwhudson: Indeed.
<thumper> heh
<mwhudson> it write locks and then unlocks the branch
<mwhudson> thus triggering the puller to run
<wgrant> Isn't there a convenient in-LP method to do the upgrade now?
<thumper> wgrant: yes (ish)
<mwhudson> not as convenient as a stupid shell script
<thumper> mwhudson: is non-2a-branches ordered corrently for upgrades?
<thumper> mwhudson: also what about the ones that are done already?
<mwhudson> thumper: doesn't matter
<thumper> ok
<mwhudson> thumper: upgrade will just upgrade the repo if it can't open the branch
<mwhudson> thumper: ones that are done already will cause "already at most recent format" type errors
<mwhudson> but well, that's not really a problem
<thumper> mwhudson: running now
<thumper> mwhudson: bzr: ERROR: RemoteRepository(bzr+ssh://bazaar.launchpad.net/~ubuntu-branches/debian/squeeze/autoconf-nonfree/squeeze/.bzr/)
<thumper> is not compatible with
<thumper> CHKInventoryRepository('bzr+ssh://bazaar.launchpad.net/~ubuntu-branches/debian/sid/autoconf-nonfree/sid/.bzr/repository/')
<thumper> different serializers
<mwhudson> yeah
<thumper> why?
<mwhudson> doesn't actually seem to matter in the end though :-)
<poolie> thumper: known bug about cross-format stacking
<mwhudson> dunno
<poolie> sucks
<poolie> feel free to bump up the bug
<thumper> what is the issue around it?
<poolie> bzr doesn't support stacking across branches of varying formats
<poolie> and lp configures package branches to be stacked on each other
#launchpad-dev 2010-02-16
 * mwhudson afk for lunch
<Ursinha> jtv, no I don't :/ I'm off for carnival
<Ursinha> jtv: if you send me one email with the problem, I can take a look when around
<jtv> Ursinha: hi!  Never mind, I found the cause.  Enjoy!
<thumper> Ursinha: damn, you're off aren't you
<thumper> Ursinha: when are you back?
<thumper> poolie: we currently report format 2a badly on the branch pages
<Ursinha> jtv: heyyy I thought you weren't here :)
<thumper> poolie: what should we say?  Just "2a"?
<jtv> Ursinha: libpqxx release tonight
<poolie> yes
<Ursinha> thumper: yes, I am, but should return on Wed.
<Ursinha> jtv: oh, I see
<thumper> Ursinha: your wed not mine right?
<Ursinha> thumper: yes :)
<thumper> hmm...
<thumper> ok
<Ursinha> thumper: what can I do
<Ursinha> for you
<thumper> I'll have another team lead thingy not done
<thumper> Ursinha: I was going to talk to you about only marking things for qa when available on edge or staging
<Ursinha> thumper: go ahead, what is it
<Ursinha> hmm
<thumper> Ursinha: rather than when in stable/db-stable
<Ursinha> thumper: hm, right
<Ursinha> thumper: so devel/db-devel, right?
<thumper> Ursinha: no... but only tagging for qa when it is possible to qa
<thumper> Ursinha: right now I see things that need to be qa'ed but can't because they aren't on edge or staging
<thumper> Ursinha: it would be good to only get the notification to QA when you can
<thumper> Ursinha: that way it is less likely to be put into the "later" basket, and done ASAP
<Ursinha> thumper: I thought that when a branch reached stable, it could be QAed
<thumper> right now I need to poll edge or staging to see if I can QA
<thumper> Ursinha: no...
<Ursinha> thumper: :/
<thumper> Ursinha: it has to be rolled to edge
<Ursinha> thumper: right
<Ursinha> thumper: I'd have to find a way to check if the revision is already there
<Ursinha> shouldn't be that hard..
<wgrant> There's a script for thatâ¢
<Ursinha> but it would demand some work, and I wonder how would I fit this in the way tagging script work
<Ursinha> wgrant: yes
<wgrant> utilities/on-edge
<thumper> Ursinha: if your script grabbed the branch and opened it locally, you could use bzrlib to check the ancestry
<Ursinha> wgrant: hmm
<Ursinha> wgrant: cool, thanks
<Ursinha> thumper: so I guess it's not hard to do
<thumper> Ursinha: that would be really cool
<Ursinha> thumper: I can do that when I return
<thumper> Ursinha: I'll send you a chocolate fish when its done :)
<Ursinha> thumper: hehe :)
<Ursinha> thumper: I thought the script was behaving like that, only showing items to test already "testable"
<Ursinha> I'll fix that
<thumper> Ursinha: awesome, ta
<Ursinha> jtv: I heard you're coming to visit me here :)
<Ursinha> thumper: np :)
<jtv> Ursinha: so it seems
<wgrant> >>> sprb.buildstate
<wgrant> <DBItem BuildStatus.FULLYBUILT, (1) Successfully built>
<wgrant> Yay.
 * mwhudson finally notices the icon screwage on edge
<wgrant> mwhudson: Just the branch sprite, or something else too?
<mwhudson> probably just the branch sprite
<wgrant> It's odd that only that one has been hit.
<wgrant> Do I want to call attributes source_package_recipe_build or sourcepackagerecipebuild?
<wgrant> There is no consistency.
<mwhudson> well
<mwhudson> the old way is sourcepackagerecipebuild
<mwhudson> the new way is source_package_recipe_build
<mwhudson> soyuz is old
<wgrant> That's what I thought. Thanks.
 * thumper afk for a bit
<thumper> mwhudson: I'd like to talk some more about recipe stuff if you are up to it
<thumper> I find that talking helps clear my mind on the issue
<thumper> or...
<thumper> I could just finish that other branch...
<thumper> hmm...
 * thumper forgets about recipes for a bit
<thumper> ...
<thumper> mwhudson: I could do with a mid-impl call if you have some time :)
<mwhudson> thumper: mid-imp sounds fine
<mwhudson> thumper: skype me up
<mwhudson> thumper: http://jcalderone.livejournal.com/tag/sixty%20seconds
<wgrant> stub: Regarding the patch to drop SourcePackageRecipeBuildUpload, I get a test failure in the person visibility consistency checker. If I drop the foreign key from SPRBU -> Person.id, it's all OK. Can I do that?
<stub> wgrant: Yes, that is good.
<wgrant> stub: Thanks.
<mwhudson> noodles775: hello again (!)
<noodles775> Hi mwhudson :)
<wgrant> Morning noodles775.
<noodles775> Hiya wgrant.
<thumper> hi noodles775
<thumper> noodles775: is your birthday in july?
<noodles775> thumper: nope - august... why so?
<thumper> noodles775: I was wondering about the 775
<thumper> noodles775: and I though July '75
<thumper> t
<jml> good hello
<noodles775> Ah, just noodles was already taken, not sure why I added th eextra 7.
<noodles775> Morning jml
<jml> a truck with flatpack bookshelves is arriving any minute now.
<noodles775> Yay.
<lifeless> hi jml
<jml> lifeless, hello
<wgrant> Hmm, is BjÃ¶rn doing DB reviews now?
<jml> wgrant, yes. was the email not sent to the public list?
<wgrant> jml: Not that I saw.
<jml> wgrant, sure. anyway, yes, BjÃ¶rn is doing db reviews.
<wgrant> jml: OK, thanks. Was just a bit confused by the extra review request that sprung up on one of my MPs a couple of days ago.
<jml> wgrant, np.
<adeuring> good morning
<jml> good morning
 * jml has flatpack shelves now
<wgrant> But can you put them together?
<jml> well, first step is getting them upstairs
<jml> putting them together... I've done it before
<thekorn> allenap, hi, have you seen the test failures? I would like to help fixing them, but I've no clue about the correct fix (implementing this method by returning an empty list vs. moving this method to something like IHasTags)
<mrevell> Morning
<jml> mrevell, good morning
<mrevell> Guten morgen jml
<wgrant> Why doesn't +activereviews show the number of pending reviews?
<jml> don't know.
<noodles775> Do we have any examples of where we've presented disabled form widgets on LP? I can't find any...
<noodles775> I can see that zope.scheme items can take an optional constraint, but that is validation related, whereas I'd like to present a checkbox that is disabled in some situations.
<noodles775> (setting readonly=False causes the field to render as a normal text field, rather than a checkbox).
<noodles775> If not, I can create a new widget to do so, but just wanted to check first.
<jml> noodles775, I can't think of any.
<wgrant> The only disabled widgets I can think of are the textboxes on the code import registration view, but that uses BeautifulSoup...
<noodles775> OK, thanks guys.
<wgrant> Is this for the Archive.private restriction?
<noodles775> wgrant: yep.
<noodles775> wgrant: just for the ui presentation of it of course - it needs to be read only on the model with a validator there.
<wgrant> Right.
<deryck> morning, all.
<adiroiban> danilos: hi, can you please advise my how I should process with bug 516317 ? Should I fix it together with bug 127171, create a new branch, or wait for you to fix it?
<mup> Bug #516317: Product:+changetranslators should become +settings <ui> <Launchpad Translations:In Progress by danilo> <https://launchpad.net/bugs/516317>
<mup> Bug #127171: Rosetta experts not allowed to "Change translators" <Launchpad Translations:In Progress by adiroiban> <https://launchpad.net/bugs/127171>
 * jml offline
<danilos> adiroiban, hi, I have a branch which fixes it and a few other issues at the same time, it's almost ready
<danilos> adiroiban, I believe it fixes 127171 as well :)
<adiroiban> danilos: ok. I'll wait for that
<adiroiban> :)
<danilos> adiroiban, I am renaming it 'translation-settings' because I am also moving 'official_rosetta' setting there so it's easier to configure a project for translation in one place
<danilos> adiroiban, if you are impatient and have a mostly ready branch, you can land it and let me worry about solving all the conflicts and everything else :)
<adiroiban> no problem
<adiroiban> I don't think it is worth landing it
<danilos> adiroiban, if it's done, it's worth landing it because I haven't ran the full test suite on my branch yet :)
<adiroiban> but maybe you can reuse some of the tests
<danilos> adiroiban, and, it's a work you invested time in, and I always appreciate that :)
<adiroiban> i think that landing it will only create more work for you
<adiroiban> so maybe you can reuse some of the tests from there
<adiroiban> danilos: it is OK if I will assing you the bug 127171 ? It is now assigned to me and targeted for the next milesone.
<mup> Bug #127171: Rosetta experts not allowed to "Change translators" <Launchpad Translations:In Progress by adiroiban> <https://launchpad.net/bugs/127171>
<danilos> adiroiban, yeah, it'll probably help
<danilos> adiroiban, iow, just go ahead and land it :)
<adiroiban> danilos: I'm confused. Should I land it, or assing it to you and leave you the task of fixing this bug?
<danilos> adiroiban, land it :)
<danilos> adiroiban, I'll worry about other things
<adiroiban> ok
<jtv> adiroiban: shame we didn't have time to talk more at fosdem...  I introduced Milo to a new snack, and if you don't know them yet, I'll get you one next time.  :)
<adiroiban> :)
<jtv> abentley: is there a way to get a branch URL, but with a preference to (say) a read-only http URL?
<jtv> a preference *for
<abentley> jtv, you can, but you might have to hand-craft it.
<jtv> abentley: this is for hosted branches btw... alternatively, can I predict what protocol I'll get?
<jtv> (and will read-only URLs go away completely?  This is untrusted access so read-only would be a nice touch)
<abentley> jtv, the protocol you get is entirely up to you.
<jtv> abentley: me the branch owner or me the script that wants to read the branch contents?
<abentley> you, a launchpad developer.
<jtv> So rewriting the branch URL could be messy.  (BTW I just realized: the branch may well be mirrored instead of hosted)
<abentley> jtv, I don't understand what you're trying to do.
<abentley> jtv, I don't understand whether you're writing a cron script or a script that accesses Launchpad over the API
<abentley> jtv, so I really don't understand what answers are relevant to you.
<jtv> abentley: I'm writing the code that produces a BuildFarmJob for a given Branch object from the db.  Based on that BuildFarmJob, the slave will have to export the branch contents and operate on them.
<jtv> The thinnest interface I can think of for doing that is to pass a branch URL to the slave, after which the slave can read branch contents from that URL.
<jtv> I'm not sure the warehouse URL would be good to expose to a slave; the slave is not trusted.
<abentley> jtv, this will only work for branches which support anonymous access.  Is that acceptable?
<jtv> abentley: yes
<abentley> jtv, so you can just use the http URL.
<jtv> abentley: yes, that's what I was thinking; makes it easier to set up the firewall rules.
<jtv> abentley: is there a reliable way to get that url?
<abentley> jtv, over http, there's no distinction between mirrored and hosted branches, and the URL rewriting happens elsewhere.
<abentley> jtv, I don't know if there's a way to construct an http url directly, but if not, you can just prepend "http://bazaar.launchpad.net" to the bzr_identity.
<jtv> abentley: would it make sense to add, say, an http_url property to IBranch?
<abentley> jtv, I don't think so.  Branches have lots of URLs, even multiple http URLs.  A public_url method might make sense, though.
<jtv> abentley: as a compromise, public_http_url?  I know it seems overly specific but the "http" part matters for our firewall setup.
<danilos> jtv, abentley: you'd definitely want to involve someone who knows our production config into this discussion, since I know we'll at least have to open firewall rules for this
<jtv> danilos: yes, we know, thank you  :-)
<abentley> jtv, no, our API is fat enough.  I'd want a method that can produce sftp, bzr+ssh, or http URLs.
<jtv> abentley: so... a method that takes a protocol parameter, and which for now only supports http?
<abentley> jtv, I can't think of a reason to only support http.
<jtv> abentley: just that I don't want to write, and you don't want to maintain, something that nobody uses yet.  :)
<abentley> jtv, I am quite happy to maintain something like that, considering it's a 7-character difference at most.
<abentley> jtv, I would even be willing to update our existing code that generates URLs to use it.
<jtv> abentley: in that case, how about you tell me the URL patterns, I write a branch, and you review it?
<jtv> Or is that too incestuous?
<abentley> jtv, works for me.
<jtv> Actually, if you just tell me other places that compose URLs like this, I'll have the patterns too :-)
<abentley> jtv, looking.
<abentley> jtv, lp.code.xmlrpc.branch is the bit that actually converts an lp: URL to a public URL.
<jtv> (the "os.path.join" is going to do funny things with the URLs if we ever port to Windows :)
<abentley> jtv, that may be the only place we do this conversion.
<jtv> _getUniqueNameResultDict?
<abentley> jtv, yes.
<jtv> so we can push the core of that method down into the model code.
<abentley> jtv, yes, that would make sense.
<jtv> abentley: thanks, I'll get to work on that then.  At last a good opportunity to start playing with pipelines.
<abentley> jtv, cool.
<abentley> jtv, I recommend avoiding reconfigure-pipeline for launchpad developers.  Instead, just create a branch as you normally would, then make a lightweight checkout and work in that.
<jtv> abentley: do I push the lightweight checkout to bzr separately to set up an MP?
<jtv> *to launchpad
<jtv> not to bzr
<abentley> jtv, no, pushing a lightweight checkout is the same as pushing its branch.
<abentley> jtv, you can also use lp-submit to set up the MP.
<jtv> abentley: I've already got a branch with committed changes; does this approach still help in that case?  Or would I have to merge my existing branch into the new one?
<abentley> jtv, still works.  Just create a lightweight checkout of your existing branch, and you can start using pipelines.
<jtv> abentley: oic
<jtv> abentley: the plugin complains about an API version mismatch.
<abentley> jtv, which version of bzr are you using?
<jtv> 2.0.4
<abentley> jtv, and you're using bzr-pipeline from source?
<jtv> abentley: yes, did a "bzr branch" into my plugins dir & renamed to bzr_pipeline
<abentley> jtv, you need to use lp:bzr-pipeline/stable
<abentley> jtv, that version of pipeline doesn't contain the lp-submit command.
<jtv> bzr: ERROR: No module named pipeline.commands
<jtv> You may need to install this Python library separately.
<abentley> jtv, you should rename it to pipeline, not bzr_pipeline.
<noodles775> intellectronica: did we ever standardise on a way to ensure that model validation can be re-used in browser views?
<jtv> abentley: thanks, that works... how do I submit when ready?
<noodles775> I have a memory of you bringing it up at one point.
<intellectronica> noodles775: i must admit i'm not sure what exactly you're on about
<noodles775> intellectronica: sorry. So you've got a model validation that storm uses to ensure a new value is consistent before it adds it to the db.
<noodles775> And you want to do a very similar check in your views form validation.
<abentley> jtv, use the web UI, and if there's a previous pipe, specify it as the prerequisite branch.
<jtv> abentley: right ho
<noodles775> intellectronica: perhaps I mis-remembered, I thought you had brought this up once before (related to API validation checks).
<intellectronica> noodles775: is this a check you can do without storm? if it is, then you can use a validator with the interface attribute, and use the interface (or specializations of it) for both the model object and the form
<noodles775> intellectronica: yep, we're using the same IFace for the form schema and the model, but I'm not sure what you mean by a validator with the interface attribute.
<noodles775> Currently the model validation is done via the storm_validator attribute of BoolCol.
<intellectronica> noodles775: iirc you can pass validator function to the constraint argument of zope interface fields
<intellectronica> let me see if i can find an example
<noodles775> Thanks.
<intellectronica> noodles775: see some validators in lib/canonical/launchpad/interfaces/validation.py
<noodles775> Thanks intellectronica.
<noodles775> I did look at constraints earlier, but from memory it was only allowing a constraint on the new value... whereas I need to compare it with the old. But I'll check there.
<noodles775> intellectronica: right, so the fact that I need storm to my check means I can't use a constraint. I'll keep looking, ta.
<sinzui> mars: target questions to launchpad-registry because we have answer contacts. Assigning them to the registry-team does not generate a notification.
<mars> sinzui, will do, thanks
<leonardr> flacoste, do you think it's okay to change client.js so that the launchpad website uses the 'devel' version of the web service when making ajax requests?
<mars> flacoste, leonardr, fwiw I think it is OK for the JS to use the latest webservice.
<flacoste> leonardr: agreed
<mars> we can decouple them later if (and only if) we have problems with API drift
<mars> err, problems with rapid API change
<leonardr> cool
<mars> (and we would be trading rapid API change for API drift problems.  Anyway, I'll take the former for now)
<leonardr> flacoste: as i was telling mars and gary, my only concern is that our test coverage might not be good enough to detect ajax failures caused by api changes
<leonardr> but i don't truly know how good it is
<mars> leonardr, I think it is good enough
<gary_poster> flacoste: I feel this is a call for BjornT, ideally, but he is not around.  I asked leonardr to ping you because, as he says, this is an assertion that our Windmill tests are pretty good, or that we are willing to experience pain until they are.
<gary_poster> I don't know the Windmill tests.
<flacoste> windmill tests are fine
<gary_poster> flacoste: ok, good enough. leonardr ^^^ that's explicit :-)
<leonardr> all right
<leonardr> we'll do it
<gary_poster> thanks
<leonardr> mars, do i have to do anything special to rebuild the *-min.js files?
<jtv> abentley: do I need to check whether the branch is public?
<jtv> abentley: also, is it appropriate to use the same list of accepted schemes that the url field accepts?
<abentley> jtv, I think that's a nice-to-have.
<jtv> abentley: when it's not public, there's basically no public URL right?  I could return None, or raise, or just pretend things will work.
<abentley> jtv, I don't know what list of schemes the URL field accepts.
<jtv> abentley: it's in IBranch: BranchURIField(..., allowed_schemes=['http', 'https', 'ftp', 'sftp', 'bzr+ssh'],
<jtv> OTOH that doesn't include lp, so maybe it's not quite the same thing..?
<abentley> jtv, that list is not the list of schemes we support for codehosting.
<sinzui> mars: answers have history, you do not need to write dates in white boards: https://answers.edge.launchpad.net/launchpad-registry/+question/101172/+history
<jtv> abentley: it'd be easiest for me to accept any protocol string at all.  Maybe I'll just do that.
<abentley> jtv, I think raising an exception would make sense for non-public branches.
<abentley> jtv, I think accepting any scheme is okay.  We can tighten it later if needed.
<abentley> jtv, but it only makes sense to raise an exception for http, when the branch is private.
<jtv> abentley: pidgin died.  You said the non-private check only makes sense for http.
<abentley> jtv, yes, that was the last thing I said.
<jtv> abentley: https..?
<abentley> jtv, either the API generates URLs that work with LP, in which case, https should be forbidden altogether, or it generates whatever URL you request, and the private check doesn't make sense.
<jtv> abentley: for my use case, either kind of function would do.  Right now I'm thinking "does whatever you want, but asserts that you're not making obvious mistakes."
<abentley> jtv, to my mind, requesting an https URL is an obvious mistake.
<jtv> abentley: that's fine; I just wanted to know if that was something that was supported.
<abentley> jtv, at present, only http, sftp and bzr+ssh are supported.
<jtv> abentley: so that's the ones the public codehosting API supports, plus sftp.
<abentley> jtv, yes.  sftp is basically a legacy service now.
<jtv> abentley: so it may make sense to crib my list from PublicCodeHostingAPI with sftp in a "grandfather clause" appendix?
<abentley> jtv, yes, that would make sense.
<danilos> jtv, abentley: I am again jumping into the discussion, but have private branches been considered? atm, we just need to be careful not to expose data from them publicly
<danilos> abentley, ah, I see it has
<danilos> jtv, abentley: ok, ignore me, thankyouverymuch :)
<jtv> danilos: and this affects our existing plans how..?  :-p
<james_w> jelmer: hey, did your delete branches API ever land?
<jelmer> james_w: yep, last cycle
<james_w> jelmer: I don't see it on https://edge.launchpad.net/+apidoc/#branch
<james_w> or is it implicit?
<jelmer> Hmm, I don't see it either
<abentley> rockstar, chat?
<rockstar> abentley, sure, lemme find my headset in this mess.
<rockstar> abentley, I don't see you on skype.
<rockstar> abentley, I heard you at first, then I didn't anymore.
<abentley> I'm here
<rockstar> abentley, I heard you again, and then you dropped off.
<abentley> rockstar, making a test call.
<abentley> rockstar, test call worked.
<rockstar> abentley, yeah, as did mine.
<abentley> rockstar, grr.  Maybe xmpp chat...
<kfogel> adeuring: mystery... when I view source for (say) https://bugs.edge.launchpad.net/gwibber, the bugfilters-portlet stats box on the right (which says there are "489 Open bugs", according to my browser) doesn't contain the string 489 anywhere.  That number appears nowhere in the HTML source.  I do see where the "Open bugs" is, but... what's going on?  Where are the numbers?
<kfogel> adeuring: where I expect the number 489 to be, there is just this: <td class="bugs-count"></td>
<kfogel> it's empty
<adeuring> kfogel: that's pulled in via an extra HHTP request
<kfogel> adeuring: cool (because we'll want to do that for patches count too, I think)
<adeuring> kfogel: we had lots of timeouts, when this data was directly in the HTML data, so we get the data for some portlets later
<kfogel> but how does this work?
 * kfogel looks
<adeuring> kfogel: are you sure that conting patches is also so slow?
<adeuring> s/conting/counting/
<kfogel> adeuring: well, no I'm not sure... but if the rest of the portlet already does things this way, it would seem like a good idea to do that here too
<kfogel> adeuring: (I also have a good guess that it is as slow as any other thing in that box, though I could be wrong)
<adeuring> kfogel: ah, right.
<kfogel> adeuring: I'm puzzled by how the content is fetched asynchronously, though.  There doesn't appear to be any unique ID in the HTML identifying the particular <td>...</td> where (say) the open bugs count goes, or the critical bugs count, or any of the others.
<kfogel> How does the content get put in the right place?
<adeuring> kfogel: the portlet data is pulled from LP via Javascript. Look for lines like "Y.on('domready', function()"
<adeuring> kfogel: or "Y.one('#bugfilters-portlet-content').set(..."
<kfogel> adeuring: oh, it replaces the *whole table* ?
<adeuring> kfogel: I think the portlets are empty without the JS generated stuff
<kfogel> adeuring: well, according to the source they would have the names, just not the counts.  let me try turning off js to see
<kfogel> yup
<kfogel> adeuring: yup -- try it.  the field names are still there, just no data (which makes sense -- because even without the data, at least you can click the link to get to all open bugs or whatever)
<kfogel> so it's robust: even when no js, it's still useful
<kfogel> this is great, I think I know what to do then
<kfogel> thanks
<adeuring> kfogel: just add your data to the prortlet "page"
<kfogel> adeuring: yup
<adeuring> the URL is gwibber/+bugtarget-portlet-bugfilters-stats for your example
<adeuring> kfogel: that's plain JSON data; tehre is probably not even a template for it.
 * adeuring is talking nonsense. This is HTML data, not JSON...
<kfogel> adeuring: lib/lp/bugs/templates/bugtarget-portlet-bugfilters-content.pt
<adeuring> kfogel: right
<kfogel> adeuring: I'm noticing that some of the other items in that box are conditional, but in a weird way.  For example:
<kfogel> <tr tal:condition="view/pending_bugwatches_url">
<kfogel> adeuring: why would something be conditional on the... URL?
<kfogel> I could understand being conditional on the pending_bugwatches_count, but on the url??
<kfogel> I mean, the URL is true no matter what, right?
<adeuring> kfogel: well, yes... but it is generated in the view class, and for whichever reason, it seems that it can be None the empty string or whatever. Don't ask me why ;)
<kfogel> adeuring: Okay, I won't ask you why, but I'm not going to imitate it either (seeing no reason to).
<adeuring> kfogel: yeah, that sounds sane ;)
<adeuring> kfogel: the docstring in the view class has a bit of an explanation: For certain bugtarget, we don't want to or cannot show this sort of data.
<adeuring> I am not sure if it is that much better to use the numbers insteand as a flag...
<adeuring> We may want to show the number zero
<kfogel> adeuring: I think I'm going to go with showing zero.  UI-wise, that's a bit less confusing to users who are expecting the item to be there.
<adeuring> right
<kfogel> adeuring: you're in class BugsStatsMixin in bugtask.py, right?
<adeuring> kfogel: i think it is BugsInfoMixin
<kfogel> adeuring: hmm, right below that is BugsStatsMixin which inherits from BugsInfoMixin
<kfogel> the former has the URLs, the latter has the counts
<adeuring> kfogel: ah, right
<kfogel> adeuring: but for the URL, I'm just appending "+patches" to the context url already
<kfogel> adeuring: no need for a specialized url function here, right?
<adeuring> kfogel: no, I dont't think so. Unless the portlet is displayed for some sort of bug target where we don't have the +patches view.
<kfogel> adeuring: AFAIK that is not the case
<adeuring> kfogel: right
<kfogel> (and if it were the case, the right fix would probably be to implement +patches there anyway!)
<kfogel> adeuring: I'm removing the "story-patch-report" tag from https://bugs.edge.launchpad.net/malone/+bug/172501
<kfogel> does that seem reasonable?
<mup> Bug #172501: reject non-code patch attachements <patch-tracking> <story-patch-report> <Launchpad Bugs:Fix Committed by adeuring> <https://launchpad.net/bugs/172501>
<kfogel> adeuring: that will leave the "patch-tracking" tag there
<adeuring> kfogel: that's OK. I also wnated to checkk if I indeed fixed that bug already. Cant remeber....
<kfogel> adeuring: fix committed
<adeuring> kfogel: argh, right...
<adeuring> confised it woth another bug...
<adeuring> ...confused it...
<kfogel> adeuring: doing the same thing with other bugs that aren't about the +patches view per se.  for example
<kfogel> https://bugs.edge.launchpad.net/malone/+bug/283941
<mup> Bug #283941: Upstream report should show which reports have patches <patch-tracking> <story-patch-report> <Launchpad Bugs:In Progress by adeuring> <https://launchpad.net/bugs/283941>
<adeuring> kfogel: sure, np
<kfogel> deryck: I just ran across this tag "story-patch-report-external" on bug https://bugs.edge.launchpad.net/malone/+bug/403443
<mup> Bug #403443: bug watch should notify when a patch is attached to an upstream bug <bugwatch> <patch-tracking> <story-patch-report-external> <ubuntu-upstream-relations> <Launchpad Bugs:Triaged> <https://launchpad.net/bugs/403443>
<kfogel> deryck: is that a tag we're trying to use consistently?
<deryck> kfogel, on phone.  will explain shortly.  sorry
<kfogel> deryck: I might change to "patch-tracking-external", and put it on bug 283941 as well.
<mup> Bug #283941: Upstream report should show which reports have patches <patch-tracking> <Launchpad Bugs:In Progress by adeuring> <https://launchpad.net/bugs/283941>
<kfogel> deryck: okay, will wait, np
<deryck> kfogel, off now. sorry again.
<deryck> kfogel, so the history there...
<kfogel> deryck: listening :-)
<deryck> kfogel, was that jorge had two lines of development he was wanting -- exposing patches LP knows about and getting data about patches in other places (i.e. "external") into the LP patches reports....
<deryck> kfogel, so that tag was my way of trying to store those kind of second phase, external-related bugs and not lose them
<deryck> kfogel, you can use or discard it as you see fit now.
<kfogel> deryck: I'm going to change it to patch-tracking-external
<kfogel> deryck: and reserve "story-patch-report" for stuff related to the +patches view, which is 90% of how we were already using that tag.
<deryck> kfogel, sounds good
<kfogel> deryck: tagging question: in a bug that's really just an XXX cleanup bug related to an existing story (story-patch-report), do we give it the story-patch-report tag?  I'm thinking of https://bugs.edge.launchpad.net/malone/+bug/515584
<mup> Bug #515584: BugsPatchesView:batchedPatchTasks() should use a Zope form, instead of validating form values manually <story-patch-report> <Launchpad Bugs:Confirmed for kfogel> <https://launchpad.net/bugs/515584>
<deryck> kfogel, in that case, yes, because I think that bug should be fixed before leaving this work.
<deryck> kfogel, that's not hard to do so we shouldn't let that debt linger.
<kfogel> deryck: *nod*
<kfogel> deryck: and you want to keep all our "story-foo" tags unofficial, right?
<deryck> kfogel, yes.
<kfogel> deryck: ok
<deryck> kfogel, though don't get me started on the official/unofficial distinction and how I hate it. ;)
<kfogel> deryck: :-)
<kfogel> deryck: final tagging philosophy question: most bugs are cleaned up now, but some have *both* the "patch-tracking" tag and the "story-patch-report" tag.  My feeling is, they only need the latter, since we're working on that story right now.  Should I make it so?
<deryck> kfogel, yeah, I think that's fine.  But there's certainly no harm in multiple-overlapping tags.  it is tagging after all.  anyone can put whatever tag they like.
<kfogel> deryck: I'm just trying to get unwanted bugs out of people's faces in search results, more than anything else, but yeah, it's no huge harm either way
<deryck> kfogel, gotcha
<kfogel> deryck: ok, all bugs that ever had "patch-tracking", "story-patch-report", or "story-patch-report-external" are now consistified into a grand system that, if y'all are lucky, I might explain to you someday.
<kfogel> no bugs had "patches-view" or "patch-report" AFAICT
 * jml offline for a while.
<adiroiban> is the last line a valid Python code http://paste.ubuntu.com/377709/ ?
<salgado> adiroiban, yes, it is
<mrevell> night all
<kfogel> adeuring: very puzzling phenomenon: in this diff http://paste.ubuntu.com/377792/ , if I remove the "|nothing" from the tal:content, then when I visit https://bugs.launchpad.dev/patches-view-test I get an OOPS.  But when I leave the "|nothing" in, everything works fine and it shows "6 Bugs with patches" as it should.  If the count is returning 6 (which is correct), then why would the "|nothing" matter at all?
<kfogel> deryck: (you might be able to answer above question I aimed at adeuring)
<deryck> kfogel, you know I don't know.  the is a tal mystery to me.
<kfogel> deryck: ok :-)
<deryck> kfogel, what OOPS do you get without nothing?
<kfogel> deryck: http://paste.ubuntu.com/377800/  (slightly different diff now, but not in any important ways)
<kfogel> It's like bugs_with_patches_count() doesn't exist.
<kfogel> I wonder if this has to do with the double load -- first the
<kfogel> page load and then the javascript to fetch the contents for the stats box.
<kfogel> maybe the view is defined differently each time
<deryck> kfogel, ah, right.  That's likely it.  In one context the method exists, in another it doesn't.
<kfogel> deryck: the only reason I care is that I wanted to do something fancy whereby if it was one patch, it would say "Bug with a patch" instead of the plural "Bugs with patches".  But that's hard to do now because I can't set a tal:condition on a number we don't have yet.  On the other hand, none of the other stats do that, so maybe this new item will just have to sink to the level of its surroundings :-).
<deryck> kfogel, I vote sink. :-)  It will be uniform, too, unless you see a nice way to do it.
<kfogel> deryck: if we did do it, we'd want to do it for everything.  sigh -- I really like plural polish, but it's gonna cost too much right now.  Oh well.
<deryck> kfogel, working back through your statement, I'm not sure I get "a number we don't have yet."  The view rendering the portlet has the number, no?
<kfogel> deryck: well, I'm a little mystified by how this code is succeeding right now.  As you can see, the text "Bugs with patches" (and now "Bug with a patch") is in the .pt file.  That text is still there after the portlet is rendered.  But all tests indicated that there is at least evaluation of the portlet during which the view does not contain the method in question.
<kfogel> then later, it does (because the method returns the correct answer)
<kfogel> but at the time when I do the conditional to evaluate plurality, the method apparently does not exist
<kfogel> whoa
<kfogel> deryck: never mind -- test-o or sometihng on my part
<kfogel> my plurality thing *is* working now
<deryck> ah, ok.  Nice :-)
<kfogel> deryck: so does that mean I get to commit it? :-)
<deryck> kfogel, if it passes review and ec2.... sure. :-)
<kfogel> http://paste.ubuntu.com/377809/  just fyi
<deryck> kfogel, looks good at a glance to me.  I worry about adding another query to this page, but that's not your code's fault.
<kfogel> deryck: ah, just as an aside: this is not something we necessarily want to abstract out either.  Do 'find lib/lp -name "*.pt" -print | xargs grep plural' and you'll see many different ways we handle plurals, and they are often context-specific.
<deryck> kfogel, I was wondering if there was a particular tal'ism for handling this.
<kfogel> deryck: well, the whole point is the portlet loads separately, so the user doesn't really pay the full cost -- they can start using the page before the portlet is done loading.
<kfogel> (responding to your earlier comment)
<deryck> ah right
<kfogel> deryck: lib/lp/app/templates/base-layout-macros.pt:  <tal:plural
<deryck> stupid me
<kfogel> but it wouldn't help me much here, because I have to change the whole phrase
 * deryck looks
<kfogel> oh wait
<kfogel> it might work
 * kfogel reading moer
<kfogel> moer
<kfogel> dang it
<kfogel> "more"
<kfogel> deryck: I think maybe I can use that
<deryck> ok, cool
<deryck> in Django, this is so simple. ;)  {patches_count|pluralize}  and done. ;)
<kfogel> hah, lib/lp/registry/templates/distroseries-needs-packaging.pt is like the *only* place that uses it right now
<kfogel> not even
<kfogel> hmm
<kfogel> deryck: for your entertainment: http://paste.ubuntu.com/377818/  (works)
<deryck> kfogel, nice
<leonardr> flacoste: i just commented on https://code.edge.launchpad.net/~leonardr/launchpadlib/multiversion/+merge/19343 . do you agree?
<flacoste> leonardr: well, an alternative would be to use edge/devel by default, no?
<leonardr> flacoste: do you want devel to be the edge default permanently?
<flacoste> leonardr: that might actually be a good idea
<flacoste> leonardr: anyway, i think this is best decided in conjunction with the Ubuntu release team, they may have sorts of policy that i'm unaware of
<leonardr> flacoste: i'm talking with the release team now. i just came over here to confer with you about this
<thumper> morning
<kfogel> deryck: UI reviews to beuno still, or someone else?
<deryck> kfogel, no beuno is gone now
<deryck> kfogel, noodles is our only ui reviewer now.
<kfogel> deryck: *sob*
<deryck> I know
<leonardr> flacoste: slangasek says "integration with web apps is a small enough use case in the distro that it's not something there's a policy for :)"
<kfogel> deryck: meaning michael.nelson in launchpad, not "noodles" (who is Jonathan McDowell)
<kfogel> deryck: that almost messed  me up for a moment :-)
<adeuring> kfogel: sorry for the late answer.I have not yet a clue why you got the OOPS. But hiding it is via "| nothing" is not the best idea. It hides "serious" exceptions as well, and that makes debugging quite difficult.
<kfogel> adeuring: oh.  I just took my hint from the other code in that .pt... Do you have a recommendation?
<kfogel> I have working code right now; I'm reluctant to break it unless I know how to make it work again :-).
<deryck> kfogel, yeah noodles775 is michael's nick here, I believe.
<adeuring> kfogel: yes, we have places in the templates where "| nothing" is used. But your reviewer might ask you for a good justification to use it in tis case ;)
<adeuring> kfogel: but carping aside, can you tell me the branch where you get this odd OOPS?
<kfogel> adeuring: right now my justification is "because it breaks if I don't do that".  I *think* the reason is that the method doesn't even exist in the view the first time the portlet template is loaded, it only exists the second time (the javascript reload).
<kfogel> adeuring: lp:~kfogel/launchpad/255868-patches-view-from-bugs-page
<adeuring> kfogel: OK. So, then you don't need to load it at all the first time?
<kfogel> adeuring: ...maybe that's what that tal:condition was for, that I was puzzled about earlier!
<kfogel> it's all coming back to me now...
<kfogel>   ...the sunken ship, the diamonds...
<leonardr> flacoste: ok, i have two plans
<deryck> have a nice afternoon/evening all.... I'm out.
<leonardr> flacoste: i've spoken with slangasek and i have two plans
<flacoste> which are?
<leonardr> the plans differ depending on whether we want 'devel' to be the permanent default for the edge service
<leonardr> the restriction placed on my by slangasek is that if there is code (rather than changing defaults) it needs to be in by thursday
<flacoste> ok
<flacoste> and by in, that means an ubuntu upload
<flacoste> can we make this?
<leonardr> if the ec2 test i'm running now succeeds, definitely
<flacoste> ok, you'll need to ping Lucio, the guy that james_w said was maintaining this now
<flacoste> i don't know his irc nick though
<flacoste> or was it Luca
<kfogel> adeuring: no, those tal:conditions are slightly different -- they are to url instead of to count.  (If it sounds like I'm a little mystified by how this thing works, it's because I am.  Trying to figure it out.)
<leonardr> so, plan #1 is to change the launchpadlib trunk so that the default is "beta" instead of "devel" and give that to slangasek/lucio/luca
<leonardr> plan #2 is to do a little more work so that different instances can have different defaults, set the default for edge to 'devel' and the default default to 'beta', and give *that* to the packagers
<leonardr> i like plan #2 but i don't think we have to do it for lucid
<leonardr> in either case, after the freeze, once 1.0 is deployed, we'll change launchpadlib so that the default (global default or everything-but-edge default) is "1.0" and do another release
<adeuring> kfogel: the usage of that template is quite confusing. It is used for +bugtarget-portlet-bugfilters-info and for +bugtarget-portlet-bugfilters-stats. In one case, the view class BugListingPortletInfoView is used, in the other case BugListingPortletStatsView. And only the latter defines the .*count properties.
<adeuring> so you must use this ugly "|nothing" variant.
<flacoste> leonardr: sounds good
<kfogel> adeuring: oooooooooooh
<kfogel> adeuring: I think this maybe wasn't designed for a newcomer to understand immediately
<kfogel> :-)
<adeuring> kfogel: yes; I needed also some time to understand what is going on.
<adeuring> And I think we should file a bug about this craziness.
<kfogel> adeuring: May I ask you to file that one?  I think you will be better able to articulate what is going on (and I will understand better after reading the bug).
<adeuring> kfogel: sure
<leonardr> flacoste, i'm still getting windmill test failures when i run the ec2 tests
<leonardr> not a good sign for my chance of landing this branch soon
<leonardr> but, i have another idea for cheating a little bit
<leonardr> if i make the 'beta' change to launchpadlib, i can verify that it works against a stock launchpad
<leonardr> and do it that way
<leonardr> it doesn't actually require launchpad to be multiversion, only to support 'beta'
 * thumper caffeinates
 * jelmer thinks thumper is onto something
<EdwinGrubbs> sinzui, bac: ping
<sinzui> hi EdwinGrubbs
<EdwinGrubbs> sinzui: hi, I was looking at https://dev.launchpad.net/Registry/UpstreamLinkUbuntu#Project community services (New)
<EdwinGrubbs> and it seems to overlap heavily with the application tabs and with the "Get Involved" sidebar links.
<sinzui> Edwin yes, it does, I think I mentioned that in my notes.
<sinzui> EdwinGrubbs: bac and I are calling this Contributor Connections at the moment
<sinzui> EdwinGrubbs: It is different from Involvement, because the former is about getting information upstream, the other is for hosted features only
<EdwinGrubbs> sinzui: it seems like the tabs should indicate whether that application is active for a project, instead of having to click on the tab or having to look inside a portlet
<sinzui> EdwinGrubbs: There are a lot of bugs about that. How do you intend to resolve the conflict between many communities: off-site-upstream, hacker using a mirror, translators, answer contacts?
<sinzui> EdwinGrubbs: Even if I host my project on launchpad, another user may choose to use answers and translate my app
<sinzui> Do I have the right to tell this person to go away?
<sinzui> In the case of Ubuntu, the answer is No, All the the project maintainer can do is state what he officially uses.
<EdwinGrubbs> sinzui: that sounds like a permission issue, which doesn't seem to help decide where to display the LP/offsite info and edit buttons.
<sinzui> EdwinGrubbs: No, we are still at an impass. The problem is really in the mind of users who choose to host their project in launchpad. Launchpad is not a project hosting service. It is a set of community services that are attracted to projects and teams
<lifeless> errr
<lifeless> someone should tell our staff and users that then
<sinzui> EdwinGrubbs: if there was no barrier between reporting a bug and lp sending it upstream automatically, then we Launchpad would include Bugs in the Involvement portlet when ever it knows where the bug tracker is
<EdwinGrubbs> sinzui: that seems to say, if it is easy to submit a bug, look over here in the UI, but if it is hard, look for that info someplace else.
<sinzui> EdwinGrubbs: Yes. that is fair statement.
<sinzui> EdwinGrubbs: launchpad is successful when it is moving data, when ubuntu bugs do not go upstream, it is a failure. The same is true for bugs in my projects that do no flow upstream
<sinzui> EdwinGrubbs: We offer the Involvement portlet so that maintainers can promote the features that use launchpad for.
<lifeless> surely 'report a bug' should link to the upstream bug tracker if its not tracking in lp
<sinzui> EdwinGrubbs: but communities still need to know where the crucial services are hosted.
<EdwinGrubbs> sinzui: I guess what I'm getting at is that the Involvement portlet could indicate when information is missing that could be filled in.
<sinzui> lifeless: If the bugs team can make that happen, we should
<sinzui> EdwinGrubbs: that is a change in focus. How do you propose to resolve it?
<EdwinGrubbs> sinzui: the involvement portlet could either have grayed out links or it could just have a little warning at the bottom of it. I'm not sure if that is what you are asking.
<thumper> EdwinGrubbs: hi
<thumper> EdwinGrubbs: are you aware of the branch sprite issue?
<sinzui> EdwinGrubbs: I am asking you to reconcile the demand of one community who want to host  a service offsite against another community that want to host it on launchpad
<EdwinGrubbs> thumper: nope, what is that issue?
<thumper> EdwinGrubbs: they aren't showing
<sinzui> EdwinGrubbs: I can say support is offered on a forum, and anther person can offer support using Answers. Someone can say the code is in GNOME's git, but Ubuntu needs the code in bazaar.launchpad.net
<sinzui> EdwinGrubbs: solve that in the involvement portlet and you will be a hero.
<EdwinGrubbs> thumper: are you just talking about the yellow bzr icon? On which page is it not showing up?
<thumper> EdwinGrubbs: https://code.edge.launchpad.net/ubuntu/+source/autoconf-nonfree
<thumper> EdwinGrubbs: and proposing a merge
<thumper> EdwinGrubbs: and a merge proposal page
<EdwinGrubbs> thumper: that was definitely removed in the branch that updated all the sprites so that they could be automated. Sorry about that. I will work on fixing the 4 missing css classes and get that landed today.
<thumper> EdwinGrubbs: awesome, ta
#launchpad-dev 2010-02-17
 * mwhudson lunches
<bjf> has anyone tried interacting with lpapi from groovy?
<thumper> mwhudson: ping me when you want to talk about code import scheduling
<thumper> bjf: not that I know of
<thumper> bjf: but I don't know everything :)
<bjf> thumper, was just curious, looks like a nice language and I've got a bunch of lp work going on
<bjf> thumper, it also talks to databases well and I'm doing that as well
 * thumper nods
<thumper> grrrr!!!!!
<thumper> attachment fail!
<EdwinGrubbs> thumper, rockstar: can I get a review for the fix of the missing branch sprite?
<thumper> yes
<EdwinGrubbs> thumper: you can pretty much ignore the changes in the icon-sprites.positioning file since that is autogenerated. https://code.edge.launchpad.net/~edwin-grubbs/launchpad/bug-521934-missing-sprites/+merge/19454
<thumper> ok
<thumper> EdwinGrubbs: done
<EdwinGrubbs> thumper: thanks
<thumper> rockstar: ping
<rockstar> thumper, pong
<thumper> rockstar: quick call?
<rockstar> thumper, sure.
<mwhudson> thumper: say 15:30 for a call about code imports?
 * mwhudson actually wants to get some programming done
<thumper> mwhudson: sure
<mwhudson> thumper: i think i've broken the back of the incremental import thing \o/
<mwhudson> for git at least
<thumper> w00t
<thumper> mwhudson: call soon?
<mwhudson> thumper: now, or rather in about 1 min, works for me
<thumper> ok
<mwhudson> thumper: https://bugs.edge.launchpad.net/launchpad-code/+bug/497645/comments/2
<mup> Bug #497645: code imports should run fewer jobs at once <code-import> <Launchpad Bazaar Integration:Triaged> <https://launchpad.net/bugs/497645>
<mwhudson> https://bugs.edge.launchpad.net/launchpad-code/+bug/510490
<mup> Bug #510490: importds and DB authentication <Launchpad Bazaar Integration:Triaged> <https://launchpad.net/bugs/510490>
 * thumper back after dinner to finish some stuff off
<mwhudson> thumper: incremental git imports all but ready for review
<mwhudson> (one test needs some love)
 * mwhudson EODs
<thumper> is anyone else having issues with firebug and firefox?
<thumper> as soon as I go inspect element, or click on the bug on the status bar, firefox crashes
<stub> I had it crashing as soon as I tried to get to the error console a week or three ago
<stub> I think it is still doing it too
<thumper> :(
<thumper> stub: I bet the answer will be "upgrade to lucid"
<noodles775> thumper: depending on what you need, the chromium inspector might do the job too.
<thumper> noodles775: chromium has a whole other pile of issues rendering launchpad
<thumper> noodles775: chromium doesn't seem to like the css or the cert
<thumper> noodles775: know how to tell it to just FJDI and accept that I know what I'm doing?
<noodles775> thumper: you can usually just click through it?
 * noodles775 runs dev server
<thumper> noodles775: I did click through, but I get no css
<noodles775> thumper: ah, you're probably looking at a code.lp.dev?
<thumper> noodles775: yes
<noodles775> look at lp.dev first...
<thumper> WTF?
<noodles775> Working now?
<thumper> yeah
<noodles775> Great.
<noodles775> thumper: btw, just in case you haven't already, you'll need the chromium-browser-inspector pkg too.
<thumper> noodles775: I'm looking at adding a description to the merge proposal (for the first comment)
<thumper> noodles775: I now have two multiline JS editors on the one page
<thumper> noodles775: it looks interesting...
<noodles775> fun!
<adeuring> good morning
<noodles775> Moin adeuring
<adeuring> hi noodles775!
<thumper> noodles775: have you talked to jml about the recipe stuff
<thumper> ?
<noodles775> thumper: not since two days ago.
<thumper> noodles775: probably worth scheduling a chat
<thumper> noodles775: jml is planning on talking with james_w about the multiple series on Thursday (after lucid code freeze)
<mrevell> Morning
<jtv> hi mrevell!
<mrevell> Hey jtv!
 * gmb could've sworn he ran `make schema`; why's combine-css running?
<deryck> Morning, all.
<gary_poster> question for someone on code team: I have two branches that were not imported because of sqllite db locks afaict: https://code.edge.launchpad.net/~gary/zc.buildout/system-python-2-bootstrap-changes and https://code.edge.launchpad.net/~gary/zc.buildout/system-python-3-option-cleanup .  I don't see a bug for this.  Should I file one?  Is there a workaround to get my two branches imported?
<gary_poster> code team, another question: afaict, I cannot make a MP to request a merge from one imported branch to another (that is, I don't see a way to make a MP request for https://code.edge.launchpad.net/~gary/zc.buildout/system-python-1-simple-fixes ).  If that's the case, then I might as well delete these branches anyway.  Am I right?
<gary_poster> abentley, rockstar ^^^
<rockstar> gary_poster, import branches cannot be proposed for merge.
<rockstar> gary_poster, if in doubt, file a bug.  We can always close it as a dupe.
<deryck> allenap, since you're work is really a feature outside of the two on the board, I'm going to take your bugs off the bugs backlog.  just FYI.
<allenap> deryck: Okay.
<deryck> allenap, you can continue to pull from the story tag until we make bug sync the feature 1 story.
<allenap> deryck: Cool. I'm on a skunkworks project ;)
<deryck> heh
<deryck> indeed.
<deryck> allenap, for a day or two anyway ;)
<gary_poster> rockstar ack thank you
<allenap> deryck: Do you still want a card for bug 282178?
<mup> Bug #282178: Make IPerson an IHasBugs and make sure calling searchTasks on it works <api> <ubuntu-qa> <Launchpad Bugs:In Progress by thekorn> <https://launchpad.net/bugs/282178>
<deryck> allenap, I do for that, yeah.  As a bugs lane coding task for you, since you're in progress on that for bug fixing.
<allenap> deryck: Okay, I think I get it :)
<deryck> allenap, the idea is that we have no actual work in progress that isn't accounted for on the board, and the card limit numbers keep us from taking on too much WIP.
<deryck> allenap, so if you do work outside the board, it defeats the purpose of the board.
<deryck> allenap, but then there's that skunkworks matter, so it's not perfect now. ;)  But we'll get there as we transition to it. :)
<abentley> rockstar, gary_poster: I thought import branches could be proposed for merge.  We've seen that with couchdb.
<rockstar> abentley, you can propose for merge against an import branch, but the branch itself can't be a source_branch of the mp.
<deryck> flacoste, ping
<flacoste> hi deryck
<abentley> rockstar, Ah.
<allenap> thekorn: I fixed up those test failures: http://paste.ubuntu.com/378410/
<kfogel> noodles775: when you get a chance, UI review on https://code.edge.launchpad.net/~kfogel/launchpad/255868-patches-view-from-bugs-page/+merge/19438  (there are screenshots attached to bug #255868, so you can just look at those if you want to avoid building the branch).
<mup> Bug #255868: Project summary page should show links to patches <story-patch-report> <ubuntu-upstream-relations> <Launchpad Bugs:In Progress by kfogel> <https://launchpad.net/bugs/255868>
<kfogel> noodles775: use the most recent two screenshots; the ones before that are from a now-abandoned UI
<noodles775> kfogel: sorry, I'm EODing now, but either I can do it first thing in the morning, or you could check whether sinzui would like to do the UI review.
<kfogel> noodles775: oh, I didn't know sinzui could.  thanks
<kfogel> noodles775: will ask him
<kfogel> sinzui: when you get a chance, UI review on https://code.edge.launchpad.net/~kfogel/launchpad/255868-patches-view-from-bugs-page/+merge/19438  (there are screenshots attached to bug #255868, so you can just look at the two most recent screenshots there if you want to avoid building the branch).
<mup> Bug #255868: Project summary page should show links to patches <story-patch-report> <ubuntu-upstream-relations> <Launchpad Bugs:In Progress by kfogel> <https://launchpad.net/bugs/255868>
<noodles775> He's wanting to build up ui-reviews (at least he was, I keep repeating that, tell me if that's not the case anymore sinzui!)
<sinzui> noodles775: I do not have a choice. I think UI reviews are hard and stressful, but We really need to do them
<noodles775> sinzui: yes, but I mean particularly whether you'd *like* to do them... ie. with the one earlier today, whether you'd prefer if I just did them now that you've built the number of ui reviews that you've done?
<noodles775> (I remember you saying that you hadn't been getting any, so I've been specifically defering to you)
<sinzui> yes, I certainly was not getting reviews, and I think you should continue asking for my reviews. But what is the criteria for graduation?
<noodles775> sinzui: The chat with beuno - I'd hoped that that was what he wanted to chat with you about the other week before he left. Maybe we should ask him, as his last official service to LP :L
<beuno> yes
<noodles775> :)
<beuno> I want to gradtuate sinzui
<beuno> if he will let me
<sinzui> kfogel: This change looks consistent. I do not have much to say about it. But I have a question about this layout verses the older layout? Do we know if users are seeing this list of reports better than the old position next to the old chart? I think I miss the icons of the past (but not the CVE icon which always looked super scary)
<kfogel> sinzui: when you say the "old" layout, do you mean the one formerly tried in this bug for this specific link, or do you mean something about the portlet box?
<kfogel> sinzui: (by the way, I reassigned the UI review to you, from michael)
<sinzui> I mean the inline information. This list is now 7 long. IT is not as effective as a smaller list.
<sinzui> kfogel: Yes I saw
<kfogel> sinzui: I mean, yes, it's now increased by one item (but not sure what that has to do with "the old position next to the old chart")...
<kfogel> or with icons?
<sinzui> kfogel: deryck: I am looking at http://launchpadlibrarian.net/39277302/255868-screenshot-bugs-with-patches-in-project-group-portlet.png We have now reached the magic number of 7 in the box. If we want to add more reports, I think we need to consider breaking the list up. What I am really asking is if there is evidence now that this list is more or less usable than before
<kfogel> sinzui: I don't think we have evidence either way yet.  Do we have any strategy for collecting such evidence?
<deryck> sinzui, I don't know of any evidence, no.
<beuno> sinzui, lets try and have a call today?
<sinzui> beuno: okay
<deryck> sinzui, but beuno suggested this link in there back at a UDS session. :-)  And nothing was added until now.
<kfogel> sinzui: should we consider that a separate question from the UI review of this particular change?  That is, we may want to further break up the portlet box, but that can be a separate question, before or after this change, I think.
<sinzui> kfogel: deryck: okay. As it said at the start, the addition is consistent and I think it is good to land. I think though we have exhausted this list as a means to add more reports
<deryck> sinzui, agreed
<kfogel> sinzui: "But isn't more always better?"
 * kfogel ducks
<deryck> sinzui, I think we need 3 separate types of filters -- a "me" section, status filters, and other interesting junk. :-)
<deryck> they wouldn't all have to be portlets if those use cases were met.
<sinzui> deryck: yes, I think that is what I want to see on my project pages. One day we will do that
<sinzui> kfogel: I approved the UI, I would approve the code, but I wonder why there is no test for the presence of the link. Is it okay of the link disappears from the page?
<kfogel> sinzui: writing test now
<kfogel> sinzui: may I submit the code to you for review when the tests are done?
<sinzui> okay. show me when you are ready and I will give the code my +1
<kfogel> sinzui: I like the pre-approval, thanks :-).
<kfogel> sinzui: btw, if no patches, it says "0 bugs with patches".  Having the link disappear in the empty case has both advantages and disadvantages; I went with showing it because I thought it's best for the portlet to maintain a consistent size and shape and contents, and for people to see explicitly that there are no bugs with patches rather than wonder if the feature simply stopped working.
<sinzui> kfogel: I know that, I wrote the plural macro with bac. I want "0 bugs with patches" to show all the time. Otherwise users need to hunt for that information, wasting time.
<kfogel> sinzui: oh, you wrote that macro?  Thanks!  (Don't know if you saw, but I'm using it.)
<sinzui> Yes, I saw, which is why I know that "0" is plural without seeing the page.
<bac> sinzui: the blueprints did have a portlet on the distroseries-index page but it is conditional
<sinzui> bac: I suspected, but since I see lots of blueprints, I think the condition may be smoking crack
<bac> ah, right.  yes i'm looking into it
<sinzui> bac: I would not spend a lot of time on this. I think latest bugs and blueprints are flawed. I think they should only show items that younger than 3 months
<kfogel> sinzui: re testing: I'm trying to use find_portlet() from lib/canonical/launchpad/testing/pages.py.  But it demands the portlet's name, as text between <h2> tags, and in this case the portlet has no name.  Is there some typical way to examine the text in the bugfilters-portlet in a test?
<sinzui> do not use find portlet, it is a 1.0 test helper
<kfogel> sinzui: oh
<kfogel> sinzui: ok :-)
<sinzui> kfogel use find_tag_by_id which is good and also faster because you can pass existing soup object
<kfogel> sinzui: I'm starting out with page tests, just to see that the right thing happens, but am I right in thinking I'll also need to write a windmill test b/c this portlet is populated with data by a javascript call?
<sinzui> no windmill, we just need to see that the link is in the portlet list
<kfogel> sinzui: *nod*
<kfogel> sinzui: should be done soon then
<salgado> rockstar, should I assign bug 514400 to you?
<mup> Bug #514400: Make combine-css skip work when it is not needed <Launchpad Foundations:Triaged by salgado> <https://launchpad.net/bugs/514400>
<rockstar> salgado, yes.  In fact, I'm making headway on that now.
<salgado> cool!
<rockstar> salgado, what's weird is that lazr-js has the ability to do this already, but the launchpad Makefile seems to ignore it.
<salgado> rockstar, but can we use that even for combine-css or is this something we could use in the jsbuild target only?
<rockstar> salgado, I'm not sure what you mean.  combine-css and the jsbuild stuff use similar code paths from lazr-js
<salgado> rockstar, combine-css uses just lazr.js.combo.combine_files()
<kfogel> sinzui or anyone: want to play Captain Obvious on my page test?  http://paste.ubuntu.com/378501/
<sinzui> find tags by class does not except a soup object.
<kfogel> sinzui: *nod* thx
<sinzui> kfogel: find_tags_by_class is another helper that should not be used
<kfogel> sinzui: oh
<kfogel> sinzui: do we mark things as obsolete when we decide they're obsolete, in some way that would get a warning when the thing is used at run time?
<sinzui> it is slow, and prone to failure as pages change. We use ids to test
<thekorn> allenap, great!
<sinzui> kfogel: place a id on any element you want to test.
<rockstar> salgado, yeah, so does jsbuild.
<rockstar> salgado, well, no, I take that back.  They both use a subclass of lazr.js.build.ComboFile
<kfogel> sinzui: so I should change the generated html to be more testable?
<sinzui> yes
<kfogel> sinzui: ok
<kfogel> sinzui: but, oddly, I just got "None" for the portlet, using this code:
<kfogel>     >>> def show_bugs_with_patches_from_portlet(contents):
<kfogel>     ...     portlet = find_tag_by_id(contents, 'portlet-bugfilters')
<kfogel>     ...     print portlet
<kfogel> oh
<kfogel> duh
<kfogel> nm
<kfogel> wrong id I ihnk
<sinzui> kfogel: in many cases when you are updating a page, you really just want to update the existing story to show what the user seed
<kfogel> sinzui: ah -- maybe I should be looking for the existing tests of that portlet, instead of adding a new test to patches-view.txt, then?
<sinzui> kfogel: isn't there already a story that verifies the contents of the portlet, and did it fail when you changed the page?
<kfogel> sinzui: dunno, am in EC2 right now
<kfogel> sinzui: but I'll grep around for it.  thanks for the advice_
<kfogel> sinzui: lib/lp/bugs/stories/xx-bugs-statistics-portlet.txt   :-)
<sinzui> yep
<rockstar> salgado, hi
<salgado> hi rockstar
<rockstar> salgado, could I call you?
<salgado> rockstar, I'm with gary reviewing a branch over on #launchpad-foundations; can it be in, say, 15 minutes?
<rockstar> salgado, yes, just ping me when you're done.
<salgado> will do
<salgado> rockstar, I'm ready now
<rockstar> salgado, cool.  Skype?
<mwhudson> good morning
<adiroiban> hi. the code from lib/lp/services/worlddata is part of registry project?
<sinzui> adiroiban: launchpad-foundations
<sinzui> salgado: ping
<adiroiban> sinzui: thanks
<salgado> hi sinzui
<sinzui> salgado: I am looking at OOPS-1506G1386 which is a very common occurrence in production. It is caused by an email address that belongs to an account but not a person
<sinzui> salgado: validate_action_add_email tries to do the right thing by checking if you own the address, or tells you who owns it. It does in the or case.
<salgado> I was looking at that today as well: bug 423447
<mup> Bug #423447: Trying to add an email that's already registered as a SSO account but not a person, oopses <oops> <Launchpad Foundations:Triaged by salgado> <https://launchpad.net/bugs/423447>
<sinzui> salgado: I am trying to decide how to fix this. I think I need to use ensure person on the email.person *before*  it checks if the person is the current user or someone else
<salgado> sinzui, that might create a Person entry for someone who's not a Launchpad user
<sinzui> salgado: exactly my concern
<sinzui> salgado: This event implies these are the same users, so we need to merge accounts, but we cannot until both accounts has a person, which sounds like a bad design :(
<salgado> yeah, that's exactly why I said this is a tricky one in that last comment on the bug report
<salgado> we could "merge" accounts, maybe?
<sinzui> salgado: yes that sounds right (and lightweight), but we have never exposed an account in traversal. I think we need a new kind of login/authtoken
<salgado> sinzui, yes, I think we'd need that, but this might become a non-issue once we split the auth/main databases for real
<sinzui> yes
<sinzui> salgado: if we do not know how to fix this, we could at least avoid the oops by adding a third condition that states the email address is owned by a email.account.displayname. We provide an action for the user to choose a merge. the action will create the Person for the account and then redirect to +requestmerge
<salgado> sinzui, that sounds good to me.  another option would be to change the view that handles ACCOUNTMERGE tokens to create the missing Person, if needed
<sinzui> no, that is ugly too. What happens when the user decides that the SSO account is the master, and the current person/account is the dupe?
<salgado> since an SSO account is identified by its email address, I don't think that's a problem
<salgado> we'd move the email address to the remaining account, so to the user it'd seem as if there ever was just one account
 * sinzui ponders if he looses U1 data if his email address goes to an account that has not used U1
<kfogel> sinzui: My mods to make that bugs portlet test expect the new stuff don't seem to take effect -- I'm wondering if I'm missing something obvious.  http://paste.ubuntu.com/378578/
<thumper> morning
<sinzui> salgado: I think adding a 3rd condition and creating the person as needed by ACCOUNTMERGE is a right, but I really do not know who other apps use account, and I wonder is bad things are happening know when we merge persons.
<sinzui> kfogel: the output implies that there are more lines to change, or worse, we are over testing the whole portlet.  I see 5 failure *before* the part of the test you changed.
<kfogel> sinzui: I'm accustomed to seeing "ghost failures" that precede an actual failure, but what puzzles me is that the change in my branch is minimal (just adds that new line to the portlet), and yet the same test currently passes in db-devel pristine.
<adiroiban> when exposing a new interface via LP API, after adding export_as_webservice_entry, should I do any other steps to be listed in +apidoc?
<sinzui> kfogel: I think the test failures are legitimate, though I question why everything is tested twice for a story...this looks like state testing that I would do in a unittest.
<sinzui> kfogel: You changed the obvious print_portlet_contents calls, but not the print_portlet that often precede the test you changed
<kfogel> sinzui: no idea how I missed that.  Thank you for the clear eyes.
<kfogel> sinzui: (after a certain point, a lot of similar stanzas start to look the same, sigh)
<sinzui> kfogel: this is not a story. it is a bad test, though this is the right place to make your addition.
<kfogel> sinzui: sigh.  I want to file an XXX, but... every place I turn (and adeuring confirms this is not unusual) there's something like that, where we all agree it's not ideal and that it should be fixed.  I could have, almost literally, spent my entire day filing XXXs.
<sinzui> kfogel: yes. That is why adding to this test is the right thing at this moment. We don't need another note stating the obvious
<kfogel> sinzui: :-)
<kfogel> sinzui: is it necessary to add a test anywhere expecting a non-zero patches count?
<sinzui> Looks like it in this test. I would consider using "..." in places where we really do not want to do duplicate verification
<sinzui> adiroiban: no the API is generated from the introspection of the interface
<kfogel> sinzui: sorry, didn't understand "Looks like it in this test."  (agree about the ... thing, in the general case)
<sinzui> kfogel: this test looks like it really wants you to risk a hand injury repeating uninteresting information
<kfogel> sinzui: agreed, but I can use ... to elide that stuff.  My question is, should we also test for a case where there are more than 0 bugs with patches?  If so, I'm tempted to do it in patches-view.txt instead of in xx-bugs-statistics-portlet.txt, but would like your opinion.
<sinzui> kfogel: yes, you are right on all points
<kfogel> sinzui: cool, thanks for the help.  off to do that now, then will submit to you for review
<kfogel> sinzui: is there any way for me to "import" print_portlet() and print_portlet_contents() from that other test, or do I need to move those two methods to some other place first?
<kfogel> e.g. ./lib/canonical/launchpad/testing/pages.py
<sinzui> kfogel: you would need to move the two helpers into a module first, then reimport them into a doc test.
<adiroiban> sinzui: hm... but it looks like +apidoc/index.html is mapped to lib/canonical/launchpad/apidoc/index.html, and this is a static file. make apidoc is not updating this file.
<kfogel> sinzui: (is pages.py (see above) the right module?)
<sinzui> adiroiban: make build generates the wadl doc
<jtv> adiroiban: your branch has landed
<adiroiban> sinzui: thanks. Then something is fishy on my side as after running make build, the wadl files and apidoc/index.html were not updated
<sinzui> hmm
<adiroiban> jtv: thanks. I'm waiting to hit edge and I will test it
<jtv> cool
<bac> hi sinzui - you free for a quick call?
<adiroiban> jtv: do you have time to answer some questions regarding adding something to the Launchpad API? I have something like this: http://paste.ubuntu.com/378609/
<adiroiban> but make apidoc, make build or make run will not update the +apidoc/index.html
<sinzui> adiroiban: it does not update, delete it first
<jtv> adiroiban: all I can add to what sinzui says is, "hey, great that you're doing that!" :-)
<adiroiban> it worked. thanks! Can I add this note to https://dev.launchpad.net/API/ImplementingAPIs ?
<sinzui> adiroiban: I think we need to document that trick some where. I think I discovered it as 45 minutes of shouting at my computer
<sinzui> bac: I am available now
<sinzui> bac: this is the report we are linking to from the new portlet https://edge.launchpad.net/ubuntu/lucid/+needs-packaging
<adiroiban> sinzui: I have added a tips and tricks section on ImplementingAPIs wiki page. I put this info togheter with some tips from wgrant to simplify the testing of APIs
<wgrant> sinzui: Hm, that view name is suboptimal.
<wgrant> "needs-packaging" is a bug tag meaning that some software needs to be packaged.
<sinzui> wgrant: rock, file bugs and tell me how to make this work
<rockstar> thumper, this looks much better: https://devpad.canonical.com/~rockstar/floating.png
<thumper> rockstar: +1 on that
<deryck> Have a nice <whatever-day-part-applies>, all.  I'm out.  cheers.
<rockstar> thumper, great.  I'll pass that on to abentley.
<mwhudson> thumper: i'm re-requesting mirrors on those brancehs james_w mentioned, i think that'll clear most of the formats up
<thumper> mwhudson: cool, I was going to ask;)
<james_w> thanks mwhudson
<james_w> that's based on what the db thinks fwiw
<mwhudson> james_w: ok, that's good to know
<mwhudson> (and a good thing to check, i think)
<thumper> yeah, if the db thinks it is 2a then the puller and scanner both worked
<thumper> mwhudson: we should talk about the recipe stuff some more, perhaps around 1pm?
<mwhudson> thumper: okay
<Ursinha> how could I searchTasks using the order_by parameter? I'm trying to order_by date_created and it gives a KeyError
<poolie> Ursinha: it's 'datecreated'
<Ursinha> poolie: ahhh
<Ursinha> poolie: thanks :)
<poolie> i found this out by grepping the code :-/
<poolie> np, you're welcome
<wgrant> I normally work it out by doing an advanced search and looking at the query string.
<poolie> i think i tried that
<poolie> there might be an intermediate level of mapping
<Ursinha> poolie: maybe it would be nice to have a list in the api docs
<poolie> i agree
<Ursinha> poolie: I'll file a bug
<poolie> you could add a tip to the api faq in help.l.n or wherever it is
<wgrant> It would be nice if bug search didn't suck, too.
<Ursinha> wgrant: :)
<poolie> there are some interesting recent bugs in https://bugs.edge.launchpad.net/launchpadlib/+bugs
<poolie> mostly 'be faster'
<rockstar> thumper, any idea where the launchpad.branches is exposed for launchpadlib?  It thinks getByUniqueName is exposed, but I don't see where.
<thumper> rockstar: not really, jml did that bit, but try grep :)
<rockstar> thumper, grep doesn't seem to be very helpful.
<rockstar> Or maybe it's just too noisy and I need to adjust the squelch on it.
<rockstar> Ah, I see now.
<thumper> rockstar: where? I didn't see it
<Ursinha> poolie: I guess it's somehow bug 256940
<mup> Bug #256940: Make it possible to discover enumerated values <launchpadlib :Triaged> <https://launchpad.net/bugs/256940>
<thumper> wgrant: ping
<rockstar> thumper, IBranch has pass-thru methods for IBranchLookup
<rockstar> Unfortunately, it doesn't seem to work.
<wgrant> thumper: Hi.
<thumper> wgrant: do you know if sourcepackagename is exported anywhere?
<thumper> wgrant: I was considering exporting the text of the name from the sourcepackage object
<wgrant> thumper: Exporting it is forbidden. Just export a text field.
<thumper> wgrant: yes... that is what I ment, but do you know if it is done anywhere yet
<wgrant> thumper: See ISourcePackagePublishingHistory
<thumper> wgrant: defined where?
<wgrant> thumper: lib/lp/soyuz/interfaces/publishing.py
<thumper> ta
<thumper> wgrant: just wanted to be consistent in naming
<wgrant> Good idea.
<thumper> fooey
<thumper> it is exported but as "name"
<thumper> yay (not)
<wgrant> Heh.
<thumper> oh well, at least I don't need to do that
<james_w> there is source_package_name elsewhere I think
<thumper> james_w: yeah, on the publishing history that wgrant mentioned
<thumper> james_w: I'm just exposing the sourcepackage on the branch
<thumper> through the api
<james_w> and source_name
<james_w> and package_name
<james_w> so, clearly you need to pick a new one
<thumper> james_w: how about package_source_name?
<james_w> unused, good choice!
<thumper> w00t
<poolie> hello james_w
<james_w> hi poolie
<poolie> james_w: jam was having some trouble getting import-packages running to test it
<poolie> i don't know if he already posted or spoke to you about it
<james_w> he has
<poolie> k
<james_w> I think it's running now, if a little slow for him
<jml> rockstar, I'm pretty sure the getByUniqueName API works.
<rockstar> jml, yes, there was a previous facepalm moment that was not announced on IRC.  :)
<jml> rockstar, :)
<leonardr> gary, flacoste: the new launchpadlib was packaged and uploaded to debian earlier today
#launchpad-dev 2010-02-18
<thumper> jml: could you enlighten me to how it's hocked up?
<jml> thumper, IBranchSet is exported as a collection, getByUniqueName is a read operation on that collection
<thumper> gotcha
<thumper> jml: thanks
<jml> thumper, np
<thumper> mwhudson: when is good for you?
<mwhudson> thumper: argh
<mwhudson> thumper: about an hour?
<thumper> mwhudson: ok
<mwhudson> i haven't had lunch yet
<thumper> mwhudson: at least it'll timebox our discussion
<thumper> mwhudson: I have to go get the girls at 3
<mwhudson> thumper: ok
 * mwhudson lunches
<thumper> sinzui: ping
<mwhudson> thumper: hello
<thumper> mwhudson: hi
<mwhudson> thumper: ready for a call now, but if you can wait a few minutes more you won't have to listen to me eating my lunch :-)
<thumper> mwhudson: I can wait a few minutes :)
<thumper> mwhudson: now?
<mwhudson> thumper: yep
<mwhudson> thumper: https://code.edge.launchpad.net/~vcs-imports/pydoctor/trunk
<thumper> https://edge.launchpad.net/ubuntu/+source/gwibber/2.29.90-0ubuntu1
<thumper> https://edge.launchpad.net/~gwibber-daily/+archive/ppa
<thumper> https://edge.launchpad.net/~gwibber-daily/+archive/ppa/+build/1514631
<mwhudson> thumper: https://edge.launchpad.net/builders/
<mwhudson> thumper: https://edge.launchpad.net/~launchpad/+archive/ppa/+build/1056751
<poolie> did abentley or anyone write a script that will automatically feed approved reviews to pqm?
<poolie> or semiautomatically?
<abentley> poolie, not that I know of.
<poolie> maybe i will then
<abentley> poolie, however, Tarmac behaves that way.
<poolie> it automatically pulls?
<poolie> do you think we should switch?
<abentley> It automatically merges approved merge proposals and commits.
<poolie> right
<abentley> I think we should switch.
<poolie> so presumably one should only use status:Approved if you really mean 'approved with no further changes'
<poolie> not conditionally approved
<abentley> poolie, true.
<abentley> poolie, when we get a chance to work on merge queues again, that will be less of a concern.
<poolie> i think it's ok, as long as people use it that way
<poolie> it's probably a good idea anyhow
<poolie> in piloting i find myself wanting a state that's not "in progress" but not "needs review"
<poolie> like, it needs a pp to fix it
<poolie> somewhere in between
<abentley> poolie, if we can generalize it sufficiently, cool.
<abentley> poolie, another option was to assign merge proposals to people.
<poolie> i don't think adding another state is precisely right
<poolie> maybe that's it
<poolie> because actually i want to distinguish 'the author is going to go away and fix this' from 'i will'
<poolie> so maybe assignment is right
<lifeless> sounds like a bug
<abentley> lifeless, what sounds like a bug?
<lifeless> something that gets assigned to someone to do, and wanting to track that
<poolie> yeah, but it's one of those vague difficult bugs
<abentley> lifeless, ah.  I thought you meant the discussion suggested a bug was present.
<poolie> it's questionable to file it until you can make it concrete imo
<lifeless> poolie: I guess I'm saying that maybe merge proposals lie on the same spectrum as specs and bugs
<poolie> oh i see
<poolie> IAssignable
<lifeless> while I'd like to see be less crisp about what they are, and more malleable about what we want them to do
<lifeless> s/while/which/
<mwhudson> grr what's locking bugbranch
<mwhudson> oh probably the scanner
<poolie> lifeless: i was interested by the idea of using rpc signing rather than https
<poolie> http://developer.amazonwebservices.com/connect/entry.jspa?externalID=1928
<poolie> "use ssl if at all possible though it is slower"
<lifeless> poolie: yes
<poolie> so i think we should fix lplib (or lazr or whatever) to hold the connection
<lifeless> poolie: you still get to avoid round trips though - and note that more and more corporates are deploying SSLBump and equivalents
<lifeless> poolie: ofh for sure.
<poolie> 'still get to' how?
<lifeless> poolie: using message signing
<poolie> using message signing and http?
<poolie> yes
<poolie> it's kind of a shame it's not enough
<mwhudson> can you upload files to the librarian without a database connection?
<mwhudson> hm
 * mwhudson eods
<thumper> gah
<thumper> can't get balsamiq to start
<thumper> :(
<adeuring> good morning
<mrevell> Morning
 * wgrant is surprised to see the product/project rename actually happening!
<deryck> Morning, all.
<bigjools> morgen
<adeuring> morning deryck
<didrocks> hey guys, I've at last a LP checkout working in a karmic VM. I'm tried to export and import GPG key from the launchpad API (cf bug #389872). Following jml's explanation, I have added the export_readâ¦ export_writeâ¦ decorator to function but I don't have any idea of other stuff to add. For the attribute, appart from adding exported() to them, same thing, quite lost
<mup> Bug #389872: GPG keys not exposed in API <api> <Launchpad Registry:In Progress by didrocks> <https://launchpad.net/bugs/389872>
<didrocks> is there any doc or help I can get? :)
<thekorn> didrocks, there is https://dev.launchpad.net/API/ImplementingAPIs which explains how doing such things,
<didrocks> thekorn: sweet, it seems to be what I need, let get it a try :)
<didrocks> thanks
<thekorn> didrocks, did you push your code somewhere?
<didrocks> thekorn: not yet, I'll try to get it working first and check locally before pushing
<thekorn> aha, ok
<wgrant> It's not the easiest export.
<wgrant> Because there's no URL yet.
<thekorn> right, this part is missing on the wiki page I mentioned above
<wgrant> So you'll need to add a new method to PersonNavigation, and you'll need a browser:url directive in lib/lp/registry/browser/configure.zcml
<didrocks> wgrant: hem, I'm already fighting to do the right thing, seems more difficult then :)
<didrocks> (and I need to do the same for ssh)
<didrocks> well, I pushed my branch into ~didrocks/launchpad/devel but it still needs work and I'm unsure to know understand enough the documentation for the decorators handling
<deryck> adeuring, gmb, kfogel -- just a reminder to update the kanban board before the standup, to make walking the board quicker.
<adeuring> OK, I'll try it...
<deryck> matsubara -- the qa burndown seems not to be updating any longer for 10.02.
<deryck> Assuming I'm still looking at the right page.
<matsubara> deryck, I'll take a look
<deryck> matsubara, thanks!
<deryck> adeuring or gmb -- could I get one of you to review a branch for me?
<adeuring> deryck: sure
<deryck> I need a bugs dev to make sure I haven't missed something that exists somewhere else.
<deryck> adeuring, thanks!  https://code.edge.launchpad.net/~deryck/launchpad/enable-tracking-link-perms-512378/+merge/19600
<deryck> adeuring, feel free to ping me in #launchpad-review to follow up.
<adeuring> deryck: r=me
<deryck> adeuring, excellent, thanks!
<BjornT> deryck: why don't you check whether the user has edit permission for the project?
<deryck> BjornT, because I'm dumb. :-)
<deryck> BjornT, this is why I wanted someone from bugs.  I felt I was missing something obvious.
<deryck> but I guess that's obvious beyond bugs even.
<BjornT> deryck: you can do a tal:condition on something like context/required:launchpad.Edit
<deryck> BjornT, ah, cool.  I never knew that actually.
<deryck> much simpler, too.
<directhex> i just came across bug #139855. is there any drive from canonical to resolve the bug, or in the short term is the easiest way to track usage by mirroring a PPA off-site where you have your own access.log to parse?
<mup_> Bug #139855: Display stats about PPA usage <feature> <Soyuz:Triaged> <https://launchpad.net/bugs/139855>
<deryck> adeuring, BjornT -- updated:  https://code.edge.launchpad.net/~deryck/launchpad/enable-tracking-link-perms-512378/+merge/19600
<deryck> adeuring, BjornT -- look good?
<adeuring> deryck: yes
<bigjools> directhex: it's not a priority at the moment, although we'd love to do it at some point
<deryck> adeuring, gmb, kfogel -- call, in two?
<gmb> yep
<kfogel> deryck: yup
<directhex> bigjools, the post by jml presents an action list. how much of that list can be worked on by normal plebs without secret canonical web server access?
<BjornT> deryck: looks good
<deryck> BjornT, thanks!
<bigjools> directhex: most if not all of it
<bigjools> directhex: some work was done already to scan logs
<directhex> bigjools, i'll take a look then. my cpu/ram seem to be on fire, courtesy of "bzr branch lp:launchpad".
<bigjools> yeah it's a big branch
<directhex> 243M	launchpad/
<directhex> right, there we go
<directhex> bigjools, as i read this source, a thought occurs. how exactly do i *test* my changes?
<bigjools> directhex: depends on what you want to implement, there's a lot of work to do
<bigjools> UI, backend, database schema etc
<directhex> bigjools, the UI is the kind of easy bit i could probably do, but unfortunately the backend stuff is a prerequisite
<bigjools> most of the existing testing is done in the test suite, unit tests or doc tests
<bigjools> well you could start by proposing some schema changes, we'll get those done, and then you can poke values in locally when doing the UI
<bigjools> the scanning part is a separate task
<directhex> bigjools, is the bug report the best place to discuss my thoughts?
<bigjools> directhex: yes, absolutely
<fran6co> hi
<fran6co> i need some help setting up launchpad in my machine, anybody that want to give a hand with it?
<fran6co> well, I managed to install it and make it work, but the thing that i don't get is the cronscripts
<fran6co> i couldn't find info about how to i should run them
<fran6co> thanks in advance
<matsubara-lunch> deryck, burndown is up to date
<deryck> matsubara-lunch, thanks!
<deryck> mrevell, call time?
<sinzui> jelmer: ping
<jelmer> sinzui, hi
<sinzui> bug 523844 does not look like a bug, it looks like a question, and I think it is one should should be able to answer yourself.
<mup> Bug #523844: experimental Debian distroseries <Launchpad Registry:New> <https://launchpad.net/bugs/523844>
<sinzui> jelmer: do you see the Add series link: https://edge.launchpad.net/debian
<jelmer> sinzui: Ah, yep
<sinzui> jelmer: I think you have a better understanding about creating a debian series than myself.
<jelmer> sinzui: I hadn't realized I could do that sort of thing myself now. :-)
<jelmer> sinzui: I'd like to double-check that experimental isn't missing by intention though, who should I talk to about that?
<sinzui> jelmer: No idea. I can fix the code, but distroseries were designed for Ubuntu. Launchpad really ignores all other distros
 * sinzui tries to edit lenny to see if experimental exists
<sinzui> Experimental does not appear to be supported
<sinzui> but I think that is because series cannot go backwards
<sinzui> jelmer: try creating the series on staging, the editing it to fix the status
<sinzui> jelmer: you may need to hack the URL because only losas are really offered the link
<mrevell> msg deryck I'll get Skype up and running
<mrevell> heh
<sinzui> jelmer: I just confirmed that you can hack the url for +edit ti set a active series to experimental
<jelmer> sinzui: ah, thanks
<jelmer> sinzui: I'll see if I can confirm that it's ok to create an experimental distroseries and then I'll give it a try. I'll close the bug.
<sinzui> jelmer: the missing series status sounds familiar, I think this is a bug that also affects projects
<jelmer> sinzui: In particular I'd like a distroseries named experimental, I'm not too worried about having its status set to Experimental
<james_w> erm, https://edge.launchpad.net/debian/experimental
<james_w> is that not what you are looking for
<james_w> ?
<deryck> gmb, ping
<gmb> deryck: Hi
<deryck> rockstar, you around?
<kfogel> deryck: terminology question: do we have a name for a not-yet-filled-in-by-separate-javascript-request portlet?  Do we call the raw, unfilled portlet a "base" or a "template" or a "raw" portlet or anything like that?
<deryck> kfogel, you mean the name of the template file itself?
<kfogel> deryck: well, no, I have that.  I mean a word for the unfilled portlet (except I don't know if "unfilled" is what we'd use).
<kfogel> like if you were writing a test, and one phase of your test examined the unfilled portlet, and another phase pulled the real contents and examined those, what would be the words to describe each phase?
<adiroiban> gmb: did you had time to put this branch for ec2 test https://code.edge.launchpad.net/~adiroiban/launchpad/bug-522188/+merge/19395 ?
<deryck> kfogel, in just the test comment or doc portion?  In plain English not code, is that what you mean?
<kfogel> deryck: well, in a doc comment, but also in the names of two methods I'm abstracting out of xx-bugs-statistics-portlet.txt (I'm moving them to pages.py, because they're needed in multiple tests now).
<kfogel> deryck: right now, they're called, respectively, print_portlet() and print_portlet_contents().
<deryck> kfogel, ah, ok. yeah, I'm not aware of any convention for that.  Those names seem fine to me.  Or print_empty_portlet for the first?
<kfogel> deryck: the first of those names is unclear on its own; then if you see the second name it makes more sense, but I'd like to avoid that window of confusion.  I'll probably go with "unfilled" and "filled" (as "empty" implies no content at all, which isn't the case).
<kfogel> deryck: I just wanted to check that there wasn't already a term in use within Launchpad for these two states.
<deryck> kfogel, not that I'm aware of, no.
<kfogel> deryck: ok, thx
<deryck> np
<gmb> adiroiban: Strange, I thought that branch had landed; let me check
<adiroiban> gmb: I have not received any email from buildbot. Same for this branch https://code.edge.launchpad.net/~adiroiban/launchpad/bug-340664/+merge/18452
<gmb> adiroiban: Hmm. I'll look into that.
<adiroiban> I am not sure how buildbot is sending notification... as some of my branch did land without a notification from buildbot
<gmb> adiroiban: How did only some of it land? That doesn't make sense.
<adiroiban> I mean that for some of my branch that landed
<adiroiban> i received an email from buildbot
<adiroiban> others were landed without a notification from buildbot
<kfogel> jtv: saw the characters displayed correctly in my email from your review response.  yes, they're right :-)
<jtv> kfogel: different chars this time.  Got the full text from google translate.  Is that last character the same one I hear in the polite version of "hello"?
<kfogel> jtv: yup -- it means "you'
<kfogel> "you"
<jtv> In the polite form, right?  I always thought it sounded more like nÃ­Å
<gmb> adiroiban: Ah, looks like your branches were submitted close to the same time but we were in testfix mode so they didn't land and I subsequently forgot to resubmit. Apologies; I'll do so now.
<adiroiban> gmb: no problem. thanks
<sinzui> adiroiban: See my answer to your question. I want to know if you see what I see
<adiroiban> sinzui: http://bayimg.com/eaKPPAAcJ
<adiroiban> this is whay I see
<adiroiban> only âReactivate MLâ
<sinzui> :(
<sinzui> adiroiban: I will purge and report a bug
<adiroiban> thanks!
<kfogel> deryck: just fyi, jcastro and I are having this conversation over in the community channel (and are about to turn it into a phone call): http://paste.ubuntu.com/379256/
<sinzui> adiroiban: It is purged
<kfogel> deryck: it's a dicey UI question w.r.t. the bugs-with-patches stats.  your opinion welcome.
 * deryck looks
<adiroiban> sinzui: yep. Many thanks!
<kfogel> sinzui: (you might be interested in above two comments addressed to deryck too)
<sinzui> adiroiban: this may not be time to celebrate: bug 471274
<mup> Bug #471274: No option to purge list archives when purging a mailing list <feature> <mailing-lists> <oem-services> <Launchpad Registry:Triaged> <https://launchpad.net/bugs/471274>
<deryck> kfogel, simple fix -- just have the X bugs with patches be inclusive of closed bugs  OR have +patches exclude closed by default and add filter by status.
<kfogel> deryck: I think I like the former, actually.  Since people can sort by state once they get to the +patches page anyway.
<adiroiban> sinzui: that's ok. we didn't have top secret stuff there :)
<kfogel> deryck: thanks.
<deryck> kfogel, sure
<mwhudson> jelmer: i'd like to upgrade dulwich & bzr-git so i can get my changes to do partial imports landed, will lp:dulwich and lp:bzr-git be ok?
<mwhudson> i guess lp:bzr-git is required :-)
<mwhudson> jelmer: thanks for merging my branch btw
<sinzui> bac: ping
<sinzui> EdwinGrubbs: ping
<bac> hi
<bac> sinzui
<sinzui> bac: look at https://edge.launchpad.net/ubuntu/lucid/+source/alacarte The "Unspecified" is a db hit, and when you list a lot of projects on a page, it is a costly repeat
<sinzui> bac: I have never liked this this ambiguous, possibly pathetic attempt to say the license is unknown. Do you want to argue for this feature's continued existence?
<bac> sinzui: not i
<jelmer> mwhudson, yeah, both trunks should be fine
<bac> sinzui: but that lookup is no more costly than anything else, is it?  getting rid of it b/c it is ugly is one thing but i can't see how it is hammering us more than any other attributte
<kfogel> sinzui: advice -- I've abstracted out two functions that were formerly in a .txt test file.  Now they live in pages.py -- see http://paste.ubuntu.com/379268/ .  But, the code refers directly to "http://bugs.launchpad.dev/", because the functions are specific to the bugfilters portlet.  I think that's okay, but I also notice this is the only .py code in that directory that hardcodes a service URL.  Is this ok?
<sinzui> bac: true, but if +packaging shows 20 of these packages, we are doing repeat queries for the license, plus the bug supervisor, tracker, branch, and translations. I care about the last four items, but the license is a formatter quirk, and the only person who can change the license if the project owner
<sinzui> kfogel: I see. I think both helpers are kind of bogus because they are making assumptions about the markup or host. They break every year
<sinzui> kfogel: as long as the bug team is willing to do the maintenance, there is no problem
<kfogel> sinzui: well, I think we are (will check with the team).  my question is really about the host aspect -- whether that's likely to break ever.  But if you think it's no worse than any of the other assumptions in that test, then I'll leave it.
<kfogel> s/test/test function/
<sinzui> The host is less brittle then the markup. I would question why the test helper is going into 1) a deprecated module, and 2) a module used by by non-bugs apps. I think the tests should be in lp.bugs.tests.bug
<sinzui> kfogel: ^
<sinzui> kfogel: actually, lp.bugs.tests.bug is exactly where the helpers belong. doctests. stories, and unittests can import them as needed
<kfogel> sinzui: thx, will move
 * sinzui wonders why lp.bugs.pagetests exists
 * kfogel wonders how one is supposed to know it's deprecated
<sinzui> kfogel: it is canonical.launchpad. we build in lp.<app>
<sinzui> kfogel: everything in canonical.launchpad was left behind during the open source apocalypse
<kfogel> sinzui: ahhh.  Yes, I remember that, but as I wasn't developing at the time, I didn't have a reflexive understanding of it.  got it
<sinzui> oh dear, as I feared lp.pagetests do not work because that is not a legitimate directory
<sinzui> deryck: ping
<deryck> hi sinzui
<sinzui> lp.bugs.pagetests should not exist. The tests in those directories are not played by the test runner. They should be moved to lp.bugs.stories
<sinzui> I was about to report a bug
<sinzui> deryck: just doing a move to the stories directory should make the command work
<sinzui> ./bin/test -vvc -t cve-pages -t xx-bug-text-pages
<deryck> sinzui, oh, wow.  Didn't realize we had anything in pagetests still.
<kfogel> deryck: since this came up as I was adding some test infrastructure, if there's a simple rearrangement/move to do, I can just do it on my branch here...
<kfogel> sinzui: ^^
<deryck> sinzui, I'll move them this evening or tomorrow am.
<deryck> ah
<kfogel> deryck: lib/canonical/launchpad/testing/pages.py is the file I was adding to, when sinzui noticed
<deryck> kfogel, ok, you could move the tests under lp.bugs.pagetests to lp.bugs.stories if you don't mind.
<deryck> kfogel, see the command above sinzui gave.  it should run after the move.
<deryck> kfogel, but I don't mind doing this tomorrow AM first thing if you don't want to mess with it.
<sinzui> deryck: I think this is an issue where someone found some tests in the old location, and tried to move them to the now location, but created a bad directory in the process.
<kfogel> deryck: I don't -- nearly complete non-understanding of the totality of this issue, now that I look carefully at the commands and paths above, coupled with a desire to close out my WIP, makes me defer to you :-).
<deryck> sinzui, ah, right.  that would make sense.
<deryck> kfogel, sure.  it's trivial for me.  So happy to do it.
<sinzui> deryck: kfogel: The tests are pre-3.0. they might not pass
<deryck> gah!
<deryck> right
<kfogel> ?
<deryck> they haven't been running this long  rm -rf anyone? :-)
 * deryck ducks
<deryck> sinzui, kfogel -- I'll sort it out proper tomorrow.  I have another test to fix, so no worries.
<kfogel> deryck: thx
<sinzui> deryck: you have my rs to delete any test that duplicates the authority of another test.
<deryck> sinzui, will do.  thanks.
<mwhudson> hmm
<mwhudson> jelmer: do you have any clever ideas about managing ~/.bazaar/svn-cache on the import slaves
<mwhudson> jelmer: these directories are about 7 gigs on the slaves already
<jelmer> mwhudson: do we need clever ideas?
<jelmer> mwhudson: ah
<jelmer> mwhudson: are those sizes really a problem?
<mwhudson> and as we add more slaves, each slave is going to have to do the work to rebuild the cache
<mwhudson> which seems pretty wasteful
<mwhudson> jelmer: well, it depends how much more it grows, i guess
<mwhudson> 10 gigs isn't really a problem, i guess
<mwhudson> 100 gigs we'd need to know about
<mwhudson> 1000 would be a definite problem
<jelmer> mwhudson: it'll be a while before you'd hit 100 gigs I think
<jelmer> we should have a better mechanism in place in bzr to store this sort of cache data before that I hope
<jelmer> mwhudson, alternatively, we could see if we could turn off the cache for bzr-svn and see how much extra bandwidth that requires
<jelmer> mwhudson: since lp only does pulls we might be able to get away with that
<mwhudson> jelmer: hmm
<mwhudson> jelmer: the "determining revisions to fetch" step takes a considerable amount of time for large repos
<mwhudson> jelmer: isn't that the step that the cache helps with?
<jelmer> mwhudson, I wonder why that is, it is much quicker locally
<mwhudson> jelmer: f'ex https://code.edge.launchpad.net/~timmie/grass6/grass_trunk
<mwhudson> looks like "determining revisions to fetch" too 7 hours here (!)
<mwhudson> *took
<jelmer> whoa
 * jelmer tries locally
<jelmer> mwhudson, does a lot of terminal out cause the slave process to hang perhaps?
<jelmer> terminal outPUT
<mwhudson> and copying revisions took 5
<mwhudson> jelmer: i don't think so, it's only outputting one line a minute
<mwhudson> it's possible the machine was slammed while this was happening
<jelmer> mwhudson: I've seen it with more importants so I don't think this was a glitch
<jelmer> *imports
<jelmer> man, my brain is sleeping
<jelmer> mwhudson, I'll see if I can reproduce it here locally doing a pull. If not, I wonder how we can profile the problems on the lp slaves..
<mwhudson> other things we can do are preserve the svn-cache between runs like we do the git.db
<mwhudson> or maybe just gzip the caches, they seem to compress pretty well
<mwhudson> jelmer: the reason i'm asking is because i've been asked about how much disk space to give new code import slaves
<jelmer> mwhudson: the svn-cache is not preserved between runs, the home directory of the slave is cleared?
<mwhudson> jelmer: uh no
<mwhudson> that's not what i meant
<mwhudson> we currently preserve git.db files by copying them from the imported branch onto escudero, and fetching them before updating the import
<mwhudson> we *could* copy the ~/.bazaar/svn-cache/$foo directory to and from escudero after and before an import in the same way
<mwhudson> with some trickery to know which file to get, i guess
<jelmer> I don't think that's worth the effort to be honest
<jelmer> unless the "fetching revision info" step is taking too much time
<mwhudson> oh right
<mwhudson> that step seem to hardly be taking any time
<mwhudson> hm, but maybe the cache already existed for that branch
<mwhudson> seems like it took 10 minutes for that repo in fact
<mwhudson> that's more than you'd want to spend on every import, for sure
<mwhudson> but i guess once per slave isn't that bad
<jelmer> mwhudson: it'll only happen once per slave per repo
<jelmer> after that it should be seconds
<mwhudson> jelmer: but it's not really about time, it's about disk space
<mwhudson> right now anyway
<mwhudson> if we preserve them somewhere else, we can only have the ones on the slave that are running at a given time
<jelmer> if it's about disk space we can look at whether we actually need the cache
<jelmer> mwhudson, there's definitely something wrong on launchpad
<jelmer> "determing revisions to fetch" for the branch you just linked to took less than a minute here
<mwhudson> !!
<maxb> ooi, why isn't "determining revisions to fetch" the equivalent of a single "svn log" operation?
<mwhudson> jelmer: is this a clue?
<mwhudson> 2010-02-16 21:43:44 WARNING Upgrade to svn 1.5 or higher for faster retrieving of revision properties.
<jelmer> maxb: if you're not using the cache, it is
<maxb> Shouldn't 'svn log' be fast enough to make a cache pointless?
<jelmer> maxb: there's more you can do on a svn repo than svn log
<jelmer> maxb: more than bzr pull I mean
<mwhudson> jelmer: the slaves seem to be on subversion 1.4.6dfsg1-2ubuntu1.1
<jelmer> mwhudson: Hmm, I don't get that here
<jelmer> mwhudson: Ah, it might be useful to upgrade the version of subversion on the slaves
<jelmer> mwhudson: but it also looks like we're hitting a very slow code path when we're using svn 1.4, so we should probably fix that anyway
<mwhudson> jelmer: last time i talked about upgrading svn apart from as part of a distro upgrade i was told it was impossible
<jelmer> mwhudson: I'll see if I can reproduce the issue in a dapper vm
<mwhudson> jelmer: hardy should be fine...
<maxb> mwhudson: interesting, was a reason given?
<mwhudson> maxb: cscvs uses both python-subversion and pysvn
<maxb> That sounds merely slightly complex, not impossible
<mwhudson> and at least one of those is really hard to build
<mwhudson> well
<mwhudson> maybe we didn't ask you :-)
<mwhudson> anyway, it might be better now
<maxb> I've built pysvn on Slackware 10.0, I'm sure I can build it on hardy
<maxb> :-)
<mwhudson> backporting from say feisty to dapper might have been more awkward than karmic -> hardy
<kfogel> sinzui: several days in a row now, my machine has gradually slowed down to a crawl, becoming more and more unusable; running lp tests seems to be the cause, though I'm not 100% sure yet.  'top' indicates that python2.5 consumes most of the CPU.  Have you experienced anything similar?
<sinzui> kfogel: the testrunner does take most of one cpu, but all is better when the test completes
<kfogel> sinzui: the "all is better" part doesn't always seem to happen for me
<sinzui> kfogel: I never run into swap, so I think memory may be important. I have 2 gigs @ 32 bit usually allocated to development
<kfogel> sinzui: well, up to several seconds just to switch virtual desktops, maybe 1/2 second keystroke delay now.  I'm rebooting.  back in a bit
<kfogel> deryck: in a searchTasks() query, is there a recommended way to get the search to include all statuses?  If you do status=None, it reverts to default statuses, which don't include (e.g.) BugTaskStatus.FIXRELEASED.
<kfogel> deryck: should I just do status=any(*UNRESOLVED_BUGTASK_STATUSES, *RESOLVED_BUGTASK_STATUS) or something?
<sinzui> Is anyone with postgress guruness awake?
<kfogel> deryck: (well, that syntax may be wrong but you get the idea)
<kfogel> sinzui: sorry, no guruness, but if you ask I'll look and see if I know the answer off the top of my head.
<deryck> kfogel, yes, the idea is right, but not the syntax.  Pass in a single combined list.
<kfogel> deryck: will do
<deryck> I'm out.  Later all.
<sinzui> kfogel: we have bad data in production that I though we fixed in November. patch-2207-09-0.sql drops packaging_uniqueness and adds packaging__distroseries__sourcepackagename__key  which is a more restrictive UNIQUE rule. But I do not see this patch applied when I look at the db
 * sinzui thinks he needs to email stub and jtv to understand the screwup
<kfogel> sinzui: I think so -- this is probably more about lp db patching process than about postgresql per se
<sinzui> I also need to find the query that identified the insane data too
<mwhudson> sinzui: do you need to "drop index" not "drop constraint" ?
<mwhudson> just a guess
<sinzui> maybe
<mwhudson> strange for it not to error though
<sinzui> the new constraint was not added, so there is either a big failure or two small failures
<kfogel> sinzui: I tried to request a code review (non-ui this time) from you for https://code.edge.launchpad.net/~kfogel/launchpad/255868-patches-view-from-bugs-page/+merge/19438  -- it wouldn't let me, I guess because you had already approved it for ui.  Can you do a code review?
<sinzui> kfogel: I saw it and am reading right now
<kfogel> sinzui: thanks (hunh -- you got the email, and yet the page doesn't show you as a pending reviewer)
<sinzui> kfogel: r=me
<kfogel> sinzui: thanks
<bac> sinzui: Q: in the new portlet under "Needs linking" is that really "needs linking or more info", i.e. using getPrioritizedUnlinkedSourcePackages?
<bac> sinzui: b/c some of those can be *linked* but still lacking signficant information
<bac> if that's the case i need to change the heading
<kfogel> sinzui: is it normal for EC2 to try to merge devel into a db-devel-based branch before testing?  http://paste.ubuntu.com/379375/
<sinzui> bac: yes getPrioritizedUnlinkedSourcePackages
<bac> sinzui: so "Needs linking or more information"?  very wordy
<sinzui> bac: the ones that are missing information need to be fixes by upstream contributors most of the time
<kfogel> sinzui: 'utilities/ec2 test --headless -b launchpad=db-devel' is probably the answer
<sinzui> bac: Maybe we needed to be clear that the packages need to be associated with an upstream project...I do not like the use of "link"
<bac> kfogel: yes, if you're not using devel you need to specify db-devel.  there is a wiki page devoted to how to work with db-devel
<sinzui> kfogel: if you branched from db-devel, then yes you need to merge to that, but I have read your changes and they could be extracted and applied to a devel branch so that edge users can see it
<kfogel> sinzui: my changes depend on stuff that's only in db-devel, though (the entire +patches view is not present in devel yet, IIRC)
<kfogel> sinzui: so these particular changes would work, but the link they add would be a broken link
<sinzui> fair enough, send it to staging
<bac> sinzui: http://people.canonical.com/~bac/link-portlet.png
<bac> that's a lot of use of "link"
<sinzui> bac: yes, which is why I wonder is linked is right. We can let the users decide
<bac> sinzui: so you want to go with linked now?
<bac> or do you want to try to come up with a new term now?
<jelmer> mwhudson, hmm, looks like subvertpy doesn't actually compile with svn 1.4 at the moment..
<sinzui> bac: if you can, that would be grand. I wont make it a condition for landing since I think we need users to help find the words that explains the connections between packages an project releases, and communities and kinds of contributions
<mwhudson> jelmer: i guess we'd better be careful when upgrading that then!
<jelmer> mwhudson: fixing it atm
<bac> sinzui: i'm having a hard time coming up with an alternative right now.  "associate"/"association" is the only thing i can think of and it would look silly in that many places i think
<mwhudson> jelmer: ok
<sinzui> bac: yes, it is hard
<sinzui> bac: I now see that the bug I am working on is in fact a duplicate of the one I asked for help with on the list: look at this: https://edge.launchpad.net/gtk/+packages
<bac> i think i'll do a first pass as is knowing that inertia is a terrible thing
<bac> sinzui: so any clue why that db patch is not applied?
<sinzui> no, and I wish I could find my dup finder query. I could clean up the dup data in an evening if I could get a list
<mwhudson> oh
<mwhudson> i know why the patch isn't applying
<mwhudson> the filename is wrong
<mwhudson> sinzui: ^
<sinzui> rock. I am incompetent
<sinzui> mwhudson: I do not see the file name wrong, but maybe the revision is wrong?
<sinzui>     INSERT INTO LaunchpadDatabaseRevision VALUES (2207, 9, 0);
<sinzui> 9 should be 09?
<mwhudson> sinzui: no
<mwhudson> ot
<mwhudson> it's called .0.sql
<mwhudson> all the others are -0.sql
 * sinzui bangs head
<sinzui> thanks mwhudson I will update the bug. We cannot land a name fix until the data is cleaned up again
<mwhudson> sinzui: np
<mwhudson> i guess it's "unusual rollout requirement" time
<sinzui> mwhudson: thanks a lot. there are only two bad condition in the data, I got lucky when I saw one in the list of 10,000 packagings
<jelmer> mwhudson, 1.4 compatibility has been restored
<mwhudson> jelmer: cool
<sinzui> spm: ping
<sinzui> hmm, losas are sprinting aren't they
<mwhudson> not yet, but spm isn't around
<mwhudson> mbarnett might still be?
 * mwhudson lunches
 * thumper is back but grabbing lunch
#launchpad-dev 2010-02-19
<jelmer> mwhudson, I can reproduce the determining revisions to fetch slowness
<thumper> jelmer: awesome
<thumper> jelmer: what was it
<jelmer> thumper: building against svn 1.4 slows down bzr-svn significantly
<jelmer> because we attempt to fetch all revision properties manually for each revision in the repository and don't cache the results
<jelmer> something that can apparently take 7 hours for some repositories where it takes less than a minute when built against svn 1.5...
<thumper> Ursinha: ping
<thumper> jelmer: hmm.. interesting
<thumper> jelmer: on a side note, has bzr-hg been fixed for the infinite recursion bug?
<jelmer> thumper: I haven't had a chance to look into it yet.
<poolie> hi jelmer
<jelmer> 'morning poolie
<wgrant> sinzui: Is there a good reason that admins cannot promote others to admins?
<thumper> wgrant: admins of what?
<wgrant> thumper: Teams.
<thumper> wgrant: they can can't they?
<thumper> wgrant: I thought they could
<wgrant> No. Only owners can.
<thumper> really?
<wgrant> Admins can demote, but not promote.
<thumper> hmm...
<thumper> seems a bit screwy to me
<thumper> if you trust them to be admins, let them be admins, not half-admins
<wgrant> Right.
<sinzui> wgrant: I am certain there is not *good* reason admins cannot promote
<wgrant> It has been like that for as long as I remember, and I have always wondered about it.
<thumper> wgrant: patches accepted :)
<maxb> Submitting a patch loosening security policy can feel weird
<wgrant> Yes.
<wgrant> Particularly a security policy that has been around for perhaps 5 years and nobody knows the purpose of.
<maxb> I don't suppose there's any light at the end of the tunnel w.r.t. removing xx-resetpassword-of-sso-account.txt ?
<maxb> I miss being able to do a clean test run
<wgrant> The OpenID RP stuff should cause that to vanish.
<wgrant> But I'm not sure how far off that is...
<Ursinha> thumper: pong
<thumper> Ursinha: I was wondering if you had used (QAed for me) the API to get branches by date
<Ursinha> thumper: not yet
<thumper> ok
<Ursinha> thumper: but will soon
<thumper> :)
<maxb> wgrant: I don't suppose you've already debugged ""GpgmeError: (32, 176, 'Unknown error code')"" on lucid, have you?
<wgrant> maxb: Try http://paste.ubuntu.com/379455/
<wgrant> (in sourcecode/pygpgme)
<wgrant> Other than that, most stuff seems to work with stock Lucid + ppa:launchpad.
<wgrant> I haven't run the whole test suite lately, though.
<wgrant> The need to downgrade python-setuptools has mysteriously vanished in the past week or so.
<maxb> nifty
<maxb> I'm actually not working on 2.6 or lucid right now, having got distracted by "No really, don't load my system bzrlib.plugins.*"
<wgrant> Is there a codeimport queue visible somewhere?
<wgrant> I'm wondering if I would be better to branch from Git locally rather than waiting for my new import to complete.
<mwhudson> lolz
<mwhudson> mwh@grond:unicod-branch-names-bug-449528$ ls lib/canonical/launchpad/images/*gray*
<mwhudson> lib/canonical/launchpad/images/persongray.png  lib/canonical/launchpad/images/teamgray.png
<mwhudson> mwh@grond:unicod-branch-names-bug-449528$ ls lib/canonical/launchpad/images/*grey*
<mwhudson> lib/canonical/launchpad/images/edit-grey.png  lib/canonical/launchpad/images/link-grey-arrow.png
<mwhudson> wgrant: no, it's not visible
<mwhudson> (there's already a bug report about that)
<thumper> mwhudson: looking for a grey tick?
<mwhudson> should be getting more machines soon though...
<mwhudson> thumper: well
<mwhudson> thumper: playing around with the icon for "partial success", yeah
<mwhudson> tried a few things, i think a grey tick works best of the things i've tried
 * thumper nods
 * thumper is working through today
<mwhudson> i've made one in gimp
<mwhudson> i was actually trying to decide what to save it as :-)
<thumper> I thought we use american spelling in the code, so gray
<mwhudson> wfm
<wgrant> Partial success? Does that mean incremental imports are sufficiently close that UI is a concern?
<mwhudson> wgrant: yes
<mwhudson> the gimp still has a pretty weird ui
<wgrant> mwhudson: The single window mode is coming in the next release.
<thumper> just in time to be removed by default?
<mwhudson> wgrant: hooray
<mwhudson> thumper, wgrant: http://people.canonical.com/~mwh/partial-success.png
<thumper> mwhudson: looks good
<wgrant> thumper: It's still probably a year away.
<thumper> mwhudson: did you just desaturate the image?
<mwhudson> thumper: yeah
<wgrant> mwhudson: Nice.
<wgrant> It has a tooltip, I hope?
<mwhudson> wgrant: yeah
<mwhudson> "Partial Success"
<thumper> mwhudson: where is our list of contributor agreement signers?
<wgrant> We're mostly not in the team :(
<mwhudson> thumper: private url given to you in private channel
<thumper> :)
<lifeless> also the public team - you should get added wgrant :)
<wgrant> Does anybody actually use Empathy?
<jelmer> wgrant: Yep
<jelmer> mwhudson, thumper: Does the "determining revisions to fetch" step always take very long or just on the initial import?
<mwhudson> jelmer: just on the initial import
<jelmer> mwhudson: I don't think there's a lot we can do to work around that other than upgrade the svn on lp to 1.5
<mwhudson> jelmer: ok, want to fire off an rt asking how much work that would be?
<mwhudson> or i can
<wgrant> jelmer: With MSNP?
<jelmer> wgrant: yep
<jelmer> mwhudson: how urgent is it? Lucid will have a recent enough version of svn
<wgrant> jelmer: Odd. I keep trying to use it, but give up every time because it silently drops incoming messages once the conversation times out after a minute or so.
<wgrant> It doesn't drop messages for you?
<jelmer> wgrant: not that I have seen
<mwhudson> jelmer: maybe not all that urgent, it's only a performance thing after all
<wgrant> Subject: None
<wgrant> Thankyou Launchpda.
<maxb> FYI, I already have svn 1.6 *and* pysvn for hardy in a PPA
<_thumper_> mwhudson: https://code.edge.launchpad.net/~ian-clatworthy/bzr/whats-new/
<thumper> mwhudson: a bad error... know what caused it?
<thumper> mwhudson: my guess is the scanner, I just don't know why
<thumper> mwhudson: also, we should get bzr 2.1 final into LP
<mwhudson> thumper: no, that's the puller
<mwhudson> thumper: 2.1 is probably trivial, drop the tarball in, update versions.cfg and ec2 land
<thumper> mwhudson: does the puller set the stacked branch in the DB?
<mwhudson> thumper: yes
<thumper> ah
<mwhudson> that's a pretty nasty error indeed
<mwhudson> maybe duelling pullers?
<mwhudson> a remirror will probably clear it up
<thumper> hmm..
<thumper> mwhudson: it is past beer o'clock
<thumper> mwhudson: are you tweaking the code import dispatcher?
<mwhudson> thumper: yeah
<mwhudson> thumper: i'm thinking about it
<thumper> mwhudson: thinking about doing it Monday?
<mwhudson> or maybe i'll stop instead and go and sit outside until emma calls and tells me she's finished work
<mwhudson> thumper: yeah
<thumper> you know if you try again, you'll miss something
<thumper> do it Monday
<thumper> have a beer
<mwhudson> ok :-)
<thumper> I hope it is sunnier in Wellington than here
<thumper> we used to have sun
<thumper> but it has gone again
<mwhudson> it's a very very nice day today actually
<thumper> do you think the description should go above or below?
<mwhudson> sunny, not too windy
<thumper> I'm thinking above
<mwhudson> bit cool perhaps
<mwhudson> thumper: did you make that screeny?
<thumper> yeah
<thumper> 6
<thumper> http://people.canonical.com/~tim/description6.png
<mwhudson> oh right
<mwhudson> i'm not really sure
<mwhudson> i guess it makes sense for it to go high up the page
 * mwhudson EOWs
<poolie> cheerio
<poolie> thumper: really a description and a commit msg and a comment?
<thumper> poolie: the editable description is instead of the initial comment
<poolie> oh ok
<poolie> more like a bug then?
<thumper> poolie: trying to be
 * thumper EOWs too
<Ursinha> thumper: are you still there?
<Ursinha> thumper: if so, do you have an example of usage of the branches search by date?
<kfogel> thumper: looking rocketfuel-setup, I see that it does 'bzr branch lp:~launchpad-pqm/launchpad/devel $LP_TRUNK_NAME' after attempting 'bzr launchpad-login $lpusername'.  This means that if the user has a Launchpad login (and successfully logs in) that the 'bzr branch' will happen over bzr+ssh://, right?
<kfogel> thumper: oh, I see your answer in https://answers.edge.launchpad.net/launchpad/+question/101139
<kfogel> thumper: which I think says that
<poolie> hello kfogel
<kfogel> poolie: hello
<wgrant> BjornT: Can you please review https://code.edge.launchpad.net/~wgrant/launchpad/sprbu-columns-to-sprb/+merge/18995 reasonably soon?
<henninge> Morning!
<henninge> How do I find out the current database user in a unit test?
<adeuring> good morning
<BjornT> wgrant: sure. sorry, i had missed that one, thanks for reminding me.
<wgrant> BjornT: Thanks.
<mrevell> Hi
<deryck> Good morning, all.
<wgrant> bigjools: The circular import may not be a good reason to not do it, but is there a good reason *to* do it?
<bigjools> because Attribute sucks and it takes 60 seconds to do the right thing
<bigjools> and on that note, I am heading out for a liquid lunch while my power is off
<wgrant> Why is db-devel so ancient?
<deryck> BjornT, ping
<BjornT> deryck: pong
<kfogel> deryck, adeuring: just fyi (re my mail about failing test): the EC2 run that I submitted right after that has now passed, so I'm going to pqm-submit my changes for bug #255868.  sinzui already reviewed.
<mup> Bug #255868: Project summary page should show links to patches <story-patch-report> <ubuntu-upstream-relations> <Launchpad Bugs:In Progress by kfogel> <https://launchpad.net/bugs/255868>
<deryck> kfogel, sounds good.  Was going to look later, but glad it's passing now.
<deryck> kfogel, it looked to me like your ec2 run was against devel instead of db-devel, based on the log messages you sent.
<adeuring> kfogel: I haven't received your mail (yet)...
<kfogel> adeuring: was to bugs team members (all)
<kfogel> deryck: based on the two outputs (failure and then success), you're right, and yet the command line was the same and specified db-devel.  hmmm.
<adeuring> kfogel: our mail server seems to be a bit slow sometimes...
<kfogel> deryck: I'll keep the records; would like to figure out what happened here -- a lost EC2 run costs a lot.
<deryck> kfogel, you use `utilities/ec2 land` to make sure the syntax is good, too.
<deryck> kfogel, I think most of us use that now, rather than ec2 test.
<kfogel> deryck: yeah, should just do that
<kfogel> ok, submitted.  out for a while
<BjornT> bigjools: ping?
<bigjools> BjornT: yo
<BjornT> bigjools: we currently have both DistributionSourcePackage and DistributionSourcePackageCache in the db. it feels like we should combine those two, but i'm not sure what the latter is used for. can you enlighten me?
<bigjools> BjornT: it's for searching; the distro search page uses it
<bigjools> gets updated every day
<bigjools> there's an fti on the various columns
<BjornT> bigjools: would  make sense add settable columns to it? i.e., have it represent a distribution source package in the db, which can have bug reporting guide lines for example.
<bigjools> BjornT: I don't follow what you mean
<BjornT> bigjools: basically, i think that the DistributionSourcePackage python class should be connected to a table in the db. this table should both have some properties for the package, for example bug reporting guidelines, which is settable by users, as well as acting as a cache for certain properties, like the latest publication record.
<BjornT> bigjools: would it make sense to use DistributionSourcePackageCache for this? is would it be better to keep it for searching only?
<bigjools> BjornT: there is already a DistributionSourcePackageInDatabase !
<bigjools> I would keep it for searching only
<bigjools> DistributionSourcePackage and DistributionSourcePackageInDatabase should be merged; that was always the intention
<bigjools> and you'll notice the latter has the bug_reporting_guidelines on it
<BjornT> bigjools: yes, i know. my question was whether it would make sense to merge DistributionSourcePackageInDatabase and DistributionSourcePackageCache
<bigjools> BjornT: I don't think it does
<bigjools> partly because the latter is also archive-specific
<bigjools> i.e. it gets generated with knowledge about publications
<BjornT> bigjools: ok, makes sense. DistributionSourcePackageCache should probably get renamed as to not cause that much confusion.
<bigjools> yeah, that's cool
<bigjools> naming is the hardest part of programming, after all :)
<BjornT> bigjools: do you have time to have a quick call to talk me through https://code.edge.launchpad.net/~wgrant/launchpad/sprbu-columns-to-sprb/+merge/18995? i'm interested in the model's PoV
<bigjools> BjornT: yeah can do
<bigjools> call me on skype when you're ready
<BjornT> cool
<jtv> BjornT, can I bounce something off you?
<jtv> BjornT: we have a bunch of fake transaction managers in tests.
<jtv> Would it be a good idea to make that a test helper?
<BjornT> jtv: could be, if they are similar enough.
<jtv> BjornT: looks like.  In fact this is one reason why I still like to pass transaction managers around: it's a reasonably clear injection point for tests.
<adiroiban> gmb: hi. Any news regarding the landing of this MP https://code.edge.launchpad.net/~adiroiban/launchpad/bug-522188/+merge/19395 ?
<gmb> adiroiban: !??! It appears to have disappeared again. This is vexing. I'll go and see if I can find it.
<adiroiban> :))
<gmb> adiroiban: It looks like the ec2 run just sort of died and didn't email anyone to say that it had. Starting a new run now; I'll keep an eye on it.
<adiroiban> gmb: np. I will remind you on monday if it will fail again
<gmb> adiroiban: If that happens I'm going to take a very large axe to our ec2 utility.
<adiroiban> :)
<jtv> abentley: preparing to land your branch-url branch
<abentley> jtv, rock.
<jtv> abentley: that's a whopping 4-branch landing.  :)  Can I just land the last branch (after pumping) and the whole pipeline will be included?
<abentley> jtv, yes.  As long as all your stuff is fully merged into mine.  You can check that with bzr missing.
<jtv> abentley: it says "you have 15 extra revision(s):"
<jtv> followed by... yes, 15 revisions.
<jtv> abentley: does that mean I'm good to go?
<abentley> jtv, well, it depends whether the extra revisions are in my branch or yours.
<jtv> abentley: I merged yours into mine.
<abentley> jtv, okay, never mind then.
<jtv> abentley: and all those 15 revisions are mine.
<abentley> jtv, the general answer to your question is yes.
<jtv> \o/
<abentley> If you want to land multiple branches, and all the branches have been merged directly or indirectly into one of them, you can just land that one.
<abentley> pumping is just a convenient way of ensuring all the branches are merged into the last pipe.
<jtv> convenient indeed... it's been fun!
<sinzui> EdwinGrubbs: I like your proposal. I'll reply with some questions about how communities complete for the involvement portlet.  Let's assume we are going to extend the Involvement portlet instead of creating a new portlet
<deryck> Have a good weekend, everyone.
<maxb> I have a MP that is "review approve"d but not "merge approve"d. Would someone possibly be around who could flip that setting and ec2 land it?   https://code.edge.launchpad.net/~maxb/launchpad/bug-497731/+merge/19732
<rockstar> gary_poster, hi
<gary_poster> hi rockstar
<gary_poster> (I'm about to turn into a pumpkin btw, but can reply using cell phone in a bit)
<rockstar> gary_poster, so, if I need to create a new script in bin/ then I need to create a .in file in buildout-templates/bin, but where do I tell buildout about it?
<gary_poster> rockstar, if it is one of those kinds of templates--free form--then just putting it in the directory is sufficient.  rerunning bin/buildout will notice and build.  Note that there are other kinds of scripts that let you keep your code in the tree (see lp-windmill, for instance, and how it is hooked up in buildout.cfg [scripts] entry-points, but I imagine you've seen those and rejected them with reason.
<wgrant> Can somebody please ec2 land https://code.edge.launchpad.net/~wgrant/launchpad/sprbu-columns-to-sprb/+merge/18995 for me?
<lifeless> lp-project-upload looks nearly-useful
<lifeless> but I really don't want to depend on ubuntu-dev-tools for an upstrea project
<lifeless> any suggestions on where I should ask pitti to move the code to ?
<wgrant> lifeless: Maybe launchpad-tools, the collection of useful Launchpad-related scripts?
<lifeless> wgrant: I'm thinking lptools
<lifeless> https://bugs.edge.launchpad.net/lptools
<lifeless> is that what you were meaning?
<lifeless> dobey: btw
<wgrant> lifeless: I meant a hypothetical thing, the lack of existence of which was a bug.
<lifeless> dobey: _puhlease_ beat statik into giving you some work time to do QA [we do allow for this when we open source something, so that we're not doing abandon-ware]
<lifeless> wgrant: hah. Well it exists but is named differently.
<wgrant> Why has db-devel not been merged into for a couple of days?
<dobey> lifeless: was off today, so was out doing other things in the nice warm sun :)
<lifeless> fair 'nuff
<lifeless> dobey: I'd love to see you do an upstream release :)
<dobey> lifeless: During this cycle, I don't even really have enough work time to finish the stuff for Lucid. But I think after Lucid, I want to switch to our ops+ team for 6-12 months, and do a lot of work on infrastructure/tools, and lptools would certainly fall into that realm
<dobey> lifeless: yeah, I know. I would if I had any time to fix the bugs and do it :)
<lifeless> dobey: just doing a release would be helpful
<lifeless> it would mean that the packaging would be a little more sane.
<dobey> lifeless: I'll see if I can't do that. I need to do a release for changeup also
<lifeless> changeup ?
<dobey> A fairly simple thing to make it possible to automatically restart ubuntuone-syncdaemon, gwibber-daemon, and similar 'behind the scenes' user-level services, on upgrades
<lifeless> dobey: also if you can merge the non-packaging parts of bzr+ssh://bazaar.launchpad.net/~lifeless/debian/sid/lptools/sid/ that would be really nice.
<dobey> what's non-packaging in there?
<lifeless> dobey: e.g.: bzr merge lp:~lifeless/debian/sid/lptools/sid/; bzr revert debian; bzr revert --forget-merges; bzr commit -m "whatever"
<lifeless> dobey: have a look :>
<dobey> hrmm
<lifeless> a bunch of work from ted
<lifeless> desktop files
<lifeless> a preferences dialog
<dobey> sounds frightening
<lifeless> https://code.edge.launchpad.net/lptools/+activereviews
<lifeless> would be good to merge
<dobey> I wish I could get the same list of reviews from the API, as is shown on http://code.launchpad.net/~dobey/+activereviews
<lifeless> it has most of the changes I'm talking about
<lifeless> dobey: you can now I believe
<lifeless> or not quite; you can get reviews-requested-for-an-object
<dobey> yeah, I just haven't had time to look at his branch because it's so big
<lifeless> which is not the same
<lifeless> dobey: JFDI, honestly, everyone is running his branch.
<james_w> dobey: you can now get those two reviews on your +activereviews page with lp.me.getRequestedReviews() or similar
<lifeless> james_w: I thought that didn't handle group membership yet?
 * lifeless really wants a direct match to +activereviews too
<james_w> but those reviews are requested of dobey n'est pas?
<lifeless> in this case
<james_w> hence "those two reviews"
<lifeless> it may be that +me/activereviews isn't showing what it should either ;>
<james_w> it's a start
<james_w> so, one API request to get them all would be good
<james_w> and just returning a list of merge proposals is fine, as you can sort client side
<james_w> so we need a parameter to getRequestedReviews() to do a recursive lookup, then a modification to the internal method it calls
<dobey> There's a 3rd review on a private branch that isn't directly requested of me, but is via group membership
<james_w> do you know if there is a way to get the transitive closure of team memberships for a person?
<james_w> dobey: ah, I can't see that one, indeed you can't get that easily with this API yet
<james_w> I'd like to fix that, but it's more than just exposing something over the API like the current version
<dobey> I don't really care about sorting. That's a trivial problem. :)
<dobey> I just want all the reviews pertaining to me
#launchpad-dev 2010-02-20
<lifeless> dobey: don't suppose you're still around?
<lifeless> what should be done with stable and db-devel are auto-merge-conflicting
<wgrant> Somebody needs to resolve the conflicts and then just submit to db-devel.
<wgrant> They're pretty easily resolved.
<wgrant> So it needs somebody with about two minutes of time and PQM privileges.
<lifeless> do you have pqm access?
<lifeless> [if not why not :P]
<wgrant> No :(
<nigelb> https://launchpad.net/~bongcaivang has been assigning bugs to himself.  anything can be done?
<wgrant> nigelb: eg?
<lifeless> wgrant: I sggested nigelb pop in here
<lifeless> wgrant: a new ubuntu spammer, need someone with the CHR bit set.
<nigelb> wgrant: https://bugs.launchpad.net/ubuntu/+source/ubuntuone-client/+bug/447090
<mup> Bug #447090: can not sign in <apport-bug> <i386> <ubuntuone-client (Ubuntu):Fix Committed by bongcaivang> <https://launchpad.net/bugs/447090>
<wgrant> lifeless: You have the power.
<lifeless> wgrant: do I?
<wgrant> lifeless: Yeah. ~lifeless -> ~launchpad -> ~registry
<lifeless> ah, I did't knw they had done that for all
<nigelb> here's the list of all the bugs he's assigned to himself (ugh
<nigelb> https://bugs.launchpad.net/~bongcaivang/+assignedbugs
<wgrant> In fact, ~canonical-bazaar is sufficient too.
<cody-somerville> To do what exactly? deactivate his account?
<wgrant> cody-somerville: Suspend, but yes.
<lifeless> suspended
<wgrant> lifeless: Emailed too?
<nigelb> thanks lifeless :)
<lifeless> wgrant: I filled out the reason field
<nigelb> we've been unassigning and he's be reassigning for quite some time
<wgrant> lifeless: The user will be unable to see that.
<nigelb> whole yday was cat and mouse
<lifeless> wgrant: I'm assuming that that would email him
<wgrant> I don't believe so.
<lifeless> bah
<wgrant> But let me check.
<lifeless> care to spend 30 seconds to check?
<wgrant> You might also want to look at the CHR docs on wiki.c.c. There might be something there.
<wgrant> lifeless: Suspending does not email the user.
<lifeless> wgrant: wasn't anything in the cheatsheet the other day (and all them should be faqs anyhow)
<wgrant> And there can't be any hooks that do it, because the second thing the button handler does is unset the preferred email.
<lifeless> not documented
<wgrant> Damn.
<wgrant> Send the user a nice email and ask them to contact feedback@ for objections, I guess.
<lifeless> yeah
<lifeless> feedback@launchpad.net right?
<wgrant> Right.
<lifeless> hmm, is that 6 or 7 bugs today
<wgrant> More more more!
<lifeless> I can't email them
<lifeless> suspended accounts are 404
 * lifeless asks a question
<wgrant> I have their old page open.
<lifeless> do you have their email
<wgrant> PMed.
<lifeless> thanks
<lifeless> mailed them,
<lifeless> bcc feedback@ so CHR folk have a record
<wgrant> Great.
<nigelb> lifeless, wgrant: both of you work for canonical?
<lifeless> nigelb: I do; wgrant doesn't, though we should fix that :P
<nigelb> hehe, file a bug ;)
 * wgrant just got hit in the face by a Windmill.
<wgrant> Must the test suite really pop up Firefox windows?
<lifeless> wgrant: /o\
<lifeless> wgrant: xvfb-run
<wgrant> lifeless: Indeed.
<wgrant> I just didn't think my test expression would catch any Windmill tests.
<lifeless> nobody expects the windmill
<wgrant> Very true.
<wgrant> BjornT: Is ILFA actually exportable?
<wgrant> It's not exported at the moment, and everywhere else just exposes URLs.
<wgrant> (and Julian suggested it in the bug)
<BjornT> wgrant: oh, i was quite sure it was exportable, but it seems that they aren't.
#launchpad-dev 2010-02-21
<MTecknology> Please enable the 'multiverse' component in /etc/apt/source.list'
<MTecknology> s/source/sources/ **
<MTecknology> be nice if it said which line it wanted changed too
<MTecknology> Need to get 438MB of archives. After unpacking 1,210MB will be used.
<MTecknology> wow
<wgrant> Add "-o Apt::Install-Recommends=no" to the apt-get command line.;
<MTecknology> wgrant: little bit of a difference :P
<MTecknology> Any ideas what went wrong here?  http://paste.ubuntu.com/380738/
<wgrant> MTecknology: You didn't follow the instructions.
<wgrant> You have not run launchpad-database-setup
<MTecknology> ./utilities/launchpad-database-setup
<MTecknology> I rtan that
<MTecknology> I ran it wrong.....
<MTecknology> wgrant: thanks
<MTecknology> wgrant: so now that it's running I go to http://launchpad.dev and I see "It works!"
<wgrant> MTecknology: You probably haven't restarted Apache sufficiently.
<MTecknology> wgrant: I ran /etc/init.d/apache2 stop before runni make run; I only installed openssh-server and vim before running that script
<wgrant> MTecknology: ls /etc/apache2/sites-enabled
<wgrant> Do Zopeless emails not get send at all in the dev config?
<MTecknology> wgrant: I said that in this channel too - just use your imagination :P
<MTecknology> wgrant: http://paste.ubuntu.com/380760/
<wgrant> MTecknology: That worked fine.
<wgrant> The appserver is running.
<MTecknology> wgrant: I just meant line 24,25
<MTecknology> wgrant: but I can'
<MTecknology> t get to it..
<wgrant> MTecknology: What does launchpad.dev say now?
<MTecknology> Unable to connect
<MTecknology> I put an entry in my hosts file '192.168.1.6   launchpad.dev'
<wgrant> Don't you mean 127.0.0.1 launchpad.dev?
<wgrant> Er, 127.0.0.88 launchpad.dev
<wgrant> rocketfuel-setup also does all that for you.
<MTecknology> I'm running that in a remote vm
<wgrant> Ah. Then you'll need to alter the Apache configuration to listen on a non-loopback interface.
<MTecknology> oh
<MTecknology> wgrant: wow.. that's a massive apache config
<wgrant> It's not that big..
<MTecknology> the ones I use for websites are generally 50 or less
<wgrant> Yes, but Launchpad isn't a website.
<wgrant> It's a horribly oversized behemoth of a web application.
<MTecknology> :P
<MTecknology> wgrant: kinda too bad you can't have a core then a different web app for each of the services available
<MTecknology> like bugs, answers, etc.
<wgrant> Indeed.
<MTecknology> 20 different bzr branches put together in a careful manner so they're all inner-twined; looks messy :P
<wgrant> Huh?
<MTecknology> find . -name .bzr | wc -l
<MTecknology> 20 branches
<wgrant> One main branch, and 19 external dependencies.
<wgrant> Because somebody hates using packages.
<MTecknology> oh
<wgrant> See sourcecode/
<wgrant> Those are all external deps, not part of LP.
<wgrant> Except launchpad-loggerhead, but that one's just strange.
<MTecknology> 1.1GB of launchpad source and deps :P
<MTecknology> wgrant: I think you guys should release a version that handles users, teams, and branches :P
 * wgrant is not really a Launchpad guy.
<MTecknology> I think the launchpad guys should do that :P
<MTecknology> That's the part I really want to dig around in
<MTecknology> wgrant: thanks for the help
<mwhudson> launchpad-loggerhead should just be part of the lp tree
<wgrant> mwhudson: I thought I heard talk about it maybe being moved soon after the open sourcing.
<mwhudson> possibly from me!
<mwhudson> mostly a matter of tuits plus a little making sure things don't fall over on rollout
<wgrant> Yeah.
<thekorn> ahrscheinlich zu kalt ;)
<thekorn> upps, soory
<Pilky> beuno: ping
<mwhudson> morning
<mwhudson> erg
<thumper> morning
<Pilky> anyone around who does (or is interested) UI/UX/Usability stuff on launchpad, wanted to discuss some ideas I'd had
<thumper> Pilky: what sort of ideas have you had?
<Pilky> well a while ago I posted some minor UI change mockups to the list, but I've been working on mocking up some more dramatic changes, aimed at making the UI far more approachable
<thumper> Pilky: interestingly we've recently changed some ways we are dealing with ui changes
<thumper> Pilky: the way to get the main designers to look at it is to post to the launchpad-dev mailing list with [RFC-UI
<thumper> damn
<thumper> [RFC-UI] in the subject
<Pilky> ok cool
<thumper> Pilky: this will get the right people looking and commenting
<Pilky> thanks for the advice, I'll send a post now
<thumper> ok
<wgrant> When was that change announced?
<wgrant> Ah, there.
<Pilky> thumper: I've posted to the list if you're interested
<thumper> Pilky: ok, cool
<wgrant> Are there no provisions to put aside some regular amount time in each team for polish?
<thumper> wgrant: yes there is, it is called slack :)
<wgrant> This sort of thing seems to be common:
<wgrant> 1) Release big new unpolished feature of complete UI overhaul.
<wgrant> 2) People file bugs about all these trivial but glaring UI issues.
<wgrant> 3) These bugs sit around for three years.
<wgrant> s/of/or/
<Pilky> wgrant: I think the main UI issues with launchpad aren't really trivial
<wgrant> Pilky: No, but there are lots that are.
<Pilky> the core issue is that it has one of the least appealing and approachable UIs out of all the major code hosting sites
<wgrant> Those are the ones that are obviously bad.
<Pilky> oh yeah there's lots of things like that
<Pilky> but what I've tried in the mockups I posted to the list is to fix the core problem
<Pilky> w
<Pilky> which in turn fixes some of the trivial problems
<wgrant> I mean things like bug #524322.
<mup> Bug #524322: Excessive gaps in branch subscriber portlet <trivial> <ui> <Launchpad Bazaar Integration:Triaged> <https://launchpad.net/bugs/524322>
<wgrant> It should have been starting every Launchpad developer in the face for months.
<mwhudson> wgrant: i can honestly say i've never noticed that bug
<wgrant> Intriguing.
<Pilky> how is the UI actually managed for launchpad, is there someone who does all the UI work, or is it up to the team working on a feature?
<Pilky> because there are some really good bits of UI dotted around launchpad, eg the icons next to links
 * wgrant likes the mockups.
<Pilky> but they can be right next to some bad bits of UI
<wgrant> Launchpad no longer has a UI designer.
<jml> Pilky, Launchpad is a massive application that has been developed for five years.
<wgrant> jml: But the important templates were completely redesigned 6 months ago.
<Pilky> jml: yeah I appreciate the size of launchpad
<jml> wgrant, which is still only a fraction of the UI. It's much, much more than templates.
<wgrant> The really important one (the home page) was, as I recall, designed a couple of days before release by one person.
<wgrant> jml: True.
<Pilky> jml: but the issue is the current UI makes it seem like a big application, which can seem quite daunting
<Pilky> I don't believe powerful has to mean complicated :)
<jml> Pilky, the way things work is that the team working on the feature tends to drive most of the stuff, UI mockups get circulated and commented on. We used to have a UI guy dedicated to us, but now we don't, so we work with them.
<jml> Pilky, sure, I completely agree.
<jml> Pilky, I'm just saying that observing properties of the UI and trying to guess how we do our UI work based on that is going to be fraught with peril :)
<Pilky> heh ok
<Pilky> but I'd definitely like to help out UI wise
<Pilky> hence me doing these mockups
<jml> Pilky, which are much appreciated.
<jml> (although I haven't looked at these last ones, I'm at a Twisted sprint atm and ignoring large chunks of my mail)
<Pilky> heh
<jml> Pilky, the fact that the new user experience of Launchpad is so bad has been bugging me for a while now
<Pilky> yeah, I think the main reason for that is the hassle of creating a new project
<Pilky> I have lots of smaller open source projects that I'd like to put on launchpad, but I just zip the source and upload to my server because it is less hassle
<jml> Pilky, that's certainly one aspect, I'm not sure it's the main one
<jml> Pilky, sinzui & co. are working on project registration stuff soon
<jml> Pilky, see https://dev.launchpad.net/RoadMap
<Pilky> hmm
<Pilky> well setting up a project was one of the next things I was going to look at mocking up next weekend.
<wgrant> Wow, BranchRevision really has a full record of every revision in every branch? That is absolutely insane.
<mwhudson> wgrant: ya think?
<thumper> wgrant: is someone landing your branches for https://bugs.edge.launchpad.net/launchpad-code/+bug/509892
<mup> Bug #509892: Store upload log for SourcePackageRecipeBuilds <wellington> <Launchpad Bazaar Integration:In Progress by wgrant> <https://launchpad.net/bugs/509892>
<Pilky> Anyway I'm heading offline, thanks for the info and feedback! :)
<Pilky> night
<wgrant> thumper: No. There's a successor that is currently in review, but I need to talk to Julian about it. https://code.edge.launchpad.net/~wgrant/launchpad/sprb-new-model-columns/+merge/19172 is good to land, if you could do the honours.
<thumper> wgrant: I've assigned your work to 10.02
<thumper> wgrant: that one is dependant on an earlier one, all good to go?
<thumper> wgrant: can I trust you to not push extra unreviewed revisions?
<wgrant_> thumper: Hopefully.
<thumper> :)
<thumper> wgrant: you need a commit message on the merge proposal
<wgrant> thumper: Fixed, sorry.
<wgrant> (The squeak-vm thing he speaks of is in Launchpad as an Ubuntu package only.)
<thumper> wgrant: ec2 landing your work
<wgrant> thumper: Thanks.
 * mwhudson officially doesn't care about this #launchpad conversation any more
<lifeless> sinzui: are you around per-chance? you seem to be the reviewer than made the 'squeak' project inactive; keithy is active there and wants to use lp for some stuff
<thumper> what's the easy way to get a launchpadlib object linked to edge?
<thumper> isn't there a one liner or something?
<wgrant> Launchpad.login_anonymously('some consumer', 'https://api.edge.launchpad.net/')
<lifeless> wgrant: lp.load() ?
<wgrant> lifeless: That grabs an object from the webservice using an existing launchpadlib connection.
<thumper> wgrant: gah, where is the Launchpad class?
<wgrant> thumper: launchpadlib.launchpad.Launchpad
<wgrant> Also, you can use launchpadlib.launchpad.EDGE_SERVICE_ROOT rather than the magic URL.
<thumper> wgrant: I don't have login_anonymously
 * thumper wonders why not
<wgrant> thumper: You are on Karmic.
<thumper> wgrant: yes I am
<james_w> 'edge' works instead of EDGE_SERVICE_ROOT on newer releases too
<wgrant> thumper: Use Launchpad's launchpadlib, or Launchpad.get_token_and_login('some consumer', EDGE_SERVICE_ROOT)
<thumper> wgrant: using login_with
<wgrant> Or that.
<thumper> WTF?
<thumper> 404
<thumper> on the token
<thumper> grr
<thumper> GRRR
<lifeless> wgrant: if load requires an existing connection, it wanting an absolute url is at odds with its core
<wgrant> lifeless: It needs to be authenticated (if only anonymously), so it needs to be run on an existing object.
 * wgrant headdesks.
<lifeless> wgrant: ?
<wgrant> #lp
<thumper> my version of launchpadlib isn't working :(
<wgrant> thumper: What's it not doing?
<wgrant> Still 404ing?
<thumper> yeah
 * thumper tries bin/py
<wgrant> What was the request URL?
<thumper> http://pastebin.ubuntu.com/381232/
<thumper> wgrant: any ideas?
<wgrant> thumper: Does EDGE_SERVICE_ROOT by any change have /beta on end?
<wgrant> Mine doesn't, but this new version might be version-aware.
<wgrant> Yes, Karmic's needs the /beta.
<thumper> yes
<mwhudson> jml: didn't you fix the default on ec2 test to be --headless?
<jml> mwhudson, no :( I included that fix as part of my subunit-by-default branch, which I haven't landed.
<thumper> mwhudson: I don't think the default was headless
<mwhudson> thumper: it's not
<jml> mwhudson, I'm still shaving that yak.
<mwhudson> thumper: but i know i talked to someone about changing it
<mwhudson> jml: ah ok
<thumper> ok
 * thumper runs to collect maia
 * wgrant wishes that there were WIP MPs for all branches.
<beuno> wgrant, me too
<beuno> thumper too
<beuno> someday!
<wgrant> A good first step would be to allow me to create a WIP MP without spamming everybody.
<wgrant> For now it seems to email people just like a normal MP :(
<beuno> that should be easy-ish to do, no?
<wgrant> Very probably.
<beuno> I'm sure thumper can hack that up in 5 minutes when he gets back
<beuno> tests included
<wgrant> Does Zopeless mail get entirely dropped in the dev config?
<wgrant> It doesn't seem to go anywhere that I can find.
<beuno> I don't know anything about that area
<mwhudson> wgrant: thumper is working on that this cycle
<wgrant> mwhudson: Yay.
<mwhudson> wgrant, beuno: but you need to stop treating the initial comment as the description
<mwhudson> or the mail that gets sent out when you do ask it for review will very likely not be useful
<wgrant> Right.
<mwhudson> so that's what he's actually working on now
<beuno> perfect, solved
<beuno> mwhudson, so he's finally moving the description into it's own field?
<mwhudson> beuno: yeah
<beuno> super great
<wgrant> Very good.
<beuno> I've experienced using MPs differently this week
<beuno> so I've felt some pains I haven't before  :)
<mwhudson> :)
<mwhudson> you should talk to thumper about them i guess
 * mwhudson lunches
<wgrant> beuno: BTW, I really hope that U1 Notes fix hasn't been rolled out, because it's still an issue.
<beuno> wgrant, the part I did has, that part that blanks your notes hasn't
<beuno> I basically removed the auto-save function that was broken in many ways
<wgrant> I meant bug 452689.
<beuno> ah
<beuno> right
<beuno> I think I flipped that my mistake
<beuno> wgrant, can you check on edge?
<wgrant> edge.o.u.c?
<beuno> yessir
<wgrant> beuno: Still buggy.
<beuno> hrm
<beuno> wgrant, can you comment on the bug?  the fix should be on edge
<wgrant> It probably has been fixed. Just not properly.
<beuno> that would be good to know ASAP  :)
 * wgrant sighs.
<wgrant> The other one is still fixed, though. Good.
<beuno> maybe it didnt roll out to edge
<beuno> it was a crazy week
 * wgrant cannot work out how to save a contact.
<beuno> I'm not super sure you can today  :)
<wgrant> So it has an edit form with lots of nice inputs, but no save functionality. I see.
<beuno> I started a week ago, give me time
 * wgrant needs multiple prereq branches :(
#launchpad-dev 2011-02-14
<huwshimi> Do we use the main loggerhead branch in launchpad? Do we make any modifications to the template or is it just a vanilla deploy?
<lifeless> huwshimi: no, yes
<lifeless> huwshimi: I want yours and sinzuis opinion on this actually
<lifeless> I'd love it if we used releases without modifications, and did our tweaks outside the loggerhead tree
<lifeless> huwshimi: theres a merge proposal / bug that shows the upstream loggerhead theme, and ours.
<lifeless> huwshimi: do we need a custom theme, or perhaps we should just run the original - thats even simpler than doing some custom css for it in our tree and glueing them together.
<huwshimi> lifeless: Do you know which bug that is?
<lifeless> not offhand
<lifeless> start with launchpad-project/+activereviews
<lifeless> follow the loggerhead mps, you'll find it pretty quickly
<huwshimi> lifeless: I don't really know the history of loggerhead. Is it mostly just a launchpad project and therefore we can do what we like with the main branch?
<lifeless> no
<lifeless> its a bzr community project we earnt commit rights too
<lifeless> and was then abandoned upstream, so was adopted by mwhudson/lp-code for a while
<lifeless> then he left and they lost cycles
<lifeless> most recently, flacoste, poolie and I agreed that the LP team will maintain it, as we maintain lazr-js etc
<lifeless> which means:
<lifeless>  - do the right thing for it
<lifeless>  - fix bugs affecting us in sensible ways
<lifeless>  - do code review and bug triage for contributions by other users, whatever the their purpose
<huwshimi> lifeless: OK, so I've been tasked with making it look like Launchpad. Somehow I need to do that in a way that is maintainable.
<lifeless> huwshimi: interesting, cool.
<lifeless> huwshimi: so, we *could* change trunk to look like LP
<lifeless> huwshimi: but if we do that, we should still keep it looking and feeling nice for standalone users.
<poolie> hi lifeless
<poolie> re removing the oauth.py copy and using the system one instead:
<lifeless> huwshimi: And we'd not want to have assests that prevent it being easily deployed.
<lifeless> huwshimi: we can definitely add dependencies on this that LP also uses if that helps
<lifeless> huwshimi: we can also make it more modular/pluggable and change out integration glue
<poolie> do you know how i persuade buildout to import oauth from the system pythonpath, rather than trying to get it from an egg?
<lifeless> poolie: it should just do it; make sure to remove the existing egg and tarball
<huwshimi> lifeless: If it is a separate entity it probably shouldn't change design every time LP does. But having said that if it is our devs doing all the work then they would have to maintain two sets of templates/css.
<lifeless> huwshimi: it is our devs
<poolie> hm apparently not http://pastebin.ubuntu.com/566810/
<poolie> maybe there's still something cached somewhere that makes it think so?
<huwshimi> lifeless: So doing it in trunk means less work, but means we can't have any LP specific stuff right?
<lifeless> did you delete the eggs/oauth* dirs ?
<wgrant> poolie: Ah, give up now.
<wgrant> You have to use the egg.
<wgrant> Since our other eggs require it.
<lifeless> wgrant: EWHAT
<wgrant> Ah, lazr.restfulclient doesn't actually require a particular version.
<wgrant> So you may be able to make it happy with the system version.
<lifeless> huwshimi: so, the constraint is 'trunk must still be a viable standalone code browser'
<lifeless> huwshimi: that doesn't mean 'no lp stuff' but its likely easiest to frame it that way
<lifeless> huwshimi: *however* we can quite straightforwardly make hook points that our glue in lib/launchpad_loggerhead uses to combine with the generic stuff in trunk
<wgrant> poolie: Perhaps try lifeless' suggestion of removing the built eggs. But I don't like your chances; I've given up trying to fix things like this before.
<lifeless> huwshimi: that is a little more engineering, but certainly not twice.
<huwshimi> lifeless: How does this decision get made?
<poolie> wgrant, thanks for the advice
<lifeless> huwshimi: engineers doing the work
<poolie> seeing as it's only removal of cruft my perseverence is not going to be very high
<poolie> it's just a principle thing
<wgrant> Heh.
<wgrant> buildout gets a bit messy sometimes :/
<thumper> wgrant: hey
<thumper> wgrant: I'm going through your review comments
<thumper> wgrant: it isn't very easy to use the same text in the initial form
<thumper> wgrant: I have changed to "Built daily" rather than "Build daily" though
<wgrant> thumper: Hmm.
<wgrant> I still do not like the two completely different UIs.
<thumper> wgrant: the original form had "Built daily" with a checkbox
<thumper> wgrant: it was changed to make it more descriptive
<thumper> wgrant: I suppose we could change it back and add the help link
<thumper> wgrant: what do you think?
<wgrant> thumper: I think that would be best.
 * thumper goes to poke the code
<wgrant> Except in that case it could possibly be better as "Build daily"
<wgrant> We have no definition that I can see for the style of checkbox labels.
<thumper> wgrant: what do you mean?
<thumper> :(
<thumper> not a fan of the boring grey titles
<wgrant> thumper: Some of our checkbox labels are imperative, others indicative.
<wgrant> It makes me sad.
<poolie> turning into mpt?
<poolie> wbn to have a style guide for that
<thumper> someone needs to make sure we're consistent
<thumper> I think we did have a style for it
 * thumper goes to look
<wgrant> Uhoh.
<wgrant> Debian is considering source-only uploads again.
<thumper> wgrant: what is the impact of that?
<wgrant> thumper: Flamewars, probably.
<wgrant> Ubuntu has mandated source-only uploads from day 1.
<wgrant> Someone in Debian proposes it every so often.
<wgrant> But it inevitably gets nowhere.
<wgrant> Besides massive threads.
<thumper> wgrant: https://dev.launchpad.net/UserInterfaceWording is the page, feel free to edit
<wgrant> thumper: It's not very thorough, unfortunately.
<wgrant> We need UI review :(
<wgrant> eg. http://blog.launchpad.net/
<poolie> i get the same problem even with my entire egg directory deleted
<wgrant> Look at that first image.
<wgrant> poolie: What if you remove the egg from download-cache?
<poolie> for oauth?
<wgrant> Yeah.
<poolie> i think i already did that
<wgrant> So buildout cannot find it.
<poolie> hm, maybe not
<poolie> wgrant, the full stop at the end of the line?
<wgrant> poolie: Yes.
<wgrant> They are little things.
<poolie> with the ironic red arrow?
<wgrant> But they make the UI look far more awful than it actually is.
<poolie> i think we need ui review in the sense that everybody checks for it
<wgrant> Indeed.
<poolie> also, how ironic to get so many mp reviews about missing full stops in comments, and for this to get through in the ui
<wgrant> Heh.
<poolie> this may be an instance of a kind of thing that is easier to spot in review in a screenshot
<poolie> or maybe not
<poolie> but the punctuation perhaps stands out more from the code punctuation there
<wgrant> Apparently not.
<wgrant> Because there is a screenshot.
<poolie> on the mp?
<poolie> ah :)
<wgrant> I'm not sure if there was one there.
<wgrant> thumper: What's the easiest way to force failed scans to be retried?
<thumper> wgrant: lock and unlock the branch
<wgrant> thumper: There are a few hundred.
<wgrant> Is that still the easiest way?
<thumper> wgrant: I have a simple plugin that allows me to do that
<thumper> um...
<thumper> why did they fail?
<wgrant> This is from the package-import incident last week.
<wgrant> Which exposed the bzr stacking bug.
<poolie> ok, even with the download cache and all the eggs deleted, it still fails with "We have no distributions for oauth that satisfies 'oauth'"
<lifeless> there is an API to force a scan isn't there ?
<poolie> i looked for buildout documentation about this but i couldn't find any
<wgrant> lifeless: Does requestMirror still work?
<wgrant> I don't really recall.
<lifeless> poolie: is this in a clean tree ?
<thumper> wgrant: if you want to use launchpadlib, I think there is a method
<thumper> no, requestMirror doesn't work
<wgrant> :(
<poolie> lifeless, as in ' make clean'd tree?
<lifeless> no, as in freshly branched
<lifeless> or bzr remove-tree and then checkout again, kindof thing
<thumper> wgrant: it is the branchChanged method, which is triggered by the unlock code
<thumper> wgrant: and nothing is exposed over the api
<lifeless> poolie: I think you're well past diminishing returns
<thumper> wgrant: what I did was to get a list of the failed branches, and wrote a bash loop to process them with 'bzr rescan'
<thumper> wgrant: which takes advantage of my bzr-expert privileges
<wgrant> Yeah.
<wgrant> Can I give you a list of branches?
<thumper> wgrant: sure
<thumper> wgrant: pastebin?
<wgrant> poolie: Could you requeue the top p-i failure? 92 packages failed due to some unidentified librarian breakage for ~20 minutes yesterday.
<wgrant> thumper: Gimme a sec.
<wgrant> Hopefully I can grep it out.
<poolie> lifeless, you mean well past this being sufficiently easy to clean up that it's worth doing?
<poolie> i agree
<lifeless> yes
<poolie> regretfully
<poolie> ok
<poolie> i shall abandon it and explain where i got to
<poolie> pisses me off to have it be so hard to remove copy and paste though
<lifeless> agreed
<poolie> but, i shall be po'd about something more important
<lifeless> this is the problem with having 4 different dep systems in play
<poolie> wgrant, will do
<poolie> right
<poolie> wgrant, actually for the sake of proceeding towards handoff, let's ask a losa (spm?) to do it instead, and i'll help
<wgrant> poolie: I would, except spm seems to be even more excessively busy than normal.
<lifeless> u1 melted down in the weekend
<wgrant> Again.
<lifeless> no
<lifeless> uniquely
<wgrant> Oh.
<wgrant> Awesome.
<poolie> i came in this morning to have all my tomboy notes conflicted
<lifeless> I doubt he'll have time for anterior irritation amelioration today
<lifeless> or posterior :P
<poolie> wgrant, hm, so that's kind of a problem if we're not going to be able to do it without them in future
<poolie> anyhow
<wgrant> poolie: Well, we are getting lots of new LOSAs :)
<poolie> indeed
<poolie> that's 92 packages failed with key bzrlib.errors.InvalidHttpResponse
<poolie> is this a bug for it?
<poolie> i guess it was just an lp operational glitch?
<wgrant> The librarian was half-broken for a while.
<poolie> maybe udd should retry it
<wgrant> No idea why.
<wgrant> RIght.
<wgrant> I'd requeue_package --auto it
<maxb> Sourceforge have fixed their svn ssl cert, happily
 * maxb is restarting imports
<poolie> https://bugs.launchpad.net/udd/+bug/718483
<_mup_> Bug #718483: should delay and retry on http 503 from librarian <Ubuntu Distributed Development:Triaged> < https://launchpad.net/bugs/718483 >
<poolie> oh, will --auto be enough maybe?
<wgrant> I think so.
<poolie> ok done
<wgrant> thumper: http://paste.ubuntu.com/566819/ should be most of them.
<thumper> wgrant: http://people.canonical.com/~tim/new-recipe-built-daily.png
<wgrant> thumper: I'm not sure if the (?) belongs on the title or description. But that looks fine.
<wgrant> Consistency in terminology is the main thing.
<thumper> ok
<thumper> hmm... almost time to collect mini-mes
<lifeless> grah
<lifeless> these bug search pages are scary messy
<wgrant> The code behind them?
<lifeless> the structure & code
<lifeless> bug 717394
<_mup_> Bug #717394: Distribution:+bugs timeouts <timeout> <Launchpad itself:Triaged> < https://launchpad.net/bugs/717394 >
<lifeless> 3.5 seconds overhead on every ubuntu bug search, the results are effectively constant
<wgrant> thumper: Did you have any luck with rescanning those branches?
<thumper> wgrant: yep, over 100 triggered so far
<thumper> it is a little slow as there are 14 vfs calls
<wgrant> Ouch.
<wgrant> Thanks.
<thumper> wgrant: all rescans initiated
<poolie> wgrant, re your comment on bug 716175
<_mup_> Bug #716175: api calls fail while launchpad is readonly <Launchpad itself:Triaged> < https://launchpad.net/bugs/716175 >
<poolie> (i ask only for my education; i'm happy to leave it low)
<poolie> the "consumer check" is to do with the oauth validation?
<wgrant> poolie: Yes.
<wgrant> Even for anonymous requests it does a consumer lookup.
<poolie> is this the kludge that we create a temporary login for anonymous requests?
<wgrant> Possibly to allow us to blacklist.
<poolie> why does it do that?
<wgrant> So we can kill off misbehaving applications, I guess.
<LPCIBot> Yippie, build fixed!
<LPCIBot> Project db-devel build (363): FIXED in 5 hr 41 min: https://hudson.wedontsleep.org/job/db-devel/363/
<wgrant> thumper: Still around?
<lifeless> we have a very spec compliant oauth implementation
<lifeless> the standard solves problems we don't have
<wgrant> Oh?
<lifeless> you can do a much more direct solution to the 'hand out a cookie to login a client' problem
<lifeless> doing nonces in a layer on top of http is rather gruesome
<wgrant> Archive:+index is looking reasonably healthy now.
<wgrant> Might almost deserve to have its timeout override removed.
<wgrant> huwshimi: Do you have a plan for desaturating the buttons?
<wgrant> There are still 1.0-style large buttons around in the facet colours.
<wgrant> They have been deprecated for years, but are still used.
<huwshimi> wgrant: Where abouts are they?
<wgrant> Hmm. They're in the facet colours, but they don't seem to all correspond to the relevant facet. https://code.staging.launchpad.net/ has a few.
<huwshimi> wgrant: Ah yes;
<wgrant> They look very odd now that everything else is grayscale.
<huwshimi> wgrant: Well not *everything* is greyscale :)
<wgrant> Ahhh, they actually are app colours.
<wgrant> But Code uses them inappropriately.
<huwshimi> wgrant: but yes, they do need some love
<wgrant> There are 6 left.
<wgrant> Time for another icing purge, I think.
<lifeless> wgrant: so where is your target brnach?
<wgrant> lifeless: https://code.launchpad.net/~wgrant/launchpad/stop-using-targetnamecache
<wgrant> But alone it is not enough to disable the script, unless we want to take it out temporarily and hope that nobody notices sort drift while my rewrite makes it through the system.
<wgrant> Wow, Distribution:+search is quite a piece of work.
<lifeless> whats the name of the feature flag context manager
<wgrant> I only know of FeatureFixture.
<wgrant> thumper: When you're back, could you try rescanning one of the branches, to check that our DB purge worked?
 * lifeless does a naughty
<poolie> wgrant, that's the only contextmgr i know of too
<poolie> wallyworld, it seems to me there's a bit of a split here (in qatagger) between
<poolie> trying to get the right accurate model, which probably requires adding features to lp
<wgrant> https://lpstats.canonical.com/graphs/LaunchpadOpenBugs/20100215/20110215/ says launchpad has like 9600 bugs open.
<wgrant> https://bugs.launchpad.net/launchpad says 5800.
<poolie> vs just something pragmatic that people can use
<lifeless> wgrant: heh
<poolie> wgrant, one difference, though I don't know if it would account for all of them, is that lpstats tends to include private bugs
<lifeless> wgrant: also, https://bugs.launchpad.net/launchpad-project is what we need to look at.
<poolie> even private bugs nobody on the team can see
<wgrant> lifeless: I know, but that's only 1000 more.
<wallyworld> poolie: sorry to cut you off - i am just going to get Lachie from school - running late. i'll talk to you when i'm back
<poolie> np, no rush
<wgrant> Ah
<wgrant> Does not exclude dupes.
<wgrant> poolie: What's happening with the forking debugging?
<wgrant> Do we know what's happening, or should I land the cowboy to turn it off?
<wgrant> Who moderates blog comments? There are a couple waiting, and I'm wondering if I should approve them as I see them.
<LPCIBot> Yippie, build fixed!
<LPCIBot> Project devel build (438): FIXED in 5 hr 40 min: https://hudson.wedontsleep.org/job/devel/438/
<poolie> wgrant, i can moderate them
<poolie> you should be able to too?
<poolie> wgrant, i presume jam is going to look at the data on the bug, if any
<wgrant> poolie: I can, but I wonder if I should.
<poolie> too late!
<poolie> but, generally
<poolie> they were obviously not spam
<poolie> i don't see any benefit in sitting on them
<poolie> if in doubt about the policy, talk to someone else
<poolie> i think the more we post the better
<thumper> wgrant: which one
<wgrant> thumper: Any of them.
<wgrant> thumper: They all failed again.
<thumper> :(
<wgrant> thumper: Bug #718511
<_mup_> Bug #718511: Failed initial scans require manual cleanup <branch-scanner> <Launchpad itself:Triaged> < https://launchpad.net/bugs/718511 >
<thumper> what cleanup?
<wgrant> poolie: I agree.
<wgrant> thumper: Deleting the branchrevisions.
<wgrant> Which we have done.
<thumper> wgrant: lp:debian/wheezy/php-xml-htmlsax3
<wgrant> Thanks.
 * thumper has to make dinner for hungry munchkins
<thumper> wgrant: if you want the rest poked, let me know and I'll kick it again
<wgrant> thumper: You will hopefully be poked in a couple of minutes. Thanks.
<wgrant> thumper: Consider yourself poked.
<wgrant> It worked, thanks.
<wgrant> And p-i only has 72 jobs remaining.
<wgrant> Excellent.
<wallyworld> poolie: wrt qa tagging, the two goals you mentioned - accurate model and easy to use - don't have to be mutually exclusive :-)
 * thumper pokes the script again
<thumper> script is running
<wgrant> Thanks.
<wgrant> It's working.
<huwshimi> lifeless: Really like the queries time indicator. I have a question though. Every document load (before images, javascript, css etc.) takes about 2.5 seconds longer than the query time. Is that just the time it takes to execute the files, render the templates etc?
<lifeless> huwshimi: we are currently 100% overcommitted in the datacentre
<lifeless> huwshimi: so things are queueing a page in advance, and thats within 3 seconds, 99% of the time
<lifeless> huwshimi: its our top rt
<lifeless> huwshimi: or approx; I'm going to pounce on mthaddon today to try and get it moving; we should have traction on it this weke
<lifeless> huwshimi: bug 716760, which needs someone to grab it and run with it, is about measuring this gap
<_mup_> Bug #716760: no measurement of in-datacentre queue timings <Launchpad itself:Triaged> < https://launchpad.net/bugs/716760 >
<lifeless> huwshimi: (sort of actual transfer times)
<huwshimi> lifeless: It's interesting, once you have one bit of information (the query times) readily available, you notice things.
<lifeless> huwshimi: I mentioned that bug to stub, but I don't know if he's going to poke at it
<lifeless> huwshimi: yes, continual partial attention
<lifeless> huwshimi: I think it would be really excellent to also show the time after the initial document - time to get other assets, fire js etc etc
<lifeless> huwshimi: I expect a similar psychological effect
<huwshimi> lifeless: For example on the lp.net homepage about 75% of our load time is that load overhead (the queries, js, images etc. load in the other 25%)
<lifeless> wgrant: bug 31820
<_mup_> Bug #31820: mysterious exception during accept on ltsp 0.75 <lp-soyuz> <soyuz-upload> <tech-debt> <Launchpad itself:Triaged> < https://launchpad.net/bugs/31820 >
<lifeless> wgrant: wtf
<wgrant> lifeless: Probably librarian down.
<lifeless> wgrant: any reason not to invalid it and wait for a reoccurence with diagnostics ?
<wgrant> lifeless: I doubt it.
<wgrant> thumper: lp:debian/wheezy/libjconv lp:debian/wheezy/atom4 lp:debian/wheezy/netrek-client-cow
<wgrant> lp:debian/wheezy/biosquid lp:debian/wheezy/quilt
<wgrant> thumper: I missed those five. All the rest are scanned, though.
<thumper> wgrant: doing those now
<wgrant> thumper: Thanks.
<wgrant> p-i should finish in a couple of minutes.
<wgrant> thumper: Thanks, all scanned successfully.
<wgrant> poolie: Apart from nexuiz-data, we appear to be done.
<wgrant> Everything is scanned, p-i processing new Ubuntu uploads.
<poolie> hooray
<wgrant> poolie: Except https://code.launchpad.net/~ubuntu-branches/debian/wheezy/xserver-xorg-video-voodoo/wheezy
<wgrant> Created, but unlinked and unpushed.
<wgrant> Ah, the squeeze branch is pack-0.92!?
<wgrant> Yeah, we have some rich-root-pack branches.
<wgrant> How very odd.
<lifeless> Ursinha: hi
<lifeless> bug 718566 should be on loggerhead not launchpad
<_mup_> Bug #718566: DatabaseError on loggerhead trying to view a revision on staging <oops> <Launchpad itself:Triaged> < https://launchpad.net/bugs/718566 >
<Ursinha> lifeless, hello
<Ursinha> yes, I know
<Ursinha> am changing right now
<lifeless> coolio :)
<Ursinha> oh, I like the query count on the top
<lifeless> Ursinha: excellent
<mrevell> Hey
<_starbuck> morning mrevell
<mrevell> _starbuck, Nice nick
<_starbuck> :)
<_starbuck> thanks
<adeuring> good morning
<al-maisan> moin adeuring
<adeuring> salÃ¼ al-maisan!
<bigjools> Ursula has gone all BG on us
<spiv> Would some kind lp committer please shove https://code.launchpad.net/~spiv/launchpad/haproxy-for-twisted-services/+merge/48427 at ec2test or whatever the next step is?
<lifeless> ERROR:qa-tagger:Something went wrong when marking bugs: 'LPQAStagingDeployment' object has no attribute 'deployed_source_revno'
<lifeless> wgrant: just bug 717363 needs qa and we can has deploy.
<_mup_> Bug #717363: prf WalkerBase.walk can fail if self.open raises an exception <product-release-finder> <qa-needstesting> <Launchpad itself:Fix Committed by sinzui> < https://launchpad.net/bugs/717363 >
<lifeless> night all
<allenap> Cheerio lifeless .
<LPCIBot> Project devel build (439): FAILURE in 5 hr 18 min: https://hudson.wedontsleep.org/job/devel/439/
<LPCIBot> * Launchpad Patch Queue Manager: [r=thumper][no-qa] Use the new, correct lazr.restful-0.16 egg.
<LPCIBot> * Launchpad Patch Queue Manager: [rs=wgrant][rollback=12363] Revert r12363. The extra subquery makes
<LPCIBot> bug searches perilously slow for non-registry-admins.
<LPCIBot> * Launchpad Patch Queue Manager: [r=benji, bac][bug=717363] Recover from an open() failure when the
<LPCIBot> product-release-finder calls walk().
<LPCIBot> * Launchpad Patch Queue Manager: [r=lifeless][bug=68203] Do not accept whitespace comments when
<LPCIBot> reporting a bug.
<bigjools> dpm: hi, around?
<dpm> hi bigjools
<bigjools> dpm: hi - do you admin the ubuntu-translators list?
<dpm> yes
<bigjools> dpm: great - can you please unsubscribe help (at) launchpad.net, or it might be launchpad-help at canonical .com
<wgrant> Could also possibly be feedback@
<stub> lifeless: What got patch-2208-41-0.sql ?
<bigjools> not according to the mail headers
<dpm> bigjools, done
<bigjools> dpm: awesome, thanks
<dpm> np
<bigjools> yay for less email
<stub> bigjools: Any fallout from switching to debversion? There are other places in our schema we could switch too (eg. ProductSeries.name and other places with version_sort_key indexes)
<bigjools> stub: no, swimmingly well actually
<bigjools> seems a lot faster
<stub> Ta
<bigjools> Archive:+index has fallen out of the top 10 timed-out pages \o/
<wgrant> 99% @ 9.67
<bigjools> details details :)
<wgrant> Which is >3s better than before
<bigjools> the debversion thing plus some other improvements I made have helped a lot
<wgrant> Yup.
<wgrant> We have lost a few thousand soft timeouts a day :)
<bigjools> well the latest report is for the weekend which is not as busy, let's see tomorrow's
<wgrant> True.
<bigjools> wgrant: so, the last soyuz-report cleanup we can do is to turn the routine errors into non-oops
<wgrant> bigjools: Right.
<wgrant> The key errors.
<bigjools> FatalUploadError etc
<wgrant> Although I'm not sure about that NoneType has no email thing.
<wgrant> Is that the account/person mess again?
* henninge changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | firefighting: - | On call reviewer: - | https://code.launchpad.net/launchpad-project/+activereviews
* henninge changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | firefighting: - | On call reviewer: henninge | https://code.launchpad.net/launchpad-project/+activereviews
<bigjools> don't see that
<wgrant> 2 occurrences on the 11th.
<wgrant>         OOPS-1868PPA545, OOPS-1868PPA546
<bigjools> could be
<bigjools> wgrant: your config changes seem ok then, I think
<wgrant> bigjools: Thanks.
<wgrant> It makes the inheriting configs much simpler.
<bigjools> FSVO
<henninge> jtv: Any news on my MP?
<wgrant> The diff is bad. But the configs are nice.
<bigjools> who can I assign loggerhead questions to?
<wgrant> It's LP-maintained now.
<wgrant> So your squad :)
<jtv> henninge: no, was trying to figure out the XPI bug.  But I can get back to your MP if it's urgent.
<jtv> There was some mention of it not being critical any more.
<jtv> Does that still apply?
<henninge> jtv: it's not release-critical anymore because we had the release
<henninge> jtv: the bug still is and re-enabling Ubuntu imports depend on it.
<henninge> Yes, it is urgent. ;-)
<jtv> Ah!
<jtv> henninge: in that case,
<jtv> âwhy do you skip unreadable/nonexistent files instead of requiring the user to get their act in order?
<jtv> â why do you require a distroseries to be specified among the options but assume a distroseries to be specified?
<jtv> â where does the circular import on IDistributionSet come from, since this is a script?
<henninge> jtv: as mentioned, this is adapted from reupload-imports
<henninge> I had meant the script for my personal benefit initially but thought it might be useful for others, too.
<henninge> So, the answer to the first question is "to keep it simple" but I guess a warning would be appropriate.
<jtv> henninge: there is a warning, but it doesn't say what the problem is.  I say just dump the check.
<henninge> The answers to the other two are: "Because I copied that from the originial script.
<henninge> I like robust scripts, though.
<jtv> henninge: I don't recall off the top of my head but there may be a way to specify to optparse that a particular option is required.
<jtv> Typo in the test: is_trackging
<jtv> henninge: also, a copyright notice of 2009 on a new script in scripts/rosetta/ may be taking "adapting the existing script" a bit far.  :)
<henninge> I agree ...
<jtv> henninge: doesn't is_upstream_import_on_sourcepackage duplicate logic you had somewhere else?
<jtv> I thought you already implemented that part.
<henninge> no, I don't think so.
<jtv> Then sorryâwasn't sure so I thought I'd ask.
<jtv> Oh, there's a misplaced import in there.
<allenap> Does anyone know if the bug expiry stuff is meant to be running now?
<jtv> henninge: lines 777â778 of the diff
<henninge> jtv: you already reviewed this code, so maybe you are remembering that.
 * henninge looks
<jtv> ohhh could be
<stub> bouncy bouncy
<henninge> jtv: hm, I think I had circular problems on that one but I can check that again.
<jtv> henninge: even then, please move it to the top of the method and comment it.
<jtv> henninge: I'm having trouble figuring out what exactly _makeMessages in the test is supposed to do.  It crams a lot of scenarios into one method and it's not clear to me how the parameters are meant to interact.
<henninge> hm, does the docstring not help?
<jtv> That's after reading the docstring!
<henninge> Maybe I should add that is_tracking only makes sense if the other two are False?
<henninge> I had tried to think of other interfaces for the method but this was the best I could come up with.
<jtv> Why not have multiple methods?
<jtv> I mean, it helps to know that is_trackging doesn't combine, but it'd help more to have smaller and more self-evident chunks.
<jtv> Also, what's "this message" in the docstring?
<henninge> sorry, got called to the door
<henninge> jtv: there is a finite permutation of possible combinations that the method I am testing has to deal with.
<henninge> I had a table on paper and tried to have something else in code.
<henninge> Maybe I should change it to not use the tri-state bools.
<jtv> Well more parameters probably aren't the answer either.  The method already spends a fair portion of its space just trying to figure out what it was that the caller requested.
<jtv> Can't you do something along the lines of "makeIdenticalUpstreamMessage"?
<jtv> The tests themselves look good, by the wayânice and thorough, well-documented.
<jtv> (repeated typo in the tests: identcal)
<jtv> They're also nice and concise.
<wgrant> Hahaha
<wgrant> real	1m4.066s
<wgrant> Down from 18 hours.
<jtv> wgrant: what is?
<wgrant> jtv: update-bugtask-targetnamecaches.py
<jtv> wgrant: add some sleep()s to avoid surprises.
<jtv> Also, nice cache of future "optimizations" for those lazy days.
<wgrant> Indeed.
<henninge> jtv: expanded docstring
<henninge> http://paste.ubuntu.com/566935/
 * jtv looks
<henninge> jtv: I guess I could use _makeUpstream, _makeUbuntu and _makeSuggestion methods and pass other messages as is_identical into it.
<henninge> s/method/message/ on line 21 of that paste
<jtv> henninge: I think it'd be better to split it up, yes.
<henninge> I will have to carry pofile and potmsgset around which I was trying to avoid ...
<jtv> henninge: yes, that is a bit of pain.  But I suppose potmsgset would be implicit in the TM in most places.
<henninge> yes, I was just thinking about that.
<deryck> Morning, all.
<henninge> Hey deryck!
<jtv> hi deryck
<wgrant> henninge: Can I throw a review your way?
<henninge> wgrant: sure but please don't aim for any tender parts ...
<wgrant> henninge: https://code.launchpad.net/~wgrant/launchpad/bug-718004-rewrite-targetnamecache/+merge/49612
<stub> yay 70% packet loss between ISP and the USA. Think I'll go read a book.
<henninge> wgrant: That looks very good. Is it ok for you, though, if I finish this after lunch?
<wgrant> henninge: I'm about to vanish for the night, so I won't be around, but that's fine.
<henninge> wgrant: ok, g'night ;)
<wgrant> henninge: Thanks for reviewing.
<gary_poster> deryck, I figure if I exposed my ignorance enough I'd force someone, probably you, to respond :-) thanks, and I look forward to the "now this is what to do" email ;-)
<deryck> heh
<deryck> gary_poster, actually, I think bac is doing the right thing ;)
<deryck> finishing that reply now
<gary_poster> heh, ok cool, deryck, thanks
<adeuring> henninge-lunch: could you have another look at https://code.launchpad.net/~adeuring/launchpad/bug-702468/+merge/49205 ?
<bac> thanks deryck.  i was hoping you'd give your 2 cents.
<bac> gary_poster: are you fixing my blog post?
<bac> mrevell-lunch: does the 'front-page' *category* not put it on launchpad.net?  you have to add a tag?
<mrevell> bac, Yeah, it's a tag, rather than a category
<bac> what does the category do?
<gary_poster> bac, I was approving a comment, and was going to ask if you wanted me to meddle.  I approved it, and now I'll leave the rest alone for you :-)
<bac> can we get rid of that one as it confuses people like me
<bac> mrevell: ^^
<bac> gary_poster: thanks
<gary_poster> np
<bac> mrevell: do all comments have to be moderated?
<mrevell> bac, I didn't know there was a front-page category. Sure, let's kill it. As for comments ... from new people they do.
<mrevell> Once you've had one comment approved, it's open season.
<bac> mrevell: thanks
<LPCIBot> Project db-devel build (365): FAILURE in 2 hr 51 min: https://hudson.wedontsleep.org/job/db-devel/365/
<LPCIBot> Launchpad Patch Queue Manager: [rs=buildbot-poller] automatic merge from stable. Revisions: 12370,
<LPCIBot> 12371, 12372 included.
<jtv> crazy busy day
<jtv> henninge-lunch: test_accept_upstream_no_ubuntu says: "If there was already an upstream translation, but no ubuntu one, the suggestion replaces both."  There is only one translation to replace, of course; and might it be better to say "accepting the suggestion" rather than "the suggestion"?
<pindonga> I was wondering, how do you manage to automatically sign the deb packages created from recipes? (ie, without the user entering his passphrase?)
<pindonga> I'm trying to achieve this myself, but on a local machine
<maxb> pindonga: The signing keys of PPAs are stored by Launchpad. Not even the owner of the PPA has access to the private key.
<pindonga> maxb, so you have a script that does the signing
<pindonga> I'm using bzr dailydeb
<pindonga> and I want to automate the signing of the packages I create
<pindonga> you know any way I could do?
<maxb> In the way you mean, packages are not signed.
<maxb> Apt archives are what is signed
<pindonga> this is essentially for a plugin for tarmac I'm building
<maxb> What software are you using to build the Release and Packages files?
<pindonga> maxb, when I run bzr dailydeb and use the dput option it will push the dsc/changes stuff to launchpad
<deryck> henninge, abentley, adeuring -- coming, sorry.  2-3 more minutes.
<pindonga> for that it will sign the package
<pindonga> sorry, not pacakge
<pindonga> but changes/dsc files
<pindonga> I was wondering if there is a way to have the credentials stored in a way that doesn't require the user to put it in
<pindonga> bzr dailydeb uses debsign internally to sign the files
<jml> !!!
<maxb> Whether or not you have a passphrase set on the key is your own decision.
<pindonga> max, ah, so if I create a key without a passphrase that should work
<maxb> Obviously an unprotected key that enables any possessor to impersonate you to Launchpad is a dangerousl thing
<pindonga> maxb right
<jml> http://paste.ubuntu.com/566971/
<maxb> pindonga: I think we should continue this on #launchpad, as it is not -dev related
<pindonga> maxb, sure
<pindonga> thanks anyway
<jtv> henninge: you have my voteâsorry for the delays.
<jml> I just got this error when running a clean build of a branch.
<allenap> bigjools: Have there been any problems recently when adding new builders? I think I've found a bug where the buildmaster will create a new SlaveScanner for new builders every 5 minutes until it's restarted.
<jml> http://paste.ubuntu.com/566971/
<deryck> abentley, if you'll point me at that mp, I'll review later, after I finish a handful of emails.
<henninge> jtv: thanks!
<abentley> deryck, first I have to create the MP.
<deryck> ah, ok :-)
<jtv> henninge: on an unrelated note, I see that POTMsgSet.setSequence has a case for TranslationTemplateItem.sequence < 0.  I have _no_ idea what sense that would make.  Can you think of anything?
<henninge> jtv: I was not aware of that and nothing comes to mind about it.
<abentley> sinzui, could you please have a look at https://code.launchpad.net/~abentley/launchpad/merge-translation-script/+merge/49099 ?
<jml> allenap: because NewBuildersScanner doesn't update self.current_builders?
<allenap> jml: That's the badger.
<jtv> henninge: thanks, just checking my sanity.  :)
 * sinzui looks
<jml> allenap: I don't know of any production issues, but I guess I've just confirmed there's a logic bug in the code.
<henninge> jtv: I don't see what you mentioned, though ...
<allenap> jml: Okay, I'll file a bug. How would you triage it? I'm tempted to mark it critical.
<bigjools> allenap: urgh
<henninge> jtv: can you paste the version of setSequence you are seeing? Or give me the revno?
<jtv> henninge: there's a check for sequence >= 0 (which should always be the case), and then there's an else: pass.
<jml> allenap: I guess critical, but that's probably only because I'd just fix the bug :)
<henninge> oh, that's what you meant
<allenap> jml: Yeah, that was my plan too :)
<bigjools> allenap: critical
<henninge> jtv: that's because our style guide used to require that else
<abentley> sinzui, thanks.
<jtv> henninge: it's not thatâit's that there's an "elif" instead of an "else" in the first place.
<henninge> jtv: hm, you mean that condition should be checked in an assert?
<jtv> henninge: an "assert sequence >= 0" would have made more sense!  Maybe this grew out of an "if sequence > 0" that then also absorbed the "sequence == 0" case.
<bigjools> has anyone managed to land anything via ec2 lately?
<henninge> jtv: I agree
<bigjools> I'm on attempt #4 and it's still bloody failing in windmill
<jtv> Thanks.
<jtv> bigjools: I did, yesterday.
<bigjools> :/
<jtv> bigjools: I did run rocketfuel-setup first, IIRC, after hitting our windmill problem.
 * henninge relocates
<bigjools> how does that affect ec2?
<jtv> bigjools: newer AMI maybe
<jtv> I don't know how those propagate.
<bigjools> it just picks up the latest automatically
<sinzui> abentley: r=me
<abentley> sinzui, thanks.
<bigjools> actually, now it's showing connection refused errors in librarian tests
<bigjools> ec2 testing is about as reliable as a chocolate teapot lately.
<bigjools> and now bzr lp-land is crashing!  I am doomed.
<gary_poster> "about as reliable as a chocolate teapot
<gary_poster> "
<gary_poster> -> an analogy I would have never thought of
<gary_poster> and like :-)
<bigjools> gary_poster: the other one is a chocolate fireguard - fairly standard stuff this side of t'pond :)
<abentley> deryck, https://code.launchpad.net/~abentley/launchpad/share-existing-packagings/+merge/49634
<bigjools> bac: hey, the no-email-for-own-bug-actions is totally awesome
<deryck> abentley, got it, thanks.  Will review very shortly.
<bac> bigjools: yes.  danilos and gary_poster did a great job.  (i just got to announce it)
<bac> bigjools: and it was only six years in the making
<bigjools> is there a reason it wasn't done for questions?
<bac> bigjools: that is on our plate.  look for it in 2017
<bigjools> heh
<deryck> heh
<deryck> yeah, bac, danilos and gary_poster are to be praised for this work.  many lp users will rejoice.
<bigjools> we're not worthy
 * bigjools does the Wayne's World thing
 * deryck joins in since it was time for a stretch break anyway
<sinzui> henninge: do you have time to review https://code.edge.launchpad.net/~sinzui/launchpad/packagebugs-search-0/+merge/49479
<gary_poster> lol :-)
<henninge> sinzui: r=me
<sinzui> thank you henninge
<allenap> henninge: Do you have time to review a fix for bug 718761? https://code.launchpad.net/~allenap/launchpad/new-builders-scanner-calm-down/+merge/49645
<_mup_> Bug #718761: NewBuildersScanner schedules a new SlaveScanner every 5 minutes for new builders <Launchpad itself:In Progress by allenap> < https://launchpad.net/bugs/718761 >
<henninge> allenap: I am currently looking at another review but will pick yours up right after that.
<allenap> henninge: Thanks :)
<dobey> hey all
<dobey> did something break the API this morning?
<dobey> i keep seeing this in tarmac:
<dobey> NotImplementedError: Can't look up definition in another url (https://api.launchpad.net/1.0/#branch)
<leonardr> dobey: you're probably using edge, which is gone
<dobey> leonardr: ^^ that URL doesn't have edge in it though... hrmm
<leonardr> dobey: edge redirects you to api.launchpad.net, but launchpadlib thinks it's using edge
<leonardr> it's telling you that api.edge.launchpad.net served it urls from another server (api.launchpad.net)
<jml> is there a step in the nodowntime rollout process for marking "Fix committed" bugs as "Fix released"?
<jcsackett> jml: i believe people have done that manually.
<jcsackett> or did you mean in documentation?
<deryck> bigjools, did you ever get past your windmill test failure?
<jml> I meant the latter but I would have tolerated being pleasantly surprised wrt automation :)
<bigjools> deryck: yes but I didn't change anything
<bigjools> I get other errors in ec2 now :/
<bigjools> none of them to do with my branch of course, and since it was attempt #4 at ec2 I gave up and pqm-submitted it
<deryck> bigjools, ah, ok.  I've got a call and review and then can look to see if something stands out to me.
<leonardr> dobey: try this branch and see if it helps you https://code.launchpad.net/~leonardr/launchpadlib/fake-edge
<leonardr> henninge: would love a review of https://code.launchpad.net/~leonardr/launchpadlib/fake-edge/+merge/49651
<dobey> leonardr: i did this: https://code.launchpad.net/~dobey/tarmac/no-more-edge/+merge/49649
<leonardr> dobey: you don't need to import lpnet_service_root, you can just use the string 'production'
<leonardr> but you have the right idea
<leonardr> similarly 'staging'
<dobey> well i'd rather import the strings than using literals everywhere.
<LPCIBot> Project devel build (440): STILL FAILING in 5 hr 14 min: https://hudson.wedontsleep.org/job/devel/440/
<LPCIBot> Launchpad Patch Queue Manager: [r=lifeless][ui=none][bug=607960] Permit turning targetnamecache off
<LPCIBot> via malone.disable_targetnamesearch feature flag.
<leonardr> dobey: those constants aren't relaly meant to be re-imported
<leonardr> maybe i should change them to be the strings
<abentley> henninge, could we mumble about https://bugs.launchpad.net/launchpad/+bug/716586 ?
<_mup_> Bug #716586: Diverged translation cannot be converged again. <regression> <upstream-translations-sharing> <Launchpad itself:Triaged> < https://launchpad.net/bugs/716586 >
<henninge> leonardr, abentley: otp atm
 * jcsackett sighs.
<jcsackett> local codehosting is not the easiest thing to get working.
<abentley> henninge, oic
<abentley> jcsackett, what problem are you having?
<jcsackett> abentley: a general inability to push or pull branches to try and test mp stuff.
<jcsackett> abentley: when i try to push, i get an error about needing to supply create-prefix.
<jcsackett> when i try to pull, i get a NOT A BRANCH error.
<jcsackett> i'm rebuilding something right now, when that's done i can supply you more specific errors.
<abentley> jcsackett, are you running "make run_all"?
<jcsackett> abentley: no, i was not aware of run_all. i take it that gets everything else up and running?
 * jcsackett prepares to edit wiki page on development codehosting.
<abentley> jcsackett, yes.  "make run" doesn't start codehosting.
<jcsackett> abentley: fantasitc. thanks. :-)
<abentley> jcsackett, I've never used "make run_codehosting" like that page says, but I guess that works, too.
<jcsackett> abentley: make run_codehosting wouldn't run for me, thus trying make run.
<jcsackett> oddly, make run_all *is* working. i should find out why codehosting wasn't.
<abentley> jcsackett, that would be very good to know.
 * jcsackett nods.
<jcsackett> thanks, again, abentley. this has gotten me past the current hurdle. :-)
<abentley> jcsackett, np
<abentley> jkakar, did you get anywhere with that ClassAlias issue?
<jkakar> abentley: Hey, sorry about that, the other day... I looked at it and then got distracted.
<jkakar> abentley: So yeah, I looked at it, but I don't understand why you want to use ClassAlias in that code...
<jkakar> abentley: It's usually useful when you self-join a table, which didn't seem to be the case, if I remember correctly.
<jkakar> abentley: Anyway, it did seem a touch odd that the ClassAlias wasn't being used, because I would expect it to be.
<abentley> jkakar, I want to use the POTemplate table two ways: I want to find Products that link to POTemplates, and I want to find packages that link to POTemplates.
<jkakar> abentley: Can you paste the query again, please?
<abentley> jkakar, looking...
<jkakar> abentley: Ta.
<abentley> jkakar,  http://pastebin.ubuntu.com/565962/
<abentley> jkakar, the english version is "Find me packagings where the product and the sourcepackage each have POTemplates"
<jkakar> abentley: Hmm, are the POTemplates the same for a product and sourcepackage, or do products and sourcepackages each have their own POTemplate?
<jkakar> If they're not the same, I suspect you'll need to run two queries or use subselects or something.
<abentley> jkakar, the products will have different POTemplates from the sourcepackages.
<jkakar> abentley: In that case, I don't think the join you have here is going to work.
<abentley> jkakar, The direct SQL works just fine.
<jkakar> abentley: The SQL that's generated at the bottom of the paste you mean?
<abentley> jkakar, no, I wrote this as SQL first and got a losa to run it against staging.
<abentley> jkakar, I'm trying to Stormify that sql.
<jkakar> abentley: Is that SQL at the bottom your hand-written working SQL or what's generated?  If it's generated, can you paste the working SQL, please?
<abentley> jkakar, that's the generated SQL.  The SQL I want is: https://pastebin.canonical.com/43146/
<joey> jml: yeah, needs love
<joey> jml: fixed!
<jml> joey: thanks :)
<joey> my pleasure
<joey> and you're very welcome
<joey> I have one action I can't do on -meeting so I  had to mail off for a group request for that
<abentley> henninge, ping me when you can chat.
<henninge> abentley: now
<henninge> ;-)
<jcsackett> abentley: so i thought i had this working, but it turns out the pushing wasn't putting anything into launchpad.dev, but just pushing into my home directory on my machine. thoughts?
<jkakar> abentley: Hmm, my conversion of that SQL looks a bit different than yours: https://pastebin.canonical.com/43286/
<jkakar> abentley: Does that work?
<abentley> jcsackett, OTP
<jcsackett> abentley: ah, sorry.
<henninge> abentley: https://translations.qastaging.launchpad.net/mailclipper/trunk/+pots/testcase/fr/+translate
<jml> sinzui: would your squad be able to finish up the two remaining High priority feature-flags bugs?
<sinzui> maybe
<abentley> jkakar, I can try it, but shouldn't my version have generated a FROM statement with "AS" in it?
<sinzui> jml: I think the answer is yes. Looks like jcsackett and wgrant have booked their next few days. I will be taking a new bug in a few hours
<jml> sinzui: https://bugs.launchpad.net/launchpad/+bug/670019 and https://bugs.launchpad.net/launchpad/+bug/708999.
<jkakar> abentley: Yes, I'd expect that.  I don't understand why that isn't happening.
<_mup_> Bug #670019: audit facility for feature flags <feature-flags> <lp-foundations> <Launchpad itself:Triaged> < https://launchpad.net/bugs/670019 >
<_mup_> Bug #708999: code.branchmergequeue, code.incremental_diffs.enabled feature flags not documented <code> <feature-flags> <Launchpad itself:Triaged> < https://launchpad.net/bugs/708999 >
<jml> sinzui: thanks.
<abentley> jkakar, okay, I'll file a bug then.
<jkakar> abentley: Thanks.
<bac> hi deryck
<deryck> bac, hey
<allenap> danilos: Hi there. Could you triage bug 714521 if possible? If I do it, it's likely to be a guess.
<_mup_> Bug #714521: Partial translation export feature gone <regression> <upstream-translations-sharing> <Launchpad itself:New> < https://launchpad.net/bugs/714521 >
<danilos> allenap, well, I wanted to leave it for deryck's team to triage, but considering it's a regression, I am sure it should be critical according to our 'regression' policies
<allenap> danilos: Ah, yes :) I'm getting used to this slowly :) I'll update it.
<danilos> allenap, it would be nice to get Henning's opinion as well, because he marked it as 'invalid'
<allenap> Cool, I'm in favour of that.
* henninge changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | firefighting: - | On call reviewer: - | https://code.launchpad.net/launchpad-project/+activereviews
<cr3> can someone help me understand why bugattachment.data.open() seems to return a buffer with a read() method, as per the api documentation, but the open method in the LibraryFileAlias class doesn't return anything.
<henninge> leonardr: I won't be able to look at your mp, I am sorry.
<leonardr> ok, i'm sure i can get someone to look at it since it's a partial solution to an active emergency
<allenap> wgrant: Mind if I ask you about a build issue? I think bug 718790 is something the user can correct himself - i.e. avoid running javadoc on .class files - but I wanted a second opinion as I am not at all familiar with builds.
<_mup_> Bug #718790: FTBFS with java due to illegal characters <Launchpad Auto Build System:New> < https://launchpad.net/bugs/718790 >
<abentley> jkakar, filed as https://bugs.launchpad.net/storm/+bug/718864
<_mup_> Bug #718864: ClassAlias does not generate AS in FROM clause <Storm:New> < https://launchpad.net/bugs/718864 >
* flacoste changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | firefighting: API scripts talking to edge are broken | On call reviewer: - | https://code.launchpad.net/launchpad-project/+activereviews
<abentley> henninge, if a suggestion is not diverged, there's no way of knowing whether it was suggested for upstream or ubuntu, right?
<henninge> abentley: right, and suggestions can not be diverged anyway.
<henninge> i.e. potemplate should only be not None if one (and only one) of the is_current_* flags is set.
<abentley> henninge, thanks.
<cr3> nevermind, folks, found answer in HostedFile in lazr.restfulclient
<jml> Off for a bit. Back later.
* flacoste changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | firefighting: - | On call reviewer: - | https://code.launchpad.net/launchpad-project/+activereviews
<leonardr> is there anyone who can help me understand PATCHPlugin in launchpad/javascript/client/client.js?
<benji> leonardr: I too seek enlightenment.
<jtv> I, on the other hand, seek review: https://code.launchpad.net/~jtv/launchpad/bug-715854/+merge/49676
<jtv> Any takers?
<leonardr> jtv, i'll take it if you'll take https://code.launchpad.net/~leonardr/launchpadlib/fake-edge/+merge/49651
<jtv> leonardr: OK
<jtv> leonardr: I wonder whether we ought to use the new LP import format for launchpadlib.
<jtv> src/launchpadlib/launchpad.py: from launchpadlib.uris import STAGING_SERVICE_ROOT, EDGE_SERVICE_ROOT
<jtv> leonardr: similar for the list constant in src/launchpadlib/tests/test_launchpad.py
<lifeless> moin
<jtv> hi lifeless
<leonardr> jtv: i think i need to push a more recent version
<leonardr> yeah, you didn't have the version that included the deprecation warning. sorry about that
<jtv> leonardr: it does have that
<leonardr> sorry, i meant...
<leonardr> the version that will give you a deprecation warning if you use EDGE_SERVICE_ROOT
<leonardr> it's a minor change
<jtv> That's not in _dereference_alias?
<leonardr> i changed EDGE_SERVICE_ROOT to 'edge'
<jtv> Ah
<leonardr> which will trigger the warning when it goes into _dereference_alias
<jtv> as opposed to 'production'
<leonardr> right. it'll get converted to production after the warning
<jtv> I see
<jtv> leonardr: my one gripe is with the test.  It seems to test multiple things at once:
<jtv>  * Looking up edge on the web service gives you production.
<jtv>  * A web lookup for edge also gives you production.
<jtv>  * Exactly one of these gives the warning.
<leonardr> well, that's easy to fix
<jtv> Nobody said it had to be hard. :)
<leonardr> jtv: problem with catching the warning. the warning only happens once, and i don't know how to guarantee which test it happens in
<jtv> surprisingâ¦ why is that?
<leonardr> ordinarily a deprecation warning only happens the first time you do the deprecated thing
<jtv> I would maybe the firstâ¦
 * leonardr rereads the module doc
<jtv> I would _have expected_ maybe the first.
<leonardr> it's not
<jtv> Then can you guarantee that this test won't fail arbitrarily, e.g. when run in combination with specific other tests?
<jtv> Arrgh, I have to go.
<leonardr> jtv: pushed a new revision that should please you
<leonardr> i'll take a look at your branch now
<jtv> cool, thanks
<abentley> lifeless, stub still needs to sign off on database reviews, right?
<lifeless> yes
<lifeless> https://dev.launchpad.net/PolicyAndProcess/DatabaseSchemaChangesProcess
<lifeless> both of us need a db review requested
<leonardr> jtv: reading your mp i kind of get what's going on but i don't know what a file reference actually is. is that the file in which a translatable string occurs?
<abentley> lifeless, could not find that page on dev.launchpad.net.
<jtv> leonardr: yes, the place in the project's source tree where it's found.
<jtv> (I had that in the MP at one point but must have taken it out again)
<lifeless> abentley: I find it from the policy and process list when I need to
<abentley> lifeless, I mean it's there if you know the URL, but not linked to in https://dev.launchpad.net/PatchSubmission or anything.
<jtv> leonardr: I'm done with yours.  Have to run now!
<lifeless> abentley: ah, it probably needs to be referenced
<lifeless> statik: hi
<lifeless> I can has review? https://code.launchpad.net/~lifeless/launchpad/bug-704446/+merge/49580
<jcsackett> lifeless, have a moment to talk?
<jcsackett> regarding the revision that had to be rolledback?
<lifeless> sure
<lifeless> irc/voice?
<jcsackett> uh, voice might well be easier.
<jcsackett> mumble or skype?
<lifeless> s
<lifeless> whats yoiur skype id?
<jcsackett> lifeless: j.c.sackett
<jcsackett> i'm online now.
<lifeless> durham ?
<gary_poster> leonardr: http://code.google.com/p/httplib2/issues/detail?id=97 !  cool.
<leonardr> yay
<lifeless> nice
<lifeless> nasty bug there
<thumper> morning
<thumper> deryck: ping
<jam> lifeless: I just got rejection emails for proposals against loggerhead trunk...
<jam> Is there any context I can give you that will help?
<jam> (emails, merge proposal, etc.?)
<lifeless> jam: what mail was rejected?
<lifeless> jam: as in the address that was sent to
<jam> lifeless: one was a new merge proposal, one was a comment follow up to an existing proposal
<jam> lifeless: the rejection message was: From: launchpad-bugs-owner@lists.canonical.com
<jam> So I assume launchpad-bugs@lists.canonical.com
<LPCIBot> Project devel build (441): STILL FAILING in 5 hr 12 min: https://hudson.wedontsleep.org/job/devel/441/
<LPCIBot> Launchpad Patch Queue Manager: [r=adeuring][bug=631206] Make Builder:+history load in 2 seconds
<LPCIBot> instead of timing out.
<jam> lifeless: digging into the content, it looks like it was actually launchpad-bugs@lists.ubuntu.com that it was getting sent to
<jam> (From an old Received: line)
<lifeless> ok thats ~launchpad
<lifeless> you proposed to trunk yea?
<lifeless> flacoste: hi
<flacoste> hi lifeless
<lifeless> we up ?
<flacoste> i'm ready for our call
<thumper> leonardr: mumble?
<leonardr> thumper, i'm on it
<jam> lifeless: yes, this is to lp:loggerhead
<leonardr> https://code.launchpad.net/~thumper/launchpad/daily-ajax/+merge/49295
<lifeless> flacoste: https://devpad.canonical.com/~lpqateam/ppr/lpnet/latest-daily-categories.html
<jam> lifeless:  should I be adding sinzui to my loggerhead request-for-review for the ui stuff?
<jam> or should I be targetting ~loggerhead-team and asking for ui, or how is that done?
<lifeless> jam: generally ask here
<lifeless> jam: I suggeset huwshimi, who is tasked with making loggerhead look more like lp, and sinzui
<sinzui> lets give huwshimi a go first
<jam> huwshimi and sinzui: https://code.launchpad.net/~jameinel/loggerhead/revert-colors/+merge/49705  is the link
<jam> sinzui: fine with me, just making sure I conformed to whatever protocol is applicable
<sinzui> why is the person picker showing me a non-active user?
<jml> huwshimi: care to join me on mumble?
<huwshimi> jml: Sure
<jml> ... or not. I can't seem to sign on.
<poolie> hello jml, huwshimi, leonardr, all
<huwshimi> jml: You look online to me now
<jml> poolie: hello
<huwshimi> poolie: Morning
<poolie> can i have a mini-preimpl discussion on https://bugs.launchpad.net/launchpad/+bug/643224
<_mup_> Bug #643224: dkim whitelist should be in configuration <dkim> <feature-flags> <lp-foundations> <Launchpad itself:Triaged> < https://launchpad.net/bugs/643224 >
<jml> huwshimi: skyping you instead
<poolie> specifically as to whether the the approach described is ok
<thumper> leonardr: how do I change an exposed method so it doesn't appear in 'devel' api?
<thumper> leonardr: I've changed the setRecipeText from being a callable method to being a mutator on recipe_text
<thumper> leonardr: also, do I need to annotate the new mutator in any way to make it say devel only?
<poolie> lifeless, can you have a quick glance at 643224?
<leonardr> thumper: put @operation_removed_in_version("devel") at the top of the annotations
<thumper> leonardr: I'm not entirely clear on this point
<thumper> leonardr: ok
<leonardr> you don't need to annotate the mutator because there is no preexisting code that will break now that this field can be edited
<thumper> ok
<leonardr> i think you can make that work if you really want to, but i don't remember how--i think you would use operation_for_version
<thumper> leonardr: is a mutator a write_operation?
<thumper> leonardr: or more specifically, do I need @export_write_operation as well as @mutator_for ?
<leonardr> thumper: yes, you define a write operation and _then_ make it a mutator
<leonardr> so if you want it to be a mutator in one version and not another, you would define it in the earlier version, then put @operation_for_version() on top, and put @mutator_for on top of that
<wgrant> allenap: There are probably better people at 4am, but sure.
<thumper> leonardr: I'm just pushing a change, then I'll get you to look at the diff, and how I've done it to see if there is a more sane way
<leonardr> ok, i'll take a look
<jml> huwshimi: http://www.ubuntu.com/community
<thumper> leonardr: line 196(ish) of https://code.launchpad.net/~thumper/launchpad/recipe-inline-edit-recipe-text/+merge/49585
<leonardr> thumper: you might not need two different methods. try putting @operation_for_version("devel") right on top of @operation_removed_in_version("devel")
<thumper> leonardr: what of the mutator?
<thumper> leonardr: it seems a bit weird to stack them like this
<poolie> leonardr, re your comment about LPNET_SERVICE_ROOT='production'
<leonardr> thumper: @operation_removed_in_version basically clears the slate. you can start from scratch on top of it
<thumper> leonardr: do the annotations work from the outside in, or the inside out?
<leonardr> thumper: they work from bottom to top
<leonardr> the only restriction is you can't put an older version on top of a newer version
<thumper> leonardr: do I need two @export_write_operations ?
<leonardr> thumper: yeah, you have to start all over
<leonardr> including the parameters, i think
<thumper> leonardr: so if I'm starting over, why do I need the @operation_for_version("devel") ?
<thumper> isn't that the default?
<thumper> as I don't need it for the current one
<leonardr> thumper: you might not need it. i'd put it in just to make it clear that you're reinstating it for devel
<thumper> hmm.... ok, let me try
<thumper> well... it works
<leonardr-afk> ok, if you can do without two methods i think that's better (although you could argue two methods is actually more readable)
<leonardr-afk> but i think if you do it with one method you should explicitly state @operation_for_version as a way of signalling its return
<wgrant> sinzui: https://code.launchpad.net/~wgrant/launchpad/icing-purge/+merge/49590 seems like your sort of thing. Could you take a moment to review it?
<thumper> leonardr-afk: yep, I've done that
<sinzui> wgrant: I will
<jam> lifeless: any chance you could teddy bear on ideas for diagnosing the file handle leak issue? https://bugs.launchpad.net/launchpad/+bug/717345
<_mup_> Bug #717345: Updates to SFTP server leak file handles in use_forking_server mode <Launchpad itself:In Progress by jameinel> < https://launchpad.net/bugs/717345 >
<jam> I just ran hammer-ssh with 200-way concurrency
<wgrant> sinzui: THanks.
<jam> and got 186 concurrent connections for 60s
<jam> without falling over anywhere
<jam> at least that I can see
<jam> a bit hard to check every one of 2k connections
<wgrant> jam: You were requesting a smartserver on every connection?
<jam> wgrant: yep
<jam> wgrant: 'echo hello' which is the cheapest request we have
<jam> but still the full stack
<wgrant> :(
<jam> wgrant: and running locally, and running 'lsof' while connecting and afterwards shows 0 stale handles, etc.
<wgrant> jam: :(er
<jam> yep
<jml> g'night all
<wgrant> It sounds like productions children must just not be dying :/
<jam> so apparently qastaging and dev is immune from whatever was killing production...
<jam> now, I'm not triggering failures, and some other stuff, which I can try to inject
<jam> but something really smells here
<jml> g'night all.
<jam> wgrant: and I just checked again, all of those spawned connections return a valid response
<jam> so not only is it connecting, the requests are getting through and responded to
<jam> I could try changing the test script to actually do something more than just 'hello', but I'm not sure what that would tell me yet
<lifeless> jam: sure
<lifeless> jam: did you want voice or irc?
<jam> lifeless: irc is ok. I'm not sure what else to fill in just yet, other than what I mentioned above to wgrant. I'm trying to redo the hammer_ssh script a bit to do more actions per connection
<lifeless> jam: one possibility is expensive queries and then detach
<lifeless> jam: you thought error cases might be important too
<lifeless> jam: also on loggerhead, both sinzui and huwshimi are online know, if you want to mention the theme thing to them.
<jam> lifeless: detach? meaning disconnect, or do you mean run in the background?
<lifeless> disconnect
<lifeless> huwshimi: sinzui: bug 718968
<_mup_> Bug #718968: trunk should use old trunk color scheme <ui> <loggerhead:Confirmed for jameinel> < https://launchpad.net/bugs/718968 >
<lifeless> jam: I've commented on the bug too, fwiw
<jam> lifeless: I ran 300, and it only got 1900 connections total
<jam> so I think I'm hitting a cpu bottleneck with the ssh handshake
<wgrant> jcsackett: So, what's youre codehosting issue?
<lifeless> jam: it has a fd limit of 1024
<lifeless> jam: so wouldn't expect it to get passt 255 concurrent
<jam> lifeless: sure
<jam> well, theoretically 341
<jam> (1024/3)
<wgrant> jam: Plus the SSH FD.
<jcsackett> wgrant: broadly, it doesn't work. more specifically, when make run_all or make run_codehosting, i can neither push branches too, nor get branches from lp.dev
<lifeless> jam: network, in, out, err == 4
<jam> wgrant: right
<wgrant> jcsackett: Sounds like you need SSH config. Have you run utilities/make-lp-user?
<jam> lifeless: right, and I'm guessing I'm currently bottlenecked on "network" only
<jcsackett> wgrant: i have.
<jam> which is why I'm trying to set the connection to do a bit more before it exits
<jam> though now after a couple secs it hangs waiting for a child to exit. but I'll work on that
<jcsackett> i'm not sure it's ssh; the error i get when pushing is that a path or parent directory doesn't exist.
<wgrant> jcsackett: http://paste.ubuntu.com/567140/ is what I have in my ~/.ssh/config.
<wgrant> jcsackett: replace ppa-user with the username you gave to make-lp-user.
<wgrant> jcsackett: It is probably SSH. It's connecting to your system sshd instead of the LP one.
<jcsackett> wgrant: ah, that makes sense.
<jcsackett> trying that now.
<huwshimi> jam: In regards to the colours in loggerhead and making it themeable, I'm in the process of skinning LH so that it looks like launchpad.  Whatever we do we should make it easy to maintain different themes for LH vanilla and LH Launchpad or make them the same.
<jcsackett> wgrant: http://paste.ubuntu.com/567142/
<jam> huwshimi: atm, I'm just proxying for max kanat-alexander, who was the one that preferred the old default theme (non-lp)
<jam> and beuno who was the one that said they were requested that the themes look different
<jcsackett> wgrant: on the plus side, that is a *different* error, which frequently implies progress. :-P
<huwshimi> jam: In that case making it themeable might be a good way to go, right?
<jam> huwshimi: themeable would be great
<jam> and a good answer for the bug
<jam> but far outside of my scope :)
<huwshimi> jam: ok sure.
<wgrant> jcsackett: Ah, you need the forking service now.
<wgrant> jcsackett: Or to disable the forking service.
<jcsackett> wgrant: ?
<wgrant> jcsackett: Set use_forking_daemon to False in the development config.
<wgrant> This is the new thing that jam is debugging.
<jam> wgrant: I doubt that is the failure, but hey, eliminate variables all you want :)
<jam> note the IP address is also custom
<jam> specifically 127.0.0.88 or something like that
<wgrant> jam: Does run_codehosting start the forking service now?
<jam> wgrant: yep
<wgrant> Oh :(
<jam> otherwise it wouldn't work :)
<wgrant> It wouldn't be the first time that development codehosting didn't work :)
<lifeless> jam: huwshimi: I might have a quick chat to poolie about the theme
<jam> lifeless: fine with me, time for me to EOD
<lifeless> jam: kk
<wgrant> lifeless: Did your testfix make it through?
<wgrant> I don't see it.
<jcsackett> wgrant: we may have success. no error msgs on the push, i'm syncing branches and checking the web interface.
<wgrant> jcsackett: Excellent.
 * wgrant works out why it didn't work.
<sinzui> wgrant r=me
<jcsackett> wgrant: looks like that was it.
<jcsackett> thanks!
<wgrant> sinzui: Thanks.
<wgrant> sinzui: I wanted you to look over it because I thought you might have left them for a reason last time.
<wgrant> jcsackett: Great,
<sinzui> wgrant: I remarked that something was using the translate button a few months ago, well maybe 6 months ago
<wgrant> jcsackett: Odd. It works fine for me locally with the forking service.
<lifeless> wgrant: huh, let me see
<wgrant> lifeless: I bet you forgot [testfix]
<wgrant> But PQM should have mailed you :)
<wgrant> Ooooh, sub-second local LP pushes with the forking service.
<lifeless> beuno: hey, on bug 718968 - question for you
<_mup_> Bug #718968: trunk should use old trunk color scheme <ui> <loggerhead:Confirmed for jameinel> < https://launchpad.net/bugs/718968 >
<wgrant> That is awesome.
<lifeless> wgrant: I put the testfix at the end
<lifeless> silly regeex ordering
<wgrant> jcsackett: Is there a launchpad-forking-service running?
<wgrant> lifeless: Hah.
<lifeless> resubbing in a sec
<wgrant> Thanks.
<wgrant> Test runs with that breakage are valid, but have 4 errors?
<lifeless> yes
<wgrant> Great.
<lifeless> wgrant: so, staging appears to have restored successfully; how are you estimating its age ?
<wgrant> lifeless: I normally look for when the stream of new bugs finished.
<wgrant> As long as bugs.staging.launchpad.net doesn't time out...
<wgrant> Which it does.
<wgrant> Ah, there we are.
<wgrant> Doesn't look very restored.
<wgrant> Where is staging-restore.sh?
<wgrant> I can't find it in any of the usual places.
<wgrant> It restored fine on the 5th. It should have presumably restored again on the 12th, but I wonder if it didn't do it because there was no new rev to update to.
<lifeless> I got the restore log 2 days back
<wgrant> You mean the cronspam?
<wgrant> That it sends every time it updates?
<wgrant> lifeless: https://pastebin.canonical.com/43311/
<wgrant> It always restores on a Saturday. This time it did not.
<wgrant> Perhaps a backup was missed due to the rollout.
<wgrant> Or something like that.
<lifeless> ah yesm wrong ting
<wgrant> staging_restore.sh does not always restore staging :(
<wgrant> LOSA ping.
<spm`> yo
<wgrant> spm: What is in sourcherry:/srv/staging.launchpad.net/dumps_from_prod/?
<beuno> lifeless, hey, will look in a bit
<spm> are you trying to determine what's up wit hthe staging restore?
<wgrant> spm: Yes.
<wgrant> Last weekend's did not.
 * beuno replies
<spm> there's nothing helpful in the staging restore email?
<wgrant> Config schemas with production database servers referenced: Australia says argh
<wgrant> spm: The emails are not useful.
<wgrant> The logs on carob only report when a new backup is found.
<wgrant> So all the information I have is that it finds a new backup every Saturday except the last one.
<spm> 2011-02-09 06:38 & 2011-02-14 06:35 <== no restore
<spm> oh I wonder. possibly the rollout on friday night may have messed things up there.
<lifeless> spm: are you able to trigger a qastaging restore ?
<spm> sure
<wgrant> spm: That was my suspicion.
<lifeless> ssh: connect to host bazaar.launchpad.net port 22: Connection refused
<spm> lifeless: as in a DB restore (to confirm)
<lifeless> spm: yes
<wgrant> lifeless: Again? :/
<wgrant> lifeless: Still works fine from here.
<lifeless> wgrant: may be intermittent
<wgrant> Well, it's happening from at least two places, so I guess we have to investigate...
<spm> erm. is tehre a reason we'd have 49 x /usr/bin/python2.6 /srv/bazaar.launchpad.net/production/launchpad-rev-12351/eggs/bzr-2.2.2_lp_2-py2.6-linux-x86_64.egg/EGG-INFO/scripts/bzr launchpad-forking-service --path /srv/bazaar.launchpad.net/var/launchpad_forking_service.socket --pid-file /srv/bazaar.launchpad.net/var/launchpad_forking_service.pid
<spm> all dating from Feb 11.
<wgrant> Are they all from Friday?
<wgrant> Yeah.
<wgrant> Kill them all.
<wgrant> I said the process count was infalted.
<spm> ta
<wgrant> But nobody mentioned those :/
<wgrant> Actually.
<wgrant> If you haven't killed them all.
<wgrant> Maybe keep a couple around.
<wgrant> jam might like to investigate.
<lifeless> spm: (yes to db restore of qastaging)
<wgrant> Although that's not enough to cause much of a problem.
<wgrant> #44028
<_mup_> Bug #44028: cups reports "printer fault" and refuses to print <cupsys (Ubuntu):Invalid> < https://launchpad.net/bugs/44028 >
<wgrant> Er. ECHAN, but yeah.
#launchpad-dev 2011-02-15
 * thumper throws up his hands in frustration
<thumper> damn windmill testing
<wgrant> lifeless: Erm, your PQM submission had a very odd failure.
<wgrant> lifeless: I guess it could have been the codehosting connection refusal.
<thumper> anyone else getting failures in  lp.services.features.browser.tests.test_feature_info.TestFeatureControlPage ?
<wgrant> thumper: Yes.
<wgrant> lifeless has a testfix for that.
<wgrant> But PQM is being crap.
<wgrant> There are four failures relating to this change. See the latest buildbot run for details.
 * thumper sighs
 * thumper gets frustrated at jQuery selectors
<thumper> possible pebkac
<mwhudson> jquery?
<thumper> mwhudson: used for windmill tests to select elements
<mwhudson> oh right
<thumper> yep PEBKAC
<lifeless> wgrant: I have submitted, again, again.
<lifeless> 4th attempt to ec2 land this db patch
<wgrant> lifeless: Thanks.
 * thumper sprinkles some '[0]' values over the jquery selectors
<lifeless> grrr
<lifeless> http://checkip.amazonaws.com -> efail
<lifeless> just for me
<wgrant> Why? Proxy?
<lifeless> dunno
<lifeless> there is a nz wide proxy :(
<wgrant> Awesome.
<wgrant> Yay, landed.
 * wgrant lands stuff.
<wgrant> lifeless: '[ui=none]' :(
 * thumper lands daily-ajax
<wgrant> So, we need to do something about all these codehosting OOPSes. I think UnknownRemoteImportanceError probably wants to generate informational OOPSes?
<wgrant> ExternalBugtracker for BugTrackerType 'GOOGLE_CODE' is not known.
<wgrant> That seems thoroughly useless.
 * wgrant removes.
<mwhudson> wgrant: i don't think you mean codehosting
<wgrant> mwhudson: Er, checkwatches.
<wgrant> No idea where codehosting came from.
<spm> wgrant: about the same place as "ssh codebounce.c.c" comes from
<wgrant> Although there are lots of codehosting OOPSes too.
<wgrant> They are next after checkwatches.
<lifeless> spm: oh hai
<wgrant> So HA
<lifeless> spm: I can has those haproxy?
<spm> DENIED
<spm> not really, I just like saying that.
<lifeless> spm: you've been saving that up for weeks
<spm> months....
<lifeless> spm: ok, so not denied... urlof screen cap ?
<wallyworld> anyone know how to get the text to show up beside the image for a lazr-btn styled submit button? by default, using class=lazr-btn hides the text and just shows the image
<thumper> anyone? https://code.launchpad.net/~thumper/launchpad/recipe-inline-edit-recipe-text/+merge/49585
<thumper> wallyworld: call?
<thumper> wallyworld: mumble? and I'll try to answer your question
<wallyworld> thumper: ok
<poolie> hi lifeless
<poolie> can i borrow your ear briefly?
<wgrant> lifeless: That's a nice Message query improvement you have there.
<lifeless> whats wrong with these two lines
<lifeless>         comments[0].text_for_display = ''
<lifeless>         assert len(comments) > 0, "A bug should have at least one comment."
<lifeless> wgrant: yeah, should be pretty swet
<lifeless> poolie: sure
<lifeless> poolie: voice or irc?
<lifeless> spm: thank yo
<poolie> pots? i can call you?
<beuno> len of the list is > 0!  guaranteed success, can't beat that
<wallyworld> anyone want to claim a code review? https://code.launchpad.net/~wallyworld/launchpad/request-build-popup/+merge/48864
<wallyworld> thumper: do you have a bug number handy for the recipe "build now" work
<wgrant> spm: qas is down... is this the DB restore?
<spm> wgrant: aye
<spm> hrm. actually I could bring it back up for now. the db restore is into an _new db. need it so?
<wgrant> Not really.
<wgrant> How long will the DB restore take?
<wgrant> I am hoping for another deployment tonight to unbreak nightly.sh.
 * StevenK appears.
<wgrant> Hi StevenK.
<wgrant> When are you back?
<wgrant> thumper: What's changed in the new lazr.restful?
<StevenK> At the airport, waiting for boarding.
<StevenK> Plane delayed 40 minutes already.
<wgrant> Where've you been? Adelaide?
<StevenK> Right.
<poolie> wgrant, i commented on https://code.launchpad.net/~wgrant/launchpad/bug-708999-ff-docs/+merge/49758
<poolie> happy to discuss that
<spm> wgrant: how long? no idea. based on staging restores somewhere around 12+ hours would be my wag.
<wgrant> spm: :( OK
<wgrant> poolie: Yeah, we do need a boolean standard :(
<wgrant> I reused the existing [empty|nonempty] notation, but am not very happy with it.
<poolie> i think at least changing the docs would be good
<poolie> we can file a bug for booleans
<poolie> i regret not adding such a mechanism in the first place but at least it's fairly easy to change
<wgrant> Do you have a preferred replacement for '[empty|nonempty]'?
<poolie> [true|]
<poolie> maybe?
<wgrant> I guess, yeah.
<wgrant> I can probably leave out the "if true" then, too.
<poolie> i can really imagine people either putting 'non-empty' or more likely thinking it's something to do with how to handle empty queues
<poolie> i did, on the first reading
<wgrant> poolie: I initially had the "if non-empty" at the start, but felt that made it less readable by putting the useful information later.
<wgrant> But this looks OK now.
<wgrant> poolie: http://people.canonical.com/~wgrant/launchpad/feature-info.png
<wgrant> But we really need some standardisation of names and values :/
<lifeless> uhm
<lifeless> [true|] seems less clear to me
<lifeless> this is a dsl, its not actually python.
<lifeless> just saying.
<wgrant> For some of the keys true makes sense.
<wgrant> eg. visible_render_time and code.incremental_diffs.enabled
<lifeless> and disable_targetnamesearch
<wgrant> Right.
<lifeless> but
<wgrant> So only really code.branchmergequeue needs to be 'enabled'
<lifeless> False
<lifeless> is not the opposite of true
<wgrant> Right.
<lifeless> hmm
<lifeless> this might be even easy
<wgrant> It is tempting to define a new boolean standard.
<wgrant> Describe it on +feature-info as empty or non-empty.
<wgrant> And then make all of the existing rules fit it.
<poolie> yes
<poolie> i think for now, if it's just empty or not
<poolie> i care more about making the comment clear than what's in the value domain
<wgrant> lifeless: You had a suggestion?
<poolie> lifeless, i see launchpadlib's tip still has uris in uris.py, surprisingly
<poolie> s/surprisingly
<poolie> i think leonard was only saying hypothetically he could change them
<lifeless> wgrant: BugTask:+index may be easy
<wgrant> My current thoughts are that I should standardise all boolean flags to empty/non-empty, call this value domain 'boolean', document the empty/non-emptiness of this new domain on +feature-info, and beat the existing boolean feature flags until they have names and implementations compatible with this new standard.
<wgrant> lifeless: Oh?
<lifeless> wgrant: sure, sounds reasonable
<poolie> or not
<poolie> wgrant, that sounds fairly reasonable
<poolie> i wonder if 'defined to be ""' vs 'not defined' will be a bit confusing, for flags that essentially default to be on
<wgrant> Flags won't default to be on.
<wgrant> The only one that does that is disable_targetnamesearch.
<wgrant> And that is one I will rename.
<lifeless> wgrant: mmm
<lifeless> wgrant: why rename it?
<wgrant> Because it is inverted.
<lifeless> its inverted so that when we deploy it doesn't suddenly disable the search
<lifeless> it has the vector it needs
<wgrant> Right, but that can also be achieved by adding the FF beforehand.
<lifeless> and the name matches the vector, doesn't it ?
<wgrant> Or we could add a default to getFeatureFlag.
<lifeless> wgrant: or just accept that sometimes we need to disable rather than enable
<wgrant> :(
<lifeless> wgrant: I think changing the name for the reason you give is pointless consistency
<lifeless> wgrant: harmful even
<wgrant> OK.
<wgrant> lifeless: Do you also object to my renaming of malone.advanced-subscriptions.enabled to malone.advanced_subscriptions.enabled, code.branchmergequeue to code.branchmergequeues.enabled, code.recipes_enabled to code.recipes.enabled?
<wgrant> I fear that if I do not then yet more variants will spring up.
<wgrant> Whereas if we standardise now, people will see there is a standard and follow it.
<wgrant> Because currently it is a little crazy.
<poolie> +1 to that
<poolie> adding a default is a bit harder in TAL
<poolie> we could set a default in the code that registers the flag definition
<wgrant> Ahh, true.
<poolie> i have also been wondering about features that grow from being experimental and off-by-default to being stable and on-by-default
<poolie> we want to be able to garden the dynamic rules
<poolie> but, ideally, while still being able to flip things off in response to late-breaking emergencies
<poolie> (but perhaps not; the code may have rusted in place)
<wgrant> Right. The 'default' column on +feature-info seems a little pointless if there is no default.
<poolie> if there's a default we probably shouldn't have things expressed in the negative sense like 'hidden' or 'disabled'
<poolie> i guess the question is whether the default is expressed in just one place, or diffused throughout the code
<poolie> at the moment i suspect the latter
<wgrant> Right, that was my point with lifeless' new disabled_targetnamesearch.
<wgrant> I don't like it. But without defaults it's hard to sensibly do the opposite.
<wgrant> For now I'll do the boolean standardisation. We can think about defaults later.
<lifeless> uhm
<wgrant> Hm?
<lifeless> so, I don't care about the renames; all the ones you list are flags we're going to delete in the next 4-5 months.
<wgrant> We hope.
<lifeless> I think the _ is better than the -
<wgrant> Yep.
<lifeless> I could care less about the namespacing; I think the cost of communication will outweight several years development :)
<wgrant> Waitwaitwait.
<wgrant> Could, or couldn't?
<lifeless> I have a small aesthetic preference for namespacing
<lifeless> but I really think it doesn't matter at all
<lifeless> if you rename
<lifeless> you need to communicate very clearly
<lifeless> check losa docs
<lifeless> check outstanding rts
<lifeless> and check with teams setting these.
<wgrant> Of course.
<lifeless> thats a lot of work; if you want to do it, its your choice. But you asked for my opinion.
<lifeless> which is 'why bother'
<wgrant> If I don't, then someone will need to police all reviews until we are standardised.
<lifeless> why?
<wgrant> Because conventions are good, but people don't follow unimplemented conventions.
<lifeless> what does this standard buy us; what is the benefit? how does it make us more efficient or less fragile ?
<lifeless> conventions are conventions, they are most definitely not intrinsically good
<wgrant> Sure.
<wgrant> But code.branchmergequeue is not a clear name.
<wgrant> If its values were [enabled|disabled] it might be.
<lifeless> Look, I don't /object/ to a mass rename if you do all the work, I just think I weigh the costs and benefits very differently here. I think the cost in losa time is a huge shame.
<lifeless> and I certainly wouldn't want reviews being bounce back if a clear but 'nonstandard' name were chosen; feature flags are meant to be low key and cheap constructs.
<lifeless> I agree that code.branchmergequeue is unclear.
<thumper> wgrant: you mean with respect to my change?
<wgrant> thumper: You upgraded us to the proper 0.16.0, with no-qa.
<wgrant> I am wondering if anything significant has changed that needs testing.
<wgrant> Since things went bad with the last upgrade.
<thumper> wgrant: the only change is that non-5xx error codes no longer have tracebacks
<wgrant> OK, great. Thanks.
<poolie> wgrant, i think the main change we want is to say that in future it should be 'code.branchemergequeue.visible' or whatever
<poolie> to establish the pattern for future things
<wgrant> poolie: Right.
 * lifeless is saying that structure really doesn't matter
<lifeless> the silos are gone
<lifeless> what matters now is *doing*
<lifeless> and with that, I'm back to working on timeouts, a pasttime I hope you all share my passion for
<wgrant> lifeless: Our favourite BugTask:+index?
<lifeless> yes
<wgrant> Excellent.
<wgrant> What.
<wgrant> Why does lib/lp/services/features/templates/feature-info.pt have several <p>s without </p>s?
<wgrant> This is not the 1990s :(
<wgrant> What :/
<poolie> :)
<wgrant> What.
<wgrant> If I attempt to close one of the <p>s after another element, TAL complains the <p> is not open.
<poolie> i wonder if tal has special inferences for it?
<wgrant> Hmmm.
<wgrant> PARA_LEVEL_HTML_TAGS = frozenset([ # List of HTML elements that close open paragraph-level elements # and are themselves paragraph-level. "h1", "h2", "h3", "h4", "h5", "h6", "p", ])
<wgrant> XHTML does not work that way, TAL...
<mwhudson> chameleon ftw
<wgrant> So it uses XML namespaces heavily, but apparently does not use an XML parser.
<wgrant> How odd.
<huwshimi> wgrant: Those <p>s look like they shouldn't be there anyway
<wgrant> huwshimi: I suspect they are there to space it nicely. But I am hoping to remove them.
<wgrant> For now they are <p/>s instead.
<wgrant> I am also tempted to delete zope.tal in retaliation for it being braindead, but that may be slightly excessive.
<huwshimi> wgrant: Well they should be at least after the <h2>s and certainly not wrapping the <table>
<wgrant> huwshimi: Right.
<wgrant> I initially presumed they were wrapping the h2 and text.
<wgrant> But the h2 closes them immediately.
<wgrant> Because pre-XHTML HTML is cool.
<huwshimi> wgrant: It's nice that it closes those tags for us, but not nice in that it doesn't help us write good html
<wgrant> No, it's not nice that it closes them at all!
<huwshimi> wgrant: What do you mean?
<wgrant> It makes us write something that looks like an XML document and makes us specify namespaces and all that. And then treats the document like something that is not XML.
<wgrant> That is not useful.
<wgrant> It is stupid :/
<wgrant> I guess it may have made sense when it was written.
<huwshimi> wgrant: Right, so it's trying to be smart in a way that is unhelpful. It is good however that we don't deploy broken html pages.
<wgrant> huwshimi: Ah, but a real XML parser would crash, not render broken XHTML.
<lifeless> hmm
<lifeless> whats a good bug to look at that has synchronising bugwatches ?
<wgrant> Ones that work?
<wgrant> Bug 184131 has two.
<_mup_> Bug #184131: Apache 2.2 SNI support <Apache2 Web Server:Fix Released> <apache2 (Ubuntu):Fix Released> <apache2 (Debian):New> < https://launchpad.net/bugs/184131 >
<wgrant> Except the debbugs one is broken at the moment.
<huwshimi> wgrant: It would be good if it broken depending on the spec (i.e. it breaks when we don't add alt tags to images etc.)
<lifeless> trying to see why we process bug watched messages
<lifeless> (at all)
<wgrant> lifeless: Ah, you want a bug with synced comments?
<lifeless> yes
<lifeless> build_comments_from_chunks is -bong-
<lifeless> AFAICT our templates don't use half the work it does
<stub> I suspect it is not using a real XML parser because they are slow, and Chameleon's very reason for existence is because someone got sick of ZPT being slow.
<wgrant> lifeless: Bug #476397 has several imported comments.
<_mup_> Bug #476397: does not display correctly an inkscape pattern fill <apport-bug> <i386> <Poppler:Confirmed> <poppler (Ubuntu):Triaged by desktop-bugs> < https://launchpad.net/bugs/476397 >
<lifeless> ah
<lifeless> right
<lifeless> wgrant: thanks, very helpful
<wgrant> lifeless: You'd not seen that feature before?
<lifeless> wgrant: I was trying to figure out if the loop on imported bug messages was global state for the template or local to the comment
<lifeless> it was local to the comment, so we can skip the size(bug messages) lookup entirely.
<lifeless> also
<lifeless> bugtask:+index is doing lazy evaluation *of the bug watches*
<lifeless> and the irony is
<lifeless> its doing that in a region of code with a comment saying 'we want to avoid one query per item in the for loop'.
 * lifeless facewalls
<wgrant> Haha.
<lifeless> check out build_comments_from_chunks if you want to see the evidence
<lifeless> stub: hi; there was something I wanted your advice on, some less-normalisation I think.
<lifeless> stub: reckon yoiu could spare a few minutes to look at this?
<stub> Sure
<lifeless> its uhm, forget the bug number - binarypackagerelease.packagename though
<lifeless> forces materialisation to sort archive pages
<wgrant> binarypackagename and sourcepackagename are two normalisations that I don't really agree with.
<lifeless> i'm just trying to find the bug, without luck
<wgrant> getPublishedBinaries?
<wgrant> That one?
<lifeless> bug 706200
<_mup_> Bug #706200: Archive:EntryResource:getPublishedBinaries timeouts <qa-ok> <timeout> <Launchpad itself:Triaged by julian-edwards> < https://launchpad.net/bugs/706200 >
<wgrant> Yeah.
<stub> I'd happily drop the *PackageName tables - never liked them.
<wgrant> Hah.
<lifeless> +1
<wgrant> Julian wasn't precisely happy with them last time I talked to him about this.
<wgrant> So it sounds like everyone agrees.
<stub> I wouldn't even call it denormalization since the *PackageName tables contain a single value and are write once - not 3NF as implemented by SQL databases.
<stub> Oh - I recall the rationale I think...
<wgrant> huwshimi: I can see why the <p>s are there.
<wgrant> huwshimi: Removing them leaves too little space between a table and the following h2.
<huwshimi> wgrant: Oh yeah?
<wgrant> h2 needs more like margin-top: 1em
<wgrant> It now has 0.3em.
<stub> There was no SourcePackage or BinaryPackage tables, so *PackageName was there to ensure via referential integrity that bugs could only be targetted to valid packages. It is pretty weak, but I wasn't able to argue against it well enough back then.
<huwshimi> wgrant: Is that a general thing? I'm not sure where we use h2s currently
<wgrant> huwshimi: I'm not sure where else we use them either.
<wgrant> stub: That's completely invalid.
<wgrant> stub: So we can throw them out, it seems.
<stub> Could be, or I could be recalling wrong. It was a subject of much debate anyway, and I lost.
<wgrant> stub: It's a very plausible reason.
<wgrant> It's just also completely invalid.
<lifeless> wgrant: bah, indices exist for a reason
<stub> Actually, a lot of our indexes don't exist for a reason (any more). I'll be detecting and cleaning out the useless ones soon ;)
<lifeless> \o/
<wgrant> lifeless: Hm?
<wgrant> huwshimi: Most of the existing h2s seem to be in portlets (that have their own padding) or surround div.actions (that have their own margin)
<wgrant> lifeless: Where did I not use indices?
<wgrant> Hah.
<wgrant> code.recipes_enabled lies.
<wgrant> It is already empty|non-empty, not on|off as it says.
<wgrant> In fact I think they all are.
 * wgrant headdesks.
<wgrant> Hmm.
<wgrant> memcache defaults to enabled.
<wgrant> That's a bit awkward with the standardised boolean type.
<lifeless> hmm
<lifeless> we need a 'slices to range query' helper
<lifeless> 'nother day
<wgrant> lifeless: Huh?
<wgrant> Oh.
<wgrant> Yeah, that would be handy sometimes.
<wgrant> lifeless: Do you have an opinion on the memcache feature? Should I detect the absence of a flag and default to on?
<wgrant> I think that's probably best unless we have a pressing need for more global defaults.
<wgrant> unless/until
<wgrant> poolie: What do you think?
<wgrant> lifeless: What was the point of your indices comment? Have I missed an index somewhere?
<lifeless> wgrant: hmm? unset is on at the moment because its a killswitch
<lifeless> wgrant: it should stay that way.
<lifeless> wgrant: I think you're making flags more complex and harder to use, if you're running into trouble.
<wgrant> It is not trouble. There are easy ways to do it. I just want to set a good standard.
<lifeless> wgrant: so, its a killswitch, it should stay as one
<lifeless> 'normally on'
<wgrant> Right.
<wgrant> The value on prod will need to change, but that is easy enough.
 * poolie looks
<poolie> i think that making memcache behave as "if not set, on" just inline in the code is reasonable for now
<poolie> i think "memcache" is a pretty broad name for a flag too
<poolie> if it actually means tal caching or something
<wgrant> That's what I've done. I check if there is no flag, and default to on.
<poolie> cool
<wgrant> It's actually not just TAL caching.
<wgrant> It's the whole client.
<poolie> ok
<wgrant> http://people.canonical.com/~wgrant/launchpad/feature-info-again.png
<wgrant> It turns out that most of the current https://launchpad.net/+feature-info is a lie :(
<poolie> wgrant, i like your screenshot
<poolie> yes, but a Bright Shining Lie
<wgrant> huwshimi: Some <th>s seem to be black, others grey. Is this intentional?
<huwshimi> wgrant: I don't think so
<poolie> wgrant,  so again i think saying "this turns off memcache altogether" would be good too
<poolie> but we can get there
<wgrant> huwshimi: Compare https://launchpad.net/+feature-info andÂ https://code.launchpad.net/launchpad/+activereviews
<lifeless> oh my
<lifeless> the ss queries on bugtask:+index are -huge-
<wgrant> lifeless: ss?
<wgrant> Strucsubs?
<poolie> "process'" is odd grammar
<wgrant> poolie: Is it?
<wgrant> It's not mine, but it seems right to me.
<poolie> process's? process?
<huwshimi> wgrant: yeah that's wrong
<wgrant> huwshimi: Missing thead?
<wgrant> Yeah.
<huwshimi> wgrant: That's one way to solve it :)
<lifeless> wgrant: yes
<lifeless> wgrant: visit bug 2 in the same data with LP_DEBUG_SQL=1
<poolie> i guess it's a known bug that there seems to be a different acl for triaging while filing vs triaging after filing?
<wgrant> poolie: Not a bug.
<lifeless> also wasDescriptionModified is fugly
<wgrant> poolie: It was quite intentional at the time.
<lifeless> poolie: I wouldn't assume so, I think its surely a bug
<lifeless> possibly not filed.
<wgrant> poolie: Bug supervisors were meant to be able pre-triage their bugs, but normal users were to be discouraged from doing so.
<poolie> wgrant, something about people being more likely to triage badly if it's on the initial form?
<poolie> hm
<wgrant> This annoys me to no end.
<poolie> there's a weird mispattern here about it being good to make actions hard if some people will abuse them
<wgrant> People are more likely to set to Confirmed if the initial form tells them that they can, I guess.
<poolie> true
<wgrant> But it is very inconvenient.
<poolie> i suffer this on lp:kanban for instance
<lifeless> setting confirmed is irrelevant
<lifeless> because confirmed should die in a fire
<wgrant> lifeless: Yes.
<wgrant> There were proposals at UDS-Jaunty to remove it.
<poolie> that kind of gets into the new/confirmed/triaged thing: does confirmed mean "multilpe people reproduced this" or "someone knowledgeable reproduced it" or "a developer read it"?
<wgrant> But that never eventuated.
<poolie> :)
<poolie> i have a fire
<wgrant> It's too warm today to destroy things in fires.
<wgrant> Except for zope.tal.
<lifeless> zomg
<lifeless> whats wrong with this
<lifeless> tal:replace="view/visible_oldest_comments_for_display/count:len">23</span>
<wgrant> poolie: Any comments on http://people.canonical.com/~wgrant/launchpad/feature-info-again-again.png? I've hopefully improved the subscriptions.
<wgrant> lifeless: What is count:len?
<lifeless> wgrant: count is a sql query
<lifeless> as in .count()
<lifeless> IIRC my tal, :len is a formatter to say 'this is a length'. IMBW
<wgrant> Ah.
<lifeless> but - we're doing a sql query to figure out how many rows we told the sql engine to get for us.
<lifeless> or something approximately like that
<wgrant> Although it may be less than what we requested.
<lifeless> bizarre
 * lifeless is willing to live with that discrepancy
<lifeless> at least for a bit
<wgrant> Oh.
<wgrant> In that context.
<wgrant> The requested value should always be right, right?
<wgrant> Since if there are fewer comments then we display all of them.
<lifeless> right
<wgrant> So that is pointless.
<wgrant> Kill it.
<lifeless> right
<lifeless> another related change
<lifeless> currently the threshold is 'visible comments > limit'
<lifeless> I am changing that to 'comments > limit' because 'visible comment' requires interpretation
<lifeless>  /sigh
<wgrant> lifeless: You mean you have to ask the DB to look at every comment?
<poolie> wgrant, i see you stripped off the warning about scopes not being used?
<poolie> or is that just screenshot cropping?
<poolie> it looks very wide but i assume it scales
<wgrant> poolie: Just screenshot cropping.
<lifeless> wgrant: check out get_visible_comments in browser/bugtask.py
<poolie> so memcache default=enabled is just pure documentation at this point?
<wgrant> lifeless: I did not want to see that.
<wgrant> poolie: No.
<poolie> ie it's describing a policy embedded in the code, rather than actually setting the default?
<wgrant> Right.
<poolie> k
<wgrant> I think once more things want defaults it may become more than just documentation.
<poolie> right
<poolie> it looks really nice
<poolie> i wonder if we should make them sentence case to be consistent with the usual thing for ui and docstrings
<poolie> anything in particular you want comments on?
<poolie> oh while you're there you could link to the dev wiki about this, perhasp
<poolie> so we can add more docs
<wgrant> A good idea.
<wgrant> On both counts.
<wgrant> Was mostly wondering if the descriptions are OK now.
<wgrant> But those comments are good too.
<wgrant> The memcache one would be easier to word as a negative :(
<wgrant> poolie: Hm, there's no really relevant page on the dev wiki, I don't think.
<poolie> hm
<poolie> i'll create https://dev.launchpad.net/Foundations/FeatureFlags as at least a stub
<poolie> there's a bug open asking for something beyond the lep
<wgrant> Foundations is dead.
<poolie> what would be a good URL?
<poolie> it seems like there should be a /Guide/ or /Components/ or something
<wgrant> I'd like a namespace, but I don't think we have a relevant one.
<poolie> so although Foundations doesn't exist as a team, it still exists on the dev wiki home page
<poolie> we can always rename with a redirect later
<wgrant> OK.
<poolie> ... unless someone has a better name now?
<wgrant> I don't.
<poolie> bug 692480
<_mup_> Bug #692480: need better docs on how to use/test feature flags <feature-flags> <Launchpad itself:Triaged> < https://launchpad.net/bugs/692480 >
<poolie> i may well be wrong but it looks like edge's api is 30% faster than lpnet
<wgrant> poolie: Less load.
<wgrant> So smaller queues.
<huwshimi> Bye people.
<poolie> that's measuring across many requests
<wgrant> lifeless is trying to fix this.
<wgrant> huwshimi: Night.
<poolie> i'm surprised it's so high it shows up above the rtt
<lifeless> 2.5 seconds
<lifeless> last time I measured it
<lifeless> its sneaking up
<lifeless> we've got 2 new 8 core appservers, a complete reconfig of the per appserver process thread count and an haproxy rearrangement to address this
<poolie> \o/
<lifeless> eta is probably 3-4 weeks
<lifeless> I /think/ we'll last that long
<poolie> 2.5s... queue time before service?
<wgrant> Overhead on edge is usually around 600ms.
<wgrant> lpnet is rarely below 1.5s.
<wgrant> In my quick tests here.
<wgrant> poolie: Yes.
<lifeless> poolie: yes; just subtract the time to first byte and the reported render time.
<poolie> ouchy
<lifeless> poolie: (and rtt)
<poolie> just wondered why my attempted optimizations (which also included moving things to lpnet) seemed to be making things so much worse
<poolie> silly me
<lifeless> well, lpnet is where we want you
<wgrant> Hmmm, yeah, for the API that would reallllly suck.
<lifeless> because those edge resources will make lpnet faster as we move them across
<poolie> it makes kanban generation 30% slower overall
<poolie> usure
<lifeless> the thing making edge faster isn't queuing so much
<cody-somerville> hmmm... are you guys saying edge is faster currently then lpnet? :P
<lifeless> as that we don't have as many bloated pages served on edge
<lifeless> *that* drives the queue times
<lifeless> cody-somerville: yes, and if you use it I will personally block your uid there
<lifeless> (just saying)
<poolie> lifeless, what do you mean?
<wgrant> Speaking of which... in the debacle last night did we tell the complainers to use lpnet instead?
<lifeless> wgrant: which debacle?
 * cody-somerville hardly ever uses his personal uid... just saying :P
<wgrant> lifeless: The edge API incident.
<lifeless> cody-somerville: :P
<poolie> what happened?
<lifeless> wgrant: I haven't read about that yet; we should.
<wgrant> lifeless: edge was serving api.launchpad.net URLs for an hour.
<poolie> oh, very nice of it to pitch in like that :)
<lifeless> wgrant: (I mean I haven't read the detail .. I knew the outline)
<wgrant> Ah.
<wgrant> Well, my edge config minimisation broke things because our vhost config is terrible :(
<lifeless> yeah
<lifeless> wgrant: as a data point, this is the sort of reason I was ignoring the edge config
<wgrant> Also because people don't remove cruft from configs, though.
<lifeless> wgrant: nonzero chance of trouble, short term win, long term we're killing it.
<cody-somerville> You guys should have like a duplicated setup for like staging this stuff :P
<wgrant> It would not have broken if people had removed cruft instead of trimming it.
<poolie> lifeless, so the difference is basically just that there's less load on edge relative to it's hardware
<poolie> ?
<lifeless> cody-somerville: we do, but the config is the thing that is different
<lifeless> poolie: shorter requests are more common on edge
<poolie> huh
<lifeless> 99th percentile on lpnet is 3s, on edge its 2.something
<wgrant> lifeless: Oh, really?"
<wgrant> Didn't know that.
<lifeless> api's 99th percentile is 2s across both
<poolie> becaues it gets relatively more api load?
<lifeless> edge has slower servers
<wgrant> Since the reports were merged it's less easy to check.
<lifeless> thats my current theory.
<lifeless> not that I really care; its going bye bye
<cody-somerville> If launchpad has no load, how fast would it be?
<cody-somerville> *had
 * lifeless sobs   /home/robertc/launchpad/lp-branches/working/lib/lp/bugs/browser/bugtask.py(986)wasDescriptionModified()
<wgrant> lifeless: Heh heh heh.
<lifeless> cody-somerville: about 1.5 to 2 seconds faster on every page
<lifeless> cody-somerville: you're seeing the render times on pages now, right ?
<cody-somerville> well... 'query/external actions issued' time
<lifeless> thats the render time
<lifeless> on *top* of that there is
<lifeless> RTT
<lifeless> tcp window opening
<lifeless> SSL
<lifeless> in datacentre queuing
<poolie> well, for this i think i have ssl keepalives
<lifeless> python GIL contributes maybe 25% to the render time
<poolie> imbw
<lifeless> in datacentre queuing is doing a variable 1.5-2 seconds
<lifeless> no load would remove in datacentre queuing
<lifeless> and the GIL contention issues
<lifeless> slow sql would still be slow
<poolie> pfft
<cody-somerville> I guess my question is... these seem like 'common' problems. How do other websites manage to have better performance? Is it a code issue or are we just need investing the cash to have a powerful enough 'cloud' to serve lp as fast?
<cody-somerville> err... s/need/not/
<wgrant> The DC overhead fix is scheduled.
<wgrant> The GIL contention will also soon be minimised by these same changes.
<cody-somerville> LP faster would make LP like 50% better. Folks would have so much more tolerance for other issues. At least thats my theory.
<wgrant> It's mostly a matter of people not really caring about performance for 5 years, and it's taking a while to fix it.
<lifeless> cody-somerville: I agree; and its on the basis that we're buying the new front end servers etc.
<wgrant> cody-somerville: Definitely.
<lifeless> wgrant: well, folk gave a crap, but I think the tools in use, and the particular sorts of analysis needed to solve performance were challenging for the team.
<lifeless> wgrant: there are places in LP that have been heavily optimised along one or two axis
<wgrant> lifeless: Sure.
<cody-somerville> I don't think folks really recognized how big of an impact performance had on the overall perception of the service - in the same fashion that in the early days folks didn't recognize the impact the UI had on the overall impression.
<lifeless> I think thats totally untrue
<cody-somerville> maybe thats just a personal reflection
<lifeless> *every* current and emeritus LP dev I know cares passionately about performance and UI
<lifeless> on the UI side there was a bunch of conflicting directives
<lifeless> the distro wanted bugzilla
<lifeless> in terms of UI, not actual bugzilla
<lifeless> Mark wanted what he'd now call crisp and clean
<cody-somerville> Yea... but thats relatively 'recent'
<lifeless> lots of churn resulted
 * cody-somerville remembers launchpad from one or two 'ui refreshes' ago.
<lifeless> cody-somerville: on day one he wanted the same basic outcome; the language for describing it has evolved.
<cody-somerville> ah
<lifeless> anyhow
<lifeless> there is a /tonne/ of discipline needed to make the site fast
<lifeless> and zope is not intrinsically kind - zope is optimised for same-machine ~0 latency DB calls.
<lifeless> which we don't have
<cody-somerville> Ugh... omg... I cant believe the snow plows are already starting! Its only 3am!!! :(
<cody-somerville> errr...
<cody-somerville> 2am
<LPCIBot> Project devel build (442): STILL FAILING in 5 hr 40 min: https://hudson.wedontsleep.org/job/devel/442/
<poolie> wgrant, bug 719180
<_mup_> Bug #719180: need central defaults for flags <feature-flags> <Launchpad itself:Triaged> < https://launchpad.net/bugs/719180 >
<poolie> not sure if that should be higher
<wgrant> It's not hard, and there is a squad assigned to take the remaining high-priority FF bugs. So perhaps.
<wgrant> That may be a Strategist question, I'm not sure.
<lifeless> wgrant: there isn't one, is there? it got close off
<wgrant> lifeless: The remaining bugs were thrown at Green this morning.
<lifeless> oh, nice.
<wgrant> That's why I was working on it.
<lifeless> isn't green on maintenance?
<wgrant> Yes.
 * lifeless twitches
<lifeless> who threw them at you?
<wgrant> lifeless: The Strategist hiimself.
<wgrant> There were only two or so.
<poolie> also https://bugs.launchpad.net/launchpad/+bug/719182
<_mup_> Bug #719182: boolean interpretation for feature flags <feature-flags> <Launchpad itself:Triaged> < https://launchpad.net/bugs/719182 >
<wgrant> And it within a maintenance squad's mandate to finish off bugs resulting from a feature -- normally it would be the squad that did the feature initially, but that's obviously not possible here.
<poolie> what else should go into https://dev.launchpad.net/Foundations/FeatureFlags#preview
<wgrant> poolie: That looks excellent.
<wgrant> Particularly the conventions bit :)
<poolie> thanks :)
<poolie> just going to fill out the two blank sections of course
<lifeless> wgrant: kk
<poolie> wgrant, also hard_timeout, contrary to its docs, seems to be in milliseconds
<poolie> not seconds
<wgrant> poolie: Hah, so it is.
 * wgrant fixes docs.
<poolie> lifeless,  did you add server.?
<poolie> as a scope
<poolie> any particular reason it uses a dot rather than a colon?
 * wgrant vanishes for a while.
<wgrant> Thanks poolie, lifeless.
<poolie> thank you; have a good night
<lifeless> poolie: you added it
<poolie> hah
<poolie> ok
<lifeless> poolie: if I may, -1 on colons
<poolie> so team.admins ?
<lifeless> erm
<lifeless> on flags, not scopes
 * lifeless confuses the issue
<poolie> oh, yeah, i agree with that
<lifeless> let me start over
<poolie> was writing about scopes
<lifeless> so, server. was what you had
<lifeless> I left it as is when doing team: because of diminishing returns etc
<poolie> istm colon makes sense for scopes because it's followed by user parameters, but dots are better for flags which are more like tokens
<poolie> but i mostly want to document stuff that will avoid unnecessary inconsistency
<lifeless> cool
<poolie> to make things faster maybe you can read <https://dev.launchpad.net/Foundations/FeatureFlags#preview> and tell me if you either disagree with anything, or think anything ought to be there
<lifeless> I may have seemed grumpy earlier; I'm glad folk care about this
<lifeless> I think there is a balance though
<lifeless> what happened to the LEP ?
<poolie> it still exists
 * lifeless thought we'd agreed to clean it up and keep it current
<poolie> but people complained that it had too much speculative stuff
<poolie> oh, i thought we agreed we would add separate documentation
<lifeless> right, which I have a note here to go delete.
<poolie> i think "here's what you need to know to use it" is separate from "here's what we thought about in writing it"
<poolie> the LEP still has some ideas we could add later
<poolie> they link to each other
<lifeless> there is a risk in wikis of ending up with a link farm
<lifeless> I think this runs very close to that
<poolie> true
<lifeless> anyhow the prose is fine
<poolie> we could certainly shove some of the LEP into LEP/ff/Future
<poolie> or /Archive
<lifeless> it shouldn't really be in 'Foundations' as those team silos are gone
<poolie> as i said i would be happy to see the dev wiki get a /Guide namespace or something
<poolie> we can rename and redirect
<poolie> i just wanted to get _some_ developer-oriented docs
<lifeless> sure
<lifeless> any as I say, the prose there seems fine, both content and style
<poolie> if you want to take a view of what the dev wiki should look like that would be great
<poolie> cool, thanks
<lifeless> Personally, I would expect to find in a LEP enough useful stuff to use something we built
<lifeless> much like one does in a PEP
<poolie> well
<poolie> eventually the PEPs are redundantly documented in What's New and in the actual user manual
<poolie> so it's kind of analogous
<lifeless> wgrant: /count:len means 'call len() on the thing and show it as a count' as far as I can tell..
<poolie> oh, https://bugs.launchpad.net/launchpad/+bug/692480/comments/1
<_mup_> Bug #692480: need better docs on how to use/test feature flags <feature-flags> <Launchpad itself:Triaged> < https://launchpad.net/bugs/692480 >
<poolie> maybe i should just move it to /FeatureFlags now?
<lifeless> poolie: when I need to know about context managers, I read the context manager PEP; I think if we're not going to delete the LEPs the same should be true.
<lifeless> yes, /FeatureFlags would be a reasonable home
<poolie> so i would probably start with the user manual and drill down to the pep
<poolie> perhaps we should make it part of the lep process that by the time it's implemented, the lep is up to date
<poolie> your impatience before was good
<poolie> i just heard some frustration with the lack of concise docs at the epic, so i felt i ought to add some
<lifeless> thank you
<lifeless> uhm
<jtv-afk> Morning folks.  Still massive test fail?
<lifeless> I'd roll the testing one into the main page for it I think
<poolie> yeah, that would be good
<lifeless> jtv: nope
<poolie> "This page is about SOMETHING. More details can be found in CategoryCategory."
<poolie> good to know it's about something
<poolie> lifeless, ok testing stuff rolled in
<poolie> you're probably right they should merge actually
<poolie> with future stuff or background shunted off into appendices
<wgrant> poolie: Shall I drop the /Foundations from the link?
<lifeless> poolie: its sounding nicer and nicer
<lifeless> \o/
<lifeless> partial loading bugtask:+index in place
<wgrant> lifeless: Just messages for now?
<lifeless> wgrant: its still loading all activities, yes.
<wgrant> :(
<wgrant> Still, not so bad.
<lifeless> also found some lovely bugs
<lifeless> did you know a bug import change the 'original description' for a bug ?
<wgrant> Yup.
<wgrant> There's a bug for that.
<lifeless> well, fixed.
<jtv> lifeless: the same tests that were failing in ec2 are also failing for me locally, even in devel.  So why not still massive test fail?
<lifeless> jtv: buildbot is green
<wgrant> lifeless: Since the index is explicit now?
<lifeless> wgrant: yes
<wgrant> Yay.
<poolie> wgrant, yes
<poolie> yes drop /Foundations/ from the link
<wgrant> Already done.
<wgrant> Thanks.
<jtv> lifeless: rf-setup seems to have fixed most failures, but not all
<lifeless> jtv: what failures are you seeing?
<jtv> lifeless: in ec2test.tests.test_remote, a bunch of cases where a human-readable string was expected and something crypt-y comes out instead ("VnA5Gzâ¦") and one where "utf-8" was expected on an email header but "utf8" was received.
<jtv> And perhaps more, but there's rather a lot of output.
<jtv> 7 failures in that test (on devel)
<wgrant> jtv: Are you on natty?
<jtv> Maverick still.
<wgrant> Hmm.
<jtv> I've been seeing these over the past 7 hours or so.
<jtv> On EC2, I even saw the number of failures in that particular test increase over that time.
<wgrant> Hmm.
<poolie> night
<lifeless> jtv: sorry, no insights here; buildbot is happy though, which is (a) reference platform
<jtv> Is nobody else seeing failures from ec2test.tests.test_remote?
<wgrant> Not I.
<wgrant> I have had two ec2 runs today.
<wgrant> One failed because lifeless' testfix bounced, one succeeded after that landed.
<jtv> And mine failed in the same way that devel failed locally.  :(
<jtv> It's as if email is encrypted but the tests expect it not to be.
<adeuring> good morning
<jtv> hi adeuring
<wgrant> jtv: passes fine for me locally.
<wgrant> jtv: Checked your bzr config?
<bigjools> morning all
<wgrant> jtv: That is copied to the ec2 slave.
<jtv> wgrant: I wouldn't know what to look for.  I didn't mess with it recently.
<jtv> hi bigjools
<jtv> wgrant: there's a gpg_signing_command there
<jtv> and create_signatures = always
<wgrant> I don't sign my revisions, although I probably should.
<wgrant> I suppose it's possible that's relevant.
<jtv> I thought that was required setup.
<wgrant> Clearly not :)
<stub> jtv: Sounds like Python mail library issues.
<wgrant> I have seen the utf8 vs utf-8 issue before.
<wgrant> But I do not remember in what circumstances.
<stub> jtv: In particular, I'd guess the 'encrypted' message is base64. We have a monkey patch lurking somewhere that is supposed to change the default from base64 to quoted printable.
<wgrant> Aha.
<stub> And the preferred character set to UTF-8
<jtv> That sounds like it would help these testsâ¦ but why did this start failing for me so recently?
<stub> You switched to Python 2.7 or something?
<jtv> Not that I know.
<jtv> Discovery: the failures I've been getting until 8 hours ago were actually completely different ones than the ones I've been seeing since then.
<stub> jtv: You get this on a fresh branch?
<jtv> devel
<jtv> also, on EC2 with a branch that was fresh yesterday.
<jml> good morning
<lifeless> hi jml
<jtv> morning jml
<wgrant> Evening jml.
<bigjools> helleau jml
<stub> jtv: So my guess is that it is something to do with test ordering or which subprocess layer is being run.
<jtv> stub: I'll try running just one failing test in isolation then.
<jtv> That fails too.
<stub> jtv: With tests being run that haven't imported canonical.launchpad.mail or lp.services.mail.sendmail failing because the monkey patch hasn't installed.
<stub> Stick an 'import lp.services.mail.sendmail' at the top of the test's .py file?
 * jtv tries
<lifeless> it might be useful to have something run over all our tests and run them one by one in a new process; file bugs on any that fail.
<jtv> stub: yup, that fixes it!
<stub> del Charset.CHARSETS['utf-8']
<stub> Charset.add_charset('utf-8', Charset.SHORTEST, Charset.QP, 'utf-8')
<stub> Charset.add_alias('utf8', 'utf-8')
<jml> lifeless: wouldn't be too hard to do. I suspect some of our tests would behave better with such an runner
<jtv> stub: that's in that module?
<stub> So those lines and associated comments and imports needs to move to lib/lp_sitecustomize.py. I think that is the 'correct' solution ('correct' is an odd term to use when talking about monkey patches)
<jtv> stub: strange that this should affect me so selectivelyâ¦ have I got the wrong locale configured or something?
<stub> jtv: Are you running a subset of tests or the complete suite? If you run a subset of tests, you will be running things in a different order.
<jtv> stub: it happens either way.
<stub> Dunno then :)
<stub> Tests should be run in ascii order IIRC, and the layer forking should be deterministic.
<stub> I guess the tests might run in locale specific order... which could give weird results.
<stub> I feel old whenever I see my old comments, dated 2005...
<stub> and 2004 in the module docstring...
 * StevenK finally gets home.
<StevenK> Fail, Virgin Blue
<lifeless> StevenK: grats
<wgrant> bigjools: That's going to be *quite* a branch.
<jml> remove SourcePackageName and BinaryPackageName?
<lifeless> sourcepackagename may be tricky to remove
<adeuring> henninge: could you have a looka t this mp: https://code.launchpad.net/~adeuring/launchpad/bug-702468/+merge/49205 ?
<jml> lifeless: you think?
<jml> (yes, it will be tricky to remove)
<wgrant> lifeless: It is referenced in a few more places. But it's not fundamentally significantly harder.
<wgrant> Just a few more tables.
<lifeless> wgrant: package branches
<wgrant> lifeless: Hm?
<wgrant> No harder than bugs.
<lifeless> personally, I'd love for that constraint to be fixed, its a showstopper for turning off dput
<lifeless> wgrant: we constrain the namespace of package branches to existing sourcepackagenames.
<wgrant> lifeless: That's only because it's easier.
<jml> wgrant's right.
<lifeless> easier than what?
<jml> lifeless: easier than implementing it not constrained
<jml> lifeless: at the time
* gmb changed the topic of #launchpad-dev to: :
<gmb> Aaaaah
<wgrant> Succinct!
<gmb> Botheration.
<gmb> If someone's got a recent topic line, now's the time to save the day.
* wgrant changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | firefighting: - | On call reviewer: - | https://code.launchpad.net/launchpad-project/+activereviews
<henninge> adeuring: of course! I was already looking at it yesterday but things got in the way. Sorry.
<adeuring> henninge: no problem
<gmb> wgrant: Thankyou kindly.
* gmb changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | firefighting: - | On call reviewer: gmb | https://code.launchpad.net/launchpad-project/+activereviews
<wgrant> henninge: Thanks for reviewing that branch.
<henninge> wgrant: pleasure ;)
<bigjools> wgrant: yeah I was being slightly tongue-in-cheek in the bug comment.
<bigjools> sarcasm doesn't translate easyily to the written word
<bigjools> and my typing doesn't help
<wgrant> Come on qastaging, restore quicker.
<allenap> I would like to increase the hard time-out for Person:+delete so I can delete a team. Should I do that, or should I fix the bug first (which is probably very hard)?
<wgrant> ~ubuntu-main-sponsors?
<stub> So the packaging table allows distroseries to be NULL, although there are now rows in the database with a NULL distroseries.
<allenap> wgrant: Yes :)
<wgrant> allenap: Good luck with that.
<stub> Should this column be NOT NULL?
<wgrant> allenap: It could have many thousands of bugsubscriptions.
<allenap> wgrant: Yeah, that's what I was afraid of.
<stub> (I have two indexes on (distroseries,sourcepackagename), and only one is correct)
<stub> bigjools: ^^
<wgrant> stub: That's a Registry thing.
<stub> Ta
<wgrant> stub: I'm pretty sure our code won't handle it if distroseries IS NULL.
 * bigjools reads
<wgrant> Certainly nothing uses it.
<wgrant> Or creates them like that.
<wgrant> Because it means nothing.
 * bigjools always seems to get pinged when  "packaging" is involved :)
<lifeless> allenap: you should fix the bug
<lifeless> allenap: there is no guarantee that even a 20 sec timeout would be sufficient
<henninge> adeuring: r=me with just a little comment. Thanks!
<allenap> lifeless: Gah. Bugger. Okay, I try and figure out what it'll take.
<lifeless> allenap: bug 162510
<_mup_> Bug #162510: Person:+delete timeouts : Person merging needs to be done asynchronously <canonical-losa-lp> <chr> <feature> <lp-registry> <merge-deactivate> <tech-debt> <timeout> <Launchpad itself:Triaged> < https://launchpad.net/bugs/162510 >
<lifeless> allenap: its half done already.
<allenap> lifeless: Awesome, thank you.
<lifeless> jml: got 5 ?
<StevenK> lifeless: Actually package branches don't use SourcePackageName directly, but SourcePackage
<jml> lifeless: yeah. just let me capture this thought.
<wgrant> StevenK: SourcePackage doesn't exist.
<jtv> danilos, henninge: bug 719247
<_mup_> Bug #719247: Translations branch auto-approver not working <Launchpad itself:Triaged> < https://launchpad.net/bugs/719247 >
<wgrant> StevenK: It is a Python myth.
<danilos> jtv, yay?
<jtv> danilos: not yay
<StevenK> wgrant: ISourcePackage, lib/lp/registry/interfaces/sourcepackage.py
<jtv> wgrant: you lieâI saw a SourcePackage once and if my camera hadn't broken down just thenâ¦
<adeuring> henninge: thanks!
<henninge> adeuring: np
<henninge> jtv: We need to replace the approver.
<wgrant> StevenK: Yes, a Python myth. Goes nowhere near the DB, and most of this work is going to be in the DB interface code.
<danilos> jtv, any indication as to what might be the problem?
<jtv> danilos: none whatsoever
<jtv> henninge: this sounds like it may be an operational issue though
<henninge> jtv: without looking further into it, I remember that the auto-approver for the branch imports was too careful.
<jtv> henninge: this was a simple single-template setup though
<jml> lifeless: skype me when you are ready
<henninge> hm
<jml> wgrant: that's backwards, imo.
 * henninge looks closer
<wgrant> jml: Yes, but so are lots of things.
<wgrant> :(
<wgrant> DistributionSourcePackage is both!
<wgrant> DistributionSourcePackageInDatabase, yay.
<gmb> bigjools: Is cron.germinate Soyuz territory?
<bigjools> gmb: yes
<bigjools> it's maintained by the ubuntu platform guys though
<wgrant> Riddell's change is fine.
<henninge> jtv: that is odd indeed
<gmb> wgrant: Thanks :)
<gmb> That's what I was asking about.
<wgrant> gmb: Platform does just about all work on it.
<wgrant> But it lives in our tree so we must wave them through.
<gmb> Righto; duly noted. Thanks again.
<jtv> henninge: sbackup looks like it may be having the same problem since september.
<gmb> danilos, jtv, henninge: I'm looking at abentley's translations branch and, whilst it's small, I have no context on it to know whether it makes sense or not. Can one of you guys take a look?
<gmb> https://code.launchpad.net/~abentley/launchpad/fix-undiverging/+merge/49718
<jtv> gmb: I can
<gmb> jtv: Bedankt.
<jtv> Graag gedaan
<jtv> (prononounce each "g" as if it were your last)
<gmb> Hah.
<jtv> No, more like khh
<jtv> Check your keyboard.  If the spittle is visible there, you're doing it right.
<gmb> :)
<henninge> adeuring: If you still have a chance to fix it: The "Comment instead of docstring" exception is only for actual tests (methods that start with test_). SelectUpstreamTranslation should have a """docstring""" not  a # Comment. ;-)
<adeuring> henninge: ah, right... but I'va already started "ec2 land"...
<henninge> adeuring: if it fails, you'll have your chance ... otherwise just include that in one of your next branches.
<henninge> ;-)
<jtv> gmb: done
<gmb> jtv: Thanks
<henninge> jtv: You wrote: The script requires a distroseries to be specified among the options, but assumes a distroseries to be specified.  There may be a way to specify to optparse that a particular option is required
<jtv> henninge: I meant it assumes a distro to be specified.
<henninge> jtv: ah
<jtv> And source package, IIRC
<henninge> jtv: it defaults to "ubuntu"
<henninge> via optparse
<jtv> I see
<henninge> no, source package is checked
<StevenK> OSError: [Errno 2] No such file or directory: './lib/canonical/launchpad/icing/yui/event/event-base-ie-min.js'
<StevenK> During 'make'
<wgrant> StevenK: rm -r lazr-js
<wgrant> make clean may do that.
<wgrant> But I just remove it manually.
<StevenK> wgrant: Ta.
<wgrant> It happens occasionally on YUI upgrades.
<wgrant> allenap: Do you have any ideas for getting rid of all the UnknownRemoteImportanceErrors and "ExternalBugtracker for BugTrackerType 'FOO' is not known." errors?
<lifeless> gmb: hi
<lifeless> gmb: I think bug 719249 is a risky way to go
<_mup_> Bug #719249: The new direct subscription overlay takes too long to load <story-better-bug-notification> <Launchpad itself:Triaged> < https://launchpad.net/bugs/719249 >
<lifeless> gmb: most of the time most users won't be changing subscriptions, by preloading you are going to pay for that on *every* page load.
<allenap> wgrant: Get rid of them entirely, or from the OOPS reports? For the latter I think they should be added to (a|the) blacklist.
<lifeless> allenap: if its in a blacklist, its pointless overhead to write an oops
<lifeless> allenap: log it to the log file and move on - you don't need the http request count, backtrace, etc etc for these.
<allenap> lifeless: Okay. wgrant, I'll file a bug for lifeless's suggestion.
<wgrant> Thanks!
<lifeless> allenap: rule of thumb: if a *human* should do something *now*, we record an OOPS.
<lifeless> allenap: otherwise, we shouldn't.
<allenap> lifeless: Noted :)
<gmb> lifeless: Hmm. I see your point, but since we load all the portlets and initialise all the JS for the subscription links asynchronously already I'm not sure I entirely agree that it's risky.
<wgrant> jtv: Hi.
<jtv> hi wgrant
<bac> gmb: i've just put up a MP but am not really here today.  could you review it at your leisure?
<gmb> bac: Sure thing.
<bac> thx
<wgrant> jtv: update-stats.py is now the second-slowest piece of nightly.sh. Looking at DistroSeriesLanguage.updateStatistics, it seems to grab every POFile then sum the results of methods that return a value directly from a DB column. Am I missing something here, or can that be replaced with one query to select the sums directly?
<wgrant> jtv: Same with the POTemplate iteration in DistroSeries.updateStatistics.
<jtv> wgrant: we haven't looked at the DistroSeriesLanguage version of that for a whileâas it happens I was just looking at POFile.updateStatistics performance and that seems to be quite fast now.
<jtv> But yes, I believe that's something we simply never had time to optimize.
<henninge> wgrant is the guy that screws booster rockets onto scripts ... ;-)
<wgrant> henninge: I don't think I can do 1000x again, though :(
<henninge> wgrant: 100x is fine, too :-P
<wgrant> jtv: I gather that those attributes are caches that are relatively new?
<jtv> wgrant: not at all, but their mechanics and intricacies have changed
<wgrant> jtv: Hmm.
<jtv> The attributes are positively ancient.
<wgrant> It just seemed odd to have methods that just wrap apparently cached values.
<wgrant> I presumed they were historical relics.
<jtv> Those methods are from ancient interfaces, though adi simplified them last year.
<wgrant> But there is no evil magic that prevents me from doing the sum in the DB?
<jtv> They're still nastily spread out through the code, unfortunately.
<jtv> No really evil magic, no, though you may run into some attributes that are computed from others.
<wgrant> I don't believe there are any of those. But I will check.
<wgrant> It all just seemed too simple!
<jtv> Yes.
<jtv> danilos, henninge, this may be related to the translations branch approver problem: bug 719267
<_mup_> Bug #719267: Translation template approval times out <timeout> <Launchpad itself:Triaged> < https://launchpad.net/bugs/719267 >
<lifeless> gmb: ah, if its async its better, but still - unneeded load is waste
<lifeless> gmb: do have a look see - and remember that tests against launchpad.net are currently pessimised due to being overcapacity
<gmb> Right.
<jml> when are we getting more capacity?
<lifeless> jml: shortly after we get losa cycles
<lifeless> we have 8 cores we can expand onto with
<lifeless> 'simple'
<lifeless> reconfiguration
<lifeless> and 12 more cores coming
<lifeless> we should knock ~ 1 second of in-data centre latency off immediately
<lifeless> to test; hit a page up and note the render time shown
<lifeless> compare the time-to-first-byte in speed tracer of the same request
<lifeless> the difference is ssl + rtt + in-dc queuing
<lifeless> night all
<jml> g'night.
<bigjools> a pox on python files with a - in their name
<jml> bigjools: indeed
<jml> and also ones that can't be imported
<bigjools> that is my problem :/
<bigjools> trying my best to write a test for something I changed that is totally untested right now
<jml> yeah.
<jml> bigjools: with the file formerly known as ec2test-remote.py, I renamed it and added some tests for the current behaviour in a branch before landing my behaviour change
 * bigjools is delving into buildd module territory where all bets are off
<deryck> Morning, all.
<allenap> Morning deryck :)
<LPCIBot> Project db-devel build (366): STILL FAILING in 5 hr 40 min: https://hudson.wedontsleep.org/job/db-devel/366/
<leonardr> gmb, take a look at https://code.launchpad.net/~leonardr/launchpadlib/silently-replace-edge-with-production/+merge/49794
* leonardr changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | firefighting: - | On call reviewer: gmb, leonardr | https://code.launchpad.net/launchpad-project/+activereviews
<LPCIBot> Project devel build (443): STILL FAILING in 5 hr 16 min: https://hudson.wedontsleep.org/job/devel/443/
<LPCIBot> * Launchpad Patch Queue Manager: [r=henninge][bug=198778,
<LPCIBot> 718004] Rewrite update-bugtask-targetnamecaches to use a set-based
<LPCIBot> method, approximately 1000x faster.
<LPCIBot> * Launchpad Patch Queue Manager: [r=sinzui][no-qa] Remove unused icing.
<LPCIBot> * Launchpad Patch Queue Manager: [r=deryck, lifeless,
<LPCIBot> wgrant][bug=673530][incr] Add a popup choice widget for the build
<LPCIBot> frequency.
<bigjools> gmb: review ping please: https://code.launchpad.net/~julian-edwards/launchpad/buildd-regex-bug-719162/+merge/49796
<abentley> henninge, deryck, adeuring: My internet is busted.  May not be able to Mumble.  Is POTS an option?
<deryck> abentley, we could skype and dial a land line.
<abentley> deryck: logging off to preserve cell data...
<gmb> leonardr, bigjools: Just got back from lunch; will peer owlishly at your MPs shortly.
<leonardr> great
<leonardr> bigjools: i can take yours
<gmb> leonardr: edge_server_equivalent_string_becomes_production() and edge_web_server_equivalent_string_becomes_production() look like tests but they don't have the test_ prefix so (I think) the testrunner won't run them. Is that deliberate (or am I wrong)?
<leonardr> gmb: it's not deliberate, let me check
<gmb> leonardr: line 121 and 127 of the diff, for reference
<leonardr> gmb: yeah, just adding test_ fixes the problem
<gmb> Cool
<gmb> leonardr: r=me then.
<leonardr> ok
<leonardr> bigjools, r=me
<salgado> leonardr, does launchpadlib mock Launchpad for any of its tests or do we still need a local Launchpad to run launchpadlib's test suite?
<leonardr> salgado: you still need a local launchpad
<leonardr> launchpad has unit tests but they don't test that launchpad returns specific data
<leonardr> they test that launchpadlib behaves in a certain way
<leonardr> er, launchpadlib has unit tests...
<salgado> right
<salgado> thanks leonardr.  I was afraid that'd be the case
<leonardr> np. there is a project to make a mock launchpad but it's stalled afaik
<salgado> leonardr, do you know where I can find more info about it?
<leonardr> salgado: no, i don't remember who was doing it.
<james_w> leonardr, do you mean the one on https://code.launchpad.net/launchpadlib/+activereviews ?
<leonardr> let's see...
<leonardr> yeah, that's the one.
<leonardr> i guess it's not stalled--there's recent activity
<salgado> well, about 8 months ago was the last non-merge commit
<salgado> although there's review activity. :)
<bigjools> leonardr: thanks
<danilos> gmb, leonardr: hi guys, anyone want to take a look at my "split bugtask_index.js into two" branch?
<gmb> danilos: I'll take it
<gmb> Since I set you on that path....
<danilos> gmb, heh, excellent, thanks
<danilos> gmb, https://code.launchpad.net/~danilo/launchpad/bugtask-index-portlet-setup/+merge/49818
<danilos> gmb, diff is probably not there yet :)
<gmb> Right :)
<gmb> Time for me to get coffee
<henninge> jtv: can you help me with some SQL foo?
<danilos> gmb, heh, yeah, the diff itself is >2k lines, but you know what that is
<jtv> henninge: sure
<gmb> Right
<henninge> jtv: it might be a good idea to consolidate the input queue before re-enabling imports.
<jtv> henninge: consolidate how?
<henninge> jtv: dpm reckons there are been multiple uploads for a source package in the queue and we really only need the latest.
<henninge> jtv: so from all entries with the same sourcepackage and path, we could delete all but the latest.
<henninge> I was just wondering if that could be done in SQL or if I need a script.
<jtv> henninge: don't they all have the same uploaders?
<henninge> jtv: dpm says not.
<jtv> Sometimes not?  Never?  Not a lot?
<henninge> good questions.
<jtv> Because AFAICS that's the only duplication we're going to see: from multiple people uploading the same file.
<henninge> jtv: True
<jtv> Meanwhile, unfortunately, it's become impossible to approve some templates.
<danilos> jtv, :(
<henninge> jtv: I guess we could get some statistics on that
<jtv> Timeout while running statistics for new POFiles.
<jtv> How ironic.
<henninge> why is that?
<bigjools> allenap: thanks for fixing the buildd-manager
<jtv> henninge: we're both talking about statistics.
<allenap> bigjools: No worries, though it's not in stable yet.
<henninge> hah!
<jtv> henninge: redundant uploads should normally go into the pre-existing queue entriesâand that means all of them for package uploads.  Is there really a problem here?
<henninge> jtv: not sure
<henninge> jtv: I will see if I can put together a query to get some numbers on that.
<henninge> after lunch (well, tea really)
<jtv> henninge: one complicationâ¦ if two uploads are by different people, who's to say they don't both contain valid, original data?
<henninge-tea> jtv: dpm says just the latest matters.
<henninge-tea> but maybe he is wrong?
<jtv> Well I wonder how it's possible to tell, since it's different people.
<jtv> You'd have coordinate that with all the translation teams.
<jtv> (the impersonal "you," not you as such)
<danilos> jtv, for packaged uploads, even if they are from different people, we can safely ignore non-latest ones (in theory, that might still cause different behaviour, but doesn't matter in practice)
<jtv> danilos: ISWYM
<jtv> Even so, I'd be interested to see how much it'd save us.
<danilos> jtv, sure
<dpm> henninge-tea, jtv, my question was not whether the two uploaders upload valid data (which I assume they'd do). The question is whether, assuming one uploads valid translation data on Monday and the other one on Tuesday, why do we need to import the data from Monday, if it's the same (or contains less translations)?
<jam> spm: ping, if you haven't killed the extra processes on crowberry yet, it would be good if we could do some debugging.
<jtv> dpm: in the meanwhile we've been convinced of that, but our current counterquestion is: how many imports would it save us?
<dpm> jtv, henninge-tea, anyway, you guys know the technical details better, that was just my brainstorming to make the disruption in translations shorter
<dpm> jtv, I don't know :) But I'm guessing it would be a big saving in big uploads such as KDE
<dpm> it's a matter of finding out how many "duplicate" uploads there have been since we stopped imports
<jtv> dpm: the real disruption is the ongoing delays thoughâthere's no disruption to non-Ubuntu imports, and for Ubuntu ones it may not (âguess, to be verified) make much of a difference now relative to the existing delay.
<jtv> Also bear in mind that duplicate uploads are processed much faster.
<dpm> jtv, I'm talking about Ubuntu only, I know the other queues are not stopped
<dpm> jtv, I didn't know duplicate uploads were processed faster, I understood from your comments that was only the case if they came from the same uploader
<jtv> If they come from the same uploader, then the new upload simply replaces the old one so only 1 is processed at all.
<jtv> But processing a msgstr that's identical to the current translation of that msgid is very fast.
<dpm> jtv, anyway, I just thought I'd ask, I'll let you guys figure it out. The reason why I thought about this is because we did this once in the past for OO.o uploads (i.e. discard all but the latest upload), and it was my understanding that it did speed things up
<jtv> dpm: it did, yesâ¦ we'll have a look to see if there's a similar effect.
<jtv> OO.o would slow down anything.  :)
<dpm> :)
<dpm> the same with KDE, it's a big import
<gmb> danilos: r=me with a couple of tweaks (one of which is a longstanding bug that I put there and never noticed ;)).
<danilos> gmb, heh, sorry, I don't want to fix *your* stuff :P
<danilos> gmb, anyway, thanks :)
<gmb> Hah.
<danilos> gmb, btw, since I am not too comfortable around JS, should it really be "var namespace.blah_blah = ..." or just "namespace.blah_blah = ..."?
<danilos> gmb, (i.e. without the "var")
<gmb> danilos: Argh. Without the var.
<gmb> I always make that mistake
<danilos> gmb, heh, ok, thanks :)
<gmb> And it takes me ages to debug it. Good catch.
<danilos> gmb, for testing the advanced_subscription stuff, do I have to worry about malone-alpha memberships and such?
<gmb> danilos: Not on lp.dev, no.
<danilos> gmb, cool, thanks
<sinzui> allenap: what do you intend to do about the merge/delete person timeout?
<allenap> sinzui: I'm going to look at the code then, actually, talk to you about it :)
<allenap> sinzui: I was going to see if there's a registry job already set up that I could piggy back on.
<sinzui> yep. That was Edwin's intentions
<allenap> Cool. I'll head in that direction.
<sinzui> allenap: I updated the question because the timeout is related to a second bug. The proc is transferring the bug subscriptions to ~registry. That is just wrong
<sinzui> allenap: They are separate issues. But I know how to remove the 196 bug subscriptions. Either dholbach can use the script I posted, or make ~registry the team owner so that I can clean up the bugs first
<allenap> sinzui: Is there are bug for "transfer to ~registry on delete"? I assume it should just unsubscribe.
<sinzui> allenap: bug 613604
<sinzui> wrong
<sinzui> allenap: bug 613603
<_mup_> Bug #613603: team merge subscribes/assignes bugs to ~registry <lp-registry> <merge-deactivate> <Launchpad itself:Triaged> < https://launchpad.net/bugs/613603 >
<allenap> sinzui: I was going to do a work-around like the script you suggest, but I'll try and get this bug fixed first... it would be nice to see it work.
<allenap> sinzui: Thanks.
<sinzui> allenap: Edwin intended to use PersonTransferJob. The job could be membership change, person/team merging or person to person emails
<allenap> Cool.
<jcsackett> could someone help me figure out an lp.dev codehosting issue? i've run sync_branches but still can't get a branch scanned. :-/
<jcsackett> ah, got it. competing process hurting codehosting.
<allenap> sinzui: It looks like a lot of the logic in AdminTeamMergeView should be folded into the new job, if not into Person.merge() itself.
<sinzui> allenap: oh, thanks for bringing that up
<sinzui> I think I reported a bug that the view logic can be pushed lower I was thinking the model, but the job may also be the right place. We may want to initiate a merge/delete over the api.
 * sinzui looks for bug
<sinzui> allenap: there is no bug. I think the conversation took place in an MP. Once we got the view logic cleaned up, it looked like we could finally get the rules into something beneath the view
<allenap> sinzui: Yeah, I am kind of pointing out the obvious. I think I might move only AdminMergeBaseView.doMerge() into the job at first.
<sinzui> allenap: yes. Just do what is obviously needed. We have fumbled pushing the logic lower before. It has only been in a state that looked doable since December of last year
<allenap> sinzui: Okay, thanks.
<jkakar> FWIW, and maybe it's just from years of being used to the old, I'm finding the grey headings make things harder to see... again, could just be that I'm just very used to the old maroon, but this is my initial experience.
<sinzui> jkakar: it is harder. huwshimi and I discussed it.
<sinzui> jkakar: The colour was bad, but at least is hid the font-size and spacing issues that are *really* bad in Lp
<sinzui> jkakar: I am doing a spike this evening to fix the font-size issue. If it works, huwshimi will do a followup to address heading
<jkakar> sinzui: Awesome, thanks. :)
<jml> jkakar: what about the old blue, orange, green etc?
<jml> jkakar: is it just maroon that's easier to read?
<jkakar> jml: I miss the red 'Report a bug' link.
<sinzui> jkakar: is landscape using Canonical web guidelines
<jkakar> Somethings are a bit weird... for example, the THEADs on the +activereview page are much stronger than the headings that describe what each section is, which feels backwards.
<sinzui> jkakar: oh, yes, we discussed the participation links too. We are uncertain. We decided to try consistency first. We lost something though.
<sinzui> jkakar: wow, it's like you are reading my reviews and emails to huwshimi
<jkakar> sinzui: :)
<sinzui> yes the headers are odd, many are inconsistent because the engineer hacked the css/style rules
<sinzui> We need to tear out a lot of exceptions in templates
<jkakar> One thing I thought, particularly from looking at a merge proposal, was, "Oh, they're trying to make user content jump out and be more prominent than all the chrome"... if that's the case, great, but I think it needs a bit more iteration. :)
<jkakar> sinzui: Yeah, that kind of work is really tedious, but important.
<sinzui> jkakar: it certainly does need iteration. Unlike other updates, we are not stopping releases for a few months. I think the font-size issue is becoming a blocker. Canonical Web guildlines have some good advice, but we do not know why px is used instead of ems.
<jkakar> sinzui: I don't know if would help, but we've been through a lot of careful iteration and selection of colours in Landscape with the redesign we did last year.  It might be a place to look for ideas/inspiration.
<jkakar> sinzui: I don't why they're used, either.
<jkakar> sinzui: I think it's because designers use Photoshop and think naturally in terms of px's.  em's are a bit of a random measurement.
<jkakar> sinzui: alejandra and yaili are good people to talk to about the guidelines, especially to get insight about questions like that.
<sinzui> jkakar: thanks! I think there is something else at play. browser engines are now running on a many tiny and huge screens. I suspect the engines are using px and a baseline rather than absolute units for fonts.
<jkakar> sinzui: They're also good people to report problems regarding the guidelines, since they basically maintain them.  We found a number of issues during our redesign that resulted in tweaks to the guidelines.
<sinzui> fab, I do not think the bug I reported were seen
* gmb changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | firefighting: - | On call reviewer: leonardr | https://code.launchpad.net/launchpad-project/+activereviews
<abentley> deryck, we need to have the inverse of translation merging ("splitting") for when a Packaging is deleted, so we should probably put that on Kanban.
<deryck> abentley, ok.  Do you mind adding a bug and card for that?
<abentley> deryck, okay.
<deryck> thanks!
<LPCIBot> Project db-devel build (367): STILL FAILING in 5 hr 16 min: https://hudson.wedontsleep.org/job/db-devel/367/
<LPCIBot> Launchpad Patch Queue Manager: [rs=buildbot-poller] automatic merge from stable. Revisions: 12377,
<LPCIBot> 12378, 12379 included.
<jml> just wrapping up for the day, around for another 20 or so minutes.
<lifeless> jml: hi
<jml> lifeless: hi
<LPCIBot> Project devel build (444): STILL FAILING in 5 hr 36 min: https://hudson.wedontsleep.org/job/devel/444/
<LPCIBot> * Launchpad Patch Queue Manager: [r=henninge][bug=718761] Ensure that NewBuildersScanner only detects
<LPCIBot> a new builder once. Previously,
<LPCIBot> any builder registered after start-up would be detected as "new" every
<LPCIBot> 5 minutes.
<LPCIBot> * Launchpad Patch Queue Manager: [r=adeuring][ui=none] [r=adeuring][bug=715802] Add a vestigial page
<LPCIBot> for the new Bug:+subscriptions view.
<LPCIBot> * Launchpad Patch Queue Manager: [r=lifeless][no-qa] SQL statements logged in OOPs and ++profile++
<LPCIBot> output include the value of their bind variables.
<LPCIBot> * Launchpad Patch Queue Manager: [r=leonardr][bug=715854] Don't set filereferences when creating
<LPCIBot> POTMsgSets during translation import.
<jml> You know, that bot would be better if it just showed the failing tests
<jml> rather than the merges
<jml> StevenK: LPCIBot should show the failing tests, rather than the merges.
<lifeless> thats a great idea
<LPCIBot> Project db-devel build (368): STILL FAILING in 48 min: https://hudson.wedontsleep.org/job/db-devel/368/
<LPCIBot> Launchpad Patch Queue Manager: [rs=buildbot-poller] automatic merge from stable. Revisions: 12380,
<LPCIBot> 12381 included.
<beuno> so, what's up with the color change in bug titles?
<lifeless> colour change?
<beuno> they're not red anymore
<beuno> they're gray
<lifeless> they were red?
<lifeless> anyhow, the h1 colours are now consistent on all the domains
<beuno> heh
<beuno> yes, they where always red on bugs, bugs was already red
<beuno> so all h1's are light-ish gray?
<lifeless> thats my understanding; sinzui and huwshimi are going to tweak it some more though
<beuno> I'd propose a more title-y color, something that instead of blending into the background stands out
<sinzui> beuno: the shades will change. If Lp had a colour (aubergine, orange, etc...) we could use that
<sinzui> beuno: I am going to attempt a final font-size fix tonight
<thumper> morning
<lifeless> jcsackett: is bug 242461 a dup
<_mup_> Bug #242461: Bug search returning results for inactive projects. <404> <lp-bugs> <oem-services> <search> <Launchpad itself:Triaged> < https://launchpad.net/bugs/242461 >
 * jcsackett looks
<jcsackett> lifeless: looks like it. bug 632847 is specifically about an oops when following links b/c of that, but it's the same issue.
<_mup_> Bug #632847: Bug page OOPS when viewed in deactivated project context <404> <bad-commit-12363> <lp-bugs> <oops> <qa-bad> <Launchpad itself:In Progress by jcsackett> < https://launchpad.net/bugs/632847 >
<lifeless> jcsackett: I was looking for some other stuff and it seemed awfully familiar :)
<jcsackett> yes indeed. :-P
<jcsackett> lifeless: i'll go ahead and mark it.
<benji> Anyone know what's wrong with my lint? http://pastebin.ubuntu.com/567459/
<benji> leonardr: hi, have time for a quick review? https://code.edge.launchpad.net/~benji/launchpad/bug-649252/+merge/49876
<leonardr> benji, sure
<benji> thanks
<wallyworld> thumper: i cn hear you
<wallyworld> thumper: i'll try restarting :-(
<wallyworld> thumper: not sure what'
<wallyworld> faaaark
<wallyworld> stupid mic
 * wallyworld tries something else
<wallyworld> thumper: let me try and get it sorted and i'll get back to you
<thumper> leonardr: mumble?
<thumper> StevenK: around?
<wallyworld> thumper: i tries skype and that worked
<thumper> haha
<thumper> mumble hates you
<wallyworld> don't know what's up with mumble though
<leonardr> thumper, yeah
<leonardr> thumper, for future reference: http://pastebin.ubuntu.com/567472/
<leonardr> benji, r=me with very minor changes, sorry for the wait
<benji> leonardr: no problem; gave me time to figure out why lint hates me
* leonardr changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | firefighting: - | On call reviewer: - | https://code.launchpad.net/launchpad-project/+activereviews
<wgrant> mwhudson: We didn't know at the time that it was running out of file handles :(
<mwhudson> wgrant: yeah
<mwhudson> production debugging is such a joy!
<benji> leonardr: I don't know if you have time or not, but speaking of lint breakage, I have a branch ready for review that fixes the problem: https://code.edge.launchpad.net/~benji/launchpad/make-lint-ignore-grep-env/+merge/49884
<leonardr> sure, why not
<leonardr> benji, makes sense to me
<benji> cool
<poolie> benji, thanks for the doc feedback
<poolie> leonardr, hi
<leonardr> poolie, hi
<benji> my pleasure
<poolie> leonardr, i'm just wondering if there's a news-feed-like-thing that client authors ought to be reading to tell them about this
<leonardr> poolie, what is this? the changes to launchpadlib?
<poolie> yes, launchpadlib
<poolie> or a what's new webpage or something
<leonardr> there's no news feed. we can turn the CHANGELOG into a news fee
<leonardr> d
<poolie> i don't care so much about it being rss (i don't use that)
<poolie> as just being told "oh you shouldn't use launchpadlib.launchpad anymore" somehow
<poolie> perhaps deprecation warnings or api breaks are the most direct means to do so :)
<poolie> oh i finished reading "Restful Web Services" on the weekend btw; it was good
<leonardr> thanks
<leonardr> it's difficult to deprecate a constant, or you would have been getting deprecation warnings for a while
<poolie> i know
<wgrant> Can't you detect the deprecated service root when it is used?
<poolie> i know it's hard
<poolie> well, i guess there are two things being deprecated
<poolie> the edge server; also the symbol that points to the url
<thumper> leonardr: I seem to be having trouble with the launchpadlib test for the setRecipeText
<thumper> leonardr: it is saying that AttributeError: 'Entry' object has no attribute 'setRecipeText'
<thumper> when using version='1.0' to launchpadlib_for
<wgrant> Your WADL is probably out of date.
<poolie> for warning people not to use 'edge' we can easily warn when you use that URL
<poolie> for the name, i don't know
<poolie> just generally speaking i think python deprecation warnings are pretty poor
<poolie> because they tend to appear to users who can't necessarily do anything about them
<poolie> a warning at compile time (if there was such a thing) would be better
<lifeless> upstream python has turned off display of deprecation warnings by default
<lifeless> !
<leonardr> there's also no way to force developers to read anything or to comply with any of our standards--witness the oauth problems
<leonardr> thumper: i would also suspect out-of-date wadl, but that shouldn't be a problem in a launchpad test
<leonardr> also, if you had out of date wadl it would have setRecipeText
<lifeless> run 'make'
<leonardr> run make just to be sure
<thumper> ok
 * wallyworld takes son to doctor, back soon
<poolie> leonardr, so i'm kind of inclined to focus on making things fail in an obvious way
<poolie> and warning people ahead of time by documentation, eg blog posts
<poolie> rather than changes in the program
<poolie> not sure though
<leonardr> poolie: well, there was a blog post back in october that nobody read, since when there was a fire-drill failure yesterday one common response was "it's too much trouble to fix this"
<leonardr> i'm not saying that a deprecation warning will do any better
<leonardr> but the other thing i did yesterday was make the old code give the new behavior
<leonardr> which will help
<poolie> i think your thing of just making edge send people to lpnet is good
<poolie> considering the constraints of not being able to rewrite on the server side
<poolie> and that clients were for a long time required or encouraged to ask for edge
<poolie>  "it's too much trouble to fix this" --> meaning what?
<poolie> too hard to update clients to not use edge?
<leonardr> poolie: right. "i have 20 machines" kind of thing
<leonardr> that was how i found out it was an accident, actually--one person wouldn't take "make this trivial change" for an answer, and escalated
<poolie> anyhow
<poolie> https://code.launchpad.net/~leonardr/launchpadlib/silently-replace-edge-with-production/+merge/49794 seems reasonable to me
<thumper> leonardr: lp:~thumper/launchpad/recipe-inline-edit-recipe-text is still getting the attribute error
<leonardr> checking
<leonardr> thumper: here are a couple things to try
<leonardr> 1. don't bother removing the method and re-exporting it. just publish it as a mutator write operation
<leonardr> that might give you the behavior you want
<StevenK> thumper: My setRecipeText change just landed, too.
<thumper> StevenK: which change is that?
<StevenK> thumper: https://code.launchpad.net/~stevenk/launchpad/set-recipe-text-bad-data/+merge/48853
<thumper> leonardr: you mean remove;     @export_write_operation(),   @operation_for_version("devel") from the middle?
<leonardr> thumper: yes
<leonardr> i think we can treat this the way we treated transitionToStatus
<leonardr> no, unfortunately not
<leonardr> transitionToStatus was changed to a mutator in 'beta', not '1.0'
<leonardr> how long has this setRecipeText operation been in place, anyway? isn't this whole feature new?
<thumper> leonardr: it complains if I delete them
<leonardr> let me see the complaint
<thumper> http://pastebin.ubuntu.com/567499/
<leonardr> thumper, how new is this feature? unless it's older than version 1.0 of the web service there's no need to do this
<leonardr> ie. you can change the 'devel' version around however you want, without respect for backwards compatibility
<leonardr> thumper: you should also remove the 'operation_removed_in' annotation if you didn't already
<thumper> leonardr: how do I find out if they are in 1.0?
<leonardr> thumper: bzr annotate is probably the best way
<thumper> leonardr: when did we cut 1.0?
<leonardr> quite a while ago, possibly a year
<leonardr> "It was introduced in March 2010"
<thumper> hmm... very possible recipes weren't in 1.0
<thumper> how do we generate the wadl for 1.0?
<leonardr> thumper: it's based on the annotations
<leonardr> the thing is that once someone published setRecipeText as a named operation, it showed up in all the versions, including 1.0
<leonardr> since it wasn't explicitly excluded from other versions
<wgrant> Could we invert that behaviour?
<leonardr> but, given that it wasn't in the original 1.0, and that i suspect it was added pretty recently, i think we can just get rid of it
<wgrant> So things only show up in devel unless they are explicitly annotated?
<thumper> leonardr: I agree
<leonardr> wgrant: it would be a big project since we'd have to manually grandfather in everything else
<wgrant> leonardr: But otherwise versions are not versioned :/
<leonardr> wgrant: our policy is to let older versions have new stuff, but not to *change* behavior. we decided that anything else was a lot of work to prevent things that generally range from "slightly harmful" to "actually good"
<wgrant> leonardr: But then we need to explicitly restrict things to devel if we think we ever want them to change.
<wgrant> We should introduce features to devel and then push them back later.
<wgrant> as an explicit step.
<leonardr> we already have an explicit step "don't publish named operations that are effectively mutator methods", and setRecipeText already slipped through that crack
<wgrant> Well, that's this particular case.
<wgrant> But in general, an immature recipe API should not have slipped into 1.0.
<leonardr> but no policy i could have set would have prevented it, since the much simpler policy i set wasn't followed
<thumper> leonardr: source package recipes were not exported at all at the 1.0 time
<wgrant> leonardr: This is why we need safe defaults, I think.
<wgrant> Policies are easily missed.
<thumper> leonardr: so... the first adapter (on the bottom) should be     @operation_for_version("devel")?
<leonardr> thumper: i suggest simply removing all references to versions
<leonardr> @operation_for_version("devel") will remove setRecipeText from 1.0 and replace it with nothing
<thumper> leonardr: it seams reasonable that we should be able to annotate an Entry to say "added in devel"
<leonardr> thumper: we decided not to add support for that because it was too much work for too little benefit
<thumper> leonardr: so how do I say "this method is only in devel"?
<leonardr> thumper: that's how you'd do it. @operation_for_version("devel")
<james_w> ok
<leonardr> but it's already in 1.0, so when you fix it in devel you might as well let the fix propagate to 1.0
<thumper> leonardr: so ... http://pastebin.ubuntu.com/567508/
<thumper> hmm..
<thumper> what if we take recipe stuff out of 1.0?
<lifeless> thumper: its new right ?
<leonardr> wgrant: if we were twitter or some other very popular web service i would be very anal about this. but as it is, we don't have the manpower to maintain velocity with strict separation between versions
<thumper> lifeless: new since 1.0 yes
<lifeless> thumper: I think new stuff should be devel only
<lifeless> IMO we shouldn't change 1.0 at all
<thumper> lifeless: we are discussing how to manage this
<leonardr> ok. when you publish recipes on the web service, you can publish all of its named operations in devel only
<leonardr> you can publish all of its fields in devel only
<thumper> it would be nice to just have checked in WADL for defined versions
<wgrant> leonardr: We can take a big step towards that by just making the default devel-only.
<leonardr> you can publish all links to recipe type objects in devel only
<wgrant> leonardr: When a feature is done, we promote it to 1.0.
<wgrant> If people use a feature in devel, it's their problem if their script breaks.
<leonardr> wgrant: if the default is devel only then we (i) must manually promote hundreds of existing features
<wgrant> And they will realise it's unstable.
<leonardr> thumper: but, you can't stop something called a 'recipe' from showing up in the wadl for earlier versions, and the syntax for restricting all this stuff will be ugly
 * lifeless would like the default to be devel only; and when we cut 1.1 we shift the things that are stable from devel to 1.1
<leonardr> because our original design was based around doing version restrictions *when there was a problem with letting stuff bleed through into older versions*
<thumper> leonardr: why not?
<leonardr> we can re-evaluate this, we can always re-evaluate this, but it's going to be a big project during which imo we are not delivering value to users
<thumper> I guess it depends if you are a user of the 1.0 webservice and using things you thought were stable, but weren't
<leonardr> to clarify, our original design was based around doing version restrictions when something *changed* between versions, not when something was added
<thumper> I can see that
<leonardr> we can do this:
<leonardr> "we created the named operation setRecipeText. that was a mistake"
<leonardr> "let's leave setRecipeText alone in 1.0 and change it in devel"
<thumper> that's what we were trying to do
<leonardr> right
<thumper> but the tests weren't working
<thumper> I got a launchpadlib for 1.0
<leonardr> then i proposed checking to see how recent the feature is
<thumper> and it says setRecipeText doesn't exist
<leonardr> because maybe we can just change it everywhere
<leonardr> and short-circuit this problem
<thumper> it may well be that no-one uses this at all through the web service
<thumper> I'd be happy to just change this everywhere for this method
<thumper> but my concerns are over the versioning as a whole for the web service
<leonardr> well, as a whole the rule is to do this by the book
<leonardr> so let's do this by the book
<leonardr> and then you'll see how it works
<thumper> ok...
<thumper> so lets try to get the annotations working then
<leonardr> so, why is the test failing? i don't know. i'll investigate
<leonardr> it's possible there's a bug in lazr.restful. i don't know if we have a real case where a method is removed and then reinstated as a mutator
<thumper> ok
<thumper> can I leave this test with you to check?
<thumper> lp.code.model.tests.test_sourcepackagerecipe.TestWebservice.test_recipe_text_setRecipeText_in_one_zero
<thumper> that is the test that fails
<thumper> in the branch
<leonardr> ok
<leonardr> doing it by the book this time is annoying, but the advantage this has over 'devel only by default' is that in the majority of cases where we didn't make a mistake, we don't have to add cumbersome annotations, and people who are using 1.0 get the new stuff for free
<thumper> IMO 1.0 should not get new stuff
<thumper> but that's just me
 * thumper needs food
<leonardr> thumper: it's not just you, but the amount of work required to stop 1.0 from getting new stuff is nontrivial. and what are you doing? making 1.0 less useful
<StevenK> The other side of the coin is 'making 1.0 more stable, and doing what we said we would'
<lifeless> leonardr: FWIW we make mistakes most of the time
<leonardr> lifeless: maybe 'bleed through by default' was just another mistake then
<leonardr> thumper: the operation works in beta but not 1.0. that makes me think there may be a bug in lazr.restful
<lifeless> leonardr: the reason I say we make lots of mistakes is because we do incremental development
<lifeless> leonardr: so its baked in that we'll iterate and tweak
<lifeless> I think this is a positive thing, but the friction it has with support guarantees like 1.0 is pretty high.
<leonardr> i say this because lazr.restful has a feature that let us deprecate 'transitionToStatus' et al. in a reasonable way, and that might be interfering with what we're doing here
<lifeless> interesting
<leonardr> lifeless: so, it sounds the early-2010 web service versioning strategy conflicts with the early-2011 development strategy
<lifeless> leonardr: tell me about this deprecation feature
<leonardr> lifeless: look at lib/canonical/launchpad/rest/configuration.py
<leonardr> last_version_with_mutator_named_operations = "beta"
#launchpad-dev 2011-02-16
<leonardr> in 'beta', a @mutator_for method is a mutator but it's also published as a named operation
<leonardr> so you can set bug.status or you can invoke transitionToStatus
<leonardr> post-'beta', a @mutator_for method is only a mutator
<leonardr> there was no way to describe this behavior with annotations, so i had to add a new feature to lazr.restful
<lifeless> ok
<leonardr> i don't know how it might be happening but i have a suspicion that feature is interfering with what we're doing here
<lifeless> is that because annotations are per export ?
<leonardr> annotations can make something not be a mutator anymore, but they can't change what it means to *be* a mutator
<lifeless> ok
 * leonardr does what he should have done in the first place and looks at lazr.restful unit tests
<leonardr> thumper: the second @operation_for_version is not necessary. @operation_removed_in_version("1.0") automatically bumps the version counter up to 1.0
<leonardr> by taking that out i can get setRecipeText to show up in 1.0, but if i make it a mutator in devel it disappears from 1.0
<leonardr> so i think there is a lazr.restful bug
<leonardr> let me try to duplicate within lazr.restful where it's more convenient
<poolie> leonardr, i may well be wrong but i thought i saw lp:kanban give me an edge url to create a token
<poolie> even though i updated it to request not to use edge
<poolie> it may be something weird here
<leonardr> poolie: i'm happy to take a look, but i'm long past eod already
<leonardr> and i have this other problem to look into
<poolie> np just letting you know
<leonardr> k
<poolie> if i can reproduce it i'll file a bug
<poolie> i may well have just had an out of date tree or something
<leonardr> sounds good
<wallyworld_> anyone want to claim a code review? https://code.launchpad.net/~wallyworld/launchpad/request-build-popup/+merge/48864
<lifeless> wgrant: 4 seconds spent doing portlet calculations AFAICT
<lifeless> wgrant: I think I'm going to disable ilike temporarily, announce that, and then work on the portlet after I finish bugtask:+index landing
<poolie> wallyworld_, weak +1 on that
<wallyworld_> poolie: ?
<poolie> your mp looks good to me but i'm not very familar with what you're changing
<StevenK> wallyworld_: Why are you moving from .builds to builds_for_recipe()? That change looks spurious to me.
<wallyworld_> StevenK: it's needed because there's now 2 places (soon to be 3) that need to call that. and so moving it off SourcePackageRecipeView to a helper method that can be called elsewhere supports that
<poolie> leonardr, if you're reading scrollback, it was in fact another reference to edge from kanban
<leonardr> poolie: you mean you reproduced the problem?
<wallyworld_> StevenK: SourcePackageRecipeView and SourcePackageRecipeRequestBuildsAjaxView use it
<poolie> i did, and it's a bug in kanban not lplib
<StevenK> wallyworld_: Why can't they just use self.context.builds ?
<wallyworld_> StevenK: self.context is a recipe and the builds() logic is in the view
<LPCIBot> Project devel build (445): STILL FAILING in 6 hr 26 min: https://hudson.wedontsleep.org/job/devel/445/
<StevenK> wallyworld_: .builds for a recipe should still DTRT?
<wallyworld_> StevenK: different requirements. recipe has getBuilds() and getPendingBuilds(). the view builds() calls these in order to show all the pending builds and 1-5 recent builds
<StevenK> wallyworld_: Hmmmm
<wallyworld_> StevenK: the logic to construct the contents of the builds table was already there before my mp. i'm just using it to implement a view that returns just the builds table separately instead of the entire recipe index page
<wallyworld_> StevenK:  so that an ajax call can do stuff and then refresh the builds table with the latest info without needing a page refresh
<leonardr> thumper: i'm fairly sure there is a lazr.restful bug but i can't reproduce it within lazr.restful
<leonardr> i can come back to this in the morning if that's all right
<thumper> leonardr: sure
<wallyworld_> thumper: the request daily build stuff all working but requires js. do we want to just not show the link if js is disabled?
<thumper> wallyworld_: it should be quite possible to have it working without js too
<thumper> wallyworld_: pretty trivially even
<wallyworld_> thumper: ok.
<wallyworld_> thumper: also, where did you want the link? currently i've put it just below the build schedule field (just below where it says "daily build")
<thumper> wallyworld_: got a pic?
 * wallyworld_ fires up ksnapshot
<wallyworld_> thumper: http://people.canonical.com/~ianb/request-daily-build.png  (I don't like the + icon)
<wgrant> Can I suggest s/daily/manual/?
<thumper> wallyworld_: why do you have a + icon?
<wgrant> Requesting a daily build doesn't make a huge amount of sense.
<thumper> wallyworld_: how about (build now)
<lifeless> how about 'immediate'
<lifeless> or now
<wgrant> yeah.
<wgrant> That.
<wallyworld_> thumper: i wanted to put an icon there but am unsure which one is best. perhaps we don't need an icon at all. i think it looks plain without one. but that's just me
<wallyworld_> thumper: the + icon is a placeholder
<wgrant> It needs an icon. But I don't know of an appropriate one.
<thumper> what build icons do we have?
<cody-somerville> ugh
<wallyworld_> wgrant: while i have your attention - want to be release manager for 11.03? :-)
<wgrant> The milestone icon is almost right, maybe.
 * wallyworld_ looks
<thumper> wgrant: the clock?
<wgrant> thumper: Yes.
<cody-somerville> Clicking 'Find...' next to 'Assign to:' field when filing a bug takes you to https://bugs.launchpad.net/people/ - making you loose your bug report.
<thumper> seems reasonable
<wgrant> cody-somerville: I filed that a year or so ago.
<thumper> wallyworld_: can you try with the clock icon, and the text: Build now
<thumper> wallyworld_: then lets look at the picture
 * cody-somerville just had a funny image of flacoste using a spinner from twister to randomly select teams to assign to bugs.
<wallyworld_> thumper: i can't see the clock icon in lib/canonical/launchpad/images??? have i missed seeing it or am i looking in the wrong place?
<wgrant> wallyworld_: milestone.png
 * wallyworld_ nods
<wgrant> It is not optimal to overload the icon's meaning. It's mostly used for milestones and series at the moment. But it's the most appropriate thing we have.
<wgrant> Although for a lot of daily builds the flame icon might be appropriate...
<thumper> heh
<thumper> wgrant: there is the "building" icon
<thumper> the animated one
<wgrant> thumper: But it's animated :(
<wgrant> and I don't really like it much.
<wallyworld_> hmmm. icon='milestone' in the Link() constructor doesn't work. still get the + icon
<wgrant> What's the HTML?
<thumper> wallyworld_: it should end up being an anchor with the class="sprite milestone"
<wallyworld_> thumper: yeah, still has "sprite add" though. so Link('+request-daily-build', 'Build now', icon='milestone') doesn't seem sufficient
 * wallyworld_ tries a make clean
<thumper> wallyworld_: you need to restart to see view changes
<wallyworld_> thumper: already tried that :-)
 * thumper goes to collect the girls
<wgrant> OOPS-1872BZR108821
<wgrant> That is a lot of OOPSes.
<wgrant> And a strange OOPS.
<mwhudson> wgrant: is that from a smart server process?
 * StevenK blinks at the oops report for Soyuz
<wgrant> mwhudson: I presume so.
<wgrant> StevenK: Hmm, that's a bit concerning.
<wgrant> dpkg accepted the version...
<StevenK> Pity it doesn't contain any more information.
<wgrant> I will find the upload.
<wgrant> 2011-02-15 19:40:56 DEBUG   Verifying source file vdr-plugin-xinemediaplayer_.11-1easyVDR1.tar.gz
<StevenK> Neat
<wgrant> I'm surprised dpkg-source accepted that.
<StevenK> Same.
<StevenK> At the very least, that also warrants a dpkg bug.
<wgrant> Although maybe not.
<wgrant> Why should dpkg-source check the version string?
<wgrant> On extraction, that is.
<wgrant> On building, maybe.
 * wgrant tries.
<wgrant> The source package builds fine.
<wgrant> I wonder if binaries do.
<wgrant> Yup, and dpkg installs them.
<wgrant> Baaaaaaah
<wgrant> The upstream_version may contain only alphanumerics[33] and the characters . + - : ~ (full stop, plus, hyphen, colon, tilde) and should start with a digit.
<wgrant> "should"
<wgrant> StevenK: I guess valid_version may want to be fixed. Although we could decide that we don't want to support that, since it's apparently never been done before...
<wgrant> What do you think?
<wgrant> Interesting.
<wgrant> .1 > 1
<wgrant> In fact it seems to be greater than just about anything.
<wgrant> So it's probably a good idea to reject it.
<wgrant>   41 ProtocolError: <ProtocolError for http://downforeveryoneorjustme.com/http://bugs.opencompositing.org/xmlrpc.cgi: 405 Method Not Allowed>
<wgrant> Awesome.
<lifeless> erm wtf
<wallyworld_> thumper: http://people.canonical.com/~ianb/request-daily-build.png
<wallyworld_> i don't like the green text with that icon
<wallyworld_> blue would be better imho
<wgrant> Once we have pervasive JS that might be good.
<wgrant> But we'd need to change globally.
<thumper> wallyworld_: hmm...
<thumper> I don't like that icon
<wallyworld_> what about the copy icon?
<thumper> wallyworld_: try "yes"
<thumper> which is the copy icon?
<thumper> do we have any gear icons?
<wgrant> thumper: The copy icon is the edit icon.
<wgrant> It sucks.
<thumper> no, that blows
<thumper> sucks and blows at the same time
<wgrant> We have a gear icon. But it flashes.
<wallyworld_> thumper: hit refresh
<wgrant> build-success-publishing.gif
<thumper> wallyworld_: that's a maybe
<thumper> wgrant: which is that?
<wgrant> It's used on Archive:+packages to indicate builds that have finished but are not yet published.
<lifeless> grr flakey code
 * thumper was looking for that page that shows all the images
<wgrant> thumper: +graphics
<wgrant> Possibly for the sole purpose of making it ungreppable.
<thumper> thanks
<lifeless> yay for race condition bugs
<lifeless> goes to show, don't make things faster.
<wgrant> lifeless: A race on BugTask:+index?
<lifeless> wgrant: bug message
<wgrant> Ah.
<lifeless> see bugcomment.py - group_comments_with_activity
<thumper> wallyworld_: what about just using the source-package-recipe icon?
<lifeless> if you have two bug comments in the same datetime - and the test can generate 7 in the same datetime sometimes - then the sort order depends on sorted() being stable
<lifeless> but its not guaranteed stable
<wgrant> Hah.
<wgrant> That code was changed recently.
<wgrant> The window was just added.
<lifeless> orly?
 * wallyworld_ looks
<wgrant> Previously it only grouped things at the same second.
<lifeless> wgrant: I think you misunderstand
<lifeless> wgrant: bug comments ordering is the question
<LPCIBot> Yippie, build fixed!
<LPCIBot> Project db-devel build (369): FIXED in 6 hr 28 min: https://hudson.wedontsleep.org/job/db-devel/369/
<lifeless> wgrant: same datetime - doesn't matter if its per second or per day or per ms grouping
<lifeless> wgrant: race still exists, need to disambiguate by message.index
<lifeless> (though of course, thats *wrong* if we don't fix the bugimporter to use the date of import not the date on the remote tracker.
<lifeless> but I'll leave that for the next person that comes in and goes wtf.
<lifeless> anyhow, Iwas getting that one time in 3, but I think I've fixed it
<lifeless> the race that is
 * wallyworld_ needs to add source-package-recipe entry to style-3-0.css.in
<lifeless> lp:~lifeless/launchpad/bug-607935 seems to be passing all tests, fingers crossed.
 * lifeless waits for diff to update
 * thumper shelves current work to wait for LP.client.cache updates
<lifeless> I can has review? https://code.launchpad.net/~lifeless/launchpad/bug-607935/+merge/49915
<wgrant> lifeless: https://bugs.launchpad.net/bugs/7217/+watch/68410
<_mup_> Bug #7217: epiphany-browser: epiphany crashes trying to print <epiphany-browser (Ubuntu):Fix Released by seb128> <epiphany-browser (Debian):Fix Released> < https://launchpad.net/bugs/7217 >
<wgrant> We record errors along with their message and the relevant OOPS.
<wgrant> I think I will clarify the message and drop the OOPS for most known failures.
<wgrant> Perhaps "clarify" is not the correct term for expanding on "Unknown"
<wgrant> Except that most of the exceptions appear at the bugtracker level, not the bugwatch level.
<wgrant> Argh.
<lifeless> wgrant: cool
<wgrant> No, not cool.
<lifeless> wgrant: easy answer - show the bugtracker health against the bugwatch
<wgrant> lifeless: Against all 20000 bugwatches?
<lifeless> wgrant: when rendering in a bug
<lifeless> they are related, no?
<wgrant> Well, sure. But I was hoping for a solution that didn't require DB changes :)
<wgrant> But I need new enum values for bugwatch warnings anyway.
<wgrant> So we might as well just do this properly.
<lifeless> tis always a tradeoff
<lifeless> you could squelch the oops, log at info or whatever so there is a record, and file a bug for future improvements.
<lifeless> right now users are equally badly off in this proposal.
<wgrant> We need a new BugTrackerActivity table and BugWatchActivityStatus.WARNING
<wgrant> Once we have those we can notify users adequately.
<lifeless> what would the table store ?
<wgrant> It would store bugtracker-wide errors.
<wgrant> Because of the batch update mechanism, most of the exceptions don't have an associated bugwatch.
<wgrant> Has the description textarea on +filebug shrunk, or is it just me?
<lifeless> wgrant: do we know the bugwatch(es) that we're /trying/ to update?
<lifeless> wgrant: anyhow, such a schema change sounds reasonable
<lifeless> wgrant: I only suggest a lower key thing to keep in the spirit of maintenance vs features
<wgrant> lifeless: We don't.
<lifeless> wgrant: so the api just triggers 'changes since date' or some such?
<thumper> wallyworld_: how's that image going?
<wgrant> Something like that. I forget the details.
<wallyworld_> thumper: &^!@^!@! make was failing
<lifeless> seems to me we know then - that case would be 'all bugs'
<lifeless> [for that tracker]
<wgrant> lifeless: That is a lot of bugwatchactivities.
<wallyworld_> thumper: turn out you have to make sprite_image separately or else it all falls in a heap
<wgrant> wallyworld_: The sourcepackagerecipe icon wasn't already a sprite?
<wgrant> Wait, what is the sourcepackagerecipe icon?
<wgrant> Oh, right. Package with a branch.
<wallyworld_> wgrant: no. not a sprite already
<wallyworld_> but why isn't the sprite_image target invoked for a make clean build or even a make build?????
<wallyworld_> if i commit this change, then others will have the same issue, no?
<wgrant> The sprite image is committed to the tree.
<wgrant> Because it changes once in a fairly long time.
<wallyworld_> oh ok
<wallyworld_> i didn't realise that. surely though commiting binary blobs that are generate from source is bad :-(
<wgrant> Plus the location calculation is not entirely deterministic.
<wgrant> Yes, it is.
<wgrant> Rather bad.
<wallyworld_> it's not as if sprite_image takes more than asecond or so to run either
<wgrant> But people don't seem to mind including large trees of other people's code in our tree.
<wgrant> So what's a small blob? :)
<wallyworld_> so why the fark don't we just run it as part of make
<wallyworld_> i just wasted a looong time figuring this out :-(
<wallyworld_> wgrant: one small blob is a blob too many :-(
<wallyworld_> thumper: http://people.canonical.com/~ianb/request-daily-build.png
<wallyworld_> thumper: can we/i fix the !!@%@!^& makefile?
<wgrant> wallyworld_: You may want to check with a LOSA.
<wgrant> Also make the positioning stuff deterministic.
<wallyworld_> or do you agree with commtting the sprite image blob?
<spm> hrm?
<wgrant> When I rebuilt it Monday a couple of things changed order.
<wallyworld_> wgrant: so what did you do to account for the ordering change
<wgrant> spm: Where is 'make build' done on production?
<wgrant> wallyworld_: Nothing, because I didn't end up committing my changes.
<spm> on each prod server
<wallyworld_> spm: we are committing a binary blob (the sprite image file) when imho it should be built
<wgrant> wallyworld_: We need to be careful that we don't end up with different files across the frontends, I think.
<wgrant> Or caching may get a bit odd.
<wallyworld_> wgrant: so how did you know things changed order?
<spm> wallyworld_: by sprite - this is a picture?
<wgrant> wallyworld_: icon-sprites.positioning changed.
<wallyworld_> spm: yeah, it's a binary blob of all our little icons that are displayed next to links etc - a combination of the individual png files if you like
<wallyworld_> wgrant: so in that case both files would need to be committed and it should just work
<spm> so why do you want to build that on each server? it'd be pretty static - build and control it yourself?
<wgrant> wallyworld_: Which files?
<spm> esp aiui, this'd only need building on 2 servers - can you get the granularity down such that it *only* builds on 2 of the 30 odd?
<wallyworld_> spm: committing blobs which are built from source artifacts is imho fundamentally wrong and bad practice
<spm> building on 30 servers, something needed on 2, is also ... wrong. :-)
<wallyworld_> spm: make build should do it
<wallyworld_> it only takes a second
<wallyworld_> spm: the makefile targets should only build what's required
<wallyworld_> for each deployment target
<spm> yesssss. so does every other little build. which is why  a nodowntime rollout takes from 60-90 minutes.
<wallyworld_> s/target/environment
<wallyworld_> spm: it could be done before rollout though - part of creating the rollout directory image
<wallyworld_> oh, you said nodowntime
<spm> actually - step back a bit here - *what* builds this? what tools are needed?
<wallyworld_> spm: make sprite_image builds it
<spm> does that need imagemagik or something?
<spm> I guess - how often do these change?
<wallyworld_> its a tool in the bin directory which also does other sprite related stuff (css file generation etc)
<wgrant> It uses PIL.
<wgrant> So it should be fine.
<wallyworld_> spm: they don't change often
<spm> just checking we don't suddenly introduce a bunch of crackful deps to add to prod. :-)
<wallyworld_> it's more a philosophical thing, plus i just wasted shitloads of time cause i expected make to build what it needed
<spm> heh
<wallyworld_> by philosophical, we are talking about what consistutes best practice etc
<spm> ha. don't make me pull out the page on argument fallacies. ;-)
<spm> I guess in the overall scheme of things, go for it. add it to the makefile.
<wallyworld_> spm: i likely won't change the makefile, once i build a bridge and get over my frustration
<wallyworld_> spm: clearly oeple smarter than me did it that way for a reason :-)
<spm> rules are there so you think about breaking them. not to be followed blindly. :-)
<wallyworld_> :-)
<wallyworld_> try telling that to my wife
<spm> wives are an exception to all rules, except their own. QED.
<wallyworld_> you are a wise man
<poolie> hi spm
<spm> heya poolie
<spm> wallyworld_: my 2c for something like this: that doesn't change often and all, denormalise - commit the binary blob and wear the overhead there - to get a bigger saving in prod/deploy/build. but each case has exceptions.
<wallyworld_> spm: fair enough
<poolie> +1 for naked juggling
<wallyworld_> poolie: ha de ha ha
<spm> if it had funky deps on code changes and such? then you make build on each server per normal; but if it's largely standalone. shrug.
<wallyworld_> poolie: i don't think you really would want to see that
 * spm afks to fetch lad from school
<wallyworld_> thumper: you around?
<wgrant> Daily builds for this recipe will not occur.
<wgrant> There is no PPA.
<wgrant> That sounds a little dramatic.
<wgrant> wallyworld_: Do you know much about the daily build archive widget?
<wallyworld_> wgrant: depends on what you want to know :-)
<wgrant> wallyworld_: The picker popup needs spinners.
<wgrant> The list takes a while to load, leaving the popup empty for a couple of seconds.
<wallyworld_> wgrant: because it takes time to load them up?
<wgrant> Then when you move to another page the number changes immediately, but the contents take a couple of seconds.
<wgrant> In both cases I think the contents should be replaced with a spinner.
<wgrant> Or something like that.
<wallyworld_> wgrant: yeah, i just wrote a popup form and added a "Loading..." spinner for the same reason
<wgrant> Also there is a vast amount of padding at the bottom of the popup.
<wgrant> Ah, because there is an unused footer slot.
<wgrant> Deleting that makes things marginally more sensible.
<wgrant> It seems to be a general picker issue.
<wgrant> is that lazr-js?
<wallyworld_> wgrant: i think the best thing is to file a bug so it's in the system
<wgrant> wallyworld_: Sure, I'm just wondering where to file it.
<wgrant> Is it LP, is it Code, is it lazr-js?
<wallyworld_> wgrant: yeah, the base picker.js is in kazr-js from memory
<wgrant> Otherwise the AJAXyness of SourcePackageRecipe:+index is pretty awesome.
<wallyworld_> wgrant: i'd file against launchpad since the issue manifests itself in the recipe area. that's imho
<wallyworld_> wgrant: once my mp lands it will have even more ajax
<wallyworld_> the request builds link uses a popup form with a no-pageload refresh of the current builds table
<wgrant> wallyworld_: Ah, other pickers have a search widget where the search icon turns into a spinner.
<wgrant> The daily builds ones do not.
<wgrant> I will file against launchpad.
<wallyworld_> wgrant: that was my thinking - that the root cuase lay in how the picker was being used, hence a lp issue
<wgrant> wallyworld_: bug #719785, bug #719788, bug #719795
<_mup_> Bug #719785: Recipe owner and archive pickers need spinners <recipe> <ui> <Launchpad itself:Triaged> < https://launchpad.net/bugs/719785 >
<_mup_> Bug #719788: Recipe archive picker has an empty footer <recipe> <ui> <Launchpad itself:Triaged> < https://launchpad.net/bugs/719788 >
<_mup_> Bug #719795: Recipe archive picker items are twice the required height <recipe> <ui> <Launchpad itself:Triaged> < https://launchpad.net/bugs/719795 >
<wallyworld_> wgrant: thanks. thumper will be ecstatic :-)
<wgrant> Heh.
<wgrant> I also have some suggestions for the builds table, but I think they might cause thumper to leap over the Tasman and strangle me.
<thumper> hmm....
<thumper> wgrant: I'm open to suggestions
<thumper> wgrant: do the pickers not have any spinners?
<wgrant> thumper: They don't :(
<thumper> wgrant: that's a problem
<wgrant> thumper: I'd prefer the builds table to be more like Archive:+packages. A single row per source, only showing binaries if they are not yet built or have failed.
<StevenK> wgrant: Hm, do you know off-hand if the sampledata has another account enabled except admin@c.c?
<wgrant> Archive:+packages' implementation is rather flawed. But you have the opportunity to experiment and improve upon that.
<thumper> wgrant: not really
<thumper> wgrant: we are moving off recipes into maintenance shortly
 * thumper goes to make dinner
<wgrant> StevenK: no-priv@canonical.com, carlos@canonical.com, test@canonical.com, cprov@ubuntu.com
<wgrant> thumper: :(
<StevenK> wgrant: Thanks
<StevenK> Not allowed here reproduced for recipes with a disabled archive.
<StevenK> Which is a little O.o
<StevenK> Oh, let me guess. The security proxy is such that if the archive is disabled, only the owner can see it?
<wgrant> That's right.
 * StevenK grumbles
<StevenK> I'll fiddle the browser code to give us 'Disabled archive' rather than unathorized
<lifeless> wgrant: are you talking the +dailybuilds thing, or the per-user recipes list?
<wgrant> lifeless: SourcePackageRecipe:+index
<lifeless> thumper: will the daily builds page we spoke about get fixed before you rotate off?
<wgrant> Can I search through OOPSes for a particular exception without grepping through $lots?
<lifeless> not yet
<wgrant> Oh, I can see lp:oops-tools now. That is handy.
<wgrant> Ahhhh
<wgrant> Most of these OOPSes used to be in a warnings subsection of the checkwatches OOPS report.
<lifeless> still covered by zop
<wgrant> Should they be?
<wgrant> Remember we have UFD and go.
<wgrant> *co
<wgrant> OffsiteFormPostError, UnexpectedFormData, InvalidBatchSizeError all OOPS but are in a separate section of the report.
<wgrant> Still, some of these checkwatches OOPSes record no information beyond what is put in the DB, so they can go.
<lifeless> unless we're going to take action on it, they shouldn't be logged.
<lifeless> We have significan space issues with oops
<wgrant> Oh, do we?
<lifeless> yes
<wgrant> :(
<lifeless> plus rsync overhead, processing on devpad etc
<wgrant> True.
<lifeless> oops should be things /we/ need to take action on
<spm> ^^ this! :-)
<wgrant> lifeless: Caps!@
<wgrant> Also, bug #28506 is probably not the one you are looking for...
<_mup_> Bug #28506: It should be possible to install xchat instead of xchat-gnome <ubuntu-meta (Ubuntu):New> < https://launchpad.net/bugs/28506 >
<lifeless> wgrant: context?
<lifeless> do you mean 268508 ?
<wgrant> lifeless: Your blog post says 28506
<lifeless> grah
<lifeless> edit-fail
<lifeless> the right link is earlier
<lifeless> fixed
<lifeless> wgrant: thanks
<wgrant> lifeless: I'd also retitle it something like "Bug searches no longer match target names", I think.
<wgrant> But perhaps not.
<lifeless> thats more accurate and less relevant or understandable IMO - I did consider it
<lifeless> but something like 99.9% of multi context searches will be package related
<wgrant> s/target/source package/, then.
<lifeless> that would be a lie
<wgrant> But caps and short is important.
<wgrant> Thanks.
<wgrant> If it didn't show up on https://launchpad.net/ I would not be so pedantic.
<poolie> wgrant, i agree
<poolie> 'read the blog' is bad style too
<lifeless> 'read the blog'?
<wgrant> Down the bottom of the recent blog post listing.
<wgrant> I don't have much of a problem with that, though.
<LPCIBot> Project devel build (446): STILL FAILING in 5 hr 47 min: https://hudson.wedontsleep.org/job/devel/446/
<LPCIBot> * Launchpad Patch Queue Manager: [r=jtv][bug=710591] Fix translation import script to use old-style
<LPCIBot> imports on source packages if no upstream project is configured.
<LPCIBot> * Launchpad Patch Queue Manager: [r=gmb][no-qa] Split portlet setup for subscriber widgets from
<LPCIBot> bugtask-index.js.
<LPCIBot> * Launchpad Patch Queue Manager: [r=jtv][ui=none][bug=716586] Allow converging diverged translation
<LPCIBot> * Launchpad Patch Queue Manager: [r=gmb][bug=718849] Do not escape revision author if the name and
<LPCIBot> email are None.
<LPCIBot> * Launchpad Patch Queue Manager: [r=leonardr][bug=719162] Make buildd pointer check regexes work on
<LPCIBot> natty
<lifeless> grrrrah
<lifeless> series_buglistings makes me sad
<thumper> lifeless: probably not...
<thumper> lifeless: but on the bright side, if it is a timeout, it'll get addressed early
<poolie> hi jml?
<jtv> gahmorning folks
<poolie> hi jtv
<henninge> Hi jtv!
<poolie> are you in nl?
<jtv> yes
<jtv> hi henninge!
<lifeless> stub: if you have time, would appreciate a review of https://code.launchpad.net/~lifeless/launchpad/bug-607935/+merge/49915
<stub> k
<jtv> hey stub
<stub> jtv: Tell me about your standing desk and where you got it.
<poolie> i'd like to know that too
<stub> 1) Fly to Bangkok
<jtv> stub: HomeProâit's not a desk and I've seen the same equipment in other countries.
<jtv> Basically built myself a rack.
<stub> Assemble it yourself stuff?
<stub> Or some sort of stackable shelving?
<lifeless> does anyone know of existing statistics apis within lp ?
<StevenK> lifeless: As in download statistics? More information?
<lifeless> as in 'get the open bug counts for all the active series of ubuntu'
<lifeless> I mean in the source tree, not the json APIs
<mrevell> Hello
<poolie> hi mrevell
<poolie> would you like to have a look over https://dev.launchpad.net/LEP/BuildFromBranchIntoPrimary for me?
<mrevell> poolie, Sure. With a view to user testing?
<adeuring> good morning
<poolie> yes, or communication/announcement/etc
<poolie> well, more communication
<StevenK> bigjools, wgrant: I suspect allenap could use a hand to QA r12383.
<allenap> Indeed. I don't have the foggiest about how to QA that one.
<bigjools> we need to get mup to show commit messages when it sees rNNNNN
<allenap> Well, apart from saying dogfood.
<bigjools> pedigree chum to the rescue
<allenap> bigjools: Good idea.
<mrevell> Thanks. I'll take a look later  my morning  poolie and email you some proposals.
<allenap> Who's the mupmaster?
<bigjools> gustavo IIRC
<poolie> thanks
<poolie> ok i've got to go, may be on again later
<bigjools> allenap: I'm not sure that's QA-able.  I don't recall seeing any weird effects in production from the bug  - although I expect the memory would grow.
<wgrant> Aha.
<allenap> bigjools: We would need to add builders after starting the buildmaster, then observe how often they're scanned. But that sounds pretty hard to do.
<bigjools> allenap: yes
<allenap> The code change is a single line so perhaps we can rely on visual inspection alone.
<wgrant> It is not at all hard to do.
<wgrant> Add a new virtual builder that does not exist.
<wgrant> It will remain enabled.
<wgrant> Despite the scans failing.
<allenap> Interesting :)
<bigjools> you would need to rely on the log output
<wgrant> In other news, I have a bzr-sftp locally with 6 fifos opened without connections.
<wgrant> I may have reproduced the leak.
<wgrant> Yay.
<wgrant> poolie: Around?
<poolie> wgrant, yay, that's greaot
<poolie> i have to go out now
<poolie> may be back in about an hour
<poolie> vila is here
<wgrant> k
<vila> *blink*
<vila> paramiko leaks ?
<wgrant> vila: No, I am trying to reproduce the forking service disaster.
<vila> got that, and no, paramiko may leak threads, not sockets (IIRC)
<vila> when you say fifo you mean pipes ?
<wgrant> Right.
<vila> k, np, just checking
<vila> wgrant: did you check with jam ? I think it was working on it yesterday
<wgrant> vila: it seems that I may have actually exceeded the FD limit for a moment, so it crashed and failed to close sockets for a few seconds, which I initially missed due to the logs going by so quickly :(
<wgrant> So the FD leak was because we were already exhausted.
<wgrant> Still, we should probably avoid leaking in that case.
<vila> that's my understanding yes, the symptom is hitting the FD limit but the cause is the leak
<stub> bigjools: Where you working on dropping sourcepackagename/binarypackagename links for a few selected cases, or across the board?
<bigjools> stub: otp
<bigjools> one mo
<stub> np
<bigjools> allenap: I'll QA that buildd-manager change for you
<bigjools> allenap: although there is a buildd-manager on staging so you could do it with some losa fu
* danilos changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | firefighting: - | On call reviewer: danilos | https://code.launchpad.net/launchpad-project/+activereviews
<danilos> stub, hi, is there any better (as in less intrusive) way to change a DB column default value other than with a DB patch?
<danilos> stub, (I guess I can use the constructor, but that kinda sucks)
<stub> danilos: The only way to change the DB column default is with a DB patch. If you are using the constructor, you are doing something different.
<danilos> stub, right, I understand that, thanks
<bigjools> stub: since changing everything is such a ridicuously massive and invasive change, I was planning on adding a name column to spr/bpr for now, and then gradually migrating certain things
<stub> If you put a default value into Python code for some reason, we should update the DEFAULT on the column too or else confusion will happen.
<bigjools> we may never be able to get rid of BPN/SPN because of bug targets
<stub> bigjools: I thought one or two tables at a time might be best.
<stub> bigjools: You dropping the foreign key too?
<bigjools> which FK?
<stub> bigjools: The existing references to the spn/bpn tables
<bigjools> stub: where possible, yes, I need to analyse
<stub> bigjools: There is no point having both a TEXT column containing the name, and a foreign key reference to the (hopefully identical) name in spn/bpn.
<bigjools> quite
<bigjools> it depends on what code needs it and how easy it is to fix
<stub> bigjools: So if there is a db patch adding the new TEXT column and not dropping the old spn/bpn references, open a bug and have XXX's referencing it in the model code.
<bigjools> mais oui
<bigjools> I need to audit all the stuff that uses bpr.name / spr.name
<danilos> stub, that's why I was asking if there was a less intrusive change, just changing it in python is effective for our tests, but I'd like to fix the default in DB as well
<stub> Don't consider it intrusive :-)
<dpm> jtv, would you have some minutes to see how we can recover the en-US.xpi Lucid + Maverick template and reupload it now that the Firefox bug in LP has landed?
<jtv> dpm: ah yesâ¦ can we do that after lunch?
<danilos> stub, heh, it's intrusive to my development process: I can land entire branch onto devel, and this one little bit (DB patch) has to go to db-devel :)
<dpm> jtv, sure, I'll ping you later on then
<stub> danilos: The db patch doesn't have to be part of the same branch.
<stub> But yeah, two branches to land vs. one.
<danilos> stub, so, can you please take a look at https://code.launchpad.net/~danilo/launchpad/db-bug-718809/+merge/49952 ? :)
<stub> k
<stub> danilos: So what do we do with the 1.3 million entries that have been set to the previous default of True ?
<stub> (And 131 people have already twiddled their knob)
<danilos> stub, we want to keep it as is
<danilos> stub, at least that's my understanding: we do not want to change values for existing users, just for new ones
<stub> So existing users keep the original behavior (unless they twiddle the setting), and new users default to no selfgenerated notifications?
<danilos> stub, and I believe that's pretty explicit in bug 718809 description
<_mup_> Bug #718809: New users should default to not receiving email for their own actions <story-better-bug-notification> <Launchpad itself:In Progress by danilo> < https://launchpad.net/bugs/718809 >
<danilos> stub, that's right
<stub> r=stub
<danilos> stub, cheers
<danilos> stub, do you think I should update the sample data before landing?
<stub> danilos: I wouldn't. I don't think changing the default for new users warrents changing the sampledata (by definition, contains existing users).
<stub> Matches production better too ;)
<danilos> stub, right, but 'make lint' issues a warning already, so I somehow suspect that somebody has failed to update it after running all the DB patches (probably some which migrate data)
<danilos> stub, so I am only asking if I should re-make sampledata after a clean 'make schema' :)
<stub> Up to you then. I'm not fussed myself.
<danilos> stub, cool, I'll just ignore it then and leave it for someone who actually does a more invasive patch :) ta
<stub> If you are landing now, go for it to minimize future conflicts.
<danilos> stub, heh, ok then
<danilos> stub, actually, the differences seem irrelevant: http://paste.ubuntu.com/567629/
<danilos> stub, should I file a bug against the schema linter about this?
<stub> If you want. Seems benign enough. I don't know what is doing the check or how easy it is to modify.
<stub> danilos: Maybe not - we actually do want it to bitch in this case, to ensure devs keep up to date with PG releases and in particular are using the correct major release.
<danilos> stub, oh, I thought you'd be the one who has implemented the check: the lint warning is very long and kind of annoying as you can see in the MP :)
<stub> (different releases causing ordering to be different and making diffs huge)
<stub> Not me :-)
<danilos> stub, right, but this is a minor rev and that sucks :)
<stub> Think I run lint? Ha!
 * stub runs
<danilos> stub, haha
<danilos> now, how does an OCR find a reviewer? :)
<deryck> Morning, all.
<danilos> deryck, good morning :)
<bigjools> danilos: (or any translations dude), is this valid or a dupe?  https://bugs.edge.launchpad.net/launchpad/+bug/719903
<danilos> bigjools, it is definitely a bug, but it's likely related to the recent translations work; henninge, do you know if it's a dupe?
<danilos> deryck, or perhaps you? :)
<henninge> danilos: dunno, busy atm
 * deryck looks at bug
<deryck> bigjools, danilos -- not a dupe that I know of.
<bigjools> any code people around?  this one's a bit weird https://bugs.edge.launchpad.net/launchpad/+bug/719901
<_mup_> Bug #719901: No web access to lp:language-selector <Launchpad itself:New> < https://launchpad.net/bugs/719901 >
<bigjools> https://code.launchpad.net/~ubuntu-core-dev/language-selector is fine but  https://code.launchpad.net/~ubuntu-core-dev/language-selector/ubuntu is not
<wgrant> bigjools: Probably the private bug.
<wgrant> I can see it fine.
<wgrant> Because I can see apport bugs.
<wgrant> What is the traceback you see?
<bigjools> meh didn;t think to look down there, yeah it's a bug
<bigjools> wgrant: why can you see apport bugs then?
<wgrant> bigjools: Because I'm in ~ubuntu-dev.
<bigjools> ah right
<gary_poster> Can anyone be my bzr pipeline mentor at the moment? :-)  sync-pipeline didn't work, as Aaron warned me privately it might not since I used reconfigure-pipeline.  I got
<gary_poster> Creating new pipe at lp:~gary/launchpad/bug164196/.bzr/pipes/bug164196
<gary_poster> bzr: ERROR: Permission denied: "~gary/launchpad/bug164196/.bzr/pipes/bug164196/": : Cannot create branch at '/~gary/launchpad/bug164196/.bzr/pipes/bug164196'
<gary_poster> I then tried lp-propose
<gary_poster> and that seems to want to propose everything for the same push branch
<gary_poster> which doesn't seem to work so well either
<gary_poster> danilos, do you happen to have any wisdom for me?  I think you use pipelines
<jml> easy documentation branch up for review: https://code.launchpad.net/~jml/launchpad/silence-sphinx-errors-and-warnings/+merge/49965
<danilos> gary_poster, hum, you can always deal with branches individually, and perhaps even remove the branch from LP (if it has no MPs already) and then re-push (or maybe even just bzr push --overwrite)
<gary_poster> danilos, so just bzr push branch 1, then bzr next and bzr push branch 2 to someplace explicitly?
 * gary_poster tries
<danilos> gary_poster, well, you should have separate branches in the repo as well, so you should be able to access them; not sure how it works if you are using a shared repo with working dirs included
<danilos> jml, I'm taking a look
<danilos> jml, what does double colon do?
<jml> danilos: signals that the indented bit up ahead is quoted text.
<jml> http://docutils.sourceforge.net/docs/ref/rst/restructuredtext.html#literal-blocks
<danilos> jml, ok, was it the intention to remove it on line 22?
<danilos> jml, thanks
<jml> danilos: yes, because that's a list.
<jml> let me double check the output
<jml> yeah, looks right.
<danilos> jml, r=me, thanks, looks much better like this :)
<jml> danilos: thanks.
<danilos> bigjools, btw, I am somehow guessing your removal of a tag from deryck is unintentional: https://bugs.edge.launchpad.net/launchpad/+bug/719903 :)
<bac> hi mrevell
<jml> build is broken.
<mrevell> hi bac
<bac> hi leonardr
<leonardr> hi bac
<bac> leonardr: regarding my merge proposal from yesterday, the whole point was simply to add the 3rd party stuff and update the tools.  i should've been more explicit, but after 'make jsbuild' you'll see the gallery-accordion is included in launchpad.js and the accordion CSS is included in combo.css
<leonardr> bac: ok, r=me
<bac> thanks leonardr
<abentley> deryck, sorry I'm late.  Did I miss the whole meeting?
<deryck> abentley, yeah.  it was only adeuring and I, since henninge_ is also away momentarily.
<abentley> ah, okay.  Well, we've got our chat in an hour, right?
<deryck> abentley, sorry, crashed.  if I missed anything else you said.
<abentley> deryck, no worries.  I just said "Well, we've got our chat in an hour, right?"
<deryck> abentley, yes.  we can catch up then.  Thanks!
<bigjools> danilos: crossing edits.... we *really* need to fix that :/
<wallyworld> gary_poster: hey, can you recommend anyone in your squad who i could persuade to be the next release manager? :-)
<jml> om nom nom
<gary_poster> wallyworld, thanks for reminding me.  I was going to actually poll them, after reading your well-penned email of the past 12 hours.  Let me do so...
<wallyworld> gary_poster: thank you. i lied about being able to ride a unicycle
<wallyworld> gary_poster: i've asked a few people (not from your squad) already but haven't had any luck so far
<bigjools> wallyworld: I can ride a unicycle
<wallyworld> bigjools: i don't believe you - prove it. go on, i double dare you
<bigjools> heh, shall I sent you a picture?  I need to take it first of course.
<wallyworld> bigjools: you mean you'll need to google it and the fire up gimp
<gary_poster> wallyworld, my world is ruined by your lack of unicycle-riding!
<wallyworld> gary_poster: is there anything else i can do naked to make up for it?
<bigjools> wallyworld: I don't need to use gimp of course, google will find everything you ask for :)
<gary_poster> I've heard of bigjools' vaunted unicycle riding prowess for years now, but it actually has not been proved in my presence, I now realize
<bigjools> heh
<gary_poster> wallyworld lol no really that's fine :-)
<wallyworld> hmmm. but he does know how to wield a gun. i've seen it with my own eyes
<bigjools> wallyworld: are you threatening to go naked?  I'm sure that'll get a new RM pretty quick actually
<wallyworld> bigjools: well, i was going to attach a photo but the super macro lens on the camera broke so you couldn't see anything anyway
<bigjools> ouch :)
<gary_poster> wallyworld, benji has gracefully acquiesced to being volunteered.  I don't think he's hoping for any of the naked activities in recompense, but you can clarify that with him directly.
<benji> to be perfectly clear: "naked activities" void the offer
<gary_poster> lol
<wallyworld> benji: i've never said this to another man when i haven't been drunk, but i love you :-)
<benji> heh
<benji> I have that effect on people.
 * gary_poster enjoys laughing
<bigjools> get a room guys
<benji> :)
<wallyworld> bigjools: it's room 216 at the Shady Pines if you're interested
<bigjools> I'll take my cameras out
<wallyworld> you have a macro lens then?
<bigjools> I think I'd better stop here before I get into trouble :)
<bigjools> sinzui: is it possible to turn a project into a project group?
<sinzui> No
<bigjools> thanks
<henninge> deryck: back ;-)
<deryck> hi henninge.  wb! :-)
<leonardr> danilos, please review https://code.launchpad.net/~leonardr/lazr.restful/promote-write-operation-to-mutator/+merge/49976
<leonardr> i'm here if you have questions
<allenap> Is the removal of colour on the "Report a bug", "Ask a question", etc. links intentional?
<allenap> The "Get Involved" links.
<bigjools> and bug title
<bigjools> as someone said the other day, did someone die? :)
<gary_poster> flacoste, putting benji as release manager helps out the team, but hurts the squad's feature development.  Should the RM be on a bug rotation squad, or should the decision ignore current rotations?
<bigjools> gary_poster: I had that very discussion with my team this morning
<flacoste> gary_poster: i say ignore rotations
<gary_poster> ok thanks flacoste
<flacoste> i don't think it should come from bug rotation as a rule
<jml> allenap: yes.
<allenap> jml: I was quite fond of the colour :-/
<sinzui> allenap: yes
<jml> allenap: of which one exactly?
<sinzui> allenap: but there is an open discussion about whether to use an official Lp colour for these special links
<sinzui> we do not have any official colours
<jml> allenap: in any case, it's a work-in-progress. there was some discussion about this yesterday on either this channel or #launchpad
<jml> allenap: I'll ask huwshimi to send an email about it all to the list.
<sinzui> allenap: before we return to colour, we want to get the font-sizes corrected. I have a work-in-progress
* abentley changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | firefighting: - | On call reviewer: danilos, abentley | https://code.launchpad.net/launchpad-project/+activereviews
<allenap> jml: I was fond of all of them. I just wondered if it was an accident really. If someone has a plan then I don't really want to bikeshed, tempting though it is.
<allenap> sinzui: Interesting.
<jml> also, the build has been broken for about an hour now.
<adeuring> henninge: could you help me with qa for bug 702468?
<_mup_> Bug #702468: First upstream translation does not replace Ubuntu-only translation <qa-needstesting> <upstream-translations-sharing> <Launchpad itself:Fix Committed by adeuring> < https://launchpad.net/bugs/702468 >
<henninge> adeuring: I am currently doing something similar for my fix, so I don't think you need to do anything. It requires l o s a help for imports and such.
<henninge> I will cover that in my QA.
<adeuring> henninge: cool, thanks!
<danilos> leonardr, back from lunch, taking a look now
<leonardr> tx
<gary_poster> danilos and abentley, https://code.launchpad.net/~gary/launchpad/bug164196-1/+merge/49973 https://code.launchpad.net/~gary/launchpad/bug164196-2/+merge/49977 and https://code.launchpad.net/~gary/launchpad/bug164196-3/+merge/49980 are available for review.
<gary_poster> abentley, thank you for your pipeline help.  I had used reconfigure so I ran into the problem you described with sync.  For now I just pushed individually.  Next time hopefully I'll start without reconfigure
<gary_poster> Then I can wee what the expected behavior is :-)
<abentley> gary_poster, you're welcome.
<gary_poster> heh, see
<danilos> leonardr, I assume there are tests for mutators working correctly otherwise? (since your test only "proves" that it stops providing a method)
<leonardr> danilos: yes, there are. a mutator is just a write operation that doesn't show up in lookups
<leonardr> i could add a test that the mutator works in 3.0
<danilos> leonardr, right, thanks
<danilos> leonardr, if there are already tests for that elsewhere, no need to imho
<danilos> leonardr, just one more question: is it still possible to keep a method and make it a mutator? (i.e. have set_value keep working in 3.0)
<leonardr> danilos: no, because there's no point in having a named operation that duplicates the effects of a mutator
<leonardr> the situation we have here is where we made a mistake--this should have been a mutator method all along
<leonardr> but we can't change an old version for backwards compatibility reasons
<danilos> leonardr, well, I can imagine someone wanting to keep both so you don't have to port client code, no?
<danilos> leonardr, maybe that'd be hidden by things like launchpadlib instead, so I am just wondering
<leonardr> danilos: this falls into the category of 'if you don't want to port code, don't use the new version'
<leonardr> it's an easy change to make
<danilos> leonardr, ok, as long as it's recognized as being a use-case (or not), I think it's fine; some explicit documentation might be useful, if you agree
<danilos> leonardr, anyway, with that considered, r=me
<leonardr> thanks
<danilos> leonardr, thanks for the fix! :)
<danilos> leonardr, oh, are you also getting lazr.restful updated for LP as well?
<danilos> abentley, have you started on any of Gary's branches?
<abentley> danilos, no, was on a call.
<leonardr> danilos: that's next. i'm going to do a release and thumper will incorporate the new version with his branch that needs the fix
<danilos> abentley, ok, I am starting with the first one, and I'll make sure to claim the one that I am working on
<danilos> leonardr, excellent, sounds good
<abentley> danilos, cool.
<danilos> gary_poster, re -1, in lib/lp/bugs/model/bug.py you modify addCommentNotification to add activity parameter, but you don't modify addChange which I believe does a similar thing for non-comment changes â is that intentional or not?
<gary_poster> danilos, intentional.  addChange uses change objects, which handle activities themselves.
<danilos> gary_poster, ok, cool
<danilos> gary_poster, r=me on the non-DB stuff, and I am guessing stub will give you patch number 45 :)
<gary_poster> danilos, cool, thank you :-)
<abentley> gary_poster, re: your other branch, it seems like you're changing the meaning of date_emailed to be "date_processed".  Did you consider changing that in your patch?
<abentley> gary_poster, did you consider making is_omitted a "status" enumeration instead?  Values could include PENDING, SENT, SKIPPED.
<danilos> abentley, whoops, I just claimed that one as well, perhaps gary_poster will have to request another review from you
<gary_poster> abentley, date_emailed: you are quite right.  I didn't consider it for very long, because I didn't think the win merited the database disruption.  In the abstract, I'd agree 100%, and if stub said that I shouldn't worry about the disruption, I'd be fine with it.
<abentley> gary_poster, I see.
<abentley> danilos, hmm.  You shouldn't have been able to claim the review after I did.  Stale page?
<danilos> abentley, perhaps
<gary_poster> danilos, abentley, atually things are fine.  danilos is on -2 and abentley is on -3
<danilos> gary_poster, ah, ok :)
<abentley> gary_poster, danilos: confirmed.
<gary_poster> abentley, enumeration: I didn't, and I suppose this is a "in for a penny, in for a pound" sort of thing.  By that I mean that I questioned the wisdom of including the flag now at all, without proof of its necessity, but I leaned towards wanting it anyway.  An enum would be cleaner, and I'd be fine with that if you wanted to request it.
<abentley> gary_poster, since we're creating a field either way, I recommend doing a status enum.
<gary_poster> abentley, cool, +1
<abentley> gary_poster, in the doctest, why are you reversing the order of the notifications?
<gary_poster> abentley, so that you see them in the order they happened in the doc test
<gary_poster> we get the last eiht
<gary_poster> eight
<gary_poster> but then they are in reverse chronological order
<abentley> gary_poster, ah.  Missed the meaning of "-" in "-id".
<abentley> gary_poster, I think ResultSets support slicing.  If so, getting a ResultSet might be clearer.
<jtv> henninge: we've been seeing some problems with the branch approver not approving templates that it can't figure out a decent name for.  How about we let the domain default to the project name?  (The approver will back off when it sees duplicate names anyway)
<henninge> jtv: sounds reasonable
<henninge> jtv: glad you found the reason for those non-approvals.
<gary_poster> abentley, ok, I agree in principal, and will see if I can figure out what needs to be done.
<jtv> henninge: I thought this case was covered somehow, but AFAICT it was simply something we hadn't gotten to yet.
<abentley> gary_poster, cool.  I don't feel strongly about these changes, so r=me.
<gary_poster> ok, thank you very much abentley
<danilos> gary_poster, fwiw, the variable name "person_causing_change" was chosen as the most descriptive one, after my reviewer (bac, I think) had trouble following code, so perhaps "actor" is not the best choice :)
<gary_poster> danilos :-P :-) ok, I can revert that part
<gary_poster> it just helped with line length
<danilos> gary_poster, not necessarily if you think you'd find it obvious who 'actor' is when you first look at the code
<bac> gary_poster, danilos: i think it changed from "person" so "actor" is still an improvement.
<danilos> bac, true
<danilos> gary_poster, bac, it's just hard for me to say if it's clear enough with so much background already
<danilos> gary_poster, so, I'd say it's up to you
<jam> vila, wgrant: From what I've been able to piece together, hitting the FD limit causes us to half-create connections, which then leak handles for future connections
<gary_poster> ok, cool, bac, danilos.  I'll see if I can fit the longer version in nicely, and if it is annoying, I'll leave it with "actor".  Thank you
<jam> so hitting the limit is both a symptom and a cause
<benji> leonardr: I sent a general email to the -dev list, but I want to bring https://dev.launchpad.net/yellow/JavaScriptAPIAccessDraft to your attention specifically in case you have things to add.
<vila> jam: but you shouldn't even approach the FD limit or you can't scale right ?
<leonardr> benji, will take a look
<danilos> gary_poster, now, a more relevant question: is there no better way to figure out if a change is of a certain type than process activity.whatchanged? it kind of looks fragile to me
<jam> vila: well, what I don't understand is why the existing code doesn't have the same problem. However, we can't serve more than 200 concurrent connections from a single process anyway.
<vila> jam: so I thought you were approaching it *because* you had leaks
<jam> It looks like this might be a little bit less
<leonardr> benji: it occurs to me that we might be duplicating some work. do you know about blue squad's project to make things more widget- and event-oriented?
<jam> vila: nope, just more connections than we thought
<jam> somebody mentioned that there might be *5* handles per child
<jam> which would only give us 200 max connections, instead of 250
<jam> and the issue is that if we hit 200 for a second, then we leak a couple of handles
<jam> so that then our max becomes 180
<jam> then 150, then ...
<jam> etc
<vila> what's the point of forking if you don't keep the fd local to the forked process but not in the parent ?
<jam> vila: wrong fork
<jam> there are 3 processes involved
<jam> 1 is the master Conch twisted ssh process
<benji> leonardr: nope, I wasn't aware.  I was mostly writing down how things are, but I didn't know that a project to improve it was ongoing.
<jam> which handles taking SSH encrypted data over the socket
<jam> and turns it into stdin/stdout for a child process
<jam> this has always been a single process
<jam> though it used to "exec" a new bzr child
<vila> O_o
<jam> (subprocess.Popen)
<jam> what seems to have changed
<jam> sec
<jam> what *I* changed
<jam> is that instead of "subprocess.Popen()"
<jam> now connect to this existing service
<jam> and ask it to fork
<jam> so that it already has all the state ready for the response
<jam> What may have also changed
<jam> is that when we hit the fd limit
<jam> we don't fail gracefully
<jam> and that we may have more handles per-child now
<gary_poster> danilos, gmb gave acivities to me as the best way around, if I linked the notifications and activities as I did in the first branch.  whatchanged is the raw source for the (new, computed) attributes "attribute" and "target".  Using them would be more apt, perhaps, but arguing that it was more robust would be...doable but not super strong, IMO.  The argument would have to be that target and attribute were actually
<gary_poster> what we cared about, and by pointing at those two, if there were changes, we'd at least be pointing at the semantically correct values so maybe they would have a higher chance of being robust in the future.  Actually...
<jam> though we shouldn't.
<leonardr> benji: ok, i'll look at your document, our system will build on top of what you're describing rather than replacing it
<jam> If we do, then it is probably a bug for me.
<jam> vila: And the way we will scale up is by putting more Conch SSH process behind an HAProxy load balancer
<jam> which we want to do anyway
<gary_poster> ...danilos, what I could do is move the normalization of duplicateof into the computed attributes
<gary_poster> that would be more correct
<gary_poster> and then I could rely on target and attribute
<gary_poster> rather than whatchanged
<gary_poster> which are really one and the same, but, eh, maybe it is better :-)
<benji> leonardr: is what you're working on similar to the PATCHPlugin?  I.e., it integrates with the YUI widget machinery to let you do REST calls as the result of widget/form operations?
<danilos> gary_poster, it's just that it kind of seems wrong to me to process data that we have produced based on data we still have (if we still have it)
<danilos> gary_poster, also, get_key would definitely benefit from having a docstring about what is it really doing :)
<leonardr> benji: yes, it uses the PATCHPlugin
<vila> jam: oh
<gary_poster> danilos, not sure I follow "process data that we have produced based on data we still have" in context.  If I do understand you, which I might not, then we do not have that data anywhere other than in these activity objects.
<leonardr> it will also let you set up a hook function to be run when a certain field of the context changes
<gary_poster> danilos: get_key docstring, yeah, I saw that when I was writing the MP, but I was suffering from fatigue ;-)
<gary_poster> I'll add one
<benji> leonardr: is your work purely for in-place editing or it it also for forms with ok/cancel buttons
<danilos> gary_poster, right, my question is basically, do we have it in a form that's more suitable for processing (i.e. as target/attribute)? if not, whatchanged is what we'll have to live with
<danilos> gary_poster, and fwiw, I wouldn't mind a unit test or two for get_key either
<gary_poster> danilos, target/attribute is the closest we can get (as far as I've been told and as far as I've seen myself), and those are computed attributes based off of whatchanged
<gary_poster> so I think asking me to use target/attribute is reasonable, a good idea, and yet also kind of minor
<danilos> gary_poster, right, then I guess it's ok as-is
<gary_poster> ok
<danilos> gary_poster, heh, if it is minor, please do, if not, no worries
<gary_poster> unit test for get_key, ok :-)
<gary_poster> danilos, I meant minor as in not that important, but it probably won't be too bad.  I can give it a whirl.
<gary_poster> rephrase, I meant minor as in not that important, but it probably will be relatively easy.
<danilos> gary_poster, oh, my only worry is that "whatchanged" will be changed in the future, and our code might be silently broken
<danilos> gary_poster, so, if we make sure we have parsing in only one place, that reduces the chances of that happening
<gary_poster> danilos, I worried about that a bit, which is why I went a bit overboard in the tests for this.  I could go even farther "overboard" and considered it
<jam> vila: https://bugs.launchpad.net/launchpad/+bug/717345/comments/14
<_mup_> Bug #717345: Updates to SFTP server leak file handles in use_forking_server mode <Launchpad itself:In Progress by jameinel> < https://launchpad.net/bugs/717345 >
<jam> see the last paragraph
<gary_poster> and would be willing to, as I mentioned in the cover letter
<gary_poster> danilos, ok so summarizing so we're clear, here's what I'll do
<gary_poster> 1) add docstring and unittest for get_key
<leonardr> benji: for ok/cancel buttons you'll add a custom widget and hook it into this system
<benji> interesting
<jcsackett> deryck: do you have any idea why diff overlays on lp.dev don't size remotely the same as diff overlays on lp.net? display appears to be YUI related, which is why i'm coming to you.
<gary_poster> 2) move duplicateof code to target computed attribute code, and add test
<gary_poster> 3) change get_key to use target/attribute
<gary_poster> danilos, yeah?
<danilos> gary_poster, yep, sounds good
<gary_poster> cool, on it
<danilos> gary_poster, great, thanks
<gary_poster> thank you
<jtv> henninge: can you think of any problems with my approach?  po/messages.pot for project Foo will have domain Foo, name foo.
<deryck> jcsackett, no idea off the top of my head
<jcsackett> well darn. :-P
<benji> leonardr: do you know when your work will be usable by others?
<deryck> jcsackett, I'm grabbing something to eat before the tl call next hour, but I can help you look it over after all that
<deryck> different CSS linked in devmode maybe?
 * deryck goes to eat now
<leonardr> benji: the event-based stuff is still in the theoretical stage
<benji> -
<jcsackett> deryck: enjoy, and thanks. :-P
<leonardr> for the widget stuff, there's no reason why you can't do it now--you just have to decide to buckle down and consolidate your javascript code into a widget
<henninge> jtv: You will need some name mangling
<henninge> jtv: project "Foo Bar", domain "Foo_Bar", name "foo-bar"
<jtv> henninge: selbstverstÃ¤ndlich :)_
<henninge> jtv: oh, that was the project name, like in the URL, right?
<henninge> so that's quite useful already
<jtv> Yes, but even so, I take it the mangling is already there.
<henninge> hm
<jtv> Because a filename can contain all the weird stuff that project names can.
<henninge> right
<jtv> If not, well, then I'll add it and we'll have fixed another subtle bug.  :-)
<jcsackett> sinzui: i have just discovered there will be maintenance folks in my apartment all afternoon, which makes mumbling hard. move to sometime soonish or to tomorrow morning?
<sinzui> jcsackett: can talk in 1.5 hours or after. Any time really. I wanted to talk to you anyway about the UI for seeing feature flag changes
<jcsackett> sinzui: so like 1:30? should work for me.
<lifeless> moin
<sinzui> jcsackett: okay
<danilos> gary_poster, I've approved your branch, but if you want, I'd be happy to take a look at incremental diff (as I said in a comment)
<danilos> abentley, thanks for the review, btw :)
<gary_poster> danilos, awesome, thanks.  how much longer are you around today?
<abentley> danilos, np.
<danilos> gary_poster, probably another 30 mins or so
<gary_poster> danilos, ok, I won't make it by then.  I'll decide whether to wait for you once I'm done with the revision.  I think you'll be happy with the changes.
<danilos> gary_poster, yeah, I am confident I will be, but I might stick around for a full hour, so try me :)
<gary_poster> danilos :-) ok cool
<danilos> gary_poster, oh, one more thing, can you please check if the test runner really picks up a unittest.TestCase tests (eg. a TestGetEmailNotifications) from a file with no explicit loadTestsFromName() call (I vaguely remember that not happening in the past, but I am not too sure about it)
<gary_poster> sure danilos
<jtv> good night folks!
<danilos> gary_poster, actually, I've tried it out myself, and it does :)
<gary_poster> ok cool :-)
<danilos> jtv, good night
<gary_poster> thank you
<jtv> zzz
<leonardr> benji: the only feature i know that collection objects have is 1) the 'entries' attribute which has the default page
<leonardr> and 2)  the 'lp_slice' method, which lets you get stuff off of the default page
* danilos changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | firefighting: - | On call reviewer: abentley | https://code.launchpad.net/launchpad-project/+activereviews
<benji> leonardr: thanks, I'll add that
<leonardr> but i suspect nobody uses collections, judging from what i've seen
<gary_poster> leonardr that doesn't mean we shouldn't
<benji> yeah, I share your suspicion, that's probably why I couldn't find any examples to learn from
<jml> although using lp.testing.TestCase is better than unittest.TestCase for a whole heap of reasons.
<leonardr> benji: look at lib/lp/code/browser/sourcepackagerecipe.py for an example that uses the InlineEditorPicketWidget
<leonardr> the widget itself is tested in lib/lp/app/doc/lazr-js-widgets.txt
<leonardr> that's the direction we'd like to move in. it would make the javascript much more comprehensible
<jml> lifeless: is edge getting less traffic now?
<henninge> danilos, jtv-afk: Did I hear you complain about queue approval timing out a lot ?
<danilos> henninge, it was mostly jtv, the last approvals I did a week or so ago did not time out for me
<henninge> danilos: I cannot approve a template import on qastaging, it times out in ... POFile.updateStatistics!
<henninge> which is in _copyPOFilesFromSharingTemplates
<danilos> henninge, that's very interesting, if that's timing out, I would expect different filters on +translate page to time out
<danilos> henninge, oh
<henninge> danilos: do we have a way to exempt a page from the timeout?
<danilos> henninge, you can up a timeout per page, I think POFile:+translate already has an exception
<henninge> how?
<danilos> henninge, look at https://launchpad.net/+feature-rules
<henninge> cheers
<danilos> henninge, you might need an admin to modify the value though
<lifeless> jml: per appserver yes
<lifeless> jml: less than it was before? not entirely sure.
<lifeless> flacoste: got a minute? I want to talk capacity ...
<jml> lifeless: ok.
<flacoste> lifeless: i've got 10 minutes
<jml> lifeless: was curious since we are still getting edge crashes.
<lifeless> jml: thats because we're still using edge
<lifeless> jml: we can't get rid of it transparently because of some bugs in launchpadlib/wadl
<bigjools> good night all
<jml> lifeless: well, yes. Was asking to test a theory. If the crashes are induced by high load and we've got less traffic then I would expect fewer crashes.
<jml> lifeless: although I didn't know that we can't get rid of edge transparently
<jml> that is a pain.
<lifeless> jml: indeed
<lifeless> jml: you can run stats and normalise by appserver
<jml> lifeless: my curiosity is far more idle than that :)
<lifeless> some appservers have crazy configs though; two in particular are configured for 32 threads (they were test instances from the scaling tests, not meant for prod deployment as-is)
<jml> huh
<lifeless> I have a branch that fixes this as part of the single threaded appserver RT
 * deryck goes to finish lunch
 * jml goes to get food
<lifeless> allenap: btw thats a dupe
<lifeless> allenap: I think if you search for person picker you'll find the original
<jcsackett> sinzui: mumble? i have had to relocate but think i have found a quietish spot.
<jkakar> Out of curiosity, is it possible to change the name of project or project group?  I mean the name used in URLs for bugs, branches, etc.
<jcsackett> or given i have already had to move, we can do the usual time if you would prefer.
<sinzui> jcsackett: get comfortable, then ping me
<abentley> jcsackett, why does your bugs-in-deactivated-pillars branch request reviews from both Robert and Launchpad code reviewers?
<dobey> jkakar: i think you have to ping a l-osa to change it
<sinzui> jkakar: a ~registry member can change the launchpad-id in the url We want to let the owners do it bug we have not solved the case about know when the URL is not permanent
<sinzui> almost everyone in this channel can change a project name
<jcsackett> abentley: lifeless was helping me out with some performance issue, i thought i would solicit comment from him directly, but leave it open to usual review.
<jkakar> Cool, thanks.  Was mostly curious if it was possible/something we did for folks.
<sinzui> jkakar: we can also provide an alias so that old urls do work eg gaim->pidgin
<jkakar> sinzui: Ooh, very nice. :)
<abentley> jcsackett, ah, cool.  I thought maybe you only wanted a review from Robert, but had requested a second review instead of changing the existing one.
<jcsackett> abentley: a very reasonable guess.
<jcsackett> sinzui: ready whenever, assuming mumble is cooperating.
<abentley> jcsackett, you're changing model code, but you're testing browser code.  That makes this an integration test, so you still need a unit test IMO.
<abentley> jcsackett, I recommend using person_logged_in instead of login/logout, which is already importd.
<abentley> s/importd/imported.
<abentley> jcsackett, I don't understand why you're using both.
<jcsackett> abentley: i can agree on the test, and i'll restructure the login/logout bit.
<jcsackett> laziness, is the answer, regarding using both. :-P
<abentley> jcsackett, you can just change it to a unit test.  I don't think you need an integration test
<lifeless> jcsackett: looking now
<lifeless> jcsackett: you've done something different to https://bugs.launchpad.net/launchpad/+bug/421901/comments/14
<_mup_> Bug #421901: Person:+bugs timeouts <lp-bugs> <timeout> <Launchpad itself:Triaged> < https://launchpad.net/bugs/421901 >
<henninge> abentley: Your diverged fix looks good, I just QA'd it.
<abentley> henninge: cool, thanks.
<henninge> lifeless: wasn't there some easy way to generate the list of bugs for the Deployments table on LPS?
<lifeless> henninge: there is a bug asking for that
<henninge> oh, ok.
<henninge> I seem to live in the future, then. ;)
<jcsackett> lifeless, abentley: i'll respond shortly. otp.
<henninge> deryck: QA done and I requested the deployment.
<henninge> I will go home now
<deryck> henninge, awesome.  many thanks!
<deryck> henninge, all the revs before you are qa-ok?
<henninge> deryck: yes, they were Abel's and Aaron's, I QA'ed them, too.
<deryck> excellent.  thanks!
<deryck> henninge, enjoy your night.  Sorry you had such a long day.
<henninge> it dragged on a bit ...
<jml> is there a way to specify a build-dep w/ buildout?
<jml> i.e. not install_requires
<lifeless> jml: I don't think it discriminates between build and run, does it ?
<jml> lifeless: that's my question.
<lifeless> heh
<jcsackett> lifeless: read your comment. the reason for the different join is nothing better than distraction. i'm pushing up changes now--we def want the active flag in the join.
<jcsackett> abentley: i'm also creating the unit test. i will push that up shortly.
<jml> Error: Picked: Sphinx = 1.0.7 :(
<benji> jml: it looks like you need to nail the version by adding "Sphinx = 1.0.7" to versions.cfg
<jml> benji: thanks.
<jml> benji: I'm thinking now that maybe I should just make the test skip if sphinx isn't available.
<allenap> lifeless: Yeah, someone else has marked it as a dupe now. It struck me that it was probably a dupe, but the dupe finder had no luck, and, as it turns out, the master bug's title doesn't really give enough away to help.
<lifeless> allenap: well it should be easier to find now
<benji> Sphinx isn't an onerous dependency, I wouldn't worry about making in required
<benji> and it would be good if there is no barrier to an existing LP developer contributing to the docs
<jml> benji: good point.
<LPCIBot> Project devel build (447): STILL FAILING in 6 hr 19 min: https://hudson.wedontsleep.org/job/devel/447/
<flacoste> jml, lifeless: http://people.canonical.com/~flacoste/lp-bugs-chart/
<flacoste> deryck: an example of YUI charts ^^^
<deryck> hurrah!
<deryck> you lucky link-of-to-yahoo-cdn hacker you!
<lifeless> http://webnumbr.com/launchpad-timeout-bugs#
<lifeless> http://webnumbr.com/launchpad-oops-bugs.all
<lifeless> http://webnumbr.com/launchpad-critical-bugs
<flacoste> i prefer my stacked graphs
<flacoste> :-)
<jam> bzr st
<jam> mt
<benji> flacoste: are the fractional bugs in the first chart the result of bugs that take longer than a week to fix?
<flacoste> benji: no, they are a workaround for http://yuilibrary.com/projects/yui3/ticket/2529972
<benji> ah
<flacoste> i replace 0 with 0.1 to prevent the graph from being screwed up
<benji> too bad non-zero infinitesimals don't exist
<leonardr> thumper, i need to leave in a few minutes--can we do a quick mumble?
<thumper> leonardr: sure
<thumper> StevenK: we can't hear you
<StevenK> thumper: Not trying to talk yet
<deryck> Later on, everyone.
<wallyworld___> thumper: https://code.launchpad.net/~wallyworld/launchpad/recipe-build-now/+merge/49968
<wallyworld___> go /nick wallyworld
<allenap> Does anyone know a neat way to capture log output in a test? Like a context manager or a fixture?
<jcsackett> self._
<jml> allenap: addDetail(), there's an example in lp.testing.TestCase, iirc.
<jcsackett> ...damn clipboard...
<allenap> jml: Sorry, I meant how to capture log output by, for example, calls to log.debug()?
<jml> allenap: oh, you add one of the mock handlers
<allenap> jml: Ah, yes, like FakeLogger.
<jml> allenap: yeah.
<allenap> Thank you :)
<poolie> hi jml?
<jml> poolie: hello
<poolie> hey there
<poolie> sorry it's so late, i had to do some chores here first
<jml> that's ok
<wgrant> jam: That's what I found, yes.
<wgrant> jam: So once we hit the FD limit, analysis is useless.
<wgrant> jam: We need to watch it before it hits :/
<jam> wgrant: well, I can reproduce here, and I think I have a handle on how to clean up
<jam> so that it isn't useless
<jam> I've got 3 patches so far, finishing up the 4th
<wgrant> I saw it mostly leaking sockets to the forking master.
<wgrant> With only a few fifo handles staying around.
<poolie> jml, shall we talk on the phone?
<poolie> hi jam, wgrant
<jml> poolie: sure, although skype or mumble are to be preferred
<poolie> let's try mumble
<jml> ok
<wgrant> jam: I was able to stably run hammer_ssh against dev with 120 concurrent connections. But FDs occasionally spiked to over 900, because there were more than 200 ssh processes. Any idea what was going on there?
<wgrant> 150 was too much, but possibly only because hammer_ssh doesn't actually limit properly.
<wgrant> jam: Do you think we need to retain the master socket and stderr?
<jam> wgrant: you mean in hammer? or in the Conch service?
<jam> we definitely do in the Conch
<wgrant> jam: In the Conch service.
<jam> because it is used
<wgrant> Oh :(
<wgrant> We can't just wait for our sockets to close to detect exiting?
<jam> as for hammer not rate limiting... it spawns another connection once it closes the handles to the spawned ssh
<jam> we could have it wait to spawn until the ssh actually exits
<jam> just change the line that sasy "if len(pool) < 200" to "if len(pool) + len(stopping) < ..."
<wgrant> Aha.
<jam> wgrant: fifo's don't exit like that
<jam> unfortunately
<wgrant> Bah.
<jam> which is why we hold open the socket
<jam> because it *does*
<jam> roughly
<jam> that socket is actually connected to the master, which will inform when the child is finally closed
<wgrant> Right.
<wgrant> jam: Did you obtain the process listings that I discovered the existence of yesterday?
<jam> wgrant: yeah, that is patch 1,2 and I think 3
<wgrant> Ah, I see you followed the logs.
<jam> wgrant: Conch asks for a new child, starts connecting but can't finish
<jam> then the child sits around waiting forever
<jam> https://code.launchpad.net/~jameinel/launchpad/lp-serve-child-hangup/+merge/50055
<jam> tells the child to hang up after 2 minutes
<wgrant> jam: Oh yeah, I worked that out quickly enough.
<wgrant> jam: I also looked into the log rotation issue late last night.
<wgrant> It strongly suggested that we reached exhaustion at 10:29, not 11:01.
<wgrant> Yet the first bad process is from 11:01.
<jam> wgrant: certainly, at least the first one
<wgrant> jam: But 117*5 is only 600ish. The base is ~10. So we have 400 handles unaccounted for.
<wgrant> Unless there were connections staying around with fifos open after the forking service had died?
<jam> wgrant: we are missing the log that might tell us about other bad children.
<jam> If the server cannot process a request but the child doesn't hang around.
<wgrant> Right.
<wgrant> I wonder.
<wgrant> Once we have it cleaning up, we may also want to bump the FD limit.
<wgrant> Argh.
<wgrant> Except that I tried that, and then select crashed instead.
<jam> wgrant: interesting. I've certainly been reproducing it by *lowering* the FD limit
<jam> at 100, I can hammer with 15, but not with 20
<jam> ulimit -n 100, hammer --load=20 starts failing, hammer --load=10 passes just fine
<wgrant> jam: Well, I was hoping we could get it to stop leaking on failure, then quadruple the failure point so it's safe to run on prod.
<wgrant> So we can watch and see if there are any other leaks.
<mwhudson> wgrant: probably need to use the poll reactor in that case
<wgrant> Probably.
<mwhudson> probably should anyway actually
<jam> mwhudson: you mean select.poll vs select.select?
<jam> or however that translates into Twisted objects
<mwhudson> jam: yes, twisted has a few reactors
<mwhudson> the default is select
<mwhudson> but we'd probably want to use the poll or epoll reactors
<mwhudson> and have to, if we want to support > 1024 (?) fds
<mwhudson> or you can recompile python with MAX_FD lifted or whatever it is :)
<poolie>    robert tries to correct deryck on his understanding of
<poolie>    iterations, but fails miserably because deryck is right (or at
<poolie>    least taking notes and has the last word, muuuuhahhhh!) :-)
<poolie> :)
<mwhudson> yeah, that made me laugh too :)
<sinzui> wgrant: mumble?
<wgrant> jam: Which leaks haven't yet been eliminated?
<poolie> jam, couldn't we pass the fd across the unix socket?
<jam> poolie: still takes up a fd
<poolie> this seems to be getting complicated
<wgrant> I think we need to bust the 1024 FD limit.
<wgrant> Which means increasing the ulimit and moving to the new reactor.
<jam> wgrant: or just getting the haproxy stuff working and using 2 services
<wgrant> True.
<poolie> well, using poll or similar is probably a good idea anyhow
<poolie> jam, so it seems like allowing there to be a state where the child bzr process is started, but it's not connected to conch is just giving an opportunity for trouble
<poolie> so i'm suggesting using fd passing as a way to eliminate that, rather than to reduce the number of fds used
<jam> poolie: you can still run out of fds while transferring them over the socket.
<jam> I also don't know how to do the socket transfer stuff, though I can learn
<huwshimi> sinzui: Let me know when you're available to talk about that stuff
<jam> so you still have the possibility of the child being open because it couldn't transfer the handles
<poolie> gary_poster, is there any reason your apidoc couldn't move to people.c.c and therefore be public?
<mwhudson> poolie: i think it documents shipit
<mwhudson> or something lame like that
<wgrant> But the shipit code is basically public anyway :(
<sinzui> huwshimi: I expect 2.5 hours. My daughter are providing an itinerary for the next few hours
<gary_poster> hi poolie.  yeah, it probably does, though that coud be adjusted, if it still worked. :-/
<huwshimi> sinzui: Sure, no problems :)
<poolie> jam, i would expect you would set up the fds before forking
<poolie> therefore not start the child at all if something goes wrong
<poolie> that's why i say we could avoid the issue of things being half-
<poolie> started
<jam> poolie: interesting idea, but quite a bit more complex to change the current code. I'm sure it can be done, but it changes a lot of the ordering
<poolie> gary_poster, maybe we could even just have two copies, one with shipit and one without
<poolie> but as you say people can run it up themselves
<poolie> jam, i wonder if it would be worth it if it's safer
<poolie> i don't think of using one thread in python to stop another as being a very safe technique
<jam> poolie: works quite well in my testing, especially since the one is only blocked on an os.open
<poolie> the first iteration worked well in your testing too ;-)
<jam> poolie: true, though I'm also testing under these conditions
<jam> with running out of file handles, and seeing the cleanup
<jam> I'm willing to iterate on this a bit
<jam> but it is my EOD right now
<poolie> sure, i'm just saying i think it's inherently safer to avoid this stage, all else being equal
<poolie> i've used fd passing before
<lifeless> jam: poolie: we've 3 weeks to get the next iteration prepped, by default
<jam> poolie: then perhaps you'd like to write a patch?
<poolie> if i get a chance i will see if it's available in python and how it would fit in
<jam> poolie: the 3-4 patches I've put up today, I can see that I can force the service to run out of handles, it complains for a while but still serves what it can
<jam> at the end, everything gets cleaned up
<jam> and lsof shows it back at pristine state
<jam> poolie: I don't see 'sendmsg' in the socket documentation: http://docs.python.org/library/socket.html#module-socket
<jam> And "google python sendmsg recvmsg" is a bit lacking
<jam> http://bugs.python.org/issue1194378 seems to be an implementation
<jam> but looks to only be py 2.7/3.1
<jam> which turns out to be a dupe of: http://bugs.python.org/issue6560
<jam> which doesn't seem to have a resolution
<jam> just a patch
<jam> anyway, maybe I'll see you guys later tonight.
<poolie> jam, hm, we might need our own C glue, and perhaps at that point it's not worth it
<spiv> ctypes might do.
<spiv> I'm pretty sure I've heard of people doing this in Python before though.
<spiv> (but perhaps with C glue)
<spiv> There's http://pypi.python.org/pypi/sendmsg/
<poolie> as a c extension
 * poolie is looking into bug 720403
<_mup_> Bug #720403: bzr server error: "file exists" while taking lock <code-hosting> <Launchpad itself:Confirmed> < https://launchpad.net/bugs/720403 >
<poolie> hi spiv
<spiv> Right, but at least it's a C extension you don't have to write from scratch yourself :)
<wgrant> sinzui: Do you have any suggestions for reducing the DMP spam?
<lifeless> wgrant: dmp ?
<wgrant> I think I will stop logging errors as errors, and perhaps increase the check interval.
<wgrant> lifeless: distribution mirror prober
<sinzui> mirror prober
<lifeless> ah
<sinzui> wgrant: I do not.
<lifeless> are you talking about oopses from it, or jus tlogging ?
<wgrant> Just logging.
<wgrant> It logs some normal probe errors at warning/error
<wgrant> And sometimes overruns its window.
<lifeless> it should parallelise
<lifeless> its heavily driven by mirror speeds
<wgrant> It is parallised.
<wgrant> Although not completely.
<wallyworld> thumper: is there a bug for the "build now" work?
#launchpad-dev 2011-02-17
 * poolie thinks, again, there ought to be an incipient mp for every branch 
<poolie> whether it's been actually proposed or not
<poolie> spiv, the thing is writing the extension is easy, deploying it is the hard part
<maxb> poolie: but how would you know what the destination branch is?
<lifeless> maxb: all of them :P
<lifeless> poolie: why is deploying the extension hard?
<thumper> wallyworld: not really
<thumper> wallyworld: file one
<poolie> i'm just once-bitten by attempting to change the oauth deployment
<thumper> how does one document a flag?
<poolie> thumper, read dev.launchpad.net/FeatureFlags and if that's not enough, ask me
<thumper> ok
<poolie> if it's not enough i'd really like to make it better
<thumper> poolie: https://launchpad.net/+feature-info lists undocumented features
<thumper> poolie: how do they become "documented"?  The docs don't cover that
<poolie> hm
<poolie> ah, see "adding a new flag"
<poolie> i'll expand that a bit
<poolie> or are you just documenting an existing one?
<thumper> I'm wanting to add a new flag, but I should also document the missing ones
<wgrant> The missing ones are documented in a branch of mine.
<thumper> poolie: yeah, that section is very easy to miss
<thumper> wgrant: thanks
<thumper> poolie: who is able to edit the feature flags on prod?
<poolie> losas
<poolie> only
<poolie> well, ~admin
<poolie> i think that's correct
<poolie> is that better now?
<poolie> thumper, please try to match the naming guide in that page too
<poolie> or object to it and we can change it
<wgrant> But if you invent another non-standard scheme I will cry :)
<poolie> exactly, so please don't
<poolie> the naked juggling is distracting enough without people criying too
<thumper> StevenK: when do you want to chat?
<poolie> thumper, all good?
<thumper> poolie: the only thing making me cry right now is how we are currently hiding recipes
<wallyworld> thumper: can you remember where that code is where you check for None in the formatter?
<thumper> wallyworld: yes
<wallyworld> thumper: and please tell me where it is mr smarty
<thumper> wallyworld: lib/lp/code/browser/sourcepackagerecipe.py
<wallyworld> thanks
<thumper> archive_picker
 * wallyworld sad - there's actually no existing tests for NoneFormatter
<lifeless> StevenK: btw
<lifeless> StevenK: there is a jenkins openid plugin now
<lifeless> whoa
<lifeless> where did the loggerhead exceptions go
<wallyworld> StevenK: how's the recipe popup build request review going?
<StevenK> wallyworld: You mean, the one I wasn't actually doing? :-)
<wallyworld> StevenK: you asked me questions about it so i assumed you had claimed it :-)
<StevenK> wallyworld: Nope :-)
<wallyworld> StevenK: np. i'll find another sucker^H^H^H^H^H esteemed developer to do it
<lifeless> wallyworld: I'm a little confused
<lifeless> wallyworld: I thought you were working on manually requested builds
<wallyworld> lifeless: i always confused
<lifeless> wallyworld: but I saw you file a but about manually requested daily builds that aren't stale
<wallyworld> lifeless: yes, but the request build popup work was done before that
<lifeless> which is a /totally/ different use case
<wallyworld> so i have 2 recipe related branches up for review
<wallyworld> the 2nd depends on the first
<wallyworld> due to shared javascript
<wallyworld> lifeless: make sense now?
<wallyworld> wgrant: want a small review? https://code.launchpad.net/~wallyworld/launchpad/add-link-to-noneformatter/+merge/50088
 * wallyworld has 3 active reviews to try and get off his list
<LPCIBot> Yippie, build fixed!
<LPCIBot> Project devel build (448): FIXED in 5 hr 59 min: https://hudson.wedontsleep.org/job/devel/448/
<LPCIBot> * Launchpad Patch Queue Manager: [r=leonardr][bug=649252] add an unsubscribe link to non-verbose bug
<LPCIBot> notifications
<LPCIBot> * Launchpad Patch Queue Manager: [r=leonardr][no-qa] isolate bin/lint.sh's use of grep from the
<LPCIBot> environment
<LPCIBot> * Launchpad Patch Queue Manager: [r=danilo][ui=none][no-qa] Fix rST and Sphinx errors in our top-level
<LPCIBot> docs, and make the main page more interesting.
<lifeless> whats the glue to say 'does this class implement ISomething or other' - when querying, not declaring
<sinzui> hi huwshimi
<huwshimi> sinzui: Hey therwe
<huwshimi> *there
<sinzui> huwshimi: I recall writing about the loss of colour from the involvement portlet. but I cannot find it in my email :(
<sinzui> huwshimi: So pretending I have dot written about this. do you want to mumble to discuss the issues that users have raised?
<huwshimi> sinzui: Sure, that'd be great
<thumper> lifeless: what do you think about converting feature flag values of "off" and "false" to False ?
<lifeless> thumper: i'm pretty unkeen
<lifeless> thumper: what issue are you trying to solve?
<thumper> lifeless: I'm adding a feature flag "code.recipe.beta" to control the message and the beta icon
<thumper> lifeless: so we'll add a rule that says code.recipe.beta -> "on"
<thumper> lifeless: but if we set a rule to say code.recipe.beta -> 'off'
<lifeless> thumper: theres a rule for this already isn't there ?
<thumper> lifeless: I want it to work
<thumper> no
<thumper> we have code.recipe_enabled
<thumper> which is different
<thumper> I'd like to rename that to "code.recipe.enabled"
<thumper> enabled controls whether we barf on recipe actions
<lifeless> thumper: so this is to let you go to non-beta without tying it to a deploy ?
<thumper> beta is just to show the messages
<thumper> yes
<thumper> exactly
<lifeless> and then you'll delete both rules entirely.
<thumper> aye
<wallyworld> thumper: can you tell me where our existing web service api tests are? i've had a boy look but can't see them
<lifeless> I would just do the current thing - 'if <thing>: ... '
<thumper> yeah, I know I _can)
<lifeless> thumper: the default is None/'', and you can add rules of ''
<thumper> I just think it would make it more clear
<lifeless> I have think it makes it more awkward
<thumper> getFeatureFlag('some.feature') should be False if some.feature == "off" or "false"
<lifeless> Not at that layer
<lifeless> and I really don't see the argument for a higher layer at the moment
<lifeless> it seems like cruft
<thumper> find
<thumper> fine
<thumper> we can agree to disagree and I'll not change it
<lifeless> I agree that there is some confusion
<wallyworld> lifeless: what do you have against implementing expected semantics for 'off'/'false' meaning Bool(False) ?
<lifeless> one of the reasons I have the opinion I have is that 'no rule implies no value' is kind of predictable. 'no rule implies a specific value' needs some sort of schema
<lifeless> wallyworld: the current /layer/ is simple strings. Its schemaless and typeless.
<lifeless> wallyworld: this makes it extremely robust in some ways, and frustrating in others.
<wallyworld> ok.
<lifeless> wallyworld: given that we version the interpreter for this data (the appserver processes) multiple times a day, a failure in this area would be extremely messy.
<wallyworld> yep :-)
<lifeless> I've no strong objection to a higher layer being built : but I see no benefit to building it either for simple booleans because the current implementation meets our functional needs
<wallyworld> but that's what tests are for :-)
<lifeless> wallyworld: tests cannot catch this
<lifeless> we had one failure from features already, took the site down
<lifeless> -the whole site-
<wallyworld> i'll take your word for it. not familiar enough with it to have an opinion either way. but on the surfaces, everything should be testable in an ideal world
<lifeless> wallyworld: the rules are in the production DB
<wallyworld> why not also in stable or staging?
<lifeless> stable is a branch, not a DB, and the rules can have commercial-in-confidence values in them.
<lifeless> as for staging, it /may/ have the same rules, or it may have different ones
<wallyworld> lifeless: sorry ETERMINOLOGY. i just meant the test databases
<lifeless> I'd rather see small helpers when folk solve tricky problems here built up than the core of the layer become typed.
<wallyworld> i'd say there's an argument that our test environment should replicate the prod one, in this case having the same core database values. but i take the point about the privacy issue
<lifeless> and booleans have a comprehensive solution now: there's nothing you can add to make our handling of booleans more pithy, correct or powerful
<lifeless> wallyworld: folk need to be able to change rules on [qa]staging to be able to check their patches work etc
<lifeless> wallyworld: it would be disruptive to forcibly sync those on a high frequency basis
<wallyworld> agreed. but people also should do some testing with data reflective of what's in production if possible (akin to progression vs regression testing in a way)
<wallyworld> lifeless: so since i have your attention, do you know where out web service api tests are?
<lifeless> what sort of such tests
<lifeless> like
<lifeless> tests for a given api
<lifeless> or tests that the api is wired up
<lifeless> or ... ?
<wallyworld> i want to use a web api client to invoke something annotated with export_write_operation() for example to see that it works
<wallyworld> so i guess tests that the api is wired up
<wallyworld> given that the underlying functionality is unit tested separately
<lifeless> thats generally done by a smoke test of that api
<lifeless> AIUI
<wallyworld> but testing that what comes over the wire is as expected
<lifeless> test.*webservice, spread throughout the tree
<wallyworld> oh. thanks, i'll take a look. i just wanted a starting point to current practices before diving down the wrong rabbit hole :-)
<wallyworld> lifeless: ah i see my problem now. there's web services already on the domain model i'm working with but no tests written, so i couldn't find any examples to start from
<wallyworld> for the new web services i'm adding
<StevenK> Argh. Where is the representation of an archive as a link hiding?
<wallyworld> StevenK:  PPAFormatterAPI ?
<lifeless> are
<lifeless> canonical.librarian.ftests.test_web.LibrarianWebTestCase.test_404
<lifeless>  canonical.librarian.ftests.test_web.LibrarianWebTestCase.test_oldurl
<lifeless> still known as intermittent failures?
<StevenK> lifeless: Yes, they failed yesterday
<wallyworld> StevenK: or something like that?
<sinzui> huwshimi: http://pxtoem.com/
<StevenK> wallyworld: Thanks, but that isn't in the trace back
<wallyworld> StevenK: pastebin?
<StevenK> AH!
<StevenK> The information is spat out before the traceback
<StevenK> wallyworld: You could give me a clue tracing back from the page template to the code, though.
<wallyworld> StevenK: ok. wanna throw the tb in pb?
<StevenK> wallyworld:             <tal:recipe-builds repeat="build view/builds">
<StevenK> wallyworld: I think that's the line. If not, it's sourcepackagerecipe-index.pt, unchanged from devel
<wallyworld> StevenK: should we mumble?
<StevenK> wallyworld: Sure
<thumper> simple review lifeless? https://code.launchpad.net/~thumper/launchpad/recipe-feature-flag/+merge/50092
<thumper> just pushing some lint cleanup
<wallyworld> thumper: get in line :-P
<wallyworld> take a number
<thumper> wallyworld: ok, you do it :)
<wallyworld> thumper: i'll do yours if you do mine :-)
<thumper> ok
<thumper> which one?
<StevenK> thumper: Check your indenting line 43 of the diff
<wallyworld> thanks. i've got 3 and our wip limit is exceeded on the board
<thumper> StevenK: that was in the lint cleanup
<thumper> StevenK: the diff is updating as we speak
<wallyworld> thumper: which one? there's the simple link formatter which could be done first
<thumper> wallyworld: hit me
<wallyworld> thumper: https://code.launchpad.net/~wallyworld/launchpad/add-link-to-noneformatter/+merge/50088
<wallyworld> the recipe ones will take a lot more effort
<wallyworld> but still need to be done
<thumper> wallyworld: TestXHTMLRepresentations?
<thumper> wallyworld: shouldn't that be TestNoneFormatter or something?
<wallyworld> ?
 * wallyworld looks
<thumper> wallyworld: have you looked at test_tales?
<thumper> wallyworld: 'cause I don't understand you test_shorten_traversal
<wallyworld> thumper: didn't see test_tales. since tales.py was in lib/app/browser i looked for existing tests in lib/app/browser/tests
 * thumper nods
<thumper> from lp.testing import test_tales
<thumper> or more accurately
<thumper> from lazr.restful.testing.tales import test_tales
<wallyworld> thumper: the test_shorten_traversal was my attempt to unit test that method based on my reading of it's intended behaviour
<thumper> wallyworld: since there is no docstring, I can't tell what it should do
<thumper> :)
<wallyworld> thumper: yeah, but there's no docstring on the traverse() method that is being tested either, so it's catch 22
<wallyworld> thumper: i'll look at the existing test_tales stuff and fix the tests accordingly
<lifeless> \o/ tests pass
<thumper> wallyworld: ok
<wallyworld> thumper: the TestXHTMLRepresentations was a cut'n'paste gone wrong :-(
<thumper> wallyworld: good :)
<thumper> wallyworld: I was wondering if I was understanding it :)
<wallyworld> thumper: mp diff updating as i type this
 * wallyworld dislikes how long it takes for the diff to be updated
<StevenK> wallyworld: Patches welcome?
<wallyworld> StevenK: isn't it controlled by a cron script?
<StevenK> Two, in fact
<beuno> hey, remember when Launchpad devs didn't use merge proposals at all?  those where the days
<thumper> hi beuno
<wallyworld> ah, when i was a wee lad, i had to walk to school through 10 feet of snow with no shoes
<thumper> beuno: notice we just went over the 50k mark?
<beuno> and speaking of merge proposals!
<beuno> thumper, I did, and I \o/ed
<thumper> :)
<beuno> it kinda blew my mind, I didn't realize the numbers where growing so quick;y
<thumper> beuno: still mostly canonical people using them
<StevenK> I guess the biggest consumer of MPs is LP?
<beuno> I bet U1 can give LP a run for its money
<wallyworld> thumper: just to double check - you don't know of any existing web service tests for recipes that i can't find?
<wgrant> LP only has like 7000 branches.
<StevenK> beuno: I wonder how to test that theory
<wgrant> So it can't be many of the MPs.
<thumper> https://lpstats.canonical.com/graphs/BranchMergeProposalsNewPerDay/20100218/20110218/
<StevenK> I'm pondering crafting some SQL to get the answer
<lifeless> NotFound: Object: <zope.browserpage.metaconfigure.IcingFolder object at 0x883cdd0>, name: u'lp.js'
<lifeless>  ?
 * beuno tries to manually add up all the branches for u1
<wgrant> thumper: That's a really encouraging graph.
<thumper> https://lpstats.canonical.com/graphs/BMPNewRegistrants/20100218/20110218/
<beuno> wgrant, how do I find out how many branches a project has?
<poolie> hi all
<beuno> it only seems to tell me about active branches
<thumper> https://lpstats.canonical.com/graphs/BranchMergeProposalsProjectsNewMergeProposals/20100218/20110218/
<poolie> hi beuno
<beuno> heya poolie!
<thumper> beuno: select show all branches
<beuno> I did, "Any status"
<beuno> Ubuntu One Servers has 345 active branches owned by 31 people and 3 teams. There were 716 commits by 19 people in the last month.
<beuno> but that's not useful
<lifeless> flacoste: hhhaa - XXX flacoste 2008/04/24 This should be moved to a
<lifeless> flacoste: I've half-implemented said setTarget
<StevenK> In SQL: "PackageBuild.archive = Archive.id AND False" ... D'OH
<wgrant>    10294 | launchpad                               | f        |  5103
<wgrant>     9514 | ubuntuone-servers                       | f        |  3462
<wgrant>     1186 | bzr                                     | f        |  1473
<wgrant>     8920 | drizzle                                 | f        |  1279
<wgrant>    23444 | examiner                                | f        |  1109
<wgrant>    13681 | openlp                                  | f        |   939
<wgrant>    12784 | ubuntuone-client                        | f        |   754
<wgrant> From October.
<wgrant> Right column is the count
<wgrant> So U1 probably wins by now.
<wgrant> :(
<StevenK> wallyworld: U1 client already does?
<StevenK> BAH
<StevenK> wgrant: ^
<beuno> right, we have desktopcouch, bindwood, android clients, iphone
<wgrant> beuno: Are they in a project group?
<beuno> wgrant, yes, all under ubuuntuone I think
<wgrant>              name             | count
<wgrant> ------------------------------+-------
<wgrant>  launchpad-project            |  5385
<wgrant>  ubuntuone                    |  5042
<wgrant>  bazaar                       |  2339
<wgrant> But that is outdated :(
<beuno> well, still surprising to see more activity in LP
<beuno> October is not that far away
<StevenK> wgrant: Tempted to get that run on prod :-)
<StevenK> wgrant: With "LIMIT 3" or so
<thumper> wgrant: go on, ask spm
<wgrant> thumper: staging's only a week out of date; you can do it :P
<thumper> wgrant: ok
<wgrant> From the start of 2010:
<wgrant>  launchpad-project            |  2375
<wgrant>  ubuntuone                    |  2271
<wgrant>  bazaar                       |  1079
<wgrant>  ayatana                      |   959
<wgrant> So we still win!?
<wgrant> thumper: SELECT project.name, COUNT(*) FROM branchmergeproposal JOIN branch ON branch.id=branchmergeproposal.target_branch JOIN product ON product.id = branch.product JOIN project ON project.id = product.project GROUP BY project.name ORDER BY COUNT(*) DESC;
<wgrant> thumper: watch out with your recipe FF-only branch.
<wgrant> thumper: request-daily-builds uses the config option.
<wgrant> And we can't make an FF apply only to a script, AFAIK.
<thumper>  launchpad-project |  8912
<thumper>  ubuntuone         |  6695
<thumper>  bazaar            |  4294
<thumper> I wrote it from scratch
<spm> [15:20:50] <thumper> wgrant: go on, ask spm <== have I finally got you lot scared of me. *FINALLY*!?!?
<StevenK> spm: Ask wgrant?
 * StevenK cackles
<spm> heh
 * StevenK notes that Storm doesn't like 'is'
<StevenK> But it returns a BOOLEAN!
<lifeless> its interesting - bazaar has 1/5th the devs, and 1/2 the mps
<wgrant> lifeless: But Bazaar also doesn't suck.
<beuno> I guess bazaar probably has the most community as well
<wgrant> True.
<wgrant> spm: r12395 to nodowntime, pls.
<beuno> and, launchpad was the first of all those to start using mps, if we look at totals
<beuno> bazaar slowly moves away from bundlebuggy after a while
<wgrant> True.
<wgrant> That's why I took the numbers from 2010 onwards.
<wgrant> When everyone should have been using MPs.
 * beuno nods
<lifeless> \o/
<lifeless> one query for the milestone targeted bugs portlet.
<wgrant> lifeless: But how slow?
<lifeless> wgrant: https://bugs.launchpad.net/launchpad/+bug/717394/comments/2
<lifeless> wgrant: I expect to be close to that, am assessing the generated sql now
<lifeless> argh, search builder fail
<lifeless> constraining on distro series using any -> noop, no constraino.
<thumper> beuno: ...
<thumper>  launchpad-project |  3879
<thumper>  ubuntuone         |  3525
<thumper>  bazaar            |  1548
<thumper> (
<lifeless> SELECT BugTask.distroseries, COUNT(BugTask.bug) FROM BugTask, Bug WHERE Bug.id = BugTask.bug AND ((BugTask.status = 10) OR (BugTask.status = 15) OR (BugTask.status = 20) OR (BugTask.status = 21) OR (BugTask.status = 22) OR (BugTask.status = 25)) AND Bug.duplicateof is NULL GROUP BY BugTask.distroseries
<thumper> for branches created after the start of 2010
<beuno> ah, laz U1 devs!
<beuno> *lazy
 * thumper is getting the munchies
<lifeless> still, i need to go; I'll finish it off later
<StevenK> Mmmmmm, smells like rain here
<lifeless> wgrant: pushing to lp:~lifeless/launchpad/bug-717394 if you are interested. Its a little ugly, but EMENOCARE
<lifeless> pushed
<lifeless> wgrant: I thought you requested a deploy?
<StevenK> lifeless: It's on LPS
<wgrant> lifeless: henninge added one to the deployed section of LPS, but didn't poke anyone about it. I extended it to the latest rev and moved it to the right place.
<wgrant> Just now.
<lifeless> wgrant: cool; probably want to poke someone
<wgrant> I did.
<StevenK> lifeless: He did?
<wgrant> spm: Are you sufficiently poked?
<StevenK> Haha
<StevenK> "Are you not entertained!?"
<poolie> thumper, so lifeless's point was to just keep using non-empty strings as boolean true?
<thumper> poolie: yes... although I agree with you
 * thumper EODs
<poolie> good night
<thumper> poolie: we can talk about this more tomorrow if you like
<StevenK> Can haz review from someone?
<StevenK> lifeless: You know, I think identi.ca doesn't know your new suburb
<poolie> wow
<poolie> wgrant, the lag seemed to be suddenly going up to several seconds
<wgrant> Yeah.
<wgrant> I guess Mumble isn't so good when you're on the same side of the world.
<poolie> adequate consultation is good
<poolie> routing via london's not optimal
<wgrant> Not quite.
<poolie> i guess we possibly could have an australasian server
<poolie> i wonder how hard it is
<wgrant> Huh.
<wgrant> What happened to the loggerhead OOPSes/
<wgrant> ?
<wgrant> There were <1000 yesterday.
<wgrant>   lib/lp/contrib/
<wgrant> Oxymoron!
<StevenK> wgrant: Naming is hard
<wgrant> StevenK: We already have lib/contrib :(
<StevenK> Yes, and it needs to die too
<wgrant> Now I have two directories to destroy :(
<StevenK> Awesome, lib/contrib/oauth.py has no license
<StevenK> wgrant: Seems like spm is not sufficently poked.
<wgrant> Insufficiently present, would be my guess.
<wgrant> poolie: Is there something wrong with the oauth 1.0.1 egg?
<wgrant> Or did you just try to use a package because it is less evil?
<poolie> wallyworld, hi
<wallyworld> poolie: yellow
<poolie> the thing with tim's patch is when he says 'on/off' he doesn't actually mean the strings 'on' or 'off'
<poolie> how silly of you :)
<poolie> we can either fix it so you can actually use those values
<poolie> which is the bug i mentioned
<wallyworld> i thought that's what gerFeatureFlag() returns?
<poolie> or we can update his docs
<poolie> you're right
<poolie> it just returns the string at the moment
<poolie> so unless it's handled at a lower level he needs to check == 'on'
<wallyworld> so his condiitonal will fail
<wallyworld> as written
<poolie> well, it won't work in the way you'd expect
<poolie> how do you say that in TAL?
<poolie> i think it's a bit longwinded
<poolie> wgrant, i believe the egg still has the bug in it
<poolie> ubuntu's package is 1.0+a bit
<wgrant> poolie: We have the 1.0 egg.
<wallyworld> poolie: so, won't "if not getFeatureFlag(RECIPE_ENABLED_FLAG)" give a crap result if the ff is set to "off" ????
<wgrant> There is a 1.0.1 egg aka 1.0a, which is probably what Ubuntu has.
<StevenK> wgrant: Can you see the point of lib/canonical/lazr/security.py at all?
<wgrant> StevenK: Probably to support callsites that were added before stuff was moved to lazr.restful.
<wgrant> I can only see one left, though, in a test.
<StevenK> Right
<StevenK> I'm tempted to just fix it
<wgrant> Delete delete delete.
<wgrant> You have my full support.
<StevenK> For once. :-P
<wgrant> You have my full support to delete anything in lib/canonical, as long as it doesn't break tests :P
<StevenK> I'll delete lib/canonical/database, then.
<StevenK> That won't break tests, that will break EVERYTHING.
<wgrant> Indeed.
 * wallyworld goes to take kid to soccer
<StevenK> wgrant: So, fix the test to import from lazr.restful or just bin it completly?
<wgrant> StevenK: LP doesn't use protect_schema directly.
<wgrant> Check that it's tested in lazr.restful and bin it.
<StevenK> wgrant: Looks to be used extensively in doc/webservice.txt
<poolie> wallyworld, yes it will give a crap result
<poolie> back in a sec
<wgrant> StevenK: But is it tested directly?
<poolie> wallyworld, anyhow, that's my point
<poolie> there are various approaches
<StevenK> wgrant: I can't see a unit test directly
<wgrant> [XXX leonardr 2009-04-02 bug=354441 This test should be moved into the same lazr module as the code it tests, security.py (in lazr.restful as of this writing.)]
<wgrant> I'd follow that advice.
<poolie> wgrant, my first attempt at removing the code from contrib fell in a heap because the same bug is in the code in the egg
<poolie> hooray for tests
<wgrant> poolie: Yeah, but I think our egg is outdated.
<wgrant> There is a newer one.
<poolie> ok, in that case upgrading the egg and re-landing my first branch should fix it
<wgrant> Where is that branch?
<wgrant> I will throw it at ec2 now.
<poolie> bug 715000?
<wgrant> Not that one.
<wgrant> Because we gained a new contrib module this morning, I must remove this one swiftly.
<wgrant> Ah, the branch is gone.
<poolie> >  Because we gained a new contrib module this morning, I must remove this one swiftly.
<poolie> why?
<poolie> ah i might have pulled into that branch
 * poolie look
<wgrant> Yeah, you repurposed the branch to use the package instead.
<poolie> cherrypicking r12325 of lp:launchpad should do it
<poolie> it was merged and then it failed in buildbot
<wgrant> Oh, true.
<wgrant> I forgot that bit.
<wgrant> Or just s/contrib.oauth/oauth.oauth/, I guess.
<poolie> wgrant, basically just that plus deleting the contrib thing
<poolie> yep
<poolie> for going to use the packaged version there was just a bit mor estuff
<poolie> are you just trying to remove cruft, or was there some other motivation too?
<wgrant> poolie: Removing cruft.
<poolie> thanks
<stub> So the new SSH server is falling over under load, or because it is leaking?
<wgrant> stub: It uses more handles per connection than before, but as far as jam and I can see it does not leak until it has already exhausted its handles.
<wgrant> And jam has a fix for those post-exhaustion leaks, which should allow recovery.
<wgrant> We do not know why production reached exhaustion.
<poolie> it seems like previously it would stall or block connections when load spiked
<stub> wgrant: Can we put it behind HAProxy to keep the load under control?
<poolie> and with jam's change it would get bogged down
<poolie> there's separate work underway towards that
<wgrant> stub: I suppose we could.
<wgrant> We want haproxy soon anyway for other reasons.
<spiv> The numbers of observed connections in production don't match the analysis of how many connections it would take to reach exhaustion, so we're still missing something.
<spiv> (Fixing the known robustness bugs is still a good idea, of course)
<wgrant> There were 117 forking services at the probable time of exhaustion.
<wgrant> Which is only 600 FDs.
<wgrant> Unless it was being slow and hadn't noticed that lots of children were dead.
<wgrant> So it still held their handles.
<stub> But howmany 'recently' that might not have been cleaned up?
<stub> Mmm...
<wgrant> It's hard to say, because the logs are gone.
<wgrant> I think we should fix the leak, bump the limit (which means moving to a new reactor), and then deploy while watching very carefully and lsofing regularly.
<wgrant> Last time we knew something was up 40 minutes before service really degraded.
<wgrant> So it is not *too* dangerous.
<stub> Right. Since there will always be a hard limits though, so we will want limiting from HAProxy anyway (just might be much higher once the leak is fixed). And it will help diagnose if it is a leak or slow gc.
<spiv> I don't know any reason not to add --reactor=epoll to the twistd invocation (in the init script, I guess?)
<wgrant> Sure.
<wgrant> spiv: Let me try that locally and throw 600 connections at it.
<spiv> (And we may as well do that for all twistd invocations everywhere)
<poolie> spiv, what _is_ the state of the haproxy stuff in production
<poolie> per robert's comments in bug 717345
<poolie> it's not yet live but ready to go live?
<poolie> spm^^ ?
<spiv> poolie: see bug 702024 (or RT 40480); I'm stuck waiting for more information from spm about how it was tested on staging, because that failed but AFACIS it's working perfectly in a dev vm.
<spiv> poolie: the support code in the production code base, and the web status port is deployed
<wgrant> 2000 FDs,s till going fine...
<spiv> wgrant: you had to raise the per-process fd limit, I assume?
<wgrant> spiv: Yes.
<wgrant> Hah.
<wgrant> hammer_ssh.py hit the FD limit at a little over 500 connections.
<wgrant> But the server is fine.
<spiv> :)
<wgrant> Running with an FD limit of 10000 and --reactor=epoll
<wgrant> Although it does like its RAM...
<poolie> i'd like to see some Launchpad Enchantment Proposals
<poolie> what is the rule for who can triage while filing?
<poolie> i'm in ~kanban and yet i cannot do it for kanban/+filebug
<wgrant> The bug supervisor.
<wgrant> If the bug supervisor is not set, then nobody can.
<wgrant> It does not revert to the owner.
<poolie> i see; thanks
<wgrant> (probably a bug, but so are lots of things!)
<poolie> > Awesome feature.  I'm happy to volunteer gcalctool to use this.
<poolie> excellent
<poolie> what a good start
<wgrant> poolie: Is that a BFBIP volunteer?
<poolie> yes, robert ancell
<poolie> k, got to go
<wgrant> Ah, yes, found it.
<poolie> have a good night
<wgrant> You too.
<lifeless> StevenK: I've told it to not show location, but it didnae listen
<lifeless> poolie: hi
<lifeless> poolie: am back if you wantt to talk
<cody-somerville> Is the PPA publishing cycle still at 5 minute interval?
<wgrant> cody-somerville: Yes.
<cody-somerville> Deletions get processed during that cycle, right?
<cody-somerville> and is there a graph showing delay? (ie. # of minutes late starting new cycle vs. time or something)
<wgrant> Files are removed from the indices as part of each publisher run.
<wgrant> But they are removed from disk less frequently.
<wgrant> */30, IIRC.
<wgrant> Yes, */20
<wgrant> Er/
<wgrant> */30, I cannot type.
<lifeless> stub: ping
<wgrant> cody-somerville: https://lpstats.canonical.com/graphs/PPALatencies/20110217/20110218/
<stub> lifeless: pong
<lifeless> now a good time for the weekly call ?
<stub> lifeless: time is fine. my sleep patterns the problem.
<stub> lifeless: sure
<lifeless> no worries; went and saw a movie ;)
<lifeless> stub: https://lpstats.canonical.com/graphs/WildcherryDiskUsage/
<LPCIBot> Project devel build (449): FAILURE in 6 hr 3 min: https://hudson.wedontsleep.org/job/devel/449/
<LPCIBot> * Launchpad Patch Queue Manager: [r=stub][bug=607935] Reduce overhead when showing only some bug
<LPCIBot> comments.
<LPCIBot> * Launchpad Patch Queue Manager: [r=leonardr][no-qa] Add YUI3 gallery-accordion widget to tree.
<lifeless> explain select count(distinct branchrevision.branch) from branchrevision, branch where branch.id=branchrevision.branch and branch.last_scanned is NULL;
<stub> https://pastebin.canonical.com/43510/
<stub> Bug 711071
<_mup_> Bug #711071: Distribution:+bugtarget-portlet-bugfilters-stats timeouts <dba> <timeout> <Launchpad itself:Triaged> < https://launchpad.net/bugs/711071 >
<lifeless> stub: https://bugs.launchpad.net/launchpad/+bug/717394
<_mup_> Bug #717394: Distribution:+bugs timeouts <timeout> <Launchpad itself:Triaged> < https://launchpad.net/bugs/717394 >
<jtv> morning folks
<adeuring> good morning
<jtv> hi adeuring
<adeuring> hi jtv!
<StevenK> allenap: Hello, Mr OCR! Would you have time to review my branch today?
<jtv> StevenK: if not, I could take it.
<lifeless> \o/ 1.4 seconds off of bug search in ubuntu coming right up
<lifeless> and, I'm off to sleep
<mrevell> Morning
<allenap> StevenK: Sure. I'm only doing a half day today, but you're first in line :)
* allenap changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | firefighting: - | On call reviewer: allenap | https://code.launchpad.net/launchpad-project/+activereviews
 * allenap assumed abentley is no longer reviewing.
<jtv> allenap: I already claimed it :)
<allenap> jtv: Woah, swifty.
<jtv> a timezone ahead :)
<jtv> StevenK: "an" private archive..?
<jtv> lib/lp/code/model/tests/test_sourcepackagerecipe.py
<StevenK> jtv: I can't see that in my diff, but looking
<jtv> Near the bottom.
<jtv> Very close to it.
<StevenK> Oh, it's "an" disabled, not private
<jtv> Sorry!
<jtv> Force of habit I suppose
<StevenK> jtv: Fixed and pushed
<jtv> StevenK: also in that same method by the way: it may be clearer to change the name slightly to say what should happen, not just what scenario you test.  For instance, test_getBuilds_ignores_disabled_archive instead of test_getBuilds_disabled_archive
<StevenK> jtv: Agreed, fixed locally.
<StevenK> jtv: Anything else you can see, or shall I commit and push?
<jtv> StevenK: I've got more, hang on
<jtv> StevenK: two questions about test_view_with_disabled_archive.
<StevenK> jtv: Shoot
<jtv> First, are all cases covered?  Owner can see builds for private archive, other users can't?
<StevenK> jtv: In this case, even the owner won't, which I think is fine. They disabled the archive, anyway.
<jtv> Then I think I misunderstood something in the MP
<StevenK> They will of course be able to change the daily build archive if they wish.
<jtv> So the owner sees private _archives_, not _builds for_ private archives.
<StevenK> *disabled*
<StevenK> Stop doing that :-)
<jtv> Argh
<jtv> Yes, sorry.
<jtv> So: the owner sees their disabled archives, but not builds for their disabled archives?
<StevenK> jtv: We are only talking about the page for the recipe here -- if the daily build archive is disabled, anyone will see the name of the archive, but no one will see builds into it. And of course will be able to change it, as I said.
<StevenK> And of course, the owner will be able to change it, I mean.
<jtv> OK.
<jtv> Then what I hope is my final stupid question: it looks like you're adding Archive to your join.  How will that affect performance?
<jtv> Damn, penultimate.  My other one was "why does test_view_with_disabled_archive create a browser with a specific person, rather than either the default or self.factory.makePerson()"
<StevenK> jtv: I suppose I could make another person rather than using no-priv. Would you prefer that?
<jtv> StevenK: I think so, yes.  makePerson() really says "some arbitrary regular person."
<StevenK> jtv: Archive isn't terrible large, so I can't see it adds much to the query.
<jtv> But if there's a default you can use, that's even better.
<StevenK> Well, I can use no-priv, but sampledata makes me sob.
<StevenK> jtv: Also fixed locally.
<jtv> What I mean is, doesn't getUserBrowser give you some arbitrary, unprivileged person by default?
<StevenK> Oh. Not that I noticed when I read it.
 * StevenK re-checks
<StevenK> jtv: You're right, fixing.
<jtv> StevenK: also, please keep an eye on its performance!  Archive isn't _that_ small, and I think I caught it slowing down a join quite a lot recently.
<StevenK> jtv: I will compare before and after on qastaging for page loads when I QA. If it's a large jump, I will file a bug.
<jtv> StevenK: great, thanks.
<jtv> Final note: in lp/code/model/sourcepackagerecipe.py, the And() you add has its arguments list formatted weirdly.  Could you add a line break right after the opening parenthesis?
<StevenK> jtv: It matches other uses in the file ...
<jtv> StevenK: no time like the presentâ¦
<StevenK> jtv:
<StevenK> http://pastebin.ubuntu.com/568101/
<StevenK> Rah
<StevenK> jtv: I prefer it before, TBH.
<jtv> Hmm okay, let's be pragmatic then. :)
<jtv> Approved.
<StevenK> jtv: With the And() change, or you don't mind now?
<jtv> StevenK: I don't mind now
<jtv> Anyway, I already voted.
<jtv> henninge: I think this was you, but I'm not used to hearing such foul language from you!
<jtv>         self.assertEqual('my-domain', make_name('my - do@ #*$&main'))
<henninge> jtv: That's what reading comic books does to one's language!
<jtv> tsk tsk
* allenap changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | firefighting: - | On call reviewer: - | https://code.launchpad.net/launchpad-project/+activereviews
<jml> I've added a dependency, and that dependency makes deprecation warnings when it loads
<jml> we shouldn't do anything about the deprecation warnings
<jml> how can I silence them?
<wgrant> jml: lib/lp_sitecustomise.py
<wgrant> s/mise/mize/
<wgrant> We filter Crypto DeprecationWarnings there already.
<jml> wgrant: thanks.
<jml> next question, what's the correct way of running a script from an egg dependency within a test?
<wgrant> Is it importable?
<jml> not really. it's the sphinx-build script from Sphinx. I could sort of re-implement sphinx-build by importing sphinx and calling main() directly, if pressed.
<jml> but am not in a rush to do so.
<wgrant> I'd be doing that, unless the script itself is non-trivial...
<jml> hmm.
<jml> more robust against path snafus, less against changes to the sphinx-build interface. you're probably right.
<wgrant> Plus it feels less like evil.
<jml> I'm not sure
<jml> my evilometer can't really distinguish the two.
<wgrant> Calling Python from Python via a subprocess is somewhat evil.
<jml> well, the evil is that sphinx doesn't provide a decent API
<wgrant> Ah :(
<wgrant> My programmatic invocations of it are limited to Makefiles.
<jml> I had thought about calling the Make target.
<LPCIBot> Project db-devel build (374): FAILURE in 5 hr 56 min: https://hudson.wedontsleep.org/job/db-devel/374/
<LPCIBot> * Launchpad Patch Queue Manager: [rs=buildbot-poller] automatic merge from stable. Revisions: 12396,
<LPCIBot> 12397 included.
<LPCIBot> * Launchpad Patch Queue Manager: [r=stub][no-qa] Make the BugMessage.index non-null now that its fully
<LPCIBot> populated and maintained by code.
<LPCIBot> * Launchpad Patch Queue Manager: [rs=buildbot-poller] automatic merge from stable. Revisions: 12394,
<LPCIBot> 12395 included.
<jml> sphinx-build also doesn't return a non-zero return code when it fails
<wgrant> Yeah, I noticed that.
<wgrant> Fortunately docs aren't terribly critical, I guess.
<wgrant> Still, Sphinx is really likable in lots of ways.
<wgrant> I can forgive it a bit.
<jml> No, but that's not an excuse for not building a good command line tool
<wgrant> Although I cannot forgive docutils for being what it is.
<jml> grar caching
<jml> wait, huh. Why does TestCase.makeTemporaryDirectory not delete the directory in tearDown?
<bigjools> you mean - our code has bugs?!  <shock>
<jml> never mind. I'm a muppet.
<bigjools> waldorf?
<wgrant> Hmm, that's pretty nice.
<wgrant> nightly.sh takes 5 hours now.
<wgrant> Not >24
<jml> wgrant: that's good. 5 is much less than 24.
<wgrant> Although update-bugtask-targetnamecaches takes 50% longer on prod than mawson :(
<jml> *sigh*. running directly from Python means mangling stdout & stderr. I guess I'll try it and see if it's faster.
<wgrant> Sounds like it may be similar to docutils in its general levels of sensibility. Which I guess isn't surprising.
<bigjools> wgrant: mawson is for soyuz testing....
<deryck> Morning, all.
<jml> boring branch: https://code.launchpad.net/~jml/launchpad/prevent-new-sphinx-errors/+merge/50136
<jml> where is zeca used these days?
<jml> specifically, is zeca.tac invoked anywhere?
<maxb> tests that need to talk to a keyserver?
<wgrant> It is still used by tests.
<wgrant> eg soyuz-upload.txt
<jml> any ideas about prod though?
<wgrant> jml: It was never used in prod.
<wgrant> It would possibly be even less reliable than SKS.
<wgrant> It was never intended for anything more than tests.
<jml> ok. I'll move the tac/ out of the root daemons/ directory then.
<wgrant> Renaming it wouldn't go astray either...
<jml> wgrant: yeah. I'm moving canonical.zeca to lp.services.keyserver.
<wgrant> jml: Could you put it in lp.testing.keyserver, maybe?
<jml> wgrant: yeah, I think so.
<wgrant> lp.services.keyserver sounds like GPGHandler.
<jml> hmm.
<jml> does zeca even *need* config?
 * jml doesn't think so
<wgrant> It needs a root.
<wgrant> Indeed, that is its only key.
<wgrant> If you think you can make it static, you are sadly mistaken -- we need to override all those for parallel testing :(
<jml> is config the easiest way to do that?
<jml> does start-soyuz-dev.sh get used much?
<wgrant> The existing facilities to do that are based on the config infrastructure.
<wgrant> I use it often, but more often for buildd-manager than zeca.
<gmb> allenap: Are you OCRing today?
<allenap> gmb: Not really, I'm only doing a half-day and I've got a lot on my plate. If it's short I might take a look though :)
<gmb> allenap: 57 lines of JS refactoring? Pweeese?
<allenap> gmb: Okay.
<jml> wgrant: ok. I'll fix it rather than delete it then.
<allenap> As it's you :)
<gmb> allenap: I shall send you a Woodfordes voucher by way of thanks. https://code.launchpad.net/~gmb/launchpad/speed-up-subscription-overlay-bug-719249/+merge/50148
<wgrant> jml: It will probably be used more often when we have another 20 people trying to work out what Soyuz does.
<wgrant> Rather than just using mawson.
 * gmb notes that he needs to find out whether Woodfordes does vouchers.
<jml> yeah.
<jml> I just wish we had some consistency with the means by which we fire up dev services
<wgrant> You mean like a Makefile?
<jml> and indeed some rationale for scripts/, bin/ *and* utilities/
<jml> wgrant: if such were used universally
<wgrant> jml: They pretty much are, except for Soyuz because I was lazy.
<jml> I love that 'zeca' is also a package and a password.
<wgrant> I do not know its etymology.
<jml> A Brazillian singer.
<wgrant> Ah.
<jml> I have to say I don't like the newer standard_test_template
<henninge> Anybody in for a 103 line review?
<henninge> https://code.edge.launchpad.net/~henninge/launchpad/bug-720673-import-origin/+merge/50157
<henninge> deryck, abentley, adeuring: ^ ?
<abentley> henninge, sure.
<henninge> abentley: thank you!
<abentley> henninge, you're still using edge? :-P
<henninge> abentley: ups, that's my browser history playing tricks on me ;)
<abentley> henninge, r=me
<henninge> abentley: thanks!
<abentley> henninge, also, could we mumble about bug #702477?
<_mup_> Bug #702477: Translation credits on new POFiles show the dummy string for the other side <upstream-translations-sharing> <Launchpad itself:Triaged by abentley> < https://launchpad.net/bugs/702477 >
<henninge> abentley: sure, give me 5 minutes
<abentley> henninge, cool.
<henninge> abentley: now
<henninge> ;)
* jcsackett changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | firefighting: - | On call reviewer: jcsackett | https://code.launchpad.net/launchpad-project/+activereviews
<bigjools> henninge: do you know what's going on here? https://answers.edge.launchpad.net/launchpad/+question/125218
 * jml pushes up a branch that renames 'zeca'
<jml> https://code.launchpad.net/~jml/launchpad/zeca-is-keyserver/+merge/50166
<bigjools> want that reviewed? :)
<jml> bigjools: yes please.
<jcsackett> sinzui: could you follow up on my review of https://code.launchpad.net/~jml/launchpad/prevent-new-sphinx-errors/+merge/50136, pls?
<sinzui> I will
<jcsackett> thanks.
<jtv> Revenge for the Fake Librarian!
<jtv> I just got a test case down from 29s to 3s.
<jtv> bigjools: ^^^
<bigjools> nice!
<jtv> The main trick is that you don't have to commit as much.
<bigjools> very nice
<jtv> Also of course, when run individually, you can save on setup by using a lighter layer.
<jtv> That's another 8s or so.
<bigjools> jtv: know what's going on here? https://answers.edge.launchpad.net/launchpad/+question/125218
<jtv> Gar, the incorrect use of technical terms is dizzying
<jtv> bigjools: working on it.
<bigjools> jtv: also: https://answers.edge.launchpad.net/launchpad/+question/140873 and  https://answers.edge.launchpad.net/launchpad/+question/140873
<jtv> bigjools: aren't those the same?
<bigjools> grar paste fail
<jtv> They very much look the same.
<bigjools> https://answers.edge.launchpad.net/launchpad/+question/137882
<jtv> bigjools: OK, that's me busy for the rest of the day :)
<bigjools> sinzui: did you hear back from the guy here? : https://answers.edge.launchpad.net/launchpad/+question/145187
<bigjools> jtv: I aim to please
<jtv> your aim is <tshock!>OW!
<sinzui> bigjools: thanks for asking. No. he has not. I think we should suspend
<leonardr> large but interesting branch needs review: https://code.launchpad.net/~leonardr/lazr.restful/include-html-field-representations/+merge/50174
<bigjools> sinzui: ok, I'll get on it
<sinzui> bigjools: Chex I have a question from a user I cannot answer. It concerns a user investigating spam that claims to be from him: https://pastebin.canonical.com/43532/
<LPCIBot> Yippie, build fixed!
<LPCIBot> Project devel build (450): FIXED in 5 hr 49 min: https://hudson.wedontsleep.org/job/devel/450/
<jml> what benefit do we get from chunkydiff?
<jml> a better question, are the errors from pagetests more readable than the errors from standard doctests?
 * jml experiments
<abentley> jml, I dunno, but I would *love* to have bzr's assertEqualDiff.
<bigjools> jml: I've never noticed any difference between doc and page test error output
<jml> yeah, identical.
<henninge> abentley: I am stuck on a stupid timeout on qastaging for approving a template to be uploaded to cheese ...
<abentley> henninge, ah.
<henninge> before that it was waiting for an admin to make me maintainer of the project
<henninge> ...
<henninge> abentley: I am about EOD now. I cannot continue on this today, I am sorry.
<abentley> henninge, understood.
<jtv> How anti-climactic.  Some preparation, a whole day of revising and adding tests, and then everything just passes with just a tiny tweak of two functions.
<jtv> TDD will never sell until they learn to threaten us with eternal damnation.
<jml> huh
<jml> TDD isn't spending a day writing tests before you change code
<jtv> There were a lot of tests to be revised.
<jtv> Small change, large consequences.
<jml> guh, DNS down
<jml> anyway, https://code.launchpad.net/~jml/launchpad/flush-out-canonical/+merge/50192 up for review
<jtv> jml: it's massive!
<jml> jtv: not really.
<jml> jtv: I mean, it's bigger than usual, but it's not like it's 1500 lines of actual programming.
<jtv> jml: anyway, fell for itâI spoke up so I volunteered.  Looking now.
<jml> jtv: thanks.
<jtv> And may I say thank you for doing this.
<jml> jtv: my pleasure.
<jtv> jml: doesn't pydoc have a dedicated tag for things that are deprecated?
<jml> back.
<jml> jtv: don't know. will look it up.
<jml> also, we use rst-based epytext
<jtv> ah
<jtv> that gives me something I can search for, thanks
<jtv> (I'd been wondering)
<jml> answer is "yes", http://epydoc.sourceforge.net/manual-fields.html
<jtv> Cool!
<bigjools> mmm "make compile" is taking forever....
<jtv> bigjools: -jâ
<bigjools> that'd be nice
<bigjools> -j2 for now though.  still takes ages
<bigjools> 5 minutes so far!
<bigjools> 2 spare cores = sad Phenom
<leonardr> jcsackett, can you look at https://code.launchpad.net/~leonardr/lazr.restful/include-html-field-representations/+merge/50174? it's a big branch but it should be fun
<bigjools> jml: I am trying to bolt ftp into poppy-sftp
<lifeless> moin
<jml> bigjools: oh yes.
<bigjools> the maze of twisted objects is making my head hurt
<jml> lifeless: hello
<lifeless> jml: hello
<bigjools> morning lifeless
<jml> bigjools: am happy to help navigate.
<bigjools> jml: I will take you up on that tomorrow, thank you
<lifeless> jml: you might want to deop yourself; joey fixed us to have permissions, but its nominally bad form to have ops all the time
<bigjools> just trying to get something basic working first
<jml> lifeless: thanks. I didn't notice.
<lifeless> flacoste: you too
<jtv> jml: I'm surprised we're not using some off-the-shelf base64.
<jml> jtv: me too.
<jtv> I'll admit I'm not sure it's all url-safe.  And I believe there are different choices for the 64th digit.
<jml> jtv: I'm also not sure why we sometimes use md5 and sometimes use sha1
<lifeless> flacoste: ping
<jtv> jml: if we keep the algorithm secret, it's harder to crack.
<jtv> (yes, I AM joking)
<jml> jtv: at least now the intent of the calling code is more clear.
<jtv> jml: yes, it's a pleasure to read.
<jml> jtv: stub is the original author, it seems (r1268!)
<jtv> long live stub
<jtv> A factory of metaclasses, oh dearâmy meta stack just overflowed.
<bigjools> twistd has finally beaten me for the day, I quit
<benji> jml: half-overhearing your conversation makes me wonder if base64.urlsafe_b64encode in the stdlib is of interest to you
<jcsackett> leonardr: looking now.
<jtv> jml: ISTM base does go horribly wrong on 2's-complement minimal numbers.
<jml> benji: If I'm reading the code correctly, I think it's too late.
<jml> benji: since we've already stored stuff keyed with the old algorithm.
<benji> Oh, there are otherwise encoded URLs in the wild?  That would be too late.  (Unless we were highly motivated to change for some reason.)
<jml> right.
<jml> not 100% sure though.
<jml> I'm just moving the bit that does the encoding from one place to another :)
<jml> (yay abstraction!)
<benji> having a wierd way and a standard way is worse than just a wierd way :)
<jcsackett> leonardr: this will probably take me a bit. :-P
<jml> exactly.
<jml> jtv: I don't know. I'm not sure it really matters, tbh.
<lifeless> matsubara: hi
<lifeless> matsubara: I think this is another bug in wallyworlds bind variables patch
<matsubara> lifeless, hi
<lifeless> matsubara: note the values has (u"...."
<jtv> jml: yeahâ¦ though given the recent hullabaloo with basically all java and php breakingâ¦
<lifeless> thats python syntax, not sql
<matsubara> lifeless, i don't know what patch is that
<jml> jtv: I can add a check to reject negative numbers. It's not used for positive ones right now.
<jml> s/not/only/
<lifeless> matsubara: he changed the oops sql layer to record the values of things getting
<jtv> jml: sure, that'd do it for me!
<lifeless> %s substituted etc
<jtv> jml: also, simpler.
<bigjools> good night
<lifeless> bigjools: night
<jtv> good night bigjools
<lifeless> matsubara: so that we get better diagnostics
<lifeless> however, its backfired spectacularly;)
<leonardr> jcsackett, feel free to ask questions
<matsubara> lifeless, hmm
<jtv> jml: in lib/lp_sitecustomize.py, s/eminate/emanate/
<jtv> jml: Also, any idea whether that comment pertains to all deprecation warnings (which emanate from zope), or to all deprecation warnings that emanate from zope?
<matsubara> lifeless, it doesn't seem to happen often though. I could identify only one oops triggering the regex problem
<jml> jtv: no idea. I just moved it.
<jtv> jml: fair enough, I'm ready to vote
<lifeless> matsubara: indeed - much of our code just embeds literal sql today; this will change
<lifeless> matsubara: I've just commented via mail with a suggested addition regex, and a proposed bugfix for the existing one so it doesn't timeout
<jtv> jml: done.
<leonardr> jcsackett, fyi, one thing that i shouldn't have put into the branch (and that i remove in the follow-up):
<leonardr> if not invalid_field.endswith('_html'):
<leonardr> that's the wrong way to do that
<jcsackett> leonardr: dig.
<jml> jtv: thanks!
<jtv> np
<lifeless> \o/
<lifeless> bug 1 rendering reliably on staging.
<_mup_> Bug #1: Microsoft has a majority market share <iso-testing> <ubuntu> <Clubdistro:Confirmed> <Computer Science Ubuntu:Invalid by compscibuntu-bugs> <EasyPeasy Overview:Invalid by ramvi> <GNOME Screensaver:Won't Fix> <Ichthux:Invalid by raphink> <JAK LINUX:Invalid> <The Linux OS Project:In Progress> <metacity:In Progress> <OpenOffice:In Progress by lh-maviya> <Tabuntu:Invalid by tinarussell> <Tivion:Invalid by shakaran> <Tv-Player:New> <Ubuntu:I
<lifeless> who would have thought it.
<jml> lifeless: nice
<sinzui> jcsackett: I see you have a fix for diff and it was the viewport
<jcsackett> sinzui: alignment to the viewport, yes.
<jcsackett> once it was isolated it was trivial.
<sinzui> jcsackett: r=me for code and ui
<leonardr> jcsackett, when you're done: https://code.launchpad.net/~leonardr/lazr.restful/more-tests-for-combined-representations/+merge/50202
<jcsackett> leonardr: that's the followup?
<leonardr> yeah
<jcsackett> sinzui: thanks
<lifeless> I wonder if someone would like to qa https://bugs.launchpad.net/launchpad/+bug/718809
<_mup_> Bug #718809: New users should default to not receiving email for their own actions <qa-needstesting> <story-better-bug-notification> <Launchpad itself:Fix Committed by danilo> < https://launchpad.net/bugs/718809 >
<lifeless> if they do, we can deploy 4 more revs
<flacoste> hi lifeless
<jtv> jcsackett, are you very busy?
<jtv> If not, â https://code.launchpad.net/~jtv/launchpad/bug-719247/+merge/50204
<lifeless> hi flacoste
<flacoste> lifeless, you pinged me earlier?
<flacoste> also , where do you see that I have ops?
<lifeless> flacoste: you had ops here and in #launchpad
<flacoste> now removed?
<lifeless> jml deoped you here, but that will happen on every join until you tell chanserv not to do it by default
<lifeless> flacoste: you can get it back with /msg chanserv op #launchpad-dev
<jml> lifeless: ahhh, that's what's going on.
<lifeless> jml: same story for jtv, bigjools etc having voice
<jml> lifeless: do you know what the setting is?
<jml> lifeless: because afaict, you have the same flags as I do.
<lifeless> we had someone being hostile/silly on #launchpad the other day and I pinged joey cause the ops list was -staaaale-
<lifeless> I do
<jml> figured it out
<lifeless> it should have been
<lifeless> flags #launchpad-dev lifeless -O
<lifeless> but chanserv whinged, so I've asked joey to action it
<jml> yeah, that's what I've got.
<jcsackett> jtv: i can get to it, it's just got two in the queue before it.
<jtv> jcsackett: then I will have to leave before you get to itâ¦ the documentation's fairly extensive though.
<jcsackett> jtv: no worries.
<jtv> thanks
<lifeless> flacoste: I did, about the chat with #is; how did it go?
<mhall119> using launchpadlib, is there a way to get the url for a person's mugshot?
<flacoste> lifeless: went well, our tickets are blocked waiting on resources, but are next in line
<lifeless> thanks
<mhall119> lp.people['mhall119'].mugshot gives me a lazr.restfulclient.resource.HostedFile
<mhall119> lp.people['mhall119'].mugshot.open() gives me a lazr.restfulclient.resource.HostedFileBuffer
<mhall119> for displaying in LD, I just need a URL that the browser can use to load the image
<mhall119> we currently use "https://edge.launchpad.net/api/beta/~%s/mugshot" % (identity)
<lifeless> the mugshot_link, but note the caveats:
<mhall119> if lp.people[request.user.username].mugshot.open() doesn't throw an exception
<lifeless>  - privacy means that we may hand out a link to our appservers which will cause a request to use, a 302 and then a request to the librarian
<lifeless> mhall119: didn't we go over this in excruciating detail a couple of weeks ago? IIRC you want the external url added to the object where possible, or something?
<leonardr> lifeless: i don't think there is a web_link for a hosted file
<mhall119> lifeless: yes, but it was Ronnie asking then, I'm playing catchup
<lifeless> leonardr: indeed - they've been using the api link directly
<lifeless> leonardr: which doesn't do what they need quite, AIUI
<lifeless> mhall119: the channel is logged I think; you might like to read that discussion first
<mhall119> lifeless: mugshot_link seems to be exactly what I want, not sure why we didn't use it in the first place
<mhall119> if no mugshot is given, will mugshot_link be None, or raise an exception?
<lifeless> mhall119: whats the value you see at the moment ?
<leonardr> mhall119: mugshot_link will always have a value, but sending a GET to that url may give you a 404
<leonardr> >>> l.people['david'].mugshot_link
<leonardr> u'https://api.launchpad.net/1.0/~david/mugshot'
<mhall119> ok..
<lifeless> yeah, its *exactly* the same thing you are using at the moment
<mhall119> leonardr: that's the problem I'm trying to fix too
<mhall119> getting text instead of an image
<mhall119> any ideas on that front?
<jml> 347 packages in canonical and lp, 788 directories
<leonardr> mhall119: what kind of program are you writing?
<lifeless> jcsackett: https://code.launchpad.net/~lifeless/launchpad/bug-717394/+merge/50110
<mhall119> leonardr: http://loco.ubuntu.com
<jcsackett> lifeless: on the queue. :-P
<leonardr> mhall119: is it javascript or server-side html generation?
<mhall119> leonardr: server-side
<mhall119> we store the URL in our database so we don't have to call LP every time
<leonardr> mhall119: i suggest issuing a request for the mugshot link. you'll either get a 404 or a redirect to the librarian, and you can store the redirected url
<leonardr> just be sure to re-check every once in a while in case they change their mugshot
<leonardr> will that work?
<mhall119> leonardr: yours doesn't give either a 404 or 302
<leonardr> mhall119: it's a 303
<mhall119> ah, ok
<mhall119> leonardr: so given that 303, is there a way to get the correct url?
<leonardr> mhall119: it should be in the Location header
<mhall119>  oh, nevermind, https://api.launchpad.net/1.0/~david/mugshot gives a 404
<jcsackett> leonardr: r=me on the first one.
 * jcsackett prays for no more ~800 line diffs.
 * jcsackett glances at lifeless's with sneaking suspicion...
<leonardr> jcsackett: sorry, i should have done the Accept header parser as a separate branch
<jcsackett> leonardr: all good. :-)
<jcsackett> i just enjoy kvetching.
<leonardr> in that case you should pray for more 800 line diff
<leonardr> s
<lifeless> jcsackett: +360 - 145
<lifeless> jcsackett: is small
<lifeless> of course, because it hits lots of contexts, thats 730 lines flattened
<jcsackett> lifeless: yeah, it's not actually that bad.
<jcsackett> i'm on a bit of cold meds, so my sense of humor is distorted.
<lifeless> jcsackett: btw I thought about making setUpTarget2 a template function, but the trade off was pretty average, and the code harder to follow so I left it as first crafted
<lifeless> jcsackett: nasal hijinks?
<jcsackett> lifeless: thus far. there's been a round of flu/bronchitis among some friends. i'm hoping mine is just a cold.
<lifeless> grah
<lifeless> be well
<lifeless> the any/all helpers are a bit awkward to work with
<lifeless> flacoste: so, optional reviews
<lifeless> flacoste: I raised the thread as we discussed
<flacoste> lifeless: yep, looks like we are good to make this the standard process
<flacoste> lifeless: simply update the wiki pages
<lifeless> flacoste: I'd like to roll it into the main process docs. What do you think?
<flacoste> lifeless: sure
<lifeless> jcsackett: so my branch has only 4 code changes: expose open_bugtasks_search, add BugTaskSearchParams.setTarget, add BugTaskSet.countBugs, use it from BugTargetView
<lifeless> matsubara: hi
<lifeless> matsubara: did my suggesetion in the bug make sense?
<matsubara> lifeless, I'm adding some tests cases so we can see if the regex change will work for us
<matsubara> lifeless, from what I tested manually the first regex worked, the second one didn't work so well
<lifeless> matsubara: I think we need test cases yes :)
<lifeless> matsubara: but broadly you need two changes:
<lifeless>  1) make the single quote string regex not take so long
<lifeless>  2) add a regex for u"" strings
<jcsackett> lifeless: dig. having made coffee i'm properly looking at yours now.
<matsubara> lifeless, cool. thanks for the feedback. once I have the mp ready, I'll ask you to take a look
<matsubara> :-)
<lifeless> sweet
 * thumper afk to take car in to get serviced, back asap
<benji> that reminds me, I need to take cdr in for service too; I wonder if the local Lisp shop is still open
<leonardr> jcsackett, can i get you to take a look at one more small revision to https://code.launchpad.net/~leonardr/lazr.restful/more-tests-for-combined-representations/+merge/50202 ?
<leonardr> specifically, revision 185
<jcsackett> leonardr: still r=me.
<leonardr> great
<lifeless> jcsackett: hi
<jcsackett> lifeless: hello.
<lifeless> would a voice call to go through the patch help? I'm only asking because I wasn't expecting it to chew up hours of your time .
<matsubara> lifeless, https://code.launchpad.net/~matsubara/oops-tools/720358-regex-fix/+merge/50225
<lifeless> jcsackett: (and I feel guilty about taking up that much time)
<jcsackett> lifeless: it hasn't chewed up hours. you pinged while i was in the midst of other reviews.
<jcsackett> and i've had some distractions. i'm finishing making notes now.
<lifeless> jcsackett: oh, phew :)
<jcsackett> lifeless: yeah, no worries. :-)
<jcsackett> now, OCR? that has chewed up hours. :-P
<LPCIBot> Yippie, build fixed!
<LPCIBot> Project db-devel build (375): FIXED in 5 hr 57 min: https://hudson.wedontsleep.org/job/db-devel/375/
<jcsackett> lifeless: r=me.
<jcsackett> sinzui, there's a stack of reviews i've thrown in that i need follow up on. not sure if you've seen them or not already.
 * jcsackett moves on to last branch he was pinged for.
<thumper> leonardr: want to approve https://code.launchpad.net/~thumper/launchpad/recipe-inline-edit-recipe-text/+merge/49585 ?
<lifeless> oh, rightm you're being mentored
<lifeless> sinzui: -yo-
<jcsackett> lifeless: yes, i'm the diet coke of reviewers right now. :-)
<lifeless> that reminds me of cola/coke/pepsi
<lifeless> http://piumarta.com/software/cola/
<lifeless> jcsackett: ^
<matsubara> lifeless, thanks!
<lifeless> de nada
<jcsackett> lifeless: that sounds rather ambitious.
<lifeless> it is
<lifeless> some interesting shit
<dobey> heh, that is not what i think of when i hear pepsi/jolt in reference to a piece of code
<lifeless> http://piumarta.com/software/peg/ is also interesting to compiler geeks
<jcsackett> peg i have heard of.
<jcsackett> never looked at, but heard of.
<sinzui> lifeless: jcsackett: I am back. I can review again
<lifeless> sinzui: \o/
 * StevenK grumbles at buildbot
<lifeless> sinzui: I want to have a ui discussion about bugtask:+index with someone
<lifeless> sinzui: would you be suitable, and if so, when would be good for you?
<thumper> In mumble, if I use "push to talk" what do I push?
<lifeless> there is a config option to choose a key
<sinzui> lifeless: in a few minutes. I need to get a drink and hope that me second computer prefers an older kernel
<lifeless> sinzui: kk
<thumper> StevenK: can you hear me?
<StevenK> thumper: Didn't have headphones on, try again?
<thumper> leonardr: you around?
<leonardr> yup
<thumper> mumble?
<StevenK> thumper: Configure -> Settings -> Shortcuts
<lifeless> sinzui: shout when you are ready
<thumper> sinzui: can you look at jcsackett's reviews of leonardr's branches?
<thumper> sinzui: I want them landed asap :-)
<StevenK> leonardr: lib/canonical/lazr/security.py and lib/canonical/lazr/doc/checker-utilities.txt is the test
<thumper> sinzui: if you can't, I'll be happy to mentor the reviews
<sinzui> thumper: I am reading it now
<thumper> sinzui: awesome, thanks
<thumper> I'm so freaking excited about this lazr.restful change
<thumper> sinzui: got a minute?
<thumper> sinzui: for mumble?
<lifeless> thumper: I'm queued already for sinzui :)
<thumper> lifeless: :(
<lifeless> thumper: ^^
<thumper> lifeless: my minute with sinzui is talking through the background of the changes that he is currently reviewing
<thumper> lifeless: so it is more timely :)
<poolie> hi thumper
<thumper> hi poolie
<poolie> hi lifeless, your statement about pithiness was to pithy to be understandable :)
<thumper> poolie: we should have a chat sometime
<poolie> how about 60m from now?
<LPCIBot> Project devel build (451): FAILURE in 5 hr 26 min: https://hudson.wedontsleep.org/job/devel/451/
<LPCIBot> Launchpad Patch Queue Manager: [r=jtv][bug=715325] Don't return builds for disabled archives using
<LPCIBot> get{Pending, }Builds() on recipes.
<thumper> poolie: yeah, that should work
<lifeless> poolie: :)
<leonardr> thumper, i don't think test_recipe_text tests anything? to test that you can edit the recipe text, you would have to call lp_save()
<thumper> do we have to call lp_save for mutators?
<leonardr> thumper: you don't have to call lp_save() when you invoke a named operation
<leonardr> but the purpose of a mutator is to hide a named operation so that it looks like normal field access
<thumper> leonardr: but you do for mutators?
<leonardr> and in that case you do need to call lp_save()
<thumper> leonardr: if you have several mutators that are being saved, which order are they called?
<leonardr> ie. on the client, side, there is no 'mutator'
 * thumper adds a lp_save()
<leonardr> it's just that recipe_text is editable
<leonardr> r=me apart from that
<thumper> ok
<leonardr> thumper: if you have several mutators, i believe they're called in the order in which the fields were defined in the annotated interface
<thumper> ok
<thumper> as in the order the fields that are being mutated were defined?
<leonardr> yeah
<thumper> ok
<thumper> that makes sense
<leonardr> that gives you the most control
 * thumper nods
<StevenK> leonardr: As a stupid question, how do I run the lazr.restful tests? There's no makefile that I can see.
<leonardr> StevenK: python bootstrap.py; bin/buildout; bin/test
<thumper> StevenK: it can take a while the first time
<thumper> StevenK: it is worthwhile setting a cache for buildout
<leonardr> yeah, put this in ~/.buildout/default.cfg
<leonardr> [buildout]
<leonardr> eggs-directory=/home/[username]/.buildout/eggs
<leonardr> download-cache=/home/[username]/.buildout/download-cache
<leonardr> and create those directories
<leonardr> first buildout will take a few minutes, subsequent will be instantaneous
<thumper> leonardr: I've just privmsg'ed that to StevenK too :)
<leonardr> k
<StevenK> I didn't even have setuptools installed, so I'm still on the first step :-)
 * leonardr leaving but will be back to do a lazr.restful release
<jam> jcsackett: could I get you to look at https://code.launchpad.net/~jameinel/launchpad/lp-forking-serve-cleaner-childre/+merge/50031
<jam> hmm... looks like I put the comment in the wrong spot, just a sec
<jam> pushed an update
<lifeless> sinzui: ping
<sinzui> hi lifeless
<lifeless> sinzui: is now good ?
<sinzui> May I have 10 more minutes to finish a review?
<lifeless> of course
<jcsackett> jam: everything where it should be on that MP? I can look at it now.
<lifeless> matsubara-afk: lp-oops is timing out for me now
<lifeless> matsubara-afk: on OOPS-1716ED446
<sinzui> lifeless: I am available now. irc, mumble, or skype
<lifeless> skype works best I think
<StevenK> leonardr: https://code.launchpad.net/~stevenk/lazr.restful/move-test/+merge/50239 If you're still around
<jml> turns out my branch to rename zeca fails with an obscure error in an at-first-glance unrelated test
<jml> who'd have thunk it!
 * jml fixes
<thumper> jml: I'd a thunk it
<lifeless> sinzui: https://bugs.launchpad.net/ubuntu/+bug/1
<_mup_> Bug #1: Microsoft has a majority market share <iso-testing> <ubuntu> <Clubdistro:Confirmed> <Computer Science Ubuntu:Invalid by compscibuntu-bugs> <EasyPeasy Overview:Invalid by ramvi> <GNOME Screensaver:Won't Fix> <Ichthux:Invalid by raphink> <JAK LINUX:Invalid> <The Linux OS Project:In Progress> <metacity:In Progress> <OpenOffice:In Progress by lh-maviya> <Tabuntu:Invalid by tinarussell> <Tivion:Invalid by shakaran> <Tv-Player:New> <Ubuntu:I
<jam> jcsackett: everything should be ready
<jcsackett> jam: i will look at it soon.
<StevenK> wgrant: Did you want to close all of the bugs referenced by the rollout?
<jcsackett> sinzui: fire alarm just went off in my apartment building. hopefully we'll be back in before standup.
<jam> jcsackett: this is another one which should be ready for review https://code.launchpad.net/~jameinel/launchpad/twisted-close-on-failure/+merge/50067
<wgrant> StevenK: Done.
<jml> getPublisher meaningfully changes state. How interesting.
<thumper> StevenK, wgrant: does a distro "always" have a current distroseries?
<wgrant> No.
<thumper> or, I guess, does "ubuntu" always have one?
<wgrant> Wait.
<wgrant> With status CURRENT?
<wgrant> Or do you mean a development focus?
<thumper> wgrant: whatever I need for "maverick" now
<thumper> wgrant: I'm wanting to provide a reasonable default for daily builds
<thumper> so the default initially will be the current distroseries
<thumper> development focus right now is natty
<wgrant> There will almost always be one, but it will be missing occasionally.
<wgrant> And for new distros.
<wgrant> Which are coming soon.
<thumper> wgrant: if a distro has at least one distro series, is the distro guaranteed to have a development focus?
<thumper> also... why don't we force distros to have at least one series like projects?
<wgrant> Because distroseries are special and do magical Soyuz things.
<wgrant> Although it's less bad now.
<jml> did anyone debug that error with ec2 things not having any output in 600s?
<wgrant> You should use Distribution.currentseries.
<jcsackett> sinzui: i'm back. false alarm, evidently.
<wgrant> thumper: .currentseries implements the development focus logic, and will always return something if there are any series.
 * thumper looks at the distro model code
<thumper> wgrant: that would give recipes a default of making for natty now not maverick
<thumper> ...
<wgrant> I know.
<StevenK> lifeless: See new comment on RT#43352
<_mup_> Bug #43352: ipp jobs not purged; purging causes 100% cpu usage <cupsys (Ubuntu):Fix Released> <gnome-cups-manager (Ubuntu):Invalid> < https://launchpad.net/bugs/43352 >
<thumper> I wonder if that is good enough
<lifeless> StevenK: otp
<wallyworld> thumper: ping me when you are free? do you want to discuss the "Build now" ui review points?
<thumper> wallyworld: I was just going to have a quick chat with poolie
<thumper> wallyworld: after that?
<wallyworld> thumper: no hurry. that's why i mentioned when you are free :-)
<jcsackett> jam: what confidence do we have that this try/finally all works, given there's no test coverage?
<thumper> poolie: skype or mumble?
<thumper> wgrant: when is a distroseries FROZEN ?
<StevenK> thumper: Just before release.
* jcsackett changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | firefighting: - | On call reviewer: - | https://code.launchpad.net/launchpad-project/+activereviews
<wgrant> Around release time, or when it has just been created and is not yet initialised.
 * thumper nods
<thumper> ok
<wgrant> Embedding all this in a single status is a little insane, but that's how it is :(
<jml> wgrant: is https://bugs.launchpad.net/launchpad/+bug/661931 still an issue?
<_mup_> Bug #661931: make check sometimes fails on EC2 without test failures <lp-foundations> <Launchpad itself:Triaged> < https://launchpad.net/bugs/661931 >
<wgrant> jml: I don't think so.
<wgrant> All that remains is the Windmill hang, AIUI.
<jml> For which I just filed bug 720998
<_mup_> Bug #720998: ec2 test times out <build-infrastructure> <Launchpad itself:Triaged> < https://launchpad.net/bugs/720998 >
<thumper> wallyworld: join mumble and lets talk about your reviews
<thumper> wallyworld: poolie has now been bumped to after lunch
<wgrant> jml: Does getPublisher do much beyond creating directories?
<jml> wgrant: well, presumably it gets a publisher
<wgrant> s/do much/do much evil/
<jml> wgrant: oh right, nothing obvious.
<jml> wgrant: I deleted a line in a test that was "unused_variable = getPublisher(...)"
<wgrant> Hah.
<wgrant> That all sort of sucks. I rewrote the underlying config stuff last year, but didn't end up making it all the way to the top.
<jml> because I guess functions that start with 'get' are effectively read-only
<jml> wgrant: it's all about incremental improvement
<wallyworld> thumper: was otp, ok to go now
<jml> I got this error while running sphinx.main() in the Launchpad test suite on EC2:
<jml> actual = "Making output directory...\nException IndexError: IndexError('list index out of range',) in <bound method _LockWarner.__del__ of <bzrlib.lockable_files._LockWarner object at 0x10f65d50>> ignored\n"
<jml> uhh, the error is the exception
<jml> hmm...
<jml> that gives me a though
<jml> .t
<lifeless> wgrant: hi
<lifeless> wgrant: I was considering another deploy, but danilos patch asn't qa'd; wondering if thats something you might like to do?
<lifeless> StevenK: I see, cool.
<sinzui> lifeless: ie6=0.5%, ie7=1%
<lifeless> nice
<lifeless> so we should ask our stakeholders / commercial customers only
<wgrant> lifeless: Sure.
<wgrant> lifeless: Looks good. I shall request the deploy.
<lifeless> wgrant: -woot-
<wgrant> This means we have deployed every day this week, I think.
<sinzui> 16% of our visitors are using mobile devices
<wgrant> Ah, no, missed Tuesday.
<wgrant> But I think timezones meant they were close anyway.
<sinzui> I bet they hate bug 1
<_mup_> Bug #1: Microsoft has a majority market share <iso-testing> <ubuntu> <Clubdistro:Confirmed> <Computer Science Ubuntu:Invalid by compscibuntu-bugs> <EasyPeasy Overview:Invalid by ramvi> <GNOME Screensaver:Won't Fix> <Ichthux:Invalid by raphink> <JAK LINUX:Invalid> <The Linux OS Project:In Progress> <metacity:In Progress> <OpenOffice:In Progress by lh-maviya> <Tabuntu:Invalid by tinarussell> <Tivion:Invalid by shakaran> <Tv-Player:New> <Ubuntu:I
<wgrant> lifeless: Can we remove the security deployment section from LPS?
<wgrant> production-stable seems to be pretty dead.
<lifeless> wgrant: raise it for discussion with losas
<lifeless> I wanted to initially but there was lots of concern about when it would be needed, so it stayed as a compromise
<wgrant> k
<wgrant> I am at least tempted to move it below the stable section.
<lifeless> if it goes, a bunch of processes can be cleaned up too
<huwshimi> sinzui: Wow, the number of mobile users is way higher than I expected it
<jam> jcsackett: sorry about the delay, was picking up my son. I have run the specific code quite thoroughly, and there are some tests that already cover it.
<jam> It is a little bit hard to inject specific failures into that exact point. If you feel it is critical, we can try to figure something out
<jam> but I did do a lot of manual testing.
<jcsackett> jam: ah, in your MP you said there were no tests; i took that to mean no testing. :-)
<jam> no new tests covering what changed
<jcsackett> dig.
<jcsackett> jam: r=me, but i'm mentored, so sinzui will have to follow up, and it is EOD in our tz. he may still follow up today, or tomorrow morning.
<jam> k, thanks jcsackett
<jcsackett> jam: np. :-)
<wgrant> lifeless: So, are you OK with https://code.launchpad.net/~wgrant/launchpad/bug-708999-ff-docs/+merge/49758?
<wgrant> It seems rather controversial.
<wgrant> But it looks like improvements on it are even more controversial.
<lifeless> otp
<wgrant> Some people's interpretation of HTTP is pretty creative.
#launchpad-dev 2011-02-18
<wgrant> jml: Still around?
<leonardr> sinzui, thanks for your review. did you see the follow up? https://code.launchpad.net/~leonardr/lazr.restful/more-tests-for-combined-representations/+merge/50202
<jml> wgrant: yes
<wgrant> jml: Which version of bzr-pqm do you have installed?
<jml> wgrant: 1.4.0dev
<jml> why?
<wgrant> I want the exact package version. Because your zeca move added [ui=none], but I removed that two weeks ago.
<jml> 1.4.0~bzr75-0ubuntu1-launchpad1~maverick1
<wgrant> (it ended up with two sets of tags: one without [ui=none] from ec2, and another from presumably lp-land)
<jml> oh
<wgrant> Hmm. That is old.
<wgrant> 1.4.0~bzr77-0ubuntu1-launchpad1~maverick1 is the latest version.
<wgrant> You've not upgraded lately?
<jml> also, I appear to have something symlinked in my plugins
<jml> wgrant: just today
<jml> wgrant: maybe when I upgraded to natty I didn't re-enable my sources.list for launchpad?
<wgrant> That could do it.
<jml> should I use 'natty' or 'maverick' for that PPA?
<wgrant> natty
<wgrant> We support that now.
<wgrant> Although the test suite doesn't entirely work, I don't think.
<jml> not surprised.
<leonardr> thumper, maybe you want to take the (short) follow-up branch, that's all that's holding back a lazr.restful release
<lifeless> wgrant: so
<lifeless> let me look
<lifeless> wgrant: I'm not entirely happy with the memcache change; it seems ugly to me
<lifeless> wgrant: perhaps we should talk about it ?
<thumper> leonardr: sure, email me with details
<thumper> I'm just moving back home from the shared office space
<lifeless> matsubara-afk: I can't access the oops in https://bugs.launchpad.net/launchpad/+bug/636158
<_mup_> Bug #636158: BugTask:+index times out with many bug tasks/nominations (eg. Bug #1) <lp-bugs> <pg83> <timeout> <Launchpad itself:Triaged> < https://launchpad.net/bugs/636158 >
<lifeless> matsubara-afk: spm has updated and reenabled the lp-oops db updater
<wallyworld> thumper: you back online?
<lifeless> woo
<lifeless> http://webnumbr.com/launchpad-timeout-bugs#
<thumper> wallyworld: am now, but checking on something
<wallyworld> thumper: ok. ping me when you are done
 * thumper nods
<matsubara-afk> lifeless, spm: thanks. looks like update_db is working fine. instructions on how to update are here: https://dev.launchpad.net/Foundations/QA/OopsToolsSetup
<matsubara-afk> lifeless, not sure why OOPS-1716ED446 is not working though
<thumper> wallyworld: now?
<wallyworld> ok
<matsubara-afk> lifeless, only thing in the apache log is: [Fri Feb 18 01:09:25 2011] [error] [client 122.63.10.108] Premature end of script headers: oopstools.wsgi, referer: https://bugs.launchpad.net/launchpad/+bug/636158
<lifeless> matsubara-afk: thanks; also there are uncommitted changes in the tree; we should fix that to be automatically deployable (like lp itself)
<_mup_> Bug #636158: BugTask:+index times out with many bug tasks/nominations (eg. Bug #1) <lp-bugs> <pg83> <timeout> <Launchpad itself:Triaged> < https://launchpad.net/bugs/636158 >
<matsubara-afk> lifeless, I reverted some debugging code I had left there. the other uncommitted things are config changes which shouldn't be in the vcs (e.g. passwords) and some minor stuff that I have bugs open for but didn't get to it yet
<lifeless> matsubara-afk: so the password can be switched to ident
<lifeless> matsubara-afk: and other config stuff should either be a separate tree if confidential, or just committed in some fashion
<lifeless> matsubara-afk: I say should because the benefits to automation are immense
<matsubara-afk> lifeless, right. I think if we can switch the db password to ident then the rest of it can be committed
<matsubara-afk> the basic auth stuff is deprecated and I think I can remove that too
<lifeless> cool
<wgrant> lifeless: What is particularly ugly about the memcache change?
<huwshimi> It appears the difference in URL between viewing a branch in loggerhead and viewing the branch details in LP is just a subdomain change (https://code.launchpad.net/~huwshimi/launchpad/tag-label-687546 and https://bazaar.launchpad.net/~huwshimi/launchpad/tag-label-687546). I can't find a situation where this is not the case. Can anyone verify this or know how I can go about doing so?
<wgrant> huwshimi: The branch path is always the same.
<huwshimi> wgrant: Awesome. No chance that this ever differs?
<wgrant> Obviously the subpaths can be different.
<wgrant> No.
<wgrant> But lp-loggerhead should already have a link back to code.launchpad.net... can you reuse that?
<huwshimi> wgrant: It does?
<wgrant> huwshimi: http://bazaar.launchpad.net/~launchpad-pqm/launchpad/devel/changes, click the branch name in the top left.
<huwshimi> wgrant: Ah right. I missed that as it does not work locally
<huwshimi> wgrant: Thanks for that
<wgrant> lifeless: With that BugTask:+index query... isn't it basically just SELECT blah blah FROM Message JOIN BugMessage ON BugMessage.message=message.id WHERE BugMessage.bug = %s AND BugMessage.index = 0"
<wgrant> ?
<wgrant> The query used there seems hugely excessive.
<lifeless> wgrant: its using the indexed_messages property
<lifeless> yes it can be made much better
<wgrant> Ahh.
<wgrant> I noticed it was timing out on that a lot on qastaging last night. But couldn't check the OOPSes to see how bad it really was.
<lifeless> its better than it was
<lifeless> its not shipping them all over the wire
<lifeless> the old one was a 10 second query ;)
<wgrant> Yep.
<lifeless> should be fixed now, just checking I haven't broken anything
<wgrant> Excellent.
<wgrant> What are message parents used for, anyway?
<lifeless> FIIK
<wgrant> Heh.
<lifeless> We model full email threading
<wgrant> I know.
<lifeless> we just don't show it anywhere
<wgrant> And MPs used to use it.
<wgrant> But not any more.
<wgrant> It's also used in bugmail, I guess, but not in the UI anywhere AFAIK.
<lifeless> https://code.launchpad.net/~lifeless/launchpad/bug-721056/+merge/50266
<lifeless> thumper: ^
<lifeless> (diff generating now)
<wgrant> lifeless: Drop about 6 seconds off an already 10 second page?
<wgrant> Seriously?
<lifeless> yes
<wgrant> Wow.
<lifeless> 5.7 specifically
<wgrant> https://devpad.canonical.com/~lpqateam/qa_reports/deployment-stable.html needs a "Do it" button
<wgrant> Given that it looks like we are able to manage 1-2 rollouts a day now...
<wgrant> lifeless: Do you want to discuss my FF branch?
<wgrant> Ah, you replied.
<wgrant> That was easier than expected. Thanks.
<lifeless> I'd previously said it was fine I think
<lifeless> it got sidetracked and confused
<lifeless> wgrant: the check for Contains is still weird IMO
<lifeless> but I haven't read enough of the rest of the file
<wgrant> lifeless: It is weird, but it's the same weird as the two other tests.
<wgrant> Consistent weird is better than inconsistent weird.
<StevenK> wgrant: I think it would be awesome if the "Do it" button would figure out the current losa and physically tap them on the shoulder.
<lifeless> bonus points for greying out about now in the week
<lifeless> spm: hi
<StevenK> lifeless: Your diff for bug-721056 looks odd
<lifeless> spm: last interrupt, I semi-promise.
<StevenK> lifeless: You have interface changes, but no model changes.
<wgrant> StevenK: That's fine.
<lifeless> StevenK: there are no model changes
<wgrant> Although I'd reject that MP, I think.
<wgrant> Exposing an _method?
<StevenK> Haha
<lifeless> consenting adults
<StevenK> And showing a _ in public? Shameful!
<lifeless> what really wants to happen is to push the description changed check into Bug
<lifeless> but I want to make the damn thing fast first
<lifeless> and then make it pretty
<wgrant> lifeless: Nothing uses indexed_messages any more.
<wgrant> Apart from one test.
<lifeless> wgrant: and the API
<wgrant> Bah.
<wgrant> Kill it.
<wgrant> With screw-compatibility-i-want-performance fire.
<lifeless> do you know how I know the API uses it ?
<wgrant> Because it times out hundreds of times a day?
<lifeless> ... we had timeouts on Bug.messages in the first week I was ta
<lifeless> wgrant: not anymore :)
<wgrant> Oh, right, I remember that.
<lifeless> wgrant: it probably will start timing out when we hit 12 second timeouts
<StevenK> lifeless: I do note you've stopped pasting the top 10 timeouts.
<lifeless> StevenK: not at all
<lifeless> I pasted them yesterday
<lifeless> today the oops processor was off overnight so the stats would be meaningless
<wgrant> I don't think any got through.
<StevenK> Right, but how many lines actually made it to the channel?
<lifeless> StevenK: the first 3, which is what I pasted
<StevenK> I think Freenode has wised up and disabled paste for you :-P
<lifeless> possibly
<lifeless> the flood rules are positively fascist
<StevenK> The reasoning is clear. Use a pastebin!
<wgrant> As fascist as you on OOPSes? :P
<StevenK> Haha. Maybe Freenode *are* LP users ...
<lifeless> wgrant: not quite; it lets some through after all
<wgrant> Haha
<wgrant> wallyworld: Bug #720474 looks very similar to bug #687623.
<_mup_> Bug #720474: Need "Build now" capability for daily recipe builds <recipe> <Launchpad itself:In Progress by wallyworld> < https://launchpad.net/bugs/720474 >
<_mup_> Bug #687623: No ability to manually trigger a build 'like the daily build system' <lp-code> <recipe> <Launchpad itself:Triaged> < https://launchpad.net/bugs/687623 >
<wallyworld> wgrant: so it does
 * wallyworld has a closer look
<wgrant> I think we need to SourcePackageRecipe:+index'ify all of LP.
<lifeless> wgrant: what do you mean?
<wgrant> lifeless: That's a new verb that means "completely AJAXify"
<lifeless> ah
<lifeless> that would be nice; we need to decide what to do with commercial/stakeholder ie7 users
<wallyworld> wgrant: i'll mark 720474 as a dup of 687623. the wording is slightly different but the intent appears to be the same
<wgrant> wallyworld: I'd go the other way, but yes, they have the same intent.
<wallyworld> wgrant: i thought we were supposed to mark the higher # and the dup?
<wallyworld> s/and/as
<wgrant> wallyworld: Some say that.
<thumper> bah
<wgrant> But the most informative, searchable one should win, IMO.
<thumper> damn it
<wallyworld> wgrant: yes, i see that point too. but both will be searchable even if one is a dup, no?
<wgrant> Hahahah
<wgrant> No.
<lifeless> yes
<wallyworld> thumper: what's wrong
<lifeless> wgrant: the dupe finder searches dupes
<thumper> making description optional requires a db patch :(
<thumper> as there's a NOT NULL constraint
<thumper> bah humbig
<thumper> bug
<wallyworld> thumper: patch away!! i'm not RM anymore :-D
<lifeless> we should do that in the main bug search, once its performance is tolerable
<wgrant> lifeless: ah, true, for the dupefinder.
<wgrant> Indeed.
<lifeless> wallyworld: if the older bug is less useful, you can either make the newer a master, or improve the older ones subject
<lifeless> if you're not working on a bug, taking the older as master is a good idea because of the slight emphasis on older-first bugfixing within a bucket
<wallyworld> lifeless: the older one has more background info but the newer one is more succinct. i'll update the older one and dup the newer
<lifeless> sinzui: so, I found a another low hanging fruit for bugtask:index, will reevaluate when that has landed and been deployed
<lifeless> huwshimi: sinzui and I talked over bugtask:+index today, like you did the day before. I have a raft of bugs I'm going to file once I have some data on the relative performance impact of the various issues
<StevenK> lifeless: I wanted to hear your thoughts on bugmessage vs bugcomment when you have a spare cycle?
<lifeless> wallyworld: coolio
<lifeless> StevenK: hey, so - what in particular
<huwshimi> lifeless: That's great. sinzui did mention it and it sounds like we're thinking along the same lines.
<huwshimi> lifeless: Well at least that there's stuff to be dealt with
<lifeless> huwshimi: great minds
<huwshimi> lifeless: Feel free to subscribe me to those bugs or something once they're created
<lifeless> sure
<lifeless> probably be tuesday before I file them given latency to gather the needed data
<huwshimi> lifeless: In particular the subscribers list is going to get a huge hit once the yellow squad's work gets released
<lifeless> thats already out of the performance critical path
<wgrant> That's in a separate request, but it's still not good.
<lifeless> wgrant: actually
<lifeless> check BugTaskView
<wgrant> lifeless: It calculates them anyway?
<lifeless> yes
<wgrant> YAY
<lifeless> so there is stuff that needs addressing
<lifeless> a lot of detail to cover
<lifeless> but things like
<lifeless>  - have a deleted bugtask state
<lifeless> (don't show those)
<lifeless>  - don't show declined nominations
<lifeless> accordian the activities as well as messages
<lifeless> schema changes to let messages-to-show be queried from the db rather than done in python
 * wallyworld wonders why stuff wrapped by <noscript> doesn't show up if i disable javascript using the noscripts extension?
<lifeless> wallyworld: try disabling javascript entirely instead
<wallyworld> that's the hard way! so much easier to clock an icon in the location bar
<pjdc> wallyworld: iirc noscript has an option on whether to show <noscript> elements
<wallyworld> pjdc: thanks. i'll have a closer look. first time i've used noscript.
<lifeless> wallyworld: so I was basing that on my undesrstanding about what noscript does
<pjdc> if this is your first trip into the options window, try not to scream.
<lifeless> wallyworld: if you haven't tried /actually/ turning it off, you might want to try that :P
<wallyworld> i've never seen a need for noscript apert from testing launchpad :-)
<wallyworld> lifeless: jeez, i don;t look that stupid do i? i did turn js off using noscript and i know it's off cause the other ajax links have been disabled
<wallyworld> :-)
<lifeless> wallyworld: well AIUI it doesn't disable the interpreter
<lifeless> wallyworld: it filters the content
<lifeless> wallyworld: the result being that as far as the browser is concerned javascript is still enabled
<wallyworld> lifeless: yeah, i think you're right. i'll look for the noscript option to display noscript tags
<StevenK> wgrant: Is getting buildd-manager and a builder working on a development still a path of pain, misery and suffering?
<wgrant> StevenK: Yes. But it is fairly short, well-documented path of pain, misery and suffering.
<wgrant> Plus there are scripts to do most of it.
<wgrant> Unless jml broke them this morning.
<wgrant> Let me just check if the docs are up to date.
<wgrant> StevenK: https://dev.launchpad.net/Soyuz/HowToUseSoyuzLocally looks OK.
<wgrant> Although installing launchpad-buildd outside a VM is somewhere between unwise and idiotic.
<StevenK> Clearly
 * StevenK ponders ec2 demo
<wgrant> That works.
<wgrant> I've done it there before.  But it's easy enough to run lp-buildd in a VM.
<StevenK> You assume much about my setup :-P
<wgrant> Hah.
<wgrant> Grngggh.
<wgrant> login_as(someteam) creates a new member and adds it to the team.
<wgrant> Which sends email :(
<StevenK> wgrant: The wiki pages references edge :-P
<wgrant> StevenK: Yes, but it'll still work, even faster!
 * wgrant invokes a lifeless-shield.
<StevenK> Haha
<poolie> someone asked in dallas about merging #launchpad and -dev
<wgrant> Also, soyuz-sampledata-setup in devel doesn't do natty.
<poolie> seems like a good idea...
<wgrant> poolie: Until you have three dev conversations and three user conversations at once.
<wgrant> Which doesn't happen in APAC, but does in AMEU.
<spm> lifeless: yo
<poolie> i wondered about that
<wgrant> Anyone want to review https://code.launchpad.net/~wgrant/launchpad/silence-of-the-prober-ii/+merge/50273?
<StevenK> wgrant: I can't see a definition of RedirectToDifferentFile anywhere? Did it already exist?
<lifeless> spm: hi
<lifeless> spm: uhm, staging is borked
<lifeless> spm: I think its the NULL on bugmessage
<lifeless> spm: a restore from prod will fix, and its friday so we're due one anyhow
<lifeless> StevenK: edit the wiki page
<lifeless> (to remove edge !)
<wgrant> StevenK: Yes, it already existed.
<lifeless> wgrant: did you end up following up the issue with the qastaging restore ?
<wgrant> StevenK: It was considered a very strange failure.
<wgrant> lifeless: Yes, it just needed the DB swapped. Is done.
<wgrant> StevenK: But now some mirrors are showing their love for HTTP by redirecting instead of 404ing, because who needs good response codes?
<StevenK> lifeless, thumper: O hai, can one of you can mentor my review of https://code.launchpad.net/~wgrant/launchpad/silence-of-the-prober-ii/+merge/50273?
<StevenK> DiscoveryFailure: Error fetching XRDS document: <urlopen error [Errno 1] _ssl.c:480: error:140770FC:SSL routines:SSL23_GET_SERVER_HELLO:unknown protocol><br />
 * StevenK pouts at his ec2 demo
<wgrant> StevenK: Restart Apache.
<StevenK> wgrant: Graceful or bullet-in-the-head?
<wgrant> Alternatively the Apache config mangling may be broken now.
<wgrant> StevenK: The latter.
<wgrant> You used to need to do that after rf-setup, but it now restarts it itself.
<thumper> StevenK: done
<wgrant> Thanks.
<StevenK> wgrant: Welcome
<StevenK> thumper: And thanks
<wgrant> lifeless: aaaaaaaa
<wgrant> Branches without owners?
<thumper> what?
<StevenK> wgrant: Restarting apache results in the same traceback.
<wgrant> thumper: It has just been suggested that official package branches shouldn't have owners at all.
<thumper> no
<thumper> no
<thumper> no
<thumper> no
<thumper> and NO
<wgrant> StevenK: Config mangling on testopenid.dev:443 is broken.
<StevenK> $ grep -c 'testopenid.dev' /etc/apache2/sites-available/local-launchpad
<StevenK> 0
<StevenK> :-(
<StevenK> wgrant: However, I can't find an apache config section locally either.
<wgrant> StevenK: It'll fall through to the default *:443 vhost.
<wgrant> Which happens to proxy to the appserver anyway.
<StevenK> There is no default *:443 vhost
<wgrant> It was probably mangled.
<poolie> lifeless, hi, still here?
<wgrant> Does Zopeless serve any purpose these days?
<wgrant> It isn't very Zopeless.
<LPCIBot> Yippie, build fixed!
<LPCIBot> Project devel build (452): FIXED in 5 hr 40 min: https://hudson.wedontsleep.org/job/devel/452/
<lifeless> poolie: kindof
<wgrant> And the only difference I know of is that it sometimes has different mail delivery techniques.
<huwshimi> Have a good friday/weekend everyone.
<wgrant> Night huwshimi.
<poolie> night huw
<lifeless> wgrant: 'owner', not Owner
<poolie> is sending a diff on bug description changes really the most useful thing?
<poolie> they often seem pretty unreadable to me
<LPCIBot> Project db-devel build (377): FAILURE in 4 hr 56 min: https://hudson.wedontsleep.org/job/db-devel/377/
<LPCIBot> Launchpad Patch Queue Manager: [r=abentley,
<LPCIBot> stub][bug=164196] makes sure that the omitted notification objects
<LPCIBot> left over from duplicate actions are marked as processed so
<LPCIBot> that they are not perpetually considered for subsequent
<LPCIBot> notification cronscript runs.
<wgrant> poolie: I'm pretty sure the diff is buggy.
<poolie> a wdiff might be better
<poolie> or just sending the whole new thing if the change is substantial
<wgrant> Possibly.
<lifeless> I think showing the old is pretty useful
<lifeless> particularly now we don't mail  ones own changes by default
<BjornT_> poolie: showing a diff helps showing what the actual change was. a better diff algorithm would be good, though, since the current one is quite naive.
<lifeless> I hear bzr has a good one
<poolie> well
<poolie> i think i'm more often interested in just knowing what the description is
<poolie> perhaps a smarter diff would be better
<lifeless> poolie: I think there is an option you can turn on
<lifeless> where lp will give you the description on each mail
<BjornT_> poolie: i find having to re-read the whole description for small changes is a waste of time, especially if i want to know what got added/changed. i don't want to create a diff manually in my head
<poolie> BjornT_, it's a tough tradeoff
<poolie> neither feels just right
<poolie> lifeless, but that's _every_ mail which is too much
<poolie> perhaps html text with redlining would be best
<poolie> but that sounds a bit hard
<lifeless> difftastic!
<poolie> maybe a tweaked version of that
<poolie> i feel like ideally it would show the whole new description with highlighted/struck out changes
<poolie> there are probably still failure modes for this but it would be better
<poolie> it's not really a top priority
<poolie> i just got an especially unreadable one
<poolie> o/ bjornt btw
<lifeless> poolie: the obvious failure mode is 100 line descriptions
<lifeless> poolie: like most of my timeout ones :)
<poolie> mm
<poolie> i think 100 line descriptions aren't inherently a failure mode
<poolie> i was thinking more of the 'oh i'll sync on paragraph breaks' type thing
<poolie> if you're editing a huge description all the time it may suck
<poolie> but i'm not sure it will suck any more than at present
<lifeless> we have a general concept that we show deltas
<lifeless> I think that works pretty well
<BjornT_> poolie: showing highlighted/struck out changes would be good, but quite hard to do with text/plain. maybe an option to get html mail would work for those who don't mind.
<lifeless> whatever we do, keeping that seems reasonable to me
<poolie> perhaps a wdiff of the relevant paragraphs would be the best of both worlds
<lifeless> BjornT_: what do you think of the idea of having a bugtask status 'deleted' which would be hidden from search by default, and not shown in the default bugtask:+index view
<lifeless> BjornT_: the difference between it and invalid/wontfix would be the hiding of it in bugtask:+index
<lifeless> alternatively, we could hide invalid tasks by default in that view
<wgrant> Also Deleted tasks don't count for strucsubs.
<BjornT_> lifeless: yeah, i was going to suggest simply hiding invalid tasks. that would make that status more useful, and make it more obvious it's not considered a bug
<lifeless> wgrant: same thing can be done for invalid
<lifeless> BjornT_: indeed, invalid is pretty unambitious at the momemt
<wgrant> lifeless: That is difficult, because then invalidating the single task on a bug would remove all its subscribers, leaving the reporter's protests to go nowhere.
<lifeless> wgrant: maybe this makes opinion more useful
<lifeless> wontfix -> its a defect but won't be fixed here. Useful for closed source projects and for open source projects with frozen contexts - e.g. old releases of bzr.
<lifeless> opinion -> dispute over it being a defect
<lifeless> invalid -> truely not a bug and no dispute expected
<poolie> that's not quite how i would describe wontfix/invalid, but anyhow
<poolie> perhaps you could just not allow deleting the last task?
<lifeless> poolie: we're discussing a possible change :) redefinitions R us
<poolie> istm the main case for this as i understand it is to delete incorrectly added secondary tasks
<lifeless> if a spammer reports a bug
<lifeless> you want to get rid of it entirely
<lifeless> thats not all that common, but I don't know that we should special case it
<BjornT_> poolie: not sure about wdiff. it's not too bad for minor changes, but for bigger changes it seems quite unreadable
<StevenK> lifeless: Have you ever seen a spammer report a bug? I've only seen them add comments.
<poolie> well, there could be a policy that only ~admin or whatever can delete the last task
<adeuring> good morning
<mrevell> Hi
<stub> jtv: Do you have a branch for the BranchRevision.id dropping that I can pick up? Or where you still following up on that?
* adeuring changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | firefighting: - | On call reviewer: adeuring | https://code.launchpad.net/launchpad-project/+activereviews
<jml> man
<jml> this _LockWarner thing plaguing anyone else?
<danilos> wgrant, thanks for QAing 718809
<jml> Has anyone had ec2 failures in the last few hours that mention _LockWarner?
<jtv> stub: don't have that done yet, no.  An unbroken series of crises took over.
<stub> jtv: Sure. But if you have a branch, I can take it over.
<stub> Or I can start a new branch :-)
<jtv> stub: the branch I have doesn't have much useful stuff in itâ¦ hang on, I'll dig it up.
<jtv> stub: https://code.launchpad.net/~jtv/launchpad/drop-branchrevision-id
<stub> Ta
<jtv> I think the main thing was a view that we can also drop.
<jtv> Look how far we've comeâloggerhead got me to the file in one go.
<jtv> http://bazaar.launchpad.net/~jtv/launchpad/drop-branchrevision-id/view/head:/database/schema/patch-2208-99-0.sql
<jtv> Not much to see, sorry.  :/
<stub> np
<wgrant> jml: I've had two successful ec2 runs today. No _LockWarner.'
<jml> wgrant: thanks.
<jml> it's happened in two unrelated branches for me.
<wgrant> Target: bzr+ssh://bazaar.launchpad.net/~launchpad-pqm/launchpad/devel/ r12404
<jml> r12400 for me. I doubt anything has changed.
<jml> not sure why the devscripts tests are printed as RemotedTestCases (e.g. ERROR: devscripts.ec2test.tests.test_remote.TestWebTestLogger.test_got_line_no_echo (subunit.RemotedTestCase))
<bigjools> jml: another spurious ec2 error I've had is all the devscripts tests failing, did that just happen to you?
<jml> bigjools: yes. what was the error?
<bigjools> jtv: thought you were off today?
<bigjools> jml: LockWarner
<jml> bigjools: right. That's exactly what happened to me.
<jtv> bigjools: I am, I am
<jml> bigjools: did you file a bug?
<bigjools> jml: I didn't, sorry.
<jml> np
<jml> I'll do so now
<jml> maybe it'll magically accumulate information and get solved while I make breakfast.
<bigjools> I was trying to land an important branch at the time and it totally slipped my mind :(
<bigjools> having breakfast solves many problems
<wgrant> jml: Your lib/canonical cleanup branch is very nice.
<jml> wgrant: thank you.
<jml> wgrant: I hope it lands one day.
<jml> https://bugs.launchpad.net/launchpad/+bug/721166 filed btw
<_mup_> Bug #721166: Tests sometimes fail on EC2 due to _LockWarner garbage <build-infrastructure> <Launchpad itself:Triaged> < https://launchpad.net/bugs/721166 >
<wgrant> jml: "one day"?
<jtv> jcsackett: thanks for the reviewâupdated my MP
<jtv> (yes yes bigjools, I _am_ off today)
<bigjools> jtv: uh huh :)
<wgrant> Can someone explain to me why ZopelessTransactionManager exists, given that everything seems to load the ZCA anyway?
<jml> wgrant: it's failed 2 out of 2 ec2 test runs
<wgrant> jml: :(
 * jml resubmits
<bigjools> wgrant: hysterical raisins?
<wgrant> bigjools: I am hoping so.
<bigjools> wgrant: iirc it has a bug as well
<bigjools> can't do multiple commits or something
<bigjools> but maybe that was fixed
<wgrant> Hah, stub has a kill-zopeless branch.
<wgrant> From 2009.
<stub> Be my guest
<stub> Big steaming pile of techdebt
<wgrant> Zopeless scripts also don't keep email within the transaction. Is that unrelated to initZopeless except by name?
<bigjools> ah yes that was the other gem
<stub> unrelated
<wgrant> That's what I've suspected for a while.
<wgrant> But I thought I must have been missing something, given the whole name thing...
<stub> Does appserver keep them within the transaction?
<wgrant> Yes.
<wgrant> They use the normal Zope queued delivery utility.
<bigjools> I don't see how you can keep emails atomic like that
<stub> Cool (although I shouldn't say that since I probably write it...)
<wgrant> Whereas scripts use SMTP.
<wgrant> Grrr.
<wgrant> It is not in fact unrelated :(
<wgrant> Zopeless mail delivery happens when canonical.lp.isZopeless() is true, which checks if ZTM is installed. But I guess that's easy enough to untangle.
<stub> extra points for tieing it into the transaction machinery :-)
<wgrant> No.
<wgrant> Soyuz relies on it being untied.
 * stub looks for his cluebat
<wgrant> Rejection emails work by being sent before the failed transaction is aborted!
<wgrant> Because why not.
<stub> that can be untangled too. I think email being sent on aborted transactions is a bug in most cases.
<wgrant> Definitely.
<wgrant> Only a couple of scripts should require the old behaviour.
<bigjools> that will be hard to fix
<wgrant> Indeed.
<stub> jtv, danilos, henninge: About 10% of POTranslation is waste and not linked from translationmessage. Probably not enough to worry about garbage collecting?
<stub> (2.5 million out of 29 million)
<henninge> stub: although that sounds like low-hanging fruit?
<danilos> stub, it'd be great to clean it up some time in the future, but we are generally better off completing message sharing migration, imho
<danilos> stub, also, there's other stuff worth cleaning up first, like removing some columns from TranslationMessage table
<stub> It is. The query to discover the unlinked records takes 10 minutes to run, so it could go in garbo-daily
<henninge> It sounded to me like that could be done in single sql query ..
<stub> henninge: It could, but I'd rather not do it in a 20-30 minute transaction
<danilos> stub, then JFDI :)
<danilos> stub, right, up to you then
<stub> k
<danilos> stub, if locking happens on entire potranslation table instead of per-row for those 20-30 mins, it will cause issues for several translation processes
<danilos> (including web UI)
<stub> Nah - stick it in garbo so transactions are short and I don't have to remember to run the job again in 2 years time
<bigjools> jml: so I got a bit further with hooking ftp into poppy-sftp.  The problem I have now is allowing anonymous access to write files - I think I might need to write a custom FTPAnonymousShell, do you have any thoughts?
<jml> bigjools: not off the top of my head, and I have to go right now, sorry.
<jml> back at ~1
<bigjools> ok np
<bigjools> I shall continue to experiment
<wgrant> Also note that
<wgrant> # if the host is empty it can be overridden by the standard PostgreSQL
<wgrant> # environment variables, this feature currently required by Async's
<wgrant> # office environment.
<wgrant> D:
<bigjools> ha
<bigjools> wgrant: dput logs in as "anonymous" right?
<wgrant> bigjools: Yes.
<bigjools> wgrant: hmmm poppy allows anything!
<wgrant> As it probably should.
<wgrant> But our default dput.cf uses anonymous.
<StevenK> bigjools: What are doing to poor poppy-sftp?
<bigjools> StevenK: I'm adding ftp to it
<wgrant> And renaming it.
<bigjools> heh
<StevenK> bigjools: Why?
<wgrant> zeca died. poppy can die too.
<bigjools> because zope's ftp server is shit
<StevenK> bigjools: Why now is a better question :-)
<bigjools> because I want to
<bigjools> it's been itching too long and I  need to scratch now
<StevenK> Note that the SFTP server uses parts of the FTP implementation with gay abandon
<StevenK> bigjools: The whole thing needs a serious refactor
<bigjools> StevenK: I saw no evidence of it using FTP stuff
<bigjools> ftp is quite different
<bigjools> when I do this I can also fix a loooooooong standing bug where we want to make sure the changes file is signed during the session
<StevenK> from lp.poppy.filesystem import UploadFileSystem
<StevenK> from lp.poppy.hooks import Hooks
<StevenK> bigjools: ^
<bigjools> that's not ftp-specific...
<bigjools> I'll need to tweak it a bit to re-use that stuff but it won't be too hard
<StevenK> The only thing that used it was poppy, so it's poppy-specific
<Ronnie> we discovered in the loco, that the live-database is not the same as the developer and django specifications (at least for the field approval_date in teams). In the live DB its an datetime, while the developer version is a date field. Django does not complain and handle things OK, but now on the webpages, dates are shown as datetimes. I added an migration file, so this should be fixed next release.
<Ronnie> But if that field did not migrate, its possible that other fields may be wrong too. can i get the schema of the life database somehow, to compare it with the django-settigns ?
<Ronnie> cjohnston, mhall119: ^^ above is interesting for both of you too
<wgrant> Ronnie: Wrong channel?
<Ronnie> oops yea
<Ronnie> wgrant: what was the ISD channel?
<wgrant> Ronnie: #canonical-sysadmin, you mean?
<Ronnie> yes, i guess those guys can access the DB
<bigjools> StevenK: poppy/twistedsftp.py has got zero comments and zero docstrings.  :/
<bigjools> correction. one docstring
<wgrant> I wish we had a Bugs team.
<deryck> Morning, all.
<deryck> wgrant, as in the former bugs team, or as in a team to deal with all the bugs? :-)
<deryck> stub, ping
<bigjools> morning deryck
<wgrant> deryck: Users having issues with bugtask conjoining and handling statuses. There may be improvements that could be made on the Launchpad end. And there is no expert team to escalate to.
<wgrant> Except for this nebulous "stakeholder escalation process"
<wgrant> Which is bordering on completely useless for this sort of thing.
<deryck> ah
<deryck> wgrant, couldn't a maintenance team look at it?  Or it seems more complex to require feature-level work?
<wgrant> deryck: It is unclear what the best solution is.
<wgrant> (Apart from deleting task conjoinment)
<deryck> right
<wgrant> I think this is going to become a significant issue.
<wgrant> Previously issues could be escalated to a team of people who knew the domain and what was what.
<wgrant> Now there is... what is there?
<deryck> wgrant, you're right that it likely won't get dealt with for awhile, but FWIW it had no chance of getting scheduled if you had escalated to the bugs team, too. so...
<deryck> wgrant, at least here, once the critical queue is cleared and features are caught up, there is some path to getting it scheduled.
<deryck> and it's not like you can't ask one of us former bugs people what to do. ;)
<wgrant> deryck: Sure, but the old team could have offered suggestions for working around it, and probably would have been able to work out plausible improvements on the LP side.
<deryck> right
<wgrant> There is no longer any path for that sort of thing.
<wgrant> There is the escalation process to get a random team assigned to implement the feature in a few months.
<deryck> wgrant, I'm not sure what you mean "path for that sort of thing" -- if it's just "I need domain knowledge advice or help" why not just ask someone?
<deryck> wgrant, I see your point, don't get me wrong.  but maybe you're over process driven here?
<wgrant> deryck: Asking the old team people is cheating.
<wgrant> Because that will not work for long.
<bigjools> ?!
<deryck> why?
<wgrant> People leave, codebases evolve, knowledge rots.
<deryck> sure, but knowledge spreads too. and it starts with asking. ;)
<wgrant> There isn't necessarily anybody who knows what is going on in that area.
<deryck> in what area?  conjoined bugtasks?
<wgrant> Well, that stemmed from an attempt at a workaround for the initial issue.
<wgrant> Let me explain.
<deryck> ok, yes.  let's start with the real issue :-)
<wgrant> apw finds that "Incomplete with response" is fairly useless, since it's not visible anywhere except searches. So he wants to be able to have a script automatically revert Incomplete bugs to their previous status when the reporter replies. He tried to use the date_* attributes, but eg. date_triaged gets unset when moving back to Incomplete. So he tried to parse bugactivity records. Which then runs into the fact that conjoined bugtasks ...
<wgrant> ... appear in the API as normal tasks.
<wgrant> So we have a whole lot of holes in the Bugs API and/or model.
<wgrant> It is unclear what we want to support directly.
<deryck> hmmm
 * deryck looks at something
<wgrant> Previously the relevant team could make a reasonable judgement on what needed to happen.
<wgrant> So, I have enough knowledge to offer a workaround.
<wgrant> But I am not in a position to suggest ways in which Launchpad should improve.
<wgrant> I do not know who is.
<wgrant> And offering workarounds without making an effort to remove the obstacles sounds like a pretty bad idea.
<deryck> wgrant, bdmurray had a work around which I don't recall now.  But to fix this case, I would make a new IBugTask attribute called is_conjoined and export it and replace the private method BugTask._isConjoinedBugTask with it.
<deryck> easy peasy and profit ;) :)
<wgrant> Sure. But that is a fix four layers below an issue that should possibly be fixed in LP directly.
<deryck> wgrant, sure.  agreed.  but conjoined bugtasks aren't going to be fixed anytime soon, and this question does come up somewhat frequently concerning the api
<wgrant> deryck: Sure, but this completely avoids the issue of whether the Incomplete issue is something we need to tackle.
<wgrant> If people offer workarounds like this, nobody will realise that the higher-level issues need solving.
<deryck> wgrant, ok, I thought you were saying fixing conjoined bugtasks was the issue.  Sorry.  you have so many issues! ;)
<deryck> wgrant, so the fix for the incomplete issue is to have the bugtask status revert to what it was previously when the original reporter responds and do away with incomplete with response.
<wgrant> deryck: Right, but is that a desired feature?
<deryck> wgrant, there's a bug somewhere detailing all the related conversation around this.
<deryck> wgrant, I would say so.  let me find the bug.
<LPCIBot> Yippie, build fixed!
<LPCIBot> Project db-devel build (378): FIXED in 5 hr 26 min: https://hudson.wedontsleep.org/job/db-devel/378/
<LPCIBot> * Launchpad Patch Queue Manager: [rs=buildbot-poller] automatic merge from stable. Revisions: 12401,
<LPCIBot> 12402, 12403 included.
<LPCIBot> * Launchpad Patch Queue Manager: [rs=buildbot-poller] automatic merge from stable. Revisions: 12400
<LPCIBot> included.
<deryck> wgrant, bug 569298
<_mup_> Bug #569298: Toggle from Incomplete/Expired when bug reporter responds <lp-bugs> <Launchpad itself:Triaged> < https://launchpad.net/bugs/569298 >
<wgrant> deryck: Thanks.
<deryck> wgrant, I think you could sniff for the previous state from the activity log and use that rather than a blanket toggle to new status, and avoid having to do a LEP as I suggest in the bug.
<deryck> or activity table, rather
<wgrant> It's impossible to reliably interpret BugActivity :(
<wgrant> Due to task moves.
<StevenK> bigjools: What don't you understand about it? :-)
<bigjools> StevenK: the bit in the file
<StevenK> bigjools: As in SFTPFile?
<bigjools> no, the whole file
<bigjools> ooo I spotted another comment.  It's an XXX that doesn't meet our XXX guidelines.
<StevenK> bigjools: If you ask me questions about it rather than "I don't get it" I'll answer them
<bigjools> StevenK: when I have specific questions I'll ask them, but it's hard to know  where to start because there's no clue as to what does what.  Perhaps a better initial question is, "how does this stuff fit in and work?"
<bigjools> which is another way of saying, this stuff needs to be brought up to our coding standards :)
<StevenK> bigjools: The .tac is where to start
<bigjools> StevenK: that part is ok, I figured it out.  Now I just need to know what's going on inside poppy/twistedsftp.py
<StevenK> bigjools: An SFTPServer instance is created per client
<StevenK> bigjools: Depending on what the client does, methods are called in it, such as makeDirectory and openFile
<bigjools> stating the obvious :)
<bigjools> StevenK: I will try and call you next week about this
<bigjools> I need higher bandwidth and it's late for you
* benji changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | firefighting: - | On call reviewer: adeuring, benji* | https://code.launchpad.net/launchpad-project/+activereviews
<henninge> Anybody got an idea why the hard_timeout feature for TranslationImportQueueEntry:+index might not be working? https://qastaging.launchpad.net/+feature-rules
<henninge> The page still times out after 14 seconds.
<henninge> Also, it worked last week or so but the database has been renewed since.
<jml> bigjools: hi
<bigjools> jml: just going to lunch!  But I am making progress, I've even got an ftp server that can mkdir :)
<jml> bigjools: sweet
<bigjools> I have probably violated every twisted convention in the book but I'm sure you'll put me right
<jml> :)
<bigjools> once I get this DTRT I will mercilessly refactor
<bigjools> anyway, back in a bit
<deryck> henninge_, ping for standup
<henninge> deryck: coming
<deryck> ack
* bac changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | firefighting: - | On call reviewer: adeuring, benji*, bac | https://code.launchpad.net/launchpad-project/+activereviews
<benji> to whom it may concern: I plan on doing a review of https://code.launchpad.net/~jameinel/launchpad/twisted-close-on-failure/+merge/50067 in a little while.
<abentley> henninge, would it be right to say that 702477 is incomplete, and the other translation credit issues are new bugs?
<henninge> abentley: possibly ...
<henninge> abentley: although  I think all the issues would and should be fixed in one branch that updates the handling of translation credits to the new model.
<henninge> abentley: so maybe the bug should be generalized?
<abentley> henninge, No, I think if we generalize something, it should be the card.
<henninge> or that
<abentley> deryck, henninge: I propose we change my current card to "translation credits need upates for new model", and move it back to the Next column.
<abentley> deryck, henninge: because right now, there is nothing I can do about it.
<deryck> abentley, I accept that proposal :-)
<henninge> abentley: I concur
<deryck> abentley, I think the splitting work is more important for you to do anyway
<deryck> abentley, henninge -- I think that sort of bug could be pushed to when we leave feature work and do maintenance even. or done while/after jml does review even.
<benji> bac: I'm ready any time.
<benji> I'm also in the wrong chan.
<danilos> stub, hi, still around?
<stub> sort of
<danilos> stub, I'd just like you to take a very quick look at a DB query I'd like to run on production :)
<stub> sure
<danilos> stub, https://pastebin.canonical.com/43604/
<danilos> stub, it should update 23k rows and in a transaction without a commit it took <4s on staging
<abentley> deryck, henninge: the one-time merger script took ~5 hours to run.
<jml> bigjools: how's it going?
<jml> !!!
<jml> more test failures.
<bigjools> jml: ok thanks, I've figured out a bunch of stuff.  The twisted FTPShell is roughly equivalent to SFTPServer so I'm cargo-culting that stuff to just get it working before I figure  out what I need to refactor.
<jml> *nod*
<stub> danilos: looks fine - 23.9k rows on prod
<bigjools> jml: I need to work out how to add a disconnection hook into FTPShell though
<stub> danilos: Code is in place to keep this maintained in the future?
<danilos> stub, well, I'd like to have this run only as a special-rollout-requirement when code gets in place (I am just about to submit a branch up for review)
<danilos> stub, (or run it after the rollout)
<stub> ok. Do we have structuralsubscription entries that should not have a corresponding bugsubscriptionfilter?
<danilos> stub, no, we shouldn't, but we have them today
<danilos> stub, I was thinking of introducing a constraint as well, but not sure how to best do it with multiple-tables
<stub> You can't sanely
<danilos> stub, right, that's what I suspected as well
<stub> Is this a 1:1 relationship, or can there be many bugsubscriptionfilters referencing a single structuralsubscription?
<danilos> stub, there can be many BSFs referencing a SS
<danilos> stub, it's a "at least one" BSF for a SS
<stub> All seems fine. Add it to special rollout requirements, or we can land this as a db patch if you prefer.
<danilos> stub, my team would prefer to get it up on qastaging to ease further development, which is why I am not going with a patch
<stub> k
<danilos> stub, thanks for the input
<jml> bigjools: logout(), I think.
<abentley> adeuring, benji, bac: Could you please review https://code.launchpad.net/~abentley/launchpad/pomessage-sequence/+merge/50344 ?
<adeuring> abentley: sure, I'll look
<abentley> adeuring, thanks.
<bigjools> jml: ha, I found a Twisted bug
<jml> bigjools: I don't believe you. Twisted is unique among all software in its perfection.
<bigjools> ftp.AnonUserDeniedError inherits from FTPCmdError but doesn't pass it an errorMessage required when formatting the response
<jml> bigjools: looks like it passes None
<bigjools> jml: which makes RESPONSE[self.errorCode] % self.errorMessage blow
<jml> ah
<deryck> abentley, that doesn't seem *too* bad :-)
<bigjools> bizarrely, the FTPAnonymousShell uses AnonUserDeniedError with no problems :/
<jml> well, passing anything would make it blow, right? 'foo' % (anything,) fails
<bigjools> oh - huh yes :)
<jml> another windmill hang, plus librarian connection refused errors.
<jml> I was thinking on cleaning up the top-level directory of the tree
<jml> does anyone know what the *-includes directories are for?
<jml> or parts/
<jml> or the many zcml files
<jml> or benchmarks/
<jml> or version.txt
<benji> bac: I've finished my review of https://code.launchpad.net/~jameinel/launchpad/twisted-close-on-failure/+merge/50067 and added you as a requested reviewer.
<bac> thanks
<jml> flacoste: you've been around a while, know what any of those things are for?
<adeuring> abentley: r=me. must have been boring work ;)
<flacoste> parts is a buildout managed directory
<abentley> adeuring, thanks.
<flacoste> some recipe will put stuff in there
<flacoste> jml: benchmarks was for the funkload benchmarks we run in the ShipIt sprint
<flacoste> version.txt was used when we included the release number in the footer
<flacoste> might still be there, but hidden, not sure
<flacoste> but it's not used for use, since it's several number behind
<jam> benji: thanks for the review
<flacoste> overide-includes and package-includes are convenient way of adding ZCML files to the config without having to edit site.zcml
<flacoste> jml: ^^^
<jml> flacoste: thanks.
<jml> flacoste: is 'benchmarks' worth keeping, do you think?
<benji> jam: my pleasure, hopefully it's not too stream of consciousness, I intentionally left in some of my thought process because I think it communicates a bit better
<flacoste> jml: nah
<henninge> have a good weekend everybody!
<jam> Can someone help me land: https://code.launchpad.net/~jameinel/launchpad/lp-forking-serve-cleaner-childre/+merge/50031
<jam> I assume the correct way to land it is using 'ec2land', but I haven't ever set up ec2
<jml> does zope look for magically named zcml files, or do they have to be specified somewhere?
<jml> jam: I can land it for you, if you'd like
<jam> jml: thanks.
<jml> jam: ec2: ERROR: Branch doesn't have linked bugs and doesn't have no-qa option set. Use --no-qa, or link the related bugs to the branch.
<bigjools> jml: AFAIK they have to be specified
<jam> jml: no bugs, so --no-qa. It is just code cleanup. It is a bit related to a bug, but not enough to be considered anything like a fix for it.
<bigjools> jml: the top level zcml is set in the buildout.cfg
<jml> bigjools: thanks.
<bigjools> there's some wildcard includes though
<jml> jam: ok. ec2 instance starting. I'll let you know when it's detached. You ought to get an email.
<jml> I wonder if anyone uses start-gdb much.
<jam> benji: responded to your response :) The diff may need to update still
<benji> jam: cool, I'll look at it in a bit
<jam> benji: diff has been updated
<jml> flacoste: do override-includes and package-includes have extra stuff in them on production? i.e. can I move them without a special roll-out request?
<flacoste> jml: i'm 99.9% sure they don't
<jml> good enough.
<jml> flacoste: thanks.
<flacoste> if they have, it's before my time
<flacoste> i never heard of it
<flacoste> jml: site.zcml is the default, the app server one is configured in the config (launchpad.conf)
<jml> also, any clue about script.zcml, ftesting.zcml
<jml> and script-testing.zcml
<flacoste> script.zcml is hardcoded by execute_zcml_from_script
<flacoste> ftesting.zcml is hard coded somewhere in the test framework
<flacoste> and script-testing.zcml is probably similar
<jml> I can't find it when grepping the Launchpad tree
<jml> but I'll poke a bit more.
<flacoste> ftesting.zcml is in zope iirc
<flacoste> script-testing.zcml is LP specific
<flacoste> it might also be unused nowadays
<jml> do we actually *use* ftesting / ftests concepts any more?
<flacoste> FunctionalLayer uses it
<flacoste> through the FunctionalSetUp fixture
<flacoste> or whatever it's called
<jml> hmm.
<flacoste> execute_zcml_for_script actually switches between 'script.zcml' and 'script-testing.zcml'
<flacoste> based on whether it detects we are running within the test runner or not
<flacoste> jml: ftesting.zcml is the default ZCML loaded by FunctionalTestSetUp
<flacoste> which is used by all FunctionalLayer tests
<jml> flacoste: that's great, thanks. it means I can move them.
<flacoste> jml: yes, you can, you'll have some plumbing to do :-)
<jml> flacoste: also, do you know if zdaemon.conf is used?
<flacoste> jml: die, die, die!
<jml> :D
<flacoste> iow, it's not
<flacoste> at 99.9% confidence level
<flacoste> we don't even have the runzope script
<bigjools> abentley_: would you mind investigating this branch scanning problem please?  I've no idea what's up.  https://answers.edge.launchpad.net/launchpad/+question/142589
<flacoste> i need a reviewer for https://code.launchpad.net/~flacoste/launchpad/bug-721324/+merge/50365
<flacoste> a pretty simple branch
<flacoste> benji, bac: want to put your teeth onto that one?
<benji> flacoste: absolutely
<bigjools> ok I think I'll quit for the day on a good note, having just got ftp finally worked into poppy-sftp.
<bigjools> g'night all
<bigjools> have great weekends
<LPCIBot> Project db-devel build (379): FAILURE in 5 hr 35 min: https://hudson.wedontsleep.org/job/db-devel/379/
<LPCIBot> Launchpad Patch Queue Manager: [rs=buildbot-poller] automatic merge from stable. Revisions: 12404
<LPCIBot> included.
<benji> bac: I'm done with https://code.launchpad.net/~flacoste/launchpad/bug-721324/+merge/50365 and I added you as a requested reviewer.
<jml> what's the motivation for the optional bzr plugin system?
<lifeless> jml: the what?
<jml> lifeless: optionalbzrplugins
<jml> in Launchpad.
<lifeless> https://launchpad.net/optionalbzrplugins ?
<jml> in the Launchpad tree
<lifeless> I think its so that the importer can add that path and get those plugins but the scanner doesn't and won't get them.
<lifeless> ditto puller.
<lifeless> and I think the why is because the scanner and puller don't have the same safety net in case someone puts in a pull url that is http but backed by svn not bzr
* bac changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | firefighting: - | On call reviewer: benji, bac | https://code.launchpad.net/launchpad-project/+activereviews
<jml> .lifthat sounds plausible.
<jml> lifeless, rather.
<lifeless> I guessed ;)
<lifeless> whoa
<lifeless> static websites on s3
<lifeless> tiny change and *boom*
<bac> benji: i'll take danilos branch.
<benji> ok
<lifeless> booyah
<lifeless> https://bugs.qastaging.launchpad.net/ubuntu/+bug/1
<_mup_> Bug #1: Microsoft has a majority market share <iso-testing> <ubuntu> <Clubdistro:Confirmed> <Computer Science Ubuntu:Invalid by compscibuntu-bugs> <EasyPeasy Overview:Invalid by ramvi> <GNOME Screensaver:Won't Fix> <Ichthux:Invalid by raphink> <JAK LINUX:Invalid> <The Linux OS Project:In Progress> <metacity:In Progress> <OpenOffice:In Progress by lh-maviya> <Tabuntu:Invalid by tinarussell> <Tivion:Invalid by shakaran> <Tv-Player:New> <Ubuntu:I
<lifeless> down to 7.6 seconds cold
<lifeless> 5.34 hot
<lifeless> wiiin
<jml> https://code.launchpad.net/~jml/launchpad/flush-top-level/+merge/50374
<jml> Takes the top level from 54 files & directories down to 31.
<lifeless> I find it annoying that to add a recipe I have to go elsewhere than where I go to edit a recipe
<cjohnston> Can someone please take a look at the spam: https://bugs.launchpad.net/ubuntu/+source/firefox/+bug/588595/comments/22
<_mup_> Bug #588595: Ubuntu download button renders inccorrectly in Firefox 3.7 (trunk) <ppa> <Mozilla Firefox:Fix Released> <Ubuntu Mozilla PPA Bugs:Fix Released> <Ubuntu Website:Invalid> <firefox (Ubuntu):Invalid> < https://launchpad.net/bugs/588595 >
<lifeless> jml: https://bugs.launchpad.net/launchpad/+bug/721460
<_mup_> Bug #721460: Cannot use 'lp:bzr' as a branch in a recipe <recipe> <Launchpad itself:Triaged> < https://launchpad.net/bugs/721460 >
<lifeless> cjohnston: for operational stuff, #launchpad is better
<lifeless> that said, its the end of the week - is this new spam, are they still adding spam?
<cjohnston> Just happened today
<cjohnston> 1 hour ago according to LP
<cjohnston> lifeless: mostly want to see about maybe removing the user
<lifeless> cjohnston: do they have karma?
<lifeless> usually folk with karma have been hacked
<cjohnston> ehh.. I get a page gone when trying to visit the user
<lifeless> already suspended then
<jml> Got bitten by bug 720998 again.
<_mup_> Bug #720998: ec2 test times out <build-infrastructure> <Launchpad itself:Triaged> < https://launchpad.net/bugs/720998 >
<lifeless> cjohnston: file a question to get that bit of spam removed
<cjohnston> against launchpaditself?
<flacoste> congratulations to benji for achieving reviewerhood
<lifeless> flacoste: hes still mentored isn't he?
<benji> flacoste: thanks
<flacoste> lifeless: nope, bac graduated him
<lifeless> benji: congrats
<lifeless> cjohnston: yes answers.launchpad.net/launchpad
<benji> lifeless: thanks
<benji> I got a little square hat with a tassel and everything.
<cjohnston> ty
<cjohnston> lifeless: i guess someone already did it https://answers.launchpad.net/launchpad/+question/145908
<jcsackett> benj: care to use your full reviewerhood to take a look at https://code.launchpad.net/~jcsackett/launchpad/dumb-date-formats/+merge/50385 ?
<jcsackett> benji ^
<jcsackett> darn nick typos.
<benji> jcsackett: sure
<jcsackett> thanks!
<lifeless> flacoste: btw, I've just overhauled the deployment calendars to be crisper and easier to update .... now is your chance to say 'wtf, put it back', if you disagree with the approach :)
<flacoste> lifeless: looks good, but can you follow-up to my emails with the new link, given that you broke it :-)
<lifeless> flacoste: I don't think its broken, renames leave a forward in place
<lifeless> oh, but its broken
<lifeless> I'll put a redirect in place
<lifeless> flacoste: done
<flacoste> thx<
<lifeless> flacoste: the link works, I won't add noise by noting this on the list
<flacoste> fair enough
<lifeless> it will be obvious to anyone looking, and the redirect can hang about for a while without trouble
<flacoste> like you saw, it was broken
<flacoste> that's why i asked
<flacoste> but since you repaired it
<lifeless> yeah, bad assumption on my part
<flacoste> everything is fine
<lifeless> sorry!
<benji> jcsackett: https://code.launchpad.net/~jcsackett/launchpad/dumb-date-formats/+merge/50385 is approved, I had a few questions/suggestions but it looks good.
<jcsackett> benji: thanks, i'll address those concerns.
<benji> cool
<lifeless> hmm
<lifeless> is the scanner down? flacostes new commits haven't shown up
<lifeless> flacoste: hi
<lifeless> flacoste: still around ?
<flacoste> lifeless: i am
<LPCIBot> Project devel build (455): FAILURE in 5 hr 28 min: https://hudson.wedontsleep.org/job/devel/455/
<LPCIBot> * Launchpad Patch Queue Manager: [r=allenap][bug=719249] The advanced bug subscription overlay is now
<LPCIBot> pre-populated in order to speed it up a bit.
<LPCIBot> * Launchpad Patch Queue Manager: [r=lifeless,
<LPCIBot> wallyworld][bug=720503] Use feature flags for recipe enablement and
<LPCIBot> beta message showing
<LPCIBot> * Launchpad Patch Queue Manager: [r=gmb][ui=none][no-qa] Add kubuntu-mobile to list of tasks to be
<LPCIBot> generated by cron.germinate.
<lifeless> flacoste: man, interrupts. _. privmsg if you're still here
<lifeless> flacoste: btw, we're getting traction on bug search and bug display
<flacoste> i saw!
<flacoste> that's pretty cool
<lifeless> I should have an analysis of hotspots for the 'show all comments' case next week
<lifeless> once we've hit the 'ok the current definition is as efficient as we can do with current tech
<lifeless> we'll need to look at pagination of bug comments and so on
<lifeless> loading 1400 for bug one is just nuts.
<lifeless> it would be really cool to do a facebook style incremental loader though
<lifeless> sinzui suggested an enhanced BatchNavigator which would add more results in a given direction
<lifeless> without a page load, for ajax enabled users
* benji changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | firefighting: - | On call reviewer: bac | https://code.launchpad.net/launchpad-project/+activereviews
<jcsackett> sinzui: standup? or skipping, since it's just the two of us?
<jcsackett> sinzui: well, i guess skipping. :-P
<LPCIBot> Project db-devel build (380): STILL FAILING in 5 hr 32 min: https://hudson.wedontsleep.org/job/db-devel/380/
<LPCIBot> Launchpad Patch Queue Manager: [rs=buildbot-poller] automatic merge from stable. Revisions: 12405,
<LPCIBot> 12406, 12407, 12408 included.
<maxb> Why would ~registry be subscribed to lp:~debian-bazaar/bzr/unstable ? Is this one of these "Dump stuff on ~registry to allow deleting the associated thing" cases?
<wgrant> Right, deleting a team is achieved by merging it into ~registry.
<wgrant> This is somewhat insane.
<lifeless> unsubscribe it
<wgrant> Gone.
<jam> lifeless: can you ec2land  --no-qa https://code.launchpad.net/~jameinel/launchpad/twisted-close-on-failure/+merge/50067 ? (You're welcome to add bug #717345 to it, but that won't be complete until the third patch lands)
<_mup_> Bug #717345: Updates to SFTP server leak file handles in use_forking_server mode <Launchpad itself:In Progress by jameinel> < https://launchpad.net/bugs/717345 >
<jam> (Since you were kind enough to mark it approved.)
#launchpad-dev 2011-02-19
<lifeless> jam: have you set a commit message?
<wgrant> lifeless: Do qastaging OOPSes not sync very often?
<lifeless> wgrant: sadly no
<lifeless> jam: its on the way
<wgrant> :(
<jam> lifeless: thanks!
<jam> wgrant: if they are in the rsync list, you can get them yourself
<jam> rsync tellurium::codehosting-qastaging-logs/ is stuff I used quite a bit
<lifeless> jam: the point is to get them into the lp-oops microsite
<jam> lifeless: ah, so nicely formatted, rather than just raw oops
<jam> sure
<lifeless> formatted and analysed
<jam> anyway, thanks for the submission, I'm off to have some family time
<lifeless> ciao
<wgrant> Yeah, lp-oops is far better at formatting and analysing than I.
<wgrant> lifeless: Bug 721591
<_mup_> Bug #721591: Please make *-backports not automatic starting with Natty <soyuz-publish> <Launchpad itself:New> < https://launchpad.net/bugs/721591 >
<wgrant> I am wondering how we should restrict to >= Natty...
<wgrant> It's a bit too permanent for FF.
<lifeless> its a series thing right ?
<wgrant> Yes.
<wgrant> We could have a flag on distroseries.
<wgrant> But it will probably be on forever.
<lifeless> I've marked the bug incomplete
<lifeless> it doesn't describe what it needs
<wgrant> It does.
<wgrant> What is incomplete about it?
<lifeless> maybe you understan it
<lifeless> but I read the spec
<wgrant> The spec says what needs to be done:
<lifeless> and it looks like LP doesn't need to change at all, its an APT and synaptic change.
<wgrant>  - change soyuz to set NotAutomatic: yes in backports release files
<wgrant>  - add "ButAutomaticUpgrades: yes" - don't install automatically, do upgrade automatically
<lifeless> then put that in the bug in lp
 * wgrant comments.
<lifeless> bugs that say 'follow a spec over ->' when that spec is a wall of text, are pretty unapproachable
<lifeless> wgrant: fix up the description
<lifeless> thats more useful than a comment
<wgrant> True.
<lifeless> wgrant: so, the way I'd approach this general problem, if doing this greenfield would be:
<lifeless> use the prototype pattern
<lifeless> setup a 'template' distroseries
<lifeless> and have 'make a new distroseries' use the template for that distro
<wgrant> We already do that.
<lifeless> anyhow, this seems specific to releases of Ubuntu, so its a series specific thing
<wgrant> Right.
<lifeless> I don't think a flag makes sense, its not 'server config' its data driven config
<wgrant> It just seems sort of wasteful to have it as a flag on DS when there is just a single transition.
<lifeless> s/flag/featureflag/
<wgrant> Right.
<lifeless> wekk
<lifeless> I'm fine doing it however you like
<lifeless> I will note:
<lifeless>  - if its a config option somewhere, that implies that users may want to change it
<lifeless>  - if they /will not/ or /must not/ then an option is wrong
<lifeless>  - feature flags are things we want to change, as is launchpad-lazr.conf
<lifeless> this in fact a suite rule note a series rule, now that I'm more awake.
<wgrant> But it can't be hardcoded, because it is data-based.
<lifeless> sure we can, if we want to hard code it.
<wgrant> It can become a suite flag when we model suites.
<lifeless> its less flexible, and might be a poor engineering choice to hard code it.
<lifeless> but it might be a good choice to hard code it.
<lifeless> I relaise that doesn't help you all that much
<lifeless> man getCurrentSourceReleases seems a bit inefficient
<wgrant> Convoluted, but it doesn't look toooo inefficient.
<wgrant> Am I missing something?
<lifeless> https://bugs.launchpad.net/launchpad/+bug/636158
<_mup_> Bug #636158: BugTask:+index times out with many bug tasks/nominations (eg. Bug #1) <lp-bugs> <pg83> <timeout> <Launchpad itself:Triaged> < https://launchpad.net/bugs/636158 >
<lifeless> wgrant: its querying the same stuff from distro level then distroseries level according to my casual trace here
<wgrant> Oh, completely forgot that it was used by Bugs.
<lifeless> per nomination
<lifeless> not nomination
<lifeless> task
<lifeless> anyhow
<lifeless> it may be tolerable as a single lookup in a paeg
<lifeless> but we're doing a fair few
<wgrant> Do you have a modern qas OOPS for that?
<wgrant> Oh.
<wgrant> Mine has synced now.
<lifeless> OOPS-1875D380 is prod yesterday
<wgrant> Finally.
<wgrant> Yes, but prod isn't fast yet.
<wgrant> OOPS-1875QS36
<wgrant> Hah.
<wgrant> Longest statement is 319 ms. Not bad.
<wgrant> Lots of repeated SPN and VPC queries :(
<lifeless> yes
<lifeless> I'm looking at SPN atm
<wgrant> And Person itself.
<lifeless> VPC will be some missing/incimplete eager loads
<wgrant> Hmm.
<wgrant> VPC is eagerloaded.
<wgrant> But not completely, yeah.
<lifeless> e.g. list(Person query) rather than PersonSet.eagerLoadPersonsByIds
<wgrant> I wonder how expensive it would be to record tracebacks for queries.
<lifeless> I'd like to do it on demand
<lifeless> or when we're down to <50 queries on most pages
<lifeless> it is nontrivial cost, but not terrible (unless doing 1000's of times :P)
<wgrant> Sure, only on demand.
<wgrant> Like ++oops++tracebacks or something like that.
<lifeless> but the thing to remember is there is disk overhead in the oops, serialising etc.
<lifeless> wgrant: +1
<wgrant> Hmm.
<wgrant> It's querying for people that don't make it into the final page.
<wgrant> That's a bit odd.
<lifeless> I would really be fine all the time, once we're down to sensible timeline counts
<wgrant> eg the Thai Loco Team.
<lifeless> wgrant: owners of messages perhaps
<lifeless> we possibly want to turn memcache off too
<wgrant> Ohh.
<wgrant> I think it's doing the release targetting permission check.
<wgrant> These are all product owners.
<lifeless> let me guess, thats expressed in python?
<wgrant> Of course.
<wgrant> It's also buggy.
<lifeless> persistence layer should make this -so- much nicer
<wgrant> Perhaps the rewrite can fix both.
<lifeless> memcache misses for all of it
<lifeless> I think we should turn memcache off for this
<wgrant> Er.
<wgrant> Well, this is on ++oops++, so it is definitely going to miss :)
<wgrant> But yes, I think the utility is extremely limited in general.
<lifeless> also a few single bug subscroption lookups
<lifeless> I bet its per question or something
<lifeless> repeated lookups for totem
<wgrant> It's nice that we have a pathological test case like this.
<lifeless> )
<lifeless> it does two __nonzero__ calls on bugwatches
<lifeless> thats not cached by the resultset
<wgrant> At least __nonzero__ isn't so bad now.
<lifeless> depends on the query
<lifeless> it may still materialize 50K rows
<lifeless> classy
<lifeless> we prejoin bugwatch
<lifeless> then we directly query bugwatch
<lifeless> twice
<wgrant> We do similar things in lots of places.
<lifeless> thrice
<lifeless> and more
<lifeless> I wonder if its a per-task check
<wgrant>         if check_permission("launchpad.Driver", target):
<wgrant> Yay, security adapters.
<lifeless> its bracketed by milestone lookups
<wgrant> See, tracebacks handy.
<lifeless> browser/bugtask.py - line 3163
<lifeless> wgrant: of course; I use them all the time ;)
<wgrant> Also, LaunchBag needs to die.
<wgrant> lifeless: A line number that high is illegal.
<lifeless> I might work on redoing oops tools tomorrow
<wgrant> On something searchable?
<lifeless> theres a few bugs I'd like to address
<lifeless> move the format to json
<lifeless> make pageid a primary focus for it
<lifeless> make it realtime
<wgrant> lifeless: The targetting is not actually the expensive bit. It is just normal task attribute permission checks.
<wgrant> Yay.
<wgrant> Easily cachable.
<lifeless> rabbitmq loading of oopses
<lifeless> get rid of the on disk store; just keep em in the db
<lifeless> [but make it properly searchable as a necessary condition]
<lifeless> public project
<lifeless> flacoste: I bet you're gone now
<jml> I might fix the build tomorrow!
<jml> so I can land jam & my branches.
<jml> g'night.
<wgrant> Night.
<lifeless> night
<LPCIBot> Yippie, build fixed!
<LPCIBot> Project devel build (456): FIXED in 5 hr 22 min: https://hudson.wedontsleep.org/job/devel/456/
<LPCIBot> Launchpad Patch Queue Manager: [r=adeuring][no-qa] Fix factory.makePOTMsgSet default sequence.
<lifeless> SpamapS: hey
<lifeless> SpamapS: if you're around... where is the cassandra stuff, and are there lucid builds
<lifeless> jml: around?
<lifeless> jml: hai
<lifeless> jml: oh hai
#launchpad-dev 2011-02-20
<LPCIBot> Yippie, build fixed!
<LPCIBot> Project db-devel build (381): FIXED in 5 hr 47 min: https://hudson.wedontsleep.org/job/db-devel/381/
<SpamapS> lifeless: re the cassandra stuff, https://launchpad.net/~cassandra-ubuntu
<SpamapS> lifeless: stable releases has 0.7.0 .. I'm working on 0.7.2
<SpamapS> lifeless: and yes, it installs fine on lucid and maverick
<lifeless> \o
<lifeless> I'll have to see about getting it into cat
<lifeless> wgrant: are you around?
<wgrant> lifeless: Yes.
* lifeless changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | firefighting: - | On call reviewer: - | https://code.launchpad.net/launchpad-project/+activereviews
<lifeless> wgrant: https://code.launchpad.net/~lifeless/launchpad/rabbit/+merge/50488
<lifeless> if you're interested
<lifeless> I'd like another set of eyeballs before I land it
<wgrant> lifeless: I spent a while looking for the definition of BRANCH_NICK
<wgrant> Then I noticed the docstring.
<wgrant> Er, comment.
<wgrant> os_exec's name sucks.
<wgrant> This daemonisation stuff looks far more crackful than required.
<wgrant> What's wrong with subprocess?
<lifeless> I don't know yet
<lifeless> I was handed some working code that was run from Makefile and totally unsuitable for parallel testing etc
<wgrant> Ah.
<lifeless> I've done the minimal migration
<lifeless> theres more to go
<wgrant> Maybe the rabbit binaries don't daemonise and set PIDs and stuff.
<lifeless> I don't know which bits matter, and which don't.
<wgrant> It is awful and you need to clean it up, but yes, get it in the tree.
<wgrant> It looks like it will mostly work, but it's fairly insane.
 * wgrant considers destroying Zopeless.
<lifeless> heh
<lifeless> go look at the original
<lifeless> :)
<lifeless> ec2 away
<lifeless> hopefully all the deps are already presen
<lifeless> t
<lifeless> I'm seriously considering having the production rabbit present on every node
<lifeless> it would make reasoning about many failure modes a lot easier
<wgrant> Hmm.
<wgrant> lifeless: Not just on the DB servers?
<wgrant> lifeless: Hmm, ~canonical can't see U1?
<wgrant> :(
 * wgrant pokes staging in the eye.
<wgrant> Subclasses overriding private attributes... grr.
<wgrant> There is minified JS in our bzr history wtf wtf wtf
<lifeless> wgrant: probably not on the db servers at all
<wgrant> lifeless: Hmmm.
<lifeless> wgrant: why would it be on the db servers?
<wgrant> lifeless: I like keeping my critical data stores on machines that are meant to keep critical data stores.
<lifeless> rabbit sits in a spectrum
<lifeless> if (say) devpad croaks
<lifeless> we don't care if in-progress OOPSes are lost
<lifeless> all our machines have RAID level redundancy atm
<lifeless> the db servers aren't big because they have more redundancy, they are big because they are central load points
<wgrant> But the DB servers stay up.
<lifeless> so do the appservers
<wgrant> And if they go down then we are screwed anyway.
<lifeless> we can fail over to a slave
<lifeless> with rabbit by default the queued messages on a down load will return when the node returns
<lifeless> so it needs to be a pretty critical boom moment to permanently lose them
<lifeless> and whatever machine we bring another queue processor up on would establish new queues
<wgrant> Appservers having the only copy of any data is scary.
<lifeless> they have that all the time at the moment
<wgrant> Do they?
<lifeless> in progress requests
<wgrant> The persistent data stores are on four other machines, not appservers.
<lifeless> rabbit isn't a persistent data sttore
<wgrant> It persists across a request boundary, so it is persistent.
<lifeless> by that token so is an oops report
<lifeless> which means we have 5 persistent data stores on the app servers
<lifeless> (I disagree with that definition btw)
<wgrant> I don't want a taken-out appserver to reverse actions taken by a completed request.
<wgrant> Losing queue items does that.
<lifeless> I think you need to learn a little  more about rabbit
<wgrant> 14:32:14 < lifeless> with rabbit by default the queued messages on a down load will return when the node returns
<lifeless> right
<lifeless> we wouldn't have queues on the appservers
<lifeless> we'd have a rabbit on them
<lifeless> if something got buffered there, it means the network is down
<lifeless> the only use case for that is oops forwarding atm
<lifeless> separately
<wgrant> So each queue has a master node that will not be on an appserver?
<lifeless> no, dropping a queues items won't reverse actions from a completed request
<lifeless> those actions clearly wouldn't have happened if that was possible
<lifeless> in rabbit, a queue lives on a node
<lifeless> exchanges forward messages to queues based on bindings
<wgrant> Right.
<lifeless> using oops as an example
<lifeless> I'd put the queue of oopses to process on a local rabbit where the oops site is
<lifeless> I'd make it entirely ephemeral because I don't care if we lose a few in the event of failure
<lifeless> (I care if we lost tonnes, but not a few)
<wgrant> Oh, right, it lets you configure the reliability? It's been ~a year since I last looked at it in depth.
<lifeless> each message
<lifeless> and each queue
<lifeless> you can be transactional
<lifeless> or you can be fire and forget
<lifeless> we're going to have a bunch of differing requirements
<lifeless> and how we deal with failures of various sorts is going to require case by case thought.
<lifeless> simply saying 'transactional persistent, done' would be a recipe for later pain.
<wgrant> Indeed.
<lifeless> anyhow
<lifeless> it doesn't have 2pc
<lifeless> AFAICT
<wgrant> :(
<lifeless> https://dev.rabbitmq.com/wiki/AckingSchemes#section_1 kindof talks about it
<lifeless> but hte images are fucked
<wgrant> As fucked as some of our import webs?
<lifeless> it looks like you build it on top of the characteristics
<lifeless> righto, amqp 1.0 does 2pc - but IIRC 1.0 kindof foundered
<lifeless> as a ill performing mess
<wgrant> Heh.
<lifeless> best you can do with 0.8 is queue things and not release until the db is committed
<lifeless> (thats what pgamqp does)
<lifeless> ok, wow time ;)
<StevenK> lifeless: Hah
<StevenK> lifeless: 3rd 85 yesterday
<lifeless> cool
<lifeless> grrr, rabbit failed on ec2
<lifeless> is it easy to get a shell into an ec2 instance while its running tests ?
<wgrant> lifeless: It gives you the ssh command near the end of the output.
<StevenK> lifeless: ssh -v ec2test@<ec2 ip or hostname>
<lifeless> next weekend I guess
 * jml is around fo reals
<lifeless> hai
<lifeless> and bai
<lifeless> jml: I am on that bridge ;) - I have a patch where I'm trying to figure out what package to put the subunitstream model class in
<lifeless> grrr
<lifeless> dict.update() shouldn't be forbidden ... zopeSecurityProxy fail
<thumper> morning lifeless
 * thumper needs coffee 
 * thumper wanders off to find said coffee
<lifeless> hi thumper
<jelmer> 'morning thumper, lifeless
<thumper> hi jelmer
<jml> lifeless: uhh, I see.
<jml> lifeless: there's a relatively easy way to fix the {}.update thing.
<jml> lifeless: I'd suggest "testdb" if you can't think of anything better.
<jml> lifeless: you could add 'checker.BasicTypes.update({dict: checker.NoProxy})' to lp_sitecustomize, which would completely disable security proxy checking for dicts. I don't think that would be a bad thing.
<lifeless> jml: I actually think the current structure of <domain>/<layer> is really annoying; I'd much rather we had <layer>/<domain>
<lifeless> jml: wouldn't that let things escape securityproxying? if an unproxied object is in the dict
<lifeless> wgrant: hey
<lifeless> wgrant: when you are around; can more than one distro share a distroseries row?
<StevenK> lifeless: I strongly doubt it.
<lifeless> StevenK: ok cool
<thumper> StevenK, wallyworld_: standup
<thumper> ?
<StevenK> thumper: Aye, coming
<lifeless> is there a helper for sql strings like Or for storm expressions?
<lifeless> I have a list of sql fragments
<lifeless> and want on length 1 -> foo[0]
<lifeless> on length 2+ -> "(" + " OR ".join(foo) + ")"
<thumper> StevenK: https://code.launchpad.net/~thumper/launchpad/choosing-recipe-name/+merge/50530
<wgrant> lifeless: I hope not, because that would break everything.
<jml> lifeless: oh yes, I think you're right.
<jml> lifeless: I don't find our "current" thing annoying. What I find annoying is that we haven't fully switched to it.
<jml> lifeless: however, if you wanted to change, and were actually serious about finishing that change and could guarantee that we wouldn't have a mutiny, then I wouldn't mind a <layer>/<domain> approach. They seem equally preferable to me.
<wallyworld_> thumper: sorry, was having breakfast
<thumper> wallyworld_: ok
<thumper> rachel's kindle just arrived \o/
<wallyworld_> you're just happy now cause you don't like sharing
<lifeless> jml: I'm waiting for the squad culture to bed down a bit
<lifeless> jml: I think it will become obvious to others, the more they work cross <domain> that the current structure is a nuisance
<wgrant> Hmm.
<lifeless> things like the buildmaster are good as they are
<wgrant> lifeless: Except it should be in services.
<lifeless> that is, they actually are components
<lifeless> wgrant: sure, or services should die in a fire and promote its contents
<wgrant> The LP_DB* envvars make me sad.
<lifeless> excuse the hyperbole
<lifeless> wgrant: yes
<wgrant> But I guess I should still support them :(
<wgrant> Apart from that Zopeless is just about gone.
<lifeless> or find and remove their uses
<lifeless> wgrant: nice
<lifeless> oh frell
<lifeless> I have the rabbitmq code in my bug-636158 branch
<lifeless> ><
<wgrant> Hah.
<wgrant> You are good at that.
<mwhudson> the problem with "finding the uses" is that they may well be in initscripts and so on that aren't in the tree
<wgrant> mwhudson: Exactly.
<wgrant> And it's easy enough to support them in a much less OMG MUST KILL way than they are now.
<wgrant> So I will.
<wgrant> There is also a set of options that some scripts take that do the same thing :/
<lifeless> mwhudson: losado grep
<wgrant> Also, shipit needs to die.
<lifeless> thats feature level work
<wgrant> Yes :(
<lifeless> it needs some apis
<lifeless> launchpadlib fixed to be usable in appservers
<lifeless> then its doable
<wgrant> Sure. I wasn't saying it was doable. Just that it pisses me off when I have test failures because bzr grep doesn't catch stuff outside the tree.
<wgrant> Why did staging not try to restore over the weekend?
<jml> wgrant: +1
<lifeless> I need a reviewer
<lifeless> http://bazaar.launchpad.net/~lifeless/launchpad/bug-636158/revision/12413
<jml> pisses me off even more when I have deployment failures because of the same.
<lifeless> this is a refactoring
<lifeless> it has some stuff I'm not entirely happy with, but I don't konw if there are existing facilities in tree, or if I should make some.
<wgrant> lifeless: My first comment is that the method should probably be on IPublishingSet, not IDistroSeriesSet.
<lifeless> why ?
<wgrant> Because all this publishing-related stuff on DS bloats DS massively.
<wgrant> It doesn't belong there.
<lifeless> mmm, thats fair enough, but why does it belon on publishing set ?
<lifeless> why would someone look there rather than distro/distroseries ?
<wgrant> Because in the end you are asking for the latest publication.
<lifeless> thats an implementation detail
<lifeless> isn't it ?
<wgrant> No.
<wgrant> I don't think so.
<lifeless> theres a related oddness
<wgrant> There's more than one.
<lifeless> distro.getCurrentSourceReleases != distro.development_series.getCurrentSourceReleases
<lifeless> and it appears to be deliberate
<lifeless> but I'm not sure users will understand
<wgrant> I didn't know that both existed.
<lifeless> indeed
<wgrant> Distribution's finds the latest in any series?
<lifeless> there would be much less duplcicate code if they didn't both exist
<lifeless> correct
<wgrant> That's braindead.
<wgrant> But Bugs uses it.
<lifeless> indeed
<lifeless> indeed
<wgrant> I don't recall anywhere else that does.
<lifeless> I'm in correctness-preserving refactoring at the moment
<lifeless> so I am not going to fiddle
<wgrant> And I filed a bug on the Bugs behaviour a few years ago.
<wgrant> :(
<lifeless> but - it makes me care less about the duplication I have that I was worried about
<lifeless> because we can just delete the method in a separate pass
<wgrant> Right.
<wgrant> Make sure you file a bug.
<lifeless> where is the bug you filed ?
<lifeless> (because that would be the one to use)
<wgrant> Bug #279513
<_mup_> Bug #279513: Distribution source package tooltip in bugtask table shows most recent SPP <lp-bugs> <ui> <Launchpad itself:Triaged> < https://launchpad.net/bugs/279513 >
<wgrant> Not as long ago as I thought.
<lifeless> so
<lifeless> it could show most highest version
<wgrant> Or latest in the dev series.
<lifeless> right
<lifeless> which I think is better and simpler
<wgrant> Definitely.
<thumper> hmm... it seems that Matcher and Mismatch aren't in the __all__ for testtools.matchers
<thumper> I'm guessing they should be
<jml> yeah, they should.
<jml> they might even be in trunk.
<lifeless> They were skipped deliberately
<lifeless> because from testtools.matchers import * shouldn't import the base classes
<lifeless> this was perhaps a mistake
<lifeless> erm
<lifeless> 400 Bad Request Bad Request Your browser sent a request that this server could not understand. Apache/2.2.14 (Ubuntu) Server at bugs.launchpad.net Port 443
<lifeless> wtf
<wgrant> What was the request? I've seen that on bad hostnames, but that's about it.
<lifeless> commenting on that bug
<lifeless> jml: as you are around
<lifeless> jml: testtools
<lifeless> I reviewed and so on; what did you think
<jml> lifeless: I haven't had a chance to look
<lifeless> kk
<lifeless> ok, I can has review please: https://code.launchpad.net/~lifeless/launchpad/bug-636158/+merge/50534
<lifeless> thumper: ^ if you have the time, would be awesome
 * thumper is writing some groovey matchers
 * thumper enlunchenates
<jelmer_> lesigh
<jelmer_> thumper: I almost looked up enlunchenates in my English->Dutch dictionary
<lifeless> Lol
<lifeless> grrr
<lifeless> so milestones pretend to be series related
<lifeless> but they are really pillar related
<lifeless> modelling fail
<jml> lifeless: sinzui will tell you a story about that one.
<lifeless> we need to say 'series in ... or milestone in ...'
<lifeless> rather than 'series in... group by series, milestone'
<lifeless> and our bug task search compiler is byzantine
<jml> what's the sanest way to synchronously download a small thing via the web in Python?
<lifeless> bzrlib.transport ?
<lifeless> its a sane way
<lifeless> but perhaps not /the/ sanest way
<lifeless> jml: whats the story?
<jml> lifeless: fetching a very small json dict
<jml> bzrlib.transport is fine
<lifeless> jml: I meant whats the story that sinzui would tell me
<jml> lifeless: oh, I forget the details. I think for 3.0 he & beuno wanted to do a big refactoring of series/milestone relationship and got blocked.
<jml> lifeless: not many people find "series" intuitive.
<wallyworld_> thumper: can you ping me when you're back?
<wgrant> lifeless: They made releases sane on top of milestones, but milestones atill aren't completely sane.
<wgrant> lifeless: Also, milestones do have a series...
<wgrant> But they also have a product, possibly because some old data doesn't have a series, which could be fixed :)
<jml> how do we feel about this output?
<jml> http://paste.ubuntu.com/569855/
<lifeless> jml: interesting
<lifeless> jml: would be cool to put that in aws-status
<jml> lifeless: it's somewhat specific to Launchpad ec2test.
<lifeless> why is it specifc?
<wgrant> jml: How does it do that? SSH in?
<jml> lifeless: it could be made more general, but right now it looks for special information about what the test run is and whether or not it's successful.
<jml> wgrant: no, just fetches a JSON file that remote.py creates & keeps up-to-date.
<wgrant> Ah, handy.
<thumper> wallyworld_: ping
<lifeless> jml: so, I know the bug you're looking at
<wallyworld_> thumper: mumble?
<thumper> wallyworld_: ok
<lifeless> jml: and I suspect this won't meet the goal of it, though IMBW
<lifeless> jml: won't this fail if e.g. your ip has changed?
<lifeless> and isn't the goal of the bug to find things that have gone awol ?
<lifeless> oh man
<lifeless> bug task bug summaries are so broken right now
<lifeless> pop quiz
<jml> lifeless: if you don't have permission to hit those web pages, then yes, it will fail.
<jml> lifeless: it can still show instances and uptimes though.
<lifeless> should a bug be counted once per distroseries for aggregation into milestones ?
<jml> which will be good enough for that case.
<lifeless> jml: I would like this stuff to live outside the lp tree
<jml> lifeless: too late?
<lifeless> I think its great you're improving it
<lifeless> jml: its never too late
<wgrant> What's the process for shipit branches?
#launchpad-dev 2012-02-13
<wallyworld_> wgrant: i hadn't forgotten the qa - i needed to finish some stuff first. did you check that the branch badges displayed ok?
<wgrant> wallyworld_: Yep, seems to work fine.
<wallyworld_> ok. thanks.
<wgrant> I had 12 QA items of my own, so it wasn't much extra work :)
<wallyworld_> np. was just uploading a new oops-amqp to pypi for james_w
<wallyworld_> wgrant: so now that oops-amqp is at 0.0.6 on pypi, what causes the ubuntu package to get updated?
<lifeless> wallyworld_: nothing; they probably have a watch file tht will alert the package maintainer
<lifeless> wallyworld_: we don' worry about that at the moment
<wallyworld_> ok. thanks. just wanted to be sure i didn't stuff anything up
<lifeless> wallyworld_: you did an sdist upload -s ?
<wallyworld_> yes
<lifeless> wallyworld_: close off any fix committed bugs in lp; and you're done AFAIK
<wallyworld_> had to register my ssh key etc
<wallyworld_> wgrant: StevenK: if i ask nicely, can one of you +1 rick's review so i can get this landed? https://code.launchpad.net/~wallyworld/launchpad/private-by-default-ui-885503/+merge/92273
<lifeless> interesting - https://www.ebayopensource.org/index.php/Turmeric/HomePage
<StevenK> Sorry, you lost me after Java.
<wgrant> wallyworld_: Looking
<StevenK> lifeless: But teams don't have gpg keys. You'd ask the owner the sign the CoC and be responsible for the PPA?
<lifeless> StevenK: there are a number of ways it could be implemented
<lifeless> StevenK: one; force every member to be a CoC signer
<lifeless> two; reject uploads from non CoC signers
<lifeless> three; I'm not sure right now, but give me a few minutes :P
<StevenK> 1) Could result in "Damn it, I can't create a PPA because x, y and z haven't signed the CoC, and they're on holidays."
<lifeless> indeed
<wgrant>  * 471 Exceptions
<wgrant> Yay
<lifeless> nice
 * wgrant fixes 110 of them
<wgrant> wallyworld_: Done
<wallyworld_> wgrant: thank you
<wgrant> wallyworld_: Also, you can't land this until the DB patch is deployed.
<wgrant> Oh, you won't be thanking me :)
<wallyworld_> yeah, hopefully tomorrow
<wgrant> I hate our hand-generated SQL.
<wallyworld_> wgrant: thanks for review - i've responded
<wgrant> wallyworld_: My premise was that the product argument to person.hasCurrentCommercialSubscription should not exist.
<wallyworld_> why? i think it's fine to ask the question "does the user have a commercial subscription to this product". we also discussed this on the standup last week and curtis was ok with it
<StevenK> Didn't we say that on the call on Friday too? :-)
<wgrant> That's not a sensible question.
<StevenK> wallyworld_: Right, but both wgrant and I disagreed.
<wallyworld_> 2 vs 2 :-)
<wgrant> A product has a commercial subscription.
<wgrant> A person is not involved.
<wallyworld_> the current api says that a person is involved - the one before i modified it i mean
 * StevenK stabs this branch, pulls the knife off and it stabs it again.
<wallyworld_> since i can't land till db patch lands, let's discuss again tomorrow on the call?
<wgrant> That was written two weeks ago to see if a person has any projects which have subscriptions.
<wgrant> It's a suboptimal hack to make private teams more accessible.
<wgrant> It must not be perpetuated.
<wgrant> Replied.
<wallyworld_> why suboptimal hack? i think it's ok to have such a method
<james_w> thanks wallyworld_!
<wallyworld_> james_w: np. the 0.0.6 version is on pypi too fwiw
<wgrant> It's conflating two checks that your method should be performing
<wgrant> Aha
<wgrant> Now freenode is alive.
<james_w> wallyworld_, great, thanks
<StevenK> wgrant: Can you please look at http://pastebin.ubuntu.com/839965/ which is the test failure I'm debugging, along with a current diff before I slam my head through my monitor and the wall behind it in frustration?
<wallyworld_> wgrant: re: Unauthorised - that is used in many other places for similar semantics
<wgrant> NAFAIK
<wgrant> All the 'raise Unauthorized's that I can see are cases of inadequate role membership.
<wgrant> statik: Was visibility formerly public?
<wgrant> Er
<wgrant> StevenK: ^^
<StevenK> wgrant: The Unauthorized I'm not concerned with yet, it's the rest of the doctest failures
<wgrant> StevenK: They are likely cascading failures.
<StevenK> wgrant: What makes you think that?
<StevenK> Oh, that the team hasn't actually be turned private?
<wgrant> Right.
<StevenK> wgrant: visibility doesn't appear in configure.zcml, just IPersonCommAdminWriteRestricted
<StevenK> It has a require permission .Commercial, and it's allowed in IPerson's interface
<wgrant> <allow /> == <require permission="zope.Public" />
<StevenK> Right.
<StevenK> It was publically readable, and only settable for .Commercial
<StevenK> Now, how do I fix that ...
<wallyworld_> wgrant: inadequate role membership and not having the required previliges to turn on private bugs is pretty much the same thing, no? i don't thing bad request is a correct alternative regardless
<StevenK> wgrant: So I guess IPersonEditRestricted is the wrong place for visibility
<wgrant> wallyworld_: It's not really a lack of privilege of the person. It's a lack of privilege of the project.
<wgrant> wallyworld_: So it's slightly different, and not really something we use Unauthorized for elsewhere right now.
<wallyworld_> which still results in the person being authorised
<wallyworld_> hmmm
<StevenK> wgrant: Getting closer. First failure is now "- A private team cannot change visibility."
<wallyworld_> if the person were in a different role eg ~registry then they would be authorised
<wgrant> wallyworld_: Right, but that's not usually how people would solve it.
<wgrant> In order for Joe Random to solve the error, he has to change the project.
<wgrant> Not become a member of ~registry.
<wallyworld_> sure, but i used it to illustrate that it can be considered a limitation of what the person is allowed to do
<wgrant> It could go either way, but I think BadRequest is slightly better.
<wallyworld_> but badrequest is for stuff like protocol issues etc
<wallyworld_> afaiui
<wgrant> We use a derivative of BadRequest for most user errors.
<wallyworld_> why?
<wgrant> Because in most cases they're bad requests.
<wallyworld_> really? the syntax is bad?
<wgrant> Hm?
<wgrant> Bad Request doesn't meant the HTTP syntax is bad.
<wallyworld_> i thought it was a request that could not be fulfilled because not all info was provided or whatever
<wallyworld_> different to not being allowed to do something
<wallyworld_> so like leaving out a mandatory parameter = bad syntax
<wallyworld_> but here there is no such issue but there is an authorisation issue
<wgrant> A 403 is possibly more appropriate, but the rest of the API uses 400 here.
<wgrant> 401 is not appropriate for normal users, as becoming a ~registry is not a way out for them.
<wallyworld_> i wasn't suggesting becoming ~reg was a way out
<wallyworld_> i think 403 is ok
<wallyworld_> i'd like to know why the rest of the api uses 400 in these cases
<lifeless> so 400 is definitely wrong per rfc2616 but
<lifeless> httpbis changes this
<lifeless> http://tools.ietf.org/html/draft-ietf-httpbis-p2-semantics-18#section-7.4.1
<wgrant> The RFC isn't relevant here.
<wgrant> It's 15 years behind the web.
<lifeless> specifically, it says "The server cannot or will not process the request, due to a client
<lifeless>    error (e.g., malformed syntax)."
<lifeless> wgrant: a little patience kemosabi
<lifeless> wgrant: and you might find your local friendly http nazi supporting your case
<wgrant> Heh
<lifeless> 400 has become the defacto 'go away and it is your fault'
<wgrant> Exactly.
<lifeless> you could return 401 if you want to trigger *http* authentication
<lifeless> but
<lifeless> we don't use http authentication anywhere
<lifeless> so we should never use 401 or 403, more or less
<wgrant> 403 is fine
<wgrant> 401 is illegal, but we do it in the API anyway
<wallyworld_> lifeless: isn't 'go away, it's your fault' different to 'you sent all the right info to do this but you are not allowed ie forbidden"
<wallyworld_> so why should we never use 403?
<lifeless> same reason 401
<wgrant> IIRC 401 says the response MUST include WWW-Authenticate
<lifeless> 401 and 403 are talking about the ability and limits of dealing with http authentication credentials
<wgrant> 403 doesn't say anything about HTTP auth
<wallyworld_> what he said
<wallyworld_> hence 403 is ok here i think?
<wgrant> I still think 400 is more in line with the rest of the application, and indeed the rest of the modern world.
<wallyworld_> well the world is wrong :-)
<lifeless> http://tools.ietf.org/html/draft-ietf-httpbis-p2-semantics-18#section-7.4.4 if you want to read it
<wallyworld_> i still don't get why the world would be using an incorrect code
<wallyworld_> ok, i'll read it
<lifeless> I see no value in spec lawyering without reference to the current spec
<wgrant> Current *draft* spec that's already obsolete? :)
<lifeless> final stage about to be ratified
<wallyworld_> reading that link reinforces my view that 403 is ok here
<wallyworld_> but i'll make it 400 if that's what i need to do
<StevenK> Hmmmmm.
<wallyworld_> even though it's not a 400 ie all necessary info for the server to process the request was provided correctly
<StevenK> I think hasCCS has a bug
<wgrant> wallyworld_: "due to a client error" in the new spec.
<wallyworld_> but it's not a client error
<wgrant> It is.
<wgrant> It's asking for something which can't be done.
<lifeless> wallyworld_: so the sequence is - you start with nothing, you make a request, server says 401 (gotta be authenticated), so you auth and try again and it says 403( you picked the wrong usercode)
<wallyworld_> agree so far
<wallyworld_> or a user code that's not allowed
<lifeless> wallyworld_: 401 (which references http://tools.ietf.org/html/draft-ietf-httpbis-p7-auth-18) makes no sense if you are not talking about HTTP auth
<wallyworld_> yes, i should not have used a 401 in the mp. i should have used 403
<lifeless> 403, which layers on 401 - telling you to go back and reauthenticate - makes no sense unless 401 makes sense.
<lifeless> 403 is flat out wrong for us.
<wgrant> 403 doesn't layer on 401
<wgrant> In practice it did 15 years ago.
<wgrant> But even 1.1's 403 doesn't mention 401, or HTTP auth at all.
<lifeless> 403 is what you use to trigger a browser login window regardless of their current login status.
<wgrant> No, that's 401
<wgrant> 403 doesn't; the webapp uses it.
<StevenK> Meh, let's just raise 402
<wgrant> 402 is actually almost appropriate here, for like the first time ever :)
<wallyworld_> hah
<StevenK> I know :-)
<StevenK> (402 is Payment Required, for those playing at home.)
<StevenK> AH! I get it!
<StevenK> The logged in user has a subscription, but the new team doesn't.
<wgrant> Ah, finally.
<wgrant> https://launchpad.net/launchpad renders in <1s now.
<wgrant> Next stop <500ms.
<StevenK> wgrant: If I change TeamEditView to have class schema(ITeam):\n visibility = copy_field(...  I get TypeError: ('Could not adapt', <Person at 0xfb05050 team-name-100028 (Team Name 100028)>, <InterfaceClass lp.registry.browser.team.schema>) in the tests.
<wgrant> StevenK: That's inconvenient.
<StevenK> But it works for TeamAddView and ITeamCreation
<StevenK> wgrant: Can I force Zope to give me a better error?
<wgrant> StevenK: You may have to do it in setUpFields with copy_field. See BugTaskEditView
<StevenK> wgrant: Down to one failure.
<wgrant> :(
<StevenK> Hmm?
<wgrant> failplanner is fail
 * StevenK stabs longpoll
<StevenK> wgrant, wallyworld_: Do either of you want to review https://code.launchpad.net/~stevenk/launchpad/make-person-visibility-mutate/+merge/92715 ?
<wallyworld_> StevenK: i'll do it
<StevenK> Grrr. Now I have to wait 7 hours to see if combo-url breaks qas.
 * StevenK glares at wallyworld_.
<wallyworld_> what have i done this time?
<StevenK> wallyworld_: I was expecting combo-url to be the first branch to land this morning, and it wasn't.
<wallyworld_> the early nerd gets the worm
<StevenK> I tossed it at ec2 at 7:40!
<wallyworld_> mine was a small test fix which i lp-landed
<wallyworld_> how long till gary makes bb run in 10 seconds?
<StevenK> Years?
<wallyworld_> StevenK: 41	+ visibility = data.get('visibility')..... i think you will find you need to del data['visibility']
<wallyworld_> if you have made visibility field readonly, unless you remove it from data, it should error
<wallyworld_> if your tests pass as is, i'm not sure why
<StevenK> wallyworld_: It was made readonly in the interface due to the mutator method. Like I said, I had to do magic to get it to NOT be readonly for the two views.
<wallyworld_> StevenK: IPerson.visibility is readonly and doesn't updatecontextfromdata() attempt to do "context.visibility = data('visibility')"
<StevenK> It doesn't seem to.
<wgrant> Hm, I would expect it to.
<wallyworld_> it does field.set(...)
<StevenK> wallyworld_: I'm happy to delete it from data after I grab it
<wallyworld_> i think you need to but i'm concerned no test failed
<wallyworld_> there's several other places where a readonly field is set using a mutator and it has to be deleted from the data dict
<StevenK> wallyworld_: I think what is perhaps going on is the mutator is setting it, and then it gets changed to the same value
<wallyworld_> ah, that may explain it perhaps
<StevenK> wallyworld_: http://pastebin.ubuntu.com/840063/
<wallyworld_> StevenK: i think that is ok. it's more explicitly correct if that makes sense
<wallyworld_> StevenK: i'm wondering if the new code in setupFields should be moved to conditionallyOmitVisibility() and the method renamed to setupVisibility(0 so that it all lives together?
<wgrant> lifeless: Is it evil to have partial indices where distribution IS DISTINCT FROM 1? :)
<StevenK> wallyworld_: wgrant didn't think it belongs in conditionallyOmitVisibility(), and I agree.
<wgrant> StevenK: It might be OK if you rename it.
<wallyworld_> not if it's not renamed
<wgrant> But I forget what it was.
<wallyworld_> but it's code that's all to do with initing the view wrt the visibility field
<StevenK> wallyworld_: The other thing is I only need to do setUpFields for the Edit view
<StevenK> wallyworld_: The class schema is fine for the Add view, so it doesn't need to be done
<wallyworld_> let me have a closer look
<wgrant> StevenK: I'd use the same for both.
<wgrant> No point duplicating it.
<StevenK> I was happy with class schema, but it gave the can not adapt error for Edit
<wgrant> Sure, so it's not useful.
<wgrant> If you have one solution that isn't vile and works for both, use it for both.
<StevenK> wallyworld_: What do you think of http://pastebin.ubuntu.com/840071/ ?
<wallyworld_> StevenK: i think that looks nicer thanks. maybe s/delete_visibility/omit_visibility to user consistent terminology?
 * wallyworld_ stabs unity. another crash, another set of duplicate icons on the launcher
 * StevenK wields s/delete\(_visibility\)/omit\1/g
 * wgrant curses the bug sort tiebreaker.
<wgrant> Hmm
<wgrant> By doing partial indices by openness I can get the main query on https://bugs.dogfood.launchpad.net/ubuntu down to 6ms
<wgrant> Which is possibly a good thing.
<wallyworld_> StevenK: should the transitionVisibility method check that new visbility != old visibility before it complains?
<StevenK> wallyworld_: Hmmmm.
<StevenK> Probably
 * StevenK wishes for a Perl-ism
<wallyworld_> StevenK: and there's no difference between turning it on vs off?
<StevenK> #define it ?
<wallyworld_> visibility
<wallyworld_> in terms of permissions i mean
<StevenK> Sorry, I don't get it. Please be clearer.
<wallyworld_> so for the private bugs setting, you need more privileges to turn private bugs on than to turn it off
<wallyworld_> but i guess for visibility it's symetrical
<StevenK> Right
<wallyworld_> ok
<StevenK> You can either it change it, or you can't.
<wallyworld_> np. sorry
<wgrant> It's probably not symmetrical.
<wgrant> eg. if my commercial subscription expires, surely I can unprivate my team?
<wallyworld_> StevenK: i'm also wondering why we're not using validate() to setField error
<wallyworld_> oh, atm, field is not shown if not permitted
<wallyworld_> so no need
<wallyworld_> this is turning out very much like the private bugs branch
<StevenK> wallyworld_: Yes.
<StevenK> wgrant: Mmmmm. I think I'd like to do that as a seperate branch, this one is already hairy enough.
<wallyworld_> StevenK: what is nagging at me for this branch is that the rules for visibility allowed or not are in 2 places - the view to omit the field or not plus the model to do the save
<wgrant> Well I think I'd like postgres to be smart and do a union over sorted index components, but it won't :)
<StevenK> wallyworld_: Right. Perhaps I need to refactor it out.
<wallyworld_> in my branch, i now have a checkPrivateBugsTransition() method which raises if not allowed. the validate() calls this and uses ex.message to set the field error
<wallyworld_> and the model nutator also calls checkxxx() which raises if the api is used
<wallyworld_> for example
<wallyworld_> so it's trying to put all the permission checks in one place
<StevenK> wallyworld_: So I can refactor out into IPerson.checkAllowVisibility if you think that's appropiate
<wallyworld_> StevenK: i think the same technique is used in the team edit view for the team moderated/restricted etc radio buttons
<wallyworld_> StevenK: i think that works
<wallyworld_> StevenK: with those moderated/restricted radio buttons, i catch the exception and use the message to put in that little info line at the top of the widget
<StevenK> wallyworld_: Anything else?
<wallyworld_> StevenK: don't think so.you'll need to add different tests depending on what changes you make. there should probs be a -ve test for the test_person_with_cs_can_create_private_team test
<StevenK> I'm so over this branch
<wallyworld_> StevenK: that's how i feel about the private bugs one :-(
<StevenK> wallyworld_: Er, but if the user has no CS, they don't see the field?
<StevenK> There are existing tests for that case, the branch adds an end-to-end test to make sure the team gets created.
<wallyworld_> StevenK: yes, you are right. i was thinking about any future asymetric permission workflow
<StevenK> Which we haven't done yet. :-P
<wallyworld_> StevenK: so i've got to run to the shops before it hails (sky is green and hail is expected). i'll +1 after dinner
<StevenK> Bleh
<wallyworld_> or sooner if you want
<wallyworld_> give me 20 minutes?
<StevenK> wallyworld_: Yeah, sure.
<wallyworld_> thanks. will do it as soon as i can
<wallyworld_> StevenK: r=me with a suggestion for a test for the ImmutableVisibilityError
<wgrant> stub: Do you have much idea how much DB IO we'd save by not reading large fields like Bug.description during searches?
<stub> Are we scanning them, or looking at the fti indexes?
<wgrant> FTI, so the description column isn't used at all.
<wgrant> Except Storm asks for it, even though it won't use it.
<stub> I have no real metrics. It would certainly be faster to display a list of 70 odd bugs (a batch from a search results). I don't know how much.
<wgrant> It's hopefully the only large toasty field that Storm asks for
<wgrant> So it seems like we could save a lot of toast table reads.
<stub> For the actual search, the fti index is used - removing description would make it smaller but I don't think it would save that much.
<wgrant> Oh, blah, fti gets rechecked even after the GIN.
<wgrant> So I guess we can't skip toast reads for that, only description.
<wgrant> stub: I'm going to experiment with using GIN for tag searches tomorrow.
<wgrant> Since they're currently the slowest common search in my new schema.
<stub> wgrant: So the trick with PG < 9.1 is that you need to determine if the query is going to do a full index scan and fail it, rather than generate an OOPS because the index gets asked to do something it doesn't support.
<stub> With 9.1, GIN is drop in - we just get slower writes and faster reads
<wgrant> Yeah, but I'm hoping we'll be on 9.1 soon enough :)
<wgrant> Since it makes Ubuntu searches 5x faster.
<wgrant> s/it/GIN/
<stub> Yes. I'll be converting all our GIST indexes to GIN.
<wgrant> stub: Could you please pastebin index usage numbers from bug and bugtask?
<stub> k
<stub> I should snapshot these like I do the disk space ones for generating deltas
<wgrant> Yeah, that sounds pretty easy and potentially very useful.
<wgrant> But not critical for my uses right now :)
<stub> https://pastebin.canonical.com/60003/
<stub> couple of obvious dead ones there...
<wgrant> I'll be deleting most of them in a month or so anyway :)
<StevenK> bugtask__binarypackagename__idx seems utterly useless
<mrevell> Hello!
<wgrant> StevenK: Considering the column should never have existed, and hasn't been used since at least 2005, yeah.
<wgrant> It's all part of The Plan.
<StevenK> wgrant: But like you say, it's probably not worth just deleting the unused indices yet
<wgrant> Right, considering I will be taking a chainsaw to those tables in the coming little while.
<StevenK> I'm looking forward to it.
<StevenK> stub: Is it easy to see if any indices aren't used at all, ignoring those on bug and bugtask?
<stub> Yes. There are false positives though, such as indexes rarely used but need to be there anyway to support that once-every-three-months operation
<StevenK> If they're used once-every-while, they should have some non-negative read and scan results, right?
<stub> I also can't generate deltas, so the numbers are from whenever the stats were last reset, and I don't know when that was :)
<adeuring> good morning
<StevenK> I can't imagine we reset the stats at all. Some of the those numbers look very large.
<stub> https://pastebin.canonical.com/60004/
<stub> That needs to go in the database report really
<StevenK> Holy crap, 208
<StevenK> Er, 280
<StevenK> bugnotificationarchive ? Really? Is that even used?
<stub> That really should be filtered by num tuples in the table or something too. There are a lot of indexes there to support deletions and such if the tables grow.
<StevenK> stub: If you don't think the gardening is much of a help, that's fine, I'm effectively offering to grab a pair of shears.
<stub> The gardening will help. Just needs a little thought on each one though, which I'll double check on review.
<stub> Leave all the indexes on person
<stub> And libraryfilealias
<stub> And the primary keys, and uniques...
<stub> Hmm... can I filter them out?
<StevenK> Hmm, like on the indexrelname?
<stub> _key, _unq, _pk, _pkey are good hints, but we don't always follow conventions unfortunately.
<mrevell> Everyone say hello to czajkowski, who joins the LP team at Canonical today :)
<stub> StevenK: https://pastebin.canonical.com/60006/ is better to work from. It skips all the unique indexes, and shows the number of tuples in the table (for indexes that would be used if we had more data)
<stub> Hello czajkowski, who joins the LP team at Canonical today
 * czajkowski waves hi 
<wgrant> Morning czajkowski.
<stub> What timezone are you in czajkowski?
<czajkowski> UTC, I live in London
<stub> In the office or at home?
<czajkowski> at home, will pop in from time to time in the office though as I live nearby
<adeuring> morning czajkowski
<czajkowski> wgrant: adeuring ello :)
<stub> I hated commuting on the tube for the few months I was there.
<czajkowski> it's about a 15-20 min walk from here
<StevenK> stub: Haha, did you see tm__potmsgset__language__not_used__idx in that?
<nigelb> 34
<nigelb> ugh
<StevenK> rick_h: O hai!
<rick_h> StevenK: howdy
<rick_h> how went the whole thing? Saw your email, will probably get a reply going with some notes.
<StevenK> rick_h: Sweet. Buildbot finished ~15 minutes ago, I was hoping that either qas would be a burning ball of fail or working fine before I went to bed.
<rick_h> StevenK: heh ok. So it's not hit qas yet then?
<StevenK> rick_h: qas takes ~40 minutes to update.
<rick_h> ok
<StevenK> Next cron for qas kicks off at :48
<StevenK> I doubt it hit the :18 run
 * rick_h crosses fingers
<jelmer> mogguh StevenK, rick_h
<StevenK> jelmer: G'tag (if you remember that one :-)
 * rick_h looks up mogguh
<jelmer> StevenK: yeah :)
<StevenK> rick_h: Did you get a chance to look at the AJAX log thing?
<rick_h> StevenK: not yet, created a ticket for it and it's on the board
<StevenK> s/ticket/card/
<StevenK> Ticket makes think of RT.
<rick_h> sorry, created a bug and a card and the card is on the board :)
<rick_h> ticket == bug sorry
* benji changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: benji | Firefighting: - | Critical bugtasks: 4*10^2
<deryck> Morning, all.
<deryck> Oh happy uninterrupted-duty day!
<rick_h> lol
<flacoste> morning deryck!
<deryck> abentley, adeuring, rick_h - https://plus.google.com/hangouts/extras/talk.google.com/orange-standup
<deryck> mrevell, my wife's van died at her shop where I'm working. And a tow truck is coming to get it. I'll have to go out when he arrives (in 10 mins likely).
<deryck> mrevell, should we push back our call with czajkowski or schedule for tomorrow?
<czajkowski> deryck: ello
<deryck> hi czajkowski!  You have no idea how happy the Orange Squad is that you have joined Launchpad! :)
<czajkowski> I'm going to hope this is a good thing :)
<deryck> czajkowski, it is! :)
<deryck> flacoste, look at my luck with this stupid van ^^ :)
<mrevell> czajkowski, deryck: Tomorrow is fine if you're not sure when you'll be done with the tow truck. If not, how about 16utc? I won't be available then but you and czajkowski can talk.
<deryck> mrevell, either way is fine with me.  Shouldn't take but about 10-20 minutes to get him a key and on his way.  but unfortunately, it just hits during our call.
<deryck> mrevell, I called when I got in this morning, but he's only now able to come here.
<mrevell> No prob :) Let's say 16utc, if that suits you czajkowski.
<flacoste> deryck: is it again your wife's doing this time? her general delicacy in handling mechanical/technology device?
<deryck> ha!
<czajkowski> mrevell: deryck sounds good to me
<deryck> flacoste, one can never know for sure. :)
<deryck> mrevell, czajkowski -- sounds good to me too.
<mrevell> great :)
<deryck> flacoste, I think it's just an alternator though, which is just unfortunate timing.
<flacoste> ah right
<flacoste> that sucks
<sinzui> jcsackett, are you back yet. I shot myself in the foot. I want to talk about invalidating a cache to apply more than one commercial subscription to a projects
<jcsackett> sinzui: i'm back now. my apologies, my timing estimates didn't account for a traffic accident on the way home.
<rick_h> jcsackett: quit running into people. It's not good for you
<jcsackett> rick_h: thankfully, i wasn't *in* the accident. i was just blocked by it. :-P
<sinzui> That accident was a MiB conspiracy. Your domicile is bugged
 * jcsackett laughs
<sinzui> mumble?
<sinzui> jcsackett, or hangout
<jcsackett> let's go mumble. google+ was crashing my computer yesterday and i don't yet trust it again.
<rick_h> deryck: so finally changed my last .html file for a bit hopefully! https://code.launchpad.net/~rharding/launchpad/combo_yui_tests5/+merge/92092
<deryck> nice!
<rick_h> deryck: if you get a few ^ first one is on ec2 and once that goes can land these other 4 side by side
<deryck> rick_h, is that a review request? :)
<rick_h> deryck: yes please :)
<deryck> rick_h, will do :)
<rick_h> I htink benji would hunt me down if I hit up ocr
<deryck> meh. what's 4200 lines among friends.
<rick_h> heh, across 34 files
<sinzui> jcsackett, bzr+ssh://bazaar.launchpad.net/~launchpad-purple-squad/%2Bjunk/disclosure_mockups/
 * benji prepares the specially-trained rick_h hunting dogs.
<czajkowski> deryck: nice to put the face to the nick btw
<deryck> czajkowski, thanks!  Likewise.
<abentley> deryck: With further optimization, I completed the branch scan in 3 minutes!
<deryck> abentley, woah!  nice!
<abentley> deryck: I had to use store._connection.build_raw_cursor().executemany to do it, so there's some work required to de-hackify it, but at least we know it's possible.
<deryck> abentley, nice that it's possible, and yuck that it's so non-obvious to do multiple inserts.
<lifeless> mornin
<sinzui> czajkowski, https://dev.launchpad.net/MaintenanceRotationSchedule
<abentley> lifeless: mornin.
<abentley> lifeless: I'd like to chat with you about task queues.  Are you up for that sometime today?
<lifeless> sure
<lifeless> how many hours till your EOD ?
<danilo> benji, heya, I've put a few merge proposals up for pygettextpo (https://code.launchpad.net/pygettextpo/+activereviews): it's used only by LP but I want to make it more complete and usable (and perhaps able to replace gettext_po_parser.py in Launchpad as well)
<benji> danilo: ok, I'll take a look
<danilo> lifeless, flacoste: I wonder if we should create a separate maintainer team for pygettextpo: I do want to contribute quite a bit of code to it, and I am guessing I might be the one with the biggest interest in driving that :)
<danilo> benji, thanks!
<lifeless> danilo: we can, but you should be in lp-canonical-emeritus anyway, which should give landing rights
<danilo> lifeless, right, cool, I am (in canonical-launchpad-emeritus)
<danilo> lifeless, (and actually, I was still in ~launchpad, so I left that team)
<lifeless> :P
<lifeless> So, while that remains true I wouldn't expect any unusual friction; you still count as a reviewer, the policies don't seem onerous or irrelevant, ...
<lifeless> but like I say, we certainly can give it separate governance if thats useful
<abentley> lifeless: EOD in 3+ hours.
<lifeless> abentley: ok, cool. I have 2 other calls (one with the UK) but we should be able to find a timeslot.
<danilo> lifeless, sure, I don't have a particular need for it, as long as this works
<lifeless> danilo: if it doesn't, lets try to fix it first (its a new thing after all) and then iterate from there
<abentley> lifeless: cool.
<danilo> lifeless, agreed, thanks
<abentley> lifeless: I can't find a non-hacky way of using cursor.executemany via Storm, but my tests show a significant performance win for branch scanning.  Do you know why it's not provided?
<rick_h> benji: if you have a sec to review: https://code.launchpad.net/~rharding/launchpad/ajax_log_930141/+merge/92827
<benji> rick_h: sure
<rick_h> StevenK: ^^ for you since I know you were asking about it
<rick_h> benji: ty
<flacoste> lifeless: i wouldn't speaking a little later today, that could make it convenient to talk to abentley
<lifeless> cool, thanks
<sinzui> Sweet, bringing thunderbird to focus also kills X.
<lifeless> abentley: lets try for a chat now?
<abentley> lifeless: sure.  Preferred chat tech?
<lifeless> abentley: I have a call with elmo but not sure just when; uhm skype is le trivial for me
<flacoste> lifeless: let me know when you are available
<sinzui> wgrant, I cannot hear you
<lifeless> flacoste: in 5 ?
<flacoste> lifeless: might be best to reschedule
<StevenK> rick_h: Yay!
<StevenK> rick_h: That looks awesome.
<flacoste> lifeless: unless your agenda is short
* benji changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: - | Firefighting: - | Critical bugtasks: 4*10^2
<lifeless> flacoste: we could time box to 15m and see? If you're past EOD lets just adhoc it tomorrow
<flacoste> lifeless: timeboxing to 15 minutes suits me
<wgrant> sinzui, wallyworld_: https://launchpad.net/ubuntu/precise/+localpackagediffs
<wgrant> See the column headers.
<lifeless> wgrant: ok, you have mail
<lifeless> I am popping out for a little bit, back soon
<sinzui> wgrant, StevenK, jcsackett, wallyworld_: I published part one of the terminology http://blog.launchpad.net/
<wgrant> lifeless: Thanks.
<wgrant> jelmer: Do you know what caused the performance regressions in your revision_history() branch last time?
* wgrant changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: wgrant | Firefighting: - | Critical bugtasks: 4*10^2
<StevenK> wgrant: Do I need to drop all indices first, or will dropping the table take care of everything?
<wgrant> StevenK: You can't drop the table directly. Change its schema to todrop instead.
<wgrant> upgrade.py does the special slony dance to drop them from there.
<StevenK> wgrant: Ah yes. But will the indices disappear as well?
<wgrant> StevenK: This is Entitlement?
<StevenK> wgrant: Yes.
<wgrant> StevenK: So, yes, indices go with the table automatically.
#launchpad-dev 2012-02-14
<lifeless> flacoste: http://blog.mountaingoatsoftware.com/agile-succeeds-three-times-more-often-than-waterfall
<StevenK> wgrant: I should fix security.cfg in the code branch, or the db branch?
<wgrant> StevenK: You can do it in the DB branch, so might as well do it there.
<wgrant> In fact, you have to do it there.
<wgrant> Because of person foreign keys.
<StevenK> Hmm, so I have to drop the column from person too
<StevenK> That is going to be fun.
<wgrant> Hm?
<StevenK> Oh, it's a join, not a column
<wgrant> Yes.
<rick_h> StevenK: ping
<StevenK> rick_h: O hai
<rick_h> StevenK: hey, I started landing the test fixes
<StevenK> I saw.
<StevenK> I think.
<rick_h> StevenK: in my dev work I noticed that make jsbuild doesn't update the build dir files
<rick_h> So I saw combobuild
<rick_h> but that didn't update an existing file either
<rick_h> I kept having to make clean && make to get the build dir updated to work on a js file as I worked
<rick_h> which was a tad slow as you can imagine
<rick_h> I tried to see if the combobuild didn't work right because it wasn't listed in PHONY, but that wasn't it.
<rick_h> I didn't get to look at all of it before EOD
<StevenK> bin/combo-rootdir will exit with a message if build/js/lp exists
<rick_h> StevenK: right, it basically ran without cp'ing the files
<rick_h> StevenK: we need some way to work on a .js file and getting it rebuilt like make jsbuild
<StevenK> rick_h: Right, so now that icing depends on the YUI directory inside build/js, I can't just blow away the entire directory with impunity like I used to.
<rick_h> StevenK: right, I have on my todo to work on a script you can run using pyinotify to auto rebuild changed files
<rick_h> but have things keeping me from getting to it yet
<rick_h> plus, that's going to be a pain since we've got build exceptions :/
<StevenK> rick_h: Right, so I've not come up with a good plan yet.
<rick_h> StevenK: ok, well that's part of it. Wanted to make sure I wasn't expecting something to work that wasn't
<rick_h> sounds like it's on the todo list then
<rick_h> which is ok, now that I know
<StevenK> I think I can delete everything but yui and then cope if yui is a symlink
<rick_h> yea, I was thinking of just making a temp make command that blew away /build/lp and rebuilt it
<StevenK> Meh, don't do that
<StevenK> Fix combo-rootdir to not be so naive :-)
<rick_h> it was a sledgehammer solution :)
<StevenK> rick_h: I'll look at fixing combo-rootdir today
<rick_h> yea, I'll look at it then. Thanks for filling me in
<rick_h> ok, well no big deal. No one's really using it yet
 * StevenK is destroying code and tables
<rick_h> StevenK: so qas ok? Is it setup I can run combo loader on qas if I set the FF?
<StevenK> rick_h: You can't set the flag, and no, it's still waiting for that RT.
<rick_h> ok, cool
<lifeless> wgrant: I have some bad news w/respect heat
<lifeless> wgrant: we're missing some update cases.
<lifeless> wgrant: I suspect dupes, subscribers and affects-me-too are not updating the bug record heat
<wgrant> lifeless: Are too.
<wgrant> at least affects-me-too is.
<wgrant> Subscribers work too
<wgrant> And I can't test duping non-destructively.
<StevenK> wgrant: Easy review https://code.launchpad.net/~stevenk/launchpad/kill-entitlements/+register-merge
<StevenK> BAH
<StevenK> When did that stop working?
<wgrant> Oh, you're afflicted too?
<wgrant> wallyworld_: ^^
<wgrant> We have a sane reproducer.
<StevenK> wgrant: https://code.launchpad.net/~stevenk/launchpad/kill-entitlements/+merge/92888
<lifeless> wgrant: really? oh good. I was watching heat fail to change on prod when doin stuff the other day
<wallyworld_> wgrant: what's up?
<wgrant> wallyworld_: The MP bug
<wgrant> We have a reproducer who is not stupid :)
<wallyworld_> bug nr?
<wgrant> Bug #929422
<_mup_> Bug #929422: Fails to refresh the URL when making a merge request <Launchpad itself:Triaged> < https://launchpad.net/bugs/929422 >
<StevenK> That's it
<wgrant> Apparently it replaces the page content inline?
<wgrant> That sounds extremely evil.
<StevenK> It should stop doing so.
<wallyworld_> if you can come up with a solution that can avoid replacing the page content, i'm all ears
<wgrant> wallyworld_: Redirect!
<wgrant> window.location = whatever
<wallyworld_> isn't that another server request though
<StevenK> It used to redirect and it was fine!
<wgrant> wallyworld_: Don't you have to make the request to get the content anyway?
<wgrant> It will just be a normal navigation request rather than an AJAX one.
<wallyworld_> hard to explain - an xhr request is made and it may error so we need to stay on the current page
<wgrant> Sure
<wallyworld_> so that if there is an error
<wallyworld_> if can be displayed
<wgrant> And when it succeeds it will give a 303, right?
<wallyworld_> atm it gives 200
<wgrant> Or 201 or something like that.
<StevenK> wallyworld_: The URL can updated via JS
 * wgrant reads the code.
<wallyworld_> right, so that's all i need to do
<wgrant> No
<wgrant> That's evil.
<wallyworld_> i just forgot to update the url via js
<wallyworld_> why?
<wgrant> We should avoid it if we can; it's not widely supported.
<StevenK> wgrant: Thank you for the +1
<wallyworld_> is it part of the standards?
<wgrant> It's part of HTML5
<wallyworld_> so only ie < 8 or something will have an issue?
<wgrant> wallyworld_: No, not even IE9 supports it.
<wallyworld_> sigh. i hate ie
<wgrant> It's a new API and what you're doing is reasonably evil.
<wallyworld_> fsvo
<wgrant> Replacing the page content entirely is unprecedentedly evil.
<wgrant> That is not arguable :)
<wallyworld_> i'll take your word for it i suppose
<wallyworld_> i mean, it's merely doing what the browser does when it gets stuff to render
<wgrant> document.write is evil :)
<wallyworld_> why? it's a public api, or?
<wgrant> There are lots of evil public APIs.
<wallyworld_> so the issue is that the lp form infrastructure returns the html resulting from doing a form submit, and the xhr call needs to do something with this data or else just throw it away and waste the rendering effort
<wgrant> I had assumed that you used an API call.
<wgrant> I didn't realise until a few minutes ago that you actually POSTed to +register-merge.
<lifeless> so, FWIW
<lifeless> I think avoiding roundtrips is good; but is this actually doing that ?
<wallyworld_> yes
<lifeless> in which case, the UI impact is a regression, but it is otherwise tolerable, at least in the absence of e.g. doing the form via *stache
<wallyworld_> i posted to +register-merge to reuse all the existing code and avoid writing a bunch of new stuff and keep easy backward compat with non js browsers
<lifeless> I heartily approve
<wgrant> I approve of eliminating roundtrips, but I disapprove of introducing yet another non-standard way of doing forms.
<wgrant> Particularly one that is fragile and uses widely discouraged APIs like document.write.
<wallyworld_> except i need to figure out a non evil way to handle the form submission via the xhr call without adding a rt and wasting rendering effort
<wallyworld_> maybe a server side redirect?
<lifeless> thats what LP normally does
<lifeless> Are you sure the xhr call isn't following it itself?
<wgrant> There's still likely to be a round-trip, but it's not too terrible for this sort of page.
<wallyworld_> it would cause the server side rendering to be duplicated
<wallyworld_> perhaps
<wgrant> The way we normally avoid duplication is by not POSTing to the view.
<wgrant> But using an API instead.
<lifeless> did you set out to eliminate roundtrips, or do something else?
<wgrant> That's how all the other AJAX forms work, TTBOMK.
<wallyworld_> lifeless: i set out to convert what was a bog standard html form into a page that allowed the server to check for issues and send back an error message for display without leaving the page
<wallyworld_> and still act like a form submission on success
<wallyworld_> and live within the lp form infrastructure
<lifeless> so, neither fish nor fowl :)
<wallyworld_> yeah, guess so
<wallyworld_> wanted to avoid too much new code / change etc
<lifeless> so, deconstructing this situation
<wallyworld_> perhaps should look at using an api
<StevenK> There is one?
<lifeless> the new thing here was doin it in the picker
<wallyworld_> the work was already 5 or 6 branches long
<StevenK> IBranch.createMergeProposal()
<wgrant> eg. AJAX milestone creation is the normal form mapped to the API
<lifeless> you could have, for instance, let the picker choose anyone, and show an error on submission
<lifeless> would not have been as nice
<wallyworld_> lifeless: the picker does a check, but the submission also needs to do a check
<lifeless> wallyworld_: I know
<lifeless> wallyworld_: the picker check is redundant
<wallyworld_> so i did both
<wallyworld_> lifeless: yes, but it gives a nicer user experience
<lifeless> indeed
<lifeless> which I applaud
<wallyworld_> catch the error early
<wallyworld_> in the workflow
<wallyworld_> so, i needed a way to hit submit button but not leave the page and refresh if there is an issue, hence i made the xhr call, but the lp form stuff gives back a 200 with the new html for the resulting page on success
<wallyworld_> hence my use of document.write to render - it's the same data over the wire after all
<lifeless> wallyworld_: I don't quite follow that last bit; why couldn't it just do what our other forms do - render with an error?
<lifeless> (I'm not sayin that is good or optimal; just trying to understand)
<wallyworld_> lifeless: sure. there is no single field  in error - it's a confirmation popup
<wallyworld_> not an error
<wallyworld_> and the form error stuff is a page refresh
<wallyworld_> which i didn't want
<wallyworld_> the user sees the confirmation popup and hits yes and a normal submission then occures
<wallyworld_> or if no, the popup goes away and they keep editing the mp
<lifeless> I understand, but how is that different to a fresh page with a 'please click here to ok adding the user as a subscriber to  ...' ?
<wallyworld_> yuk.
<wallyworld_> better , more web 2.0 experience
<wallyworld_> i didn't think doing a separate confirmation html page was a good thing to implement
<wallyworld_> so if no objections, let me see if a server side redirect works. might double render, not sure, but better than the current bug
<lifeless> StevenK: can you check with webdev toolkit and see if a 30x is returned from the appserver?
<StevenK> ... better?
<StevenK> I'll be putting up an MP shortly
<lifeless> wallyworld_: so, as a datapoint, *all* our forms today double the same form as the confirmation page for all data entry issues
<lifeless> wallyworld_: e.g. with red boxes around bad data etc
<lifeless> wallyworld_: fitting in with that may well have been much less effort, and certainly would have been more consistent
<wallyworld_> lifeless: but here, there is no bad data
<wallyworld_> there's no error
<lifeless> wallyworld_: model it as a default-hidden field and there is
<lifeless> that said, what we do today isn't all that nice
<lifeless> it is however js-free
<wallyworld_> i was trying to avoid propagating the non-nice stuff
<wallyworld_> and we are allowed to do js only stuff now, right?
<lifeless> that is still vague, sadly.
<lifeless> we need to pin down the exact non-js use cases.
<lifeless> there are, AFAIK, three:
<wallyworld_> well, too late, horse has well and truly bolted
<lifeless>  - login
<lifeless>  - apport bug filing
<lifeless>  - google indexing (all stuff we want indexed must have a server side render)
<lifeless> but, TTBOMK we haven't got a solid ratification around that bein the complete set, yet.
<wallyworld_> lifeless: yes, it was extra work, but you need to do that to establish a new way forward to get away from the old, "bad", stuff, even if we decide we want to do it another way, we need to blaze a trail
<wallyworld_> see what works, what doesn't
<lifeless> sure
<lifeless> I haven't, I hope, been critical so far
<wallyworld_> oh, no not at all
<lifeless> just trying to ensure I have a clear understanding
<wallyworld_> i appreciate a robust discussion, especially since i broke stuff :-(
<wallyworld_> i have a thick skin :-)
<wallyworld_> i do like the new behaviour, so would like to make that work rather than reverting to a confirmation page
<StevenK> You have to, to be a Queenslander
<wallyworld_> :-P
<wgrant> Damn, beat me to it.
<lifeless> anyhow
<lifeless> so, here is my take on it
<lifeless> we probably need two forms of forms
<lifeless> the bug filing / login case
<lifeless> everything else
<lifeless> we -don't- need server side rendering of forms, google doesn't need to index forms
<lifeless> the current approach for populating and validating forms / form widgets / etc is API calls, not view calls.
<lifeless> I'd rather we push on that unless we think its flawed, than overload the behaviour of server side forms
<lifeless> which this seems to do to me
<wallyworld_> lifeless: validation is done via form submission
<wallyworld_> the lp form view converts the post params into objects etc and then calls validate
<lifeless> wallyworld_: why? I mean, I know that is what you did, but API calls that mutate/create do their own validation
<wallyworld_> i'm relying on the standard lp form submission code to get the form data and convert to model objects
<wallyworld_> using the schema
<wgrant> The API does that too.
<wallyworld_> yes. but i also then resuse the exisiting form validation callbacks
<wallyworld_> but using the standard submit
<wgrant> Aren't they on the API too?
<wallyworld_> s/but/by
<wallyworld_> they are? how? one defines validate() on LaunchpadFormView
<lifeless> the parameters are typed
<lifeless> with schema validators attached to that (e.g. Choice())
<wallyworld_> sure, i'm not talking about param conversion, but validation
<wgrant> That should all be in the model.
<wgrant> The view is just meant to present it as HTML.
<wallyworld_> the LaunchpadFormView subclass offers a validate() method
<wallyworld_> which people use
<wallyworld_> and then call field.setError
<wgrant> Sure, but that should only be for formatting model errors as HTML.
<wgrant> Since the model should be doing the validation, not the form.
<wgrant> s/form/view/
<wallyworld_> should be but doesn't always
<wallyworld_> i'll have to recheck this particular case
<wallyworld_> to see what it does
<lifeless> wallyworld_: if the model does not do the validation, that is a bug
<wgrant> Exactly.
<wallyworld_> then we have lots of bugs i think
<wgrant> Because there is an API for this.
<lifeless> wallyworld_: We have had many OOPSes - criticals - due to exactly that.
<lifeless> this discussion has been good, I think I can suggest a path now
<lifeless> (the one above), will send to list
<wallyworld_> lifeless: wgrant: the RegisterMergeProposalView does do it's validation in the view - so bug :-(
<wallyworld_> there is no well defined pattern that is in common use across lp for this
<lifeless> well, I beg to differ
<lifeless> we have two patterns and a bunch of sometimes poorly migrated forms
<lifeless> the migration *should* have been to move all validation to API's
<wallyworld_> the code i look at often suggests otherwise :-)
<lifeless> so there is an expected layout
<lifeless> If there is an API, it *must* do its own validation.
<wallyworld_> agreed
<lifeless> If there is a view and an API, the view should catch appropriate errors from the model, rather than duplicating validation code
<wallyworld_> yep, but mostly don't
<lifeless> If there is a view and no API, its old unmigrated code.
<lifeless> wallyworld_: -> lots of bugs
<wallyworld_> yep
<wallyworld_> lifeless: would you consider it a bug if a factory method makeFooBar() caused a side effect of leaving behind an admin login such that a check_permission in a test erroneously passes?
<StevenK> I would.
<wgrant> I ran into a similar thing last week.
<wgrant> Except it was logging out.
 * wallyworld_ stabs factory 
<wgrant> I was tempted to wrap the factory in an interaction preserver.
<StevenK> Fix the factory to do with celebrity_logged_in():
<wallyworld_> in this case, makeRegistryExpert() is evil
<wallyworld_> yes, will fix, just wanted to check
 * wallyworld_ wonders how many tests will fail now
<StevenK> wallyworld_: You will probably get fallout
<StevenK> Ah, you guessed :-)
 * wallyworld_ sighs
<StevenK> Welcome to Launchpad.
<wallyworld_> is it too late to run away?
<wallyworld_> StevenK: what's bad is that the method just above uses the correct pattern, so somehow that was not noticed
<lifeless> wallyworld_: yes; an undocumented sideeffect is almost certainly a bug
<StevenK> lifeless: Yes, it returns a 303.
<lifeless> StevenK: thanks
<lifeless> wallyworld_: ^ :)
<lifeless> wallyworld_: server side is doing the right thing
<wallyworld_> for the register mp form?
<lifeless> yes
<StevenK> POST => 303 ; GET <MP ID> => 200
<StevenK> And then Firebug refreshes the Net panel, because it's evil.
<wallyworld_> ok, so if the xhr call gets back stuff with code=303 and content type = text/html and responseText = the page html, what do i do?
<wallyworld_> i could have sworn i got code = 200
<lifeless> you do
<lifeless> the browser follows the 303
<wallyworld_> ah right, i remember now
<lifeless> question your assumptions, always :)
<wallyworld_> yes, i did figure that out
<StevenK> wgrant: https://code.launchpad.net/~stevenk/launchpad/smarter-combo-rootdir/+merge/92897
<wallyworld_> i had ust forgotten
<wallyworld_> but i now recall my dilema
<wallyworld_> the server side 303 is mapped to a 200 by the browser
<wallyworld_> and the repsonseText has the html to render
<StevenK> It looked like two seperate requests to me
<wallyworld_> not if i recall correctly
<wallyworld_> but i could be wrong
<StevenK> In Firebug there was a POST, which returned a 303 See Other, and then a GET which returned 200 OK
<wallyworld_> but that's all at the browser level
<wallyworld_> i don't think the xhr call sees that
<wallyworld_> the xhr call does a post and the browser does it's stuff and ends up giving a repsonse with code = 200
<lifeless> StevenK: yes, thats right
<wallyworld_> so the xhr call is none the wiser that the redirect has occurred
<wallyworld_> so, even if i use an api, that's still an extra round trip
<wallyworld_> 1. the api call to do the submission, 2. the call to get the correct url on success
<wallyworld_> whereas now it's just the one form submission
<lifeless> wallyworld_: actually no, an API can be less roundtrips
<wallyworld_> it's only one now atm
<lifeless> wallyworld_: load the form template (moustache) on the initial page load; get the data back from the API (a single POST : reply) and render in client
<lifeless> wallyworld_: its 2 POST:303 + final url:200
<wallyworld_> it will be a huge job to convert the mp page(s) to mustache
<wallyworld_> i was trying to keep it all simple
<wallyworld_> ah yes, it is 2.
<wallyworld_> i'll have a look after i get this current branch done
<wallyworld_> now that i have figured out how factory is screwing me over
<lifeless> sure, like I said though, we need a consistent story
<lifeless> what you're doing isn't consistent, nor is it any faster than the API can/should be in principle [modulo headdesk bugs, of course]
<lifeless> it might be more efficient for porting existing forms
<wallyworld_> but it is a much nicer user experience
<wallyworld_> if we stay consistent, we keep the crappy ui forever
<lifeless> no, you misunderstand me
<lifeless> we have other ajax forms with lovely UI
<lifeless> that don't do what you have done
<lifeless> of course we have to break consistency to migrate to something else
<wallyworld_> do they need to be backwards compatible though?
<wallyworld_> mine does
<lifeless> well, its not
<lifeless> (because it breaks non-js browsers)
<wallyworld_> no, it works
<lifeless> so you have all the overhead of server side widget stuff etc
<wallyworld_> for non js-browsers, it just submits as normal
<lifeless> you've got me tied up in knots
<wallyworld_> but you don't get the popup confirmation but you do get an informational
<lifeless> to be functionally equivalent you have to be able to ack the confirmation box
<lifeless> so you must have added a widget, which you said 'yuck' to before
<wallyworld_> here, it's not quite functionally equivalent
<wallyworld_> non js just get the informational message
<wallyworld_> after form submission
<wallyworld_> js get the popup
<lifeless> ok
<wgrant> wallyworld_: Doing it the old way (setting document.location) is no more roundtrips.
<wgrant> wallyworld_: You can tell XHR to not follow the redirect.
<wgrant> wallyworld_: So you'll get back a 303 with a Location
<wgrant> Then you set window.location = location
<wgrant> Done
<wallyworld_> ok, cool. didn't know that
<lifeless> so thats a decent answer here
<wgrant> Oh, blah, you can't actually block redirects yet.
<wgrant> So the cheapest may be to say if is_ajax then return a 201 instead.
<wgrant> It's still evil, but a far, far milder variety.
<wallyworld_> can't block redirects? not part of the standard yet?
<wgrant> Right, it may be in future.
<wallyworld_> i did look for how to do that but didn't see anything i could use
<wgrant> Right. So the view needs to return a 201 like the API, or you need to use the API.
<wgrant> BugSecrecyEditView already does something similar.
<wallyworld_> yes, i wrote the ajax bit
<wallyworld_> but it's used as a portlet inside a page
<StevenK> wgrant: Can haz review?
<wgrant> StevenK: Why don't we want to clobber yui(2)?
<wgrant> lifeless: For optimising counts, have we tried doing a "count = SELECT COUNT(*) FROM (real query LIMIT 10000); if count == 10000 then count = 'shitloads';" sort of thing?
<lifeless> wgrant: we haven't
<lifeless> wgrant: I can see a few ways that that won't help much
<lifeless> wgrant: but yeah, sure, that could be a good approach.
<lifeless> also grr @ bad triage from users that that pump and run
<wgrant> In some cases it won't help, sure.
<wgrant> But a lot of common queries just end up being an index walk.
<StevenK> wgrant: Because it isn't going to change.
<wgrant> StevenK: Not even with a YUI upgrade?
<StevenK> wgrant: YUI2 won't change, and we can't upgrade YUI past 3.3 currently
<StevenK> We can once we stop using yui-deps and related garbage that needs deleting.
<wgrant> Sure, we can't do it now, but when we want to...
<StevenK> wgrant: combo-rootdir will likely change again to no longer create a symlink for yui once the icing dependancy dies
<StevenK> So this isn't the final solution
<StevenK> And then it can stop being buildout generated.
<StevenK> wgrant: Ah, thanks. I wasn't sure that I'd convinced you.
<wgrant> wallyworld_: Sorry, missed the review email before. I've just replied.
<wgrant> StevenK: Indeed.
<wallyworld_> wgrant: np. i'm just about to put up the fix for the mp registration
<wgrant> Excellent, thanks.
<StevenK> YAY
<StevenK> http://www.smh.com.au/environment/weather/weather-settles-after-a-day-of-storms-hail---and-a-tornado-20120214-1t2su.html << Sydney weather is whacked
<wallyworld_> it's quite simple in the end
<wgrant> wallyworld_: The cheap fix I suggested should be about a 10 line diff + tests.
<wallyworld_> wgrant: sadly, other model code calls check_permission so i figured it was 'tolerated'
<wgrant> wallyworld_: There's only a couple of other places.
<wgrant> PersonRoles makes it pretty cheap to avoid now.
<wallyworld_> well, only for those simple checks
<wgrant> Sure.
<wallyworld_> the logic needs to be broken out and the security adaptor call that
<wgrant> In the complex cases (mostly bugs, branches) it's already delegated to the model.
<StevenK> And that is going to change anyway?
<wgrant> Hm?
<StevenK> We're going to change the complex case of bugs and branches for disclosure?
<wgrant> Yes, but the precise rules are irrelevant here.
<wgrant> It's the code structure around them that is under scrutiny.
<wgrant> Tomboy fails at crash survival :/
<wgrant> It mustn't have flushed my notes for about 6 hours yesterday :/
<lifeless> less crashing thanks
<StevenK> wgrant: I have a test here that is doing setupBrowser(auth='Basic mark@ex...:password')  I thought you removed that terribleness?
<StevenK> loltpg
<wgrant> StevenK: Hm, I suspect that's no longer doing what it thought.
<wgrant> StevenK: Especially since the domain seems to be ellipsised.
<wgrant> All the passwords are now 'test'
<wgrant> Tests still use basic auth.
<StevenK> wgrant: I'm happy to fix it with a hammer (in a Jeremy Clarkson style) while I'm here.
<StevenK> wgrant: What should it be doing instead?
<wgrant> StevenK: Which test?
<StevenK> test_adddownloadfile_nonascii_filename in TestProductFiles
<StevenK> lib/lp/registry/tests/test_product.py
<wgrant> So, you can usually just say self.getUserBrowser(user=someone)
<wgrant> It also takes a url arg.
<StevenK> Does that work under TestCase, or does it have to be BrowserTestCase?
<wgrant> TestCaseWithFactory is sufficient.
<wgrant> BrowserTestCase should probably be eliminated.
<StevenK> +1
<StevenK> wgrant: user= is an IPerson, so I have to reach into IPersonSet to grab mark out?
<wgrant> Yeah
<wgrant> Or you can continue using setupBrowser
<wgrant> But I suspect that test is bad
<wgrant> Because 'password' is no longer the right password.
<StevenK> It isn't password, it is test
<wgrant> Ahhh
<wgrant> I thought I sedded all of them out :)
<StevenK> I just thought auth='Basic ...' was sort of evil and needed to be killed
<wgrant> It is vaguely.
<wgrant> But it's still used widely.
<wgrant> I didn't want another 20000 line diff to my name.
<StevenK> Haha
<StevenK> Exception-Type: Unauthorized
<StevenK> Exception-Value: (<zope.browserpage.metaconfigure.SimpleViewClass from /home/steven/launchpad/lp-branches/moar-factory-cs/lib/lp/registry/browser/../templates/productrelease-file-add.pt object at 0x11e60950>, 'browserDefault', 'launchpad.Edit')
 * StevenK blinks
 * StevenK stabs wallyworld_.
<wallyworld_> ouc
<wallyworld_> h
<wallyworld_> ok, i give in
<StevenK> wallyworld_: Where is the fix for +register-merge? It's getting quite annoying.
<wallyworld_> it's up from review
<wallyworld_> for
<StevenK> I have created a large number of MPs today, though.
<StevenK> wallyworld_: Do you mind reviewing https://code.launchpad.net/~stevenk/launchpad/moar-factory-cs/+merge/92910 , so I can give wgrant a break? It's nice and small.
<wallyworld_> ok
<wallyworld_> StevenK: i think you need to pass in some attributes to the favtory method
<StevenK> wallyworld_: Are you sure, the test passes? :-)
<wallyworld_> eg there's one test where sales_system_id is 'foo' and then later 'new' and we need to be sure it's being reset
<wallyworld_> an empty test also passes but doesn't test what's required :-)
<StevenK> wallyworld_: Ah ha.
<StevenK> wallyworld_: The voucher is redemmed, which ignores whatever is on the CS. Then we remove it, and recreate it. No voucher, so it's new.
<StevenK> The factory method uses sales_system_id='new' all the time
<wallyworld_> yes
<wallyworld_> and the test as originally written first used an id of 'foo'
<StevenK> Yes, and it tested for 'hello'
<StevenK> So the 'foo' was clearly ignored
<wallyworld_> yes, seems so in this case
<StevenK> wallyworld_: So, fix, or "meh" ?
<wallyworld_> perhaps it's ok then
<wallyworld_> let me double check
<StevenK> wallyworld_: Pushing a change, reformating the asserts
<wallyworld_> StevenK: i think we can ignore it
<wallyworld_> yeah, there's a lot of asserts that are the wrong way around
<StevenK> You can see one of them in the current diff, line 63
<wallyworld_> and 45 and ....
<StevenK> Yeah, fixed that too
<StevenK> assertIs is better
 * wallyworld_ taps fingers waiting for diff
 * StevenK too
<wgrant> Huh
<wgrant> Dynamic bug listings just magically work with a StormRangeFactory.
<wgrant> I'm shocked.
<wallyworld_> StevenK: r=me
<StevenK> wallyworld_: Thanks!
<wallyworld_> np
<wgrant> Anyone feel like reviewing https://code.launchpad.net/~wgrant/launchpad/fix-debug-timeline-js/+merge/92916?
<wgrant> -1/+2
<wgrant> Ooh
<StevenK> wgrant: Oooh?
<wgrant> Flattening access information into bugtaskflat brings COUNT(*) on the 92000 open Ubuntu bugs down from 1100ms to 250ms when hot, because it doesn't have to join against BugSubscription.
<StevenK> wgrant: I find myself curious why the death of Mochi killed debug_timeline, and why you have href="javascript:void(0)"
<wgrant> StevenK: removeElementClass doesn't exist, so I assume it was mochi.
<wgrant> I do that so the link does nothing, because I am too lazy to change it to only create the link when it adds the handler.
<wgrant> Otherwise it will try to take you to the href :)
<StevenK> wgrant: r=me, land it
<wgrant> Thanks.
<wgrant> I've been using it on DF for a while, so I know it works :)
* wgrant changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: - | Firefighting: - | Critical bugtasks: 4*10^2
<wgrant> lifeless: How significant is the overhead of rabbiting an oops?
<wgrant> lifeless: I'd basically like to set the soft timeout of some bug searches to 0 when making large changes.
<StevenK> stub: Thank you for the +1.
<StevenK> stub: I'll set it to approved, but I need to wait for the previous branch to hit db-devel before landing this.
<stub> yup. np.
<stub> StevenK: mrevell might cry to see entitlements go, but nothing on that is scheduled as far as I know.
<StevenK> stub: They've never been used, ever. CommercialSubscription handles everything
<stub> Yer, I was wondering if it was one of those eternal 'we want to use it soon' bits. I'm all for it going.
<StevenK> If wgrant and I had figured it was this easy to rip out, we may have done it as part of the Doomed 37
<wgrant> Curtis confirmed that it can die.
<StevenK> stub: Working from https://pastebin.canonical.com/60006/ , what do you think of https://pastebin.canonical.com/60117/ as a start?
<stub> bugnotificationattachment__message__idx is a worry as we should be using that when we delete a message (I suspect we don't delete, just hide at the moment, but that should probably change)
<stub> bugsummaryjournal.viewed_by - looks like it is missing its foreign key definition?
<wgrant> StevenK: That may be deliberate.
<wgrant> Er, stub.
<StevenK> stub: I'm happy to drop anything we're not 100% sure about and play it safe.
<wgrant> It may rely on the triggers on BugSubscription to update it.
<stub> wgrant: yer, just refreshing my memory
<StevenK> I did ponder adding the unused accesspolicygrant index to that list just to see wgrant be very sad.
<wgrant> I'll be modifying that schema soonish anyway :)
<stub> So yes, it shouldn't have a FK constraint. And it can go - looks like we never select on it directly, but only when we already have a subset of rows using product/distro etc.
<StevenK> stub: Shall I drop bugnotificationattachment__message__idx from the patch?
 * stub ponders
<stub> The table has 0 rows...
<StevenK> bugnotificationattachment is empty?
 * StevenK greps
<stub> Yup
<StevenK> COMMENT ON TABLE BugNotificationAttachment IS 'Attachments to be attached to a bug notification.';
<StevenK> ... That sounds like bong
<stub> Maybe some odd 'send patches' use case that will never land.
 * StevenK blames
<stub> So remove the 'DROP INDEX' from your branch. If it goes, it goes because we drop the whole table.
<stub> I hate the hwdevicedb schema
<StevenK> Right
<StevenK> stub: Yes, I'm not amused that 13 are HW indices
<stub> Same for hwdevicenamevariant - 0 rows. Leave the indexes. If they go, go because we drop the whole table.
<stub> And hwdmihandle
<stub> And hwtestanswer
<stub> all of those actually....
<StevenK> hwtest* ?
<stub> No point removing an index on a table containing 0 rows. No gain, and as we can see helps point to potentially bogus tables
<StevenK> Leaving bugjob, bugsummaryjournal, hwdmivalue and packagesetinclusion
<stub> Yes, hwtest*
<wgrant> I excluded hw* from my purge because I hope to delete them all in one hit within a year :)
<stub> packagesetinclusion is 0 ows too
<StevenK> Why is that empty?
<StevenK> packagesets are widely used
<stub> Maybe nobody is using nested yet?
<stub> Maybe we haven't implemented the UI for that?
<StevenK> stub: Well. There is packagesetinclusion and flatpackagesetinclusion
<StevenK> Compare and constrant the schema :-)
<stub> Looks like we neglected to populate packagesetinclusion
<StevenK> I know flatpackagesetinclusion is used, since I hit it on the derivation work
<stub> Right. So that will be the exploded version, or should be.
<stub> So flatpackagesetinclusion contains 118 rows, each of which parent==child.
<StevenK> Heh, handy
<stub> So back to my earlier theory that nobody is using nested packagesets
<StevenK> Right
<StevenK> I'll dump it then
<StevenK> So my 17 is down to 3.
<stub> :-)
<stub> The revision ones look juicy if that makes you feel bad.
<StevenK> Oh, which?
<StevenK> I was ignoring any index that had n_live_tup > 0
<stub> revision, revisionauthor, revisioncache
<stub> Ignoring? The higher that number, the bigger the win
<StevenK> Oh, crap
<wgrant> StevenK: Don't dump PSI
<wgrant> It's just a feature that Ubuntu isn't using yet.
<StevenK> stub: What does the number mean, then?
<stub> The number of rows in the table
<stub> number of live tuples
<StevenK> Ahh, and the indices listed are utterly unused.
<StevenK> Right
<stub> These are all indexes that are unused, yes.
<stub> So unused indexes on hwsubmissiondevice are a win as it has 100 million rows
<StevenK> Right.
<StevenK> stub: I'm concerned sl_log_[12] is in your list. :-)
<stub> It is?
<lifeless> wgrant: its proportional to the # of queries
<StevenK> Line 145 and 146
<stub> oh, thought I'd filtered those
<stub> Forgot 'public schema only'
<lifeless> wgrant: you can test locally to get a handle on it
<StevenK> stub: Right, let ignore those 3. Going by largest number first, do you want to get the DB to do the heavy lifting, and ORDER BY n_live_tup LIMIT 15 and I'll hack my patch to drop those?
<StevenK> stub: If you think we can do more than 15 in a FDT, feel free to bump up the LIMIT
<stub> StevenK: Why the cleanup push? You trying to get loc credit before the market boom? Have you decided on a beer:loc exchange rate you are going to start selling at?
<StevenK> stub: No particular reason
<StevenK> Our DB is large, unwieldy and can be used to scare small children.
<stub> Dropping indexes should be fast. Should be able to cram a lot of them in.
<stub> I could time it, but...
<StevenK> I could time it on DF, but that would be pointless
<StevenK> Especially since wgrant keeps being evil to DF.
<StevenK> stub: I thought index deletion scaled with the size of the index?
<stub> We still need to investigate the targetted ones. eg. LibraryFileContent.sha256. The column should probably be dropped instead as it is WHUI (not sure why it was urgent before)
<stub> StevenK: rm file_on_disk;sync;
<StevenK> Because there is a little panic surrounding SHA1 and the librarian uses it quite heavily
<stub> So we shouldn't drop either the column nor the index, and add the code to the Librarian to populate it on upload.
<StevenK> I'll let lifeless rule on that.
<stub> We don't want to drop indexes we will need because that will cause timeouts in the future (because we are unlikely to notice the indexes we need are not there until production).
<StevenK> Right
<stub> message__parent__idx unused. The DB model supports threading, but we never display it. UI decision
<StevenK> stub: I'm paring down your list. Ignoring all indices whose n_live_tup is 3 digits or less and splitting out sl_log*, bug*, lp_*, hw* and specification__date_last_changed__idx, I'm down to 76.
 * StevenK grep's out the sha256 index, based on previous discussion
<StevenK> stub: I'm concerned there are 6 FTI indices implicated
 * StevenK ignores branch__access_policy__idx too, glancing at wgrant.
<stub> StevenK: Right. Probably shouldn't drop the fti indexes, and consider dropping the column (and fti trigger) instead
<StevenK> stub: Right.
<StevenK> Hah. sourcepackagepublishinghistory__ancestor__idx
<StevenK> That one is my fault.
<stub> wgrant: Have all the no-more-bug-heat-rewriting changes been rolled out?
<StevenK> stub: I'm down to 68 -- nothing obvious is jumping out, but I can pastebin what I'm down to.
<wgrant> stub: No, we have to wait 6 days for all the bugs to be updated.
<wgrant> stub: The new function was deployed last night, and the garbo job updates everything weekly.
<stub> ok. I want to rebuild the fti indexes, hopefully for the last time.
<wgrant> bug.fti will hopefully die soon :)
<wgrant> Although just to be replaced with an equivalent.
<StevenK> Pity
<wgrant> Er, not bug.fti, but its index.
<stub> Have you been investigating tsearch2, or are we finally looking at an external text search engine?
<wgrant> Nah, still plain old tsearch.
<wgrant> Just on a flattened table.
<wgrant> Performs pretty well with GIN and bitmap index combination.
 * StevenK ignores DSD too, since he isn't sure about its indices at all.
<StevenK> stub: https://pastebin.canonical.com/60118/
<wgrant> I'm surprised that branch__date_created__idx isn't used
<wgrant> Likewise the codeimport* ones.
<StevenK> I'm concerned about karmacache and oauth*
<wgrant> stub: What about master vs slave?
<stub> wgrant: We never list 'all branches in order of date created'. And I don't think it will ever be used for ordering when tuples have been selected using some other selection criteria.
<wgrant> stub: Except that we list the first 5 branches in order of date created.
<wgrant> https://code.launchpad.net/ bottom middle
<stub> wgrant: Good call. Should pull the index usage details on the slaves too.
<stub> wgrant: probably using 'id' instead, since it guarantees order in tests.
<wgrant> True.
<stub> Hmm... that is interesting. A lot more unused indexes on _prod_1 than _prod_2. We point all our scripts to _prod_2 IIRC, and prod_1 actually benefits from this lack of load balancing because it will have fewer hot indexes.
<wgrant> Hmm.
<stub> StevenK: Ignore indexes that are not also listed in https://pastebin.canonical.com/60120/ https://pastebin.canonical.com/60121/
<stub> StevenK: Or I can check it during review
<StevenK> stub: My current list is https://pastebin.canonical.com/60118/
<stub> https://pastebin.canonical.com/60119/ is the reordered master list
<stub> Will we ever delete a GPG key? I guess not, as even if a user removes it we need to keep it around since there are old published archives signed with it.
<wgrant> stub: We've done it once AFAIK.
<wgrant> That may be why those indices exist.
<czajkowski> aloha
<stub> StevenK: branch__merge_queue__idx will be needed if merge_queue is ever used. Is that exposed by the API, or is automatic merging a proposed feature in lp?
<wgrant> Morning czajkowski.
<czajkowski> wgrant: morning :)
<adeuring> good morning
<czajkowski> adeuring: hiya
<adeuring> hi czajkowski!
<mrevell> Hi!
<czajkowski> mrevell: ello
<StevenK> stub: merge queues are a half-done feature
<StevenK> stub: I'd like to see them finished, since PQM needs to die a horrible, painful and above all drawn-out death
<stub> We still can't disable or remove code imports, huh?
<wgrant> stub: Suspend
<wgrant> And you delete them by deleting the branch.
<stub> oauthaccesstoken allows you to specify limitations on an access token, but it isn't used and thus the __product, __project etc. indexes show up
<stub> I guess that isn't on the radar at the moment
<stub> Should be be dropping those columns instead?
<StevenK> stub: Are you happy enough with that patch that I should claim -10 as my very own, put an MP up for the branch and we can nut out the rest in review?
<stub> oauthaccesstoken + oauthrequesttoken
<stub> StevenK: Sure
<wgrant> stub: I don't think the approach they were designed for is adequate.
<StevenK> stub: I wonder if any of those columns even contain data
<wgrant> So they should probably go.
<stub> StevenK: They don't.
<wgrant> they've certainly never been used.
<adeuring> wgrant: thanks for the notice on bug 726320
<_mup_> Bug #726320: <type 'exceptions.TypeError'>: add_repository() got multiple values for keyword argument 'src_type' <apport-crash> <i386> <natty> <aptdaemon (Ubuntu):New> < https://launchpad.net/bugs/726320 >
<StevenK> So perhaps we should not kill the indicies and just drop the columns
<adeuring> wgrant: do you have already an idea how to dealwith sorting by rank(Bug.fti, ftq('whatever'))
<wgrant> adeuring: StormRangeFactory is a bit of a challenge due to all the new sort orders.
<wgrant> adeuring: And that, yeah, I don't know :/
<stub> StevenK: Do you want to open a bug or me?
<StevenK> stub: I'm dealing with allocated.txt and pushing up a branch, so can you?
<stub> We can drop the indexes now, but we need a bug open for the column removal so that doesn't get lost (well... )
<stub> sure
<wgrant> adeuring: Sorting by milestone, person, reporter names has the same sort of issue.
<wgrant> adeuring: We need to return more data than usual.
<stub> Fodder for salgado to reduce his loc impact :-)
<stub> We should have a tag for that actually...
<adeuring> wgrant: ah, right, each sort vaule must appear in the result set
<StevenK> stub: 'here-salgado-salgado' ?
<adeuring> wgrant: but adding that shoulkd not be difficult
<wgrant> adeuring: Yeah, it's just a bit messy due to three types of queries that are produced.
<stub> StevenK: Anyone working on features would find it useful ;)
<wgrant> adeuring: some of them UNIONs, others wrapped in extra layers to eliminate dupes.
<wgrant> I'm reworking that stuff atm, which is why I advise not to touch it.
<wgrant> Hopefully get down to just one or two types of queries.
<wgrant> And then work out how to get SRF working on top of them.
<adeuring> wgrant: right -- my basic idea was anyway to use stormrangefactory ;)
<wgrant> adeuring: My changes make this a bit less important, because scans are about 5x faster in the new schema.
<wgrant> But I expect to do it afterwards anyway.
<wgrant> stub: We have a concrete and immediate plan for divorcing the SSO DB, FYI.
<stub> wgrant: Cool. What is the plan?
<wgrant> I expect to have it testable by next week.
<wgrant> stub: SSO becomes an xmlrpc-private client.
<wgrant> New method on LP takes an OpenID identifier and returns the data SSO needs.
<stub> wgrant: And what happens when Launchpad is unavailable. Nobody cares?
<wgrant> stub: SSO just won't return the extra data (Launchpad ID, timezone and teams).
<wgrant> That appears to be acceptable to ISD.
<stub> Cool. The HA aspects where what made it a pain and the current system.
<stub> Typical for HA, when you meet all the goals you end up with a system you don't want to maintain :-)
<wgrant> stub: Heh.
<wgrant> Well, LP is far HAer than it used to be :)
<wgrant> We no longer have ludicrous periods of downtime.
<stub> If you twist your definition of HA enough, yeah.
<wgrant> HAer, not HA :)
<stub> From a traditional POV, we gave the finger to HA and instead opted for controlled scheduled downtime.
<wgrant> Yeah
<wgrant> Of reasonable duration, importantly.
<wgrant> 90 minutes of auth unavailability is clearly unacceptable.
<wgrant> A minute, which is what we'll probably be at once this is done, seems OK.
<stub> We currently offer 99% uptime I think (99.65% with fastdowntime, plus unscheduled failures). That figure is passed on to any other systems relying on LP in addition to their own downtime.
<wgrant> stub: How do you figure 99.65%?
<wgrant> Outages are currently either 90 or 150 seconds.
<stub> 5 minutes per day - what we promise rather than what we actually use.
<wgrant> At 90s that's 99.90% for the day
<wgrant> Ah
<wgrant> Right.
<stub> Need some real metrics for the real answer, including the unscheduled outages :)
<wgrant> Most unscheduled outages hit SSO too, fortunately :)
<stub> wgrant: How do we know when the bug heat update has completed? Some garbo output to watch for?
<wgrant> stub: SELECT COUNT(*) FROM bug WHERE heat_last_updated < '2011-02-13';
<wgrant> s/2011/2012/
<stub> Cool. I guess we can drop the heat_last_updated soon. That covered by a bug or kanban task or something?
<wgrant> Might wait for things to settle before doing that.
<stub> I'll file another tech-debt bug
<stub> Bug #931987   if you want to claim it
<czajkowski> ello are uploads not being picked up ?
<czajkowski> wgrant:  sorry not sure where to ask that question in here or -ops
<wgrant> czajkowski: ops, probably.
<wgrant> It's possibly related the pgbouncer issue.
<czajkowski> k thank
<tumbleweed> wgrant: ta
<czajkowski> tumbleweed: yo're here
<tumbleweed> I haven't gc-ed channels in a while...
<rick_h> morning
<czajkowski> rick_h: aloha
<StevenK> rick_h: I note your lack of reply. However, I fixed your combo-rootdir issue in devel.
<rick_h> StevenK: lack of reply?
<rick_h> StevenK: I saw that MP, will pull that and try it out thanks!
<StevenK> rick_h: You said you'd reply to my combo-url mail on the list.
<StevenK> rick_h: Pull it out? Just merge devel? :-)
<rick_h> StevenK: ah, yea well my tests came back failing last night
<rick_h> part of my reply was going to be an update on the new test templates and such
<StevenK> Ah, kay
<rick_h> I got a bunch of js timeouts so both failed and I'm working on figuring out wtf this morning
<rick_h> no landing from me :/
<StevenK> rick_h: The AJAX log branch is seperate, I'm guessing?
<rick_h> yea, but that new test came back with js timeout
<StevenK> Strange
<rick_h> so something bigger perhaps? or maybe I just caught a bad ec2 day
<StevenK> I threw two branches at ec2 today, both passed nicely.
<rick_h> *sigh* well, I'm peeking and will try to relaunch in a bit
<rick_h> but yea, so I didn't reply to the email yet
<rick_h> I hate it, tests all pass locally peachy, only way to "check" is to toss things at hours of ec2 and cross your fingers
<StevenK> Welcome to Launchpad. :-/
<deryck> adeuring, rick_h -- https://plus.google.com/hangouts/extras/talk.google.com/orange-standup
<sinzui> jcsackett, Do you have time to review https://code.launchpad.net/~sinzui/launchpad/listify-cached-licences/+merge/92893
<jcsackett> sinzui: good timing, i have another branch i need to review this morning anyway, so certainly. :-)
<jcsackett> looking at yours now.
<sinzui> jcsackett, fab, I am looking at your first branch now
<sinzui> jcsackett, replied with a patch
<jcsackett> sinzui: i have replied to yours as well, with some notes/nitpicks. r=me.
<jcsackett> sinzui: i will patch mine, thanks for the code. wasn't sure how to test just registering, nor did i think it was strictly necessary. happy to add it though. :-)
<rick_h> hmm, so if I fire up an ec2 test run and ssh to the box I should be able to find the files and manually fire off test commands while it runs right?
<rick_h> looks like no, I can't boooo
<rick_h> deryck: ok, big duh...the 8 tests that fail are the 8 updated ones to look in the build dir
<deryck> rick_h, aren't they all being updated to look in the build dir?  or just in this branch it's only these 8 that are changed?
<rick_h> deryck: only those 8 are changed. I'm trying ot see if hte build dir is done via the test runner
<rick_h> make is run
<rick_h> which should make the build dir
<deryck> rick_h, yeah, that's what I was saying this morning; it smells like the files aren't there.
<rick_h> deryck: yea, the make command is in run_demo_server so wonder if the YUI js tests are run in a different place and the build dir is gone/not there yet
<rick_h> deryck: so yea, guessing the issue is how the ec2 test script is run. Working on getting my head around it
<rick_h> but that makes sense now
<lifeless> inline adverts in the twitter stream. Disappointing twitter, disappointing.
<czajkowski> abentley: are you about for a quick question, not urgent , can ping tomorrow it's re https://answers.launchpad.net/launchpad/+question/184925
<benji> when doing a make in a fresh devel checkout I get "ImportError: No module named convoy.meta"; I followed the instructions in Steve's "Convoy is ready for others
<benji> " email, so I'm wondering what step I left out
<rick_h> benji: convoy should be installed via an apt-get upgrade
<rick_h> it's part of the ppa
<benji> rick_h: I would have thought the same thing
<lifeless> rick_h: what do we import convoy for ?
<lifeless> benji: what flaour of Ubuntu are you running?
<benji> oh; I wonder if it's not packaged me
<rick_h> lifeless: convoy.meta helps walk the list of YUI modules we defined
<rick_h> so thatit knows how to import it
<benji> lifeless: an older one ;)
<benji> let me look exactly
<lifeless> benji: tsk!
<benji> lifeless: I have a newer VM, let me fire it up and see if it works there
<rick_h> benji: ah, ok that makes sense then
<lifeless> benji: you know we're meant to upgrade @ beta release each ubuntu release ? :)
<benji> lifeless: I mostly do, I forgot which VM I was running
<rick_h> lifeless: do things run well on precise now?
<rick_h> I've been waiting for all that pgsql stuff to settle before upgrading the laptop, but looking forward to
<lifeless> rick_h: so what I encourage folk to do, and I do myself, is run my host in $ubuntu-current (usually upgrading around alpha 1), and do dev in an LXC lucid container
<rick_h> I need to play with lxc I suppose. I've not messed with it at all yet
<benji> rick_h: it's pretty cool, if a bit young still
<benji> rick_h: yay, my newer VM works fine
<abentley> czajkowski: I wish that were a quick question, but I've looked at it several times and not come up with an answer.
<abentley> czajkowski: It looks like it's failing to gpg-sign a revision.
<abentley> czajkowski: I don't know why it's failing, but I don't think it should be trying to sign in the first place.
<abentley> lifeless: this is a CVS import.  We don't gpg-sign those, do we? https://code.launchpad.net/~rpm5/rpm/trunk
<lifeless> I don't think we sign any imports
<lifeless> IMBW
<lifeless> but if we sign any, I expect we sign all
<rick_h> benji: awesome
<czajkowski> abentley: ok just confused as I've never seen that kind of thing before again apologise for the question for the coming weeks
<lifeless> abentley: actually IIRC we did setup signing of imports way back with baz, it was a religion then
<lifeless> abentley: so I expect our cscvs infrastructure does indeed sign things
<abentley> lifeless: Maybe I'll pull the successful imports and see if they're signed.
<abentley> lifeless: 10254 commits not signed
<lifeless> abentley: from a cvs import ?
<lifeless> svn ones are all bzr-svn now, which won't sign as we're not invoking the commit codepath; ditto hg and git.
<abentley> lifeless: yes, https://code.launchpad.net/~rpm5/rpm/trunk.  62 commits were signed (but with keys my box doesn't know)
<lifeless> abentley: verra interesting
<lifeless> abentley: I wonder if the different import slaves have different configs for bzr or something
<abentley> lifeless: Yes, I wonder too.
<abentley> lifeless: Also wonder how many of our CVS imports are failing.
<lifeless> whats the easiest way to get a js repl ?
<benji> lifeless: FF Ctrl-shift-K
<lifeless> benji: ok, and thats bare in a new tab....
<lifeless> benji: so I should be less vague; I want to play with handlebars.js
<lifeless> ah, http://tryhandlebarsjs.com/ looks the ticket
<benji> lifeless: hmm, maybe find a page that already loads it?  Or do some document.createElement("script");
<benji> or that :)
<lifeless> abentley: had you seen http://engineering.linkedin.com/frontend/client-side-templating-throwdown-mustache-handlebars-dustjs-and-more in evaluating moustache ? [just curious]
<abentley> lifeless: no.  Thanks for pointing it out.
<abentley> lifeless: "...none of the templating options worked well across client and server unless the server could also execute JavaScript..."
<lifeless> its an interesting statement, but missing details needed to evaluate it
<lifeless> ... which is a bit disappointing
<abentley> lifeless: Agreed.
<abentley> lifeless: I don't agree that Mustache has "clean" syntax.  Am I silly for wanting loops to be distinct from conditionals?
<lifeless> mmm, I could go either way
<lifeless> its very pithy as it stands
<rick_h> lifeless: just create an index.html with handlebars in it. Then just use things like the normal dev tools if you want, but I find just using livereload and that test.html file enough to tinker away
<rick_h> lifeless:  https://github.com/mockko/livereload/blob/master/README-old.md
<rick_h> lifeless: another cool js toy area http://jsfiddle.net/
<rick_h> lifeless: that's cool because you can share a fiddle via link to others, interactive pastebin
<wgrant> jelmer: Hi, you have a couple of bits of QA.
<rick_h> StevenK: ping, you see my email to the list today?
<StevenK> Wha? E-mail?
<StevenK> rick_h: I've been on mumle since I started, e-mail is a secondary concern
<StevenK> Bah. mumble
<StevenK> rick_h: make build can't run combobuild
<StevenK> If you just run make (Which is the 'inplace' target), that will run combobuild
<rick_h> StevenK: ok, so what's the "right" way for me to get combobuild run in tests?
<rick_h> StevenK: my issue is I can't ec2 test/land anything because make check only runs make build and combobuild never gets run
<rick_h> StevenK: if I break out combobuild to be two parts, one that just copies files into build/js/lp and one that uses convoy to generate meta.js can build run combobuild? (the first part)
<StevenK> rick_h: Yes. I can do that for you today.
<rick_h> StevenK: ok, if you've got time. I thought I couldn't just run combobuild form make build, but couldn't recall why
<rick_h> but yea, all the JS tests now point to the build/js/... for their test files so I need that to be populated for make check
<StevenK> rick_h: Because convoy won't be installed everywhere, and machines like banana and nutmeg use make build
<StevenK> And gandwana for that matter. I should fix that.
<rick_h> StevenK: ok, I wasn't sure 100%
<rick_h> the other thing would be to break out the meta generation stuff of convoy out but think that'd be making for more problems
<StevenK> Right
<StevenK> I don't think there is any gain there
<rick_h> my only concern with the splitting of meta.js generation is that when we run that, we only want to generate the meta.js for the non-minified files
<rick_h> so I think what we do now is, copy files, generate meta.js, minify files
<rick_h> so it may be a pita to have to break that into 3 steps ugh
<StevenK> You think? :-)
<StevenK> I'm happy with 2 steps
<rick_h> well I know convoy only looks at the root directory and generates metadata for all files in there
<rick_h> if we say copy/min files in one step, and generate meta.js with convoy in another
<rick_h> it'll have dupe metadata info
<StevenK> Right
<StevenK> Perhaps convoy needs a patch to ignore minified files
<rick_h> yea, maybe an --exclude regex or something
<rick_h> nvm, already has an exclude regex for the cmd line version
<StevenK> So we can --exclude '-min.js' ?
<rick_h> well except I don't think we call it from the cmd line do we? We import extra_metadata?
<StevenK> utilities/js-deps -n LP_MODULES -s $BUILD_DIR/lp -o $BUILD_DIR/lp/meta.js >/dev/null
<StevenK> That is called from combo-rootdir
<rick_h> right, js-deps so that's got to be updated to pass the args to main() or whatever
<StevenK> What?
<StevenK> It already handles this
<StevenK> Run utilities/js-deps --help
<rick_h> yea, sorry, main() catches it
<StevenK> You're trying to solve problems that don't need solving :-)
<rick_h> sorry, thoguht it was handled outside of the main we imported
 * rick_h is taking a second to trace code :)
<rick_h> ok, well anyway. Sounds like a plan to me. We can keep it two steps if we can make sure it skips the -min.js files
<wgrant> Bah
<wgrant> Only got Product:+index 99% down by 25%.
<wgrant> Although mean down by 45%, which I guess is better.
<lifeless> so, mean means all users notice it
<lifeless> 99% means outliers notice it
<lifeless> if both shift by the same amount, and 99% isn't capped by timeouts, then you removed a constant overhead
<wgrant> I know.
<lifeless> :P
<wgrant> Ah, there we go, the remaining major win is trivial.
 * wgrant fixes.
<wgrant> lifeless: Do you know what the current stats targets are on prod?
<wgrant> That query with massive planning overhead that I found last week is still only 50ms on DF cold, and 10ms hot.
<wgrant> Including planner.
<lifeless> I can find out if you need
<jelmer> wgrant: yes, sorry - it's first on my list of things to do tomorrow
<lifeless> should we roll it back in the interim ?
<jelmer> lifeless: roll back my change you mean? it's only just landed
<wgrant> It landed a couple of days ago
<wgrant> That's two or three LP deployments :)
<lifeless> jelmer: it's 10 commits back, landed on the 13, its now the 15th, adjusting for tz's its been in trunk for 36 hours
<lifeless> jelmer: anything in trunk is a pipeline stall for the whole team, QA on it *has* to happen promptly.
<lifeless> ideally the person landing, but failing that anyone with knowledge - and yes, that makes things with specialist knowledge benefit from explicit handoffs for QA.
<lifeless> jelmer: if you can tell us what you were going to do, we may be able to do it
<jelmer> lifeless: it only landed on qastaging 21 hours ago according the qa tagger
<lifeless> even so, thats plenty of time to have either qa'd or handed off the qa
<wgrant> lifeless: What's the process for live index creation these days?
<wgrant> Apply before or after landing?
<wgrant> It should take about 60ms to create :)
<lifeless> wgrant: land on qastaging, add and QA there, then prod.
<wgrant> land on devel, add and QA on qastaging, then prod?
<lifeless> yes
<wgrant> It turns out that ProductReleaseFile.productrelease is unindexed.
<wgrant> So I guess the table is just pathetically small.
<lifeless> 74K rows
<wgrant> Oh, right, the PRF.
<wgrant> p-r-f, sorry
#launchpad-dev 2012-02-15
<wgrant> hah
<wgrant> 30ms query in all product traversals
<wgrant> with pillarname.alias_for IS NULL, when the alias_for index is partial on NOT NULL.
* StevenK changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: StevenK | Firefighting: - | Critical bugtasks: 4*10^2
 * wgrant dies quietly.
<wgrant> lifeless: https://code.launchpad.net/~wgrant/launchpad/bug-932438/+merge/93109 would enjoy your perusal.
<wgrant> StevenK: Could you please review https://code.launchpad.net/~wgrant/launchpad/bug-932451/+merge/93111 and https://code.launchpad.net/~wgrant/launchpad/bug-932433/+merge/93110?
 * wgrant headdesks repeatedly
<wgrant> goddammit
<wgrant> did nobody actually think when they were writing any of the queries on Product:=index
<wgrant> Half of them are unindexed.
<wgrant> So they only work at all because the tables only have 100000 rows.
<wgrant> Oh look, now the query is 0.1ms
<StevenK> wgrant: r=me * 2
<wgrant> Thanks.
<wgrant> And another two missing indices.
<wgrant> That should be about 400ms off /unity now, I think.
<lifeless> wgrant: you forgot to request from stub as well
<wgrant> Blah, true.
<wgrant> If you hold off for a minute I'll have another two indices in there.
 * StevenK peers at his ec2 run
<StevenK> It has dropped the entitlement table, but a method under IPerson is still trying to query it.
<wgrant> StevenK: Person merge will query anything that has foreign keys.
<wgrant> You'll need to drop the foreign keys before you move to todrop.
<wgrant> Only foreign keys to branch and person matter.
<StevenK> Oh, my DB patch is wrong
<wgrant> Well, it's right.
<wgrant> It just won't work.
<wgrant> We should possibly fix upgrade.py to drop todrop relations even when unreplicated, but I was never brave enough to do it.
<StevenK> wgrant: I need to ALTER TABLE person DROP CONSTRAINT entitlement_{approved_by,person,registrant}_fkey; ?
<wgrant> StevenK: ALTER TABLE entitlement, but roughly yes.
<StevenK> wgrant: But won't they get deleted if I just drop the entitlement table?
<wgrant> They will, but you can't drop the entitlement table.
<wgrant> You can only move it to todrop and then let upgrade.py have its way with it.
<wgrant> It will eventually drop in a replicated environment, but not AFAIK in an unreplicated one.
<wgrant> And we are possibly going to move the replicated drop to after service is restored.
<wgrant> Which means you'd have to drop the FKs early anyway.
 * wgrant lunches.
<StevenK> wgrant: http://pastebin.ubuntu.com/842474/
<wgrant> StevenK: Looks reasonable. Apply it locally and check that entitlement has no more FKs onto person or branch.
<StevenK> I didn't see any on branch, but I'll check
<StevenK> And then commit/push/re-toss at ec2
<wgrant> There probably aren't any.
<wgrant> But those are the two tables that matter.
<StevenK> I was tempted to pg_dump launchpad_dev | grep entitlement | grep CONSTRAINT or some such
<wgrant> \d entitlement should do fine
<wallyworld> huwshimi: got a minute for a css question?
<StevenK> wgrant: distribution, product and project too :-/
<wgrant> StevenK: They don't matter.
<StevenK> Oh, they will get binned when the table does?
<StevenK> But the person FKs are thpecial?
<wgrant> They will all get binned when the table does.
<wgrant> Person and branch FKs are special because our code queries to find all of them.
<wgrant> On branch deletion and person merging.
<StevenK> Right
<wgrant> To check for references that need breaking or updating.
<wgrant> Oh, LFA also.
<wgrant> But there probably aren't any of those.
<lifeless> wgrant: this needs a faq I think
<lifeless> wgrant: on the dbpatches pages
<lifeless> wgrant: its a vicious special case that folk don't touch often enough to have paged in
<wgrant> lifeless: Yeah
<wgrant> Probably.
<wgrant> But tests catch it, so it's not so bad.
<lifeless> wgrant: having a canned answer would avoid confusion.
<lifeless> wgrant: would you like to write one?
<lifeless> (I would like /someone/ to write one)
<wgrant> I two massive DB patches to finish today; maybe after that :)
<wgrant> lifeless: Is there a good reason we don't use HSTS?
<wgrant> I believe there are a couple of types of URLs (PRF downloads) that we force to HTTP for probably no good reason.
<wgrant> But HSTS is probably useful for performance as well as security.
<lifeless> get chrome to put it in their hardcoded list ?
<lifeless> wgrant: its never been discussed TTBOMK
<lifeless> back soon
<wgrant> lifeless: HSTS is a header
<wgrant> http://en.wikipedia.org/wiki/HTTP_Strict_Transport_Security
<lifeless> wgrant: I know
<lifeless> wgrant: (what in my answer suggested I didn't ?)
<wgrant> lifeless: You mentioned Chrome's hardcoded list.
<lifeless> yes
<lifeless> wgrant: http://www.chromium.org/sts
<wgrant> lifeless: Perhaps eventually.
<wgrant> But we should do the header first :)
<mwhudson> that's a pretty random list of sites
<StevenK> And IE doesn't support it. This is my surprised face.
<wgrant> +download/blah is the only thing we don't currently server over HTTPS to browsers
<wgrant> And we'll have to start HTTPSing for some things soon.
<wgrant> (due to private projects)
<lifeless> +download/blah should be https
<wgrant> So hopefully https://rt.admin.canonical.com//Ticket/Display.html?id=30043 isn't an issue
<wgrant> any more
<wgrant> lifeless: You would damn well think so.
<wgrant> But Plone seems to be a bit screwed.
<wgrant> Or was in 2008.
<lifeless> so
<lifeless> this exemption should be backed up by logic in LP
<lifeless> +1 on nuking it, with a short list discussion to make sure noone is surprised
<wgrant> Great.
<wgrant> Then I shall deploy HSTS and X-Frame-Options and HttpOnly
<lifeless> do we generate http urls for +download ?
<wgrant> (the last of which requires a minor zope.session upgrade)
<wgrant> I believe so.
<lifeless> as long as you stop generating them the apache rule won't be in the way
<lifeless> (though that isn't a reason to preserve it)
<wgrant> Yep
<wgrant>         return str(URI(url).replace(scheme='http'))
<wgrant> There's the guilty one.
<StevenK> Kill it
<wgrant> poolie: What are your not-4-year-old thoughts on https://bugs.launchpad.net/launchpad/+bug/174186?
<_mup_> Bug #174186: https redirects pose download problem <infrastructure> <lp-bugs> <lp-foundations> <Launchpad itself:Invalid> < https://launchpad.net/bugs/174186 >
<wgrant> StevenK: could you review https://code.launchpad.net/~wgrant/launchpad/remove-extra-privacy/+merge/93122? As discussed on the call this morning.
<StevenK> wgrant: The CTE flag never made it to flags.py? :-(
<wgrant> Nah, it was only meant to last a couple of days.
<wgrant> But I forgot to delete it.
<StevenK> first_product = self.factory.makeProduct(name='bunny')
<StevenK> I thought I saw something odd in the diff
<StevenK> Had it read it closer
<StevenK> wgrant: r=me
<wgrant> Thanks.
<poolie> hi wgrant
<poolie> wgrant, i think it's funny when things flip from fixreleased to invalid :)
<poolie> it's not a practical issue for bzr atm afaik
<poolie> i would leave it ~wontfix
<wgrant> poolie: It's really Fix Released, but the Great LP Project Merge let the Invalid malone task take over.
<wgrant> So it's currently fixed.
<poolie> how is it fixed?
<wgrant> But I am going to unfix it unless someone brings up a very good reason to not.
<wgrant> Check the download links on https://launchpad.net/bzr/+download
<wgrant> They're the only internal HTTP links in Launchpad
<poolie> wow
<poolie> that's crazy
<StevenK> All because of Plone
<StevenK> I knew Plone was evil, this just confirms it
<poolie> please do fix it, i think it's a bug, arguably a serious one, that those links are insecure
<wgrant> One would think so.
<wgrant> Thanks.
<poolie> i do think it's nice if there is at least the option to get the final file over http
<poolie> ie if the librarian speaks http if you ask for it on the final url
<wgrant> You can s/https/http/ in the librarian URL and it will still work.
<wgrant> If you really must get a compromised copy of the file.
<poolie> yep
<poolie> i think that's a decent accommodation to the specific case of broken clients or ridiculous firewalls
<wgrant> I might just make the change and then publicly shame anybody who complains that their stuff is broken.
<lifeless> public shaming isn't a terribly friendly strategy
<wgrant> Neither is downloading code over HTTP :)
<poolie> the issue like this that does currently annoy me is that curl, by default, doesn't follow redirects
<poolie> but, lp probably can't do anything about that
<poolie> i guess if we wanted to help people like this
<lifeless> we could generate librarian urls directly in the page
<poolie> and i'm not saying it's a priority
<lifeless> except for private context
<lifeless> *content*
<poolie> right, put the final url
<poolie> and then people can more obviously make the protocol change themselves
<lifeless> OTOH its a special case for pretty low return
<lifeless> curl should really follow redirects over HTTPS
<poolie> i don't know why it has that default
<poolie> also it just gives you an 0 byte file rather than erroring
<poolie> oh well
<lifeless> poolie: (curious) have you filed a bug ?
<wgrant> There's an option to make it follow the redirect.
<StevenK> wallyworld: Did you notice you got r14800 ?
<wallyworld> no
<wallyworld> what's the significance?
<StevenK> wallyworld: Your MP fix is r14800
<poolie> lifeless, nup i assumed it was intentional
<StevenK> It's a 00 number!
<StevenK> They're special or something
<wallyworld> oh, ok :-)
<wallyworld> i feel much better knowing that
<poolie> StevenK: +2 damage
 * StevenK grumbles at the lack of QA
<wgrant> StevenK: https://code.launchpad.net/~wgrant/launchpad/bug-932518/+merge/93127 is as discussed above, +4/-118. But you've done a few of mine today, so feel free to ignore if you've had enough :)
<StevenK> And here I was figuring it was payback for yesterday. :-P
<wgrant> I only took like two of yours yesterday.
<wgrant> I was a bit surprised when you decided to throw one at wallyworld.
<StevenK> wgrant: I figured I had annoyed you enough. But, r=me
<wgrant> Thankyou sir.
<wgrant> StevenK: I might point out that all the bugs I've squashed today have been bugs that I also filed today :)
<StevenK> They're still squashed. :-P
<StevenK> wgrant: https://code.launchpad.net/~stevenk/launchpad/two-part-combobuild/+merge/93125
<wgrant> Curses.
<wgrant> StevenK: It's not a problem any more that meta.js won't be in the minified output?
<StevenK> wgrant: No, we were using meta.js only, and the difference between them is only 4K
<wgrant> StevenK: Also, why do we want to do this?
<wgrant> Doesn't this mean prod will now minify twice?
<StevenK> wgrant: We want to do this for Rick's work of the tests using JS under build/js
<StevenK> And yes, for the short term
<wgrant> Ah, right.
<wgrant> r=me
<StevenK> wgrant: Thanks!
 * StevenK wonders how to make visibility the third field
<wallyworld> huwshimi: ping
<huwshimi> wallyworld: Yo
<wallyworld> huwshimi: i have a ccs question for you https://pastebin.canonical.com/60211/
<wgrant> lifeless: Am I allowed to make code changes in db-devel as long as they're not necessary on production?
<huwshimi> wallyworld: Is there a reason that description is in the <a> then?
<wgrant> lifeless: I want to have tests for these triggers.
<wgrant> lifeless: Which probably means introducing new Storm classes etc.
<wallyworld> huwshimi: so a user can click anywhere in the text to fire the link
<wallyworld> huwshimi: i don't understand why the text-decoration: none is being ignored
<huwshimi> wallyworld: It's probably to do with specificity and how the underline is applied to the a tag. You might be able to try ".yui3-ichoicelist a .choice-description:hover" (notice the "a" there).
<StevenK> I thought FormFields was a list. :-(
<wallyworld> huwshimi: thanks, will give it a go now
<StevenK> Since I don't want to add it near the end
<StevenK> Well, at the end
<lifeless> wgrant: no
<wallyworld> huwshimi: no go sadly.
<huwshimi> wallyworld: Do you use chrome or firefox?
<lifeless> wgrant: thats the short answer; the long answer is that its really hard to be -completely- confident nothing will leak
<wallyworld> huwshimi: ff
<huwshimi> wallyworld: Hmm.. I wanted to check something, but I'm not sure how to do it in Firefox
<lifeless> wgrant: I suggest you do the tests in a branch, review it as a branch + prereq, and land separately
<wallyworld> huwshimi: also bad in chrome
<wgrant> lifeless: k, thanks
<wallyworld> just checked
<huwshimi> wallyworld: Oh, yeah I just wanted to check something in Chrome's inspector
<huwshimi> wallyworld: As in, if you inspect the span if the css rules are being crossed out
<wallyworld> huwshimi: since it's the same in both, you can check in chrome
<wallyworld> i'll have a look also
<huwshimi> wallyworld: is this code only in your branch?
<wallyworld> huwshimi: yeah
<huwshimi> wallyworld: oh wait, so the color: green IS being applied to the span?
<wallyworld> huwshimi: yes
<huwshimi> wallyworld: Oh!
<wallyworld> hence my confusion
<huwshimi> wallyworld: Wait, so when you hover medium everything gets underlined but the span doesn't go green right?
<wallyworld> huwshimi: the span goes green as expected BUT everything is underlined
<wallyworld> and i don't want the text in the span underlined
<wallyworld> so the color is being applied but not the decoration style
<huwshimi> wallyworld: strange, I'd expect the green to only apply when you're hovering the span text
<wallyworld> yes, sorry
<wallyworld> it only applies when you hover
<wgrant> The browser/reset a:hover won't have a colour
<wgrant> So it won't override it
<wgrant> It *does* have a text-decoration
<huwshimi> wallyworld: So what you need is this: ".yui3-ichoicelist a:hover .choice-description:hover {"
<wallyworld> let me try that
<huwshimi> wait
<huwshimi> wallyworld: .yui3-ichoicelist a:hover .choice-description {
<huwshimi> I left a :hover at the end
 * StevenK feels utterly dirty.
<StevenK> +            field = self.form_fields.__FormFields_seq__.pop()
<StevenK> +            self.form_fields.__FormFields_seq__.insert(2, field)
<wallyworld> huwshimi: sadly no. i'll keep trying
<huwshimi> wallyworld: Can you push your branch somewhere and I'll have a quick try?
<wallyworld> huwshimi: lp:~wallyworld/launchpad/choice-popup-descriptions-932424
<huwshimi> wallyworld: Cheers
<wallyworld> huwshimi: you need to turn on ff disclosure.enhanced_choice_popup.enabled
<huwshimi> wallyworld: OK, np
<wallyworld> and run bin/combine-css
<wallyworld> and make jsbuild
<wallyworld> thanks in advance :-)
<wallyworld> huwshimi: then, just goto any bug and click the bug status popup
<wallyworld> or importance
<huwshimi> wallyworld: So the problem is that the child element (the span) is doing what it's told. With the "a: hover span" it's not underlining the text when we hover the span just as we want it to. The problem is the span was never underlining the text, that's coming from the parent element, the <a> and child elements can not override a parent's.
<huwshimi> wallyworld: So what you'll have to do is move the "Medium" into another span. Set a:hover { text-decoration: none; } and then a:hover .medium-span { text-decoration: underline; }
<huwshimi> wallyworld: Does that all make sense?
<wallyworld> huwshimi: i'm digesting it
<wallyworld> huwshimi: so a child element can't override a parent? seems quite limiting
<huwshimi> wallyworld: Welcome to CSS
<huwshimi> :)
<wallyworld> it just seems broken
<wallyworld> must be a reason for it
<wallyworld> huwshimi: thanks for looking, really appreciate it
<wallyworld> i'll try it out
<huwshimi> wallyworld: No problems. Let me know how you go
<wallyworld> huwshimi: will do. may have to go and get the kid from school before i get it going
<StevenK> wallyworld: Got one sec?
<wallyworld> StevenK: yes
<wallyworld> gone
<wallyworld> want another?
<StevenK> wallyworld: Y.one('#field.subscriptionpolicy');
<StevenK> wallyworld: Is that by class or id?
<wallyworld> won't work
<wgrant> # is ID
<wallyworld> by id
<wgrant> But also by class
<wgrant> id field, class subscriptionpolicy
<wallyworld> but Y.one doesn't like ids with '.'
<StevenK> The linked bug was marked as fixed in 3.0.0
<wallyworld> StevenK: Y.one("[name='field.subscriptionpolicy']")
<wallyworld> will work
<wallyworld> assuming the element comes from a lp form
<wallyworld> where zope sets the name
<wallyworld> otherwise Y.DOM.byID() will work
<wallyworld> or something like that
<StevenK> lp.buildmaster.tests.test_builder.TestSlave.* just failed for me on ec2 :-/
 * StevenK stabs them
<wgrant> The librarian probably stole its port.
<StevenK> lp-land and move on?
<wgrant> Yup
<adeuring> ood morning
<wgrant> stub: That milestone thing seems pretty odd.
<wgrant> Given that the primary parent of a milestone is a series.
<wgrant> stub: It's also a small extremely read-heavy table, queried in some places where we don't actually have the series object, just the ID.
<wgrant> So it will be some pretty awkward refactoring to avoid a harmless index, and use an index on a deprecated column instead.
<stub> Ok. If we need the index to avoid a join with distroseries or productseries
<wgrant> I've found one place where it is cleaner to query on product.
<wgrant> And productseries isn't necessary at all.
<wgrant> But many others really want to deal in series :/
<stub> Yup. So no real harm adding them.
<wgrant> Did you see my comment about productreleasefile?
<stub> Did you do timings on the productreleasefile index btw?
 * stub refreshes
<wgrant> On its creation, or the queries?
<wgrant> On DF the example query I used goes from 60ms to 0.5ms
<wgrant> I can pull it up from history if you want.
<stub> oh, brain fade. I'm too used to the id columns being the first column
<wgrant> Heh, yeah, that is a bit odd.
<stub> Did productrelease used to be the primary key? Can't think why it would be out of order like that unless it was added later.
<wgrant> I assume it was (productrelease, libraryfile)
<wgrant> But surely not, since SQLObject doesn';t support that.
<wgrant> So maybe it was just for storing a single tarball per release? :/
<stub> It might have if you tried hard enough
<stub> Not sure when the table arrived. Doesn't matter anyway.
<stub> r=stub
<wgrant> Thanks.
<wgrant> It would be nice if we could globally profile planning overhead :/
<wgrant> stub: Can you apply those live, please?
<wgrant> Or should I land first?
<wgrant> I've tested them live on DF.
<stub> I'll apply them now
<wgrant> Thanks.
<stub> wgrant: Done
<wgrant> stub: Thanks!
 * wgrant tries and lands.
<rick_h> morning
<rick_h> thanks for the combo build updates StevenK, will try landing again this morning woot!
<StevenK> Hehe
<deryck> adeuring, abentley, rick_h -- https://plus.google.com/hangouts/extras/talk.google.com/orange-standup
<deryck> abentley, adeuring -- it doesn't kill the hangout when I leave, does it?
<adeuring> deryck: works fine
<deryck> ok, cool
<timrc> Hello.  The Commercial Engineering team would like to increase the default size of a PPA owned by ~oem-archive upon creation.  I do not believe this is something we can currently do via the API.  If that's true, is there a way to increase the default on a per-team basis with a little magic on your end?
<bigjools> it would be an easy patch for you guys to do
<bigjools> (to set it via the API)
<timrc> bigjools, okay, that is what I needed to know
<timrc> one other thing...
<timrc> bigjools, the understanding that I have which is based on the understanding of my boss is that we can jack the default up to 1TB with the understanding we'll never actually consume that much space? Does this match your own understanding?
<bigjools> timrc: echan for this
* abentley changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: abentley | Firefighting: - | Critical bugtasks: 4*10^2
<deryck> rick_h, give me 2 minutes and I'll join you on hangout.
<rick_h> deryck: k
<deryck> rick_h, https://plus.google.com/hangouts/extras/talk.google.com/go-for-deryck
<jelmer> is anybody else getting xmlrpc errors related to the uniqueness of owner_name, when pushing to qastging?
<sinzui> jelmer, I am seeing null value in column "owner_name" violates not-null constraint when pushing a new branch
<jelmer> sinzui: thanks, so it's not just me
<jcsackett> sinzui: are we going for a JS/mustachy approach on the +sharing, or ... ?
<sinzui> jcsackett, Yes.
<jcsackett> sinzui: cool.
<jcsackett> i had *assumed* so, but was suddenly wondering. :-P
<sinzui> jcsackett, The search form and filters stumped the punters. We will do some paper mockups to revise what and how we display them. Assume the table is good to implement for stakeholders to try in the next two weeks
<jcsackett> sinzui: dig. the side portlet summary bit too, i'm guessing? (seems *very* uncontroversial).
<sinzui> They are
<jcsackett> righton.
<sinzui> We do need to update the text about sharing Some. This is really an audit and retraction state, we do not allow users to grant it and we do not explain who someone might accomplish sharing Some. Dan and I will work on that
<lifeless> bigjools: hai
<bigjools> lifeless: yellow - needed to talk to you as it happens
<lifeless> bigjools: we should talk a little about the ppa admin / commercial admin thing
<bigjools> now is not a great time
<lifeless> sure
<lifeless> what did you want tto ttalk aabout ?
<bigjools> I want to poke you to make a testtools release and push it to debian so I can sync it to precise
<bigjools> jono fixed a nasty bug
<lifeless> yes
<lifeless> then did a release
<lifeless> it had a regression
<lifeless> another release is needed, I'm not sure if he has done that yet
<bigjools> arses
<lifeless> jml: ^ oh hai
<jml> what's up?
<lifeless> jml: backscroll
<jml> oh right testtools
<jml> haven't any feedback on my subunit branch
<jml> wanted that before I released, since the regression is changing a private API that subunit relied on
<lifeless> I can do something about that
<lifeless> jml: it conflicts
<jml> lifeless: it didn't when I submitted it, and I'm pretty busy right this second.
<lifeless> jml: I haven't changed trunk since
<lifeless> jml: given your other branch which duplicated stuff done in trunk a couple weeks back, I'm fairly sure you worked off of a stale base of some sort
<jml> lifeless: ok whatever, it *did* conflict when I submitted and I'm pretty busy right this second.
<lifeless> so, let me see if I can figure out what it is doing
<lifeless> reviewed
<jml> lifeless: thank you.
<jml> lifeless: I'm unlikely to get to that until tomorrow morning UK time, but will try to do so then.
<sinzui> danhg, wallyworld is landing a change that will allow Lp to show the definition of an enum: http://people.canonical.com/~ianb/enhanced-choice-picker.png
<sinzui> danhg, I think we need to rewrite some bug definitions before putting the feature into beta
<rick_h> woot! combo loader tests passed yay
<jcsackett> rick_h: nice!
 * rick_h sees a light in the tunnel...now to determine if it's aimed at him
<deryck> rick_h, awesome
<mtaylor> any losas around want to help a brother out with another merged-account-openid-confused issue? https://answers.launchpad.net/launchpad/+question/187825
<mthaddon> mtaylor: only one at the moment (me) and I'm in a meeting I'm afraid - can you assign it to ~canonical-losas and someone will get to it at some stage
<mtaylor> mthaddon: I surely will! thanks!
<mthaddon> thx
<mtaylor> mthaddon: it won't let me assign - is subscribe someone else good enough?
<mthaddon> mtaylor: I've assigned it to be sure
<rick_h> bwuhahaha, the quad-ec2 land.
<lifeless> flacoste: got a minute ?
<flacoste> lifeless: sure
<czajkowski> evening
<thumper> morning
<czajkowski> thumper: hey
<mhall119> is there a way to get a history of all previously merged branches from launchpadlib?
<lifeless> deryck: hey, do you ahve time for a quick chat this avo ?
<deryck> lifeless, I don't actually.  and was trying to keep my tomorrow afternoon open for hacking, too. just trying to finish my current branch.
<deryck> lifeless, should we try to get in something just as you come online tomorrow?
<lifeless> lets do
<lifeless> will be a short call, I promise
<deryck> ok, sounds good.
<lifeless> thanks!
<lifeless> czajkowski: hey
<czajkowski> lifeless: evening
<lifeless> oh, I was going to talk about trivial vs nontrival when I realised I had misread the bug.
<lifeless> czajkowski: also good evening ;)
<czajkowski> lifeless: mind If I ask you a q re a bug while I'm here
<lifeless> of course not
<czajkowski> lifeless: thanks re : https://bugs.launchpad.net/launchpad/+bug/931131
<_mup_> Bug #931131: recipes attempt to build packages with version older than previous build <recipe> <Launchpad itself:Confirmed> < https://launchpad.net/bugs/931131 >
<czajkowski> it's been confirmed by the janitor, but after you commented on it, wasn't sure if it should be marked triaged?
<lifeless> it hasn't been triaged
<lifeless> (not the importance is undecided still)
<lifeless> I'll triage it now; yes you are right the status [and importance] needed setting
<czajkowski> ok, just didnt want to change it today without askin so I can know in future
<lifeless> czajkowski: don't stress, you *will* make mistakes, and they will be easy enough to fix :)
<lifeless> czajkowski: you've seen https://dev.launchpad.net/BugTriage right ?
<lifeless> czajkowski: I have to pop out for a bit; have a good evening.
<czajkowski> lifeless: I did and will keep it open during the day, but was going through stuff with mat today and just wanted to check
<czajkowski> thanks
* abentley changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: - | Firefighting: - | Critical bugtasks: 4*10^2
<wgrant> jelmer: Hi
<czajkowski> wgrant: jcsackett aloha
<wgrant> Evening czajkowski.
<jcsackett> aloha, czajkowski.
<czajkowski> jcsackett: hi
<czajkowski> folks having a good day/end of day ?
<jcsackett> passably good. :-)
<wgrant> lifeless: Do you know what's going on with the two branch scanner revs?
<wgrant> One is bad, but not rolled back AFAICT.
<lifeless> I have not heard
<lifeless> jelmer: ^ ohai
<jelmer> hi wgrant, lifeless
<jelmer> I'm in the process of rolling them both back
<jelmer> the other one is probably fine but I can't really QA one without the other
<wgrant> jelmer: Any reason this wasn't reverted a few hours back? It's just missed another buildbot run.
<czajkowski> nn folks
<jelmer> wgrant: I didn't have the time to (it's well past EOD here)
<wgrant> k
<wgrant> In general, rollbacks are the most critical thing that can be done. If you don't have time, please poke someone else to do it.
<jelmer> ok
<StevenK> +            # We'd like visibility near the top. Eyes closed, please.
<StevenK> +            field = self.form_fields.__FormFields_seq__.pop()
<StevenK> +            self.form_fields.__FormFields_seq__.insert(2, field)
<lifeless> jelmer: thanks. And for context - LP has been blocked for 2 days now; our goal is multiple deploys *per day*, so this is a massive impediment
<jelmer> lifeless: Sorry. I didn't mean to impede. It's been a while since I've landed something and landings seem to be happening a lot faster now.
<lifeless> jelmer: tis ok; I'm explaining why you're getting nagged so much :)
<sinzui> StevenK, self.form_fields = self.form_fields.select(['name', 'visibility'])
<sinzui> StevenK, http://pastebin.ubuntu.com/843663/ show a reselection for field to omit/change the order of setup fields
<wgrant> jelmer: Thanks.
 * wgrant murders buildbot.
<StevenK> sinzui: https://code.launchpad.net/~stevenk/launchpad/better-name-field/+merge/93317
<sinzui> thank you
<sinzui> wallyworld_, I think these two bugs relate to the errors users see when creating teams because the form is governed by invariant rules: bug #463563 bug #604289
<_mup_> Bug #463563: Create a new team UI has optional renewal period, then returns an error when not supplied <lp-registry> <teams> <Launchpad itself:Triaged> < https://launchpad.net/bugs/463563 >
<_mup_> Bug #604289: on open teams / mailing lists 'renewal' doesn't make much sense <lp-registry> <teams> <Launchpad itself:Triaged> < https://launchpad.net/bugs/604289 >
<wallyworld_> sinzui: thanks, will look
<sinzui> wallyworld_, I report Bug #933159 about the restricted policy
<_mup_> Bug #933159: +newteam form does not state that restricted is valid subscription policy for private teams <disclosure> <privacy> <teams> <Launchpad itself:Triaged> < https://launchpad.net/bugs/933159 >
<wallyworld_> sinzui: ah, yes. i saw that issue recently but forgot to report a bug. will fix
<sinzui> wallyworld_, I think we have a closed bug about this issue. I has dogged me for years
<wallyworld_> sinzui: closed?
<wgrant> lolzors
<wgrant> hhhhhhaaaaaaaaaaa
<wgrant> why do all these tables have no relevant indices
<wgrant> (today's winner is bugbranch.branch)
<sinzui> wallyworld_, I think there was a bug describing several issue with the form. Many were fixed, but this issue was not separated into a new bug
<lifeless> indices are for wusses
<wallyworld_> sinzui: ah, ok. not an uncommon issue. bug reports often are not granular enough :-)
<wgrant> Hmm
<wgrant> lifeless: Does https://pastebin.canonical.com/60335/ have similar planning overhead to yesterday?
<wgrant> There's about 70ms of 75ms unexplained.
<sinzui> StevenK, approved with some non-binding comments
<wgrant> Needs to be on prod, since stub was playing with the staging stats yesterday
<lifeless> Time: 44.003 ms to explain (no analyze)
<wgrant> OK
<wgrant> I think we may want to drop the stats target globally, and raise it only on specific columns.
<wgrant> (it's 2500 on prod now, rather than the default of 100)
<lifeless> I buy that; but how much by?
<wgrant> We'd have to test. Should be pretty easy to do on qastaging.
<lifeless> we should also report a bug
<lifeless> and check on pg9.1
<lifeless>  /9.2
<wgrant> I'm not sure it's a bug.
<wgrant> It's analyzing 25x more data.
<lifeless> it may be doing it inefficiently
<wgrant> I wonder if this explains week 47-49 on https://lpstats.canonical.com/graphs/PPR/20111016/20120216/
<wgrant> lifeless: Ah, the default is actually 10.
<wgrant> No, that was 8.2
<wgrant> 8.4 is 100
<wgrant> "The default limit is presently 100 entries. Raising the limit might allow more accurate planner estimates to be made, particularly for columns with irregular data distributions, at the price of consuming more space in pg_statistic and slightly more time to compute the estimates."
<wgrant> Slightly more time... but we are running at 25x more, so perhaps more than slighlt.y
<lifeless> -> bug
<wgrant> not sure if serious
<wgrant> Increasing performance sensitive parameter globally to 25x the recommended value
<lifeless> did we file one that explain doesn't count planning time?>
<wgrant> That does not make a bug.
<wgrant> Not sure.
<wgrant> We have stub early today, though :)
<wgrant> Can discuss then.
<lifeless> tomorrow
<wgrant> Bah, true.
<wgrant> Anyway, this seems to be a reasonably serious and global performance issue.
<StevenK> sinzui: .select doesn't want a list
#launchpad-dev 2012-02-16
<wgrant> Hmm
<wgrant> So pillarname is the most read table on the system by around 50%. And the main query that used it was 30x slower than it needed to be.
<wgrant> Our performance situation is hilarious.
<wgrant> == Most Bloated Tables ==
<wgrant> logintoken || 96% || 555 MB of 576 MB
<wgrant> Is autovacuum broken?
<lifeless> no
<wgrant> Oh-ho-ho
<wgrant> I think it's all slony
<wgrant> The performance regression in week 47 last year (the week of Nov 19) coincides with slony jumping from 20% to 85-110% master CPU usage.
<wgrant> I wonder if that's when we upgraded.
<wgrant> Compare https://devpad.canonical.com/~lpqateam/dbr/weekly/db-report-2011-11-11-2011-11-18.txt and https://devpad.canonical.com/~lpqateam/dbr/weekly/db-report-2011-11-18-2011-11-25.txt
<wgrant> No major write load changes.
<wgrant> The upgrade was the 18th.
<wgrant> To slony2
<wgrant> That is slightly suspicious, if you ask me.
 * StevenK blinks at FormFields.select()
<StevenK> It's second argument is *names, and when I pass it a list, (Pdb) p names
<StevenK> (['name', 'displayname', 'visibility', 'contactemail', 'teamdescription', 'subscriptionpolicy', 'defaultmembershipperiod', 'renewal_policy', 'defaultrenewalperiod', 'teamowner'],)
<wallyworld_> StevenK: who is a lp.dev user with launchpad.Commercial?
<StevenK> admin ?
<wgrant> wallyworld_: commercial-member@canonical.com
<wgrant> or admin, yeah
<wallyworld_> thanks
<wgrant> wallyworld_, sinzui: Why BugTaskVisibilityPolicy?
<wgrant> Isn't it global to all artifacts?
<wallyworld_> wgrant: it's not
<wallyworld_> unless something went wrong with the landing
<wgrant> Oh, the commit message was wrong.
<wgrant> The code is right.
<wallyworld_> bugger, didn't change it when i changed the code
<wallyworld_> StevenK: i think the rule in checkAllowVisibility() is wrong? if ff is on, it no longer allows lp.Commercial if ff is on
<wgrant> I had also intended that it be AccessPolicyType, but we can discuss that.
<wgrant> Since it's not an actual policy.
<wallyworld_> sure, easy to change
<wgrant> Just a class of policy.
<StevenK> wallyworld_: Damn good point
<StevenK> wallyworld_: I can subtly fix that in this branch if you want
<wallyworld_> StevenK: yes please :-)
<wallyworld_> and add a test
<wallyworld_> StevenK: just noticed, the public/private field on Team+edit suffers the same issue as the private_bugs field
<wallyworld_> StevenK: when you edit a private team, the public/private field says Public
<wallyworld_> it's not being initialised from the context properly
<StevenK> Right
<wallyworld_> StevenK: lp:~wallyworld/launchpad/private_bugs-init-932721 has my solution
<wallyworld_> to the problem which works
<lifeless> StevenK: thats a list in a tuple
<StevenK> lifeless: Well, yes.
<StevenK> lifeless: I needed a *
<StevenK> Which blows the list apart into arguments, I think
<lifeless> if thing = ([foo,bar],)
<lifeless> quux(*thing) == quux([foo,bar])
<StevenK> thing = [foo, bar]
<lifeless> then
<lifeless> quux(*thing) == quux(foo, bar)
<StevenK> Right, so I was right
<lifeless> (but your names above was ([foo, bar],)
<StevenK> lifeless: I was calling select(list), where it was 'def select(self, *names)'
<StevenK> Calling select(list) got me ([foo, bar],)
<StevenK> Calling select(*list) got me (foo, bar)
<lifeless> kk
 * StevenK grumbles at IPerson.checkAllowVisibility()
<mwhudson> what is the intent of the register merge button doing ajax rather than a direct post?
<mwhudson> is it supposed to generate the diff immediately?
<StevenK> No
<StevenK> It is supposed to tell you when stuff is wrong so you can fix it
<wgrant> It'll be unbroken in a few hours.
<mwhudson> StevenK: ah ok
<mwhudson> StevenK: 'wrong' ?
<StevenK> wallyworld_: ^
<wgrant> Requesting a review from someone who can't see the branches.
<wallyworld_> StevenK: ?
<wallyworld_> it will warn you if you are about to expose private branches
<wallyworld_> and ask you to confirm
<mwhudson> ah ok
<wallyworld_> mwhudson: since now nominated reviewers are subscribed to the source/target branch if the branch(es) would otherwise have been invisible to the reviewer
<StevenK> BAH
<StevenK> I see what broke the test
<StevenK> We only want to render the context for edit
<StevenK> Otherwise it looks for IPersonSet.visibility, and then the toys get de-pramed
<StevenK> wallyworld_: I think I'm done with your changes.
<wallyworld_> StevenK: excellent
<StevenK> wallyworld_: I'll get you to review the changes I added, if you don't mind.
<wallyworld_> np
<StevenK> But that will about ten, since the kettle is calling
<wallyworld_> can relate to that
<StevenK> wallyworld_: Sorry, I also cleaned up after lunch, but r14801 only on https://code.launchpad.net/~stevenk/launchpad/better-name-field/+merge/93317
 * wallyworld_ looks
<StevenK> wallyworld_: Feel free to add your +1 on the MP if you want.
<wgrant> StevenK: Do you plan a fastdowntime tonight?
<wallyworld_> StevenK: why not use check_permission('launchpad.Commercial', self)? it was just the placement that was wrong, not the method call itself
<StevenK> wallyworld_: Because it wasn't working, and check_permission() in model code is always a bit suspect
<StevenK> wgrant: That sounds like a good plan
<StevenK> I think the mass index death patch is first
<wgrant> It is indeed.
 * StevenK reaches for staging logs
<wgrant> Hmm
<wgrant> Now, unity, give me back my coding terminal.
<wgrant> Please?
<StevenK> wgrant: gvim? :-P
<wgrant> It's still there in Alt-Tab, but has otherwise disappeared.
<wallyworld_> StevenK: ah, didn't notice it was in model. in such circumstances, i would instantiate and call the security adaptor.'
<StevenK> wallyworld_: Example?
<wallyworld_> so that we know the same security checks are used
<StevenK> wallyworld_: I quite like the use of IPersonRoles
<wgrant> IPersonRoles is the right thing to do here.
<wallyworld_> StevenK: except that it used to be launchpad.Commercial
<wallyworld_> and now it's different
<StevenK> in_commercial_admin() is identical
<StevenK> For this purpose
<wallyworld_> so we really want to allow in_admin as well?
<StevenK> Both should be allowed
<StevenK> .in_admin == ~admins
<wallyworld_> sure, but it's new behaviour
<wallyworld_> just checking that it's really required
<StevenK> Not really, now it's explicit
<StevenK> admins have lp.Commercial
<wallyworld_> right, didn't know that
<StevenK> admins also have lp.Moderate and I think they have lp.Special
<wgrant> No
<wgrant> lp.Special exists solely to exclude admins.
<wgrant> That's its entire purpose.
<StevenK> Scoff
<lifeless> which is a bit backwards way of doing it
<lifeless> its on the wrong dimension
<wallyworld_> StevenK: r=me
<StevenK> wallyworld_: Thanks!
<StevenK> wallyworld_: Did you want to scribble the MP, or you don't care?
<wallyworld_> np. be good to get all this stuff landed
<wallyworld_> StevenK: already have :-)
<StevenK> wgrant: I'm not sure I trust dbupgrade.log
<wgrant> StevenK: Oh?
<wgrant> The times in there should be accurate.
<StevenK> 2012-02-15 11:15:36 INFO    2209-10-0 applied just now in 0.1 seconds
<lifeless> yes, any ?
<StevenK> I don't believe it. That patch drops 63 indices
<lifeless> dropping an index is cheap
<StevenK> So stub said, delete a file, sync, move on
<lifeless> little more than that but yeah
<lifeless> (metadata updates, locks, etc)
<StevenK> I'd just expect dropping 63 to take more than 100ms
<lifeless> did you confirm my question about FK indices ?
<lifeless> StevenK: its one transaction, so only a few syncs, not one per index.
<StevenK> I saw it, I didn't think we were dropping any
<lifeless> branch.owner ?
<lifeless> I haven't gone looking for others but I'd expect that to be the index we need for efficient referential-integrity queries
<StevenK> But we don't select via owner. The index has been untouched
<wgrant> It was branch__owner_name__idx that was dropped
<wgrant> Not owner
<wgrant> owner_name probably hasn't been used since unique_name was introduced.
<mwhudson> that index was probably a hang over from before source package branches
<lifeless> ah, cool.
<lifeless> StevenK: the indexes I'm concerned about *wouldn't be used*
<lifeless> or at least, very rarely
<wgrant> Person merges would use them.
<StevenK> lifeless: So, I trust stub's judgement on this
<lifeless> sure
<lifeless> I will ask if he checked
<lifeless> which is all I was asking you
<lifeless> that someone took the time to remember this step
<StevenK> wgrant, I, and stub spent a few hours talking over all of them
<wgrant> I disagree with the removal of some.
<wgrant> eg the ones on incrementaldiff
<wgrant> They're only unused because the feature is disabled.
<wgrant> But I don't care enough to revert the patch.
<StevenK> Index creation can be done without downtime
<wgrant> They're easy to add back later if we don't want to delete the feature.
<wgrant> Right.
<StevenK> wgrant: FDT request up
 * StevenK glares at his laptop
<StevenK> If I double a text file on my desktop to bring up gedit, Unity crashes
<StevenK> If I use gnome-do to bring up the same file, it doesn't
 * wgrant blames nautilus
<wgrant> And thumper.
<StevenK> It's oneiric, so his care factor is probably a large negative number.
 * StevenK stabs _lock_actions more.
<cody-somerville> Why does buildmailman.py take so long to run?
<StevenK> Because mailman is large
<cody-somerville> and wtf
<cody-somerville> ImportError: Bad magic number in /home/cody-somerville/Projects/launchpad/lp-branches/devel/lib/lp/services/timeline/timeline.pyc
<cody-somerville> fixie fixie
<cody-somerville> anyhow, either launchpad has gotten slower to start or my machine has gotten slower :(
<cody-somerville> hmm... ImportError: No module named convoy.meta. Am I missing a dep?
<StevenK> Yes
<StevenK> launchpad-developer-dependencies depends on it
<cody-somerville> What is convoy anyhow?
<cody-somerville> and can I copy the lucid build with binaries of it into natty series of ppa?
<StevenK> Why are you developing LP on Natty?
<StevenK> I was going to remove the Natty packages from the PPA
<cody-somerville> :( cause I like Natty
<StevenK> cody-somerville: We offically support the latest LTS and the latest release only.
<StevenK> However, I'll fix the convoy recipe to build for Natty.
<cody-somerville> \o/
<cody-somerville> StevenK, You're the best.
<StevenK> cody-somerville: And to answer your question: Convoy is a WSGI app for loading multiple files in the same request.
<StevenK> Our plan is to use it for JS rather than the horrible launchpad.js
<cody-somerville> Is it faster?
<StevenK> It will result in domReady happening quicker, because your browser no longer has to digest 3MiB of JS
<StevenK> cody-somerville: launchpad.js is a concatenation of every JS file in our tree + YUI. convoy means that we only send down the files that are needed for each page in one request
<cody-somerville> Cool. Are you taking advantage of xsendfile header?
<StevenK> No, we're not using XSendFile
<wgrant> stub: I found some more queries today which have massive planner overhead due to the stats things.
<wgrant> stub: Should we drop the target down to something more sensible except for problematic cols?
<stub> wgrant: I'm considering that, yes. Unfortunately, for the problematic issues we would need to set it to a value we know is too low (100).
<wgrant> stub: What if we drop to like 500 and see how it goes?
<stub> I'm trying to assemble a test case for upstream, so the more the merrier.
<wgrant> stub: Also, slony 2 is a performance regression.
<stub> Dropping it to 500 push the issue below the radar, but it will still be lurking. Not sure if that is the best idea.
<wgrant> The unidentified global performance regression from November coincides exactly with the slony 2 upgrade, which coincides with 15x more sl_log_* tuple reads and slony going from 15% DB CPU usage to 110%
<stub> So I have a collection of ideas I'm not happy with :)
<wgrant> True
<stub> wgrant: There is a known regression in Slony that is fixed with the next patch level that matches what you are seeing.
<wgrant> http://www.slony.info/bugzilla/show_bug.cgi?id=167 is possibly relevant
<wgrant> Yeah
<wgrant> We should upgrade to 2.1, or possibly increase the intervals back to their defaults.
<wgrant> See if that alleviates it.
<stub> Is it affecting perceived performance? atm we have CPU and disk to spare I think, and the sl_log_ entries are going to end up in RAM anyway.
<wgrant> Compare https://devpad.canonical.com/~lpqateam/dbr/daily/db-report-2011-11-16.txt and https://devpad.canonical.com/~lpqateam/dbr/daily/db-report-2011-11-18.txt
<wgrant> I agree that it shouldn't be a problem.
<wgrant> But the PPR graphs say otherwise.
<wgrant> https://lpstats.canonical.com/graphs/PPR/20111017/20120217/
<wgrant> Week 47 is the slony upgrade
<wgrant> web increases by ~0.8s during that week and stays there
<wgrant> It's possibly not related.
<wgrant> But slony eating more than a core sounds undesirable anyway.
<stub> wgrant: I have a suspicion we did a point release upgrade of PostgreSQL to 8.4.9 at the same time
<wgrant> 21:18 < mthaddon> stub: any objections if we upgrade postgresql while we're at it? (security update)
<wgrant> Indeed.
<stub> Hmm... or would have have been the previous month? That point release was out 26 sept, but not sure how long it took for the packages to be made, dribble through to lucid and actually get deployed.
<wgrant> Buildbot broke in late October
<stub> Right. So the PPR graph is quite probably the planner issue.
<wgrant> Yeah
<wgrant> Fix join selectivity estimation for unique columns (Tom Lane)
<wgrant> This fixes an erroneous planner heuristic that could lead to poor estimates of the result size of a join.
<wgrant> That doesn't look irrelevant.
<stub> It was suspected when I've discussed this on IRC, but I'll need to prove it to Tom.
<wgrant> Fortunately we have a nice sacrificial sourcherry where we can reproduce the issue...
<stub> Not really sure how PostgreSQL downgrades work :)
<wgrant> No, but patching out the planner changes does :)
 * stub gets dragged out to lunch
<wgrant> stub: Setting branch.owner and teamparticipation.{person,team} to 2500 is enough to get it to 40ms on DF
<wgrant> For one query I tested.
<wgrant> https://pastebin.canonical.com/60335/ is the one.
<wgrant> Ouch
<wgrant> *Up* to 50ms if I remove the second part of the UNION
<wgrant> stub: http://paste.ubuntu.com/844047/ is a reasonably minimal (but LP prod-specific) example with timings.
<wgrant> So it gets pretty serious pretty quickly.
<wgrant> And reproduced all those results to within 1ms on clones of those tables with only the relevant columns and no indices.
<stub> wgrant: You are *upping* the stats to improve the planner time?
<stub> 2500 is the current production and staging default
<czajkowski> aloha
<wgrant> stub: No, this is on DF, which defaults to 100
<wgrant> The first column is the list of columns that I changed to 2500 and reanalyzed.
<stub> wgrant: "Setting branch.owner and teamparticipation.{person,team} to 2500 is enough to get it to 40ms on DF". Did upping the stats to 2500 improve planner time to 40ms, or are you talking about query time?
<wgrant> stub: enough to get it to 40ms from 1ms
<wgrant> Remember that on DF planning is quick
<stub> oic
<wgrant> because of the tiny stats
<wgrant> stub: Hm, did you tweak stuff on prod? I have yesterday's 200ms query taking ~15ms now.
<stub> I applied a patch changing the statistics target on the TeamMembership.person column to 100
<wgrant> Ah, assumed that wasn't live, since it was on db-devel.
<stub> (all columns on that table could happily go to 100, but I didn't want to be invasive yet as there is a reasonable chance I'll be changing the default)
<stub> Guess I should have landed it on devel now we have twiddled the pqm rule.
<wgrant> Yeah, otherwise scheduling fastdowntime gets confusing.
<wgrant> For us mortals without access to prod LDR :)
<stub> Should wire that up on a web page in Launchpad. We never made a /+admin or similar jump off page did we?
<wgrant> I've been considering that for a while.
<wgrant> Just needs to be something trivial and text line +opstats, I guess.
<wgrant> s/line/like/
<wgrant> There's still significant overhead on a lot of queries, but it's vastly reduced on yesterday's.
<wgrant> Heh
<wgrant> My pillarname fix worked well
<wgrant> Reads are down by 99%
<wgrant> Wow
<wgrant> pillarname has in fact dropped off the DBR entirely, from being the top read table by like 50%
<stub> wgrant: When do Bug.access_policy and branch.access_policy get dropped or repaired?
<stub> wgrant: What was the pillarname fix?
<wgrant> stub: PillarNameSet.getByName was doing a full seq scan on pillarname
<wgrant> That's the API that's, you know, used to traverse to products and distros.
<stub> Yer. Always wondering why that table was so popular for reads :)
<wgrant> It's probably read by more requests than any other table.
<wgrant> But the reads should each be roughly one tuple.
<wgrant> stub: Hmm, I should probably drop them with the rest of the schema changes.
<wgrant> Forgot that detail.
<wgrant> branch.access_policy will probably come back at some point, but bug.access_policy is dead to us.
<StevenK> I love it when a branch goes through ec2 first time
<StevenK> Yay for 4 changes as well as knowing what tests are impacted
<wgrant> stub: Thanks
<wgrant> Those columns are gone in -2
<stub> wgrant: So people get access to an artifact either via being granted access to a policy or being granted directly to the artifact?
<wgrant> stub: Right. Same as the old schema, just with multiple policies per artifact, and modelled slightly differently with denormalisation.
<wgrant> So to see an Ubuntu security bug you need to either be in ubuntu-security (which has a grant for the ubuntu security policy), or have a specific grant for the bug.
<stub> wgrant: I don't understand why any of the 'FK to be added later' columns need to have their foreign keys added later rather than now.
<wgrant> stub: Person merging and branch deletion need special code to handle foreign keys to their tables.
<stub> ok
<wgrant> -3 adds that code, -4 will add the foreign keys
<wgrant> The ugliest part of fastdowntime :(
<wgrant> We really should rework that code to make it easier, but I'm not quite sure how.
<stub> Hey, you guys wanted it like this :-)
<wgrant> Pfft.
<stub> You could add the columns to the 'ignore these columns' sets in -1 and -3, but probably no real gain
<stub> well... one less fast downtime
<wgrant> Ah, yes, could this time, because there's an initial patch.
<wgrant> Might as well do that.
<stub> I'm pretty certain person merge has the ignores stuff you need. I don't know about the branch removal code.
<stub> Probably not - ignores doesn't make sense there except for this sort of db patch juggling.
<wgrant> Branch removal doesn't actually check. It just has a doctest that lists all the FKs.
<StevenK> Oh, twitch
<StevenK> That's a pointless test
<wgrant> Oh?
<StevenK> How does listing all FKs on branch help in any way?
<wgrant> Because it will fail if you've added one, hinting that you need to add new rules to handle it.
<stub> AccessPolicyArtifact needs a PRIMARY KEY
<wgrant> Curses, you're right.
<czajkowski> morning
<wgrant> Morning czajkowski.
<czajkowski> has anything happens since last friday with launhcadlib , a user could login on friday and today is seeing errors and not not able to use it
<wgrant> czajkowski: The user is using staging. staging's database gets reset over the weekend, so the credentials from last week are no longer valid.
<czajkowski> wgrant: thank you
<wgrant> They may need to use a keyring manager to remove the cached credentials.
<czajkowski> nods
<czajkowski> thanks for explaining
<wgrant> np
<StevenK> czajkowski: That's one thing to keep in mind -- "I'm having a problem with LP" covers a multitude of sins, you should get into the habit of asking if they're using production, edge, staging or qastaging if they don't say so themselves.
<wgrant> (in this case the error showed it was staging)
<czajkowski> StevenK: wgrant nods ok
<StevenK> czajkowski: mrevell can explain the gory history about that four instances (and there is a fifth, but non developers tend not use it) if you care. :-)
<lifeless> StevenK: czajkowski: s/edge//
<czajkowski> thanks folks
<StevenK> Oh, right, edge redirects now
<wgrant> lifeless, StevenK: Not for the API
<cody-somerville> How long does the Soyuz test suite take to run these days?
<StevenK> Like an hour?
<StevenK> buildmaster doesn't add much
<cody-somerville> omg.
<StevenK> cody-somerville: The entire testsuite on buildbot is now taking about 5.5 hours
<cody-somerville> lol. I just ran it in 123.1 seconds - Tests with failures: runTest
<cody-somerville> Okay. So I'm looking to export the authorized_size attribute on PPAs. StevenK: I'll buy you a beer at UDS if you do it for me :)
<wgrant> stub: Declared the PK, dropped an index that was redundant with it, and renamed one which had an obsolete name.
<StevenK> cody-somerville: And if I'm not at UDS? :-)
<wgrant> It's a trap, because we don't go to UDS :)
<cody-somerville> lol
<StevenK> cody-somerville: To export it read-only, it's fairly easy
<StevenK> To *change* it over the API, that's a little harder
<cody-somerville> I want to be able to change it (provided one has the necessary permissions)
<stub> ta. just looking at the triggers
<StevenK> Yes, I thought you might. :-)
<cody-somerville> Isn't it just an int field?
<StevenK> cody-somerville: So, you can wrap it in exported(), and write a bunch of tests to see :-)
<cody-somerville> If only that was easier :(
<stub> wgrant: SECURITY DEFINER functions also need to add a 'SET search_path TO public' clause for security reasons (patch-2209-00-5.sql has an example).
<StevenK> cody-somerville: webservice tests aren't too bad
<cody-somerville> when I run make schema, I get 'ProgrammingError: text search configuration "default" does not exist'
<stub> Not that it should be possible to exploit that in our setup, but defence in depth and all that.
<wgrant> Yep
<wgrant> cody-somerville: Have you run launchpad-database-setup?
<wgrant> It's in utilities/
<cody-somerville> I have in the past
<StevenK> You probably need to again
<cody-somerville> ugh. it destroyed my database, lol.
<stub> cody-somerville: In a good way?
<stub> That script scares me
<cody-somerville> No. I thought I could provide a different account name as argument and it would just destroy stuff belonging to that
<cody-somerville> but nope. it just went ahead and dropped the entire cluster
<wgrant> That's why it gives you a warning :)
<wgrant> Oh, but it lies.
<wgrant> Because it indeed drops the whole cluster.
<cody-somerville> 'THIS SCRIPT WILL DESTROY ALL POSTGRESQL DATA for the given user'
<stub> That isn't a lie, it just doesn't tell you everything
<cody-somerville> I gave it a non-existent user as an argument, lol
<cody-somerville> welp, at leasts the tests are running correctly now
<wgrant> stub: Is there any way we can tell what's reading 1.5 million branch tuples/sec?
<wgrant> That seems pretty unlikely to be legit.
<stub> wgrant: Those stats are not linked to particular queries or connections unfortunately.
<wgrant> Yeah.
<wgrant> Will see if the go away in 15 minutes when everything's disabled.
<stub> wgrant: The only way I can think of would be to turn on statement + plan logging and analyze that.
<wgrant> That will narrow it down to the webapp.
<wgrant> stub: Ah, thanks for the approval.
<wgrant> Unless it's translations_import_queue_gardener it must be the webapp...
<gmb> Folks, I know I'm being dumb, but what do I need to do to make this go away:
<gmb> Traceback (most recent call last):
<gmb>   File "utilities/js-deps", line 4, in <module>
<gmb>     from convoy.meta import main
<gmb> ImportError: No module named convoy.meta
<gmb> (when running `make`)
<StevenK> Upgrade your dependencies
<gmb> StevenK, That's what I thought... I'll try again.
<wgrant> gmb: apt dependencies
<wgrant> Make sure launchpad-developer-dependencies is installed
<wgrant> And that you're not running natty, which IIRC you are.
<gmb> Ahahahahahahhaahahasdhashdaqhrwuqwhrquiwhrqui3hr1ui32hr1ui3rh13iurh3stabstabstabstabdie
<StevenK> Haha
<gmb> wgrant, I'm on Precise.
<wgrant> Oh, OK
<gmb> For juju goodness.
<gmb> And lxc
<wgrant> One of the other MacBookers was on Natty at the 'dome
<gmb> and other crazy-ass stuff.
<gmb> wgrant, Yeah, but I'm running in a VM; I'm too lazy to do the work to get Ubuntu working properly on the metal.
<wgrant> Heh
<gmb> And whaddaya know, lp-dev-deps isn't installed any more.
<gmb> Goodness knows why.
<gmb> Thanks fellas.
<bigjools> gmb: release upgrade removes it
<gmb> bigjools, Yeah, I must've forgotten to re-add it afterwards.
<jml> bigjools: testtools 0.9.14 released, fixing the subunit incompatibility.
<jml> I really want that in precise.
<StevenK> jml: Feature freeze is today, get cracking
<rick_h> morning
<jml> StevenK: I don't know what to do. Honest.
<jml> StevenK: otherwise I would.
<StevenK> Looks like it has been synced from Debian in every release except precise
<bigjools> jml: awesome
<jml> StevenK: normally lifeless takes care of it, but he's been busy this cycle.
<bigjools> jml: I'll try and get it poked in
<jml> bigjools: thanks.
<wgrant> stub: I think I've found the thing that's reading branch a lot. The planner makes very bad life choices: http://paste.ubuntu.com/844328/ is the >1s query issued by BranchLookup.getUniqueName, <1ms if I extract the distroseries bits into a CTE.
<stub> \o/
<wgrant> Branch.owner is extraordinarily skewed, but the planner should be able to tell that those joins map perfectly through unique indices :/
<wgrant> Unless I'm missing something.
<stub> Yes, it is expecting a user to not have many branches
<wgrant> Yeah, which is a reasonable assumption.
<wgrant> I can't fault it for that.
<wgrant> But I can fault it for not doing the dead-simple plan which evaluates the four unique joins individually :)
<stub> It might know that id XXXX has lots of branches (if the sample size is high enough). But it won't know that that Person.name matches that id.
<wgrant> Right, which is why it normally works, because we usually query by ID.
<wgrant> But in this case the indices should tell it that the WHERE clause constrains each join to a single row, surely.
<stub> The planner estimates agree with that
<stub> It was expecting a single row at every stage of the plan, but things went pear shaped when it found that user actually owned 21k branches
<StevenK> That's what happens for ubuntu-branches, though
<wgrant> I'm still not sure how that can work out cheaper than the plan I suggest, but I guess index reads might do it.
<StevenK> I say go for the CTE
<stub> Yes, planner being too thick. For every table in there we can guarantee a single row due to uniqueness on the id column, except for Branch which *probably* will return 1 row but that is a guess, not a guarantee.
<stub> Well, 0 or 1 rows
<wgrant> It can reason through the unique indices and say there's only one branch row.
<wgrant> Because it knows there's only one row for every element of one of its unique keys.
<stub> I think the problem is that it doesn't know that. It could, but not implemented.
<wgrant> Seems like it would be a useful sort of thing to know :/
<stub> wgrant: Paste your rewritten query?
<wgrant> http://paste.ubuntu.com/844355/
<wgrant> Joining the CTE is slow. It may be the one-row hint provided by the = that does it.
<wgrant> It's still not doing the right thing, but it works.
<wgrant> (it does a nested loop from branch onto person)
<stub> So you want to filter the Branch table on those criteria? Where do you start? Join with distroseries, you end up with a working set of maybe 20k. Join with SourcePackageName you end up with a working set of maybe 30. But join with Person, you are probably going to end up with 1 row in your working set.
<stub> Rather than rewrite for a CTE, we might be able to craft an index that will be used for this and similar queries
<stub> Say (person, distroseries, sourcepackagename)
<stub> Suspect we always limit on both distroseries and sourcepackagename, never just sourcepackagename?
<wgrant> This query is the only one that appears to be particularly problematic.
<wgrant> Because it's called as the sole query behind an XML-RPC API
<wgrant> So has none of the objects.
<wgrant> So the stats are useless.
<wgrant> Because it's all string-based.
<wgrant> So, yeah, never just sourcepackagename.
<stub> Index fails for me
<wgrant> Likewise.
 * wgrant sleeps.
<stub> Night. I can't improve on what you have.
<wgrant> stub: Thanks. I might land that tomorrow, then.
<stub> http://paste.ubuntu.com/844395/ without the CTE
* jcsackett changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: jcsackett | Firefighting: - | Critical bugtasks: 4*10^2
<rick_h> https://code.launchpad.net/~rharding/launchpad/gallery-accordian_fix/+merge/93406
* rick_h changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: jcsackett, rick_h* | Firefighting: - | Critical bugtasks: 4*10^2
<rick_h> jcsackett: can you look over this when you get a sec pls? https://code.launchpad.net/~rharding/launchpad/gallery-accordian_fix/+merge/93406
<czajkowski> bigjools: did the person on this ticket contact you on irc to resolve the issue https://support.one.ubuntu.com/Ticket/Display.html?id=4997
<jcsackett> rick_h: sure, i'll take a look in a sec.
<czajkowski> sinzui: you about ?
<sinzui> I am
<czajkowski> ready for G+ ?
<sinzui> I am
<czajkowski> sinzui: https://plus.google.com/hangouts/133f83a9936535af8066fcbc7938f0325c3f4d8b?authuser=0&hl=en-GB#
<czajkowski> sorry for the delay
<czajkowski> we've been cleaning the RT system
<jcsackett> did our diff builder get much smarter? or has it always just shown one line when a file has just been moved?
<sinzui> czajkowski, bzr+ssh://bazaar.launchpad.net/~launchpad/lp-dev-utils/trunk/
<mabac> jcsackett, thanks for reviewing https://code.launchpad.net/~linaro-infrastructure/launchpad/workitems-model-classes/+merge/92174. would you also land it for us?
<abner`> folks, I'm planning to have a new bts installed in my server, but there's a complicated requirement here: I need two instances in two servers being synchronized. Any idea if it's possible to be done with launchpad?
<sinzui> czajkowski, this is the command to disable a translation project because the project is already registered ./disable_projects.py --msg=2 gmusicbrowserhutranslate
<jcsackett> lifeless: you around?
<bigjools> jml: https://launchpad.net/ubuntu/+source/python-testtools
<jml> bigjools: \o/
<jml> bigjools: thanks :)
<bigjools> jml: my extreme pleasure :)
<jcsackett> mabac: yes, that's on my todo list. :-)
<mabac> jcsackett, cool, thank you very much! :)
<sinzui> czajkowski, this is the command to disable one or more projects that sends an explanation to the project maintainer: ./disable_projects.py totten25
<sinzui> czajkowski, http://opensource.org/docs/osd
<syst3mw0rm> Hi
<syst3mw0rm> I would like to know the status of grackle project...as mailman is investigating the possibility of using grackle as new archiver framework
<sinzui> syst3mw0rm, It is not yet operational
<sinzui> syst3mw0rm, There are two needed parts to start testing. I am working on the client library right now. It might be done tomorrow
<syst3mw0rm> sinzui, what is the second part ?
<sinzui> syst3mw0rm, The Cassandra-based store is yet to be built and might be a month a way give that my squad does not have time to work on it
<syst3mw0rm> sinzui: Well, what do you think about the idea of using grackle as new archiver framework ?
<syst3mw0rm> for mailman
<sinzui> syst3mw0rm, I have been using a memory implementation store to test the client lib. I suspect the memory implementation will evolve into a a reference implementation that can be used to create other backends.
<sinzui> syst3mw0rm, That is my dream, but I think Cassandra's java is a nuance. I think something else with a python heritage would be more suitable.
<syst3mw0rm> so you mean that grackle can be used with other backends as well, and you will be using cassandra-based store for launchpad ?
<sinzui> syst3mw0rm, I already have a branch queued to land that makes out mailman use grackle
<syst3mw0rm> can you provide me with the link ?
<syst3mw0rm> sinzui, why did you choose cassandra when you think that cassandra's java is a nuance ?
<sinzui> syst3mw0rm, correct. grackle is providing a fast way to inject a message and a mechanism to get JSON encoded messages back. We have already completed the JS/AJAX lib that will allow the browser to read the archive doing a minimal number of server queries
<sinzui> I do not like java. I do not want to maintain a java-stack and python stack to run an archive.
<sinzui> As a developer, or someone deploying the code, I want few dependencies, not many
<sinzui> syst3mw0rm, Canonical like Cassandra it is not considered an extra dependency since it is the preferred blob store
<syst3mw0rm> ok
<sinzui> And to be fair to Cassandra, It was created to store messages by Facebook. I was surprised that an archive did not already exist
<syst3mw0rm> So, what they used to store the messages when Cassandra was not open source ?
<sinzui> syst3mw0rm, yes. I think Facebook has a one or more message stores in Cassandra. Their messages though are short, and without attachments.
<syst3mw0rm> sinzui: can you give me link of the branch, where makes mailman use grackle ?
<sinzui> syst3mw0rm, If I were to hack on the store on my own time. I would keep the mbox and use a db (sqlite default, but allow mysql, drizzle, postgresql) sot the indexes and json data. I think that will be fast enough for giant lists an easy to deploy
<sinzui> The mbox would only been needed to get the non-text parts
<sinzui> syst3mw0rm, https://code.launchpad.net/grackle
<sinzui> I pushed some changes a few days ago
<syst3mw0rm> sinzui:  non-text part parts, like attachments ?
<sinzui> yes
<sinzui> ^ syst3mw0rm
<syst3mw0rm> sinzui: ok.
<syst3mw0rm> sinzui: The link you have provided just have some unit tests..
<lifeless> morning
<sinzui> syst3mw0rm, there is code in there for the client lib. It is very small. The memory implementation is in the test. I will extract it after I have complete the client API
<sinzui> syst3mw0rm, the javascript lib will be moved into grackle soon. It is in Lp at the moment because it had the YUI infrastructure we depend on.
<syst3mw0rm> sinzui: nice..Can i also get involved with development of grackle client lib... ? As i am planning to make mailman use grackle..it would be better if i am quite familiar with grackle internals..
<sinzui> syst3mw0rm, that would rock. I expect to have the client completed this week. That allows ud to refine it next week, integrate the js, and work on the server. wgrant was working on the server, but he has more important things to work on. We can work on the server.
<syst3mw0rm> sinzui: seems like a great opportunity..
<syst3mw0rm> so i think i will get myself familiarized with cassandra till you finish the client part..and then we can take it forward form there itself..what do you say ?
<syst3mw0rm> we are going to work on cassandra itself, right ?
<syst3mw0rm> sinzui: ^
<lifeless> with, not on
<lifeless> being familiar with how cassandra modelling is done is a good idea though
<sinzui> syst3mw0rm, maybe. I certainly will use Cassandra when working on Canonical's time. On my own time I would write something that easier for someone to deploy with mailman
<lifeless> sinzui: cassandra is no harder to deploy than postgresql once packaged :)
<syst3mw0rm> lifeless: "with, not on" ?
<lifeless> syst3mw0rm: we don't expect to be making changes to cassandrra
<sinzui> lifeless, There are worlds ouside of Ubuntu's JuJu paradise.
<lifeless> sinzui: oh, I know - I wasn't meaning juju
<sinzui> lifeless, Most mailman installs would want a batteries-included option that meets the needs of 80% of mailman installs.
<lifeless> sinzui: cassandra will run single-node quite happily
<lifeless> sinzui: your time is of course your time; I just don't see much reason to do a different backend for other users: I expect most mailman installs are from Ubuntu packages already
<lifeless> sinzui: (or Debian with packages)
<sinzui> lifeless, I will not ever install a java stack to get mailman running for my biz/org The mediocre choices of archivers we have is partially because they are awesomely easy to setup at the moment of mailman's install
<syst3mw0rm> sinzui, lifeless i think that i should discuss it with barry warsaw who maintains mailman, so he might be having better idea about which backend service might be good for mailman installations...What do you guys say ?
<sinzui> syst3mw0rm, I certainly will talk to barry about the replacement of pipermail. That is why I think the server/store implementation needs to have options
<sinzui> python can needs of most cases, galaxy-size hosts need enterprise level backends
<syst3mw0rm> sinzui: your last statement isn't clear to me...
<syst3mw0rm> are you saying that we should have two options, cassandra for galaxy-size hosts and python based service for most of the other cases ?
<lifeless> that is what sinzui is saying
<lifeless> where Launchpad counts as galaxy size
<lifeless> salgado: nice finding of deletable code !
<sinzui> lifeless, I would not have said that two years ago. The terrible performance we see now is cause by some monster lists we now host
<salgado> lifeless, heh, it was danilo who told me about it :)
<sinzui> salgado, there are still map views and locate tables
<sinzui> personlocation tabled
<sinzui> tables
 * sinzui stops typing
<salgado> sinzui, map views?!  those I'd like to delete
<sinzui> I think we have two view, possibly unregistered sitting in lp.registry.browser.team
 * salgado goes find them!
<sinzui> wtf: salgado person.index still calls this: <div tal:content="structure context/@@+portlet-map" />
<sinzui> I suck I should have removed that 14 months ago
<salgado> I'm glad you did not
<salgado> the +map view on teams is still registered though
<salgado> yay, I'll kill 'em all
<salgado> (https://launchpad.net/~linaro-infrastructure/+map)
<lifeless> useful!
<lifeless> deryck[lunch]: ohhai
<sinzui> salgado, I removed a vocab a few months ago that was not being used.  There is not obvious way to locate an unused vocab, But i think there might be 2000 deletable lines
<salgado> sinzui, other vocabs, you mean?
<sinzui> yes
<salgado> niice, I'll have a look and see if I find any others
<abner`> folks, I'm planning to have a new bug tracking system installed in my server, but there's a complicated requirement here: I need two instances in two servers being synchronized. Any idea if it's possible to be done with launchpad?
<sinzui> salgado, tar up all the windmill directories to in the tree and put it in the dev wiki. you might delete 20,000 lines of code that is never executed
<lifeless> abner`: in principle yes, in reality no for two reasons; the federation system isn't all that mature (e.g. we have bits turned off because of issues) and we don't have an LP<-> federation module written
<lifeless> abner`: but you could just use postgresql replication ..
<salgado> sinzui, why put that in the wiki?
<abner`> lifeless, I have been trying to do this replication on the db layer, but it's not attending to all requirements.
<sinzui> I think deryck wanted to keep them as reference to what we did test when writing yui replacement tests
<abner`> lifeless, how launchpad stores the data? is it one big db with all projects inside?
<lifeless> we have 5 or 6 data stores (db, mailman, bazaar, debbugs replica, debian replica, librarian) with data stored according to its nature
<lifeless> there is a big DB, but its not the largest store if you measure by disk size; and possibly not largest by addressable-items either
<deryck> lifeless, I'll ping in about 30-1hr for call.  cool?
<czajkowski> lifeless: sorry for the multiple pings in -meeting
<czajkowski> i added the PPA CoC topic to the CC agenda to get some discussion/answers
<abner`> lifeless, hmm.. I'm asking because during the synchronization/replication I would need to transfer only the data of a specific project, not all project. So having an unique db with all project can also be a problem.
<lifeless> deryck: ok
<lifeless> czajkowski: my irc client doesn't see any pings
<sinzui> salgado, I am disappointed to see that windmill is only 2593 linez
<czajkowski> lifeless: what irc client do you use ?
<lifeless> czajkowski: irssi; it needs a : to invoke highlight
<salgado> sinzui, come on, every windmill line, when removed, should probably be worth like a dozen lines of proper code ;)
<lifeless> czajkowski: I could change that, but its good ATM - if someone doesn't want to invoke me, they don't accidentally do so.
<czajkowski> ah yes same here
<czajkowski> good to know
<czajkowski> anyway we got an answer yes they do need to sign the CoC for ppas and it should be enforced
<lifeless> czajkowski: thanks fo rletting me know about the discussion
<czajkowski> shall get the mins later and post to the list tomorrow
<lifeless> great
<salgado> sinzui, btw, I got excited and started removing the location-related (except timezone) from IPerson but then realized .latitude and .longitued are exported on the API.  do we need to follow a process to remove those or can we just assume nobody will care as those things have been removed from the UI anyway?
<czajkowski> lifeless: in fact before I head out, here you go http://ubottu.com/meetingology/logs/ubuntu-meeting/2012/ubuntu-meeting.2012-02-16-17.02.moin.txt and you can see the discussion http://ubottu.com/meetingology/logs/ubuntu-meeting/2012/ubuntu-meeting.2012-02-16-17.02.moin.txt
<lifeless> salgado: keep the attribute, return None for them, for old API versions
<lifeless> salgado: for devel, unexpect it entirely
<lifeless> czajkowski: I used my backlog :P
<salgado> lifeless, right, makes sense
<lifeless> s/unexpect/unexport/
<czajkowski> well in case others were curious :)
<sinzui> salgado, yes, what lifeless said.
<salgado> lifeless, what about stuff that was only exported on the beta API? I guess those can be removed altogether?
<lifeless> IMO yes
<lifeless> same as devel. YMMV though, I think some folk used beta heavily?
<barry> lifeless: hiya
<lifeless> barry: oh hey
<lifeless> sinzui: and syst3mw0rm: were talking about mailman archivers
<lifeless> and we got to a point where consult-barry was raised. I thought, why wait?
<barry> syst3mw0rm: hi, thanks for joining us here
<barry> lifeless: good thought :)
<syst3mw0rm> barry: no problem
<sinzui> barry, we were discussing ideals of archivers. We are using java-based Cassandra as the message store for Lp. I believe that is over kill for most mailman installs. A simpler store possibly taking the batteries-included approach would work
<lifeless> I was putting forward the opinion that apt-get install cassandra is pretty simple :>
<barry> right, anything java-based is overkill for standard mm installs :)
<barry> lifeless: certainly for lp, that would be fine.  mm3 though runs on lots of *nixen
<barry> sinzui: do you have apis designed for the backend store?  what are the basic requirements for that store?
<sinzui> barry yes
<sinzui> in, fact we have accidentally written a memory-based reference implementation for testing
<barry> sinzui: would it be possible to use sqlite + filesystem ?
<sinzui> I think it will be easy to play tests against another implementation or mock/fake to verify conformance
<barry> or should i say storm + possibly filesystem
<sinzui> barry, friend, man-of-my-mind, yes. I think mbox + sqlite will work for 80% of the cases
<barry> sinzui: that would be perfect, if by 'mbox' you mean http://docs.python.org/library/mailbox.html
<sinzui> barry, sqlite could also be a step to switch to drizzle, mysql, etc ...
<sinzui> barry, yes, I do mean that
<barry> sinzui: exactly.  and if done through storm, it's an easy-ish config var to change.  e.g. we have postgres support in mm3 bzr right now
<barry> sinzui: beauty.  that would let us use maildir for example
<sinzui> oh, yes. I forgot that
<barry> (maildir on disk format)
<barry> sinzui: btw, i *love* the idea of a backend store that does all the threading and whatnot, and vends the necessary info through rest for whatever formatter you want to use
<syst3mw0rm> barry: i guess you are talking about the grackle's idea ?
<sinzui> barry, I don't have any code yet, but I image the mbox is only used to store the attachments and is the portable-part of the archive. The db manages, indexes, threads, and stores JSON for fast returns os massage data
<barry> syst3mw0rm: yep
<salgado> lifeless, I can't seem to find how to tell exported() to do so only for certain versions of the API... any pointers? :)
<barry> sinzui: yep.  mbox would store the raw email messages too, which you might eventually want to have access to
<barry> sinzui: the nice thing is that the db would also store things like takedown flags.  and the really nice thing is that the consumer of the json can do things like email obfuscation algorithms, or body massaging (e.g. hyperlink bug tags, etc.)
<lifeless> salgado: something_for_version IIRC
<salgado> oh, a different thing, like operation_for_version I guess
 * salgado digs
<sinzui> barry, we are in agreement. we even intend to have the Cassandra version do the same.
<lifeless> salgado: oh oh, I know what you mean
<lifeless> salgado: its a dict at the end of the export
<salgado> oh, right
<lifeless> takes a mapping of versions and shit. Uhm, grep for '\<beta\>' and you'll likely find it quickly
<barry> sinzui, syst3mw0rm: this would be so cool.  the state-of-the-art in floss archivers has been abysmal for at least a decade.  this would leapfrog everything that i'm aware of
<syst3mw0rm> barry: seems cool..but frankly, it will take some time to digest totally whatever you were discussing...:)
<sinzui> barry, We will use Lp as a pass-through to do linking in real-time because of permission/data changes. The page code/js does not know it is talking to Lp...it can talk directly to the archive (a wsgi app) if you want
<barry> syst3mw0rm: no worries :)
<barry> sinzui: rock
<lifeless> sinzui: hmm, the js will need to be querying LP; I think yo umean 'LP will be offering the same API the archiver does'
<lifeless> sinzui: so they can be substituted transparently
<lifeless> (Do I understand correctly?)
<sinzui> lifeless, correct
<barry> lifeless: is there *anything* y'all can do to bump up the timeout on https://launchpad.net/ubuntu/+search?  it's so painful, but such a basic interface
<lifeless> barry: patches appreciated
<barry> :(
<lifeless> let me link you the bug
<lifeless> bug 816870
<_mup_> Bug #816870: Distribution:+search (package search) timeouts <critical-analysis> <timeout> <Launchpad itself:Triaged> < https://launchpad.net/bugs/816870 >
<lifeless> the count alone is 11 seconds to complete
<barry> yay!  4 hotter
<lifeless> the actual search will then be (probably but not always) less than that
<lifeless> we'd likely need 20seconds to have -any- chance of reliability
<barry> lifeless: jfdi!
<lifeless> and thats as high as we /can/ go before other components in the chain will time it out
<barry> i'm willing to wait :)
<lifeless> barry: when we raise such timeouts, we consume more resources, increases contention for service points -> more issues elsewhere
<lifeless> barry: this needs to be fixed, not tolerated. Thus my 'patches appreciated' comment.
<barry> lifeless: i know.  i'm just being curmudgeonly.
<lifeless> if we could just toss hardware at it, and tolerate the timeouts, it would be a different matter; but anyone who uses LP will be on master most of the time, and thats much harder to scale
<lifeless> and this is all DB server CPU time, so we only have 16 concurrent queries we can run
<lifeless> -> not many.
<lifeless> barry: I'd love it if you fixed it.
<barry> lifeless: no, i do appreciate the constraints
<wallyworld_> sinzui: jcsackett: wherefor art thee?
<sinzui> Fighting
<wallyworld_> unity?
<jcsackett> wallyworld: be there in just a moment.
<thumper> what?
 * thumper saw the word
<sinzui> I lost X for 30 minutes
<wallyworld_> thumper: unity is making me sad today
<StevenK> thumper: Do you hilight on unity or something?
<thumper> wallyworld_: why?
<thumper> StevenK: no, but quassel has a summary buffer
<wallyworld_> thumper: dash broken, random stuff left on screen
<thumper> which shows all chatter on all channels
<thumper> and I happened to see it
<sinzui> I am not sure unity is my problem. X went belly up and I had no harware detection for display, mouse, or keyboard
<wallyworld_> thumper: i just upgraded this morning. often when  stuff crashes and i go to report the bug, it says i am not running an official ubuntu package and so won't do it
<thumper> wallyworld_: which are you using?
<wallyworld_> thumper: let me check
<thumper> there have been X graphics driver issues
<thumper> visual corruption et al
<wallyworld_> thumper: 5.2.0+bzr1975ubuntu0+644.really1977
<thumper> wallyworld_: apt-cache policy unity
<wallyworld_> thumper: https://pastebin.canonical.com/60465/
<wallyworld_> thumper: the dash issue is that when i click on an icon to launch an app, nothing happens
<thumper> wallyworld_: hmm... that is the latest and greatest
<wgrant> wallyworld_: https://devpad.canonical.com/~lpqateam/dbr
<wallyworld_> thumper: yes, i like the bleeding edge :-)
<thumper> wallyworld_: so bleed
<wallyworld_> thumper: indeed. i was just mentioning it :-)
* jcsackett changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: - | Firefighting: - | Critical bugtasks: 4*10^2
<thumper> wallyworld_: I think I've found your bug
<wallyworld_> thumper: you're a legend
<thumper> wallyworld_: I didn't say I've fixed it :)
<wallyworld_> thumper: that's ok. finding it is the firs tstep :-)
<wallyworld_> thumper: and fewer others will be affected hopefully
<jcsackett> wallyworld_: ok, i've made my vote on the MP and added you directly as a requested look. enjoy. :-P
<wallyworld_> jcsackett: thanks, i think :-)
 * jcsackett grins
<jcsackett> wallyworld_: are you the reviewer today, or just planning on looking at that MP out of the goodness of your heart?
<jcsackett> wallyworld_: when you have a moment (say when your day fully begins), can you look at https://code.launchpad.net/~jcsackett/launchpad/trivially-add-sharing-ui-stuff/+merge/93501? it's much shorter than the other MP. :-)
<wallyworld_> jcsackett: i had already clicked on it but haven't read it yet :-)
<wgrant> wallyworld_: https://code.launchpad.net/~wgrant/launchpad/bug-933853/+merge/93508 is the branch performance fix I mentioned earlier.
<StevenK> http://pastebin.ubuntu.com/845139/
<StevenK> sinzui: http://pastebin.ubuntu.com/845139/
<wgrant> oops
<wgrant> lifeless: I was going to do a FF cleanup today anyway...
* wallyworld_ changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: wallyworld | Firefighting: - | Critical bugtasks: 4*10^2
<wgrant> baaah
<wgrant> work items landed
#launchpad-dev 2012-02-17
<wallyworld_> lifeless: there's a mp to remove person location stuff. just double checking that you've approved that?
<lifeless> I haven't reviewed it
<wallyworld_> lifeless: i don't mean a review, just that you are ok with the work item?
<lifeless> I'm aware of it; I wouldn't expect approval to be needed
<wallyworld_> i assume no one uses the location stuff then
<wgrant> Actually, someone generated images from it just a couple of weeks back.
<wgrant> But it's not possible to set it any more; I agree it should be removed.
<lifeless> maps were nuked way back
<lifeless> this is hangover; we need tz and should keep that
<wgrant> Damn. Missed buildbot by 90s.
<wallyworld_> ok. just wanted to check, thanks
<jcsackett> sinzui: it would appear changing my login email on kanban deleted all my cards.
<lifeless> \o/
<jcsackett> oh hurray! it just changed my icon to the same one wgrant uses, so my eye slid over them.
<wgrant> Oh, I wondered why I had so many cards.
<wgrant> Didn't think about that.
<jcsackett> yeah, i updated gravatar and now all is well.
<jcsackett> wgrant: we need to get you a gravatar acocunt, if only so i stop seeing bastardized power button symbols everywhere. :-P
<wgrant> Heh
<wgrant> jcsackett: btw, https://code.launchpad.net/~wgrant/launchpad/multipolicy-3 is the WIP branch for the new AccessPolicy model.
<wgrant> Should be pretty much stable by the end of today.
<jcsackett> wgrant: awesome! thanks.
<jcsackett> i look forwards to adding actual data to the sharing elements soon. :-)
<rick_h> StevenK: ping, have a sec?
<StevenK> rick_h: Hmmmmm?
<rick_h> StevenK: bit OT, I'm curious about the packaging of something
<rick_h> StevenK: I see the convoy stuff seems to have branches that are just the debian dir?
<StevenK> The packaging stuff we use is everything
<rick_h> ok, yea I couldn't figure out how to see ours
<rick_h> StevenK: so do you just have a packaging branch then with the debian dir?
<StevenK> rick_h: https://code.launchpad.net/~launchpad/+recipe/launchpad-convoy
<rick_h> and pull trunk, merge over, etc?
<StevenK> rick_h: That recipe builds into the LP PPA.
<StevenK> rick_h: Our packaging is in lp:~launchpad/convoy/trunk; but it contains everything
<rick_h> ok, I've got to hit up the recipe docs then. So does that recipe merge in the main convoy into ours then to build?
<StevenK> rick_h: The recipe checks out lp:convoy
<StevenK> rick_h: It then merges lp:~launchpad/convoy/trunk into it
<rick_h> ok, and the recipe updates changelog/etc then in some generic sense?
<StevenK> Right
<rick_h> ok
<StevenK> It generates a version, sets the message to Auto build
<rick_h> gotcha, ok makes sense
<rick_h> I'm tinkering with a pyinotify driven JS build script and trying to package up the js minification stuff out of tree
<rick_h> and wanted to see if I could figure out how to package it up in some way in ppa first
<StevenK> Heh, okay
<StevenK> Packaging is ... fun to learn
<rick_h> yea, I've tinkered and done a few things
<rick_h> but I've never used the recipe stuff and was surprised to see the landscape branches that were pure debian dir files
<rick_h> StevenK: any word on the RT?
<StevenK> Nope
<rick_h> all the test branches landed so starting to peek at actually looking at opening up, finding/fixing bugs, and optimizing
<StevenK> If we want a higher priority on it, we can ask lifeless to raise it at the meeting
<rick_h> thanks a ton for all the help with it
<rick_h> meh, I know there's some discussion up the chain started which is why I wondered if you'd heard anything
<StevenK> rick_h: I've gotten most of the pieces ready for webops when they want it
<rick_h> cool
<rick_h> now I'm tried of doing make jsbuild so want it to auto build :)
<StevenK> Two configs diffs are in my home directory on carob
<StevenK> rick_h: sinzui mentioned an issue with it too -- not all of our views are under LaunchpadView
<StevenK> rick_h: I'm looking at that today
<rick_h> ??
<rick_h> Ah, like the graph that was broken?
<rick_h> or don't use base-layout-macro?
<StevenK> rick_h: We added a property to LaunchpadView that constructs the combo loader URL.
<rick_h> ah
<StevenK> If the main page view doesn't have LaunchpadView as a parent, it will oops
 * StevenK hugs /usr/bin/sponge
 * wallyworld_ stabs X - hard to work with all this rubbish being left behind on the screen
<huwshimi> wallyworld_: Are you talking about the window shadows?
<wallyworld_> huwshimi: yeah
<huwshimi> wallyworld_: The compiz update that's supposed to fix it just came through
<huwshimi> I'm just about to restart to make sure
<wallyworld_> huwshimi: awesome! thanks! will look
<huwshimi> brb
<huwshimi> wallyworld_: All fixed for me :)
<wallyworld_> huwshimi: excellent. will update
<huwshimi> ugh, but something killed thunderbird :(
<wallyworld_> huwshimi: you mean it won't start?
<huwshimi> wallyworld_: It starts, I just get blank panels
<wallyworld_> noooooooooooooooooooo
<huwshimi> wallyworld_: Like this: http://ubuntuforums.org/attachment.php?attachmentid=212818&d=1329440555
<huwshimi> oh, you'll have to login to see that image
<wallyworld_> yeah
<wallyworld_> no login
<wallyworld_> oh well, might need to revert to previous tb version
<huwshimi> wallyworld: How do you revert a version these (post synaptic) days?
<wallyworld> huwshimi: install synaptic :-)
<wallyworld> that's what i used
<huwshimi> wallyworld: haha
<wallyworld> huwshimi: i wasn't joking :-)
<huwshimi> wallyworld: I know, it's just funny
<wallyworld> :-)
<mwhudson> command line is pretty easy too :)
<mwhudson> apt-get install mozilla-thunderbird=$VERSION
<mwhudson> err maybe ==
<huwshimi> mwhudson: How do I find out the previous version? :)
<wallyworld> i use synaptic to see what the previous versions are
<mwhudson> not sure :)
<huwshimi> mwhudson: It's ok :)
<StevenK> apt-cache policy
<lifeless> huwshimi: apt and dpkg log updates
<lifeless> huwshimi: in /var/log/
<huwshimi> lifeless: cheers
<wallyworld> StevenK: want a small +22/-6 review? https://code.launchpad.net/~wallyworld/launchpad/optional-extended-choice-popup-933743/+merge/93518
<lifeless> stub: oh hai
<stub> yo
<lifeless> encaffeinated?
<stub> lifeless: yup
<lifeless> shall we call/sykpe/hangout/mumble/voip/smoke signal/...
<lifeless> stub: ^
<stub> lifeless: whatever. skype is up, hangout virgin
<stub> Just try and ignore the hedge trimmer outside my window :-/
<wgrant> Hangouts should really take an s/out//
<wgrant> The plugin seems to be even shittier than Flash.
<StevenK> stub: I guess your stats patch needs an FDT?
<wgrant> No
<wgrant> It was live
<wgrant> So Monday is workitems.
<wgrant> Tuesday is my -0, Wednesday hopefully my -2
<StevenK> Ah, so I should request DB r11382?
<wgrant> Doesn't really matter.
<wgrant> But you could, yes.
<StevenK> Well, either I do it tonight, or you do your rev which includes it all on Monday
<wgrant> "it all" is the null set.
<wgrant> So it makes little difference :)
<StevenK> It includes another patch that isn't in devel
<StevenK> So it changes the merge into devel a little
<wgrant> True, but it affects nothing.
<StevenK> wgrant: How about a NDT of r14820?
<wgrant> There's nothing interesting.
<wgrant> Apart from your rev, but that's just one rev.
<StevenK> I was about to say :-)
<StevenK> Where is wallyworld's work ...
 * StevenK kicks buildbot
<wgrant> 10 hours away
<wgrant> ie. next week
<StevenK> The death of lp/contrib is interesting. To me.
<wgrant> The death of lp/contrib in this manner is a mistake.
<wgrant> I nearly reverted it.
<StevenK> Nearly?
<wgrant> I entirely object to the principle of it, but not quite enough that I will revert it.
<lifeless> stub: https://launchpad.net/launchpad-results
<StevenK> Is utilites/on-edge even useful anymore?
<wgrant> s/edge/qastaging/
<StevenK> wgrant: But it's disgusting
<StevenK> And probably mostly deprecated by the deployment reports.
<StevenK> wallyworld: How do you feel about a 470 line diff?
<wallyworld> StevenK: already on it
<wallyworld> StevenK: r=me. can we delete those old bug redirection views?
<StevenK> I'm not sure. I thought about it, and figured it was just easier and less risky to bring them into line.
<wallyworld> sure
<wallyworld> brad's comment on one of them seems to indicate it can be gone.
<StevenK> It's still mentioned in bug's ZCML
<wallyworld> i think that is also obsolete, but not sure
<wallyworld> since the redirection is done elsewhere
<StevenK> The fact that it deals with IBug:+index in the ZCML makes me worry that it isn't
<wallyworld> yeah, safest to leave as is i guess
<wallyworld> StevenK: after you land your branch, feel like a small +22/-6 review for me?
<StevenK> wallyworld: Sure.
<wallyworld> thanks!
<StevenK> https://code.launchpad.net/~wallyworld/launchpad/optional-extended-choice-popup-933743/+merge/93518 ?
<wallyworld> StevenK: yeah
<StevenK> That's +22/-10 :-P
 * wallyworld gets out a birch branch
 * StevenK deals wallyworld a penalty card for 'Lying'
 * wallyworld accepts the punishment
 * StevenK peers at the removal of the math imports
<StevenK> wallyworld: ^ That was lint cleanup?
<wallyworld> StevenK: yeah
<wallyworld> lint said they were not used
 * wgrant implements multirow and subselect inserts in Storm
<StevenK> wallyworld: r=me
<wallyworld> StevenK: thanks :-)
 * wallyworld lands
<StevenK> And utilities/qa-ready has a copy-and-paste job from on-edge
 * StevenK deletes them both for being useless
<wgrant> lifeless: I guess you're the Stormiest around. Could you glance at a small branch to vaguely implement bug #411556 and bug #61300?
<_mup_> Bug #411556: Storm should support multi-row inserts <Storm:New> < https://launchpad.net/bugs/411556 >
<_mup_> Bug #61300: Missing files in lout-doc <lout (Ubuntu):Fix Released> < https://launchpad.net/bugs/61300 >
<wgrant> Bug #613300
<_mup_> Bug #613300: Storm needs to support INSERT INTO ... SELECT <Storm:New> < https://launchpad.net/bugs/613300 >
<wgrant> stub: Thanks. Can you please apply it?
<stub> Yup. On it.
<StevenK> wallyworld: One more
<wgrant> stub: Thanks.
<stub> wgrant: Done except for _prod_2, which is blocked on the backups completing.
<StevenK> Which will take hours?
<wgrant> stub: Thanks.
<stub> Could be another hour or two
<wgrant> 3.5 hours, but yes.
<stub> seems to normally be finished before 3pm my time, which is the fdt window
<wgrant> nightly.sh completed Thu Feb 16 08:51:20 UTC 2012
<wgrant> nightly.sh completed Wed Feb 15 09:15:18 UTC 2012
<stub> Need to reschedule that if it is overrunning the fdt window
<StevenK> FDT is 1000UTC
<stub> oh right - changed that to avoid webops shift change
 * wgrant hopes that archive will now completely drop off the list
<wgrant> Bah. #6 -> #27 :(
<wgrant> Must be another evil query.
<lifeless> wgrant: did stub review it ?
<wgrant> lifeless: No, that was an index.
<wgrant> https://code.launchpad.net/~wgrant/storm/bulk-insert/+merge/93527
<wgrant> I'm probably missing something, but it seems to work so I might use it anyway.
<lifeless> the first hunk is confusing
<lifeless> I will write it in the review
<lifeless> you can get stub or jkakar or whoever is around to land; I'm well past EOW
<lifeless> (we're about to enter phase 2 of operation put-cynthia-to-sleep
<lifeless> )
<wgrant> Heh
<wgrant> Good luck
<wgrant> Thanks
<wgrant> Heh
<wgrant> I think abentley is trying to work around Storm's disability in another way
<wgrant> http://bazaar.launchpad.net/~abentley/storm/executemany
<wallyworld> StevenK: sorry was picking up kid from school and buying dinner food
<StevenK> wallyworld: Too late, self reviewed
<stub> wgrant: I can do the second review, but would be better if jkakar or jamesh or gustavo does the second review - someone who understands the internals and trade offs better (but to me, the patch looks good)
<wgrant> stub: Yep, that's why I didn't ask you.
<wgrant> Was waiting for one of the trinity.
<wgrant> yay
<wgrant> lolz
<wgrant> The main scriptactivity query is unindexed.
<wgrant> And bugbranch
<wgrant> That leaves BPPH as the only table with unexplained completely insane read volumes.
* wallyworld changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: - | Firefighting: - | Critical bugtasks: 4*10^2
<StevenK> Haha
<StevenK> wgrant: I think BPPH can be explained with one word: Publication
<stub> wgrant: I think I wanted scriptactivity trimmed so indexes would be unnecessary :-/
<stub> Seems to have become an audit trail though.
<wgrant> stub: Heh
<czajkowski> aloha
<mrevell> Hello
<czajkowski> morning is there a reason as to why people cannot remove their opengpg keys ?  a user wants to add a new one and delete their old one ?
<maxb> Well, there are some data integrity concerns, in terms of keys being used to track who uploaded packages
<czajkowski> maxb: thanks for your help
<salgado> I don't think we can QA bug 933592 (it's only about adding some model classes); should I just mark it as qa-untestable?
<wgrant> erk
<wgrant> That's illegal
<wgrant> Code changes aren't meant to land on db-devel.
<wgrant> salgado: ^^
<salgado> oh, perfect
<salgado> wgrant, how does it work? we need to get the schema changes deployed first and then land the code changes?
<wgrant> salgado: Right.
<salgado> shall we revert that landing, then?
<wgrant> Hm, didn't that DB patch already get deployed early in the week?
<wgrant> Last friday, in fact.
<wgrant> This branch should have landed on devel instead.
<wgrant> So it's not harmful, just illegal. You probably want to land on devel and pretend that the db-devel landing never happened.
<salgado> oh, could be. how can I tell when a db patch has been deployed?
<wgrant> https://wiki.canonical.com/InformationInfrastructure/OSA/LaunchpadProductionStatus logs deployments, and I normally comment in the linked bug.
<wgrant> But it looks like this time there was no bug.
<salgado> yep, there wasn't one for the schema changes
<wgrant> Anyway, it was deployed just over a week ago.
<wgrant> DB patches tend to be deployed within a couple of working days.
<salgado> ok, sorry about that. I'll create a new MP against devel and ask somebody to land it for me
<salgado> mabac, ^
<wgrant> Far easier for me to just pqm-submit it now :)
<mabac> salgado, hi there. just a sec while I catch up
<mabac> oh...
<salgado> wgrant, oh, that works too :)
<salgado> wgrant, although if there are schema-related changes the merge will fail, right?
<salgado> I mean, if one of our merges from db-devel into our branch included any changes in database/schema
<wgrant> Those checks are no longer enforced, but are generally done manually.
<mabac> wgrant, sorry about that. :) now I know what to do next time
<wgrant> But it only checks the delta between devel and your branch
<wgrant> So anything that's already in devel doesn't violate the check.
<salgado> mabac, yeah, just so you're aware of the policy, I didn't know about it either
<mabac> salgado, thanks. learning new stuff every day
<mabac> salgado, wgrant let's see. the entire schema change is included in the models branch. I didn't quite get if that's going to be a problem
<wgrant> mabac: It's not a problem, since I merged that into devel a week ago.
<mabac> wgrant, ok, makes sense since it didn't show up in the mp diff
<salgado> wgrant, right, but I merged db-devel into our branch a couple days ago and that brought in a bunch of new db patches.  I can't seem to find whether those patches have been deployed or not
<mabac> wgrant, scratch that. the diff was from db-devel of course
<wgrant> salgado: There's one unmerged DB patch ahead of your branch, but that was applied live. I'm merging that into devel separately now.
<wgrant> Then I'll merge yours in a sec.
<salgado> wgrant, ok, thanks a lot for taking care of this
<wgrant> I have three patches in the queue -- cleaning it up is entirely selfish :)
<salgado> heh :)
<mabac> wgrant, thanks
<wgrant> The reason we're able to do DB patches with so little downtime is that they are kept separated from code changes. If we let normal code changes also land to db-devel, the DB patches are no longer necessarily passing the test suite with the deployed code.
<wgrant> That's why this restriction is in place.
<wgrant> OK, submitted.
<salgado> wgrant, ah, makes sense
<wgrant> Landed on devel and bug marked untestable.
<salgado> yay!
<mabac> great! thank you
<wgrant> So, you can land any further branches on devel at your leisure.
<mabac> salgado, I have added some code according to your email: lp:~linaro-infrastructure/launchpad/workitems-widget mostly hunting for references to whiteboard and mimicking that.
<salgado> mabac, cool, are you going to be around a little longer? we can talk about that after I review your l-i-t branch?
<mabac> salgado, sure, I have a couple of hours left of today. thanks for picking up that review!
* bac changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: bac | Firefighting: - | Critical bugtasks: 4*10^2
<deryck> abentley, rick_h -- https://plus.google.com/hangouts/extras/talk.google.com/orange-standup
<salgado> on bug 933910 I've just deleted unused code. should I mark it qa-untestable?
<matsubara> bac, hey there, could you do a review for me?
<matsubara> bac, https://code.launchpad.net/~matsubara/lp-devops-dashboard/932768-lplib-credential/+merge/93207
<bac> matsubara: sure
<matsubara> thanks bac
<salgado> bac, hi there.  on bug 933910 I've just deleted unused code. should I mark it qa-untestable?
<_mup_> Bug #933910: Remove TranslatableMessage & co <qa-untestable> <tech-debt> <Launchpad itself:Fix Committed by salgado> < https://launchpad.net/bugs/933910 >
<bac> salgado: both of your reviewers mentioned some uneasiness with the branch pending a thorough QA.  so can you exercise translations functionality to see if anything breaks?
<salgado> oh, sure, I'll do that
<bac> matsubara: in the environment where the devops dashboard is run, does it have access to the Gnome keyring?  it looks like your update to download-cache got a new version of launchpadlib which now (and for some time, actually) prefers to store credentials in the gnome keyring.
<bac> matsubara: if you have strong preference to use your existing credentials file then your changes look good.  but i wonder why you cannot use the keyring.
<matsubara> bac, it's run on devpad
<matsubara> not sure if we have a gnome-keyring there. I wonder if is there a way to update the keyring with the existing credentials
<matsubara> it's run by a cronjob under the lpqateam user
<bac> matsubara: it it possible to just reauthenticate?
<matsubara> I think so. not sure which user that token belongs to but it should be easy to find out or create a new token
<bac> matsubara: actually, you're going to have a hard time given the restricted nature of devpad.  i think the code changes you've made are probably the right way to go.
<bac> matsubara-lunch: actually, you're going to have a hard time given the restricted nature of devpad.  i think the code changes you've made are probably the right way to go.
<bac> matsubara-lunch: i've approved the merge proposal but cannot mark it as such since i'm not in the QA team.
<james_w> would someone please push https://code.launchpad.net/~james-w/python-oops-datedir-repo/bson-compat/+merge/92560 through for us please?
<rick_h> james_w: pulling down
<james_w> \o/ rick_h
<rick_h> james_w: oh, this isn't lp
<rick_h> heh, need to read better
<bac> rick_h: are you doing the push for james_w?  if not, i will.
<rick_h> bac: working on it, tring to get the branches pulled/rnu tests on it
<bac> rick_h: oh, ok.  thanks.
<bac> just didn't want to duplicate effort
<rick_h> bac: thanks
<rick_h> bac: ok, can you push it, evidently I've got something wrong in my setup for this setup.py and it hates me
<rick_h> along with the buildout, and time sinking like no tomorrow
<bac> rick_h: ok.  will do so after lunch.  <-- james_w
<james_w> thanks
<rick_h> bac: ty
<sinzui> bac: will you have time to review a branch that fixes some stupid team links https://code.launchpad.net/~sinzui/launchpad/usless-team-links/+merge/93616
<rick_h> bac: if you get a second please: https://code.launchpad.net/~rharding/launchpad/fix_yui_934225/+merge/93622
<matsubara> bac, thanks!
<lifeless> flacoste: I think you underestimate the frequency, and depth, of confusion caused :)
<lifeless> flacoste: I seem to have put the cat amongst the pigeons with that thread; I think due to my comment in brackets; oops!
<rick_h> lifeless: think of the kittens you meany :)
<lifeless> every time you disable a feature flag, yaweh kills a kitten?
<rick_h> lifeless: I'm pulling out the jsmin stuff out of utilites into a separate package, is it enough for it to be a python package or would a .deb be the one true way?
<lifeless> rick_h: AFAICT we have to keep two deps systems - .deb and buildout - indefinitely.
<rick_h> lifeless: ok, so as long as I can get the python package on pypi, just a python package is good enogh?
<lifeless> rick_h: we can't do seamless upgrades of .deb's on appservers, so yeah a python package is fine
<lifeless> (by seamless upgrades I mean that when a deb version X is upgraded to Y, any appservers still consulting X are now out on a limb and not safe if they do late imports or try to restart;
<lifeless> this /shouldn't/ matter if the thing is api compatible, but its still a risk (and how often are things 100% compat)
<lifeless> rick_h: yes, pypi is good enough
<rick_h> lifeless: ok, thx. Sounds like a plan
<lifeless> rick_h: have you seen the CreatingNewProjects wiki page ?
<rick_h> lifeless: no, haven't looked
<rick_h> will now
<salgado> lifeless, hey there.  I didn't quite understand your suggestion for updating the work items... it's not clear to me whether you're saying it's fine to just update all objects normally and flush() at the end or something else?
<lifeless> salgado: you raised a concern about race conditions between sequence changes and other changes
<rick_h> lifeless: wrt the longpoll discussion, I've not peeked at it a ton, but would moving the longpoll to only pages that need it help at all? prevent all lp urls from making lp connections?
<lifeless> salgado: I was noting that using a flush() to well, flush, the other changes, and then doing sequence changes, would give you some safety
<rick_h> lifeless: from seeing the JS it looks like it's started on every page in base-layout-macros?
<lifeless> rick_h: well, that would probably reduce the impact, but consider that we'd like every page to be able to update in realtime...
<salgado> lifeless, did I? I was just pointing out that even if we manage to update all the sequences with a single update we could still end up issuing multiple updates for the status column
<rick_h> lifeless: yea, reaching for quick/dirty helpers
<lifeless> rick_h: I mean, if LP:
<lifeless>  - was fast
<lifeless>  - had a built in dashboard / notification list
<lifeless>  - easy to navigate
<lifeless>  - fast search
<lifeless> folk might open less tabs
<lifeless> -> virtuous circle
<lifeless> salgado: ah, well I read it wrongly then :P
<lifeless> not unheard of :>
<flacoste> rick_h: unless i'm mistaken, we actually only opens the connection on page that requires it now
<rick_h> lifeless: definitely agree, but since things a bug page doesn't use LP that I recall if the issue is 6 tabs and we can get it to 6 MP pages or what not might make a sig. impact on usability
<lifeless> rick_h: we wouldn't want it to be a timebomb though
<salgado> lifeless, ah, that explains things, then ;)
<lifeless> rick_h: consider that a bugs page that shows new comments is bug desired
<lifeless> rick_h: bug searches that update when someone adds/removes a bug to the resultset
<rick_h> flacoste: ah, yea I see in the comment that it checks for a lp.cache.longpoll, missed that
<lifeless> rick_h: longpoll is a gateway to a -really- wonderful UI
<rick_h> lifeless: ok so nvm, it's more prevelent than I've realized
<lifeless> rick_h: also remember the desire to move to one domain for the appservers
<lifeless> rick_h: if there is no code., then the impact would be less isolated
<lifeless> so, I don't think we need to panic; if flacoste can find resources to finish it, we can disable the flag until a scheduled rotation to do the work, and then we'll be all good.
<lifeless> I *really* want this working, but I don't want us paying interest we can avoid in the meantime.
<lifeless> flacoste: (did you see what I did there :P)
<rick_h> lifeless: yea, I can imagine 6tabs being a pita to work around
<lifeless> rick_h: the thing is that even wgrant, who a) knew about the issue, and b) isn't exactly clueless, got confused and spent 5-10 minutes asking webops questions about operational status, only to realise it was this foot-gun.
<flacoste> lifeless: first, let's agree on what finished means
<lifeless> flacoste: good point
<flacoste> lifeless: and if finished means it requires a 'rotation', then let's scrap this here and now
<flacoste> we finish this on maintenance or we scrap it
<lifeless> so, lets look at https://bugs.launchpad.net/launchpad/+bugs?field.tag=longpoll
<lifeless> 876325 is small but fiddly
<lifeless> 906482 is the key user facing thing, and wildcard polling domains are probably sufficient to address it
<lifeless> its not perfect (but I'm not arguing for perfect)
<lifeless> bug 850790 we may be able downgrade
<_mup_> Bug #850790: Enable dynamically allocated development services to communicate their configuration to other processes <longpoll> <Launchpad itself:Triaged> < https://launchpad.net/bugs/850790 >
<lifeless> bug 809139 just needs some examination of how rabbit reports its zomg's in its logs and either nagios or oops glue for that
<_mup_> Bug #809139: rabbit deployment lacking automated error gathering <longpoll> <Launchpad itself:Triaged> < https://launchpad.net/bugs/809139 >
<lifeless> bug 903186 looks shallow - a normal operation (race condition probably) generates an oops that we don't want, we should filter on the longpoll server
<_mup_> Bug #903186: Closed: Method(name=close, id=40) (404, "NOT_FOUND - no queue 'longpoll.subscribe.091adf46-9097-4d50-97bc-8d20e8c18417' in vhost 'launchpad.net'", 60, 20) content = None  <longpoll> <oops> <Launchpad itself:Triaged by redsquad> < https://launchpad.net/bugs/903186 >
<flacoste> lifeless: right, like i said 906482 is what was the killer
<lifeless> 902117 may be unactionable
<flacoste> at the time, Julian thought that implementing Bayeux was the solution
<flacoste> and that would have taken too much time
<lifeless> or it may be a precursor to 903186
<flacoste> (not to say useless in the end)
<lifeless> 901844 will be fiddly but isn't hard
<lifeless> <fin>
<lifeless> flacoste: so, I don't know if that needs a rotation or not
<flacoste> it doesn't
<lifeless> flacoste: it -may- have a long tail, but I don't see a lot of reason to expect one.
<flacoste> we don't have LP rotation available anyway
<flacoste> so it will have to be done on maintenance
<flacoste> and 4 bugs is definitively maintenance turf
<lifeless> flacoste: I missed some
<lifeless> https://bugs.launchpad.net/launchpad-project/+bugs?field.tag=longpoll
<lifeless> flacoste: nothing major though
<lifeless> flacoste: fiddly integrate-with-ops for the most part
<flacoste> yep
<lifeless> important to do before going live
 * lifeless is sssstubborn on that
<flacoste> agreed
<flacoste> agrees
<lifeless> flacoste: so, I'd still like to disable the flag for now
<lifeless> flacoste: because it is causing confusion and taking up time, even in our tiny pool of users
<lifeless> flacoste: (and, we should generally have the same experience of LP as our users)
<flacoste> lifeless: i'm +0 here really, my only concern was about the 'we need to schedule a squad to finish this' part :-)
<lifeless> heh :>
<flacoste> we could probably have a notification-service-beta team
<flacoste> for those who really want the feature, or develop it
<lifeless> that just adds permutations for debugging headaches
<flacoste> and are not confused or serial tabs opener ;-)
<lifeless> I'd be totally happy with the feature being back on for devs when the connections issue is resolved
<rick_h> bac and if you get another sec pls https://code.launchpad.net/~rharding/launchpad/standardize_js_934401/+merge/93637
<bac> rick_h:  ok
<bac> sinzui: working on yours now
<sinzui> fab
<deryck> flacoste, I'm -1 on disabling this. if only because tabs are the drug and lifeless is the junkie. I love him too much to see him drown in his own multitasking.
<deryck> :)
<flacoste> lol
<rick_h> dr deryck with the presciption of "quit doig that crap to yourself!" :)
<deryck> :)
 * sinzui waits for quote page update
<lifeless> deryck: lol :P
<lifeless> deryck: not that I'm not the one running into issues and getting confused :>
<deryck> lifeless, heh, I know. Just having some fun.
<bac> hi rick_h, why do you cp the yui2 stuff instead of creating a symlink as done for yui3?
<rick_h> bac: the yui3 one is a symlink from build/js/yui -> build/js/yui3.3.0
<rick_h> bac: the cp is a copy of what we're doing in bin/combo-rootdir
<bac> rick_h: ok, thanks
<rick_h> I'm reloading to see if the yui3.3.0 is a symlink or a copy, it might be able to be a symlink, not compared
<wgrant> abentley: Hi, https://code.launchpad.net/~wgrant/storm/bulk-insert may be of interest.
<abentley> wgrant: You're still using execute, so this looks complementary to executemany.
<wgrant> abentley: It does it in a single statement.
<wgrant> So executemany is not required.
<abentley> wgrant: In my tests, a single insert statement via execute took 3x as long as via executemany.
<wgrant> abentley: That's a bit of a worry.
<wgrant> Because executemany indeed just calls execute lots of times, so there's even more statement parsing overhead.
<abentley> wgrant: I can repeat my tests on monday, but at one point I had something much like your code and the branch scanner too 10 minutes.  Then I changed it to use executemany, and it took 3 minutes.
<wgrant> abentley: Did you use execute directly with a string, or did you use storm?
<abentley> wgrant: Directly with a string.
<wgrant> Huh
<abentley> wgrant: executemany is intended for bulk operations, so it doesn't surprise me that it's more efficient.  It does surprise me that you say it just repeatedly executes the statement.
<wgrant>     if (!PyArg_ParseTupleAndKeywords(args, kwargs, "OO", kwlist,
<wgrant>                                      &operation, &vars)) {
<wgrant>     while ((v = PyIter_Next(vars)) != NULL) {
<wgrant>         if (_psyco_curs_execute(self, operation, v, 0) == 0) {
<wgrant> It just parses the Python argument structures and then throws them into execute :/
<wgrant> Not preparing the statement or anything.
<wgrant> INSERT INTO ... VALUES is also intended for bulk operations, and should win because it's only parsed once... I shall attempt to reproduce your results.
<abentley> wgrant: The branch I was scanning was bzr+ssh://bazaar.launchpad.net/~irar/gcc-linaro/slp-for-any-nops-4.6/
<abentley> wgrant: I'm scanning it with ~abentley/launchpad/branchscanner-timeout-bug-808930
<wgrant> abentley: Oh, real code, even better. Thanks.
 * wgrant tries.
<wgrant> abentley: How big's that branch?
<wgrant> Huh
<wgrant> Do people really survive with <6 Launchpad tabs open?
<jelmer> what's the issue with having multiple tabs over
<jelmer> *open?
<wgrant> abentley: my way (ie. insert_many == store.execute(Insert(columns, table=[table], expr=rows))) looks to be maybe 10% faster, but I haven't tried on gcc-linaro yet.
<wgrant> jelmer: The current longpoll implementation means that each MP page (and hopefully eventually more than just MP pages) keeps a persistent HTTP connection
<jelmer> I'm in the lp team, but I haven't seen any issues with longpoll IIRC
<wgrant> Apparently it's exceptional to have lots of LP tabs open.
<wgrant> (since firefox limits concurrent connections to each hostname to 6)
<cody-somerville> hmm... I think my test run has hung on setting up something in the functional layer :(
<wgrant> abentley: Scanning lp:launchpad (deleting the branch and truncating branchrevision cascade each time) takes 2:15 with your insert_many, and 1:55 with Storm bulk Insert.
<wgrant> http://paste.ubuntu.com/846588/ is the diff
<wgrant> That Storm rev is just lp:~wgrant/storm/bulk-insert merged into the normal LP storm branch.
<abentley> wgrant: well, I'll certainly look into that on my next workday.  Which is actually Tuesday.
<wgrant> abentley: Oh, Canada has Monday off too?
<wgrant> Thought that was just the US.
<wgrant> k
<wgrant> The difference may be more pronounced on prod, where there's greater DB latency than a UNIX socket.
<abentley> wgrant: Yeah, Family Day.
<wgrant> ... intriguing.
<wgrant> (er, truncating revision, not branchrevision, obviously)
<abentley> wgrant: Well, Ontario has it, anyway.  This stuff varies by province.
<wgrant> Ah
<cody-somerville> \o/ yay Family day.
#launchpad-dev 2012-02-18
<cody-somerville> Although it's a provincial holiday so most of the people here in Ottawa who work for Federal government won't have it off, lol.
<cody-somerville> oops. so much for family day.
<cody-somerville> Does setting authorized_size to None disable the quota checking
<cody-somerville> ?
<cody-somerville> ah. It's not possible.
<cody-somerville> However, weirdly distribution archives still have the authorized_size attribute but it is None.
 * cody-somerville wishes running just a single doctest didn't take so long.
<wgrant> Heh
<cody-somerville> Actually, I guess it is possible.
<cody-somerville> but it'll probably cause the upload processor to crash
<wgrant> lpnet DB CPU usage has dropped by around 70% of a core in the last week, it appears.
<wgrant> cody-somerville: I expect that it's possible.
<wgrant> But the form might not allow it.
 * wgrant tries.
<cody-somerville> wgrant, it allows it
<wgrant> Ah
<wgrant> Maybe if a PPA has it set it will just crash
<wgrant> You're probably right.
 * wgrant checks the code.
<cody-somerville>         MEGA = 2 ** 20
<cody-somerville>         limit_size = self.archive.authorized_size * MEGA
<cody-somerville> ^^ that'll crash
<cody-somerville> You can't multiple None
<wgrant> Indeed. Is that under an if archive.is_ppa?
<wgrant>         if upload.is_ppa:
<wgrant>             self.checkArchiveSizeQuota(upload)
<wgrant> So yeah
<cody-somerville> It probably shouldn't do that, it should probably check
<cody-somerville> and the code fixed to see if authorized_size is None
<cody-somerville> holy crap
<wgrant> Yes, but that would make sense.
<wgrant> And you know our policy about Soyuz.
<cody-somerville> why does it take so long to do a commit?
<wgrant> If a test commits we have to tear down the DB.
<wgrant> Which takes a couple of hundred ms
<cody-somerville> No. bzr commit
<wgrant> If it doesn't commit, we can just abort the txn.
<wgrant> Ah
<cody-somerville> it's been going at it for like 5 minutes
<cody-somerville> repacking stuff
<wgrant> Ah, your repo hasn't been touched in a while?
<cody-somerville> I just branched it like yesterday
<wgrant> bzr repacks every order of magnitude of revisions, I believe.
<wgrant> Hm, it's possible we've hit a threshold, but I wouldn't think so.
<cody-somerville> It says 'Saving data locally - Stage: repacking chk:chk node n/n'
<cody-somerville> it's already repacked the inventory
<cody-somerville> and other stuff
<cody-somerville> I've got another 14k chk nodes to go, lol
<wgrant> Hm
<wgrant> There's about 118000 revs in devel
<wgrant> So I wouldn't expect a repack to happen around now.
<cody-somerville> ugh. now it's repacking texts and there is 400k of them, lol.
<wgrant> But maybe I misunderstand.
 * cody-somerville has a party to go to and really wants to get this branch up for review before he goes :(
<lifeless> wgrant: its OOM(base 10) size of pack, and every multiple of each OOM, of that size
<lifeless> so 118000 will be doing a 1K rev pack, probably, but that should be near-instant
<wgrant> Ah
<lifeless> also a brand spanking new repo should be under all the thresholds and not pack at all
<lifeless> however, 'branched yesterday' does not imply brand new repo
<lcanas> hi guys, I'm having a look at the launchpadlib to get data from the platform. I would like to get the bugs for a given project and I can't manage to find out how .. any clues?
#launchpad-dev 2012-02-19
<lifeless> wgrant: in support of heat updates being wonky:
<lifeless> https://bugs.launchpad.net/launchpad/+bug/816870
<_mup_> Bug #816870: Distribution:+search (package search) timeouts <critical-analysis> <timeout> <Launchpad itself:Triaged> < https://launchpad.net/bugs/816870 >
<lifeless> 36 heat, but AFAICT it should have 40
<lifeless> 2*6 * 6*4 + 2*2
<lifeless> 44 if the subs from dups are really honoured
<lifeless> wgrant: I just added a dup, and its off by 8 => 2 for the subscriber, 6 for it being a dupe -> I do think there is a glitch somewhere
<lifeless> or perhaps memcache
<poolie> hi all
<poolie> lifeless, if i was to put up a patch that re-added g+ (and maybe other) 'share' features, implemented just as a button linking to a url on the remote site, do you think that would be accepted?
<lifeless> that would have less (but still some) disclosure implications; it would have no security implications. It may have taste/design/usabilility aspects which I'm not the best person to address (such as it wouldn't show the counts -> wouldn't grow LP's capabilities at all).
<lifeless> the remaining disclosure implication would be the invitation to share private objects, definitely addressable.
<lifeless> poolie: speaking of patches
<lifeless> poolie: you've not removed the duplication in the show-timeline-to-developers code you landed
<lifeless> poolie: I'm feeling a little let down, as you committed to do so as part of landing it without delay
<poolie> i was just thinking about that too
<poolie> i will do it soon
<poolie> my january was a bit fuller than expected, what with two hospital trips
<poolie> i know we said 'january'
<lifeless> I understand, thank you
<lifeless> january FSVO january will be fine :)
<poolie> this is a good example of why not to make decisions based on planned future work :)
<lifeless> hah, indeed
<poolie> so i think it seems interesting enough that it's worth fixing the bugs and unifying the code
<poolie> the bugs being mostly cross-browser js things
<poolie> it got one or two upvotes
<lifeless> its nice
<lifeless> I wouldn't want it removed
<lifeless> I just don't want it to be costly in a costly area
<lifeless> flacoste: I'd kindof like to blog about our slack set-based approach; that ok with you? [given that you're orchestrating it, I don't want to steal your thunder...]
<wgrant> poolie: I fixed most of the bugs last week
<wgrant> poolie: Because I needed it
<wgrant> poolie: It works in Firefox now, toggles, and jumps to the right place when it shows.
<poolie> wgrant, oh hooray
<poolie> thanks very much!
<poolie> and i'm so glad you found it worth doing
<poolie> wgrant, so the only remaining significant thing is to unify the rendering with the oops code?
<wgrant> I think so
<wgrant> oops/profile
<wgrant> lifeless: Ah, duh
<wgrant> lifeless: The count at the top of the page is users affected including dupes.
<poolie> that's a feature :)
<poolie> possibly showing the details in a tooltip or something would be good though
<wgrant> That bug's users_affected_count == 4, so 2*6 + 4*4 + (2 + 2 on dupes)*2 == 36
<wgrant> lifeless: Not a regression, and technically correct by the docs, but we should possibly fix it.
<poolie> wgrant, ?
<wgrant> poolie: Hm?
<poolie> i wondered if you considered the inclusion of dupe-affected users a regression or something
<wgrant> Ah, no, it's just that it's displayed including dupes, but heat calculation uses the raw value without dupes.
<poolie> oh ok
<wgrant> Since heat calculation is now more transparent (because the hilarious aging stuff is gone), this has only become obvious recently.
<poolie> don't talk to me about heat :)
<wgrant> Last time someone talked to me about heat, I deleted 7/8 of it :)
<wgrant> wallyworld_: Morning
<wallyworld_> wgrant: hello
<wgrant> wallyworld_: Can you (no-)qa your picker thing?
<wallyworld_> ok, give me a few minutes
<wgrant> Thanks :)
<poolie> wgrant,  :)
<lifeless> wgrant: can you create an appropriate bug please
<lifeless> wgrant: I doubt this will be the last we hear of it
<lifeless> wgrant: we may want to keep the updateheat garbo job but change it to use a featureflag to determine the oldest date of heat which is *invalid*
<wgrant> Possibly, yeah.
<lifeless> wgrant: then we can just land new algorithm, set the flag to the right date, and watch it run
<lifeless> (and then stop automatically)
<wgrant> That sort of thing is probably going to be useful in several places.
<lifeless> yes
<wgrant> eg. the bug denorm tables
<wgrant> Once I work how HTF we can recalculate BugSummary.
<wgrant> lifeless: bug #936607
<_mup_> Bug #936607: Bug heat calculation's affected users count doesn't match the displayed value <bugs> <Launchpad itself:Triaged> < https://launchpad.net/bugs/936607 >
<lifeless> thanks; uhm, I might tweak the title for search
<wgrant> Search is hopeless anyway, but feel free :)
<lifeless> bug 936607
<_mup_> Bug #936607: Bug heat appears wrong due to affected users from duplicates not being included <bugs> <Launchpad itself:Triaged> < https://launchpad.net/bugs/936607 >
<lifeless> I figure folk will try bug heat wrong / bug heat broken etc; fingers crossed
<wgrant> Yeah
 * wallyworld_ takes car to mechanic - stupid fuel pump is broken :-(
<lifeless> wallyworld_: needs a software upgrade?
<wallyworld_> lifeless: i wish that's all it was. i used a rubber mallet to get it started
<wallyworld_> but that's impractical
<wallyworld_> hopefully won't cost toooo much
<lifeless> wgrant: so, sso detangle. You're all set on that ?
<wgrant> lifeless: Yeah. Hopefully make some progress on it soon.
<lifeless> wgrant: I think its more urgent than continued db efficiency in the medium term
<lifeless> wgrant: because it is a one off cost with a fairly long deploy tail
<wgrant> Yeah, but it's also a bit more work and harder to do in small chunks of downtime throughout my days.
<lifeless> yah
<lifeless> I hear a rumour you have mondays to self-direct
<wgrant> I've got stuff to finish off this week, but I expect so.
<lifeless> :P
#launchpad-dev 2013-02-11
<StevenK> wgrant: https://code.launchpad.net/~stevenk/launchpad/use-timeout-for-checkwatches/+merge/147586
<wgrant> StevenK: Have you tested locally that it actually works?
<StevenK> Haha
<StevenK> You tell funny jokes
<StevenK> Now I have to remember how to do that
<StevenK> wgrant: 2013-02-11 01:48:53 INFO    Updating 1 watches for 1 bugs on http://bugzilla.abisource.com
<wgrant> StevenK: But does the timeout work?
<StevenK> wgrant: I've also discovered mantis and trac also call their own urllib2.open, so I've changed them too
<StevenK> I'll check the timeout itself after lunch
<wgrant> Great
<StevenK> 2013-02-11 02:19:13 INFO    Error updating http://bugzilla.abisource.com/: http://bugzilla.abisource.com: <urlopen error timed out>
<StevenK> 2013-02-11 02:23:26 INFO    Updating 1 watches for 1 bugs on http://tracker.ardour.org
<StevenK> 2013-02-11 02:23:56 INFO    Error updating http://tracker.ardour.org: http://tracker.ardour.org: <urlopen error timed out>
<StevenK> (ardour is Mantis)
<StevenK> wgrant: Are you happy enough with that?
<StevenK> That's XMLRPC and Mantis
<StevenK> I can dig for Trac and old Bugzilla if you wish
<wgrant> StevenK: Seems fine to me
<wgrant> No further changes?
<StevenK> Let me push them up
<StevenK> I'm tempted to refactor Mantis and Trac so there is one callsite for urlopen in base
<wgrant> That seems like a good idea, and it shouldn't be too hard
<StevenK> Current idea: Set a self.url_opener on both Mantis and Trac and teach urlopen in the base class to use it if it's set
<StevenK> For bonus points, set a variable to the function and call it as a function
<StevenK> wgrant: The MP is updated
<wgrant> StevenK: Can you just default url_opener to urllib2.urlopen?
<StevenK> How does that help? I can't call .open on that?
<wgrant> Ah, true.
<wgrant> r=me
<wgrant> StevenK: You lose!
<StevenK> Blink
<StevenK> I hate the checkwatches tests
<StevenK> -        def raise404(request, data):
<StevenK> +        def raise404(request, data, timeout=None):
 * StevenK peers at these branch vocab tests
<StevenK> There is a test to make sure a vocab doesn't return branches owned by a OPEN or DELEGATED team
<wgrant> StevenK: That sounds like the series branch picker
<StevenK> Which is on product
<StevenK> Since that deals with IProduct or IProductSeries
<StevenK> Right, thanks to some hideous code, the branch vocab tests pass
<StevenK> Except my last lot of refactoring broke them
<StevenK> wgrant: OMG, these changes work
<wgrant> :)
<StevenK>  6 files changed, 196 insertions(+), 199 deletions(-)
 * StevenK accidently deletes IBranchCollection.search
<StevenK>  6 files changed, 197 insertions(+), 445 deletions(-)
<adeuring> good morning
<wgrant> StevenK: Oh
<wgrant> StevenK: You still lose at buildbot
<wgrant> StevenK: Look at those test counts
<wgrant> There's one very long error which apparently breaks the subunit stream
<wgrant> lp.bugs.tests.test_bugtracker.TestMantis and checkwatches.txt, from my ec2 run
<StevenK> Bleh
<StevenK> I did wonder why the test count was quite low
<StevenK> wgrant: So I managed to destroy another ec2 run of yours?
<wgrant> StevenK: Yup
<wgrant> And I've only run two batches of them in the last month...
<wgrant> you sabotaged both!
 * StevenK adds another stroke to his "wgrant's ec2 runs I've doomed by accident" board
<wgrant> Lies
<StevenK> Probably shouldn't be too bad to sort out
<StevenK> wgrant: Testfix landing.
<wgrant> StevenK: Thanks
#launchpad-dev 2013-02-12
<StevenK> Failed tests:      1
<StevenK> Bwahaha
<StevenK> And that's only because I intentionally broke the interface
<StevenK> wgrant: Those Connection is closed errors were caused by my cleanup to the BranchCollection.store property
<wgrant> Ah
<StevenK> I'm not sure why, but I've reverted them and they'll stay that way
<wgrant> StevenK: https://launchpad.net/~tonyyarusso/+archive/ppa/+packages?field.name_filter=&field.status_filter=published&field.series_filter=edgy
<wgrant> Spot any problems there?
<wgrant> Only 3 PPAs, one copy archive, and the Ubuntu primary archive seem to be afflicted
<StevenK> Hahaha
<wgrant> Poor Dapper
<wgrant> Some 38000 of its files are missing
<StevenK> Didn't we inject the missing ones?
<StevenK> I can recall a script
<wgrant> I don't think we ever ran it
<wgrant> We ended up hacking p-d-r instead
<wgrant> We ran into this when we obsoleted dapper
<StevenK> Yeah
<wgrant> And it failed to remove the files because they didn't exist in the DB any more
<StevenK> Right
<StevenK> So they should be gone anyway,
<wgrant> Yeah
<wgrant> But there are 9 binaries that built in dapper in the primary archive, are still published somewhere else, and are deleted
<StevenK> Now I remember why I don't like hearing about archive consistency
<StevenK> It makes me want to curl up under my desk and cry
<wgrant> Apart from the copy archive the numbers seem more manageable, so I might track down all PENDING/PUBLISHED pubs with missing files and get them revived.
<StevenK> Ignoring the 38,000 for Dapper?
<StevenK> Not sure what state Dapper's publications are in
<wgrant> Those are obsolete
<StevenK> Aside from 'disarray'
<wgrant> They should technically still exist, but they don't really matter and would have be recovered from old-releases
<wgrant> The only things that will break are those that are still publishe
<wgrant> d
<StevenK> So the new GC can remove them again?
<wgrant> Well, not yet removed
<wgrant> No
<wgrant> Anything that is obsolete is meant to stay forever, by current policy
<StevenK> Sure
<wgrant> We only remove *pre-release* binaries.
<StevenK> Oh, 38,000 of Dapper's released binaries are missing
<StevenK> Handy
<wgrant> Yes
<wgrant> Only 9 of those are still published somewhere
<StevenK> See previous comment about under my desk and sobbing
<wgrant> Ah, no, 4 PPAs
<StevenK>     raise ValidationFailed("directories differ")
<StevenK> ValidationFailed: directories differ
<StevenK> wgrant: ^ Seen that before?
<wgrant> StevenK: I have no context
<StevenK> wgrant: http://pastebin.ubuntu.com/1638432/
<wgrant> StevenK: cscvs, run away
<StevenK> Haha
<wgrant> More than likely unrelated to your changes
<StevenK> I'll continue to ignore it
<StevenK> Fighting with horrible doctest changes
<StevenK>  12 files changed, 212 insertions(+), 618 deletions(-)
<StevenK> Oh, ugh
<StevenK> These tests rely on searching on branch.url being sane
<StevenK> Which I broke because branch.url is only used for mirrored branches and horrible
<StevenK> Haha
<StevenK> I mention mirrored branches and mwhudson joins
<wgrant> Yay, electricity.
<StevenK> Haha
<StevenK> OH
<wgrant> StevenK: How's it going?
<StevenK> One failure
<StevenK> I broke HostedBranchRestrictedOnOwnerVocabulary, and I was contemplating deleting it, but productseries requires it
<StevenK> (IProductSeries.translations_branch, that is)
<wgrant> Hmm
<wgrant> 66 BPRs are affected
<wgrant> 6 of those are from intrepid, so we'll need to restore dists/intrepid temporarily
<StevenK> Hmm, I think my query is broken
<wgrant> StevenK: Howso?
<StevenK> Because the test breaks :-)
<StevenK> Pretty good indication
<wgrant> Heh
<StevenK>     + ~a-branching-user/product-two/hosted
<StevenK>       ~a-branching-user/product-two/hosted
<StevenK>       ~a-team/product-two/another_hosted
<StevenK> That's my slightly fixed query, but it's still broken
<StevenK> wgrant: http://pastebin.ubuntu.com/1638511/
<wgrant> StevenK: That'll return the branch twice if the user owns it directlyt
<StevenK> Which is exactly what I'm seeing, yeah
<wgrant> And it will also not do what you want if the user has no TPs (though that never happens, due to the self participation)
<StevenK> Right, so the Branch.ownerID == self.user.id was not needed
<StevenK> wgrant: So I think this crap actually works, shall I push it up?
<wgrant> StevenK: Might as well
<StevenK> My smoke test revealed another failure
<StevenK> wgrant: https://code.launchpad.net/~stevenk/launchpad/new-branch-search/+merge/147840
<StevenK> But it can wait until tomorrow if you want to do necromancy instead
<wgrant> StevenK: waaaat
<wgrant> StevenK: What's this def search_branches stuff?
<wgrant> What was wrong with having it on BranchCollection with every other branch search method?
<StevenK> wgrant: The BranchCollection stuff makes my brain drip out of my ears
<StevenK> IAllBranches and related friends
<wgrant> That doesn't mean you should just create a separate parallel and broken in slightly different ways implementation :)
<StevenK> How is search_branches broken?
<wgrant> It's code
<wgrant> => it has bugs
<wgrant> BranchCollection is well-tested
<wgrant> search_branches is not
<wgrant> And there's no reason for divergence AFAICS
<StevenK> wgrant: Ah, but you did say I should ignore the current stuff and re-implement
<wgrant> The current *algorithm* :)
<wgrant> The problem is the algorithm, not BranchCollection
<StevenK> Right, and I wasn't sure I could even delete IBranchCollection.search() until two hours ago
<StevenK> Which meant I was in a position to have both
<StevenK> If I needed
<adeuring> good morning
<cjwatson> Hm.  I may need to fix bug 604427 before I can test my bpph-phase branch properly
<cjwatson> (Because I need to change overrides for ddebs, and I can't do that right now ...)
<wgrant> cjwatson: Why can't you?
<cjwatson> changeOverride refuses because the archive would change
<cjwatson> i.e. "OverrideError: Overriding component to 'universe' failed because it would require a new archive."
<wgrant> Ah
<wgrant> That's probably just a missing bit in changeOverride
<cjwatson> Which is arguably a bug for ddebs
<wgrant> If it's a ddeb it should go through .debug_archive
<wgrant> As it does in IIRC publishBinaries
<wgrant> Two lines missing :)
<cjwatson> Ah, yeah, OK
<wgrant> I may be way off, it's been a while
<cjwatson> Separate branch or should I roll it in?
 * wgrant checks the code
<cjwatson> Thing is, I suspect phased-update-percentage will leave around a lot of ddebs because there's no really convenient way to override them
<wgrant> Well
<wgrant> That's not a problem
<wgrant> Because there are no ddeb publications for the primary archive today
<cjwatson> Well, indeed, but when there are it's a bit of a timebomb
<wgrant> There are about 5 other bugs that are similarly bad and filed about that.
<wgrant> I'd put the change in the same branch
<cjwatson> But I guess we need to fix basically all of https://bugs.launchpad.net/launchpad/+bugs?field.searchtext=ddeb before we can enable ddebs in primary
<wgrant> Exactly.
<cjwatson> OK, I'll amend bpph-phase accordingly, thanks
<wgrant> StevenK: Have you tried these searches?
<StevenK> wgrant: Which searches?
<StevenK> Shall I cowboy this onto DF?
<wgrant> How effective is the new search algorithm?
<wgrant> You could.
<StevenK> wgrant: Pending merge from you on DF, I'm wary of conflicts.
<wgrant> I wonder if the "term.startswith('~') => look up by exact unique_name match" rule should be replaced with "'/' in term => look up by unique_name prefix match"
<wgrant> And the name match could perhaps restrict to the relevant pillar
<wgrant> StevenK: You can revert that
<StevenK> Not sure how to revert just that pending merge
<wgrant> No need
<wgrant> bzr revert
<StevenK> Right
<wgrant> Revert the whole thing
<wgrant> If you want JS then unshelve and update the revno in that diff
<wgrant> So, the idea behind my suggestions is that normally you want to search in the relevant context
<StevenK> Probably do, but updating DF first
<wgrant> But you need to be able to override
<wgrant> So if you specify a full path it will still work
<wgrant> But searching for just a name fragment will only return stuff from your projet
<StevenK> Depends on the vocab used
<wgrant> Yes, similar to the person picker rework
<StevenK> If just BranchVocabulary is used, then it will match everything
<wgrant> Right
<wgrant> But we can do the same thing for the bugtask branch picker that we did for the bugtask person picker
<wgrant> So it knows which pillars are relevant
<wgrant> Should be fairly easy
<StevenK> So I'm not done? :-(
<wgrant> And if the context is a branch (eg. for +register-merge) then it should probably use the branch's pillar
<wgrant> You're knee-deep in branch search, you might as well make it not completely terrible given we already have the code
<StevenK> The RestrictedOnBranch vocab already does that
<wgrant> Ah, good
<StevenK> Er, RestrictedOnProduct
<wgrant> So mostly just bugtask, and maybe the series branch picker
<wgrant> Though the latter may already be done
<wgrant> But in general there's probably almost no need for a generic branch vocab
<wgrant> Unlike people, branches are well-scoped
<StevenK> vocabulary='BranchRestrictedOnProduct',
<StevenK> For IProductSeries.branch
<wgrant> Great
<wgrant> So maybe only the bugtask one needs tweaking
<wgrant> Is there a trigram index on Branch.name yet?
<wgrant> If not, we'll need one
<StevenK> No, I was going to do that after
<StevenK> Since the index can hit prod before a deployment
<wgrant> I suspect we'd be somewhat better off doing a sort of FTI
<wgrant> eg. splitting on _- etc.
<wgrant> But substring will do for now
<StevenK> Shall I change IBugBranch.branch to BranchRestrictedOnProduct ?
<wgrant> StevenK: No
<wgrant> StevenK: Because a bug doesn't have a single product
<wgrant> You should be able to look at the bugtask subscriber vocab, I think
<wgrant> To see how to handle multiple products
<wgrant> At least I think it does
<wgrant> Also, all this should be about pillars
<wgrant> Not products
<wgrant> Or maybe even products and sourcepackages, rather than pillars
#launchpad-dev 2013-02-13
<StevenK> The bugtask subscriber vocab is built in place
<StevenK> wgrant: mawson is updated, my patch cowboyed and JS unshelved, but it only renders timeouts
<StevenK> Ah, /auditor finally works
<lifeless> zomg'
<StevenK> lifeless: Ah, the auditor project page on DF
<lifeless> oh
<StevenK> wgrant: Right, so the exact matches are great for DF
<StevenK> Very fast
<wgrant> StevenK: I'd certainly hope so!
<StevenK> Maybe we want to restrict the MP vocab
<wgrant> One thing I thought of is that you probably want to do a case-insensitive match
<wgrant> On Branch.name
<StevenK> wgrant: So searching for 'devel' on proposing a LP branch returns too many matches
<StevenK> But I'd honestly expect to usually paste in a URL
<wgrant> StevenK: You're conditioned to paste a URL because search is so useless today
<wgrant> It usually times out, and when it doesn't time out it doesn't return useful results unless you're very accurate
<StevenK> Maybe we want to search on unique_name too
<StevenK> But I discounted that
<wgrant> Prefix search on unique_name makes sense if there's a / in the term
<wgrant> Otherwise I don't think it's valuable
<StevenK> wgrant: Thoughts on using BranchRestrictedOnProduct for BMP's vocab?
<wgrant> StevenK: Remember that BMPs work for more than products
<StevenK> wgrant: Prefix search? Then searching for launchpad/devel won't work
<wgrant> StevenK: That's true.
<wgrant> But mmm
<StevenK> contains_string is probably too heavy handed for unique_name
<wgrant> Yes, but it's all we have without schema changes
<wgrant> And I don't think it's going to be toooo long for a trigram to be effective
<wgrant> But you should try.
<StevenK> You mean like http://pastebin.ubuntu.com/1641700/ :-)
<wgrant> StevenK: Right, that sort of thing
<wgrant> Ideally a unique_name match would be component-wise rather than character-wise, but we can't really do that atm
<StevenK> Right
<StevenK> Let me patch that onto DF
<StevenK> wgrant: Oh, shall we TGRM up name and unique_name ?
<wgrant> StevenK: Try with and without
<wgrant> Max length of unique_name today is 141
<wgrant> Which isn't great, but not fatal
<StevenK> What's that one?
<wgrant>  ~openerp-commiter/openobject-addons/trunk-configuration_wizard_improvement-atp-improve-general-apply-button-color-in-configuration-wizard-pja |         141
<StevenK> That's digusting
<StevenK> *disgusting
<wgrant> Then
<wgrant>  ~78luphr0rnk2nuqimstywepozxn9kl19tqh0tx66b5dki1xxsh5m-launchpad-a811i2i3ytqlsztthjth0svbccw8inm65tmkqp9sarr553jq53in/syncany/delimiters       |         135
<wgrant>  ~78luphr0rnk2nuqimstywepozxn9kl19tqh0tx66b5dki1xxs-phn5hho65-a811i2i3ytqlsztthjth0svbccw8inm65tmkqp9sarr553jq5/sysbench/sysbench-pg           |         131
<wgrant> Then more openerp
<StevenK> I can't complain about the 135 or 131, those are effectively our fault
<wgrant> Yep
<wgrant> https://code.launchpad.net/~chipaca/ubuntuone-client/dobey-needs-a-nice-self-contained-dbus-signal-to-let-the-applet-know-when-content-queue-changes
<StevenK> Apparently, the searches are all ~ 2 seconds
<wgrant> ie. awful
<wgrant> get a plan
<StevenK> wgrant: http://pastebin.ubuntu.com/1641721/
<wgrant> StevenK: Heh
<wgrant> StevenK: FYI I prefer to name cowboyed indices and tables temp_* so they can be easily found and removed
<StevenK> It's still 440 ms with the TRGM
<StevenK> wgrant: Would gist rather than gin help?
<wgrant> StevenK: No
<wgrant> GiST vs GIN is mostly a write vs read tradeoff
<wgrant> GIN is slower at writes, but faster at reads
<wgrant> So we want GIN
<StevenK> But then we're stuffed even with the index
<wgrant> Not if it's a relatively consistent 4s
<wgrant> It's not create
<wgrant> great
<wgrant> But it's better than 2s consistent.
<wgrant> And will be faster on prod
<wgrant> StevenK: Is this unique_name or name?
<StevenK> The former
<wgrant> Which is rare
<wgrant> though most of the length will be in name too, see how it performs
<StevenK> That index is just creating now
<StevenK> TRGM for name is 300ms
<wgrant> Reasonable.
<StevenK> I was expecting more on the order of 100
<StevenK> Not 300 or 400
<wgrant> Not with a long large GIN index
<StevenK> What's the \d magic to see the index size?
<wgrant> \di+
<StevenK> Holy crap, they're 200MiB all up
<StevenK> wgrant: So, push the changes for this branch, and create a new branch for two shiny TRGMs?
<wgrant> I think so
 * StevenK stabs the branch scanner and reaches for carob
<StevenK> wgrant: https://code.launchpad.net/~stevenk/launchpad/branch-trgm-indicies/+merge/148082
<StevenK> wgrant: And new-branch-search is updated
<adeuring> good morning
#launchpad-dev 2013-02-14
<StevenK> wgrant: http://pastebin.ubuntu.com/1647128/ is that what you meant?
<wgrant> StevenK: Yeah
<StevenK> wgrant: https://code.launchpad.net/~stevenk/launchpad/new-branch-search/+merge/147840 is updated
<wgrant> cjwatson: Not going to take care of soyuz.derived_series_sync.enabled too?
<cjwatson> I tend to do one flag at a time or it gets too tangled for my poor brain, esp on three hours of sleep
<cjwatson> But I guess if you like.  When was it enabled everywhere?
<cjwatson> And same question for soyuz.derived_series_upgrade.enabled
<wgrant> derived_series_upgrade isn't enabled anywhere, I don't think
<wgrant> cjwatson: soyuz.derived_series_sync.enabled was enabled on 2011/08/16, but I don't think derived_series_upgrade ever was
<wgrant> So derived_series_sync can probably be turned on unconditionally
<cjwatson> Yeah, was just digging that out
<cjwatson> Agreed
<cjwatson> Now I remember the QA saga that that was
<cjwatson> And https://lists.ubuntu.com/archives/ubuntu-devel-announce/2011-July/000877.html
<cjwatson> wgrant: Done
<wgrant> cjwatson: Thanks.
 * wgrant goes on a review sptree
 * StevenK waits for his review to bubble to the top
<StevenK> wgrant: The field needs to be lowercased too.
<StevenK> wgrant: Uh?
<StevenK> wgrant: field is just Branch.name or Branch.unique_name
<StevenK> wgrant: I dropped the CountableIterator since the only real callsite of search is the branch vocabs, and they wrap what is returned in a CountableIterator
<wgrant> StevenK: The field needs to be lowercased
<wgrant> unique_name and name are not lowercase implicitly
<wgrant> So matching a lowercase value against them will make some things impossible to find
<wgrant> (and it won't use the index)
<StevenK> Oh, I can't use field.contains_string ?
<wgrant> You might be able to use field.lower().contains_string
<StevenK> Yeah, that looks good
<StevenK> wgrant: Right, everything addressed with the exception of CountableIterator, since I'm ignoring that
<wgrant> You addressed that.
<StevenK> I meant addressed in terms of code changes
<StevenK> wgrant: http://pastebin.ubuntu.com/1647698/
<wgrant> StevenK: s/http/http:/? And what happens if the URI fails to parse?
<wgrant> eg. http:hahahahhaveanoops
<StevenK> wgrant: No, I want to deal with http://bazaar.launchpad and https://code.launchpad
<StevenK> I could use or
<wgrant> Ah, right
<wgrant> Still need to handle URI parse failures
<StevenK> wgrant: I should catch the InvalidURIError and return [] ?
<StevenK> None, I guess, since only search calls _getExactMatch
<wgrant> Yeahj
<StevenK> Which means we'll try and match it against name, but eh
<StevenK> wgrant: http://pastebin.ubuntu.com/1647727/
<wgrant> StevenK: Oh, bzr+ssh too maybe?
<wgrant> Maybe just try to parse the whole thing as a URI
<wgrant> And extract the path
<wgrant> Ignoring scheme
<StevenK> I hadn't considered bzr+ssh
<wgrant> Though lp: doesn't have a host, so it's a bit special
<StevenK> wgrant: So I can reflow _getExactMatch to do lp: or ~ and then just parse the whole thing
<wgrant> Actually, lp:foo passes to foo as the path
<wgrant> So no special handling required
<StevenK> Except that getByUniqueName won't work
<wgrant> Ah, indeed
<StevenK> wgrant: http://pastebin.ubuntu.com/1647748/
<StevenK> wgrant: I can write a bzr+ssh test if you wish
<wgrant> StevenK: Oh
<wgrant> If you're just going to look up by URL anyway then why do you need the lp: special acse?
<wgrant> If you remove that the first thing it will do is try it as a URL anyway...
<StevenK> Excellent point
<wgrant> And if it starts with ~ then the URL lookup will fail
<StevenK> wgrant: http://pastebin.ubuntu.com/1647760/
<wgrant> So it will try to find it as a unique name...
<StevenK> Ah, but it will try and parse it as a URI
<StevenK> When it isn't
<wgrant> And?
<wgrant> +            path = URI(term).path[1:]
<wgrant> is probably wrong
<wgrant> Might want .strip('/') instead
<wgrant> Rather than just dropping the first character
<wgrant> (which will fail for lp:foo)
<StevenK> lp:foo won't get there
<StevenK> That's handled by the .getByUrl call
<wgrant> True
<wgrant> But still
<wgrant> [1:] is evil and pointless
<wgrant> And easily avoidable :)
<StevenK> URI('~launchpad-pqm/launchpad/devel')
<StevenK> == InvalidURIError
<wgrant> Sure
<wgrant> I
<wgrant> 'd hope so :)
<StevenK> Right, so the term.startswith('~') has to stay then
<wgrant> That's a uniquename
<wgrant> Oh
<wgrant> But failing to parse as a URI short-circuits out now
<wgrant> That's probably wrong
<StevenK> except InvalidURI can do path = term if you wish
<wgrant> Exactly
<StevenK> wgrant: Now that you've made me rewrite _getExactMatch :-) http://pastebin.ubuntu.com/1647772/
<wgrant> StevenK: Sounds reasonable
<StevenK> wgrant: So, bzr+ssh search test, or commit and land it and its index friend?
<wgrant> Perhaps the latter
<StevenK> wgrant: Okay, now that I'm off the phone, I'll land them both
<wgrant> Great
<wgrant> StevenK: You've tested the latest code on DF?
<wgrant> StevenK: To confirm that it uses the indices?
<StevenK> Spoil my fun
<StevenK> wgrant: Indices recreated, code updated
<StevenK> 0.8 seconds
<StevenK> For a search of PAD/DEVEL
<wgrant> The plan is the more interesting bit
<StevenK>                Index Cond: (lower(unique_name) ~~ '%pad/devel%'::text)
<StevenK>  Total runtime: 152.058 ms
<wgrant> Right
<wgrant> Seems sensible
<StevenK> And faster
 * StevenK sadfaces at buildbot
<adeuring> good morning
#launchpad-dev 2013-02-15
<StevenK> wgrant: http://pastebin.ubuntu.com/1654509/
<wgrant> StevenK: Mmm
<wgrant> StevenK: That's a bit of a regression
<wgrant> I wonder if LimitedView might be better reserved for the "you can only see this archive because a build was copied out of it into a public archive" case
<wgrant> Which might mean that an intermediate permission is required
<StevenK> wgrant: Well, we can override getBuildRecords on IArchive to return [] if the requestor is a subscriber and it's a private archive
<wgrant> StevenK: Ew
<wgrant> A proper solution would not involve hacks like that, and would remove the +packages hack that is in place today
<StevenK> Yes, which I did, and you say it's a regression
<wgrant> StevenK: The regression is that +index no longer works
<StevenK> I didn't know the baselayot template will only show the not shared with you string if you hold LimitedView
<wgrant> Sure
<wgrant> It's still a regression
<wgrant> Not realising that something was going to break does not make it not broken :)
<StevenK> Sure. But what can I do instead?
<wgrant> Use a non-LimitedView permission, I guess
<StevenK> launchpad.NotQuiteView doesn't help either
<StevenK> Do we have a list of them?
<wgrant> StevenK: lib/lp/permissions.zcml
<StevenK> Hmmmm
<StevenK> None of them are a good fit
<StevenK> wgrant: Hmmmm, where is view/macro:is-page-contentless defined? My quick grep didn't turn up much
<StevenK> Bah
<StevenK> It's in app/browser/tales
<StevenK> And it's as I feared, it checks for !View
<wgrant> Luckily it is code
<wgrant> You could alternatively reassign View, I guess
<StevenK> wgrant: I've added launchpad.SubscriberView, does that make me terrible?
<mwhudson> i think it would have upset stevea about 5 years ago
<StevenK> Haha
<StevenK> The comments in lib/lp/permissions.zcml make some interesting reading
<StevenK> mwhudson: Never mind that, most of the launchpad team managed to annoy stevea in Budapest
<StevenK> wgrant: Reassign View?
<wgrant> StevenK: You could make +packages etc. require more than View, potentially
<StevenK> Indeed
<StevenK> +packages requires View in the ZCML, but the view wants Append
<StevenK> +delete requires Edit
<StevenK> So it's a mess
<wgrant> Hmm?
<wgrant> Append is for uploading/copying packages in
<wgrant> +packages does not need Append
<StevenK> -            # To see the +packages page, you must be an uploader, or a
<StevenK> -            # commercial admin.
<StevenK> -            if not check_permission('launchpad.Append', self.context):
<wgrant> Oh, you mean for a private archive
<StevenK> Yes
<wgrant> Anyway, Append is clearly wrong here
<wgrant> Because you can't protect the view with it
<StevenK> It is already is if it's private
<wgrant> No
<wgrant> There's a hack that you are to entirely disregard
<wgrant> The view is not protected
<StevenK> Right, +edit and +delete require .Edit, and +builds and +packages require View
<StevenK> And +subscriptions requires Append
<wgrant> Right
<wgrant> The key is that subscribers need to be able to see +index but not +packages
<StevenK> Yes
<wgrant> So you need multiple view permissions
<wgrant> And LimitedView probably doesn't fit
<wgrant> => you probably need another viewish permission
<wgrant> Not Append or Edit
<StevenK> So I've created SubscriberView
<wgrant> Right, but you need to consider how that will interact with things that like View
<StevenK> But that trips straight over the is-page-contentless, because that fires if you don't hold View
<wgrant> I think lazr.restful checks for LimitedView now instead, so it's probably fine
<wgrant> But stuff like is-page-contentless will not be
<lifeless>  OOPS-cb18303a07bb367312fd6366522207cc
<wgrant> Hmm?
<wgrant> What exactly did you expect? :)
<lifeless> not a timeout ;)
<wgrant> It's an unbatched page that probably isn't linked from anywhere
<wgrant> Oh, it is batched
<wgrant> But not very well
<StevenK> The BTF UNION BTF UNION BTF query is horrible
<wgrant> Still, AFAIK it's unlinked and unreleased and well known to be horrible
<wgrant> You got what you ordered :)
<lifeless> so delete it
<StevenK> Person:+subscriptions 19 13.25
<wgrant> I'm not sure it's necessarily worth deleting at this point
<lifeless> whats the current top freq timeout ?
<wgrant>       455 /    0  POFile:+translate
<wgrant> Most of the time
<lifeless> thats not so bad
<wgrant> Sometimes someone runs a dreadful bot that calls getMergeProposals on project groups, which can exceed that
<lifeless> well done
#launchpad-dev 2015-02-09
<cjwatson> rpadovani: Heh, I wondered when somebody would notice.  Not quite ready to give a timeline yet but we're working pretty hard on it
<rpadovani> \o/
<cjwatson> wgrant: For the AccessArtifact index changes, is there a reasonable way to test performance?  Maybe timings on dogfood?
<cjwatson> (And is it possible to test this kind of schema change in something transaction-like, so that I'm not actually mutating the live schema?)
<wgrant> cjwatson: Sure, postgres DDL is fully transactional.
<wgrant> I've realised that my NOT VALID suggestion won't work directly, though, since the lock reductions got reverted. But we can easily hack the catalog if it's too slow.
<cjwatson> So I just do BEGIN / blah / ROLLBACK?
<wgrant> Dogfood is good for timing, yep
<cjwatson> Is DDL timing within a transaction reasonably reliable?  I mean it doesn't just defer most of the work until commit?
<wgrant> COMMIT is by necessity instantaneous.
<cjwatson> http://www.postgresql.org/docs/9.3/static/sql-begin.html has an admonition about statements being executed more quickly inside a transaction
<wgrant> Well, that's true, it won't fsync before the COMMIT.
<wgrant> (or do synchronous commit)
<wgrant> But the vast majority of the work is done before the COMMIT.
<cjwatson> Oh, and how would you recommend getting an accurate timing?
<wgrant> \timing on
<cjwatson> Thanks
<cjwatson> wgrant: Timings in the MP now
<cjwatson> https://dev.launchpad.net/PolicyAndProcess/DatabaseSchemaChangesProcess suggests that 1.5 seconds may be tolerable?
<wgrant> I guess AccessArtifact is only private stuff, so if it's hot it should be easy.
<wgrant> We need to ensure it's hot beforehand, though; wildcherry is crap.
<cjwatson> 224143 rows on DF
<wgrant> That could well be fine. They're narrower rows than I'd remembered.
<wgrant> cjwatson: Speaking of which, when is your meeting preference this week?
<cjwatson> 8MB
<cjwatson> (table size)
<wgrant> Yeah, the rows are basic two ints + header each.
<cjwatson> wgrant: Either day is workable, but I think Tuesday as in the calendar is best.
<wgrant> Sounds good.
<wgrant> Easier than working out how to move an event in Google Calendar.
<cjwatson> :-)
<cjwatson> I think the only unaddressed bit of the db-git review is the flag/shortcut question.  If the enum approach sounds workable I can try implementing that today.
<wgrant> I knew there was something I was missing, yeah.
<wgrant> We probably want two separate flags to make the unique constraints and queries more sensible.
<wgrant> Though an enum wouldn't be terrible, I'm not sure it's worth it.
<cjwatson> So owner_official boolean NOT NULL, official boolean NOT NULL, CONSTRAINT blah CHECK (NOT owner_official OR NOT official); owner_official true implies lp:~cjwatson/launchpad, official true implies lp:launchpad, both false implies unofficial repo
<cjwatson> (or s/official/preferred/g or s/official/default/g, something like that)
<wgrant> Yep, exactly.
<wgrant> Then the unique indices and queries are trivial.
<cjwatson> Gets rid of a lot of duplication in the GitShortcut definition, certainly.
<cjwatson> Might need a little bit of care in personmerge.
<wgrant> Yeah, but branches are already handled specially anyway due to name conflicts.
<wgrant> I'd assume that the default flags get unset upon move.
<cjwatson> If there's a conflict, at least, eys.
<cjwatson> *yes
<wgrant> target_default has to be unset regardless, but owner_default could stay if it doesn't conflict, yeah.
<cjwatson> Why does target_default have to be unset if there's no conflict?
<cjwatson> Just changing the owner around shouldn't matter for that.
<wgrant> I guess if it's just the owner changing, true.
<cjwatson> target_default probably ought to be unset if you're changing the target, granted.
<wgrant> Yeah, you're right, I'd forgotten that changing the user counted as a move.
<cjwatson> I think  owner_default OR NOT target_default  would make a better constraint, actually.
<cjwatson> We don't want to have launchpad-pqm be the owner of launchpad and to have lp:~launchpad-pqm/launchpad != lp:launchpad.
<cjwatson> So if target_default is true then owner_default must also be true, to avoid silliness.
<wgrant> Yeah, indeed.
<wgrant> cjwatson: You have enough to go on now?
<wgrant> The only niggle I have with the model now is the owner == target duplication, but I don't think we can sensibly avoid it completely.
<cjwatson> I think so.  I'm not sure how to avoid that either, I've been going back and forward on None vs. IPerson.
<cjwatson> target=None is more Branch-like, and the root of why it's coming up now is that I've simplified away the various places where we pass separate project and sourcepackage arguments there to a single target.
<cjwatson> Should I introduce IGitTarget?  That's the one major piece of the Branch* cluster that I was able to omit, but if I introduced that then I could pass a non-None target for personal repositories.
<wgrant> Yeah
<cjwatson> But the downside there is that it makes the webservice implementation less obvious.
<wgrant> What would the non-None target be?
<cjwatson> PersonalGitTarget(owner=foo)
<wgrant> Ah
<cjwatson> Or whatever
<wgrant> But yeah, that makes the webservice impossible.
<wgrant> Simplifying everything to take a single abstract target= is definitely the right way to go. I took Bugs most of the way in 2012, and it worked fine.
<cjwatson> We'd have to go back to project= distro_source_package=, or else export IGitTarget on the webservice.  Neither is very palatable.
<wgrant> But personal bugs aren't a thing.
<cjwatson> So I think target=owner for those is the least bad alternative.
<wgrant> I agree.
<wgrant> Oh, it's IS meeting week. I'd best sleep.
<cjwatson> night
<rpadovani> cjwatson, o/ Do you have ever thougth to apply Launchpad to Google Summer of Code? :-)
<cjwatson> rpadovani: I've only been full-time on LP for a month, give me a chance :-)
<cjwatson> I don't know that kind of history
<jelmer> IIRC there was one year in which we had joint Bazaar/Launchpad participation in the SoC
<wgrant> I don't recall Launchpad ever doing it, but Bazaar may have.
<jelmer> wgrant: Yeah, you're right
<jelmer> https://developers.google.com/open-source/soc/2007/#bzr
<jelmer> I think what I must be remembering is that Bazaar once participated under the umbrella of Ubuntu
<wgrant> Ah yes, that sounds plausible.
#launchpad-dev 2015-02-10
<timrc> cjwatson, So small world.  One of my new coworkers at HP is Susie Gray who apparently you know from her time in Cambridge.
<cjwatson> timrc: indeed :)
<cjwatson> wgrant: git-basic-model review> I'm not really sure how the launchbag is used, in general, and when one needs to update it.  Can you point me at a reference?
<cjwatson> wgrant,blr: I'm around when you are
<blr> cjwatson: ready whenever suits
 * wgrant wanders in.
#launchpad-dev 2015-02-11
<cjwatson> OK, I think that's all the BUILDERFAILed builds I can see retried.
<cjwatson> wgrant: You mentioned making sure AccessArtifact is hot before applying db-git.  Do you think that's still necessary given the staging timings, and if so how do we go about it?
<wgrant> cjwatson: Yes, it's still necessary. A SELECT COUNT(*) FROM accessartifact; a few times should do it.
<wgrant> wildcherry IO is sloooooooooow.
<cjwatson> OK, I'll put that in the RT.
<cjwatson> wgrant: https://wiki.canonical.com/InformationInfrastructure/OSA/LaunchpadProductionStatus#Unusual_Rollout_Requirements look right?
<wgrant> cjwatson: I'd say "immediately before", but one of us will be handholding anyway I imagine.
<cjwatson> tweaked
<wgrant> Thanks, looks good.
<cjwatson> That's https://rt.admin.canonical.com/Ticket/Display.html?id=78737 now.
<cjwatson> Hopefully wildcherry will rest in peace soon.
<wgrant> Quite.
<ePierre> Hi everyone
<ePierre> I was checking this page yesterday regarding the Launchpad API : https://help.launchpad.net/API/launchpadlib
<ePierre> and I saw a link to the reference documentation: https://launchpad.net/+apidoc/
<ePierre> however it looks like the current version of the API is planned to be deprecated in 2 months
<ePierre> if I want to develop a little Python tool to retrieve and process info from Launchpad, should I still go with launchpadlib?
<wgrant> ePierre: launchpadlib is the way to go, but unless your application is being distributed to end-user systems it should probably be using the 'devel' API version.
<wgrant> '1.0' will be around for longer than the page states (and even 'beta' still works, as we've had no reason to break it so far), but new applications that can be easily updated should use 'devel' by default.
<ePierre> wgrant, ok thanks!
<ePierre> For now it's just gonna be for personal use (I just would like to tinker a bit with Python and Launchpad to retrieve info more easily than using the web interface)
<wgrant> ePierre: Definitely use devel, in that case.
<cjwatson> wgrant: https://code.launchpad.net/~cjwatson/launchpad/git-basic-model/+merge/248976 should be ready for re-review now.
<wgrant> Ah, you beat me.
<wgrant> Just finished with the second pass, just about there.
<cjwatson> Oh zope.publisher why do you despise me
<wgrant> cjwatson: https://code.launchpad.net/~wgrant/launchpad/xbia-refactor/+merge/249295 and https://code.launchpad.net/~wgrant/launchpad/xbia/+merge/249296 could do with a review if you have time today.
<wgrant> What's it doing to you?
<cjwatson> wgrant: will do
<cjwatson> Unhelpful response to unexpected exceptions in XML-RPC methods.  (Hacking in traceback.print_exc)
<wgrant> Ah.
<cjwatson> Er traceback.print_exception
<cjwatson> Otherwise you just get "fatal: remote error: Unexpected Zope exception: TypeError: object.__new__() takes no parameters" and have to drink heavily.
<cjwatson> </mjg59>
<wgrant> Nothing useful in the appserver log?
<cjwatson> No, but there is if one hacks zope.publisher :)
<wgrant> Heh
<cjwatson>         # Create an appropriate Fault object. Unfortunately, we throw away
<cjwatson>         # most of the debugging information. More useful error reporting is
<cjwatson>         # left as an exercise for the reader.
<wgrant> ,,,
<cjwatson> wgrant: Does it matter (e.g. for determinism of build creation) that the return value of determine_architectures_to_build is no longer sorted?
<cjwatson> wgrant: And do we have agreement with Debian ftpmaster (or buildd admins?) on the specification?
<wgrant> cjwatson: createForSource sorts the output, I think, but let me check.
<wgrant> cjwatson: It's roughly compatible with the Debian proposal, at least in the ways we're likely to use it.
<wgrant> (though the Debian proposal hasn't been touched in about a year)
<wgrant> Steve needs this now, or gross hacks will abound.
<wgrant> We figure it'll only be a handful of packages, so even if they adopt a conflicting proposal it's pretty trivial to migrate.
<cjwatson> The output isn't sorted AFAICS.
<wgrant> createForSource sorts by processor.id
<cjwatson> Only for arch_indep_das
<cjwatson> needed_architectures is unsorted
<wgrant> are you looking at the new code?
<wgrant> Oh, or did I break it in the refactor branch, and fix it in the final...
<cjwatson> Ah, you would appear to have done
<wgrant> Yeah, sorry, you're right.
<cjwatson> OK, I guess we can land in one step.  Let me just reassure myself about the rest of the second branch.
<cjwatson> Shouldn't there be some kind of assertion in determine_architectures_to_build that if we need an arch-indep build then we actually get one?
<cjwatson> Or will that fail later?
<wgrant> It may legitimately fail.
<wgrant> If I have an "Architecture: powerpc all", it will only build on powerpc. If I don't have powerpc, it shouldn't create any builds.
<cjwatson> So that'll just return the empty dict.  OK.
<wgrant> Right.
<cjwatson> r=me with a couple of comments
<wgrant> Thanks.
<wgrant> dpkg-architecture is really slow.
<wgrant> The build start time estimate tests are the worst thing in the world.
<cjwatson> Re GitRepositorySet.set*Default, I was thinking of having those be internal methods which were called by methods that require edit permission on the target; I'm just not far enough up the stack to implement that yet.  But you're right that it shouldn't require edit permission on the repository, and I should make the intended use more clear.
<cjwatson> s/Set//
<wgrant> In that case the automatic clobbering behaviour probably makes less sense.
<wgrant> Moving them off Edit indeed sounds sensible.
<cjwatson> wgrant: I've addressed your second batch of git-basic-model comments.  Thanks.
#launchpad-dev 2015-02-13
<lifeless> wgrant: haskell lp auth working
<lifeless> wgrant: next release of authenticate-oauth will be usable out of the box
<wgrant> lifeless: Just needed the body -> header tweak?
<lifeless> yeah and just on +access-token
<wgrant> Yep.
<lifeless> https://plus.google.com/+RobertCollins/posts/JByEJs1dJJv
<wgrant> Haskell hasn't melted your mind yet?
<lifeless> nope
<lifeless> there's some initial get-to-grips on how monads compose to burn through
<lifeless> I nabbed a little of erikd's time to help be bootstrap on that
<wgrant> :)
<lifeless> and now I'm terrorising ghc happily
<lifeless> I'm not superproductive yet
<lifeless> I don't think I'm going to tackle wadl processing at this sstage though :)
<mwhudson> heh, my first (and only) ghc bug report was nearly 15 years ago: http://code.haskell.org/~dons/haskell-1990-2000/msg06263.html
<lifeless> mwhudson: what was your 'real' thing ?
<mwhudson> lifeless: undergraduate computational maths project
<mwhudson> modular arithmetic iirc
<mwhudson> hm, i still had the code a while ago
<mwhudson> there were lectures on pascal for people who hadn't programmed before
<mwhudson> but i implemented my projects in common lisp, python, haskell and bash...
<cjwatson> we had lectures on ML, on the basis that that way nobody knew WTF they were doing even if they'd been self-taught programmers for >10y
<cjwatson> (explicitly so!)
<mwhudson> cjwatson: a pretty sound approach!
<mwhudson> oh heh, iirc the haskell was cracking rsa
<mwhudson> for super exciting key lengths like 32 bits
<cjwatson> we had an end-of-first-term assignment to decrypt RSA with some tiny key length using pencil and paper
<cjwatson> when deciphered, the plaintext was "ONLY EIGHT MORE TERMS"
<lifeless> nice!
<mwhudson> :)
 * mwhudson zzz
<cjwatson> blr: OK, I finally have your turnip charm deploying with the local provider now that I've figured out WTF was up with HTTP proxies.
<cjwatson> blr: I had to apply http://paste.ubuntu.com/10207023/ - does that make sense?  check-rev isn't a command, I don't believe you can "juju set turnip" before you "juju deploy", and it doesn't look like the charm takes a revision config option anyway
<cjwatson> (if that makes sense, I'll send an MP, but with doc updates as well)
<cjwatson> blr: I probably can't sanely hook it up to a local LP until we make turnipserver much less hacky and actually driven by the juju config, though
<cjwatson> blr: would it make more sense perhaps, rather than running turnipserver, to (write and?) run separate processes for each element of the stack?  turnipserver is a reasonable stopgap when developing, but this would fit better with longer-term scalability needs
<cjwatson> wgrant: https://code.launchpad.net/~cjwatson/launchpad/git-basic-model/+merge/248976, https://code.launchpad.net/~cjwatson/turnip/auto-create-repository/+merge/249058, and https://code.launchpad.net/~cjwatson/launchpad/git-namespace/+merge/248995 should all be ready for re-review now.
<cjwatson> and I think that's a reasonable point to declare EOW
<blr> cjwatson: I think we want to be careful there, CI ran into difficulties coupling their code to juju.
<blr> cjwatson: ah quite right, an error in the makefile which I think is corrected in one of the more recent revisions of the charm
<blr> cjwatson: and yes, I think we're going to want to run the processs independently, absolutely.
<blr> cjwatson: how revisions are managed still needs work, might return to that after I have the block storage sorted.
<blr> regarding the first point, I think it would really hinder development to have to rely on juju in any way in development.
<blr> ugh, the word 'development' is a bit overloaded, but hopefully that makes sense.
<cjwatson> blr: I'm definitely not suggesting coupling development of turnip itself to juju - what did you read as suggesting that?
#launchpad-dev 2015-02-14
<blr> cjwatson: ah, that's good - the "driven by juju config" bit :)
<wgrant> cjwatson: Will hopefully have those all approved before you return. Thanks.
<wgrant> I guess cjwatson meant that it was configurable at all, which is necessary for both juju and LP usage.
<blr> wgrant: right, from what I gather, on the pyramid end anything exported to the environment takes precedence over configuration in foo.ini - not certain what the convention for twisted is, so would appreciate your thoughts on that.
<blr> cjwatson: will catch with you on my tuesday to get a sense for what you need specifically for your work.
<cjwatson> blr: Right, as wgrant says I meant in general configurable at all (probably with defaults roughly equivalent to what it's doing now), and in this specific case driven by the juju config.
<lifeless> ugh stab. LP API datetimes use +00:00 rather than Z
<lifeless> standards... the thing where there are so many to choose from
<wgrant> lifeless: That's from Python's isoformat.
<wgrant> Also +00:00 and Z are both ISO8601, so not even multiple standards :P
<lifeless> yeah
<lifeless> still ugh. found a fix - use ZonedTime
<lifeless> see aeson issue 197
<lifeless> night
<wgrant> Night :)
<lifeless> oh, and yeah, I have a aeson prser for bug/bugimportance/bugstatus now. yay
#launchpad-dev 2015-02-15
<blr> wgrant: regarding turnip configuration, do we want to be running turnip with twistd and a tac?
<lifeless> turnip ?
<jelmer> lifeless: some sort of root vegetable :P
<blr> deploying on vegetables is more sustainable.
<lifeless> but nowhere near as healthy :)
<wgrant> blr: Most of our other Twisted services do that. twistd can be a little limiting at times, but I don't envisage problems in this case.
<wgrant> I suspect/hope twistd will die out as sensible init systems become universal.
<wgrant> But for now it's probably a good idea.
<blr> wgrant: cool, will do that then.
<blr> wgrant: good weekend?
<wgrant> blr: The aircon died, but otherwise pretty good :)
<wgrant> You?
<blr> ugh, imagine that's not much fun. Got the winter vegetables in, alas, no turnips.
<wgrant> Heh
<blr> wgrant: hoping to have a chat with pitti today about his experiences with nova-compute-lxc - thomi mentioned he posted to canonical-tech last year while working on the uci-engine.
<blr> well, this evening more likely, given timezones.
<wgrant> blr: If you're wanting to test deployment on OpenStack, I'd get creds to our bootstack and use the real thing.
<wgrant> The standard for sensible juju work nowadays is local provider for most development and testing, then real OpenStack when you want to do more.
<blr> wgrant: okay, just thought it sounded appealing provisioning the entire stack locally - although at a guess, it would be painful.
<wgrant> blr: Oh yeah, it's certainly appealing. But I'm not sure it's worth the pain.
<wgrant> Ah, found the buildd-manager double-upload bug.
<wgrant> Yay
<wgrant> https://bugs.launchpad.net/launchpad/+bug/1422199
<mup> Bug #1422199: buildd-manager can repeatedly upload builds if handleStatus fails in the wrong place <buildd-manager> <buildfarm> <oops> <Launchpad itself:Triaged> <https://launchpad.net/bugs/1422199>
<xnox> wgrant: good morning =) how is monday looking so far?
<wgrant> xnox: Rather good.
<wgrant> xnox: Well done on the launchpadlib py3k port.
<xnox> wgrant: tah. Well, pitti only found half a dozen bugs in it, when he tried to port apport.
<xnox> mostly around marshalling binary data around.
<wgrant> Heh, of course.
 * xnox can't run full test-suite, as i didn't port _all_ the testsuite deps - which is like the rest of lazr.*
<wgrant> Yeah. Would be nice if it could run against an external lazr.restful.
<xnox> plus i'm not sure test-suites would have helped here. Cause i had to port them to str/bytes world view as well.
<xnox> yeah, but too crazy for me to implement =)
<xnox> i find pitti to be a better testsuite.
 * xnox hides
#launchpad-dev 2017-02-14
<xnox> cjwatson_, shall we do a rename in launchpad s/pocket/channel/?
<cjwatson_> xnox: eh, not worth the ginormous pain
#launchpad-dev 2018-02-16
<gQuigs> thoughts on dropping "Read about installing" part of the PPA page
<gQuigs> ?
<gQuigs> instead replace with a link to the two wiki pages, https://help.launchpad.net/Packaging/PPA or https://help.launchpad.net/Packaging/InstallingFromAPrivatePPA depending if it's a private PPA or not
<gQuigs> added my suggestion to bug https://bugs.launchpad.net/launchpad/+bug/631868, I'm happy to do work on this, if it's something we want to do
<mup> Bug #631868: Adding PPAs pop-up confusing <help> <lp-soyuz> <ppa> <ui> <Launchpad itself:Triaged> <https://launchpad.net/bugs/631868>
#launchpad-dev 2020-02-10
<tomwardill> landing rocketfuel changes branch
<cjwatson> tomwardill: Er
<cjwatson> +getopt_output="$(getopt -o '' -l no-workspace,lpusername:,assume-yes -- "$@")" || exit 1
<cjwatson> ++eval set -- "$getopt_output"
<cjwatson> Second line is a syntax error isn't it?
<tomwardill> huh
<tomwardill> err, sure I tested that
<tomwardill> one mo
<tomwardill> ...
<cjwatson> You can switch it back to Needs Review if you're quick
<tomwardill> yeah, just did
<tomwardill> hopefully quick enough
<cjwatson> Yeah
<cjwatson> Also the "while :; do" indentation is still weird
<cjwatson> Shouldn't it start at column 1?
<tomwardill> oh, so it should
<tomwardill> it was originally tabbed, not spaced
<tomwardill> so that's what I thought you meant :)
<cjwatson> Ah - no, I meant the vertical position :)
<tomwardill> okay, both fixed
<cjwatson> Cool, looks better now :)
 * tomwardill clears throat...
<tomwardill> "landing rocketfuel changes branch
<tomwardill> "
<tomwardill> (take 2)
<cjwatson> tomwardill: Could you have a look at https://code.launchpad.net/~cjwatson/lp-source-dependencies/+git/lp-source-dependencies/+merge/378804 and https://code.launchpad.net/~cjwatson/launchpad/+git/launchpad/+merge/378805 ?  More attempts to get staging working
<tomwardill> on it
<tomwardill> +1 to both
<cjwatson> Thanks
<cjwatson> (I think we should seriously reconsider the policy of using sdists for most things and wheels only where necessary.  Installing sdists involves executing code and is generally a much slower and much more complex operation than installing wheels.  For pure-Python packages it makes basically no difference to auditability, or if anything probably improves it since you're just unpacking zip files ...
<cjwatson> ... rather than doing complicated things with setup.py.  For packages containing extensions we do indeed need to make sure that we trust the builds; but we could build them once somewhere trustworthy and commit that, which we have to do anyway in several cases.)
<cjwatson> I remember wgrant being opposed at one point to lp-source-dependencies containing very many wheels, but it's possible I misunderstood him, that he's changed his mind, or that I can persuade him. :-)
<wgrant> For pure python maybe
<wgrant> But for extensions we'd have to do weird things to separate them by arch, series, etc.
<cjwatson> We already do by arch
<cjwatson> Series perhaps
<wgrant> Also a full build really doesn't take that long
<cjwatson> Uh
<cjwatson> It really does
<cjwatson> It massively slows down our production deployments
<wgrant> That might advocate for wheels being included in the deployment artifact
<cjwatson> (We mostly don't notice locally because most people don't rm -rf download-cache/wheels/)
<wgrant> I'm pretty sure the build on my desktop took under ten minutes
<wgrant> After I did that
<cjwatson> Yeah, it's a lot worse on a lot of our hardware
<cjwatson> The staging deployment hadn't even finished failing everywhere after I got up this morning, so it spent something like nine hours just doing five or so builds
<cjwatson> There's a problem with pip's wheel caching - the wheels are cached by hash of the source path, so the download cache isn't preserved usefully when we sync it out to deployment targets
<cjwatson> But maybe we could copy them out to some other directory with predictable paths and then use that in the deployment artifact
<wgrant> That is so weird
<cjwatson> (sdists also involve dealing with the usual setup_requires nightmare)
<cjwatson> Mostly much better with setuptools invoking pip to do that now, except that that's actually made things worse on Python 3 due to https://github.com/pypa/pip/issues/6264
<cjwatson> On Python 2 we dodged that bug due to luck
<cjwatson> I think we're going to have to finish the job of abolishing our use of --system-site-packages
<cjwatson> Which would make things much less confusing anyway, but there are a couple of remaining hard problems to solve
<cjwatson> Basically python-apt and python-pil (and maybe python-tdb for code imports, but we can probably just add that to the virtualenv)
<cjwatson> For python-apt I have an evil plan that looks more or less like https://paste.ubuntu.com/p/4xHRt4xZwS/
<cjwatson> We'd have to keep up with updates (but we have to do that anyway for lots of our Python dependencies) and there may be some care involved with libapt compatibility, especially across series, but those problems seem tractable
<cjwatson> I'm not sure I'm comfortable taking that sort of approach for python-pil / pillow though, because ouch @ image parsing libraries on our security boundary
<cjwatson> Aside from some build system stuff, we only use it for checking sizes of images uploaded in forms.  I wonder if we could plausibly just call out to imagemagick for that
<pappacena> Thanks for the comments on my MP, cjwatson. I'm splitting the ArchiveSigningKey renaming into this new one: https://code.launchpad.net/~pappacena/launchpad/+git/launchpad/+merge/378826 while I'll check the other feedbacks
<tomwardill> that discussion of development environments in London was ace, as it caused me to actually fix all the broken things in mine
<cjwatson> pappacena: Great, thanks, will look
<roadmr> is this a good place to whine about Launchpad timeouts on specific operations? :)
<cjwatson> Only if they come with OOPS IDs
<tomwardill> and cake
<roadmr> https://oops.canonical.com/oops/?oopsid=OOPS-e0d2dfcb5398a0477ac9d343b669f4c1 ð
<roadmr> that's a sample OOPS but - it used to sporadically timeout when adding people to the sso-2f-testers team - these days it *always* times out (but still adds the person)
<roadmr> maybe the list of people is now too big? it's 1300 or so I think
<cjwatson> Yeah, looks like it needs some bulk-loading.  Could you file a bug and quote the OOPS ID please?
<roadmr> totally
 * cjwatson probably won't have time but hopefully somebody else will :)
<roadmr> yes, and it's not a showstopper, just annoying
<roadmr> thanks!
<roadmr> https://bugs.launchpad.net/launchpad/+bug/1862670
<mup> Bug #1862670: timeout/oops when adding people to a certain LP team (but adding still succeeds?) <Launchpad itself:New> <https://launchpad.net/bugs/1862670>
<wgrant> cjwatson: I'm sorry did you just propose fixing a security issue by calling out to imagemagick?
<cjwatson> wgrant: eh there's a reason I haven't just put together a patch for it yet ...
<cjwatson> pillow also seems made of CVEs though
<cjwatson> I happened to have a look at it and the last security update was, err, last week
<cjwatson> and we only pick that up when we happen to restart appservers for other reasons
<cjwatson> surely in 2020 CE there exists a simple thing to find the dimensions of images that isn't composed of cheese and security holes
<cjwatson> https://pypi.org/search/?q=image+size has several things in pure Python that look 100x simpler and probably usable
<pappacena> Silly Zope question here, but... I'm trying to register a class as a component here, but I'm getting a strange error saying " __init__() takes exactly 4 arguments (1 given)". My configure.zcml directive looks quite similar to other examples in Launchpad... anyone knows what am I missing?
<pappacena> https://www.irccloud.com/pastebin/i7QTFuxN/configure.zcml
<pappacena> this is my configure.zcml
<cjwatson> pappacena: what's ArchiveSigningKey's __init__?
<pappacena> ahm... it's empty
<cjwatson> pappacena: normally something registered as a utility would have at most an __init__(self) without extra options
<pappacena> ahhh, no no!
<cjwatson> pappacena: the thing in class= is really just a factory - whatever you put there is called with () to create the component that's registered as the utility
<pappacena> it's not empty... I've declared that receiving paremeters! ð¤¦ââï¸
<pappacena> Ok, I moved one step... let me keep on doing things here. Thanks, cjwatson!
<cjwatson> np
#launchpad-dev 2020-02-11
<tomwardill> cjwatson: SnapBuild has a `calculateScore` with a hardcoded number of 2510, is that important?
<cjwatson> tomwardill: Yes (https://help.launchpad.net/Packaging/BuildScores).  Just returning a constant 2510 is fine for now (self.archive.relative_build_score doesn't make sense here), though we'll probably want to add a relative_build_score to the OCIRecipe at some point so that we can score individual recipes up or down
<cjwatson> Build scores are less important than they were when the build farm was hideously underprovisioned, but of course they do still matter sometimes
<tomwardill> righto, makes sense
<tomwardill> cjwatson: I'm unclear on the process of inferring `virtualized` for a recipe build (essentially, what is involved and where that lives).
<tomwardill> I don't know what the rules are that make something virtualized vs not
<tomwardill> is there somewhere I should be looking to find it?
<tomwardill> this was in my head, once upon a time
<cjwatson> tomwardill: require_virtualized is a column on the recipe, defaulting to true and only settable by (some variety of) admins, because setting it to false may allow privilege escalation depending on the current configuration of the build farm.  virtualized on an individual recipe build is derived from a logical combination of the configuration of the Processor it's building for and the ...
<cjwatson> ... appropriate OCIRecipe.require_virtualized
<cjwatson> tomwardill: Conventionally set in FooBuildSet.new, in this case probably "not distro_arch_series.processor.supports_nonvirtualized or recipe.require_virtualized"
<tomwardill> right, that makes sense
<cjwatson> So if the Processor doesn't "support" running non-virtualized (really, is forbidden from doing so) then all its builds are virtualized; otherwise, it depends on the recipe
<cjwatson> SnapBuildSet.new and LiveFSBuildSet.new take archive.require_virtualized into account too, and actually that's kind of weird, I wonder why
<cjwatson> The archive is a source rather than a target in both those cases
<cjwatson> I can't really think of why that's the case and I suspect it's a bug
<cjwatson> It makes sense in BinaryPackageBuildSet.new because the archive is a target and the build is in some sense "within" it
<tomwardill> huh, interesting
<tomwardill> factory.py just caused my VSCode to force close
<tomwardill> that's a new one
<pappacena> Well, maybe 5k lines of code is too much for VSCode... such a week software. hehe
<tomwardill> hah
<tomwardill> I think it's more that I ran out of ram
<tomwardill> 34Gb commited caused some problems
<tomwardill> cjwatson: https://code.launchpad.net/~twom/launchpad/+git/launchpad/+merge/376132 is ready again, if you've got some time to look over it to check I've not wrecked anythign with the post-review changes :)
<tomwardill> oh damn, forgot the personmerge
 * tomwardill does the thing (also, yay todo lists)
<cjwatson> https://code.launchpad.net/~cjwatson/launchpad/+git/launchpad/+merge/378908 should sort out the deployment trouble from earlier today
<cjwatson> tomwardill: thanks, let me know once you're done with personmerge and I'll re-review
<tomwardill> cjwatson: just pushed
<tomwardill> ooh, my buildtheworld thing can actually install launchpad.
<tomwardill> Now for the hard part of config hacking until it works
<cjwatson> nice
<cjwatson> tomwardill: Could you run utilities/format-new-and-modified-imports over that whole MP?
<cjwatson> system-site-packages avoidance symlink hack seems to at least sort of work.  Trying to productise it a bit and test it now
<cjwatson> And removing a collection of small hacks in various places that only existed due to that
<wgrant> Nice
<cjwatson> Hm, we never properly landed the cowboy of python-tdb onto code import workers, did we
<cjwatson> wgrant: Could you review https://code.launchpad.net/~cjwatson/meta-lp-deps/remove-python-subversion/+merge/378802 and https://code.launchpad.net/~cjwatson/meta-lp-deps/python-tdb/+merge/378925 ?  I doubt anyone else has enough background to review those usefully at the moment
<wgrant> cjwatson: lgtm
<wgrant> What was the cowboy?
<wgrant> Oh you mean installing it at all
<cjwatson> Yeah
<cjwatson> The undocumented dep hit us when we deployed alnitak and izar
<cjwatson> (I love the Arabic star names scheme.  Every so often I'm out with Judith at something astronomy-related and I see a machine name in a star map)
<cjwatson> Alnitak's in Orion so you see it all the time
 * cjwatson discovers a variety of ridiculous ancient dependencies (e.g. cscvs â honest-to-goodness original python-sqlite)
<cjwatson> One of those "considerably less effort to just back away and leave it there" things
<cjwatson> Will probably have to port it to SQLite 3 at some point if CVS imports haven't died by then
#launchpad-dev 2020-02-12
<tomwardill> cjwatson: done and pushed
<SpecialK|Canon> cjwatson: yay for symlink hack!
<SpecialK|Canon> cjwatson: good email (re livefs webhooks)
<SpecialK|Canon> ilasc: please make sure ^ go in the weekly!
<SpecialK|Canon> on the subject of naming things
<ilasc> SpecialK|Canon noted.
<SpecialK|Canon> one of the things Przemek inherited when we last worked together was a Django/Celery service I'd built to manage webhooks
<SpecialK|Canon> called...Spiderman
<SpecialK|Canon> because it...shoots webs
<SpecialK|Canon> (I'll get my coat)
<cjwatson> SpecialK|Canon: I put it in the weekly yesterday
<cjwatson> FWIW
<cjwatson> These days I try to do that as a post-deployment step, because otherwise I forget
<Spads> this reminds me of when I wrote a STONITH mechanism into something, and took great glee in naming everything based on that metaphor.  other_node_to_be_shot_in_the_head = gethostbyname(...)
<Spads> head_in_which_other_node_is_shot = IMPI(...).port()
<Spads> etc
<cjwatson> OK, I'm generally not fond of the "shoot in the head" metaphor, but "head_in_which_other_node_is_shot" breaks the unpleasant bits of the metaphor enough to make it funny
<SpecialK|Canon> cjwatson: hah way ahead of me, thanks :)
<SpecialK|Canon> and agreed re STONITH
<ilasc> cjwatson: thank you for all the help on the webhooks for livefs and for adding it to the weekly!
<cjwatson> my pleasure
<tomwardill> cjwatson: https://code.launchpad.net/~twom/launchpad/+git/launchpad/+merge/376431 is rebased on top of the latest concrete branch and ready for another look when you get chance
<cjwatson> tomwardill: approved, but a few small comments and one critical one.  Go ahead and land once you've fixed those
<tomwardill> cjwatson: righto, will do :)
<cjwatson> tomwardill: (er, by which I mean concrete-oci-recipes, not the next one along yet)
<tomwardill> indeed :)
<tomwardill> Just got the email
<tomwardill> okay,I can go from nothing to 'running' launchpad in one command
<tomwardill> shame I can't login once it's running...
<cjwatson> Not bad though
<tomwardill> cjwatson: ah, those tests arrive in the next MP (oci-ocirecipebuild)
<tomwardill> although come to think of it, that is missing anything that tests that, as virtualized being inferred is new
<cjwatson> Right, so maybe it's quick to just move the relevant bits over
<tomwardill> yeah, doing tha tnow
<tomwardill> blergh, the next one has a bunch of rebase fails in it I've just found
<tomwardill> wee, this is a fun pile of yaks. Currently adding more interface declarations to IOCIRecipe, because I missed the Admin ones
<tomwardill> cjwatson: does https://git.launchpad.net/~twom/launchpad/commit/?id=c5e1d572e1ee61ffa33d73abb73cc87b0b67958a look reasonable?
<tomwardill> had to backport some more infrastructure from the next MP
<cjwatson> tomwardill: LGTM
<tomwardill> cool, will land that, and then rebase/fix the next one
<tomwardill> cjwatson: https://code.launchpad.net/~twom/launchpad/+git/launchpad/+merge/376431 is up I think
<cjwatson> wgrant: I fixed up the type naming in https://code.launchpad.net/~cjwatson/launchpad/+git/launchpad/+merge/373765 (db-bugsummary-statement-triggers) slightly.  OK now?
<wgrant> cjwatson: Yay!
<cjwatson> OK, let's see how long an LP test run takes on my home server now it has an SSD to play with
<cjwatson> Older CPU than my laptop (Ivy Bridge), but seems a bit faster in some situations so we'll see
<cjwatson> Er, server is Ivy Bridge, laptop is Broadwell
<cjwatson> I tried before but it was too slow to be pleasant on rotating rust
<cjwatson> Would be nice to be able to run LP tests somewhere that isn't my laptop
#launchpad-dev 2020-02-13
<SpecialK|Canon> cjwatson: https://www.madhatterboardgames.co.uk/ - fabulously lightweight side-business; limited browsing potential but they seem great for "do you have X please?"
<SpecialK|Canon> Their site seems far more featureful on desktop, but still, no real stock list
<SpecialK|Canon> Meanwhile twom linked me to a _lovely_ ukulele cover of Heal The World: https://www.youtube.com/watch?v=SE_ZSTqgv7E
<tomwardill> I would pay reasonable money for a copy of Agricola All Creatures Great and Small
<tomwardill> s/Great/Big/
<SpecialK|Canon> tomwardill: I suggest emailing them then!
<tomwardill> done
<cjwatson> SpecialK|Canon: thanks
<tomwardill> updated https://code.launchpad.net/~twom/launchpad/+git/launchpad/+merge/376431 . Will land unless anyone says otherwise :)
<tomwardill> cjwatson: where did the DB patch that adds OCIRecipeBuildJob end up? I'm completley failing to find the right branch for it.
<cjwatson> tomwardill: I think maybe I had it in the most recent one I landed but then removed it for now?  If so I haven't resubmitted it anywhere yet
<tomwardill> cjwatson: yeah, I've found it in an old DB patch that I had a copy of
<cjwatson> tomwardill: Yeah, removed in 35651746e6218615c1f4259b64f0f0e6c7cbe3ab since it's only useful for doing registry pushes AIUI
<tomwardill> cjwatson: I think it's a dependency of the next branch (BuildJob), to get towards BuildBehaviour.
<cjwatson> tomwardill: Why do we need BuildJob to have BuildBehaviour?
<cjwatson> A FooBuildJob is a job that runs over a (typically completed) FooBuild; a FooBuildBehaviour is the mechanism for dispatching a FooBuild to the build farm
<cjwatson> Usually the dependency is the other way round, if anything
<cjwatson> But only because you can't really have a completed build without having a behaviour for it
<tomwardill> huh, well my trivial answer is 'because that's the order I did the branches in', but you are right. I should be able to rebase the buildbehaviour on top of the RecipeBuild
<cjwatson> I'll try to work on the registry push DDL soon, but yeah, shouldn't be needed here
<tomwardill> cool, I'll do that
<tomwardill> going to land OCIRecipeBuild
<tomwardill> then Behaviour should rebase on top of master
#launchpad-dev 2020-02-14
<tomwardill> cjwatson: https://code.launchpad.net/~twom/launchpad/+git/launchpad/+merge/379201 is ready (rebased and resubmitted on master)
<tomwardill> although I've not actually checked it with the buildd branch...
 * tomwardill tries to work out how to do that
<pappacena> Folks, I'll top-approve https://code.launchpad.net/~pappacena/launchpad/+git/launchpad/+merge/379021, removing auditor (and django) from LP's code. Any objection?
<tomwardill> dooooooo iiiiiiitttt
<SpecialK|Canon> pappacena: no objection, but for the future, I'd personally love to see the kind of message you've got in the LP MP, in the commit message instead
<SpecialK|Canon> oh though if it ends up in the merge commit I guess that works
<SpecialK|Canon> (Mostly I'm thinking from a browsing-git perspective "ok this definitely removes auditor but _why_")
<pappacena> I guess only the "Commit message" field from LP MP goes to the git history, and not the description. In fact, I'm adding way more details in the field that *doesn't* go to the git history today. I'll start putting more details in the "commit message" field.
<SpecialK|Canon> Oops yes
<SpecialK|Canon> That feels like a slightly odd distinction
<SpecialK|Canon> I think it throws me slightly every time
<SpecialK|Canon> But yes, thank you
 * tomwardill attempts to work out how buildd-manager works
<tomwardill> Idle!
<tomwardill> woo
<cjwatson> tomwardill: reviewed
<tomwardill> cjwatson: ta. Will look after lunch
<cjwatson> commit message: I've taken to including full details in the actual git commit messages that are part of the branch I'm pushing, and normally only the first line in the MP's commit message that ends up in the merge commit, as otherwise it gets pretty duplicative
<SpecialK|Canon> Ah; I've found myself using `git log --merges` a lot in a few places, so really liked the duplication
<SpecialK|Canon> (but that's a purely personal foible)
<tomwardill> cjwatson: do you know of existing examples of TrialTestCase with pushConfig and similar LP test methods?
<tomwardill> oh,. might just be able to lift the pushConfig method, I think that's the only one I need
<cjwatson> tomwardill: MakeSnapBuildMixin.makeSnap does exactly this
<tomwardill> ah, so it does
<tomwardill> I was looking for instances of the `pushConfig` method
<cjwatson> Could I ask for some py3-related reviews?
<cjwatson> https://code.launchpad.net/~cjwatson/lazr.restful/newer-grokcore-martian/+merge/379034 and https://code.launchpad.net/~cjwatson/launchpad/+git/launchpad/+merge/379036 are various dependency upgrades
<cjwatson> https://code.launchpad.net/~cjwatson/lazr.restful/py3-division/+merge/379205 is an easy bit of porting
<cjwatson> https://code.launchpad.net/~cjwatson/launchpad/+git/launchpad/+merge/378460 is a rather larger bit of porting; the top commit in that branch is very large and formulaic, but https://git.launchpad.net/~cjwatson/launchpad/commit/?id=7e44f2849205ce68c9d6e2987874f3d7536c7429 is worth reviewing separately
<tomwardill> cjwatson: how silly an idea would be to use the 'current' series of the distribution as the distro_series of an OCIRecipeBuild? (Ideally, I think just selecting the latest LTS would be a reasonable approach, but I can't figure out if that's possible)
<cjwatson> (Are people generally getting MP review requests by email?  I feel people mostly don't look unless I explicitly ask)
<cjwatson> tomwardill: It's not ideal because it usually won't be an LTS and because it couples OCI builds to a not-very-related piece of process.  It would be OK for a first pass to unblock you, as long as it has a great big XXX on it and we commit to figuring out something better soon
<tomwardill> I get them by email, but I think I need to make my filter smarter to avoid them getting lost in the snapstore ones
<cjwatson> Mm
<tomwardill> I should also check that folder more often tbh
 * tomwardill pokes at filter
<cjwatson> Yeah - I dislike nagging too much and I figure my responsibility is more to review than to be reviewed, but at the same time I don't want things I do to be stalled
<tomwardill> yeah, I'll fix my filter and make a point of checking it more often
<cjwatson> Cool
<tomwardill> well, I would, if I could make gmails UI behave
<tomwardill> ooh, it works, and it even told me about your comment reply
<tomwardill> cjwatson: that error handling change makes it much nicer
 * tomwardill has run into that urlopen thing before
<cjwatson> tomwardill: which change, sorry?
<cjwatson> oh, in six-urllib?
<tomwardill> https://git.launchpad.net/~cjwatson/launchpad/commit/?id=7e44f2849205ce68c9d6e2987874f3d7536c7429
<tomwardill> that is much nicer than parsing the length of args
<cjwatson> Yeah, it looked less awful this way
<cjwatson> requests would probably be better still, but one thing at a time
<tomwardill> heh
