#launchpad-dev 2010-04-05
<stub> What's the bestest way of running a single story nowadays?
<deryck> Morning, all.
* deryck changed the topic of #launchpad-dev to: Launchpad Development Channel | Week 4 of 10.03 | PQM is open | https://dev.launchpad.net/ | Get the code: https://dev.launchpad.net/Getting | On-call review in irc://irc.freenode.net/#launchpad-reviews | Use http://paste.ubuntu.com/ for pastes
* wgrant changed the topic of #launchpad-dev to: Launchpad Development Channel | Week 0 of 10.04 | PQM is open | https://dev.launchpad.net/ | Get the code: https://dev.launchpad.net/Getting | On-call review in irc://irc.freenode.net/#launchpad-reviews | Use http://paste.ubuntu.com/ for pastes
<salgado> maxb, around?  have you tried running the python2.6 branch of LP recently?
<barry> gmb: ping
<gmb> barry, Hi
<barry> gmb: hi.  responded to your email.  thanks for your help with this!
<gmb> barry, No worries. Sorry for letting myself get sidetracked *again*
<gmb> barry, FYI, I've converted Mark's export to Launchpad interchange format and I'm going to try doing the import locally to see what happens to the patches.
<barry> awesome, thanks
<gmb> I'll reply to the thread with what I find out. Hopefully should have some answers today.
<barry> cool
<kfogel> barry: got a sec?  In my spare time I'm working on bug #231023
<mup> Bug #231023: on status change, show new status in Subject: <email> <Launchpad Bugs:Confirmed> <https://launchpad.net/bugs/231023>
<kfogel> barry: ... and although I'm adding all sorts of instrumentation to lib/canonical/launchpad/mailnotification.py:BugNotificationBuilder.build
<kfogel> I can't seem to get any of it to take effect in launchpad.dev.  Is that the right place?
<barry> kfogel: sure
<kfogel> barry: and lib/lp/bugs/doc/bugnotification-email.txt should also pass/fail based on changes I make in that code, right?
<barry> kfogel: that's what i would think.  did you put a break point in .build()?
<kfogel> barry: well, I'm doing this: http://paste.ubuntu.com/409510/
<kfogel> barry: IOW, I added an "x" suffix to "Bug" just to make sure things are working, and then log the whole subject to a tmpfile.
<maxb> salgado: hey. Not recently, no
<kfogel> barry: last week, I'd run that code and then go into launchpad.dev and make all sorts of bug changes -- but my tmpfile never showed up.
<kfogel> barry: trying again now
<barry> kfogel: it's always helpful to stick a pdb.set_trace() in there and see if the code you think is getting hit actually is ;)
<kfogel> barry: that's a good idea
<salgado> maxb, I've asked because I'm trying to resurrect it but am blocked on what seems to be http://reinout.vanrees.org/weblog/2009/11/13/distribute-buildout-recursion-fixed.html (which should be fixed on our version of buildout)
<maxb> Ah. For any insane buildout behaviour, I would direct you towards gary :-)
<salgado> maxb, if only he were around today... :<
<salgado> :(
<kfogel> barry: even this patch http://paste.ubuntu.com/409529/ is not throwing me into the debugger
<kfogel> barry: this is when I go to https://bugs.launchpad.dev/firefox/+bug/5 and make various changes (I add a comment, change status, change assignee, etc)
<kfogel> barry: I'm wondering if there's something I'm supposed to do to get the entire notification system "turned on"?
<deryck> mars, hi.  Do you have a minute to help me work out something for a Windmill test?
<mars> deryck, yes, in a few minutes
<deryck> mars, ok, thanks!  Just ping when you have 2-3 minutes.
<kfogel> deryck: you have backscroll to see my above conversation with barry?  If you know why I can't seem to get launchpad.dev's bug notification system operating, pls let me know.
<mars> deryck, online, let me know when you have a moment, and I can help you
<deryck> kfogel, so lp's notifications system is really just a set of calls to notify() and then running the notification script.  I would guess the script acts based on config settings, but I've not actually run it locally.
<deryck> mars, cool.  thanks.  So I'm working on bug 494257
<mup> Bug #494257: Users not notified in email when subscribed by someone else <email> <ubuntu-qa> <Launchpad Bugs:In Progress by deryck> <https://launchpad.net/bugs/494257>
<mars> deryck, ok
<deryck> mars, and the issue there is that the notify(ObjectCreatedEvent()) stuff happens in browser code, not model code.  So it's never hit from JS use of API...
<deryck> mars, so I want a windmill test to verify this and do TDD on the fix for it.  So see:  http://bazaar.launchpad.net/~deryck/launchpad/not-notified-someone-else-subscribed-494257/revision/10607
<deryck> mars, the event listener stuff there doesn't seem to work, and I'm wondering if BugsWindmillLayer will support this approach to testing this?
<mars> deryck, the first thing I have to ask about this is: does windmill share process and configuration with the application server?
<mars> I know that we can now use the ObjectFactory with windmill
<mars> but that fact does not imply that the event listener in your windmill test is being registered with the application server process
<kfogel> deryck: "notify()", huh?  Sounds like that's a major entry point that I've... not encountered yet in code, though I've heard it referred to ;-).  Where is the "notification script"?  Is it ./cronscripts/send-bug-notifications.py ?
<deryck> kfogel, yes, that's the script.  and look through model/bug.py for notify calls.
<kfogel> deryck: thx
<deryck> np
<mars> rockstar, ping, do you remember who set up the windmill objectfactory integration?
<mars> rockstar, it was done recently IIRC
<deryck> mars, I think it was flacoste and BjornT at various times, but neither is available today, I believe
<mars> deryck, yeah, may have been BjornT
<deryck> mars, this is actually what I'm asking you, is can this be done?  i.e. is there access to the app server in windmill in this way
<mars> deryck, I don't know.
<deryck> mars, ok, cool.  thanks anyway :-)
<mars> no problem.  Hopefully BjornT knows.
<deryck> I thought BjornT might be the only one to know.  but was casting about for help. :-)
<mars> I think he is the only person who knows the entire system structure.  Others know parts of it, and can assemble the whole thing if need be.
<Ursinha> sinzui, hi
<Ursinha> sinzui, is bug 548578 really ready to qa? I mean, are you able to qa that on staging or edge?
<mup> Bug #548578: scripts/cache-country-mirrors.py is generating files with perms 0600 <qa-needstesting> <Launchpad Registry:Fix Committed by sinzui> <https://launchpad.net/bugs/548578>
<Ursinha> or anywhere else
<sinzui> Ursinha, yes it is ready for QA because it will be test by spm on the machine that is affected
<Ursinha> sinzui, right
<thumper> morning
<jelmer_> hi thumper
<kfogel> Whoa. pqm.launchpad.net says "Now playing...  * 1 request(s) for other project(s).
<kfogel> "
<kfogel> What does that mean?
<salgado> kfogel, it means it thinks the submission's target is not a LP branch
<kfogel> salgado: ah, okay.  Pity it's not more... informative about that.
<salgado> yeah, it could do a lot better
<kfogel> salgado: I guess I'm trying to find out if those messages are about the two branches I just submitted.
<kfogel> salgado: I should wait for the email?
<salgado> kfogel, yeah, you'll get FAILURE emails shortly, but you might re-run the pqm-submit commands with --dry-run to find out what's wrong
<salgado> unless you're not landing launchpad branches?
<kfogel> salgado: my intention was certainly to land launchpad branches.
<kfogel> salgado: what PQM thinks my intention is, I cannot say.
<salgado> kfogel, and to make things worse it won't clearly tell you what went wrong in the FAILURE emails. we can only infer that by looking at what was sent to PQM,
<kfogel> salgado: Yes.  In retrospect, it was probably a mistake to put PQM in charge of all NATO air defense systems.
<bdmurray> Can I set bug 544387 to fix released?
<mup> Bug #544387: date_created for a bug subscription not exported in the API <qa-ok> <Launchpad Bugs:Fix Committed by brian-murray> <https://launchpad.net/bugs/544387>
<wgrant> bdmurray: Yes.
#launchpad-dev 2010-04-06
 * thumper screams at the horridness of some integration tests
 * thumper facepalms
 * thumper afk for lunch
<wgrant> sinzui: Was there a reason that you landed my sampledata branch to db-devel rather than devel?
<sinzui> wgrant, I do not think pqm will permit sample data changes to be made to devel
<wgrant> sinzui: As I mentioned in the MP, I have had sampledata changes landed to devel before.
<wgrant> I remember because people complained that I had broken their local test suite.
<wgrant> But this doesn't affect the test data, so it should be fine.
<sinzui> wgrant, sorry, that was not my understanding
<wgrant> sinzui: Well, nothing seems to actually make it clear what can go where.
<sinzui> after several botched landings, a checker was added to PQM landings to ensure no db changes could be merged in devel. My understanding wad no changes in the database part of tree
<sinzui> That may only be the schema directory
<wgrant> I touched current.sql in devel in January.
<wgrant> r10174
<sinzui> The checker is not smart. it will let you land a new enum on edge. Those are fun oopses to read on lpnet
<wgrant> Yeah :(
<sinzui> wgrant, the rule is in Makefile check_merge
<sinzui> your change were definitely landable in devel
<wgrant> sinzui: Ah! I didn't know it was public.
<wgrant> Good find.
<wgrant> Even security changes can. Excellent.
<sinzui> I can land you branch in devel now
<wgrant> That would be wonderful. Thanks.
<thumper> sinzui: sample data can land in devel
<thumper> sinzui: as can security.cfg changes
<thumper> sinzui: just no patches
<sinzui> thumper, I do not think security.cfg can land.
<thumper> sinzui: I think it can
<wgrant> They can.
<wgrant> It greps out security.cfg.
<sinzui> oh, read that the wrong way
<thumper> sinzui: LibraryFileAlias:StreamOrRedirectLibraryFileAliasView
<thumper> sinzui: timing out a lot
<thumper> do you know why?
<sinzui> no
<wgrant> I saw recent talk about making it handle offline librarians better. Could that be related?
<thumper> I don't think so wgrant
<thumper> it seems to be load dependant
<thumper> sometimes it just doesn't respond
<wgrant> Yay.
<thumper> sinzui: got a minute?
<thumper> sinzui: I have a failure with test_notifications (lp.registry.tests.test_mlists.TestMailingListImportScript)
<thumper> sinzui: mail it being delivered locally instead of being caught in the test mailer
<sinzui> I did not know that was in the tree
<thumper> sinzui: what was in the tree?
<sinzui> thumper, I do not know anything about that test. I thought the import script was added to the tree as a convenience to losas
<thumper> :(
<thumper> sinzui: is it barry?
 * sinzui thinks we are talking about the mbox archive importer
<sinzui> barry, certainly knows that best
 * thumper stabby
<thumper> ââ¹
<wgrant> thumper: This is with your branch that suspiciously simplifies the Zopeless mailer?
<thumper> wgrant: why yes
<wgrant> I thought that was too easy.
<thumper> wgrant: I found one bug in the sendmail simplification
<thumper> wgrant: and this one
<thumper> wgrant: that's it
<wgrant> Hmm.
<thumper> found it
<thumper> although it is FUCKING INSANE!
<wgrant> Uhoh.
<thumper> about to ec2 land the big branch \o/
<wgrant> sinzui: Did you send my branch off to PQM?
<wgrant> I don't see it yet.
<sinzui> wgrant, ec2. pqm was being a bitch to me
<wgrant> sinzui: Ah, that would do it. Thanks.
<thumper> 'ec2 land' isn't working for me
<thumper> SFTPError: Garbage packet received
<thumper> others getting this?
<wgrant> thumper: Merge devel, apparently.
<wgrant> How would I go about getting a cron.publish-ftpmaster log or two? Do I need to talk to $REAL_SOYUZ_PERSON, or can somebody else see them?
<wgrant> spm, maybe? ^^
<spm> wgrant: getting may be difficult in a policy sense; what are you after in particular?
<wgrant> spm: Well, all the data processed by them is public. But I'm mostly interested in the more detailed timestamps that jpds introduced in 10.03.
<spm> hrm. looks like someone has got shell debug mode on in that script...
<spm> awesome. 2 differeing sets of time/date logging in the same logfile.
<spm> 2010-04-05 23:27:54 DEBUG   Skipping release files for warty/PROPOSED
<spm> Tue, 06 Apr 2010 00:28:00 +0100: Not re-signing /srv/launchpad.net/ubuntu-archive/ubuntu/dists.new/karmic/Release
<wgrant> Yeah. Some is probably from cron.publish-ftpmaster, some will be from publish-distro.py (inside c.p-f), and some from apt-ftparchive (inside p-d)
<spm> fwiw, fyi et al, the first +TZ is preferred over the 2nd.
<wgrant> Yeah, much more sortable.
<wgrant> And readable.
<spm> heh; yes; but more importantly. standardised and accepted. :-)
<spm> Tue, 06 Apr 2010 <== looks quite differnet in a diff lang setting :-)
<wgrant> True.
<lifeless> spm: subunit?
<spm> lifeless: yarp - aware of. haven't look at yet...
<wgrant> spm: Are edge updates coming back at some point soon?
<spm> wgrant: they're actually back atm - I gather they were re-enabled overnight last night our time
<wgrant> spm: Ah, so it should update in a few hours. Thanks.
<spm> 1 hour I believe is when it kicks off - 0800 BST; takes about 45-50 mins to run.
<wgrant> Ah.
<wgrant> Is the S there Standard or Summer?
<lifeless> I think the answer is 'yes'
<lifeless> that is, that BST refers to the normal and daylight savings offsets
<wgrant> \o/
<wgrant> That's about as bad as sites (eg. LP) calling Australian Eastern Standard Time 'EST'.
<spm> wgrant: I don't see the problem with 'EST'? Surely everyone is totally Australian Centric such that no one else would ever confuse us by also using 'EST' !?!?!?
<wgrant> spm: I found a similar case last night while watching the STS-131 launch blog... it quoted times in EST. Now, public NASA times are traditionally given in US Eastern time. But it turns out the website had detected my location and was in fact talking about AEST.
<wgrant> *That* was confusing for a while.
<spm> hahahaha I'll bet!
<adeuring> good morning
<spm> wgrant: edge rollout failed spectacularly. fyi.
<wgrant> spm: YAY
<wgrant> I guess there were some pretty big cullings over night.
<wgrant> (auth DB, and a whole lot of code)
<spm> I believe so. this looks like a configs mess from a Q&D looky
<wgrant> Yay.
<mrevell> Morning
<wgrant> Do we have any Soyuz people today?
<wgrant> If so, #launchpad.
<wgrant> It looks like cron.publish-ppa's process-accepted.py is not running properly.
<wgrant> Or possibly ppa:hakaishi/qshutdown's publish flag is off, but I don't think that's it.
<wgrant> Still no Soyuz people alive?
<james_w> wgrant: jelmer is your best bet today
<wgrant> Well, there is an operational issue. Somebody probably needs to inspect cron.publish-ppa logs and work out why https://edge.launchpad.net/~hakaishi/+archive/qshutdown/+build/1627084 isn't published yet.
<wgrant> (process-accepted.py appears to have skipped it)
<jelmer> wgrant: hi
<wgrant> Morning jelmer.
<wgrant> jelmer: Can you see the relevant log, or does that need a LOSA?
<jelmer> wgrant: I don't think I can see the relevant log, at least I wouldn't know where to look
 * jml sniffles his way into #launchpad-dev
<wgrant> Morning jml.
<jml> actually, I'm calling it quits on today
<jelmer> not feeling well ? :-(
<jml> nope
<leonardr> salgado, who can i direct to look at OOPS-1556K2375? it looks like an OpenID data problem
<salgado> leonardr, that'd be myself
<leonardr> salgado: let me forward you the guy's email
<salgado> leonardr, did you forward it to me?
<leonardr> salgado: i cced you on my reply
<leonardr> i can forward if you didn't get it
<salgado> leonardr, please. I didn't get it
<leonardr> salgado, done
<salgado> leonardr, still haven't gotten it.  to what address are you sending?
<leonardr> salgado: salgado@canonical.com, which is what my address book has for you
<leonardr> is it your full name@canonical?
<salgado> that should work
<leonardr> salgado: should i try your full.name@?
<salgado> leonardr, no need to.  it should reach me at some point
<leonardr> ok
<leonardr> salgado: doh, i never *sent* the email. it was in my outbox
<salgado> heh
<deryck> gary_poster, hi.  Concerning the general problem nature of that ajax-form bug...
<gary_poster> deryck: hey, yeah
<deryck> gary_poster, we need a general solution for history when writing ajax apps.  Perhaps yui3 already provides us with all we need?
<gary_poster> ah.  yeah, it is supposed to, IIRC.
<gary_poster> deryck: mars is out right now, but I'd ask him when he returns.  other YUI3 people could probably confirm as well
<gary_poster> deryck: do you want to push that back to foundations to document the approach, and then I'll push it back to malone for the changes to that particular form?
<gary_poster> ("document the approach" I hope will be a wiki page link to YUI docs somewhere)
<deryck> gary_poster, I don't think we need to do that yet.  I'll follow up with mars to make sure we have everything we need.  we can document as we fix it if so.
<gary_poster> deryck: awesome thank you.  feel free to push back over here if you think it's appropriate though
<deryck> gary_poster, cool.  will do.  thanks.
<gary_poster> ty
<gary_poster> deryck: are you guys the lucky stiffs in charge of hwdb?  If so, did you see my comment to https://bugs.edge.launchpad.net/checkbox/+bug/550973 and are you OK with it?  Are you further ok with me assigning it to malone, or is there something else more appropriate I should do with it?
<mup> Bug #550973: checkbox should send a referer header when it POSTs data to Launchpad. <checkbox:In Progress by cr3> <Launchpad Foundations:New for adeuring> <https://launchpad.net/bugs/550973>
 * deryck looks again....
<deryck> gary_poster, yes, we can target this at malone.  however... are you saying that the exemption for checkbox should *not* be added?  adeuring already has a branch for this.
<gary_poster> deryck: no, I'm saying it *should* be added, permanently.  The only difference, I think, between what adeuring has done and what I'm suggesting is that we should include an explanation of this exception along with the others in the comments for that part of the code.
<gary_poster> I'll clarify that in the bug
<deryck> ah, gotcha.
<deryck> gary_poster, super.  So definitely re-target to us, too.
<gary_poster> deryck: ack thank you will do
<deryck> adeuring, since you've already started work on the above bug 550973, we need to get a card on the board for it, too.
<mup> Bug #550973: checkbox should send a referer header when it POSTs data to Launchpad. <checkbox:In Progress by cr3> <Launchpad Foundations:New for adeuring> <https://launchpad.net/bugs/550973>
<dhastha_> danilos, I've got an error Unable to install launchpad-developer-dependencies after ./rocketfuel-setup
<danilos> dhastha_, you need to run it with sudo, have you done that?
<adeuring> deryck: done. Though I did not rally start any work on that bug.
<danilos> dhastha_, btw, you are using Ubuntu, right? 9.10?
<deryck> adeuring, ah, ok.  I saw a linked branch.
<deryck> adeuring, simple to fix, though, right?
<dhastha_> danilos, no am using 10.04 beta version
<adeuring> deryck: well.... I am not how exactly the "precise specs" should look like... Sure, I can add the referer header stuff, but I am not sure if gary_poster has anything else in his mind.
<adeuring> deryck: also; i wonder if should alternatively switch the submission HWDB report to the webservice API
<dhastha_> danilos, no problem i installed with sudo apt-get
<danilos> dhastha_, right, so, can you check if you have Launchpad PPA in your "Software Sources" (go to System -> Administration menu)
<adeuring> that way, we would use a alraedy more os less well defined interface
<danilos> dhastha_, right, then that part should be fine, you should be able to re-run rocketfuel-setup and it should complete everything it needs to do
<dhastha_> while re run rocketfuel-setup   downloadings are going there
<gary_poster> adeuring, deryck: I tried to clarify my suggestion in https://bugs.edge.launchpad.net/malone/+bug/550973 .  I'm vaguely +0 on switching to the webservice, but that's not up to me.
<mup> Bug #550973: checkbox should send a referer header when it POSTs data to Launchpad. <checkbox:In Progress by cr3> <Launchpad Foundations:Invalid by adeuring> <Launchpad Bugs:New for adeuring> <https://launchpad.net/bugs/550973>
<deryck> gary_poster, ack.  thanks
<adeuring> gary_poster: ah, right... I added such a cimment only the test...
<adeuring> to the test..
<gary_poster> adeuring: cool.  yeah, I suppose you could refer to the test instead.  I just want to be able to read the code and know what's going on from there, directly or indirectly
<adeuring> gary_poster: sure. I'll add such a comment. We can't get rid of this sort of special handling of the +hwdb/+submit URL for at least one year in order to support older HWDB client versions, so an API method wouldn't make this comment obsolete
<gary_poster> adeuring: cool
<salgado> leonardr, still haven't gotten that email.  has it gone out this time?
<leonardr> salgado: i guess not... my email client is acting up
<leonardr> salgado: ok, now it's sent for sure
<salgado> leonardr, cool, thanks
<rockstar> bac, my branch landed yesterday about time for your EOD.  I thought I should just let you know.
<bac> rockstar: i did see it and my follow up branch landed as well
<bac> thanks for the note
<rockstar> bac, niiiice</borat>
<rockstar> sinzui, can you explain to what's going on with the stylesheets currently?  I don't see a style-3-0.css but I do see a style-3-0.css.in which I would assume buildout would use to make the stylesheet.
<sinzui> EdwinGrubbs, modified it so that it can be regenerated when the sprites change
<sinzui> rockstar, ^
<rockstar> sinzui, so I need to edit the .in file and re-run make?
<EdwinGrubbs> rockstar: yes, you can just run "make combine-css".
<sinzui> rockstar, It is the first thing I would try. combine css may be the real command since that is how I get my changes uncached
<rockstar> EdwinGrubbs, I think I remember reviewing this, but now I see it makes me a sad panda.
* flacoste changed the topic of #launchpad-dev to: PPAs are not being published: working on a fix | Launchpad Development Channel | Week 0 of 10.04 | PQM is open | https://dev.launchpad.net/ | Get the code: https://dev.launchpad.net/Getting | On-call review in irc://irc.freenode.net/#launchpad-reviews | Use http://paste.ubuntu.com/ for pastes
<EdwinGrubbs> rockstar: if you are just testing changes and don't want to run make after each edit, you could edit lib/c/l/icing/build/combo.css directly, which is what combine-css creates.
<rockstar> EdwinGrubbs, well, it's not really that bad.  I just use Firebug a lot more before I edit the stylesheet.
<rockstar> EdwinGrubbs, I'm just bitching because I'm a bitch.  :)
<deryck> kfogel, ping
<kfogel> deryck: hey
<kfogel> deryck: what's up?
<deryck> kfogel, you've got two branches hanging 'round the kanban board.  Did these land yet?  I can move them if so.
<kfogel> deryck: hunh, sorry.  let me take a look
<kfogel> deryck: moved them to landed lane; let me see if they're live yet
<kfogel> deryck: so at least the fix in bug #538219 does not appear to be live on edge yet.  Is that expected?
<deryck> kfogel, I think we've got some issues updating staging and edge after rollout.  So yes, not unexpected.
<kfogel> deryck: that's the one where a file ending in ".debdiff" was not believed to be a patch by Launchpad, forcing the user to confirm it.
<kfogel> deryck: ok, won't worry then.  For now, they're in landed lane; I can QA them both when edge or staging is updated.
<deryck> kfogel, cool, sounds good.  I thought they were landed but didn't want to move them without asking.
<leonardr> barry, do you know much about python pickling?
<maxb> I seem to remember there being some public documentation for what vcs-imports reviewers are looking for when reviewing, but I can't find it. Can anyone point me?
<kfogel> maxb: besides https://help.launchpad.net/VcsImports ?
<maxb> kfogel: I seem to remember reading something more reviewer-oriented. I think it even mentioned not doing vcs-imports from sourceforge?
<kfogel> maxb: hunh, no that I don't remember
<kfogel> I mean, we check to make sure the source repository is real and that it's trunk not branch (if svn).
<maxb> https://dev.launchpad.net/ReviewingCodeImports
<kfogel> Not sure why we'd leave out sourceforge... I think we may have suspended for a time due to some kind of perf issue on our end or their end, but I thought that got fixed.
<maxb> that's what I was after
<kfogel> maxb: ah, yes -- and it's apache.org, and the issue is apparently still current
<bdmurray> bac: just to confirm you are going test / land those branches right?
<bac> bdmurray: yes, i will
<bdmurray> bac: cool, thanks!
<barry> leonardr-lunch: hi.  still need to know about pickling?  yeah, i know something about it :)
<leonardr> barry: thanks, but the problem is now moot
<barry> leonardr: cool
<mars> rockstar, ping, do you have a moment to help test something on launchpad.dev in FF3.6 Lucid?
<rockstar> mars, I can as soon as I'm off the phone here (and possibly picked up my wife, depending on how long this call goes)
<mars> rockstar, ok, thanks.
<rockstar> mars, is it something you can give me details to and I can just fire the deferred when its done?
<mars> rockstar, simple: try the "Choose..." link on https://bugs.launchpad.dev/jokosher/+bug/12/+addbranch, using trunk/.  It should open a person picker.  For me, it does not.
<mup> Bug #12: "Next 10 messages" changes Display Settings <Launchpad Translations:Fix Released by daf> <https://launchpad.net/bugs/12>
<mars> ^ That is a DailyWTF bug title if I ever saw one
* flacoste changed the topic of #launchpad-dev to: Launchpad Development Channel | Week 0 of 10.04 | PQM is open | https://dev.launchpad.net/ | Get the code: https://dev.launchpad.net/Getting | On-call review in irc://irc.freenode.net/#launchpad-reviews | Use http://paste.ubuntu.com/ for pastes
<ubuntujenkins> hello i am trying to get the launchpad code as per https://dev.launchpad.net/Getting . how ever I am being asked to install the texlive packages. I have the texlive as per http://www.tug.org/texlive/acquire-netinstall.html which is required by the ubuntu-manual project. will launchpad work with this version of tex live?
<ubuntujenkins> its not the texlive packages
<mars> ubuntujenkins, assuming that there is no massive difference between the tug version and the main repositories, then I would think it should work.  Personally, I would just try it and see :)
<ubuntujenkins> mars:  the tug version is better than the repo it has more of tex live in it but how do i stop it installing by the script?
<mars> ubuntujenkins, that I do not know.  Someone who knows apt well would be able to tell you.
<mars> I've never tried to install something with apt while forcing it to skipping a package dependency.
<ubuntujenkins> the script only asks for "launchpad-developer-dependencies apache2 apache2-mpm-worker" but launchpad-developer....... depends on tex live
<ubuntujenkins> thanks mars I will see if anyone knows
<mwhudson> that seems a bit odd, for launchpad to depend on texlive
<ubuntujenkins> I am trying installing it in synaptic I unmarked the texlive stuff before i did the install
<mars> mwhudson, I think it is a dependency of a dependency
<ubuntujenkins> I think so as well
<mars> ubuntujenkins, found one path: use equivs to create a dummy package: http://osdir.com/ml/ubuntu-users/2010-02/msg02052.html and http://ubuntuforums.org/showthread.php?t=726317
<mars> anyway, I'll wait and see what an expert says :)
<ubuntujenkins> thanks mars I am getting somewhere but we will see when it is finished
<wgrant> ubuntujenkins: Why does the manual project not use the system LaTeX?
<wgrant> That's pretty hostile to development.
<mars> or at least, why haven't they packaged the TUG version into a PPA or something
<ubuntujenkins> I am not 100% sure but there isn't enough in the packages for our stuff to work. We have a great guy working on the laytex stuff.
<ubuntujenkins> thats a though mars
<wgrant> ubuntujenkins: Has a bug been filed?
<ubuntujenkins> I don't know wgrant i will ask
<wgrant> If you can't develop a manual for Ubuntu without using non-packaged LaTeX, one of us has a problem that should be fixed.
<ubuntujenkins> we are using 2009 which is only packaged in lucid, but you are right it should be fixed
<ubuntujenkins> apparently there isn't a bug that has been filled, as he is not sure on what the exact stauts of the packages is. They fail to work for our needs and are about 1gb smaller is size
<mars> ubuntujenkins, that should just be a backport then, not a big deal from what I hear.  Especially if you use a PPA to build for karmic.
<ubuntujenkins> mars:  I have just realised a ppa for text live is harder than, you may think there is a script included that means you can install stuff with out using synaptic etc
<ubuntujenkins> how big is the launchpad source these days?
<mars> ~/canonical/lp-branches/fix-ie-caching$ du -sh .
<mars> 156M	.
<lifeless> including pyc files
<lifeless> :P
 * mars *shrug*
<wgrant> Plus all the deps.
<mars> does du follow symlinks?
<ubuntujenkins> thanks
<lifeless> no
<ubuntujenkins> I have no clue how i am going to understand all of this stuff
<lifeless> follow the guide :P
<mars> ubuntujenkins, "this stuff" is pretty broad :)
<mars> If you listed all of the topic areas that a software engineer experienced with this particular project was functional in, I wonder how many you would get
<ubuntujenkins> once i get it set up i will start poking around. I need to work out which bits i need to look at, and understand it all in general
<mars> ubuntujenkins, when you know what you want to fix, give us a shout in this channel.  We can show you a path to get it done.
<mars> And trust me, you will learn a lot just fixing one tiny little bug.
<ubuntujenkins> thanks mars i think i will be, i have been asked to look into adding a feature to rosseta/launchpad translations. I am sure i will learn a lot. I started a manual sub project as a "tester" and now I am one of the devels
<ubuntujenkins> it is to do with how launchpad deals with fuzzy translations, it was removed a while ago but. the team would like it improved and hopefully implemented as an option for translations. I think thats it anyway
<ubuntujenkins> this is what i was told "It'd not just a matter of allowing fuzzy translations.  We also want to highlight the differences between the original English string and the current English string and the original (fuzzy match) translation, so the translators can easily see what's changed and adjust the translation accordingly."
<mars> ubuntujenkins, have you thought about sending a mail to the lp-dev list, and CC'ing the rosetta team?
<ubuntujenkins> not yet i will do, when i have some more time and a bit of an understanding of launchpad i am rather clueless at the moment.
#launchpad-dev 2010-04-07
<ubuntujenkins> thanks for your help guys, night
<mars> mwhudson or thumper, ping
<mwhudson> mars: i
<mwhudson> hi
<mars> hi mwhudson.  Would you happen to have a moment to help debug a failing test I have?
<mars> I need someone else to verify that it fails on their system
<mwhudson> mars: sure
<mars> mwhudson, thanks.  Could you please tell me if this fails:
<mars> mwhudson, xvfb-run -s '-screen 0 1024x768x16' bin/test -t test_newly_linked_branch_diff_popup
<mwhudson> mars: hang on, updating devel
<mars> I assume you have xvfb installed.  If not, then just drop that part.
<mwhudson> mars: ok, running
<mwhudson> mars: bah, need to run make schema
<mars> boo
<mars> that is really annoying :)
<mars> I wonder if it could be made to fail earlier
<mars> 'make check_schema'
<mars> I've had that happen many times
<mars> confirm_dbrevision_on_startup() seems to do all the work to raise a database revision error.  How to get that function into a script...
<mwhudson> mars: anyway, that test passes for me
<mars> mwhudson, perfect, thanks for the help.
 * wgrant wonders why make clean decides to obliterate a whole lot of data directories.
<mwhudson> lunchtime
<dhastha> need help: error in make schema  Missing ./download-cache.
<dhastha> Developers: please run utilities/link-external-sourcecode.
<dhastha> make: *** [download-cache] Error 1
<dhastha> i got this error while building a pristine trunk (devel) instance
<spm> dhastha: a pristine trunk doesn't have all the external code that is needed. per the message, (i guess...) run utilities/link-external-sourcecode
<dhastha> i am getting error while run utilities/link-external-sourcecode
<dhastha> spm, Invalid line at line "4".
<dhastha> Invalid line at line "7".
<dhastha> bzrlib.errors.ParseConfigError: Error(s) parsing config file /home/dhastha/.bazaar/locations.conf:
<dhastha> spm, how to rectify those errors?
<spm> dhastha: I'd suggest you look at the file suggested and see why that appears broken as a 1st step. ?
<thumper> dhastha: have you followed the setup instructions on the dev wiki?
<mwhudson> heh
<mwhudson> i can tell the uk is in summer time now, i'm getting mails from the future about bugs
<wgrant> That's still not fixed?
<wgrant> Bug #262040
<mup> Bug #262040: Bogus timestamps on some unbatched bugmail <email> <Launchpad Bugs:Triaged> <https://launchpad.net/bugs/262040>
<mwhudson> hey look i commented on that bug roughly a year ago
<mwhudson> what are the chances
<wgrant> Heh heh.
<wgrant> spm: Has the PPA process-accepted.py gone and died again?
<spm> i hope not...
<wgrant> spm: Ah, no, just very late.
<spm> where does that one run again? soyuz is a mini nightmare of odd corners and crontabs...
<wgrant> It's germanium. Some of my binaries appeared 5 minutes late, but they did appear.
<wgrant> cron.publish-ppa on germanium.
<spm> bleh. that'd be right. i was on cocoplum and cesium.
<wgrant> Heh.
<spm> now if only i could find which of the 6 users it runs under... /me gives up and looks for a logfile for clues
<wgrant> lp_publish
<spm> is that called by something else then? cronscripts/publishing/cron.publish-ppa cronscripts/publishing/cron.daily-ppa scripts/process-death-row.py and keys and htaccess
<spm> are the only tasks of note that run
<wgrant> cron.publish-ppa calls process-accepted.py.
<spm> ahh there it is. the 1st. yup
<spm> one of the "possibilities" for our sprint coming up; is to drag eg Julien by the ear and nail him to the floor until he completely explains the entire soyuz "WTF" and how it "works".
<wgrant> Hahaha.
<wgrant> '"works"' is right.
<spm> with diagrams, if we're feeling really vindictive
<wgrant> https://wiki.ubuntu.com/CelsoProvidelo/SoyuzInfrastructureOverview but worse :P
<mwhudson> there are already diagrams!
<mwhudson> some of them may even be accurate
 * spm is NOT game to click on that link before breaking for lunch. maybe after. and after I've digested. but not before.
<lifeless> wgrant: what was the oops that you needed looked at in the weekend ?
<wgrant> spm: Refreshing https://edge.launchpad.net/~wgrant/+archive/openttd/+build/1654520 regularly shows that shipova is being dodgy and keeps dropping the build (note the fluctuating start time). Can you marks shipova not-OK or something? I've seen others complaining about this one over the past couple of weeks too.
<wgrant> lifeless: Good point.
<spm> wgrant: not sure I can; one sec...
<wgrant> spm: https://launchpad.net/builders/shipova/+edit; uncheck the OK checkbox; save. unless you mean policy-wise.
<spm> policy/perms wise; looks like I can. see if that stops it
<spm> and is now showing disabled with a nice message as to why :-)
<wgrant> Great. And my build is going OK.
<wgrant> Thanks.
<spm> \o/
<wgrant> I knew there was a dodgy builder around, but hadn't managed to work out which.
<wgrant> spm: Also, do you know why cocoplum OOPSes aren't syncing to the OOPS viewer thingy?
<wgrant> There are some interesting cocoplum OOPSes which cannot be viewed.
<spm> wgrant: I'ev just done a full manual sync - see if that works?
 * spm afks for coffee... brb
<mwhudson> thumper: were you asking for http://pythonpaste.org/webob/ earlier?
<thumper> yeah, but found it elsewhere, thanks
<mwhudson> :)
<thumper> mwhudson: do you know if it has taken over as the prime development from paste?
<mwhudson> thumper: i think so but i don't know so
<mwhudson> it's only a replacement for bits of paste aiui
 * wgrant is interested in OOPS-1553FTPMASTER11 and OOPS-1553FTPMASTER16
<wgrant> Maybe they will exist now.
<mwhudson> seems not
<wgrant> Grrr.
 * spm has a look on the filesystem...
<spm> wgrant: when were they generated?
<spm> roughly?
<spm> actually - roughly to "which day" would work.... :-/
<mwhudson> i can't even see how you'd get an oops prefix like that
<wgrant> Actually, now I think about it...
<spm> because for the days of the 5th, 6th and 7th of April, there is only 1 oops. 81309.FTPMASTER1
<wgrant> While the OOPS prefix was FTPMASTER...
<wgrant> The log was from a process that runs on cesium.
<spm> Ah ha!
 * wgrant gives the production configs an A+ in confusion.
<wgrant> It was Friday or so, IIRC.
<spm> :-)
<spm> I think I can see why the oops *may* not show up. drwx------ 6 lp_buildd  lp_buildd     4096 2010-04-02 14:28 lp_queue/ <== oops dir on cesium
<wgrant> Heh.
<spm> bingo. there they are.
<spm> all their perms are crap too. /me envisages another bug report for tdoay
<wgrant> lalalala that's not Soyuz any more lalalala
<mwhudson> funky umask?
<spm> not likely - we've seen too many cases of these shenanigans before acorss all parts of LP to blame umasks; but I will check.
<spm> btw, have we ever mentioned how much we hate the twisted 1Mb log rotation evilness? <== grrrrr.
<spm> wgrant: try now?
<wgrant> spm: I don't actually have access to OOPSes.
<spm> hahaha. thought you did :-)
<spm> ad it still doesn't show. that may need a diogo fix
<wgrant> :(
<spm> authentication failed for user "uploader"
<spm> DB error - I'd guess it's been fixed since, as there are no more since then. Date: 2010-04-02T13:38:29.286748+00:00
<mwhudson> wgrant: OperationalError: FATAL:  Ident authentication failed for user "uploader"
<wgrant> Ah, great. Thanks.
<wgrant> I didn't think it had happened since, but there appeared to be quite a few that day.
<spm> post rollout; and I'd suspect no one informed us of the new DB user in advance. Or we missed it. :-)
<spm> wgrant: what *&^%*&ing process/script generated those oops? I'm guessing cronscripts/buildd-queue-builder.py ?
<wgrant> spm: buildd-manager.
<wgrant> buildd-queue-builder is one of four scripts which do the job that buildd-manager does.
<spm> Oh ho ho. the daemon itself.
<wgrant> But it is way obsolete.
<wgrant> Oh, wait.
<wgrant> No. process-upload.py, launched by buildd-manager.
<wgrant> It will soon be buildd-manager itself, but not yet.
<spm> ahh; kk. thanks!
<spm> I think I will stop trying for accuracy or precision where soyuz bugs are concerned and just go with "it's broken. pls fix" reports.
<thumper> testfix?
<thumper> grr
<mwhudson> spm: buildbot kickage pls
<thumper> mwhudson: it says wrong python used
<thumper> mwhudson: do you know what is going on?
<spm> damn. new there was a reason I wanted to stay logged onto prasÃ©
<mwhudson> thumper: yes
<spm> mwhudson: kicked
<mwhudson> thumper: buildbot is fixed, it needs a restart and a force
<thumper> spm: kicked or restarted?
<spm> restarted type of kicking
<thumper> spm: thanks
<thumper> mwhudson: my big branch passed all tests, and is on its way to landing on db-devel
<mwhudson> thumper: woo
<thumper> oh ffs, pqm still thinks it needs testfix
<thumper> how do I smack it around?
<spm> sudo spm fix! <== probably
<thumper> sudo spm fix plz
<spm> plz. command not recognised
<spm> thumper: ?? waterfall looks good? or are you referring to db_devel?
<thumper> yes
<mwhudson> it looks like buildbot-poll should be moving out of testfix
<spm> ah yup. we're in failure according to the pqm configs
<mwhudson> thumper: when did you get the failure from pqm?
<mwhudson> oh
<mwhudson> silly pqm configs
<thumper> mwhudson: after it was booted
<mwhudson> spm: any log files to say why?
<spm> that hsould sort itself every 5 mins. hrm. wonder if just forcing a build should clear it?
<spm> Starting buildbot-poll at Wed Apr  7 05:34:35 BST 2010
<spm> Builder lp at http://lpbuildbot.canonical.com:8010 failed building r10635 [failed shell_7].
<spm> Builder db_lp at http://lpbuildbot.canonical.com:8010 succeeded building r9202.
<spm> so i read that as devel is the cause; and a build is underway to rectify that as we speak. ???
<mwhudson> oh right
<mwhudson> that sort of makes sense, the tip of devel did fail to build
<spm> yup
<mwhudson> i wonder if there's a way to fake it
<thumper> so I have to wait then?
<mwhudson> we could kill the current build and then force a build
<thumper> meh
 * thumper goes to guitar lesson
<spm> well the script that generated the above is run every 5 and is more or less getting it's status from the 'last build - I believe. and setting the failure/success mode accordingly.
<cody-somerville> wgrant, Did you say that process-accepted is dying again?
<wgrant> cody-somerville: No. It was just being slow.
<cody-somerville> hmmm... yea, I see at the tail of the cron.ppa.log that it failed to grab the lockfile several times.
<wgrant> What was it doing in the previous runs that took so long?
<wgrant> p-a normally takes <20s, AFAICT.
<wgrant> I guess a couple of big PPA index regenerations could easily throw p-d over 5 minutes, but I don't know of any PPAs that big.
<cody-somerville> looks like a bunch of nightly builds.
<wgrant> They must be huge.
<wgrant> Even chromium was only a couple of hundred MiB last time I looked.
<cody-somerville> Looks like there is a bunch of PoolFileOverwriteError exceptions that cause numerous PPAs to be processed each run.
<cody-somerville> And the fact that the publisher seems to publish suites and pockets that aren't even dirty probably doesn't help.
<wgrant> Bug #387049
<mup> Bug #387049: Copy backend does not detect file conflicts <ppa> <Soyuz:In Progress by stevenk> <https://launchpad.net/bugs/387049>
<wgrant> They must be dirty -- maybe they have pending deletions?
<wgrant> Domination is probably also significantly slower than it needs to be, but I have a simple fix for that.
<wgrant> Source domination, that is.
<cody-somerville> Looking here, an entire second per PPA is eaten processing suites and pockets that don't exist in PPAs at all and/or probably don't exist in that specific PPA.
<wgrant> In which step? Domination?
<cody-somerville> Step A: Publishing
<wgrant> Hmm.
<wgrant> Bad queries must be bad.
<cody-somerville> I see here a PPA that had nothing to be published and it took 2 seconds just for the publishing step.
<wgrant> I've almost fixed B and C', but I've not looked at A yet.
<wgrant> I wasn't aware that it took long at all.
<cody-somerville> On average, it looks like it takes half a second per file that actually has to be published plus a static 2-3 second overhead.
<wgrant> I know the indices on the publishing tables are stupid.
<wgrant> Maybe most of that overhead is just initial query time.
<underdrk> morning
<underdrk> is it normal for rocketfuel-setup to use mor than ~300 meg ram?
<underdrk> its eating trough all the ram on my virtualbox system getting killed eventually
<wgrant> underdrk: That's probably bzr.
<wgrant> noodles775: Morning.
<underdrk> wgrant: yup
<wgrant> noodles775: I need Soyuz acks on four MPs, when you have a moment.
<underdrk> so, what is bzr doing that it needs ~300m ram?
<wgrant> underdrk: Not sure.
<underdrk> shoudn't it just download and store stuff on disk?
<noodles775> Hey wgrant, jkust on the phone. Will look in a tick.
<wgrant> underdrk: Probably. But the LP branches are a little ancient and special, so may be causing some interesting behaviour.
<wgrant> noodles775: No urgency. Thanks!
<underdrk> wgrant: hmm, thats a pitty
<underdrk> ill see if I can make it download an a machine with more ram and then scp it into my virtualbox
<wgrant> underdrk: Your VM has only a few hundred megabytes of RAM assigned?
<spm> assign some more swap and see if that helps it continue; vs dying. hopefully without io thrashing as well. ???
<wgrant> I don't like your chances of running much of LP with <1GB on i386, or much <2GB on amd64.
<spm> heh; given I just killed a 4Gb of memory importd that was running away a tad; yes; agreed :-)
<wgrant> spm: Linux as usual?
<spm> ruby actually; but not a bad guess.
<wgrant> Ah.
<spm> we have 3 nasties: linux kernel; gcc and ruby.
<wgrant> gcc shouldn't be so bad since incremental bzr-svn was deployed a week ago, right?
<wgrant> Ah, yes, it's even finished its initial import now.
<spm> oh cool. was just updating the related bug report; and noted that gcc was the 3rd of the triumvirate of pain, so :-)
<adeuring> good morning
<simon-o> hi :-)
<wgrant> Hm: http://forum.biosinteractive.com/viewtopic.php?f=5&t=7&p=28#p28
<wgrant> People are not happy with Launchpad.
<underdrk> wgrant: for a development platform yes, I'd hope half a gig would be enough
<underdrk> but, ill mount some more swap
<wgrant> jelmer: Morning.
<wgrant> noodles775: Thanks for those.
<noodles775> wgrant: thank you :)
<wgrant> buildd-manager is a whole lot more understandable after that lot.
<noodles775> Yeah, it's great seeing the code continually improved!
<wgrant> So, I have a branch that removes 9500 lines of unused code.
<noodles775> !
<wgrant> That seems to violate the 800 line rule by a little bit.
<mrevell> Hi
<james_w> how do I find out if PQM is in textfix?
<dhastha> need help: error occuring while running ./utilities/link-external-sourcecode ~/launchpad/lp-sourcedeps
<dhastha> Wanted to link /home/dhastha/launchpad/lp-sourcedeps/download-cache to ./download-cache but source does not exist
<jelmer> james_w: if there's an email to the mailing list with a test failure and pqm rejects your requests, while its error message mentions requiring [testfix]
<jelmer> dhastha: hi
<dhastha> jelmer, hi error while installing launchpad locally
<jelmer> dhastha: how did you create lp-sourcedeps?
<james_w> jelmer: yeah, I got rejections, I want to know if it's safe to submit again yet.
<dhastha> jelmer, after run rocketfuel-setup
<wgrant> Odd. buildbot merged 20 minutes ago.
<wgrant> Which suggests that thins weren't too unhealthy.
<jelmer> dhastha: perhaps rocketfuel-get fixes it?
<dhastha> jelmer, after run rocketfuel-get i got lp-branches and lp-soucedeps files automatically
<dhastha> jelmer, sorry i run rocketfuel-setup only, i don know about rocketfuel-get
<james_w> wgrant: so it's probably ok now?
<james_w> I got rejected some hours ago
<jelmer> dhastha: did rocketfuel-setup interrupt halfway through?
<wgrant> james_w: Ah, if it was that long ago, then go for it.
<james_w> ok
<james_w> anyone know how to land skipping ec2?
<james_w> I just did "ec2 land", but it seems wasteful to run the tests again?
<dhastha> jelmer, no error in rocketfuel-setup
<jelmer> james_w: use "bzr lp-land" ?
<dhastha> jelmer, there is no error in database-setup also
<mwhudson> james_w: you can't tell if we're in testfix, it really sucks
<mwhudson> (there's a bug report, but i don't have any ideas on fixing it)
<dhastha> jelmer, error starting in  make schema
<james_w> what's PQM's address for LP?
<james_w> (I have searched dev.launchpad.net for all this, but I couldn't find it)
<StevenK> james_w: Heh heh, I had exactly the same problem about a week ago.
<jelmer> pqm_email = Canonical PQM <launchpad@pqm.canonical.com>
<StevenK> Bah, jelmer! I was about to paste that
<dhastha> jelmer, download-cache folder is not available in my lp-sourcedeps folder so only error occuring. isn't it?
<jelmer> dhastha: yeah, I wonder what went wrong there. What happens if you run rocketfuel-setup again?
<jelmer> dhastha: it should be taking care of creating lp-sourcedeps
<wgrant> dhastha: Does lp-sourcedeps contain anything at all?
<wgrant> If it doesn't contain any significant volume of data, remove it and run rocketfuel-setup again.
<dhastha> jelmer, eggs, sourcecode two folders only are there
<dhastha> wgrant, k
<danilos> dhastha, hi, is wgrant's suggestion helping?
<dhastha> danilos, i removed all the datas and trying once again
<danilos> dhastha, cool
<deryck> Morning, all.
<persia> LP rollouts are the third thursday of the month, right?
<wgrant> It depends.
<wgrant> This is a 5 week cycle (hence this is week 0)
<wgrant> persia: See https://dev.launchpad.net/Releases/2010Calendar
<wgrant> (no, there is not a release the day before Lucid)
<persia> Oh, perfect.  THanks.
<thumper> oh, we are in a week 0?
<thumper> groovey
 * wgrant points at /topic
<wgrant> (although I set it)
<dhastha> danilos, Permission denied (publickey).
<dhastha> bzr: ERROR: Connection closed: Unexpected end of message. Please check connectivity and permissions, and report a bug if problems persist.
<dhastha> danilos, same error
<danilos> dhastha, what do you have in ~/launchpad? and what user did you run rocketfuel-setup under?
<wgrant> dhastha: You don't have your SSH key registered with Launchpad?
<danilos> wgrant, he does
<wgrant> dhastha: What does 'bzr launchpad-login' say?
<dhastha> rocketfuel-setup file only i am having in ~/launchpad. while i run i am getting that error. but am having ssh key registered
<danilos> dhastha, perhaps it's related with the keyserver, can you try changing it as suggested on https://dev.launchpad.net/Getting:
<danilos>     ## Sometimes the keyserver from Ubuntu doesn't respond, especially on Karmic
<danilos>     ## Try changing keyserver.ubuntu. to pool.sks-keyservers.net
<wgrant> danilos: That's for OpenPGP keys.
<wgrant> This is an SSH key problem.
<danilos> wgrant, right, dumb me
<danilos> dhastha, ok, so the only thing I can suggest is to make sure whatever environment bzr gets executed in, it has SSH_AGENT env variables properly set (i.e. don't run it with sudo options where it unsets some of these "unsafe" variables)
<dhastha> danilos, making local branch of launchpad trunk. bzr: ERROR: [Errno 13] Permission denied: '/home/dhastha/.bazaar/bazaar.conf'
<dhastha> danilos, finding revision going on
<danilos> dhastha, ok, you'll probably have to revert those privileges to your own user using something like "sudo chown -r dhastha.dhastha /home/dhastha/.bazaar"
<maxb> And be sure not to run rocketfuel-setup under sudo in the future. that's not right
<danilos> that was my fault, I suggested it might be needed
<wgrant> jtv: Is your soyuz-sampledata-setup failure fixed by my sampledata reversion yesterday?
<wgrant> jtv: Its output was accidentally added to the sampledata a couple of weeks ago.
<jtv> wgrant: that's what I was hoping for yesterday, but unless I was less up-to-date than I thought, it doesn't seem to help.
<jtv> wgrant: oh wait, did you end up landing that on db-devel or not?
<wgrant> jtv: It landed on db-devel a couple of days ago, and on devel a little over 24 hours ago.
<wgrant> devel r10627
<wgrant> db-devel r9194
<jtv> wgrant: it may be a "missing link" in how the revisions flowed on this system... I'll try again.
<deryck> gmb, I think we should make Bug #557252 a priority.  Seems a simple fix with many gains.  Would you agree?
<mup> Bug #557252: Creating CalculateBugHeatJob is very slow <story-bug-heat> <Launchpad Bugs:Triaged> <https://launchpad.net/bugs/557252>
<gmb> deryck, I just triaged it as medium :). But if we've got someone to spare to fix it then yes, we should deal with it, if only to make garbo-hourly happier
<deryck> gmb, yeah, that sounds fine.
 * wgrant loves bugmail from 'Tomorrow 00:32'
<kfogel> wgrant: http://fts.ifac.cnr.it/cgi-bin/dwww/usr/share/doc/emacs21-common/etc/future-bug
<wgrant> kfogel: Heh.
<wgrant> (Bug #262040 is the problem.)
<mup> Bug #262040: Bogus timestamps on some unbatched bugmail <email> <Launchpad Bugs:Triaged> <https://launchpad.net/bugs/262040>
<bac> reviewers meeting starting in one minute
<dhastha> danilos, are you there?
<kfogel> Anyone know why PQM is suddenly being very coy about showing details?  http://people.canonical.com/~kfogel/images/pqm-opacity.png
<danilos> dhastha, yeah
<kfogel> danilos (or anyone): know why PQM is suddenly being very coy about showing details?  http://people.canonical.com/~kfogel/images/pqm-opacity.png
<kfogel> ?
<danilos> kfogel: hum, probably submissions for non-LP branches, or something is weird with the PQM configuration
<kfogel> danilos: no, it's been like this for a while, and I know I had a submission in the queue then.  I think the configuration is messed up.  What's the best way to report that: RT, or file a bug?
<danilos> kfogel: it's probably best to check with LOSAs
<kfogel> danilos: will do
<danilos> kfogel: thanks :)
<deryck> adeuring, Bug 550973 is really just about adding the referrer header exception and a good comment.  So I moved it to easy bugs backlog.
<mup> Bug #550973: checkbox should send a referer header when it POSTs data to Launchpad. <checkbox:Fix Committed by cr3> <Launchpad Foundations:Invalid by adeuring> <Launchpad Bugs:In Progress by adeuring> <https://launchpad.net/bugs/550973>
<adeuring> deryck: OK, I know. The exception already exists, BTW
<deryck> adeuring, oh, hmmm.  So it's landed?
<adeuring> deryck: yes, as an RC branch, IIRC. But the comment that gary wants is still missing.
<adeuring> deryck: at least LP receives and prcesses again HWDB submissions
<gary_poster> yeppers
<deryck> adeuring, ok.  so even more easy then :-)
<adeuring> deryck: I know; I just wnat to finish this mantis related bug. Otherwsie, I'll never finish that branch, I'm afaid. After that, I'll add this comment
<deryck> adeuring, oh, yeah, I didn't mean to imply it should be done now.  Just noting that I moved it to a different queue.
<adeuring> deryck: OK ;)
<jml> hello again
<dhastha> daniloff, are u there?
 * maxb imagines that the "off" suffix means no
<dhastha> need help: error occuring while run launchpad locally.     make run
<maxb> dhastha: that is not much detail for anyone to be able to help you with. Please pastebin the error
<dhastha> maxb, http://paste.ubuntu.com/410637/
<maxb> Hmm, I guess your rocketfuel-setup never finished successfully?
<maxb> Do you have a /home/dhastha/launchpad/lp-sourcedeps/download-cache/dist/setuptools-0.6c9-py2.5.egg ?
<dhastha> maxb, finished successfully. no error found
<dhastha> maxb, upto download-cache only am having. there is no dist as well as setuptools.0.
<maxb> What do you see if you run 'bzr info' in the download-cache dir?
<dhastha> Bound branch (format: pack-0.92)
<dhastha> Location:
<dhastha>       branch root: .
<dhastha>   bound to branch: bzr+ssh://bazaar.launchpad.net/~launchpad/lp-source-dependencies/trunk/
<maxb> And 'bzr status' ?
<dhastha> bzr: ERROR: No WorkingTree exists for "file:///home/dhastha/launchpad/lp-sourcedeps/download-cache/.bzr/checkout/".
<dhastha> dhastha@dhastha-desktop:~/launchpad/lp-branches/devel/download-cache$ bzr statusbzr: ERROR: No WorkingTree exists for "file:///home/dhastha/launchpad/lp-sourcedeps/download-cache/.bzr/checkout/".
<maxb> OK, run 'bzr checkout' and it should create a dist/ directory
<dhastha> bzr: ERROR: File exists: u'/home/dhastha/launchpad/lp-sourcedeps/download-cache/.bzr': [Errno 17] File exists: '/home/dhastha/launchpad/lp-sourcedeps/download-cache/.bzr'
<maxb> Hrm
<dhastha> maxb, bzr checkout  returns above error
<maxb> Ok, try this: 'bzr unbind; bzr checkout; bzr bind lp:lp-source-dependencies'
<dhastha> same above error
<dhastha> maxb: rocketfuel-setup had taken somuch of time to complete
<maxb> dhastha: How fast is your internet connection? One option would be to just throw away download-cache and re-fetch it
<dhastha> maxb, 60kb per sec. How to refetch that particular file only?
<maxb> Unfortunately, that branch is huge
<maxb> dhastha: Could you please run 'ls .bzr' in download-cache?
<dhastha> branch  branch-format  branch-lock  checkout  README  repository
<maxb> OK, 'rm -rf .bzr/checkout' and then try 'bzr checkout' again
<dhastha> no error found
<maxb> Great, do you now have dist/ and cmmi/ directories in download-cache?
<maxb> Also, please re-run 'bzr info'
<dhastha> there is no one folder there
<dhastha> while run bzr info       it returns Checkout (format: unnamed)
<dhastha> Location:
<dhastha>        checkout root: .
<dhastha>   checkout of branch: bzr+ssh://bazaar.launchpad.net/~launchpad/lp-source-dependencies/trunk/
<maxb> OK, this part is now sorted.
<mrevell> Night all
<maxb> dhastha: Now that this has been sorted out, you should re-run rocketfuel-get
<dhastha> maxb, k sure
<rockstar> sinzui, hi
<sinzui> hi rockstar
<rockstar> sinzui, so, I'm trying to add a method to lp.registry.models.product.Product - I've made IProduct inherit from IHasRecipes and added the method getRecipes to IProduct, but my test still says there is no getRecipes method.
<rockstar> sinzui, so I'm assuming that there's some sekrit zcml somewhere, but I don't see anything suspicious in lp/registry/configure.zcml
<rockstar> sinzui, I seek enlightenment grandfather.
<sinzui> rockstar, verifyObject() says there is no problem?
<sinzui> rockstar, you added the method to IProductPublic?
<rockstar> sinzui, verifyObject (as used by assertProvides) says there is a problem, so I'm assuming that proxy is getting in the way.
<rockstar> sinzui, no, I added it to IProduct
<rockstar> Er, I made IProduct inherit from IHasRecipes, which has the method.
<rockstar> And then created the method in Product
<sinzui> okay
<rockstar> sinzui, I guess using IProductPublic might be okay.  As far as I know, we don't have a story for private recipes.
<sinzui> rockstar, I do not see IHasRecipes in any zcml, so there are not permissions to see it. My tree may be out of date
<rockstar> sinzui, yeah, it landed this morning.
<sinzui> rockstar, the other way to explore this is to make IProductPublic inherit IHasRecipes and see if the attr is visible
<rockstar> sinzui, ack.  I'll try that.
<rockstar> sinzui, it worked. Screw it.  I'm leaving it that way.
<sinzui> :)
<sinzui> rockstar, take a look at the other interfaces in the file first to be sure it is what you want. Most registry objects define permission on the base Interfaces for webservice API reasons
<rockstar> sinzui, yeah.  I don't think we care that recipe listings are public.  As it looks now, Recipes can't be private.
<dhastha> maxb, error occuring while make run:    http://paste.ubuntu.com/410703/
<maxb> dhastha: Could you please pastebin 'dpkg -l' (it will be quite long)
<dhastha> maxb, http://paste.ubuntu.com/410704/
<maxb> dhastha: You do not have launchpad-developer-dependencies installed
<dhastha> maxb, http://paste.ubuntu.com/410709/
<maxb> dhastha: https://dev.launchpad.net/Running, under the "Building" heading
<thumper> morning
<thumper> sinzui: I've cc'ed you on an email
<thumper> sinzui: would be happy to talk about it at some stage
<dhastha> maxb, success. Thanks a lot
<maxb> np
<sinzui> leonardr, anyone: How to I go about making the fascist happy?
<sinzui> You should not import copy_field from lazr.restful.declarations:
<sinzui>     lp.registry.interfaces.distribution
<sinzui> ah I see:
<sinzui> from lazr.restful.interface import copy_field
<leonardr> glad you figured it out since i had no clue
<sinzui> :)
<thumper> what was the reason for: Module canonical.launchpad.webapp.login, line 255, in login
<thumper>     email = removeSecurityProxy(account.preferredemail).email
<thumper> AttributeError: 'NoneType' object has no attribute 'email'
<thumper> on login again?
<thumper> user in #launchpad having issues
<mwhudson_> i would have less wtfs if buildout and buildbot didn't look so similar
#launchpad-dev 2010-04-08
<wgrant> Can someone please land https://code.edge.launchpad.net/~wgrant/launchpad/amputate-buildergroup/+merge/22198, https://code.edge.launchpad.net/~wgrant/launchpad/filter-apt_pkg-deprecationwarnings/+merge/22317 and https://code.edge.launchpad.net/~wgrant/launchpad/bug-538844-master-side/+merge/22681?
<mwhudson> wgrant: ok
<wgrant> mwhudson: Thanks.
<mwhudson> wgrant: they're all headless now
<wgrant> mwhudson: Great.
 * thumper struggles with self
<thumper> mwhudson: ping
<StevenK> I just merged a branch with devel, and it has conflicts because apidoc.moved didn't exist and then it moves the existing apidoc to apidoc.moved
<StevenK> So, I'm confused
<mwhudson> thumper: pong
<thumper> mwhudson: got a few minutes for a skype call?
<mwhudson> StevenK: delete the apidoc & apidoc.moved directories i think
<thumper> mwhudson: I'm struggling with some delegate stuff
<mwhudson> thumper: ok
<StevenK> mwhudson: Then bzr up or rocketfuel-get?
<mwhudson> StevenK: no, i think the problem is that the directory was deleted in devel
<mwhudson> but it contains ignored files in your tree
<mwhudson> and then i think it was re-added and deleted again or something
<StevenK> mwhudson: I've just checked, and I've got the same conflicts in devel too
<thumper> lp:~thumper/launchpad/package-branch-listing-speedup
<wgrant> StevenK: That happens because your local copy of the directory contains ignored *.pyc, so bzr won't remove it itself.
<StevenK> Ahhh
<lifeless> actually its a really freaking annoying limiation
<wgrant> lifeless: bzr really-ignore-i-dont-care *.pyc?
<lifeless> wgrant: our initial plan is a lost+found style approach
<wgrant> Ah.
<StevenK> lifeless: Will it get a flagged a conflict, or more of a "These files want to be deleted, do you care?"
<lifeless> neither
<wgrant> mwhudson: lp:wgrant/launchpad/bug-538844-master-side failed one test; I've just pushed up a fix.
<mwhudson> wgrant: ok, i'll ec2 it again
<thumper> mwhudson: interested in reviewing that branch, or do you want to pass on it?
<mwhudson> thumper: oh yeah, i'll have a look now
<wgrant> mwhudson: Thanks.
 * wgrant now needs somebody to land https://code.edge.launchpad.net/~wgrant/launchpad/more-buildmaster-cleanup/+merge/22735 and https://code.edge.launchpad.net/~wgrant/launchpad/remove-dbnote/+merge/22736
<mwhudson> wgrant: i like remove-dbnote on general principle
<mwhudson> wgrant: started on both of them
<wgrant> mwhudson: Thanks.
<thumper> I have a feeling it would be easier to start from scratch with the email on needs review branch
<thumper> that's now confirmed
<thumper> it will be easier to start again
<thumper> bzr pull --overwrite FTW
<thumper> make clean schema
<wgrant> Because your MP email job stuff makes it all so much easier?
<thumper> yep
<thumper> got rid of all the horrible edge cases
<thumper> also added the description to the mp since I started that branch
<wgrant> True.
 * wgrant is waiting for implicit WIP MPs and nice diffs.
<thumper> wgrant: getting there...
<StevenK> Failed to load application: No module named psycopg2
 * StevenK stabs things
<wgrant> StevenK: Just that branch?
<wgrant> What's the full traceback?
<StevenK> wgrant: Just that one branch, yes
<StevenK> wgrant: And http://paste.ubuntu.com/410869/
<wgrant> Ooh, trying to develop Soyuz on amd64. Brav.
<wgrant> +e
<wgrant> That is an odd one, if it works in bin/py as you say...
<StevenK> bin/py and then import psycopg2 doesn't give an error, no
<mwhudson> fricking wifi
<thumper> dinner time
<wgrant> Hm.
<wgrant> GRUB is telling me "biosdisk read error"
<wgrant> This cannot be a sign of good disk health.
<spm> ew. i suspect not; or you've just found a really ikky bug.
<wgrant> It's fortunately just a family machine that has no important data.
<wgrant> But still, ew.
<adeuring> good morning
<wgrant> buildbot is upset with me, even though the two branches that I landed are completely unrelated and they both passed EC2...
<stub> wgrant: Not upset at you, just upset. Your branches landed on launchpad/devel, and that buildbot couldn't checkout the source code for some reason. I'll give it a kick.
<wgrant> stub: Ah, OK, thanks.
<wgrant> It's nice of it to tell me that I broke it, but give no details of what the failure was.
<stub> db-devel on the other hand seems to have a genuine failure:
<stub>     AssertionError: <lp.code.model.branchtarget.ProductSeriesBranchTarget object at 0x1278ea50> doesn't support code imports
<wgrant> That'll be james_w's code import export work conflicting with the productseries +setbranch creation overnight.
<mwhudson> looks like james_ws sutff?
<wgrant> The former in devel, the latter in db-devel.
<mwhudson> ah
<wgrant> ...
<wgrant> I just had an EC2 run fail because it got a connection timeout while sending the submission email to PQM.
<wgrant> The test suite otherwise passed -- can somebody please submit lp:~wgrant/launchpad/more-buildmaster-cleanup?
<wgrant> I find it interesting that it failed to email PQM, but managed to email me.
<noodles775> wgrant: there seems to be network issues between amazon and bzr.launchpad.net... buildbot is failing to update-sourcecode consistently right now.
<wgrant> noodles775: lovely...
<james_w> who landed the +setbranch stuff?
<james_w> not to point fingers, but so that I can find the branch to work out the fix
<wgrant> bac
<wgrant> https://code.launchpad.net/~bac/launchpad/bug-524302/+merge/22180
<james_w> thanks
<james_w> ok
<james_w> it seems he based his work on top of mine, so I don't know how that will have passed tests
<james_w> productseries aren't really valid branch targets for code imports
<james_w> you can't push a branch to lp:~james-w/product/productseries/foo
<wgrant> I was wondering why ProductSeries were branch targets at all.
<wgrant> It doesn't make sense in the current model.
<james_w> yeah, dunno
<james_w> we could either fake it, having it import to the product namespace and setting the official link
<james_w> but that sounds like there may be unexpected consequences for people on occasion
<james_w> so we could just change this new branch to use IBranchTarget(product) and then make the link itself
<wgrant> Isn't IPerson an IBranchTarget too? How does that work with code imports?
<james_w> it doesn't
<wgrant> Well, how doesn't it fail in the same way?
<james_w> it would
<james_w> except that this isn't a view on IPerson is it?
<wgrant> Shhhhh details.
<thumper> product series aren't branch targets
<james_w> thumper: the code disagrees
 * thumper looks at the code
<james_w> ah no, this was just /really/ bad timing, as he used the changed methods before the restrictions were tightened, so that's my fault really, I didn't count on anyone adding any users of the method and just fixed what was in tree for the first part
<thumper> WTF?
<thumper> james_w: are you fixing it?
<thumper> we could have an adapter from product series to branch target
<thumper> but product series aren't branch targets
<james_w> well, I'm not sure I'm going to fix that
<james_w> I'm going to fix the build at least
<thumper> can someone please file a bug about that
 * thumper isn't working
<james_w>     ProgrammingError: permission denied for relation account
<james_w> ^ anyone familiar with that?
<james_w> just doing factory.makeProduct()
<james_w> when it does IPerson.getByName()
<noodles775> james_w: that's in a test, or in a console? (if a console, I was going to suggest running make schema first).
<james_w> doctest
<james_w> I had the same idea, make schema is going now
<james_w> and it's done, just 14 minutes later
<james_w> https://code.edge.launchpad.net/~james-w/launchpad/bug-524302/+merge/23006
<james_w> could someone review ^ please
<james_w> argh, conflicts!
<noodles775> james_w: let me know when you've resolved the conflicts. jelmer and I are on OCR today.
<james_w> noodles775: please take another look
<wgrant> james_w: Note that you can just run database/schema/security.py to reset perms.
<james_w> ah, ok
<wgrant> No need for a full >5min make schema.
<wgrant> Is PQM happy yet?
<noodles775> PQM is, buildbot's not (same issue I mentioned before, failing update-sourcecode but continuing test run).
<wgrant> I've three branches that passed EC2 but presumably got rejected by an upset PQM.
<noodles775> Right, yes it's in testfix (sorry).
<wgrant> Ah.
<stub> Did a fix for GpgmeError: (32, 176, 'Unknown error code') land? Cause I'm still seeing them.
<wgrant> stub: No, it hasn't landed yet.
<wgrant> jelmer: ^^?
<stub> Is it https://code.edge.launchpad.net/~jelmer/pygpgme/bug452194/+merge/21635 ?
<wgrant> That's the pygpgme fix.
<jelmer> the fix for pygpgme itself has landed
<wgrant> But there's another fix that is needed to update sourcecode.
<jelmer> but my change to update launchpad to update sourcecode hasn't landed yet, because pygpgme in the lp tree needs to be upgraded to 2a first
<wgrant> Ahh.
<stub> https://code.edge.launchpad.net/~launchpad-pqm/pygpgme/devel says it is 2a now
<stub> Oh - that is the repo, not the branch
<stub> I can't see an upgrade button - does it need to be upgraded manually by a losa because pqm is too stupid to hit the button itself?
<mthaddon> yep, I think so...
<stub> mthaddon: Speaking of losas.... feel free to upgrade that branch :)
<wgrant> Wasn't the problem client-side?
<wgrant> In that update-sourcecode won't change the format, so the pull from rich-root to poor-root will fail?
<jelmer> mthaddon: it's the production pygpgme branch that needs to be updated
<jelmer> perhaps I should just patch update-sourcecode to deal with rich root upgrades?
<jml> jelmer: I think that would be a good idea
<jml> jelmer: the only reason I didn't do that for the subunit upgrade was because I wasn't sure what we used in production
<jml> jelmer: but it turns out that we use update-sourcecode, so making it handle rich-root upgrades would not introduce any new differences between dev & prod
<maxb> jml: Hello. As the author of https://dev.launchpad.net/ReviewingCodeImports, could I ask you to comment on whether it is safe for community members to be in ~vcs-imports, in the "[Launchpad-dev] Code Import Reviews" email thread?
<jml> maxb: I am merely the copier-and-paster from internal wikis, not the author. But I do know a little about the subject.
<maxb> ah, ok
<jml> maxb: in a nutshell, it ought to be safe (that's certainly the intent) and it probably is, but I would recommend someone flick through the relevant bits of code before opening it up
<jml> maxb: in the past, we abused the ~vcs-imports team as a short-hand for "Launchpad Code team"
<jelmer> jml: *nods*
<deryck> sinzui, ping
<sinzui> hi deryck
<abentley> leonardr-afk, do you know what /lp/translations/interfaces/webservice.py is for and how it's used?
<james_w> abentley: it collects things that lazr.restful should scan for exports it should use to generate the wadl (and possibly for serving them too)
<james_w> at least if it's the same as the one in /code/
<abentley> james_w, why do only code and translations have one?  Why doesn't it have an __all__?
<abentley> james_w, is it a magic name? or is it configured somewhere?
<james_w> because the others haven't caught up to the new way of doing things yet. Because it's not for importing from?
<abentley> james_w, if lazr.restful isn't importing those symbols, how does it know what they are?
<james_w> it's importing the module and walking over the contents I imagine
<abentley> james_w, if they're imported so they can be used from the module, they should be listed in an __all__.
<james_w> lib/lp/code/configure.zcml:  <webservice:register module="lp.code.interfaces.webservice" />
<james_w> I'm not in this to argue with you
<abentley> james_w, basically, I think the lint error is correct, but I'm happy for you to land your code as is.
<james_w> ok, thanks
<abentley> james_w, I want to talk to leonardr-afk about potentially changing it, but that doesn't need to block you.
<james_w> great
<abentley> james_w, context: I just submitted an approval review.  Unfortunately, I didn't sign it because enigmail isn't available on Lucid, and it was rejected.
<james_w> ok
<bdmurray> bac: https://code.edge.launchpad.net/~brian-murray/launchpad/api-export-messages-count/+merge/22833 is still unmerged as of yet afaict
<bac> bdmurray: did you get ec2 email about it?
<bac> bdmurray: thanks for pointing it out.  i'll have a look to see what happened
<bdmurray> bac: I seem to recall a failure landing it due to multiple? changes trying to be landed at once
<bdmurray> bac: yeah there was a buildbot failure
<bdmurray> bac: and I'm certain ;-) it wasn't my change
<bac> bdmurray: i don't have a failure message for it.  i'll resubmit now.
<bdmurray> bac: cool thanks
<sinzui> bac: db-devel hates you: https://lpbuildbot.canonical.com/builders/db_lp/builds/672/steps/shell_7/logs/summary
<bac> sinzui: wow
<sinzui> bac: ProductSeriesBranchTarget is an adapter that used by the Involvement menu to go from PR to BT.
<sinzui> bac: I wonder if something else changed...surely ec2 would have noticed that.
<sinzui> bac: Thumper was asking me about the adapter today, maybe there is an underlying change.
<bac> sinzui: ok.  let me try to figure out what's going on
<sinzui> bac: I forwarded you thumper's email and my reply
<bac> thanks
<deryck> jml, leaving aside the "don't look the project overview page" issues, how would you suggest dealing with the hot bugs list when it isn't useful to some (usually small) projects?
<jml> deryck, I don't know. Sorry.
<deryck> jml, fair enough.
<jml> deryck, I don't mean to scupper the approach you've suggested
<deryck> jml, no, not a problem to call it to question... just wondered if you had another idea.  Multiple lists seems the only way to handle that.  Short of crazy configuration options, which I don't favor.
<jml> deryck, I'm going to keep thinking about it.
<deryck> ok, cool.  thanks.
<dhastha> danilos, R u there?
<leonardr-afk> abentley, do you want to change lazr.restful or launchpad?
<abentley> leonardr, launchpad, most likely.  I think that lp/translations/interfaces/webservice.py et al should be providing an __all__ if their imports are for external use.
<leonardr> i agree, i didn't know about webservice.py
<leonardr> i thought we were running against the regular interface modules
<gary_poster> sinzui, mars, stuartm is wondering if we have fewer new spammer accounts since the new SSO was put in place.  Apparently the new SSO story makes the captcha more enforced.  Can either of you quickly give any data about that?
<sinzui> no
<mars> nope
<sinzui> gary_poster, we have fewer accounts since we took links off of person and ppa pages owned by users with no karma
<sinzui> gary_poster, we had two spam accounts registered in the last 24 hours
<mars> sinzui, how did you catch those?
<gary_poster> sinzui: ok, and is that rate typical?
<sinzui> They registered project
<mars> ah
<sinzui> gary, the spammer used SSO to successfully attack via our wikis. We locked them down last month
<sinzui> gary_poster, lpnet oopses show that we are under constant attack via search data injection. 1/3 of all oops!
<gary_poster> sinzui: ok, thanks.  so, to be super clear, you are saying "no, the new SSO server does not appear to have reduced our spammer registrations" right?
<gary_poster> lovely
<mars> the background noise of the internet
<sinzui> correct.
<gary_poster> thank you sinzui
<mars> sinzui, mean time to compromise a directly exposed unpatched windows box on the internet was, what 36 seconds last I heard?
<mars> no surprise that our site would be bombarded in a similar way
<sinzui> gary_poster, mars: I think some more perspective is need...
<sinzui> Dec: 2500+ using profile + project
<sinzui> Jan: 100- using PPA
<sinzui> Feb: 50- using PPA (because we ignored the issue)
<sinzui> March 10- using SSO + wiki
<sinzui> April 2: looking for project vulnerabilities
<sinzui> Jan-April 20+ search attacks each day
<sinzui> gary_poster, mars: I review googles results, project, and oops report for new attacks every day,
<sinzui> I check the database every Sunday
<mars> silly question: what is wrong with this pqm-submit message?  http://pastebin.ubuntu.com/411169/
<sinzui> thumper, bac and I want to talk when you are available
<thumper> sinzui: can you wait until 21:00 UTC?
<thumper> just under 2 hours from now
<thumper> I need to get dressed and eat
<sinzui> thumper, thanks
<jml> g'night all
<thumper> sinzui: did you want to have a quick chat before my standup?
<sinzui> thumper, yes. bac and I want to know how we can fix the Involvement menu adapter
<thumper> ok
<thumper> lets talk
<sinzui> bac: can you talk
<thumper> sinzui: start me off, where is the code for the involvement menu adapter
 * sinzui looks
<sinzui> thumper look in lp/registry/browser/pillar.py for the reason
<sinzui> thumper the adapter is lp/code/model/branchtarget.py ProductSeriesBranchTarget
<thumper> sinzui: but where is the pillar.official_codehosting found?
<sinzui> thumper, product
<thumper> ah
<thumper> pillar = nearest(self.context, IPillar)
<SlonUA> any ideas about such error Transport error: Server refuses to fulfill the request (403 Forbidden) for http://bazaar-internal.launchpad.dev/00/00/00/55/.bzr/branch-format
<thumper> what I don't understand is the need for the product series branch target
<thumper> sinzui: I can't see where it is used
<thumper> SlonUA: where are you getting it?
<sinzui> thumper, the involvement menu is shown on many kinds of objects and we are showing links based on the pillars selection of official links
<SlonUA> thans for notice .. it's after make sync_branches
<thumper> SlonUA: probably bad permissions on the local filesystem for the bzr branches
<thumper> sinzui: right, but a product series isn't a pillar
<SlonUA> thumper: 1) create branch on dev, 2) push by bzr 3) make sync_branches
<sinzui> thumper, yes, it is hidden via adaption.links are made form the context, and this bug https://bugs.edge.launchpad.net/launchpad-registry/+bug/482256 shows that users want to submit code for a product series, but could not
<mup> Bug #482256: Involvement menu 'Submit code' link OOPSes from the project series <oops> <Launchpad Registry:Fix Released by sinzui> <https://launchpad.net/bugs/482256>
<mwhudson> SlonUA: /var/tmp/bazaar.launchpad.dev/mirrors probably has too restrictive permissions
<thumper> SlonUA: yes, but in the local setup of lp.dev you should have done 'sudo make install' at some stage
<SlonUA> thumper: hm ... make run_codehosting is running under root
<sinzui> thumper, I ring. bac can catchup with us later
<SlonUA> mwhudson: drwxr-xr-x 3 root root 4096 2010-04-08 21:24 mirrors
<mwhudson> SlonUA: hm, strange
<SlonUA> thumper: all instalation was under : sudo -i
<mwhudson> i wouldn't run launchpad as root though!
<SlonUA> any issues with apache !?
<SlonUA> mwhudson: agree it's was under chroot before .. i have issues .. so i have VM
<thumper> def product_series_to_target(product_series):
<thumper>     return ProductBranchTarget(product_series.product)
<SlonUA> mwhudson: the same issue with 777 for mirrors ...
<mwhudson> SlonUA: well, http://bazaar-internal.launchpad.dev/00/00/00/55/.bzr/branch-format should be served by apache
<mwhudson> so maybe read apache's error.log?
<SlonUA> mwhudson: i read my mind .... looking =)
<SlonUA> mwhudson: [error] [client 192.168.241.11] client denied by server configuration: /var/tmp/bazaar.launchpad.dev/mirrors/00/00/00/57/.bzr/branch-format
<mwhudson> SlonUA: well, that looks like an apache problem then
<SlonUA> mwhudson: yeah .. any ideas to quick look !?
<mwhudson> SlonUA: well, i guess you've installed the local-launchpad-apache configuration and gracefulled apache?
<SlonUA> mwhudson: btw; no special settings for apache was done .. only make install and remote access tunning + one ip using
<mwhudson> SlonUA: 'remote access tuning' ?
<mwhudson> you might need to do some more of that i guess
<SlonUA> mwhudson: <VirtualHost *:80>
<SlonUA>   ServerName bazaar-internal.launchpad.dev
<SlonUA>   LogLevel debug
<SlonUA>   DocumentRoot /var/tmp/bazaar.launchpad.dev/mirrors
<SlonUA>   <Directory /var/tmp/bazaar.launchpad.dev/mirrors/>
<SlonUA>     Order Deny,Allow
<SlonUA>     Deny from all
<SlonUA>     Allow from 127.0.0.0/255.0.0.0
<SlonUA>     Options SymLinksIfOwnerMatch
<SlonUA>     AllowOverride None
<SlonUA>     Options Indexes
<SlonUA>   </Directory>
<SlonUA> </VirtualHost>
<SlonUA> sorry pals for spam =)
<SlonUA> mwhudson: is such part correct !?
<mwhudson> SlonUA: looks ok, i guess you are actually accessing it from localhost?
<SlonUA> so, i run make sync_branchesh from localhost .. but ... i have hosts with external ip
<SlonUA> mwhudson: oh. u mean ... i got it
<SlonUA> just fix it to :
<SlonUA> <Directory /var/tmp/bazaar.launchpad.dev/mirrors/ >
<SlonUA> Order deny,allow
<SlonUA>     Allow from localhost 127.0.0.0/255.0.0.0
<SlonUA>     Options SymLinksIfOwnerMatch
<SlonUA>     AllowOverride None
<SlonUA>     Options Indexes
<SlonUA>   </Directory>
<SlonUA> mwhudson:  something like this
<SlonUA> mwhudson: i think i will try to experiment .. to understand it ... thank u for ur time =) ..
<thumper> jelmer: ping
<thumper> jelmer: I'm wanting to update launchpad's bzr-git and dulwich
<thumper> jelmer: should I use tip?
<jelmer> thumper: please do
<thumper> jelmer: has the names or locations of the cache files changed?
<thumper> jelmer: because we'd need to update the launchpad code that stores and looks for them
<jelmer> thumper: yes, caches are now in .bzr/repository/git/
<thumper> jelmer: entire directory?
<jelmer> thumper: yep
<thumper> jelmer: fixed file names? or can it change?
<jelmer> thumper: it'll only have two files in it at the moment
<thumper> jelmer: best to just tar up the directory?
<thumper> mwhudson: is that easy with the code we have to do an entire directory?
<jelmer> thumper: but that will change in the future, once Bazaar makes it easier to reuse pack files
 * thumper nods
<mwhudson> thumper: not trivial, but not very hard i imagine
<thumper> mwhudson: we have some tar.gz stuff in other parts
<thumper> mwhudson: so shouldn't be too hard
<mwhudson> thumper: exactly
<james_w> wgrant: hey, any idea how I can get a publishing record for every upload done in lucid after it was opened?
<james_w> if I grab the total of Published + Superseded with a created_since_date of after it was opened I think that will duplicate some?
<wgrant> james_w: You'd need Deleted as well.
<wgrant> But yes, that will give you duplicates.
<wgrant> Perhaps stuff them into a dict by (source name, source version)?
<james_w> ok
<james_w> I don't really care about Deleted I don't think
<james_w> it doesn't need to be exact anyway
 * wgrant looks for someone to land https://code.edge.launchpad.net/~wgrant/launchpad/move-rescueiflost-tests/+merge/22737
<jelmer> thumper, mwhudson: I've pushed the fix for the -0000 / +0000 issue
<mwhudson> jelmer: i guess lp:dulwich is still out of date?
<jelmer> mwhudson: at least until the moment you hit "import now" :-)
#launchpad-dev 2010-04-09
<mwhudson> losa ping!
<spm> mwhudson: yo!
<mwhudson> spm: it's the pick-a-staging-import-slave and run http://pastebin.ubuntu.com/411296/ time
<mwhudson> (hopefully with correct urls this time)
 * thumper heads for coffee
<spm> mwhudson: nah; just make em up; urls are overrated
<thumper> mwhudson: that doesn't store the git tree properly though
<mwhudson> spm: :)
<mwhudson> thumper: oh yes, you're right
 * thumper really heads for coffee
<spm> mwhudson: so raspberry again?
 * mwhudson adjusts his iron-o-meter
<mwhudson> spm: no?
<spm> that was a joke - at myself.. but :-)
<mwhudson> oh good
 * mwhudson turns the gain back down
<spm> :-)
<mwhudson> spm: i'll need to whip up another patch first so no hurry
<spm> mwhudson: hrm. yes cause the apply patch in that paste is telling me it's applied already
<spm> mwhudson: the otehr steps are done; fwiw
<mwhudson> oh heh
<spm> thumper: I guessing this is realted to the rt we discussed yesterday? https://pastebin.canonical.com/30326/
<mwhudson> jelmer: does bzr-git now depend on bzr 2.2?
<mwhudson> jelmer: http://pastebin.ubuntu.com/411354/
<jelmer> mwhudson: Have you recompiled dulwich?
<mwhudson> ah
<mwhudson> yes, but probably only with python 2.6
<mwhudson> spm: can you apply this patch to strawberry's launchpad tree: http://pastebin.ubuntu.com/411369/
<spm> raspberry?
<mwhudson> some kind of fruit
<mwhudson> not an antartic base
<spm> sweet, that narrows it down fopr me to ~ 80% of our servers... ;-)
<spm> mwhudson: patched
<thumper> mwhudson: well that saved me some work for today :)
<thumper> mwhudson: testing the kernel?
<mwhudson> thumper: the bzr-git thing?
<mwhudson> thumper: yeah, just hit go like a minute ago
 * thumper watches
<jelmer> oh, I didn't know you were going to try the kernel again
<jelmer> the unusual modes thing isn't fixed yet
<jelmer> although if the batch size is 5k you might not notice it
<wgrant> Is it faster now?
<jelmer> wgrant: a full kernel import should be less than 6 hours now
<wgrant> Wow.
<lifeless> mwhudson: and not a penguin?
<wgrant> Isn't that almost three orders of magnitude faster?
<lifeless> I think we should just use the last 4 hex digits of their IP address
<jelmer> wgrant: two things made it fast - there was a bug in the sqlite cache that made it store a few billion rows rather than less than a million
<wgrant> Haha.
<lifeless> jelmer: !
<jelmer> wgrant: and InventoryDirectory.children in Bazaar should be -Devil
<lifeless> jelmer: in 2a it should be tolerable.
<lifeless> jelmer: what was it doing that sucked? We can tune it further...
<jelmer> lifeless: it's still very slow in 2a - I haven't compared it to pack-0.92
<SlonUA> mwhudson: hi
<lifeless> jelmer: pack based is full inventory load
<lifeless> jelmer: so editing one dir is definitely cheaper in 2a :>
<lifeless> bbs
<jelmer> lifeless: I'm now walking the git tree objects rather than InventoryDirectory.children
<wgrant> jelmer: I notice that you have a branch to move upload processing into buildd-manager itself -- why that rather than letting a separate daemon handle upload processing, thus making buildd-manager less horribly slow?
<SlonUA> mwhudson: r u going create guide for launchpad.dev usage as remote access + code_hosting
<wgrant> We are wasting a *huge* amount of buildd time at the moment because of the synchronous upload processing.
<jelmer> lifeless: that works well too so there's no need to tweak anymore from bzr-git's POV
<jelmer> wgrant: It's the first step towards doing things in a more asynchronous manner
<jelmer> wgrant: and it at least avoids the overhead of importing all of launchpad each time
<wgrant> jelmer: I'm not sure I see how embedding the synchronous process further into buildd-manager helps with asynchronicity.
<wgrant> But yes, removing that overhead is a good step.
<SlonUA> pals, could u say how much stuff will be run under 'make run_all'
<wgrant> You know, that import does seem to be going a little more quickly.
<jelmer> wgrant: :-)
<lifeless> SlonUA: I doubt a guide for production use of launchpad.dev will be made - it doesn't make sense :)
<lifeless> SlonUA: as for what is started by make, you can look at the makefile; or is there some reason you care about what stuff starts ?
<SlonUA> lifeless: thanks for notice ... just interesting about targets in make .. yup i will look
<SlonUA> lifeless: about guide .... yes we have two ... remote access and code-hosting ... but for use together we hadn't ... but i done already ... i just use vm for launchpad.dev
<lifeless> SlonUA: what do you mean by remote access?
<SlonUA> lifeless: access to launchpad.dev from another host
<jelmer> mwhudson: I'm curious how fast the second batch will be, since it will have to reconstruct some of the base texts
<wgrant> It doesn't look like the initial 'finding revisions to fetch:generating index' is any faster.
<jelmer> wgrant: that really can't be optimized much further, at least not in bzr-git
<mwhudson> no, so maybe we should import even more than 5000 revisions in one go
<jelmer> mwhudson: yeah, that'd make sense
<wgrant> It looks like only half the import time was actual rev imports.
<wgrant> Impressive.
<jelmer> lifeless: git imports now actually spend most of their time in add_inventory_by_delta()
<lifeless> jelmer: interesting
<lifeless> jelmer: how well does that perform for you ?
<jelmer> lifeless: reasonably
<jelmer> lifeless: it just happens to be the main bottleneck left
<mwhudson> it seems like the second kernel import is quite a lot slower
<mwhudson> still 100 times faster than what's in production, mind...
<thumper> will be interesting to watch how it slows
<thumper> hmm, I wonder why postgresql 8.4 didn't restart automatically on restart
<thumper> mwhudson: I'm removing the is_personal_branch from IBranch and adding supports_short_identities to BranchTarget
<thumper> mwhudson: after a talk with jml this morning
<mwhudson> thumper: +1
 * mwhudson afk for lunch/emma collection
<thumper> mwhudson: watching the git import speed on strawberry
<mwhudson> thumper: it seems to be doing about 150 revs a minute?
<thumper> it spends < 50% of the time doing the import
<thumper> 25min for 5000
<thumper> so about 200 per minute I guess
<thumper> the rest is determining revisions, packing et al
<thumper> and overhead
<thumper> do you think 5k is big enough?
<mwhudson> mm
<thumper> 10k maybe?
<mwhudson> 10k sounds about right
<mwhudson> could even go higher i guess
<mwhudson> lemme check the memory usage
<thumper> is the same number used by bzr-svn for the incremental imports?
<mwhudson> currently yes
<mwhudson> but that'll be very easy to change to something separate
<thumper> actually the number per minute seems to vary wildly
<wgrant> Hm, it's already up to 50 minutes per run?
<thumper> wgrant: but seems relatively constant
<thumper> wgrant: and I'm not sure that strawberry is particularly fast
<wgrant> IIRC the production import started at ~3 hours, and is now >20.
<thumper> wgrant: much of that we think is dealing with the gazillion row sqlite db
<thumper> wgrant: let it run for a day and see where we are
<mwhudson> when select count(*) from trees; takes an hour...
<wgrant> Right, but I'm just saying that initial looks can be deceiving.
<wgrant> Heh.
<thumper> mwhudson: want to get those bzr-git changes into a branch for me to review?
<wgrant> And it's already taking almost twice as the first couple...
<mwhudson> thumper: yeah ok
<thumper> wgrant: don't panic
<thumper> mwhudson: how's the non-git work going?
<wgrant> I guess even at this speed it will be done in two or so days.
<wgrant> A tad better than production!
<mwhudson> thumper: i think the first pipe that changes codehosting is basically done, barring the fact that most of the new docstrings are "XXX"
<thumper> :)
<thumper> this import run is down to 50/minute
<thumper> for the last 10 minutes anyway
<wgrant> thumper: It looks like around 200 from here...
<thumper> variable...
<wgrant> How do production's DB indices compare to the distributed schema?
<lifeless> what do you mean ?
<wgrant> Is the dev schema kept up to date with index changes that presumably occur first on production?
<lifeless> usually yes;
<lifeless> every now and then you'll see stub add a db patch saying 'already done'
<lifeless> but usually they are done in dev and cp'd to prod
<wgrant> Aha.
<stub> They have to be the same - the same tools used to upgrade dev databases are used to upgrade production databases.
<stub> Is the destination select in the PPA copy form broken? I'm trying to copy packages to Lucid but getting told "The The following sources cannot be copied:  slony1 1.2.20-1~launchpad.hardy1 in hardy (same version already has  published binaries in the destination archive)"
<wgrant> stub: Where are you copying to/from?
<wgrant> Both archive and series are important.
<wgrant> mwhudson: Landed; thanks.
<stub> wgrant: I'm trying to copy from https://edge.launchpad.net/~maxb/+archive/launchpad/+copy-packages
<stub> I need to get the packages rebuilt for lucid
<wgrant> stub: You can't use an intra-PPA copy to do a rebuild.
<wgrant> That would result in different binaries with the same name.
<stub> Which makes me think the form is bogus, since it is asking me for the series.
<wgrant> That's useful for when you want to copy to another PPA.
<wgrant> But yes, it should more obviously forbid source-only copies to the same PPA.
<stub> So I can copy it to my PPA with the same series, and then rebuild it in my PPA?
<wgrant> You could do that, but you won't be able to copy it back in.
<stub> This all seems terribly arbitrary and confusing to an outsider.
<stub> Garh.... deleted the previous copy (for karmic, now useless), and still get the same error message.
<stub> I think I need to delete the PPA
<wgrant> You cannot ever have different binaries of the same name again.
<wgrant> The restriction is not arbitrary and confusing -- you cannot have multiple distinct files of the same name in the same PPA, because that is a lie and is very confusing.
<wgrant> However, we may eventually have a facility for requesting binary rebuilds which append some as-yet-undecided string to the version.
<wgrant> (note, however, that this is the Soyuz definition of 'eventually')
<adeuring> good morning
<spm> morning adeuring!
<spm> thumper: ref that configs merge proposal. 1. +1, 2. unless I hear otherwise in the next 2-3 mins :-) I'll submit the merge for you
<spm> thumper: scratch that merge via me - appear to be having key issues between me/bzr/pqm dued to a new key of mine; so will need to fix that. aka yak shaving. probably quicker if you submit when ready.
<wgrant> noodles775: Can I move lp.soyuz.browser.archive.traverse_distro_archive to Distribution.getArchive and export it? It does violate the separation a little more than now, but such is life with the webservice...
<spm> mwhudson: did you see this earlier? staging codehost: https://pastebin.canonical.com/30326/ appears to be a diff one to your other patch I've been using.
<mwhudson> spm: that one's thumper's fault!
<mwhudson> spm: i guess hacking out the update_preview_diff section from the config will let it start it up
<spm> mwhudson: oh fun :-)
<spm> mwhudson: semi seriously tho - is it problematic that these sorts of changes can get past the test suite to blow up staging? ie is this testable and hence do I need to bug report?
<wgrant> Speaking of configs... is edge going to update at some point?
<spm> heh; not if staging blows up.
<wgrant> But I thought the config key removals were reverted :(
<spm> the updates typically run; crash; so we roll back.
<mwhudson> spm: i guess buildbot/ec2-when-run-by-a-launchpad-dev could test that all the configs in the lp:lp-production-configs branch load
<spm> interesting. the edge updates aren't actually disabled atm; and should be running right now.
<wgrant> mwhudson: You mean like is suggested in bug #557271?
<mup> Bug #557271: Unable to remove config entries from the schema <Launchpad Foundations:New> <https://launchpad.net/bugs/557271>
<mwhudson> spm: at least thumper's branch is only on db-devel
<spm> haha. failing to build/sync. blech.
<maxb> stub: Hi. It would be rather horridly confusing if "1.2.20-1~launchpad.hardy1" was to be rebuilt in lucid, no? :-)
<spm> maxb: (mock indignation for stub) I Fail To See The Problem (but privately hahahaha oops)
<mwhudson> wgrant: great mind's think alike!
<mwhudson> (or fools never differ, depending on your pov)
<wgrant> maxb: What are we going to do about Python 2.6's changed output (tarfile trailing slashes, "[111] Connection refused" vs "(111, 'Connection refused')", that sort of thing)? Should we adjust tests to cope with both, or just fix them in the 2.6 branch and do another mass merge on migration day?
<maxb> Given doctests don't allow for "this or that", I think fix-in-branch-and-mass-merge for those
<wgrant> Well, we can easily and somewhat safely ellipsise the latter case.
<wgrant> But I guess since we're going to have to mass-merge some stuff, we might as well fix it properly in the mass-merge.
<maxb> The tarfile madness might be more amenable to making compatible, I'm not sure
<lifeless> use better assertions ;)
<wgrant> Also, are you going to finish off your assertion SyntaxWarning branch?
<lifeless> assertThat is your friend
<wgrant> initialise-from-parent.py is the only 2.6 Soyuz-related failure that isn't trivially fixable.
<maxb> wgrant: I'm a bit blocked on what to do re initialise-from-parent.py. Creating an entire distroseries in the test is beyond my skill level at present
<maxb> I was thinking of just seeing if the test works if I make it init from warty instead of hoary in the present sampledata
<wgrant> maxb: The error is with the pending builds, right?
<wgrant> If so, as a quick fix I would just make the builds no longer pending.
<wgrant> Until we can replace most of these abominable tests.
<maxb> i.e. modify the sampledata hoary in the test setup?
<wgrant> Exactly.
<wgrant> We already do similar things in other places.
<wgrant> Just flip the builds to FAILEDTOBUILD or similar.
<maxb> And the test harness will magically notice that I've dirtied the DB, and reinit it after my test?
<wgrant> Right.
<wgrant> If you commit, the DB will be torn down and recreated when your test finishes. If you don't commit, there's no problem with not tearing it down.
<maxb> And in this case I shall need to commit, since it needs to exec a separate initialise-from-parent.py
<wgrant> True.
<wgrant> It probably already explicitly commits.
<wgrant> Since otherwise the DB wouldn't be marked dirty, even though i-f-p.py alters it.
<maxb> oh, actually yes, it already commits to actually create the to-be-inited series
<noodles775> wgrant: re. the move of traverse_distro_archive, perhaps the bug would be a good spot for the conversation? (that way Julian can see the history etc.)?
<wgrant> noodles775: Probably true.
<wgrant> maxb: So, two lines of code should fix it...
<wgrant> adeuring: I'm confused. Why should the bug page have to deal with missing LFCs? garbo-daily shouldn't be removing files that are in use, right?
<adeuring> wgrant: in theory, you are right. But we had the case that LFC records were missing, see the bug mentioned in the commit message.
<wgrant> adeuring: Right, i reported that bug.
<wgrant> But it was more of a "why is garbo-daily removing these at all?" bug.
<adeuring> wgrant: Ah, I see. Admittedly, I don't have a definitive answer. My guess is that a LOSA deleted the LFCs because the files contained private data
<adeuring> Regarding the code: Only those bug attachments are deleted that are useless anyway because thes don't have an LFC record. So, no harm done, I think
<adeuring> s/thes/they/
<wgrant> If the LFCs are deleted for a good reason, that's fine.
<wgrant> maxb: There's no modern 2.6 branch beyond salgado's?
<maxb> Not at present, no
 * wgrant wonders if we want a fresh ~launchpad-committers one soon.
<maxb> given salgado is going leave according to the wiki page - yes!
 * maxb runs tests on a tweaked initialise-from-parent.txt
 * wgrant is collecting some fixes that are safe for devel.
<spm> mwhudson: also lazr.config.interfaces.ConfigErrors: ConfigErrors: schema-lazr.conf does not have a mpcreationjobs section. <== edited away both this and the prior and it started
<mwhudson> https://code.staging.launchpad.net/~mwhudson/linux/trunk <- slowly slowing down, but mostly i think it's the bits before and after (eg packing) that are getting slower
<mwhudson> spm: woo
<spm> https://pastebin.canonical.com/30343/ for the now obvious details
<wgrant> mwhudson: Yeah, but I don't think it's going to slow down to even the initial production time.
<mwhudson> spm: um, i guess someone (tm) should land a lp-production-configs branch
<dhastha> Danilos: are you there?
<mwhudson> wgrant: yeah
<spm> mwhudson: https://code.edge.launchpad.net/~thumper/lp-production-configs/merge-proposal-jobs/+merge/22995 perhaps? I tried, but apper to be having key fail atm
<wgrant> It'll overtake production in 7 hours.
<mrevell> Morning all
<spm> hey mrevell!
<mrevell> Howdy spm :)
<mwhudson> spm: well that doesn't appear to delete the old sections
<mwhudson> wgrant: also i think pear is ~twice as fast as strawberry
<spm> mwhudson: oh thumpers branch? oh right. yes. bleh.
<wgrant> mwhudson: Handy.
<wgrant> Interesting.
<wgrant> Latest import was only 48 minutes.
<stub> So how do I get the hardy packages at https://edge.launchpad.net/~stub/+archive/launchpad2 rebuilt for Lucid?
<noodles775> stub: currently, assuming they require rebuilding, you'll need to change the target in the changes file and re-upload.
<stub> So I can't do that with the Launchpad UI? Ok.
<stub> I seem to be able to do the copy without a rebuild, so maybe that will work.
<stub> I'm certainly being offered way too many choices that appear to be impossible due to deb packaging limitations.
<jml> good morning
<noodles775> stub: you can indeed copy to another series including the binaries, but it's only safe *if* you are certain they do not need to be rebuilt. See https://help.launchpad.net/Packaging/PPA/BuildingASourcePackage#Versioning
<noodles775> and yes, it is confusing.
<maxb> stub: Why are you rebuilding slony1 in lucid at all?
<stub> maxb: So I can test the staging rebuild procedure locally
<maxb> So, you want a pg8.3 version?
<stub> Yes
<stub> I can't recall right now why I couldn't use the 8.4 version...
<maxb> Well, 1.2.20 is already in lucid for 8.4, so you shouldn't need the plain slony1 source from my PPA at all (as you can see it's greyed out "Newer version available" in your PPA)
<maxb> What you probably want to do is take the slony1pg83 source, and increment its version to 1.2.20-1launchpad~stub1, and build that in lucid
<deryck> Morning, all.
<stub> maxb: I should learn how to do that one day :-)
<maxb> stub: It'll take me 30 seconds, shall I just do it?
<stub> maxb: That would be nice, ta :-)
<maxb> oh waitaminute, didn't I already put a pg83 slony1 for lucid into ~launchpad/ppa ?
<maxb> yes, yes I did :-)
<stub> maxb: Ahh... yes. I'm going around in circles. Unfortunately, those packages are broken. postgresql-8.3-slony1 contains nothing but the documentation, and is missing the important stuff in /usr/lib.
<maxb> !
<maxb> oops
<stub> https://bugs.edge.launchpad.net/launchpad-foundations/+bug/559046 (which I opened before getting distracted)
<mup> Bug #559046: Slony-I 1.2.20 for pg8.3 package missing critical files <Launchpad Foundations:New> <https://launchpad.net/bugs/559046>
<stub> I was going to try the rebuild in my ppa to see if it was a simple fix :)
<maxb> stub: Oops, I suck. Fix uploaded.
 * maxb glares at the build queues with loathing
<wgrant> maxb: Most of that is just an archive rebuild.
<maxb> The amd64 queue is still 2 hours for fresh single uploads :-(
<wgrant> But it looks like i386/amd64 could actually be 40ish minutes behind.
<wgrant> Oh :/
<wgrant> Fortunately you have quite a few buildd admins around.
<maxb> stub wants the packages more than I, and he's a rubber ducky :-)
<wgrant> Very true.
<maxb>     from lp.soyuz.interfaces.build import BuildStatus
<maxb>     ImportError: cannot import name BuildStatus
<maxb> ^ That's in a doctest I'm trying to modify. Anyone have any idea why that breaks?
<maxb> especially given I copied the import line from the code I'm testing!
<jelmer> maxb: I think BuildStatus was moved recently
<jelmer> it might also just be a glitch because of circular imports..
<maxb> ugh, I must have copied it from one branch and then merged devel....
<wgrant> maxb: I moved it to lp.buildmaster.interfaces.buildbase a month or so ago.
<jml> "make schema" seems to fail on API doc generation for me... looks like a fairly shallow makefile bug:
<jml> mv lib/canonical/launchpad/apidoc.tmp/* lib/canonical/launchpad/apidoc
<jml> mv: target `lib/canonical/launchpad/apidoc' is not a directory
<jml> make: *** [lib/canonical/launchpad/apidoc/index.html] Error 1
<jml> is this a known issue?
<wgrant> You've resolved the conflict that occurred when you pulled?
<wgrant> I've not seen that in any of my branches.
<jml> I resolved the conflict by deleting lib/canonical/launchpad/apidoc
<jml> maybe I did it the wrong way
 * jml tries against stable
<wgrant> I think that's the right solution.
<jml> yeah, it happens with stable
<jml> "make clean; make schema" to reproduce the error
<wgrant> Oh, does schema not depend on build?
<wgrant> Hm, it does.
<jml> I'll fix it now. https://bugs.edge.launchpad.net/launchpad-foundations/+bug/559159 for reference
<mup> Bug #559159: "make clean; make schema" fails with API doc error <build-infrastructure> <Launchpad Foundations:New for jml> <https://launchpad.net/bugs/559159>
<jml> oh hmm
<jml> lib/canonical/launchpad/apidoc _is_ supposed to be versioned and present
<jml> and empty :\
<wgrant> Oh, right, that's why it conflicted.
<wgrant> I misremembered :(
<jml> it's kind of weird that it's not just created on build
<wgrant> It used to have content.
<jml> but only weird, not actually bad
<jml> crap
<jml> that's messed up a lot of my branches
<jml> ok. now that I've got a working build, I'm going offline to code & eat lunch
<jtv> abentley: hope you may be able to help with this... fun & games trying to run a codehosting cron job on dogfood: https://pastebin.canonical.com/30356/
<jtv> abentley: might that indicate that I'm running it on the wrong server, or on a branch that's on the wrong server?
<wgrant> dogfood doesn't have codehosting...
<wgrant> jtv: ^^
<wgrant> dogfood uses production codehosting.
<jtv> Ah, so I guess I'd need to run the script there, against the dogfood config.  :/
<wgrant> Which script?
<jtv> rosetta-branches
<wgrant> Ah.
<wgrant> That sounds disastrous.
<wgrant> You'll probably need to stuff with DB perms to let yo urun it from crowberr and blah blah.
<jtv> That too, probablyâbut at least that part is familiar.
<jtv> dogfood isn't.
<wgrant> dogfood isn't really familiar to anyone.
<wgrant> I'm not quite sure of the point of dogfood now that most stuff is reasonably testable locally. It seems like staging could be given a build farm for the rarer production-like test cases. staging has codehosting.
<jtv> And the work we're doing right now involves buildfarm, codehosting, and translationsâso maybe per-team islands are not an effective way to test this any more.
<wgrant> I would think not.
<wgrant> dogfood seems to mostly be used for QA now, which could be done on staging.
<jtv> If staging had a build farm.
<wgrant> AIUI it was often used for development or pre-merge testing previously.
<wgrant> Right.
<jtv> There's also the easy, fast-turnaround patching, database access etc.  I wouldn't discount thatâbut then we all need that AFAICS.
<wgrant> True.
<SlonUA> hi pals ... could u help me ... =)
<SlonUA> i have redirection to https://launchpad.dev from http://bazaar.launchpad.dev/~bla/vla/devel/files
<SlonUA> so, just redirection from  http://bazaar.launchpad.dev to  https://launchpad.dev
<SlonUA> i can't 'View the branch content'
<maxb> SlonUA: Please report what the command `getent hosts bazaar.launchpad.dev` prints for you
<wgrant> The problem is probably that branch-rewrite.py isn't running.
<SlonUA> maxb: 1sec
<SlonUA> 192.168.241.11  bazaar.launchpad.dev ... i use remote access
<maxb> Remote access in incompatible with bazaar.launchpad.dev
<maxb> *is
<maxb> "A full Launchpad development setup requires two IP addresses on the local machine, on which to run two HTTPS listeners - one for main Launchpad, and one for Loggerhead (code browsing) of private branches. As most developer workstations have only one non-local IP address, and as the second one is only required for Loggerhead on private  branches, you may well not bother to set up an additional IP address. If you do want to do this, identif
<maxb> y a suitable IP address, and add it to your machine's network configuration now. "
<maxb> quoted from Running/RemoteAccess
<abentley> jtv, sorry, just starting work now.  Looks like wgrant helped you?
<abentley> wgrant, +10 on staging buildfarm
<jtv> abentley: helped me realize what depths of shit I'm in, but yes.  :-)
<wgrant> abentley: It shouldn't actually be hard to get just a build farm up and running. A full archive is harder, though.
<maxb> SlonUA: Oh, sorry, except that's only required for *private* branches.
<SlonUA> maxb: i see .. thanks .. so ..
<maxb> SlonUA: Anyway, you should read Running/RemoteAccess again carefully, because I think you've not quite performed the tweaking described there correctly
<maxb> In particular, the "Amending the Apache configuration" part
<SlonUA> maxb: ohh .. thanks .. i will read again
<abentley> wgrant, Do you mean that it's hard to get a full copy of an archive on disk, or do you mean that building to archive is harder?
<wgrant> abentley: Lots of config.
<wgrant> The former, though.
<wgrant> Hmm, I guess we really need on-disk archives for SPRBs, since they need build-deps.
<wgrant> But for jtv's purposes it doesn't matter.
<wgrant> jtv just needs codehosting and a buildd-manager.
<jtv> Well... and a build slave, and the Translations db.  Not asking for much.  :)
<dhastha> error while make run:    http://paste.ubuntu.com/411614/
<wgrant> noodles775: New buildfarm project? Yes please...
<wgrant> They are sort of split between Soyuz, Code and slightly Translations now...
<noodles775> Yep.
 * wgrant wonders who needs to be convinced.
<noodles775> wgrant: we can point Julian to the bug next week, I think he'll be keen too.
<SlonUA> maxb: hi again
<maxb> hello
<SlonUA> maxb: i have to ip on my vm ... but https://bazaar.launchpad.dev/ work perfect ... but http://bazaar.launchpad.dev/ still redirect to https:/launchpad.dev
<SlonUA> i mean 2 ips
<maxb> pastebin your apache configuration
<SlonUA> maxb: http://pastebin.com/Qe3VpmHp
<maxb> SlonUA: huh. looks ok to me. Sorry, I'm not sure what to check next
<SlonUA> maxb: thank for look in .. i think i will hack launchpad-lazr.conf
<SlonUA> maxb: btw .... in section [codehosting]
<SlonUA> it will be correct: port: tcp:5022:interface=bazaar.launchpad.dev
<SlonUA> instead port: tcp:5022:interface=127.0.0.88
<jml> g'night all
<sinzui> flacoste, ping
<flacoste> hi sinzui
<sinzui> flacoste, I discovered the cause of the adaption insanity. We use provideAdapter() in the browser.pillar module, import that module cause the registrations to be wrong
<sinzui> flacoste, I fixed the issue by switching to ZCML
<flacoste> sinzui: provideAdapter in the browser.pillar modules sounds like a bad idea
<sinzui> we have a few navigation menus registered that way. It is all fine until you intend to subclass
<flacoste> sinzui: you mean registered using an annotation? or a call to provideAdapter direclty?
<sinzui> directly
<flacoste> directly in the module is never a good idea
<sinzui> so I have learned
<flacoste> well, unless gary disagrees
<gary_poster> reading back
<sinzui> provideAdapter(InvolvedMenu, [IInvolved], INavigationMenu, name="overview") was the code
 * sinzui wrote it
<sinzui> in any case, I have accidentally fixed an issue for EdwinGrubbs.
<gary_poster> +1 on zcml (or grok) in the abstract sounds safer.  I don't have enough context to have a strong opinion, but if there's a problem with what you did, sounds like you found a good solution. :-)
<sinzui> gary_poster, you can reproduce the insanity by adding
<sinzui>     import lp.registry.browser.pillar
<sinzui> to lp.registry.browser.productseries, then run
<sinzui>     ./bin/test -vvc -t pillar-views
<sinzui> gary_poster, As I said, I have learned something. I will propagate this in the reviewers meeting
<gary_poster> sinzui: great.  (sorry for latency, on call)
<mars> sinzui, around?
<sinzui> mars, I am
<mars> hi sinzui, have a moment to tell me your thoughts on killing the old style.css?
<sinzui> yes
<mars> sinzui, do you have mumble set up?
<mars> or Skype
<sinzui> I have mumble, but need to choose a public server
<mars> sinzui, why not the company server?  It has a channel
<sinzui> The email I got from IS implies they are not giving out anymore accounts until they implement the new login
<mars> sinzui, skype then?
<mwhudson> jelmer: hmmm... latest completed linux incremental import on staging -- 20 mins determining which revs to import, 30 mins importing, *90* minutes repacking, twice (!)
<mwhudson> hm
<mwhudson> actually i think it's packing three times
<mwhudson> that's pretty daft
<maxb> Is that the "pack every 1000 revisions" thing?
<mwhudson> no
<mwhudson> that wouldn't be so bad if it were happening, i think, because aiui that only packs the 1000 new revisions
#launchpad-dev 2010-04-10
<jelmer> mwhudson: we need a better story for dealing with batched imports
<jelmer> mwhudson: bzrlib should just have a base class that takes care of regular repacking, etc
<mwhudson> jelmer: please put your thoughts on https://bugs.edge.launchpad.net/launchpad-code/+bug/559678 :)
<mup> Bug #559678: incremental import does full repack 3 times <code-import> <Launchpad Bazaar Integration:Triaged> <https://launchpad.net/bugs/559678>
<thumper> the import is going well though...
<jelmer> mw	will do
<jelmer> mwhudson: iwll do
<jelmer> argh, lagggisch \
<jelmer> thumper: it's still a lot slower than a local import
<jelmer> thumper: that should be fixed once we start caching Tree objects
<thumper> when will we start caching Tree objects?
<mwhudson> the currently running import is way slower
<mwhudson> i think strawberry is a vm slice though, so that might be a factor
<jelmer> thumper: when jam finishes his pack collection refactoring
<dhastha> Need help: An error occurred during a connection to translations.launchpad.dev.
<dhastha> SSL received a record that exceeded the maximum permissible length.
<dhastha> (Error code: ssl_error_rx_record_too_long)
<rockstar> dhastha, are you using trunk?
<dhastha_> rockstar, yes i asked trunk. I restarted apached. now its working
<lifeless> mwhudson: jelmer: repacking should already be incremental only
<lifeless> jelmer: you're probably overriding / changing something.
<lifeless> fetch knows how to incrementally repack imported packs only.
<jelmer> lifeless: I'm doing manual commit_write_group calls every 1k revisions
<jelmer> lifeless: I'd like that logic to be in bzrlib rather than the individual plugins
<lifeless> jelmer: where the code is is orthogonal to whatever bug you have
<lifeless> why is it doing a full pack 3 times?
<jelmer> lifeless: I don't know why it's doing 3 repacks, I just see repacks regularly during imports
<lifeless> it should incrementally pack, not full repack
<jelmer> lifeless: I don't know if it does a full or incremental repack
<jelmer> it doesn't seem like a full repack to me, but it might be different for mwhudson
<lifeless> unless either a) you call pack(), or b) you insert a fetch stream with a differing serialiser (and that would be incremental using the pack hint)
<jelmer> lifeless: basically I'm gathering the pack hints for all write groups I commit (every 1000 revisions) and then specify that list of pack hints to target.pack()
 * jelmer clarifies his comment
<lifeless> jelmer: after 10 write groups, bzr will autopack anyway.
<lifeless> jelmer: I think you should be less agressive about triggering a manual pack
<lifeless> the only things you will get major benefit from packing are those things that have never been packed.
<jelmer> lifeless: I'm only triggering a pack after all revisions have been fetched and I specify the gathered pack hints
<lifeless> jelmer: yes, I get that. its still more agressive that needed, I suspect
<jelmer> lifeless: The fact that after 10 write groups bzr autopacks anyway is one of the reasons why I'd like to see a helper in bzrlib that can take care of determining how often to create write groups
<jelmer> lifeless: rather than optimizing the fetchers in bzr-git/bzr-svn and bzr-hg independently
<lifeless> jelmer: I'm not arguing for or against code placement.
<jelmer> lifeless: I realize that
<lifeless> jelmer: its a distraction in this discussion
<jelmer> I guess my comment probably isn't very relevant for this bug
<lifeless> which is, after all, on the weekend.
<lifeless> I suspect we need to change the pack hint definition slightly so you can tell if its the pack you added, or a packed version of same/similar.
<lifeless> if you wanted to do that, I don't think there are backwards compat implications, as its currently defined as an opaque hint
<jelmer> I might have a look at that at some point
<lifeless> so what I'm suggesting is
<lifeless> you do something like (if not hint.was_repack: hints.add(hint)
<lifeless> rather than your current simple 'add the hints together'
<jelmer> thanks
<jelmer> My brain clearly has stopped working so I'm getting some sleep, way too late here
<jelmer> gnight
<lifeless> gnight
<jml> sidnei, I've just submitted two really simple patches to zope-dev. Could I tempt you into landing them & maybe even releasing the respective components?
<sidnei> jml, appended to my todo. :)
#launchpad-dev 2010-04-11
<jml> sidnei: thanks
<dhastha> need help:  What is the actual command to run launchpad documentation? I used   "bin/test -vv -m lp.translations -t name-of-the-file.txt"
<dhastha> but it shows /usr/bin/test: extra argument `-m'
<dhastha> wgrant, How to run documentation using  /bin/test ?
<wgrant> dhastha: bin/test -vvt name-of-the-file.txt
<dhastha> wgrant, it return /usr/bin/test: missing argument after `browser-helpers.txt
<wgrant> dhastha: Why is it running /usr/bin/test?
<wgrant> It should be running ./bin/test
<wgrant> Which directory are you in?
<dhastha> wgrant,     Am in  ~/launchpad/lp-branches/devel/lib/lp/translations/doc. I have to run txt files in this. I already used ./bin/test   but it return bash: ./bin/test: No such file or directory
<wgrant> dhastha: cd ../../../..
<dhastha> wgrant, Now am in dhastha@dhastha-desktop:/$.     ./bin does not have test. It returns ./bin/test: No such file or directory
<wgrant> dhastha: That should have taken you to ~/launchpad/lp-branches/devel, not to /.
<dhastha> wgrant, see this error:  http://paste.ubuntu.com/412381/
<wgrant> dhastha: -t, not -m
<dhastha> wgrant, Running tests at level 1
<dhastha> Total: 0 tests, 0 failures, 0 errors in 0.000 seconds.
<dhastha> ** 1 import policy violations **
<dhastha> There were 1 imports of names not appearing in the __all__.
<dhastha> You should not import copy_field from lazr.restful.declarations:
<dhastha>     lp.registry.interfaces.distribution
<wgrant> dhastha: Your test specification did not match any tests.
<wgrant> bin/test -vvt brwoser.helpers.txt
<wgrant> s/brwoser/browser/
<wgrant> And also s/./-/
<wgrant> dhastha: Why did you add the '-m lp-translation'?
<dhastha> wgrant, Mr. Danilos told me  to run document using  "bin/test -vv -m lp.translations -t name-of-the-file.txt .
<wgrant> dhastha: 1) the '-m BLAH' isn't normally necessary -- it just restricts which modules are searched in; and 2) 'lp.translations' != 'lp-translation'
<dhastha> wgrant, can u see this pls  http://paste.ubuntu.com/412385/
<wgrant> dhastha: The test ran successfuly.
<thumper> mwhudson: it seems that the kernel import on staging is taking longer than an hour to get onto the worker causing a keyboard interrupt, and killing the import
<thumper> mwhudson: we should take a look at the branch on the import slave
<thumper> mwhudson: an hour seems somewhat excessive to grab the branch from another internal server
<wgrant> win 18
<wgrant> Argh.
<mwhudson> thumper: i d
<mwhudson> thumper: there are a few things we can do about that
<mwhudson> not build a working tree, stacking
<jml> sidnei: one of my zope patches has been landed and released, btw
<lifeless> cool
<jml> which means I'm still blocked on getting the other patch landed
<lifeless> :(
<thumper> morning
<adiroiban> hi, can someone help me and send this branch to ec2-test lp:~adiroiban/launchpad/bug-548999? Thanks!
<wgrant> james_w: Don't we just need to change SPRB to send the append_version properly?
<james_w> wgrant: yep
<james_w> shouldn't really be a difficult change
<wgrant> RIght.
 * wgrant just had a thought: didn't we discuss in Wellington having a special PPA with a non-ancient bzr-builder? We don't have facilities to do that yet...
<james_w> yeah, I don't remember if that was the solution we decided would be best
#launchpad-dev 2011-04-04
<lifeless> ugh
<lifeless> lazr stuff uses the whle zope stack doesn't it :(
<lifeless> can has review? https://code.launchpad.net/~lifeless/lazr.batchnavigator/keyoffset/+merge/56091
<lifeless> thumper: ^ if you're around, its trivial
<wgrant> lifeless: The branch name is a lie.
<lifeless> it is
<jelmer> lifeless, what does "newest = false" do?
<jelmer> nevermind
<jelmer> lifeless, r=me
<lifeless> jelmer: thanks!
<lifeless> seeking teddy bear for short design call
 * wallyworld hates debugging 200+ line doc tests
<lifeless> me too with s/200+ line//
 * wallyworld meant to type 2000 not 200
<huwshimi> A review for someone who has magic Monday reviewing powers: https://code.launchpad.net/~huwshimi/launchpad/ie-comment-button-414747/+merge/56094
<thumper> huwshimi: done
* wgrant changed the topic of #launchpad-dev to: devel in RC until r12735 releasable | https://dev.launchpad.net/ | On call reviewer: - | https://code.launchpad.net/launchpad-project/+activereviews
<huwshimi> thumper: Thanks :)
<thumper> np
<thumper> FFS
<thumper> pgAdmin III fails on natty with trust authentication :(
<wgrant> thumper: Were you looking at the hwdb OOPSes?
<thumper> wgrant: nope, not right now
<thumper> wgrant: I'm looking at branch stacking problems on rename
<wgrant> Someone was talking about it on Friday.
<wgrant> I don't recall who.
<wgrant> thumper: Great!
<thumper> I was talking about looking at it, but changed to look at the stacking issues
<wgrant> Ah, right.
<wallyworld> wgrant: lazr restful upgrade broke some tests. i've finally got them fixed. in ec2 now.
<StevenK> Can haz review? https://code.launchpad.net/~stevenk/launchpad/dsd-cant-request-resolved/+merge/56097
<wgrant> wallyworld: What broke/
<wgrant> ?
<wgrant> wallyworld: Note that we're RC, so it won't make it through PQM.
<wallyworld> wgrant: a bugs doctest. the issue was not our stuff but another change in 0.18.1 which short circuited patch requests
<wallyworld> wgrant: np. i'll land after rc if it passes ec2
<wgrant> Ah, right.
<wallyworld> wgrant: plus, the notifications stuff i did sometimes adds up to an extra 2 queries to a request to get the notifications from the session, so some query count tests needed tweaking
<wgrant> wallyworld: Only for webservice requests?
<wallyworld> wgrant: yes, afaik. only for requests come in via a WebServicePublicationMixin implementation
<wgrant> Great.
<wallyworld> StevenK: i'll grab it but someone will need to mentor it
<wgrant_> StevenK: Are you going to harrass mawson?
<wgrant> Hmmm.
<wgrant> It is tempting to batchnavigate BugTask:+index...
<wgrant> That and gandwanacide should just about finish off those timeouts.
<wgrant> Making it pretty and AJAXy can come later.
<poolie_> hi thumper
<lifeless> wgrant: that would be cool
<poolie_> hi lifeless,wgrant
<lifeless> wgrant: still have the milestone gangcrush to finish off bugtask:+index
<lifeless> hi poolie_
<wgrant> lifeless: Mm, that's not so bad.
<wgrant> Afternoon poolie_.
<lifeless> wgrant: the milestone thing... the thing that was breaking cve:+index ? :)
<lifeless> wgrant: its pretty shallow conceptually, but a bit ugly pragmatically
<StevenK> wgrant: Updating code now.
<wgrant> StevenK: Great.
<StevenK> wgrant: jtv's branch is merged as of r10383.
<wgrant> StevenK: I know,.
<wgrant> We avoided merging it to devel.
<huwshimi> Am I right in thinking we don't have any javascript link checks in Launchpad?
<wgrant> huwshimi: What do you mean?
<StevenK> wgrant: You said we need to check both, so should I avoid rf-get?
<wgrant> StevenK: r10383 introduced a new script, but also changed the old one.
<wgrant> StevenK: We need to test the cron.publish-ftpmaster still works, and that publish-ftpmaster.py does too.
<wgrant> Speaking of the devil.
<lifeless> StevenK: wgrant: also see my mail (just sent) to -devel about commingling of changes during complete downtime with stuff that really only needs partial downtime.
<poolie_> thumper, i was wondering about your branch name on bug 377519
<_mup_> Bug #377519: Stacked on location breaks if the stacked upon branch is renamed <branch-stacking> <lp-code> <Launchpad itself:In Progress by thumper> < https://launchpad.net/bugs/377519 >
<StevenK> wgrant: make stop'ing, and merging
<poolie_> are you planning to fix it by exposing the branch id?
<huwshimi> wgrant: Well during our build process or anything
<huwshimi> wgrant: Actually build would probably be a bad place to do it
<wgrant> huwshimi: How does one check JavaScript links?
<wgrant> Or do you mean lint?
<huwshimi> wgrant: Ah crap. Yeah lint
<wgrant> huwshimi: pocket-lint's JS checking is broken at the moment. sinzui was looking at that last week.
<wgrant> It's meant to work, but broke at some point.
<huwshimi> wgrant: Ah I see, there is a bug reporting the exact issue I'm trying to address (bug 742619)
<_mup_> Bug #742619: JS syntax checking is broken in packaging <Launchpad itself:Fix Released by sinzui> <pocket-lint:Fix Released by sinzui> < https://launchpad.net/bugs/742619 >
<jtv> StevenK: where do DSD.source_pub and DSD.parent_source_pub get set?
<jtv> Oh, stupid question
<jtv> They don't
<jtv> properties
<lifeless> poolie_: yes, he is
<poolie_> :/
<poolie_> i would have liked us to be involved in the preimplementation discussion of that
<poolie_> since we'll have to support it
<StevenK> wgrant: dogfood is up and running on r10388. Let's break it into tiny pieces.
<wgrant> StevenK: Have you run the cronscript there before?
<poolie_> i don't think it's an awful decision but i do wonder where the discussion happens
<poolie_> if any
<wgrant> poolie_: Why do you have to support it?
<StevenK> wgrant: Once, I think
<lifeless> poolie_: well, he was working on a complex scheme to rewrite all the branches affected by renames, and I asked if he'd considered just exposing the persistent id
<lifeless> poolie_: we chatted a bit, compared races, vuln, implementation overheads, and decided this was simpler
<lifeless> poolie_: here and in -ops for part of it
<poolie_> wgrant, because people will report bugs about this against bzr
<poolie_> just now?
<lifeless> poolie_: on friday
<lifeless> poolie_: so there are two discussions; meta and change specific. Lets have one at a time :) - which is your preference for first-up ?
<poolie_> either; let's do the meta one
<poolie_> i think, ideally, when people change an interface between two systems, they'd let people on the other side know
<poolie_> obviously in this case you know a fair bit about the impact on bzr
<poolie_> but, it's kind of disturbing
<StevenK> wgrant: Stale lockfile found, and removed.
<poolie_> i think we tend to post on the bug saying "i'm planning to fix this by responding to /+branchid/whatever and then rewriting the existing branches..." or whatever it is
<poolie_> it doesn't need to take a long tihme
<lifeless> poolie_: uhm, so I think that drives siloisation and adds a roadblock: we can debate the /size/ of the block (I'll acknowledge it could be quite low) - but what do we get for it ?
<lifeless> poolie_: I'd rather encourage engineers to gather enough data to be confident - and socialise things sufficiently to be confident - and then get out and do it
<lifeless> poolie_: I may be abbreviating the logic here; we can switch to voice if you want more bandwidth, or tell me to expand on stuff
<StevenK> wgrant: set -x in this script sucks
<StevenK> wgrant: And there's some partner errors
<poolie> i'm not saying anyone needs affirmative permission from the other team to change things
<poolie> that would cause siloization and blocking
<poolie> i just find it disturbing there's no communication at all
<lifeless> perhaps I'm categorising this differently
<poolie> probably
<lifeless> I see it rather like bzr choosing to use a different published api from lp - something that already exists
<poolie> we need to work hand in hand to give a good system here
<poolie> changing it without saying anything undermines the trust we need to do that well
<lifeless> I'm extremely surprised now
<jtv> I need to move to an English-speaking country.  Someone just told me something in Indonesian and I understood far too much.
<poolie> :)
<poolie> uh, maybe we should go to voice, but i should finish something here first
<poolie> sorry to leave you hanging
<poolie> wrt the actual change, is lp going to rewrite existing branches to use numeric ids?
<lifeless> thats not entirely clear
<lifeless> the basic plan is:
<lifeless>  - do the work to expose numeric ids safely [secure, private branches support etc] on http and sftp and bzr://
<lifeless>  - using a feature flag start offering the stacking targets numeric path rather than symbolic path, and see how it works
<lifeless>  - we can look at a mass rewrite in future if it works well
<poolie> ok...
<lifeless> by offering I mean in the policy config file that bzr reads from LP
<poolie> suggesting bzr stack on the numeric id
<lifeless> yes
<poolie> i think it's either something like this, or limiting renames
<poolie> in a way, this is like introducing immutable names and then another set of names on top of them
<lifeless> sure
<lifeless> though we already have aliases for branches, so there is very little new stuff here
<poolie> true
<poolie> there's another bug about recipes breaking when branches are renamed
<lifeless> its a duplicate
<poolie> is it proposed to also rewrite recipes into numeric ids?
<lifeless> recipes actually track branches by id in the db
<lifeless> the breakage is when the thing that a recipe branch is stacked on is renamed, AIUI
<wgrant> That's right.
<lifeless> the recipe branch becomes unusable
<wgrant> All the cases I've seen have been stacking breakage.
<poolie> oh, ok
<poolie> i see, it was just duped on the weekend
<wgrant> Yeah, I missed that last week :/
<poolie> will there be a public way to discover the id for a branch?
<poolie> (or maybe there already is in the api?)
<poolie> i guess this disturbs me because ... it seems like a lot of launchpad development happens behind closed doors
<poolie> even from the point of view of someone about as close to the project as you can be without actually working on the team
<poolie> i don't understand why people don't just say "the plan is X" on the bug
<poolie> (to go back to the meta question)
<poolie> it's fine for you guys to decide to just do this
<poolie> it just feels wasteful to have all these "oh, when did lp start doing X?" questions, even after reading most of the bug mail and all of the devel list
<lifeless> poolie: FWIW, I feel somewhat disconnected on recent bzr stuff; its much harder than I expected to track it unless I read all #bzr backlog
<poolie> it may be a grass-is-greener thing
<lifeless> poolie: I think a policy saying 'say what your intent is on the bug' is problematic in a few ways
<lifeless>  - changing bugs leads to 'me too' and 'have you thought of X' comments [even from you!] which can be a bit of a drain to deal with
<poolie> :) indeed
<lifeless>  - the change is going to get reviewed during code review, and if its 'feature scale' jml, and if its important blogged about on the blog anyway
<poolie> on the first point: that effect does happen, but it's pretty much saying "working in the open is annoying"
<lifeless>  - plans often don't survive contact with the enemy^Wcodebase; theres some sort of balance between communicating eagerly and drowning in out of date data
<lifeless> I disagree that its equivalent to 'working in the open is annoying'
<poolie> on the 3rd; i agree, i'm just wishing for a bit more
<poolie> well
<poolie> having bug reports, or talking about what you're doing at all, encourages people to comment
<poolie> some of those comments will be helpful and some will be noise
<poolie> so do you see any value in such a policy, or do you think it's just a waste?
<poolie> by policy i mean more like 'custom' than 'legally mandated'
<lifeless> I think its a waste - its treating a symptom not a cause
<lifeless> I don't know what the cause is
<poolie> anyhow, the root request is
<lifeless> but the fact you felt trust was betrayed by this suggests that its not at all about communication of intent
<poolie> i'd like to know if launchpad's going to change the way it interacts with bzr
<poolie> i realize in theory there is an abstraction, but in practice we deal with lots of things that cross that abstraction
<poolie> probably several dupes of that bug were originally filed against bzr, for instance
<lifeless> this gets my hackles up, and I don't know quite why
<poolie> some possibilities:
<poolie> * it really should be a shiny abstraction and i shouldn't need to know about it
<poolie> * i ought to read the lp commits too
<poolie> * i ought to find out about it on demand when it becomes a problem, not ahead of time
<lifeless> The closest thing I am bringing to mind when I think about similar problems I face is my first statement as TA nearly a year ago: (paraphrased) I'm a resource not a control
<poolie> * lp shouldn't describe this in the bug ahead of time but rather on the blog or release notes or something after it happens
<poolie> yeah, interesting comparison
<poolie> perhaps i should be asking myself why tim didn't choose to talk to or tell me about it
<poolie> or you
<poolie> ?
<lifeless> I didn't because right now is the first time I saw you since the discussion with Tim on Friday; and it had slipped my mind since then because its not in my critical-quadrant : its vapourware [as its not done yet and may have sandtraps] and its [in my assessment] low risk right up to the point where we start changing the policy
<lifeless> I was up to 4am this morning after Lynne had an asthma attack in the weekend - had to go into the 24 hour clinic.
<poolie> sorry to hear that
<lifeless> This tends to clear ones mind of dander
<lifeless> shes ok, its just context.
<lifeless> like, if the discussion happened monday and I saw you tuesday, vs this
<lifeless> also we have serious scaling issues with batch nav, and most of my [shredded] brain today has been looking at that, dealing with stale trees that won't bootstrap on lucid, design etc
<poolie> it's not an emergency, especially if it's not contemplated to be turned on suddenly
<lifeless> and coordinating the downtime on wed [which is largely a communication exercise, but we're still in the process of changing to make it faster and more robust, so requires care]
<poolie> so i guess overall
<lifeless> I think to do our job well, the LP team needs tools and space; you need to trust that if an engineer thinks an LP change will impact the bzr *team*, that they, or their line manager, will raise it.
<lifeless> just as LP needs to trust that bzr won't jump CPU use by 20% in a point release [for instance]
<lifeless> if the LP team doesn't have that trust from you at the moment, we need to figure out why that isn't the case, and what we need to do about it.
<poolie> yeah, that is the key issue
<poolie> i wonder
<lifeless> LP needs the room to make mistakes (and fix them rapidly) - because rapidly fixing things that do break is cheaper overall than avoiding every breakage [but different changes have different risks - there is no golden rule]
<poolie> i'm trying to think of any other changes in lp that have affected us since orlando and the squad structure
<poolie> i can't think of any
<lifeless> there is the beta redirect
<lifeless> bah
<lifeless> edge
<poolie> i don't think there have been a lot of changes there either way
<lifeless> which failed and was reverted
<poolie> yes, that was .. kind of ok, but kind of messy
<lifeless> its queued up in RT
<poolie> i guess at the moment i just come back to that being told this would change would have built more trust
<poolie> if you tell me about the probably-won't-hurt thing, i know you'll tell me about the dangerous things
<lifeless> I want the bzr integration in LP to be not at all special: special cases have higher friction and get less traction from self-directed folk unless they have a vested interest
<poolie> agree
<lifeless> poolie: still sounds like you have a lack of trust, or a need for control
<poolie> it does
<poolie> i have the impression lp has sometimes broken things in ways we could have avoided if we'd been asked
<poolie> so, that causes a feeling of not trusting that they will ask quite enough
<lifeless> it seems to me there are many things that could cause that:
<lifeless>  - top down management [above both bzr and lp | above the relevant engineer] deciding something
<lifeless>  - lack of knowledge [unknown unknown: engineers involved didn't know enough to know that they didn't know enough]
<lifeless>  - lack of attention to detail / cavalier changes / something
<lifeless>  - deadlines
<poolie> i guess, in particular, stacking has had a lot of bugs that cross both systems
<poolie> so, to me, it seems obvious to check on both sides before changing it
<lifeless>  - previous miscommunication [thought that they were doing the right thing on a prior agrement about direction]
<poolie> (even a pretty shallow change, like this)
<lifeless> well, you've gone from general issue [trust] to specific issue [stacking is a special case]
<poolie> we've probably seen all of those causes
<poolie> ah, i'm saying stacking seems like a good example of the general issue
<poolie> so looking at this list
<lifeless> so, telling you about small changes as a predictor for telling you about dangerous changes, is a poor metric
<poolie> this is perhaps a case where it's easy to put in more protection to guard against previous mistakes
<poolie> indeed
<poolie> (easy but possibly fallacious)
<poolie> so, in short, you think i should just trust lp to tell us if it is dangerous?
<lifeless> as its a poor metric, I don't think it would build trust - it would diminish it [because it would fail to predict, and you would feel betrayed]
<poolie> seems pretty easy on both sides
<poolie> how will we know if it's not working?
<lifeless> poolie: it? this specific change?
<poolie> let's back up
<poolie> > so, in short, you think i should just trust lp to tell us if it is dangerous?
<poolie> i meant that
<lifeless> ok
<poolie> that's a nice scalable algorithm
<poolie> i had got the idea that it wasn't working well enough
<lifeless> I expect failures in the algorithm.
<lifeless> I think thats ok.
<lifeless> I expect failures everywhere.
<lifeless> A /huge/ chunk of my non-timeout-coding-changes work has been about making it easier for us to change, to undo things so we can get back to known good states; to reduce the number of things changed at once to make diagnosis easier etc.
<lifeless> The risk is that a really dangerous change [high impact, high probability of going wrong, lots of user costs and black marks for lp and bzr] will get through
<lifeless> the reason I say that *that* is a risk, not 'some bad changes will get through', is because that sort of change can be hard to recover from.
<poolie> yeah
<poolie> of course in the past changes were very hard to undo
<lifeless> But OTOH thats why we have the oversight we do: most changes are seen by 2 people (engineer, manager, or engineer, reviewer), larger ones have sysadmins see them, more chance of random eyeballs on the diffm etc
<poolie> both through the deployment cycle, and because people would have moved on
<lifeless> right
<lifeless> thus the squad structure & nodowntime deployments
<poolie> so small-medium size problems can be tolerated more
<lifeless> right
<lifeless> not appreciated, but tolerated
<lifeless> what we trade to get that is [we hope] less churn on engineers, more focused time looking at the problem - things that *help* with flow, with analysis and good solutions.
<poolie> thanks for talking it over
<lifeless> I'm happy to
<poolie> so the short story is, relax, poolie, trust people will ask if something really does need external input or risk-assessment or coordination
<poolie> and if it turns out things are actually slipping through too much, we can debug then?
<lifeless> right
<poolie> fair enough
<lifeless> I realise the outcome of a messup is likely to be dramatic and widespread with bzr
<poolie> probably this new setup is different enough we should have a blank scorecard
<lifeless> ditto the archive / ppas though
<poolie> yeah
<poolie> and, rightly or wrongly, i feel we field a lot of the user support and complaints
<poolie> certainly some fair fraction of it
<poolie> anyhow, let's try that
<lifeless> I'm sure you do
<lifeless> the best thing we can do to reduce that is to fix bugs like this so that folk don't encounter problems ;)
<poolie> indede
<lifeless> we have mearly 400K branches on LP
<lifeless> thats a looooot of room for bugs to cause support tickets
<maxb> On the topic of mentioning possible solutions on bugs, I'm pretty sure I mentioned this possible option (branchid in URLs) on a bug several months ago :-)
<maxb> lifeless didn't like it much at the time, I recall
<poolie> i thought so too :) so that added to my surprise
<poolie> i thought there was a religious prohibition against exposing database keys
<poolie> though, i guess mps already break that, and perhaps also others
<StevenK> poolie: Talk to MPs
<lifeless> sometimes the simplest thing is the best thing
<lifeless> bug ids
<lifeless> merge proposals
<lifeless> questions
<StevenK> Bugs and questions are already "special" anyway
<StevenK> How else are you going to identify them?
<lifeless> StevenK: ask for a unique name
<poolie> so in some ways i actually wanted to ban renames, because they break people's bzr checkouts etc in confusing ways
<lifeless> StevenK: like e.g. stack exchange
<poolie> possibly avoiding that is better
<StevenK> lifeless: Oh, awesome. I just fixed bug nmnfgolwnpgwnepgnwgp
<maxb> Banning renames is quite harsh
<poolie> people can re-push if they want to change ownership etc
<lifeless> poolie: my experience with users is that when you ban something, they will do it by hand
<lifeless> poolie: and make the situation even worse
<poolie> lifeless, stackexchange's primary key is a number
<poolie> the name is just icing, i'm pretty sure
<poolie> yeah, maybe so
<poolie> i'm happy to give this a try
<lifeless> put it this way; if the lp team cannot make sensible decisions as a bzr hosting site, sourceforge and savannah have /no hope/
<poolie> or i should say happy for you to give this a try
<poolie> i think the main place this comes up is in changing ownership, and perhaps we should address that more directly
<poolie> :)
<poolie> indede
<lifeless> we need to debug and fix *that* problem, if it exists.
<maxb> poolie: Do you think this is actually going to break anything? Other than involving stacking at all, it seems pretty un-scary
<lifeless> stub: we forgot to speak on thursday
<lifeless> stub: is now convenient ?
<poolie> maxb, i reckon it will have a couple of non-critical bugs
<poolie> if i could tell you exactly what they will be i would
<lifeless> StevenK: you know that lp:~stevenk/launchpad/unmatched-brackets-annoyance is breaking a half-open range, right ?
<poolie> anyhow, to draw a line under it, i'm not trying to veto the change; thanks for talking it over; i'll trust lp more
<lifeless> poolie: cool, thanks.
<poolie> we can talk tomorrow if you'd like to catch up on bzr
<lifeless> poolie: thursday would be good actually, I have a tonne of bits to sort out
<StevenK> lifeless: wgrant already pointed out the fail and the branch and MP are already deleted.
<stub> lifeless: Gimme a minute... the coffee is still kicking in.
<lifeless> stub: ping me in a few then
<lifeless> hmm
<lifeless> UDS coming up
<lifeless> I suspect we'll see more of https://bugs.launchpad.net/launchpad/+bug/735970 soon
<_mup_> Bug #735970: Person:+specworkload timeouts <timeout> <Launchpad itself:Triaged> < https://launchpad.net/bugs/735970 >
<poolie> :) probably
<lifeless> ! 5% of open LP bugs were reported by me
<StevenK> I would have expected higher
<wgrant> Bah, only 3% are mine.
<lifeless> StevenK: it is, I'm rounding
<lifeless> 7.7M requests yesterday
<wgrant> Interesting.
<lifeless> non-opstats
<lifeless> previously these were spikes
<wgrant> Hmm. 3.1% of open bugs are mine, and 3.0% of all bugs.
<wgrant> Oddly stable.
<lifeless> I think we're either in UDS swell, or we've got some new api clients [most of the growth is api.]
<wgrant> But you said there were 500M new non-API requests.
<lifeless> yeah
<lifeless> we were tending to do 6M a day
<lifeless> so growth of 1.7M and 1/3 is non api. domains
<lifeless> wgrant: StevenK: wallyworld__: was my post on our capacity planning/deployment changes useful ?
<lifeless> should I do more such things? less? include other details?
<wallyworld__> lifeless: haven't read it yet sorry
<lifeless> wallyworld__: no need to apologise
<wallyworld__> lifeless: was it email? i'm a bit behind
<lifeless> email
 * wallyworld__ looks
<wgrant> lifeless: Wasn't really useful for me, since I've been following it all pretty closely anyway.
<wgrant> lifeless: But I think it was probably generally useful.
<lifeless> wgrant: NullBT is truely dead right ?
<wgrant> lifeless: The number of instances of that string in the tree is 0.
<wgrant> So I presume so.
<lifeless> we can trim a feature rule then
<wgrant> Ah, forgot about that.
<wgrant> Assuming we get nodowntime out before tomorrow morning, we should be able to drop the timeout on Wednesday.
<lifeless> wgrant: thursday
<lifeless> wgrant: IFF it looks good
<StevenK> To 8s?
<wgrant> Oh?
<wgrant> Oh, right, release.
<lifeless> wgrant: downtime please :)
<wgrant> lifeless: Also, we can drop the publicrestrictedlibrarian rule after the release.
<wgrant> lifeless: true, true.
<lifeless> cool
<StevenK> Can we remove edge now?
<lifeless> StevenK: no
<wallyworld__> lifeless: i found it really useful. i had never seen that information anywhere before (it may be on the wiki but i haven't seem it). thanks for posting it
<lifeless> StevenK: we need to do the redirect for everything but bzr and api clients
<lifeless> wallyworld__: cool, thanks
<StevenK> lifeless: I've been thinking about it. Couldn't we hack the prod apache's to also answer for edge?
<wgrant> lifeless: We can drop the Archive:+index override, I think.
<wgrant> The latest PPR shows nothing above 12s.
<wgrant> And even that is a tiny fraction of 0.1%.
<wgrant> 99% under 6.81
<lifeless> sob - src/lazr/batchnavigator/README.txt is not end user docs.
<lifeless> wgrant: cool!
<lifeless> StevenK: no
<lifeless> StevenK: for apis, the wadl includes the domain and thats hashed
<lifeless> StevenK: you cannot serve the wadl for domain A from a server that thinks its domain B
<lifeless> StevenK: and, LP doesn't believe in multiple root-urls, its a server wide config
<lifeless> stub: yo
<stub> lifeless: yo. Caffeinated, showered, human (fsvoh)
<lifeless> \o/
<lifeless> calling
<stub> lifeless: skype just hung
<lifeless> wops
<stub> Wow... needed -9
<lifeless> -o-
<lifeless> must be running zope
* henninge changed the topic of #launchpad-dev to: devel in RC until r12735 releasable | https://dev.launchpad.net/ | On call reviewer: henninge | https://code.launchpad.net/launchpad-project/+activereviews
<LPCIBot> Project windmill build #138: STILL FAILING in 41 min: https://lpci.wedontsleep.org/job/windmill/138/
<adeuring> good morning
<StevenK> henninge: Hi! Would you like to mentor wallyworld__'s review of https://code.launchpad.net/~stevenk/launchpad/dsd-cant-request-resolved/+merge/56097 ?
<henninge> StevenK: sure
<thumper> poolie: sorry, just back for a stint
<thumper> poolie: had caitlin at the doctor, and now she is at the emergency dept awaiting an xray
<thumper> I'm home making dinner now
<thumper> after talking with lifeless, we are going to stack on the branch id yes
<thumper> using a particular alias
<poolie> thumper, sorry to hear that
<poolie> hope she's ok
<poolie> that sounds good
<poolie> thanks for working on it, it's quite a headache bug
<henninge> StevenK: r=me
<StevenK> henninge: Thank you!
<lifeless> wgrant: https://devpad.canonical.com/~lpqateam/ppr/lpnet/latest-daily-timeout-candidates.html still has plenty of risky pages
<lifeless> oh wow
<lifeless> https://launchpad.net/ubuntu/natty/+cve
<lifeless> notice htat all the cves for a source package are listed against each bug for that selfsame package
<allenap> lifeless: Thanks for reverting the state of that test runner bug. I thought that --no-qa would leave the bug alone, but that was a stupid assumption. Gah, I hate tracking QA in bug tags.
<lifeless> no worries
<lifeless> if you don't want it linked, don't link it ;)(
<lifeless> noqa does what it says on the box - no qa needed
<lifeless> allenap: I'm really glad you did that patch
<lifeless> allenap: It might be worth mailing the list to let folk know that test ids are changing - it will affect folk using testr run --failing
<allenap> lifeless: Okay, good idea (to email). Fwiw, I like linking related branches even if they don't fix the bug; their presence may help others understand the problem better, and it's a shame not to link them because of the QA process we have.
<wgrant> bigjools: Evening.
<bigjools> helleau
<wgrant> bigjools: We're now running entirely on poppy-sftp, and it seems to be working fine.
<lifeless> allenap: its not the qa process thats the issue; its the 'close bugs that were linked
<lifeless> '
<lifeless> allenap: that is a separate bug related process
<bigjools> wgrant: \o/
<lifeless> s/bug/but/
<bigjools> wgrant: when it's settled for a bit we can stop it throwing oopses for badly-signed uploads
<wgrant> bigjools: Yup.
<allenap> lifeless: Okay, yes. At the moment we QA each branch, and tracking that in the bug seems wrong unless we restrict ourselves to one branch per bug. We may also want to consider QA on a fixed bug as a whole, and that makes sense to track on the bug. You know all of this already, I just want to express that I find it frustrating and interrupting.
<bigjools> wgrant: I'm rather pleased that it seems to only have had that one bug :)
<wgrant> bigjools: Well, plus at least one other non-critical.
<wgrant> (the DB user)
<bigjools> e_notabug
<jtv> henninge, dear former teammate, are you up for a review?  http://launchpad.leankitkanban.com/Boards/Show/14028617
<jtv> Ahem
<jtv> I *said*
<jtv> "Copy"
<jtv> https://code.launchpad.net/~jtv/launchpad/bug-745542/+merge/56106
<jtv> Good thing that didn't happen on schoolgirlswithdungeondimensionbeasts.com
<jtv> Hmm I sense potential there
<henninge> jtv: interesting board, much more sophisticated than ours.
<jtv> What do you know, that domain name's still available!
<lifeless> allenap: sure, I agree its suboptimal
<henninge> who would have guessed
<henninge> jtv: Anyway, I'll have a look at your MP in a minute.
<jtv> thanks
<lifeless> allenap: In this particular case though, even if we had an entirely separate qa system, we'd still not want the bug linked, because we close linked bugs
<allenap> lifeless: I guess there's a tension there: the convenience of automatically closing bugs restricts us from linking related branches to bugs.
<lifeless> allenap: yes
<allenap> If I break up a bug fix into multiple branches (via a pipeline say) to make review simpler I either have to file bugs for each step, or wait until the end to land it as one.
<lifeless> allenap: huh, no
<allenap> So there's a motivation to create larger branches to avoid the bureaucracy.
<lifeless> allenap: I disagree with your assertion that you need bugs for each step
<allenap> lifeless: If I want to land each step then I do.
<lifeless> allenap: why?
<allenap> lifeless: QA.
<allenap> And auto-closing.
<lifeless> so lets separate them
<lifeless> autoclosing is I think generally something we should remove
<lifeless> because more and more bugs are not fixed by the deploy
<lifeless> but by enabling a feature flag to expose the thing
<allenap> Good point.
<lifeless> for qa, we can only qa one thing on a bug at a time, but that doesn't preclude multiple landings
<allenap> Okay, fair. Without auto-closing that works.
<lifeless> as long as you say clearly somehow [e.g. we could use tags, or comments, or whatever] that the landing is incremental
<allenap> Ah, there is an incremental tag already :-/
<lifeless> then the human coordinating the deploy [ wgrant, me, matsubara etc etc etc ] can not close it for you
<allenap> Which I had forgotten all about!
<wgrant> allenap: --incr is a bit broken at the moment :(
<lifeless> the --incr to lp-land seems broken
<allenap> Oh, okay.
<lifeless> but thats a small matter of code
<allenap> Is that in qa-tagger?
<lifeless> IIRC yes
<lifeless> it currently doesn't tag at all
<lifeless> but it needs to say something like incremental-rev-XXXX, and then in the deploy report treat that as qa-untestable [or better yet have it be orthogonal to qa'ability
<lifeless> I haven't hit frustration-powered-fixing level yet
<lifeless> hopefully someone else will beat me to it
<wgrant> Exactly, I just go along and tag them untestable.
<allenap> I'll see if I can get bothered enough :)
<wgrant> It is irritating, but not worth fixing it :P
<jml> bigjools: hello
<bigjools> yo
<jml> bigjools: today is the day
<jml> bigjools: the day that is happening right now
<jml> bigjools: would you like to talk about derived distros?
<bigjools> jml: I would
<jml> bigjools: is now good?
<bigjools> it is
<bigjools> I'm enskypeinating
 * jml waits
<bigjools> jml: "call refused"
<deryck> Morning, all.
<henninge> jtv: r=me
<jtv> henninge: dankeschÃ¶n
<henninge> bitteschÃ¶n
<jtv> henninge-lunch: I followed up to your reviewâ¦ please let me know if that satisfies you!
<jtv> I'm off soon myself.
* benji changed the topic of #launchpad-dev to: devel in RC until r12735 releasable | https://dev.launchpad.net/ | On call reviewer: henninge, benji | https://code.launchpad.net/launchpad-project/+activereviews
<gary_poster> jml, hi.  Sorry, I did not forget to prep Friday for today's mtg, but I did run out of time.  I should have prep ready for you within an hour or an hour and half, with other stuff going on.  For better and worse, I don't think I have a lot to write up.
<jml> gary_poster: ok, thanks.
<dpm> hi all, I was trying to find out what's going on with the gdm translations, which don't seem to get imported from package uploads. My guess is that translations for gdm are now being imported from bzr upstream branches. However, the branch set for https://launchpad.net/ubuntu/natty/+source/gdm is not correct, as we're not using 'master' for natty, but 'gnome-2-32'.
<dpm> Could someone confirm if this makes sense?
<dpm> Can LP do imports of upstream branches other than the main upstream branch?
<james_w> dpm, for git?
<dpm> james_w, yeah, gnome's git in this case
<james_w> dpm, that's not currently possible
<dpm> thanks james_w, just to have some background info, do you (or anyone else) know if this is a planned feature or something getting worked on?
<james_w> dpm, it is being worked on, but I don't know what's left to do
<james_w> #bzr could tell you
<jelmer> james_w, dpm: It's very close
<dpm> james_w, ah, cool, thanks. jelmer, is this a bzr feature, rather than a LP one, then?
<jelmer> I have one lp:bzr branch that needs fixing and there's some fairly trivial changes that have to be made in bzr-git
<jelmer> dpm: Yes - you'll be able to specify e.g. "git://git.samba.org/samba.git,RELEASE_3_5" as a URL on Launchpad
<jelmer> sorry, "git://git.samba.org/samba.git,branch=RELEASE_3_5" as a URL on Launchpad
<dpm> jelmer, excellent. Is there any way I can track find out when the feature is completed? Is there e.g. an open bug I can subscribe to?
<jelmer> dpm: bug 380871 covers it (but is broader than just this)
<_mup_> Bug #380871: support for colocated branches <code-import> <lp-code> <Bazaar:In Progress by jelmer> <Bazaar Git Plugin:Fix Released by jelmer> <Bazaar Hg Plugin:In Progress by jelmer> <Launchpad itself:Triaged> < https://launchpad.net/bugs/380871 >
<dpm> ok, thanks jelmer!
<danilos> benji, btw, have time to review that branch of mine? :)
<benji> danilos: I'll trade reviews for voice lessons.
<danilos> benji, haha, it's a deal
<rvba> henninge-lunch: benji: Hi, I'd appreciate a review of https://code.launchpad.net/~rvb/launchpad/dds-add-missingpackages-page2/+merge/56136
<henninge> jtv-afk: agreed and replied
<henninge> rvba: looking
<deryck> henninge, ping for standup
<henninge> deryck: yeah, setting up mumble right now ...
<jml> anyone seen this librarian test failure before?
<jml> *ahem*
<jml> _this_ test failure: http://paste.ubuntu.com/589261/
<deryck> jml, I've seen that before.  Once.  It's been awhile, though.
<deryck> jml, I don't recall what it was, sorry.  I *think* it was masking another failure somewhere....
<deryck> but could be recalling wrong.
<jml> deryck: ok, thanks.
<jml> deryck: I can't see any bugs filed for it, so I'll file one just in case.
<deryck> jml, thanks, I know I didn't file a bug.
<rvba> jml: if I remember correctly I've been told this is a spurious failure ....
 * rvba checks his logs
<wgrant_> jml: Known spurious... I thought it was filed.
<jml> filed https://bugs.launchpad.net/launchpad/+bug/750274
<_mup_> Bug #750274: librarian.txt fails sometimes <build-infrastructure> <librarian> <spurious-test-failure> <Launchpad itself:Triaged> < https://launchpad.net/bugs/750274 >
<deryck> bac, ping
<bac> hi deryck
<deryck> hi bac.  lib/lp/app/javascript/tests/test_accordionoverlay.html is failing because the linked test file is missing....
 * bac looks
<deryck> bac, yeah, just wondering if this was intentional.  Or if I'm stupid and doing something wrong.
<bac> deryck: oh, i'm sure it isn't the latter
<bac> or the former, either
<deryck> heh
<bac> deryck: that test is deprecated and should've been removed
<deryck> bac, ok, cool.  I'll do a branch to drop it then.  Thanks!
<bac> deryck: thanks
<gary_poster> jml, I'm feeling like I'm behind the times today :-/ but I have a first draft of what I intended to have for you.  https://dev.launchpad.net/LEP/BetterBugSubscriptionsAndNotifications/FeatureReviewNotes
<gary_poster> I suspect the parts with "XXX" are the parts that you want to see most.  I'm trying to get those filled in now.
<bac> mrevell: ping
<mrevell> hello bac
<jml> gary_poster: thanks. :)
<wgrant_> sinzui: Morning.
<sinzui> I am afraid it is
<bigjools> jml: I updated the LEP with my NRCP interpretations
<wgrant_> sinzui: spm unbroke lists.staging.launchpad.net this afternoon, so I was able to mostly QA the mhonarc changes.
<sinzui> \o/
<sinzui> how was it unbroke?
 * sinzui was going to ask a losa to make ~launchpad the team to access all lists in staging to satisfy the retarded code
<wgrant> sinzui: It was running the new apache_openid, rather than the old mpopenid.
<wgrant> sinzui: Yeah, I suggested whitelisting ~launchpad, but then decided we should try to unbreak it properly first.
<sinzui> thank you very much. I think staging should be running what we run in production
<wgrant> sinzui: So the old module (dpkg name apache-openid, python name mpopenid) was installed, and the config updated to match the old values in prod.
<wgrant> This means that private teams are broken, but at least public teams work :)
<wgrant> sinzui: There is some very odd syncing going on.
<wgrant> sinzui: It has twice today overwritten the archives with prod ones... but apparently not the mboxes.
<sinzui> ?
<wgrant> https://lists.staging.launchpad.net/launchpad-users
<wgrant> I've twice sent mail to that list today.
<sinzui> I think I am familiar with the script that should do that
<wgrant> Each time it updated to the new theme, using an mbox from December + my new message.
<wgrant> But then an hour later it was on the old theme.
<wgrant> And I tried creating new lists.
<sinzui> I can follow up...since I had dedicated my day to getting that qaed
<wgrant> They worked for a while, but then the MX rejected them after the archive was erased.
<wgrant> (the ML still exists in the DB)
<wgrant> So I think the mailman update script in fact does a mailman restore, even if it was just for an upgrade.
<wgrant> sinzui: It would also be great if this was easily transferrable to qas.
<sinzui> The restore script does an rsync
<wgrant> It should probably only do that on a full restore.
<sinzui> I agree, and will work on the test.
<wgrant> Thanks :)
<wgrant> LOSA ping.
<sinzui> The test is one of the few that still want to be on the mailman layer, but a solid unitest on a lower layer is all that is needed
<Chex> wgrant: hello there
<bigjools> ooo what is this new AJAX Log
<sinzui> Maybe I will abolish the mailman layer tests today. That would be very satisfying
<wgrant> Chex: Hi! We merged db-stable into devel 12ish hours ago, so qastaging needs a bit of hand-holding which spm didn't have time to complete.
<wgrant> Chex: I believe the DB is probably patched -- could you try restarting the services and see if they come up?
<Chex> wgrant: sure thing, hang on
<wgrant> bigjools: I take that to mean that the deployment just finished.
<wgrant> bigjools: The AJAX log isn't completely awesome yet, but it will be soon.
<wgrant> bigjools: (eg. it doesn't log URLs yet, since YUI makes them impossible to get)
<bigjools> is it just showing the XHR requests?
<wgrant> Right.
<wgrant> Timings and status and OOPS information for now.
<wgrant> (yes, a bug page makes like 10 requests on load)
<bigjools> it's one way of not timing out single requests :D
<bigjools> it needs a close button or click-outside-the-box-to-close event
<wgrant> Does anyone have a bug description that they need to edit?
<Chex> wgrant: ok, looks like the restart worked OK
<wgrant> Chex: Indeed, thanks a lot.
<benji> danilos: I got some tests failures and had a couple of very small comments on https://code.launchpad.net/~danilo/launchpad/refactor-overlay-setup/+merge/56146
<wgrant> deryck: Thanks for cleaning up the windmill stuff. How did it go in ec2?
<deryck> wgrant, 5 failures remaining.  Working through them today, then handing off to wallyworld when I EOD.
<wgrant> deryck: Great!
<wgrant> Its return to the default test run will be a mixed blessing, though :(
<wgrant> Slooow.
<deryck> yeah, it really is.
<deryck> Once this is done, I turn my attention to an alternative.
<danilos> benji, regarding the switch indentation, I let Emacs js-mode handle it, I don't have a preference, and re the comments, they were in there originally, I believe they contain useful information
<danilos> benji, perhaps they should not be written as uncommented code though :)
<danilos> benji, I'll also go through all the method/function comments and fix them as you suggest
<benji> danilos: re. indentation; I only have a slight preference so I'm good with it staying
<benji> re. comments: right, just a default that throws an exceptoin would be good
<benji> danilos: re. comments: do you know if there is any reason to have the double astrisks?  I figured there is some API documentation tool (akin to javadoc) that extracts them, but I don't know that we actually use one
<danilos> benji, yeah, there seems to be a pattern, I am unfamiliar with it myself as well
<deryck> oh crap.  We're in rc only.
<deryck> I had forgot we still bother with that. ;)
<bigjools> not sure why we do it at all
<gary_poster> jml, is there a very focussed LP team of yours that I can give the feature flag for the structural subscription stuff?  Ideally, for instance, you'd have a "LP product team" team or a "LP feature review" team
<jml> gary_poster: ~pelicans
<gary_poster> cool thanks jml
<gary_poster> jml, https://launchpad.net/~pelicans does not exist
<jml> gary_poster: it's private.
<gary_poster> oh
<jml> gary_poster: in which case, let's simplify to "no"
<gary_poster> :-)
<gary_poster> could you make a feature review team, then, jml?
<gary_poster> or do something else like that?
<jml> gary_poster: yes, but not right now
<gary_poster> ideally it would be easy for you to add and remove people specifically for this purpose
<gary_poster> ok, cool jml.  lemme know when and I'll finish up setting up the wiki page.
<jml> gary_poster: will do.
<gary_poster> thx
<rvba> henninge: I've pushed 2 minor changes to my branch since you took it for review
<henninge> rvba: np, still on it.
<rvba> henninge: great, thx
<henninge> rvba: what's your revno?
<rvba> henninge: 12731
<rvba> henninge: oops sorry: 12733
<henninge> ah, I was starting to wonder.
<henninge> rvba: that's the one I got. ;)
<rvba> ok, everything is ok then
<henninge> rvba: is there a derived series in the demo data or how do I create one?
<rvba> henninge: yes there is https://launchpad.dev/deribuntu/deriwarty/+missingpackages
<rvba> henninge: you have to enable the feature flag to access this page "soyuz.derived-series-ui.enabled default 1 on"
<henninge> I did but the page is pretty boring ... ;)
<rvba> henninge: indeed, there is no difference of the right type to show up on the new page ... one has to change the status of some of the existing differences
<henninge> rvba: where/how do I do that?
<rvba> henninge: hang on
<rvba> henninge: update distroseriesdifference set difference_type=2;
<rvba> this will change the type of the existing differences so that they show up on the new +missingpackages page
<henninge> oh, I cannot do that in the UI?
<henninge> ok
<henninge> rvba: ah, now I see stuff
<rvba> henninge: AFAIK no, not directly. These differences are computed by jobs.
<henninge> ah, I understand
<rvba> I probably should have included a few differences of the right type in current-dev.sql for good measure ;-)
<henninge> I remember noodles worked on this
<henninge> rvba: there is an UI issue.
<henninge> also, it is confusing that the comment is empty.
<rvba> henninge: yes, the comment slot needs some more work.
<rvba> henninge: what is the issue?
<henninge> http://people.canonical.com/~henninge/screenshots/review-rvba.png
<henninge> rvba: The "blacklisted" radio buttons on the right.
<henninge> They look completely misplaced.
<rvba> henninge: I see ... I haven't touched these yet (they pre-existed before I started to work on this branch). But I completely agree with you.
<rvba> s/this branch/this page/
<henninge> rvba: did they actually look this way too, overlaping with the comment?
<henninge> also they are aligned to .... nothing
<henninge> oh, right-aligned, I guess
<rvba> yes
<henninge> strange
<rvba> but I completely agree, this is very strange
<henninge> rvba: to me it is strange that this passed UI review!
<rvba> the reason might be that this is a feature that is in the works
<henninge> rvba: Is it OK if we continue this tomorrow? I won't get through your branch before I have to EOD.
<henninge> rvba: possible
<rvba> will be tested on a very few number of users first
<rvba> do you have a suggestion to fix this in a way that would be consistant with the rest of the UI?
<henninge> I am not even clear what it is referring to
<henninge> is it a filter?
<rvba> no, is a status for the difference
<rvba> you can change the status to blacklisted so you will not see it in the list ... next time you load the page
<henninge> ah, I remember the orignial page that Michael did.
<rvba> I guess he's the one who did the floating blacklisted slot in the first place :-)
<rvba> henninge: note that this is less ugly when there is more data to display (https://dogfood.launchpad.net/deribuntu/dangerous/+localpackagediffs)
<henninge> rvba: yes, that looks better.
<henninge> rvba: but I don't see the Blacklisted thing, probably because of permissions.
<henninge> rvba: but that's not our subject right now
<rvba> henninge: indeed
<henninge> rvba: I'll have to continue tomorrow morning, I am sorry. I have to leave soon. I hope that is ok.
<rvba> henninge: sure, no problem. Just ping me tomorrow.
<henninge> cool, thanks
* henninge changed the topic of #launchpad-dev to: devel in RC until r12735 releasable | https://dev.launchpad.net/ | On call reviewer: benji | https://code.launchpad.net/launchpad-project/+activereviews
<danilos> benji, I've fixed all the problems you reported, hopefully the branch is much better now: I've only left the lint issues, though, they are in very long string construction
<danilos> benji, (test failures due to my loading the test_ss.html from the wrong branch :/)
<benji> danilos: cool, taking a look now
<danilos> benji, heh, a left-over commented-out "debugger;" statement, removing that one as well
<benji> cool
<benji> danilos: approval plus a patch for some more-or-less lint issues: https://code.launchpad.net/~danilo/launchpad/refactor-overlay-setup/+merge/56146
<allenap> benji: Thanks for the review :)
<benji> my pleasure
<danilos> benji, thanks; btw, what's your take on "concatenating strings with + is slower than array.join('')"? probably doesn't matter in this case where we've got ~60-70 strings being joined, but maybe worth considering
<benji> danilos: good question; in modern CPythons it's roughly the same speed, I don't know about the various JS engines out there.  As you say, since we don't do it in any tight loops or anything it probably doesn't matter
<benji> on the other hand, the code might look /slightly/ better with trailing commas instead of trailing <space><plus>, but that's pretty minor
<danilos> benji, fwiw, I dislike multi-line vars as well, but I've seen them used so I continued the practice; I have not yet developed my personal JavaScript style and I am unsure about what is our existing one
<benji> danilos: yep, I figured the code in question already existed and you just moved it, I wonder if pocketlint would have found it
<deryck> we seem to have lost the "this is a duplicate bug, comment only if ..." message.
<deryck> does anyone know, was this intended?
<deryck> sinzui, do you perhaps know anything about this ^^?
<sinzui> deryck: I do not know about that. I saw it a few days ago
<deryck> sinzui, ok, no worries.  I'll ask around.
<deryck> I'm guessing lifeless knows.  But it's still too early for him.
 * deryck smells performance improvements at play
<sinzui> deryck: I see it on this bug that I marked as a dupe a few hours ago: https://bugs.launchpad.net/launchpad/+bug/749419
<_mup_> Bug #749419: Cannot undo bug status change if I lack permission to set the previous status <Launchpad itself:New> < https://launchpad.net/bugs/749419 >
<deryck> sinzui, yeah, it's still on lpnet, but not on qastaging.  I only notice because of failing windmill tests.
<sinzui> wtf. Someone linked Lp source code to a firefox plugin
 * sinzui gets his axe
<deryck> heh
<deryck> I wondered about that, too.
<deryck> ok, I feel like I'm going crazy.  Now I see the warnings again.
<james_w> deryck, seen http://www.phantomjs.org/ ?
<deryck> james_w, ah no.  I haven't.  Will look closely at that.
<james_w> deryck, probably not equivalent to windmill/selenium, but if you wanted to build your own :-)
<james_w> someone may already be doing that though
<deryck> yeah, maybe so.
<deryck> I certainly don't want to build it.
<deryck> I'm starting to think YUI test in the context of the app server is really the best idea.
<deryck> unless you really want end to end smoke test kind of thing.
<bac> benji: i was able to finish adding the edit links everywhere...so there's nothing for you to do -- except review it, perhaps?
<bac> benji: a MP has been submitted but hasn't shown up yet.
<benji> cool, I'm working on one now and I'll do yours next
<bac> benji: great
 * bac afk for a bit
<lifeless> morning
* lifeless changed the topic of #launchpad-dev to: Performance Tuesday! | devel in RC until r12735 releasable | https://dev.launchpad.net/ | On call reviewer: benji | https://code.launchpad.net/launchpad-project/+activereviews
<lifeless> statik_: ping ?
<benji> ok bac, I'm done with https://code.launchpad.net/~bac/launchpad/add-edit-links/+merge/56247
<bac> thanks benji
<mtaylor> lifeless: do you know anything about the launchpad openid group extension as it relates to jenkins?
<lifeless> mtaylor: not really, other than that we funded the openid plugin
<mtaylor> lifeless: ok. I was trying to figure out why group information is coming through properly for me but not for other people
<lifeless> well
<lifeless> you need to be registered
<lifeless> IIRC that is, we only give out group info to trusted agents
* benji changed the topic of #launchpad-dev to: Performance Tuesday! | devel in RC until r12735 releasable | https://dev.launchpad.net/ | On call reviewer: - | https://code.launchpad.net/launchpad-project/+activereviews
<lifeless> jcsackett_: don't forget to qa on db-stable ;)
<poolie> hi lifeless
<lifeless> hi poolie
<poolie> we can have a catch up call today if you like
<lifeless> thursday would be better
<lifeless> as I mentioned yesterday, I'm flat out just now
<poolie> oh, sure
#launchpad-dev 2011-04-05
<lifeless> how does one specify a new dependency to buildout ?
<mwhudson> lifeless: setup.py
<lifeless> mwhudson: I'm looking in lazr.batchnavigator
<lifeless> mwhudson: I have a test-only dependency, or should I not worry about the subtlety
<mwhudson> setup.py *might* support test_requires?
 * lifeless goes with damn-the-torpedoes
<sinzui> thumper: wgrant mumble?
<thumper> sure
<wgrant> Sure.
<thumper> wallyworld: you around?
<wgrant> He's on mumble.
<wallyworld> yep
<wallyworld> thumper: sinzui: i can't hear you guys. looks like my audio is foobar again. will have to restart :-(
<thumper> wallyworld: ok
<sinzui> wgrant: does mumble need stabbing?
<wgrant> sinzui: Thumper and I are talking OK...
<thumper> sinzui: I can hear him
<thumper> sinzui: and you
<sinzui> maybe I fell off the earth
<sinzui> I feel like that actually
<huwshimi> sinzui: we can hear you
<thumper> sinzui: it is just you
<sinzui> fab
<sinzui> mumble gave up on it's own
<wallyworld> thumper: sinzui: sorry, my audio has totally died for some reason :-(
<sinzui> wallyworld: okay. mumble booted me from the conversation twice
<wallyworld> sinzui: it's not mumble. i have no output at all. speakers/headphone all silent :-(
<poolie> lifeless,  shall we stick with https://bugs.launchpad.net/launchpad/+bug/525758
<_mup_> Bug #525758: A project's branch privacy policy is not visible on the branch settings page <confusing-ui> <lp-code> <privacy> <Launchpad itself:Triaged> < https://launchpad.net/bugs/525758 >
<lifeless> cyes
<lifeless> yes
<lifeless> poolie: mail sent; I hope I captured everything - if I didn't, please correct me ;)
<poolie> very nice, thanks
<poolie> ok, https://bugs.launchpad.net/launchpad/+bug/750871
<_mup_> Bug #750871: want an api and ui to set branch privacy policy <Launchpad itself:Triaged> < https://launchpad.net/bugs/750871 >
<poolie> lifeless, it looks like you could create a Branch through the api with private=true
<poolie> which would avoid a race of making them private after they're created
<poolie> in that case it would be enough to just have bzr call it
<poolie> (arguably a bit messy to need to do this as well as sshing in, but not too bad)
<poolie> do you think that won't work?
<lifeless> you still need a policy setup to permit privacy at all
<lifeless> + having to use bzr push --use-existing-dir
<lifeless> wgrant: ugh
<lifeless> wgrant: we broke 'redirect to a real context'
<lifeless> https://bugs.launchpad.net/launchpad/+bugs/5011
<poolie> well, if bzr called the api it would hopefully be smart enough to expect a directory there
<wgrant> lifeless: We didn't.
<lifeless> wgrant: it was already broken ?
<wgrant> lifeless: The container is '+bug'
<wgrant> lifeless: Not '+bugs'
<poolie> ah, so the default policy, even if you have an entitlement to private branches, is to disallow them?
 * lifeless fails
<lifeless> poolie: right
<lifeless> poolie: noone can has private branch until a policy is setup permitting a group they are in to have them
<lifeless> poolie: I think 750871 is a dupe of 525758 isn't it ?
<lifeless> ah, no its not
<poolie> they're arguably the same, but  one is "show it" and the other is "let me change it"
<poolie> i was going to file one more for "change it per branch"
<lifeless> well, that already works, just only one way
<poolie> huwshimi, it seems like pressing enter in the tags field no longer saves the changes
<lifeless> a search for 'branch privacy' is pretty interesting
<huwshimi> poolie: Can you file a bug please?
<poolie> i will
<poolie> oh also https://bugs.launchpad.net/launchpad/+bug/252172
<_mup_> Bug #252172: Changing a branch's status/privacy isn't obvious <confusing-ui> <lp-code> <privacy> <Launchpad itself:Triaged> < https://launchpad.net/bugs/252172 >
<poolie> lifeless, is bug 444498 a dupe of bug 347772?
<_mup_> Bug #444498: branch privacy for packages (of projects with privacy enabled) <lp-code> <Launchpad itself:Triaged> < https://launchpad.net/bugs/444498 >
<_mup_> Bug #347772: Privacy policy support for package branches <disclosure> <lp-code> <package-branches> <privacy> <Launchpad itself:Triaged> < https://launchpad.net/bugs/347772 >
<lifeless> poolie: no
<lifeless> but I just edited it before you asked anyhow :)
 * wgrant hunts for someone to review https://code.launchpad.net/~wgrant/launchpad/bug-750640/+merge/56276
<lifeless> poolie: 444498 is about extending privacy to package branches when the upstream has a commercial ticket
<lifeless> 347772 is about permitting distro contributors to do private branches for embargoed changes (BFB into primary archive will need that)
<lifeless> wgrant: is +        fields.append('Source', source) right ?
<lifeless> wgrant: if source is None, will that DTRT
<wgrant> lifeless: Yes, see eg. the handling of Essential.
<poolie> huwshimi, https://bugs.launchpad.net/launchpad/+bug/750880
<_mup_> Bug #750880: pressing enter in bug tags field doesn't save changes <Launchpad itself:New> < https://launchpad.net/bugs/750880 >
<lifeless> wgrant: its almost worth doing a Matcher there
<lifeless> wgrant: but your helper is awkward, a nicer spelling is included in my review
<wgrant> Thanks.
<lifeless> huwshimi: hi
<huwshimi> poolie: Does that happen when you hit the save button as well?
<huwshimi> lifeless: Hello.
<lifeless> huwshimi: I don't recall any feedback from you on my suggestion about your easy-ui bugs & priority
<wgrant> lifeless: Treating it as a dict is wrong, but I guess it's OK for tests.
<lifeless> huwshimi: so I thought I'd poll you for some
<poolie> huwshimi, there is no save button
<poolie> as i suspected it's due to a problem in tag suggestion
<lifeless> wgrant: your test treats it as one :)
<poolie> or so it seems
<huwshimi> poolie: OK then this looks like a new issue
<lifeless> wgrant: so either match on a vector (and definitely do a matcher), or be much more pithy :>
<poolie> i think i'ts fallout from the previous fix
<poolie> hm so https://bugs.launchpad.net/launchpad/+bug/120306 suggests there is already a ui for this
<lifeless> poolie: tag edits are saving for me
<poolie> it may be to do with using a new tag?
<lifeless> poolie: yes there is, as I said. But public->private is deliberately not supported
<poolie> then you can't toggle it, can you? :)
<lifeless> poolie: no :>
<lifeless> well
<lifeless> you can toggle once
<lifeless> anyhow
<lifeless> I think that we need to revisit it
<poolie> ok, so i'll file a new one, mentioning it may be complex
<lifeless> but I don't think its a proximate cause of pain
<lifeless> blah
<lifeless> the words are not coming today. I mean 'changing that wouldn't make a lot of difference'
<huwshimi> lifeless: I'm not sure it's appropriate for us to elevate these bugs. If we did we would have to raise up a few hundred bugs to high.
<lifeless> huwshimi: but you think they are very important for the user experience, right? Or did I misread your email ?
<huwshimi> lifeless: The idea is not that people should have to do these bugs, but a way of signifying that these are some good bugs to look at if people want to do some UI work.
<lifeless> huwshimi: I think this is a flawed way to analyze the problem
<lifeless> huwshimi: You made a compelling argument that fixing these bugs is both relatively cheap and will have a large impact on usability and product perception.
<poolie> lifeless, it's exactly the button these users are wanting to push
<poolie> i agree it would be better if they'd never gotten in to this state
<poolie> anyhow, https://bugs.launchpad.net/launchpad/+bug/750889
<lifeless> poolie: but they want to push it because another critical step has been missed.
<_mup_> Bug #750889: need to be able to make public branches private <Launchpad itself:Triaged> < https://launchpad.net/bugs/750889 >
<huwshimi> lifeless: In as much that fixing any UI bugs is a good thing.
<lifeless> huwshimi: well, the UI isn't a special case for users: a bug is a bug
<lifeless> hard to use is hard to use
<poolie> ok now to fix the actual branch
<lifeless> huwshimi: If I understand correctly, you are postulating a disconnect between 'this is important (to the UI)' and 'this is important'
<lifeless> huwshimi: is that correct?
<huwshimi> lifeless: I don't think the bugs I've tagged are necessary any more important than other UI bugs (in some cases exactly the opposite). All I am doing is identifying bugs that are cheap to fix.
<lifeless> huwshimi: now I'm really confused
<huwshimi> lifeless: One sec.
<StevenK> lifeless: You've screwed up the topic a bit
<lifeless> huwshimi: you said in your email that these were high leverage bugs, holding three properties: easy to fix, only need ui changes, make a big difference to users
<lifeless> bwah
* lifeless changed the topic of #launchpad-dev to: Performance Tuesday! | https://dev.launchpad.net/ | On call reviewer: - | https://code.launchpad.net/launchpad-project/+activereviews
<poolie> lifeless, shall we have a bug for 'bzr push --private'?
<lifeless> poolie: I think that would be good
<poolie> istm that's a bit less important than getting the policy set right
<poolie> but may be worth filing anyhow
<thumper> lunch I think before tackling the inmemory xmlrpc mock for the stacking changes
<lifeless> huwshimi: the ajax log stuff is awesome
<lifeless> I've having a few wtf moments
<lifeless> (at latencies)
<huwshimi> lifeless: Yeah it's nice to see that stuff all the time. We're actually under for quite a few queries, and most of the rest are are not a long way over
<lifeless> huwshimi: I had some come in at 24 seconds this morning ><
<huwshimi> lifeless:  Oh really!@
<lifeless> yeah
<lifeless> I may have hit browser concurrent connection limit or something
<lifeless> I opened a few tabs
<lifeless> wgrant: why isn't it a regression ?
<wgrant> lifeless: What regressed/
<lifeless> wgrant: our hiding of merge proposals
<lifeless> wgrant: when we added a feature (prereqs) we backslid on disclosure
<wgrant> Mmmmm. Maybe barely.
<wgrant> We're not really disclosing stuff we shouldn't, though.
<wgrant> So it's more that prereqs are broken when the prereq is private but the proposed branch is not.
<wgrant> I think that makes it a broken feature, not a regression.
<lifeless> disclosure is defined as being viral
<lifeless> thats not working here
<lifeless> thats a regression
<wgrant> Viral to what extent?
<wgrant> A private bug does not make a branch private.
<wgrant> A private branch does not make a bug private.
<wgrant> A private prereq should possibly not make an MP private.
<lifeless> a branch link is invisible if either the branch or the bug is invisible
<lifeless> a mp is invisible is any of the items on it are invisible
<wgrant> Except a bug or blueprint or
<lifeless> one step removed
<lifeless> mmm, viral is the wrong word
<lifeless> I would love to see 4 /    0  None fixed
<wgrant> Hmm, it's the librarian.
<wgrant> Erm.
<wgrant> https://lp-oops.canonical.com/oops.py/?oopsid=OOPS-1920MPJ4
<wgrant> Reads the same file 376 times.
<lifeless> wgrant: I actually meant the 'None' part :)
<wgrant> lifeless: Do you know of a bug for late evaluation of attachment LFCs on BugTask:+index?
<lifeless> no
<wgrant> Have you looked at that at all?
<lifeless> shouldn't be happening
<lifeless> perhaps fallout from librarian code changes you did?
<wgrant> Which?
<wgrant> It only affects attachments that appear in comments.
<lifeless> removing the feature flag
<lifeless> oh
<wgrant> No, that's unrelated.
<lifeless> hadn't seen that at all
<wgrant> https://lp-oops.canonical.com/oops.py/?oopsid=OOPS-1920A1682 for example.
<wgrant> 3s selecting attachment LFCs.
<wgrant> Fixing now.
<lifeless> win
<wgrant> Rendered in 10.9s for me just now :/
<lifeless> why does it need the LFC's at all ?
<wgrant> (with ?comments=all)
<wgrant> It shows the file size.
<lifeless> whatnow
<wgrant> Xorg.0.log (23.1 KiB, text/plain)
<lifeless> in the comment ?
<wgrant> This is in the comments, not the portlet.
<thumper> hmm...
<lifeless> 178 queries/external actions issued in 5.27 seconds
<wgrant> lifeless: ?comments=all
<lifeless> we really should show the real url in the oops report
<lifeless> 521 queries/external actions issued in 10.93 seconds wheeeee
<wgrant> lifeless: The query string is there if you look hard, but yes.
<wgrant> lifeless: Exactly.
<lifeless> wgrant: gets truncated sometimes
<lifeless> wgrant: so, are you going to eager load, or make the attachment the same as the portlet
<wgrant> lifeless: It should be really cheap to eager load.
<lifeless> cool
<wgrant> I mean, LFC isn't that fat.
<wgrant> Hmm, except the hashes. But it's still <100kb of extra data.
<lifeless> sure
<wgrant> Bug search really, really sucks.
<wgrant> This will possibly fix half of the BugTask:+index timeouts.
<wgrant> I hope.
<wgrant> Mm, although I don't know what the OOPS report shows.
<wgrant> Since it shows more entries for BugTask:+index in "top timeouts" than there were timeouts.
<lifeless> rotfl
<thumper> lifeless: how can I tell if a repository is stacked or just pretending?
<thumper> lifeless: how can I open the repository to find the number of revisions?
<thumper> I have a branch that appears to be stacked locally using my new branch id alias
<spiv> thumper: I'm not sure what "just pretending" could mean
<thumper> but I want to confirm for myself that it is doing what it says
<thumper> spiv: I mean is saying it is stacked, but the branch repository has all the revisions
<spiv> Either a repository has at least one fallback repository, or it doesn'tâ¦
<thumper> can it do that?
<thumper> does it confirm that the fallback repository is valid?
<spiv> Probably you can have the state where the stacked repo has all the revisions (e.g. stacking on an empty repo)
<spiv> But I don't know why you'd care?
<thumper> humour me
<spiv> bzrlib always opens all the fallbacks
<spiv> Unless you explicitly do Branch.open(ignore_fallbacks=True)
<spiv> I'm guessing âcan be openedâ is what you mean by âis validâ?
<thumper> open(base, _unsupported=False, possible_transports=None)
<thumper> no ignore_fallbacks
<thumper> repository? or bzrdir?
<spiv> *Branch*
<mwhudson> thumper: you can look at len(branch.repository.all_revision_ids())
<spiv> The fallback location is defined in the branch.conf
<mwhudson> (iirc)
<spiv> If you open a repository directly, without going via a branch, it will have no fallbacks configured.
<lifeless> thumper: it opens the fallbacks at Repository.open() / Branch.open()
<lifeless> thumper: it will fail eagerly
<spiv> lifeless: not at Repository.open
<lifeless> thumper: this may change in future but hasn't yet
<lifeless> spiv: yes, true, I'm handwaving
<lifeless> because I hadn't read your prose yet :P
<spiv> thumper: (BzrDir.open_branch also accepts ignore_fallbacks=â¦)
<thumper> http://pastebin.ubuntu.com/589515/
<thumper> seems to be working
<thumper> info -v tells me what I care about
<thumper> and that is the 3 revisions bit
<spiv> thumper: but that said, it sounds like you just want to open the branch, and if it can be opened then the repository and its fallbacks must exist.
<spiv> There may be a good reason why you want to check whether the revisions exist in the branch's repo or the stacked-on repo, but it's not obvious to me why you should need to check that.
<spiv> thumper: I would expect info -v in that context would be counting revisions in the fallbacks as well as that direct location.
 * spiv â lunch
<thumper> spiv: no, it doesn't
<thumper> spiv: which is good
<thumper> interesting
<thumper> bzr info -v http://bazaar.launchpad.dev/~thumper/wikkid/soundex doesn't show the count of revisions in the repository
<jtv> lifeless: I'm experiencing delays in Q/A for db-devel r10383.  What are the chances of that one becoming a blocker for this rollout?
<wgrant> jtv: 0
<wgrant> jtv: The merge has already occurred.
<wgrant> r10381
<jtv> so no?  thanks, that's good to know
<wgrant> Right.
<wgrant> It is very scary and needs to be QAd at some point, but it's not 11.04-criticasl.
<jtv> Well the interesting part is that besides the big change, I'd also want to see that the minor changes to the original script are still safe.
<lifeless> jtv: FTR I mailed the -dev list on monday about this
<lifeless> jtv: I'd like to know if I was unclear etc so I can communicate better in future
<wgrant> jtv: I meant that both are very scary.
<wgrant> jtv: That script hasn't been changed significantly in a Long Time.
<wgrant> Even though these changes look pretty safe.
<jtv> lifeless: I see it nowâI suspect thunderbird has been hiding it from me.
<jtv> wgrant: it's extra-scary even because I know so little of what to expect from the scripts that it invokes.  Did you see the run-parts directories in cronscripts/publishing/distro-parts?
<lifeless> bbiab
<poolie> wow the new +authorize-token is quite slick
<poolie> well done leonard, or whoever
<wgrant> The system-wide one? Yeah.
<wgrant> (Pdb) from lp.bugs.model.bug import Bug
<wgrant> (Pdb) Bug.get(16)
<wgrant> <Bug at 0xfe2cb90>
<wgrant> (Pdb) print self.context.bugID
<wgrant> 16
<wgrant> (Pdb) print self.context.bug
<wgrant> None
<wgrant> What?
<wgrant> Hmm. Maybe I have a slave object, I guess, but...
<wgrant> ... and attempting to check self.context._store crashed pdb. Yay.
<lifeless> win
<lifeless> wgrant: this is a new bugtask ?
<wgrant> lifeless: Yes. I create it, test the query count, add attachments, test the query count. The query count test seems to be removing it from the store, which I guess shouldn't surprise me too much.
<lifeless> its invalidating the world
<lifeless> grab it again from the set
<lifeless> (at a guess)
<lifeless> psatebin ?
<wgrant> lifeless: Retrieved, still broken :/
<lifeless> wgrant: ok, thats new then
<lifeless> I've not seen *that* before
<lifeless> time to edit z3batching
<lifeless> if I'm not back in an hour, send in a search party
<wgrant> Heh
 * StevenK goes to tell the people in the next apartment where they can store the masonry drill they've been using since 9am.
<StevenK> It's in the room right next to me, so I have a splitting headache
<wgrant> lifeless: http://pastebin.ubuntu.com/589533/
<wgrant> makeBugAttachment works if you drop line 59 of that diff.
<thumper> :-(((
<thumper> WTF?
<thumper> running a test I get a failure that looks hard to explain
<thumper> so I run the test with -D
<thumper> it stops at: -> result.addSuccess(test)
<thumper> (Pdb) c
<thumper> so I continue
<thumper> Ran 1 tests with 0 failures and 0 errors in 11.213 seconds.
<thumper> stupid POS
<lifeless> wgrant: that makes no sense at all
<wgrant> lifeless: The test, or the problem?
<lifeless> wgrant: the problem
<lifeless> wgrant: there is no store.reset etc etc in that matcher
<lifeless> nor a store.invalidate
<wgrant> A store.invalidate won't remove the objects, either... it should just mark everything AutoReload, right?
<lifeless> store.invalidate is pretty shallow yes
<lifeless> store.reset is harsher, IIRC
<wgrant> Right, I think reset does some really bad stuff.
<wgrant> I guess I'll just create a fresh task for now :/
<lifeless> wgrant: I'd really like to know whats up
<lifeless> wgrant: have you tried to bisect?
<wgrant> lifeless: Bisect the matcher?
<wgrant> I probably should.
<lifeless> wgrant: yes
<thumper> can I get someone else to run TestBranchChangedErrorHandling ?
<thumper> I have it failing on devel
<thumper> unless I add -D
<thumper> in which case it kinda passes
<spiv> thumper: I'm pretty sure 'bzr info -v' of a branch does count revisions from the stacked-on repository.  What makes you think otherwise?
<thumper> spiv: it shows the full history count for the branch, but the repository says 3 revision
<thumper> spiv: since the output says that, that is why I think that way
<spiv> thumper: not in quick testing locally
<spiv> thumper: and not according to my reading of the code
<thumper> spiv: is for me
<thumper> spiv: http://pastebin.ubuntu.com/589515/
<spiv> It's probably affected by running over a smart server.
<spiv> Probably that's a bug.
<thumper> but very handy
<spiv> You're probably the first person to think so :)
<wgrant> WTF :/
<lifeless> wgrant: ENODETAIL
<wgrant> Set up a browser? Fine. Browse to a question? Fine. Browse to a bug? You can't use makeBugAttachment any more.
<wgrant> Somehow browsing to BugTask:+index manages to break the subscribers, I think.
<wgrant> lifeless: ^^
<wgrant> makeBugAttachment doesn't work at all, even on a new bug.
<wgrant> (this affects anything usin setupBrowserForUser, not just the matcher.
<lifeless> wgrant: I thought it wasn't the matcher ;)
<wgrant> So did I, but I didn't have evidence.
<StevenK> wgrant: Pity me, I'm debugging the packagecloner
<lifeless> StevenK: pity the fool ?
<wgrant> Oh, LaunchBag.
<wgrant> Fucking LaunchBag, please die.
<StevenK> wgrant: Then remove it?
<wgrant> I have some misgivings about the output of *canonical*_url depending on what you've traversed through lately.
<StevenK> wgrant: Perhaps it wants to make sure you take your shoes off so you don't dirty the carpets.
<wgrant> Hah
<lifeless> wgrant: rotfl
<StevenK> Sigh, I thought the build was linked from BPPH
<wgrant> StevenK: You have to go through BPR.
<wgrant> lifeless: No.
<StevenK> Ah, I see it.
<wgrant> Unless you are rolling around to squash LaunchBag flat, in which case it might be acceptable.
<lifeless> sigh at domain name spam
<lifeless> pop quiz
<lifeless> nvm
<jtv> lifeless: is there no way to resolve a qa-bad other than roll back the entire revision and then land a fixed re-run of the branch?
<lifeless> jtv: you can just land a fix
<lifeless> jtv: have it claim to rollback the bad revision
<jtv> lifeless: thanks.  I think https://dev.launchpad.net/QAForContinuousRollouts sort of skips over the exact meaning of --rollback; I thought it had bzr implications.
<lifeless> jtv: feel free to tweak it; its purely a workflow tag for us
<lifeless> if the commit says '[rollback=1234] fix 1234', then its pretty clearly not a rollback
<jtv> lifeless: so it just pairs the revisions to mark the fact that the newer revision fixes a qa-bad on the older one?
<lifeless> IWBN to have qatagger have a unbreak=1234 that does the same as rollback but doesn't confuse people
<lifeless> jtv: yes
<jtv> OKâ¦ I guess https://dev.launchpad.net/QAProcessContinuousRollouts needs updating as well.
<poolie> lifeless,  oh was your complaint about domain spam by any chance inspired by pda.lv?
<poolie> i just hit that
<lifeless> 'Internet copyright of "canonical" and Bdjikln company'
<lifeless> Dear President & CEO, .... - addressed to info@ and delivered to me
<jtv> So CEO is Russian for "occupant"?
<jtv> No reviewers today?
 * StevenK pushes wgrant forward.
<jtv> a volunteer!
 * jtv pins suspiciously plastic-y medal and branch on wgrant: https://code.launchpad.net/~jtv/launchpad/db-bug-751054/+merge/56298
 * wgrant is looking.
<wgrant> jtv: Done.
<wgrant> StevenK: Want to mentor?
<jtv> thanks
<jtv> only fair :)
<StevenK> wgrant: Done
<StevenK> jtv: Well, wgrant is on the schedule as OCR today, which is why I picked on him
<jtv> wgrant: I'm not sure what you mean about me saying that I reverted to the devel versionâ¦  what I said was "I really just copied the devel version of the script into a db-devel branch."  What's missing?
<jtv> It's not actually exactly reverting, since the devel version had changed in the meantime.
<wgrant> Mmm
<adeuring> good morning
<wgrant> lifeless: Thanks.
<wgrant> Bah, real test failure.
<wgrant> Odd...
<wgrant> I can't reproduce it, but it happens on ec2.
<StevenK> wgrant?
<wgrant> Anyway, I should go.
* gmb changed the topic of #launchpad-dev to: Performance Tuesday! | https://dev.launchpad.net/ | On call reviewer: gmb | https://code.launchpad.net/launchpad-project/+activereviews
<jtv> gmb: got a branch for you to review if it's convenient!  https://code.launchpad.net/~jtv/launchpad/db-bug-244328/+merge/56310
<gmb> jtv: Sure.
<jtv> Splendid, thanks
 * jtv runs out for some simple, urgently-needed nutrients
<jml> are we in RC mode?
<StevenK> Just testfix, I think.
<jml> ffs.
<gmb> jtv: r=me
<jml> not that it matters!
<jtv> gmb: thanks
<jtv> StevenK, wgrant: do you chaps know anything about the pending merge on df current?
<wgrant> jtv: You can drop it.
<jtv> \o/ thanks
<wgrant> That's weeks old, though, and not actually applied any more...
<wgrant> jml: Um, that makes you feel *better*?
<wgrant> jml: I can see why, but it also scares me more.
<wgrant> Because something has changed.
<wgrant> Although what exactly I cannot tell, since the assertion message suggests that the text matches the regexp fine...
<jml> wgrant: it's intermittent, and it's probably because of code.
<jml> rather than environment
<bigjools> wgrant: grar, every time I see a test change for the publisher I want to shout loudly when the file is left in lp/soyuz
<jml> and it's only that test rather than other tests in the file
<wgrant> bigjools: That orig.tar.gz? Yeah.
<bigjools> wgrant: no, lib/lp/soyuz/tests/test_publish_archive_indexes.py and friends
<bigjools> should be lp/archivepublisher
<wgrant> Oh.
<wgrant> Yeah.
<wgrant> I guess I should have moved it.
<wgrant> but the code itself is in Soyuz.
<wgrant> So maybe not.
<jml> wgrant: I mean, I feel terrible that we tolerate having such things in our test suite, but believe me that's a small fraction of the terrible I feel, and I'm trying to take my wins where I can.
<bigjools> along with most of the code in the soyuz model ....
<wgrant> bigjools: Shall I move it before I land the branch, or keep it with the code it tests?
<bigjools> wgrant: rs=me if you want to move it
<wgrant> I think we should keep it in Soyuz until we split the classes, though :/
<bigjools> maybe
<jtv> bigjools: can I just run publish-ftpmaster on df or will that open a hole that the Old Ones may use to break through into our world?
<jtv> (LPCONFIG=dogfood)
<bigjools> jtv: the cron.publish-ftpmaster  script can run unaltered
<wgrant> jtv: Please back up dists/ first, though.
<jtv> (And of course I'm just naming one of the more likely failure modes as an example; please let me know of other risks as well)
<bigjools> nothing will cross from the Other Side
<jtv> Phew.
<bigjools> wgrant: dists/ is probably already fucked
<wgrant> bigjools: Some of it is, some of it is not.
<wgrant> It's reasonably easy to judge, and useful for comparisons when trying to unfuck the publisher :)
<jtv> bigjools: that's /srv/launchpad.net/ubuntu-archive/ubuntu/dists/ ?
<bigjools> jtv: yes
<jtv> right ho
<bigjools> wgrant: unfuck the publisher?  You have a spare year? :)
<jml> deryck: have you seen bug 751187?
<_mup_> Bug #751187: Upstream translations not being imported for some Ubuntu packages <Launchpad itself:New> < https://launchpad.net/bugs/751187 >
<jml> deryck: not sure if it's relevant to the work you guys are doing
<deryck> jml, I haven't seen.  I'll look now.
<jml> deryck: thanks.
<jtv> deryck: there's the possibility that it's a simple outdated timestamp.
<jtv> It may pay to have a look at the import queue entries.
<wallyworld_> deryck: hi. my laptop is acting up. no sound, flaky network :-( did you get my recent email? i just want to make sure you don't waste time on the windmill stuff since i've got it sorted
<deryck> wallyworld_, no, but my mail setup is flaky.  mail not getting to me after upgrade of server.
<deryck> I'm looking into my mail problem now.
<wallyworld_> deryck: cool. i got rejection messages. my sound is totally borked for "no" reason. very frustrating.
<deryck> my busted mail is very frustrating.
<jtv> wgrant: I don't suppose you should be here at this time, but if you are: I got publish-ftpmaster to run in 27 minutes or so, but bigjools tells me that's a good thing.  Perhaps you could take a look?
<wgrant> jtv: That is plausible if no release pocket was dirty.
<wgrant> Let's see.
<wallyworld_> deryck: did you upgrade the kernel recently? that's what did my sound in i think. or did you upgrade to natty?
<jtv> wgrant: the output is in /srv/launchpad.net/codelines/current/jtv7.log
<wgrant> I was about to ask.
<wgrant> Thanks.
<wgrant> 2011-04-05 11:10:57 DEBUG   Publishing The Maverick Meerkat-RELEASE
<wgrant> 2011-04-05 11:10:57 DEBUG   Attempting to publish pending sources.
<wgrant> 2011-04-05 11:11:00 DEBUG   Added /srv/launchpad.net/ubuntu-archive/ubuntu/pool/main/h/hello/hello_2.6-1lp5.dsc from library
<wgrant> It really finished in only 27 minutes?
<wgrant> Dirty maverick-release should have put you through at least 3-4 hours of pain...
<deryck> wallyworld_, yeah, I have a server I share with a friend and he upgraded over night.
<jtv> wgrant: I ripped out all the explicit gc and the cache invalidations.
<wgrant> jtv: Ah, you can close one of my bugs, then.
<jtv> No longer necessary with the generational cache.
<wgrant> jtv: But that wasn't the bit that made it take hours.
<jtv> What was?
<wgrant> (I had a branch that did the same thing, but it didn't definitely improve performance)
<wgrant> Queries when generating override and file lists.
<wgrant> Huh.
<wgrant> Only took 7 minutes to generate binary file lists.
<wgrant> WTF?
<wgrant> This is reason to be extremely suspicious about its results.
<jtv> Mind you, the various calls to publish-distro etc. are all in one process now, so we may be getting better cache re-use.
<jtv> (When combined with not continually invalidating or flushing the cache, of course)
<jtv> But by all means, please be suspicious about the results!  That's what I really need here.
<wgrant> ... oh.
<wgrant> You deleted publishingtunableloop.
<wgrant> *That* is the important bit.
<jtv> Yes
<jtv> Because it did explicit gc and cache invalidation.
<wgrant> As long as we're not running into RAM usage issues (due to materialising a huge resultset in one hit), that is fine and should be a massive benefit.
<jtv> And essentially nothing else.
<wgrant> The issue was that it did binary file lists in 16ish batches of 10000 each, each of which took 8-9 minutes for the DB to calculate due to the compound sort key.
<jtv> Yeahâ¦ I figured (and documented in the MP) that if there are still problems of this nature, we're probably better off taking a fresh, post-StupidCache, post-bash look at them.
<wgrant> Actually, I think I did test this once before.
<wgrant> By hacking tunableloop to always use a batch size of 200000.
<jtv> It pains me to admit it, but the loop tuner was never the right tool for this particular job.
<wgrant> But it wasn't quite so wildly successful, possibly because of the remaining cache crap.
<jtv> Bear in mind by the way that we no longer have the infinite-caching problem.
<wgrant> RIght.
<jtv> So if we still have memory problems in this script, it's going to show up as excessive database waits rather than as swap or OOM.
<jtv> But I'd be very much interested in your appraisal of the data that this run produced.
<wgrant> launchpad@mawson:/srv/launchpad.net/ubuntu-archive/ubuntu$ ls -l dists/maverick/main/source/Sources
<wgrant> -rw-r--r-- 1 launchpad launchpad 3763543 Apr  5 11:31 dists/maverick/main/source/Sources
<wgrant> That shouldn't exist any more, should it?
<jtv> wgrant: that's done by one of the run-parts scripts, which I'm not running in this case.
<jtv> Basically Ubuntu-specific.
<wgrant> jtv: Ah! Well, apart from that it looks pretty reasonable. I will throw more at it tomorrow, when I am meant to be working.
<wgrant> Well done on destroying PublishingTunableLoop.
<wgrant> Probably won't help productive by more than a couple of minutes, but helps DF massively...
<jtv> (But actually glad you're spotting these things, because it makes it much more likely that you'd have spotted any noticeable oversights)
<jtv> Thanks very much for looking into this.
<wgrant> s/productive/production/, grar.
<jtv> Next time let's run domination in parallel.  :-)
<wgrant> Hmm?
<wgrant> You can run judgeSuperseded in parallel with index generation if you really want to, but that's about it...
<jtv> Just guessing: maybe domination is embarrassingly parallelizable across architectures.
<wgrant> Indeed.
<wgrant> But it should also be blindingly fast now.
<jtv> And it takes a fair bit of the time, actually.
<wgrant> Maybe on mawson.
<wgrant> On prod each pocket should be no more than 1.5s or so.
<jtv> On this run, I see about 4 minutes going into it.  That's a sizable percentage of overall runtime.
<jtv> Not that parallel runs would necessarily help mawson.  :)
<StevenK> I think parallel runs on mawson might make the chassis itself glow red hot.
<wgrant> jtv: You can see it speed up significantly as the caches get hotter.
<jtv> Kewl!
<bigjools> StevenK: rofl
<wgrant> amd64 takes 59s, armel 15s, i386 13s, and it continues.
<jtv> yeah nice
<wgrant> Source still takes 90s, but it's more like 2s on prod (was 150s until I fixed it six months ago)
<wgrant> wildcherry has fewer issues with caches, given that it doesn't have 4GiB and all of the LP services.
<henninge> rvba: Just to let you know, I am working on your review.
<jtv> wgrant: by the way you'll notice complaints about stuff not being signed.  You probably saw this coming, but that's because gpg signing also is in a run-parts script now.
<rvba> henninge: great.
<henninge> rvba: Here is as far as I have come and there are a few things that need attention.
<wgrant> jtv: Yeah, I'll walk through everything, get more stuff publishable, populate partner, add fake keys, enable the rest of run-parts, etc. tomorrow and see what breaks.
<henninge> rvba: if you want you can start on those. http://paste.ubuntu.com/589661/
<wgrant> Then I may change my qa-hellno to qa-ok.
<jtv> wgrant: I'm so glad I don't know how to do all of that.  I backed up before my run using cp -r /srv/launchpad.net/ubuntu-archive/ubuntu/dists /srv/launchpad.net/dists.bak
<StevenK> wgrant: You mean it isn't qa-overmydeadbody ?
<bigjools> jtv, wgrant: I'm not entirely convinced about those complaints being to do with lack of signing
<jtv> wgrant: do you want me to put that old dists back in place?
<bigjools> since it's never been an issue on DF before
<jtv> Invalid archive signature?  It could be reading the wrong file or something I guess
<jtv> bigjools: you're rightâ¦ files seem to be missing
<bigjools> jtv: which equates to a further path issue
<jtv> Yes, and it's not the partner path
<bigjools> well, in fact the original path issue :)
<jtv> Hmm maybe notâ¦  In the example I'm looking at, the diff, dsc, and orig tarball are all there but the deb isn't.
<jtv> Where should that deb have come from?  (If it's the librarian, then I can think of a reason)
<wgrant> bigjools: I didn't see those errors. I'm trying not to look at it in depth yet.
<wgrant> bigjools: But if it's "Invalid archive signature" on a deb, it could be from the failed librarian transfers when I ran the careful publisher.
<wgrant> I forget the full list, but fennec was one of them.
<wgrant> On most archs its debs failed to transfer properly from production.
<wgrant> Is that one of the ones with errors?
<jtv> wgrant: fennect looks to be behind a sizable portion of the errors, yes
<wgrant> As for partner, there are no fresh uploads and probably nothing existing on disk to index. I will sync and upload stuff tomorrow to test it out.
<wgrant> jtv: Aha, great.
<wgrant> So it's probably nothing to worry about, but I'll check it out.
<jtv> About half the pages of errors are completely full of fennec-related lines
<jtv> Then there's python-htmltmpl
<jtv> and a bit of microcode.ctl
<wgrant> I don't recall those, but possibly because they sound arch-indep.
<wgrant> So there would be around 1/5 of the errors.
<jtv> I think python-htmltmpl is again repeated lots of times (per arch I guess) and that seems to be most of the rest.
<wgrant> I may manually regrab the files tomorrow and see what happens.
<wgrant> As well as -C'ing partner.
<wgrant> Hopefully that won't take too long.
<jtv> One thing that bothers me a lot is the carrying-over of old directories.  For all I know we might not notice if, for instance, we stop generating a file and keep copying an old copy instead.
<wgrant> jtv: Yeah, but we have no choice :/ dists isn't all generated by the publisher.
<wgrant> Custom uploads end up in there too.
<wgrant> Custom uploads are bad in a few ways.
<wgrant> And good in not many :(
<jtv> Well, here's hoping that we're making some real progress in manageability and transparency of the process.
<jtv> Oh, we have another buildbot failure in progress.  :(
<wgrant> What has combusted now?
<jtv> Dunno summary won't load
<wgrant> That's why I asked :/
<wgrant> It's just hanging for you too?
<jml> I don't see any failures on the waterfall
<jtv> Oh drat I'm mis-reading it.
<wgrant> I see nothing in stdio.
<wgrant> jtv: Ah, good.
<jtv> Sorry for the noise.
<LPCIBot> Project windmill build #139: STILL FAILING in 1 hr 11 min: https://lpci.wedontsleep.org/job/windmill/139/
<wgrant> Not helping, Jenkins.
<jtv> wgrant: my fix for the partner archive path is in db-devel now, so if you update df's codebase tomorrow it should be in there.
<wgrant> jtv: Great, thanks.
<deryck> jml, can you send me a ping at my work email, see if I got mail sorted out now?
<jml> deryck: sure.
<wgrant> I may merge db-stable into devel again tomorrow, since there are no DB changes.
<jtv> I'll clean up my pending merge first.
<wgrant> jtv: Which branch is that?
<wgrant> jtv: It's fairly critical to sensible running the script.
<jtv> The gc/cache cleanup
<wgrant> i mean where is it on LP, so I can merge it again tomorrow.
<jtv> Alternatively, I could just commit it.
<deryck> hurray, mail!  Thanks, jml!
<jtv> wgrant: Oh, I was about to give you the pointer.  :)
<jtv> wgrant: lp:~jtv/launchpad/db-bug-244328
<jtv> Feel free to attach your own kill-kill-kill bug.
<rvba> henninge: just pushed the corrections from the first batch of comments you made (Thanks already for the careful review :)).
<bigjools> wgrant: hopefully jtv answered all your questions, I was at lunch
<jtv> bigjools: wellâ¦ "is there a God" was a bit of a sticky point
<henninge> rvba: I'll have  a look. I just sent off the complete review.
<rvba> henninge: great, I'll look into it.
<wgrant> bigjools: I think so.
<bigjools> jtv: contentious but ultimately easy to answer :)
<jtv> bigjools: we've been over this.  You don't count.
<LPCIBot> Project windmill build #140: STILL FAILING in 46 min: https://lpci.wedontsleep.org/job/windmill/140/
<rvba> henninge: a small question about your comment of my use of @@. I though this was the only way to access an attribute on the default _view_ attached to this object. But since it's not used in the code base, I'd like to know how else this can be done ...
<henninge> rvba: oh, this is an attribute of the view?
<rvba> henninge: yes
<henninge> ah yes ;)
<henninge> rvba: view/attribute
<rvba> henninge: but the view available in the page is attached to another object
<rvba> henninge: let me explain:
<henninge> hm
<rvba> I need to access an attribute of the view attached to an object inside a loop
<rvba> the current view is attached to the context
<rvba> I need to access another view
<henninge> rvba: I understand
<henninge> rvba: now the notation makes sense, too.
<henninge> rvba: now I wonder, too, why it is not used anywhere else
 * henninge greps again
<rvba> indeed ...
<rvba> if you create a small template fragment that is used inside the loop then it's not necessary to use this @@ because you can make the association (view/object) for this fragment only
<rvba> but if all you need is to get an attribute from a view, adding a 3 lines template fragment looks overkill.
<henninge> rvba: I think the usual approach would be to push this into the model
<henninge> rvba: In this case, DistroSeriesDifference should have parent_source_package and source_package attributes.
<henninge> rvba: and you'd use "difference/parent_source_package/fmt:url" in the template.
<rvba> henninge: I confess I don't really like putting things like this into the model
<rvba> henninge: but if it's the way it's done in the rest of the code I'll do it
<henninge> rvba: that's my interpretation, yes
<rvba> henninge: what does this /fmt:url do exactly?
<henninge> rvba: it gets the canonical_url for the object,  the same thing the function does.
<rvba> henninge: ok, I get it.
<henninge> so you wouldn't need that for tal:condition, but for tal:attributes
<rvba> ok
<henninge> rvba: to me it sounds better to not have the different view depend on each other too much.
<henninge> s/view/views/
<rvba> henninge: good point
<henninge> rvba: I'd say it makes the code more fragile.
<henninge> we have a pretty close relationship between view and template in Launchpad.
<henninge> rvba: Ha!
<henninge> pub = IResultSet(pubs).one()
<henninge> Database code does not belong in view code, that much I know. ;-)
<henninge> rvba: so this is a strong indicator that these should be pushed into the model.
<rvba> henninge: that's right. You win :)
<henninge> :-D
<henninge> Launchpad wins!
<rvba> henninge: all right ... I'll go though the rest of your remarks and get back to you when it's done
<bigjools> henninge: you should explain why "Database code does not belong in view code" so that rvba learns something :)
<henninge> bigjools: Hey, I am only going through the motions here ... :-P
<rvba> I think it's pretty standard isn't it? Views are for fetching data from models and getting it ready for the templates.
<bigjools> the main reason is security
<henninge> rvba: it's also because you should only handle security proxied objects in view code.
<bigjools> there we go :)
<henninge> rvba: yeah, it's actually the main reason
<henninge> ;-)
<rvba> ok, makes sense
<henninge> rvba: plain database queries will give you naked objects.
<henninge> can't have that in public ...
<rvba> right
<rvba> henninge: thanks again, I'm learning a lot thanks to this review process I reckon :)
<bigjools> rvba: generally if view code needs to get a database object, it does a query via the context, or uses a secured utility
 * jml mumbles something about sorting & views.
<bigjools> allenap: "field.summary": u"Even The Register likes it." - just made me lose a mouthful of coffee
<allenap> bigjools: Hehe :)
<bigjools> allenap: you have a typo in lib/lp/registry/help/distribution-add-series.html
<allenap> bigjools: Dang.
<bigjools> "its version is used instread"
<allenap> bigjools: Nothing wrong with that ;)
<allenap> That's how they say it round here.
<bigjools> if you're dyslexic
<allenap> bigjools: I'll fix it now.
<bigjools> allenap: I could make more comments about round ther e:)
<bigjools> but they would apply to round here too
<jcsackett> anyone happen to know if logging something as error in our cronscripts automatically creates an OOPs?
<sinzui> jcsackett: it is not automatic
<sinzui> jcsackett: an text can be logged as an error. We want an exception.
<jcsackett> sinzui: right, i was double checking. b/c once i was able to get tests for the hardwardb stuff running, the call site I saw that corresponded to the OOPs error msg didn't cause an exception. :-(
<sinzui> jcsackett: look for globalErrorUtility or a subclass. it will call handling()
<jcsackett> sinzui: ah, okay.
<abentley> deryck: I've got the form showing: http://people.canonical.com/~abentley/translations-usage.png Just want to confirm this is really what you want before I go implement saving.
 * deryck looks
<deryck> abentley, yeah, looks good to me.
<abentley> deryck: alrighty then.
<bac> hey deryck, is this type of markup (found in several bugs pt files) still valid?  i don't see that file being built anymore.
<bac> http://paste.ubuntu.com/589741/
<deryck> bac, we shouldn't be doing that anymore.
<bac> deryck: thanks for the confirmation
<deryck> bac, np
<bac> deryck: would you mind looking at the top of soyuz/templates/archive-copy-packages.pt?  i assume it is wrong too but don't know why it was ever there.
<deryck> ok, looking....
<deryck> bac, yeah, it's wrong too
<deryck> not needed.
<deryck> anything in lib/lp/APPNAME/javascript gets pulled in automatically.
<bac> so anything loading specific js files from icing/build is wrong
<deryck> bac, yup.
<deryck> bac, it's loading files twice in devmode but no harm to production.
<bac> deryck: well those files don't exist in devmode, so it is oopsing trying to load them
<deryck> bac, ah, right.
<bac> which confuses people, e.g. me
<deryck> gotcha
<bac> gmb: can you have a look at https://code.launchpad.net/~bac/launchpad/remove-old-devmode-js/+merge/56400
 * bac lunches
<gmb> bac: Sure
<LPCIBot> Project windmill build #141: STILL FAILING in 1 hr 15 min: https://lpci.wedontsleep.org/job/windmill/141/
<gmb> bac: r=me
<henninge> rvba: still around?
<rvba> henninge: yes
<henninge> why did you have to use structured().escapedtext?
<henninge> I mean, I thought just structured() would have been enough?
<rvba> I did that as first but then I get the __repr__ version of the object in my page ... and it's not pretty
<rvba> s/as first/at first/
<mrevell> Night!#
<jml> flacoste: you have disappeared from our secret squirrel IRC server!
<flacoste> jml: really!?!
<jml> flacoste: yep.  flacoste has quit (Connection reset by peer)
<bigjools> night all
<henninge> rvba: really? that's interesting.
<henninge> rvba: you mean this gives you a _repr_ of the structured object?
<henninge> 871	+      <p><tal:replace replace="structure view/explanation" /></p>
<rvba> henninge: yes
<henninge> I wonder if wgrant has already been changing things ...
<rvba> I saw quite a few .escapedtext in the code ...
 * henninge checks places he knows
<henninge> hm, those that I had don't go straight to TAL.
<henninge> rvba: see, now I learned something, too ... ;-)
<rvba> henninge: :)
<rvba> henninge: the proper way might be to do this:
<rvba> <tal:replace replace="structure view/explanation/escapedtext" />
<rvba> not much difference though
<rvba> but the code is full of these
<rvba> henninge: I'll have to get going is a few
<rvba> s/is/in/
<henninge> rvba: I am not done yet, I am sorry. We will have to do another short round tomorrow morning.
<henninge> I'll finish my review of your changes tonight, so you can have them in the morning.
<rvba> henninge: sounds perfect
<rvba> henninge: see you tomorrow
<henninge> rvba: see you! ;)
<bac> hi deryck, sinzui: we've got a few places in our JS that uses Y.fail.  they seem to be untested and calls to it actually fail.  do either of you have any experience with it?
<deryck> bac: it's a test module method, no?  We shouldn't be using that in regular code.
<bac> deryck: that is my impression too.  but i see it in many places
<bac> deryck: should we just be throwing an error?
<deryck> bac: yeah, I think we either want to use Y.error directly or the showError mechanism we have.
<bac> deryck: it is in five modules, including lp.js and client.js
<deryck> hmmmm, ok
<abentley> deryck: how would I get the URL for a translation_group from its name in Javascript?
<abentley> deryck: I figure translation_groups.getByName, but I don't know how to get translation_groups.
<bac> deryck: and i just discovered it is part of 'test', so you have to include 'test' in production code, which is vile.
<deryck> bac: yeah, agreed.  so the client module looks like it's trying to log the fail. I think Y.error or even a simple Y.log('some msg', 'error') is enough there.
<bac> and it points to none of those paths ever being exercised...
<deryck> bac: and I think that ^^ could be the basic pattern.  I'm certain we don't want Y.fail in js code.
<deryck> right
<bac> ok
<deryck> abentley: are they exported in the api?  Can you get them from an api get?
<abentley> deryck: yes, they're exported in the API.  I don't know how to get them from an API get.
<deryck> I'm not sure either.  I'm thinking and looking at code.
<abentley> deryck: it seems like there should be a generic answer to "how do I get the URI for a collection", and then the rest would fall into place.
<deryck> yeah
<deryck> abentley: yeah, so it's the same as an entry in the client, right.  Y.lp.client.Collection.  but like you say you need the URI.  I really don't know how you get that.  trying to find an example somewhere.
<deryck> abentley: where are you getting the name from?
<abentley> deryck: I'm getting it from the form data.
<abentley> deryck: from the form I posted a screenshot of earlier.
<deryck> right
<abentley> deryck: Unless I'm doing it wrong?  (I'd love to be doing it wrong!)
<deryck> abentley: are you doing an XHR post of the form on submit?  Or trying to handle each piece with the API?
<abentley> deryck: the latter.
<deryck> abentley: can you not make it easier by doing a single form post with the data?
<abentley> deryck: It's not something I'd thought of 'till now.
<abentley> deryck: how would that work?
<deryck> abentley: do an XHR post to the page that normally handles this step.  on success flash green and go to next part of page.  report if it errors.
<deryck> abentley: we may not do this anywhere else.  I can't find an example.  But you would just use Y.io directly.
<abentley> deryck: could you give me more detail or an example of the first piece?
<abentley> deryck: That is, making the XHR post.
<deryck> abentley: typing up something....
<deryck> abentley: something like this: http://pastebin.ubuntu.com/589828/
<rvba> henninge: still around?
<henninge> rvba: yes and no ;)
<rvba> henninge: I see ... just a quick question then if you don't mind
<henninge> rvba: go ahead
<rvba> henninge: in your review you advise me to move back getParentPackageSetsNames into the views ... but my understanding of our previsou co
<rvba> sorry ...
<rvba> previous conversation was that you wanted these methods in the model to avoid the clumsy @@ inside the template
<henninge> rvba: sorry, there must have been some misunderstanding, then.
<rvba> henninge: I understood that the main point was security.
<rvba> henninge: but still, I'll have to use @@ to be able to call a method from a view (but not the "main" view) associated with an object in the template.
<henninge> rvba: yes, that's why (parent_)source_package_release belong in the model.
<abentley> deryck: thanks.  I think I can hack that.
<henninge> rvba: no, you don't ;)
<deryck> abentley: cool.  hope it helps.
<rvba> henninge: then I think I missed something
<henninge> rvba: which view is the one that the template belongs to?
<henninge> rvba: ah here, DistroSeriesMissingPackages
<rvba> henninge: yes
<henninge> that's the view that needs the formatting code.
<rvba> henninge: but I'll need to call a view that is acting on the difference, not the distroseries
<rvba> that's the part I don't understand properly I guess ... how to call a method, not on the main view of the template ... but on the view associated with the many differences displayed on the page
<henninge> rvba: aha, difference is from a tal:repeat
<rvba> henninge: right
<rvba> henninge: there is my problem :-)
<rvba> henninge: hence the @@
<henninge> rvba: yes, you have too options to avoid the @@
<henninge> rvba: option 1: construct the list in TAL
<rvba> henninge: yes, I can use ','.join directly inside the TAL
 * rvba greps
<henninge> nah, avoid python in TAL at all costs.
<rvba> right
<henninge> rvba: you could do a tal repeat over difference.source_package_release
<henninge> and use tal:condition on the "last" variable (I forget the details of tal:repat here) to add a comma or not.
<henninge> rvba: option 2: construct it in the view
<rvba> henninge: got it ... quite ugly though.
<henninge> yes, I agree
<rvba> (I'm talking about the repeat)
<henninge> I know
<henninge> rvba: but then you will not be able to loop over differences/batch but will have to create your own list.
<henninge> which works ok
<rvba> still looks a little bit of a hack to me though :-)
<rvba> but I'm glad to use any method that works
<henninge> rvba: currently differences is the content of view/cached_differences.
<rvba> ye
<rvba> s
<henninge> rvba: you can instead create a propety that iterates over cached_differences internally and yields dicts of information
<rvba> I think I get the idea: create a simple list wrapper with the differences and the formatted packagesets
<rvba> right
<henninge> yup
 * henninge goes back to do other stuff
<rvba> henninge: thanks for your time, it's late for the two of us ... I'll sleep over this and see if the morning brings a new idea.
<henninge> rvba: great idea
<henninge> ;-)
 * henninge cannot sleep yet, though
 * rvba meither
<rvba> s/meither/neither/
<lifeless> morning
<lifeless> deryck: hi
<deryck> hi lifeless
<lifeless> deryck: bug heat decay
<deryck> ah yes
<deryck> we talked about that at the thunderdome, no?
<lifeless> deryck: stub has identified *something* causing rapid index bloat in bug/bugtask
<lifeless> deryck: we talked about the rendering stuff - threshold and the curve fitting function
<deryck> ok, gotcha.  new issues now.  the joys of heat! :-)
<lifeless> deryck: stub is still gathering data, but it may be the bug heat decay (because every row in bug (and now bugtask)) is being rewritten at high frequency
<lifeless> I'm being opportunistic, since you're still here ;) and asking about how important the decay aspect is
<lifeless> like, if we removed the 'time since last alteration' component from the heat function, do we think folk would care?
<deryck> lifeless: it's very important actually.  the decay helps bugs not sit in the top 10 hot bugs too long.  bugs can gain heat quickly and they need to decay quickly or it gets odd looking on that bugs home page.
<deryck> lifeless: the decay happens via the offline job or cron, though.  So it could be run less frequently.
<lifeless> deryck: did we try it without the decay at any point ?
<lifeless> deryck: I ask because hot bugs are often lightning rods anyway
<lifeless> deryck: which means they get meaningless comments that have the effect of nullifying the decay
<deryck> lifeless: we tried various forms of heat.  we never tried without the decay alone.
<deryck> lifeless: decay helps mitigate the lightening rod.  comments don't add heat, either.
<deryck> lifeless: so I think the decay bit runs in garbo.  perhaps we could go daily instead of hourly?  I don't recall how frequently it runs actually.
<lifeless> deryck: the frequency the task runs isn't the issue
<lifeless> deryck: its the rewriting of the entire table over the course of a week
<lifeless> deryck: *if* this is the problem, its baked into the use of a decay function at all.
<lifeless> deryck: it may not be
<deryck> lifeless: ah, ok.
<lifeless> deryck: like I say, I'm gathering data.
<deryck> lifeless: and, to be clear, there isn't a decay funtion per se.  There is one calculation function. part of that function handles decay
<lifeless> deryck: yep, I'm on that
<deryck> ok, cool
<lifeless> update_bug_date_last_updated updates bug_last_updated, which is the key thing for the decay, on every IObjectModified event
<lifeless> which includes
<lifeless>         for="lp.bugs.interfaces.bugmessage.IBugMessage                 lazr.lifecycle.interfaces.IObjectCreatedEvent"
<lifeless>         handler="lp.bugs.subscribers.buglastupdated.update_bug_date_last_updated"/>
<lifeless> deryck: so commenting on a bug freshens its heat
<lifeless> deryck: that might not have been the intent, but it is the implementation :)
<deryck> lifeless: I don't think that's entirely correct.  update_bug_date_last_updated has nothing to do with heat, IIRC.
<lifeless> in trusted.sql
<lifeless>     days_since_last_update = (datetime.utcnow() - date_last_updated).days
<deryck> lifeless: sorry to seem dumb, I'm in the middle of natty upgrade and get really look
<lifeless>     total_heat = int(total_heat * (0.99 ** days_since_last_update))
<deryck> lifeless: right.  that heat function *uses* date_last_updating but the event that causes date_last_updated to be updated doesn't cause the heat function to run.
<lifeless> deryck: right
<lifeless> deryck: but the reason we have to recalculate the heat on every bug
<lifeless> deryck: is because date_last_updated is part of the function
<lifeless> [and date_created]
<lifeless> deryck: I was analysing the 'commenting on a bug does not affect its heat' - but it does
<lifeless> deryck: because it resets date_last_updated
<lifeless> deryck: anyhow, will leave you to your upgrade :) stub is gathering concrete data to pin down the cause of bloat
<deryck> lifeless: well indirectly.  commenting affects date_last_updated.  And that factors into heat.  But commenting does not trigger a recalculation of heat.  Changing those dates does not trigger a recalculation of heat.
<deryck> lifeless: ok, fair enough :-)  Good luck with it.
<lifeless> deryck: thanks
 * sinzui just discovered multitouch is now enabled on his macbook
<jcsackett> sinzui: really? how well is it working?
<sinzui> three fingers: resize/move via love handles. four fingers: opens the dash
<sinzui> contacts synced for the first time in 4 months this morning too. The test will be my next update though. My primary computer has only synced twice in 18 months
<lifeless> deryck: I see the confusion I was causing; I was saying that commenting on a bug stops the 'if days_since_last_update > 0:' condition firing
<deryck> ah ha
<lifeless> deryck: any bug commented on less than 1 day before the background task to update heat, will have an unscaled heat
<deryck> lifeless: I see what you mean now
<lifeless> deryck: and also that bugs which are in the top 10 will likely get a continual stream of comments - they are lightning rods
<deryck> right
<lifeless> deryck: all they need is one comment every week or so to stop them dropping out
<deryck> right
<jcsackett> sinzui: nifty.
<lifeless> deryck: which is why I'm wondering if there is any /practical/ impact of dropping the date based scaling
<sinzui> oh, hell
 * sinzui gets children
<lifeless> deryck: IFF stub finds that it is the heat routine causing issues
<deryck> lifeless: right.  given that, maybe there's no harm.
<deryck> lifeless: we could try and see.
<lifeless> yeah
<lifeless> there is no rush, not till we know the bloat cause
<lifeless> thanks for the time!
<deryck> np!  Thanks for bringing it up.
<deryck> Later on, everyone.
<lifeless> wgrant: would you like the honours ?
<LPCIBot> Project windmill build #142: STILL FAILING in 1 hr 10 min: https://lpci.wedontsleep.org/job/windmill/142/
 * mwhudson_ raised an eyebrow at https://launchpad.net/c++
<lifeless> -lol-
<lifeless> terrible project name
<mwhudson> yeah
<mwhudson> jml: hooray for the upgrade to twisted 11 being so easy!
<lifeless> 933, in _get_nodes
<lifeless>     found[idx] = cache[idx]
<lifeless> RuntimeError: maximum recursion depth exceeded
<sinzui> jcsackett:
<sinzui> ping
 * sinzui is on stupid medication today
<jcsackett> sinzui: pong.
<jcsackett> sorry, didn't hear the notification.
<sinzui> jcsackett: mumble?
 * wallyworld sad. coffee machine broken :-(
<lifeless> gmb: ping
<lifeless> gmb: lp:~gmb/launchpad/bug-1-timeout - this will cause timeouts on bug message collections via the API I think
<poolie> hi lifeless
<lifeless> gmb: s/timeouts/bad-data/ - it needs rolling back, I've explained in the bug
<lifeless> hi poolie
<wgrant> lifeless: Which honours?
<lifeless> wgrant: deleting shipit
<wgrant> Woot.
<wgrant> I wasn't sufficiently up to date on email.
<wgrant> But yay.
<wgrant> Is production sufficiently gone?
<lifeless> see chat with chex in -ops an hour back
<wgrant> So that looks like a yes.
<wgrant> And if not, then the unused shipit appservers will break, oh no.
<wgrant> This is very good news.
<wgrant> We can finish purging Account.
<wgrant> And then delete AccountPassword and eventually Account itself!
<wgrant> And we will be sensible again!
<lifeless> VPC 4 eva
<wgrant> Yes.
<wgrant> lifeless: Have you rolled back gmb's thing?
<lifeless> not yet
<lifeless> I have to pop out for a bit
<lifeless> if you want to do it yourself, that would be cool
<lifeless> I can't see any good reason for him to be collecting /all/ bug messages for bug 1 in his code
<_mup_> Bug #1: Microsoft has a majority market share <iso-testing> <ubuntu> <Clubdistro:Confirmed> <Computer Science Ubuntu:Invalid by compscibuntu-bugs> <EasyPeasy Overview:Invalid by ramvi> <GNOME Screensaver:Won't Fix> <Ichthux:Invalid by raphink> <JAK LINUX:Invalid> <The Linux OS Project:In Progress> <metacity:In Progress> <OpenOffice:In Progress by lh-maviya> <Tabuntu:Invalid by tinarussell> <Tivion:Invalid by shakaran> <Tv-Player:New> <Ubuntu:I
<lifeless> so I suspect the change is actually entirely different
<LPCIBot> Project windmill build #143: STILL FAILING in 1 hr 10 min: https://lpci.wedontsleep.org/job/windmill/143/
<lifeless> change needed is entirely different, I mean
<wgrant> OK, I'll check it and roll it back.
<lifeless> thanks
<lifeless> have we seen an OOPS for this in the oops reports?
<lifeless> -> bbiab
<jml> mwhudson: hah!
<mwhudson> jml: ?
<jml> mwhudson: it took way longer than you might think, because of two separate spurious test failures during ec2 and a testfix mode when I went to land today
<jml> mwhudson: but the patch was small :)
<mwhudson> jml: well, ok, so "modulo usual launchpad development craptitude"?
<jml> mwhudson: yeah, modulo that, it was a piece of cake
<jml> mwhudson: I wish every Python library cared as much about backwards compatibility.
<jml> anyway, off to bed
<jml> g'night.
#launchpad-dev 2011-04-06
<sinzui> wallyworld: do you want to try mumble?
<wallyworld> sinzui: now i have no mic. i got my sound working eventually (seems a recent kernel update killed it for others too). i ended up having to upgrade to natty. but now it won't recognise my mic anymore :-(
<wallyworld> so i'm doing the phonon/pulse dance :-(
<thumper> wallyworld: is your mumble working
<wallyworld> thumper: ^^^^^^
<thumper> i silent wallyworld is a good wallyworld
<thumper> :)
<wallyworld> sinzui: sorry, no work :-(
 * wgrant deletes shipit.
<wgrant> Yay, this even lets us delete a celebrity.
<huwshimi> wgrant: As soon as I saw the announcement I knew you'd be happy :)
<thumper>  how is it coffee time again already
<wallyworld> thumper: at least you have a working coffee machine :-(
<thumper> hahah
<LPCIBot> Project windmill build #144: STILL FAILING in 46 min: https://lpci.wedontsleep.org/job/windmill/144/
<lifeless> gmb: did you find an OOPS for that bug 1 timeout ?
<_mup_> Bug #1: Microsoft has a majority market share <iso-testing> <ubuntu> <Clubdistro:Confirmed> <Computer Science Ubuntu:Invalid by compscibuntu-bugs> <EasyPeasy Overview:Invalid by ramvi> <GNOME Screensaver:Won't Fix> <Ichthux:Invalid by raphink> <JAK LINUX:Invalid> <The Linux OS Project:In Progress> <metacity:In Progress> <OpenOffice:In Progress by lh-maviya> <Tabuntu:Invalid by tinarussell> <Tivion:Invalid by shakaran> <Tv-Player:New> <Ubuntu:I
<lifeless> gmb: I'd be happy to eyeball it and see if I can suggest a cause
<lifeless> thumper: stacked branches
<lifeless> thumper: I see the code is going well
<thumper> lifeless: yup
<lifeless> thumper: did I correctly read that private http branch access isn't supported?
<thumper> lifeless: landing the branch id support now
<thumper> yes
<thumper> that is right
<thumper> we don't support any private branches over http
<lifeless> oh
<lifeless> thats easy then
<lifeless> I thought they worked in loggerhead
<thumper> that's different
<lifeless> well
<lifeless> loggerhead backs onto http
<thumper> it raises permission denied
<lifeless> inside the dc
<thumper> and loggerhead goes to https
<wgrant> It uses internal by-ID HTTP, right?
<thumper> lifeless: different rule
<wgrant> Like, raw by-ID, not +branch-id
<thumper> right
<lifeless> wgrant: its mapped to that
<lifeless> wgrant: I'm not sure how it handles stacked-on urls
<lifeless> which is why I'm asking these questions
<thumper> um...
<thumper> it should be the same way
<thumper> ...
 * thumper thinks
<thumper> I'll take a look
<wgrant> It must translatePath, right?
<lifeless> thumper: until we change the virtual policy file
<lifeless> thumper: this isn't going to magically affect people
<thumper> lifeless: yes, I'm not landing that pipe yet
<lifeless> thumper: so I'd check it on qastaging or whatever
<thumper> yep
<lifeless> just worth noting 'private stacked branches should work in loggerhead still' :)
<thumper> yes they should
<lifeless> \o/
<thumper> landing incremental support is good though
<lifeless> totally
<thumper> we can slam it on things like this
<wallyworld_> thumper: i got my sound working again. if you are free at some stage, i'd like to mumble to discuss some code
<thumper> wallyworld_: sure
<thumper> now is fine
<wallyworld_> ok
* StevenK changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: StevenK | https://code.launchpad.net/launchpad-project/+activereviews
<lifeless> aaaaaaaaaaaaaaaaaaaaaaaargh
<lifeless> I just read '    def getBatches(self):' in lazr.batchnavigator
<mwhudson> thumper/lifeless: loggerhead still uses that nasty transform_fallback_location hook i added to bzrlib i think
<thumper> heh
<thumper> mwhudson: ping
<mwhudson> thumper: hello
<thumper> mwhudson: I was going to ask you about how loggerhead talked
<mwhudson> thumper: ah, i think i may have been lying above
 * mwhudson tries to remember if he wrote this code
<mwhudson> aaah right
<mwhudson> thumper: basically, it uses translatePath under the hood, it actually accesses branches in a pretty similar way to how codehosting does now
<thumper> right
<thumper> it seems likely then that my change to the vfs to support +branch-id alias will work fine there too
<lifeless> I wonder if we should look at openid support for bzr
<lifeless>  / oauth
<mwhudson> thumper: yes
<thumper> \o/
<poolie> rather than ssh keys?
<poolie> http://productblog.37signals.com/products/2011/01/well-be-retiring-our-support-of-openid-on-may-1.html
<mwhudson> thumper: it seems like loggerhead should survive almost any rearrangement of how virtual paths map to real ones, basically
<mwhudson> (this is not true of the external http service)
<thumper> I've tweaked the rewrite map code to work with the new alias
<mwhudson> ah ok
<poolie> seriously, lplib started silently sending me to staging in natty? :(
<mwhudson> poolie: i realized the other week that what i wanted for openid to be useful was my browser to know to send me to my chosen OP whenever openid was offered as an option
<lifeless> poolie: for https access to private branches
<poolie> mwhudson, if it had been decently integrated into browsers it would have had a better chance
<poolie> probably chicken/egg
<lifeless> poolie: yeah, I saw that blog post
<mwhudson> poolie: did anyone actually try that?  it sort of seems that the standard was at the wrong level for it to be feasible
<lifeless> mwhudson: entirely wrong level
<lifeless> mwhudson: I don't recall /any/ chatter on http-wg about it
<poolie> well, exactly
<wgrant> poolie: lplib has always defaulted to staging, AFAIK...
<mwhudson> it works nicely enough, although with a bunch of unnecessary hair i guess, for canonical/ubuntu SSO
<poolie> even if you specify a different root
<poolie> i understand what it is; i'm filing a bug
<poolie> adding new parameters to python methods other than at the end is not a good idea for api compatibility
<wgrant> What's changed now?
<wgrant> get_token_and_login?
<lifeless> Distribution:+bugtarget-portlet-tags-content needs attention
<wgrant> Yeah. It almost looks cold-cachey, but it's hard to say.
<lifeless> wgrant: it probably just needs the countBugs aggregate function applied to it
<lifeless> wgrant: I bet most of the queries are table scans
<wgrant> lifeless: IIRC it's a single 10s query in a lot of cases.
<wgrant> I looked at it a while ago.
<lifeless> lets have a looksee
<poolie> wgrant, https://bugs.launchpad.net/launchpadlib/+bug/752095
<_mup_> Bug #752095: Launchpad constructor parameters changed incompatibly in natty <launchpadlib :New> < https://launchpad.net/bugs/752095 >
<wgrant> Grar.
<wgrant> "Organize your workflow with Launchpad.
<wgrant> "
<wgrant> From 37signals.com/accounts
<lifeless> 1 5211.0 2
<wgrant> Do people not Google these things :(
<lifeless> wgrant: 2 x 5000ms queries
<wgrant> poolie: Hm, is the constructor really a public API?
<wgrant> poolie: I've never seen it recommended, I don't think.
<lifeless> wgrant: also nonddy queries
<lifeless> wgrant: totally identical
<poolie> https://help.launchpad.net/API/launchpadlib recommends it
<poolie> and, its name suggests it's public
<lifeless> wgrant: and looking at every tag when in fact I think the portlet only shows official tags
<poolie> for fks sake
<poolie> how gratuituous, after all the pissing and moaning about compatibilyt
<lifeless> wgrant: so, kill the duplicate call, constrain by the official tags, win.
<wgrant> poolie: Bah :/
<poolie> i know api stability is hard in python
<poolie> this one is just a bit annoying because it doesn't just give a typeerror, it happily does nearly the right thing
<lifeless> wgrant: ok, I've analyzed it up
<lifeless> wgrant: we can make it a 300ms portlet
<lifeless> https://bugs.launchpad.net/launchpad/+bug/736002
<_mup_> Bug #736002: Distribution:+bugtarget-portlet-tags-content timeouts <timeout> <Launchpad itself:Triaged> < https://launchpad.net/bugs/736002 >
<LPCIBot> Yippie, build fixed!
<LPCIBot> Project windmill build #145: FIXED in 1 hr 2 min: https://lpci.wedontsleep.org/job/windmill/145/
<wgrant> Hmmm.
<wgrant> lifeless: I'm trying to QA the bugattachment LFC preloading thing.
<lifeless> wgrant: e-pic fail ?
<wgrant> The bug in question times out on qas, even without ?comments=all
<wgrant> Looking at an OOPS, the query take 20ms.
<lifeless> hit up ++profile++show
<wgrant> But there is a 4s gap afterwards.
<lifeless> wgrant: does it timeout on prod without ?comments=all
<wgrant> lifeless: No, it only takes 5s.
<wgrant> Erm.
<wgrant> Got a 502 from qas when asking for a profile.
<wgrant> Odd.
<lifeless> wgrant: you've marked it ok, was that optimistic ?
<wgrant> Erm, that must have been the wrong bug.
<wgrant> lifeless: http://paste.ubuntu.com/590009/ only returns a few hundred rows on qas?
<lifeless> wgrant: 750949
<wgrant> Er....
<lifeless> [Bug 750949] [NEW] BugTask:+index times out with lots of attachments .. ** Tags added: qa-ok
<_mup_> Bug #750949: BugTask:+index times out with lots of attachments <qa-ok> <timeout> <Launchpad itself:Fix Committed by wgrant> < https://launchpad.net/bugs/750949 >
<wgrant> Oh, right, bug number, not row count.
<lifeless> ** Tags removed: qa-needstesting
<wgrant> Fixed.
<lifeless> 342 rows
<wgrant> Right.
<wgrant> So 4s after that is completely insane.
<wgrant> Also, qas seems to be dead now.
<wgrant> Maybe it's updating slowly.
<lifeless> 40ms
<wgrant> Right.
<wgrant> That's slightly longer than the OOPses show, but still not too terrible.
<wgrant> Ah, now it's sometimes timing out before it even gets to attachments.
<wgrant> So I think it might just be generally unwell.
<wgrant> Random multi-second sleeps in there too.
<wgrant> So I'm going to re-OK this and blame it on asuka being shit.
<lifeless> check the asuka load graph
<lifeless> huwshimi: btw
<lifeless> huwshimi: ajax log on qastaging is bokred
<huwshimi> lifeless: Oh
<wgrant> lifeless: Ugh, any idea what's got it so high?
<wgrant> Spiked massively an hour ago.
<lifeless> askalosa
<wgrant> Huh.
<wgrant> Interesting.
<wgrant> spm: Hi.
<spm> damn. why does my /ignore not work anymore. :-(
<wgrant> spm: :(
<wgrant> spm: Something made asuka reasonably angry load- and RAM-wise right around midnight UTC.
<wgrant> any idea what that is?
<wgrant> (it is somewhat recovering now, but still not ideal)
<huwshimi> lifeless: Where abouts is it not working?
<wgrant> In fact it appears to now have flattened out at a heightened level.
<wgrant> Whereas it normally drops back down to normally quickly after this sort of spike.
<spm> wgrant: hrm. not offhand. tho the load in general for asuka is pretty spiky. I'd suggest a combo of a rollout of something and/or cronjobs firing off
<wgrant> spm: Spiky and *11*.
<wgrant> I tried to use capital digits there.
<wgrant> Sadly they don't exist.
<spm> wgrant: heh, I see about 15 of those over the past week. :-/
<wgrant> Indeed, but they're fairly irregular.
<spm> ish
<wgrant> I guess I'll see if it's less unhappy in a few hours.
<spm> we should probably fork qas onto a separate box
<wgrant> Or just toss the scripts elsewhere and see what happens.
<wgrant> I hear we have at least two free appserver boxes now ;)
<StevenK> Haha
<wgrant> Maybe we can convince gandwana and potassium to serve their sentences on qas, too.
<wgrant> Since I doubt anybody else wants them :)
<spm> heh, I am not typically privy to the next hop destinations of servers. :-)
<lifeless> spm: there is a ticket to move the scripts off
<spm>  /forcenickchange lifeless ticketmaster
<lifeless> huwshimi: huh, iz working now
<lifeless> it wasn't working before
<wgrant> lifeless: It just updated.
<wgrant> lifeless: May have been icing skew?
<lifeless> perhaps
<lifeless> spm: I prefer KeyMaster, thanksverymuch
<spm> sigh. delusions of grandeur.
<lifeless> wgrant: and w/out ?
<wgrant> 4s
<wgrant> Slightly faster than prod.
<lifeless> +1
<wgrant> But not much.
<lifeless> nie work
<mwhudson> hee hee, http://bazaar.launchpad.net/+branch/linaro-android-build-tools/files/head:/ works
<lifeless> mwhudson: of course
<lifeless> mwhudson: or is that unexpected somehow ?
<wgrant> It's pretty new.
<wgrant> And probably an unintended side-effect.
<mwhudson> lifeless: it's new-ish at least
<thumper> bah!
<thumper> I'm trying to make a unit test that does the equivalent of: http://pastebin.ubuntu.com/590022/
<thumper> anyone got any ideas?
<wgrant> You should be able to add a header to a testbrowser...
<thumper> hmm...
<wgrant> Using the remarkable browser.addHeader method.
<thumper> right now it oopses
<thumper> which is what i'm wanting to fix
 * lifeless hates on doctests
<StevenK> lifeless: Then remove them?
<lifeless> StevenK: its the README
<StevenK> Bwaha
<lifeless> anyone see a reason for start=0 to be present on the first batch url ?
<wgrant> No.
<lifeless> may be some test fallout... shurg
<lifeless> I've already caused that
<wgrant> Test fallout is dealable with.
 * lifeless loves matchers
 * wgrant unbreaks publish-distro.d
<wgrant> StevenK: You has QA.
<wgrant> Hmm.
<wgrant> We need a publisher on qastaging.
<StevenK> wgrant: The QA tagger should move to someone else
<wgrant> StevenK: Oh?
<StevenK> Indeed, it's moved to r12746.
<wgrant> Yeah.
<wgrant> Waiting for my fixed publish-ftpmaster.py to run before I try PPA publishing.
<StevenK> On DF?
<wgrant> Yeah.
<wgrant> I've turned on the run-parts things and made them work.
<StevenK> That could take eons
<wgrant> So we might get a full run.
<wgrant> No!
<wgrant> jtv deleted PublishingTunableLoop, so it should only take about 10 minutes to generate file lists now.
<wgrant> Indeed, now into a-f itself.
<StevenK> I wonder how much that will speed up the publisher on cocobanana
<StevenK> wgrant: Do you need to land a branch of fixes for the run-parts fixes?
<wgrant> Yes :(
<wgrant> StevenK: Not by more than a minute or so, I don't think.
<wgrant> But let me check.
<wgrant> Seconds at most.
<wgrant> Since wildcherry can keep the necessary indices in RAM.
<StevenK> So the massive speed up is for DF's benefit only
<wgrant> Right.
<wgrant> I guess it'll be around a 15-20s improvement for natty on prod.
<wgrant> ie. nothing at all.
<lifeless> done
<wgrant> gpg: signing failed: secret key not available
<wgrant> :(
<wgrant> lifeless: What's done? Batchnav?
<lifeless> yeppers
<lifeless> phase 1
<wgrant> Nice work!
<wgrant> Does it work>?
<lifeless> who knows
<wgrant> Argh.
<lifeless> ?
<lifeless> bzr+ssh://bazaar.launchpad.net/~lifeless/lazr.batchnavigator/keyoffset/ if you want to peek, mp coming up now
<wgrant> publish-ftpmaster.py's signing hook doesn't work.
<wgrant> Since the GPGHandler utility overrides GNUPGHOME to a temp dir.
<lifeless> does this break the deply?
<wgrant> No.
<wgrant> this is the rev we didn't merge.
<wgrant> And we weren't even planning to move to this new script immediately either.
<lifeless> I can has review?
<lifeless> https://code.launchpad.net/~lifeless/lazr.batchnavigator/keyoffset/+merge/56499
<lifeless> StevenK: ^
<wgrant> 1400 lines.
<wgrant> That's almost into Soyuz territory.
<lifeless> StevenK: if you whine that its too large, I'll whine that its using copied code from zope
<lifeless> wgrant: I can halve it by deleting comments and tests.
<lifeless> wgrant: Lines of code is a terrible complexity metric
<lifeless> wgrant: particular because of context
<lifeless>  12 files changed, 659 insertions(+), 209 deletions(-)
<lifeless> is a better-but-still-not-great indicator
<StevenK> lifeless: I think you linked the wrong bug
<lifeless> StevenK: thats merge proposals being noddy
<lifeless> StevenK: bug 726195
<_mup_> Bug #726195: merge proposals show closed bugs <code-review> <Launchpad itself:Triaged> < https://launchpad.net/bugs/726195 >
<lifeless> which is nontrivial
<lifeless> because we should capture the bugs against the MP
<lifeless> so the branch can get more links and not alter the MP
<lifeless> this would also make some queries faster
<StevenK> lifeless: I'm not sure about your choice of 'memo' as a variable name -- especially since it's shown in URLs
<lifeless> StevenK: cosmetic and easy to change if you have a better label
<lifeless> StevenK: its going to be base64 junk
<lifeless> StevenK: not hand guessable or enterable
<StevenK> Well, I'm not sure what you're trying to show with it
<lifeless> StevenK: its the db constraints needed to get back to a particular batch, serialised.
<lifeless> e.g. for a bug search it might be ABBD23427834EfJ
<lifeless> meaning 'bug heat < 43 and bugtask.id > 34'
<lifeless> http://en.wikipedia.org/wiki/Memoization
<StevenK> Right
<StevenK> lifeless: I'm ... I don't know, half-way? Breaking for lunch, so my stomach doesn't break out and insist.
<lifeless> StevenK: thats cool
<lifeless> StevenK: I'm going to start on a 'lets see if this breaks lp' branch of lp
<wgrant> Aha! It no longer crashes!
<wgrant> We may have victory.
 * wgrant carefully publishes a few suites.
<wgrant> lifeless: https://code.launchpad.net/~wgrant/launchpad/bug-752149/+merge/56500
<wgrant> Oh, actually, even simpler.
<lifeless> wgrant: tell me when
<lifeless> I kindof wish we'd done an april fools thing
<lifeless> like skinning lp like sourceforge and announcing a merge
<wgrant> Heh.
<wgrant> Even Debian did one this time.
<lifeless> I know
<wgrant> lifeless: Diff updated.
<lifeless> sadness, make schema day
<lifeless> we should make that incremental
<wgrant> lifeless: You mean like, say, running upgrade.py?
<lifeless> mmm, or make it obvious how to do it incrementally if one knows how
<lifeless> wgrant: well its not always safe
<lifeless> wgrant: but yes, upgrade.py on launchpad_ftest_template etc
<wgrant> Also, make schema isn't *that* slow any more.
<lifeless> tell that to my wadl compiler
<lifeless> I'm just wearing my curmudgeon hat today
<wgrant> lifeless: Thanks.
 * wgrant glares at mawson.
<wgrant> spiv: Hi.
<lifeless> ok, launchpad bugs needs to die
<lifeless> can we just delete that catchall ?
<lifeless> let folk use the shocking mute functionality ?
<wgrant> We should probably blackhole it for now.
<wgrant> I was hoping the bug subscription feature work would think about how to fix not just bug notifications, but apparently not :/
<poolie> lifeless, i'm aware there's a "bugs i reported" function
<poolie> nm, i'll comment on the bug
<lifeless> wgrant: well, by removing the thing, I think we can subscribe things more directly and appropriately
<wgrant> lifeless: I guess we'll also find out what's going there pretty quickly.
<wgrant> And work out how to fix it.
<Ursinha-afk> lifeless, hi, did you see any other occurrences of bug 727552?
<_mup_> Bug #727552: when first rev is bad, with a later rollback, qatagger incorrectly reports it deployable atthe top <qa-tagger:In Progress by ursinha> < https://launchpad.net/bugs/727552 >
<Ursinha-afk> I can't reproduce that
<spiv> wgrant: hi?
<wgrant> spiv: Your Codehosting Twisted evil needs QA.
<StevenK> Eyes bleeding ...
<spiv> wgrant: ok.  What would that look like?  Would verifying that 'bzr push' of a new branch still works, (to staging/qastaging) suffice?
<lifeless> so
<lifeless> sob
<lifeless> GoogleBatchNavigator - facepalm
<StevenK> lifeless: Burn it with Holy Fire?
<StevenK> spiv: Well, how did you notice the slowness in the first place?
<lifeless> Ursinha-afk: well I saw it right before I reported the bug, unless its been specifically fixed i suspect its still the case
<lifeless> StevenK: jam noticed it locally in a vm
<lifeless> spiv: pushing to qastaging is good - be sure to trigger some failures
<spiv> StevenK: jam noticed it.
<lifeless> StevenK: can't burn it trivially, it wants a refactor of the base class, for now I'll just copy and paste and cry
<spiv> StevenK: I saw the profiling data he generated, applied my knowledge of which bits of Twisted are particularly crummy, jam did a crude hack confirming our hypothesis, I wrote a proper workaround (and proper fix for upstream, plus micro-benchmark for upstream).
<spiv> lifeless: practically everything we do triggers failures
<spiv> lifeless: that's the issue! ;)
<spiv> lifeless: jam noticed this issue from a simple initialize BzrDir RPC.
<lifeless> spiv: I know, familiar with it
<lifeless> spiv: I mean trigger something we expect to return an error to you, the user.
<spiv> Ok.
<lifeless> spiv: because if its going to break, I'd expect that to be where it does.
<lifeless> e.g. unknown project
<spiv> So, 1) push up a new branch to somewhere that works, ~user/proj/branch, and 2) try a push that should fail, e.g. to ~user/no-such-project/foo ?
<lifeless> or a project you don't have permissions to push into
<lifeless> yes, exactly
<spiv> Ok.  When will the code be on qastaging?
<wgrant> It is now.
<spiv> Great!  I'll do that QA now.
<lifeless> spiv: when the qatagger comments on  abug saying 'x landed', its reached a staging server.
<lifeless> spiv: the specific one depending on the branch it landed on
<spiv> Sounds handy, although I'm sure to have forgotten that by the next time I have another LP change to land and QA.
<StevenK> db-stable => staging, stable => qastaging
<spiv> Or just as likely, the process is likely to have changed ;)
<StevenK> Our QA process hasn't changed for a few months.
<spiv> StevenK: yes, my point exactly
<lifeless> spiv: I think we'll make it faster, but not change this aspect
<spiv> StevenK: just how recently do you think I've landed something? :)
<lifeless> the notify a human now aspect is very useful
<lifeless> spiv: 6 weeks?
<spiv> lifeless: and it didn't pass QA :P
<spiv> (I'm not complaining, I think you have a good process that's working well: the fact that wgrant was able to ping me about this in a timely fashion shows this process is coping adequately with infrequent contribuors like myself)
<StevenK> lifeless: Pass the bleach, r=me
<lifeless> StevenK: thanks!
<StevenK> lifeless: With a little caveat that you can ignore if you wish
<wgrant> spiv: lifeless and I keep track of our QA status pretty closely,.
<StevenK> I tend to watch it, I just don't nag.
<spiv> StevenK: I don't need nagging, I just need my hand held because I don't have much familiarity with your processes any more.
<StevenK> spiv: That was a dig at wgrant and lifeless, rather than you.
<spiv> wgrant: my testing worked.  Do I just change the tag to qa-ok now?
<wgrant> spiv: Yup.
<spiv> wgrant: done.  Thanks!
<wgrant> Thank you!
<wgrant> Much better than me having to do it myself tonight :)
<spiv> :)
<lifeless> wgrant: is there anything you need from me ?
<poolie_> lifeless, hi, i was looking at outstanding bugs assigned to me
<poolie_> what do you think i should do on https://code.launchpad.net/~mbp/launchpad/690021-rlimit/+merge/43733
<poolie_> your last comment seems to suggest 'land it'
<poolie_> i think landing it and seeing what happens is probably the best option
<poolie_> or just abandoning it
<wgrant> lifeless: Don't think so.
<lifeless> poolie_: I would like to see it landed
<poolie_> as is?
<lifeless> poolie_: we know the branch that causes this (linux ;))
<lifeless> so we can see if it loops as tim fears it may
<lifeless> I would be ok with landing it as is, as long as when we deploy this rev we know to look for this potential problem
<lifeless> we can rollback just the import slaves *if* its a problem
<poolie_> ok, that makes sense to me
<poolie_> thanks; hooray for unsticking things
<wgrant> The lucid non-release pockets are big :(
<lifeless> StevenK: https://code.launchpad.net/~lifeless/launchpad/bug-752153/+merge/56505
<StevenK> lifeless: Oh, and it copies code due to your earlier comment
<StevenK> lifeless: r=me
<lifeless> StevenK: thanks
<lifeless> next step, migrate a real collection, like FTBFS, to use a custom IRangeFactory
<thumper> simple fix: https://code.launchpad.net/~thumper/launchpad/fix-dud-referer/+merge/56508
<thumper> I was trying to fix bug 44919
<_mup_> Bug #44919: UnicodeDecodeError while POSTing forms with non-ascii characters. <infrastructure> <lp-foundations> <oops> <Launchpad itself:Triaged> < https://launchpad.net/bugs/44919 >
<thumper> but the oopses I looked at weren't related to that failure
<thumper> and I didn't notice
<lifeless> I have a suggested tweak
<thumper> so I filed bug 752218
<_mup_> Bug #752218: Referer header with non-ascii oopses on not found page <oops> <Launchpad itself:In Progress by thumper> < https://launchpad.net/bugs/752218 >
<thumper> lifeless: for me?
<StevenK> thumper: r=me
<lifeless> thumper: yeah, its on the revier
<lifeless> *review*
<thumper> ok
<thumper> lifeless: the whole point of getting the referer is to show a link to the user to say "go back to where you came from"
<thumper> if we replace the dud characters
<lifeless> thumper: oh
<thumper> it'll fail
<lifeless> thumper: rightyho
<thumper> and it's bogus anyway
<thumper> normally spam
<thumper> from the oopses I looked at
<lifeless> thumper: yeah. the underlying bug here - which you might like to put a comment about
<lifeless> is that we shouldn't be decoding the referrer at all
<lifeless> urls that we did not generate can legitimately be in /any/ encoding
<wgrant> The moral of the story: Python 2 sucks, let's use Python 3.
<lifeless> because it makes this worse
<thumper> the problem is that the http request has a byte string
<thumper> but when we try to put it into the tales
<thumper> it gets coerced into unicde
<thumper> and dying
<lifeless> it has to stay a byte string the whole way through
<lifeless> I know thats way out of scope to fix
<thumper> it will never though to include in the page
<lifeless> and I'm ok with your branch
<lifeless> I'm simply noting that if someone comes along and goes 'wtf we drop it' - it would be worth a mention at that point in the code
<thumper> most the templating engines these days are unicde based
<thumper> lifeless: I'll add a comment
 * thumper wonders what happens to the 'o' in unicode
<lifeless> its fine that they are unicode based, but they /render/ to something defined as bytes, not as ascii or utf8.
<thumper> eventually yes
<thumper> but the unicode intermediate step causes pain
<lifeless> yuuuup
<lifeless> oh, ffs, timeouts connecting to amazzzon again
<lifeless> wgrant: that ftbfs collection timeout
<lifeless> wgrant: has teh script been rewritten
<lifeless> ?
<lifeless> bug 730396 I think
<_mup_> Bug #730396: DistroSeries:EntryResource:getBuildRecords timeouts <timeout> <Launchpad itself:Triaged> < https://launchpad.net/bugs/730396 >
<wgrant> lifeless: Not yet.
<lifeless> wgrant: 20ms batches of FTBFS
<lifeless> wgrant: except for the last one, because the table structure is gnarly, as I keep whinging about
<lifeless> wgrant: this is about 30 times faster
<wgrant> lifeless: Nice!
<lifeless> https://bugs.launchpad.net/launchpad/+bug/730396
<lifeless> bah
<_mup_> Bug #730396: DistroSeries:EntryResource:getBuildRecords timeouts <timeout> <Launchpad itself:Triaged> < https://launchpad.net/bugs/730396 >
<lifeless> https://bugs.launchpad.net/launchpad/+bug/730396/comments/14
<_mup_> Bug #730396: DistroSeries:EntryResource:getBuildRecords timeouts <timeout> <Launchpad itself:Triaged> < https://launchpad.net/bugs/730396 >
<lifeless> poolie: I think its good to question the canned searches we have on bugs.l.n/+me
<poolie> me too
<lifeless> poolie: I may have come across as pushing back on you earlier; i didn't intend to
<poolie> i was having a kinda irritable day
<poolie> i'm finding unity really stunning in concept and rather flaky as actual software
<poolie> oh, also the lplib thing was annoying
<wgrant> Yes, it's really great to use, except that it crashes.
<wgrant> Lots.
<wgrant> And when it doesn't crash it glitches.
<poolie> its failure mode of talking to staging instead of lpnet was a big timewaster
<lifeless> wgrant: you need to ask the devs to take the if username=='' check out
<poolie> anyhow, i agree about reconsidering the canned searches, but on the other hand this is probably not the most important change there
<poolie> one thing i would like a lot is to make them show most-recent-first
 * wgrant headdesks.
<wgrant> lifeless: Have you looked at BugTargetBugTagsView?
<wgrant>         popular_tags = [tag['tag'] for tag in sorted(
<wgrant>             other_tags, key=itemgetter('count'), reverse=True)[:10]]
<lifeless> wgrant: can you please take that blunt spoon out of my eyeballs?
<wgrant> I knew it was bad, but wow...
 * lifeless considers breaking list comprehensions in lp
<lifeless> wgrant: there are 6K unique tags in lp
<lifeless> sorry
<lifeless> in ubuntu
<wgrant> Yeah.
<wgrant> It is stupid.
<lifeless> wgrant: criminal
<lifeless> wgrant: it looks like *product* bug tag portlets may show every tag
<lifeless> wgrant: check out the launchpad portlet for instance
<wgrant> lifeless: I don't think so.
<lifeless> https://bugs.launchpad.net/launchpad -
<wgrant> launchpad-project's is a little odd, because it inherits the old disabled ones.
<lifeless> look at it.
<wgrant> Hmm.
<lifeless> do we really have ~100 official tags?
<lifeless> including 'arm'
<lifeless> and 'boobytrap'
<wgrant> boobytrap is official.
<wgrant> arm is official.
<lifeless> so is arm
<lifeless> wtf
<wgrant> boobytrap is one of Julian's.
<lifeless> well, apparently we do.
<lifeless> we should nuke about half of these.
<wgrant> story-better-bug-notification is unofficial, but probably in the top 10.
<lifeless> oh, it shows the top 10.. yay
<lifeless> uhm
<lifeless> denormalisation time?
<lifeless> possibly a better query will be faster
<wgrant> Well, I was going to try to improve the query, and if that didn't work just drop the top 10.
<lifeless> add a rank clause for starters
<wgrant> Or drop the top 10 if there are official tags, or something like that.
<wgrant> Why?
<lifeless> 6K results back is pointless
<wgrant> ORDER BY COUNT(Bug) LIMIT 10?
<lifeless> count(tag); yeah
<wgrant> Er, count(tag)?
<wgrant> What good would that do?
<wgrant> Oh, count(BugTag), you mean?
<lifeless> bugtag.tag
<lifeless> the point is to only return the 10 most popular
<lifeless> wgrant: one change I think would be uncontentious
<lifeless> wgrant: include official tags in the top 10 calculation
<lifeless> wgrant: so that if (say) 5 of the official tags are also top 10, one sees 15 tags total.
<wgrant> rofl, for the first 8 months it showed the bottom 10 instead of top 10.
<lifeless> erm count(official) + 5
<lifeless> wgrant: \o/
<lifeless> yay testing.
<wgrant>     def _getSearchURL(self, tag):
<wgrant>         """Return the search URL for the tag."""
<wgrant>         # Use path_only here to reduce the size of the rendered page.
<wgrant>         return "+bugs?field.tag=%s" % urllib.quote(tag)
<wgrant> :/
<wgrant> I've not seen that reason used anywhere else.
<lifeless> right, where was I before I knocked the battery out
<StevenK> Fail
<lifeless> thanks, I hadn't noticed
<wgrant> ...
<wgrant> Grar.
<wgrant> That tag cloud is calculated after taking privacy into account.
<wgrant> But is cached publicly.
<lifeless> hmm
<lifeless> the queries I saw had no privacy clause, just private=False
<lifeless> or was that cause they were anonymous requests?
<wgrant> User: Anonymous User
<lifeless> k
<lifeless> I don't know that memcache saves us much here anyhow
<lifeless> same argument as usual: if misses suck, we're in trouble
<lifeless> if misses don't suck, do we care
<wgrant> If it wasn't loaded in a separate request I think it would be worth preserving it.
<wgrant> Ah, getUsedBugTags* only has twoish callsites, fortunately.
<wgrant> Because both of those methods are broken and need a rewrite :(
<lifeless> wgrant: you might like my countBugs aggregat ehelper
* mthaddon changed the topic of #launchpad-dev to: Launchpad down/read-only from 08:00-09:30 UTC for a code update | https://dev.launchpad.net/ | On call reviewer: StevenK | https://code.launchpad.net/launchpad-project/+activereviews
<LPCIBot> Project devel build #612: FAILURE in 5 hr 41 min: https://lpci.wedontsleep.org/job/devel/612/
<StevenK> Baaaaaaaaaah, windmill
<StevenK> When did you come back?
<wgrant> If it's failed again, it is going to be killed again in 5 minutes.
<wgrant> Although I guess it's not actually testfixed us yet, so it can stay for a while.
<wgrant> But the moment it fails in buildbot I am killing it again.
<StevenK> Heh
<StevenK> wgrant: Not even a second chance?
<wgrant> It has had three already :)
<StevenK> The fact that it failed in Jenkins and not Buildbot makes me sad.
<wgrant> Jenkins has always been better at exposing fragile tests :/
<StevenK> Two time out errors makes me suspcious
 * StevenK runs 'bzr rm lib/lp/soyuz/pas.py' to make him feel better.
<lifeless> bug  740750 makes me very sad
<_mup_> Bug #740750: API timeouts broken and returning no useful data... <regression> <Launchpad itself:Triaged> <lazr.restful:Triaged> < https://launchpad.net/bugs/740750 >
<lifeless> anyhow, I need to talk to an api plumbing person
<adeuring> good morning
<lifeless> wgrant: I might grab a little of your time tomorrow to bounce ideas about glueing $stuff together
<wgrant> lifeless: Sure.
<wgrant> Oh, it's time.
<lifeless> wgrant: on tags
<lifeless> wgrant: is it constrained to open bugs only now ?
<wgrant> lifeless: Yes, but it seems to fail to exclude dupes.
<lifeless> s/open/only open/
<mrevell> Hello, good morning
<lifeless> I wonder if there is a poor non-api collection I could improve
<lifeless> wgrant: is that ftbfs collection also in the web ui?
<wgrant> lifeless: Something very like it, yes.
<wgrant> lifeless: https://launchpad.net/ubuntu/+builds uses Distribution.getBuildRecords.
<wgrant> /~wgrant/+archive/ppa/+builds for Archive.getBuildRecords.
* StevenK changed the topic of #launchpad-dev to: Launchpad down/read-only from 08:00-09:30 UTC for a code update | https://dev.launchpad.net/ | On call reviewer: - | https://code.launchpad.net/launchpad-project/+activereviews
<lifeless> https://launchpad.net/ubuntu/natty/+builds?build_text=&build_state=failed&arch_tag=all
<lifeless> https://launchpad.net/ubuntu/natty/+builds?arch_tag=all&build_state=failed&build_text=&start=3150&batch=75
<lifeless> would seem to match the behaviour
<mpt> Is there a graph anywhere of how much faster Launchpad has been getting?
<mpt> lifeless? ^
<lifeless> mpt: its a bit noisy
<lifeless> but
<lifeless> https://lpstats.canonical.com/graphs/PPR/
<lifeless> this doesn't show the in-dc queueing problem which I discussed in my capacity email to the -dev list
<lifeless> for isntance
<lifeless> https://lpstats.canonical.com/graphs/PPR/20100407/20110407/
<wgrant> And the in-DC queueing was a *big* problem.
<mpt> thanks
<lifeless> we don't yet have metrics for in-dc queuing
<lifeless> we have a nagios check coming for one known symptom
<jkakar> Maybe there's already a bug filed, but sometimes when Launchpad is in read-only mode I get two of the yellow notification boxes, when viewing a bug, that tell me Launchpad is in read-only mode.  It's seems to be consistent for particular bugs, I think, but there doesn't seem to be an otherwise obvious pattern.
<lifeless> jkakar: I want to nuke readonly mode
<lifeless> jkakar: its a source of longer downtime, ironically enough
<jkakar> lifeless: That sounds like an excellent fix for the issue. :)
<stub> jkakar: Are you using two browser windows by any chance?
<jkakar> stub: Nope, one instance of Minefield (FF4) with several tabs open.
<stub> jkakar: Several tabs open on Launchpad? Its a known issue that if you are using two browser windows to Launchpad the race conditions can cause notifications to pop up on the wrong browser
<jkakar> stub: Ah, that could be it, yes.
<jkakar> stub: It happened (sometimes) when I was viewing a milestone listing and Ctrl-clicked several bugs to open in new tabs.
<stub> That would do it. The key to 'which notifications should I display' is a cookie, and that same cookie is used for all browser windows of course.
<lifeless> mpt: for instance, see translations 99th percentile goes from 8 down to 4 seconds
<lifeless> mpt: on the year-wide graph
<lifeless> mpt: the red line is the over-everything metric, which is very slow to move (because > 50% of renders are API, and API is already generally pretty good
<lifeless> )
<mpt> lifeless, thanks, ivanka just wanted an answer to the question "Why is Launchpad down for maintenance", and that graph was the most effective rationale I could think of :-)
<lifeless> heh
<lifeless> mpt: so the the problem is the technology
<wgrant> ... that covers most problems with Launchpad.
<lifeless> mpt: schema changes are tricksy in postgresql w/replication
<lifeless> https://dev.launchpad.net/Database/LivePatching
<lifeless> https://dev.launchpad.net/LEP/ReliableDBDeploys
<lifeless> mpt: both those links are technical, but I'm sure you can translate ;)
<mpt> in my copious spare time
<lifeless> we just gave you 90minutes of time ;)
<mpt> zing!
<lifeless> mpt: anyhow, short story, we've dropped nearly 2 seconds off the 99th percentile
<lifeless> mpt: and 9 seconds off the hard timeout cap (which is set to maintain < 0.1% failure rate)
<wgrant> And we'll hopefully drop it again tomorrow...
<lifeless> wheee
<lifeless> https://bugs.launchpad.net/ubuntu 1.91 seconds
<wgrant> Ah, great. It was 2.8 when I tried before.
<wgrant> Must have warmed up.
<lifeless> 2.8 is still better than 11 :)
<wgrant> Very true.
<lifeless> https://bugs.launchpad.net/ubuntu/natty was 2
<lifeless> https://bugs.launchpad.net/bzr 1.68
<lifeless> https://bugs.launchpad.net/bazaar 1.81
<lifeless> bzr 1.27 hot
* mthaddon changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: - | https://code.launchpad.net/launchpad-project/+activereviews
<lifeless> 2.16 for launchpad-project
<lifeless> 1.58 hot
<lifeless> before this change
<lifeless> only 20% of page loads for the ubuntu bugs-index were under 5 seconds
<lifeless> 50% under 8
<lifeless> this is a massive win
<lifeless> BugTask:+text needs a fixin
<lifeless> wow
<lifeless> Bug:+text - 65K hits a day
<lifeless> ProjectGroupSet:+all is half fast and half slow
<lifeless> I thought I fixed it :(
<lifeless> oh, or maybe it wasn't deployed
<lifeless> wgrant: I think Distribution:+bugtarget-portlet-tags-content needs fixing before we drop the timeout
<lifeless> 12s 99th percentile
<wgrant> lifeless: Whitelist whitelist.
<lifeless> wgrant: I think we should wait till monday;
<wgrant> lifeless: Do you have IP addresses for BugTask:+text? I suspect Debian.
<jam> wgrant: fixing is better than whitelisting :)
<lifeless> wgrant: that gives us a complete day on thurs->friday
<wgrant> lifeless: True.
<wgrant> lifeless: Plus we'll have an ndt before Thursday.
<lifeless> yup
<lifeless> so, logs will tell us
<lifeless> I don't have an IP handy
<jam> lifeless: so I saw that spiv qa'd his twisted patch, how does that work in a deployment sense?
<jam> Since codehosting is still downtime required
<wgrant> lifeless: lucas imports Ubuntu bugs into UDD, and I think that still uses BugTask:+text
<jam> but you are trying to only do a db deploy during this downtime
* danilos changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: danilos | https://code.launchpad.net/launchpad-project/+activereviews
<lifeless> jam: we either schedule downtime, or it gets rolled out next db deploy
<jam> lifeless: so currently codehosting *does* get rolled out during db deploy
<jam> (even if that particular patch missed this downtime)
<lifeless> yes
<lifeless> its still a full deploy
<lifeless> but I'm trying to get things detangled
<lifeless> there are complicating factors
<lifeless> hmm
<lifeless> I think we need the PPR to go into 5ths of a second
<wgrant> lifeless: Yeah :(
<lifeless> night
<henninge> rvba: r=me ;-)
<rvba> henninge: \o/
<wgrant> bigjools: Want to land your config branch now? I'll remove the schema keys tomorrow.
<rvba> henninge: thank you for your patience and precious advices.
<wgrant> henninge: Thanks for encouraging the proper use of structured()!
<henninge> wgrant: welcome
<henninge> wgrant: I was surprised, though, that .escapedtext has to be used explicitely.
<wgrant> henninge: Yes :/ That's a bit silly.
<henninge> it is
<rvba> henninge: wgrant I've put the call to escapedtext in the template though
<henninge> I saw that. I have no preference either way.
<wgrant> It belongs in the template.
<henninge> rvba: well done ;)
<deryck> Morning, all.
<jml> morning
<jkakar> Hiya jml!
<jml> jkakar: hello
<rvba> danilos: Hi! Could you take a look at https://code.launchpad.net/~rvb/launchpad/dds-add-unique-packages/+merge/56548 for me?
<danilos> rvba, hi, sure thing
<danilos> rvba, is it intentional that this pages uses the same template as +localpackagediffs?
<rvba> danilos: yes
<danilos> rvba, if so, you'd probably want to rename the template so it better indicates what is it about
<rvba> danilos: that's right.
<rvba> danilos: actually the template name is distroseries-localdifferences.pt. The 3 pages sharing this templates are all used to display local differences ... of different types.
<danilos> rvba, ok, then I guess it's fine to keep the name
<rvba> yep
<danilos> rvba, the branch looks pretty good otherwise, but I have another naming question: I believe it's our practice to name all our view classes as SomethingSomethingView, so it'd be nice to change that as well
<danilos> s/as well//
<danilos> rvba, other than that, r=me
<rvba> danilos: great, I'll look into the naming thing.
<danilos> rvba, basically, just append "View" to the class name, and, seeing there's more of them like that (for the other pages), we might want to fix those (in a different branch), or discuss this on the mailing list :)
 * rvba is doing that right now.
<danilos> cheers
<jml> mrevell: could I please skype you to test out my new webcam?
<mrevell> sure
<mrevell> jml, please go ahead, caller
<mrevell> jml, I see you
<mrevell> but I don't hear you jml
<bigjools> shakin that ...
<bigjools> wgrant: around?
<wgrant> bigjools: Mostly.
<bigjools> wgrant: just wanted to run something by you quickly as it seems like I have a too-easy solution to something :)
<wgrant> bigjools: Sure.
<bigjools> wgrant: when we open a new distroseries, one of the steps taken is to run the publisher in careful mode
<bigjools> wgrant: I am thinking that we can avoid that by creating all the publications as pending instead
<wgrant> bigjools: That will take many hours to publish.
<bigjools> there must be a catch
<wgrant> bigjools: It will hash a hundred gigabytes or so.
<wgrant> == slow
<bigjools> hmmm
<bigjools> why is careful publishing quicker
<wgrant> Hmm. Is it a full -C publish?
<wgrant> Or just -A?
<bigjools> -A
<wgrant> -A is indices only.
<bigjools> yes
<bigjools> so it avoids the file checks
<wgrant> -C implies -P, which tries to rewrite all of the files from the DB.
<bigjools> arse
<wgrant> Yis.
<bigjools> so there's the catch
<wgrant> Yes.
<wgrant> Explicit pocket dirtying would be nice.
<wgrant> Can has?
<bigjools> heh
<bigjools> that would also force file publishing no?
<wgrant> No.
<wgrant> It would just mark an extra suite as dirty, requesting normal domination and index generation.
<wgrant> (it would also make the publishable safely killable)
<bigjools> yeah
<bigjools> might have to do this - we need a way of making new distroseries "just work" with no shell access
<wgrant> Yeah. It's pretty simple, too.
<wgrant> New (archive, distroseries, pocket) table, set by the init job, unset at the end of the publisher.
<bigjools> but.... pockets ... :/
<bigjools> some distros don't have pockets, so it'd need to be (archive, suite)
<wgrant> bigjools: Right, once we have explicit suites.
<wgrant> But until then suite == (distroseries, pocket)
<wgrant> I hope soon we'll delete pockets.
<bigjools> that won't work for distros other than ubuntu
<wgrant> But that's not how it is now.
<wgrant> Sure, but neither will SPPH, BPPH, PackageUpload, PackageBuild, or just about anything.
<bigjools> some distros want to keep updating the release "pocket" even after release
<wgrant> They will all need s/(distroseries, pocket)/suite/, and that can be done when we have a Suite table.
<bigjools> yay
<wgrant> And then pockets can GTFO LP.
<wgrant> I'm not sure whether Suite wants to include Archive... I guess we'll see what's best.
<bigjools> probably not, there could be an archivesuite table later
<bigjools> but... use cases first :)
<wgrant> Yes.
<bigjools> bug 174375
<_mup_> Bug #174375: Distribution drivers permissions may need redesign <distributions> <lp-registry> <permissions> <ubuntu-platform> <Launchpad itself:Triaged> <ubuntu-community:Confirmed for techboard> < https://launchpad.net/bugs/174375 >
<wgrant> D:
<bigjools> *weep*
<wgrant> Yes.
<wgrant> I have quite deliberately avoided trying to analyse that :)
<LPCIBot> Yippie, build fixed!
<LPCIBot> Project devel build #613: FIXED in 5 hr 41 min: https://lpci.wedontsleep.org/job/devel/613/
<henninge> deryck: I'll be relocating quickly now. I may be 5 minutes late for the call
<deryck> henninge: np
 * danilos -> away
<deryck> Just FYI for everyone, I've asked abentley *not* to be on call reviewer today.
<deryck> We have some work to get done on Orange Squad and I need him today.
<deryck> I won't list myself as on call reviewer, but if anyone needs a review feel free to ping me.
<jml> deryck: thanks for the heads up
<deryck> np.  resending in email, too.
<bigjools> sinzui: how's the allergy meds going? :)
<sinzui> I am stoked to have survived the night after several choking bouts
<sinzui> I think I need to move north or to a desert for a month
<bigjools> sinzui: are you feeling up to talking about distro series permissions?
<sinzui> sure
<abentley> sinzui: you can come here, but it may not be north enough.
<bigjools> sinzui: I want to let distro drivers add new series.  The pollen in the nose however is the Ubuntu team
<sinzui> yes. That is a very large team
 * bigjools recalls the epic warthogs email 4 years ago
<abentley> deryck: chat?
<deryck> abentley: sure.  mumble?
<sinzui> bigjools: I believe IDerivativeDistro already supports this behaviour
<abentley> deryck: yes.
<bigjools> sinzui: ah... that old chestnut
<bigjools> sinzui: addseries needs lp.append on IDerivativeDistribution
<sinzui> I believe lp.append checks for admin, owner, or driver
<bigjools> ok
<bigjools> how does it decide that something is IDerivativeDistribution?
<sinzui> there was something about lp.driver that scared me when I tried it
<sinzui> I wish I could remember what the problem was
<bigjools> ah, I have found the hack
<bigjools>         if self.name == 'ubuntu':
<bigjools>             alsoProvides(self, IBaseDistribution)
<sinzui> bigjools: the same kind of hack we have in Person to be an ITeam
<sinzui> I thought that hack was using full_functionality with checks the celebrity
<sinzui> ^ bigjools
<bigjools> in this case I might not need to worry about this
<bigjools> sinzui: yes, it does I think
<sinzui> bigjools: Do you still intended to introduce IRemixDistribution so that projects like balitx do not get builds and translations
<bigjools> sinzui: so everything is an IDerivativeDistribution unless it's Ubuntu?
<sinzui> bigjools: That is it
<bigjools> sinzui: I wasn't planning on changing that at all
<bigjools> soyuz is quite happy
<bigjools> well, once you add some series and architectures
<bigjools> sinzui: See https://dogfood.launchpad.net/deribuntu/dangerous
<sinzui> bigjools, ah understood
<sinzui> bigjools: you may have an awesome blog post to do. You may be celebrated by  the 101 Ubuntu derivatives.
<bigjools> sinzui: well, possibly.  This stuff is being done for Linaro and OEM
<bigjools> the derivative management will only be available if you get explicitly allowed to use it
<abentley> deryck: http://people.canonical.com/~abentley/translations-usage.png
<sinzui> So a driver of baltix could create a series that is based on natty. The series will be added with everything that automatically happens with a new series, but features exposed in the derivative UI will not be available?
<bigjools> sinzui: right - there's a separate initialisation phase that copies the packages from the parent
<bigjools> sinzui: BTW I just made myself a driver on a non-ubuntu distro and the addseries link disappeared
<bigjools> but it worked if I was a maintainer
<sinzui> oh.
<sinzui> dear
<sinzui> I bet the link needs to be guarded by append. I recall the link was added back to the page in haste and admin/owner may have been used
<bigjools> actually I should say, the link disappeared when took myself out of the maintainer positionb
<bigjools> +addseries gives a 403 as well
<sinzui> :(
<bigjools> this is odd then, I own it and I am a driver.
<bigjools> but I can't edit it
<sinzui> bigjools: did rvba change permission after changes distro owner rules. Distro owner cannot edit  their distro because for Ubuntu, that was hundred of people. Only admins could change the distro
<sinzui> bigjools: I am sure we had IDerivativeDistros permissions working.
 * sinzui looks for tests
<sinzui> bigjools: lp/registry/browser/tests/distroseries-views.txt shows drivers creating series
<bigjools> hmmm
<bigjools> sinzui: https://dogfood.launchpad.net/deribuntu
<bigjools> check out the driver
<bigjools> I own that team
<bigjools> I cannot get to +addseries
<sinzui> bigjools: the test is wacky. I see check_permission('launchpad.Driver', view)
<bigjools> sinzui: ah!
<sinzui> We expect launchpad.Append
<jcsackett> danilos: do you have time to review https://code.launchpad.net/~jcsackett/launchpad/hardwaredb-should-ignore-bad-data/+merge/56472
<jcsackett> a big part of the diff is just a new sample data file being added and some tests being moved.
<sinzui> bigjools: ModerateDistributionByDriversOrOwnersOrAdmins is now lp.Moderate
<bigjools> sinzui: one sec, OTP
<al-maisan> sinzui: OMG .. that name was almost Java-level verbose ;)
<sinzui> bigjools: so drivers, owners, and a admins can create series because they have lp.Moderate, but the views and links check lp.Append. We broke this last August when we tried to remove exceptional permissions. We may need to revert this if we find that we cannot fix the lp.Append guards on the link and view
<sinzui> al-maisan: security.py likes to live dangerously close to the 78 character length limit
<al-maisan> so I see
<danilos> jcsackett, sure thing, I can handle one more review today :)
<jcsackett> danilos: awesome. thanks. :-)
* danilos changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: - | https://code.launchpad.net/launchpad-project/+activereviews
<danilos> jcsackett, is it important for test data to be this big? can you not cause the same problem with a much smaller input file?
<jcsackett> danilos: possibly; i cribbed that file from the "good" sample data file. almost all elements in the rng appear to be required. :-/
<danilos> jcsackett, ok, fair enough
<jml> ducking out to get coffee
<danilos> jcsackett, I won't be looking at the file in detail then, but I suppose at least long comment sections like the one containing XXX can be removed :)
<jcsackett> danilos: sure; i'll do what i can to clean it up.
<jcsackett> danilos: currently, it's literally the "good" file with a single bad dmi section.
<danilos> jcsackett, that'd be much clearer if the file was potentially constructed on-the-fly: copy the good one, change one entry
<jcsackett> danilos: that's certainly a thought. i'll confess to laziness. i'll see about doing that.
<danilos> jcsackett, iow, I don't like that there's a huge file without it being clear that there's one tiny little bit inside that _is_ important
<jcsackett> danilos: dig. i can certainly see the objection.
<danilos> jcsackett, if you just feel too lazy, perhaps just add a big comment at the top indicating more of what we discussed here (though, I certainly prefer a better solution :))
<danilos> anyway, off to the code now
<sinzui> bigjools: This is my summary of the permission issue https://pastebin.canonical.com/45750/
<jcsackett> danilos: in order, i'll try: modifying the data on the fly; cleaning everything possible out of the file; adding a big comment.
<danilos> jcsackett, excellent, thanks a bunch :)
<danilos> jcsackett, btw, is _logError used anywhere where you'd still want it to record an OOPS?
<jcsackett> danilos: it's used in several places. that's why by default it still creates the oops. to my knowledge, right now the one call site that's been modified to pass create_oops=False is the only place we want to supress it, but we may discover more.
<danilos> jcsackett, ok, sounds good then, with a minimal test case for the context manager (so nobody breaks it in the future), r=me
<jcsackett> danilos: dig. i'll add that.
<danilos> jcsackett, thanks! it seems a useful feature, worth emailing the list about I'd say :)
<jcsackett> danilos: cool. i'll do that once it lands. :-)
<jml> back
<bigjools> sinzui: right sorry, off TP now
<bigjools> sinzui: I'm not sure the analysis is entirely right since when I visit addseries I get a permission error, so it's not just a UI issue.  Or is this because I am tripping up on the view?
<bigjools> ooo a patch, I like those :)
<sinzui> bigjools: no one has launchpad.Append, the permission was removed for distros...but Lp lets you still use it for links and views :(
<bigjools> sinzui: argh :/
<bigjools> I'll hack mawson and see what happens
<sinzui> I see maps.ubuntu.com arrived 9 months too late
<bac> hi sinzui, i'm trying to isolate a windmill test failure following the instructions at https://dev.launchpad.net/Windmill
<bac> sinzui: trying to run a single test with 'bin/test --test=test_bug_commenting' finds 0 tests.
<bac> has it changed?
<sinzui> bac: I do not know  you can run a single js test. the setup I wrote created a single test suite for all the files in the directory
<bac> sinzui: the above is the example from the wiki.  perhaps it never worked?
<sinzui> bac: deryck[lunch] did the last round of revisions.
<timrc> Either there is some latency, http://maps.ubuntu.com is broken, or my IP does not translate correctly but I'm not seeing an Ubuntu badge over Austin, TX
<sinzui> bac: the magic has to be in the test harness class. it may be that some harnesses were updated
<bac> sinzui: ok.  thought i'd try you since you're here.  i'll wait for him to return from lunch
<bac> sinzui: i'm reading through the thread from yesterday now to see if there are clues
<sinzui> timrc: i see a single badge over Austin in  http://maps.ubuntu.com/map/
<timrc> sinzui: hm not Iâ¦ but it may be someone else (as I just broadcast this :))
<timrc> sinzui: other ip geolocating say they can't figure out where I am :/
<bac> sinzui: why did you say the maps are 9 months late?
<sinzui> bac: I mispoke. I was hoping Ubuntu would get a OSM map implementation. That is was I proposed. Instead it got a Google implementation
<bigjools> sinzui: your fix worked
<bac> sinzui: ah
<sinzui> bigjools: \o/
<bigjools> indeed!
<timrc> they should accept postal codes as well ;)
<bigjools> maps.u.c hopelessly gets my location wrong too
<sinzui> bac: I see that test_bug_commenting is a real windmill test
<bac> sinzui: yes
<sinzui> bac: the windmill layer is disabled at the the moment, or at leas when I checked it yesterday
<sinzui> bac: you need to specify both the layer and the test to convince the testrunner to run it
<bac> sinzui: really?  i got ec2 failures for windmill tests last night
<bac> oh, i see
<sinzui> bac: bin/test has a windmill and mailman exclusion rule
<sinzui> bac: i tried to remove the mailman exclusion rule yesterday and failed. I saw windmill also had the same rule
<bac> thanks sinzui, this works:   bin/test --layer=BugsWindmillLayer --test=test_bug_commenting
 * bac updates wiki
<sinzui> bac: you can just use --layer=Windmill
<sinzui> layer matching is pretty generous
<bac> even better
 * sinzui loves multi-touch love handles
<jcsackett> sinzui: can i ask you to take a look at https://code.launchpad.net/~jcsackett/launchpad/set-question-message-visibility/+merge/56588? it's the retarget of the set-question-visibility work to devel instead of db-devel so i can get it out and working.
<sinzui> jcsackett: I just looked at it
<sinzui> I have nothing to say. I can just approve it now
<jcsackett> sinzui: that works for me. :-)
<jcsackett> thanks.
<deryck> well that took forever
<deryck> sinzui, bac -- did you guys solve your testing questions I see in the scrollback?
<sinzui> deryck: yes
<bac> deryck: yes
<deryck> awesome
<bac> the wiki was out-of-date...but no more
<sinzui> deryck: ./bin/test excludes the windmill and mailman layer by default. They must be explicitly passed as an arg
<bac> deryck: i do have another problem.  test_duplicate_finder is failing for me in my branch (where i did make a mod) but also in trunk
<deryck> sinzui: this should be fixed now with wallyworld_'s recent landing
<deryck> bac: and you're up to date on trunk?  With the changes wallyworld_ landed?
<bac> yep
<bac> deryck: it fails waiting forElement on the dupe search field
<deryck> bac: you on Firefox 4?
<bac> deryck: nope, 3.6.16
<deryck> hmmm, ok.
<bac> are we to avoid upgrading to 4?
<deryck> bac: for Windmill yes.  wallyworld_ and I are still trying to work out something going wrong there.  See his mail.
<bac> ok
<deryck> bac: as for the test, I can't run it yet.  Still futzing with my natty upgrade.
<bac> deryck: no good deed...
<deryck> bac: you could paste me the error, though.  I can look at that and see if anything stands out.
<bac> deryck those windmill tests are passing now.  go figure.  i'll take my chances with ec2.  time for a super-quick review?
<deryck> bac: sure, hit me!
<bac> deryck: i'm still waiting on the MP to show up
<deryck> bac: no worries.
<cody-somerville> Why is the text area for further information on the filebug page so small? :(
<benji> so users of Chrome or Firefox 4 feel special when they resize it?
<bac> deryck: https://code.launchpad.net/~bac/launchpad/bug-751397/+merge/56617
<deryck> bac: thanks.  got it and looking at it now.
<bac> thx
<bac> deryck: i'm going to step out for a bit
<deryck> bac: no worries.  I've got TL call top of the hour, so if I don't finish by then, I'll catch the rest after.
<jml> hey, can anyone spot a bug in this script? http://paste.ubuntu.com/590375/
<jml> manual inspection says 20 critical bugs were closed in the last week, the script says 17
<benji> jml: this modification http://pastebin.ubuntu.com/590387/ gives me this output http://pastebin.ubuntu.com/590388/
<jml> benji: interesting
<benji> I'm trying to figure out what the deal is with the date_closed on those three.
<jml> benji: why add 'not t.date_closed'?
<benji> because the comparison fails otherwise (I'm in hack mode at the moment so don't hold me to that)
<jml> benji: ok thanks. I'll explore some more :)
<benji> also note that the script will take quite a while longer to run this way
<jml> yeah.
<benji> jml: the three bugs that have date_closed == None are all status:Invalid
<jml> benji: are they old, too?
<benji> jml: you could say that: 2005 and 2006
<jml> benji: date_closed data isn't consistent in old bugs.
<jml> benji: thanks for helping, very much appreciated.
 * jml is off for the evening.
<benji> glad to help
<jcsackett> sinzui: can you take over #launchpad?
<jcsackett> also: mumble, or is our two-man standup later enough?
<deryck> bac: r=me on the js cleanup + addition tal condition branch.
<bac> thanks deryck
<bac> hey deryck, what woes are you having with natty in vm?
<bac> you running vmware?
<bac> i haven't tried it yet...
<deryck> bac: no, parallels and parallels tools won't build against natty kernel.  So worse than 2d Unity experience.  Really crappy desktop experience without a suitable video driver.
<lifeless> flacoste: https://lpstats.canonical.com/graphs/PPR/20100407/20110407/ is looking nice now
<flacoste> lifeless: indeed
<sinzui> sorry jcsackett. I have lost what little sense I had because of allergies and drugs
<lifeless> sinzui: get well soon
<sinzui> jcsackett: lets do our talk at 5:00 since it will only be U and I about at that time
<jcsackett> sinzui: sounds good. and i hope your allergy season ends soon. i remember having terrible allergies when i lived up your way.
<sinzui> lifeless: I think the plants should get a room if they want to get it on
<lifeless> sinzui: amen
<lifeless> jkakar: http://www.rfc-editor.org/rfc/rfc6202.txt may interest you
<lifeless> flacoste: hi
<lifeless> flacoste: we have /huge/ numbers of official bug tags for lp
<flacoste> lifeless: we do
<lifeless> flacoste: I'd like to zerg through and zap a tonne; any objection in principle? if not I'll assemble a list and mail -dev for specific objections
<flacoste> lifeless: be my guest!
<lifeless> thanks
<lifeless> flacoste: also, have you seen how fast https://bugs.launchpad.net/ubuntu is? woot
<flacoste> indeed :-)
<lifeless> flacoste: https://dev.launchpad.net/PolicyAndProcess/PreReleaseQAProcess looks stale too
<flacoste> lifeless: yes
<lifeless> flacoste: or perhaps poorly named
<lifeless> flacoste: carte blanche ?
<flacoste> lifeless: sure, i'll watch the edits and review
<jkakar> lifeless: Ooh, thanks.  I've added it to my "to-read" list. :)
<lifeless> yay deleting old wiki pages
<lifeless> flacoste: I'm done on the doc sweep; mailed you about some pages I couldn't decide on
<lifeless> sinzui: hi
<sinzui> hi lifeless
<lifeless> sinzui: heya
<lifeless> sinzui: I wanted to touch base and confirm my understanding that jcsackett will be continuing with a button to hide spam in the UI ?
<sinzui> lifeless: One of us will be doing that.
<lifeless> sinzui: \o/
<poolie> hi all
<sinzui> He is still exposing some of api goodness needed to support it :(
<lifeless> sinzui: I am thrilled to confirm that; for some reason I had started to doubt my memory
<lifeless> hey poolie
#launchpad-dev 2011-04-07
<poolie> lifeless, for my education, is there a second category of oopses for things like 404s
<poolie> like "probably not our fault"
<poolie> ?
<lifeless> I think at the moment that we do some second-pass filtering in the oops web UI
<lifeless> Thats wasted overhead though
<lifeless> OOPS are fairly expensive to create
<lifeless> so I plan to remove that
<poolie> btw hearty +1 to stopping launchpad-bugs backscatter
<poolie> and i'm sure jam & co would agree too
<lifeless> the rule of thumb for oopses is:
<lifeless>  - if its logged we should do something about it
<lifeless>  - (corollary) if its something we cannot do anything about, we should not log it
<lifeless> sinzui: thoughts on https://bugs.launchpad.net/launchpad/+bug/744888/comments/4 ?
<_mup_> Bug #744888: Muting bug 1 times out <bad-commit-12754> <story-better-bug-notification> <timeout> <Launchpad itself:Triaged> < https://launchpad.net/bugs/744888 >
<sinzui> lifeless: I thought Snapshot was taught to always ignore collection fields
<lifeless> sinzui: Thus I'm asking you :)
<sinzui> ah, but bugs and questions want to snapshot for the last message I think. Maybe it was not done
<lifeless> a collection snapshot could just record the tip message (for some apporpriate order)
<lifeless> and the delta is 'messages newer than X'
<lifeless> [e.g. see my batch navigator work]
<poolie> lifeless, i wonder if there needs to be a category like in bzr where we log a traceback but do not show it to the user
<poolie> ie we don't think it's a problem, but if it turns out that they think it's a problem, they still have the data
<lifeless> poolie: we have that [we use it for nearly-timedout-renders]
<lifeless> poolie: but I don't think an OOPS is appropriate for the case we're discussing
<lifeless> oopses gather a lot of data to permit diagnostics and correction
<poolie> sure
<lifeless> this case is pretty clear - the nonce /has/ been used. Whatever went wrong was in the request before the one having trouble
<poolie> i agree with avoiding the waste and being clear about what's actually a problem
<poolie> thumper, thanks for fixing bug 377519
<_mup_> Bug #377519: Stacked on location breaks if the stacked upon branch is renamed <branch-stacking> <lp-code> <qa-needstesting> <Launchpad itself:Fix Committed by thumper> < https://launchpad.net/bugs/377519 >
<poolie> if that's going to be rolled out gradually, i would like to add a note on the bug saying that people won't immediately be able to verify it as fixed
<poolie> or, they won't immediately see the problem go away
<poolie> is that correct/ok?
<poolie> anyhow, it was a one off
<thumper> poolie: it isn't fully fixed
<poolie> i was just curious
<thumper> poolie: I forgot to put the --incr flag on the landing
<thumper> yes feel free to add a comment
<lifeless> poolie: it won't be fixed fixed until we do something for existing branches
<lifeless> I think communicating expectations to our users is a great idea
<poolie> :-)
<lifeless> I'd edit the summary at the top to describe whats going on
<poolie> i really liked how you did that the other day
<poolie> i forget which bug it was
<lifeless> one reason to do this is that bug search only searches the subject and summary, not the comments.
<lifeless> this was a performance thing resulting from the way the fti was assembled way back; I haven't had time to review and examine how to enable searching in commenets
<lifeless> without messing up performance
<wgrant> wallyworld_: BTW, Unity distracted me from saying this on the call, but I have no problem with Windmill being back.
<wgrant> wallyworld_: I want it to be back on by default, but it's simply too expensive if it fails.
<wallyworld_> wgrant: np. i agree
<wgrant> wallyworld_: So I will not hesitate to disable it upon the first spurious failure in buildbot.
<lifeless> sinzui: ok so we faded out there
<lifeless> sinzui: is my suggestion a) broken, b) sensible, c) we need to modify how snapshotting deals with collections ?
<lifeless> sinzui: or d) none of the above
<sinzui> lifeless: sorry. I very distracted. IAre we certain that is needed? was it cargo culted from old code?
<sinzui> as for bug and questions looking at messages, I think that is lazy design. I believe there are two emitters of the modifications and they know if a message was appended
<lifeless> sinzui: bug one has several thousand messages
<lifeless> sinzui: I'm certain we either need to not snapshot it, or snapshot it much more efficiently
<lifeless> sinzui: *and* I'm certain that what is snapshotted internally shouldn't match the API attributes for this object either, as the API has different needs vs in-appserver deltas.
<sinzui> So the publisher emitting the event should be clear what message was added instead of letting the subscriber guess. I do not think snapshot should be working with deap data
<sinzui> deep
<huwshimi> wallyworld_: From about line 450: http://bazaar.launchpad.net/~huwshimi/launchpad/private-objects-298152/view/head:/lib/lp/bugs/javascript/bugtask_index.js
<lifeless> sinzui: ok, cool.
<lifeless> so we doNotSnapshot it, see what breaks, fix the signal to contain the new message if needed
<sinzui> yep
<lifeless> wgrant: speaking of bugs.. 744849
<wgrant> Bug #744849
<_mup_> Bug #744849: populate-archive.py unusably slow <Launchpad itself:Triaged> < https://launchpad.net/bugs/744849 >
<wgrant> Yeah.
<lifeless> man, that was the easiest critical fix ever.
<wgrant> Oh?
<lifeless> search for -de in a person picker
<huwshimi> Um, what the failure? http://paste.ubuntu.com/590501/
<wgrant> huwshimi: Aha, that has shown up on Jenkins before.
<wgrant> huwshimi: Spurious, but I'd never seen it elsewhere. Thanks.
<lifeless> huh
<lifeless> temp_dir - replace that with fixtures.TempDir
<lifeless> more robust implementation
<huwshimi> Should I just lp-land it?
<wgrant> huwshimi: If that was the only failure.
<huwshimi> wgrant: It was, assuming the rest of the tests ran ok
<wgrant> huwshimi: How many tests did it report?
<huwshimi> wgrant: 13988
<wgrant> huwshimi: That sounds right.
<wgrant> Land it.
<huwshimi> wgrant: Thanks maate
<huwshimi> *-a
<thumper> hmm
<thumper> is this thing on?
<lifeless> irc is
<_thumper_> :(
<_thumper_> quassel still having problems with the sqlite file
<bac> lifeless: i got bit by the temp_dir issue too.  unfortunately the version of fixtures we're on atm does not have TempDir.  you see any problem in this change until we can update fixtures and use it?  http://pastebin.ubuntu.com/590522/
<lifeless> bac: bin/py -m pydoc fixtures shows TempDir as being available
<lifeless> bac: so I think its usable now
<lifeless> bac: why do you say it isn't ?
<bac> er, did that just happen?  let me update
<lifeless> bac: no, TempDir is ancient
<bac> lifeless: b/c i looked in the egg and didn't see it
<lifeless> bac: :!grep TempDir -r eggs/fixtures-0.3.5-py2.6.egg/
<lifeless> eggs/fixtures-0.3.5-py2.6.egg/EGG-INFO/PKG-INFO:        TempDir
<lifeless> ...
<bac> lifeless: argh, bitten by emacs file completion...
<lifeless> bac: I don't mind if you fix temp_dir, but deleting code seems worth a little extra time
<lifeless> it does need spelling very slightly differnetly (see the pydoc for TempDir)
<StevenK> wgrant: Should I disable the windmill job on Jenkins?
<wgrant> StevenK: Probably. But don't delete it... I suspect we might need it back soon.
<StevenK> Hah
<StevenK> wgrant: Oh, sure, I was just going to disable it.
<StevenK> wgrant: Your hate of Windmill is showing. :-)
<wgrant> I don't hate it.
<wgrant> I just don't trust that it isn't go to break again.
<wgrant> And breakage is really really bad.
<StevenK> Loathing?
<wgrant> So, the tests that we currently implement through Windmill are necessary.
<wgrant> They catch real breakage.
<wgrant> But they are slow and unreliable.
<StevenK> RARGH, the package cloner is creating builds in the *parent*
<StevenK> Bad package cloner!
<wgrant> Heh.
<lifeless> wgrant: I'm really glad you hold that position now; its (I think) much more useful than the fire n brimstone one you had a few months back.
<wgrant> lifeless: I have held this position for several months.
<StevenK> Oh, duh. The first createMissingBuilds call is *me*
<wgrant> Perhaps the breakage had greater emphasis back then, but the breakage was also itself greater.
<lifeless> wgrant: Well, 6 months back then :) - anyhow, I like the current attitude; I didn't mean to imply you had a particularly bad one in the past.
<jtv> wgrant: how did publish-ftpmaster work out on dogfood yesterday?
<lifeless> wgrant: sorry, bad news..
<lifeless> wgrant: that bug still freaks out in prod
<thumper> lifeless: is there a relatively simple way to see if a recent oops contains a particular string?
<lifeless> yup, grep
<lifeless> they are all in the lpnet logs dir on asuksa
<lifeless> a wildcard match will let you pass the directories you need to grep
<lifeless> grep -i string */2011-04-04 -r
<lifeless> etc
<lifeless> bbiab, shopping run for lynne
<thumper> are they linked on devpad anywhere?
<lifeless> oh
<lifeless> I meant devpad
<lifeless> asuka is staging
<lifeless> <- 6am starts bad
<wgrant> lifeless: Bah. Worked for me at least once :/
<wgrant> jtv: I filed a few bugs.
<jtv> wgrant: ohâ¦ anything serious?
<wgrant> jtv: I enabled the run-parts stuff.
<wgrant> jtv: Only one significant non-run-parts bug, IIRC.
<jtv> wgrant: do you have a list somewhere?  I take it these are all things that need resolving before we can use the new script.
<wgrant> http://bugs.launchpad.net/launchpad/+bugs?field.bug_reporter=wgrant&orderby=-datecreated
<wgrant> I think #752279 is a somewhat bigger issue.
<_mup_> Bug #752279: publish-distro.d scripts expect partner to have dists.new <derivation> <soyuz-publish> <Launchpad itself:Triaged> < https://launchpad.net/bugs/752279 >
<wgrant> It's not clear that the rsync is working.
<jtv> wow that's taking long to load
<wgrant> lifeless: :/ it just render in 8.1s for me on prod.
<wgrant> +ed
<wgrant> Waiting for the OOPS to sync :(
<wgrant> jtv: All of those bugs except 752279 have workarounds in the DF tree.
<wgrant> For 752279 there might be a debug print left in the script, since the logging doesn't seem to be working.
<wgrant> I carefully published partner as well as lucid's non-release pockets yesterday, so we have an archive of reasonable size to work with.
<wgrant> Just needs a bit of SQL to dirty them.
<jtv> wgrant: partner is supposed to do the same dists.new dance as the primary archive.  If that's not working, I'd look for a path problem first.
<wgrant> jtv: Yeah, I realised that after I filed the bug.
<wgrant> jtv: The script currently crashes in a location that cannot be determined because the atexit crashes.
<jtv> There is no atexit.
<wgrant> Hm.
<jtv> Unless it's in the LaunchpadScript's lock-cleanup code or something, but that shouldn't matter.
<wgrant> If you give me a moment to retake my desktop from the dead grasp of Unity, I will get you the traceback.
<wgrant> jtv: It's in the finally.
<wgrant> http://librarian.dogfood.launchpad.net/57722542/7gpgHGXJT50D5VFRhqsoFWlJEla.txt
<wgrant> 13:35:47 < jtv> Unless it's in the LaunchpadScript's lock-cleanup code or something, but that shouldn't matter.
<wgrant> 13:36:09 < wgrant> If you give me a moment to retake my desktop from the dead grasp of Unity, I will get you the traceback.
<wgrant> 13:37:34 < wgrant> jtv: It's in the finally.
<wgrant> 13:37:37 < wgrant> http://librarian.dogfood.launchpad.net/57722542/7gpgHGXJT50D5VFRhqsoFWlJEla.txt
<jtv> wgrant: thanks
<jtv> wgrant: I was sort of expecting problems there.
<wgrant> Yeah.
<jtv> Though I don't really see what "directory not empty" means in this case.
<wgrant> jtv: I suspect it's renaming a directory on top of an existing one.
<wgrant> mv works fine there; it just moves *into* it.
<wgrant> rename won't do that.
<jtv> That's exactly the problem I was half-expecting.  :)
<jtv> Basically, it uncovers a bug in the old script AFAICS
<wgrant> Yeah.
<wgrant> It won't fail.
<wgrant> But it will do something completely wrong.
<jtv> Question is, is it okay to delete the directory that's in the way?
<wgrant> Mumble.
<jtv> Coffee.
<wgrant> Please forgive me for pressing Ctrl+Alt, oh Unity.
<wgrant> No, it will have none of that.
<wgrant> lifeless: Uhhh
<wgrant> lifeless: Your OOPS.
<wgrant> SQL time: 1381 ms
<wgrant> Non-sql time: 11600 ms
<jtv> wgrant: I'm on mumble but don't see you.
<wgrant> jtv: Oh, that was me mumbling to indicate that I'm not sure.
<wgrant> Not saying that we should talk about it... although maybe we should.
<wgrant> lifeless: wampee, too. This looks fairly unideal.
<lifeless> wgrant: you think contention ?
<thumper> I wish we had the actual oops id in the filename instead of some other random shit
<lifeless> thumper: next iteration will have a search engine
<thumper> having to open the file to find the oops id is a PITA
<lifeless> thumper: and no filenames
<wgrant> lifeless: I don't know, but this is a pretty serious issue that warrants immediate investigation.
<wgrant> I forget K's threading config, but it sounds big.
<lifeless> wgrant: check lp-prod-configs
<wgrant> I am.
<lifeless> wgrant: 4
<lifeless> lpnet11
<wgrant> Yeah.
<wgrant> Is that 2 concurrent requests in hap?
<lifeless> 6
<lifeless> or perhaps 4
<lifeless> check the html dump mthaddon gave me earlier
<wgrant> Yes, but it had spaces in the name.
<wgrant> And I can't be bothered typing all those backslashes.
<lifeless> \\\
<lifeless> have mine
<wallyworld_> thumper: what app did you use to get your recent screen capture?
<thumper> record my desktop
<wallyworld_> thanks
<thumper> or something like that
<wgrant> jtv: :(
<wgrant> Huh.
<thumper> \o/ another unicode bug bites the dust
<wgrant> :)
 * StevenK wishes Storm supported INSERT INTO
<lifeless> StevenK: add it
<lifeless> I can give you some tips if you like
<lifeless> ok, trying for the shops again... bbiab
<StevenK> Writing SQL via format string is beginning to wear thin
<StevenK> And trying to add another column to copy sucks
<wgrant> StevenK: How're you doing it?
<wgrant> The way I did it in PublishingSet.publishBinaries isn't *that* unmaintainable.
<wgrant> But it's certainly still not pretty.
<StevenK> wgrant: I don't think I need it.
<StevenK> The test case is making my primary archives with requires_virtualized true
<StevenK> Which is probably wrong
<StevenK> wgrant: Do you agree?
<wgrant> StevenK: It's valid either way.
<wgrant> I don't know what your problem is, though.
<wgrant> Virt primary archives work fine most places I've tried, though.
<StevenK> wgrant: This is for IDS, and IDS doesn't copy DAS.supports_virtualized
<wgrant> StevenK: Ahh.
<wgrant> That's a bit awkward.
<StevenK> So if the archive is requires_virtualized, IDS can't create builds
<wgrant> But, er, why is IDS creating builds?
<wgrant> I guess for failed ones from the previous series.
<StevenK> Right
<wgrant> If IDS has a "rebuild copied sources" option, I will cry.
<StevenK> ... it does
<wgrant> What the fuck?
<wgrant> How is that going to work?
<wgrant> Somebody didn't think this through :P
<StevenK> Read the code? It works
<wgrant> I guess if you create the archive disabled and add external deps with a pre-bootstrapped archive you might get somewhere.
<wgrant> But, er, creating an empty archive with lots of builds isn't going to get you very far at all :)
<StevenK> wgrant: I seem to recall Julian was against the idea of copying supports_virtualized, but I can't recall the reason
<wgrant> StevenK: We don't normally want to support PPAs for a while after the series is initialised.
<StevenK> wgrant: The idea is for "Scorched Earth rebuilds" with different compiler flags and such, but yes, it will need magic to happen
<StevenK> Exactly
<StevenK> So I think I'll change the test harness to make sure Archive.requires_virtualized == False
<wgrant> Evil magic with sources.list.d is lamont's plan, I believe.
<thumper> https://code.launchpad.net/~thumper/launchpad/fix-unicode-path-oops/+merge/56688
<thumper> for those that love unicode
<lifeless> jtv: is today a loliday too ?
<jtv> lifeless: not that I know, too
<jtv> I mean, no
<jtv> why?
<lifeless> thanks
<lifeless> I have a call scheduled with stub on thursdays
<lifeless> if hes on leave I can naff off
<lifeless> and you are lower latency than canonicaladmin
<jtv> It's always good for a human to hear that their unique usefulness lies in a slight latency advantage over computers for simple menial tasks.
<jtv> Keeps us in our place.
<huwshimi> lifeless: There are some unofficial tags that need a cleanup/removal as well. For example 'post-3-ui-cleanup' should probably just be converted to UI now. Is there any reason I shouldn't just fix that one myself now?
<lifeless> oh Facepalm: we have a cache of messages on bug, and we don't use it. --fail--
<lifeless> huwshimi: JFDI
<huwshimi> lifeless: Sure :)
<lifeless> jtv: us humans are uppity creatures :)
<jtv> yesâ¦ can't wait for skynet to take over
<jtv> wgrant: I'm fixing the $ARCHIVEROOTS escaping problem.  I think the right way to do it would be a double-quoted, escaped string containing double-quoted, escaped strings: "\"directory1\" \"directory2-with-weird-\$\" \"directory\`3\`\""  It may affect quoting in the plugin scripts, though I think it'll be cleaner overall.
<huwshimi> lifeless: Is bug #438540 made invalid due to the substring search change made recently? Or was that relating to something else?
<_mup_> Bug #438540: assigned to search doesn't do partial matches <lp-bugs> <post-3-ui-cleanup> <ui> <Launchpad itself:Triaged> < https://launchpad.net/bugs/438540 >
<lifeless> looking
<wgrant> jtv: Shell escaping makes me even more sad than JS escaping in classic HTML.
<jtv> And that, I hope, is very sad indeed.
<jtv> The thing about JavaScript escaping is the false promise:
<lifeless> huwshimi: no, different search - thats the person picker
<jtv> you think you can escape JavaScript, but that's not what it means.
<lifeless> huwshimi: 'person *to assign to*
<lifeless> huwshimi: refresh the bug, I've clarified it
<huwshimi> lifeless: Ah right. Yeah I didn't read the description properly
<huwshimi> lifeless: Is our AJAX timeout set to 30 seconds?
<wgrant> huwshimi: Server-side it's the same as everything else.
<wgrant> 11s.
<lifeless> huwshimi: if you see a longer duration one of the following is true:
<lifeless>  - hit max concurrent http connections in your browser
<lifeless>  - poor network e.g. ssl reneg, lost packets, whatever
<lifeless>  - the request stopped doing all db activity before 11 seconds and just played with itself for the rest of the time
<lifeless> huwshimi: the first one is fairly common, I think.
<lifeless> [not an exhaustive list, but roughly in terms of frequency/probability I think]
<huwshimi> Interesting. Whenever I get an AJAX failure it always dies at 30 seconds. I wonder if that's being set by YUI? If so we should change that to match our 11 second timeout, right?
<lifeless> no
<lifeless> not unless we can guarantee:
<lifeless>  - never have more than N (unknown) concurrent ajax requests to the same host (even from other browser windows)
<lifeless>  - guarantee network conditions won't go quirky
<lifeless> huwshimi: 30 seconds is the default network SYN attempt timeout IIRC
<lifeless> huwshimi: so I think you have issues, perhaps the same as smpillaz, with your network path to LP
<lifeless> huwshimi: do you find bzr push very slow sometimes (> 15 seconds to get started)
<huwshimi> lifeless: I haven't noticed that
<lifeless> ok
<lifeless> well, we need to find and fix this
<lifeless> but I don't think dropping an in browser timeout is sensible given the legitimate reasons it might queue in your browser.
<huwshimi> lifeless: If I have Launchpad open in a bunch of tabs is that where I might hit the max concurrent connections?
<lifeless> yes
<lifeless> or even on one tab
<lifeless> bugtask:+index seems to do 8 separate ajax requests
<lifeless> which will probably exceed your concurrency limit by some-N (I haven't checked chromium, but it could be as much as 6)
* lifeless changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: - | https://code.launchpad.net/launchpad-project/+activereviews | qastaging down for a schema test, back soon
<huwshimi> lifeless: So what I was doing was opening a bunch of bugs in separate tabs and then editing each one. I was timing about about 40% of the time (rough guess). What I just tried was opening one bug in a new tab and editing it, closing the tab and starting again. I had 3 tabs open max. The first two were fine, the third timed out.
<jtv> Error: Couldn't find a distribution for 'zope.publisher==3.12.0'.
<jtv> Is that caused by the aggressive cleanups people have been doing?  It's on a fresh db-devel branch, after rocketfuel-get/rocketfuel-setup/bzr pull/make clean
<huwshimi> wgrant: Can you explain to me a bit more about bug #433342? I'm not sure where you're talking about.
<_mup_> Bug #433342: 1em extra space on top of every page <lp-web> <post-3-ui-cleanup> <Launchpad itself:Triaged> < https://launchpad.net/bugs/433342 >
<wgrant> jtv: I see a zope.publisher-3.12.0.zip in download-cache.
<jtv> I don't.  Only zope.publisher-3.12.1.tar.gz.
<StevenK> wgrant: Do you know anything about hello-partner in maverick's NEW queue on mawson?
<wgrant> StevenK: No, I only published partner carefully, I didn't upload anything.
<wgrant> I blame jtv and bigjools.
<StevenK> wgrant: It's 16 weeks old!
<wgrant> Oh.
<wgrant> That could be me, then.
 * jtv has an alibi
<wgrant> I did do something around then.
<wgrant> huwshimi: It seems to be fixed now.
<huwshimi> wgrant: Sweet.
<StevenK> Changed-By: William Grant <william.grant@canonical.com>
 * StevenK glares at wgrant 
<wgrant> StevenK: Yeah, I didn't realise it was still there, so I thought it was more recent.
<StevenK> Description: hello-partner - The classic greeting -- proprietary edition
<wgrant> 16 weeks old... really?
<StevenK> Does it say "Hello, 640K is enough for anybody!" ?
<wgrant> What's the date on it?
<StevenK> Date: Sat, 18 Dec 2010 16:55:52 +1100
<StevenK> That's from the .changes
<wgrant> Ah, so the week I started.
 * StevenK rejects it
<wgrant> https://code.launchpad.net/~lamont/launchpad/lp-buildd-78/+merge/56700 needs a review.
<wgrant> Already deployed on the machines that matter.
<wgrant> It redefines revolting, but it works!
<StevenK> That is *hideous*
<wgrant> Yse.
<StevenK> wgrant: DOne
<wgrant> Thankyou. I believe IS now owes us new eyes.
<spm> IS or lamont personally?
<wgrant> Either will do.
<spm> and what's so bad about that? looks downright elegant to me. nicely formatted over a few lines and everything!
<spm> damn. my nose just grew about 5 inches
<jtv> spm: I'm sure Little Red Riding Hood's "grandmother" had a good one to explain that away, but I forget what it was.
<spm> heh
<wgrant> WindMIIIIIIIIIIIL
<wgrant> L
<StevenK> wgrant: It just failed?
 * wgrant disabled that test.
<wgrant> s/disabled/disables/
<wgrant> StevenK: lucid_db_lp
<StevenK> GAAAAAAAAAAAAVIN!
 * StevenK found a bug in +initseries
<wgrant> Oh?
<StevenK> If you don't select any packagesets it feeds packagesets=[""] to DistributionJob
<wgrant> Bah, qas is down.
<wgrant> Bad lifeless.
<lifeless> wgrant: bad schema 2004
<wgrant> I hope PQM will accept a devel testfix for a db-devel failure.
<lifeless> yes
<lifeless> stub: hi
<stub> lifeless: yo
<lifeless> we're slated for a call I think
<lifeless> you coffeed up etc etc?
<stub> Yup. Coffeed up. Hungry but breakfast can wait :)
<lifeless> k, let me plug mic in
<adeuring> good morning
<jtv> hi adeuring
<adeuring> hi jtv!
<bigjools> morning
<StevenK> O hai
<jtv> hi bigjools
<bigjools> the decision to wear shorts this morning feels like a bad one right now
<StevenK> Haha
<jtv> I'm not going to ask.
<henninge> Morning all ;)
<henninge> danilos: Hi!
<mrevell> Good morning
<danilos> hi Henning, how's it going?
<henninge> danilos: great, thanks ;)
<henninge> danilos: This one is especially for you: ;-)
<henninge> https://code.launchpad.net/~henninge/launchpad/devel-714521-restore-partial-exports/+merge/56713
<henninge> danilos: but I can ask allenap if you don't want to ...
<danilos> henninge, heh, yaaay :)
<danilos> henninge, it's actually quite a simple branch, r=me :)
<danilos> henninge, do take care of the lint issues if they are real, though :)
<henninge> danilos: they are not
<henninge> or at least I know there is a but for them
<henninge> danilos: thank you
<danilos> henninge, heh, then fix the linter, come on :)))
<jam> lifeless: One page that I'm noticing the performance of: "https://code.launchpad.net/~jameinel": "80 queries/external actions issued in 9.83 seconds"
<jam> I'm noticing it because I'm deleting old junk branches, and they all redirect to the main page.
<lifeless> jam: yup
<jam> do you know which api that is? (Just so I can keep tabs on it, probably can't do much myself just yet)
<jam> just hit 10s this time
<jam> so I'm very close to hard-timeout on it.
<lifeless> jam: bug 746866
<_mup_> Bug #746866: Person:+branches timeout: sometimes-slow bug-branch link query <timeout> <Launchpad itself:Triaged> < https://launchpad.net/bugs/746866 >
<jam> lifeless: so I should link fewer of my branches to bugs then ? :)
<lifeless> no
<lifeless> its fixable
<lifeless> most of these are
<lifeless> just noting that the problem is known
<jam> lifeless: hence the :). But yeah, I'll follow that bug.
<lifeless> there is a 10x improvement in that bug waiting for coding and execution
<jam> lifeless: from your comments, that was just an SQL query change, not a new index, etc?
<lifeless> yup
<lifeless> (reading my own comments here ... could be wrong :)
<jam> lifeless: it looks like the original query finds all the teams you are part of, and then all of the bugs linked to those teams, ouch!
<lifeless> jam:  4 /    4  Person:+branches - so 4 hard and 4 soft timeouts yetserday
<jam> lifeless: sounds pretty low on the stack. and only really affecting heavy users
<lifeless> jam: not quite; it finds all the branches you have access to via private team subscriptions, but only the bugs linked to your branches
<jam> lifeless: http://paste.ubuntu.com/590646/
<jam> that sure looks like give me all bugs connected to teams I'm in
<lifeless> it is
<lifeless> but
<jam> I realize the Bug.id can be optimized by postgres
<lifeless> note that its in And()
<jam> and inject it
<lifeless> AND (Bug.private = FALSE OR ...)
<lifeless> jam: oh right, uhm so that bit isn't usually an issue
<jam> lifeless: sure, but it means the planner has to be really smart, vs your rewrite
<lifeless> jam: right
 * thumper closed three critical bugs today with no code \\o/
<thumper> they have all been fixed in the interim
<thumper> just some work investigating to make sure
<lifeless> nice
<jam> is there a clean way to delete a branch procedurally? I'm looking to do perf testing where I push a new branch many times, and I'd rather just delete all of these right after they are created.
<jam> Right now, I just go via the web api, and URL hack +delete and submit a lot
<lifeless> there is a web api to delete branches
<lifeless> spivs new stuff will be in the next downtime deploy
<jam> lifeless: https://launchpad.net/+apidoc/1.0.html#branch only shows me "canBeDeleted" I don't see an actual Delete call
<lifeless> jam: lp_delete()
<lifeless> jam: file a bug on the apidoc not mentioning that DELETE method is supported on objects
<jam> bug #753334
<_mup_> Bug #753334: apidoc doesn't mention DELETE methods <Launchpad itself:New> < https://launchpad.net/bugs/753334 >
<jam> lifeless: just got a hard OOPS for code.lp.net/~jameinel
<lifeless> stub: whats the next db patch #?
<lifeless> we really should fold https://bugs.launchpad.net/launchpad-project/+patches into +activereviews
<wgrant> Let's not open up that can of worms again.
<lifeless> wgrant: when was it ever open ?
<wgrant> There have long been thoughts about merging them by turning patches into branches.
<lifeless> different discussion
<lifeless> a small incremental step would be to merge the reports with patches being a separate section
<wgrant> lifeless: Still around?
<lifeless> hmm
<wgrant> lifeless: Do you know how to tell lazr.restful to rename a field in just 1.0 and beta?
<lifeless> I imagine export it twice
<wgrant> You can do something like this:
<wgrant>         ("1.0", dict(exported_as="datebuilt")),
<wgrant> But that renames it in 1.0 and later, not 1.0 and earlier.
<jtv> wgrant, care to look at the quoting fix?  It was a bitch to get it just a little right, so I didn't go for fool-proof.  https://code.launchpad.net/~jtv/launchpad/db-bug-752178/+merge/56719
<wgrant> jtv: This also fixes the 10-sign-releases quoting bug.
<wgrant> You should link that as well.
<jtv> cool
<stub> lifeless: 2208-60-0
<lifeless> wgrant: so, you export renamed in beta and then renamed in devel
<lifeless> stub: thanks
<wgrant> lifeless: Yeah, but I think that's a bit backwards.
<wgrant> lifeless: The field is really called date_finished, but in beta and 1.0 it should be called datebuilt.
<jtv> wgrant: that's the GNUPGHOME one?  Bug 752179?  Then I'm not sure it does.
<_mup_> Bug #752179: PublishFTPMaster runs hooks with a bad GNUPGHOME <derivation> <soyuz-publish> <Launchpad itself:Triaged> < https://launchpad.net/bugs/752179 >
<wgrant> I think that's the common case.
<wgrant> jtv: Not that one.
<wgrant> Hm.
<wgrant> Maybe I didn't file a bug for that one.
<wgrant> But there was a mismatched quote in it, which you've deleted.
<jtv> wgrant: ah that oneâno, didn't see an explicit bug for that one.
<wgrant> It was all a bit of hackery to get it to run, so I probably forgot it.
<jtv> Well no matter now!
<wgrant> Indeed.
<wgrant> '"$PATH":%s
<wgrant> Should $PATH really be quoted?
<jtv> Ideally, yes.  What if there's a space in there?
<jtv> It's bad form, but not impossible.
<wgrant> True.
<jtv> I might have "~/Ubuntu One/bin" in my path, for instance.
<jtv> What derived series do we have set up on dogfood at the moment?
<wgrant> But I don't think quotes are necessary to preserve that...
<wgrant> jtv: deribuntu should have one, and maverick is derived from sid.
<jtv> Great.
<wgrant> jtv: Indeed, testing locally shows those quotes are not required.
<wgrant> Unless you can show otherwise.
<jtv> Hmmâ¦ not required when you do PATH=$PATH:/foo but required when you do env PATH=$PATH:/foo
<wgrant> Ah, of course.
<wgrant> But if you're executing in a shell, do you really need the env?
<wgrant> And if you're using subprocess, you can modify the env in the subprocess call.
<jtv> wgrant: not sure tbhâ¦ not using subprocess though.
<jtv> wgrant: according to `man sh`, "it is only field splitting or pathname expansion that can create multiple fields from a single word."  Question now is, when exactly does field splitting happen?  It does say "If a parameter expansion occurs inside double-quotes: [â¦" Field splitting is not performed on the results of the expansion, [â¦]."
<jtv> It also says "Field Splitting is performed on fields generated by [parameter expansion etc.] unless the IFS variable is null.
<jtv> "
<lifeless> stub: 10x performance win from cloning owner into bugmessage
<jtv> Codd, are you listening?
<lifeless> stub: hot case, and cold is estimated at 50% shorter
<stub> Sounds like a win. I'm not sure if other comment logs would benefit like questions - do they have the same usecase?
<lifeless> stub: https://code.launchpad.net/~lifeless/launchpad/bug-421901/+merge/56725
<lifeless> stub: possibly, I'm sure oops reports will tell us ;)
<lifeless> stub: but questions are so fundamentally broken right now, I am inclined to tackle them separately
<stub> lifeless: wrong target branch
<stub> lifeless: Do we want to keep the column name 'owner'? Its always been a rather bogus name.
<lifeless> bwah
<lifeless> stub: I think it would be mega confusing to mirror it under a different name
<deryck> Morning, folks.
<stub> lifeless: We can also ignore the triggers if we want - that column never gets modified.
<lifeless> stub: https://code.launchpad.net/~lifeless/launchpad/bug-421901/+merge/56729
<lifeless> stub: person delete/merge will update the column
<stub> It will? Oh yeah, it will.
<lifeless> stub: plus I figure we'll have a few of these so having 'correct' triggers we can point at to copy will be usefl
<stub> Too many and we need to work out how to best split trusted.sql :)
<lifeless> yeah
<lifeless> trusted.sql.d
<lifeless> stub: any reason those functions can't be in the db patches ?
<stub> We use DB patches because we can't back out schema modifications. I like to keep the stuff we can back out outside of the db patches so we can make use of revision control.
<stub> No need to worry about getting patch numbers if you just want to change stuff in security.cfg, trusted.sql or comments.sql
<lifeless> kk
<lifeless> stub: was just thinking about the consequence of that though
<lifeless> which is that we have to run it all every time
<stub> Yes, but it is fast.
<lifeless> ok
<stub> The weirdness is we can't remove things from there sometimes due to how we build development databases (any function mentioned in the baseline dump needs to exist in trusted.sql or things fall over)
<lifeless> stub: https://code.launchpad.net/~lifeless/launchpad/bug-421901/+merge/56729 - against db-devel
<lifeless> stub: I'll do a garbo migrator tomorrow in a separate patch
<stub> lifeless: r=stub
<lifeless> thanks
<lifeless> night
<jtv> wgrant: afaict the dash and bash manuals contradict each other on the $PATH issue we discussed earlier.  Both seem to behave like the bash manual says, but I'm reluctant to leave out the quotes based on that.
<wgrant> jtv: ... yay :/
<jtv> â¦indeed
<soren> jtv: Sorry, what part of dash's docs suggest that you need additional quoting?
<soren> I don't see it.
<jtv> soren: they say that field splitting occurs after parameter substitution, except when it happens in double quotes.
 * jtv looks up chapter & verse
<soren> How does that relate to this?
<jtv> Well isn't field splitting what would break if there was a space in the substituted value?
<jtv> FOO=bar splat ls
<jtv> From FOO=$GARGL ls
<soren> Not exatly.
<jtv> Ah!
 * soren tries to whip up an example
<soren> Ok, now I have no idea what's going on anymore.
<jtv> soren: glad we're in full agreement then :)
<jtv> Mind you, `FOO=$GARGLE ls` behaves differently from `env FOO=$GARGLE ls`
<jtv> (in this regard)
<soren> When GARGLE is what?
<soren> Something with a space in it?
<jtv> Yes.
<jtv> For instance,
<jtv> UU="~/Ubuntu One"
<jtv> PATH=$PATH:$UU/bin which my_personal_script
<jtv> â finds ~/Ubuntu One/bin/my_personal_script
<jtv> But
<jtv> UU="~/Ubuntu One"
<jtv> env PATH=$PATH:$UU/bin which my_personal_script
<soren> Yeah, that won't work.
<jtv> And I'm trying to build some kind of systematic notion of why that is, based on the manual.
<soren> Ok.
<jtv> Failing horribly, so far.
<soren> So, when you do this:
<soren> FOO=bar cmd
<soren> You're using a functionality in the shell to extend the environment of the cmd process with FOO=bar.
<soren> When you're using env, the shell doesn't know that what you're doing is essentially the same.
<jtv> But where does it say that these two expansions are different?
<soren> ...so it happily ends up calling this argv: ['env', 'PATH=~/Ubuntu', 'One/:<the rest of your path>', 'cmd']
<jtv> Yes, that part we know.  But where in the dash manual can I find where that difference comes from?
<soren> env doesn't recognise "One/blahblah" as more stuff it needs to feed into the environment, but thinks it's the command to be run.
<soren> Ah.
<soren> I still don't really see how this is related, though.
<soren> PATH is separated by colons.
<jtv> But may contain spaces.
<soren> Always.
<soren> Yes, that's fine.
<jtv> Spaces that I don't want to get expanded.
<soren> SUre.
<jtv> I think I found something:
<jtv> Leading VAR=val words are "stripped off and assigned to the environment," as opposed to "expanded as described in the section called Expansions" as the rest of the command is.
<soren> Yes.
<soren> That's it.
<soren> Where do you see that?
<jtv> But then where does it say that parameters in those VAR=val words are expanded at all?
<jtv> That's in man dash (1), under Simple Commands.
<soren> Yeah, just found it.
<jtv> I guess the difference is that the expansion of those parameters happens one parameter at a time.
<jtv> And so somewhere in there it must say that no field splitting is done for those values.
<soren> It would appear so, yes. There's no harm done in doing something this, though: PATH="~/Ubuntu One/bin:$PATH" foo
<jtv> Right.
<jtv> Unless there's quotes in $PATH, perhaps.
<jtv> This is all so maddening.
<soren> Quotes in $PATH should survive that.
<soren> Is this pertaining to an existing patch or is it still work-in-progress?
<jtv> It's pending review.
<jtv> Or not, if I messed it up horribly.  :)
<soren> link?
<jtv> https://code.launchpad.net/~jtv/launchpad/db-bug-752178/+merge/56719
<jtv> I probably don't need the "env" invocations.
<soren> jtv: I think what you're doing (in terms of escaping and protecting against escaping) is sound. That said, it looks like your executeShell helper might benefit from being able to take an env dict so that you avoid these issues altogether.
<jtv> I considered that, but I'm not sure if that will have any unexpected effects on pipes and such.
<soren> I think your current approach will hold more surprises in that regard.
<soren> For instance, you're only extending the environment for the first command of your pipeline.
<soren> (If you just naÃ¯vely prepend the "env FOO=bar" stuff that is)
<jtv> That's what I meanâI think my current approach exposes that more clearly.
<jtv> Not too conveniently, mind you, but the interaction is visible where you create it.
<henninge> Hi deryck! ;)
<deryck> Hi henninge
<henninge> deryck: Do you have some time to chat about that javascript bug?
<deryck> henninge: sure.  give me 2 minutes to warm coffee and I'll meet you in mumble
<henninge> deryck: cool
<henninge> I mean 'hot'
<henninge> ;)
<henninge> bug 724727
<_mup_> Bug #724727: Single-line inline editor causes page to shift when editing <easy> <javascript> <ui> <Launchpad itself:Triaged> < https://launchpad.net/bugs/724727 >
<henninge> deryck: ^
* jcsackett changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: jcsackett | https://code.launchpad.net/launchpad-project/+activereviews | qastaging down for a schema test, back soon
<bigjools> ummm is ajax bug subscription broken?
<wgrant> bigjools: I think it it may be for malone-alpha people.
<bigjools> ossum
<bigjools> wgrant: fyi https://bugs.launchpad.net/ubuntu-font-family/+bug/744812
<_mup_> Bug #744812: FontConfig/Qt stack choke on Ubuntu Medium font meta-data (No medium in Inkscape and too bold in Qt apps) <kubuntu> <Ubuntu Font Family:Confirmed> <ubuntu-font-family-sources (Ubuntu):New> < https://launchpad.net/bugs/744812 >
<wgrant> There's a bug for that.
<wgrant> Bug 753387
<_mup_> Bug #753387: Can't subscribe to bug: Uncaught TypeError: Cannot call method 'get' of null <Launchpad itself:Triaged> < https://launchpad.net/bugs/753387 >
<wgrant> Have we really gained 10000 bugs in a week? :/
<wgrant> Well, a bit over a week.
<bigjools> release approacheth
 * bigjools apologises for blocking pqm with a config branch
<danilos> jcsackett, hi, do you have time to review https://code.launchpad.net/~danilo/launchpad/add-subscription-link-tests/+merge/56765?
<jcsackett> danilos: sure.
<bigjools> is anyone looking at the PQM db-devel conflict?
<bigjools> rvba, did you look earlier?
<rvba> bigjools: no
<danilos> jcsackett, thanks
<jcsackett> danilos: i see conflicts.
<jcsackett> is there a missing rev to be pushed up?
<danilos> jcsackett, no, but they must be very recent since I merged a few hours ago, let me resolve them
 * bigjools fixes it
<jcsackett> danilos: dig.
<danilos> jcsackett, resolved, now pushed as well, diff should be updated in a few minutes
<jcsackett> danilos: cool. i will reload and review in a few minutes then. :-)
<bigjools> rvba: this was missed in your review I think, but don't forget to put classes in the __all__
<rvba> bigjools: ok
<bigjools> rvba: is the DistroSeriesMissingPackagesView in stable newer than the DistroSeriesMissingPackages in db-devel?
<rvba> bigjools: DistroSeriesMissingPackages has been renamed  DistroSeriesMissingPackagesView
<bigjools> (that's the one missing from __all__ BTW)
<bigjools> ok thanks
<danilos> jcsackett, now should be fine :)
<jcsackett> danilos: yup, i'm reviewing now.
<jcsackett> danilos: do yui tests continue if they an assert fails, or does the test stop?
<danilos> jcsackett, they stop, and I see your point :) I should make reverting stuff happen inside the tearDown or something of the like
<danilos> jcsackett, (if that was the point you were getting to :)
<jcsackett> danilos: :-)
<jcsackett> it was indeed.
<jcsackett> danilos: other than that, r=me.
<danilos> jcsackett, thanks
<danilos> jcsackett, fwiw, other test cases do not even bother restoring LP.* stuff, perhaps I shouldn't either
<jcsackett> danilos: yeah, i'm not sure it's a huge problem, actually. it's possible the monkey patching is only within test scope.
 * jcsackett does not know enough about javascript testing.
<danilos> jcsackett, yeah, I am in the same boat, so I wanted to (well, provided all tests succeed :), be on the safe side
<jcsackett> deryck: you around, oh javascript and YUI test guru?
<deryck> jcsackett, doing a review.  Can I have 10 minutes and then chat?
<jcsackett> deryck: sure, i can wait ten. just have a question when you're available about some testing stuff.
<jcsackett> danilos: i'm going to mark the reveiw as approved, and follow up with deryck for my own edification. i think you're probably fine. :-)
<deryck> jcsackett, if chat, ask away here.  if voice, I'll ping when ready.
<danilos> jcsackett, yeah, thanks
 * danilos is also reading up on http://developer.yahoo.com/yui/3/test/
<jcsackett> deryck: chat. i was just wondering if when you patch a function for testing purposes (like replacing it with a function that just asserts it was called) is that only within test scope, or does it continue to be patched after the test suite finishes?
<jcsackett> we have many yui tests that do like namespace._some_function = somenewfunction and then don't revert it.
<jcsackett> (well, maybe not many, but they exist. and it's pertinent to danilos branch that i'm looking at).
<deryck> jcsackett, can you post review branch for me to see an example?
<jcsackett> https://code.launchpad.net/~danilo/launchpad/add-subscription-link-tests/+merge/56765
<jcsackett> deryck ^
<jcsackett> actaully, let me get you a smaller paste so you don't have to read through all of that.
<jcsackett> deryck: http://paste.ubuntu.com/590785/
<jcsackett> in that example, module._show_add_overlay gets altered to assert that it's called and gets the right data. there's code to revert it in this instance, but some tests don't do anything. i was wondering if the code to revert it was necessary.
<danilos> jcsackett, oh, btw, that one is probably not the best example since I can just move the "reversion" to the top of the call so asserts won't stop it even
<danilos> (though, that's a different topic)
<deryck> jcsackett, yes, this is the right way to do it.  you can't be sure you have the right function later without doing this.
<jcsackett> danilos: fair. it was an example i had handy. :-)
<jcsackett> deryck: dig. so instances where one doesn't see this is a bug.
<deryck> right
<jcsackett> thanks, deryck!
<jcsackett> so, danilos, i would just move the revert back to the top, as you said, and then all is well. :-)
<jcsackett> er, to the top, not back to the top.
<danilos> jcsackett, heh, done
<danilos> jcsackett, btw, thanks for the review! :)
<jcsackett> danilos: you're welcome. :-)
<rvba> jcsackett: Hi! I'd appreciate a review of https://code.launchpad.net/~rvb/launchpad/dds-missing-base-version-747202/+merge/56311
<jcsackett> rvba: looking now.
<jcsackett> rvba: i don't think you need "python: not(view.show_package_diffs_request_link) and context.base_version" in your template, because you updated "view.show_package_diffs_request_link" to include "context.base_version" as part of the check.
<jcsackett> i think "not view/show_package_diffs_request_link" should be sufficient.
<rvba> jcsackett: that's right
<jcsackett> in fact, right now i don't think the current condition can ever be true.
<jcsackett> because if context.base_version is true, "not(view.show...)" will be false.
<jcsackett> prehaps i am misunderstanding somethign.
<rvba> no, I think you're right ... I guess a test is missing then
<rvba> :w
<rvba> (oups, sorry)
<jcsackett> no worries. i do that all the time. :-P
<rvba> jcsackett: I'll add another test for that
<jcsackett> rvba: yeah, double check how that condition works, and put a test in place. i'm marking it as needs info for now, since the boolean logic there needs some investigation. ping me when the fix is up and i'll take another look. :-)
<rvba> jcsackett: ok, thanks for the review
<jcsackett> rvba: you're welcome. i look forwards to seeing the branch with those changes. :-)
<henninge> deryck: it's not a bug: http://paste.ubuntu.com/590807/
<deryck> henninge, ah, right, ok, then.  Thanks!
<rvba> jcsackett: actually ... after checking it I think the test was right.
<rvba> python: not(view.show_package_diffs_request_link) and context.base_version => can be true
<deryck> abentley, one down.  r=me with minor comment on style.
<rvba> formally because not(A and B)=Not A or not B
<rvba> jcsackett: but I still have to add a test ;-)
<abentley> deryck: thanks.  I'll look into that.
<abentley> deryck: not surprising my brace style was inconsistent.  I've never used this style of braces 'till now.
<benji> jcsackett: hi, I have a fresh MP for you: https://code.edge.launchpad.net/~benji/launchpad/add-edit-tests/+merge/56786
<deryck> abentley, no worries.
<jml> edge!
<jcsackett> it keeps popping up.
<benji> heh
<jcsackett> benji: i see many conflicts.
<benji> jcsackett: darn, let me take a look
<rvba> jcsackett: just pushed my changes (added a test, moved up a small template fragment to be more consistent)
<jcsackett> rvba: okay. i will take a look in a few moments. :-)
<rvba> jcsackett: great.
<deryck> abentley, second one is r=me.
<deryck> abentley, we spent so much time on these in discussion earlier, the reviews are easy. :-)
<deryck> I'm just taking my time to hopefully not miss anything.
<abentley> deryck: heh.
<abentley> deryck: Well, if you see better ways to do things, please let me know.
<deryck> abentley, certainly, I will.
<abentley> deryck: I'll be interested to know if I got the inheritance right for FormErrorHandler.
<deryck> abentley, in the final branch, right?
<abentley> deryck: right.
<deryck> abentley, ok, I'll look closely at that
<jcsackett> rvba: r=me.
<rvba> jcsackett: thanks a lot.
<jcsackett> rvba: i did just see actually that there's a text conflict in the MP again. you'll need to fix that before landing, of course. :-)
<rvba> jcsackett: sure. will do.
<bigjools> ummm I just somehow managed to make an input box in firefox when editing a wiki page think that the text is right-justified.  How would I fix that?
<maxb> bigjools: Ctrl-Shift-X
<bigjools> maxb: how obvious...!  Thanks.
<deryck> abentley, all done.  r=me.
 * deryck wipes brow from reviewing three js branches
<deryck> abentley, the inheritance looks good for FormErrorHandler, too.
<abentley> deryck: cool.  Creating an instance of ErrorHandler in order to create a subclass of it just seems weird.
<deryck> abentley, yeah, it is a bit weird actually.  but that's prototyped based objects for you. :-)
<deryck> ok, need a break now.  lunch!
<timrc> lifeless: sorry if my activity on the wiki the other day raised an eyebrow :) we're trying to clearly state our requirements moving forward w/ archive management in OEM and the Derivative Distributions work Launchpad is committed to do is something we're very interested in eventually using
<timrc> we don't have good documentation on what our requirements and use cases are, so that's what we're trying to codify (maybe the wrong word)
<bigjools> timrc: use cases would be awesome :)
<timrc> bigjools: aye :)
<bigjools> good night all
<mrevell> Good night all.
<lifeless> good morning
<lifeless> is anyone on the db-devel conflict?
<lifeless> timrc: not at all, I thought it was great to see it captured
<lifeless> anyone here witha windows machine ?
<abentley> lifeless: sure.
<lifeless> abentley: could you qa huw's patch to let IE make comments on bugs? Its on qastaging
<lifeless> https://bugs.launchpad.net/launchpad/+bug/414747
<_mup_> Bug #414747: No comment submit button on IE <bug-page> <ie> <javascript> <lp-bugs> <qa-needstesting> <ui> <Launchpad itself:Fix Committed by huwshimi> < https://launchpad.net/bugs/414747 >
<lifeless> abentley: if you have ie 7 or 8, that is
<abentley> lifeless: Okay, I'll see what I can do.
<lifeless> abentley: thank you!
<abentley> lifeless: I'm getting lots of timeout errors.
<jcsackett> ah, the joys of not realizing you're disconnected...
<abentley> lifeless: qastaging is not usable due to timeouts.
<lifeless> abentley: oh :( which bug ?
<abentley> lifeless: bug 414747
<_mup_> Bug #414747: No comment submit button on IE <bug-page> <ie> <javascript> <lp-bugs> <qa-needstesting> <ui> <Launchpad itself:Fix Committed by huwshimi> < https://launchpad.net/bugs/414747 >
<lifeless> abentley: I meant which bug were you testing with on qastaging :)
<abentley> lifeless: I don't understand.
<lifeless> abentley: what url on qastaging timed out
<lifeless> abentley: https://bugs.qastaging.launchpad.net/launchpad/+bug/414747 I just added a comment there successfully
<_mup_> Bug #414747: No comment submit button on IE <bug-page> <ie> <javascript> <lp-bugs> <qa-needstesting> <ui> <Launchpad itself:Fix Committed by huwshimi> < https://launchpad.net/bugs/414747 >
<benji> jcsackett: it took a while but I have a revamp of that ealier MP for you to look at if you have time: https://code.edge.launchpad.net/~benji/launchpad/add-edit-tests-2/+merge/56832
<abentley> lifeless: Oh, I thought it applied to code review comments, so I was testing with https://code.qastaging.launchpad.net/~james-w/launchpad/more-matchers/+merge/32057
<jcsackett> benji: happy to take a look at it now. :-)
<benji> cool
<lifeless> abentley: ah! it may, but it was filed against bug pages specifically
<lifeless> jcsackett: hi
<abentley> lifeless: worked.
<abentley> lifeless: on bug 240067
<_mup_> Bug #240067: Launchpad projects need wikis <feature> <lp-foundations> <ubuntu-platform> <Launchpad itself:Triaged> < https://launchpad.net/bugs/240067 >
<lifeless> abentley: cool; what version of IE did you use? {so I can put it in the bug }
<abentley> IE 8.0.6001.18702
<lifeless> thanks, appreciate this
<abentley> lifeless: you're welcome.
<jcsackett> lifeless: hello.
<lifeless> jcsackett: hiya
<jcsackett> hello again, lifeless. :-)
<lifeless> jcsackett: wanted to reassure you I'm not going to mess with your spam stuff
<lifeless> jcsackett: though we now have clear data that centralisation in the schema is a performance pessimisation
<jcsackett> lifeless: which is a shame.
<lifeless> jcsackett: well, schema != model
<jcsackett> true.
<jcsackett> and i only really care about code fragmentation in the model.
<lifeless> if we say, for instance 'a message belongs to one context only'
<lifeless> then we can serialise a few different ways with no /requirement/ for code duplication in python
<jcsackett> i think that sounds just fine. :-)
<lifeless> we may need to hack on or around storm, but thats a one time code
<lifeless> s/code/cost/
<lifeless> jcsackett: separately, I've commented on the end of Bug:45419
<jcsackett> as i said, i think there's ways to keep what i've done and do what you're suggesting (and anything that helps the bug pages is a win). i just wanted to peep up and make sure that was a consideration.
<lifeless> jcsackett: you need to unlink that bug, so future landings won't trigger qa-needs for it
<jcsackett> i saw your comment.
<lifeless> jcsackett: cool
<lifeless> jcsackett: related to that the db-stable qa report also had qa needed by you for your landing on db-stable
<lifeless> jcsackett: I presume its the same patch?
<jcsackett> lifeless: yeah, just retargeted to devel.
<lifeless> jcsackett: (its now showing qa-untestable because I marked up the bug [which is shared] as untestable per your comment)
<flacoste> lifeless: i'm getting 503 trying access launchpad.net
<lifeless> jcsackett: something you might like to do [I do this] is to just look at the two reports daily and search for jcs on the page.
<lifeless> flacoste: the home page?
<flacoste> yep
<flacoste> got disconnect with the mumble server also
<flacoste> might me connectivity to the data centre from here
<jcsackett> the part that was landed is actually qa-able, i just wasn't able to qa it because of machine problems until about 15 min ago. everything worked nicely.
<jcsackett> which wasn't really a surprise, since it worked on the first non-deployed landing as well.
<lifeless> blog.launchpad.net works (matters for the front page only of lp.net)
<lifeless> flacoste: reprorudced
<lifeless> its just the home page
<jcsackett> benji: r=me.
<benji> jcsackett: thanks
<lifeless> gary_poster: I have an unreasonable hunch that https://bugs.launchpad.net/launchpad/+bug/754058 is due to subscription js changes
<_mup_> Bug #754058: post bug filing notification is cleared by an ajax request a few seconds after page load <regression> <Launchpad itself:Triaged> < https://launchpad.net/bugs/754058 >
<gary_poster> lifeless well :-P then :-)
<lifeless> gary_poster: or due to the notification-via-ajax stuff wallyworld_ did
<gary_poster> I don't have enough context yet
<gary_poster> lemme read
<lifeless> gary_poster: but I don't know which is more likely; I mention to you now so you know
<lifeless> gary_poster: I'm also going to mention to wallyworld_.
<lifeless> gary_poster: thanks!
<thumper> :(
<thumper> neither bigjools nor deryck are around
<lifeless> thumper: you need them ?
<thumper> lifeless: I was wanting to talk to them, yes
<gary_poster> lifeless: "post-filing note" means you file something and then you add a comment immediately after?
<lifeless> gary_poster: no, the blue notification
<lifeless> gary_poster: we have this thing where a project can set 'instructions to show people after they file a bug'
<lifeless> it gets shown as a page notification
<gary_poster> oh
<lifeless> actually the more I think about this
<lifeless> the more I think its wallyworld_'s thing having an unexpected interaction
<lifeless> I think his code wipes the notification area rather than combining things, or something like that
<gary_poster> lifeless, yeah, I duped and see what you mean now, and lots of interactiony things are possible, but we haven't touched that code AFAIK, or things near it
<gary_poster> so, IOW, I'll leave it be unless directed otherwise, if that's alright
<lifeless> gary_poster: totally fine
<gary_poster> cool.  thanks for heads up
<mwhudson> argh, why isn't lib/lp/services/command_spawner.py written using twisted?
<lifeless> I'll talk to wallyworld_ today; if its not his stuff, will let you know monday
<mwhudson> twisted is good at this sort of thing :)
<lifeless> lies!
<lifeless> gary_poster: is there anything I can do for you at the moment?
<gary_poster> lifeless, thinking...your help on the bug 1 timeout was much appreciated btw
<_mup_> Bug #1: Microsoft has a majority market share <iso-testing> <ubuntu> <Clubdistro:Confirmed> <Computer Science Ubuntu:Invalid by compscibuntu-bugs> <EasyPeasy Overview:Invalid by ramvi> <GNOME Screensaver:Won't Fix> <Ichthux:Invalid by raphink> <JAK LINUX:Invalid> <The Linux OS Project:In Progress> <metacity:In Progress> <OpenOffice:In Progress by lh-maviya> <Tabuntu:Invalid by tinarussell> <Tivion:Invalid by shakaran> <Tv-Player:New> <Ubuntu:I
<lifeless> gary_poster: was happy to help
<gary_poster> lifeless, I think we are OK in terms of high-level things.  Once I dig into #753000 more and determine if we need a new DB constraint I may look your or stub's way.  I can't think of any larger-size mysteries to point you at though.  Thank you for asking.
<_mup_> Bug #753000: NotOneError caused by duplicate stuctural subscriptions <merge-deactivate> <oops> <story-better-bug-notification> <Launchpad itself:Triaged> < https://launchpad.net/bugs/753000 >
<lifeless> gary_poster: kk
<thumper> abentley: ping
<abentley> thumper: pong
<thumper> abentley: I was wanting to talk to you about some bzrlib stuff, but bigjools just got back to me, so I'll follow up in email, it is nothing urgent, just wikkid
<abentley> thumper: cool.
* jcsackett changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: - | https://code.launchpad.net/launchpad-projet/+activereviews | qastaging down for a schema test, back soon
<lifeless> flacoste: we should start reporting on regressions now
<lifeless> we've been tagging them for a month now, I think
<lifeless> https://bugs.launchpad.net/launchpad/+bugs?field.tag=regression
<flacoste> lifeless: what do you have in mind?
<lifeless> flacoste: a stat in the TL call, and a graph
<flacoste> lifeless: we already have a graph
<lifeless> \o/
<lifeless> flacoste: then I'm interested in a little regular discussion about it
<flacoste> lifeless: as part of the critical bugs burndown, regressions have their own category
<flacoste> same thing in the critical bugs fixed chart
<lifeless> do we have the ration (regression count in period / bugs filed in period)
<flacoste> lifeless: you mean, new regression filed / new bugs filed, no we don't have that
<flacoste> lifeless: i can work on that next week
<lifeless> if you think its useful
<flacoste> i do
<lifeless> hmm
<lifeless> I wonder
<lifeless> regressions per landing would be interesting too
<flacoste> i actually think that a new bugs filed in last 24h with some details on categories would be helpful
<lifeless> or landings per regression
<lifeless> flacoste: a portlet like that rock
<wallyworld_> lifeless: yes, that's my issue. i'll fix it
<lifeless> wallyworld_: if you want to talk around the interactions, I'm available anytime
<wallyworld_> lifeless: np. i know what's causing it.
<lifeless> cool
<lifeless> [whats causing it?]
<LPCIBot> Project devel build #618: FAILURE in 5 hr 55 min: https://lpci.wedontsleep.org/job/devel/618/
<wallyworld_> lifeless: https://code.launchpad.net/~wallyworld/launchpad/improper-notification-removal/+merge/56855
<wallyworld_> lifeless: the cause was that it was expected that each new patch request would need to clear existing notifications
<wallyworld_> so that old ones would not be left hanging around
<wallyworld_> soory, i didn;t see your earlier question - was working on the fix
<lifeless> wallyworld_: no worries
<wgrant> Bah.
<wgrant> Anybody looking at the conflict from 6 hours ago?
<thumper> wgrant: I was just about to start
<thumper> but if you want to , it's all yours :)
#launchpad-dev 2011-04-08
<wgrant> I was hoping to go and eat something first.
<wgrant> So feel free.
<thumper> ok
<thumper> I'll take it
<thumper> wgrant: standup?
<thumper> wgrant: or do you want to do it after food?
<wgrant> thumper: Well, there's no Curtis, but I guess we could do it anyway.
<thumper> wallyworld_: mumble?
<wallyworld_> thumper: just untangling my cords
<wallyworld_> thumper: https://bugs.launchpad.net/launchpad/+bug/698138
<_mup_> Bug #698138: Assigning a non-contributor to a bugtask should show a warning <regression> <Launchpad itself:In Progress by wallyworld> < https://launchpad.net/bugs/698138 >
<thumper> I'm busy upgrading the laptop in a non-x session as compiz / unity / whatever has become unresponsive
<thumper> all praise to quassel for being awesome
<rockstar> gmb, are you around-ish?
<lifeless> can has review? https://code.launchpad.net/~lifeless/launchpad/bug-421901/+merge/56861
<mwhudson> lifeless: the change to pgsql is just refactoring right?
<lifeless> yes
<lifeless> reuse ftw
<mwhudson> just checking my eyes
<mwhudson> :)
<lifeless> :)
<mwhudson> lifeless: this test will fail if there's more than 5000 bugmessages in sampledata!!
<lifeless> mwhudson: oh hoes
<lifeless> *noes* heh
<lifeless> if someone puts 5K messages in sampledata, I'll ask mark for a company credit card to pay for my flight to their house...
<mwhudson> yeah
<lifeless> mwhudson: though actually its not 5K it would fail at
<lifeless> thats the *chunk* size
<mwhudson> and actually even then, it might not fail, i admit i don't know the details of tunableloop
<mwhudson> lifeless: right
<lifeless> the default window it gets is something like 20 minutes, or more on a many-core machine
<lifeless> its about 500 bm updates/second
<mwhudson> lifeless: there is a limit there, but it's comically high i agree
<lifeless> 60*20*500 -> something big
<lifeless> 0.0M
<lifeless> bah
<lifeless> 0.6M
<mwhudson> i guess i'm a little surprised that the launchpad dbuser has the right to disable triggers
<lifeless> it doesn't
<lifeless> connecting with None as the user
<lifeless> tells postgresql to use the unix uid
<mwhudson> aah ok
<lifeless> which on all out test environments has such access
<lifeless> I head-desked on this for about an hour digging around to get it right
<lifeless> its *already* commented in the place I found, so i didn't copy the comment.
<lifeless> but I did name the method 'root_connection'
<lifeless> to hint at this :)
<lifeless> I can add more prose, though I'm not sure it will save anyone time
<mwhudson> oh right
<mwhudson> lifeless: would superuser_connection be acceptable as a name?
<lifeless> sure
<mwhudson> i didn't grasp the sense that root_ was intended in
<lifeless> ok
<lifeless> changing
<mwhudson> thanks
 * mwhudson grimaces at notNull=False
<lifeless> double negatives ftw
<mwhudson> it's almost triple, in some sense :)
<lifeless> its a bit [perhaps a lot] annoying that storm doesn't have the concept of 'can query on this but do not retrieve it ever'
<mwhudson> you mean, stuff that shouldn't be retrieved when the object is materialized?
<lifeless> yeah
<mwhudson> to make the data sent back smaller?
<mwhudson> hm yeah
<lifeless> nor used when inserting the object
<lifeless> this is a field we need in constraints
<wgrant> That seems like it should be pretty easy...
<lifeless> not in memory, not in inserts and not in select clauses.
<wgrant> huwshimi: Hi.
<huwshimi> wgrant: Hello
<wgrant> huwshimi: Seen bug #753423? We turned it off on prod last night.
<lifeless> we'd also need to expose it in subselects etc
<_mup_> Bug #753423: Privacy notification overlay broken in Firefox <javascript> <ui> <Launchpad itself:Triaged> < https://launchpad.net/bugs/753423 >
<lifeless> so its actually a bit complex.
<huwshimi> wgrant: huh. What version of Firefox was that?
<mwhudson> lifeless: basically it's something the query compiler should see, but nothing else?
<mwhudson> lifeless: holy pee administered btw
<mwhudson> ... and back to performance reviews
<lifeless> mwhudson: yeah
<mwhudson> (procrastination driven development anyone)
<lifeless> zigactly
<wgrant> huwshimi: Seen in 4 and 3.6
<huwshimi> wgrant: Ok thanks.
 * mwhudson giggles
<mwhudson> Conflict adding file versions.cfg.THIS.moved.moved.moved.moved.moved.  Moved existing file to versions.cfg.THIS.moved.moved.moved.moved.moved.moved.
<lifeless> \o/
<lifeless> procrastinating again ?
<mwhudson> how did you guess!
<lifeless> no idea
<lifeless> none at all
<lifeless> wgrant: want the good news or the bad news
<thumper> https://code.launchpad.net/~thumper/launchpad/fix-unicode-path-oops/+merge/56688 needs review
<lifeless> mwhudson:   5 https://edge.launchpad.net/bazaar/resolve_lp_path (BazaarApplication:PublicCodehostingAPI)
<StevenK> lifeless: We have to use Buildbot for the next 15 years?
<lifeless>        OOPS-1923EA1350, OOPS-1923EE1204, OOPS-1923EE1205, OOPS-1923EE1211, OOPS-1923EE1212
 * thumper closed twitter tabs to avoid twitter induced procrastination
<wgrant> lifeless: BugTask:+index didn't go down at all, and searchTasks is even higher?
<thumper> about to close facebook soon too
<lifeless> wgrant: it did go down
<lifeless> wgrant: it was over a hundred
<lifeless> wgrant: I think we've done the low hanging fruit
<wgrant> WTF, +login timing out?
<wgrant> I wonder if they were around the DC breakage time.
<lifeless> possibly
<mwhudson> lifeless: is that timeouts?
<wgrant> Yes, they were.
<wgrant> So our results from today could be a little off, I guess :(
<lifeless> mwhudson: yes, probably from the inter-DC fail
<mwhudson> oh ok
<lifeless> will need to reevaluate on Monday
<wgrant> It would be nice to see the distribution over time of each OOPS clsuter.
<lifeless> I is
<lifeless>   1 AssertionError: Build 434541 lacks a corresponding source publication.
<lifeless>    GET: 1 Robots: 1  Local: 0
<wgrant> I suspect a lot of the spiky OOPSes are just people hitting refresh lots.
<lifeless>       1 https://launchpad.net/ubuntu/+source/sugarcrm/4.5.0h-0ubuntu2/+buildjob/434541/+index (BinaryPackageBuild:+index)
<lifeless>        OOPS-1923K942
<lifeless> db inconsistency ?
<lifeless> wgrant: I'm positive they are
<wgrant> lifeless: Sort of.
<wgrant> lifeless: There are two causes.
<wgrant> lifeless: 1) gina sucked when Ubuntu was imported, created some builds where there should not have been any.
<wgrant> 2) Incorrect builds were created when hppa was reintroduced.
<lifeless> wgrant: and we assert on this *6 years* later?
<wgrant> lifeless: The assertion was commented out for years, because the sampledata was broken.
<wgrant> lifeless: it was reeneabled just a few months ago, after the sampledata was fixed.
<wgrant> It was not realised that production data was broken too.
<wgrant> (the brokenness is not a significant issue, so we should probably just recomment the assertion)
<wgrant> lifeless: How are we going with the new appservers?
<wgrant> SQL time: 995 ms
<wgrant> Non-sql time: 10592 ms
<wgrant> Not. Cool.
<wgrant> And this is just going to get worse as we drop the timeout, because average CPU:SQL ratio will increase.
<StevenK> wgrant: Rargh! It took me WEEKS to uncommet that assertion
<lifeless> wgrant: exactly
<wgrant> StevenK: But you destroyed tonnes of doctests.
<lifeless> wgrant: they are coming
<StevenK> wgrant: WEEEEEEEEEEEEEEEEEEEEKS!
<StevenK> Still, I suppose it was 6,000 lines of doctests
<lifeless> wgrant: 2 of the 4 have memory in place, we have the name of the 3rd, and the 4th has had its memory order
<wgrant> lifeless: wampee and soybean are 24GB now?
<StevenK> lifeless: Can share name? Just curious what it is.
<lifeless> https://lpstats.canonical.com/graphs/WampeeMemory/ https://lpstats.canonical.com/graphs/SoybeanMemory/
<lifeless> StevenK: I'd need to look up the rt
<lifeless> StevenK: tom pastebinned a template generation command line that faceplanted
<StevenK> chaenomeles. I can't even pronounce that.
<wgrant> lifeless: Hm, soybean seems to have a bit of headroom :)
<lifeless> yeah
<lifeless> the issue is shuffling ids around
<wgrant> lifeless: What sort of CPU expansion capabilities do these guys have?
<lifeless> its just a pita
<lifeless> wgrant: we can add another 8 cores + another 24GB of RAM AIUI
<lifeless> wgrant: /possibly/ more beyond that
<wgrant> Ah, nice.
<lifeless> dell 'G6' if that means anything to you
<wgrant> Since it would be nice to continue avoiding scaling horizontally.
<lifeless> right
<StevenK> Er, I thought they were DL380 G6s?
<lifeless> StevenK: sounds right
<StevenK> Which is *HP*
<lifeless> erm
<lifeless> talk to ISD :)
<spm> wgrant: why do you want to avoid scaling horizontally? genuinely curious, as horizontally scaling has always been the holy grail to aim for in every system I've designed
<lifeless> spm: we need to scale horizontally as needed, but:
<wgrant> We need to be able to scale both ways.
<lifeless>  - there is pressure to reduce datacentre rackspace footprint
<wgrant> Vertical is probably cheaper.
<wgrant> At least to an extent.
<wgrant> CPU + RAM probably cheaper than new machine, if we can fit it.
<lifeless>  - the base cost for a machine is fairly high mgmt wise at the moment (logging etc overhead)
<mwhudson> according to hp.com you can fit 192 gigs of ram in a G6 :-)
<spm> at what cost....
<mwhudson> i imagine that's howlingly expensive though
<lifeless> spm: anyhow, the choice of new machine or upgraded machine is ultimately down to TCO
<lifeless> spm: elmo has expressly indicated a preference for scaling of the *capacity* of individual appserver machines in the medium term
<lifeless> spm: we will always keep the ability to scale horizontally (and continue working on being able to do that at the DB, archive, codehosting etc)
 * spm recalls get a very blank look from the senior architect at $job-1 when asked what the ROI on her new exciting vision for revamping the entire departments systems was. it was like the question hadn't even arisen. ho hum.
<lifeless> spm: the lowest configuration I'd be truely happy with is N+1 redundancy across the board; then scale in whatever way is cheapest.
<spm> fair enough. was curious, thanks!
<StevenK>  4 files changed, 24 insertions(+), 57 deletions(-)
<StevenK> Excellent.
<mwhudson> huh, i don't really understand the configurator, but it looks like 144 gigs of ram can be got for about $6k
<spm> $job-2 we used to aim in all our designs to ROI in 6-9 months. any longer than that was invariably excessively optimistic. 3 months was preferable.
<mwhudson> which is a lot less than i expected
<wgrant> I guess the larger DIMMs are a lot more expensive than smaller ones, but it probably has a few slots...
<spm> which was a hilarious experience to bring into the public service at $job-1. when in a single 1 month project you can save your client enough that they essentially get you "free" for the next 11 months, your contract renewal is a doddle. :-D
 * wgrant waves to Windmill.
<wgrant> Bye.
<mwhudson> heh
<mwhudson> are you killing it, or is it dying without help?
<wgrant> I am killing it, since it's still failing like before.
<wgrant> Except now there's a timeout set, so it at least doesn't hang and break the world.
<StevenK> wgrant: You're disabling it again?
<wgrant> Yes.
<StevenK> Okay, re-enabling the windmill job on Jenkins. :-)
<wgrant> Heh.
<StevenK> Last failure on Jenkins was Windmill, too.
<wgrant> Yeah.
<wallyworld_> huwshimi: did you have a chance to look at the screencast?
<huwshimi> wallyworld_: I did briefly. One sec.
<wgrant> Yay, fast test suite again.
<lifeless> wgrant: you nuked w/m ?
<wgrant> Yeah.
<wgrant> :(
<wgrant> At least it doesn't crash the testrunner any more, though!
<wallyworld_> wgrant: it was just the one test that failed. surely it wouldn't be too hard to fix it.
<wallyworld_> at the least, it would be good if we could still run the javascript tests
<wgrant> wallyworld_: No, it was the same breakage as last time.
<StevenK> wallyworld_: Like we said before, we do.
<wgrant> Except that it doesn't hang any more, since there was a timeout added.
<StevenK> wallyworld_: On Jenkins.
<wgrant> wallyworld_: It's not specific to any one test.
<huwshimi> wallyworld_: It would be nice if it did the fade back to the search when you click 'no'. Also there's a little design work that needs to be done on the confirmation screen, but I'm happy to have a look at that once your branch is ready.
<wgrant> wallyworld_: It is an infrastructure issue.
<huwshimi> wallyworld_: But other than that it looks great. Really nice work
<wallyworld_> huwshimi: cool. there was no ready made css that looked suitable for the confirmation screen so i just used whatever. i'm open to suggestions. i'm writing some tests now - there's no exiting ones for our lp picker sadly, just the kazr one
<huwshimi> wallyworld_: OK no problems. If you want I can push a branch with some changes once it's ready to do so.
<wallyworld_> wgrant: StevenK: i really wish the windmill ones were run as part of the build process, else they will bitrot. in any case, could we not run the javascript ones?
<wgrant> wallyworld_: They still rely on Windmill.
<wgrant> wallyworld_: And I believe it's the Windmill infrastructure that is at fault here.
<wgrant> wallyworld_: I really wish we ran them too.
<wgrant> But I wish we sucked less more.
<wgrant> And frequent test breakage prevents us from sucking less.
<wallyworld_> wgrant: yes but given they are mich lighter weight i'm hoping they won't hang or whatever
<wallyworld_> huwshimi: my current work branch (sans tests) is https://code.launchpad.net/~wallyworld/launchpad/assign-non-contributor
<lifeless> wallyworld_: if its an infrastructure issue, then it doesn't matter which tests are run within that set, its all/nothing.
<wallyworld_> huwshimi: if you flick me some css i can add it in
<huwshimi> wallyworld_: Cool thanks.
<lifeless> wallyworld_: if its a specific test issue, we can fix that test specifically as needed; I havent studied wgrants analysis, but if he thinks it infrastructure, I'm inclined to trust it
<wallyworld_> lifeless: sure. i'm thinking though that the js tests are much lighter weight and perhaps less prone to the issues
<wallyworld_> lifeless: likely is infrastructure since i *think* different tests fail
<lifeless> wallyworld_: that assumes that its a test issue, not infrastructure, no?
<wgrant> It's possible (probable, even) that the JS tests are safer to run.
<wgrant> Because they're less likely to expose infrastructure issues.
<wgrant> If someone wants to try putting them in a different layer and reenable them, by all means do so.
<wallyworld_> lifeless: the full windmill tests (ie not he js ones) are more likely to fail from what i've seen, even though windmill is used as the harness for the js tests
<wgrant> It is, as usual, the client.open call failing.
<wallyworld_> wgrant: if i get a chance, i'll look to do that. i'd like as much testing done "in band" as we can
<wgrant> We previously couldn't get that from failed test runs, but had determined it experimentally.
<wgrant> (since the testrunner hang and was killed without a traceback)
<wgrant> We at least have tracebacks now.
<wallyworld_> yes, true
<wgrant> I'll file a bug.
<wallyworld_> wgrant: i'll mention it to deryck tonight. he will be inetrested i'm sure
<wgrant> wallyworld_: Indeed.
<wgrant> Bug #754256
<_mup_> Bug #754256: WindmillLayer tests occasionally hit a timeout in client.open <spurious-test-failure> <Launchpad itself:Triaged> < https://launchpad.net/bugs/754256 >
<wallyworld_> wgrant: thanks. and unless a few remaining issues with onkeypress events are sorted, windmill tests won't work with natty (ff4) anyway :-(
<wgrant> We may want to include extra debugging info and switch it back on until it fails.
<wgrant> Now that we can get useful info out of it.
<wgrant> Or even run buildbot with -D for a while, or something like that.
<wallyworld_> yeah. i'll see what deryck suggests since he's more across this than i am
<wgrant> wallyworld_: It fails to start with a profile error for me :/
<wgrant> Maybe I should wipe some of my ~/.*
<wallyworld_> wgrant: ah. i know how to fix that
<wallyworld_> wgrant: you need to symlink a couple of things
 * wallyworld_ looks
<wgrant> Oh, new unity today.
 * wgrant prepares to reboot.
<lifeless> wgrant: you're familiar with the bfj stuff right ?
<wgrant> lifeless: Yeah.
<lifeless> what would prevent us removing BFJ
<wgrant> The table, or the class?
<lifeless> lets assume I'm right that the table should be folded into the two adjacent tables (packagebuild and translationsbuild)
<wgrant> I'd prefer to keep the current *object* model. Database model I'm not fussed about.
<lifeless> sorry, translationtemplatesbuild
<wallyworld_> wgrant: you need to symlink /usr/lib/firefox/defaults/profile -> /etc/firefox/profile
<wgrant> wallyworld_: Ah, thanks!.
<lifeless> wgrant: well, I don't care for or against the object model. the db model is killing performance.
<wallyworld_> wgrant: or lib64
<wgrant> lifeless: Right.
<lifeless> wgrant: table scans on million row tables - bad
<wallyworld_> whatever is there
<wgrant> wallyworld_: None of those three paths exist.
<lifeless> wgrant: and binarypackagebuild + sourcepackagerecipebuild need to be rolledup too
<lifeless> wgrant: BFJ + PB + BPB -> one table
<lifeless> wgrant: BFJ + PB + SPRB -> one table
<lifeless> wgrant: BFJ + TTB -> one table
<wallyworld_> wgrant: you should have a /usr/lib/firefox ? if so mkdir defaults and then do the symlink
<wallyworld_> wgrant: or you saying /etc/firefox/profile doesn;t exist?
<wgrant> wallyworld_: /etc/firefox/profile doesn't exist.
<wgrant> lifeless: So, we need to keep BFJ around, at least a bit.
<lifeless> wgrant: hy
<lifeless> why
<wallyworld_> wgrant: ok. i think what i did was start ff3 and created a new empty profile. then copied that to /etc/firefox/profile. not sure if a ff3 install creates /etc/filefox/profile or not
<wgrant> lifeless: Unless you want to do a big UNION.
<wgrant> lifeless: eg. for builder history.
<lifeless> wgrant: do we care about builder history?
<wallyworld_> wgrant: we really need windmill to support ff4's directory layout :-)
<lifeless> wgrant: beyond a couple of days
<wgrant> wallyworld_: Yeah :(
<lifeless> ff4 writes its profiles to /etc/firefox?
<wallyworld_> lifeless: ah that could be it. i couldn't recall exactly why i symlinked to that directory. but that i think is the correct answer
<wgrant> I think it's more that Windmill looks for the default profile in one of those directories, to make a copy of its own.
<wgrant> And Firefox 4 doesn't really have a default profile, AFAICT...
<wallyworld_> lifeless: it's a one line change to windmill. but i would like to solve the other ff4 specific test failures first
<wgrant> or it's very well hidden.
<LPCIBot> Yippie, build fixed!
<LPCIBot> Project devel build #619: FIXED in 5 hr 46 min: https://lpci.wedontsleep.org/job/devel/619/
<wgrant> lifeless: So, it should be fairly simple to alter BuildFarmJob/PackageBuild to direct inheritance, and flatten the tables.
<wgrant> The only difficult bits will be the things that query for BFJs or PBs directly, then adapt them to the real objects.
<wgrant> Basically... flatten tables, merge *Derived into *, change SPRB and BFB and TTB to inherit from * instead of *Derived... fix queries.
<lifeless> yeah
<lifeless> this may be 'undo the refactoring that was done', which if true is sad in a way, but the performance measurements are pretty clear about the status quo.
<wgrant> lifeless: The old one wasn't ideal, either. The table split (which this is undoing, and then going further) was not the primary goal -- it was removing duplicated code and cleaning up too.
<lifeless> a big question
<lifeless> which I don't know enough to reason about
<lifeless> is is it easy enough to Just Do
<lifeless> or is it big enough to need a feature squad
<wgrant> It's not small enough to Just Do, but not big enough for a feature squad.
<lifeless> or perhaps a few carefully chosen denormalised columns (like BugMessage.owner is)
<wgrant> I guess it depends what Just Do means.
<wgrant> It's a few days of work.
<lifeless> oh
<lifeless> thats small enough to JustDo
<lifeless> it will fix about 10 timeouts
<lifeless> thats *cheap*
<wgrant> The hairy bits are things like getBuildsForBuilder and findBuildCandidate.
<wgrant> Everything else is actually pretty simple.
<huwshimi> Woah! I have crazy scrollbars!
<lifeless> so fBC should be easy enough.
<wgrant> Since it's not all duplicatastic any more.
<lifeless> wgrant: gBFB - how deep does that need to go, realistically ?
<lifeless> wgrant: like, why do we care about what was built on $builder-N 4 years ago ?
<wgrant> lifeless: Probably only a few days, yeah.
<wgrant> lifeless: Other non-trivialities include URLs... We could share a sequence, but that sounds mildly evil.
<lifeless> wgrant: urls for ?
<wgrant> lifeless: BPBs and SPRBs.
<lifeless> composite key
<lifeless>  /bpb-1234
<lifeless>  /sprb-1234
<lifeless> or /+bpb/1234
<lifeless>    /+sprb/1234
<huwshimi> A review (fixes privacy notifications in Firefox): https://code.launchpad.net/~huwshimi/launchpad/privacy-notification-firefox-753423/+merge/56872
<huwshimi> wgrant: If you're interested ^
<wgrant> huwshimi: Just saw the email. Looking.
<wgrant> huwshimi: Have you tried other browsers?
<wgrant> Well, I guess all that's left is Opera and IE.
<huwshimi> wgrant: It doesn't work in IE because our javascript is broken in IE. I'm just double checking Opera.
<huwshimi> wgrant: brb
<wgrant> huwshimi: Heh.
<wgrant> huwshimi: You can't have the content margin there from the start?
<wgrant> huwshimi: It's not ideal that the page jumps once the JS loads.
<huwshimi> wgrant: Right, sorry back.
<huwshimi> wgrant: It's not ideal.
<huwshimi> wgrant: I'm not sure how else to modify the body classes and add html to the main template without JS.
<wgrant> huwshimi: Can't you add a class in the template if the FF is activated?
<wgrant> (and the object is private)
<huwshimi> wgrant: To the <body>?
<wgrant> Something like that.
<huwshimi> wgrant: Oh and yes, it works in Opera (whatever version I have on my computer)
<StevenK> wgrant: Do you remember where ppa-run and such are hiding on DF?
<wgrant> huwshimi: Excellent.
<wgrant> StevenK: ppa-run?
<StevenK> wgrant: There's a script on DF that I used as a cheat-sheet for how to accept uploads
<wgrant> StevenK: Upload, run process-upload.
<wgrant> If you don't remember all the incantations, maybe follow my HowToUseSoyuzLocally.
<wgrant> I didn't know there was a script... I just do it all manually.
<wgrant> There's not that much to it, particularly with --builds in cron.
<huwshimi> wgrant: Do you actually know if it's possible to modify the body classes and add new divs before content or do you just imagine that it should be possible? :)
<huwshimi> wgrant: Cause from what I'm looking at it doesn't look very possible with the current setup
<jtv> Can someone walk me through the process to upload a debian package on dogfood?
<wgrant> jtv: Have you uploaded it? That's the first step.
<jtv> Yes
<wgrant> Well, I guess the first step is obtaining a package.
<wgrant> Do you have a package?
<jtv> Yes, I've uploaded it.
<huwshimi> wgrant: If you take a look at base-layout.pt you'll get some clues about what's currently possible.
<wgrant> Great, so it's in /srv/launchpad.net/ubuntu-queue/incoming?
<wgrant> huwshimi: Let's see.
<jtv> wgrant: trying to figure that out
<wgrant> huwshimi: Could you put something in view/context/fmt:public-private-css?
<wgrant> huwshimi: That's already injecting classes into body.
<jtv> wgrant: yup, it's there
<wgrant> jtv: ^Rprocess-upload^R^R or so until you see an invocation that includes /srv/launchpad.net/ubuntu-queue
<wgrant> https://dev.launchpad.net/Soyuz/HowToUseSoyuzLocally is probably a useful guide to have open.
<jtv> wgrant: by the way, why "ubuntu-queue" when it's a debian package?
<wgrant> jtv: Er.
<jtv> Oh.
<wgrant> You're trying to upload a package to *Debian*, rather than a Debian package?
<jtv> Oh dear, there's a difference?
<wgrant> Well, Ubuntu uses Debian packages.
<StevenK> Yes
<huwshimi> wgrant: I guess. It doesn't seem like an appropriate place to do it though
<wgrant> Uploading *to Debian* on something production-like *may* work, but is untested.
<jtv> Well yes, but it'd be a bit meaningless for me to say "Debian" when I just mean "not RPM," right?
<wgrant> It certainly won't create any builds or anything.
<jtv> Oh
<jtv> That could make it rather hard for me to Q/A DSD packageset filtering for the debian case by "just uploading a package."
<wgrant> jtv: Well, you could try.
<wgrant> jtv: It won't create any builds, but the primary archive shouldn't care about that.
<wgrant> I'll run the processor so I can watch it blow up, unless you have already.
<jtv> If it doesn't work, will I know whether I caused the problem or it failed somewhere else?
<wgrant> I'll hopefully be able to work that out.
<jtv> I suppose I don't really need any builds, as long as it generates a DSDJ.
<wgrant> Right.
<StevenK> And an SPPH
<StevenK> wgrant: I'm about to IDS up DF
<wgrant> Keep in mind that this ranks very high on the sick, wrong, and abuse of -C almost-anything scales.
<StevenK> wgrant: Or I can wait until you're done, just in case
<wgrant> StevenK: Are you trying to kill Nafallo?
<StevenK> Pffft, IDS only takes 7 minutes.
<wgrant> ... on prod.
<StevenK> No, on DF
<wgrant> Huh.
<StevenK> I ran it yesterday
<StevenK> I'm expecting it to be quicker on qas
<wgrant> jtv: You ran it and it got rejected?
<wgrant> /srv/launchpad.net/ubuntu-queue/rejected/upload-ftp-20110408-035934-000012/~/ubuntu/libpqxx_3.2-1sid1.dsc
<wgrant> Not a terribly helpful upload path; you probably wanted /debian instead of /~/ubuntu
<jtv> wgrant: ran what exactly?
<wgrant> The upload has been processed.
<wgrant> and rejected.
<wgrant> Someone ran process-upload.py.
<jtv> Yay.
<jtv> Ah, yes.
<jtv> But yes, I wanted to upload to debian.
<wgrant> We'll need -C absolutely-anything, I suspect, which is why I want to run it myself.
<jtv> Ahhh, ubuntu is set in "incoming" in my .dput.cf.
<wgrant> Since Debian has no ComponentSelections or SourcePackageFormatSelections or ArchivePermissions.
<wgrant> Or even a SectionSelection, for that matter.
<wgrant> StevenK, jtv: https://code.launchpad.net/~wgrant/launchpad/remove-archivepublisher-config/+merge/56873 would like a review.
<jtv> wgrant: I'll take it.  Meanwhile, I replaced "ubuntu" with "debian" in my .dput-cf (well, I cloned the "dogfood" section) but that doesn't seem to be right.
<wgrant> jtv: Do you still have a ~/ in there?
<StevenK> wgrant: r=me
<wgrant> StevenK: Thanks.
<jtv> That was quick :)
<wgrant> It's not a very difficult branch.
<StevenK> It's +0/-6
<StevenK> So a no-brainer
<jtv> wgrant: yes, I doâ¦ just copied what I had for ubuntu.
<wgrant> jtv: That's for PPAs.
<jtv> ~%(dogfood)s/ubuntu
<wgrant> Please don't make me hack non-Ubuntu PPA support onto mawson.
<wgrant> [df]
<wgrant> method = ftp
<wgrant> fqdn = upload.dogfood.launchpad.net:21
<wgrant> incoming = %(df)s
<wgrant> login = anonymous
 * jtv backs off slowly, keeping both hands in view
<wgrant> That's what I use.
<wgrant> dput df:debian whatever_source.changes
<jtv> And that'll do both ubuntu and debian?
<jtv> ah!
<StevenK> wgrant: Bug 753249 makes me sad.
<_mup_> Bug #753249: +initseries calls deriveDistroSeries() with incorrect arguments <derivation> <Launchpad itself:Triaged> < https://launchpad.net/bugs/753249 >
<jtv> And the %(df)s is the section name?
<wgrant> jtv: Right.
<wgrant> jtv: It is replaced with the argument after the colon.
<jtv> Great.  Trying the upload again.
<wgrant> I'm not sure if absolutely-anything will be sufficient, but we'll see.
<jtv> Grrr package has already been uploaded
<wgrant> -f
<jtv> Oh, I just edited the changelog and uploaded again.
<jtv> It's almost done I think.
<jtv> Almost thereâ¦
<jtv> Done!
<wgrant> There it is.
<wgrant> 2011-04-08 05:21:47 DEBUG   Unable to find distroseries: unstable
<wgrant> Overriding...
<wgrant> Nearly there.
<wgrant> Bah.
<jtv> Should I say "sid" instead of "unstable"?
<wgrant> P-a-s crashing due to no nai.
<wgrant> 2011-04-08 05:27:19 DEBUG   Thank you for your contribution to Debian GNU/Linux.
<wgrant> jtv: ^^
<jtv> wgrant: what is that in English?
<jtv> Something crashed, something succeeded?
<wgrant> 2011-04-08 05:27:19 DEBUG   Sent a mail:
<wgrant> 2011-04-08 05:27:19 DEBUG       Subject: [debian/sid] libpqxx 3.2-1sid2 (Accepted)
<wgrant> 2011-04-08 05:27:19 DEBUG       Recipients: "Jeroen T. Vermeulen" <jtv@canonical.com>
<StevenK> It broke, and he fixed it
<jtv> Ah.  Thanks!
<jtv> And there's my DSDJ!
<jtv> Where's the DSDJ runner?
<wgrant> (FTR, I changed the upload path to debian/sid, added a ComponentSelection for main in sid, amputated pas.py, then process-upload -C absolutely-anything)
<jtv> Wow, this stuff isn't easy is it?
<wgrant> That was actually a lot easier than I expected.
<jtv> What is pas.py by the way?
<StevenK> jtv: Don't ask
<wgrant> Do not go there.
<StevenK> You don't want to know
<wgrant> Heh.
<jtv> StevenK: gee thanks, I already asked didn't I?
<jtv> Oh well, I'm quite happy for the monster to stay under the bed.
<wgrant> In this case the monster *is* the bed.
<lifeless> hmm
 * jtv has a brief, swimming vision of Ripley and the child hiding under the bed and the facehugger on top squeaking "oh dear oh dear there's monsters under the bed"
<lifeless> are bugs 739144 and 701525 dups
<wgrant> Bug #739144, bug #701525
<_mup_> Bug #739144: Code review comment via email truncated, most content lost! <email> <regression> <Launchpad itself:Triaged> < https://launchpad.net/bugs/739144 >
<_mup_> Bug #701525: Merge-proposal reply truncated <Launchpad itself:Incomplete> < https://launchpad.net/bugs/701525 >
<wgrant> I think so.
<wgrant> I thought someone already duped them.
<jtv> Is that the same truncation that also happens on the web page?
<wgrant> .. oh.
<wgrant> A third instance.
<wgrant> That is odd.
<lifeless> I was looking for a dup for 754303
<lifeless> which I could *swear* I saw
<lifeless> but the search eludes me
<wgrant> I've seen something like it before, yeah... but I can't think of the summary.
<poolie> i guess it's not news if staging is timing out?
<lifeless> nope
<lifeless> I mean
<poolie> k
<lifeless> if it doesn't work how ever many times you try
<lifeless> and it works on prod
<lifeless> it may be an issue
<lifeless> but considering you may need 40 or 50 attempts to seed the cache
<poolie> sure, but intermittent is ok
<poolie> np
<lifeless> you need rather a lot of patience
<lifeless> wgrant: is bug 51007 really still present ?
<_mup_> Bug #51007: unnecessary truncation in queue output <lp-soyuz> <soyuz-ftpmaster-tools> <Launchpad itself:Triaged> < https://launchpad.net/bugs/51007 >
<wgrant> lifeless: I think so. It should be gone in 6 months, though.
<wgrant> (queue is dying)
<lifeless> wontfix ?
<wgrant> Not yet.
<wgrant> Maybe eventually.
<lifeless> well
<lifeless> if its going to be deleted
<wgrant> It hasn't been decided exactly what will happen yet.
<lifeless> and we're not going to fix in the interim
<lifeless> wgrant: so queue isn't dying ?
<wgrant> lifeless: It has to at least change significantly for DDs. But we don't know quite how yet.
<lifeless> wgrant: so, we'll provide a webUI, one that works ?
<wgrant> It will probably die, given that shell access is hopefully going to vanish.
<wgrant> Right.
<lifeless> so, lets wontfix.
<lifeless> its been *7* years
<lifeless> for a trivial show-wider-columns problem.
<wgrant> 5 years, but OK.
<lifeless> bah, yes.
<jtv> StevenK: having some weird trouble with the dsdj runner.  It doesn't seem to clean up its jobs.
 * lifeless closes a 4-digit bug
<jtv> StevenK: I always forget whether job runners are supposed to do that or not.
<StevenK> It is not
<lifeless> I think bug 674759 might be fixed
<_mup_> Bug #674759: hide-comments.py is hiding the wrong bug <canonical-losa-lp> <lp-bugs> <Launchpad itself:Triaged> < https://launchpad.net/bugs/674759 >
<jtv> StevenK: Ah sorry.  Just seeing some aborts in the debug output that mystify me a bit.  It's hard to Q/A "things shouldn't happen."  :-)
<StevenK> jtv: Oh, like what?
<jtv> I'm Q/A'ing the packageset filtering of DSDs.
<StevenK> Yes, I know, I'm interested in the output
<jtv> Jesus if these neighbours could just be a _little_ bit quietâ¦  I got the guy with the truck to turn off the radio on the external speakers, so now the baby kicks in.
<jtv> Unfortunately I can't snip anything right now; trackpad stopped working.
<StevenK> jtv: The next door apartment is having their bathroom remodelled, so I got woken up at 8:00am by them removing tiles with a very noisy power tool.
<jtv> StevenK: kill!
<StevenK> jtv: Thankfully, they stopped using it at 2:00pm
<jtv> Due to successful kill?
<StevenK> Sadly, no.
<StevenK> And I got to miss 30 minutes of it while I walked to get lunch.
<jtv> Personal tip: if you miss the comforting sound of a power tool being used near you at an ungodly hour, play some Ronald Keating.  Different sound, same feeling.
<lifeless> hmm
<lifeless> 7 seconds to add a bug comment
<lifeless> I think the structural filters are slowing things down somewhat.
<spiv> StevenK: I had the pleasure of being one of those residents that inflicted power tool noise on our neighbours, finally.
<StevenK> Haha
<StevenK> spiv: Vincent got "My First Power Drill" as a present?
<spiv> StevenK: we had a leaking pipe in the shower wall, so there wasn't really any choice, so I didn't even have to feel too bad about it :)
<spiv> Although the week and a half without a functional shower was tiresome (fortunately we also have a bathtub).
<spiv> StevenK: Vincent is quite capable of making noise without power toolsâ¦
<StevenK> Haha
<spiv> Such as by banging things together that aren't meant for banging, and then he makes even more noise when you take the things away from him!
<lifeless> bam bam
<wgrant> I'm on the conflict.
<wgrant> Non-trivial JS refactors on both branches :(
<wgrant> Ah, maybe it's the one that gary referred to.
<jtv> Anybody up for a review involving shell quoting?  https://code.launchpad.net/~jtv/launchpad/db-bug-752178/+merge/56719
<jtv> hi henninge!
<henninge> Hi jtv! ;)
<danilos> does YUI3 have anything regarding string formatting (ala sprintf)?
<wgrant> danilos: Y.Lang.substitute may help.
<wgrant> lib/lp/code/javascript/requestbuild_overlay.js uses it.
<danilos> wgrant, exactly what I needed, thanks
<jtv> wgrant: should GNUPGHOME really be unset in the 10-sign-releases script?
<jtv> Or is that just an example?
<wgrant> jtv: It needs to be set to some special value on prod.
<wgrant> On DF I just unset it to make it work.
<jtv> wgrant: thanks.  The scripts I implemented were meant for production, so we could just hard-code the production value there if we know what it is.  But what is it?
<jtv> Luckily I made it easy to use in-tree or out-of-tree scripts, so we could just let IS copy and edit the scripts.
<wgrant> He always drops out as I try to answer :(
<wgrant> jtv: Check cron.publish-ftpmaster for the prod value.
<jtv> Thanks!
<jtv> (So much for saving my IRC connection by setting my TCP keepalive wait to 1 minuteâ¦ I'll try with a shorter interval)
<adeuring> good morning
<jtv> hi adeuring!
<jtv> Will you be reviewing today?
<adeuring> hi jtv!
<jtv> adeuring: if you'll be reviewing, then https://code.launchpad.net/~jtv/launchpad/db-bug-752178/+merge/56719  please.  âº
<adeuring> jtv: sure, I'll look
 * adeuring just needs a bit more coffee
<mrevell> Hello
<jtv> adeuring: thanks, and enjoy your coffee.  :)
<bigjools> good morning world
<jtv> world julian morning good
<bigjools> wgrant: are people really so thick as to repeatedly try to upload using the same unregistered GPG key despite the warning they'll get now?
<lifeless> bigjools: you are nowhere near pessimistic enough
<bigjools> lifeless: ha
<bigjools> the sun is shining, I am feeling the opposite today :)
<wgrant> bigjools: Do we know that the "Signing key %s is not registered in launchpad." error is returned properly to the client?
<bigjools> yes
<wgrant> I know the "no public key" one is, but that is an actual gpg error.
<lifeless> wgrant: I know something we can do
<lifeless> wgrant: about tags
<bigjools> unless something broke since I tested it
<lifeless> wgrant: use the fast query, and do an exists subquery to find *a* bug that is [visible clause]
<wgrant> lifeless: Hmm. That's not really correct. But something to think about.
<lifeless> wgrant: http://paste.ubuntu.com/591153/
<lifeless> wgrant: how is it incorrect?
<wgrant> lifeless: I can file a bug in any project, tag it with some tag, and magically that tag shows up in lots of projects' portlets despite the bugs all being invisible to me.
<lifeless> why would that happen ?
<wgrant> Oh, I misread.
<wgrant> That's not a bad idea.
<lifeless> needs testing and result cross referencing
<lifeless> but it seems snappy
<bigjools> wgrant: so, we need arbitrary pockets
<wgrant> bigjools: No, we need no pockets.
<wgrant> bigjools: We will have suites instead.
<bigjools> wgrant: that won't work very well with our distroseries model
<wgrant> I guess :(
<bigjools> oh fuck sake, natty updates this morning are hanging on update-initramgs
<bigjools> update-initramfs even
<wgrant> Suite = (DistroSeries, suffix)
<bigjools> yes, which is basically a Pocket table :)
<wgrant> Maybe.
<adeuring> jtv: r=me
<wgrant> Bah.
<jtv> thanks adeuring!  And got more on the way.  :)
<henninge> wgrant: Hi!
<wgrant> henninge: Hi.
<henninge> wgrant: Looks like I deleted you mail on xss attacks. Where did you post that so I can go to the archives?
<lifeless> stub: probably want to unlink that old rosetta bug from your garbo branch :)
<lifeless> henninge: canonical-lp
<wgrant> henninge: It was canonical-launchpad
<henninge> cheers
* adeuring changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: adeuring | https://code.launchpad.net/launchpad-projet/+activereviews | qastaging down for a schema test, back soon
<stub> lifeless: Yeah, that old annoying bug.
 * stub throws away some historical data because MPs are too thick to realize bugs that have been fixed are unlikely to be fixed again.
<wgrant> bigjools: I guess the table will also need to encompass policies that are hardcoded now :(
<lifeless> stub: there is a bug open on that too; we should capture bugs against the mp not infer from the branch
<lifeless> or something like that
<bigjools> wgrant: well we need a pocketpolicy table
<wgrant> bigjools: And SuiteDependency.
<wgrant> Like ComponentDependency :(
<bigjools> yeah
<bigjools> man this is a deep rabbit hole
<wgrant> But the branches are fairly shallow.
<bigjools> talking of rabbit, can we please get rid of rabbit-mq from the lp dependencies
<wgrant> There's just an awful lot of them.
<bigjools> its package is a royal pita
<lifeless> bigjools: how so? [and no, I have a branch enabling it, just needs a little debugging to land]
<bigjools> lifeless: I do not want server daemons running on my laptop
<bigjools> so this applies to apache and all the other crap we need
<bigjools> it should start up on demand
<lifeless> bigjools: disable it
<bigjools> the rewrite daemon that apache starts uses huge gobs of ram as well
<lifeless> edit /etc/defaults/rabbitmq-server and set the start command to false; IIRC
<bigjools> lifeless: you think I haven't already? :)
<lifeless> the test suite will spin up its own rabbit
<jtv> adeuring: got a very very brief one: https://code.launchpad.net/~jtv/launchpad/db-bug-752179/+merge/56896
<adeuring> jtv: sure
<jtv> thanks
<jtv> I'm also about to propose another small one.
<wgrant> bigjools: Is there a list of this sort of thing on the LEP?
<bigjools> wgrant: I've amended it at a high level, no implementation details at all
<adeuring> jtr=me
<bigjools> wgrant: are you getting problems upgrading rabbitmq-server on natty?
<wgrant> bigjools: Nope.
<wgrant> Works fine.
<bigjools> I get dpk errors every time because the packaging is a PoS
<wgrant> I haven't disabled it, though.
<bigjools> either way it belly-aches
<lifeless> bigjools: details
<bigjools> Starting rabbitmq-server: TIMEOUT - check /var/log/rabbitmq/startup_{log,err}
<bigjools> the logs are empty because I disabled it
<bigjools> but it still bitches the same if I enable it
<lifeless> SpamapS: ^
<bigjools> the package installation also fails if there's no internet connection
<bigjools> lifeless: I get that error when doing a manual start so something's fairly hosed!
<lifeless> bigjools: sure sounds it
<bigjools> lifeless: re-installing, and it still failed.... :/
<bigjools> it now sits there as an unremovable package that blocks apt-get dist-upgrade.  Sigh.
<wgrant> Even after a purge?
<lifeless> buggy maintainer script
<lifeless> file a bug
<lifeless> and edit /var/lib/dpkg/info/rabbit*
<bigjools> wgrant: can't purge, had to do dpkg remove --force-depends to get rid of it
<wgrant> Now purge it.
<bigjools> E: Unmet dependencies. Try 'apt-get -f install' with no packages (or specify a solution).
<wgrant> dpkg --purge --force-depends rabbitmq-server
<wgrant> Otherwise we may have to hack the cache.
<bigjools> ok done
<bigjools> re-instally?
<bigjools> re-install? ...
<wgrant> Yeah
<wgrant> Hopefully --purge deleted the database.
<bigjools> Starting rabbitmq-server: SUCCESS
<bigjools> so now I just need to disable it again :)
<bigjools> thanks for the help
<jtv> adeuring: would you mind another one?  Also quite smallâ¦  https://code.launchpad.net/~jtv/launchpad/db-bug-752181/+merge/56903
<adeuring> jtv: sure
<jtv> thanks
<bigjools> wgrant: so I am trying to come up with a plan to phase in some pocket/pocketpolicy schema
<bigjools> wgrant: I think it can be done by making the schema/model and then converting little bits of soyuz one at a time.  When they're all done, then we can have other distros start to use it.
<wgrant> bigjools: Gradually move bits of policy onto a table which uses the enum, then once they're done replace the enum with an fkey everywhere?
<bigjools> wgrant: so first I want to have a pocket table.  Then we need some policies defined, probably enums which map to some code.  Then we have a pocketpolicy table (pocket, policyenum) which gets looked up instead of hardcoding the rules, whereever the rules are currently applied.  They can be gradually moved.
<bigjools> but this may need thinking over a bit more, we might need to apply policies in order for example
<StevenK> bigjools: I'd suggest a better name than 'pocket'
<wgrant> Well, the policies are pretty simple at the moment.
<bigjools> StevenK: pocket is a very strong name in Soyuz
<wgrant> Suite = (DistroSeries, suffix)
<wgrant> We will emulate Ubuntu's pockets using suites.
<wgrant> We have to fix a whole lot of stuff, we might as well stamp out pocket.
<bigjools> I'm not convinced suite is the right way
<wgrant> Why not?
* wgrant changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: adeuring | https://code.launchpad.net/launchpad-project/+activereviews
<bigjools> I can't articulate it right now but I have a feeling that it'll be hard to make it work with the existing code
<wgrant> Most stuff treats (archive, distroseries, pocket) opaquely.
<bigjools> OEM still uses pockets, they just don't call them that
<wgrant> Except for some views.
<bigjools> if we keep the notion of a pocket it will fit in nicely
<wgrant> That satisfies all of OEM's use-cases?
<bigjools> wgrant: pretty much - they have a devel, staging and "production" pocket
<wgrant> But they do insane things with series, right? Or are we going to model that with separate Distributions?
<StevenK> bigjools: You already said on the call that it's going to involve touching a lot of code -- to my mind, I still like the idea of wgrant's Suite table.
<StevenK> We already call it that anyway
<adeuring> jtv: r=me again
<jtv> and thanks again
<bigjools> yes so a suite table would be (distroseries(FK), pocket(text))
<wgrant> For now, yes.
<wgrant> It gives us the flexibility to change it in future without changing EVERYTHING.
<bigjools> well we need to change everything to use it :)
<wgrant> Right, but we need to change just about everything anyway.
<bigjools> but the most important thing is to encapsulate this stuff in *one damn place*
<wgrant> Yes.
<bigjools> then we can gradually refactor existing code
<wgrant> Ideally most stuff would just deal with an opaque (archive, suite) object, but that's going to be a bit awkward to model well.
<bigjools> very awkward
<StevenK> bigjools: So now you agree?
<bigjools> this needs to be an evolution not a revolution
 * StevenK blinks
<bigjools> StevenK: agree with what?
<StevenK> The idea of a suite table
<wgrant> I guess the DB layout doesn't really matter.
<wgrant> As long as the code only sees suites.
<bigjools> StevenK: naming is irrelevant, the idea we had was the same
<StevenK> bigjools: I was thinking about the multi-inheriance thing while walking to meet Sarah, too. It's fairly easy to model, if we have an id in Suite as well
<StevenK> Then the parent/child is a Suite, which solves the pocket crap too
<bigjools> StevenK: mmm not sure about that
<bigjools> remember that suites are inherited too
<StevenK> bigjools: If it's just distroseries as it is now, we
<StevenK> just have awkwardness in regards to pockets
<bigjools> StevenK: I need to think this through some more, it might do the trick
<bigjools> we need pocket/suite blacklisting I think
<StevenK> bigjools: This way, that particular suite is not involved in the dependency chain, so it's elegant
<wgrant> I think we probably need to do it StevenK's way.
<wgrant> Or derived series will have to have a matching set of suites.
<wgrant> Or no derivation will happen.
<bigjools> I don't see how it would need to match
<StevenK> wgrant: So it sounds fine to you?
<wgrant> bigjools: What happens if it doesn't match?
<bigjools> wgrant: you tell me
<wgrant> bigjools: Say OEM wants to have stuff inherited from natty-updates into $CRACKFUL-staging, to test updates before pushing them out to customers.
<StevenK> At the moment, they can only dervie all of natty
<wgrant> They probably want to inherit natty-release into $CRACKFUL-release.
<wgrant> But not natty-updates into $CRACKFUL-updates.
<wgrant> Since copying those blindly would be a really bad idea.
<bigjools> you've still not explained why they need to match
<bigjools> using words like "probably" don't fill me with confidence :)
<wgrant> bigjools: I manage a derivative distro. I want to test natty's updates before I push them out to my millions of users.
<wgrant> So I have natty-updates inherited by my crackful-proposed.
<wgrant> I test them there, then copy to crackful-updates.
<wgrant> If there is no explicit mapping, how does LP know to  inherit from natty-updates to crackful-proposed?
<wgrant> It will try to inherit from natty-updates to crackful-updates, and all my users will die.
<bigjools> this is all based on the assumption that you're inheriting from suites
<wgrant> I would hope you would be.
<wgrant> Because OEM doesn't want -backports.
<bigjools> there are other ways
<wgrant> Oh?
<bigjools> such as listing all the versions in each suite for a distroseries
<wgrant> But I don't care about -backports; I don't want it to show up so I have to dismiss it every time.
<bigjools> unless we blacklist it
<jtv> StevenK, you probably have better things to do right now but if you don't, got some time to explain something to me?
<StevenK> Right, so like I said, in this case the backports suite isn't in the chain, so it doesn't show up
<StevenK> jtv: I'm trying to not think about work, but bigjools seems to have solved that already.
<jtv> Maybe bigjools can thenâit'd be fairer but I saw him in another discussion.
<bigjools> wgrant: we don't need mappings for this stuff, we need a mechanism to say put that source (or sources) from that suite into this suite.  Job done.
<bigjools> but, I would like to explore all ideas
<bigjools> having many options is rarely bad
<wgrant> bigjools: I think we probably need defaults, and definitely need to be able to exclude some suites.
<bigjools> exclusion, definitely
<wgrant> Launchpad knows it's a security update, but it still makes me explicitly put it in -security :(
<StevenK> jtv: So, need what explained?
<bigjools> wgrant: we need to get more use cases, I'm not convinced (yet)
<jtv> StevenK: thanks.  The question is what publish-ftpmaster should do when it's moving $distscopyroot/dists to $archiveroot/dists.new and there's an existing dists.new in the way.
<jtv> Or rather, that's what triggered the question.
<StevenK> I don't think that should happen.
<jtv> Probably not, no.
<jtv> But weird things can happen.
<bigjools> jtv: doesn't the cleanup prevent that?
<jtv> In principle, yes.  But what if the cleanup itself fails?
<jtv> Yet with strange aeons, even Death may die.
<bigjools> then we need a cleanup cleanup
<bigjools> :)
<bigjools> at some point we have to rely on shit working
<bigjools> I suspect that overwriting dists.new is never wrong
<jtv> That is exactly what I'd like to know.
<jtv> Plus, what exactly is the role of distscopyroot?
<bigjools> provided there's transactional integrity
<jtv> (There isn't, but we try to approximate it.)
<wgrant> jtv: The dists copy needs to be outside archiveroot when the mirror sync is triggered.
<bigjools> well I mean, the publisher may set the source's status to PUBLISHED, it's there in dists.new, commit() and then boom....
<bigjools> we're fucked
<wgrant> jtv: And we need to keep a copy of dists or we're copying it every time, and that's slow.
<jtv> wgrant: okay, but what is it _for_?  What does it hold?
<LPCIBot> Project devel build #621: FAILURE in 32 min: https://lpci.wedontsleep.org/job/devel/621/
<StevenK> Oh dear
<StevenK> Ah. Slave went bang
<jtv> At this time of day, the domain name almost looks like a gentle reminder to StevenK not to bite off more than he can chew.  :-)
<wgrant> jtv: There are two copies of the dists tree. One lives in the archive root and is used by clients, the other lives in distscopyroot.
<jtv> "Sleep?  You thought you were getting time off?"
<wgrant> jtv: The publisher shuffles them around to do atomic updates.
<jtv> And the dists directories dance back and forth between these two?
<deryck> Morning, all.
<jtv> hi deryck
<bigjools> wgrant, StevenK: the other problem we have is that we want to treat some derived distros as nothing but overlays on the parent.  (like PPAs) which means we need to only publish its changes from the parent
<wgrant> bigjools: Oh, that's in scope for the initial implementation? :.
<wgrant> *:/
<wgrant> I didn't realise that.
<bigjools> wgrant: it might be, it depends on what jml decides as chief strategerist :)
<wgrant> I think that's probably not going to be doable, but we'll see.
<wgrant> Maybe with excessive pinning.
<bigjools> no, no pinning
<bigjools> stabby stabby
<bigjools> we need 2 ways of deriving basically
<bigjools> copy, or overlay
<wgrant> An overlay isn't safe without pinning.
<bigjools> why?
<wgrant> You apply an important patch to $package.
<rvba> bigjools: by pinning you mean stick with a specific version?
<wgrant> Ubuntu applies a security update.
<bigjools> (I can guess but..)
<wgrant> Your users get Ubuntu's version instead.
<bigjools> that won't happen
<bigjools> because we're overlaying only on the release pocket
<bigjools> updates need to go via the overlay
<StevenK> Which we can only do if we have Suites :-)
<bigjools> rvba: yes, it's an apt config thing
<bigjools> StevenK: not necessarily, there are many ways of implemeting it
<bigjools> keep an open mind :)
<StevenK> But if we do it for the other ...
<wgrant> bigjools: Ah, I see.
<wgrant> So it relies on having a frozen release pocket. I guess that's a reasonable constraint.
<bigjools> wgrant: ah reading the notes from yesterday, what I said can happen and also your way.  I need to ask cody-somerville how they cope with that
<bigjools> wgrant: it seems they rely on reporting
<bigjools> hmm
<bigjools> wgrant: they also create snapshots of release plus some -updates and work off that
<wgrant> I was about to ask for a forward.
<wgrant> Thanks.
<bigjools> :)
<bigjools> not sure why I didn't do that already ...
 * maxb is intrigued to see how https://answers.launchpad.net/launchpad/+question/152080 gets answered
<jtv> bigjools: I may be way behind the discussion on this butâ¦ is "multiple distroseries inheritance" really what's needed?  The notes and diagrams I've seen so far seem to be more about a separate "pull PPA packages into distroseries" feature.
<bigjools> jtv: it's both
<bigjools> jtv: they manage composite archives
<wgrant> maxb: "Where is the documentation targeted to people who're willing to run their own instance of lp?" suggests a misunderstanding of the situation :(
<jtv> bigjools: I'll have to read in more detail later thenâ¦  It's sad that evidently we thought we were ready-to-code but we weren't.  I wonder if we could reduce this kind of risk with closer "customer" involvement once coding begins.  Or would that just open the bikeshed?
<wgrant> jtv: Ready to code what?
<wgrant> jtv: The stuff that is already done?
<jtv> No, the LEP that we're implementing.
<jtv> As a whole.
<bigjools> we are ready to code the LEP
<bigjools> and were ready
<wgrant> But the LEP needs extension.
<bigjools> yes
<jtv> But these are new requirements, right?
<wgrant> Perhaps an extension.
<bigjools> the changes don't affect existing plans
<wgrant> Yeah, I think the existing Linaro plan is reasonably compatible.
<jtv> I must be missing something.  What I understood was happening was that we were supposed to include these extra requirements into the initial delivery of the functionality in the LEP.
<jtv> If that's not the case, and existing plans are not affected, then that sets my mind at ease.
<bigjools> jtv: right, we're not changing the immediate deliverables, just extending the scope
<jtv> So I can think of it as an extra LEP building on the current one?
 * jtv was traumatized by an extended scope as a child and still flinches when he sees them in the street
<bigjools> jtv: O_o
<jtv> Okay, okay, I made that up.
<bigjools> jtv: yes, I may even do a sidebar LEP :)
 * jtv pictures one of those old bibles with columns of notes about the notes
<wgrant> I'd always understood that this LEP was just the first phase, to get Linaro and possibly Ubuntu onto the new system.
<wgrant> With OEM requiring more complex rules and privacy, in at least two later phases.
<wgrant> Oh seriously.
<wgrant> More conflicts.
 * wgrant fixes.
<bigjools> did we land any schema patches yet? if not just merge the damn thing to devel
<wgrant> Yeah, one was landed a few hours ago :(
<wgrant> I may land the rev before, though.
<bigjools> yes
<wgrant> I meant to do that, but too many distractions this week :/
<bigjools> the conflicts I fixed were very odd
<wgrant> There's a lot of strucsub-related ones.
<wgrant> But this one is me+jtv.
<jtv> whawhat?
<bigjools> the one I fixed was in the distroseries stuff we've been changing but there were no actual conflicts to speak of, it looked like bzr getting it wrong
<wgrant> archivepublisher config schema.
<jtv> ah that
<jtv> yes, sorry, no way around itâI kept mine as far away as I could, which was nowhere near far enough.
<wgrant> Yup.
<wgrant> Just pqm-submitting now.
<wgrant> Yay, I beat buildbot-poller.
 * jtv goes looking for a whiteboard to map out the dists directory dance
* bac changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: adeuring,bac | https://code.launchpad.net/launchpad-project/+activereviews
<wgrant> jtv: Is 10383 still qa-bad?
<wgrant> I presume not, since you reverted cron.publish-ftpmaster.
<wgrant> Ah, not tagged properly.
 * wgrant fixes.
<bac> hi adeuring
<bigjools> I'd really like a simpler page explaining how to go about tagging when we have blocking qa failures
<wgrant> bigjools: Meh, I missed a table addition. Can only merge one rev, due to a qa-bad interlocked with the new table.
<jtv> wgrant: I'm not sure since it doesn't appear to be documented, but I think the qa-bad tag is appropriate.  I landed the fix with the --rollback option.
<bigjools> balls
<jtv> wgrant: that also means that that revision is still qa-bad, but that the problem goes away if you also merge the later revision.
<wgrant> jtv: Need to tag with bad-commit-NNNNNN as well.
<jtv> Argh
<wgrant> jtv: This is a bug tag, not a commit tag.
<jtv> You mean the bug that was qa-bad
<jtv> ?
<wgrant> Land with --rollback=NNNNNN, tag original qa-bad bug with bad-commit-NNNNNN
<wgrant> (is this documented? not sure. does it work? sort of, most of the time)
<jtv> There's a lot of documentation but all it seems to say is to, er, fix it the usual way or something.
<adeuring> hi bac
<bigjools> wgrant: hmmm I wonder if we could generalise PPAs into "overlay" distributions.
<wgrant> bigjools: What does that do?
<bigjools> wgrant: the difference is only in the sources.list we use to build with, really
<bigjools> (ignoring the components thing for now)
<wgrant> bigjools: Right, so we need to improve archive deps, I guess.
<bigjools> yeah
<bigjools> that's reasonably well encapsulated
<wgrant> We really need to gather all the Ubuntu/PPA/OEM/Linaro/whatever use-cases and work out the sorts of archives we need...
<wgrant> The post-Linaro LEP is probably going to have to be fairly revolutionary.
<bigjools> we'll never finish :)
<jtv> mv red-squad soyuz
 * henninge-lunch relocates
<jtv> bigjools, wgrant: here's the migration pattern of the dists/dists.new/dists.old directories: http://people.canonical.com/~jtv/publish-ftparchive-dists.png
<jtv> Oh, that's still without the cleanup.
<wgrant> jtv: That looks fine for a normal run.
<jtv> Hmmâ¦ I wonder if this isn't a bug in the original script.  If DONE_PUB, then all that the cleanup method does is rename {archiveroot}/dists.old to {distscopyroot}/dists.
<jtv> But the next step after setting DONE_PUB, in this simplified diagram, is to move {archiveroot}/dists.old away.
<jtv> In fact, that _is_ the next step.
<jtv> I can't come up with any set of circumstances where that would make sense.
<wgrant> jtv: Indeed, that is a bit odd.
<wgrant> Perhaps it is just for consistency.
<jtv> (Not including situations like "the disk is flapping" where it _might_ make sense but there's no real way of getting a grip anyway)
<jtv> I think the script just grew out of hand.
 * jtv goes to update the online version of that diagram
<jtv> wgrant: if you refresh now, you'll see a version with the beginning of the cleanup routine marked.
<jtv> This is all based on the old shell version, to avoid clay feet.
<jtv> wgrant: I think this also means that the publish-distro run-parts scripts shouldn't need to know about $DISTSROOT.new; the DISTSROOT parameter should include the .new part.
<jtv> It's nice to know that ultimately, $DISTSCOPYROOT/dists and $ARCHIVEROOT/dists change places.
<jtv> That was not very obvious from the script.
<deryck> adeuring, abentley -- I'm on, just mic not working, I guess.
<deryck> adeuring, abentley -- I'm guessing you guys can't hear me?
<jtv> wgrant: does this mean that $DISTSCOPYROOT/dists is really just a backup of the previous state?
<deryck> adeuring, abentley -- let me relog.
<wgrant> jtv: Right, the idea is to have two copies for speed.
 * wgrant grumbles.
<benji> PQM is unhappy and I can't tell if I'm the cause.  At one point (about 5 hours ago) it was complaining about a conflict in lib/lp/registry/javascript/tests/test_structural_subscription.js but it stopped and is now complaining about lib/canonical/config/schema-lazr.conf.
<benji> the former is mine, the latter is not
<wgrant> 23:06:38 < wgrant> jtv: Right, the idea is to have two copies for speed.
<jtv> ah thanks
<wgrant> Since it can be pretty big.
<wgrant> (it has some CD images and stuff)
<jtv> Ah, now I see how the rsync -H works.
<jtv> It's an Ouroboros.  The only copying that ever happens is the rsync (with hard-linking) back to another instance of itself.
<jtv> Now,
<jtv> where did all you zombies come from?
<jtv> (If the reference wasn't obvious: Robert Anson Heinlein, "All You Zombies," short story)
<LPCIBot> Project windmill build #150: FAILURE in 1 hr 2 min: https://lpci.wedontsleep.org/job/windmill/150/
<gary_poster> benji, thanks for pqm conflict update.  why don't we see complaints on canonical-launchpad?
<benji> gary_poster: tell me what "canonical-launchpad" is and I'll find out ;)
<StevenK> benji: The mailing list
<gary_poster> right
<wgrant> PQM does spam canonical-launchpad@ if there's a conflict.
<wgrant> Lots.
<gary_poster> :-)
<benji> I got a message about it an hour ago to launchpad@lists.canonical.com
<wgrant> I resolved that one
<wgrant> I possibly should have replied, but it should be reasonably obvious that it's resolved if your inbox isn't flooding every 5 minutes :/
<gary_poster> heh
<benji> wgrant: awesome, thanks
<gary_poster> the only way it is not obvious is if you are not familiar with that aspect of the current landing machinery
<gary_poster> which seems conceivable
<wgrant> I resolved two conflicts this evening: the earlier was strucsub JS tests, which I'm not sure I did correctly, since I don't have Windmill working locally. You might want to check that out.
<wgrant> Heh.
<wgrant> Yeah, true.
<gary_poster> :-)
<gary_poster> ok thanks wgrant
<benji> then my question becomes, why isn't my revision in devel or listed on https://devpad.canonical.com/~lpqateam/qa_reports/deployment-stable.html
<wgrant> I'd EOD'd and mostly wanted the spam to stop, but it seemed OK.
<wgrant> benji: Which revision?
<wgrant> benji: deployment-stable.html should update within 1.5 hours of your rev passing buildbot, after it's on qas.
<gary_poster> benji, since those test additions are yours, could you take a glance as he suggests?  I'm going to run make an mp before Graham leaves
<benji> bzr+ssh://bazaar.launchpad.net/~benji/launchpad/add-edit-tests-2 r12775
<wgrant> It wasn't testfixed out?
<wgrant> Or conflicted out?
<benji> <shrug>
<wgrant> Check for PQM failure emails that are to you, not launchpad@
<jtv> bigjools, may I invite you to look at how the dists directories shuffle about in publish-ftparchive?  http://people.canonical.com/~jtv/publish-ftparchive-dists.png
<benji> I got a direct email about the conflict in lib/lp/registry/javascript/tests/test_structural_subscription.js
<wgrant> benji: That was probably rejecting your branch.
<benji> ok, so I need to freshen my branch from devel so I can resolve the conflict and then use bzr lp-land to get it in? (or do I need to do the whole ec2 land dance again?)
<wgrant> No point ec2ing it, since Windmill is turned off now anyway.
<wgrant> Is the branch targetted at devel or db-devel?
<benji> devel
<adeuring> abentley: r0me
<wgrant> benji: Merge, hopefully you'll see the conflict. But it will probably conflict when it tries to merge to db-devel.
<abentley> adeuring: marse1lles
<wgrant> If you're just adding tests, should be easy enough to resolve.
<benji> wgrant: got and fixed the conflict
<wgrant> Great.
<wgrant> Commit, push, lp-land, hopefully PQM won't hate you too much.
<wgrant> I shall expect a conflict in the morning.
<rvba> adeuring: Hi! Could you take a look at this MP? https://code.launchpad.net/~rvb/launchpad/dds-link-to-derivedseries/+merge/56931
<adeuring> rvba: sure
<rvba> adeuring: thx.
<henninge> abentley: did you run the one-off merging script? I can't remember.
<abentley> henninge: No.  I included it in the RT: https://portal.admin.canonical.com/45152
<henninge> abentley: ah no, that's not what I meant.
<abentley> henninge: what did you mean?
<henninge> abentley: wasn't there a merge run of already existing package links before you started coding the jobs?
<abentley> No.  Jobs were the first thing I worked on.
<abentley> Then I worked on a one-off script.
<abentley> I requested that one-off script to be run in RT 45152.
<henninge> abentley: ok, thanks
<LPCIBot> Project windmill build #151: STILL FAILING in 40 min: https://lpci.wedontsleep.org/job/windmill/151/
<deryck> adeuring, r=me, with some minor comments.
<adeuring> deryck: thanks!
<abentley> henninge: I determined that it didn't make sense to do the merge until the jobs were being created (or we would miss some Packagings).  And since the release, I've focused on JS instead of getting these scripts run.
<henninge> abentley: but the jobs only get created when new link are created. But what about the links that already existed when message sharing was introduced?
<abentley> henninge: That's what the one-off script fixes.
<henninge> ah
<henninge> sorry, misunderstood that.
<abentley> henninge: np
<henninge> abentley: so as of now we have virtually no upstream message sharing going on.
<henninge> until that script is run
<abentley> henninge: correct.
<henninge> dpm: ^^^
<henninge> dpm: we are in a transitional stage here. So some things might not yet work as expected because the data has not been migrated yet.
<henninge> dpm: but that would not explain what happened to synapatic.
<wgrant> henninge: What's happened? It's not importing POs from the package?
<henninge> wgrant: no, it somehow reverted to old (wrong) translations it seems.
<wgrant> Huh.
<henninge> bug 740153
<_mup_> Bug #740153: Translations skewed for synaptic <synaptic> <ubuntu> <upgrade> <synaptic:Triaged> <synaptic (Ubuntu):Invalid> < https://launchpad.net/bugs/740153 >
<bigjools> jtv: looked at the png.
<jtv> Enough colours?
<adeuring> rvba: could you please add unit tests for Distribution.derivatives and DistroSeries.detDerivedSeries()?
<rvba> adeuring: ok, will do.
<jtv> bigjools: This should help a lot with figuring out the various error paths.  One question I still have though: is $DISTSCOPYROOT/dists used for anything besides this flow?  Because if not, I'd favour rsync'ing in the new stuff _before_ starting to shuffle directories.
<adeuring> rvba: Especially, I'd be curious about the new clause "DistroSeries.distributionID != self.id" and the related clause in the distroseries class
<adeuring> rvba: other than that, your branch looks goos
<adeuring> good...
<rvba> adeuring: a series is a derivation if and only if it has a parent_series and the distribution of the parent_series is not the same as it's own distribution
<dpm> henninge, thanks.  And the other question: on https://translations.launchpad.net/synaptic/main/+pots/synaptic/ca/5/+translate - why did the "In Ubuntu" suggestion did not just get accepted if permissions are correct? Is this how it is supposed to work? I.e. translations done downstream will need to be reviewed and accepted again before they make it to upstream? - I understand that some things might not yet work as expected, I'm just trying to understan
<dpm> d the expected behaviour
<rvba> adeuring: hence the clause.
<adeuring> rvba: right. what I mean is: A test that this clause is necessary would be great
<bigjools> jtv: NFI.  :(
<rvba> adeuring: ok, got it.
<bigjools> jtv: this stuff is seriously crusty
<jtv> bigjools: the diagram also reveals that the cleanup routine really doesn't do anything once the done_pub point has been passed.
<bigjools> no
<wgrant> jtv: Do what you want with DISTSCOPYROOT.
<jtv> wgrant: thanks
<henninge> dpm: the expected behavior is: If you, being a translator for Catalan in both the Ubuntu package and the upstream project, then any translation you do in Ubuntu will be mirrored to upstream *if* they are identical already.
<henninge> dpm: if they are different, they will stay different.
<jtv> bigjools: "no it doesn't do anything" or "no you're totally wrong, you simpleton"?
<bigjools> jtv: sorry, was agreeing but in a terse way :)
<henninge> dpm: in that case "different" is that upstream is untranslated.
<bigjools> E_TOOMANYSIMULTANEOUSCONVERSATIONS
<jtv> bigjools: It's okay to say "simpleton"
<jtv> Ah, OK
 * jtv trundles off to apply his newfound insight
<henninge> dpm: Although that depends on what the Ubuntu translation was before that.
<henninge> dpm: do you know if it was empty, too?
<dpm> henninge, what's the rationale on having to review untranslated strings with translations coming from downstream? I thought that they'd flow in transparently to automate the process. This is no different than having global suggestions and having to click them
<dpm> henninge, I'm not sure what you mean by empty, it's untranslated upstream:
<dpm> https://translations.launchpad.net/synaptic/main/+pots/synaptic/ca/5/+translate
<henninge> empty = untranslated.
<henninge> dpm: that translation was done long before this mechanism came into place.
<henninge> dpm: I think that they are not linked yet.
 * henninge tries on qastaging
<henninge> dpm: they seem to be linked. I could change both sides by entering a translation upstream.
<henninge> dpm: which is the expected behavior when upstream is untranslated -> the first translation overides an existing Ubuntu translation
<dpm> henninge, yeah, I can understand that, but what's up with the untranslated behaviour? Should ubuntu translations not flow to upstream if the message is linked, upstream is empty and permissions are right?
<henninge> dpm: they should but it may be a bug. OTOH this is done when the translation is entered and this one was entered long before upstream message sharing existed.
<dpm> henninge, ok, yeah, that helps, I was just trying to understand the expected behaviour. Do you want me to file a bug on that one?
<henninge> dpm: well ...
<henninge> I am not sure.
<henninge> dpm: The situation is this: normally both the sourcepackage and the uspstream translation would start out untranslated.
<henninge> dpm: so they are identical
<henninge> dpm: someone with permission translates in Ubuntu and that gets mirrored to upstream.
<henninge> dpm: let me try that out real quick
<dpm> ok
 * henninge translate on some obscure untranslated language
<henninge> eo?
<henninge> :)
<henninge> nah, that has translations.
<henninge> nds?
<henninge> De snakt platt!
<henninge> ksh
<henninge> Yeah, nobody speaks KÃ¶lsch ... ;)
<henninge> dpm: nope, not working :(
<henninge> dpm: uh oh, tracking does not work ...
<LPCIBot> Yippie, build fixed!
<LPCIBot> Project windmill build #152: FIXED in 39 min: https://lpci.wedontsleep.org/job/windmill/152/
<dpm> henninge, oops
<henninge> dpm: yeah, it only works from upstrem to downstream.
<henninge> dpm: when upstream and Ubuntu are identical, an change in upstream will be reflected in Ubuntu.
<henninge> but not the other way round although I have permissions on both sides by virtue of being a rosetta admin.
<henninge> dpm: but it may well be that the check for admin priviliges is just missing.
<henninge> dpm: please file a bug so that we look at that more closely.
<dpm> henninge, yessir!
<dpm> thanks for your help
<henninge> including the behavior when untranslated message are involved.
<henninge> dpm: BUT
<henninge> dpm: if upstream is untranslated and Ubuntu is translated then we could assume that upstream chose no to accept the Ubuntu translation.
<henninge> dpm: we might want to keep them different then.
<henninge> dpm: OTOTH that situation would also arise if somebody who is only an Ubuntu translator entered the first translation in Ubuntu.
<henninge> dpm: In that case an override by the might be advised. But we can't tell from the situation.
<henninge> In general it might be a better idea to be mare accepting then rejecting about translations.
<dpm> henninge, yeah, I'd second that in this particular case (i.e. being more accepting)
<henninge> How do I run javascript unit tests?
<sinzui> henninge:  --layer=Windmill to explicitly add that layer to the testrunner
<henninge> hm, RegistryWindmillLayer just tried to came up but died horribly ...
<wgrant> henninge: Doesn't work on Natty at the moment.
<wgrant> (our mozrunner doesn't like Firefox 4)
<henninge> wgrant: ah yes, that's what it looked like
<henninge> thanks
 * henninge will have to let ec2 sort it out.
<wgrant> henninge: ec2 doesn't run Windmill any more. But Hudson will run them in a separate job once it lands.
<henninge> argh
<henninge> wgrant: thanks
<sinzui> That reminds me that I want to get pocket-lint to use seed since most users will have that installed
<henninge> sinzui: can you still run windmill tests?
<henninge> i.e. no natty yet?
<sinzui> I have been on natty since 2010
<sinzui> There was a very painful day with no menus
<henninge> ouch
<henninge> oh well, I don't expect to have broken anything anyway.
 * henninge goes lp-propose
<sinzui> henninge: do you need to test windmill page integration or do you want to test yui unittests
<henninge> sinzui: yui unittests AFAICT
<sinzui> henninge: yui unit tests work so I just open that page to verify they pass
<henninge> sinzui: the html page that goes with the test?
<sinzui> henninge: you can open the test page (the actually harness) in any browser
<sinzui> henninge: that is right
<henninge> cool
<sinzui> it is faster than the test suite startup too
<henninge> passed!
<henninge> thanks sinzui
<sinzui> henninge: I am glad to be of service. I write js with that test page open. That lets me do test-driven development
<benji> bac: I have some structural subscription JS test sprucing up for review: https://code.edge.launchpad.net/~benji/launchpad/fix-ss-test-lint/+merge/56974
<bac> benji: on it
<benji> thanks
<bac> benji: approved with my condolences.  thanks.
<rvba> adeuring: I spoke with bigjools about this ... and I've added a test (the test for getDerivedSeries existed already) ... but I deliberately did not check for the condition (;-)) ... because of 754750. We need to refactor the whole parent/child modelling and deal with existing wrong data. So you're right, the condition should not be needed, but it is for now because of this wrong data (i.e. parent/child relationship where none should b
<rvba> e there).
<benji> bac: heh :)
<rvba> adeuring: I'll have to EOD soon (and so do you I guess) so maybe you can check it out next week if it's ok with you.
<adeuring> rvba: np Ii understand the issue with bug 754750. Let me just have a look at the diff again
<_mup_> Bug #754750: Distroseries' parent_series attribute is misleading. <derivation> <Launchpad itself:New> < https://launchpad.net/bugs/754750 >
<adeuring> rvba: r=me. thanks for the additional test and the comments!
<rvba> adeuring: great, thanks for spotting this. Have a nice we!
<adeuring> rvba: thanks! have a nice weekend too!
<henninge> adeuring, bac: Is either of you up for a JS review?
<adeuring> henninge, bac: I am a bit tired (worked quite long yesterday...) so... bac?
<henninge> adeuring: time for the weekend, I guess. ;)
<adeuring> henninge: yeah :)
<sinzui> bac: will you have time to review https://code.launchpad.net/~sinzui/launchpad/mlist-sync-0/+merge/56468
<sinzui> bac: do not hesitate to ask questions if you do. It took me a few days to understand what the original test was trying do
* adeuring changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: bac | https://code.launchpad.net/launchpad-project/+activereviews
<SpamapS> lifeless: interesting about rabbit. I'm starting to think the only sane way to run it on a roaming machine is inside a vm/container
<bac> sinzui: yes, i'll look now
<LPCIBot> Yippie, build fixed!
<LPCIBot> Project devel build #622: FIXED in 5 hr 23 min: https://lpci.wedontsleep.org/job/devel/622/
<bac> hi sinzui can you explain what test_staging_sync_list_without_team is meant to do?
<sinzui> The team was deleted, the mlist and archive are still on the system
<sinzui> the mlist+archive do not create an oops
<renata> hello everybody! I am new at working with launchpad.. I am starting to work with launchpadlib and the API. I was wondering if anyone could tell me if there is a specific API function to access distribution's packages.
<sinzui> ^ bac
 * bac looks
<renata> for example, I would like to obtain all ubuntu packages..
<renata> is it possible to do so only by giving a release name like "natty" ?
<bac> sinzui: so the fact we only see team-1 means the other is not there...therefore the test passed
<sinzui> renata: https://launchpad.net/+apidoc/devel.html distribution has two methods to get source packages,distro series has some too
<sinzui> bac: correct
<sinzui> bac: I can update the test to state that mlists+archives without a team are ignored
<bac> sinzui: yeah, some sort of explanation would be good.  most people reading that test aren't going to have a clue, assuming someone besides you ever reads it again.
<bac> sinzui: and you have the most interesting typo on the copyright year!
<bac> usually it is an off by one error, not an order of magnitude
<sinzui> renata: source package releases and binary package releases are not supported. You can get source_package_publishing_history from the source_package to learn about versions
<sinzui> bac: I claim the cough syrup made me do it
<bac> sinzui: in other tests on ec2 we've seen spurious failures with shutil.rmtree
<sinzui> hmm
<renata> sinzui tank you! I was just trying to find a list of packages to a certain distribution. I think that may do it :)
<bac> sinzui: would it be safe to add the 'ignore_errors' option
<bac> sinzui: the other tests fail when the dir is not empty
<sinzui> bac: I will look into it. The other tests will not fail of data is left behind. since each is in their own tmp dir
<sinzui> oh, well the archive setup is not in tmp, so I will add the 'ignore_errors option
<renata> sinzui, thanks a lot. your information was very useful!
<sinzui> np
<benji> bac: I have another JS MP for you, this one a bit shorter and with slightly more meat: https://code.edge.launchpad.net/~benji/launchpad/bug-750573-move-overlay/+merge/56999
<bac> benji: ok
<lifeless> benji: hi
<lifeless> benji: I'd like pointers at how the api server side glue goes about making a batchnavigator for a collection
<lifeless> benji: I want to glue lazr.batchnavigator 1.2.3's shiny new range_factory in
<benji> lifeless: hmm, let me see if I can come up with some pointers
<bac> benji: thanks for the fixes.  however, when i click on the 'submit' i still see a two-stage disappear on the overlay
<bac> benji: it is working for you w/ff4?
<lifeless> benji: if what I'm talking about sounds mysterious, I can point you at the BN changes, or describe the concept directly
<benji> bac: yeah, it looks fine in FF4 and Chromium; are you seeing the overlay dissapear and then the filter controls or the other way around?
<benji> unfortunately I think we have to choose one or the other, and I /think/ the former is slightly better
<bac> benji: other way.  when fully opened, the accordion, et al, go away leaving a smaller overlay which then disappears
<benji> lifeless: I believe I understand the root of your request.  You've been working on removing the explicit start indices from batching requests.
<bac> benji: i see the same with FF4 on os x.
<benji> bac: darn, that's what I was trying to eliminate; I may need to find a way to run this on FF3 so I can reproduce
<bac> the cancel button causes it to go away cleanly but submit shows as above
<lifeless> benji: yes, well, adding data so the query doesn't use OFFSET; the start index still stays for cosmetics
<benji> bac: oh! I can fix that, one second
<benji> bac: ok, it'll take more than a second, I'll let you know
<benji> lifeless: I think lazr/restful/_resource.py line 640 is what you're looking for
<lifeless> great
<lifeless> benji: now, leonard said that in url generation
<lifeless> just changing the next/prev links should be enough
<lifeless> or do I need to change some compilation metadata too ?
<benji> lifeless: That's a good question; I don't know off the top of my head.  Let me see if I can find anything out real quick.
<benji> lifeless: It looks like just changeing the batch links should work.
<lifeless> benji: wicked
<benji> bac: I believe I've fixed the two-step overlay hide problem, the diff is up to date on the MP
<bac> benji: cool, grabbing it now
<bac> benji: works great!
<benji> cool!
<cr3> hi folks, what's the launchpad practice for naming classes containing acronyms? HTTPFoo or HttpFoo?
<lifeless> yes
<lifeless> I suspect we have both
<lifeless> If I was guessing - e.g. to grep, I'd expect HTTP
<cr3> lifeless: grepping around has indeed returned both practices, which is understandable for such a large code base, any reason to use one practice over the other?
<cr3> it just occured to me, which is strange since I've been doing this for a while, that two acronyms in a row would suck with all caps: APIWSGIWTF
<lifeless> I think changing the case on acronyms is more surprising than not
<lifeless> OTOH theres little excuse for more than ~ 2 classes with the name of a given standard in them
<lifeless> [outside of the actual implementation of said standard]
* bac changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: - | https://code.launchpad.net/launchpad-project/+activereviews
<lifeless> sinzui: still here?
#launchpad-dev 2011-04-09
<maxb> Argh
<maxb> Why is launchpadlib so utterly careless about API compatibility ?!
<wgrant> Yes, it's pretty bad :/
<wgrant> But it's too late now :/
<wgrant> It is a little disappointing that it goes too far with maintaining compatibility of the exposed API, but the launchpadlib API itself is unstable.
#launchpad-dev 2011-04-10
<lifeless_> moin
<lifeless> _thumper_: around ?
<_thumper_> lifeless: whazzup?
<lifeless> thumper: hi
<lifeless> thumper: there is a pretty severe regression in oops reports in the API
<lifeless> thumper: I'm wondering if you can tackle it, as I suspect you know the most about what leonardr was working on
<thumper> what type?
<thumper> well... not that much
<thumper> most of the api work is known by gary and benji
<lifeless> bug 740750
<_mup_> Bug #740750: API timeouts broken and returning no useful data... <regression> <Launchpad itself:Triaged> <lazr.restful:Triaged> < https://launchpad.net/bugs/740750 >
<lifeless> this was landed right before he finished :(
<lifeless> basically, the timeouts we get from the API have had a new timeline object created before they are serialised, so we don't see /any/ data about what happened.
<lifeless> this makes it very hard to debug api timeouts (or other oopses from the api)
<thumper> lifeless: before I spend time on it i'd rather at least talk to one of gary or benji to see if they know more in this area, as I'm not familiar with it at all
<lifeless> thumper: sure - its still important tho :)
<lifeless> if that makes sense
<thumper> sure
<mwhudson> errr
<mwhudson> i'm trying to move a bug to the launchpad project
<mwhudson> if i type 'launchpad' into the project search box, I get "Too many matches. Please try to narrow your search."
<lifeless> mwhudson: use the expander and type it in
<mwhudson> is this already reported?
<lifeless> mwhudson: bug 722455
<_mup_> Bug #722455: bug task target selector widget cannot be used to select 'launchpad' <regression> <Launchpad itself:Triaged> < https://launchpad.net/bugs/722455 >
<mwhudson> lifeless: thanks
<thumper> mwhudson: yeah, that blows doesn't it
<mwhudson> sure does
<thumper> yay... email processed
<lifeless> we had three dupes
<lifeless> marked up and the oldest clarified
#launchpad-dev 2012-04-02
<wallyworld_> there's a lot of memcache errors
<wgrant> Indeed.
<wgrant> Fortunately the access logs are free of such silliness.
<wgrant> The 9138 librarian.logs are a nice touch, though.
<lifeless> wgrant: phew
<lifeless> wallyworld_: ^
<lifeless> thanks for lookig guys
<wallyworld_> the chances would have been very small given no-one knew about the page anyway
<lifeless> wallyworld_: yes. Sadly we have journos that follow us stealthily looking for things like this.
<lifeless> journos and others, but the journos we eventually find out about ;)
<wallyworld_> do they work for news corp or something?
<wallyworld_> i suppose they tap our phones also
<wgrant> Nah, they're too busy pirating Foxtel and Austar, aren't they? :)
<lifeless> wallyworld_: I detect skepticsm.
<lifeless> which is fine, I don't care if you're skeptical or not. Just as long as you *are* paranoid. :>
<wallyworld_> lifeless: none intended. poor attempt at humour
<lifeless> wallyworld_: kk
<wgrant> http://morecss.org/
<wallyworld_> wgrant: bug 933839. to me that means revoke all direct grants (pillar and artifact) for team members as well as the team. whereas what it does now is just revoke the team but leaves any member's direct access in place. agree?
<_mup_> Bug #933839: Share nothing with a user or team <disclosure> <job> <Launchpad itself:Triaged> < https://launchpad.net/bugs/933839 >
<wgrant> wallyworld_: Nobody knows how that UI looks, AFAIK
<wgrant> It can't be implicit.
<wgrant> I may have missed some mockups, though.
<wallyworld_> wgrant: there would be a new radio button otpion
<wallyworld_> atm there is "All" "Some" "Nothing" and there would be "Nothing (revoke team)"
<wallyworld_> i'm just checking i understand the Nothing (revoke team) behavour
<wgrant> All | Some | Nothing (except not really) | Nothing (yarly)?
<wgrant> Right.
<wgrant> It needs to remove explicit APGs and AAGs for the team, and any APGs or AAGs where the grantee participates in the team.
<wgrant> This is pretty handy.
<wallyworld_> yes, that matches my understanding
<wgrant> 'cause it means that I, as a random outside observer, can get everyone's access revoked :)
<wallyworld_> how? you need to be a project owner to do it
 * wgrant creates a team and subscribes it to a project's private bug.
 * wgrant quietly adds the project team to that team.
<wgrant> $projectowner says "wtf is this team doing in my sharing view, you should have nothing"
<wallyworld_> it would only be direct team members afaiui
<wgrant> Project team no longer has any privileges :(
<wallyworld_> so if it's only direct team members that wouldn't work
<wgrant> But considering only direct team members puts it in conflict with literally everything else that deals with teams in Launchpad.
<wallyworld_> perhaps, but it's the only sane way to do it
<lifeless> Wait, why would it remove transitively?
<wallyworld_> it wouldn't, that's what i'm saying
<lifeless> rephrase, why would it remove more than the grants that mention that team|person directly ?
<wgrant> lifeless: The design calls for a way to revoke a team's access and access held by any of the team's members/participants.
<lifeless> why does it want to do that ?
<wallyworld_> s/design/requirements
<wgrant> I do not know.
<lifeless> wallyworld_: can you point me at the LEP ?
<wallyworld_> just a sec, i find it
<lifeless> I suspect its an overgeneralisation from one of the oem scripts
<wallyworld_> https://dev.launchpad.net/LEP/ManagingDisclosure
<lifeless> so, Constraints and Requirements/must/4 is the thing
<lifeless> its got some nuance in there
<wgrant> It doesn't explicitly say that it requires this.
<wallyworld_> the mockups and screenshots they were based on did
<wgrant> It is the private team requirement, however.
<lifeless> wgrant: sorry, where is that ?
<wgrant> lifeless: Requirements/must/4 is the reason the private team expansion is done.
<wgrant> It's probably also the reason transitive revocation is done, but it doesn't explicitly require that.
<lifeless> wgrant: the private team expansion does not follow from 4 as I read it. Can you help me understand ?
<wgrant> "If the user has access via a team, the view must explain how to remove either the user or the team to unshare. "
<wallyworld_> i think item #1 explains why private team expansion is required
<wgrant> That implies that there is a way to see the user who is only participating in sharing by being a member of a team.
<wgrant> Ah, true.
<wallyworld_> a project owner must see who has access
<lifeless> wgrant: (4) - you can enter a userid and see their teams; this is quite different to expanding the teams.
<wgrant> lifeless: The leakage implications are only vaguely dissimilar.
<lifeless> wgrant: there is room to mitigate on one side, not on the other. I agree it has serious complications though.
<wallyworld_> so i won't start this today - we can discuss tomorrow at the standup
<lifeless> on (1), when I reviewed the LEP I read that as 'person', not as implying every concrete user.
<wgrant> 1. A lists of all the users with all or some things shared.
<wgrant> The view must also show exactly how the user has access, for example: directly or via a team
<lifeless> quoting the LEP doesn't help much.
<wgrant> That pretty clearly implies that it must be transitive.
<lifeless> yes; its also a major problem
<wgrant> Certainly.
<wallyworld_> lifeless: wgrant: whenever the team has discussed this, we've said that if a private team is granted access to a project via a policy, they are putting themselves in a position where their info has a right to be known to the project owner/drivers and so long as they are warned....
<wallyworld_> we've already done this for subscribing private teams to bugs for example in the current access model
<wgrant> wallyworld_: The only previous disclosure was LimitedView
<wgrant> This is effectively full View
<wgrant> It's not warned about, and is in many cases far worse.
<wallyworld_> not warned about yet
<wgrant> And we need to remove bug-retargetting and multi-pillar bugs.
<wallyworld_> and it's only to project owner/drivers
<wgrant> The latter of which was vetoed.
<wallyworld_> and as a project owner, i have a right not to have people spy on my project
<wgrant> Right,.
<lifeless> wallyworld_: you need to reconcile that with 'as a private team owner, I have a right not to disclosure the membership of my team'
<wallyworld_> lifeless: well, if you want to see someone else's project, then you must be prepared to tell them who is lookiing
<wallyworld_> so if a private team is granted access to *my* project, i have every right ti see who they are
<lifeless> wallyworld_: thats true; but is it the team or the members that are looking ?
<wallyworld_> both
<wallyworld_> as a member of a team with access, i have access also
<wallyworld_> if that's not acceptable, then don't subscribe your private team to a public project
<wgrant> So
<wgrant> That argument would potentially hold water if Launchpad wasn't crap.
<wgrant> But, as I went into slightly with mpt earlier, bugs are too unowned for that to work.
<wgrant> This whole thing would be far easier if Launchpad wasn't gratuitously different from every other bugtracker in history :(
<lifeless> wallyworld_: that approach cuts both ways: if you don't trust the admins of ~private-team, don't grant access to ~private-team to your project.
<wallyworld_> that's true
<lifeless> wallyworld_: Mitigating both those risks results in a system where you get limited view in both directions :
<lifeless>  - access grants grant reflexive limitedview on the team (and only that)
<wallyworld_> sounds like there's slight disagreement here when we've already embarked on an approved design :-(
<lifeless> if one wants to be sure that ~someteam is managed well, you can talk to its owner, which limitedview lets you see - and that has to be transitive (clearly)
<lifeless> well, I say clearly; I think it is.
<wallyworld_> i think so
<lifeless> you can walk up until you get a person you know.
<StevenK> wallyworld_: We missed one call in terms of the privacy banner, but I have that working.
<wallyworld_> StevenK: excellent!
<wallyworld_> StevenK: so you are now a yui guru :-)
<StevenK> wallyworld_: One thing I can't work out is on initial page load, clicking the link the overlay doesn't have a value set, but if I choose a value and then click the link again, it does.
<wallyworld_> StevenK: the current value needs to be passed into the choicesource constructor
<wallyworld_> so check the value: xxxx bit
<StevenK> wallyworld_: value: information_type.value,
<wallyworld_> and you've logged that just to be sure it is correct?
<StevenK> Not as yet ...
<wallyworld_> take a quick look just to be sure
<StevenK> information_type.value is undef. That would explain it.
<wallyworld_> StevenK: the value should be USERDATA etc, not 2, 3, 4 etc
<StevenK> And doing it again involves the same object and it keeps track of what was selected.
<StevenK> wallyworld_: So I'm getting a hold of the element correctly, but how can I return what it is currently showing? .value was my guess, and it's obviously wrong.
<wallyworld_> StevenK: you are trying to read the dom value that is not set yet
<wallyworld_> you need to seed the initial value from the model
<wallyworld_> ie the json request cache
<StevenK> Ah.
<StevenK> Back to the Python
<StevenK> wallyworld_: You have it backwards. It needs to be 3, not EMBARGOEDSECURITY.
<wallyworld_> StevenK: in the sharing table widget, it does need to be USERDATA etc. i guess we are using slightly different data models
<StevenK> Now I'm happy enough with the JS.
<StevenK> Now to stab Zope's form handling.
<wallyworld_> it would be nice to be consistent though
<lifeless> StevenK: is today auditor day?
<StevenK> lifeless: I was still deciding if I was going to do something on it, what did you have in mind?
<lifeless> just touching base to make sure you're not blocked
<StevenK> Not blocked, just feeling that the light at the end of the tunnel is an oncoming train.
<StevenK> wgrant: How can I debug why information_type doesn't show up in +filebug, assuming that I've ripped out security_related everywhere I saw it in the browser code.
<StevenK> lifeless: "Overwhelmed" would be a nicer way to say it.
<lifeless> StevenK: w.r.t. auditor ?
<StevenK> lifeless: For both auditor and this disclosure work -- but the difference between the two is I have a clear direction, a clear goal and the steps in terms of the current disclosure work.
<lifeless> StevenK: well, if you want those things for auditor, I'd be happy to help make them up.
<StevenK> lifeless: I think that will help. :-)
<lifeless> I'm available whenever you like to do that then.
<StevenK> lifeless: After lunch?
<lifeless> I don't know what that means, but sure :)
<StevenK> lifeless: 12:40 here, I'll be having lunch for ~30 at 1pm.
<StevenK> Read as 'in a little bit over an hour'
<nigelb> lifeless doesn't have lunch? lunchless?
<StevenK> Haha
<StevenK> mealless
<mwhudson> i bet lifeless can charge from usb
<nigelb> haha
<StevenK> I'm not sure I want to know where his charging indicator is.
<nigelb> tmi
<lifeless> StevenK: col
<lifeless> *cool*
<wgrant> wallyworld_, jcsackett: Bug #971241
<_mup_> Bug #971241: Sharing details page breaks when an inaccessible bug is shown <disclosure> <Launchpad itself:Triaged> < https://launchpad.net/bugs/971241 >
<wallyworld_> hmm. one could argue that a pillar driver should be able to see bugs filed against it
<wgrant> Most certainly not.
<wgrant> Owner perhaps.
<wgrant> But there is no justification for drivers to.
<wallyworld_> why not drivers?
<wgrant> (even owner is debatable)
<wallyworld_> i don't agree owner should be debatable
<wallyworld_> if i own a prject, i have a right to see what bugs are filed against it
<wgrant> wallyworld_: Because being someone who manages blueprint priorities shouldn't permit me to see embargoed security bugs.
<lifeless> wgrant: careful of cause and effect
<wallyworld_> but what if that info is required to help manage the priorities
<lifeless> wgrant: ITYM 'Being someone who manages blueprint priorities does not imply seeing embargoed security bugs.'
<wgrant> lifeless: Perhaps.
<lifeless> wallyworld_: then you will have the ability to see those bugs separately.
<lifeless> wallyworld_: e.g. via a grant to see them.
<wgrant> Now, the owner can always grant themselves access. But I think having the separation is useful for large projects.
<lifeless> wallyworld_: our job is to provide solid primitives that let folk drive the system.
<wallyworld_> so the sharing details page should just filter out bugs without view access i guess
<lifeless> (vs preconcluding particular detailed answers). We're writing a fine grained rules toolkit here.
<wgrant> wallyworld_: It probably needs to say that they exist.
<wgrant> But it can't say what they are.
<lifeless> it sounds like a case of limitedview
<wgrant> LimitedView considered harmful.
<lifeless> same as duplicate-on-private; see the thing exists but no details.
<wgrant> Indeed.
<lifeless> why do you say limitedview considered harmful ?
<wallyworld_> i'm not familiar with the implementation details of the sharing details page as i didn't write it, i'd have to look
<wgrant> lifeless: Well
<wgrant> lifeless: In a lot of cases it's difficult to do sensibly.
<wgrant> In some cases it works well.
<wgrant> But it's not the answer to everything :)
<wgrant> Anyway.
<wgrant> Drivers seeing them unconditionally is not open for debate. They must not.
<wgrant> Owners I suspect can't always see them either, but that is more arguable.
<wgrant> (eg. it's much easier to audit leaks if the owner has to explicitly add themselves in a logged fashion before they can see the embargoed security stuff)
<wallyworld_> i feel sorry for drivers trying to managing priorities in the absence of all required information eg there could be a really important embargoed security bug filed they don't even know about
<wallyworld_> i guess if the had limited view and knew the bug number they could ask
<lifeless> what does the 'sharing details page' do ?
<wallyworld_> show the artifacts (bugs and branches) a person can see
<wallyworld_> for a pillar
<wgrant> lifeless: https://qastaging.launchpad.net/launchpad/+sharingdetails/wgrant
<lifeless> k
<wgrant> wallyworld_: The Ubuntu drivers are not privy to the security bugs.
<wgrant> wallyworld_: The security team manages the security bugs.
<wgrant> From end to end.
<wallyworld_> ok
<lifeless> wallyworld_: so remember the user story: for proprietary projects (the focus), drivers could be either a) granted access to all proprietary bugs or b) granted access to all proprietary+security bugs
<lifeless> wallyworld_: thats a project policy decision, not ours.
<wallyworld_> true
<lifeless> wallyworld_: in this case, given that anyone can check +sharingdetails, the first iteration probably should just elide inaccessible bugs.
<lifeless> wallyworld_: (which it would have by default if the stock bug access query constraints were used)
<wallyworld_> and branches
<lifeless> indeed
<wallyworld_> lifeless: it wouldn't use the stock bug access queries because it would be done of the flattened table
<wallyworld_> off
<lifeless> we can look at adding limitedview onto this in the future, but it will run into the same complexity-performance things we have today.
<wgrant> wallyworld_: I would use the same queries.
<wgrant> s/I/It/
<wgrant> Just with an additional constraint to say "wgrant has access"
<wallyworld_> looking at the code, it uses findArtifactsByGrantee
<wgrant> Sure
<wallyworld_> so right now it queries directly off the flattened table
<wgrant> From that it could grab a list of bug IDs
<wgrant> And feed them into bug search.
<wallyworld_> agreed, just saying what it dies now
 * wallyworld_ has to go and get a haircut
<StevenK> lifeless: Skynet?
<StevenK> wgrant: Did you miss my earlier question?
<wgrant> StevenK: Indeed. Do you have a branch?
<StevenK> wgrant: Locally. I can paste a diff, but it's 4 files changed, 103 insertions(+), 284 deletions(-)
<wgrant> diffme
<lifeless> StevenK: yup
<StevenK> wgrant: http://pastebin.ubuntu.com/911045/
<wgrant> StevenK: That's quite the obese diff you have there.
<StevenK> I told you!
<wgrant> Also, --syntax=diff :)
<lifeless> StevenK: waiting for you to come online
<StevenK> lifeless: I have been for the last ten minutes.
<lifeless> StevenK: bah
<StevenK> In fact, I was starting Skype when I queried you with Skynet
<lifeless> skype fail fail fail
<lifeless> pkill -9 skype
<StevenK> lifeless: Book a flight to Sydney? It might be quicker and simpler.
<lifeless> foad
<lifeless> though I may come over mid april
<nigelb> StevenK: heh
<wgrant> StevenK: What does not working?
<wgrant> I see the field fine.
<StevenK> wgrant: The information type doesn't show up in Bug:+filebug
<wgrant> Oh
<wgrant> On filebug
<wgrant> I see
<wgrant> StevenK: Works better when the template doesn't refer to security_realted
<wgrant> security_related
<StevenK> wgrant: Which template? I didn't see that.
<wgrant> StevenK: The filebug template? :)
<wgrant> lib/lp/bugs/templates/bugtarget-filebug-guidelines.pt
<StevenK> Right. I was looking at the filebug.pt I think.
<bigjools> wallyworld_: which hair?
<wgrant> StevenK: Also, numeric enum values have no place outside the DB access layer.
<wgrant> But it looks like you might be using one here in the JS.
<wgrant> Hard to tell.
<StevenK> wgrant: Where am I using them?
<wgrant> +            var information_type_by_value = {};
<wgrant> But it seems to be by name, not value.
<wgrant> So it's just a lie, and not wrong.
 * jtv hopes wgrant's mother didn't hear that
<jtv> lifeless: I just ran into a funny one with oopstools' date-dir repo.  Hope I filed it in the right place: bug 971255
<_mup_> Bug #971255: Crash in _findHighestSerialFilename <Python OOPS Date-dir repository:New> < https://launchpad.net/bugs/971255 >
<StevenK> wgrant: I get the number back, I need to map it back.
<wgrant> StevenK: Whatever is returning the number is a horrible person.
<wgrant> And a bug.
<lifeless> jtv: thats probably the right place, though that method indicates you're using the old naming scheme which is awful
<jtv> lifeless: I don't think we made a conscious choice to do that â isn't this the default?
<jtv> lifeless: also, we happen to have other log files in the directory.  Their names happen to have dots in them, but not something to rely on.
<lifeless> jtv: I've commented on the bug; happy to discuss more here, or there, or both as appropriate.
<jtv> lifeless: thanks for the fast response.  I just summarized your message on there as well.
<jtv> lifeless: where do I find out how we constrain our repository object?
<jtv> Sorry, constrict.
<jtv> No idea what it means actually.  :)
<lifeless> construct
<lifeless> jtv: ^ sorry, ELOCAL for a minute there
<jtv> Sure.
 * wgrant shreds BeautifulSoup
<jtv> lifeless: We pass it an error dir and an instance id.
<lifeless> don't pass an instance id
<lifeless> pydoc oops_datedir_repo -
<lifeless>      |      :param instance_id: If None, OOPS file names are named after the OOPS
<lifeless>      |          id which is generated by hashing the serialized OOPS (without the
<lifeless>      |          id field). Otherwise OOPS file names and ids are created by
<lifeless>      |          allocating file names through a UniqueFileAllocator.
<lifeless>      |          UniqueFileAllocator has significant performance and concurrency
<lifeless>      |          limits and hash based naming is recommended.
<lifeless> (easier to have it here to reference)
<wgrant> jelmer: Can bug #296153 be closed now?
<_mup_> Bug #296153: does not mirror bzr branches over ftp <lp-code> <qa-ok> <Launchpad itself:Fix Committed by jelmer> < https://launchpad.net/bugs/296153 >
<jtv> lifeless: thanks!  I guess they'll still have timings in them to order them by time?
<lifeless> jtv: if you have the default hooks installed there is a timestamper in there,yes.
<jtv> Is there anything I need to do to make sure we have the default hooks installed?
<wgrant> (the filenames don't, though)
<lifeless> jtv: I urge you to make an oops/ subdir or something though, because oops isn't designed to have other stuff in the repository, and you may well encounter other bugs like this. The robustness that is there is to deal with oops-managed temporary files, for instance.
<jtv> I see.  Oh well.  Thanks for the quick help!
<lifeless> jtv: (defaulthooks) - they are default :) You have to take action to remove them.
<jtv> OK
<jtv> Laziness is the most reliable safety mechanism.
<lifeless> jtv: so based on this, I'm going to close the bug (or do you want to retarget it to maas)?
<jtv> Wait â it still shouldn't crash just because we passed a non-recommended option, right?
<lifeless> the only way it can get crud like this in place is if you combine its own storage area with other content
<lifeless> its not designed to do that, adding code to handle it would just be cruft, unless we want to design it to handle htat.
<jtv> But the problem content was generated by the datedir repo.  The other crud was incidental.
<lifeless> jtv: 'directory' in that call should be a date-named subdirectory
<lifeless> jtv: not the explicit path you passed in.
<jtv> I wonder why it wasn't.
<jtv> Oh wait, it was!
<jtv> So it was purely the datedir repo that broke itself.
<lifeless> jtv: one possibility is that you're constructing a datedirepository without an instance id somewhere else
<lifeless> (perhaps an earlier version on your own machine)
<lifeless> that would write oopses with hashed names
<lifeless> (but in the subdirectory)
<jtv> Yup!  Found it.
<lifeless> jtv: whats the actual name it is choking on ?
<lifeless> jtv: again though, this isn't a supported config, I don't see value in trying to handle it
<jtv> I don't know what name it was choking onâ¦ give me a minute to set things up for breaking again.
<lifeless> okidoki
<jtv> We only found out that this was not a supported config by asking you.
<lifeless> if it is an interaction between the deprecated naming system and the new, that can be fixed I think by deleting the old style logic
<lifeless> we shouldn't need it anywhere now, it was purely for migrationary purposes.
<lifeless> and this bug would indeed be evidence that we need to do that.
<jtv> Missed a step.  Back to step 1.  :/
<StevenK> wgrant: Right, that sorted that out. But the default is "(no value)" rather than Public.
<wgrant> StevenK: Indeed, you'll need to change the default.
<wgrant> And ensure that the interface forbids None.
<StevenK>     information_type = EnumCol(
<StevenK>         enum=InformationType, notNull=True, default=InformationType.PUBLIC)
<wgrant> That's no interface, that's a column.
<StevenK> Doh
<jtv> lifeless: gah.  Can't reproduce.  I have no way of telling what file was breaking it earlier.
<jtv> Ah no, maybe I need to start pserv.
<jtv> lifeless: having trouble reproducing the problem, because pserv seems to disappear every time I start it.  This needs more investigation.  But what may be happening is that one process (which constructs its datedir repo without an instance id) oopses out and then another tries to log an oops to a repo on the same directory but with an instance id.
<lifeless>  yes, that would make sense to me
<lifeless> I think we should remove the old code at this point
<lifeless> the only reason to keep it was un-upgraded analysis consoles
<lifeless> of which there are none that will be migrating to the new codebase now; (u1 are upgraded, isd are experimenting with sentry)
<jtv> Meanwhile, I don't think the filenames are timestamped.  :-(
<lifeless> jtv: the metadata is inside it
<lifeless> jtv: you can also capture the oopses directly, either by subscribing to them via rabbit, or using a test specific publisher.
<lifeless> jtv: (depending on your test environment)
<jtv> My rabbit wasn't running earlier.
<lifeless> (and you may not have a rabbit publisher setup either, but you can do so pretty easily)
<lifeless> there is a rabbit-oops-fixture in the LP code base that can probably be pulled into oops-amqp (with an optional dep on fixtures)
<jtv> What kind of usage would I get out of catching them that way?
<lifeless> you avoid the ten sorts of mess that happen by going and looking on disk after the fact
<jtv> For now I think it'd be good to minimize rabbit dependencies, given the recurring shutdown problems.
<lifeless> I have to go cook
<jtv> Don't let me stop you; I know what to fix.  Thanks again.
<lifeless> but if you want a minimal replacement, you can use repo.republish() to pickup all the oopses that are in a datedir repository and throw them at another publisher
<lifeless> so
<lifeless> make a new repo object
<lifeless> oopses = []
<lifeless> repo.publishers.append(oopses.append)
<lifeless> repo.republish()
<lifeless> # make assertions about the oopses in 'oopses' here
 * lifeless will be back later
<wgrant> wallyworld_, StevenK: https://code.launchpad.net/~wgrant/launchpad/wbr-is-void/+merge/100357 is pretty simple if someone has a sec.
<wallyworld_> i can look
<wallyworld_> wgrant: i always thought it was simply <wbr>
<wgrant> wallyworld_: In HTML, yes.
<wgrant> wallyworld_: But that's not well-formed XML, and lots of people parse us as XML.
<wgrant> <wbr /> is well-formed XML, XHTML and HTML5.
<wgrant> And it's valid HTML5 and XHTML5.
<wgrant> <wbr></wbr> is only well-formed XML and XHTML, not well-formed HTML, and not valid in any of them.
<StevenK> wgrant: Except that BeautifulSoup doesn't think so?
<wallyworld_> ok
<wgrant> Yeah, BeautifulSoup has an outdated hardcoded list of void elements.
<wallyworld_> so do any tests fail that use BeautifulSoup?
<wgrant> Every other element it assumes the end tag is missing if it's self-closing.
<wgrant> wallyworld_: Just the one. I replace it with <wbr></wbr> before passing it into BeautifulSoup.
<wgrant> Search for BeautifulSoup to find the test.
<lifeless> that seems like an odd behaviour
<wallyworld_> wgrant: can you please add an XXX then
<wgrant> lifeless: It's designed to deal with tag soup.
<wgrant> lifeless: So it's sort of justified.
<wgrant> If a non-void element doesn't have an end tag, it should probably assume the end tag is just missing.
<StevenK> wallyworld_: It isn't really an XXX.
<wgrant> StevenK: It is.
<lifeless> wgrant: <foo /> ... </foo> - where would that ever make sense ?
<wgrant> lifeless: I don't think HTML knows about <foo />
<wgrant> Does it?
<wgrant> I'm pretty sure that's just an XML thing.
<wgrant> Regularly used in HTML to make a polyglot document.
<wallyworld_> wgrant: is the space before the /> required? my eyes prefer <wbr/>
<wgrant> But only on void elements, because otherwise the parser cries.
<lifeless> wgrant: it does for foreign elements
<lifeless> http://www.w3.org/TR/html5/syntax.html
<wgrant> lifeless: That's for HTML5.
<wgrant> Which is a bit more sensible.
<wgrant> But let's see.
<wgrant> wgrant@lplucid:~/launchpad/lp-branches/wbr-is-void$ bzr grep '[^ ]/>' | wc -l
<wgrant> 10623
<wgrant> wgrant@lplucid:~/launchpad/lp-branches/wbr-is-void$ bzr grep ' />' | wc -l
<wgrant> 11095
<wgrant> wallyworld_: ^^ I win
<wgrant> (either is OK)
<wallyworld_> hmmm.
<wgrant> (but mine is perferred by almost 5%! incontrovertible evidence that mine is superior)
<wgrant> lifeless: Ah
<wgrant> lifeless: Foreign elements are MathML and SVG (ie. XML formats forced against their will into an HTML document), so don't apply here.
<lifeless> wgrant: also in html 5 <wbr/> isn't ok; but the parser-recovery rules handle it
<wgrant> lifeless: The validator is fine with it. Are you sure it doesn't parse it as an invalid valueless attribute and ignore it?
<lifeless> wgrant: which validator ?
<wgrant> lifeless: validator.w3c.org in HTML5 mode.
<wgrant> Which is still somewhat experimental, but still.
<wgrant> "Then, if the element is one of the void elements, or if the element is a foreign element, then there may be a single U+002F SOLIDUS character (/). This character has no effect on void elements, but on foreign elements it marks the start tag as self-closing."
<wgrant> So /> is valid in HTML5
<lifeless> ah, void is special cased
<wgrant> Yep
<wgrant> Confusing enough.
<lifeless> so this comes back to beautifulsoup - it *should* special case voids
<lifeless> and all others should have end tags
<wgrant> No, old SGML-subset HTML should die already,.
<wgrant> Everybody should just use XML because XML doesn't suck as much.
<lifeless> the validator is special casing voids
<wgrant> Yeah
<lifeless> other self closed elements will generate 'Self-closing syntax (/>) used on a non-void HTML element. Ignoring the slash and treating as a start tag.' from the validator
<StevenK> lifeless: It does special case voids: [16:06] < wgrant> Yeah, BeautifulSoup has an outdated hardcoded list of void elements.
<wgrant> But the relevant BeautifulSoup is the embedded one in mechanize or zope.testbrowser (not to be confused with the other embedded BeautifulSoups in our tree), so I didn't really want to mutilate it.
<lifeless> StevenK: I know it *does*, the question was *should it*
<StevenK> And then you said it should. And it does.
<wgrant> I see no problem with that.
<lifeless> StevenK: and the answer is, a) yes it should and b) the standard says to ignore the self-closing and look for an end tag.
<wgrant> We were arguing the correctness.
<wgrant> It is correct.
<lifeless> and its fallback is correct as well, which was the bit I started querying.
<lifeless> so I've cleared up a bit of fuzz in my head, which is good, and we know that the right fix is updating the void list in bs
<wgrant> Yeah
<lifeless> http://tiffanybbrown.com/2011/03/23/html5-does-not-allow-self-closing-tags/ is a cite for this btw
<wgrant> HTML5 seems to have that exception just to cope with this sort of case.
<wgrant> I wish WHATWG had just disregarded IE, specced out XHTML5, and left HTML4.01 as the last horror. :/
<wgrant> I guess in 2015 people can start using XHTML :)
<wgrant> 15 YEARS LATER
<wgrant> wallyworld_: Thanks.
<wallyworld_> np
<lifeless> didn't you hear, browsers are agile now.
<wgrant> Anyway
<wgrant> This is the last large-scale validity issue. :)
<StevenK> lifeless: Except for Safari and IE.
<wgrant> Safari's not that terrible.
<StevenK> And who knows what the truck Opera are doing.
<wgrant> Opera is fine.
<wgrant> Better than Safari, even.
<StevenK> I was conflating agile with 'make a release every two days' purely to troll.
<StevenK> What are they at now, Chrome 25.2?
 * micahg just uploaded Chromium 18.0.1025.142
<lifeless> using 4 numtets is cheating
<bigjools> I've lost track of where to download a stable one these days
<lifeless> didn't you hear, they are all stable :>
<nigelb> surely, you mean unstable :)
<micahg> bigjools: we've been pretty good about pushing stable versions out save for the last two
<bigjools> micahg: pushing where? :)
<micahg> to the archive
<bigjools> nice one
<micahg> 18 should be in -proposed later today
<wgrant> StevenK, wallyworld_: If you're still around, could you look at https://code.launchpad.net/~wgrant/launchpad/bugtaskflat-garbo/+merge/100363?
<StevenK> wgrant: r=me
<wgrant> StevenK: Thanks.
<czajkowski> aloha
<adeuring> good morning
<cjwatson> I'm working on API extensions to make it possible to turn queue into a launchpadlib script.  The wart is overriding individual binaries in a queue entry; PackageUpload currently only has an overrideBinaries method which overrides all of them, and queue traverses through queue_item.builds[i].build.binarypackages[j].override() to override individual binaries.  Would it be better to add an overrideBinary method to ...
<cjwatson> ... PackageUpload that would allow overriding one at a time, or to expose all the objects down to BPR.override and expect clients to traverse that directly?
<cjwatson> Hmm, even just writing that I think I might have convinced myself that a new PackageUpload.overrideBinary method would be better than all that object traversal ...
<wgrant> cjwatson: I'd expose overrideBinaries({name: overrides, ...})x
<wgrant> cjwatson: With the slight awkwardness that there are multiple overrides, but I guess that could be a dict.
<cjwatson> Hmm.  The client would be pretty unmanageable if the overrides were all different.  How about I just allow passing in a list of names?
<cjwatson> (I'll also need to expose something on PackageUpload that lets you get the list of binarypackagenames in a build upload.
<cjwatson> )
<cjwatson> The interface I'm trying to replace is typically used as something like "queue override -c universe binary foo bar baz"
<wgrant> That would work too, but I'm trying to minimise the number of requests.
<wgrant> Although I guess RTT for you isn't too bad :)
<wgrant> From here, every request counts.
<cjwatson> I think in practice hardly anyone would use it as -c universe binary foo bar baz -x editors blah spong, and the CLI looks distinctly unwieldy at that point
<wgrant> Yeah
<cjwatson> Even that example is ambiguous and if you start trying to make it unambiguous you end up with something like find(1) :-)
<cjwatson> You're right that there's clearly no need for YA method though.
<wgrant> Launchpad unwieldy and ambiguous? Surely you jest.
<cjwatson> Call me a fool for aspiring to something higher :-)
<wgrant> So, yeah, overrideBinaries([bpn], component?, section?, priority?) sounds reasonable.
<cjwatson> Right.
<cjwatson> Let me see what I can do, then.  (This is sort of back-burner for me but I'm trying to get stuff done before I get redirected to something else.)
<wgrant> It's pretty simple :)
<cjwatson> I've got the source override API working already.
<wgrant> Great.
<wgrant> What about permissions?
<cjwatson> Controlled by queue-admin ArchivePermissions.
<wgrant> Excellent.
<cjwatson> If you're overriding between two components you need queue admin on both.
<wgrant> Perfect.
<cjwatson> Most of that was there already for the web UI.
<wgrant> Yeah, but pretty much every large export of Soyuz stuff in the past has had obvious gaping holes.
<wgrant> So this would be a nice first.
<cjwatson> My tests are mostly about permissions at the moment.
<wgrant> :)
<wgrant> Oh good
<wgrant> This is marvellous.
<wgrant> Truly marvellous.
<wgrant> It seems that feeds are produced using BeautifulSoup
<wgrant> whhhhhy
<wgrant> So our legal HTML gets mangled unreadable, I suppose.
<stub> Because this was from the dark ages when it was the sanest way of generating the xml-like files that could only be validated using a form running on Mark Pilgrim's web site?
<wgrant>             # Unqualified hrefs must be qualified using the original subdomain
<wgrant>             # or they will try be served from http://feeds.launchpad.net,
<wgrant>             # which will not work.
<wgrant> It uses BeautifulSoup to do that :(
<wgrant> Not quite sure what's wrong with teaching canonical_url to do that...
<wgrant> Anyway, looks like I am monkeypatching BeautifulSoup :(
<StevenK> wgrant: I think we should solve it another way, and just destroy feeds.
* benji changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: benji | Firefighting: - | Critical bugtasks: 4*10
<deryck> Morning, all.
<abentley> deryck: morning.
<abentley> adeuring: After discussing queues and routing keys with Celery's main author on Friday, I think we should switch to using queues explicitly.
<abentley> adeuring: Celery is unlikely to support routing keys the way it supports queues in the near future.
<adeuring> abentley: sshall talk about it after the standup?
<abentley> adeuring: Sure.
<czajkowski> gmb: ello
<gmb> czajkowski: Wotcher
<czajkowski> gmb: care to give a talk during UAD week?  https://wiki.ubuntu.com/UbuntuAppDeveloperWeek/
<abentley> czajkowski: I figured out the national holiday issue.  I accidentally had Christmas holidays ending Feb 26.  I've asked dragnob to delete/correct it.
<gmb> czajkowski: What topic are you thinking about? Development using the LP API?
<czajkowski> gmb: aye
<czajkowski> gmb: could be handy as I want to do  ablog series on low bugs to get people involved
<czajkowski> abentley: see I wasn't going crazy :)
<gmb> czajkowski: Okay. Let me have a think, see if I can come up with anything that would fit well.
<czajkowski> gmb: thanks
<abentley> czajkowski: Yeah, but where's the evidence you weren't already crazy :-)
<czajkowski> abentley: good counter argument!
<czajkowski> can't fault you there
<StevenK> czajkowski: So you want to fix bugs and blog about it, or what are you thinking?
<czajkowski> StevenK: i sent a mail to lp-dev there on friday
<czajkowski> people to suggest a low bug
<czajkowski> then to try and get people to work on those in the community
<nigelb> Remember to ask that folks contrubting to LP need to sig the contributor agreement.
<nigelb> *sign
<czajkowski> and conversation dead :(
<rick_h> deryck: ping, so just to be sure, we want the notification to go out when someone adds an email/gpg key, and we want that timed when the initial request goes out, not when it's accepted/verified right?
<deryck> rick_h, yeah, that sounds right to me.
<rick_h> 142372
<deryck> rick_h, was that number for me?
<rick_h> deryck: no, it's this stupid yubikey. I keep finding ways to bump it
<rick_h> sorry, I keep trying different usb ports, but all are too handy it appears
<deryck> rick_h, no worries :)
<abentley> adeuring: I've tested CELERY_CREATE_MISSING_QUEUES, and it works as advertised.  Here's my current config: http://pastebin.ubuntu.com/911604/
<adeuring> abentley: cool
<reed> hi thedac, we didn't make a copy of http://people.canonical.com/~dames/openstack.mbox.tar.gz in time :( Should I file the request again?
<thedac> reed: hi, I will get that to you again
<reed> thanks thedac, I apologize for the extra work
<thedac> reed: I have placed that at the same URL
<reed> thedac, thank you very mych
<thedac> no problem
<rick_h> deryck: ping, have a sec?
<deryck> rick_h, otp.  ping when off.
<rick_h> deryck: k
<ayan> is there a Go implementaiton of the launchpad api?
<rick_h> ayan: looks like: http://blog.launchpad.net/api/launchpad-is-go
<deryck> rick_h, hey, what's up?
<rick_h> deryck: want to chat on this before I put up for mp if you don't mind
<deryck> rick_h, sure.  hangout?
<rick_h> yes please the go-deryck url?
<deryck> yeah
<lifeless> morning
<abentley> lifeless: morning.
<abentley> lifeless: How would you feel if all Jobs ran under that same db userid, e.g. launchpad_main?
<lifeless> I would worry - some of our jobs have access to privileged data and that would increase the surface area that security breaches can happen within.
<lifeless> OTOH there is a healthy (but very high latency) discussion taking place on the dev list about whether our multiple db users is worth it
<lifeless> that said, for situations where one can do 'dbuser(foo)', the degree of security we gain is minimal, its only when a db user is /not/ available that security is really increased.
<lifeless> finally, we do get reasonably significant benefits on reporting db *load* by user, which I'd be sad to lose
<jelmer> 'morning lifeless, abentley
<abentley> jelmer: morning.
<lifeless> hi jelmer
<abentley> lifeless: Okay.  Getting rid of 'em as we move to Celery would simplify it some, but not dramatically.
<rick_h> deryck: so no current messages when ssh keys are added/removed. Are we thinking we want both?
<deryck> rick_h, yeah, a user should be notified of both.
<rick_h> deryck: ok, will do.
<deryck> cool, thanks
<bac> hi flacoste
 * deryck heads back home
<lifeless> flacoste: hey there
* benji changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: - | Firefighting: - | Critical bugtasks: 4*10
<lifeless> fun shiny of the day: create trigger ... where NEW.col is distinct from OLD.col
<lifeless> s/where/when/ :)
<wgrant> lifeless: Yeah, but only in 9.1
<wgrant> That almost made me wait.
#launchpad-dev 2012-04-03
* wgrant changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: wgrant | Firefighting: - | Critical bugtasks: 4*10^2
<wgrant> Nice easy one: https://code.launchpad.net/~wgrant/launchpad/html5-charset/+merge/100541
<StevenK> wgrant: Meh, self-review :-P
<StevenK> wgrant: r=me
<wgrant> Thanks
<StevenK> No Production OOPSes? Oh dear.
<lifeless> carob may have been rebooted
<StevenK> It has not.
<StevenK> But we have Soyuz, qas and staging mails, just no prod.
<lifeless> oh
<lifeless> it will be chewing on fdt probably then
<StevenK> For an hour?
<lifeless> has on extreme cases in the past
<wgrant> StevenK: It's contending with prune.
<wgrant> postgres is not very happy.
<wgrant> It's later than normal today, but still going.
<wgrant> (you'll see it's been like an hour late for a few days)
<wgrant> It's also eating a lot of RAM, oddly.
<wgrant> lifeless: 12MB OOPSes make bson sad.
<wgrant> Ah
<wgrant> So that's why the report was so slow.
<wgrant> Something is terribly broken.
<wgrant> Although given the UA it might just be someone trying to exploit us.
<wgrant> Who knows.
<wgrant> It looks almost like someone assumed that every quoted string on the page was a URL
<StevenK> ... Odd.
<wgrant> Yeah
<wgrant> It's an archival crawler.
<wgrant> Which seems to be misbehaving.
<wgrant> Yay
<wgrant> No new bug write timeouts.
<wgrant> I am safe.
<StevenK> wgrant: Still can't figure out what to do in terms of Bug:+secrecy and multi-pillar bugs.
<wgrant> StevenK: Oh?
<wgrant> StevenK: I hadn't heard about this.
<StevenK> wgrant: Currently, the UI omits private if it's a multi-pillar bug.
<stub> Heh. Write robust code and forget WTF it does.
<wgrant> StevenK: Ah
<StevenK> wgrant: But that won't help given IBug.transistionToInformationType() will toss an exception no matter what you choose if the bug is multi-pillar.
<wgrant> StevenK: That needs to stop.
<StevenK> wgrant: And we do what instead?
<lifeless> only proprietary ones cannot be multi-pillar
<wgrant> StevenK: In the end only PROPRIETARY is forbidden to have multi-pillar
<wgrant> What he said.
<StevenK> Should I change that now, then?
<wgrant> Probably. You can remove the FF at the same time.
<StevenK> Right.
<StevenK> wgrant: https://code.launchpad.net/~stevenk/launchpad/forbid-proprietary-multi-pillar/+merge/100557
<wgrant> StevenK: What's with line 33?
<wgrant> The new cancel_url.
<StevenK> wgrant: Bug:+secrecy doesn't have a cancel link, it has annoyed me while I was working on the UI and it's one line.
<wgrant> Heh
<wgrant> k
<wgrant> + def test_hide_private_option_for_multipillar_proprietary_bugs(self):
<wgrant> StevenK: Oxymoron
<StevenK> Right, but it will change when the UI does to show information_type.
<wgrant> StevenK: The idea is that you can't make a multipillar bug proprietary.
<wgrant> StevenK: The starting condition for that test is that there is a proprietary multipillar bug.
<wgrant> That's invalid state.
<StevenK> Hmmm.
<danhg> Morning
<StevenK> wgrant: Sadly, we can't test that with the UI having not switched.
<wgrant> StevenK: Indeed.
<wgrant> StevenK: Anyway, that test is invalid. Fix it if you can, otherwise delete it.
<adeuring> good morning
<StevenK> wgrant: But no approval? :-(
* gmb changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: gmb | Firefighting: - | Critical bugtasks: 4*10^2
<gmb> Well, today's going to be a fun one if I've got only a partially-working desktop.
<gmb> Sigh
<gmb> (Also, what's the word for that feeling you get when you encounter a bit of Launchpad that you're largely responsible for and wish fervently that you'd had the time to make it less clunky?)
<lifeless> gmb: guilt
<gmb> lifeless: Yeah. At least it's fleeting, because I have no actual soul.
<czajkowski> gmb: see this is why I wanted people to reply to my thread on low bugs so I could get people to help
<czajkowski> bu nobody replied :(
<lifeless> czajkowski: well, I thought about replying
<czajkowski> but you saw to many low bugs :)
<lifeless> czajkowski: see, do you mean low as in 'unimportant' or low as in 'easy' / 'approachable' ?
<gmb> czajkowski: Scheisse. I actually meant to, and then got sidetracked. But my question was the same as lifeless's.
<lifeless> czajkowski: if you mean 'unimportant', well, it seems odd to motivate people to work on unimportant bugs.
<czajkowski> lifeless: the latter
<lifeless> czajkowski: look for bugs tagged trivial or easy.
<czajkowski> ideally what I want to do is well get 5 bugs that people would love to see done in LP, but dont ahve the time and I could try and get people more involved and woarkign on patches
<lifeless> czajkowski: there are many - hundreds - of those that are also 'high' importance.
<czajkowski> nods
<czajkowski> but thought i'd start with low first adn see if some of the smaller tweaks could get people involved and interested
<czajkowski> and a place to start
<lifeless> czajkowski: 'low' importance does not mean small.
<lifeless> it means less important. :)
<lifeless> czajkowski: but perhaps we're talking at cross purposes. Let me ask instead, why are you talking about 'low' bugs: what is attractive about them, and do you mean the same thing as I do when I say 'low' ?
<czajkowski> hmm
<czajkowski> so imo low was small tweaks to lp that people would like to see like the example I gave about cc'ing someone when you send a contact this person mail
<czajkowski> lifeless: I was also just looking to get people talking and for people t suggest bugs
<czajkowski> but that didn't happen :/
<lifeless> assertion: there are small tweaks that are important, and small tweaks that are less important. What association do you see between 'low' importance and 'small tweak' ?
<czajkowski> lifeless: I just started with low as a starting point, if you found me bugs that were high I'd also promote them :)
<lifeless> czajkowski: so, I'm a bit like a terrier here :)
<lifeless> why did you start with low; do you perhaps assume that low importance has some correlation with size of effort to fix the bug ?
<czajkowski> lifeless:  nope no correlation
<czajkowski> I start with low as a place to start
<lifeless> I find that fascinating
<lifeless> anyhow, the place I suggest you start is the tag 'trivial'
<czajkowski> lifeless: care to suggest a bug then :)
<czajkowski> at least you;ve entered into discussion
<czajkowski> which is fascinating :)
<lifeless> https://bugs.launchpad.net/launchpad-project/+bugs?field.tag=trivial
<lifeless> bug 869309
<_mup_> Bug #869309: If folders in repository have unicode names, sometimes I get KeyError <navigation> <oops> <trivial> <unicode> <loggerhead:Triaged> < https://launchpad.net/bugs/869309 >
<czajkowski> lifeless: thank you
<gmb> czajkowski: I would suggest bug 353097 and bug 249471 as good ones to tackle. The second might be slightly more complex, IDK.
<_mup_> Bug #353097: Marking a Bug as "Requiring Forwarding" text is missleading... <better-forwarding> <confusing-ui> <docs> <lp-bugs> <trivial> <Launchpad itself:Triaged> < https://launchpad.net/bugs/353097 >
<_mup_> Bug #249471: Reporting a bug with an attachment produces two info alerts <lp-bugs> <trivial> <Launchpad itself:Triaged> < https://launchpad.net/bugs/249471 >
<lifeless> nn all
<czajkowski> lifeless: nn
<cjwatson> lifeless: IME Launchpad's low importance doesn't always imply that the bug is of low importance to the user or group who reported it (and rightly so; Launchpad has to prioritise)
<cjwatson> lifeless: for instance, bug 930217 is likely to be of considerable use to Ubuntu, but it was triaged as Low, and I think that was reasonable for LP to do (we could have used up some stakeholder credit to escalate it to Critical, but I think it's better to use such credit for things we can't fix ourselves)
<_mup_> Bug #930217: Make proposed pocket useful for staging uploads <qa-ok> <Launchpad itself:Fix Released by cjwatson> < https://launchpad.net/bugs/930217 >
<wgrant> cjwatson: Don't get lifeless started on multitenancy :)
<StevenK> Haha
<cjwatson> Heh, well, my other argument would be that if you want to get more people involved then the message of "these are things we aren't currently planning to fix at all, but you can" is stronger than "these are things we're planning to fix eventually, but you can make it happen sooner"
<jml> cjwatson: perhaps we just need to spread the message that "not Critical" means "not planning on fixing at all"
<wgrant> And "Critical" means "planning on fixing eventually but it'll never actually happen" :)
<jml> "'The matter is under consideration' means we have lost the file. 'The matter is under active consideration' means we are trying to find the file."
<cjwatson> Yes Minister++
<cjwatson> or was that Yes Prime Ministere?
<cjwatson> -e
<jpds> Both.
<jml> cjwatson: I think it was Yes Minister.
<mabac> salgado, lost you
<salgado> yep, trying to reconnect
<benji> let me check my notes
<benji> ...for which channel I want to be in
<deryck> Morning, everyone.
<czajkowski> deryck: ello
<StevenK> benji!!
<benji> StevenK!!
 * benji hears string music well up in the background.
<rick_h> StevenK: you still around? I wanted to chat with you. I broke the convoy build setup you had by adding a makefile :/
<StevenK> benji: Oh, just that you fixed the unclean reactor terribleness with a branch of awesome
<benji> I hope I got it all.  I'm not very... twisty
<StevenK> rick_h: Ish. Chatting with friends and trying to avoid work.
<rick_h> StevenK: ah, gotcha. Nvm then. I'll send an official email and all that
<rick_h> go get more wine!
<StevenK> rick_h: Chatting with friends on *IRC*
<StevenK> rick_h: So I'm here at the computer, so you can ask if you wish.
<rick_h> StevenK: just wanted to have the conversation on updating hte packaging and see what needs to be done. I figure I need to update the Makefile I added to not bork whatever needs to go on
<rick_h> we've filed an RT to get it on production so figured I should unbreak things before that happens, but not sure what you had in mind when you mentioned 'unsucking' the packaging
<StevenK> rick_h: We should fix the packaging before we go forward to production.
<rick_h> StevenK: rgr, so I wanted to ping and get a list of what we need to do so that we can get it done.
<StevenK> rick_h: So I think you should organise a call/hangout with you, me, the Landscape guy that admits to knowing what convoy is, and the same for U1.
<rick_h> ok, I'll put together an email. I know sidnei is brazil-ish, not sure on the landscape guy. Will try to find some common time ground.
<jml> do any of you have any tricks for pairing on LP hacking?
<StevenK> rick_h: It's a one time call, so I'm happy to take a hit if I need to -- like 10pm or so.
<rick_h> StevenK: rgr, thanks. Don't think it'll be necessary, but will see
<rick_h> jml: I thought some people did some shared desktop stuff in hangouts recently that worked ok, but not really
<jml> rick_h: ta
<jml> How do I run ec2 from lp-dev-utils?
<rick_h> jml: you have to add it to your path. I symlinked the stuff into my ~/bin dir
<rick_h> had the same question myself heh
<jml> rick_h: thanks.
<StevenK> I symlinked into ~/bin, it was easier
<jml> please remind me what set_schema in zcml means
<StevenK> That you have permission to set the attributes exported by that class
<wgrant> interface
<jml> ah ok.
<jml> so, IIUC, you only need view permissions on IPerson to call createPPA?
<wgrant> That seems to be correct.
<jml> wgrant: where's the logic that prevents me from creating a PPA in ~wgrant?
<wgrant> jml: There probably isn't any.
<wgrant> Aha
<wgrant> It's not my fault after all
<wgrant> I apparently reviewed the commit that exports the method (despite it being before I started), but it was not added to ViewRestricted
<wgrant> When did that change...
<jml> wgrant: but ViewRestricted means "you need launchpad.View", right? surely createPPA should require launchpad.Edit?
<wgrant> jml: One would certainly think so.
<wgrant> I'm saying that your analysis is correct.
<wgrant> Not that the current state of things is in any way justifiable.
<jml> wgrant: oh ok.
<jml> wgrant: as some context, james_w and I are exploring the LP code base, hoping to make whatever changes we need so a bot can create private, commercial PPAs.
<wgrant> jml: Didn't you land a change recently to permit creating private ones?
<jml> wgrant: maybe. I tend to forget things like that.
<StevenK> Or Cody did.
<wgrant> [r=julian-edwards][bug=814567] Add an API to create private PPAs,
<wgrant> rather than changing privacy status post creation.
<wgrant> 'twas jml
<jml> yeah, found the branch.
<jml> right. that was clever of me.
<jml> I guess it's the 'commercial' bit that we need now.
<jml> You know, as an interview question, SteveA asked me how I would build a permission system. I wish I had had a really good answer.
<wgrant> ZTK's is terrible.
<wgrant> But it's one of the better ones I've seen.
<StevenK> Oh, it doesn't serve as a warning?
<StevenK> jml: One of my interview questions with mdz was him asking me which VCSes I had experience with, and I commented that I had a few personal projects in tla, which prompted him to say "You've done ... *real* work ... with tla?"
<james_w> james-w is not allowed to make private PPAs
<jml> it's hard to judge ZTK's permission system based on LP
<james_w> which is <person on which the PPA is being created> is not allowed to make private PPAs
<jml> except that you learn that it's not easy for developers to use
<jml> james_w: yeah, that's from Archive.validatePPA
<jml> james_w: which does a *manual* check for celebrity membership
<jml> james_w: rather than asked if person has launchpad.Commercial on something
<nigelb> StevenK: haha, that's a great question :P
<jml> basically, LP doesn't really have an internal language for "this user is allowed to make private stuff"
<jml> maybe it should have something like launchpad.AppendRestricted
<jml> how do I shell into the ec2 machine that the ec2 script creates?
<jml> as in, what key do I use?
<rick_h> jml: I think it creates the key and in the log it mentions the name? not 100% sure thogh
<jml> rick_h: there's a *lot* of log
<rick_h> jml: yea, but when you run it dumps out to terminal the startup bits as it builds up for the ec2 instance
<rick_h> it's in there before the instance fires up I believe
<rick_h> jml: if you look at the instance in the ec2 web tools I htink it lists what the key is that it's using? bah, I got this to work once, but not needed to do it again
<jml> hmm. nope.
<jml> I mean, there's a key there, but I can't get at it.
<rick_h> right, but does it give a hint for the filename to look for locally?
 * rick_h goes to peek at the aws console
<jml> rick_h: no, it doesn't.
<rick_h> jml: so when I click on my running instace in the ec2 web console it says the key pair name on the left?
<jml> rick_h: yes. where am I expected to find that on my laptop?
<rick_h> I don't recall, but if you updatedb/locate does it come up?
<jml> rick_h: no, it doesn't.
<rick_h> bah sorry, don't recall then. Maybe someone else knows.
<jml> rick_h: well, thank for trying.
<jml> rick_h: the trick is to ssh in as the 'ec2test' user
<rick_h> jml: doh, thanks
<jcsackett> we need an ec2 ssh command.
<jcsackett> well, maybe not need.
<jml> james_w:   ssh -A ec2test@ec2-184-73-132-244.compute-1.amazonaws.com
<jml> jcsackett: it'd be pretty easy to do.
<jml> jcsackett: I remember asking myself "why didn't I do that when I first thought of it?" when I added ec'2 list'
<jcsackett> ec2 list, btw, is a fantastic command. :-P
 * rick_h goes to look up ec2 list
<jcsackett> jml: yeah, it doesn't strike me as that difficult. might look at it when i next have slack time.
<jcsackett> rick_h: it lists all your running instances. handy way to make sure you don't leave something running.
<rick_h> ah, very cool then. I tend to hit the webui which can be a pita
<jcsackett> rick_h: yeah. and the verbose options tell you which branches are running with what instance ids. which is handy when you then want to `ec2 kill` something.
<rick_h> ooh, json decode backtrace :/
<jcsackett> basically, the less i have to jump to the webui, the happier i am.
<rick_h> yea, definitely
<james_w> jml, http://www.debian-administration.org/article/Using_GNU_screen%27s_multiuser_feature_for_remote_support
<rick_h> gmb: a bit of reading if you get some time please? https://code.launchpad.net/~rharding/launchpad/email_notice_959482/+merge/99995
<gmb> rick_h: Sure thing.
<rick_h> gmb: first real change in this area, so feel free to be a bit brutal to show me the ropes and beat proper things into my head.
<gmb> Okidoke.
 * gmb cracks knuckles
 * rick_h runs and hides
<jml> :(
<jml> https://code.launchpad.net/~jml/launchpad/create-commercial-ppa/+merge/100635 up for review
<rick_h> danhg: ping, my MP with the email and updated wording is here https://code.launchpad.net/~rharding/launchpad/email_notice_959482/+merge/99995
<rick_h> danhg: the reasoning/etc are tied into the bug and MP description, I think that'll answer most of your questions you had via email
<gmb> rick_h: Approved with comments (all about tests and documentation).
<rick_h> gmb: ty much will peek at the damage
<gmb> rick_h: The code looks fine; good work!
<rick_h> gmb: thanks, I might understand how this stuff works yet! :)
<gmb> :)
<benji> gmb: if you have a minute, I have a small branch for review: https://code.launchpad.net/~benji/launchpad/bug-972456/+merge/100645
<gmb> benji: Sure
<abentley> abel: could you please review https://code.launchpad.net/~abentley/launchpad/celery-job-feature-flag/+merge/100647 ?
<gmb> benji: Approved.
<benji> gmb: thanks
<abentley> gmb: Could you please review https://code.launchpad.net/~abentley/launchpad/celery-job-feature-flag/+merge/100647 ?
<gmb> abentley: Sure
<gmb> Merge proposals are like buses...
<gmb> Anyway.
<gmb> abentley: Approved; nice work.
<abentley> gmb: Thanks.
* gmb changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: - | Firefighting: - | Critical bugtasks: 4*10^2
<stokachu> hi, using python-launchpadlib to retrieve messages the date_created field are unicode strings instead of datetime objects, is this expected?
<stokachu> the parent bug will return a datetime object so i was curious
<rick_h> stokachu: update your version of waddlib, that was fixed a week or two ago I think
<stokachu> rick_h: ok ill do that, i saw the bug where the date_created field in a bug was fixed not sure if it extended to messages
<rick_h> I think it was fairly general
<stokachu> ok
<stokachu> did the fix make it into pypi?
<rick_h> believe so
<rick_h> yea, 1.3.1 mentions the datetime bug
<stokachu> i have waddlib 1.3.1 installed
<rick_h> ok, then you might submit a new bug on it with the steps to reproduce please (or a patch :) )
<stokachu> ok sounds good, just making sure it was an oversight
<stokachu> rick_h: im going to file a bug and hopefully get a patch but just for my sake could you look at this snip: http://pastebin.ubuntu.com/913394/
<rick_h> hmm, you're not accessing it via the object interface so not sure you'll get anything but that string
<rick_h> I think what the wadllib and such do is convert that dict access to message.date_created or something
<rick_h> based on the object definition from the wadl file
<rick_h> at least that's my understanding
<stokachu> ah makes sense, so im probably just calling it incorrectly
<rick_h> all the tests are doing things like xx.get_parameter('a_date').get_value()
<rick_h> which return a datetime object
<stokachu> hmmm ok ive been looking through the collections doc but im sure im doing something wrong
<rick_h> yea, well you're at the lplib layer which uses wadllib underneath so not sure how it exposes it
<stokachu> yea im not totally sure either, every way i slice it comes back as dict
<stokachu> bug.message is a collection object
<stokachu> when doing a bug.messages.resource_type_link
<stokachu> u'https://api.launchpad.net/1.0/#message-page-resource'
<stokachu> that doesn't exist in the api
<rick_h> sec, trying something out
<stokachu> cool no worries
<rick_h> sorry, not quite an api pro heh
<stokachu> me either, took me a week to understand that person and team were the same thing :X
<stokachu> andd my internet craps out
<salgado> is there a way to do sequence unpacking in TAL?  I thought there was but can't seem to find out how
<rick_h> stokachu: hmm, so what comes back to you is really just a dict plain and simple. It seems like there shuold be a way to get a real object out of there but don't see it with casual print dir()'ing
<stokachu> rick_h: ok, yea im getting that same impression
<jcsackett> given a set of bug ids, is there a set based way to feed them to bug search? i see bugparams can take a bug param, but that looks like a single thing, not a set thing. i'd rather not iterate over the whole set.
<rick_h> stokachu: http://pastebin.ubuntu.com/913443/
<rick_h> ok, so that's some playing, it doesn't support negative indices
<rick_h> and that comes out as a datetime object
<rick_h> not a string
<stokachu> ah
<rick_h> http://pastebin.ubuntu.com/913444/
<rick_h> is a bit better with the output
<stokachu> ok cool, thanks! :)
<rick_h> so hopefully that helps a little bit, sorry
<stokachu> rick_h: thats perfect
<stokachu> rick_h: wish it allowed negative indices
<stokachu> guess i could do message = bugs.messages[len(bugs.messages.entries)]
<rick_h> stokachu: yea, not sure if there's a way to get the length and guess the right index off of it
<rick_h> there's a property for count or size I saw while dumping out dir() of things
<rick_h> try dir(bug.messages)
<stokachu> ah lemme look into that
<salgado> rick_h, do you know if there's a way to do sequence unpacking in TAL?
<rick_h> salgado: sorry, I know next to nothing of tal
<salgado> rick_h, heh, ok.  no worries
<lifeless> cjwatson: so, I agree; you'll note I didn't end up recommending 'start with trivial+high', just 'trivial', once I felt I understood czajkowski's desire a bit better
<lifeless> cjwatson: and yeah, we have actually speculated about a very einsteinian model for importance and sorting :)
<cjwatson> right, more important to be fixable easily than to be important, in this case
<cjwatson> of course there's also the importance/urgency inequality
<lifeless> fortunately we don't have urgency, it doesn't exist :P
<cjwatson> it gives you a simpler and probably more usable model, certainly; still leaves you mapping two not-quite-parallel dimensions onto one :)
<cjwatson> I agree that that trade-off was worthwhile, but it's something contributors will likely be thinking about
<jono> hey folks
<lifeless> hi
<salgado> StevenK, hey there.  it'd be great if you could review https://code.launchpad.net/~linaro-infrastructure/launchpad/team-engineering-view-ui/+merge/100707 for us (it seems like you're the on call reviewer today?)
<StevenK> salgado: On the phone, I'll look afterwards.
<salgado> StevenK, thanks a lot!
<cjwatson> Are there any examples in LP of something whose DB/Python representation is multiple levels, but which we collapse down to one level for exposure in the web service API?  Say the Python representation is Foo.Bar.attribute, but I'd like to have the API be more like foo.attribute.
<cjwatson> I'm not quite familiar enough with the patterns and so it's hard to grep for.
<StevenK> cjwatson: SPPH does that when it exports things from SPR.
<cjwatson> Ah, of course.  Thanks.
<StevenK> return self.sourcepackagerelease.<>
<cjwatson> So just exported() in the interface and a @property in the model.  Great.
<StevenK> With a test, that should be great.
<cjwatson> I'm trying to do something similar in PackageUploadSource.
<jono> has anyone here got any experience with intltool-update? I am trying to generate a .POT file for a .desktop like file I can import into Launchpad
<cjwatson> Only in conjunction with autoconf/automake, where you do @INTLTOOL_DESKTOP_RULE@ in Makefile.am and declare the files in desktop* targets and it takes care of the rest.
<cjwatson> That's a fair bit to pull in if you aren't already using automake, though.
<jono> cjwatson, from what I understand i I have POTFILE.in in my po/ dir and just run intltool-update --pot --gettext-package=foobar it should process my file and generate a .POT file
<cjwatson>   INTLTOOL_DESKTOP_RULE='%.desktop:   %.desktop.in   $(INTLTOOL_MERGE) $(wildcard $(top_srcdir)/po/*.po) ; $(INTLTOOL_V_MERGE)LC_ALL=C $(INTLTOOL_MERGE) $(INTLTOOL_V_MERGE_OPTIONS) -d -u -c $(top_builddir)/po/.intltool-merge-cache $(top_srcdir)/po $< [$]@'
<jono> the problem is that it is not generating the file
<cjwatson> possibly helpful if you want to unpick all those expansions
<jono> I have no idea what that is :-)
<cjwatson> Ah, that's the other end, that merges the translations into an output .desktop file
<jono> gotcha
<jono> so I have the following in foo.desktop:
<jono> [Desktop Entry]
<jono> Encoding=UTF-8
<jono> _Name=MyApplication
<jono> _Comment=A useful application that does useful things
<cjwatson> something like what you said sounds right, though it needs to be POTFILES.in, not POTFILE.in
<jono> and then in a po/ dir I have POTFILES.in with foo.desktop in it
<jono> and I run: intltool-update --pot --gettext-package=foobar
<jono> but then there is no output
<cjwatson> it has a --verbose
<jono> ahhh
<jono> it says: None of the files in POTFILES.in contain strings marked for translation.
<jono>  -- which is odd as I have underscore before the keys in the .desktop file
<lifeless> jono: -> #launchpad, I suspect you'll get more help there
<jono> lifeless, it was pretty quiet earlier when I asked, but will try again
<jono> thanks cjwatson
#launchpad-dev 2012-04-04
<StevenK> wallyworld_: O hai
<wallyworld_> hellooooooo
<StevenK> wallyworld_: I'm on mumble and ready when you are.
<wgrant> wut
<wgrant> https://lp-oops.canonical.com/oops.py/?oopsid=OOPS-006ee065b5671473827829630f869283
<wgrant> Why is something trying to serialise a bug into rabbit?
<StevenK> wgrant: I was about to ask you about that.
<wgrant> Unless we're for some reason deciding to send notifications for all objects to rabbit.
<StevenK> Why would we do that? It makes no sense.
<wgrant> So it was making the bug private.
<wgrant> s/private/public/
<wgrant> Which somehow fires off a rabbit notification because why not.
<wgrant> Huh
<wgrant> So indeed
<wgrant> Anything with a base class of Storm will get rabbit notifications.
<wgrant> If the changed fields are JSONable, the world ends.
<wgrant> aren't.
<StevenK> Strange.
<lifeless> poolie: would you have time for a chat today?
<lifeless> poolie: in an hour or two from now ?
<poolie> lifeless, sure
<lifeless> poolie: how about now ?
<poolie> lifeless, oops, how about now?
<lifeless> poolie: sure, ring me?
<poolie> lifeless, engaged?
<wallyworld_> wgrant: with access policies and bugs - access policies are associated with pillars, a bug has bug tasks associated with pillars, what is a person is denied access to a pillar, it seems that they can still see the bug even if it has a bug task they cannot see
<StevenK> Access to a bug is defined via bugtasks
<wgrant> An access grant is for a bug, not a bugtask.
<wgrant> You can never see a partial bug.
<wgrant> The people who can see a private bug are anyone with a grant on the related policies or an explicit grant for the bug.
<wgrant> This means that revoking a policy grant may leave access to some bugs.
<wallyworld_> that's the crux of my question i think
<wgrant> (this can't happen with a true proprietary project, because bugs can't be shared)
<wgrant> Also, fuck bug tasks, seriously.
<wallyworld_> wgrant: the sharing details page emits bugtask urls at the moment - the page is associated with a pillar, it iterates over the bug tasks and picks the one for that pillar. i should change it to just emit the bug url i think
<wgrant> wallyworld_: It's not a problem either way.
<wgrant> You can either see all bugtasks (modulo those on deactivated projects), or you can't see any at all.
<wallyworld_> it is because the exported api on sharing service breaks
<wallyworld_> it has a param which is a List(Ibug)
<wallyworld_> and instead it is getting IBugTask from the XHR call
<wgrant> Ah
<wgrant> Sure
<wallyworld_> so i should just change the logic on the sharing page - if *any* bugtask is associated with the pillar, emit the bug url
<wallyworld_> sharing details page i should say
<wgrant> No need to consider bugtasks at all in that case.
<wallyworld_> well, the page has a context which is a pillart
<wgrant> If APGF references an AA that references a Bug, that Bug is relevant.
<wgrant> You are already context-filtered by APGF
<wallyworld_> so the url is lp.net/pillar/+sharingdetails/person
<wgrant> Sure
<wallyworld_> the details page should only show bus and branches associated with the pillar
<wgrant> And that will say SELECT bug from apgf join aa join bug where apgf.policy IN (SELECT id FROM accesspolicy WHERE product=pillar)
<wallyworld_> so how to tell if a bug is associated with the pillar - it has a bugtask targetted at the pillar
<wgrant> You're already only getting bugs that are associated with the pillar.
<wgrant> AccessPolicyArtifact (which is flattened into APGF) already handles that.
<wgrant> There are triggers which mirror BugTasks' pillars into APAs.
<wallyworld_> right ok. so there's some existing erroneous logic in the view code i'll just remove. cool. i like deleting code :-)
<wallyworld_> i didn;t write it originally so wasn't 100% sure, just wanted to check
<wallyworld_> thanks
<StevenK> wallyworld_: Setting the link has FF follow it explicity
<StevenK> wallyworld_: And moving to the JS module has broken it, so I guess I'll grab you after the call tomorrow and work through it.
<nigelb> bigjools: re:timezone, it was mentioned as part of a talk that I'm supposed to be listening to.
<bigjools> yes...? :)
<nigelb> bigjools: so... I have no clue what timezone :P
<bigjools> heh
<wallyworld_> StevenK: ok. if i get a chance before then i'll look
<wgrant> lifeless: Around?
<lifeless> kinda
<wgrant> lifeless: Is the pgbouncer ProgrammingError hack in Storm your doing?
<lifeless> no
<StevenK> wallyworld_: I can paste a diff if you want to look?
<lifeless> allenap I think
<wgrant>         if isinstance(exc, disconnection_errors):
<wgrant>             # When the connection is closed by a termination of pgbouncer, a
<wgrant>             # ProgrammingError is raised. If the raw connection is closed we
<wgrant>             # assume it's actually a disconnection.
<wgrant>             if isinstance(exc, ProgrammingError):
<wgrant>                 if self._raw_connection.closed:
<wgrant>                     return True
<wgrant> Ah
<lifeless> possibly stub
<lifeless> I wrote the pgbouncer fixture
<lifeless> that was all
<wgrant> Ah, I see.
<wgrant> Bug #959699
<_mup_> Bug #959699: test failure: test_pgbouncer_stopped <Storm:New> < https://launchpad.net/bugs/959699 >
<wgrant> lifeless: Any opinions?
<wgrant> Upgrading psycopg2 to 2.4 seems painless except for that.
<lifeless> wgrant: I think upgrading is mandatory isn't it ?
<wgrant> lifeless: Yes.
<wgrant> lifeless: Well
<wgrant> I could backport the changes.
<wgrant> But that seems like just delaying the inevitable.
<wgrant> Extending the check to also allow DatabaseError unbreaks everything.
<wgrant> But seems bad.
<adeuring> good morning
<allenap> wgrant: That's mine (the pgbouncer hack).
<wgrant> allenap: Yeah, I see you also filed a bug upstream about this thing and disabled the relevant Storm tests.
<wgrant> The new exception seems legitimate.
<wgrant> So I think I'll fix Storm to accept it and undisable the tests.
<allenap> wgrant: Tip top.
<allenap> Thank you.
<mabac> StevenK, hi there! thanks for reviewing the team-engineering view. I've run format-imports on the changed files. is there anything else that needs fixing?
<StevenK> mabac: I'd prefer the feature flag change name since it isn't reigstry. But mabac: I'd prefer the feature flag change name since it isn't reigstry. But mabac: I'd prefer the feature flag change name since it isn't reigstry. But that isn't a requirement.
<nigelb> StevenK: Is there a joke I'm missing? Or did you just type that out 3 times?
<StevenK> No, I just fail at keyboard.
<nigelb> Haha. FAIL.
<mabac> StevenK, ok thanks. I'll scratch my head on that one. is there any guidelines about feature flags I can have a look at? that bit is all new to me
<nigelb> There's a wiki.. somewhere.
<mabac> thanks
<mabac> https://dev.launchpad.net/FeatureFlags#Naming_conventions
<nigelb> Ohman. I fail at finding things on launchpad wiki.
<mabac> StevenK, so it's the "area" part that you'd prefer to change? this is a page for a team which displays blueprint work items and bugs for upcoming milestones. can the feature flag be named 'team.upcoming_work_view.enabled' ?
<jml> is it actually possible to do Launchpad development on precise?
<jml>  launchpad-developer-dependencies : Depends: launchpad-messagequeue-dependencies (= 0.110~precise1) but it is not going to be installed
<lifeless> jml: does precise still have python2.6?
<lifeless> jml: my usual recommendation - lxc - will be made now.
 * lifeless recommends lxc
<nigelb> heh
<jml> lifeless: it has python 2.6
<cjwatson> it may have python 2.6 but it won't have many useful modules for it
<cjwatson> (afaik)
<jml> I'm following instructions on https://dev.launchpad.net/Running/LXC
<jml> but I don't seem to be able to log in to the lxc I've created.
<wgrant> jml: Your normal username and password should work
<wgrant> root/root used to work, but I don't think it does any more.
<jml> wgrant: ah right. my password too, that was the trick.
<matsubara> morning!
<rick_h> morning
<lifeless> nn
<mabac> rick_h, hi there! would you land the team-work-ui for me? we've discussed the feature flag name but will leave it as is. the formatting has been fixed. https://code.launchpad.net/~linaro-infrastructure/launchpad/team-engineering-view-ui/+merge/100707
<rick_h> mabac: sure thing
<mabac> rick_h, cool, thanks!
<rick_h> mabac: can you set the commit message on your MP please?
<mabac> rick_h, oh sure
<mabac> rick_h, done
<rick_h> is there a bug to link this to?
<rick_h> mabac: ^^ or should I land this no-qa?
<jml> I'm getting a recursion error when running 'make schema': http://pastebin.ubuntu.com/914410/
<mabac> rick_h, we have tested it a bit on staging but I'm not sure.
<mabac> salgado, do you think it can land as no-qa?
<salgado> rick_h, mabac, the new page is protected by a feature flag, so I think so?
<mabac> rick_h, that's right it's just us who are going to see it now anyway
<rick_h> mabac: how about just creating a quick bug to link it to noting about adding it and then we can go through the normal qa process
<mabac> rick_h, so yes, it's ok
<rick_h> not checked out the change, but just to play it safe
<mabac> rick_h, ok will do
<rick_h> thanks mabac, sorry to drag through the process more.
<mabac> rick_h, no problem
<mabac> rick_h, bug 973322
<_mup_> Bug #973322: We need a UI for the team engineering view <Auditor:New> < https://launchpad.net/bugs/973322 >
<jml> ah, ok
<jml> the version of bzr in Launchpad doesn't support colocated branches.
<jml> bugger.
<StevenK> mabac: It should not land as qa-ok
<jml> you guys really do like to make things hard for yourselves.
<StevenK> mabac: You want to link the bug to the MP, have the webops enable the feature flag and give it a bit of testing on qas, just to make sure.
<mabac> StevenK, ok thanks
<StevenK> When it hits qas, of course. :-)
<rick_h> StevenK: yep, that's where we're headed
<mabac> rick_h, StevenK salgado I've linked the bug to the branch
<rick_h> mabac: awesome, thanks
<mabac> rick_h, salgado I'll drop offline for half an hour to go home. will resurface asap and look out for trouble ;)
<rick_h> mabac: ok, landing has started, waiting on ec2 to kick up
<mabac> rick_h, great thanks!
<jml> wuu, running tests locally
<jml> uhhhhh
<jml> why does the Launchpad  PPA have dpkg?
<jml> Get: 1 http://ppa.launchpad.net/launchpad/ppa/ubuntu/ lucid/main dpkg 1.15.5.6ubuntu4.5+launchpad1 [2,193kB]
<jelmer> jml: support for .orig.tar.xz files in source packages or something IIRC
<jelmer> jml: have you checked the changelog ? :)
<jml> jelmer: I don't really know how to do that without installing
<StevenK> Download the diff and zcat it
<jelmer> jml: you can see it on the PPA page
<jelmer> or, at least the last entry, which is usually the most relevant
<jelmer> https://launchpad.net/~launchpad/+archive/ppa/+packages
<jml> jelmer: thanks.
<cjwatson> StevenK: native packages don't have a diff
<cjwatson> dpkg in the LP PPA is mostly my fault.  It can go away once LP upgrades to precise (at least until the next time there's a requirement for a new dpkg feature)
<jml> heh
<jml> I have an approved branch, could someone please land it? https://code.launchpad.net/~jml/launchpad/create-commercial-ppa/+merge/100635
<rick_h> jml: can you create/link the MP to a bug so it goes through normal qa please?
<rick_h> jml: after that I'll ec2 land it for you
<rick_h> jml: ty, running ec2 land now
<rick_h> see you tomorrow lol
<jml> rick_h: thank you.
* abentley changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: abentley | Firefighting: - | Critical bugtasks: 4*10^2
<jcsackett> sinzui: you free for a short chat?
<salgado> abentley, I've got a small-ish one up for review if you've got some time: https://code.launchpad.net/~linaro-infrastructure/launchpad/upcoming-work-progress-bars/+merge/100808
<abentley> salgado: sure.
<sinzui> jcsackett, I am, but let me kill some windows processes to be certain I have CPU and memory to chat
<jcsackett> sinzui: ok.
<czajkowski> sinzui: is this something the new privacy stuff will address on bugs do you think?  https://bugs.launchpad.net/launchpad/+bug/973228
<_mup_> Bug #973228: "Affects me too" text should acknowledge that you are affected by a duplicate <Launchpad itself:New> < https://launchpad.net/bugs/973228 >
<sinzui> czajkowski, no, that is a feature request
<sinzui> affects me too never manages duplicates. I suspect the bug is already reported
<sinzui> jcsackett, my experiment is making this difficult. Can we try a hangout via google messenger
<czajkowski> sinzui: thanks
<abentley> salgado: can items ever be 0?
<jcsackett> sinzui: we can try, sure.
<salgado> abentley, you mean a container having no items?  that would be a bug in our code
<abentley> salgado: Okay.  Your tests need docs, but r=me.
<salgado> abentley, thanks a bunch, I'll add them
<abentley> adeuring: Could you please review https://code.launchpad.net/~abentley/launchpad/celery-rabbit-config/+merge/100813 ?
<adeuring> abentley: sure
<adeuring> abentley: r=me
<abentley> adeuring: Thanks.
<salgado> abentley, added docs and fixed one of the tests to create specs explicitly and with different priorities so that they're always in the same order
<salgado> abentley, the prerequisite branch is being ec2-landed now.  would you mind ec2-landing this one as well?  if the prerequisite one fails on ec2 this one will fail as well, but if they both pass they will land in the correct order
<salgado> there's a chance that the ec2 run of the first one may die or have spurious failures and the second one goes through, though
<salgado> abentley, actually, the prerequisite branch just landed
<abentley> salgado: Ah.
<abentley> salgado: Even if it hadn't, it doesn't matter to me whether two reviewed branches land in a single revision or separately.
<salgado> abentley, I was concerned it could affect the qa-tagger but I guess both bugs would still be linked to that single revision?
<abentley> salgado: they should.
<bac``> sinzui: you should go back to bed ... until monday.
<timrc> you guys gave sinzui a bed? wow, nice
<sinzui> timrc, my bed has a computer and tablet next to it. I just work reclined
 * sinzui sometimes wishes that was not true
<timrc> sinzui, lol
<rick_h> deryck: working on getting python-ooptools going and when it has you create the user the passwordless bit isn't owrking right
<rick_h> deryck: it has a -a flag which --help doesn't show, should that be something else?
<rick_h> deryck: from: sudo -u postgres /usr/lib/postgresql/8.4/bin/createuser --port 5433 -a -d $USER
<deryck> rick_h, seems like there was a step before that.  schema and migrations.
<rick_h> that seems to come afterwards
<rick_h> deryck: I'm working on getting the second cluster up and need to create the db
<rick_h> but need my account to have perms to do so
<deryck> rick_h, I think I did this straight out of the README. let me look at bash history
<rick_h> deryck: thanks, yea what I'm trying to follow. In the end I'm tempted to just create a user with -s -P and move on since it's a local dev env and it's how I've setup my account before on other dbs
<rick_h> deryck: but if I'm missing someting would like to fix the README
<deryck> rick_h, too long ago now for me to have bash history, but I really don't think I changed anything in the commands...
<deryck> rick_h, but it does seem like I reordered someone them to what logically made sense, and things "just worked."
<rick_h> deryck: ok
<deryck> rick_h, I also used an email from lifeless to launchpad-dev as reference too.  let me see if I can find that
<abentley> deryck: I want to test some jobs that send emails, but we usually test email in-process.  Any idea how to do this?
<rick_h> deryck:  probably https://lists.launchpad.net/launchpad-dev/msg08183.html and the email "shortcut for getting python-oops-tools running"
<deryck> rick_h, yup, was just about to paste you a bit of that.
<deryck> abentley, thinking....
<rick_h> deryck: ok, both seem to assume the current db notes are correct so I'll give it another go, maybe I've missed something. -a just doesn't show up in --help so seemed might be a typo.
<deryck> rick_h, same version of postgres as readme, i.e. 8.4 and not 9?
<rick_h> yea, the command has the full path to the 8.4/bin
<deryck> abentley, you want to test the mail sent by the job? i.e. inspect the mail? Or test that the job fires off mail?
<abentley> deryck: I am okay just testing that the job fires off mail.  Unit tests handle the actual text.
<rick_h> deryck: doh, my fault. I didn't use the patch but manually edited pg_hba.conf and didn't do it right
<rick_h> deryck: thanks, sorry for the timesink
<deryck> rick_h, np! :)
<deryck> abentley, use some sort of faker mailer that stores what's sent in a variable you can inspect after the job completes?
<deryck> s/faker/fake/
<rick_h> deryck: abentley I keep meaning to try this out sometime: http://bazaar.launchpad.net/~bloodearnest/+junk/test-email-server/view/head:/README
<abentley> deryck: I guess, but then the problem is convincing an arbitrary Launchpad job to use a fake mailer.
<rick_h> ah right, running that on qa would be a bit out of the ballpark
<deryck> abentley, yeah, true.
<deryck> abentley, do all jobs use the same mechanism to send mail? Can you monkeypatch orotherwise intercept the thing sending mail before or after and test that somehow?
<abentley> deryck: potentially yes, but it's a subprocess, so you have to trigger the monkeypatching somehow.
<deryck> abentley, ah.  yuck.  it is kind of messy then.
<deryck> abentley, sorry, I'm a bit stumped myself.
<abentley> deryck: Well, worst case, I guess we can do an environment variable like USE_FAKE_MAILER.
<deryck> abentley, ah, yeah, that would work.
<abentley> deryck: Oh, I've got it.  I can write a celery Task that does mail_helpers.pop_notifications.
<deryck> abentley, nice!
<lifeless> moin
<rick_h> jml: heads up, emailed ec2 test results to you
<salgado> abentley, in case you still have some time today, I have another branch for review (https://code.launchpad.net/~salgado/launchpad/person-upcoming-work-view/+merge/100878).  It looks big but ~95% of it is just moving code around
<abentley> salgado: looking
<abentley> salgado: 320-321 can be written containers_by_date.setdefault(milestone.dateexpected, [])
<salgado> abentley, I'm not a big fan of setdefault() but I can do that
<abentley> salgado: I'm just mentioning it.  I like it.  Especially since it returns the value, and you can do setdefault(x, []).append('y')
<abentley> salgado: It might be nice to use adaptation for GenericWorkItem.  Then you could make SpecificationWorkItem implement IGenericWorkItem directly, and provide an adapter for BugTask.
<abentley> salgado: But since this is just shufffling pre-existing code around, r=me.
<salgado> abentley, thanks!  (fwiw, I've considered using adapters for that, but didn't see much advantage in doing so)
<salgado> abentley, would you mind landing this one for me as well?
<abentley> salgado: Sorry, EOD.
* abentley changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: - | Firefighting: - | Critical bugtasks: 4*10^2
<salgado> hmm
<salgado> anybody willing to ec2-land https://code.launchpad.net/~salgado/launchpad/person-upcoming-work-view/+merge/100878 for me?
<salgado> lifeless, maybe you as you seem to be the on call reviewer today? ;) (^)
<lifeless> salgado: my landing environment is a little horked just now
<lifeless> salgado: but I can find a victim for you
<salgado> lifeless, cool, that works as well :)
<lifeless> gary_poster: hi; is there anything you guys need from me at the moment ? just asking as we have a long weekend coming up
<wgrant> salgado-afk: Your branch is holy and being reverted.
<jelmer> wgrant: we have a problem with holy branches?
<jml> rick_h: thanks.
<jml> my recent lp branch failed a bunch of doctests because it changed login_person to return the person who is logged in
<jml> (meaning you can do 'with celebrity_logged_in("admin") as admin_person:' and other fun things)
<jml> so all of the doctests that do:
<jml> >>> login_person(foo)
<jml> >>> # something else
<jml> fail, because login_person isn't supposed to be returning anything, according to them
<jml> I think this is a stupidity of doctests
<StevenK> ign = login_person(foo) ?
<jml> just wondering if anyone has any opinions on what would be the cleanest thing to do
<jml> change all the call sites
<jml> or revert the API call, creating a private _login_person that *does* return and having the public login_person keep its None-returning behaviour.
<StevenK> You could do that too
<jml> StevenK: ign = ... would wok, but it would also flag pyflakes.
<StevenK> I didn't think pyflakes dealt with doctests?
<jml> StevenK: you can get it to if you want.
<jml> obviously I want to do the thing that's the least work, but I also think LP has had one too many cracks papered over.
<jml> I guess >>> ign = login_person(foo) is better than >>> login_person\n<Person ...>
<salgado> wgrant, what's wrong with it?
<wgrant> salgado: There's an XSS in the work item title rendering.
<wgrant> There's a reason I questioned your foo_link stuff a couple of days back -- it inevitably leads to mistakes like this.
<salgado> oops, indeed
<salgado> had you told me the reason I could've avoided the mistake
<salgado> wgrant, there's another branch being ec2-landed (under abentley's account) which includes the changes you're reverting as well.  not sure there's anything to be done about that before it lands as abentley is gone
<wgrant> salgado: It'll hopefully conflict in PQM.
<StevenK> Hm, qas seems faster.
<StevenK> Of course, as I say that, it times out 3 times in a row.
<rick_h> 041510
<rick_h> ok, I didn't even touch the yubikey that time...it's on crack
<StevenK> Hm. That reminds me, when is that call ...
<rick_h> StevenK: so 9pm your time I think if I used the time website right
<rick_h> 6am my time, and should hit the other two in the morning
<lifeless> wgrant: sanity check me: the SCA stuff isn't hung off of Ubuntu, its a global concept in the code base today, right ?
<rick_h> I didn't hear back from anyone though StevenK
<wgrant> lifeless: Yes :(
<lifeless> thanks
<lifeless> jml: and theres the ack.
<wgrant> celebrities ftl
<StevenK> rick_h: Perhaps not enough notice
<rick_h> StevenK: yea, was the battle of getting the process started in time to beat convoy to produciton while giving a day or two to make sure you can show up
<rick_h> but two days should be good for yay/nay I'd think
<StevenK> You'd think
#launchpad-dev 2012-04-05
<salgado> wgrant, how http://paste.ubuntu.com/915336/ looks as a fix for that?
<StevenK> rick_h: 10UTC is 8pm.
<StevenK> rick_h: I'll ping you at about 8pm, and if it's just us we'll skip it.
<rick_h> StevenK: sorry, went off of this http://everytimezone.com/#2012-4-4,-120,6be and you're in Sydney right? according to the directory
<StevenK> rick_h: Yup, I am.
<wgrant> salgado: Looks reasonable.
<wgrant> salgado: I didn't look further once I found that one, so there may be more holes.
<salgado> wgrant, the other possible uses of 'structure' in that template will render either a format_link(obj) or a string that's hard-coded in the template (e.g. Nobody, N/A)
<wgrant> salgado: That's what I thought, great.
<wgrant> That sort of thing is strongly discouraged nowadays, but this particular case seems fairly safe.
<salgado> yeah, I should've remembered that
<wgrant> lifeless: Since you're probably the only Stormy person around, https://code.launchpad.net/~wgrant/storm/psycopg2-2.4-pgbouncer/+merge/100896 would enjoy your review.
<salgado> wgrant, so, would you like to review that whole branch or should I just pester StevenK to review the incremental changes?
<wgrant> salgado: I'll glance over it but don't really have time to review the whole thing properly immediately.
 * wallyworld_ goes to get more coffee so there is not a shortage over the extended Easter break
<nigelb> ws 10
<nigelb> urgh
<salgado> wgrant, ok, that's fine
<StevenK> salgado: While I remember, you guys do need to fill out [r=santa] or [bug=1], our tools will do it for you when we go to land it.
<StevenK> Sigh. Do *not* need to fill out
<salgado> oh, cool
<salgado> wgrant, StevenK, so, if you guys are happy with https://code.launchpad.net/~linaro-infrastructure/launchpad/team-engineering-view-ui/+merge/100707 would you mind ec2-landing it for me?
<salgado> if not just tell me why and I'll fix it :)
<salgado> late dinner now
<lifeless> wgrant: sorry, I won't get to that today
<lifeless> wgrant: stub will be around noos
 * wgrant out for a while.
 * StevenK tries take two at finding a vocab that backs onto an enum
<StevenK> stub: Hai. wgrant is out for a while, can you glance at https://code.launchpad.net/~wgrant/storm/psycopg2-2.4-pgbouncer/+merge/100896 ?
<wgrant> stub: Could you glance over https://code.launchpad.net/~wgrant/storm/psycopg2-2.4-pgbouncer/+merge/100896?
<wgrant> Storm reviewers are a little hard to come by in this TZ
<stub> wgrant: already done
<wgrant> stub: Oh, indeed. Thanks.
<wgrant> The default isolation change seems happy with that fix.
 * wgrant ec2s
<stub> Cool. We still have a forked Storm I think, so we don't need to delay on a release.
<wgrant> Yeah, already merged that up and built a new egg.
<lifeless> have a good weekend y'all
<lifeless> stub: you get a pass on catchup today, I can't be arsed ;) [unless you have something specific to cover]
<bigjools> have a nice easter lifeless
<stub> wgrant: Staging seems happy enough after I bumped that lock count up to 1024 (default is 64). The isolation level change is still my preferred solution though.
<wgrant> Night lifeless.
<wgrant> stub: Yeah
<wgrant> stub: We'll see how much of a difference it makes
<wgrant> stub: Do we graph pg_locks size?
<stub> lifeless: If interest, we seem to have run out of disk space on staging. We no longer quite have enough space for 5 full copies of the production database. Freshly build, they clock in at 320GB. I don't know if it is database size growth over the last 3 weeks putting us over the edge, or if 9.1 is a little bigger. But that doesn't really matter.
<stub> wgrant: No
<wgrant> stub: https://code.launchpad.net/~wgrant/launchpad/psycopg2-2.4.4/+merge/100912 and https://code.launchpad.net/~wgrant/launchpad/9.1-serializable-is-special/+merge/100913
<lifeless> stub: theories? different fill rate?
<lifeless> /dev/cciss/c1d0p1     808G  381G  419G  48% /srv
<lifeless> so we have enoughspace to migrate on widcherry
<stub> wgrant: Both of those are approved
<wgrant> stub: Thanks.
<stub> lifeless: production isn't an issue, if there is an increase, it is <10%. Just when we multiply that by 5...
<stub> I can't know for sure unless I restore a dump onto an 8.4 system, which I haven't done.
<lifeless> you could bring up a canonistack instance to try
<wgrant> We'll have space to do that on sourcherry once the restore is done and we're down to 3 DBs, right?
<stub> lifeless: I don't have enough historical details to tell.
<wgrant> Can bring back the 8.4 instance for a few hours
<stub> lifeless: yes, or chokecherry has enough disk I think.
<lifeless> or wild :P
<stub> lifeless: But the why isn't really an issue
<stub> Interesting, but doesn't change the situation as it isn't severe enough to block PG 9.1 migration
<stub> Staging updates are the problem. I can change the process a bit to lose one full copy, but that work needs to be done.
<stub> (Swap the new staging databases into place *before* the replica has build, which isn't too much of an issue as it is usually only done on weekends)
 * stub wonders why he is consistently typing 'build' instead of 'built'  today.
<stub> mthaddon, lifeless: Looks like db growth. Just noticed qastaging's db is 305GB while the freshly built lpmain_staging is 316GB. Should have picked that up before.
<adeuring> good moorning
<czajkowski> adeuring: ello
<mabac> hi all. :) so https://code.launchpad.net/~linaro-infrastructure/launchpad/team-engineering-view-ui/+merge/100707 was backed out and the branch has been fixed up. how do we go about re-landing it?
<mabac> should the last revision be reviewed or can the change simply be landed again?
<nigelb> StevenK, wgrant: http://pic.nym.se/nRen.gif
<StevenK> nigelb: So, you take two steps into a bathtub and then slip?
<nigelb> heh
<mabac> StevenK, would you have a look at the last revision of https://code.launchpad.net/~linaro-infrastructure/launchpad/team-engineering-view-ui/+merge/100707 and re-land if if possible?
<czajkowski> mabac: https://dev.launchpad.net/ReviewerSchedule
<czajkowski> is the schedule StevenK is EOD /asleep
<mabac> czajkowski, thanks. I'm always confused by the "Europe" tz reviewers being crossed out.
<mabac> czajkowski, so it looked to me that noone is on call? is it inside the AsiaPac time-slot still?
<czajkowski> mabac: dont think so
<StevenK> mabac: Some of the Europe TZ reviewers are struck-out because they are working on MAAS, and not LP at the moment.
<rick_h> StevenK: ping, sorry, alarm fail, pinging others by hand now. Didn't here back from them in email
<rick_h> StevenK: you did see the email right? It made it out from my mailbox?
<mabac> StevenK, ah ok, thanks
<mabac> rick_h, hi. I feel like I always pounce on you when you get online. ;) would it be possible to ask you for yet another favor?
<rick_h> mabac: what's this?
<mabac> rick_h,  https://code.launchpad.net/~linaro-infrastructure/launchpad/team-engineering-view-ui/+merge/100707 got rejected and salgado fixed it up tonight. so it needs landing again
<rick_h> mabac: ok, I'm on call review today and my mentor will be able to ok in a couple of hours. I'll see about getting it through and landing this morning
<rick_h> will take a few since it's still early here on our side of the world
<mabac> rick_h, that's absolutely fine of course. thanks a lot! :)
<czajkowski> rick_h: morning
<rick_h> czajkowski: morning
<czajkowski> rick_h: what time is it where you are?
<rick_h> StevenK: ok, well if you get this sorry. No reply in irc from the other two either. Looks like this is one failed meeting setup
<StevenK> rick_h: Indeed. :-(
<StevenK> czajkowski: If my math is right, 7am
<StevenK> rick_h: I just got home from picking up my wife and shopping, so it's okay.
 * StevenK is liking the idea of bake-at-home pizzas for dinner ...
<nigelb> StevenK: hah. sounds delicious.
<StevenK> And indeed, it's what my wife and I bought stuff for. :-D
<nigelb> WIN.
<nigelb> hah. The linode users are getting pinged out.
<asac_> hola! is there an API to query sprint attendees ... or even better to register for a sprint?
<cjwatson> I don't see any API for sprints at all
<rick_h> sprints? maybe you're thinking something like the loco system?
<cjwatson> rick_h: such as https://launchpad.net/sprints/uds-q
<cjwatson> in fact I suspect asac_ means https://launchpad.net/sprints/lcq2-12
<rick_h> lol, learn something new every day...and I work on this stuff.
<asac_> cjwatson: yeah. thats what i mean
<asac_> we have to ensure that folks that register for connect get registered for the sprint. or at least we have to block submitting our registration form if they are not registered
<asac_> so either querying who is registered through API would be nice
<asac_> but better would be to register for them because we have all the data in our form already
<asac_> flacoste: ^^
<asac_> cjohnston found: https://launchpad.net/sprints/lcq2-12/+attendees-csv for me... which probably can cover for the "get attendees in a machine readable format" use case...
<asac_> would still love to register directly though
<asac_> spinning more ideas ... is there a launchpad URL trick to pre-populate something like https://launchpad.net/sprints/lcq2-12/+register ?
<cjohnston> it is prepopulated
<asac_> yeah
<asac_> i mean ... overloading the defaults
<cjohnston> if you link them to that and they are logged in, its good to go.
<asac_> with something i already had users type elsewhere
<cjohnston> ahh..
<wgrant> asac_: https://launchpad.net/sprints/lcq2-12/+register?field.time_starts=2012-05-29T09:30
<wgrant> asac_: You can override the field defaults like that
<asac_> wgrant: lovely
<asac_> let me try for the attendee
<wgrant> asac_: You probably want https://launchpad.net/sprints/lcq2-12/+attend, though
<wgrant> That's the self-attendance form
<asac_> wgrant: no
<asac_> the problem is that the folks add the nick of launchpad
<asac_> on their own
<asac_> so he might attend
<asac_> byut then i need to check if his nick is really registered
<asac_> hmm
<asac_> wgrant: do you know if there is a URL to get the launchpad nick?
<asac_> the currently logged in?
<wgrant> What for?
<wgrant> A third-party application can't access that from a user's browser.
* rick_h changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: rick_h* | Firefighting: - | Critical bugtasks: 4*10^2
<asac_> to fill it in a external webform
<cjohnston> asac_: http://connect.linaro.org/register-connect/  has an LP username field
<asac_> wgrant: i am talking about a web html form
<cjohnston> which is automatically filled in
<asac_> wgrant: we have a sprint registration form elsewhere ... and they add their launchpad nick there ... now we want to ensure that the user is registered
<asac_> otherwise it is a mess as all users will not have a schedule
<wgrant> asac_: I'm not sure how that relates to getting the Launchpad nick of the current logged in user.
<wgrant> Don't they give you their nick?
<asac_> cjohnston: that is automatically filled in?
<asac_> wow
<cjohnston> yes
<cjohnston> because you login to LP to fill out the form
<cjohnston> so its auto populated
<asac_> wgrant: seems we do it :)
<cjohnston> I'm sure its the WP openid plugin thats used
<asac_> through openid i guess?
<cjohnston> that gives us that info
<wgrant> Yes, you get that through OpenID
<asac_> ah its even not changable
<asac_> so thats good
<asac_> so yeah ... attending url
<asac_> with just dates
<asac_> can i get the csv file from a separate 3rd party website?
<asac_> in js ?
<wgrant> Browser security will prevent you from obtaining that CSV via AJAX.
<wgrant> You could grab it with a backend application, however.
<cjohnston> asac_: what about when you click register, we load in an iframe the https://launchpad.net/sprints/lcq2-12/+register?field.time_starts=2012-05-29T09:30 url and tell them to click save
<wgrant> that won't work
<wgrant> Framing Launchpad is forbidden
<cjohnston> ahh
<asac_> wgrant: yeah, so XSS prevention i guess?
<asac_> i felt that that would happen
<asac_> that list is available to not logged in users?
<wgrant> asac_: The CSV thing is just normal browser same-origin policy stuff, nothing that Launchpad asks for.
 * jml hugs testr
<asac_> so just a wget?
<wgrant> The iframeing is for clickjacking prevention.
<asac_> let me try
<jml> sinzui: ping
<asac_> seems not
<cjohnston> wgrant: /3
<cjohnston> sorry.
<jml> I had forgotten how slow Launchpad tests are.
<wgrant> They're slow and there's 18000 of them
<wgrant> Lots of those slow doctests :(
<jml> yeah. I'm running 81 of them.
<jml> is there a list of suggestions for things I can do if I'm struggling to phrase my patch with a negative line count?
<wgrant> Refactor tests to remove duplicated code, delete dead helpers, rip out Blueprint, that sort of thing...
<jml> but not an actual hit list
<sinzui> hi jml
<deryck> Morning, ll.
<jml> sinzui: hello
<deryck> all, even
<rick_h> morn
<jml> sinzui: I'm multi-tasking a bit now, but I want to talk later about creating private PPAs
<sinzui> jml: do you want to delete the existing windmill tests that are never run? I think that add up to 2000 lines
<sinzui> fab
<flacoste> morning deryck
<asac_> hi flacoste !!
<cjwatson> 2000 lines> that might actually result in an overall net reduction since the LoC policy was announced, then ...
<flacoste> hey asac
<salgado> abentley, that PQM failure was because a branch that was a prerequisite for that one had landed and then been reverted
<abentley> salgado: Okay.  Good thing it failed, then :-)
<salgado> indeed!
<mabac> rick_h, when you get to it, this is the updated mp for the team engineering view:  https://code.launchpad.net/~linaro-infrastructure/launchpad/team-engineering-view-ui/+merge/100956 It's only the last rev that's new.
<rick_h> mabac: yep, will do, thanks.
<salgado> mabac, I've also merged the XSS fix into the two other pending branches: https://code.launchpad.net/~linaro-infrastructure/launchpad/upcoming-work-progress-bars/+merge/100808 and https://code.launchpad.net/~salgado/launchpad/person-upcoming-work-view/+merge/100878
<salgado> they have been approved already, so it should be just a matter of asking somebody to ec2-land them once the first one has gone through
<mabac> salgado, cool thanks!
<asac_> salgado: mabac: hello my friends :)
<asac_> hehe
<asac_> salgado: can you brief me on the status of the registration form API?
<salgado> asac_, hi there.  doing some lp hacking as well? :)
<asac_> i heard you mihg know something
<asac_> i am getting strong
<asac_> your merge requests inspired me :)
<asac_> now i am trying to find the time
<asac_> hehe
<salgado> I've never heard about that... danilo took all that pain for himself, I'm afraid
<asac_> ok
<asac_> flacoste: do you have any records/pointers to share?
<flacoste> asac_: sorry, got dragged in phone calls
<asac_> no problem
<asac_> i have 1 minute left
<asac_> before i have to propose a solution :)
<asac_> hehe
<mabac> asac_, I have to say that the rest of us haven't worked on the registration
<asac_> yeah
<deryck> abentley, abel rolled a rock to my scissors.  he's your pre imp. :)
<asac_> thats fine
<abentley> deryck: :-)
<deryck> rick_h, don't forget to forward that bzr connections discussion stuff to abel, too, please.  just so he's in the loop about the last time this came up.
<rick_h> deryck: will do
<deryck> rick_h, thanks!
* jcsackett changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: jcsackett, rick_h* | Firefighting: - | Critical bugtasks: 4*10^2
<rick_h> deryck: sent
<deryck> rick_h, thanks!
<bac> hi flacoste, i never got that email that you mentioned having sent.  can you resend?
<flacoste> bac: i will
<bac> thanks!
<rick_h> salgado: still around? Reviewing and trying to find where upcomingwork is called/used?
<salgado> rick_h, the view? there's a link to it from the team's main page although it's protected with a feature flag
<rick_h> salgado: not the class view, but the method in browser/team.py?
<rick_h> bah, let me check this out, not enough context in the diff
<salgado> rick_h, oh, that's the method in the TeamMenu thing... it's used to generate the link in the template
<salgado> tal:define="link context/menu:overview/upcomingwork"
<rick_h> salgado: ah ok, thanks. I was searching for upcomingwork( and came up empty
<salgado> rick_h, unless you want to review the whole thing (which is fine :), you can just review the XSS fix which is what was not reviewed by StevenK.  this branch was reverted because of that XSS hole so that's the only thing that hasn't been reviewed yet
<rick_h> salgado: yea, saw that in mabac's branch. You guys threw me off with the superceded while I had it up
<rick_h> salgado: so sorry, just giving it a run through since I've got to put my name on it :)
<salgado> rick_h, nothing to be sorry for... it's fine if you want to review it all; just thought I'd mention what happened yesterday in case you were not aware
<rick_h> salgado: thanks, yep working on keeping/catching up
<flacoste> asac_: actually, the API to add a reigstree was never implemented
<flacoste> asac_: i recall danilo wanted to implement it, but I think it was never finished (if started)
<flacoste> asac_: i can't find an email to the list, so i think it was only an IRC chat where he discussed the concept
<asac_> flacoste: do you have logs and can see if you can spot something in 1-2 minutes? otherwise its ok
<asac_> i know what to do :)
<asac_> well... at least what to propose (e.g. lets do it different)
<asac_> i will do that assuming that nothing really happened :)
<cjwatson> I could use a review of https://code.launchpad.net/~cjwatson/launchpad/publish-proposed/+merge/100985
<flacoste> asac_: not that i can easily find, my grep didn't found it :-/
<rick_h> cjwatson: will do
<cjwatson> thanks
<asac_> flacoste: allright. thanks for your help
<flacoste> asac_: sorry, for not having something more concrete :-/
<asac_> no problem at all
<cjwatson> rick_h: OK, fixed the problems you identified (thanks!)
<cjwatson> I don't have landing privileges, but I guess this needs an ack from a non-* reviewer anyway?
<rick_h> cjwatson: yea, I've requested jcsackett take a peek and I can make sure to land it once he looks at it
<cjwatson> cool, thanks
<rick_h> cjwatson: can you post a commit message on your MP please?
<cjwatson> rick_h: done
<rick_h> cjwatson: ok, starting up the land process
<cjwatson> great, thanks
<rick_h> sinzui: ping, quick yay/nay for you. I'm trying to get my email branch notification to pass tests, but hitting tests where the user isn't logged in and so I don't have access to the preferredemail attribute.
<rick_h> sinzui: so this diff seems to work for the tests that have failed so far: https://pastebin.canonical.com/63886/
<rick_h> sinzui: any alarm bells going off for you I should watch out for?
<sinzui> rick_h, i an a bit apprehensive about the change
<rick_h> ok, should I instead look at altering the tests then to make sure they're in a login state like the ones I added for this?
<sinzui> rick_h, When purple squad was hardening objects over the last 6 months, we fixed a lot of tests that did not have logged in users
<rick_h> sinzui: ok, yea there aren't many. I'm still working through the 11 failures, not all were due to this
<rick_h> I think test_escaped_message_when_removing_key was the one that I first hit
<sinzui> rick_h, I think you changes are safe, but they imply someone does not know how to write a test...it is impossible to use launchpad (zope) with out an interaction. Something really must be logged in, or else this is anonymous...
<rick_h> sinzui: ok, I'll finish going through the tests and the ones that are left I'll take a look at
<rick_h> that is, those that fail without the remove security proxy
<rick_h> if I can't figure them out I might ping for assistance to make sure I'm twisting them right
<sinzui> rick_h, I guess my concern is about the user a few lines above your change
<sinzui> rick_h, I think the model/emailaddress.py is fine because we do similar changes in models
<sinzui> rick_h, I think model/emailaddress.py should be kept because deactivated users will get spam
<rick_h> sinzui: sorry, not following you. what should be kept?
<sinzui> your second change ensures we do not send spam to deactivated and suspended users
<rick_h> sinzui: right, sorry. That one isn't goign anywhere. It fixes some of the 11 test failures. I was speaking specifically on the remove proxy thing
<rick_h> sinzui: so sounds like the suggestion is to back that out, find which of the 11 tests fail on that change, and check if the tests needs updating
<sinzui> rick_h, The only legitimate reason why for 'user = IPersonViewRestricted(event.user)'  failing is because the fields were changes by a job running as someone other than the user who owns the address.
<sinzui> rick_h, I do not think a job in involved...I think the tests are bad
<rick_h> sinzui: ok, it's happening in the ssh tests
<rick_h> doc/sshkey.txt so far
<sinzui> yep, I anticipated that and already checked that it never calls login
<sinzui> that test is bogus...it is demonstrating anything can compromise a user's sshkey, which certainly never was true
<rick_h> logintoken-corner-cases.txt is failing with the same issue as well
<sinzui> Looks like the first test needs to call login_person(name16) and login_person(name12)
<rick_h> ok
<sinzui> rick_h, these tests predate login_person() when they were created, you had to know/get the email address which was not always easy
<rick_h> xx-add-email fails blankly, and webservice/xx-person is the same unauth'd error
<rick_h> so down to 4 tests to fix
<rick_h> sinzui: thanks I'll add that and give it a go and check on the other 4 tests then and not use the removeSecurityProxy in my code itself
<sinzui> okay
<mabac> jcsackett, rick_h thanks for the review. I have changed the todo comment to the proper format.
<rick_h> mabac: ok cool thanks. Pulling update now to help land
<mabac> rick_h, great thank you
<mabac> rick_h, will it work to land the other branches salgado mentioned too then?
<mabac> salgado> mabac, I've also merged the XSS fix into the two other pending branches: https://code.launchpad.net/~linaro-infrastructure/launchpad/upcoming-work-progress-bars/+merge/100808 and https://code.launchpad.net/~salgado/launchpad/person-upcoming-work-view/+merge/100878
<rick_h> mabac: ?? sorry didn't peel at other ones
<mabac> <salgado> they have been approved already, so it should be just a matter of asking somebody to ec2-land them once the first one has gone through
<rick_h> ah no, those will have to be dealt with on their own I think, looking
<rick_h> ah ok, yea I'll have to pull each and land them one by one
<mabac> rick_h, ok. I'll peek back in a while and see if I need to do anything. thanks for looking :)
<rick_h> mabac: can you set commit messages on the mp please?
<rick_h> looks like the other two have it set
<rick_h> sinzui: ping, have a sec for this last test please?
<sinzui> yes
<rick_h> https://pastebin.canonical.com/63903/ is the test bit blowing up on me
<rick_h> https://pastebin.canonical.com/63902/ is the test failure
<rick_h> the test seems to be doing some trick with trying to help an anonymous user get access, but since it's not logged in the access to the email property fails
<sinzui> that is crack
<rick_h> I feel like trying to get the right user logged in to fix my issue breaks the spirit of the test there
<sinzui> create user, login user, do change, logout
<sinzui> login_person(ssh_user)
<rick_h> ok, but then the login(ANONYMOUS) is basically invalid/removed?
<sinzui> yes it is invalid
<rick_h> ok, sinzui any hint on this thread._local? https://pastebin.canonical.com/63907/
<rick_h> I just moved the login call below the ssh_user = getUtility line and changed it to ssh_user
<sinzui> yep
<rick_h> so https://pastebin.canonical.com/63908/ is making the thread._local error on the getByName call
<rick_h> sinzui: nvm, got it. Have to login first with the email
<sinzui> okay
<sinzui> rick_h, sorry the lag. I am still trying to find a reliable ie8 setup to test lp.dev
<rick_h> sinzui: gotcha, np. I think I finally updated the tests to pass. Had to change a bunch in xx-add-email since there's now two messages for each test
<sinzui> yep
<salgado-afk> mabac, rick_h, I've just set the commit message on https://code.launchpad.net/~linaro-infrastructure/launchpad/team-engineering-view-ui/+merge/100956
<salgado-afk> jcsackett, maybe you can land that branch?  rick_h was waiting for us to set the commit message before he'd do it but it looks like it was too late when I did it
<rick_h> salgado-afk: I'll try to get it going tonight. Sorry, family dinner time and such. Once the boy's in bed I'll run it
<salgado-afk> rick_h, no worries, I didn't have high hopes for that anyway as I thought everyone would be gone by now. :)
<rick_h> salgado-afk: meh, I'm a junkie. I'm also in tomorrow while most people are off so worst case it's first on my todo in the morning
<rick_h> but I'll get it going in a couple of hours
<salgado-afk> rick_h, oh, do it tomorrow then!
<salgado-afk> it won't make any difference to us... I just wanted to have it ready for QA on Monday
<rick_h> salgado-afk: np, I'll get them in. Just those three right?
<rick_h> or are there any others I need to know of?
<salgado-afk> rick_h, nope, just those 3 ones.  you might want to submit just the last one, actually, as it includes the previous two
<rick_h> k, not sure how it works but I'll check it out.
<rick_h> anyway, have to get the boy to the bath, bbl
<salgado-afk> thanks a lot, rick_h!
<[reed]> where should I send possible launchpad security bugs?
<[reed]> just file them a security bug in launchpad?
<rick_h> [reed]: yes, file them as security bugs in LP and it'll get notified to most everyone
<[reed]> thanks
<[reed]> rick_h: filed, thanks again.
<rick_h> [reed]: thanks, with the holiday weekend in effect might take a few, but it'll get looked at.
<[reed]> ah, yes
<[reed]> I was wondering why Canonical people were saying they were gone tomorrow
<[reed]> hah
 * [reed] doesn't have the day off :(
<rick_h> yea, US doesn't have it, but most of the other devs do
<[reed]> ah
<wgrant> [reed]: Thanks, I've commented on the bug.
#launchpad-dev 2012-04-06
<ravi_> hi, I am trying to download the stable branch of lunchpad using bzr but the process in stuck at in-between. Do I need anything else than bzr version control installed in my system?  I am using ubuntu natty and bzr version 2.3.4
<rick_h> morning
* bac changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: bac | Firefighting: - | Critical bugtasks: 4*10^2
<salgado> rick_h, hi there. did you get an email with test failures from that branch you tried to land? (I usually don't get them because it's a team-owned branch)
<rick_h> salgado: no, I submitted them both first thing this morning
<rick_h> they should be sitll running
<rick_h> I landed two, the second one has a dependent branch, but only one
<salgado> rick_h, oh, ok, perfect :)
<rick_h> salgado: if I get anything I'll make sure to forward it along
<salgado> rick_h, that'd be great, thanks!
<rick_h> np
<rick_h> salgado-afk: ping
<salgado> hi rick_h
<salgado> I saw that the branches have landed
<rick_h> salgado: yea, one failed because the commit message was rejected
<rick_h> I'm inthe middle of upgrading to precise and having issues getting my LP env set back up
<salgado> rick_h, oh, that's fine, the other ones included the one which was rejected
<salgado> let me confirm that
<rick_h> salgado: http://paste.mitechie.com/show/602/
<salgado> rick_h, yeah, don't worry about that.  all the changes are in devel now :)
<rick_h> salgado: ah ok then cool
* bac changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: - | Firefighting: - | Critical bugtasks: 4*10^2
 * cjwatson upgrades mawson
<cjwatson> (QAing r15068)
<cjwatson> failed to upgrade: http://paste.ubuntu.com/918210/
<cjwatson> help welcome whenever anyone gets a moment
<cjwatson> (me, I'm off to bed)
#launchpad-dev 2012-04-07
<cjwatson> Ah, maybe somebody needs to install the convoy package on mawson.  launchpad-developer-dependencies depends on it, but mawson doesn't have that installed.  (Should it be in launchpad-dependencies instead, if it's required to run 'make'?)
<cjwatson> really bedtime
<cjwatson> StevenK: I don't know if you saw my query about convoy on mawson earlier; looking at RT#50616 perhaps this is something you're involved with?
<_mup_> Bug #50616: ValueError while validating image file using Python Imaging Lib <lp-foundations> <oops> <qa-ok> <Launchpad itself:Fix Released by sinzui> < https://launchpad.net/bugs/50616 >
<cjwatson> StevenK: it looks like it's blocking my QA
<cjwatson> (I know it's a holiday, get back to me when you have time :-) )
<wgrant> cjwatson: Ah, there was a workaround for that which I shelved on Friday
<wgrant> cjwatson: mawson uses dev-only make targets
<cjwatson> and doesn't have launchpad-developer-dependencies installed
<cjwatson> ok, so if I just 'bzr unshelve'?
<cjwatson> -inplace: build combobuild logs clean_logs
<cjwatson> +inplace: build logs clean_logs
<wgrant> yeah
<cjwatson> ok, done, doing rest of upgrade-dogfood-launchpad by hand
<wgrant> Thanks
<cjwatson> all done
#launchpad-dev 2013-04-02
<StevenK> wgrant: Did you see OOPS-fe4a82b31c1ecb7cca1a1da04b3aaef1?
<wgrant> Yay
<StevenK> I have a branch that adds a test, but I can't my test to break in the same way.
<wgrant> StevenK: Note that it's deleting a series that contains a milestone that is referenced by a work item
<StevenK> Right, and the code to delete a series will delete all milestones first
<wgrant> Yes
<StevenK> My test is triggering: AssertionError: You cannot delete a milestone which has specifications targeted to it.
<wgrant> StevenK: It needs to be a workitem, not a spec.
<StevenK> Which requires a spec
<wgrant> Sure
<wgrant> The spec doesn't have to be targeted.
<StevenK> IntegrityError: update or delete on table "milestone" violates foreign key constraint "specificationworkitem_milestone_fkey" on table "specificationworkitem"
<StevenK> DETAIL:  Key (id)=(13) is still referenced from table "specificationworkitem".
<StevenK> wgrant: So, the delete code does not look for specificationworkitem where deleted = True
<wgrant> Ah
<StevenK>         mapper = lambda row: row[0]
<StevenK>         return DecoratedResultSet(ordered_results, mapper)
<wgrant> itemgetter
<StevenK> (ordered_results, itemgetter(0)) ?
<wgrant> Yes
<StevenK> wgrant: https://code.launchpad.net/~stevenk/launchpad/workitems-delete-series/+merge/156457
<wgrant> StevenK: Does the delete method need an assertion for specificationworkitems as it has for specifications?
<wgrant> Also, you shouldn't be tested that at the series level.
<wgrant> testing
<StevenK> wgrant: You can't remove workitems, they just set to deleted=True
<StevenK> wgrant: I can shift the test to the milestone level if you wish, but I suspect the test will fail because of the specification.
<StevenK> Deleting a milestone performs checks against it, but I suspect the same checks aren't done against the series' milestones when you remove a series
<wgrant> StevenK: But the spec shouldn't be targeted, should it?
<wgrant> And I know you can't remove them -- that's why the browser code unsets them
<StevenK> wgrant: It didn't seem to touch workitems
<StevenK> http://pastebin.ubuntu.com/5669457/ fails with MismatchError: None is not <Milestone at 0xf1377d0>
<wgrant> StevenK: With what error?
<StevenK>     self.assertIs(None, workitem.milestone)
<StevenK> MismatchError: None is not <Milestone at 0xf1377d0>
<wgrant> StevenK: But what was the form error?
<wgrant> There must have been one, if it was not deleted.
<wgrant> Or your form post was invalid.
<StevenK> wgrant: The assert above that one is self.assertEqual([], view.errors)
<wgrant> StevenK: Sure, so perhaps you didn't specify a valid form action
<StevenK> wgrant: I doubt it, as lp.registry.browser._deleteMilestone() fires
<wgrant> StevenK: Oh, I guess they're not actually proper errors, or there is a confirmation step
<wgrant> Check the body that you get back.
<StevenK> wgrant: From view.request.response?
<wgrant> StevenK: view()
<StevenK> wgrant: http://pastebin.ubuntu.com/5669521/
<wgrant> Oh
<wgrant> That wasn't using c_i_v?
<StevenK> It is using c_i_v()
<wgrant>     view = create_view(
<wgrant>         context, name, form, layer, server_url, method, principal,
<wgrant>         query_string, cookie, request, path_info, rootsite=rootsite,
<wgrant>         current_request=current_request, **kwargs)
<wgrant>     view.initialize()
<wgrant>     return view
<wgrant> c_i_v, as the name suggests, initializes the view
<StevenK> view() == view.render(), no?
<wgrant> Sort of
<wgrant> render() isn't a thing
<wgrant> I think Launchpad views use it internally
<wgrant> But I'm not sure Zope has it as a thing at all
<StevenK> But yes, I see that it calls self.initialize(), which is odd
<StevenK> Anyway, do you believe me now? :-)
<wgrant> StevenK: Did you perchance do view() outside the person_logged_in block?
<wgrant> Perhaps the form does other things when the owner is not logged in
<StevenK> wgrant: I do, yes
<StevenK> wgrant: Same traceback if I move the print into the with block
<wgrant> StevenK: Add some prints to initialize() or pdb it
<wgrant> And see why c_i_v doesn't trigger the exception
<wgrant> Or read initialize() and work out wtf is so special about it
 * wgrant finds the relevant form
<wgrant> It appears to not override it
<wgrant> StevenK: What if you manually call view.render() rather than view()?
<wgrant> That'll let us see the content of the first submission without causing the exception
<StevenK> wgrant: That thing that isn't a thing?
<StevenK> LocationError: 'link-display-name-id'
<StevenK> Let me reflow so it doesn't have to traverse milestone
<wgrant> StevenK: It's a thing for LaunchpadView derivatives, since LaunchpadView.__call__ calls render()
<wgrant> But in general it's not
<StevenK> wgrant: http://pastebin.ubuntu.com/5669535/
<wgrant> Ah, of course
<wgrant> StevenK: store.flush()
<StevenK> wgrant: Right, the flush now shows the error
<wgrant> Yeah
<wgrant> Forms often leave things unflushed
<wgrant> Often exposing bug #812176
<_mup_> Bug #812176: Exceptions during commits are not handled by the publisher <critical-analysis> <oops> <regression> <webapp-infrastructure> <Launchpad itself:Triaged> < https://launchpad.net/bugs/812176 >
<StevenK> wgrant: So, this still leave us with the original issue, that _deleteMilestone didn't do anything with workitems
<StevenK> Which you objected to
<wgrant> StevenK: I'm confused
<wgrant> _deleteMilestone clearly needs to do for workitems what it does for specs.
<wgrant> StevenK: It may also be wise to consider how milestone deletion handles spec privacy
<wgrant> It correctly bypasses it for bugs
<StevenK> Badly
<StevenK> And yeah, I've noticed that
<wgrant> I don't remember what it does for specs
<wgrant> And the MP diff is in another window
<StevenK>             user = getUtility(ILaunchBag).user
<StevenK>             return list(target.getSpecifications(user))
<wgrant> Ah
<wgrant> So broken.
<StevenK> Yeah
<wgrant> And slow
<StevenK> wgrant: Would you like another test for that case?
<wgrant> StevenK: Better that than a 403
<wgrant> Actually, it'd just plain crash, wouldn't it
<wgrant> With a 502, probably
<StevenK> Yes, it will OOPS
<StevenK> wgrant: Hmmm, my test passes, I suspect because the owner of the product has access to the spec
<wgrant> StevenK: Indeed, you'll probably need to revoke the APG or similar.
<StevenK> wgrant: I do wonder if I can be evil and just .set(access_policy=None, access_grants=None)
<wgrant> StevenK: That's evil and depends on the implementation of the underlying queries. I wouldn't.
 * StevenK fixes the passing test to revoke the correct AP
<StevenK> DETAIL:  Key (product, id)=(27, 13) is still referenced from table "specification".
<StevenK> And that's an OOPS
<StevenK> wgrant: http://pastebin.ubuntu.com/5669598/
<wgrant> StevenK: I don't understand why _get_specifications was like that before
<wgrant> _getSpecifications
<StevenK> Because Milestone doesn't use IHasSpecifications, and only provided getSpecifications()
<wgrant> Ah
<wgrant> So, having private members in an interface seems like a seriously strange idea
<wgrant> Does getSpecifications not provide a give-me-everything mode?
<StevenK> Nope
<StevenK> Neither does search_specifications(), actually
<StevenK> -        assert self.getSpecifications(user).count() == 0, (
<StevenK> +        assert self._all_specifications.count() == 0, (
<StevenK> wgrant: ^ That's in IMilestone.destroySelf(), changed after that paste.
<wgrant> StevenK: :)
<wgrant> (pointless use of count, but it'll be an OOPS either way)
<StevenK> Oh? It's a resultset
<wgrant> .any()
<wgrant> .count() is expensive
<StevenK> wgrant: http://pastebin.ubuntu.com/5669619/
<wgrant> StevenK: Is _all_specifications really on the other interface?
<wgrant> Also, those assertions are inverted
<wgrant> And _getSpecifications is pointless
<StevenK> Oh, hah
<StevenK>     def _all_specifications(self):
<StevenK>         """See IHasSpecifications."""
<StevenK>         user = getUtility(ILaunchBag).user
<StevenK>         return self.specifications(user, filter=[SpecificationFilter.ALL])
<wgrant> Security proxies will also probably ruin your day
<StevenK> _getSpecifications is called by children
<wgrant> Sure
<wgrant> But look what it does.
<StevenK> _getSpecifications() ripped out, and I think I've defeated the security proxy
<StevenK> wgrant: http://pastebin.ubuntu.com/5669642/
<wgrant> +        return list(self.context._all_specifications) + [
<wgrant> +            list(milestone.getSpecifications(self.user))
<wgrant> +                for milestone in self.milestones]
<wgrant> Isn't that filtering by user?
<StevenK> wgrant: Sure, since that's actually displayed on the page
<wgrant> Oh, right
<wgrant> StevenK: By .any() I actually mean .is_empty(), of course. And I'm still not happy with the _ method in configure.zcml.
<StevenK> wgrant: http://pastebin.ubuntu.com/5669672/  What else can I do? Milestone hardcodes everything in the ZCML
<wgrant> StevenK: Well, declaring a private attribute as accessible from the outside seems a tad iffy...
<StevenK> wgrant: I wanted to avoid rSP
<wgrant> StevenK: That might mean that it's not a private attribute.
<StevenK> wgrant: http://pastebin.ubuntu.com/5669749/
#launchpad-dev 2013-04-03
<StevenK> wgrant: http://pastebin.ubuntu.com/5669749/
<wgrant> StevenK: Having an _ method on an interface makes even less sense than having it in configure.zcml.
<wgrant> StevenK: If you really need to access it from outside, particularly through a proxy, it should not be private.
<StevenK> wgrant: I don't want to rename it because then I have to re-grow the _getSpecifications method
<wgrant> StevenK: Why?
<wgrant> We control both sides of the code
<wgrant> We can rename it on both
<wgrant> ez
<StevenK> But the ProductSeries is via IHasSpecifications, which has far wider impacts.
<wgrant> StevenK: Er
<wgrant> StevenK: IHasSpecifications contains _all_specifications?
<wgrant> wtf, so it does
<StevenK> Yes
<wgrant> wtf
<StevenK> And it's exported
<StevenK> So we're stuck with it
<wgrant> It's exported without _
<wgrant>         exported_as="all_specifications", as_of="devel")
<wgrant> Also, why is that exported :/
<StevenK> So it's _all_specifications for beta and 1.0 ?
<wgrant> Sigh
<wgrant> No
<wgrant> Blueprints aren't exported at all before devel
<StevenK> Ah
<wgrant> What do people have against sensible search methods?
<StevenK> So which one of us is going to check appserver logs for all_specifications? :-)
<wgrant> It's going to be used
<wgrant> Because there's no specification search method AFAICT
<StevenK> search_specifications
<wgrant> Not on the API
<StevenK> Right
<wgrant> Also note that the fact it's exported means that you can't change it to actually return all specifications
<wgrant> It has to use LaunchBag to get the user
<wgrant> So perhaps create a new method _SERIOUSLY_PEOPLE_THINK_ABOUT_IT_BEFORE_YOU_EXPORT_SOMETHING_all_specifications
<wgrant> And export that instead
<StevenK> Hahaha
<StevenK> wgrant: So, IHasSpecifications._all_specifications gets renamed to visible_specifications, and left exported as it is.
<StevenK> wgrant: IMilestone._all_specifications gets renamed to something, and IHasSpecifications grows a method named the same that returns all specs
<wgrant> StevenK: Right.
<StevenK> wgrant: Suggestions for the 'something' ?
<StevenK> I think ISpecificationSet._all_specifications can die horribly, since I think only two tests use it
<wgrant> StevenK: Great
<StevenK> Hmmmmm, I don't think I can do all_specifications generically
<StevenK> Since not all of the ISpecificationTargets have columns in specification
<StevenK> wgrant: Will you accept I{Milestone,ProductSeries}.all_specifications which ignore privacy?
<wgrant> StevenK: Sure
<StevenK> ForbiddenAttribute: ('set', <storm.store.ResultSet object at 0x108f6110>)
<StevenK> I thought that was the right thing
<wgrant> StevenK: It is, but it's dangerous so it's not exposed by IResultSet
<StevenK> wgrant: http://pastebin.ubuntu.com/5672432/
<wgrant> StevenK: Hm, is SpecificationSet.__iter__ actually used by anything?
<StevenK> That's slightly harder to check
<wgrant> Yeah
<wgrant> Just saw it in thebranch
<wgrant> Not really relevant
<wgrant> StevenK: That looks reasonable.
<StevenK> I don't think it's used.
<StevenK> But it's hard to grep for.
<StevenK> wgrant: I can remove it from the model and interface and claw bit a little bit of LoC and then we'll see if buildbot loves or hates it?
<wgrant> StevenK: Maybe, but there might be webservice stuff that wants it
<StevenK> wgrant: It's not exported
<StevenK> And I don't think ISpecificationSet is either
<wgrant> StevenK: Ah, true
<wgrant> BugSet is special, because bugs live in the root
<wgrant> So yeah
<wgrant> Kill it and fix any tests that break
 * StevenK waits for the scanner before he re-pushes
<StevenK> Bwaha
<StevenK> There is one test failure, and I caused it
<StevenK> wgrant: https://code.launchpad.net/~stevenk/launchpad/workitems-delete-series/+merge/156457 has updated
<wgrant> 273	+ return list(self.context.visible_specifications) + [
<wgrant> 274	+ list(milestone.getSpecifications(self.user))
<wgrant> 275	+ for milestone in self.milestones]
<wgrant> StevenK: That's going to create a list of specs and then lists of specs
<StevenK> Hmmmm
<wgrant> StevenK: Do you need to doNotSnapshot IMilestone['all_specifications']?
<wgrant> IProductSeries too
<StevenK> Oh, where is that?
<StevenK> Oh.
<StevenK> wgrant: I can't just wrap the Attributes in doNotSnapshot?
<wgrant> StevenK: You probably can, yes
<StevenK> Gah, I was hoping to come up with a elegant solution for the ProductSeriesView.specifications :-(
<StevenK> I thought chain() could do it, but my testing isn't having much success
<wgrant> chain is right
<StevenK> Bleh
<StevenK> wgrant: But it isn't quite
<wgrant> StevenK: Oh?
<StevenK> wgrant: I can't just feed the comprehension into chain because then I can't add it to the first list
<StevenK> And I'm not sure about the wisdom of the a cachedproperty returning a iterator directly
<wgrant> StevenK: Why not? list(itertools.chain(whatever.specs, *[list of lists])
<wgrant> )
<StevenK> Hah, which is 4 lines long anyway
<StevenK> wgrant: And diff updated
<wgrant> StevenK: Thanks, r=me
<StevenK> wgrant: Do you still have objections to socket.setdefaulttimeout() for checkwatches?
<wgrant> StevenK: Internal calls will probably override that already
<wgrant> Though maybe not
<wgrant> Anyway, it sounds like we have some rogue HTTP calls
<wgrant> Which are not just bad because they miss timeouts
<StevenK> I couldn't see any rogue calls when I was reading xmlrpc and bugzilla externalbugtracker yesterday
<wgrant> StevenK: We also have a traceback from one of the recent hangs, don't we?
<StevenK> wgrant: On carob
<StevenK> wgrant: ~hloeung/7728.txt
<StevenK> That looks to implicate lib/lp/bugs/externalbugtracker/xmlrpc.py (109): request, but that sets a timeout.
<wgrant> Yeah
<wgrant> That's what I thought
<wgrant> So we need to find out why that didn't work
<wgrant> You know...
<wgrant> We should take down code imports more often
<wgrant> Frees up the build farm
<StevenK> Haha
<StevenK> wgrant: I do not get it.
<StevenK> urllib2 looks fine in terms of timeout handling
<wgrant> StevenK: Could it be hanging on squid.internal?
<wgrant> We don't actually get a timeout exception back
<wgrant> So it's possible that the TCP connection is still active, but it only makes it as far as squid.internal
<wgrant> All roads lead to squid.internal, as they say
<StevenK> The connection on loganberry was to the remote IP
<StevenK> Not breadfruit
<wgrant> Ah, true
<wgrant> That's a bit upsetting
<StevenK> wgrant: And urllib2 hasn't changed significantly between 2.6 and 2.7
<wgrant> Ah
<wgrant> I may finally have uncovered the untranslated translation credit bug.
<StevenK> Is it another wrong LEFT JOIN? :-)
<wgrant> No
<wgrant> Message sharing, as usual :)
<StevenK> I do not understand that checkwatches hang. Which makes me very sad.
<wgrant> I can have rosetta make you very sad instead...
<StevenK> If you want me to help, sure. But you'll have to get me up to speed first. :-)
<czajkowski> aloha
#launchpad-dev 2013-04-04
<StevenK> wgrant: Shall we deploy, or do you think 4 revisions is not worth it?
<lifeless> you know you want to
<wgrant> StevenK: There's that translations fix that people are waiting for, despite it being pretty minor
<wgrant> It excludes like 50 strings out of more than 50000 :/
<wgrant> So we should probably deploy
 * StevenK kicks the wiki
<wgrant> (and those strings are all ""!)
<wgrant> StevenK: Ohhh, nice catch
<StevenK> wgrant: I *think*
<StevenK> I'm not sure about it at all, but it's plausible
<wgrant> Yeah
<wgrant> I'm out of other ideas
<wgrant> We fairly often encounter redirects, mostly http->https
<StevenK> Right
<StevenK> It's plausible enough
<wgrant> r=me
<StevenK> wgrant: Have you seen the current buildbot failure before?
<wgrant> StevenK: Not in years. Force, or maybe check if there's stuff running on the slave
* wgrant changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | Firefighting: - | Critical bugs: <115
<wgrant> StevenK: https://code.launchpad.net/~wgrant/launchpad/bug-503421/+merge/157033
<lifeless> yay dino.
<wgrant> Hm?
<lifeless> the cleanup crap bug you invalidated
<lifeless> the dude is back
<wgrant> Oh good
<wgrant> One wonders why he cares if I state that the Launchpad team does not.
<lifeless> also wtf headdesk bug 45419
<_mup_> Bug #45419: No way for unprivileged users to report (probable) spam or abusive content within Launchpad <canonical-losa-lp> <comments> <feature> <gatekeeper> <lp-foundations> <spam> <Launchpad itself:Triaged> < https://launchpad.net/bugs/45419 >
<lifeless> reporting spam comments there.... is spam
<wgrant> You might think so
<wgrant> But our users apparently disagree...
<lifeless> they are wrong
<wgrant> Certainly.
<wgrant> danilos: Do you have an opinion on bug #1162192? You appear to have been involved 18 months ago in the design of the daily garbo job which removes obsolete potmsgsets (sequence == 0), which seems to basically make obsolete potmsgsets pointless.
<_mup_> Bug #1162192: Partial pot-files destroy translations <lp-translations> <Launchpad itself:New> < https://launchpad.net/bugs/1162192 >
<wgrant> https://bugs.launchpad.net/launchpad/+bug/814576
<_mup_> Bug #814576: Remove unused POTMsgSets <qa-ok> <translations-handoff> <Launchpad itself:Fix Released by gmb> < https://launchpad.net/bugs/814576 >
<StevenK> wgrant: Sorry, I was out. r=me
<danilos> wgrant, hi, that does sound like a huge problem; I don't know why would I have implemented it like that, because you are right that it would make obsolete POTMsgSets obsolete
<danilos> wgrant, I thought I helped implement a potmsgset pruner for potmsgsets which do not appear in any translationtemplateitems though, not obsolete ones (obsoleteness has nothing to do with it)
<danilos> wgrant, this was probably a bad design on my part :/
<danilos> wgrant, basically, we should not have done that for non-shared templates -- my thinking was probably modelled by the multi-series template sharing mechanism :/
<wgrant> danilos: Yeah, I thought it seemed a bit strange to do it for obsolete messages, at least without a reasonably lengthy timeout. So it should just remove them when they're not referenced by any TTIs at all?
<danilos> wgrant, yeah, but that holds all too easily if there's only a single potemplate in the "sharing set" (i.e. template with the same name among different series or in ubuntu)
<danilos> wgrant, oh, I see, you are suggesting dropping the sequence=0 search
<wgrant> danilos: What's the rule for sharing? Same (distribution, spn) or product, and same template name? Or same domain?
<lifeless> wgrant: btw https://bugs.launchpad.net/glance/+bug/1073569 - ttx is having timeouts trying to edit task status
<danilos> wgrant, a bit more complex: same (distribution, spn, templatename) or (product, templatename), but also if there is a link between a (distribution, spn) to product, then they get joined into a bigger set
<wgrant> Oh right, I remembered the packaging case, but didn't manage to actually type it
<wgrant> lifeless: That's quite a bug
<wgrant> lifeless: Also, quite the most useless timeout report ever :)
<wgrant> (ie. it's a criminal offense to report a timeout without an OOPS ID)
<lifeless> wgrant: hey, I know, but I'm not a driver there and he didn't give me one
<lifeless> he promised to file a bug when he has time...
<lifeless> openstack is releasing today IIRC
<wgrant> danilos: So, if you have time to consider what the right thing to do here is I'd really appreciate it. I'm still not completely confident in translations policy etc. If you don't have time, I'll work something out :)
<lifeless> wgrant: I thought you might want to capture an oops from there even if he is swamped and forgets
<wgrant> lifeless: k
<lifeless> gnight :)
<wgrant> Night lifeless
<wgrant> lifeless: Found an OOPS, no surprises.
<wgrant> Bug notifications are retarded, and OpenStack has hundreds of structsubs
<lifeless> wgrant: ttx did get time - https://bugs.launchpad.net/launchpad/+bug/1164402
<_mup_> Bug #1164402: Reproducible timeout updating a bug with a lot of tasks <Launchpad itself:New> < https://launchpad.net/bugs/1164402 >
<Adelein> Hi
<Adelein> I have a question about pybars
<Adelein> I am trying to find a good template engine that would allow me  to share templates between client and server django
<Adelein> so I tried handlebars but I see that it has no template inheritance support
<Adelein> does anyone know if there is support for template inheritance in pybars? or any way around it?
<Adelein> or what other template engines do you use to share templates between JS and Django/python?
<Adelein> any one out there?
<wgrant> Adelein: I think someone managed to do inheritance using handlebars partials
<wgrant> But I'm not sure. Google would know.
<Adelein> hey, I did see how they do it by creating a helper in the handlebars.js, but then how to integrate it in pybars
#launchpad-dev 2013-04-05
<StevenK> wgrant: https://code.launchpad.net/~stevenk/launchpad/silence-yui-build/+merge/157272
<wgrant> StevenK: Why not make it a proper make rule?
<StevenK> Oh?
<wgrant> StevenK: Can't you just skip extracting a version if it's already extracted?
<StevenK> -		$(RM) -r $(JS_BUILD_DIR)/yui-$$V; \
<StevenK> +		[ -d $(JS_BUILD_DIR)/yui-$$V/yui ] && continue; \
<StevenK> wgrant: ^ ?
<wgrant> StevenK: Probably, unless you want to rewrite it to use make properly.
<StevenK> That may print out No such file or directory for clean trees
<StevenK> wgrant: I'm up for that, since that part of the Makefile shits me, I'm just not sure how to.
<wgrant> Why?
<wgrant> -d shouldn't print anything
<StevenK> wgrant: So having a target for build/js/yui-VERSION: sounds like a good plan, but I can't get make to do that for me
<wgrant> StevenK: $(JS_BUILD_DIR)/yui-%:?
<StevenK> wgrant: Well, I'd like a target for each version
<wgrant> StevenK: That is a target for each version
<StevenK> But it won't expand for 3.3.0 and 3.5.1
<wgrant> Won't it?
<wgrant> You'll need another rule to depend on the targets for the two versions
<wgrant> That's what YUI_BUILDS is
<StevenK> Right
 * StevenK stabs make
<StevenK> The directory exists, so skip the rule!
<StevenK> wgrant: The MP is updated
<StevenK> See what you think of that
<StevenK> The changes are happy with a clean build and a build that already has a build/js
<wgrant> StevenK: Why the YUI_TWO change?
<wgrant> I guess it does clean up the console output a lot, and we never change it
<wgrant> But you can use digits in identifiers :)
<StevenK> wgrant: Because I was sick of it making and copying every directory under yui_2.../build
<StevenK> I can change the identifier to YUI2 if you wish
<wgrant> Or YUI2_BUILD or something
<wgrant> StevenK: Why'd you remove it from the comboloader?
<wgrant> Do we not comboload YUI2?
<StevenK> That is not the comboloader
<StevenK> Compare the makefile rule versus combo-rootdir
<wgrant> It's the thing that makes code available to the comboloader
<wgrant> Isn't it?
<StevenK> They do the exact same thing, except that combo-rootdir's looks like it will make build/js/yui2/build rather than build/js/yui2/asset and friends
<StevenK> wgrant: http://pastebin.ubuntu.com/5678948/ for matching goodness
<StevenK> wgrant: It used to be, until rick did his horrible multiple YUI versions thing and then the makefile and combo-rootdir started doing it
<wgrant> StevenK: What copies them into the comboloader root, then?
<wgrant> Or does it just get symlinked...
<wgrant> I can't remember
<StevenK> wgrant: The make rule YUI_BUILDS will unpack them into the comboloader root
<StevenK> combo-rootdir will make a symlink to the default version, and populate build/js/lp
<StevenK> wgrant: A further clean up would be to destroy combo-rootdir and do it all in make
<wgrant> And then the comboloader grabs it from the symlink in /srv, which points directly at build/?
<StevenK> build/js, yeah
<StevenK> /srv/launchpad.dev/convoy -> /home/steven/launchpad/lp-branches/silence-yui-build/build/js
<wgrant> Right
<wgrant> Hm
<wgrant> So combo-rootdir is now basically nothing
<StevenK> No
<StevenK> Like I say, it makes the symlink and populates build/js/lp
<StevenK> Those can be moved into make with a bit of fiddling
<wgrant> Basically nothing :)
<wgrant> Yeah, probably worth doing that in a followup
<wgrant> Seems silly to have it split
<StevenK> I'm happy to leave this branch until Monday and finish the split
<wgrant> Sounds reasonable. Won't be much work
<wgrant> Will be much cleaner
#launchpad-dev 2014-04-04
<xnox> how to get src package name for a binary package name? e.g. binary_package_publishing_history doesn't seem to have a link/field to src package name.
<jtv> I wish I remembered... it was complicated, I think.
<jtv> Not _very_ complicated, though.
<jtv> Are you working in server-side python code, or the API, or..?
<wgrant> xnox: It's not possible atm.
<jtv> Ah.  Hi wgrant.  :)
<wgrant> Morning jtv
<jtv> Was there no way back up the reference graph?
<wgrant> xnox: There's no reason it's not possible; there should be a binary_package_build attribute exported on binary_package_publishing_history
<wgrant> Nope, no way. Longstanding bug.
<jtv> Ouch.
<wgrant> Well, there's deliberately no way to get to a source_package_publishing_history directly, as there can be multiple.
<wgrant> But there should be a BPB
<wgrant> A patch wouldn't be hard.
<jtv> I thought there'd be a way through the BPPH.
<cjwatson> If you can query for a build instead (e.g. archive.getBuildRecords) then that's a workaround
#launchpad-dev 2014-04-05
<lifeless> wgrant: am I going to be disappoint if I ask about OOPS-6d81cb892ef5856f7cfcc3285c6db4ea
<wgrant> lifeless: A retry doesn't work?
<lifeless> that was the retry
<lifeless> a direct post rather than API worked
<wgrant> It's in some part because of the linked questions.
<wgrant> (those notifications are all in-request, unlike bug notifications)
<lifeless> I remember :)
<lifeless> wgrant: qastaging is down/gone ?
<cjwatson> I think it's being shuffled around as part of staging upgrades
<cjwatson> (don't know details, just background memory of -ops scrollback)
<lifeless> ack thanks
#launchpad-dev 2014-04-06
<wgrant> lifeless: Funny story
<wgrant> lifeless: We moved it off asuka on Friday
<wgrant> And then the new machine had a hardware failure on Saturday morning
<wgrant> asuka really doesn't want to die.
<lifeless> wgrant: !
#launchpad-dev 2015-03-30
<sil2100> Hey!
<sil2100> I'm using the LP API to fetch some info about packages and I noticed something really strange in what I get back from LP
<sil2100> For instance, an exemplary code-snippet that I used in lp-shell:
<sil2100> http://paste.ubuntu.com/10707382/
<sil2100> I tried getting all versions of the bash package and tried sorting it by when it was created in the archive
<sil2100> Anyway, what worries me the most is that the latest created version seems to be 0.2.0-1build1, which is really old and not even visible in LP, and it's Published
<sil2100> While the actual version, 4.3-11ubuntu2, is somewhere later
<sil2100> Is that expected? Since I querried for packages from the vivid series I would expect seeing only those that I actually see in https://launchpad.net/ubuntu/vivid/+source/bash, for instance
<sil2100> And I'm pretty sure 0.2.0-1build1 wasn't created at the beginning of this month
<sil2100> AH!
<sil2100> I see the problem
<sil2100> exact_match is not True by default
<sil2100> hah
<sil2100> Scratch the above question, didn't know about this
<cjwatson> sil2100: Yup, that's really a historical mistake in the API design.
#launchpad-dev 2016-04-04
<lblythen> any help pls re. Launchpad Documentation Team membership?
<lblythen> 13 pending members, no-one approved for a year and a half :-(
<wgrant> lblythen: Done. Lots of people just randomly apply to every team they can find, so I don't normally approve people I don't recognise without being poked.
<lblythen> wgrant: that's great, thanks a lot :-) sorry for the delay responding.
<jelmer> The DNS record for planet launchpad appears to have been removed - was that intentional?
<wgrant> Yes.
#launchpad-dev 2016-04-05
<lifeless> a
#launchpad-dev 2016-04-07
<xnox> cjwatson, is there a direct url to https://api.launchpad.net/devel/ubuntu/precise/amd64/chroot_url
<xnox> e.g. launchpad.net/+chroot_url or some such to directly download the chroot tarball?
<cjwatson> xnox: no
<xnox> ack.
<cjwatson> xnox: note that manage-chroot exists though
#launchpad-dev 2017-04-03
<tsimonq2> cjwatson: So I'd like to bring up a discussion we had a few months ago about apt vs. apt-get in Launchpad docs
<tsimonq2> cjwatson: (for example when adding a PPA)
<tsimonq2> cjwatson: Now that we have Ubuntu 12.04 ESM provided to Ubuntu Landscape customers, should those pages still have apt-get because of them, or can we move on?
#launchpad-dev 2017-04-04
<cjwatson> tsimonq2alt: precise hasn't EOLed yet in any case ...
<cjwatson> tsimonq2alt: I'm still inclined to keep it the way it is for a while; there's no great rush to eliminate four characters :)
<tsimonq2> cjwatson: Ah ok.
<tsimonq2> cjwatson: Thanks :)
#launchpad-dev 2017-04-06
<cjwatson> wgrant: Can you walk me through what's needed for librariangc in https://code.launchpad.net/~cjwatson/launchpad/db-buildinfo/+merge/321262 ?  I didn't think that it usually had specific handling of domain tables that reference LFA.
<wgrant> cjwatson: Oh nevermind, SourcePackageRelease.changelog
<wgrant> er copyright
<wgrant> The first LFA FK on a new table requires security.cfg changes
<wgrant> But of course SourcePackageRelease has one nowadays.
<cjwatson> Ah, for UnreferencedLibraryFileAliasPruner
<wgrant> Yeah
#launchpad-dev 2018-04-04
<lifeless> cjwatson: oh good
<cjwatson> wgrant: I brought https://code.launchpad.net/~cjwatson/launchpad/snap-allow-network/+merge/336924 up to date with the changes you suggested to db-snap-allow-network
<cjwatson> wgrant: Also if we want to get https://code.launchpad.net/~cjwatson/launchpad/inrelease-by-hash/+merge/336675 in place for bionic (which I think ideally we would) that'll need to be soon
