#launchpad-dev 2009-07-03
<thumper> why am I not surprised to see wgrant in here :)
<bigjools> lol
<bigjools> hello thumper :)
<wgrant> thumper: You... you spoke in here. Nobody has spoken in months.
<thumper> bigjools: I think I like this place
<thumper> let's make it home :)
<thumper> hi jtv
<jtv> hi thumper, good night!
<thumper> I have to go make chocolate cake now :)
<noodles775> mmmm
 * bigjools cuddles under a rug with thumper
<wgrant> thumper: Terrible.
 * jtv looks for a drink that'll make him forget he saw that
<thumper> I'm wondering how to tell quassel to join this automatically
<thumper> perhaps it just knows...
<bigjools> thumper: it will do it now
<thumper> I must test at some stage
<bigjools> it remembers
<thumper> bigjools: ah, it never forgets :)
<bigjools> one might say it tracks your every move
<wgrant> What is with this surprise incursion?
<thumper> where are our losas?
<thumper> I want a dame launchpad pastebin
<thumper> !!!!
<thumper> damn
<thumper> it's too late to spel rite
<thumper> so ...
 * bigjools phones the queen
<thumper> we moving launchpad-reviews here too?
<bigjools> we could!
<thumper> bigjools: email the list
<thumper> lets make this happen
<wgrant> Oh, it's actually happening?
<thumper> bigjools: I want to organise a weekly call with you
<bigjools> thumper: I'll wait to see what Karl says first
<thumper> wgrant: we have the power !!!
<bigjools> wgrant: it could!
<thumper> bigjools: karl will like it
<bigjools> I am kicking it off, at least
<thumper> bigjools: can you do evenings?
<wgrant> This channel has been sitting idle for almost 7 months.
<thumper> I could make an effort for a 6am call or too
<thumper> two
<thumper> damn it
<thumper> wgrant: bigjools invited me in
<bigjools> thumper: how much do you pay?
<thumper> and now I'm gunna make some noise
<thumper> bigjools: what? you pay me remember
<thumper> d'uh
 * bigjools pulls empty pockets inside out
<thumper> we need to talk about soyus integration
<thumper> d'oh
<thumper> soyuz
<thumper> you can see I don't type that much
<bigjools> thumper: yes - we've just landed a load of stuff that will make the "click to upload" button a reality
<bigjools> but
<bigjools> it still needs more work
<thumper> bigjools: I want to be able to link a branch to a ppa
<thumper> bigjools: so when I push, you build
<bigjools> the buildds need an overhaul to do generic jobs
<thumper> bigjools: sound good?
<bigjools> oh yes
<thumper> bigjools: sure, we've got work to do too
<bigjools> but we need work from IS
<thumper> bigjools: lets get it done by christmas, deal?
 * bigjools shakes on it
<thumper> I'd love to present this at linux.conf.au
<thumper> in a lightning talk
<thumper> (maybe)(
<thumper> I need to organise a real talk too
<thumper> anyway...
<thumper> kitchen calls
<bigjools> send me a piece
<thumper> bigjools: can I book you for a 30 min call weekly?
<thumper> bigjools: we'll put it in the calendar
<bigjools> sure thing
<bigjools> I'll have my secretary contact your secretary
<thumper> bigjools: how does 18:00 Tues UTC sound?
<bigjools> bad :/
<thumper> wed morning for me
<thumper> hmm
<thumper> ok, make me an offer
<bigjools> let me think about it
<thumper> ok
 * thumper goes to bake
<bigjools> mornings might be better
<thumper> yeah, sure, make it my evenings
<bigjools> my evenings are hectic - kids etc
<thumper> later peoples (and mine too)
<bigjools> ciao
<jtv> henninge: if you run into that "subordinate to itself" assertion failure while QA'ing, the fix is attached to bug 385924.  I just put it up for review.
<ubot3> Malone bug 385924 in rosetta "Message-sharing migration fails: a potmsgset was found subordinate to itself" [High,In progress] https://launchpad.net/bugs/385924
<henninge> jtv: ok
<flacoste> a very elite group of people here!
#launchpad-dev 2009-07-05
<thumper> morning
#launchpad-dev 2010-07-05
<adeuring> good morning
<mrevell> Guten morgen developertrons
<lifeless> it is I, quasimodo
<lifeless> question about the lp test runner
<lifeless> if I have a test that fails in ec2
<lifeless> given the test name
<lifeless> whats the best way to run it locally
<StevenK> lifeless: bin/test -vvt <name>
<lifeless> StevenK: thanks
<lifeless> StevenK: and common causes for it passing locally failing in ec2land ?
<mwhudson> lifeless: ec2 land is hardy running python 2.5
<lifeless> hmmm
<mwhudson> at a guess anyway
<StevenK> lifeless: I'd need to see the traceback
<lifeless> easy done
<StevenK> lifeless: There isn't really a bunch of common causes for stuff like this
<lifeless> http://pastebin.org/383643
<lifeless> I've merged devel to my branch locally, made no diff
<lifeless> running: ./bin/test --subunit  -t test_runJobHandleErrors_oops_generated| testr load
<lifeless> id: 46 tests: 19 skips: 2
<mwhudson> oh
<mwhudson> well, at least partly what's going on here is a glorious lack of test isolation
<lifeless> aroo ?
<lifeless> I'd like to make oops' generated by tests not hit disk at all
<lifeless> but I don't want to make the patch bigger still
<lifeless> as my focus is profiling, not oopses
<mwhudson> otherwise it looks like a parenthesising bug?
<mwhudson> if logid.isdigit() and (lastid is None or int(logid)  > lastid):
<lifeless> hmm, worth trying for
<lifeless> I guess different fs characteristics in ec2
<lifeless> thanks!
<lifeless> does ec2land land tip or 'approved revision'
<mwhudson> tip i think
<mwhudson> (sadly)
<lifeless> \o/ I don't need to play silly-things
<lifeless> mwhudson: oh, I got ec2land running outside
<lifeless> mwhudson: nuked the paramiko check, apt-get installed boto; set pythonpath and manually made an ec2 file because its now glued into buildout and that blew up outside the vm.,
<mwhudson> lifeless: congrats :)
<lifeless> how much of that is mergable, do you think?
<jml> hello everyone
<wgrant> jml: Rabble rabble not open enough rabble rabble.
<wgrant> *ahem*. Morning.
<lifeless> wgrant: ?
<wgrant> lifeless: Referencing a recent Facebook update.
<lifeless> oh right
<lifeless> clearly true
<StevenK> lifeless: No fair complaining about a WIP MP
<lifeless> StevenK: sorry - saw it come in, replied.
<lifeless> bigjools: btw, not stalking, just subscribed :)
<bigjools> lifeless: heh - prepare for email deluge
<lifeless> bigjools: I have been for 6 years
<lifeless> bigjools: I doubt you'll shock me today
<bigjools> ok, continue delige
<bigjools> deluge, even
<wgrant> Speaking of being subscribed to merge proposals...
<wgrant> Why do I get a rejection email from some lists.c.c address for every piece of merge proposal email I send?
<lifeless> because we're muppets
<wgrant> For LP, that is.
<lifeless> yes
<bigjools> matsubara is fixing that I thought - it's an artefect of the various ownerships
<lifeless> there is a list representing reviewers
<bigjools> artefact, evem
<bigjools> oh fuck I can't type today
<lifeless> that list has a contact address - the internal lp reviews list from when lp reviews started to get too much for the dev discussion list.
<wgrant> Ah.
<lifeless> and that list, has a not-c.c policy on it
<lifeless> so it goes barf
<lifeless> just deleting the contact address on the list should fix it.
<wgrant> But won't that then spam all the members?
<wgrant> Actually, I guess it would fix it.
<wgrant> Since the people affected by the spam would be the people most empowered to fix it :P
<lifeless> they can manage their influx by setting a personal subscription
<lifeless> do no-mail, AIUI
<lifeless> see you guys tomorrow
<jtv> gary_poster: I think bug 534399 has fallen into a bit of a hole.
<_mup_> Bug #534399: Windmill breaks branch-rewrite on dev systems <Launchpad Foundations:Triaged> <https://launchpad.net/bugs/534399>
<gary_poster> jtv: indeed, thanks for bringing it to my attention.  I'll get that back on track.
<jtv> gary_poster: thanks for that!  Anything related to windmill or the build farm is probably a good candidate for smoothing out because the little complications there can be so devastating to our productivity.
<gary_poster> jtv, agreed.
<jkakar> FYI, just got this: bzr: ERROR: Invalid http response for https://xmlrpc.edge.launchpad.net/bazaar/: Unable to handle http code 502: Bad Gateway
<jkakar> A service might need to be kicked.
<bigjools> edge seems to be fairly borked
<wgrant> jml: Regarding that email, is there not a work item missing for adding a new security level?
<wgrant> Simply sending an email post-hoc is surely insufficient.
<jml> wgrant, if you think so, please say so on the thread, since others seem to disagree with you.
<lifeless> wgrant: why?
<lifeless> (lets talk briefly now, and either save a round trip, or generate a more targeted mail)
<wgrant> lifeless: Because I don't want most of my applications to be able to change my authentication tokens.
<wgrant> Even if it tells me about it later.
<wgrant> Allowing most apps to do that is *utterly stupid*.
<wgrant> Even if I immediately revert it, there is a window there.
<wgrant> A significant risk that can be avoided by simply adding a new privilege level.
<lifeless> wgrant: ok, so you mean 'when allocating an oauth key, privileged things *also* require an explicit ACK from the user when allocating permissions'
<lifeless> like
<lifeless> read[/write] public
<lifeless> read[/write] public+private
<lifeless> read[/write] public+private+security-details
<wgrant> read[/write] public
<wgrant> read[/write] public+private
<wgrant> allowed to steal my identity
<wgrant> Yes, it's pretty bad even if a read/write token gets compromised. But it's not permanent-account-hijackingly bad.
<wgrant> That level of risk should be minimised, surely.
<lifeless> so I think you have a valid concern
<lifeless> there are usability tradeoffs here.
<wgrant> There is no question of this, surely.
<wgrant> Yes, there are tradeoffs.
<lifeless> ftr I don't think we *must* or *must not* do this.
<wgrant> But I might point out what lots of Launchpad accounts have access to.
<lifeless> I'd like to explain why
<lifeless> lets say we add this new privilege level for oauth.
<lifeless> quickly will *ask for it always*
<lifeless> (because quickly wants to be able to do this if the user selects a menu item)
<wgrant> I was hoping it would obtain a privileged token at the start and then drop privileges later. But even if not, it's better than my scripts everywhere being privileged.
<lifeless> sure, no question of that.
<lifeless> what might be ideal is a trusted dialog saying 'you sure?'
<lifeless> the email is a way to ensure that no matter what, the user is told.
<lifeless> frankly script privilege is pretty useless in oauth today; noone knows what level to grant, apps can't request a specific level, and its not got any granularity.
<lifeless> Personally, given the choices of:
<lifeless>  - quickly screen scraping in the users web context
<lifeless>  - quickly using an API with an email audit notification
<lifeless>  - quickly... with extra privilege level
<lifeless> I really want to avoid the first at all costs.
<lifeless> I'm not sure that the third is really all that nice a way to tackle credential-and-contact management access control for scripts; *my* inclination would be to do something where silent takeover is impossible as a starting point.
<lifeless> wgrant: but your concerns are valid; raise them. I'm too tired to be creative right now :) - but others may not be, and there is always tomorrow. I'm glad that you only see this as 'add a small extra item', it means its mostly right.
<lifeless> wgrant: I think what I'm getting at is that privilege escalation shouldn't be unrestricted nor indefinite, but [as currently implemented] this would, and users would invariably grant the wrong access to quickly.
<lifeless> so I'd really like to see a creative solution
<lifeless> that gave just enough access for just long enough and didn't add room for user error from users that *have never used LP before*
<lifeless> sleep, take #2.
<jtv> Does anyone know how I fix these problems starting the Librarian layer?  http://paste.ubuntu.com/459509/  I got them after an unexpected shutdown.
<jtv> henninge: I just pushed the fix for test_helpers.
<henninge> jtv: cool, thanks
<jml> jtv, make sure all the librarian processes are dead and pid files are deleted.
<jtv> jml: again?
<jtv> After all the reboots, shutdowns, kills, make schema runs etc?
<jml> jtv, well, that's what I'd do to try to fix the problem.
<jtv> Well past that, I'm afraid.  :-(
<jml> ok.
<jml> jtv, what happens when you try to start the librarian manually?
<jtv> It depends.  With the twistd commandline mars gave me, it ran without complaints.
<mwhudson> jtv: what happens if you run ./bin/start_librarian ?
<jtv> I tried "make start_librarian" which exits with:
<jtv> Daemons cannot log to stdout, exiting!
<jtv> Now trying bin/start_librarian
<jtv> Gives me the same line.
<jtv> Either way, it takes pretty long to reach that stage for some reason.
<jtv> jml: remarkably, none of all this shows up in /var/tmp/librarian.log
<jtv> That just stops where my system shut down unexpectedly Friday.
<mwhudson> hm
<jml> jtv, maybe that's a config=TESTRUNNER vs config=DEVELOPMENT confusion
<jml> (except I think I got the case inverted)
<jtv> :)
<mwhudson> so basically start_librarian runs the same thing as the twistd line i gave you
<mwhudson> oh well
<mwhudson> it daemonizes i guess
<jtv> But the twistd line worked.
<jml> isn't the daemonizing thing important wrt tacrunner interactions?
<mwhudson> jtv: ./bin/twistd --python daemons/librarian.tac --pidfile /var/tmp/librarian.pid --prefix Librarian --logfile /var/tmp/librarian.log
<mwhudson> jml: but i think the librarian started by the test runner doesn't daemonize
<mwhudson> may be wrong there
<jtv> mwhudson: that returns quietly
<jml> mwhudson, I don't know.
<jml> I'm guessing that maybe there's something amiss with the way the tac runner is detecting that the librarian has indeed started.
<jtv> mwhudson: yay!  that starts up the daemon.
<jtv> This time, output appears in the log.
<mwhudson> jtv: then you can have fun exploring the difference between what that does and what start_librarian did because i can't see that there should be any
<jtv> mwhudson: and what's more, tests still break
<mwhudson> jtv: maybe it is a development/testrunner thing like jml said
<jtv> jml, any idea how I would test that?  On "make run" things seem to start normally but references to librarian files still give me an error in the browser.
<jml> jtv, take a working command and do LPCONFIG=testrunner working-command
<jml> jtv, alternatively, if it's make, then "make working-command LPCONFIG=testrunner"
<jml> for reasons that elude my poor little mind.
<jtv> Trying the same tests
<jtv> jml: I think the former should also work, but make has a feature of allowing the setting of variables in its command line like this.
<jtv> Hmm... the tests still break with LPCONFIG=testrunner
<jml> oh, that's why I said "working command"
<jml> if it's "breaking command", then do LPCONFIG=development :)
<jtv> The absence of a "working command" was thwarting me :)
<jtv> "make run" and tests both break.
<jml> I thought you had two things to compare
<jml> <jtv> mwhudson: yay!  that starts up the daemon.
<jml> oh now I see
<jml> never mind
<jml> anyway, if the tests break with both configs in the same way, then it's not a config issue :)
<jtv> Well frankly I don't...  I thought the daemon might keep running.
<jml> or not a config selection issue, rather.
<jml> jtv, you've probably been through this already, but you are doing all of this testing against one of the stable branches, right?
<jtv> They fail in _exactly_ the same way, far as I can see.  Shouldn't the tests fail from all other sorts of horrible problems with the wrong config?
<jtv> jml: no, though the same problems occur in both.
<jml> jtv, which subset of the tests are you running?
<jtv> jml: I've been running various subsets of Translations tests.
<jml> jtv, the least I can do is try running the same subset locally against db-stable to guarantee that it is indeed a problem with your local configuration
<jtv> From trunk, and now from db-stable.
<jtv> I'm fairly sure it is, given that it started happening across an unexpected system shutdown.
<jtv> I'm currently setting up a test branch based on db-stable.
<jtv> jml: this produces it on a devel-based branch: ./bin/test -vv -m lp.registry -t test_milestone
<jtv> now trying db-stable as well
<jtv> Oh blast, that wants another "make schema"
<jtv> jml: same result on db-stable.
<jtv> jml: what's really strange is that now my librarian seems to be running, but I'm still getting the same error.
<jtv> And still nothing from lsof -i TCP:111...  is it on the wrong port perhaps?
<jtv> Ah... the librarian I have running now listens on the ports set up in the development config, not the ones in the testrunner config.
<maxb> Hrm :-/
<maxb> sinzui's pocket-lint throws errors on apt-installation
<maxb> We didn't officially de-support hardy as a LP dev platform yet, did we?
<lifeless> maxb: we're still deployed on hardy, so I sure hope not.
<lifeless> maxb: OTOH All the canonical staff should be on Lucid anyhow; so they'd need hardy vms...
<maxb> sinzui's new linter only runs on python 2.6
<lifeless> heh
<lifeless> I hope thats not in the deployment code path.
<maxb> Only in lp-dev-deps on lucid for now, but it breaks the ability to sync lp-deps versions across all distroseries
<lifeless> yeah
<lifeless> is it an external package
<lifeless> (you imply that, I'm checking for clarity)
<maxb> Built in sinzui's personal PPA, copied into the launchpad PPA
<lifeless> is it a new custom package or a maverick backport? [I'm wondering where to file a bug on it]
<maxb> Fully custom so far
<rockstar> maxb, the lp-dev-deps don't get deployed to production. IS has their own packaging (AFAIK)
<rockstar> maxb, I was running a hardy chroot for a long time, until those packages brokedededed
<maxb> Indeed, but lp-deps and lp-database-deps and lp-soyuz-deps build from the same source ... and AFAIK the LP PPA is still supposed to be workable on hardy
* lifeless changed the topic of #launchpad-dev to: Launchpad Development Channel | Week 4 of 10.06 | PQM is RC; devel is open | https://dev.launchpad.net/ | Get the code: https://dev.launchpad.net/Getting | On-call review in irc://irc.freenode.net/#launchpad-reviews | Use http://paste.ubuntu.com/ for pastes
* mars changed the topic of #launchpad-dev to: Launchpad Development Channel | Week 4 of 10.06 | PQM is RC; devel is open | | Release Manager and R-C wrangler: mars | https://dev.launchpad.net/ | Get the code: https://dev.launchpad.net/Getting | On-call review in irc://irc.freenode.net/#launchpad-reviews | Use http://paste.ubuntu.com/ for pastes
<abentley> rockstar, thumper: There's a power outage in my area, which means I'm tethering, which means I can't use irc.canonical.com's unorthodox port number.
<thumper> abentley: ok
<rockstar> abentley, does that also mean no mumbling?
<thumper> abentley: can you mumble?
<abentley> rockstar, thumper: Haven't tried yet.  I do have fring on my phone if that doesn't work.
<thumper> abentley: I'm in mumble now if you want to try
#launchpad-dev 2010-07-06
<lifeless> mars: so it looks like PQM devel isn't open, the RE is looking for release-critical there as well db-devel
<lifeless> mars: if you're around, could you confirm that that is not intentional and I can talk with a losa to tweak the active config
<mars> lifeless, checking
<mars> lifeless, that is intentional - devel was to close today, especially given the staging issues.  What did you need to land?
* mars changed the topic of #launchpad-dev to: Launchpad Development Channel | Week 4 of 10.06 | PQM is RC; devel is closed | Release Manager and R-C wrangler: mars | https://dev.launchpad.net/ | Get the code: https://dev.launchpad.net/Getting | On-call review in irc://irc.freenode.net/#launchpad-reviews | Use http://paste.ubuntu.com/ for pastes
<lifeless> mars: I was hoping to land my uniquefileallocator refactoring
<lifeless> mars: its not RC
<lifeless> mars: when will it reopen ?
<mars> lifeless, I would guess Thursday early UTC, if there is no re-roll
<lifeless> ok cool
<thumper> mars: when are we releasing?
<mars> thumper, 2300 UTC
<mars> thumper, your time :)
<thumper> isn't that 45 minutes ago?
 * rockstar hates UTC math.
<mars> hehe
<lifeless> rockstar: it drives you around the clock?
<mars> thumper, 2300 UTC July 6th :P
<thumper> so in 23.25 hours
<mars> yep!
<thumper> ok
<mars> spm, so to summarize your mail: we stuck a wrench in the machine to test something, left it in there by accident.  The rest is history :)
<spm> mars: ha! coulnd't (and didn't) have phrased it better.
<spm> mars: so at this stage we have one cowboy that doesn't appear to have been landed. they're cronscripts, so perhaps not as critical, but can you confirm if we need to reapply that or not?
<mars> spm, bigjools told me that that cowboy has landed.  However I did not identify the revision.
<spm> mars: coolio; I guess one to watch for.
<poolie> lifeless: your performance plan could be described as "crash through or crash" :)
<foxxtrot> I'm getting a database authentication failure when I try to 'make run' my development launchpad instance. I used the instructions of the wiki. Is something broken in the current branch, or did I likely miss a step?
<lifeless> try make schema ?
<foxxtrot> Did that, and redid it, just to be sure.
<foxxtrot> I'll run it one more time
<maxb> It is likely some confusion between multiple postgres instances
<maxb> Please pastebin the output of pg_lsclusters
<foxxtrot> Ah, I have an 8.3 and an 8.4 cluster.
<foxxtrot> http://paste.ubuntu.com/459708/
<foxxtrot> I'm not using postgres for anything but lp on this box, but I do think I installed it a while back
<maxb> right... so I think the problem is that utilities/launchpad-database-setup will have configured the 8.4 cluster, but lp is trying to use the 8.3 cluster
<foxxtrot> Launchpad prefers 8.4, right?
<maxb> it's all a bit in transition right now
<maxb> If you are not using pg for anything else, I recommend:
<maxb> pg_dropcluster --stop 8.3 main
<maxb> followed by rerunning utilities/launchpad-database-setup
<foxxtrot> maxb, Got it running, the two postgres instances was the problem.
<adeuring> good morning
<henninge> jtv-afk: so, even after your testfix (and that's all it was), the TestSetCurrentTranslation_Ubuntu is still producing database violations when I merge db-stable.
<henninge> jtv-afk: where could the merge have gone wrong?
<henninge> Or how do I approach this?
<henninge> jtv-afk: it's all the same: IntegrityError: duplicate key value violates unique constraint "potemplate__distroseries__sourcepackagename__name__key"
<henninge> *they are* all the same, I mean
<jtv> henninge: I just got my RF working again; I'll try once again to reproduce the problem.
<henninge> jtv: I already tried it out with removed flush_order calls but that did not help.
<jtv> henninge: I wonder if there's some kind of version difference that might cause this...
<henninge> what do you mean by "version difference" ?
<jtv> IIRC we're running EC2 tests on an older Ubuntu release than we're running locally.
<jtv> Something in the software stack might be different.
<jtv> Or could it even be a schema change that's not being applied for some reason?
<jtv> henninge: not much luck reproducing the problem locally so far... trying ec2 as well
<jtv> henninge: the test fix was only for the test_helpers failure which is unrelated to the database constraints.  Do you still have a link to the failures somewhere?
<henninge> jtv: http://people.canonical.com/~henninge/merged-lp.translations.tests.log
<jtv> thanks
<henninge> jtv: I am also trying to push the branch but I have some DNS problems. Let me restart my router ...
<jtv> henninge: I found what's going on... and this time the code is relying on a new distroseries _not_ becoming the current series.  Fix is underway.
<henninge_> jtv: good news!
<jtv> I triggered this by giving a new template that should share with an old one the same name as the old one.  That revealed that the two templates are in the same sourcepackage.  The remaining question is: why doesn't this happen in local tests?
<henninge_> jtv: what do you mean by "local tests"?
<jtv> henninge_: when I run the tests on my local machine, they pass.
<henninge_> jtv: they don't for me
<jtv> henninge_: I thought yesterday you said they did
<henninge> jtv: on the current recife branch
<henninge> but not when I merge db-stable into it.
<jtv> ah
<henninge> I am pushing that right now
<jtv> well I see one possible explanation: the default status of a new distroseries may have changed, so that when you self.factory.makeDistroSeries(distribution=ubuntu) results in ubuntu.currentseries not being what it used to be.
<jtv> Missing word there.
<jtv> "it" results in ubuntu.currentseries not being what it used to be.
<henninge> jtv: this is the branch btw bzr+ssh://bazaar.launchpad.net/~henninge/launchpad/merge-db-stable-9518
<henninge> jtv: Distribution.getSeries was converted to storm
<henninge> but nothing in registry/model/distroseries.py that looks like it could have caused this.
<henninge> no related changed in the factory either, it seems.
<jtv> henninge: I couldn't find anything either...  :/
<danilos> adiroiban, henninge, jtv: please add things that you think we can consider fixing during the Epic to https://dev.launchpad.net/Translations/Reports/LaunchpadEpic2010, anything that is ugly code that you hate very much will do :)
<jtv> danilos: cool, will do
<henninge> jtv: bug 602227
<_mup_> Bug #602227: Windmill tests failing in recife branch <Launchpad Translations:New> <https://launchpad.net/bugs/602227>
<jtv> henninge: thanks...  my impression is it's a login failure.
<henninge> jtv: this is unrelated to the merge
<henninge> jtv: on some
<henninge> but I watched the first one and the dismiss button for the documentation balloon does not show up and when windmill "clicks" it does not disappear - followed by the failure.
<henninge> jtv: ^
<jtv> henninge: does not show up and then it does not disappear?
<henninge> jtv: the button does not show and the ballon does not disappear.
<jtv> oh, the dismiss button does not show up
<henninge> although I looked at the html and it seems to be there ...
<henninge> the button
<jtv> henninge: shall I try this on db-stable?
<henninge> jtv: good idea
<jtv> running...
<henninge> jtv: btw, trying it out with "make run" works just fine.
<henninge> button shows, ballon bursts ... ;-)
<danilos> henninge, jtv: it might be related to how it's stored in a cookie for the duration of the session (i.e. perhaps windmill session ends up being more persistent than a session should be)
<jtv> ooh nasty
<henninge> jtv: what's the result on db-stable?
<jtv> henninge: still waiting for the timeout on the documentation-links test
<jtv> but I'm obviously not logged in; it's asking me to.
<jtv> I canceled the login dialog, and now it's testing while logged in.
<jtv> henninge: passing tests so far
<henninge> jtv: I have to go to lunch now. ;)
<jtv> ok
<bigjools> wgrant: around?
<wgrant> bigjools: Hi.
<bigjools> wgrant: hi - just wanted your opinion on something.  I'm thinking about adding a second buildd-manager so that we have one for virtual and one for non-virtual.
<bigjools> I can't think of any cons, just pros
<wgrant> What would be the purpose of that?
<bigjools> increased throughput
<wgrant> Once it's fixed, is that really going to be a problem?
<ricotz> hello, can someone tell me what this error is? OOPS-1648ED2488
<bigjools> and isolation of problems
<wgrant> Perhaps.
<bigjools> depends on your definition of "fixed"
<bigjools> mine involves re-writing it ;)
<wgrant> I'm not terribly sure that adding a fourth concurrent uploader to the already racy set of three is a good idea, but I guess it's not too bad.
<bigjools> ricotz: what were you doing at the time? (I'm waiting for the oops data to sync to where I can see it)
<wgrant> bigjools: Don't you have a fair bit of that done already?
<bigjools> wgrant: concurrent build uploads should be fine
<ricotz> bigjools, i am trying to delete a bzr branch
<bigjools> wgrant: no, we're *really* re-writing it from scratch and building it into a new job system
<wgrant> bigjools: Concurrent source uploads can go horribly wrong. But I guess a non-virt buildd-master wouldn't be doing it.
<wgrant> Oh!
<wgrant> So it's never going to happen.
<bigjools> lol
<bigjools> yes, it will, we're committed to doing it
<wgrant> *ahem*
<bigjools> we already started designing it
<wgrant> Ah.
<bigjools> ricotz: ok, rockstar or abentley can help you there
<bigjools> wgrant: anyway, exactly yes, virt/non-virt is a good split
<ricotz> bigjools, ok
<elmo> bigjools: so, err, at one stage the long term plan was to collapse down the virt/non-virt distinction
<bigjools> ricotz: I can see the oops data now
<elmo> bigjools: once the virt feature set was on parity with non-virt (langpacks, debug builds, etc.)
<wgrant> bigjools: It's not a good split. It's a revolting hack which might improve things slightly.
<elmo> bigjools: maybe that's no longer the plan, I dunno, but if it is, separating them at the b-m level, if it's work, doesn't seem to make sense
<benji> That's the one important piece of home-office equipment I don't have yet, a coffee machine.
<bigjools> elmo: yes that can still happen, but this is a trivial change I can make that will speed up builds and then we can amalgamate the machines when we re-design the build manager
<bigjools> which as I said, is happening RSN
<wgrant> How soon is RSN?
<elmo> sure - if it's trivial, then *shrug* ignore me
<bigjools> it's in progress
<wgrant> I guess it should be pretty simple.
<bigjools> elmo: consider this analagous to lowering interest rates.  Brief pain relief until proper economic changes happen.
<bigjools> the redesign is not simple :)
<wgrant> Is the redesign using that thing that was pointed out at UDS?
<wgrant> I forget the name.
<wgrant> Celery?
<bigjools> celery
<bigjools> possibly, we're considering a few things
<ricotz> bigjools, seems they are not around, what is the error saying?
<bigjools> ricotz: it's a bug in the code
<bigjools> looks like it has a recipe build associated?
<wgrant> I noticed that code last week.
<ricotz> ok, i never used a recipe
<wgrant> It doesn't seem to cover the case where a build succeeds and produces an SPR.
<jtv> wgrant: lamont has launchpad-buildd 64 rolled out on the dogfood/staging buildfarms... is revision 63 df-tested?
<wgrant> jtv: I don't believe so.
<wgrant> But I don't know what has gone on behind the scenes.
<jtv> wgrant: then this is the time to test!
<jtv> yes, that is a bit of a sticking point...  but we now get to q/a 63 & 64.
<jtv> wgrant: my test seems to have gone down /dev/null somehow...  buildd-manager "Dispatched" it, ferraz ran it, it was "marked as done," but then there was no output.
<jtv> wgrant: any idea what the situation with your patch is?
<Ursinha> rockstar, hello :) I wonder if you can help me with an oops?
<Ursinha> hmm, maybe not
<Ursinha> :)
<Ursinha> abentley, ping
<abentley> Ursinha, hi.
<Ursinha> abentley, I see an oops in the branch scanner, do you know if the problem is known? https://lp-oops.canonical.com/oops.py/?oopsid=OOPS-1647SMS1
<abentley> I don't think the issue is understood.
<abentley> We have seen this occasionally.
<Ursinha> do you know what can be causing that?
<abentley> Ursinha, no, I don't think the issue is understood.
<Ursinha> abentley, oh, I see what you mean now
<Ursinha> abentley, I'll file a bug for that
<Ursinha> thanks
<mwhudson> abentley, Ursinha: i've always assumed that that meant the branch has been deleted while it was being scanned
<abentley> mwhudson, the branch in question exists, so I don't think that's the cause.
<mwhudson> abentley: maybe it was deleted and repushed?
<mwhudson> but maybe not, indeed
<abentley> jtv, have you ever deleted and re-pushed https://edge.launchpad.net/~jtv/launchpad/recife?
<jtv> abentley: I don't think so, no... why?  henninge's working on that branch
<Ursinha> jtv, because of OOPS-1647SMS1
<jtv> Oh, sorryâthe one in ~jtv.  That's garbage anywayâI thought --remember would work for bzr push and it didn't.
<jtv> I think I deleted it.
<Ursinha> mwhudson, abentley, does jtv answer "solve the mystery"?
<abentley> Ursinha, OTP
<mwhudson> hmm, not really
<Ursinha> ok! I'll lunch then.
<Ursinha> oops.
<Ursinha> mwhudson, go ahead
<mwhudson> Ursinha: no, go have lunch, i'm only chucking comments in from the peanut gallery
<Ursinha> mwhudson, I'll file a bug for that pasting this conversation
<Ursinha> mwhudson, bug 602323
<Ursinha> I guess the bot is dead...
<henninge> jtv: got a minute or are you busy painting the house oranje? ;-)
<jtv> henninge: I've done my 12 hours for today, sorry!
<henninge> jtv: well done! I'll get back to you tomorrow!
<henninge> ;-)
<jtv> Uh-oh!
* mars changed the topic of #launchpad-dev to: Launchpad Development Channel | Week 4 of 10.06 | LP will be read-only starting 23.00 UTC July 6th | PQM is RC; devel is closed | Release Manager and R-C wrangler: mars | https://dev.launchpad.net/ | Get the code: https://dev.launchpad.net/Getting | On-call review in irc://irc.freenode.net/#launchpad-reviews | Use http://paste.ubuntu.com/ for pastes
* Ursinha changed the topic of #launchpad-dev to: Launchpad Development Channel | Week 4 of 10.07 | LP will be read-only starting 23.00 UTC July 6th | PQM is RC; devel is closed | Release Manager and R-C wrangler: mars | https://dev.launchpad.net/ | Get the code: https://dev.launchpad.net/Getting | On-call review in irc://irc.freenode.net/#launchpad-reviews | Use http://paste.ubuntu.com/ for pastes
<Ursinha> :)
 * maxb boggles
<maxb> Someone has requested a Subversion vcs-import of http://bazaar.launchpad.net/~opensatnav-admins/opensatnav/release-1.0/files
<maxb> I have no words
 * nettrot chuckles
<lifeless> maxb: awesome
<lifeless> maxb: we should totally support that
<lifeless> maxb: (I'm serious - tarball import, keep the upstream branch if lp hosts that too, join all the dots together)
<thumper> :-o
<thumper> maxb: I think the user is confused
<lifeless> thumper: could be ;)
* Chex changed the topic of #launchpad-dev to: Launchpad Development Channel | Week 4 of 10.07 | LP will be read-only starting 23.00 UTC July 6th | PQM is RC; devel is closed | Release Manager and R-C wrangler: mars | https://dev.launchpad.net/ | Get the code: https://dev.launchpad.net/Getting | On-call review in irc://irc.freenode.net/#launchpad-reviews | Launchpad down/read-only from 23:00 - 00:30 UTC for a code update | Use http:/
* Chex changed the topic of #launchpad-dev to: Launchpad Development Channel | Week 4 of 10.07 | LP will be read-only starting 23.00 UTC July 6th | PQM is RC; devel is closed | Release Manager and R-C wrangler: mars | https://dev.launchpad.net/ | Get the code: https://dev.launchpad.net/Getting | On-call review in irc://irc.freenode.net/#launchpad-reviews | Launchpad down/read-only from 23:00 - 00:30 UTC for a code update
#launchpad-dev 2010-07-07
<mtaylor> thumper: ping
<mtaylor> thumper: I have a feature request: ... just like a project can have a devel target branch, I think a project should also be able to have a devel target series and a devel target milestone
<mtaylor> thumper: so that "file a bug against the current devel target series/milestone" and "target a blueprint to current devel target series/milestone" is really easy
<mtaylor> I'm bringing this up because we're currently merging a lot of branches with no bug/blueprint associated with them in drizzle
<mtaylor> but the number of steps I'd be asking someone to go through to add a blueprint aroud merge-request time is sort of redundant
<mtaylor> what would be great is if an unadorned branch that  files a merge request could auto-create a blueprint with the merge-request summary as the blueprint description
<mtaylor> and that is targetted to the current stuff - so that we've got a paper-trail of what was done without tons of extra dev work
<mtaylor> thumper: I kept rambling ^^^
<mtaylor> also - if there _is_ an associated blueprint, the blueprint desc could be used to pre-fill the merge request desc
<mtaylor> sort of helping describe a relationship between blueprints and merge-requests
<lifeless> mtaylor: it has all that
<lifeless> mtaylor: devel target branch is modeled as 'branch of the devel series'
<lifeless> mtaylor: but bug targeting (known as infestation) is underdeveloped - we had bad UI experiences in that space.
<mtaylor> lifeless: well great then
<mtaylor> lifeless: the "create merge-request from blueprint" or "create blueprint from merge-request" isn't there though, that would be a full-on feature request, yes?
<lifeless> it would be a patch to do it.
<lifeless> blueprints are unmaintained at the moment.
<lifeless> some folk put in slack and weekend time on them
<lifeless> but if there is something specific you want... :)
<lifeless> mtaylor: oh, and I'd love to see people caring for this
 * wgrant would love to see Blueprint put out of its misery.
<lifeless> wgrant: one way or another ?
<lifeless> wgrant: hey, while you're here, let me run something past you
<wgrant> lifeless: Sure.
<lifeless> wgrant: seems to me that the package management pages aren't really at home in the http://launchpad.net/* space, and might benefit from being in a dedicated domain like packages.launchpad.net/
<lifeless> in the same way that code.* has a more focused UI
<lifeless> (and bugs etc etc)
<wgrant> lifeless: It's difficult, because packages are structural.
<wgrant> Packages *have* bugs.
<lifeless> not saying that the multi-domain approach is good or bad, just that the soyuz stuff is neither fish nor fowl at the moment, and there is some tension with the registry [metadata only, thanks] goal
<lifeless> wgrant: so do branches ;)
<wgrant> Branches are not bug targets.
<wgrant> But yes, the Registry<-->Soyuz split is rather arbitrary, not done to well, problematic, and otherwise pretty terrible, both code and UI-wise.
<lifeless> thats true
<lifeless> wgrant: I would say that Package *names* have bugs
<lifeless> wgrant: Binary and Source package instances do not.
<lifeless> [infestation aside]
<lifeless> wgrant: thanks for letting me run that past you ;)
<wgrant> lifeless: That's true.
<wgrant> But the lack of infestations is a bug.
<lifeless> so there are two descrete concepts
<lifeless> 'can have a bug'
<lifeless> and 'can be a place that a bug fix should be done'
<lifeless> package versions by there very nature are only able to satisfy the former.
<lifeless> s/there/their/ blah.
<lifeless> containers-of-packages can do both ('there is a package in this place with the bug', and 'we want to upload a new package here with a bugfix')
<lifeless> under this model, I think there is a clear separation between registration stuff and package-instance stuff
<lifeless> what do you think ?
<wgrant> Indeed.
<wgrant> What would go on the DistributionSourcePackage Registry page if the Soyuz one was split from it?
<lifeless> so, like with other subdomains, a split doesn't mean 'do not reference', its about focus and primary presentation
<wgrant> Right.
<wgrant> But what would be the focus of the Registry page?
<lifeless> I'd expect, on a DSP registry page to give primacy to controlling things like:
<lifeless>  - acls
<lifeless>  - policy [disable uploads of this package]
<lifeless>  - history [of things we no longer keep as well as active package files]
<lifeless>  - relationships to other packages
<lifeless> Just riffing here at this point
<thumper> mtaylor: hi, sorry, was out afk
* Chex changed the topic of #launchpad-dev to: Launchpad Development Channel | Week 4 of 10.07 | LP will be read-only starting 23.00 UTC July 6th | PQM is RC; devel is closed | Release Manager and R-C wrangler: mars | https://dev.launchpad.net/ | Get the code: https://dev.launchpad.net/Getting | On-call review in irc://irc.freenode.net/#launchpad-reviews | Use http://paste.ubuntu.com/ for pastes
* Chex changed the topic of #launchpad-dev to: Launchpad Development Channel | Week 4 of 10.07 | PQM is RC; devel is closed | Release Manager and R-C wrangler: mars | https://dev.launchpad.net/ | Get the code: https://dev.launchpad.net/Getting | On-call review in irc://irc.freenode.net/#launchpad-reviews | Use http://paste.ubuntu.com/ for pastes
<mtaylor> lifeless: yes. I have put it on my todo list and plan on writing code in support of it
<mtaylor> lifeless: which means step #1 - register a blueprint - will get done at some point :)
<lifeless> mtaylor: given that LP doesn't really use blueprints for its own dev
<lifeless> mtaylor: I don't think that that is step 1. Step 1 is 'propose a UI for it on the lp-dev list'
<lifeless> discuss.
<mtaylor> lifeless: ok.
<mtaylor> lifeless: I imagine I may become the caretaker of all things blueprints...
<mtaylor> lifeless: if there is not currently one, since we really like them ;)
<lifeless> tag, you're it
<lifeless> actually sinzui IIRC has spent considerable time on it
<mtaylor> lifeless: awesome
<lifeless> but all personal.
 * mtaylor doesn't really want ownership - just expects to write some code
<lifeless> so, start with that.
<lifeless> Really.
<lifeless> JFDI :)
<mtaylor> yup
<mtaylor> that's the idea
<lifeless> step 0) get a working lp dev environment - have you got that ?
<mtaylor> yup. step 0 done
<lifeless> ok
<lifeless> step 1) make a branch to work on this; have you got that ?
<mtaylor> step 0.5 - complain about the number of steps needed for step 0
<mtaylor> ;)
<mtaylor> yup. step 1 done
<lifeless> patches appreciate.
<lifeless> step 2) if you are not sure where to start changing stuff, ask here.
<mtaylor> lifeless: pushed onto stack
<lifeless> remember that we like patches to be < 1000 lines, as a pretty hard limit, for the sanity of the reviewer.
<lifeless> unless previously agreed.
<mtaylor> yup. we're like that in drizzle
<mtaylor> sequnces of self-contained small changes that don't break shit ftw
<mtaylor> or sequences even
<lifeless> mars: when does PQM open ?
<thumper> it is supposed to be < 800
<lifeless> oh, my bad.
<thumper> PQM will open again when we have confirmed no massive fubar
<thumper> lifeless: bigger can be done with prior agreement with a reviewer
<lifeless> yeah - I mentioned that just above :P - but I had the # wrong.
<thumper> lifeless: I just noticed that after I had written the line
<thumper> :)
<lifeless> lol
<thumper> mtaylor: get yourself on the launchpad-dev mailing list :)
<wgrant> The last comment on bug #474615
<_mup_> Bug #474615: lp email 'rationale' header should use most specific criterion, or add header for direct subscription <story-better-bug-notification> <Launchpad Bugs:Triaged> <https://launchpad.net/bugs/474615>
<wgrant> Why do people not file bugs about their broken mail providers?
<thumper> wgrant: like gmail?
<wgrant> How is it Launchpad's problem that some mail providers are crap?
<wgrant> Right.
<lifeless> well
<thumper> I've filed a bug with gmail about not being able to filter on mail headers
<lifeless> see, people *use* gmail. That we want to use lp.
<lifeless> leverage.
<lifeless> wgrant: so that bug, I don't think its really a gmail flaw per se. Yes headers are better. But its still simply nice to see the most proximate subscription - its the most relevant
<lifeless> wgrant: did you mail the list about API privileges
<lifeless> wgrant: if not, I can; I've been thinking on it.
<wgrant> lifeless: I haven't really got time at the moment, I'm afraid.
<lifeless> ok
<lifeless> I'll carry your point forward and you can tell me how I do later
<wgrant> Thanks.
<lifeless> also, http://www.hemispheregames.com/osmos/ looks awesome, if totally offtopic here.
<wgrant> It is awesome, yes.
<wgrant> I discovered it a few months ago, just before the Linux port, but it still ran fine in Wine.
<spm> hrm. no elves, no swords, no sharks with lasers?!?!? can't see the point.
<lifeless> spm: addict.
<spm> heh, perhaps.... :-)
<poolie> lifeless: i was thinking of changing 'make run' to turn on tc slowness
<poolie> perhaps with an escape hatch
<thumper> heh
<lifeless> mmm
<lifeless> I think a target to start and stop tc would be awesome
<lifeless> I don't know that there is a perception mismatch for most devs though.
<lifeless> its tempting to think there is.
<lifeless> what might be really fun is an incrementally slower 'make run'
<lifeless> like, 1ms slower per second of runtime.
<lifeless> :>
<poolie> make slower
<poolie> make faster
<poolie> > not implemented yet
<lifeless> lol
<poolie> how do you mean incrementally slower?
<lifeless> poolie: just that
<lifeless> slower the longer its beein running
<poolie> oh i see
<poolie> it will add 1ms to the service time for each request?
<lifeless> for instance yes
<cody-somerville> Why would the milestone page show bugs as one status but viewing the actual bug show the bugs as a different status?
<lifeless> its caching very aggressively now.
<lifeless> Please file a bug
<cody-somerville> Ok
<wgrant> I filed a bug last week, but more dupes is good.
<lifeless> well, please me-too it.
<adeuring> good morning
<mrevell> Morning!
<mwhudson> morning
<bigjools> morning
<jml> is danilo around today?
<deryck> Morning, all.
<mrevell> jml, Danilo's on leave today
<jml> mrevell, thanks.
<mwhudson> jml: hello, how are you today?
<jml> mwhudson, bowel-clenchingly busy. finishing slides for a talk I'm giving in < 3hrs
<mwhudson> jml: oh right
<jtv> wgrant: I'm trying to Q/A the launchpad-buildd changes...  do you know of a recipe that should build without problems?  When I try I get failures that I hear are probably unrelated: http://staging.launchpadlibrarian.net/51388616/buildlog.txt.gz
<jtv> abentley, do you know of any recipes that definitely should build without problems on either staging or dogfood?  I'm trying to Q/A the latest launchpad-buildd so we can roll it out.
<abentley> jtv, https://code.dogfood.launchpad.net/~abentley/+recipe/bazaar but you may need to change the debversion
<jtv> henninge: any idea whether you'll be wanting those traits?
<henninge> jtv: I think I do ... ;)
<henninge> I mean, I'd like to have them.
<wgrant> jtv: Any recipe build is going to fail like that, since staging doesn't seem to have Soyuz.
<wgrant> Dogfood should work.
<Ursinha> sinzui, hello, good morning
<Ursinha> sinzui, something very weird is happening with the milestones of this project: https://edge.launchpad.net/iportfolio
<Ursinha> I can't see them
<Ursinha> maybe a typo lead to that?
<Ursinha> s/1010/2010/
<sinzui> I think someone put that date there
<sinzui> users allowed to screw up
<bigjools> wgrant: staging has builders, a buildd-manager and config.
<wgrant> bigjools: But no archive.
<sinzui> Ursinha, looks like you can create a date that the UI cannot support. The API does support it though
 * sinzui reports bug
<Ursinha> sinzui, hm, I see
<bigjools> wgrant: you don't need an archive to test builds
<wgrant> bigjools: You do if the primary archive's hostname is set to ftpmaster-staging.internal, and that doesn't exist.
<bigjools> wgrant: that is being fixed (dunno who set that), then it will work
<wgrant> Ah.
* mars changed the topic of #launchpad-dev to: Launchpad Development Channel | Week 4 of 10.07 | PQM is open | Release Manager and R-C wrangler: mars | https://dev.launchpad.net/ | Get the code: https://dev.launchpad.net/Getting | On-call review in irc://irc.freenode.net/#launchpad-reviews | Use http://paste.ubuntu.com/ for pastes
* mars changed the topic of #launchpad-dev to: Launchpad Development Channel | Week 4 of 10.07 | PQM is open | https://dev.launchpad.net/ | Get the code: https://dev.launchpad.net/Getting | On-call review in irc://irc.freenode.net/#launchpad-reviews | Use http://paste.ubuntu.com/ for pastes
<nettrot> Is there anyone from launchpad-foundations who can chat about Bug #602346?
<_mup_> Bug #602346: Launchpad can save HTTP requests with DataURIs <Launchpad Foundations:Incomplete by foxxtrot> <https://launchpad.net/bugs/602346>
<lifeless> any soyuz folk around ?
<bigjools> lifeless: just about to grab a cold one and finish watching the footy, wassup?
<lifeless> bigjools: hi
<bigjools> too late I'm drunk
<lifeless> bigjools: so the new private ppa private-owning-team change
<lifeless> soren can see the owning team for the font ppa
<lifeless> as in, see its main registry page.
<bigjools> yeah we know
<lifeless> Was this intentional ?
<bigjools> sorta
<lifeless> Do you have a bug saying you want to tune it more?
<lifeless> Just so I can point surprised people at it :)
<bigjools> I need to start a thread about this on the dev list, we just talked about it on the TL call
<lifeless> sure
<bigjools> no bug yet
<lifeless> I'm only asking cause of soren's question on #launchpad
<lifeless> :)
<bigjools> we're talking about having a restrictedView permission
<lifeless> in directory service terms
<lifeless> what you want is 'traversal'
<lifeless> as opposed to 'view'
<bigjools> no, it's view
<lifeless> really ?
<bigjools> partial view of a private object
<lifeless> what bits ?
<bigjools> the name of the private team is needed to form it's PPA URL for example
<bigjools> its*
<bigjools> gah hate that mistake
<lifeless> (actually, if you're drunk, and this is known, you can stay drunk and watch the sport :))
<lifeless> bigjools: I'd say that traversal implies accessing the metadata needed to present anything got at by traversing.
<bigjools> from a zope PoV, yeah
<bigjools> lifeless: I'm open to suggestions, thumper had some concerns about dealing with a new permission
<bigjools> but I'm going back to the  footy now :)
<bigjools> ciao
<lifeless> bigjools: EPIC stuff.
<bigjools> lifeless: yes
<thumper> morning
<lifeless> morning thumper
<lifeless> wgrant: lp:~leonardr/launchpad/grant-permissions-oauth :P
<lifeless> wgrant: someone used a time machine, I think.
<leonardr> wgrant, lifeless, what's up?
<lifeless> wgrant: its not quite what is needed, but its close
<lifeless> leonardr: hey! so you've done something that is very very close to what didrocks' gpg/ssh management stuff needs.
<lifeless> leonardr: which is a API way to manage security sensitive stuff
<leonardr> lifeless: ok, what do you need beyond what's in that branch (and its implied follow-ups)?
<lifeless> I don't know yet.
<lifeless> but its in the space
<lifeless> did you see the thread on lp-dev, and the LEP ?
<leonardr> no, i'm looking now
<lifeless> there is a missing bit which is 'perhaps we shouldn't let *every* API client change gpg/ssh keys, even though email notifications is a great safety net'
<lifeless> wgrant raised this and I think his points are valid, so we should try to slot something nice into the use case.
<lifeless> perhaps your tool should manage ssh and gpg keys on lp too
<lifeless> rather than quickly doing it
<leonardr> lifeless: given the way GRANT_PERMISSIONS can be used for privilege escalation, it might make more sense to have a separate access level
<leonardr> and a separate app
<leonardr> however, the credential manager could administer the granting of this new access level to this separate app
<lifeless> leonardr: if you'd like to throw some thoughts into the ring, that would be lovely
<lifeless> its a little time sensitive (isn't it always)
<leonardr> sure
<leonardr> from what i've heard here, it sounds like splitting out WRITE_SECURITY_SENSITIVE from WRITE_PRIVATE would work
<lifeless> if you have a preference as well, please indicate that too
<lifeless> I think didrocks is really able to tell us best what will work out well in the structure and what may be an issue
<leonardr> lifeless, can you give me a subject line and/or link to a web archive? my mail skills are notoriously bad
<leonardr> and i can't find this thread
<lifeless> [Launchpad-dev] Fwd: [Fwd: Quickly and Launchpad]
<leonardr> thanks, i had no idea what quickly was
<lifeless> its a mini IDE kind of thing
<lifeless> puts program boilerplate together
<lifeless> builds debs of python programs
<leonardr> cool. i'll write a response
<lifeless> sets up a development environment
<lifeless> thanks!
<leonardr> lifeless: if i understand the problem correctly, it's the danger of a malicious app making up a fake ssh key and uploading it?
<lifeless> or a new gpg key
<lifeless> I think you can add a key for an address you haven't validated previously, and it mails that address automatically
<lifeless> which the user wouldn't see
<lifeless> new gpg key would allow exposure of private branch content (but then WRITE_PRIVATE has that anyway)
<lifeless> sorry
<lifeless> new ssh key would ...
<lifeless> new gpg key would allow uploading packages
<lifeless> that the user hadn't signed, and its currently got more of a security shell around it
<lifeless> probably the whole set of interactions needs consideration
<lifeless> I'm not 100% sure wgrant is right in saying we need to do something about this - but I think we have to at least *think* critically about it
<leonardr> all right
<james_w> congratulations lifeless
<thumper> lifeless: congrats
 * bigjools hi-fives lifeless
<nettrot> I'm noticing that some of the JS Tests in Launchpad are in YUI Test, others are in Windmill. Is there a preference for new tests? And/or what is the reason for the two frameworks?
<thumper> nettrot: Windmill came first
#launchpad-dev 2010-07-08
<wgrant> Wow, congrats lifeless.
<poolie> yes, indeed
<poolie> hi wgrant
<wgrant> Morning poolie.
<poolie> i think robert's new role will have benefits for you in particular
<wgrant> Oh?
<poolie> being another step towards openness and ease of contribution
<poolie> "think or hope" i suppose
<poolie> leonardr: did you read jml's lep about api access to authentication tokens?
<lifeless> thumper: thanks
<lifeless> wgrant: thanks; poolie: thanks
<poolie> i miss you lifelesS!
<lifeless> already? I'm not gone till Monday :)
<poolie> well, i did briefly
<lifeless> and even then, I think its appropriate to think of it as expanding - like my waistline - the bzr team is definitely able to call on me to discuss stuff :)
<poolie> :)
<lifeless> oh, or did you mean as an office-mate
<poolie> yes, that's what i meant
<lifeless> righto ! - 5 hours sleep last night, but boy, better sleep than I had been getting
<lifeless> have seen allergy doctor this morning
<lifeless> so; finally; no medical stuff to do for a month; no frenzy; a feeling of peace surrounds me :)
<wgrant> lifeless: I'm sure we can invoke an architecture frenzy for you :P
<lifeless> hmm, are we in testfix mode I wonder
<lifeless>   restricted_families = archive_arch_set.getRestrictedfamilies(self)
<lifeless> ForbiddenAttribute: ('getRestrictedfamilies', <lp.soyuz.model.archivearch.ArchiveArchSet object at 0x143bc190>)
<lifeless> seems unrelated to oops infrastructure changes
<lifeless> ah yes.
<thumper> lifeless: a feeling of peace except for the frantic travelling coming?
<lifeless> thumper: compared to the last three months, its quiet
<ajmitch> lifeless: great, so I can ask you all my LP questions now? :)
<lifeless> yes
<lifeless> Not that that would seem to be any different
<ajmitch> heh
<wgrant> Can I coerce someone into ec2landing https://code.edge.launchpad.net/~wgrant/launchpad/faster-and-more-general-getBuildQueueSizes/+merge/28476 ?
<lifeless> wgrant: does that fix henninge's patch ?
<lifeless> rev 11093 seems to break everything, or perhaps its just my branch it breaks.
<lifeless> if not, then no, I can't be coerced.
<wgrant> You mean jelmer's/
<lifeless> ah yeah
<lifeless> the one reviewed by
<wgrant> That is a bit confusing.
<wgrant> I wonder why PQM doesn't use --author.
<wgrant> lifeless: Which is the test that breaks?
<lifeless> wgrant: pqm doesn't use --author because its not the author.
<lifeless> log -n0 shows the people totally accurately.
<lifeless> alias it on.
<wgrant> lifeless: Doesn't help for commit notifications.
<lifeless> wgrant: we should fix that, then.
<wgrant> Is there a testfix in progress?
<lifeless> bzr has the real data; this is a presentation issue IMO: fudging it by feeding PQM an approximation you want to see on the outside just leads to harder to work with data.
<lifeless> wgrant: I'm not sure how to look that up yet.
<lifeless> wgrant: and today is a bzr day :)
<wgrant> lifeless: Well, if you tell me which test fails, I will fix it. It's just an s/f/F/.
<lifeless> sure
<lifeless> yu have mail
<wgrant> Thanks.
<lifeless> de nada
<wgrant> Huh.
<wgrant> Did that get *CPed* without being EC2'd?
<wgrant> Crazy.
<lifeless> da wtf
<poolie> lifeless: i don't know the state of the committed/released distinction
<poolie> there are various contradictory bugs
<poolie> i don't think it's useful
<poolie> in that it makes noise while simplifying reality too much to be useful
<poolie> but maybe that's just me
<lifeless> I really don't like the noise when a release happens
<lifeless> The 'it is available to the user to use now' aspect is good though.
<lifeless> perhaps doing daily releases would address it ?
<lifeless> (e.g. edge :))
<poolie> that's one element of my dissatisfaction
<poolie> 'fix released' does not aiui reliably mean you will see the fix, even on edge
<poolie> doing daily releases would be good
<poolie> it wouldn't reduce the amount of noise
<lifeless> whats one of the bug numbers you saw
<poolie> istm that a lean view of the process is: it's waiting, it's in progress, it's done
<lifeless> I'm wondering if I'm missing a subscription somewhere
<poolie> bug 592792
<lifeless> yeah, I need to tweak my subscriptions
<lifeless> \o/ more mail.
<wgrant> poolie: You're saying that's not fixed on edge?
<lifeless> poolie: perhaps there are two clients here - the feature requestor and the dev asking 'do I have more to do' ?
<poolie> wgrant: that was a bug that sent me mail, i'm not saying it is or isn't fixed on edge
<poolie> lifeless: there are
<poolie> the feature requester wants to know "is it worth me testing whether it's really fixed"
<poolie> or "bothering to try that again" etc
<lifeless> if we try we can probably make both deeply unhappy
<poolie> heh
<lifeless> oops, thats the wrong way around :)
<poolie> istm that a state change is not enough
<poolie> without saying "this is live on edge now" or "this will be in bzr2.2b4"
<lifeless> I need a volunteer at the epic, with a 2 minute 'lightning talk' screen
<thumper> lifeless: in a certain light, isn't the point of --author to say who the author of the code is?
<thumper> lifeless: a merge isn't really code
<thumper> so pqm could be the committer, and set the author
<thumper> which is what I think many people think like
<lifeless> thumper: if you squint real real hard, but both Aaron and I have argued that this is working around a bug rather than fixing the root cause.
<thumper> :)
<lifeless> thumper: if we were sending in *patches* it would be trivially correct to use --author
 * thumper cranks up the muzak
<lifeless> code in an elevator?
<thumper> matrix soundtrack
<lifeless> nice
<wgrant> thumper: So, how do I get this testfix merged?
 * thumper looks up
<thumper> wgrant: eh?
<thumper> what's broken?
<foxxtrot> Is there an easy way to run all the windmill tests in a given JS file? I know you can bin/test PATH/TO/PYTHON/FILE.py but that doesn't work for a JS file
<wgrant> prod_lp, at least, but devel should be too.
<lifeless> wgrant: oh right
<lifeless> wgrant: whats the branch, I'll ec2land it
<wgrant> lifeless: Is that going to work with [testfix]?
<lifeless> I don't know the process yet :)
<lifeless> thumper: how does one land a testfix fixing branch
<thumper> pqm-submit
<wgrant> https://code.edge.launchpad.net/~wgrant/launchpad/testfix-getRestrictedfamilies is the branch. Trivial -- there were just some references in db-devel that were merged in while the branch was landing.
<lifeless> thumper: directly, not ec2land ?
<thumper> for a test fix, normally yes, but...
<thumper> it depends on the extent of the fix
<thumper> since buildbot is broken anyway
<thumper> adding an extra ec2 run doesn't add anything
<wgrant> All the archive and PPA tests pass fine.
<wgrant> Which is, I believe, all that failed in lifeless' run.
<wgrant> Although I haven't worked out how to get a list from the gzipped thing yet.
<lifeless> wgrant: gunzip -c | subunit-filter | subunit-ls
<lifeless> wgrant: or
<lifeless> gunzip -c | testr load; testr failing
<wgrant> Fancy.
<lifeless> nah
<lifeless> its all about the onions
<lifeless> if you want fancy
<lifeless> gunzip -c | tribunal -
<lifeless> now thats sexy
<thumper> I'd just like the failing tests back in the email body :(
<lifeless> thumper: they are
<wgrant> thumper: +1
<lifeless> aren't they ?
<thumper> no
<lifeless> oh right, I mailed bac and mars when jml forwarded me mail
<lifeless> its very shallow. Lets do it at the epic.
 * thumper nods
<lifeless> thumper: can you pqm-submit wgrants branch? the change is a three-liner, one char per line ;)
<lifeless> thumper: (given you're all setup for it)
<lifeless> I've got ec2land setup, not pqm-submit yet.
<lifeless> Hmm, I should make hydrazine talk lp targets to
<lifeless> too
<thumper> I only have devel and db-devel set up
<thumper> I should be able to do it without too much problem though
<lifeless> it should land on devel fine.
<lifeless> in devel if I do a missing on his branch I see just one commit
<wgrant> I'm not sure if this will work on production-devel, since I can't see it.
<thumper> it seems the error isn't on devel is it?
<wgrant> But it should be fine.
<wgrant> devel is broken.
<lifeless> devel is naffed too
<thumper> ah
<wgrant> I'm not sure if there's a failure yet, but it is broken.
<thumper> the devel failure was a weird twisted one
<thumper> I'll land wgrants
<wgrant> thumper: Thanks
<wgrant> lifeless: How do I run tests with testrepository?
<wgrant> quickstart doesn't say.
<lifeless> testr run
<wgrant> Ah, I presumed I'd have to pipe 'testr failing' in or something.
<lifeless> wgrant: you can (testr load - in the quickstart)
<lifeless> but you can also run tests directly if there is a .testr.conf
<lifeless> please file a bug though
<lifeless> that should be polished more
<thumper> :(
<thumper> bzr crashed
<thumper> bzr pqm-submit -m "[testfix][r=thumper] getRestrictedfamilies camel case fixup." --public-location=lp:~wgrant/launchpad/testfix-getRestrictedfamilies --ignore-local --submit-branch=../devel
<thumper> that is what I tried
<thumper> and it crashed
<wgrant> What did it complain about?
<Muscovy> Does anyone know anything about: createlang: language installation failed: ERROR:  could not access file "$libdir/plpython": No such file or directory ? It happens in an updated default install of 10.04 server.
<thumper> NoPQMSubmissionAddress
<thumper> basicly
<Muscovy> It happens during "make schema".
<wgrant> Muscovy: Make sure you have postgresql-plpython-8.3 or postgresql-plpython-8.4 installed, depending on which version of PostgreSQL you're using.
<Muscovy> Thanks, I'll try that.
<thumper> lifeless: any idea why my pqm-submit line doesn't work?
<lifeless> thumper: does your ../devel have a submit_thingy setting ?
<thumper> wgrant: I'll pull your branch into my testfix
<thumper> lifeless: yes
<lifeless> bzr info on it should say
<lifeless> thumper:  then no
<wgrant> thumper: Sounds reasonable.
<lifeless> I'd like to delete pqm-submit soon
<wgrant> But it needs to go to p-d too.
<lifeless> wgrant: the magic of auto merge should do that, no ?
<wgrant> lifeless: I really hope that production-devel isn't auto-merging anything.
<lifeless> devel being the top of the input tree
<lifeless> oh right
<lifeless> I meant after it percolates to db-devel
<thumper> lifeless: but not production devel
<lifeless> wgrant: mmm, I disagree on that hope; but there are some necessary conditions to make it safe that aren't true today.
<wgrant> lifeless: Well, OK, pre-MergeWorkflowDraft.
<wgrant> Is that still happening, what with our new overlord?
<thumper> wgrant: branch in the pipe
<lifeless> dunno, you'll need to ask him
<wgrant> lifeless: Is it still happening?
<lifeless> wgrant: :P I was referring to jml, he of the hacking-like-an-evil-overlord talk
<wgrant> Heh, indeed.
<lifeless> wgrant: not that he can answer the question, just for fun.
<lifeless> anyhow
<lifeless> I think that reducing developer friction is important
<lifeless> that process change should do that, and if there is an available hacker doing it, great.
<lifeless> I would like to look a little deeper at how concept -> production all works
<lifeless> possibly up the dial on risk mitigation and down on risk avoidance
<lifeless> then push rate-of-change up
<lifeless> I hope thats not uselessly vague. I haven't dug into the precise details of the process in some time - before buildbot - so Monday I'll be *very* busy indeed :)
<wgrant> Heh.
<lifeless> concretely, I think we court risk by doing big rollouts
<lifeless> this is observation over years
<lifeless> so we put a lot of effort into making sure nothing can go wrong in the rollout : but because its always a big change, something always does.
<lifeless> FSVO always
<lifeless> 30 man-days of changes, more or less.
<lifeless> smaller rollouts would have less risk.
<lifeless> and if the risk handling stuff is non-linear, then smaller rollouts may be better than just the ratio of sizes-included.
<wgrant> Yep.
<wgrant> But this requires the ability to handle a crisis effectively at any time.
<lifeless> it may be easier to figure out what *can* go wrong looking at a smaller rollout, and so be more prepared for what-ifs.
<wgrant> Which is certainly not the case now.
<lifeless> wgrant: not quite true. It requires the ability to handle a crisis starting within some time window T of the rollout
<lifeless> you could, for instance, have an automerge-wait-for-losa-to-hit-a-red-button situation
<lifeless> and every losa at start of shift could assess the risk, and hit the button.
<lifeless> if the relevant people are around-and-will-be-for-say-2-hours
<wgrant> True.
 * lifeless handwaves furiously
<lifeless> if you're interested in helping shape something like that, say so
<lifeless> I can chat about it with folk at the epic and see if we can give you a set of constraints and requirements
<lifeless> I mean to say: this is something that anyone interested should be able to work on.
<lifeless> and while I'm interested, I'm unlikely to have personal directed time on it immediately. But I'd love to see things become easier for everyone :)
<thumper> wgrant: I'm applying your fix to the prod-devel branch too
<wgrant> thumper: Thanks.
<spm> fwiw, recognising it is handwaving, we don't look after just LP. we have 3 or 4 (depending on how you count it) major systems. so the implicit assumption that we can treat LP as special/above the others is ... hrm... unwise is harsher than I want, but you get the idea. So 'start of shift, make a choice' stuff would not be great I'm thinking
<wgrant> spm: Your name says you only have two services, so any of the others are figments of your imagination. QED.
 * spm ponders if suspending wgrants account in retribution would 1. be overkill 2. pointless as he'd hack his way back in anyway 3. seen to be an excessive response 4.. there is no 4.
<wgrant> Heh.
<spm> lifeless: fwiw#2, yesterdays rollout was incredibly smooth. there were internal comments that a crisis needed to be manufactured so tom'd believe we were actually working. Having said that. 1hr 15 in the actual rollout portion of really quite intense very procedural Do X, then Y; isn't a walk in the sunshine either. :-)
<wgrant> Why does it take so long?
<wgrant> DB upgrades?
<spm> that's part of it, but no means all.
<spm> a basic breakdown: 10-15 mins to breakout to R/O and be able to do the DB updates. This includes shutting down the services that currently can't be kept up. code*, soyuz* etc
<spm> DB updates themselves, which can vary widely. 20-30 mins'd be the norm
<spm> re-enable the DB to be live again; restart all the app servers; verify; restart all the shutdowns.
<wgrant> Hm, OK.
<spm> And also ignores about 60 minutes of moderately intense prep to get to that stage :-)
<wgrant> Prebuilding the new trees on all machines and such?
<spm> crontabs, nagios, irc topics, etc yup
<spm> verifying the cowboys that are live are known cowboys and are included
<lifeless> spm: I know you have massive widespread responsibilities
<lifeless> spm: for start of shift, read 'appropriate time' : I find dwelling too much on the minutiae distracts from the concept.
<spm> heh
<lifeless> spm: I'd _love_ it if everything that makes a rollout slow (e.g. shutting down services to go ro) had bugs. And I knew the bug numbers.
<lifeless> spm: do you know if that is the case?
<spm> not sure tbh. some parts we have raised in the past. whether they became bugs, probably not.
<lifeless> I'm a data monster, really ;)
<spm> I hadn't noticed? This is very much news to me!
<spm> damn. left my sarcasm meter on. it just exploded.
<lifeless> :)
<spm> I guess what I'm not saying above - ideally I (personally!) don't want us to be yay/nay a rollout except in *exceptional* circumstances. like edge. it is it assumed it will work; if it doesn't we handle said failure gracefully, and that's the point we'd get involved - but the failure is not critical. Not a respond now event. More a respond soonish.
<lifeless> that makes sense to me
<lifeless> the nuance here, is that I was proposing you assess the surrounding support for dealing with stuff - controlling the timing, not the do/not do.
<spm> or perhaps: it's your code, your system (tho recognising that is an artificial distinction with ownership there...); if you want X, we'll make it so, but pls retain 'ownership' of the responsibility. kinda thiungy.
<lifeless> because you are the response team. You know if you're insanely busy, or just flat out busy.
<spm> ahh I see
<lifeless> we together provide launchpad.net - its lp-devs + lp-managers, not lp-devs alone or lp-managers alone
<lifeless> (which you know)
<lifeless> :P
<spm> I try to forget....
<lifeless> back to the example - if there was a buildd change, you might want lamont available
<lifeless> so you'd say 'delayed to $his tz'
<lifeless> but most changes are relatively shallow and would just be 'doit'
<spm> I'd, again very much personally!!!, be very keen if lp prod rollouts were much like edge is now. just a matter of course and all done purty much automagically. The key reason being a somehwat selfish one - is that makes our life much easier. if the rollout is so smooth that simplish scripting can do; then problems are also equally trivial to solve
<thumper> lifeless: hey, as your new role as TA, you can give out rc's for production-devel
<thumper> lifeless: want to give me an RC for wgrant's fix?
<lifeless> thumper: What are the implications here :) - its day -2 :P
<thumper> not much in this case, I'm just dotting the i's
<spm> actually - if there's a buildd change, I don't believe we have too many options but to wait for him. tbh, not sure of the fine detail there.
<lifeless> spm: so you see the point :)
<spm> oh yes.
<thumper> lifeless: I could equally go release-critical=thumper, but I thought you might like your name there :)
<lifeless> spm: FTR, I'm a huge fan a automation. I was surprised by Tom's apparent disinterest in the recent thread about detecting stale trees or whatever it was.
<lifeless> thumper: use yours ;)
<thumper> :)
<thumper> ok
<lifeless> thumper: When I'm doooog tired after that terrible hotel, iz not a good time to do new things ;)
<spm> which thread was that, don't recall seeing it myself?
<thumper> heh
<adeuring> good morning
<wgrant> Yay, buildbot loves me.
<spm> nah, that was me forcing a build :-)
<spm> thumper: wgrant: edge1: canonical.database.revision.InvalidDatabaseRevision: patch-2207-56-0.sql has been applied to the database but does not exist in this source code tree. You probably want to run 'make schema'. <== is that you guys causing that?
<wgrant> spm: stable is out of date.
<spm> oh bah.
<wgrant> Since it broke soon after db-stable was merged.
<wgrant> buildbot is happy now, though.
<wgrant> So it should pull soon.
<spm> we can hop. I've reverted the edge update; I guess we'll find out this timish tomorrow if edge is happy again
<spm> hope too
<wgrant> spm: Heh, it's just pulled now.
<jtv> wgrant, do you know of any recipe builds that should work on either dogfood or staging so we can Q/A your change to launchpad-buildd?
<wgrant> jtv: I don't know if staging works yet, but I may be able to get something working on dogfood.
<wgrant> ... except that I forget how it interacts with codehosting.
<wgrant> Does it use staging or production codehosting?
<jtv> It just doesn't have any.  So we may have to fake database records or something.
<jtv> Actually, if it's just for reading from a branch, it uses production codehosting.
<wgrant> Right.
<wgrant> Is buildd-manager running?
 * jtv checks
<wgrant> Oh good, I can't log in on DF.
<jtv> how jolly
<wgrant> (OOPS-1650DF10)
<jtv> On the bright side, buildd-manager is indeed running
<wgrant> bigjools: Any idea why DF won't let me log in?
<bigjools> wgrant: checking
<wgrant> bigjools: Thanks.
<bigjools> "HTTP Response status from identity URL host is not 200. Got status 404"
<bigjools> awesome
<wgrant> No provider set up?
<bigjools> some config must have changed somewhere
<bigjools> no idea what that might be
<wgrant> Can someone please land https://code.edge.launchpad.net/~wgrant/launchpad/faster-and-more-general-getBuildQueueSizes/+merge/28476 ?
<deryck> Morning, all.
<bigjools> wgrant: nice, I'll land it
<wgrant> bigjools: Thanks.
<bigjools> wgrant: try again
<wgrant> bigjools: Failure continues.
<bigjools> gnargh
<wgrant> OOPS 20
<bigjools> same problem
<bigjools> the oops report doesn't do anything useful and print the url it's trying to use
<bigjools> that would be too useful
<wgrant> Heh.
<bigjools> sigh
<wgrant> What's the OpenID host it's configured to use?
<bigjools> fuck nose
<bigjools> the config is so convoluted it's hard to work out
<bigjools>   Module canonical.launchpad.webapp.login, line 183, in render
<bigjools>     allvhosts.configs[openid_vhost].rooturl)
<bigjools> but rooturl doesn't exist
<wgrant> bigjools: You need launchpad/openid_provider_vhost set.
<bigjools> it is
<wgrant> To one of your vhosts.
<wgrant> And that vhost has its hostname set?
<bigjools> yes
<wgrant> Yay...
<bigjools> 404 means it's talking to something at least
<wgrant> Something, yes.
<bigjools> but what....
<bigjools> wgrant: please try again, I added some more info to the exception that goes in the oops
<wgrant> bigjools: Done. 22.
<bigjools> GAR
<wgrant> Oh?
<bigjools> I picked the wrong egg to hack
<wgrant> Heh.
<bigjools> try again...
<bigjools> nm
<wgrant> Any luck?
<bigjools> no
<bigjools> it's trying to access https://login.dogfood.launchpad.net/ which 404s
<wgrant> Right.
<wgrant> You could just tell it to use login.launchpad.net...
<bigjools> I *could*
<wgrant> Unless you want c-i-p fun...
<bigjools> trying one last thing before I give up
<bigjools> bah no fair, I have to travel on Sunday when the British GP is on
<lifeless> wgrant: cip?
<wgrant> lifeless: canonical-identity-provider.
<wgrant> Crazy Django thing.
<lifeless> yup
<lifeless> I remembers
 * bigjools shovels more coal into dogfood
<lifeless> night
<wgrant> Night.
<bigjools> grar
<bigjools> still no dice
<wgrant> bigjools: What're you trying to do?
<bigjools> point df at staging's openid
<wgrant> Hm. It's not working? Firewalled, maybe?
<bigjools> changing the config made zero difference
<wgrant> There are three OpenID vhosts -- you changed the right one(s)?
<wgrant> I've managed to get the wrong one before.
<bigjools> there's only one in the DF config
<bigjools> FINALY
<bigjools> and finally too
<wgrant> Yay, it works. Thanks.
<bigjools> now, food
<wgrant> Is buildd-manager alive?
<wgrant> It apparently was before.
<wgrant> But it's not dispatching a recipe build now.
<wgrant> Ah, there it is.
<wgrant> rubidium really is unbelievably slow.
<wgrant> jtv: Around?
<jtv> wgrant: Around.
<wgrant> https://code.dogfood.launchpad.net/~wgrant/+recipe/ivle-test/+build/145 just worked. The i386 build seems to be working fine too.
<wgrant> jtv: ^^
<jtv> wgrant: \o/
<jtv> I'll tell lamont that we can roll out
<wgrant> Great.
<lifeless> morning
<mtaylor> morning lifeless
<lifeless> hi mtaylor
<lifeless> 'sup ?
<mtaylor> lifeless: hanging out in dallas. enjoying sitting by the pool hacking
<lifeless> nice
<lifeless> mtaylor: hacking on blueprints yet ?
<mtaylor> lifeless: not yet
<mtaylor> lifeless: currently, actually waiting on some launchpad things to get renamed ... but I'm guessing the losas are busy
<lifeless> looks like it
<mbarnett> yeah, sorry.  I am a bit behind
<mbarnett> should be able to get to it this evening, or first thing in the morning central US time.
<lifeless> gary_poster: got a second ?
<gary_poster> lifeless, on calls till EoD but can reply for little questions
<lifeless> sure
<lifeless> its just a 'best way to do' question
<lifeless> I have this lsprof patch to extract the code into a dedicated module using an event
<lifeless> it needs either a tweaked, or a whole new, zope.app.publication
<lifeless> the tweak is adding the event; the whole new would be landing the 2 upstream patches and upgrading to latest upstream
<gary_poster> I see
<lifeless> or I could add the event in the lp tree and reference it there
<lifeless> whats the current, usual, practice when fixing something the long term way requires an upstream change? Do we submit the upstream change and then do /whatever/, or do we fork the egg temporarily ?
<gary_poster> will answer within half hour or so (sorry)
<gary_poster> (want to ask questions :-) )
<lifeless> thats fine
<lifeless> given the ec2land latency it won't make much diff to when it lands.
<lifeless> In 40 minutes I'm heading out for 2 hours - pre-flight-chores
<lifeless> but its not urgent regardless.
<gary_poster> lifeless: sorry, calls kept going
<gary_poster> lifeless: my preference order:
<gary_poster> 1) submit upstream and use upstream (sometimes impractical)
<gary_poster> 2) submit upstream and change locally in LP with one of our many existing subclasses
<gary_poster> 3) submit upstream and use upstream older egg with change (e.g., we're using 3.4.1 and upstream is at 3.5.0: we can try releasing 3.4.2 in addition to getting patch in trunk
<gary_poster> )
<gary_poster> 10) (i.e., we really don't wanna do this but sometimes we have to): make our own branch and make our own egg
<gary_poster> with (10) and possibly with (2) a LP bug is appropriate
<gary_poster> Done, and going
#launchpad-dev 2010-07-09
<lifeless> gary-afk: ok, so I've submitted upstream, but upstream zope.app.publication looks to be in the middle of a refactoring
<lifeless> so 2) - can do
<lifeless> mtaylor: please comment on HUDSON-5223
<lifeless> bah echannel
<maxb> Hmm. I wonder if I should nudge people away from importing multiple separate projects within the namespace of a single Launchpad project, when reviewing vcs-imports.
<wgrant> maxb: That certainly sounds like a good idea.
<lifeless> also towards using one project for one codebase
<lifeless> (see python-email6)
<lifeless> the most visible change with the new job for me: a whole new class of spam.
<wgrant> lifeless: Does bugmail make up most of it?
<lifeless> none of it
<lifeless> 'enterprise architecture seminar' etc
<wgrant> Oh dear.
<lifeless> tell me something
<lifeless> if we *deleted* the entire build scoring stuff
<lifeless> and just said 'distro first, ppa builds second, recipes third, rebuild-tests fourth'
<wgrant> It would work fine, yes.
<lifeless> do you think people would be happier or sadder ?
<lifeless> ok
<lifeless> clearly I need to ask actual distro folk
<lifeless> :P
<wgrant> Well.
<wgrant> In the current model it would work fine.
<wgrant> But soon it won't.
<wgrant> When we destroy the virt/non-virt distinction.
<lifeless> I don't see why that would invalidate it
<lifeless> tell me more!
<wgrant> We probably don't want the first autosync run to drown out PPA builds for days.
<wgrant> Nor do we want mass give-backs to do so.
<lifeless> what if we said
<lifeless> rather than strictly ordered
<lifeless> 10 distro to 2 ppa
<lifeless> 10 ppa to 2 recipes
<lifeless> 10 recipes to 2 rebuild tests
<lifeless> or something like that
<wgrant> Possibly.
<wgrant> I also think we should look at existing scheduling theory and see how things are meant to be done. We can do better than a CPU scheduler, since we have job time estimations.
<lifeless> well
<lifeless> right now I'd settle for 'real simple and understandable'
<wgrant> We need to draw out various use cases like those I've just outlined.
<wgrant> And work out what will work.
<wgrant> Since the current one probably doesn't work.
<wgrant> Although it's not quite clear, given how crap buildd-manager is at the moment.
<lifeless> I would like something with the following characteristics:
<lifeless>  - does not do O(anything) work after accepting a task
<lifeless>  - supports some form of 'make this thing happen sooner pretty please' (but does not need to support 'make thing happen later')
<wgrant> We definitely need a 'rush build' button, yes.
<lifeless> most of the queuing system out there are not terribly keen on lots-of-fiddle : they want simple reliable robust
<mtaylor> lifeless: is there any good way to bribe a losa?
<lifeless> yes
<mtaylor> sweet! what is it?
<lifeless> write a patch to launchpad that makes their lives easier.
<mtaylor> gah
<mtaylor> that will take longer than just waiting
<lifeless> what are you waiting for
<mtaylor> some renames
<lifeless> details
<lifeless> branches or projects
<lifeless> people or teams
<lifeless> planets or governments
<mtaylor> a project, and two teams
<mtaylor> they've all got approved questions
<lifeless> did you make a question
<mtaylor> I did
<lifeless> linky linky
<mtaylor> https://answers.edge.launchpad.net/launchpad-registry/+question/116856
<mtaylor> https://answers.edge.launchpad.net/launchpad-registry/+question/116849
<mtaylor> https://answers.edge.launchpad.net/launchpad-registry/+question/116846
<spm> mtaylor: fwiw, cake works. as do any sweet baked substances.
<mtaylor> spm: I'll mail cake right away
<mtaylor> spm: does the cake need to taste good? or is it enough that it is simply cake?
<spm> the former is preferred, but the latter is, tbh, good enough
<spm> mtaylor: blarg. first hurdle of sorts. swift-team "This team cannot be renamed because it is private." <== I believe via the DB will work, but there may be funkies. be. ware.
<mtaylor> spm: lovely
<mtaylor> spm: I like causing trouble!
<spm> ./forcenickchange mtaylor Trouble (note capital T)
<Trouble> oh. this nick is registered...
<spm> hahaha
<mtaylor> spm: I found many things that don't like being done with private teams
<mtaylor> spm: and private projects
<spm> yeah. because they're the exception and not the rule; they tend to hit funky edge cases as it were
<spm> mtaylor: so the first two should be done
<mtaylor> spm: you are so getting a cake
<spm> heh
 * mtaylor goes to look up cake import laws for australia
<spm> they're probably horrible. even carrying in mentos and the like, need to be declared.
<spm> mtaylor: that's the lot, and answers updated accordingly. woo.
 * StevenK has declared food and medication every time he's re-entered Australia. Fun. Except not.
<spm> I believe we now have to declare porn too. Tho I felt let down at not having that option to fill in, on the last trip.
<StevenK> I've not yet seen that question
<mtaylor> spm: sweet. thanks!
<mtaylor> spm: oh - hey - you know those weird edge-cases you were mentioning
<mtaylor> spm: check out https://edge.launchpad.net/swift
<mtaylor> spm: all looks good except for "swift-files trunk series is the current focus of development"
<spm> orsum
<mtaylor> and yup. ... swift-files also mentioned on https://edge.launchpad.net/swift/trunk
<mtaylor> me finds bugs! :)
<spm> Ahhh. Display Name: swift-files <== for the project
<mtaylor> ah. weird. ok cool!
 * mtaylor didn't find bugs after all
<spm> 1 form edit later, that looks better!
<mtaylor> w00t!
<spm> heh; I dunno; that I didn't see that as needing to be fixed when doing the change...
<mtaylor> I think it was my sillyness that I had the display name be the same
<spm> :-)
<mtaylor> spm: you're going to shoot me... but I just noticed one more team/user thing, you know, because I'm stoopid
<mtaylor> spm: should I file new question?
<spm> nah is cool. which one?
<mtaylor> https://edge.launchpad.net/~nova
<mtaylor> spm: 0 karma user ... ~nova-team wants to be called ~nova in my boss's perfect world
<spm> darn bosses
<mtaylor> tell me about it
<mtaylor> I would have preferred ~nova-devel and ~swift-devel myself
<spm> hrm. that may still be in use by a wiki user
<spm> nope. false alarm.
<spm> https://edge.launchpad.net/~nova-es now
<spm> mtaylor: and done
<mtaylor> spm: I think you get two cakes now
<spm> \o/
<mtaylor> of course, aussie import quarentine means they'll likely be 6 months old by the time they get to you ... but you'll be sure they do not have rabies!
<spm> hahaha
 * spm recalls an amusing little take with quarantine and ship ballast water forms filled/filed via email and clasing with the tax office dropping a million or so emails in our queues of a friday afternoon. For some reason ships are a tad unreasonable to slowing down because a mail funnel is overwhelmed.
<mtaylor> hahahahaha
<lifeless> spm: hey question
<spm> yo
<lifeless> spm: is it possible to get something that would let me read most/all of dev.launchpad.net on the plane to prague ?
<spm> wget?
 * lifeless claps slowly
<lifeless> or are you serious ?
<spm> heh, was a serious suggestion. use it to recursively get the site down to 3-5 levels
<spm> fwiw, I've used wget as a not as quick, yet dirty and working method to mirror a certain chunk of a Federal health & Ageing Departments website for DR purposes.
<lifeless> oh yay naffed css on it
<spm> which was .. fun. domino would have two 'names' that are different in it's namespace, yet identical on a mirror disk: blah/stuff/morestuff and blah/stuff being a file
<spm>  -k? from memory?
<lifeless> -m ?
<lifeless> nope, needs more, its root-parenting them
<spm> possibly. the option that says 'get all this stuff'
<spm> extra stuff
<lifeless> nah 404's
<lifeless> -k is convert links
<spm> ah yes. you will want that then.
<lifeless> except its failing
<lifeless> oh yay also the follow to action=xxx is terrible
<spm> lifeless: wget -T 10 -o $ROOTDIR/../mirror.log -nv -rkEp -l 6 -nH
<spm> is an extract of the cmd used per that mentioned above
<spm> istr the -l 6 is "levels, 6 deep", which is oft trial and errorish
<lifeless> also
<lifeless> wth is that wiki on https?
<lifeless> is public
<spm> good question. no idea. hadn't noticed that before.
<lifeless> grrrr
<adeuring> good morning
<wgrant> bigjools: Morning.
<bigjools> wgrant: hi
<wgrant> bigjools: So... the PPA log parser should work now, although I'm not sure if the production configs have the limit set.
<bigjools> wgrant: nearly, I'll sort the rest today
<wgrant> bigjools: Thanks!
<wgrant> What happens if I create a PPA named 'meta'? Does the world explode?
<wgrant> (since USER/meta seems to be used for metadata from other PPAs now)
<jml> wgrant, asking is for minions
<jml> wgrant, evil overlords do science!
<lifeless> wgrant: you've done enough tests of these things in prod.
<lifeless> wgrant: why start asking questions now ?
<wgrant> lifeless: Huh?
<bigjools> wgrant: I expect it will be fine right up until the point someone deletes the package with the meta data
<lifeless> wgrant: I mean, give it a shot.
<lifeless> wgrant: play around like you normally do :)
<wgrant> bigjools: Well, or the 'meta' PPA becomes private.
<bigjools> wgrant: it should co-exist fine with a PPA of that name
<wgrant> lifeless: I don't play around and try to break production, except for U1, and that's because I have no other option :P
<jml> I wonder how hard it would be to change zope.testrunner so that it can do its own layer composition
<jml> so that instead of layer = LaunchpadFunctionalLayer, we could say layer = (DatabaseLayer, FunctionalLayer)
<jml> or what have you
<lifeless> fairly straight forward
<lifeless> also should be fairly easy to make a testresources thing to process layers
<lifeless> fwiw
<jml> I've thought about that
<jml> a) I'm not 100% sure what that would mean
<jml> b) I don't know how to handle the "doesn't support tearDown" problem
<lifeless> fix the root cause?
<lifeless> what layers do we have that don't support tearDown ?
<lifeless> [and how can we find that out]
<jml> lifeless, http://paste.ubuntu.com/461047/
<lifeless> none of those sound like they should be hard to fix.
<lifeless> *sound*
<jml> (I told you all that subunit was worth it!)
<lifeless> I don't quite get the context on your last ;)
<wgrant> The main issue is tearing down the CA, right?
<lifeless> wgrant: there are 5 layers that don't support teardown
<lifeless> wgrant: so there are, in theory, 5 issues, no ?
<wgrant> No.
<wgrant> Those four are probably all CA-related.
<wgrant> The last, I'm not sure.
<wgrant> Remember that the layers are composed.
<lifeless> jml: ah, I see, you queried subunit for the answer
<jml> wgrant, CA?
<wgrant> jml: Component Architecture.
<wgrant> Our lovely, lovely global state.
<jml> right. specifically all of the registering done by zcml.
<wgrant> Right.
<jml> lifeless, how else would you find out? :)
<jml> undoing that is probably not all that hard.
<wgrant> It'd also be really nice if it didn't take lots of seconds to do in the first place.
<jml> wgrant, yeah, it would.
<jml> wgrant, did you know that when processing zcml, all of the registered utility classes are constructed?
<wgrant> jml: ..... seriously?
<jml> wgrant, yeh.
<wgrant> I... thought that getUtility would do that on the first call.
<jml> wgrant, it's caused the occasional weird import problem
<jml> wgrant, as did I
<lifeless> so
<lifeless> this sounds simple
<lifeless> does anyone feel like shaving it ?
<lifeless> I promise a review if they do.
<jml> heh heh
<mwhudson> i knew that, because one of the utilities classes overrides $GNUPGHOME to something stupid in its constructor
<mwhudson> https://bugs.edge.launchpad.net/launchpad-foundations/+bug/227586
<jml> lifeless, I'm already nested a couple deep. gimme a few minutes
<_mup_> Bug #227586: setting up the component architecture overwrites $GNUPGHOME <tech-debt> <Launchpad Foundations:Triaged> <https://launchpad.net/bugs/227586>
<lifeless> test isolation, thats always a good idea.
<mwhudson> the problems that causes aren't (just) in tests
<lifeless> sure
<lifeless> I have a nasty suspicion its my fault
<lifeless> anyhow
<lifeless> moving right along...
<jml> has db-stable been merged back into devel yet?
<jml> I guess I can find out myself
<wgrant> Yes.
<wgrant> A couple of days ago.
<wgrant> It broke, but it's fixed now.
<jml> wgrant, thanks.
<jml> https://code.edge.launchpad.net/~jml/launchpad/remove-product-branch-portlet/+merge/29532
<wgrant> jml: Oooh deleting sample data?
<jml> wgrant, yeah, just the branch sample data
<jml> wgrant, http://paste.ubuntu.com/461059/
<jml> I'll push my branch up too
<wgrant> jml: ... dragons?
<jml> wgrant, as in "here be dragons"
<wgrant> Ah.
<jml> all of the pagetest/windmill stuff probably isn't intrinsically harder than the others
<jml> but I'm afraid, and the list is long enough that I need to break it up in order to approach it anyway
<jml> lifeless, could you please add an approved vote too? it makes ec2 land easier to use.
<lifeless> grr
<lifeless> by which I mean
<lifeless> yes of course
<bigjools> AAAAAAAARRRRRRRRRRRRGGGGGGGGGGGGGGGGGG
<jml> lifeless, thanks.
<jml> bigjools, you seem vexed.
<bigjools> yes
<jml> wgrant, lp:~jml/launchpad/no-branch-sample-data if you want to follow along.
<wgrant> jml: s/branch-//
<jml> wgrant, well, that's the goal :)
<jml> wgrant, but one step at a time.
<wgrant> Soyuz will be... fun.
<wgrant> Although Code has done well with the factory for Soyuz stuff.
<bigjools> where fun == pain hereto untold
<bigjools> the old soyuz tests suck beyond belief
<wgrant> Um, yes.
<wgrant> I beliiiieve they're the worst in the tree.
<bigjools> why does uncommenting the code here make this return an extra row in the results? http://pastebin.ubuntu.com/461061/
<bigjools> extra_exprs is not even used in the query yet :/
<lifeless> jml: https://bugs.edge.launchpad.net/launchpad-code/+bug/603532
<_mup_> Bug #603532: make a simple 'approve' on a merge cast an approve vote <Launchpad Bazaar Integration:New> <https://launchpad.net/bugs/603532>
<jml> lifeless, thanks.
<wgrant> lifeless: Shouldn't that be handled by automatically setting the MP status when the right votes are present?
<jml> wgrant, ideally, it would work both ways
<lifeless> wgrant: thats a different story, and also nice to have.
<lifeless> wgrant: one could write a scanner to scan for mp's that have individual votes which add to 'approved' and do it per-project
<lifeless> I'd like to make that efficient,.
<lifeless> e.g. event driven yada yada.
<bigjools> Psychic Octopus has spoken
<jml> bigjools, there's not enough information there for me to even make a guess.
<bigjools> jml: well, now it works.... I'm at the stage where I don't care why it wasn't working
<jml> ok.
<bigjools> sorry for the noise
<lifeless> no worries
<jml> I just had an idea I should have had two years ago
 * jml enhappenates
<lifeless> and whats that?
<jml> Hush. I'm creating.
<deryck> Morning, all.
<lifeless> hai
<jml> deryck, hello
<jml> lifeless, http://pastebin.ubuntu.com/461067/ -- that's what.
<wgrant> jml: Randomness in tests: Australia says 'no'.
<jml> yeah, I was uncertain about that bit.
<jml> the other option (which I've used directly in other tests) is teamowner
<wgrant> Although I guess it could weed out issues... it could also make everything damn confusing.
<jml> erratic tests are bad.
<jml> wgrant, of course, the whole point is that the manner in which one gets an arbitrary team member is completed separated from one's need for it
<wgrant> jml: Right.
<jml>     # David Allouche is a member of the vcs-imports team.
<jml>     login('david.allouche@canonical.com')
<jml> ^^^ that's the sort of code I want to replace.
<wgrant> Right.
<wgrant> Could just choose the first member or something like that.
<lifeless> no
<wgrant> Owner sounds dangerous.
<lifeless> what does participation= in login do ?
<jml> lifeless, I don't know.
<jml> there's no docs on it either.
<lifeless> heh
<jml> lifeless, what question is "no" an answer to?
<lifeless> so this looks good.
<lifeless> but it could do with tests
<lifeless> no -> wgrants 'take first' - taking the first will bite, eventually.
<jml> wgrant, I don't know how deterministic choosing the first member is.
<lifeless> it will also tend to be the owner.
<lifeless> given that its currently rather difficult to make an owner outside the team, at least fo rusers of lp.
<jml> lifeless, I couldn't find any tests for the existing stuff...
<lifeless> jml: would you like a hand putting some smoke tests for this together ?
<jml> I'm also not sure about passing the celeb name as the interface.
<jml> but if it's a choice between login_as(getUtility(ILaunchpadCelebrities).vcs_imports) and login_celebrity('vcs_imports'), I think I know what I'll choose.
<lifeless> I'm not sure enough about celebs yet.
<jml> (I wish Python had a separate literal for symbols)
<lifeless> string seems fine.
<jml> hmm.
<jml> now I feel sad
<jml> lifeless, yes.
<jml> lifeless, all of this stuff boils down to setupInteraction
<jml> which lives in c.l.webapp.interaction
<lifeless> jml: yes what ?
<jml> and has tests here lib/canonical/launchpad/doc/webapp-authorization.txt
<jml> lifeless, yes I would like a hand putting some smoke tests for this stuff togethre
<lifeless> ok
<jml> lifeless, this yak shaving thing is exhausting.
<lifeless> the first times a doozy
<lifeless> I'm logging tointo my lp dev environment
<jml> I need to make a list.
<lifeless> oh wow
<lifeless> so, your patch adds login_as, login_celebrity, login_team
<lifeless> login_as needs two tests
<lifeless> or perhaps four (check participation is passed down)
<lifeless> celebrity wants one happy path
<lifeless> team wants one, and get_arbitrary wants either a test or to be private.
<lifeless> does that seem complete ?
<lifeless> jml: can you push your patch, we can divide and conquer
<jml> lifeless, sure, that seems good. let's make get_arbitrary_... private for now.
<jml> bzr+ssh://bazaar.launchpad.net/~jml/launchpad/more-login-helpers
<jml> http://paste.ubuntu.com/461073/
<jml> lifeless, so now I've found get_current_principal, I think I'm actually ok with figuring out how to write these tests.
<lifeless> ok
<lifeless> I like that you are yak shaving here.
<lifeless> The result should be really helpful to people!
<jml> woot, down another level.
<lifeless> \o/
<lifeless> want a review?
<jml> no, not yet, thanks. I'm one level deeper. I need to actually come back up before reviews make sense.
<lifeless> oh
<lifeless> I thought your stack was inverted based on your pastebin
<lifeless> Its midnight; I shall sleep; then fly.
<jml> uhh, no. the stack goes down.
<jml> lifeless, sleep well.
<jml> https://code.edge.launchpad.net/~jml/launchpad/remove-get-current-principal/+merge/29542
<salgado> jml, s/principle/principal  ;)
<jml> salgado, where?
<salgado> the function's docstring
<jml> salgado, thanks.
<mwhudson> jml: wouldn't login_team work better by creating a person, adding them to the team and logging in as that person?
<jml> mwhudson, good idea
<jml> I've got a question. It might be a tough one.
<jml> what's the difference between being logged in as an anonymous user and not being logged in?
<bigjools> test suite or API?
<jml> since you asked, both :)
<jml> (but test suite is my primary interest)
<mwhudson> jml: it has to do with whether the principal associated with the current interaction implements some interface i think
<jml> mwhudson, yeah, but what does that actually mean?
<jml> mwhudson, and why does it matter?
<mwhudson> jml: it means that some bits are different in ram somewhere
<mwhudson> jml: why does anything matter :)
<mwhudson> jml: you're not asking these questions in a way that's easy to answer :-)
<jml> why do we ever need to login(ANONYMOUS)?
<jml> why don't we model being "logged in as anonymous" as "not being logged in"
<mwhudson> oh sorry
<mwhudson> i misread
<mwhudson> if you're not logged in, there is no principal associated with the current interaction, or maybe there is no interaction
<jml> there's no interaction, it seems.
<mwhudson> for web-published stuff (like the api) logging in (possibly has anonymous) happens very very early
<jml> *** AttributeError: AttributeError("'thread._local' object has no attribute 'interaction'",)
<mwhudson> in tests and scripts and so on ... err
<jml> I wonder what an interaction is.
<mwhudson> not sure
<mwhudson> jml: it's a less specific word that request
<mwhudson> but it basically fits the same hole in the puzzle
<jml> mwhudson, I thought participation was that.
<mwhudson> oh
<jml> I might be wrong.
<mwhudson> ok, i don't remember what the difference between interaction and participation is then :-)
<jml> ok.
<mwhudson> i think one might be a more specific version of the other though?
<jml> benji, gary_poster: hello
<gary_poster> jml hey
<mwhudson> oh no
<mwhudson> the interaction can have a number of participations
<gary_poster> interactions have one or more partici...right
<jml> mwhudson, if I had to guess, I would say that a participation is the thing that bridges between a principal and an interaction
<gary_poster> but we never do that
<gary_poster> yes
<jml> gary_poster, so what's an interaction?
<gary_poster> jml, for our usage (and indeed standard usage) one request has one security interaction with the system (which is practically an instance of the security policy).  The interaction, for us, always has one participation, which is the user for that request.
<mwhudson> gary_poster: but in principle answering a request could have multiple interactions?
<gary_poster> (the multiple participation model exists to prevent privilege escalation in certain scenarios)
<jml> gary_poster, ok, that helps. some followup questions though...
<gary_poster> mwhudson in the abstract, yes
<jml> gary_poster, "practically"?
<gary_poster> jml, um, replace with "in our implementation, and the standard implementations"?
<jml> gary_poster, thanks :)
<gary_poster> np :-)
<jml> gary_poster, also, the participation is not the user for the request, at  least, not in our tests.
 * mwhudson runs
<jml> gary_poster, participation = LaunchpadTestRequest(
<jml>             environ={'HTTP_HOST': allvhosts.configs['mainsite'].hostname,
<jml>                      'SERVER_URL': allvhosts.configs['mainsite'].rooturl})
<gary_poster> yes, the participation is a request in the implementation
<gary_poster> from the security system's perspective, though, the request only has to implement a single interface
<jml> which is?
<gary_poster> that interface describes how to get the user from the participation
<gary_poster> eh, I'll look
<gary_poster> it's in zope.security someplace
<jml> from zope.security.interfaces import IParticipation ?
<gary_poster> must be it, yes
<jml> ok. pieces are drifting into place.
<gary_poster> and if you look at that interface, it only defines two bits
<gary_poster> a (back) reference to the interaction
<gary_poster> and a principal
<jml> gary_poster, so the reason that we get errors like "'thread._local' object has no attribute 'interaction'" when we try to use getUtility without first "logging in" is...
<gary_poster> an interaction has not been started
<gary_poster> as defined in zope.security.management I think; looking
<gary_poster> yeah, newInteraction
<gary_poster> (or perhaps something is being called with a security wrapper after the interaction has been ended)
<gary_poster> in any case, there is no active security interaction
<jml> gary_poster, ok, thanks.
<gary_poster> sure
<jml> gary_poster, I think as a result of this conversation I am going to maybe write a longer module docstring in c.l.webapp.interaction
<gary_poster> I'm going to run about now for travel prep and the like...
<gary_poster> ok I like improved docs
<gary_poster> I'll look at that module for the heck of it...
<gary_poster> yeah, doesn't say much
<gary_poster> jml, anything else I can help with, or may I run?
<jml> gary_poster, nothing for now
<gary_poster> ok cool
<jml> gary_poster, I'll shelve my next request :)
<gary_poster> ...
<gary_poster> ok :-)
<gary_poster> see you in Prague!
<lifeless> morning
<lifeless> gary-afk: when you get back, I've a small question about zcml and utility parameters / configuration of event driven things.
<gary-afk> morning lifeless.  ok, cool, what's up?  (fwiw, I'm mostly gone--travel prep, with occasional look-backs for pings like this)
<gary-afk> need to go in prob 5
<lifeless> Sure
<lifeless> so I have some itch scratching on the profile module thing
<lifeless> I have two functions called at start and end of requests
<lifeless> I want to decouple them from the oops system and the global config, for testing and reuse. So I plan to make a single object and have the functions just trampoline into it.
<lifeless> A Utility would match our current idiom, or perhaps just a module private.
<lifeless> Anyhow, I'd like to give the instance of this class a parameter that it can call to name-and-write-out a profile
<lifeless> because that code isn't related *doing* profiling, its related to *how we glue it in*
<lifeless> I could subclass and our zcml then specifies the subclass, but thats all kinds of wrong. So I was wondering if there is either a way to provide parameters in zcml configs, or a way to run some python code at the same point in startup that zcml parsing occurs.
<lifeless> fin
<gary-afk> reading and processing...
<lifeless> oh, one other small thing is that I'd then like to turn this into a totally external module - put it in the zope project or some such; I need to talk to flacoste about the logistics there; for now I'll probably keep it in-tree but give it its own dir and setup.py and build an egg for buildout from that.
<gary-afk> no there's not; however, that's mostly because you shouldn't need to.  (which isn't to say I haven't wanted to practically, but anyway...) you can have, for instance, a class that provides the functionality, and then have zcml register an instance of the class, set up in LP code, that actually has specified the location.
<gary-afk> A similar pattern, more for adapters, is to register in zcml a function as a factory
<gary-afk> and the function specifies custom local bits
<gary-afk> calling out to another function or class for the more general adapter bits
<lifeless> the former looks a better fit to me; are there examples of that ?
<lifeless> 'register an instance setup by LP code'
<gary-afk> well, there is in Zope code.  It's pretty easy; lemme get details for you
<lifeless> sweet, thanks
<gary-afk> lifeless: btw, handy: http://apidoc.zope.org/++apidoc++/ .  zcml section is where I look bits up in sometimes.  salgado is making the other bits available for LP devs customized to LP (like browsing LP interfaces and classes with crossreferences and stuff)
<lifeless> thanks
<gary-afk> lifeless: <utility component="my.package.my.module.instance" provides="the.interface" />
<lifeless> is using a Utility as a singleton a common pattern ?
<gary-afk> definitely
<lifeless> ok
<lifeless> and where would one put the lp code to setup 'instance'
<gary-afk> the zcml: can be simplified
<lifeless> (surely not module inline)
<gary-afk> depends on what you are doing
<gary-afk> oh, yes.
<gary-afk> if you want a factory then you can do a factory instead
<gary-afk> but yeah, looking for something in module
<lifeless> ok; so let me run this integrated thing past you once:
<gary-afk> <utility factory="foo.factory" provides=""
<gary-afk>  />
<gary-afk> I have to run now!
<gary-afk> sorry
<lifeless> configured instance is a utility
<lifeless> event hooks use utility
<lifeless> implementation out of tree
<lifeless> configuration in tree.
<lifeless> gary-afk: thanks, you've been helpful.
<gary-afk> +1, yes that's the intent
<lifeless> I'll have some fine tuning to do in Prague
<lifeless> thanks!
#launchpad-dev 2010-07-10
<nhandler> Why is ~launchpad-dev (an open team) setup so you need an admin to renew subscriptions?
<wgrant> Most probably an oversight that can be trivially fixed by a whole lot of people who are currently flying or about to fly.
<wgrant> Ah, I just got a warning too.
<nigelb> wgrant: you're flying there too?
<wgrant> nigelb: No.
<nigelb> wgrant: Gah.
<sarhan> Hi
#launchpad-dev 2010-07-11
<micahg> I'm getting a message to renew my membership in the #launchpad-dev team, this seems weird since it's an open team
<nigelb> micahg: we'll have to wait for people to wake up in prgue
<jml> nhandler, micahg: I don't know why.
<micahg> jml: should I file a bug?
<jml> micahg, I don't know, really.
<jml> micahg, it's certainly worth an email to the launchpad-dev list
<micahg> jml: k, will send one later today
<jml> ta
 * jml is off
<wgrant> https://code.edge.launchpad.net/~vcs-imports/dak/trunk is still failing, yet branching locally works fine.
<wgrant> This seems somewhat odd.
<thumper> wgrant: could be different versions of bzr-git/dulwich
<thumper> wgrant: or a corrupt local repo from an older version
<wgrant> thumper: Hmm.
<wgrant> Hm.
<wgrant> I think the new linter might actually not utterly suck.
<wgrant> This is good.
<lifeless> so who is here already?
<wgrant> Morning lifeless.
<wgrant> How's Prague?
<lifeless> hot
<wgrant> Heh, so I've heard.
<lifeless> 30'c today I hear.
<jml> hello
<dhastha> need help: I am going to do mini project for my college in launchpad translation (Native support for OpenOffice.org translations format in Launchpad Translations). If i install launchpad locally, is there any affect to my localhost
<dhastha> will my apache work properly?
<jml> dhastha, I'm not sure I understand the question.
<jml> dhastha, it will probably not break your current Apache set up.
<wgrant> It won't coexist happily with an existing PostgreSQL setup, but it can be made to afterwards.
<dhastha> jml, once a time i tried to install lauchpad locally, it was interrupted. After that if i run php, it shows download the file instead of running php
<wgrant> It may try to install apache2-mpm-worker, which will cause the removal of libapache2-mod-php5, since PHP is not threadsafe.
<wgrant> But you could tweak rocketfuel-setup to not try to install that, and just use prefork.
<jml> fwiw, I have three branches ready for review: https://code.edge.launchpad.net/~jml/launchpad/more-login-helpers/+merge/29595; https://code.edge.launchpad.net/~jml/launchpad/docs-for-interaction/+merge/29632; https://code.edge.launchpad.net/~jml/launchpad/branch-sample-data-unit-tests/+merge/29652
<dhastha> wgrant, how to install launchpad using prefork?
<jml> lifeless, my plane is scheduled to arrive at 1800, fwiw
<wgrant> dhastha: It's somewhere in rocketfuel-setup.
<wgrant> You'll need to change that.
<dhastha> wgrant, is there any possible to install both apache2-mpm-worker, libapache2-mod-php5.
<lifeless> jml: cool
<dhastha> wgrant, because i need php as well as launchpad. How can i run both?
<wgrant> dhastha: No. But Launchpad doesn't actually require apache2-mpm-worker, so it is safe to alter your version of the installation script to install install apache2-mpm-prefork.
<wgrant> s/install install/instead install/
<lifeless> also <3 the canonical sysadmins who have an Ubuntu network here already
<lifeless> jml: I'd be delighted to catch up in the evening if you like
<wgrant> Is it just Launchpaddery going on, or is there something else there as well?
<lifeless> epic for one week
<lifeless> platform team for one week
<lifeless> serialised
<wgrant> Ah.
<lifeless> some folk are double-billing it
<dhastha> wgrant, where can i get open office's custom GSI/SDF format?
<lifeless> no idea
<wgrant> dhastha: You should probably ask OpenOffice.org people.
<lifeless> dhastha: you might like to have your launchpad test environment be a vm - e.g. as per https://dev.launchpad.net/Running/VirtualMachine
<wgrant> dhastha: Have you talked to the Launchpad Translations developers about this?
<dhastha> wgrant, No. i cant find any Launchpad Translations develpers. But i informed once a time in this channel. someone replied me.
<lifeless> hmm, is there a bug about big diffs not showing in MPs - (not that they are truncated, but that no alternative is offered)
<wgrant> lifeless: There's no download link presented?
<lifeless> oh
<lifeless> its at the *other end*
<lifeless> I'll file a bug
<lifeless> scrolling up 3000 lines (or home, and then down *just the right amount* is really fiddly - and its annoying to get partway through  areview before dinign out its truncated
<wgrant> It should probably notify you at the top that it's truncated, too...
<lifeless> exactly
<lifeless> I have to wonder why we truncated anyway though.
<lifeless> thats a different conversation
<lifeless> right, filed
<lifeless> bug 604277
<_mup_> Bug #604277: polish for huge mp-diff experience <Launchpad Bazaar Integration:New> <https://launchpad.net/bugs/604277>
<jml> heh
<lifeless> jml: I seem to recall that differing postgres versions results in a freaking huge diff
<lifeless> jml: but maybe its always terrible?
<jml> lifeless, huge diff for the sampledata?
<lifeless> yeah
<jml> lifeless, it's pretty much always terrible
<wgrant> It shouldn't be.
<wgrant> It never used to be, until 8.4 came along.
<lifeless> wgrant: it has in the past been terrible too
<jml> wgrant, it did used to be.
<wgrant> Hm, not while I was watching.
<jml> I've got to go catch my flight.
<jml> lifeless, I'd really appreciate a review of more-login-helpers and branch-sample-data-unit-tests
<lifeless> jml: I'm chewing through mail looking for urgents; I need to update my draft talk to be a real talk. Then I'm all over the reviews.
<dhastha> Is there any launchpad translation developer?
<lifeless> there are a few
<lifeless> most of them will be offline right now as we're having a get together next week and people will be travelling (or in their weekends)
<lifeless> A great way to contact devs is the launchpad-dev mailing list
<dhastha> lifeless, k thank you
<wgrant> jml: So, launchpad-dev is currently bugging lots of us to contact you guys to renew our memberships. Since it's an open team, could someone flip the switch that lets us renew them ourselves?
<lifeless> neh
<lifeless> I've made it auto renew.
<lifeless> enjoy.
<lifeless> renewing for mailing list memberships is bong.
<wgrant> Ah, even better.
 * lifeless embugginates
<wgrant> Thanks.
<lifeless> lol
<lifeless> look at https://bugs.edge.launchpad.net/launchpad
<lifeless> thats one hell of a title
<wgrant> The fact that it's broken in Chromium, or something else?
<lifeless> worked for me
<lifeless> I have rev 49890 of chromiu,
<wgrant> 51881 here. The 'There are' appears to the right of the search box, so it looks like the main content of the page is just 'currently no open bugs filed against Launchpad itself.
<lifeless> hmm
<lifeless> bug 604289
<_mup_> Bug #604289: on open teams / mailing lists 'renewal' doesn't make much sense <Launchpad itself:New> <https://launchpad.net/bugs/604289>
<lifeless> https://bugs.edge.launchpad.net/launchpad shows bug 603020 in the hot bugs list for me
<lifeless> ah
<lifeless> its private.
<wgrant> Ah.
<lifeless> and I can't move it to ubuntu because you can't sensibly edit cross-pillar
<wgrant> You can edit cross-pillar easily.
<wgrant> You just can't change it to a different type of pillar.
<lifeless> thats what I mean
<wgrant> Not even through the API.
<wgrant> The restriction appears to be entirely baseless.
<lifeless> patches desired
<lifeless> https://bugs.edge.launchpad.net/malone/+bug/604290 should you choose to take it on.
<_mup_> Bug #604290: reassigning bugs to different pillar types is hard? impossible? <Launchpad Bugs:New> <https://launchpad.net/bugs/604290>
 * wgrant is currently in the depths of Soyuz.
 * lifeless throws in a lifeline
 * mwhudson is in prague
<lifeless> mwhudson: I'm just grabbing my power adapter
<mwhudson> lifeless: then hotel bar?  how long have you been here?
<lifeless> mwhudson: yes, then hotel bar
<lifeless> since 2pmish
<mwhudson> lifeless: cool, i'm going to shower but see you there in a bit
<and471> with launchpadlib, I want to get the title of a bug, I use bug.title, however this gives me the number and the project it affects
<and471> how do I get just the title the user gave?
<and471> for example for this bug https://bugs.launchpad.net/ubuntu/+source/network-manager/+bug/498478
<_mup_> Bug #498478: network manager applet should use new libdbusmenu (indicator applet) <network-manager (Ubuntu):Triaged> <https://launchpad.net/bugs/498478>
<and471> the title would just be 'network manager applet should use new libdbusmenu (indicator applet)'
<cody-somerville> and471, are you sure you have a bug or a task?
<and471> cody-somerville, I obtained the bug/task from a collection obatined from assigned_bugs = me.searchTasks(assignee=me)
<and471> cody-somerville, is there a btter way?  I need to have a collection of the bugs a user is assigned to
<and471> *better
<cody-somerville> and471, that returns tasks, not bugs
<cody-somerville> and471, but calling .bug.title on the task will return what you're looking for
<and471> cody-somerville, thanks, I am trying to test but launchpad isn't responding atm :/
<and471> cody-somerville, in the mean time
<and471> cody-somerville, could you explain the difference between a bug and a task so I don't make this mistake again?
<cody-somerville> a task connects a bug to a product
<and471> cody-somerville, thanks, and thanks for your time and help
<cody-somerville> no problem
<and471> cody-somerville, just quick, is there a convenient property of a bug, that will give you the url a user would type into their browser (ie. https://bugs.launchpad.net/ubuntu/+source/network-manager/+bug/498478)
<_mup_> Bug #498478: network manager applet should use new libdbusmenu (indicator applet) <network-manager (Ubuntu):Triaged> <https://launchpad.net/bugs/498478>
<and471> cody-somerville, or should I just substitue the bug id into bugs.launchpad.net/bugs/ID?
<and471> *substitute
<cody-somerville> thats what I do
<and471> ok
#launchpad-dev 2011-07-04
<lifeless> StevenK: thanks
<StevenK> lifeless: It's landed?
<lifeless> StevenK: tossed it at ec2
<lifeless> StevenK: *just in case*
<StevenK> Pffft.
<StevenK> Fair enough. :-)
<wgrant> StevenK: 2x QA for you
<wgrant> Bah, qa-bad.
<wgrant> StevenK: https://code.qastaging.launchpad.net/ubuntu/+source/hotwire
<StevenK> Blink
<wgrant> Yes.
<wgrant> I presume it's your fault.
<wgrant> Not sure how, though.
<wgrant> Ah.
<StevenK> Ah?
<wgrant> You said date_last_updated in lib/lp/code/templates/distributionsourcepackage-branches-grouped.pt, instead of date_last_modified.
<wgrant> Can you fix/test?
<StevenK> Oh, bah
<StevenK> I suspect you mean s/\// and /
<StevenK> :-P
<wgrant> StevenK: Any progress on the qa-bad fix?
<StevenK> Distracted by housework/phone
<StevenK> And NFI how to initialise that template
<lifeless> https://code.launchpad.net/~lifeless/launchpad/storylayers/+merge/66738
<lifeless> StevenK: do you need a hand with that qa-bad fix?
<StevenK> Yes. I'm worried that the test suite didn't pick that up
<StevenK> But I should head out and buy lunch
<lifeless> ec2 is complaining: ImportError: No module named html5browser
<StevenK> I suspect Curtis pulled his changes back into the tree, but forgot to update ec2
<lifeless> StevenK: the paralleltests branch is in devel now
<wgrant> StevenK: He did update the image.
<wgrant> lifeless: Had you merged devel lately?
<lifeless> I was based off of revno: 13322 [merge]
<StevenK> 13365 is tip, or close to
<lifeless> is that relevant to ec2?
<StevenK> It could be -- since the latest ec2 image is wallyworlds
<StevenK> And he added himself last week
<wgrant> 13345 has that change.
<lifeless> ok
<wgrant> It's possible that Curtis removed his once it was superseded.
<lifeless> can has review ? https://code.launchpad.net/~lifeless/launchpad/storylayers/+merge/66738
 * StevenK kicks the OOPS that wgrant pointed out
<bigjools> morning
<lifeless> hai
 * bigjools cleans earwigs out of keyboard yet again
<bigjools> lifeless: so did you hear about our long poll success?
<lifeless> nope
<lifeless> but I did get snippets of info during the week
<bigjools> it's pretty damned awesome
<bigjools> got an MP diff to appear the instant the job finished
 * wgrant throws boulders at American Airlines.
<lifeless> bigjools: nice
<bigjools> evening wgrant
<wgrant> lifeless: It was even non-hacky by the end!
<lifeless> wgrant: bigjools: StevenK: can I have a review please? https://code.launchpad.net/~lifeless/launchpad/storylayers/+merge/66738
<lifeless> easy stuff
 * bigjools looks
<bigjools> lifeless: get rid of the XXX at line 73, the bug is fixed/invalid
<lifeless> bigjools: I'll reopen the bug
<lifeless> bigjools: the discussion is pretty clear in it that it was missing a reproduction recipe, which this offers :)
<bigjools> mkay
 * bigjools sighs at bug crawling between LCD monitor cover and pixels inside
<spm> bigjools: sounds like a tasty breakfast on the go for you?
<bigjools> spm: well said bug is about the size of 3 pixels
<spm> lite snack?
 * bigjools just took a video of it
<spm> UQ. 1990. Doing my year project on a Decstation. was distracted by a bug running across the screen. leapt back from the desk and started peering at the side of the monitor to see WTF the bugger went. Turned to the otehr guy in the office to mention my brief fright to see the bastard pissing hisself laughing. Some (ancient) variant of xroach. bastard. :-)
<lifeless> can I ask a favour?
<lifeless> can someone send https://code.launchpad.net/~lifeless/launchpad/storylayers/+merge/66738 to ec2; my ec2 tree is updatingm but its peak period - it will be another hour before it finishes :(
<StevenK> Done
<StevenK> lifeless: --no-qa?
<lifeless> StevenK: yeah, thats fine
<bigjools> spm: lol :)
<lifeless> StevenK: thanks!
<adeuring> good morning
<jtv> Any reviewers in the house?  Here's a light little branch to get you going again after the Thunderdome: https://code.launchpad.net/~jtv/launchpad/dsd-packageset-filter/+merge/66757
<bigjools> allenap: oh I meant to say, I have landed the messaging API
<allenap> bigjools: Tip top :)
<bigjools> check out services/messaging/
<bigjools> allenap: it even has tests!
<bigjools> and they pass!
<allenap> bigjools: Blimey, professionalism from the Launchpad team, who would have thought, eh? :)
<lifeless> bigjools: thats great; does it use a network fake ? [and do you do some failure injection ?]
<bigjools> lifeless: it uses the RabbitFixture
<bigjools> no failure injection but it does test txn aborting
<lifeless> ok, so lp is unaware of the async frontend
<lifeless> ?
<mrevell> Hello Launchpad!
<bigjools> morning mrevell
<bigjools> gmb or wgrant: what happened to the twisted plugin in lazr.amqp?
<bigjools> lifeless: sorry missed your message, yes LP is unaware of it
<bigjools> other than the /longpoll redirection for the XHR
<lifeless> cool
<lifeless> nicely done to everyone
<bigjools> lifeless: it was very gratifying
<bigjools> genuinely a team effort
<rvba> bigjools: One thing that is left harcoded on the js side is the location of the longpolling server. On my local instance./longpoll/ (this hits the twisted server via apache)
<lifeless> needs a +
<bigjools> rvba: we need to try and make that come from the page template I think
<bigjools> and what lifeless said
<lifeless> bigjools: gt5 is -very- shiny
<rvba> bigjools: it's a general configuration setting, don't you think it should be in configs/ ?
<bigjools> lifeless: are you talking about the excellent game on the PS3? :)
<lifeless> rvba: nooo
<lifeless> bigjools: I am
<lifeless> rvba: firstly configs is terribly hard to change
<lifeless> rvba: secondly, having it configurable is YAGNI for now
<lifeless> rvba: so IMO make it the simplest possible spelling you can. '/+longpoll'. Or as bigjools says perhaps supply it from the view (but still aim for drop-dead simple there)
<rvba> lifeless: ok.
<bigjools> wherever it goes, let's write it just *once * :)
<lifeless> +1
<rvba> +1 ;)
<wgrant> bigjools: It's not in the 0.1 egg, but I've fixed setup.py in trunk to include it.
<bigjools> wgrant: so where does it actually live?
<wgrant> bigjools: What do you mean? twisted/plugins/lazr_amqp.py?
<bigjools> wgrant: it's not there in the egg
<wgrant> 19:30:22 < wgrant> bigjools: It's not in the 0.1 egg, but I've fixed setup.py in trunk to include it.
 * bigjools sees a circular argument coming on
<wgrant> bigjools: Hm? I noticed it wasn't in the 0.1 egg, so I fixed setup.py so it's in future ones.
<jtv> Any reviewers for a simple soyuz fix?  bigjools perhaps?  https://code.launchpad.net/~jtv/launchpad/bug-802840/+merge/66775
<lifeless> night all
<jml> flacoste: hi
<flacoste> hi jml
<jml> flacoste: sometime soon this week, I'm going to start unwinding my Launchpad team memberships & mailing list subscriptions.
<jml> flacoste: my aim is to be configured as a relatively engaged community developer who has commit privileges but isn't getting mail all the time about everything.
<jml> flacoste: sound alright?
<flacoste> jml: sounds very good
<jml> flacoste: cool.
<stub> Can we all do that :)
<bigjools> +1 to no email :)
<cr3> why does ProductSet.getByName use IPillarNameSet which results in two database queries instead of just getting the product by name directly
<bigjools> because we have bugs
<cr3> hm, probably because the pillarname table encodes aliases whereas the product table doesn't
<jml> well, I just left a whole bunch of teams
<jml> I wonder what will happen now
<bigjools> world domination
<cr3> do alias names for pillars get wiped periodically? I'm looking a team or a project I asked to be renamed a long time ago so, if my understanding of the pillarname table is correct, I would expect launchpad to redirect me
<maxb> alias names are only added if the person doing the renaming chooses to do so
<jml> ok, apparently I was subscribed to all Launchpad bugs. Not any longer :)
<bigjools> adeuring: got a sec to talk about r13274?
<adeuring> bigjools: yes. sorry for the late response
<bigjools> adeuring: I'm close to leaving so I'll make it quick!
<bigjools> adeuring: you added order_by() to a result set before calling is_empty()
<adeuring> right
<bigjools> adeuring: I was wondering why that was necessary, since Storm removes ordering on is_empty
<adeuring> bigjools: interesting; I did not know thta
<bigjools> you reviewed matsubara's branch r13332 which uses is_empty() without the order_by() :-)
<bigjools> storm does "subselect.order_by = Undef"
<abentley> bac: What's the status of getnewcache?
<flacoste> hi lifeless
<lifeless> flacoste: hiya
<flacoste> lifeless: call time?
<lifeless> yupyup
<lifeless> flacoste: skype ?
<LPCIBot> Project devel build #860: FAILURE in 6 hr 2 min: https://lpci.wedontsleep.org/job/devel/860/
<lifeless> flacoste: http://paste.ubuntu.com/634628/
<lifeless> http://paste.ubuntu.com/634629/
<lifeless> flacoste: ^
<lifeless> flacoste: https://dev.launchpad.net/ArchitectureGuide/ServicesRequirements#Services are separate projects
<matsubara> lifeless, hey there, if you have some time today, could you review: https://code.launchpad.net/~matsubara/oops-tools/805707-update-db-script/+merge/66841 for me please?
<lifeless> matsubara: sure. btw any lp reviewer can do this ;)
<matsubara> lifeless, yep, the ones available now are in your timezone :-)
<matsubara> and I'm about to leave
<matsubara> thanks lifeless
<lifeless> np
#launchpad-dev 2011-07-05
<StevenK> Hm. How to fix the qa-tagger since I forgot rollback tags
<StevenK> (But it wasn't really a rollback)
<StevenK> Is it fixed-commit-<rev> or good-commit-<rev>?
<lifeless> you'll need to track it by hand
<lifeless> I don't think anyone has landed a tag based fixup into qa-tagger
<lifeless> there is a bug noting that this problem occurs
<StevenK> Bah
<StevenK> Last Modified: 2009-07-19 ago
<lifeless> ?
<StevenK> IE, it needs more fixing :-(
* lifeless changed the topic of #launchpad-dev to: Performance Tuesday | top - 14:12:21 up 2 min,  3 users,  load average: 0.50, 0.57, 0.25
<lifeless>  https://dev.launchpad.net/ | On call reviewer: - | Critical bugs: 204 - 0:[######=_]:256
<StevenK> Fail
* StevenK changed the topic of #launchpad-dev to: Performance Tuesday | https://dev.launchpad.net/ | On call reviewer: - | Critical bugs: 204 - 0:[######=_]:256
<lifeless> StevenK: oh, your branch? failed again ?
<StevenK> lifeless: It doesn't OOPS, it just looks wrong
<lifeless> StevenK: if so the  thing to do at this point is roll it all back
<lifeless> unblock the queue so you can spend as much time on it as you want
<StevenK> lifeless: I can fix it in the same amount of time
<lifeless> StevenK: its about 30 seconds to land a bulk rollback; you can't do a test run etc in that much time
<StevenK> lifeless: Even for 3 seperate revisions?
<lifeless> StevenK: ok, so maybe a couple of minutes
<lifeless> StevenK: do you know *exactly* what is wrong?
<StevenK> Yes
<lifeless> ok, well its your call. But we have considerable data now showing that rolling forward is usually harder than anyone thinks it will be :)
<StevenK> I will self-review, it's +2,-1
<StevenK> lifeless: I suppose this is what I get for doing the original fix last night while jetlagged.
<lifeless> \o/
<lifeless> probably
 * StevenK is *still* waiting for bzr push :-/
<StevenK> lifeless: So I should put a --rollback in this landing?
<lifeless> yes
<lifeless> multiple I guess :)
<StevenK> lp-land doesn't support multiple
<lifeless> do one
<lifeless> and then copy the [] bits for the other
<StevenK> lifeless: Is the parallel job on Jenkins safe to enable?
<lifeless> StevenK: only one way to find out
<lifeless> StevenK: we're not problem free, but knowing whether its happy or not is good
<StevenK> Re-enabled, build scheduled.
<StevenK> lifeless: Uh, your parallel run is barfing horribly
<lifeless> win
<StevenK> That must be some defintion of win I'm not previously aware of
<lifeless> its a form of sarcasm
<StevenK> Well, duh
<StevenK> Replying to sarcasm with deadpan humour on IRC so *works*
<lifeless> :)
<LPCIBot> Project parallel-test build #84: STILL FAILING in 1 hr 1 min: https://lpci.wedontsleep.org/job/parallel-test/84/
<LPCIBot> Project devel build #861: STILL FAILING in 5 hr 39 min: https://lpci.wedontsleep.org/job/devel/861/
<lifeless> flacoste: perhaps you should put yourself into https://launchpad.net/~launchpad-strategist for the moment?
<wgrant> Hmmm, rabbitfixture has stopped working for me.
<StevenK> lifeless: O hai -- do you know what oops.pt is used for?
<wgrant> StevenK: I've just QAd it.
<StevenK> Ah, excellent.
<wgrant> OK, crisis time.
<StevenK> How, out of interest?
<wgrant> Made +initseries OOPS.
<StevenK> Like that's hard. :-P
<lifeless> StevenK: its used to show OOPSes
<lifeless> StevenK: the change to make it use the base template is a little questionable, because it makes OOPS rendering more able to fail, but its probably ok.
 * StevenK sighs at how fragile window focus in Natty is.
<michaelh1> Hi there.  I'm geting a HTTP 500 Internal Server Error using launchpadlib when fetching the gcc-linaro project.  OOPS is  OOPS-2012M179
<StevenK> It's known.
<StevenK> We're working on it right now.
<michaelh1> Ta
<wgrant> michaelh1: Should be working now.
<michaelh1> wgrant: ta.
<wgrant> michaelh1: It turns out that 32-bit serial IDs on high-volume OAuth tables are not a really good idea.
<michaelh1> But 32 bits should be enough for anyone?
<michaelh1> :)
<lifeless> not with 4M inc() a day
<wgrant> lifeless: Still about?
<LPCIBot> Project db-devel build #696: FAILURE in 5 hr 34 min: https://lpci.wedontsleep.org/job/db-devel/696/
<adeuring> good morning
<LPCIBot> Project devel build #862: STILL FAILING in 5 hr 44 min: https://lpci.wedontsleep.org/job/devel/862/
<bigjools> Does the bug email header info differentiate between indirect and direct subscription?
<mrevell> Hello!
<bigjools> morning mrevell
<jtv> gmb: are you reviewing today?  Here's a simple one: https://code.launchpad.net/~jtv/launchpad/bug-798297/+merge/66816
<gmb> jtv: Okay; will look at it presently.
<jtv> great, thanks
<jtv> gmb: argh, I gave you the wrong one!
<jtv> bigjools:  picked that one up and forgot to mark it.
<gmb> Haven't looked yet, so that's okay.
<jtv> I meant this one: https://code.launchpad.net/~jtv/launchpad/bug-802840/+merge/66775
<bigjools> I claimed it!
<jtv> Oh, sorry.  Your pheromones must be weak, or it may have rained.
<bigjools> you need to learn to reload your own browser
<jtv> I've started to count on the frequent reboots to do that for me.
<jtv> It's like Ajax, only simpler.
<bigjools> that doesn't help
<bigjools> FF loads the page as it was!
<bigjools> when it restores, I mean
<jtv> cache ftw
* gmb changed the topic of #launchpad-dev to: Performance Tuesday | https://dev.launchpad.net/ | On call reviewer: gmb | Critical bugs: 204 - 0:[######=_]:256
<gmb> jtv: Lazy reviewer alert: Those tests could do with some leading comments. I know you've tried to make the test names meaningful and descriptive, but it still took me several seconds to parse them, and I'm fundamentally lazy.
<jtv> gmb: leading commentsâ¦ isn't that where the opposing attorney objects and the judge tells me to behave?
 * jtv edits
<gmb> jtv: Maybe, but I'm the one wielding the loving gavel of approval here...
<jtv> Crystal.
<lifeless> wgrant: hi
<lifeless> wgrant: jus tback from parenting class
<lifeless> though I think its 'make the class faint class' atm
<nigelb> hehe
<jtv> I do so hope this is leading up to a spanking for wgrant
 * bigjools reaches for brain bleach
<jtv> gmb: comments pushed
<gmb> jtv: Right. I've already approved it.
<jtv> Ta!
 * gmb reads backscroll.
<gmb> bigjools: When you're done with the brainbleach, I need to wash my mind's eye with it.
<bigjools> be my guest
<jtv> gmb: was my branch that bad?
<gmb> jtv: Your branch was a soothing balm. Your ability to conjure up mental images, though...
<jtv> I knew that, I just enjoy watching you bring yourself to say it.
<rvba> gmb: Hi ... and thanks for reviewing my branch!
<rvba> bigjools: I'll land it then (https://code.launchpad.net/~rvb/launchpad/bug-805547/+merge/66861)
<gmb> rvba: Hi, and no problem :). (I like reviewing the small branches first; it gives me the impetus to deal with the larger ones)
<bigjools> rvba: I think your test is wrong
<bigjools> the source is not in two parents
<bigjools> the problem was that sources for unrelated packagesets were being copied
<rvba> bigjools: self.setupParent creates a few packages.
<rvba> And then 'udev' is created in each parent.
<rvba> I then create two packagesets (one in each parent).
<rvba> test1.addSources('udev')
<rvba> test2.addSources('udev')
<rvba> This should add 'udev' to each packageset in each parent.
<rvba> bigjools: don't you think?
<bigjools> yeah, I think that is the wrong approach
<bigjools> the test should check that a package in a packageset for distroseries "A" is not copied when inheriting from distroseries "B"
<rvba> I think I understood the error you got differently then ...
<rvba> (and I might be wrong)
<bigjools> I think your fix is ok
<bigjools> btw, "if spns == []"
<bigjools> len(spns) == 0 ?
<rvba> I can change that If you prefer.
<rvba> *If you prefer it that way
<bigjools> it seems better style but what do I know :)_
<rvba> :)
<bigjools> just reading your test again
<rvba> A1 (packageset p1 with source s1) + A2 (packageset p2 with source s1 [same source name]) => init (packageset p1)
<rvba> This is what I'm testing.
<rvba> Until now, when we copied from the second parent, we decided to copy the source s1 also (because we did not very that p1 was in A2) ... and this is wrong.
<bigjools> the problem was that all sources in all packagesets were getting copied for every iteration through the parent series
<bigjools> rather than just the sources in the packagesets for the parent series being considered
<rvba> exactly.
<rvba> My test test_copying_packagesets_multiple_parents_same_source triggered the problem (before the fix).
<bigjools> you comment at the top of the test is confusing me a bit: "# If a source with the same packagename is included in two parents ..."
<bigjools> that's got nothing to do with the problem
<bigjools> that's good to hear at least :)
<rvba> I would not go this far ... but it's not very precise indeed. :)
<rvba> Let me rephrase that.
<bigjools> I would have expected a test to check whether a source in p2 is not copied when initialising from s1
<bigjools> does that make sense?
<rvba> Yes.
<bigjools> rvba: it's hard to see if that's being tested in your new test
<rvba> Well, the real test is that the initialisation happens.
<bigjools> that's a smoke test, not a unit test
<rvba> And that the correct package has been copied, proof that the the initialisation was successful ...
<rvba> Here is you unit test :)
<rvba> your*
<bigjools> well if it works then ok
<rvba> bigjools: but ok, I get your point, I'll refactor the test a bit, and the comment.
<bigjools> I am just saying that I was confused :)
<rvba> And this is sign that there is a problem with the test :) ... it should not be confusing
<bigjools> splitting it up into different sections with comments would help :)
<rvba> bigjools: ok
<allenap> gmb: Got one for you if you're interested. https://code.launchpad.net/~allenap/launchpad/lp-app-longpoll/+merge/66872
<gmb> allenap: Sure
<allenap> gmb: Thanks dude.
<gmb> allenap: Wherefore this:
<gmb> 98	+    #adapts(Interface, Interface)
<gmb> 99	+    #implements(ILongPollEvent)
<gmb> ?
<allenap> gmb: It's a hint for people who are writing a concrete subclass.
<gmb> allenap: Ah, cool. That's what I thought; just wanted to check.
<allenap> I'll add a comment to that effect.
<allenap> gmb: I have to go out now, unexpectedly, but I'll be back later. That okay?
<gmb> allenap: That's fine. It's looking good so far anyway.
<allenap> Thanks.
<jml> lifeless: I guess you didn't get around to looking at the testtools gassy-failures branch.
<jml> lifeless: I'll probably merge it tomorrow if I don't hear from you.
<lifeless> sorry I didn't.
<lifeless> kk
<jml> lifeless: no worries. I'm not blocked on it, just minimizing inventory.
<LPCIBot> Project db-devel build #697: STILL FAILING in 5 hr 36 min: https://lpci.wedontsleep.org/job/db-devel/697/
<StevenK> Is anyone in Europe/US doing a deployment request? r13371 is (finally) deployable.
<StevenK> Er, that should be "Does anyone in Europe/US feel like doing a deployment request?" :-/
<LPCIBot> Project devel build #863: STILL FAILING in 5 hr 55 min: https://lpci.wedontsleep.org/job/devel/863/
<deryck> Hi, all.
<flacoste> hi deryck, happy anniversary
<deryck> Thanks, flacoste!
<flacoste> bigjools: do copy rebuilds have an entry in *pph?
<bigjools> flacoste: yes
<flacoste> bigjools: and where is the 'upload' date recorded? would it be datecreated on spph?
<bigjools> flacoste: no, the original SPR
<bigjools> datecreated on the copy SPPH is when it was copied
<flacoste> so spph.sourcepackagerelease.dateuploaded?
<bigjools> yes, that's when the package was *first* uploaded
<flacoste> no, i'm interested in when the 'rebuild request' was issued
<flacoste> so that would be spph.datecreated, right?
<bigjools> flacoste: none of the above :)  Archive.date_created
<flacoste> interesting!
<flacoste> each rebuild have their own archive!
<flacoste> convenient
<bigjools> flacoste: the SPPH datecreated is nearly right but they are not all the same
<bigjools> yes, we can only rebuild in separate archives
<flacoste> ok, i think i have all i need to do some interesting data analysis
<flacoste> thanks
<bigjools> great
<bigjools> np
<gary_poster> deryck: quick good news: Friday night I hacked a bit and got the per-test runtime for the YUI/appserver tests down from 4 seconds to .7 seconds.  I think that's much more palatable.
<deryck> gary_poster, yes, that sounds nice!
<deryck> gary_poster, now that we're back home, are you finishing off that work?  Or flacoste?
<gary_poster> deryck, I was going to do it, but *very* happy to share if anyone else wants to take it. :-)
<gary_poster> but I was just about to work on that now
<deryck> gary_poster, ok, cool.  I'm happy to take any part of it that you can pass to me.
<gary_poster> deryck, cool.  How about I work on this till Wed; if there's stuff left to get it landed I pass it to you.  Sound ok?  (I *might* ask off for Thurs and Fri, so that would make even more sense in that scenario)
<deryck> gary_poster, sounds perfect to me.
<flacoste> bigjools: populate-archive creates the archive as disabled "to allow requestor to tweak build dependencies", how do they turn it enabled to allow builds to proceed?
<deryck> gary_poster, also, my squad will do interrupt duties this week, since we should have had them last week.
<bigjools> flacoste: in the +admin/+edit page for the archive
<gary_poster> awesome thanks deryck
<deryck> np
<bigjools> flacoste: we need to do some cleanup in this area to remove these old archives when they're done with as they take up a lot of DB and librarian space.
<flacoste> bigjools: no builds will be queued until they do that right? do we record the date they enabled the archive? or we can infer it from the date on the spph?
<flacoste> bigjools: the fact that they are all in the DB currently is very useful for my analysis :-), but i agree that it might not be "feature" we want to support
<deryck> abentley, since I came in late today, let's reschedule our call tomorrow or the next day.  Cool?
<bigjools> flacoste: builds are queued as soon as the source is copied, it's just that the buildd-manager ignores builds in disabled archives
<abentley> deryck: sure.
<deryck> abentley, thanks!
<bigjools> flacoste: not sure how we can get hold of date enabled.
<flacoste> bigjools: what about packagecopyrequest.date_started?
<bigjools> flacoste: that's pretty much the same thing as the archive.date_created
<bigjools> PCR is the job that sets up the (disabled) copy archive
<bigjools> flacoste: I am guessing that you want to see how long it takes to rebuild?
<flacoste> bigjools: yes, i'm trying to assess the impact of changing rebuild priority
<bigjools> flacoste: your best bet is to add up all the build durations on the builds themselves
<bigjools> I did something similar in the lag graph
<flacoste> bigjools: we don't throw away builds?
<bigjools> no
<bigjools> and if we did, build records are not the same as the build results :)
<flacoste> bigjools: is it just me or date_completed on packagecopyrequest is bogus?
<bigjools> flacoste: I don't know, are you bogus?
<flacoste> it depends ;-)
 * bigjools gets coat
<flacoste> avg(date_completed - date_started) = 01:04:36.777614
<bigjools> I've not looked at it in a while, so not sure
<bigjools> that seems reasonable if it's 1h?
<bigjools> the job takes a while to build the archive up
<flacoste> that would 1d 4
<bigjools> ah not 1d :)
<flacoste> err
<flacoste> no you are right 1:03
<flacoste> 1h
<flacoste> ok, so date_completed is when all the package records have been copied
<flacoste> not when the rebuild is finished
<bigjools> yep
<flacoste> so it was me :-)
<bigjools> also it creates builds in the requested arches
<flacoste> bigjools: all time?
<bigjools> flacoste: yeah
<bigjools> flacoste: pulseaudio is an utter crock of shit!
<flacoste> i'm sorry to hear that it's causing you so much trouble
<bigjools> skype is refusing to use my headset
<flacoste> even though it's selected in the sound control panel?
<flacoste> bigjools: do you want to try voip?
<bigjools> flacoste: if the next "fix" doesn't work then yes
<flacoste> bigjools: i'm at extension 7356
<bigjools> flacoste: that doesn't work either :/
<bigjools> why does this have to be so hard to get right ...
<flacoste> i wonder everytime...
<Beret> can you delete tags in LP?
<jml> no.
<Beret> thx
<jml> (also, more evidence for combining #launchpad & #launchpad-dev)
<nigelb> jml: why? :(
<nigelb> I quite like the distinction. When I'm working on LP per se I'd ask here, else ask there
<jml> nigelb: because people ask user questions in here, and because #launchpad is less tracked by developers than #launchpad-dev
<jml> nigelb: also, entia non sunt multiplicanda praeter necessitatem
<nigelb> jml: less-tracked. I thought CHR was supposed to solve that. merging the channel might cause chatter merging in with work
<jml> nigelb: meh, there's little harm in that. many IRC channels deal with it just fine
<nigelb> Since the chatter in launchpad is low enough, it might be a good idea
<nigelb> jml: You got the spelling right there. I salute you sir :)
 * nigelb had to use google translate
<jml> hah.
<jml> it used to be in the #divmod topic when I hung out there
* gmb changed the topic of #launchpad-dev to: Performance Tuesday | https://dev.launchpad.net/ | On call reviewer: - | Critical bugs: 204 - 0:[######=_]:256
<timrc> Using STAGING_SERVICE_ROOT for http://pastebin.ubuntu.com/638486/, if I go to the Adminster web-view for that PPA, I do not see ARM Processors checked under Enabled Restricted families -- am I doing something wrong?
<bigjools> timrc: your paste doesn't show which PPA
<bigjools> so I suspect yes you are doing something wrong :)
<timrc> bigjools, it's a private ppa, https://launchpad.net/~oem-archive/+archive/test1
<bigjools> timrc: I see the box ticked on staging
<timrc> bigjools, lol ;) that would be my problem
<timrc> bigjools, I was looking at the production web-view
<bigjools> timrc: yes :)
 * timrc hangs his head in shame as he walks to the kabob food stand
 * bigjools considers making setup an alias for setUp in testtools to save a wasted half hour
<jml> flacoste: hi
<flacoste> hi jml
<jml> flacoste: I sent an email instead.
 * jml has to go, sorry
<flacoste> np
<LPCIBot> Project db-devel build #698: STILL FAILING in 5 hr 37 min: https://lpci.wedontsleep.org/job/db-devel/698/
<bigjools> good night all
<dobey> why would launchpad not think a person is a member of a team that person is a member of?
<dobey> so Entry.__eq__ in launchpadlib seems to be broken
<dobey> hello?
<dobey> why would the http_etag be different, on the same person resource?
<dobey> one being a reference from launchpad.people[username] and the other being a reference got through membership listing on a team
<beuno> private team?
<dobey> nope
 * beuno gives up
<dobey> i don't think it's specific to the team
<dobey> i tried another team, and it has the same wrong etag :(
<LPCIBot> Project devel build #864: STILL FAILING in 5 hr 48 min: https://lpci.wedontsleep.org/job/devel/864/
 * dobey wonders where to file the bug at least
<dobey> i guess in launchpad itself as it seems like it is a server issue, ignoring the question of whether restfulclient should even use etag as a comparison inside __eq__
<lifeless> moin
<lifeless> dobey: file a bug please
<dobey> right, i was about to :)
<dobey> bug #806163
<_mup_> Bug #806163: Different http_etag for same resource? <Launchpad itself:New> < https://launchpad.net/bugs/806163 >
<lifeless> jcsackett: is bug 806179 affecting staging/qastaging/prod ?
<_mup_> Bug #806179: PersonPickers broken from bad passed in config <Launchpad itself:In Progress by jcsackett> < https://launchpad.net/bugs/806179 >
<jcsackett> lifeless: one sec, i'll check; i found it in .dev while reviewing an MP.
<jcsackett> a guess would say yes.
<lifeless> jcsackett: does it cause test failures?
<lifeless> jcsackett: if it doesn't, the bug should be critical, and the landing that brought it in marked qa-bad
<jcsackett> lifeless: it doesn't--it's actually a result of some bad piping that until recently there wasn't any good way to test. i believe some of the YUI/appserver stuff people worked on during the epic will fix that issue.
<lifeless> right, but we can't deploy with it broken :)
<lifeless> well, we *could*, but we shouldn't.
<jcsackett> lifeless: it doesn't cause test failures, rather. it does appear on qastaging.
<lifeless> ok
<lifeless> so, lets find the bad rev, mark *its* bug as qa-bad, and escalate this new bug.
<lifeless> do you need a hand with any of that ?
<jcsackett> lifeless: don't think i should.
<lifeless> coolio
<lifeless> I'll be back in 15 or so
<lifeless> if you change your mind :)
<lifeless> allenap: btw, why can't rabbit in dev mode still use an ephemeral port ?
<allenap> lifeless: I was following the lead of other services... which I assumed didn't have ephemeral ports.
<lifeless> allenap: they don't, but its a bug :)
<lifeless> allenap: or at least, I think it is: you can't run up more than one instance in dev mode, stale processes make things break, sadness all around
<allenap> lifeless: Right. I guess the problem is going to be telling processes outside of `make run` (or whatever) about which port its running on. You've done something with that in RabbitMQLayer, so I could do the same.
<allenap> s/its/it's/
<lifeless> allenap: yeah. For instance, have the txlongpoll take a host:port on its command line for the rabbit to talk to
<jcsackett> could someone take a look athttps://code.launchpad.net/~jcsackett/launchpad/button-configs-break-pickers/+merge/66946 for me? it's a short one.
<allenap> lifeless: It does, afaik, but I'm thinking of things like running jobs that want to use Rabbit. How do I get the configuration to them? (Use the RabbitMQLayer stuff?)
<lifeless> allenap: export a LP_CONFIG
<jcsackett> lifeless: only question i do have is what we do with the qa-bad bug once the fixing branch lands--mark both bugs as qa-ok? (assuming the fix works out).
<lifeless> jcsackett: the fixing branch should be landed with --rollback=12345
<lifeless> jcsackett: it doesn't matter if you're rolling forward or back, thats the option to use.
<allenap> lifeless: Okay, I don't really get how that solves the problem.
<jcsackett> lifeless: ah. i'll keep that in mind. didn't realize that wasn't actually rollback specific.
<lifeless> jcsackett: the bad-commit-12345 marker must *stay* on the bug
<lifeless> jcsackett: that tells qa tagger to lock the rev 12345 and the rollback-of-12345 together
<jcsackett> lifeless: ah, so i should put in a bad-commit and a qa-bad? or does the bad-commit get added automatically with rollback?
<lifeless> jcsackett: the qa-bad becomes qa-good once we're all sorted.
<lifeless> bad-commit-12345 is added by hand
<jcsackett> dig.
<lifeless> allenap: oh, by creating an ephemeral config on disk, inheriting from 'development', when you export LP_CONFIG=..., all the worker processes etc will read from it
<lifeless> allenap: this is what BaseLayer does, for instance.
<allenap> lifeless: Ah, okay. However, what do we do when someone spools up rabbit using `make run` then runs a job via cronscripts/run_jobs.py? Do we have to set LP_CONFIG manually then?
<allenap> There is some convenience to having the librarian on a known port.
<lifeless> allenap: I'd report back the config to use
<lifeless> uhm
<lifeless> whats the convenience ?
<lifeless> allenap: anyhow, its a thought
<lifeless> theres obviously a -big- stack of things that would need to be migrated, but I don't think everything needs to change at once.
<lifeless> stub: hola!
<stub> YO
 * stub turns down the volume
<stub> yo
<allenap> lifeless: Okay, I'll try with an ephemeral port, but as this is only a developer convenience (afaict) it might end up being less convenient.
<allenap> On that note, night all.
<LPCIBot> Project db-devel build #699: STILL FAILING in 5 hr 34 min: https://lpci.wedontsleep.org/job/db-devel/699/
#launchpad-dev 2011-07-06
<wgrant> jcsackett: Around?
<wgrant> lifeless: Hi.
<lifeless> hola
<wgrant> lifeless: Given the utterly disastrous state of QA over the past few days, I am tempted to revert the bad picker fixes.
<lifeless> did jons branch fail ?
<lifeless> I agree we despearately need to get stable
<StevenK> Jon's branch hasn't even been reviewed
<wgrant> Jon's branch is not reviewed, not ec2'd, and not necessarily going to work, and we've already got three rollbacks in the queue.
<StevenK> And 3 more revs to QA
<wgrant> A fourth to get us back to a reasonable state cannot hurt.
 * StevenK finds his rope.
<wgrant> And it's not even easy QA.
<lifeless> I like rollbacks :)
<wgrant> Ooh, it still reverts cleanly.
 * wgrant reverts cleanly.
<StevenK> wgrant: Are you going to self-review it?
<wgrant> Yes.
<wgrant> It is done.
<lifeless> wgrant: hi; rabbit
<lifeless> wgrant: did you have some success ?
<wgrant> Not really.
<lifeless> with it in vms specifically
<lifeless> k
<wgrant> I suspect I just need to poke /etc/hosts harder.
<lifeless> I'm really disappointed with rabbit ease-of-use in this area
<wgrant> Indeed. It's entirely unnecessary, too.
<wgrant> Yay, create-lxc-aufs still works.
<wgrant> OK, got it working.
<wgrant> 127.0.0.1 localhost lucid-lp-temp-aufs-7YHq
<wgrant> Doesn't work
<wgrant> 127.0.0.1 localhost
<wgrant> 127.0.1.1 lucid-lp-temp-aufs-7YHq
<wgrant> Does
<lifeless> butwhy
<wgrant> Because it is hideous and likes to resolve its name both ways :)
<timrc> wgrant, hey, cody-somerville said you rang
<wgrant> It's like kerberos, except worse.
<lifeless> wgrant: what do you mean?
<wgrant> Hmm, I wonder if it's our fixture, actually.
<lifeless> well, u1 had the same ugly
<lifeless> and we got our fixture's core from u1
<lifeless> but I'd like to understand what you mean
<lifeless> does it manually parse /etc/hosts ?
<lifeless> or is it binding to 127.0.1.1 for some bizarre reason?
<wgrant> NAFAIK
<wgrant> Need to experiment.
<lifeless> my lxc doesn't have 127.0.1.1 in 'ip addr'
<wgrant> It's not going to.
<wgrant> It's 127.0.0.0/8
<lifeless>     inet 127.0.0.1/8 scope host lo
<wgrant> Well, that.
<lifeless> the /8 only affects routing
<wgrant> Yes...
<wgrant> And it will resolve your hostname.
<lifeless> 127.0.1.1 isn't usable unless the address is added somewhere
<wgrant> Yes it is.
<lifeless> or something daft is going on
<wgrant> ping 127.99.99.99
<wgrant> 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN
<wgrant>     link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
<wgrant>     inet 127.0.0.1/8 scope host lo
<wgrant>     inet6 ::1/128 scope host
<wgrant>        valid_lft forever preferred_lft forever
<wgrant> 127.0.0.1/8 is on the interface.
<wgrant> Hmm.
<wgrant> Anyway.
<wgrant> lo takes the whole /8
<wgrant> Always has.
<wgrant> And rabbit or the fixture somehow need to be able to resolve it both ways.
<lifeless> hah
<lifeless> searching for lo address gets a cee lo news clip ;P
<wgrant> Heh
<lifeless> I'm not sure lo always bound to all the the 127 addresses, but meh
<lifeless> wgrant: whats odd/confusing is why it *wants* 127.0.1.1
<wgrant> 127.0.1.1 has been in the default installation forever.
<lifeless> wgrant: e.g. why its not happy with 127.0.0.1
<lifeless> wgrant: no, it hasn't
<wgrant> lifeless: 127.0.0.1 reverse resolves to localhost first.
<lifeless> wgrant: feisty/etch brought it in.
<lifeless> http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=316099
<lifeless> ah, so its whinging because it gets a different name back ?
<wgrant> I presume so.
<wgrant> It's possibly the fixture being crap.
<StevenK> *However*, Feisty only added 127.0.1.1 on new installs
<wgrant> Not even 12000 OOPSes from yesterday's amusement.
<wgrant> How boring.
<LPCIBot> Project devel build #865: STILL FAILING in 5 hr 50 min: https://lpci.wedontsleep.org/job/devel/865/
<mwhudson> totally irrelevant, but apparently the 127.0.0.1/8 thing is different on os x
<wgrant> lifeless: Hi.
<wgrant> lifeless: The problem goes away if we pass fq_nodename instead of just nodename in as RABBITMQ_NODENAME.
<wgrant> lifeless: rabbitmq will use nodename@hostname if it's specified, otherwise it will do a reverse lookup to guess hostname.
<StevenK> So it's a very small patch?
<wgrant> That's how the system one works: /usr/lib/rabbitmq/bin/rabbitmq-env constructs rabbit@hostname
<wgrant> Yes.
<wgrant> Three characters.
<wgrant> I think rabbitfixture also leaks tmpdirs.
<wgrant> Need to check that out.
<StevenK> Heh
<wgrant> Like, my laptop had several thousand by the end of last week.
<poolie> hi StevenK, wgrant, lifeless
<StevenK> O hai!
<wgrant> Hi poolie.
<wgrant> Mm, doesn't seem to leak normally.
<wgrant> I guess I just killed it too aggressively.
<StevenK> A thousand times or more
<lifeless> poolie: hi
<lifeless> wgrant: cool
<StevenK> Orsum. bzrlib.lockable_files._LockWarner is now affecting ~1,500 tests on Jenkins
<spiv> Upgrade your bzrlib :P
<StevenK> spiv: That's with bzr 2.3.3-0~bazaar1~lucid2
<spiv> StevenK: The relevant change landed on trunk about a week ago
<lifeless> I think I'm blind
<lifeless> spiv: we run releases in lp though
<StevenK> I don't think I want to run LP tests against bzr trunk :-)
<lifeless> spiv: so I think 'release 2.4 please'
<lifeless> spiv: is a prerequisite
<wgrant> We often run betas.
<lifeless> the more bzrlib changes things the less I think that that is a good idea
<lifeless> what I mean is that the old one-line-of-releases, each month is as stable as possible, deprecations are always graceful : well that was more amenable to using recent developments.
<lifeless> its now harder to upgrade bzrlibs
<lifeless> and we can't be as confident about e.g. wire level support issues till the final release in the series
<lifeless> I'm not saying this is a problem per se
<lifeless> but its a consequence of the changed policies
<lifeless> zope.testing tests make my eyes bleed
<wgrant> What are you doing to it?
<lifeless> see my last comment in https://code.launchpad.net/~lifeless/launchpad/storylayers/+merge/66738
<lifeless> I am teaching it that if it gets a test with no layer, it should not do layer per-test teardown and setup, regardless of whatever layers are setup.
<lifeless> its that
<wgrant> Hah
<lifeless> or I can insert an adapting TestResult
<lifeless> between the DocFileTests in a StoryPageTestCase and the zope.testing runner
<lifeless> but the adapter would need to merge all the failures in all the elements of the story
<lifeless> I think this is probably a lesser evil
<lifeless> anyhow, we have 6 failures in our fork already
<lifeless> and no idea if they are bitrot, changes, or what have you
<lifeless> so I'm going to close my eyes and JFDI
<benonsoftware> Question: What exactly is the Launchpad Community dev team? https://edge.launchpad.net/~launchpad-dev
<lifeless> its a mailing list
<benonsoftware> Just a list?
<lifeless> you are welcome to join if you want to discuss the development of launchpad
<benonsoftware> I'm alreaady a member on it.
<poolie> lifeless, it would be nice if there was some clearer "what is this team for" concept
<poolie> some of them you're welcome to join if you're just interested; some are only for trusted peolpe
<poolie> perhaps they shouldn't all be presented as teams
<benonsoftware> if it just a list they should just have a list section
<poolie> there is a section about it being a list
<poolie> biab
<benonsoftware> What does biab mean?
<lifeless> 'Back In A Bit'
<StevenK> spiv: Out of interest, can you link me the fix for that?
<spiv> StevenK: http://bazaar.launchpad.net/~bzr-pqm/bzr/bzr.dev/revision/6002
<StevenK> spiv: That sounds like a good plan.
<spiv> Yep.
<wgrant> Probably makes the GC a bit happier.
<spiv> Hmm, probably not so much.  I doubt our lock objects were participating in reference cycles.
<lifeless> I suspect we have bugs causing us to see the warnings
<spiv> Probably.
<lifeless> ok, so this route isn't going to work
<lifeless> I'll do a big-arse adapter instead.
<spiv> To adapt it to a smaller arse?
<lifeless> if only
<lifeless> this is z.testing
 * StevenK attempts to QA stuff
<StevenK> wgrant: Can haz blessing to update DF?
<wgrant> StevenK: doit
<StevenK> wgrant: I think bug 740208 shouldn't be to bad to QA
<StevenK> It's not very clear from the code if it's an e-mail that's in a title or something
<wgrant> I checked it for descriptions earlier.
<wgrant> It seems to be OK.
<wgrant> But eeeh.
<StevenK> Heh
<lifeless> biab
<StevenK> wgrant: So it looks fine, you're just nervous about it?
<wgrant> StevenK: Yes.
<wgrant> Also considering tracking down whoever decided that erlang extensions should automatically download their dependencies from git/hg.
<wgrant> blink
<wgrant> My eyes
<StevenK> Haha
<StevenK> wgrant: Do you think it's fine to deploy if we keep on an eye on it, or not?
<wgrant> StevenK: I think it's OK.
<wgrant> git clone http://github.com/mochi/mochiweb.git
<wgrant> Check out the 'rebar' file.
<wgrant> rebar seems to be some kind of extension build system.
<wgrant> But look at what the file is.
<StevenK> I'd rather not touch a git repo :-P
<StevenK> wgrant: And we can probably deploy jtv's change, since it's FF'd
<StevenK> rvba's change is running glaciall^Won mawson
<mwhudson> wgrant: it looks lovely in github's browser, that file
<StevenK> mwhudson: URL?
<LPCIBot> Project db-devel build #700: STILL FAILING in 5 hr 40 min: https://lpci.wedontsleep.org/job/db-devel/700/
<mwhudson> StevenK: https://github.com/mochi/mochiweb/blob/master/rebar
<mwhudson> 'blob' has rarely been more appropriate
<StevenK> Right. Now to review my lunch.
<ajmitch> that blob looks like a .zip file
<wgrant> It is.
<mwhudson> it's something erlang related isn't it?
<wgrant> What else...
<StevenK> wgrant: So, the IDSJ runner just finished on mawson. It looks like it got past the problematic bit, so I'll qa-ok it, but there's another bug there.
 * StevenK files it
<wgrant> StevenK: k
<StevenK> Firefox, how you annoy me
<StevenK> Who broke click handling between Firefox 4 and 5, seriously?
<StevenK> We are probably deployable in 2 hours
<wgrant> zomg
<wgrant> Almost there.
<wgrant> Bah, bzr-hg fail.
<StevenK> Hm?
<lifeless> qa?
<StevenK> Ha
<StevenK> When r13376 hits qas, we can deploy 42 revisions
<wgrant> bzr-hg fails to work with rabbitmq-mochiweb.
<wgrant> It works with the rest :(
<poolie> file a bug, please?
<poolie> or, find one?
<poolie> which branch?
<wgrant> http://hg.rabbitmq.com/rabbitmq-mochiweb/
<wgrant> Can't branch it directly.
<wgrant> But if I try to branch a clone, it does a bit and then:
<wgrant> bzr: ERROR: An inconsistent delta was supplied involving '<unknown>', 'hg:test'
<wgrant> reason: Parent is not present in resulting inventory.
<StevenK> Right, buildbot done, waiting for qas
<StevenK> wgrant: I have prepared a bug list, too
<wgrant> Excellent.
<StevenK> 42 revisions, 28 bugs
<wgrant> Less excellent.
<poolie> wgrant, this rings a bell
<poolie> wgrant, https://bugs.launchpad.net/bzr-hg/+bug/513368
<_mup_> Bug #513368: inconsistent delta during fetch <code-import> <Bazaar Hg Plugin:Triaged> <Launchpad itself:Triaged> < https://launchpad.net/bugs/513368 >
<wgrant> poolie: Thanks.
 * StevenK whimpers at qastaging
<StevenK> How did you miss 13376?! HOW?
<wgrant> buildbot-poller being slow, maybe?
* lifeless changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: - | Critical bugs: 204 - 0:[######=_]:256
<StevenK> wgrant: But surely it pulls from stable?
<StevenK> But, perhaps
<lifeless> it does
<lifeless> but perhaps you've forgotten the 30 minute build of the tree on carob
<StevenK> Why does stable need to build on carob?
<StevenK> Since the first thing qas is run buildouyt
<StevenK> *buildout
<wgrant> StevenK: buildbot-poller pushes stable.
<wgrant> StevenK: So buildbot completing is not enough to update stable.
<wgrant> You have to hope that buildbot-poller sees it finish in a timely manner.
<StevenK> Right, so moving to Jenkins might actually solve this -- since my plan is that when devel builds successfully it will push to stable/db-devel as the last step
<wgrant> Jenkins shouldn't have permission to do that, I don't think.
<wgrant> But maybe.
<StevenK> Well, Jenkins tell tarmac to do it
<StevenK> Handwave
<StevenK> Utterly kill buildbot-poller
<wgrant> 3/4 deps packaged...
<wgrant> All of which originally grabbed $VCS checkouts of all of their dependencies
<wgrant> WTF
<LPCIBot> Project devel build #866: STILL FAILING in 5 hr 52 min: https://lpci.wedontsleep.org/job/devel/866/
<wgrant> Ahah
<wgrant> Packaged rabbitmq-management stack working.
<poolie> nice
<poolie> good night all
<wgrant> Night.
<bigjools> morning
<wgrant> Morning bigjools.
<bigjools> wgrant: did you see the QueueInconsistentStateError oops?
<wgrant> bigjools: Yes, I identified the bug and Steve got the PU rejected.
<bigjools> wgrant: good.  is it a new bug?
<bigjools> looked like it was PPA though
<wgrant> bigjools: Yes. OEM copied a package into a new PPA before it was private (the setup script hadn't called lp_save()), which triggered a delayed copy. It was then rerun with lp_save() added, so it was made private and recopied directly.
<wgrant> The delayed copy processed a couple of minutes later, but the direct copy had already been done.
<bigjools> awesome :/
<wgrant> Yay
<bigjools> the story is ....
<bigjools> delayed copies need to die
<wgrant> Yes
<wgrant> In other news, I've packaged rabbitmq-management. It's a plugin that provides an HTTP API which will allow the test suite to quickly reset rabbit.
<wgrant> So we can have test isolation.
<wgrant> Which will be nice.
<bigjools> splendid
<allenap> Morning all.
<bigjools> o/
<mrevell> Hi
<rvba> lifeless: Hi, I modified the code you rejected earlier today to throw a new exception with an additional message instead of using "print" (https://code.launchpad.net/~rvb/launchpad/rabbit-error-plugins/+merge/67000) ... if you think it's still clunky, I'll just forget about it but allenap and I though it would be nice to get a warning about potentially conflicting rabbit plugins.
<adeuring> good morning
<lifeless> rvba: I think there is a better way to tackle this; I will describe it in the proposal
<wgrant> rvba: We're going to depend on rabbitmq-management soon, so I'll be working out how to run multiple instances tomorrow, hopefully.
<wgrant> But there may be others.
<rvba> lifeless: ok thanks.
<StevenK> rvba: O hai. I QA'd your change, but hit https://bugs.launchpad.net/launchpad/+bug/806307
<_mup_> Bug #806307: IDS has no access for DistributionSourcePackage <derivation> <Launchpad itself:Triaged> < https://launchpad.net/bugs/806307 >
<rvba> wgrant: I suppose the 'conflict' is caused by multiple rabbit instances trying to use the very same port.
<wgrant> rvba: Right, I need to work out how to configure that.
<wgrant> Not sure if we can do it through an envvar like with the AMQP listener.
<wgrant> rvba: Are you running natty?
<rvba> wgrant: yes.
<wgrant> Not sure if I want to backport natty's rabbitmq 2.3.1 to everywhere, or take 2.5.0 from oneiric.
<wgrant> 2.5.0 has some major plugin build system changes.
<wgrant> 2.3.1 has everything we need.
<lifeless> rvba: done
<lifeless> wgrant: think CATS
<lifeless> s/S//
<wgrant> I forget how to check what's there.
<wgrant> Is there a madison around somewhere?
<lifeless> presumably apt-cache search on carob
<wgrant> Or must I FTP internally?
<wgrant> carob only sees stock lucid rabbitmq-server.
<rvba> lifeless: thanks ... I have used getDetails too ;) ... I guess I'll wait to see what wgrant does about this problem, if he manages to fix this in a generic fashion, the additional warning will be useless.
<wgrant> Perhaps it's in some other pocket.
 * StevenK kills parallel-test and disables the job again
<StevenK> Running for 26 hours makes me SAD
<lifeless> StevenK: sob, thanks.
<LPCIBot> Project parallel-test build #85: ABORTED in 1 day 2 hr: https://lpci.wedontsleep.org/job/parallel-test/85/
<lifeless> StevenK: any evidence of how it failed/
<rvba> StevenK: it's a good sign that the initialisation started. Thanks for QAing my change.
<LPCIBot> Project parallel-test build #86: ABORTED in 29 sec: https://lpci.wedontsleep.org/job/parallel-test/86/
<LPCIBot> * Launchpad Patch Queue Manager: [r=wgrant][rollback=13334] Revert r13334 due to bug #806179.
<LPCIBot> * Launchpad Patch Queue Manager: [r=gmb][no-qa] New lp.app.longpoll package.
<LPCIBot> * Launchpad Patch Queue Manager: [r=gmb][bug=805547] Fix packageset's copy in initialise_distroseries.
<_mup_> Bug #806179: PersonPickers broken from bad passed in config <Launchpad itself:In Progress by jcsackett> < https://launchpad.net/bugs/806179 >
<LPCIBot> * Launchpad Patch Queue Manager: [r=deryck][bug=740208] obfuscated email addresses for anonymous
<LPCIBot> webservice requests
<LPCIBot> * Launchpad Patch Queue Manager: [r=gmb][bug=802840] Ignore None component/section on sync upload
<LPCIBot> source override.
<LPCIBot> * Launchpad Patch Queue Manager: [r=stevenk][rollback=13356] Rollback revision 13356.
<StevenK> Bah, sorry.
 * StevenK smacks LPCIBot.
<wgrant> lifeless: Possibly interesting is that one of my parallel LXC workers hangs most times.
<wgrant> lifeless: Only one.
<lifeless> wgrant: fun
<wgrant> lifeless: And the whole suite passes if not run with --parallel
<lifeless> wgrant: you're doing aufs isolation right ?
<wgrant> lifeless: This is with aufs, yes.
<lifeless> wgrant: that jenkins job isn't, so its much more fragile
<wgrant> Sure.
<wgrant> rvba: You installed rabbitmq-management by grabbing the .ez files from the website and installing them manually?
<rvba> wgrant: Yes.
<rvba> Installing=moving the files to /usr/lib/rabbitmq/lib/rabbitmq_server-version/plugins
<wgrant> Yeah.
<wgrant> rvba: How is your JS branch going?
<rvba> wgrant: the generic part is up for review. https://code.launchpad.net/~rvb/launchpad/lp-app-longpoll-js/+merge/66936
<wgrant> Ah, excellent.
<rvba> The remaining parts (that I shall address after the DD work is finished) are:
<rvba> - fix up a service for the async frontend
<rvba> - use the generic stuff in the MP page (this will be just cleaning up the prototype that we've done in Dublin)
<wgrant> lifeless: Only 2.2.0 is in lucid-cat-landscape. That's just new enough that I can probably get this plugin stack backported to it. But natty's 2.3.1 backports to lucid fine without even a rebuild, and I'm not sure if 2.2.0 will work for us.
<wgrant> I guess I should test.
<jtv> Anyone mind if I kick dogfood?  bigjools, StevenK, wgrant?
<wgrant> Not I
<bigjools> jtv: are you updating it?
<jtv> Yes
<bigjools> cool, was gonna do that anyway
<StevenK> I updated it a few hours ago
<StevenK> So you may not need to
<bigjools> jtv: remember which user you're supposed to use ;)
<jtv> I made doing it right easier than doing it wrong, remember?
<lifeless> jtv: hi
<jtv> hi lifeless â saw your note, thanks for that.
<lifeless> jtv: I'd love to know what I missed
<lifeless> jtv: (I'm assuming you tried that and it didn't work...)
<jtv> Oh, no.  My query was just a quick scribble on a late-night train.  :)
<lifeless> I have an observation about postgresql - if there is an ORDER BY matching an index, pg will almost exclusively use it
<jtv> Which makes a lot of sense.
<lifeless> so if you order by id with a date constraint, it will use the primary key index
<jtv> Yes, I was simply too focused on getting to the id side of things when I wrote that version.
<lifeless> jtv: I think the heuristic is too strong, but I haven't had the time to dig into what/why is going on
<jtv> Well... it can save you some very painful plan nodes such as hash joins.
<lifeless> jtv: anyhow, it means that I now have a spinal reflex: to force an index, order by it
<LPCIBot> Project db-devel build #701: STILL FAILING in 5 hr 41 min: https://lpci.wedontsleep.org/job/db-devel/701/
<jtv> lifeless: I think I just remembered why I wrote that query the way I didâ¦ it performs reasonably for reasonable inputs even without the new index.  The simpler version is abysmal without the index no matter what the input is.  I think I'll land it my way in devel and then do it your way in db-devel after adding the index.
<jtv> No, that wasn't it.  :(
<jtv> It's been said so many times: if only the statistics would recognize monotonic columns.  :(
<bigjools> jtv: fancy doing a review?
<jtv> bigjools: if it's not too big
<bigjools> ~280 lines
<bigjools> but a lot of refactoring
<bigjools> https://code.launchpad.net/~julian-edwards/launchpad/set-previous-series-bug-805913/+merge/66942
 * jtv has at it
<bigjools> jtv: the previous_series stuff that I mentioned earlier
<bigjools> ta
<jtv> ah yes
<lifeless> jtv: we can add the index live
<jtv> oh that'd be great
<jtv> do we have a Procedure for that yet?
<lifeless> jtv: land it on devel with a non -0 patch number
<bigjools> lifeless: slony copes with index changes now?
<lifeless> normal review (stub + me for db, launchpad-reviewers for the code changes), and stub or a losa can create the index live at any point
<jtv> So the -0 patches go out as db-devel rollouts?
<lifeless> yeah
<spiv> danilos: I see https://code.launchpad.net/~danilo/launchpad/expander-anim isn't proposed for merging yet... what's the delay?  https://code.launchpad.net/~spiv/launchpad/bmp-inline-diffs/+merge/66634 depends on it.
<lifeless> so the plan is to make -0 verboten
<spiv> danilos: I'd like to get my shiny diffs landed :)
<lifeless> jtv: https://dev.launchpad.net/Database/LivePatching
<danilos> spiv, I'll be getting it landed today, some tests missing still
<jtv> lifeless: ta
<lifeless> bigjools: slony doesn't see the index
<spiv> danilos: woo!
<lifeless> bigjools: what we do is CREATE INDEX CONCURRENTLY on all the replicates
<lifeless> s/tes/s/
<spiv> Cool, I'll nag people to review mine tomorrow then.
<bigjools> ah ok
<lifeless> master first if its an index that can fail (e.g. UNIQUE), then the readonly replicas
<stub> bigjools: Slony doesn't care about indexes. it does care about constraints, which are sometimes tied up with the indexes.
 * lifeless hands over to stub
<lifeless> I get up in 6 hours :(
<bigjools> furry muff
<lifeless> jtv: anyhow, moral of the story is the index can be made live right now if needed.
<jtv> great news
<lifeless> halt()
<bigjools> lifeless: night, see you at 6am :)
<bigjools> unless you meant move the meeting TODAY?
<lifeless> bigjools: that would be wonderful but I left the suggestion too late I think :)
 * lifeless puts hand to forehead, palm out
<danilos> spiv, also, fwiw, you can land your branch without depending on mine, and mine will seamlessly provide nice animations for you when it lands :)
<spiv> danilos: well, mine merged from yours, so I can't easily land it without landing yours too :)
<LPCIBot> Project devel build #867: STILL FAILING in 5 hr 47 min: https://lpci.wedontsleep.org/job/devel/867/
<deryck> Morning, all.
<abentley> deryck: Morning.
<bigjools> +N  dist/rabbitfixture-0.2.1.tar.gz
<bigjools> +N  dist/rabbitfixture-0.3.tar.gz
<bigjools> :(
<jcsackett> morning, all.
<jcsackett> wgrant: you still up, by any chance?
<deryck> Morning, jcsackett
<jtv> bigjools: some problems with your branch I'm afraid.
<jtv> See MP.
<bigjools> jtv: music to my shell-like
<jtv> sorry
<jtv> Not tufted?
<bigjools> jtv: I don't understand your comment
<jtv> Archer ref.  Think ocelot.
<wgrant> jcsackett: Hi
<bigjools> jtv: not that one, the one in the MP
<wgrant> allenap: Do you have any more changes to rabbitfixture planned?
<jtv> bigjools: which comment?  I've been a little distracted by something or other wrong with my innards, so I may be talking nonsense.
<jcsackett> wgrant: hi. i see you pinged me last night--was it just about the rollbacks, or something else?
<bigjools> jtv: "you seem to test for the presence of a release date as an implicit side effect of SQL comparison semantics for nulls"
<allenap> wgrant: No, none yet. What have you got planned?
<bigjools> jtv: also you didn't read my blurb properly
<jtv> bigjools: oh, that one.  Well, you seem to rely on "DistroSeries.date_released < foo" to filter out DistroSeries whose date_released is null.
<jtv> Ah.
<wgrant> jcsackett: Just the rollback.
<bigjools> else you'd not have made the second comment :)
<jcsackett> wgrant: dig.
<wgrant> allenap: A trivial change to make it more robust against suboptimal /etc/hosts configs, and a more involved and as-yet unwritten one to use rabbitmq-management for cheap reset.
<bigjools> jtv: self.context.series[0] is correct, see the "addendum"
<allenap> wgrant: Tip top.
<bigjools> jtv: ok so I need an extra "Not(DistroSeries.datereleased is None)" clause?
<wgrant> != None!
<bigjools> whatever
<wgrant> Storm can't override is.
<wgrant> So that will, er, not do what you want.
<bigjools> I hate Storm
<jtv> In Storm you say != None.
<bigjools> I am going to start writing store.execute everywhere
<bigjools> sigh
<al-maisan> bigjools: :)
<bigjools> al-maisan: yes, something you can appreciate :)
<al-maisan> well, I had moments when I thought the same :)
<bigjools> hehe, I don't hate it really, but there are some weird bits
<al-maisan> like with every tool in existence
<deryck> abentley, ping for new standup time.
<bigjools> jtv: http://pastebin.ubuntu.com/638897/
<matsubara> anyone that could help me out with a test failure I got (https://pastebin.canonical.com/49400/) for this branch: https://code.launchpad.net/~matsubara/launchpad/39605-bugtask-tooltip/+merge/66928?
<matsubara> I'm not sure how to keep the query count under the limit
<stub> matsubara: I'm seeing a number of Person rows being retrieved one at a time. Pulling them in a batch will shave off some queries.
<stub> matsubara: Similarly BugActivity, Milestone and BugWatch
<matsubara> stub, hmm this is probably the for loop iterating on activities.order_by(Desc('datechanged')), right?
<stub> Maybe some code traversing attributes causing late loading of database objects? Pull them into the cache earlier.
<stub> matsubara: I don't know. I just skimmed the SQL log looking for repeated stuff.
<matsubara> stub, how can I pull everything I need on a single query?
<jtv> stub o/
<stub> person_ids = [activity.personID for activity in activities if activity.target == targetname]
<stub> store.find(Person, Person.id.is_in(person_ids)
<stub> matsubara: That might do it
<stub> jtv: o/
<jtv> stub: just wrote a "lightweight patch" for the schema, but find that "make schema" breaks because of a CREATE INDEX CONCURRENTLY.  Should I do it conventionally (non-concurrently) in the patch?
<jtv> Ah, I think I see that in the documentation now.
<matsubara> thanks stub, I'll give it a try
<stub> jtv: Don't put the concurrently in there. I add it in when applying stuff manually
<jtv> It's a bit difficult to make out what's relevant for development and what for deployment.
<wgrant> stub, matsubara: There's a helper for that: load_related.
<stub> jtv: Just a normal patch ending with something other than -0. I do the rest.
<matsubara> wgrant, cool. I'll look for it
<wgrant> matsubara: However, the important thing is not to do this in a single query, but in a constant number of queries.
<wgrant> matsubara: Which means that you're going to need to calculate it outside BugTask.
<wgrant> So the page doesn't scale by number of tasks.
<jtv> stub: I don't see any other traces of lightweight patches in devel, of self-assigned revision numbers.  Am I doing it wrong or am I getting a "first post" on both?
<matsubara> wgrant, are there some examples of how that's done somewhere else?
<stub> patch-2208-76-1.sql
<wgrant> matsubara: There should be a fair bit of it in BugTaskVie
<wgrant> +w
<stub> or any other of the non -0 patches in the tree...
<jtv> stub: I don't see any in devel though â I thought lifeless said it could land in devel.
<stub> db-devel
<stub> I don't think they can land in devel yet
<jtv> argh
<wgrant> stub: Didn't you fix the hook?
<jtv> Maybe we should just try.
<stub> Did I?
<wgrant> You certainly asked a LOSA to do it a few weeks back...
<stub> I must be running low on drugs
<wgrant> Or maybe you only got hold of the config file.
<wgrant> https://rt.admin.canonical.com//Ticket/Display.html?id=44724
<stub> It must have been terribly exciting. I vaguely recall getting pqm tweaked a while a go but don't recall what the change wass
<wgrant> Status: resolved
<stub> So yeah, what my frontal lobe extension said.
<jtv> stub: can I have db review?  https://code.launchpad.net/~jtv/launchpad/index-798297/+merge/67046
<stub> jtv: message__datecreated__idx is preferred naming
<jtv> stub: thanks & sorry
<jtv> fixing
<jtv> stub: done
<stub> jtv: So the basic reason we need this is that ORDER BY id DESC sucks when filtering the date range?
<jtv> stub: not quite â but it is one of those correlated-columns problems.
<jtv> I'm trying to find the oldest Message before a particular timestamp.
<stub> Oldest message in what context?
<jtv> Sorry, newest.
<stub> DistroSeriesDifferenceComments
<jtv> I use the id of the newest older-than-X Message to filter the message FKs on DSDComment.
<jtv> bigjools: no test for what happens when distroseries.datereleased is null?
<bigjools> jtv: look again
<bigjools> I inserted an extra series, it is not returned
<stub> jtv: The id of the newest older-than-X message will either be max(id) or NULL
<jtv> stub: yes, the code is prepared for that
<jtv> bigjools: argh, that's hard to see!  Implicit is better than explicit etc.
<bigjools> jtv: how explicit do you want?!  It tests that only stuff with datereleased is returned
<stub> jtv: I'm just wondering then why you can't do max(id) and throw away the result if it is younger than the cuttoff.
<jtv> bigjools: which is the one that isn't returned based on this â ds3 or ds4?
<jtv> stub: You mean max(id) WHERE datecreated < foo?
<bigjools> jtv: I don't think you understand what is going on here
<jtv> Probably not.  Sorry.
<stub> jtv: no, max(id). and if the corresponding datecreated is too young replace it with NULL
<bigjools> jtv: I shall add more comments and then it will become clear, this is my fault
<jtv> bigjools: no need to go all mea culpa.  :)  But I think it would be better to split these issues up into separate tests.
<jtv> Translations experience involves repeated headaches and nightmares caused by single-run, multiple-data tests.
<bigjools> jtv: I don't think that will help, it will be the exact same tests over and over
<jtv> Almost.
<jtv> Right?
<jtv> stub: I'm afraid I don't followâ¦
<bigjools> in fact they'll be slightly less useful as they won't test real world data
<jtv> DistroSeries relationships aren't that intricate, are they?  I would think what the factory spits out is good enough to test for this.
<bigjools> it is not
<bigjools> the factory is naive
<bigjools> on top of this, pulseaudio is pissing me about
<jtv> It likes that.
<bigjools> what's the ui thing that configs it?
<bigjools> I am stuck with output going to speakers even when a headset is plugged in
<jtv> Errâ¦ KDE Sound Preferences?
 * jtv is blatantly bluffing
<stub> 'the id of the newest older-than-X message' == message = store.find(Message, ...).order_by(Desc(id)).first(); if message.date_created < X: message = None
<jtv> Me, I just shout a lot and hope some neighbour will call the other and say "please tell your friend to be quiet, he keeps shouting PLEASE RESTART DOGFOOD like some crazy person."
<jml> bigjools: it's sound preferences in normal Ubuntu. 'pavucontrol' is the one provided by pulseaudio itself.
<stub> c/</>
<stub> now I'm confused.
<bigjools> jml: thanks, I'll try it
<jtv> stub: I suspect that I wasn't clear enough in explaining.
<jcsackett> is anyone available to review https://code.launchpad.net/~jcsackett/launchpad/button-configs-break-pickers/+merge/67047?
<jcsackett> i am happy to review something in exchange. :-)
<stub> My antihistamines more likely. Too much grass here.
<bigjools> jml: unfortunately no luck.  There is something else somewhere that tells PA what devices to use IIRC
<nigelb> bigjools: skype?
<bigjools> nigelb: no, as soon as it detects PA it won't use anything else
<bigjools> and now phonon won't show me anything else
<bigjools> this is infuriating
<nigelb> bigjools: see if you can find dtchen in ubuntu-devel, he probably knows more about sound than almost most of us
<jtv> stub: I think you're querying for the newest Message, or None if that one is too new.  I'm looking for the newest Message that is older than a specific date.
<jml> bigjools: pavucontrol normally does that on the Playback & Recording tabs. I find it hard to believe that Kubuntu shipped without an equivalent
<bigjools> nigelb: thanks
<stub> jtv: Right, so we need the index. yes.
<bigjools> jml: gah, you have to be *in a call" for the option to appear in pavucontrol
<jtv> (Since DSDComments are a new thing, I was tempted to hard-code a cutoff number as a shortcut :-)
<jtv> What the zark is pavu?
<bigjools> kubuntu does it differently
 * jtv is amazed
<jml> bigjools: that sucks. I reckon it would be worthwhile seeking out whatever the KDE equivalent of Gnome Sound Preferences is
 * deryck switches work locations, back soon
<jml> bigjools: because Sound Preferences actually works in natty.
<bigjools> jml: it's the config for phonon - it has a preferential list of devices to use in different scenarios (comms, multimedia, games etc) but they all got replaced by just "pulseaudio
<bigjools> :/
<bigjools> so it's a lot better than pavucontrol - when it ****ing works!
<jml> bigjools: is phonon the central sound control for kde?
<bigjools> yes
<jml> bigjools: ahh.
<bigjools> I need a phonon expert I think
<jml> bigjools: see, the gnome sound preferences now (afaict) actually configures pa, and everything uses pa, so it's all win.
<jml> bigjools: *nod*
<nigelb> jml: When do you make the switch out of LP?
<bigjools> the phonon config is supposed to show all the pulse devices too
<jml> nigelb: already done
<nigelb> jml: Ah! Nice :)
<jml> bigjools: so it's buggy?
<bigjools> and of course, the next call goes back to the speakers
<bigjools> fuck sake
<nigelb> This sounds like the time my mic took input from my speaker.
<bigjools> jml: well it might be - it was all fine a couple of days ago :/
* abentley changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: abentley | Critical bugs: 204 - 0:[######=_]:256
<benji> I need to prepare.  How about I call you in about 5 minutes?  Skype?
<flacoste> benji: sure
<benji> flacoste: what is your skype user name?
<flacoste> benji: fjlacoste
<danilos> spiv, to be able to get my branches landed, I had to remove some bits that are in your branches as well (eg. the branch.revisionexpander.js stuff); expect conflicts soon :)
<jcsackett> abentley: have room in your queue for https://code.launchpad.net/~jcsackett/launchpad/button-configs-break-pickers/+merge/67047?
<abentley> jcsackett: sure.
<jcsackett> thanks.
<jtv> danilos: getting more fallout from that Google bug we discussed.  It's like half of India is trying to create mail accounts for me.
<jtv> (I know, half of India is a lot of people butâ¦ ever break a spammer's mailbox and see their ability to send email crumble to dust?)
<abentley> deryck: I am looking into https://code.launchpad.net/~matsubara/launchpad/39605-bugtask-tooltip/+merge/66928 and BugActivity scares the hell out of me.
<matsubara> abentley,thanks for looking that merge proposal but adeuring already reviewed it
<deryck> abentley, yeah, it can be a bit hairy.
<matsubara> it wouldn't hurt to have some additional feedback though :-)
<deryck> abentley, not to mention, it's such a pain to get any useful info from it.
<abentley> matsubara: why did adeuring review it?
<adeuring> abentley: matsubara asked me :)
<matsubara> abentley, I asked him on #launchpad. I looked at the topic and saw his name there and thought he was the OCR
<abentley> matsubara: ah.  No, I am.
<matsubara> by the time I noticed that, adeuring already had done the review
<abentley> matsubara: adeuring's review looks similar to what I was in the middle of writing.
<matsubara> cool. I'm working on the changes
<abentley> matsubara: This is what I was going to propose: http://pastebin.ubuntu.com/638927/
<abentley> matsubara: Actually, ISTM that you should just grab all bug activities and find the latest for a given whatchanged string.  You can answer all information about what changed then without further queries.
<abentley> matsubara: Since I assume you care about all bug tasks on that bug.
<matsubara> yep
<danilos> jtv, yay google ;)
<abentley> jcsackett: Was "hacro" a deliberate combination of "macro" and "hack"?
<jcsackett> abentley: as i don't remember ever typing that, i would say it's more likely a typo. but if you like it, we can pretend it was intentional. :-P
<jcsackett> abentley: i think it's probably best that i fix that comment, though. :-)
<abentley> jcsackett: also the "a a".
<jcsackett> abentley: indeed.
<abentley> jcsackett: bzr gannotate shows that the unused mailing_list is hysterical raisins.
<abentley> jcsackett: I'm not very comfortable with having "true" and "false" as Python member variables.  Seems like that would be hard to use in Python code.
<abentley> jcsackett: I would think you could calculate the whole config server-side and then simplejson.dumps it.
<jcsackett> abentley: i suppose you could--this branch is just a quick fix to get the other personpicker stuff fixed quickly. bug 799847 concerns redoing the form-macros stuff so we just put important things in JSONCache and then don't need to do any view/js passing.
<_mup_> Bug #799847: form-picker-macros should not do a bunch of manual js assembly <person-picker> <Launchpad itself:Triaged> < https://launchpad.net/bugs/799847 >
<jcsackett> at which point the simplejson.dump would also be unnecessary.
<abentley> jcsackett: I don't really understand how you'd use the jsoncache that way.  Would you have one JSONCACHE entry per picker?
<adeuring> abentley: your paste is roughly what I had in mind :) But I would remove a given attribute from the set "attributes" once it is found and terminate the loop when the set "attributes" becomes empty
<jcsackett> hm. crap, that's actually fair point. benji suggested it once upon a time, but we were looking at a case with only one picker on the page.
<jcsackett> still, i think cleaner form-macro stuff is more in scope for that bug (which i'll be addressing sometime shortly).
<jcsackett> in my mind, your suggestion (or other implementations) fall in there.
<abentley> jcsackett: Okay, fair enough.
<jcsackett> abentley: i may well be pinging you to pick your brain for preimp on that bug, if you're game for it.
<abentley> jcsackett: sure.
<jcsackett> cool.
<abentley> jcsackett: This should have tests, though.  I don't think I can approve it without tests that would fail if the Python True and False were used.
<jcsackett> abentley: sure, though i'm not sure how to test this without using windmill or something to do app-server + yui.
<jcsackett> abentley: and to my knowledge, we don't have the new implementation for those tests in place yet, do we?
<abentley> jcsackett: You should be able to render it and check for the strings, no?
 * jcsackett headdesks.
<jcsackett> yeah, actually, i can just do a view test.
<jcsackett> abentley: i will do that. i'm not sure why i was thinking only YUI stuff. :-P
<abentley> jcsackett: At 192 of the full diff, it says "instead if" but should say "instead of".
<jcsackett> abentley: ack.
<abentley> jcsackett: Your XXX should also follow https://dev.launchpad.net/PolicyandProcess/XXXPolicy (currently it lacks a bug)
<jcsackett> abentley: so it does.
<jcsackett> i'll fix that as well.
<jcsackett> in the YUI knowledge sharing thing right now, so changes won't be made until after that.
<abentley> Thanks.  Ping me again when it's updated, and I'm sure we can wrap it quickly.
<jcsackett> abentley: sounds good, thanks. :-)
<LPCIBot> Project db-devel build #702: STILL FAILING in 5 hr 43 min: https://lpci.wedontsleep.org/job/db-devel/702/
<cr3> hi folks, has anyone tried building launchpad on oneiric? I'm wondering if you also get this error, probably because of spriteutils: IOError: decoder zip not available
<deryck> cr3, I know sinzui runs on oneiric.  I don't know if he had that issues.
<deryck> cr3, unfortunately, he's not around this week.
<jcsackett> abentley: i have had time to circle back to this mornings review and fix the branch: https://code.launchpad.net/~jcsackett/launchpad/button-configs-break-pickers/+merge/67047
<abentley> jcsackett: ACK.
<abentley> jcsackett: r=me
<jcsackett> abentley: thanks!
<jcsackett> lifeless: ping.
<lifeless> hi?
<jcsackett> hi, lifeless. i've got a question about how to go about landing my fix branch regarding bug 801388 and bug 806179. yesterday you told me to land with rollback, but wgrant already rolled back that branch.
<_mup_> Bug #801388: some person pickers show "assign me"/"remove assignee" when that makes no sense <bad-commit-13334> <disclosure> <person-picker> <qa-bad> <ui> <Launchpad itself:Fix Committed by wallyworld> < https://launchpad.net/bugs/801388 >
<_mup_> Bug #806179: PersonPickers broken from bad passed in config <Launchpad itself:In Progress by jcsackett> < https://launchpad.net/bugs/806179 >
<jcsackett> so now my branch includes the fixes that wallyworld put in--should i just link it to both bugs and land normally?
<lifeless> jcsackett: right, so you can just land normally once you've bypassed the merge.
<lifeless> yes
<jcsackett> do i need to remove the qa-bad tag on 801388, or will qatagger handle all of that later?
<lifeless> we've deployed past it, and its ok (so aren't rolling back)
<lifeless> so you can drop the bad-commit tag if you want
<jcsackett> ok. thanks.
<lifeless> qatagger will replace the qa-bad tag when it sees the commit
<lifeless> you can drop it by hand too if you want, but its not needed
<cr3> deryck: I finally managed to reproduce on oneiric server but it worked fine on oneiric desktop, strange
<lifeless> cr3: missing dep in the brand new world of onei?
<abentley> jcsackett: I don't understand the purpose of https://code.launchpad.net/~jcsackett/lpreview-body/body-callback-template/+merge/66125
<cr3> lifeless: seems hairier than that, installing ubuntu-desktop didn't solve the problem but I could've sworn spriteutils worked when I installed a fresh desktop
<jcsackett> abentley: it may not be a great piece of code--it was my first stab at plugin code. in short, it was to allow a means of specifying a different template for merge proposals than the one that's hardcoded currently.
<abentley> jcsackett: Right, but why?
<jcsackett> for example, i prefer reST style headers, so i've put together a template that does that.
<abentley> jcsackett: We have a standard template that's supposed to be used, and that's what it provides.  If we change to reST headers, then we should update it to use that all the time.
<jcsackett> abentley: fair point. i didn't realize the current format was mandated, given many people use different ones. (e.g. sinzui uses one that specifies his rules for a completed branch).
<abentley> jcsackett: it may be honoured more in breach than observance, but yes, that's the standard.
<jcsackett> abentley: dig. than obviously the merge proposal should not be accepted. :-)
<jcsackett> s/than/then/
<abentley> jcsackett: thanks.  Rejected.
<jcsackett> is there a clear demo of using the expander widget in lp.app.javascript anywhere? i can't find anything that uses it...
<jcsackett> but i suspect my grep fu is week.
<jcsackett> s/week/weak/
<dobey> deryck: boo. my bug is not low. :(
<deryck> dobey, then go fix it. We're open source. ;)
<deryck> dobey, Seriously, though, I'm sorry this disappoints you. We have way more bugs then we'll ever fix and until this happens more frequently or causes more problems I doubt it will be more than a low bug.
<dobey> deryck: if this was the matrix, and i could inject all the knowledge of the launchpad source directly into my skull, i totally would :)
<deryck> dobey, fair enough :)
<dobey> deryck: well, it actually breaks a feature in tarmac, and therefore blocks branch landing :(
<deryck> dobey, sounds like a bug in tarmac to me ;)
<dobey> how is launchpad returning bad data, a bug in tarmac?
<deryck> dobey, that was largely a joke.
<deryck> dobey, though I'm not clear why a person etag matters for landing branches.
<dobey> jokes don't work on irc, unless it's "that's what she said" :)
<deryck> heh
<deryck> dobey, why does tarmac care about the etag?  is it changing something about a person and needs to verify before landing a branch?
<dobey> deryck: because we check that people have signed the contributor agreement, via team membership, where cla is required. and lazr.restfulclient checks the etag in its Entry.__eq__ override
<dobey> deryck: no, we're not changing the person at all. my bot user doesn't have those privs :)
<dobey> but the eq compares self_link, http_etag, and an internal dirty dictionary, and etag being different causes it to fail
<dobey> since apparently the person != the person
<deryck> dobey, so tarmac code is not referencing the two versions of people, lazr.restfulclient is checking the team member against the person collection for you?
<lifeless> dobey: perhaps you could include your sample code?
<dobey> well, in tarmac we're getting both objects and just doing ==
<dobey> https://bugs.launchpad.net/launchpad/+bug/806163/comments/3
<_mup_> Bug #806163: Different http_etag for same person resource when accessed via people collection or team object <api> <Launchpad itself:Triaged> < https://launchpad.net/bugs/806163 >
<dobey> the subteam == person there is failing, when it shouldn't be, because of the etag issue
<deryck> dobey, so check on ID, not object.  bug in tarmac. ;)
<dobey> huh?
<deryck> dobey, again a partial joke.  just check on something unique for each user, like self_link or id (if we expose that) rather than doing an object to object comparison.
<dobey> well, yes, i could work around it, and have temporarily. but workarounds suck
<dobey> but users' accounts being broken is, i think, a more important issue. i'm surprised more people aren't hitting this or similar situations on LP
<dobey> although, i don't know why exactly we haven't had an issue until now
* abentley changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: - | Critical bugs: 204 - 0:[######=_]:256
<wgrant> Morning all.
<deryck> Later on, everyone.
<LPCIBot> Project devel build #868: STILL FAILING in 6 hr 12 min: https://lpci.wedontsleep.org/job/devel/868/
<jcsackett> later, all.
<nhandler> Not sure how many people are subscribed to debian-derivatives@lists.debian.org, but there was just some discussion about possibly getting lintian to be run against ubuntu packages in Launchpad. I just thought it was worth noting here in case anyone with more experience is interested in getting involved in the discussion (although it will probably get brought to your attention soon enough)
#launchpad-dev 2011-07-07
<LPCIBot> Yippie, build fixed!
<LPCIBot> Project db-devel build #703: FIXED in 5 hr 25 min: https://lpci.wedontsleep.org/job/db-devel/703/
<LPCIBot> Project devel build #869: STILL FAILING in 5 hr 39 min: https://lpci.wedontsleep.org/job/devel/869/
<lifeless> bbiab
<wallyworld_> anyone know what's up with qastaging?
<wgrant> LOSA ping: ^^
<hloeung> wgrant: looking
<hloeung> wallyworld_: looking
<wallyworld_> hloeung: thanks!
<wgrant> I have a suspicion it's the DB patch.
<wgrant> qastaging-nohup.out should be informative.
<hloeung> wgrant: yes it is
<wgrant> hloeung: What's the error?
<hloeung> wgrant: canonical.database.revision.InvalidDatabaseRevision: patch-2208-76-2.sql has not been applied to the database. You probably want to run 'make schema'.
<wgrant> But it's meant to ignore it unless the suffix is 0...
<wgrant> Or maybe it only ignores it in the other direction.
<wgrant> Let me check.
<lifeless> this is the 'qastaging should run db updates automatically' RT
<wallyworld_> so a qastaging deployment needs make schema manually?
<wgrant> Yeah, it will ignore it if it's applied but missing, but not if it's unapplied but present.
<lifeless> not make schema
<lifeless> never make schema
<wgrant> wallyworld_: database/schema/upgrade.py
<wallyworld_> right
<hloeung> wgrant: okay, so what do I need to do to fix it up?
<wgrant> hloeung: In sourcherry's qastaging tree, "LPCONFIG=qastaging database/schema/upgrade.py"
<hloeung> wgrant: ok running it now
<hloeung> wallyworld_, wgrant: all up and running now
<wallyworld_> hloeung: wgrant: thanks!
<hloeung> np
<wgrant> hloeung: Thanks.
<wgrant> hloeung: We already run security.py on qastaging automatically, right?
<hloeung> wgrant: let me check
<hloeung> wgrant: yes we do, ever 20 and 50 minutes past the hour
<wgrant> hloeung: Could we run upgrade.py too?
<wgrant> lifeless: Any objections?
<lifeless> I believe there is an rt/bug asking for it
<lifeless> its more likely to deadlock if the server isn't kicked out.
<lifeless> but hell, we need that option anyway for prod ;)
<lifeless> so no, no objection
<wgrant> I can't see an RT for it. I guess it could deadlock, but meh.
<wgrant> lifeless: Lots of rabbit plugins automatically listen on a default port, which makes things problematic for the fixture (as rvb saw, causing him to create the MP you looked at last night)
<wgrant> lifeless: I think we probably want to make the fixture whitelist plugins.
<wgrant> (rabbit provides no way to disable plugins, but we can create an alternative plugin directory containing symlinks to the whitelisted plugins)
<lifeless> wgrant: +1
<wgrant> Great.
<lifeless> also, are the rabbit authors just idiots?
<lifeless> <damn, my inner pessimist escaped>
<wgrant> And we can configure the management port from a commandline option, fortunately.
<wgrant> You haven't seen the plugin system, have you?
<lifeless> let me guess, they run as separate erlang processes
<wgrant> No, no.
<wgrant> The build system is a bit odd.
<wgrant> Everything is locked to the one version.
<lifeless> oh? see, thats how *every one says you should do it*
<wgrant> They all have different config mechanisms.
<wgrant> The Makefile in the root of each plugin's tree starts with "include ../umbrella.mk"
<wgrant> Because you're meant to build them in a subdirectory of their plugin umbrella project.
<wgrant> Despite them all being separate projects and VCS repos.
<wgrant> ~/create-lxc-aufs is pretty handy.
<wgrant> VM setup in 5 seconds, with automatic destruction once you close the session.
<lifeless> it would be neat if it was included in lxc proper
<wgrant> It would be neat if it was less evil.
<lifeless> EPARSE
<lifeless> :P
<lifeless> translationgroup story test is breaking me
<wgrant> Oh?
<wgrant> How many are there?
<lifeless> 01-translation-groups-page.txt  10-distro-translation-group.txt   30-show-group-translation-targets.txt  40-remove-translator.txt                   46-test-distro-structured-permissions.txt  xx-change-translation-policy.txt
<lifeless> 05-add-translation-group.txt    15-product-translation-group.txt  35-appoint-translators.txt             44-test-distro-closed-permissions.txt      55-pofile-upload.txt                       xx-link-to-documentation.txt
<lifeless> 06-edit-translation-group.txt   20-project-translationgroup.txt   36-change-translator.txt               45-test-distro-restricted-permissions.txt  60-translation-suggestions.txt
<wgrant> Hee hee
<lifeless> thats not the problem
<lifeless> its failing when catted together
<wgrant> It's still horrible.
<wgrant> Huh.
<spiv> accidentally including the xx-* in the concatenation?
<lifeless> spiv: no
<lifeless> spiv: (but a reasonable guess)
<spiv> Ok, probably something difficult and horrible then ;)
<lifeless> I've done most of the other stories and they all worked first time
<StevenK> How do I change the context of a picker when the vocab is being set inside a Choice()?
<wgrant> You may be able to do it by overriding the widget used by the form.
<StevenK> And where is the widget set?
<wgrant> They are normally automatically derived from the interface.
<wgrant> Possibly grep around for setUpWidgets
<LPCIBot> Project db-devel build #704: FAILURE in 5 hr 5 min: https://lpci.wedontsleep.org/job/db-devel/704/
<wgrant> allenap: rabbitfixture r18 breaks lazr.amqp's test suite.
<wgrant>   File "/home/wgrant/Development/lazr.amqp/trunk/eggs/rabbitfixture-0.3-py2.7.egg/rabbitfixture/server.py", line 249, in _spawn
<wgrant>     with open(self.config.logfile, "wb") as logfile:
<wgrant> IOError: [Errno 2] No such file or directory: '/tmp/tmp1jc1vI/server.log'
<wgrant> Ahh.
<wgrant> allenap: RabbitServerResources won't setUp() properly once it's been tearDown()d, as its fixtures are destroyed but not unassigned.
<wgrant> I guess __init__ should set some templates that setUp uses if present.
<wgrant> But if they're not, setUp should create new fixtures even if the existing value is not None.
<allenap> wgrant: Okay, can you file a bug? I'll try and fix it today. Sorry about that.
<wgrant> allenap: I've fixed it and added a test.
<wgrant> Seems to be happy now.
<allenap> wgrant: Tip top :)
<wgrant> allenap: What was that change for? Getting a dev rabbitmq running?
<allenap> wgrant: Yeah, but lifeless has said that I probably ought to stick to ephemeral ports even in dev. I'm still trying to figure out how to do that and not make development less convenient (i.e. having to manually specify an LPCONFIG, or something like that).
<wgrant> allenap: I don't think you can.
<wgrant> As the config has to be shared between lots of scripts.
<wgrant> And the only common ancestor is the shell.
<wgrant> And that's hideous.
<allenap> wgrant: I did have one idea.
<wgrant> Plus there's the LP process which runs inside Apache.
<wgrant> Oh?
<allenap> wgrant: Move configs/development to configs/development-base, then create a new, mostly empty development config into which things like the rabbit service can write their configuration.
<allenap> development would extend development-base.
<wgrant> Possibly... but urgh. Does that mean I'd have to restart everything after restarting rabbit, because it will reallocate its port?
<wgrant> Woo
<allenap> wgrant: Yes :)
<wgrant> :(
<allenap> wgrant: I think it's more convenient to stick to a known port for dev services.
<wgrant> Yes.
<wgrant> Grar.
<wgrant> rabbitmq reset still takes up to 10ms sometimes :/
<wgrant> But mostly around 5ms.
<allenap> wgrant: Is that with the management plugin?
<wgrant> allenap: Yes.
<mrevell> Good morrow
<allenap> wgrant: That's better than several seconds though :)
<wgrant> I have a rabbitfixture branch which starts the plugin on a randomish port, and just hacked lazr.amqp to make use of it to reset.
<wgrant> Still, eventually most tests shouldn't be running with real rabbit.
<wgrant> So it won't be that bad.
<adeuring> good morning
<bigjools> lifeless: I thought of something else I wanted to chat to you about last night
<bigjools> wgrant: https://lp-oops.canonical.com/oops.py/?oopsid=OOPS-2013BUILDMASTER1
* allenap changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: allenap | Critical bugs: 204 - 0:[######=_]:256
<allenap> rvba: Do you mind if I mark https://code.launchpad.net/~rvb/launchpad/rabbit-error-plugins/+merge/67000 as rejected?
<rvba> allenap: no problem ;).
<wgrant> allenap, rvba: I have an alternate fix for that. I have a branch that makes rabbitfixture use only whitelisted plugins
<allenap> rvba: And apologies for suggesting it :)
<wgrant> bigjools: That's pleasant.
<allenap> wgrant: Cool.
<rvba> wgrant: Great!
<wgrant> bigjools: Do you have a copy of the upload?
<bigjools> wgrant: nope, just noticed it in the oops report
<bigjools> wgrant: but notice the: "builddmaster/incoming/20110706-222926-RECIPEBRANCHBUILD-58393"
<wgrant> Yes.
<wgrant> So we can easily grab the upload and see that the maintainer is non-ASCII or something.
<bigjools> I thought we had fixed unicode stuff *ages* ago
<wgrant> Right.
<wgrant> So it's probably something obscure.
<bigjools> I shall embuggerate it
<jtv> Anyone mind if I kick mawson?
<wgrant> bigjools: Hmm, no unicode in the dsc or changes.
<wgrant> jtv: Feel free.
<bigjools> wgrant: and, still getting None returned in the build deps from slaves :/
 * jtv feels free
<wgrant> bigjools: Ja
 * bigjools goes back to bug fixing
<jtv> allenap, can I trouble you for a review?  It's this one: https://code.launchpad.net/~jtv/launchpad/dsd-packageset-filter/+merge/66757
<wgrant> Ah, I bet the regression is 13373 or so.
<allenap> jtv: Sure.
<jtv> Thanks
<wgrant> adeuring: ^^
<adeuring> wgrant: thanks!
<wgrant> adeuring: Not confirmed yet, but it's probably it.
<wgrant> Still rebuilding...
<jtv> Here goes dogfoodâ¦
<jtv> bigjools, StevenK, wgrant: permission to update the CommonTasks page to say _just once_ that LPCONFIG must be "dogfood" and that this is automatic for the launchpad user, which you should always be?
<bigjools> jtv: automatic?
<jtv> Well, set as soon as you sudo into launchpad.
<bigjools> oh someone fixed that then
<wgrant> adeuring: Confirmed: r13373 causes the regression and needs to be rolled back.
<adeuring> wgrant: cool, thanks!
<wgrant> StevenK: See, I was right to be suspicious of it!
<bigjools> feck
<bigjools> I hate starting a branch in devel only to realise that I need to switch to db-devel to add new db permissions
<jtv> bigjools: you don't need db-devel just for security.cfg.
<bigjools> it depends
<bigjools> I heart merge --uncommitted
<wgrant> bigjools: Why do you think you need db-devel?
<bigjools> I am making a change to PCJ that needs a new update permission on a table
<wgrant> devel
<wgrant> Unless it's a new table.
 * bigjools blinks
<wgrant> We run security.py before each rollout.
<wgrant> Even on qastaging nowadays.
<bigjools> ok, didn't know that
<bigjools> and, yay
<wgrant> security.py --no-revoke, that is.
 * bigjools still doesn't see the need for security.cfg
<wgrant> adeuring: So, do you know how to roll that back?
<wgrant> adeuring: It needs to be rolled back nowish.
<adeuring> wgrant: erm, no, haven't this yet,,,
<wgrant> jtv: You're handling both your QA items, I guess?
<wgrant> rvba: How about yours?
<jtv> wgrant: that's what I'm trying to do
<lifeless> bigjools: shoot
<wgrant> jtv: Just checking, so we can deploy this regression fix ASAP.
<jtv> yesyesyesi'mworkingonit
<rvba> wgrant: bigjools & I are doing the QA since there is a lot to do.
<wgrant> Great, thanks guys.
<bigjools> lifeless: have you thought about using uuids instead of configuring oops prefixes everywhere?
<lifeless> bigjools: yes
<lifeless> bigjools: see lp:oops-repository
<bigjools> because they are a FPITA
<bigjools> lifeless: is that imminent?
<wgrant> adeuring: Reversion has landed.
<lifeless> bigjools: pretty close
<adeuring> wgrant: wow, that was fast! thanks!
<lifeless> bigjools: but a background task not foregrond atm
<bigjools> lifeless: fawesome.
<lifeless> https://bugs.launchpad.net/launchpad-project/+bugs?field.searchtext=oops+prefix may interest you too
<wgrant> adeuring: So, I guess you'll not still be around in 5 hours?
<adeuring> wgrant: well, I think I will be ;)
<lifeless> bigjools: I haven't filed bugs yet; sorry - been head bashing on stories/translationgroups
<lifeless> bigjools: will tie myself to the non-coding wheel tomorrow
<gmb> Does anyone know what I can do about EC2 runs failing with this error "IOError: [Errno 30] Read-only file system: '/var/www/summary.log'"?
<wgrant> adeuring: This fix should be deployable in a little over 5 hours.
<bigjools> lifeless: np - just wanted to see what the future looked like.  It's gotta be better than oops prefixes :)
<jml> gmb: no. never seen that one before.
<mwhudson> gmb: my guess is that means your instance is ******ed
<jml> gmb: my guess is that it's either a race (writing before a disk is ready somehow), or... what mwhudson said.
<allenap> wallyworld_: Are you around?
<gmb> jml: Okay. mwhudson's concise summation of the situation accepted, I shall try staying connected to the instance and see what happens when. THe error message is a bit opaque.
<mwhudson> gmb: this is reproducable?
<gmb> mwhudson: IDK yet. ISTR that it happened twice yesterday, but I can only find one email about it so I might be making that up.
<gmb> I do know that it didn't take long to fail, so I'll start an instance and grab some food and see what blows up whilst I'm dining.
<jtv> thanks allenap
<allenap> np jtv :)
 * gmb -> lunch
<allenap> It was a pleasure.
 * jtv coughs modestly
<jtv> Anyone else getting this error when running launchpadlib_for to connect to the web service?
<jtv> IntegrityError: new row for relation "oauthrequesttoken" violates check constraint "reviewed_request"
<wgrant> jtv: Production?
<jtv> I thought it defaulted to qastaging?
<wgrant> staging
<wgrant> Unless that changed recently, which would be abhorrent.
<wgrant> We committed great evil to oauthnonce on production yesterday or the day before, but that shouldn't affect this..
<jtv> Hmm service_root defaults to http://api.launchpad.dev/ it seems, but changing that to another server did not change the error.
<jtv> And why does this happen locally?  I'd expect this to be strictly a server-side concern.
<LPCIBot> Project db-devel build #705: STILL FAILING in 5 hr 3 min: https://lpci.wedontsleep.org/job/db-devel/705/
<wgrant> Does Translations really have a nice error email to send out when an error occurs during error handling of an exportâ½
<StevenK> wgrant: Heh.
<danilos> allenap, hi, I've got a nice little half-tested branch up for review: https://code.launchpad.net/~spiv/launchpad/bmp-inline-diffs/+merge/66634 :)
<danilos> wgrant, 'nice' is debatable, but some emails are indeed going out
<wgrant> danilos: well, it's nice compared to the rest of Launchpad...
<allenap> danilos: I've already looked at that one this morning; it has conflicts.
<danilos> allenap, whoops, wrong URL
<danilos> allenap, I need a branch that introduces the conflicts :)
<allenap> Hehe.
<danilos> allenap, https://code.launchpad.net/~danilo/launchpad/expander-anim/+merge/67161
<danilos> allenap, a prereq for that other branch
<allenap> danilos: Cool. I'm looking at a wallyworld branch right now, I'll do yours next.
<StevenK> Hm, I should put up my WIP branch
<danilos> allenap, thanks
<danilos> allenap, btw, I have an actual branch which replaces a couple of existing one-off expanders with the new one on https://code.launchpad.net/~danilo/launchpad/replace-expanders-1/+merge/67169 so if you can get to that, I'd appreciate it very much (it can also help you test the code)
<danilos> allenap, (test the expander-anim code among other things)
<allenap> danilos: I'll probably be able to get to it today. If not, jcsackett is reviewing later.
<danilos> allenap, cool, ta
<Ursinha> morning launchpadders :)
<danilos> Ursinha, good morning
<bigjools> hi Mrs Ursinha :)
* jcsackett changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: allenap, jcsackett | Critical bugs: 204 - 0:[######=_]:256
<deryck> Morning, all.
<jcsackett> morning, deryck.
<gary_poster> morning deryck.  did my email make sense?  I think I'm finished with what I intend to do for that branch (unless you pass it back to me) so If you pull the branch, hopefully you'll be conflict free for the rest of the day.  bug 806744 tracks the other work I mentioned to you; I'll land that separately.
<_mup_> Bug #806744: lazr.smtptest increases fragility of Launchpad appserver layer tests <thunderdome> <Launchpad itself:In Progress by gary> <lazr.smtptest:In Progress by gary> < https://launchpad.net/bugs/806744 >
<deryck> gary_poster, ok, great.  Haven't looked at the email yet, but that all sounds fine.
<gary_poster> cool deryck.
<deryck> gary_poster, and I don't think I'll have to pass it back to you.  Should be okay finishing that myself.
<gary_poster> deryck, I figured, just saying I wouldn't take it amiss if something came up and you had to. :-)
<deryck> gary_poster, ah, gotchas. :)
<deryck> gary_poster, thanks again for your work on this!  Will be very nice when this lands.
<gary_poster> deryck, np, thanks for pushing us to do it.  I'm excited for it to land too.
<deryck> gary_poster, seb128 is in #launchpad and can't subscribe a team he owns. we assume he has to make himself and admin....
<deryck> gary_poster, is this a known bug or a feature? :)
<deryck> gary_poster, if neither, he can file a bug for us.
<gary_poster> deryck, I think that's expected but I could be wrong.  danilos, do you remember?
<gary_poster> would be fine with a bug in the abstract deryck
<deryck> gary_poster, ok, cool.
<rvba> abentley: Hi, I've replied to your comments on my MP (https://code.launchpad.net/~rvb/launchpad/lp-app-longpoll-js)
<rvba> I tried to be thorough but I'm here if you need more information.
<abentley> rvba: I see.  Sorry, I'm on a call, but I'll be looking at it soon.
<rvba> abentley: sure. Thanks.
<deryck> gary_poster, seb128 filed bug 806971.
<_mup_> Bug #806971: team owners can't subscribe their team to bug emails <Launchpad itself:New> < https://launchpad.net/bugs/806971 >
<deryck> gary_poster, I'm happy to triage since I'm on interrupt, but it might be worth someone on yellow weighing in if it's a bug or a feature. :)
<gary_poster> deryck, ack.  I'll ask danilos to look at it
<deryck> gary_poster, thanks.
<flacoste> bigjools: do you have time to review my scoring branch? otherwise i could ask an ocr?
<flacoste> bigjools: i'll remove the --raise-priority option
<abentley> rvba: thanks for your changes.
<rvba> abentley: thanks for your comments :).
<abentley> rvba: I don't understand your rationale for wanting setupLongPollManager not to take LP.cache.longpoll as a parameter.
<rvba> I wanted to call setupLongPollManage() without argument in the main template.
<rvba> To avoid putting any logic in the main template.
<abentley> rvba: setupLongPollManager() is logic too, so I don't see a big difference.
<rvba> abentley: I still think it's best to keep all the details of the way the long poll thing works inside the js file.
<bigjools> flacoste: would you mind using an OCR, I'm right in the middle of something.  FWIW it looked ok to me, apart from that option :)
<flacoste> bigjools: ok, no problem
<abentley> rvba: Okay.  r=me.
<rvba> abentley: thanks.
* allenap changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: jcsackett | Critical bugs: 204 - 0:[######=_]:256
<flacoste> jcsackett: are you available to review https://code.launchpad.net/~flacoste/launchpad/bug-805634/+merge/67077
<jcsackett> flacoste: sure.
<jcsackett> flacoste: a little confused; the whole MP talks about a --raise-priority, but the most recent commit says it removes that option?
<flacoste> jcsackett: yep, that was contentious as it would potentially block the build farm for a day or more to other users
<jcsackett> flacoste: ok. i'll ignore that part of the MP then.
<flacoste> yeah, so that becomes, new way to compute score for copy archives -- allowing for relative scoring within it
<flacoste> and some other clean-ups
<jcsackett> flacoste: r=me. you made soyuz code more understandable--that should get a medal. :-P
<danilos> gary_poster, deryck: hum, could be a bug, don't remember right now
<flacoste> jcsackett: thanks!
<bigjools> jcsackett: I resemble that remark
<gary_poster> danilos, ok.  so, the only important question is if it is a regression, I guess.  if it is, the bug is critical, if not, high or low, whatever.  deryck, do you happen to remember the old behavior here?
<jcsackett> flacoste: :-)
<deryck> gary_poster, well, we didn't provide a list before.  you could search to subscribe someone else, and search for your team.
<gary_poster> you can still do that AFAIK, yeah?
<gary_poster> deryck, danilos, I dunno, I'm inclined to mark this critical/regression and move on.  anyone disagree?
<deryck> gary_poster, no.  seb128 wants to subscribe a team to for https://bugs.launchpad.net/ubuntu/+source/gnome-desktop3/
<danilos> gary_poster, I feel the same, it's probably a minor bug in the server side code
<deryck> and the list is provided once you click through.
<deryck> gary_poster, danilos, I don't have any objections to critical and move on :)
<danilos> (btw, this conversation seems weird considering we are talking 'critical' here :))
<gary_poster> deryck, oh, I see.  I thought this was per bug, not per per pillar or whatever.  ok, cool, I'm triaging.  Thanks deryck and danilos!
<gary_poster> deryck, danilos, actually I remember this.  I triaged high, with explanation in bug 806971, if you care.
<_mup_> Bug #806971: team owners can't subscribe their team to bug emails <Launchpad itself:Triaged> < https://launchpad.net/bugs/806971 >
<deryck> gary_poster, ah, good memory.
<cr3> should mixin classes be implemented to be listed before or after base classes in the inheritance list?
<danilos> jcsackett, hi, got time for a review? :)
<danilos> jcsackett, hum, never mind, you already did :)
<jcsackett> danilos: :)
<LPCIBot> Yippie, build fixed!
<LPCIBot> Project devel build #870: FIXED in 5 hr 54 min: https://lpci.wedontsleep.org/job/devel/870/
<danilos> jcsackett, hi, now I do have another branch up with more of the expander replacements, though this one is mostly code removal: https://code.launchpad.net/~danilo/launchpad/drop-collapsibles/+merge/67224
<jcsackett> danilos: i'm happy to look at it, as soon as i finish the one i'm currently looking at.
<danilos> jcsackett, cool, thanks
<danilos> jcsackett, I am about to leave, I hope you don't mind non-OCR reviews :)
<jcsackett> danilos: no problem, i'll just leave comments on the diff if i have questions.
<danilos> cool, ta
<jml> cr3: mixins are generally after the main base class
<bigjools> right, good night all
<cr3> jml: thanks for the reassurance, I noticed the opposite in lazr.restful: class EntryResource(CustomOperationResourceMixin, FieldUnmarshallerMixin, EntryManipulatingResource):
<jml> cr3: sometimes you do it differently to take advantage of Python's MRO semantics
<cr3> how do you typically test for integrity errors, like making sure the same product cannot be created twice? I've seen very few references to the IntegrityError exception under the tests directories, so I'm wondering if there's a better
<cr3> ... a better way even :)
<lifeless> generally we don't
<lifeless> not saying thats right or wrong ;)
<gary_poster> jcsackett, are you up for a branch? https://code.launchpad.net/~gary/launchpad/bug607961/+merge/67246
<jcsackett> garyposter: sure.
<gary_poster> thanks
<flacoste> lifeless or anyone else: any tip on what to do when the TEST RESULTS email failed with a 'message size exceeded error'?
<lifeless> it means something went catastrohpically wrong
<lifeless> the error output didn't compress enough to get mailed back
<lifeless> if you run it again, but follow the status page, you can pull down the subunit before the thing shuts down
<lifeless> gary_poster: FWIW: using the tip revision datestamp to sort out etags - wicked nice answer
<gary_poster> lifeless, cool!  Glad you like it.  Based on an idea from benji.
<flacoste> lifeless: i'm running --attached now
<gary_poster> flacoste, IIRC, our default license is AGPL 3.  barry would like our bounce-detecting LMTP to be GPL 3 or LGPL 3, and has requested that this be worked out before we work on it.  He thinks it might be a general-purpose component, and indeed ISD seems like they want something similar (bug 592814).  I'm always a fan of LGPL myself.
<gary_poster> What do I need to do to get the non-AGPL license approved?  It's been awhile since I had this kind of question...
<flacoste> gary_poster: get approval from statik
<flacoste> and potentially silbs
<flacoste> but you can just ask statik, he can handle any higher approval needed
<gary_poster> ack flacoste.  I'd prefer not to run it up that flagpole until we actually have something concrete, so I'll tell that to barry.
<gary_poster> oh ok
<gary_poster> I can do that then
<jcsackett> gary_poster: r=me.
<gary_poster> thanks jcsackett!
<LPCIBot> Project db-devel build #706: STILL FAILING in 6 hr 6 min: https://lpci.wedontsleep.org/job/db-devel/706/
<LPCIBot> Project devel build #871: FAILURE in 5 hr 35 min: https://lpci.wedontsleep.org/job/devel/871/
* jcsackett changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: - | Critical bugs: 204 - 0:[######=_]:256
<jcsackett> wgrant, wallyworld, StevenK: if you're planning on getting on mumble in about 15 minutes, I can be there.
<wallyworld_> jcsackett: ok
<StevenK> wgrant: You fail. Your db-devel merge included a 2208-99 DB patch number
<wgrant> StevenK: Arghhh.
<lifeless> wtf is this about http://pastebin.com/uyXnkWMK
* wallyworld_ changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: wallyworld* (jtv) | Critical bugs: 204 - 0:[######=_]:256
<mwhudson> that does seem pretty confused
 * wallyworld_ needs more coffee already. it's going to be one of those days....
<lifeless> ok, I've tracked down the problem, folk are changing the auth for 'browser'
<lifeless> different to my pastebin
<lifeless> the problem with gpg-coc and translationgroups
<lifeless>     browser.mech_browser.addheaders
<lifeless> Differences (ndiff with -expected +actual):
<lifeless>     + [('User-agent', 'Python-urllib/2.6'), ('X-zope-handle-errors', False), ('Authorization', 'Basic test@canonical.com:test'), ('Authorization', 'Basic test@canonical.com:test'), ('Authorization', 'Basic test@canonical.com:test'), ('Authorization', 'Basic test@canonical.com:test')]
#launchpad-dev 2011-07-08
<poolie> o/ huwshimi, lifeless
<huwshimi> poolie: Hey
<lifeless> hi y'all
<wgrant> Hmmm.
<StevenK> wgrant: Your description for fix-patch-number is a little harsh :-)
<poolie> does anyone know if curl checks ssl certs by default?
<spiv> I think so, hence various bzr bugs where the workaround was "use https+urllib"
<poolie> to not verify it?
<spiv> Right
<poolie> oh, this is because we didn't depend on the ca-certs packgae
<poolie> ok
<wgrant> StevenK: Hardly!
<lifeless> wooo, death to stories
<LPCIBot> Project db-devel build #707: STILL FAILING in 4 hr 4 min: https://lpci.wedontsleep.org/job/db-devel/707/
<mwhudson> lifeless: \o/
<jtv> wallyworld_: I'm in Europe, so won't be much good to you today!
<wallyworld_> jtv: np. there's no reviews yet anyway
<jtv> Not sure that's good news, but I'll take it.  :)
<lifeless> mwhudson: yeah, bzr+ssh://bazaar.launchpad.net/~lifeless/launchpad/storylayers is landing
<lifeless>  70 files changed, 4513 insertions(+), 4618 deletions(-)
<mwhudson> lifeless: i guess this doesn't rip out the story infrastructure?
<lifeless> yeah, it goes
<mwhudson> ah cool
<lifeless> the function that walks a directory and makes the suite stays
<lifeless> but its just DocFileSuites now
<mwhudson> and the cruft in the databaselayer that sometimes doesn't reset the db?
<lifeless> thats where I started
<lifeless> https://code.launchpad.net/~lifeless/launchpad/storylayers/+merge/66738
<lifeless> the tests for that cruft were not parallel safe
<lifeless> and I was making parallel testing more reliable
<mwhudson> ahh
<mwhudson> hooray
 * wgrant lunches.
<LPCIBot> Project devel build #872: STILL FAILING in 3 hr 54 min: https://lpci.wedontsleep.org/job/devel/872/
<wgrant> StevenK: Your conflict resolution failed.
<wgrant> bug-tags.txt failed.
<StevenK> Damn
<StevenK> wgrant: Having a poke at it
<wgrant> StevenK: Thanks.
<StevenK> wgrant: When does staging need a swift kick?
<wgrant> StevenK: Once the rev after yours passes buildbot.
<wgrant> Well.
<wgrant> I guess we could do it now.
<StevenK> It's adding back id and deleting one row from LDR, right?
<wgrant> UPDATe launchpaddatabaserevision SET minor=77 WHERE major=2208 AND minor=99 AND patch=0;
<StevenK> Or that, yes
<StevenK> And there is the failure mail
<wgrant> Yup.
<wgrant> Just that one failure.
 * StevenK is waiting for make
<StevenK> So I can run it again
<StevenK> wgrant: One line fix :-/
<StevenK> Not sure why bzr merge didn't pick it up
 * wgrant seeks opinions.
 * StevenK submits it to PQM anyway
<wgrant> Should I put the reset code in rabbitfixture itself? If so, do I make it mandatory? It depends on an unpackaged plugin.
<wgrant> Test isolation without it is very slow.
* wgrant changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: wallyworld* (jtv) | Critical bugs: 210 - 0:[######=_]:256
<wgrant> lifeless: Opinions?
<lifeless> slow is bad
<lifeless> other things will want rabid fixtures
<StevenK> Rabid or rapid?
<lifeless> both!
<wgrant> Grar
<wgrant> LP needs custom suites.
<StevenK> Hm. I don't think I can change the vocab using a feature flag :-(
<wgrant> You would have to override it in the view.
<StevenK> How?
<StevenK> From what I can see +distrotask (for example) uses BugAlsoAffectsDistroMetaView which is *tiny*
<wgrant> Right, but you can probably do stuff in setUpWidgets.
<wgrant> Override the defaults.
 * spiv wonders how hard it would be to write some js goop to automatically add tooltips to every 'bug NNNN' link on every page, to show the bug title.
<lifeless> theres a bug for that :)
<StevenK> wgrant: BugAlsoAffectsDistroMetaView is a MultiStepView and it doesn't seem to call setUpWidgets at all
<wgrant> It's not a LaunchpadFormView at all?
<StevenK> The only thing in bugalsoaffects that uses LFV is BugAlsoAffectsProductWithProductCreationView
<wgrant> Directly.
<StevenK> Right
<wgrant> lifeless: You never gave a real opinion on which version of rabbit we should target. Do you have one?
<wallyworld_> spiv: should be easy since there's code to parse bug nnnn and linkify it using an XHR call. the title would just need to be added to the json data sent back from the server
<wgrant> (2.2.0 is in CAT, but not in any Ubuntu release, so ew)
<lifeless> wgrant: easiest meeting all the constraints
<lifeless> wgrant: that *is* my opinion.
<wgrant> :(
<wgrant> My preferred option is to go with the latest stable Ubuntu release, 2.3.1. Trivial to put it in lucid-cat-lp. And it has not historically been a high-maintenance package, so shouldn't be a problem if LS wants to stay on 2.2.0 forever.
<lifeless> I bet ls will upgrade happily
<wgrant> Hopefully.
<spiv> wallyworld_: tempting!  I'm trying to resist the urge to grow my todo list rather than shrink it thoughâ¦
<lifeless> wgrant: ls would be happy to upgrade
<nigelb> spiv: I remember that bug. But I doubt if there's a lookup now. Adding that lookup was deemed more expensive the last time I asked.
<nigelb> Because, theoretically a page could be slowed down by having a lot of bug numbers
<wgrant> It's fairly cheap to do it by AJAX afterwards.
<wgrant> It's expensive to do it during page rendering serverside, due to the way the nested views work.
<nigelb> In that case, if the solution is easy - moderate and someone can mentor, I might be interested in picking it up
<spiv> The main thing I'd worry about is making sure the calls are reasonably batched; I don't think one AJAX call per bug link would be great.
<wgrant> Sure. We'd need to introduce a new API call to get details of a set of bugs.
<spiv> But if it were one for all the links present at domready time, then maybe a couple more as other ajax things add more to the page, I think that'd be fine.
<lifeless> note that this is then less efficient than supplying the data when we ender
<lifeless> *render*
<spiv> Yeah.
<lifeless> we already look up the bugs
<nigelb> BUt we also linkify nonexistent bugs
<lifeless> to decide to linkify or not
<nigelb> I thought that was because we don't look up
<spiv> lifeless: even for e.g. merge proposal comments?
<wgrant> lifeless: I don't think we do any more.
<lifeless> hmm, I may be out of date
<lifeless> spiv: theres a cluster of bugs around this
<wgrant> We used to display titles in tooltips.
<wgrant> But it was killed for being too slow.
<lifeless> yah
<spiv> nigelb: so, wallyworld_ reckons it should be easyâ¦ maybe he'd be willing to mentor? :)
<wallyworld_> spiv: nigelb: of course, no problem
<spiv> Sweet!
<spiv> Looking forward to it already ;)
<nigelb> \o/
<wallyworld_> ping me when you want to start work on it and i can provide some pointers
<nigelb> Hopefully, Monday.
<wallyworld_> np. i wrote the code to do the linkification via the XHR call so I can tell you where to hook stuff up. any performance issue will be separate to that and will need to be considered of course
<wallyworld_> spiv: just been reading the backlog - from memory there's only one XHR call to return details for all the bugs on a page so it should come together nicely
<nigelb> this particular bug has been a long term itch of mine, just didn't know there was a way to fix it without affecting performanc
<nigelb> +e
<wallyworld_> nigelb: out of curiousity, how long standing? the linkification via XHR call was only put in about 6 months ago
<nigelb> wallyworld_: Its from the old days of doing this server side
<wallyworld_> ugh
<lifeless> and its landed \o/
<lifeless> a fitting time to end the week
<adeuring> good morning
<LPCIBot> Project db-devel build #708: STILL FAILING in 5 hr 57 min: https://lpci.wedontsleep.org/job/db-devel/708/
<LPCIBot> Yippie, build fixed!
<LPCIBot> Project devel build #873: FIXED in 6 hr 2 min: https://lpci.wedontsleep.org/job/devel/873/
* adeuring changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: wallyworld* (jtv), adeuring | Critical bugs: 210 - 0:[######=_]:256
<jtv> Anyone else seeing JS test failures that happen _only_ on the command line, _not_ in the browser?  http://paste.ubuntu.com/639998/
<jtv> It sounds like a test isolation problem.
<jtv> But when I comment out a different test testMultiple, which consists of just a single assertion, it passes.
<rvba> jtv: which branch?
 * jtv digs
<danilos> allenap, hi, did you see my reply for the review? :)
<jtv> I haven't pushed it.  Let me just do that.
<bigjools> Gavin is off today
<rvba> danilos: allenap is off today.
<jtv> he knows :)
<jtv> rvba: just pushed to lp:~jtv/launchpad/bug-806946
<jtv> Still waiting to be scannedâ¦
<jtv> danilos: do we have foldable branch revisions yet?  It'd be really helpful at this point.  :)
<danilos> I didn't
<danilos> jtv, nope, expander-anim is still in review :/
<danilos> jtv, as is spiv's branch the last I heard of it (it was, yesterday)
<jtv> Goes to show how much better this will make the world.
<danilos> adeuring, wallyworld_: anyone want to take on the review of https://code.launchpad.net/~danilo/launchpad/expander-anim/+merge/67161 now that Gavin is off?
<wallyworld_> danilos: i'll grab it but someone else will need to +1
<danilos> wallyworld_, right, thanks, hopefully jtv or adeuring will do that :)
<jtv> wallyworld_: my job I think :)
<wallyworld_> jtv: hey, you're there
<jtv> More amazingly, you're still here.  :)
<wallyworld_> jtv: yeah, i should have eod but am tidying up some failing tests in a branch i really want to get landed
 * jtv knows the feeling
<wallyworld_> danilos: so is gavin not around today?
<jtv> Nope
<jtv> Well, shouldn't be.
<wallyworld_> bollocks. i was going to ask him to +1 on a mp. he did the prerequisite one yesterday
<bigjools> wallyworld_: !
<bigjools> I need your halp with this pycharm thing
<wallyworld_> bigjools: no cricket today :-(
<bigjools> !
<wallyworld_> bigjools: give me a bit of time to clean up some stuff, i'll ping you
<bigjools> ok ta
<wallyworld_> jtv: could you possibly +1 this mp for me. it looks big but its a lot of moving code around in yui tests more than anything else
<wallyworld_> https://code.launchpad.net/~wallyworld/launchpad/yui-test-cleanup-807294/+merge/67281
 * jtv looks
<wallyworld_> thanks
<jtv> Oh what the !?
<wallyworld_> when landed, our yui tests will be a lot better and all lazr-js stuff will be integrated
<wallyworld_> jtv: it's not as bad as it looks - a lot of code deletion
<wallyworld_> and it's only in the tests
<jtv> rvba: that breaking JS test passes for me when I run it with "lp.registry -t javascript" instead of "lp.registry -t test_distroseries."
<jtv> (Sorry about that â on to the MP now)
<jtv> wallyworld_: actually, I may have been running into the exact race that you've been fixing.
<wallyworld_> with the test_distroseries test?
<jtv> Yup.
<jtv> And by the way, did you raise hell about the long-poll test that isn't being run?
<jtv> File a bug?
<jtv> wallyworld_: ^
<wallyworld_> jtv: i will speak to wgrant or bigjools about the long poll stuff. we need to move the javascript to the "standard" place or change the find_test rules
<jtv> Also, I'm surprised that the Y.on('domready', run_me) that you're cleaning up worked.  I'd imagine that event would already have fired by the time that executed.
<jtv> wallyworld_: especially if the long poll will take time, make sure it isn't forgotten when a high-priority job comes along.  Better file a bug.
<jtv> I wonder if "the tests are never run" qualify as Critical.
<wallyworld_> jtv: with the domready event, i more or less cargo culted the old lazr-js implementation
<jtv> Well, the whole thing looks tons better now.
<wallyworld_> jtv: i checked that it works by hand, but....
<wallyworld_> yeah, easier to write tests, waaaaaay less cut'n'paste
<wallyworld_> i will file a bug about the long poll test too
<jtv> Great
<jtv> I'm not sure the bit about lock, stock, and barrels still makes sense now that lock, stock & barrel are no longer pasted everywhere.
<jtv> Another thing that really excites me to bits is that this should open the door to log divs that, *gasp*, would be wide enough to read.
<jtv> Or is that not the "log" div?  The one that the green/red markers appear in.
<wallyworld_> jtv: bug 807426. i've marked as critical since i agree that tests not running is bad
<_mup_> Bug #807426: Long poll yui tests not run automatically <Launchpad itself:Triaged> < https://launchpad.net/bugs/807426 >
<jtv> Thanks.
<wallyworld_> jtv: the log is sent to *both* the browser console and the log div, so it is really *very* readable now
<wallyworld_> the lock, stock thing is from curtis so i am loath to remove it :-)
<jtv> I think it's specifically about the situation you rectified though.
<jtv> By the way, we're supposed to have clear criteria to say whether a bug is critical.
<jtv> I'll just submit my MP vote and then I'll look into it.
<rvba> wallyworld_: I had to modify the way the Makefile finds js files to include the js long poll stuff ... so I suppose we should do the same for the tests to be found and modify "find_yui_tests" ...
<jtv> https://dev.launchpad.net/BugTriage#Critical
<wallyworld_> rvba: yes. the method to change is build_yui_unittest_suite. or you could move the location of the javascript. but if you change the build_yui_unittest_suite, try and keep it generic so that other new code that puts javascript elsewhere can also work
<wgrant> wallyworld_: rvba is more relevant for the JS stuff.
<wgrant> I only know it from a little while struggling with a French keyboard.
<rvba> wgrant: ;)
<wallyworld_> we need to decide our packaging methodology. is it lib/lp/<silo>/javascript/<vertical> (like it is now) or lib/<silo>/<vertical>/javascript
<wallyworld_> there's arguments for and against both
<wallyworld_> even though lp uses the former almost exclusively, i prefer the latter (like longpoll has done)
<wallyworld_> it keeps everything related together better
<rvba> wallyworld_: +1
<poolie> o/ wallyworld_
<wallyworld_> hi poolie
<rvba> wallyworld_: what is the outcome of Dublin's big javascript reorg?
<wgrant> wallyworld_: IMO the latter is better.
<wgrant> wallyworld_: But we need to be aggressively factoring out non-specific stuff.
<wallyworld_> rvba: the main branch to do the reorg has landed. i've just got the one to clean up the tests approved
<rvba> wallyworld_: Great! I'll check them out then :)
<wallyworld_> jtv: thanks for the review. with the tests, i'm confident that fixing increasing that timeout will ensure the tests pass ok, so my comments in the mp may be a little too pesimistic
<wallyworld_> rvba: the upcoming branch will cut large chunks of code from the yui tests. but the main refactoring has landed
<rvba> jtv: have you figured out your js problem?
<wallyworld_> danilos: this new expander is the one to replace all the other older implementations?
<rvba> wallyworld_: maybe this has been discussed earlier but did you do something to address the intermittent test_distroseries.html.TestDeriveDistroSeriesActionsWidget.testSuccess issue?
* wallyworld_ changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: adeuring | Critical bugs: 210 - 0:[######=_]:256
<wallyworld_> rvba: yes. i increased the wait timeout to 90ms (from 30ms). i seems to have fixed it for me. it could even be set a bit higher but let's see if 90 works for everyone
<rvba> wallyworld_: ok ... it would be nice if we could come out with a proper strategy to test things like this (things with an animation) ... maybe the code should fire an event when the animation is done and then tests could perform verifications only when the event is fired.
<rvba> I think this wasted too much of your time for a simple animation :)
<wallyworld_> rvba: yes, that sort of strategy is better. i just wanted to do the minimum to get the test passing since it was not the focus of my branch
<rvba> wallyworld_: sure, I get it.
<wallyworld_> and the current implementation used a wait()
<danilos> wallyworld_,yep
<danilos> wallyworld_, few other branches with it have already been reviewed and are waiting for the animation to land
<lifeless> gmb: hi
<lifeless> gmb: I'll look more closely on monday
<lifeless> but
<lifeless> +        if linked_branches.is_empty():
<lifeless> +            return EmptyResultSet()
<lifeless> isn't needed
<lifeless> is it?
<danilos> jtv, could you ok the wallyworld_'s review on https://code.launchpad.net/~danilo/launchpad/expander-anim/+merge/67161 please? (if you agree, of course :)
<lifeless> gmb: if it is needed, you should listify: is_empty performs a query, as does __iter__
<lifeless> gmb: so you're querying the same thing twice
<lifeless> as for whether you need to eager load, I suspect that you'll need to test - check the query counter on the templates that trigger foo.linked_branches, and see if it changes when you toggle eager loading on/off
<lifeless> gmb: (and you have 3 or 4 linked branches, maybe one private, that sort of thing.
<lifeless> gmb: I suspect you don't need it as the callers will be eager loading themselves, but IMBW
<gmb> lifeless: Ah, good point re the is_empty. It's only needed because the store.find() blows up if you pass it an empty list of branch IDs.
<gmb> So I'll listify early.
<gmb> And yes, I'll play with the query counter and see what comes out. Thanks.
<gmb> Anyway, tengo hambre.
 * gmb -> lunch
<jtv> danilos: was having lunch, coming now
<jtv> danilos: does JS have different relative priorities between the ?: and = operators compared to C?  In C, "cfg.from = cfg.from ? cfg.from : {}" would evaluate as "(cfg.from = cfg.from) ? cfg.from : {}"
<danilos> jtv, that's all copy-pasted from the old slide-out/in animations, I haven't questioned its sanity
<jtv> always a big mistake :)
<danilos> jtv, fwiw, we might want to drop that altogether if we don't care about users being able to set starting/ending points up front
<jtv> danilos: at any rate, I'm afraid I won't have time to go through all this today.  :(
<danilos> jtv, that sucks because branch is getting pushed around, and I might just as well land it then because I had two reviewers on it already :/
<danilos> jtv, I'll find a different reviewer then, thanks
<danilos> adeuring, hi, I need you so much :)
<jtv> Very sorry; bad timing.
<danilos> adeuring, can you perhaps ok wallyworld_'s * review at https://code.launchpad.net/~danilo/launchpad/expander-anim/+merge/67161?
<danilos> adeuring, I've also got https://code.launchpad.net/~danilo/launchpad/replace-expanders-2/+merge/67314 up for a full review :)
<adeuring> danilos: sure
<adeuring> (sorry for the delay; had lunch)
<danilos> adeuring, cool, thanks
<danilos> adeuring, fwiw, if you haven't started yet, I'd prefer getting a review for expander-anim branch first, because that one is a prerequisite for another 3 of mine branches, and for spiv's branch as well
<adeuring> danilos: r=me on the animation branch. great work!
<danilos> adeuring, yaay, thanks
<adeuring> danilos: your other branch looks also good, but there is one issue: On the +filebug page, I see the behaviour: for the first "expand" click on an item, the expand animation runs ... and then the collapse animation runs. On a second click, only the expand animation runs. And if you try one expand  click on collapsed item A, then try another expand click on item B, then then click again on item A, you still get "expand/close" animation
<danilos> adeuring, sounds pretty weird, what browser is that? (so I can try to reproduce)
<adeuring> danilos: firefox 5.0 (natty)
<adeuring> monring deryck
<deryck> morning, adeuring
<danilos> adeuring, can you try the same thing in a webkit browser (epiphany, chromium) to see if it happens there as well (while I start up firefox and get the proper branch running :)
<adeuring> danilos: just tried epiphany -- same behaviour
<danilos> adeuring, hum, very weird
<danilos> well, tbh, I did merge with the latest stable and haven't tried it with that, and now I have to run make schema as well (are we letting them on stable already, or did we merge db-stable back into stable already)
<wgrant> danilos: Some patches can be run live nowadays.
<wgrant> So we put them in devel.
<wgrant> https://dev.launchpad.net/Database/LivePatching
<danilos> adeuring, ok, I still can't reproduce this behaviour
<adeuring> danilos: so... what shall we do?
<danilos> wgrant, oh, cool (we could run some patches live in the past as well, but we did it in a very manual way :))
<danilos> adeuring, can you look at the console to see if there are any problems being reported perhaps?
<wgrant> danilos: Right. It's still pretty manual now, but at least they get landed in the tree properly.
<wgrant> danilos: One day we will automate it.
<danilos> wgrant, cool, sounds nice (and it also means that the pqm constraint is removed stopping DB patches getting to devel)
<danilos> wgrant, it might make development a bit more annoying with more frequent 'make schema' steps
<jcsackett> morning, all.
<wgrant> danilos: Well, we should probably have a script to run upgrade.py over everything quickly.
<wgrant> launchpad_dev_template, launchpad_dev, launchpad_ftest_template, launchpad_ftest_playground should do, I guess.
<danilos> wgrant, I found upgrade.py to be very off-and-on for development systems
<spiv> danilos: I'm not really here atm, but I love the commit message on expander-anim
<danilos> spiv, heh, thanks :)
<wgrant> danilos: But make schema is much faster these days.
<wgrant> danilos: So it's not *that* bad.
<danilos> wgrant, indeed it is
<adeuring> danilos: the only messages I see are: "yui: NOT loaded: lp.app.longpoll" and "Y.lp.app.longpoll is undefined            Y.lp.app.longpoll.setupLongPollManager(); "
<danilos> adeuring, can you perhaps re-run 'make jsbuild' or something along the lines (though that should have happened anyway)
<adeuring> sure
<adeuring> danilos: no change...
<wgrant> May be worth a make clean_js as well.
<danilos> adeuring, oh, indeed, wgrant has a point considering this also does some fixing to the newly integrated lazr JS inside the tree
<wgrant> Considering what has been done to the JS build system in the last week, I think it's worth a try.
<adeuring> danilos: I did not follow your conversation.... make schema??
<adeuring> gaa make clean_js
<danilos> adeuring, right :)
<adeuring> just a second...
<adeuring> danilos: no change.
<adeuring> danilos: but one more detail:
<adeuring> danilos: (1) click "expand" on a collapsed item -> expand/collapse runs (2) click into any other desktop window (3) click on the _title bar_ of firefox -> the collapsed item opens.
<adeuring> danilos: same behaviour with epiphany
<danilos> adeuring, hum, this sounds very weird and considering this code had some funny stuff even before, can you try out the clean "stable" tree and see how it behaves there?
<adeuring> danilos: sure
<danilos> adeuring, thanks a lot and sorry for the trouble with this
<danilos> adeuring, the existing code has an event handler for the focus event so it expands when it's focused even without a click, and also some code to stop propagation of click events for different parts of the item as well, and I haven't really touched that
<danilos> adeuring, so I am worried that it'd be hard to debug this but we can at least establish if it's the new code or the old code
<gary_poster> deryck, hey, are you going to reply to jml's question about thunderdome status?  I ask both because I could help with it if you wanted, and because I want to know how your changes are going :-)
<deryck> gary_poster, yeah, I was going to reply.
<gary_poster> cool, I look forward to it deryck :-)
<deryck> cool :)
<adeuring> danilos: the expander behaves "normally" on stable
<adeuring> danilos: one more details: The "expand/collapse" show runs only for a click on the green link text. Clicks onto the xepander icon or onto the text line "New (0 commentnts) cause a regular expansion
<LPCIBot> Project db-devel build #709: STILL FAILING in 5 hr 38 min: https://lpci.wedontsleep.org/job/db-devel/709/
<danilos> adeuring, ah, I can reproduce the problem now, I'll look into it, thanks
<adeuring> danilos: cool, and sorry for spotting the exact issue so late...
<danilos> adeuring, no worries, it's very nice to catch bugs like these early on
<danilos> adeuring, the problem is that there's a big area to click (anything below the green line also works, and the actual expander icon)
<danilos> adeuring, so the problem is with the focus handling and I'll look into that right now
<danilos> adeuring, and this doesn't happen in chromium which I mostly use for testing :/
<gary_poster> deryck, thanks for update, cool!  btw, my fix for the funky asyncore errors and hangs landed on devel yesterday.  I don't know how often you are encountering them, but for me it was obviously frequent enough to be annoying.  What I did seemed to make things much better for me.
<gary_poster> So, merging with devel might help
<deryck> ok, cool.  will do that.
<flacoste> bac: are you OCR today?
<benji> flacoste: bac is on vacation today
* adeuring changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: - | Critical bugs: 210 - 0:[######=_]:256
<dobey> is it just me, or is merge proposal diff generation broken today?
<wgrant> It is.
<wgrant> The script is hung.
<wgrant> Any maintenance people around?
<wgrant> (an email has been sent about it every 20 minutes for the last 3.5 hours)
<wgrant> But as it is 3am, I shall sleep.
<flacoste> gary_poster: ^^^^
<gary_poster> flacoste, ack.
<gary_poster> flacoste, we have no code people on our team to ask for details.  Do you know them
<gary_poster> flacosete belay that :-)
<gary_poster> hm
<flacoste> gary_poster: we'll have to start to figure them out by ourselves, as the only remaining code people is abentley and he's also on leave today :-)
<gary_poster> flacoste, yeah, I figured :-)
<gary_poster> flacoste, as best I can tell from looking at the logs, the code has some problems, but it is often able to recuperate, even if it has to lose a few jobs along the way.  However, there was something like a perfect storm of errors at the time of failure, and the last log that isn't simply waiting for the log file is this:
<gary_poster>   File "/usr/lib/python2.6/traceback.py", line 57, in print_tb
<gary_poster>     if hasattr(sys, 'tracebacklimit'):
<gary_poster> exceptions.AttributeError: 'module' object has no attribute 'tracebacklimit'
<gary_poster> This is indicatative of the interpreter having gone a bit insane to my mind
<flacoste> yeah, looks like it
<flacoste> is it now always failing with this?
<flacoste> as this is run every 20 minutes
<gary_poster> No, it is silent now, other than staring at the lock file (every minute, it seems)
<gary_poster> 2011-07-08 17:26:06 DEBUG   Cronscript control file not found at file:cronscripts.ini
<gary_poster> 2011-07-08 17:26:06 INFO    Creating lockfile: /var/lock/launchpad-merge-proposal-jobs.lock
<gary_poster> 2011-07-08 17:26:06 DEBUG   Lockfile /var/lock/launchpad-merge-proposal-jobs.lock in use
<gary_poster> 2011-07-08 17:27:04 DEBUG   Cronscript control file not found at file:cronscripts.ini
<gary_poster> 2011-07-08 17:27:04 INFO    Creating lockfile: /var/lock/launchpad-merge-proposal-jobs.lock
<gary_poster> 2011-07-08 17:27:04 DEBUG   Lockfile /var/lock/launchpad-merge-proposal-jobs.lock in use
<gary_poster> My immediate recommendation is to have losas kick the process and its subprocesses
<gary_poster> Then I can file bugs for the two or three errors that seem to have led up to this state
<gary_poster> Do you agree?
<gary_poster> well, kick the process and remove the lock file, I should say
<flacoste> gary_poster: sounds good
<gary_poster> ok thanks flacoste
<LPCIBot> Yippie, build fixed!
<LPCIBot> Project db-devel build #710: FIXED in 5 hr 34 min: https://lpci.wedontsleep.org/job/db-devel/710/
<m4n1sh> gary_poster: I am working on a small writeup of LP
<m4n1sh>  is it right to name the correct the various components of LP
<gary_poster> hi m4n1sh
<m4n1sh> e.g. malone or soyez
<m4n1sh> or the name has been deprecated?
<gary_poster> "soyuz" fwiw
<m4n1sh> gary_poster: hope your are fine thesedays
<m4n1sh> :)
<gary_poster> yes, thank you m4n1sh, and you too :-)
<m4n1sh> can I have the name of various components
<m4n1sh> cant recall
<m4n1sh> soyez
<m4n1sh> malone
<m4n1sh> malone - bug tracking
<m4n1sh> soyez - package management
<dobey> soyuz :)
<m4n1sh> ah
<dobey> rosetta is translations
<gary_poster> I don't think they are deprecated.  Some internal names that you see in the code base (poppy and some other names) are not really encouraged
<m4n1sh> ah that. just slipped
<gary_poster> but yeah
<dobey> i would just call them by what people see them as in the UI
<gary_poster> soyuz, malone, rosetta, but then "answers" "blueprints"
<gary_poster> yeah, I'd be inclined to agree with dobey
<m4n1sh> I am writing a detailed explanation of what good things are there in Launchpad
<m4n1sh> I am writing like
<m4n1sh> 1) Code hosting
<m4n1sh> 2) Bug tracker - Malone
<dobey> bugs, code, archives, translations, answers, blueprints, etc
<m4n1sh> 3) Question and Answers
<m4n1sh> 4) Package Management - Soyuz
<m4n1sh> 5) Translations - Rosetta
<gary_poster> "blueprints", fwiw
<m4n1sh> I can count only 6 till now
<m4n1sh> any other?
 * gary_poster looks for product manager's feature list :-)
<m4n1sh> lol
<lifeless> hmm
<lifeless> I need some nuance explained to me with the js security model
<lifeless> who has a minute or two ?
<gary_poster> m4n1sh, try https://dev.launchpad.net/FeatureChecklist
<gary_poster> the only thing you left out was what we call registry
 * m4n1sh can only understand jQuery when he hears Javascript is mentioned remotely
<m4n1sh> if the end user does not see registry, then it is useless to mention
<gary_poster> well...
<gary_poster> they see projects
<gary_poster> and code
<m4n1sh> ah yeah
<gary_poster> I mean
<m4n1sh> and teams?
<gary_poster> yeah teams
<gary_poster> but
<m4n1sh> can I also add Mailing lists in one more feature?
<gary_poster> yeah, that's another registry-ish thing
<m4n1sh> oh
<gary_poster> m4n1sh, I'd only use the "code" names if it helps you in your write-up
<lifeless> <script type="text/javascript" language="javascript" src="http://...." async=True /> - what data on the page can that loaded script access ? I assume it can read the DOM because its been explicitly loaded by the page.
<gary_poster> (like "soyuz" and stuff)
<gary_poster> because from Launchpad's perspective, we do not talk about them
<m4n1sh> i am mentioning in brackers (codenamed Soyuz)
<gary_poster> sure
<m4n1sh> *brackets
<gary_poster> lifeless, le looks
<gary_poster> me
<m4n1sh> it is like a academic paper
<dobey> gary_poster: hrmm, branch rescanning doesn't happen to be stuck now too, is it?
<m4n1sh> so details are appriciated
<m4n1sh> thanks everyone
<gary_poster> lifeless yes, afaik.  iframes can complicate things a bit
<lifeless> as in it might not be able to access the content of an iframe, even if the iframe is from the same site as the page, because its not part of the dom and its not same-site as the script
<gary_poster> right lifeless
<lifeless> however the script can't access the cookies, nor make new json requests to the pages site
<lifeless> gary_poster: the cookies of the main page I mean, moving on from iframes
<gary_poster> dobey, the only failure we had reported was merge-proposal-jobs
<gary_poster> that may be related; me looks at logs
<dobey> yeah i didn't see any rescnning issues earlier, but i'm seeing one that seems stuck now
<dobey> lifeless: is the javascript and html hosted from the same domain?
<lifeless> dobey: no
<dobey> lifeless: that would be why
<lifeless> dobey: I'm exploring the security implications of using a service that wants us to load their js remotely
<lifeless> dobey: so there are rhetorical questions, not 'I tried and it couldn't'
<dobey> ok
<dobey> gary_poster: ah, there, it finally rescanned
<gary_poster> dobey, whew I couldn't find anything wrong :-)
<gary_poster> lifeless: https://developer.mozilla.org/En/Same_origin_policy_for_JavaScript ?
<dobey> gary_poster: i guess it's just being slow for some reason. :)
<lifeless> gary_poster: frustratingly they don't talk about the nuance here
<gary_poster> more or less cool then
<lifeless> gary_poster: that is loaded document vs included script
<lifeless> I may be incorrectly assuming that there is a difference
<lifeless> gary_poster: its easy to reason about the case where you have two tabs and one tab wants to access the other, or make new requests
<lifeless> gary_poster: but this is the case where we load a script from an arbitrary site into the tab we have sensitive data on; I welcome corrections to my paranoia
<gary_poster> lifeless, I get where you are coming from.  This is questions of the sort "if we include Google site analysis code, do they own our data and our users?"  I should know this, and I feel like I did at one point, but I've forgotten.  rockstar would know I bet.
<lifeless> right
<lifeless> I've replied to mrevell in the relevant thread
<gary_poster> lifeless, since one of the standard ways of using YUI and similar libraries is to include it in the page from a source supplied by another provider (like Yahoo's CDN), I would be strongly inclined to guess that it runs, security-wise as if it were code that were written in the page
<gary_poster> all your Dom, cookies, etc. belong to us
<lifeless> gary_poster: I share that guess - its why I'm concerned
<gary_poster> yeah
<flacoste> m4n1sh: malone, rosetta and soyuz are really deprecated names
<flacoste> m4n1sh: we removed all references to them from the UI
<flacoste> so they should be called Bugs, Translations, Packages
<flacoste> Soyuz is the only one that doesn't want to die
<flacoste> the fact that we never put it in the UI in the first place probably makes this harder
<rockstar> gary_poster, yes, bringing third party CDN code into HTTPS means you give them access to everything you're putting behind HTTPS.
<lifeless> rockstar: thanks
<gary_poster> thanks rockstar!
<lifeless> rockstar: cookies as well ?
<rockstar> Basically, they could load some malicious javascript that reads data right off your page.
<rockstar> lifeless, yes.
<lifeless> righto
<lifeless> so we probably want to stop using analytics too :(
<gary_poster> well
<rockstar> Yeah, I mean, you're assuming google is reputable.
<rockstar> And they probably are.
<gary_poster> as you said for the other company "we've got no particular reason to trust them deeply"
<rockstar> But you give them access to things that you might not want to give them access to.
<gary_poster> It could be argued that we do have reason to trust Google.  It's a matter of degree
<rockstar> Ubuntu One is using Analytics, but I'm hoping to start using some better services soon.
<gary_poster> (I'm inclined to trust them; ruining our day would not be in their best interests, AFAIK)
<gary_poster> rockstar, with JS served by you?
<gary_poster> ("better analytics")
<gary_poster> uh, "better services" :-P
<rockstar> gary_poster, not sure.  Most of the services want you to use their site.
<gary_poster> yeah that's what I've seen too
<rockstar> But then we've got another request.
<rockstar> These sites would be "pay for" services, so at that point they would also not really want to ruin our day.
<gary_poster> heh
<m4n1sh> flacoste: thanks
<lifeless> rockstar: well we don't use their search appliance for lp private data because of potential data exposure
<lifeless> rockstar: so I think you probably want to talk with ISF :)
<gary_poster> flacoste: do you have an opinion about what LP should do if all a person's registered email addresses have been disabled due to repeated email bounces?  Email activity is so central that it might not be too crazy to disable the user, or at least put them in a state that is identical to being disabled except that leaving the state is done in a special way (simply stating that an email address is no longer disabled, for
<gary_poster> On the other extreme, we could simply silently stop delivering emails to this person (silently other than the bounce processor trying to contact them once a week for N=4 weeks), and continue normally if they log in.
 * gary_poster decides lifeless meant http://en.wikipedia.org/wiki/Information_Security_Forum
<rockstar> lifeless, I'm much better at asking forgiveness.  :)
<gary_poster> Iraqi Security Forces were another option
<lifeless> gary_poster: I meant elmo :)
<gary_poster> heh
<lifeless> IS foundations
<gary_poster> ah
<flacoste> gary_poster: yeah, if it's primary contact address that is bouncing, we might as well disabled them until they sort their email out
<flacoste> gary_poster: but it's not a strong opinion ;-)
<gary_poster> cool thanks flacoste.  yeah, fair enough :-)
<lifeless> that kindof implies showing them an explanation when they attempt to log in, and a way to fix
<lifeless> FWIW I'm easy
<gary_poster> :-)
<gary_poster> I agree fwiw lifeless--if we were to go down the disabling road, we should provide a nice path for cleaning the situation up.
 * gary_poster realized about 10 minutes ago that he was writing a LEP ;-/
<lifeless> it is a nontrivial chunk of work
<lifeless> long overdue though, and worth doing
<gary_poster> bye all
<thedac> staging on asuka is down. It errors out when attempting to start: https://pastebin.canonical.com/49528/
#launchpad-dev 2011-07-09
<LPCIBot> Project db-devel build #712: FAILURE in 5 hr 39 min: https://lpci.wedontsleep.org/job/db-devel/712/
#launchpad-dev 2011-07-10
<lifeless> StevenK: reckon we could use https://bugs.launchpad.net/ubuntu/+source/lxc/+bug/807351/+attachment/2196930/+files/lxc-start-aufs on hudson ?
<_mup_> Bug #807351: it would be cool to be able to clone an lxc container onto aufs for test runs - ephemeral containers <lxc (Ubuntu):Confirmed> < https://launchpad.net/bugs/807351 >
<lifeless> StevenK: for the parallel test job
<lifeless> time for testrepository features I think
<lifeless> ugh, faceplant
<lifeless> ok, time to fix bin/test
<wgrant> lifeless: Oh? What's borked about it?
<lifeless> --list
<wgrant> Ah, yes.
<lifeless> a) it includes non test ids in its output
<wgrant> And it includes the prefixes.
<lifeless> b) it does daftness when --subunit is passed
<lifeless> wgrant: https://bugs.launchpad.net/launchpad/+bug/808513 https://dev.launchpad.net/ParallelTests#preview
<_mup_> Bug #808513: testkeyserver uses fixed path <paralleltest> <Launchpad itself:Triaged> < https://launchpad.net/bugs/808513 >
<wgrant> I quickly worked around it with:
<wgrant> +test_id_option=--subunit --load-list $IDFILE
<wgrant> +test_list_option=--list
<wgrant> Not perfect, but it got me somewhere.
<lifeless> yeah
<lifeless> that brakes with 'testr -- -t stories/gpg'
<wgrant> Yeah
<wgrant> (aufs does normally permit rm, but the kernel's new hardlink security pedantry somewhat breaks it. sysctl -w kernel.yama.protected_nonaccess_hardlinks=0 might help.
<lifeless> also have you seen the sudo glitch I mention in #tech ?
<wgrant> lifeless: (catching up on email, your JS concerns are pretty much correct)
<lifeless> thanks
<lifeless> wgrant: ah, I see why you were talking about /tmp - its how the load-list was being passed around
<wgrant> lifeless: Yes.
<lifeless> whats the env variable to export?
<wgrant> lifeless: So I bindmount /tmp/testr now and then TEMP=/tmp/testr testr run --parallel
<lifeless> lets put TEMP in ./temp
<wgrant> If you bindmount ~, sure :)
<wgrant> I didn't.
<lifeless> we need the sourcecode bind mounted minimally ;)
<wgrant> Well, I ran with an entirely separate tree.
#launchpad-dev 2012-07-02
<wgrant> cjwatson: Do you think your seriesless copy change is OK to deploy?
<cjwatson> I'm just trying it out
<cjwatson> If I can bend dogfood to my will
<wgrant> Heh
<cjwatson> Hum, copying https://dogfood.launchpad.net/~cjwatson/+archive/openssh/+packages to https://dogfood.launchpad.net/~cjwatson/+archive/dogfood-copy-target/+packages not obviously successful
<cjwatson> Oh, maybe copyPackages is just not set up for this to work
<wgrant> cjwatson: What broke?
<cjwatson> Yeah, it worked as far as it went but from_series=None is useless
<wgrant> It may uniqueify the sources.
<wgrant> Ah
<cjwatson> But the to_series=None side of it worked fine
<cjwatson> Let's try +copy-packages then
<cjwatson> https://dogfood.launchpad.net/~cjwatson/+archive/dogfood-copy-target/+packages looks OK to me, libpipeline was copied with +copy-packages
<cjwatson> So it's just the permission bug you spotted, which I think we thought was currently fairly academic because PPAs are all published to main and that's the most restrictive component for Ubuntu?
<wgrant> Indeed, looks reasonable.
<cjwatson> The only slight awkwardness with qa-oking that is that I attached the other branch to the same bug ... but I guess I can deal with that later
<cjwatson> marked qa-ok for now anyway
<wgrant> cjwatson: It'll revert to qa-needstesting once the next branch hits qastaging.
<cjwatson> Yeah
<cjwatson> I'll do a bit more testing with the relaxed feature flags on dogfood tomorrow; I want to see whether the cause for soyuz.copypackageppa.enabled being off is still valid
<cjwatson> AFAICS there'd be no reason not to experimentally lower soyuz.derived_series.max_synchronous_syncs after this is deployed, but it will probably only get any meaningful use if we can also switch on soyuz.copypackageppa.enabled
<cjwatson> At least for beta testers
<cjwatson> I can probably get the LibreOffice guys to try it ;-)
 * cjwatson goes back to bed, immediate duty done
<wgrant> Night cjwatson, thanks.
<StevenK> Bleh, buildout is still convinced it wants 'django', but it's 'Django' :-(
<nigelb> o_O LP depends on django now?
<mwhudson> StevenK: i have both Django and django in my versions.cfg :/
<StevenK> Hm, I think that actually worked.
<rick_h_> wallyworld_: r=me with a few small picks
<rick_h_> thanks for the updates!
<wallyworld_> rick_h_: awesome, thanks for the detailed review
<rick_h_> wallyworld_: yea, sorry for being picky, tring hard to take some ownership of the JS and help improve things so I appreciate the time to make the small improvements
<wallyworld_> rick_h_: no need to apologise, our js is poor in many places
<wallyworld_> so it's great you are doing this
<wallyworld_> rick_h_: many of the things you picked up on were cargo culted so there's a lot to fix
<rick_h_> wallyworld_: yea, trying to do a goold old fashioned line the in sand. :)
<rick_h_> wallyworld_: cool, I know I hate when I get long reviews so and irc isn't the best so want to make sure things come across well
<wallyworld_> well, we gotta start fixing somewhere/sometime, now is as good a time as any
<rick_h_> with you and jcsackett and sinzui doing some awesome stuff I'm feeling there's hope
<wallyworld_> well, it's getting better slowly, but there's a lot of code to fix, and way too much code without test coverage
<rick_h_> yea, it's scary sometimes, but as long as it's getting better
<rick_h_> and hopefully as things get cleaner they get easier
<wgrant> Heh
<wgrant> Bug #800515 and bug #1016454
<_mup_> Bug #800515: Enable "NotAutomatic: yes" for -proposed <soyuz-publish> <Launchpad itself:Triaged> < https://launchpad.net/bugs/800515 >
<wgrant> Dupes filed a year to the day apart
<wallyworld_> having standard templates really helps, but there's still a lot of inconsistancy in terms of patterns etc
<wgrant> 3 hours off, though :(
<rick_h_> wallyworld_: yea, it is. I've talked with deryck and one of my goals for this next year to try to get a couple of really polished JS modules as the gold standard.
<rick_h_> if you've got a question, refer to these kind of things
<rick_h_> but anway, past my bed time, but I saw your review come up and I wanted to try to unblock you asap.
<wallyworld_> rick_h_: we also have way too much code that just adds methods to a namespace instead of using widgets. i used widgets for the sharing stuff and it makes it sooo much easier
<wallyworld_> rick_h_: thanks for reviewing and unblocking, i really apprciate it
<rick_h_> right, exactly. Widgets/Plugins, and I'm working on getting YUI 3.5 which adds Views
<wallyworld_> \o/or 3.5, can't wait
<rick_h_> so looking forward to doing a real View containing several Widgets
<wallyworld_> s/or/for
<wallyworld_> yeah, mvc
<rick_h_> yea, gotten through all the code in app/blueprint/bugs and onto the code module
<rick_h_> I'll have an email in the morning with the work/hits so far.
<rick_h_> and probably turning on the combo loader for all this week unless osmeone brings up a blocker
<wallyworld_> looking forward to it
<rick_h_> so I'm pushing hard for 3.5 sooner vs later
<rick_h_> anyway, wheee
<rick_h_> night
<wallyworld_> awesome
<wallyworld_> goofnight
<lifeless> wgrant: care to update the disklessarchives diagram ?
<wgrant> lifeless: Have you finished your edits?
<lifeless> I believe so
<lifeless> it was slow on saving was all
<StevenK> wgrant: How do you feel about reviewing https://code.launchpad.net/~stevenk/launchpad/auditor-layer/+merge/112962 ?
<wgrant> StevenK: That looks pretty vile
<wgrant> Shouldn't we not be testing against the real thing?
<wgrant> How is its DB managed?
<wgrant> etc
<StevenK> Real thing, as in production auditor?
<StevenK> As this is an established pattern for at least longpoll
<wgrant> One pre-SOA service does not an SOA pattern make.
<StevenK> That ^ was the plan that lifeless and I agreed on at least.
<lifeless> wgrant: your double negative has confused me, can you rephrase ?
<wgrant> lifeless: I thought the SOA guidelines said to test against a (network) fake.
<wgrant> Not the real DB-backed service.
<lifeless> wgrant: yes, which sqlite is ~= too
<lifeless> wgrant: in memory sqlite that is
<StevenK> Not sure if the default config of auditor is in-memory, but that can be fixed.
<lifeless> wgrant: I agree its not as lean as a reimplementation with no django at all, for instance, but its certainly fitting the spirit of the SOA, that it should be cheap to spin up, config-free etc.
<wgrant> If it's throw-away SQLite, then sure :)
<wgrant> lifeless: The reference implementation shouldn't have Django at all either, but I think I lost that argument...
<lifeless> wgrant: my understanding is that it is, or will be.
<lifeless> I kindof think that that config should be in the fixture itself, to reduce the window for surprise, but for now its ok, I think.
<StevenK> wgrant: So it's still vile, or you're refusing to deal with the MP? :-P
<StevenK> wgrant: https://code.launchpad.net/~stevenk/launchpad/populate-branch-aag/+merge/112264 should be easy to review
<adeuring> good morning
<jam1> can anyone help quickly with a testing question?
<jam1> I have a test that does "login('email@address')" and I'd like to change it to use 'with person_logged_in(...)'
<jam1> however the context manager takes a user object, not an email.
<jam1> how does one get one of those from an email address
<lifeless> jam: there is a test helper module with consants
<lifeless> jam: there is also a context manager that can deal with emails, I think
<lifeless> jam: but the specific answer is PersonSet.find(email), IIRC.
<jam> lifeless: there is admin_logged_in, anonymous_* celebrity_* and person_*
<wgrant> jam: Which test?
<jam> wgrant: I'm converting a doctest to a unittest
<wgrant> An email address is usually a sign that it's doing something horribly wrong.
<wgrant> Which address?
<jam> wgrant: https://code.launchpad.net/~jameinel/launchpad/py27-xmlrpc-auth-1019292/+merge/112792
<jam> wgrant: it is just trying to assert that the XMLRPC api is using a different user than whatever the current context is
<wgrant> admin_logged_in is correct there.
<wgrant> Oh
<wgrant> Hm
<wgrant> It's using foo.bar, but just as some random user. That's a bit odd.
<lifeless> fubared
<wgrant> Since foo.bar is an admin, it's a bit silly to use it to test that it's something arbitrary user
<wgrant> Just create a new one
<wgrant> self.factory.makePerson()
<wgrant> Use that
<lifeless> jam: general rules of thumb:
<lifeless>  - using existing admins is ok but not brillirant
<jam> wgrant: ObjectFactory has no attribute makePerson
<lifeless>  - using existing persons is generally a bad idea as it ties you to sample data
<lifeless>  - better to bring up just what you need from scratch using the helpers
<lifeless> jam: on developer LOC credit; the idea is to shrink, not stay static :)
<wgrant> jam: It does so...
<wgrant> lifeless: Creating new people frequently has major test suite performance implications, so I introduced admin_logged_in which always uses an existing one.
<lifeless> perhaps wrong test base class?
<wgrant> But in this case it's just one, and it's meant to be an arbitrary person, so makePerson is correct.
<lifeless> wgrant: yes, I know - tis a workaround though.
<StevenK> wgrant: Can I have your attention now? :-)
<czajkowski> morning
<wgrant> StevenK: Indeed, sorry, let me see
<wgrant> 64	+ branch_to_artifact = dict([(artifact.branch_id, artifact)
<wgrant> 65	+ for artifact in artifacts])
<wgrant> tsk
<wgrant> bad linebreak is bad
<StevenK> wgrant: What would you recommend?
<wgrant> StevenK: http://www.python.org/dev/peps/pep-0008/#indentation
<wgrant> StevenK: Convention is to say:
<wgrant>     dict(
<wgrant>         [(artifact.branch_id, artifact) for artifact in artifacts])
<wgrant>     branch_to_artifact = dict(
<wgrant>         [(artifact.branch_id, artifact) for artifact in artifacts])
<wgrant> That's the one.
<stub> And you can probably drop the square brackets
<StevenK> It's a list comphresion in a dict() call
<wgrant> StevenK: A generator expression in a dict() call should work just as well
<StevenK> wgrant: I've pushed up a change
<wgrant> StevenK: Thanks, r=me
<Laney> https://code.launchpad.net/~laney/launchpad/db-proposed-not-automatic-pre-release/+merge/112134 updated; could somebody review/land?
<mgz> jam: using `nohup bin/test -vvv >~/test_out.log 2>~/test_err.log &` seems good
<lifeless> mgz: testr run :>
<mgz> testr was introducing subunit borkÃ©dness
<jam> lifeless: that doesn't give any progress feedback (vs tee), etc. And yeah, subunit had some 2.7 issues (skip tests would cause broken stream)
<jam> I think gmb fixed that with a newer zope.testing, though.
<gmb> Yep. The dependency update should be landing in LP devel today.
<jam> gmb: I think it already did. At least, ISTR failing to start the test suite because it couldn't find a new enough zope.testing. Though maybe that was an unrelated zope.testing change
<gmb> O.o
<gmb> jam, devel still has p15, the 2.7 stuff is p16.
<gmb> I think
 * gmb checks
<gmb> jam, Yes, Still on p15 in devel.
<gmb> p16 will be on its way to ec2 shortly
<jam> gmb: probably devel rev 15470 which bumped it to r15. That was about 1.5weeks ago, but the checkout on my desktop was a bit old
<lifeless> jam: ah, true enough
<lifeless> jam: subunit itself should be 2.7 safe; been using it for months on 2.7
<jam> lifeless: yeah, zope.testing didn't handle skips, and in 2.6 unittest didn't support them either. So testtools passed them through as success.
<jam> In 2.7, unittest gets an 'addSkip' which wasn't getting sent on to the subunit stream
<lifeless> ah
<lifeless> win ;>
<jam> so you would see the test start
<jam> but not ever finish
<jml> perhaps stopTest should emit some kind of warning if that happens.
<jam> jml: would have probably made tracking down the failure easier
* benji changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: benji | Firefighting: - | Critical bugs: 4.0*10^2
<rick_h_> adeuring: ivory https://plus.google.com/hangouts/_/ef4e3eeb2016cc8910d1dc31949a0597da77f5f0?authuser=0&hl=en-US
<rick_h_> benji: can you take a peek when you get a chance please? https://code.launchpad.net/~rharding/launchpad/code_yui35_one/+merge/113031
<benji> rick_h_: sure
<mgz> hm, test for js stuff hung at:
<mgz> write(2, "\n(test:2741): Gdk-CRITICAL **: g"..., 10
<mgz> presumably related to not having X running
 * mgz kills the layer
<jelmer> mgz: you might want to run inside of xvfb-run
<mgz> presumably, will try that when the run completes
<rick_h_> mgz: yea, I had to do that with running tests in lxc
<jam> mgz:  lp.translations.tests.test_translationtemplatesbuildbehavior.TestTranslationTemplatesBuildBehavior.test_updateBuild_WAITING_notarballERROR:root:Slave returned no filemap.
<jam> ERROR:root:Build produced no tarball.
<jam> Failure in test lp.soyuz.tests.test_packageupload.PackageUploadTestCase.test_realiseUpload_for_delayed_copies
<jam> mgz: ^^
<mgz> jam: also lp.services.mail.tests.test_stub.test_simple_sendmail
<mgz> again email header wrapping
<mgz> Failed example: message['X-Generated-By']
<mgz> Expected: 'Launchpad (canonical.com); Revision="1999";\n\tInstance="launchpad-lazr.conf"'
<mgz> Got: 'Launchpad (canonical.com); Revision="1999";\n Instance="launchpad-lazr.conf"'
<mgz> slightly different doctest issue in lib/lp/services/webapp/doc/test_adapter.txt
<ivory> sinzui: could you take a look at this? https://code.launchpad.net/~ivo-kracht/launchpad/bug-1013281/+merge/113062
<mgz> that last one is trivial, is a threading.Event object, and new in Python 2.7 returns the internal flag object rather than None
<sinzui> thank you ivory. Do you want me to land this?
<ivory> sinzui: adeuring will do it but thank you nevertheless
<mgz> Total duration of test run 18883.5 seconds.
<jelmer> benji: Thanks for the review!
<benji> jelmer: my pleasure
<Laney> benji: could you help me to land https://code.launchpad.net/~laney/launchpad/db-proposed-not-automatic-pre-release/+merge/112134 please?
<benji> Laney: sure; how can I help?  "ec2 land" it?
<Laney> I don't know the commands you guys run :P
<Laney> I guess that's the one :)
<benji> hmm, given that there is a DB change, I'll have to refresh my memory on the right way to handle that
<cjwatson> ec2 land works for db-devel
<cjwatson> Shouldn't require anything especially special
<cjwatson> (The branch is targeted correctly)
<benji> yep, the target looks good; I'm firing off a job now
<Laney> ta
<benji> Laney: the MP needs a commit message and then I'll try again.
<Laney> benji: there
<benji> Laney: PQM doesn't like newlines in commit messages.  I have removed them and submitted.  The EC2 instance is starting now.
<Laney> benji: OK. So it's just the "short description" it wants then, for future reference?
<benji> Laney: I don't know of a prescribed guideline.  I like your commit message as-is.
<Laney> well I usually try to keep the first line under 72ish when writing them
<cjwatson> benji: I have a couple of split-outs from queue-api up for review to start with; do those look like a more reasonable level of granularity?
<benji> cjwatson: I'll take a look.
<cjwatson> I'm still juggling branches locally to find the next thing to separate
<cjwatson> Probably the read-only bits of the webservice exports
<sinzui> czajkowski, test-system failures are critical, not low. Lp's mailman installation is brittle, we cannot tolerate missing test coverage
<benji> cjwatson: looking good; the only thing I spotted was some extra parens (details in the comment)
<mgz> sinzui: I had some luck poking things with mailman, manually creating parent directories makes tests pass, but then oddly the buildmailman.py in make starts failing from not having any lists
<mgz> does buildmailman.py just need fixing to be better about creating needed directories and tolerating different state?
<sinzui> mgz, I think it needs fixing for the lxc case. Mailman always has at least one list. There is a control list that out code must discount. if it was not created, Mailman goes tits up
<sinzui> mgz, Other developers have reported problems were /tmp/var/mailman was not created properly, so I think this is an old issue
<mgz> did seem odd no one else had run into problems, I'll dig a bit further and see where I get
<benji> gary_poster: indeed
<cjwatson> benji: ta
<mgz> okay, that's all the remaining py27 things I've logged as having issues, though didn't get complete run due to mailman and X issues
<rick_h_> jcsackett: ping, if you remove the renderUI in the extending class does it still work right? You shouldn't need it since it's not doing anything extra?
<rick_h_> jcsackett: r=me with the two notes.
<sinzui> StevenK, wallyworld_, jcsackett https://dev.launchpad.net/Projects/Disclosure
<sinzui> https://bugs.launchpad.net/launchpad/+bugs?field.tag=disclosure+privacy-transitions&field.tags_combinator=ALL
<sinzui> https://bugs.launchpad.net/launchpad/+bugs?field.tag=disclosure+information-type&field.tags_combinator=ALL
<wgrant_> Grar
<StevenK> Haha
<wgrant_> This 3G is so slow I'm not actually sure it's 3G at all
<StevenK> Hahahaha, Optus 3G
<wgrant_> No mobile coverage is very good around here :)
* benji changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: - | Firefighting: - | Critical bugs: 4.0*10^2
<wgrant_> ooh
<wgrant_> flashing lights
<wgrant_> maybe they have fixed it
<lifeless> wgrant: where are you ?
<wgrant> lifeless: Some HFC fault took the entire street's phone and Internet out from about 4am until 90 seconds ago.
<lifeless> wgrant: \o/
<StevenK> wgrant: Six hour outage. Pretty short for Optus.
<wgrant> Apparently their monitoring sucks enough that I was the first to report it, at around 8:10.
#launchpad-dev 2012-07-03
<StevenK> wgrant: Bwahahaha
 * StevenK stabs buildbot
<nigelb> wgrant: Maybe you are the monitoring :)
<nigelb> I'm contemplating writing a vagrant script for LP.
<nigelb> So that it spins up a virtual machine in one command.
<nigelb> Because I'm lazy.
<nigelb> StevenK: A local python job posting wanted "experience with Launchpad"
<StevenK> Haha
<nigelb> I was tempted to apply and ask if my "experience" was sufficient ;-)
<lifeless> nigelb: elliot wrote one.
<nigelb> lifeless: oh. Is it around somwhere?
<lifeless> dunno
<rick_h_> lifeless: around? Want to check on the info I should be looking into with rt combo loader/convoy?
<lifeless> -> ops
<statik> nigelb, lifeless: the vagrant script I wrote didn't actually bring up LP, it was when I was experimenting with the microservice stuff
<statik> some good docs here though https://speakerdeck.com/u/mitchellh/p/develop-and-test-configuration-management-scripts-with-vagrant
<nigelb> ah!
<wallyworld_> StevenK: around?
<lifeless> nigelb: in NZ now?
<lifeless> statik: o/ wotcha. Off the boat now?
<nigelb> lifeless: not yet :)
<nigelb> I hope I get to NZ in time for kiwipycon.
<lifeless> cool
<lifeless> I might get to entertain you then :)
<nigelb> I heard :)
<StevenK> wallyworld_: Was noming. What's up?
<wallyworld_> StevenK: i think there's a 2nd issue with the change to bug info type in the privacy portlet but i wanted a 2nd opinion
<wallyworld_> the issue is that we are now calling the transitionToInfoType api directly
<wallyworld_> but previously the view would have been used via a post
<wallyworld_> and the view would have published a change event
<wallyworld_> but we don't do that now
<wallyworld_> and in fact, the non js version does use the view so does do the event
<wallyworld_> am i missing something?
<StevenK> Which event?
<wallyworld_> ObjectModifiedEvent
<StevenK> That should be fired via the JS calls transitionToInformationType over the API
<wallyworld_> i couldn't see where in transitionToInformationType it was fired
<StevenK> It isn't
<StevenK> It's done in the LAZR machinery
<wallyworld_> oh, so the api layer fires the event?
<StevenK> Yah
<wallyworld_> ah, ok. makes sense now. thanks
<wallyworld_> i didn;t realise the lazr stuff didi that
<statik> lifeless: yep, back in real life, finishing up an 18-hour first day on the new job :)
<lifeless> enjoying it ?
<StevenK> wallyworld_: Neither did I, until I figured out why the heck I was getting two visibility change events for a bug.
<statik> yeah man I love the firehose feeling at the start of a project
<wallyworld_> StevenK: ah yeah, i do recall that now
 * StevenK stabs buildbot and sets it on fire.
<lifeless> statik: nice. And you said it was rails ?
<StevenK> building
<StevenK> ETA in
<StevenK> ~ 6 hrs 53 mins
<spm> heya statik!
<StevenK> Really, buildbot? REALLY?
<huwshimi> Why do I get the feeling Launchpad is not playing nice with my virtualhosts
<StevenK> wgrant, wallyworld_: https://code.launchpad.net/~stevenk/launchpad/no-proprietary-multi-pillar-bug-vocab/+merge/113149
<wallyworld_> StevenK: do you need both context and IBug.providedBy(context) or does providedBy error on being passed None?
<StevenK> wallyworld_: IBug.providedBy(None) returns False, so I'll drop the first bit
<wallyworld_> StevenK: i'd replace 'PROPRIETARY' in the test by InformationType.PROPRIETARY.value or whatever the correct expr is
<StevenK> wallyworld_: How? It isn't there!
<StevenK> I'm trying to look it up by the token
<wallyworld_> sure but you can get the token from the enum
<StevenK> wallyworld_: The existing test for proprietary_information_type.disabled uses PROPRIETARY
<wallyworld_> you mean 'PROPRIETARY' with the ''?
<StevenK> Yup
<wallyworld_> ok then
<wallyworld_> StevenK: i'd also simplify the if statement for the proprieryary_allowed, i don't think it adds much value having 2 booleans
<StevenK> wallyworld_: We need them. The first one is backed onto the feature flag
<StevenK> It will be simplified when proprietary_information_type.disabled dies
<wallyworld_> so why no 'if not proprietary_disabled and IBug.providedBy(context)....'
<wallyworld_> not
<StevenK> Oh
<StevenK> I get it
<wallyworld_> just a thought since it's verbose with the extra boolean and it reads ok without i think
<StevenK> wallyworld_: http://pastebin.ubuntu.com/1072538/
<StevenK> It reads better against -r submit:, but meh
<wallyworld_> i think it's not quite right?
<StevenK> The test passes
<wallyworld_> ah, i missed the () around the 2nd bit
<StevenK> I'm looking forward to display_userdata_as_private.enabled dying, the vocab will shrink in the wash remarkably.
<wallyworld_> i think it will read better without the (). if not proprietary_disabled or not IBug.providedBy(context) or len(context.bugtasks) < 2
<StevenK> wallyworld_: That will cause PROPRIETARY to leak
<wallyworld_>  it's meant to the equivalent to your condition, but with the () removed
<wallyworld_> i think i have an or where i want an and
<wallyworld_> whatever works
<StevenK> wallyworld_: So I'm going to need brackets anyway, since there is a line break in the middle
<wallyworld_> yeah, maybe put the () bit on the second line
<wallyworld_> so it's all together
<StevenK> if not proprietary_disabled and not (
<StevenK>     IBug.providedBy(context) and
<StevenK> Sigh
<StevenK>     IBug.providedBy(context) and len(context.bugtasks) > 1):
<StevenK> That will likely cause wgrant to murder me.
<wallyworld_> if not proprietary_disabled and
<wallyworld_> not(....)
<wallyworld_> whatever you feel is best, i don't mind
<wallyworld_> r=me anyway
<StevenK> wallyworld_: Did you notice me fixing your test? :-)
<wallyworld_> oh, didn't realise it was my test. thans :-)
<StevenK> wallyworld_: I thought test_vocabulary_items_project was yours
<wallyworld_> it may have been
<wallyworld_> probably was
<wallyworld_> i just can't recall right at the moment
<wallyworld_> my head is stuck in info type js mollasis
<wgrant> StevenK: Wrong
<wgrant> StevenK: The bugtask count isn't relevant -- it's the number of distinct pillars that matters.
<StevenK> wgrant: Ah, so I need .affected_pillars
<StevenK> wallyworld_: http://imgur.com/gallery/L29C6
<wallyworld_> StevenK: lol. i had heard about a google streetview car crashing
<StevenK> wgrant: http://pastebin.ubuntu.com/1072565/ is more to your liking?
<wgrant> StevenK: Something like that, yeah
<StevenK> ssl.SSLError: [Errno 8] _ssl.c:504: EOF occurred in violation of protocol
<StevenK> zsh: exit 1     ec2 land --no-qa
<StevenK> Blink
<huwshimi> Someone please approve this: https://code.launchpad.net/~huwshimi/launchpad-news-wordpress-theme/hold-me-back-im-fixing-the-type/+merge/113159
 * huwshimi is ducking out for a minute, will be back later
<adeuring> good morning
* gmb changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: gmb | Firefighting: - | Critical bugs: 4.0*10^2
<mgz> gmb: where was yellow's launchpad kanban updating script?
<gmb> mgz lp:lp2kanban.
<mgz> ta
<czajkowski> maxb: gmb morning
<gmb> Morning czajkowski.
<czajkowski> gmb: hows things up north
<czajkowski> is it grimm and wet like London :/
<gmb> czajkowski, That's what we call "summer"
<gmb> But I can spot a teeny tiny bit of blue sky
<gmb> So all is not lost.
<maxb> czajkowski: a bit grey today, I suppose, but Saturday was lovely at least :-)
<czajkowski> maxb: lets hope it's nice this Saturday too :)
<jam> gmb: you're on-call today? I think mgz has 3 nice-and-tiny branches up for review if you feel like starting off easy
<gmb> jam, I'll take a look presently. Thanks for the heads-up.
<jam> (I'm trying to see if I can get a clean run on precise today on my desktop, we're pretty close)
<gmb> Awesome
<jam> well, it won't happen *today* given the commit queue, but something like that.
<gmb> jam Incidentally, when run in parallel, I said we only ran 1800 tests. That's not true; it's just that buildbot only _reports_ that many tests. We ran 18,000. So the stream is broken again somewhere.
<gmb> Haven't had chance to try looking into it yet though.
<jam> gmb: joy... but i'm glad to hear that the test suite is running without hanging/etc.
<gmb> Indeed
<mgz> reminds me, I haven't checked what subunit does with NUL characters yet
<mgz> I don't think it minds.
<lifeless> binary safe stream
<lifeless> particularly if you are using the chunk serialiser
<lifeless> (which LP is)
<gmb> mgz, Approved all your branches.
<mgz> ta... will have to create these cards in the next lane then :)
* rick_h_ changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: gmb, rick_h | Firefighting: - | Critical bugs: 4.0*10^2
 * gmb -> afk for an appointment
<danilos> gmb, hi, I wonder if you can please ec2 submit https://code.launchpad.net/~danilo/launchpad/bug-1000642-drop/+merge/112306 for me (stub submitted it but it seems to have failed with something unrelated to the branch: https://pastebin.canonical.com/69307/)
<danilos> damn, bad timing :)
<rick_h_> danilos: I can take a peek
<danilos> rick_h_, thanks
<rick_h_> danilos: ok, off to ec2 land
<danilos> rick_h_, ta
<cjwatson> gmb,rick_h_: Might an OCR be able to have a look over https://code.launchpad.net/~cjwatson/launchpad/queue-api-readonly/+merge/113202 ?
<cjwatson> Extracted from a previous much bigger branch.
<rick_h_> cjwatson: looking up
<cjwatson> Still 920 lines, oops, I thought it would be a bit shorter than that.
<cjwatson> I can split it up further if I have to.  This seems like a fairly reasonable logical chunk though.
<rick_h_> ok, will take a peek. I'll feel better knowing it could have been worse :)
<cjwatson> The original was 1647
<rick_h_> cjwatson: r=me with two points to bring up
<cjwatson> rick_h_: Thanks.  Do you think len() is preferable there then?  It was within the same object so I figured the "privacy" (insofar as Python has any) didn't matter.
<rick_h_> cjwatson: yea, especially since you changed the other cases, it's one part consistant at that point as well
<cjwatson> len(self.sources) will involve constructing a concrete list, although maybe that doesn't matter since it's short and will be cached.
<rick_h_> I guess when I wrote the point I initially wondered if you were doing that to avoid the cache
<rick_h_> but then didn't seem like it so figured if you were going to use/go through the cache everyone should
<cjwatson> Fair
<rick_h_> to avoid one method saying one thing "there's 2 here" but the cache not saying that
<cjwatson> #245 was a drive-by what-I-thought-was-a-bug-fix, but I can't prove off the top of my head that it's correct so how about I just drop it - it's not required as part of this branch
<rick_h_> not that I see a bug with mix matched results/etc but just letting you know where my brain's at reading it
<cjwatson> Yeah
<rick_h_> yea, sorry I'm just not all a pro enough at this and chaning r/o on stuff makes me want to find a list of people writing to it to make sure no one is
<cjwatson> Not a problem
<deryck> adeuring, https://plus.google.com/hangouts/_/3ac5f6358bc446e2018dedac8a0d188891410910?authuser=0&hl=en
<adeuring> rick_h_: could you please review this MP: https://code.launchpad.net/~adeuring/lazr.jobrunner/bug-1015667/+merge/113228?
<rick_h_> adeuring: sure thing
<rick_h_> abentley: didn't you do some magic to look at the queue payload without pulling it out of the queue?
<abentley> rick_h_: Yes, you basically just read the queue without confirming that you've read the messages.  It's the core of from lazr.jobrunner.celerytask import list_queued, and abel's already looked at it.
<rick_h_> ok, so my thing with this is that the function is called 'inspect-queue' which seems innocous, but it clears things.
<rick_h_> should it stick things back on the queue?
<adeuring> rick_h_: well... the fact that the messages are gone can be a positive side effect
<adeuring> in our case
<rick_h_> right, I understand in our use case, but I'm looking at this from the outside as someone who gets this package and tries to use it
<rick_h_> would clear-queues be better?
<rick_h_> at least then someone coming along would be a bit more hesitant to run the command
<adeuring> rick_h_: ok, makes sense
<rick_h_> adeuring: ok r=me with naming clarification. Thanks.
<adeuring> rick_h_: thanks!
<abentley> rick_h_: It's mystifying to me that drain_queues(retain=True) doesn't work, but abel says it doesn't.
<rick_h_> abentley: yea, I'm not tinkered with any of this so taking that stuff at face value as far as review goes.
<rick_h_> abentley: is there some way to tell what the local branch name was from a branch pushed to LP?
<abentley> rick_h_: The branch nick in the revisions defaults to the last component of the path.
<rick_h_> right, but I pushed it to a manual name and don't have that branch locally so I must have messed up and been on an existing branch
<rick_h_> now in the confusion can't tell where the changes I made yesterday are :/
<abentley> rick_h_: Pushing it up to a manual name is not going to change the branch nicks in the revisions.
<ivory> rick_h_: could you take a look at this?  https://code.launchpad.net/~ivo-kracht/launchpad/bug-921901/+merge/113234
<abentley> rick_h_: "bzr log --long -l1 lp:$BRANCHNAME" should tell you the last path component of the branch.
<rick_h_> ty much abentley
<rick_h_> ivory: I'm not following the bug that you're fixing here. When I look at the current live site, the tag links match what the bug says they should.
<rick_h_> ivory: what was the difference in the generated urls you altered?
<ivory> rick_h_: as i understand it they should link to a project group
<rick_h_> ivory: so I'm seeing: http://pastebin.ubuntu.com/1073255/
<rick_h_> the only difference I can see is there's a trailing / in the "Should be" urls
<ivory> rick_h_: thats weird because i'm at launchpad/+bugs/... if i click on a tag
<ivory> instead of launchpad-project/+bugs/...
<rick_h_> oh oh ok, I was looking at the tags in the sidebar, not in the buglisting themselves
<rick_h_> I don't have those enabled by default in my buglisting so missed the reference.
<ivory> ah ok i should have said that they must be enabled
<rick_h_> naw, my fault "bug column tags" should have been my clue
<rick_h_> thanks, anyway, reviewing right now :)
<rick_h_> ivory: r=me
<adeuring> rick_h_: another review. please: https://code.launchpad.net/~adeuring/launchpad/bug-1015667/+merge/113245
<rick_h_> if ! tmux has-session -t coms; then tmux new-session -s coms -n email -d mutt \; new-window -n irc -d "ssh db"
<rick_h_> fi
<rick_h_> adeuring: on it
<rick_h_> oops
<adeuring> rick_h_: thanks!
<rick_h_> adeuring: r=me
<adeuring> rick_h_: thanks again!
<yaguang> hi,all
<yaguang> I am installing launchpad  in my local envirenment
<yaguang> now everything works well except for mail serivces
<yaguang> all mail are send to localhost
<yaguang> so the user can not receive any  mail notifications
<yaguang> i have installed postfix  and mailman
<yaguang> and  i can send mail by using the CLI mail  tool
<yaguang> can any one help me  on configing mail
<yaguang> any one?
<sinzui> yaguang, the development instance is  not designed to send out emails
<sinzui> yaguang, Lp distributed version is for testing and development so it is does not provide the entire Lp stack and configuration needed to run for production use.
<yaguang> so if I like to deployment a local launchpad  just the same as  https://launchpad.net
<yaguang> shouldn't  I use  a development  branch ?
<yaguang> is it ?
<sinzui> I think this is just a config issue, but I do not know what must be set. yaguang the files under lib/canonical are not free. You need to replace the images to run the instance for any non-testing use
<sinzui> yaguang, everything under lib/lp is free to change and distribute
<yaguang> thanks
<yaguang> now  I see all mail is send to  localhost
<sinzui> yaguang, There are 2 kinds of config in each dir under ./configs
<sinzui> the zcml sets up zope services, of which mail might belong. there is also a -lazr.conf file that sets most of the host names and certainly knows about email
<yaguang> the mail header now is like this
<yaguang> From root@localhost  Fri Jun 29 05:42:06 2012
<yaguang> X-Original-To: root@localhost
<yaguang> MIME-Version: 1.0
<yaguang> Content-Type: text/plain; charset="utf-8"
<yaguang> Content-Transfer-Encoding: quoted-printable
<yaguang> Date: Fri, 29 Jun 2012 05:42:04 -0000
<yaguang> From: Launchpad Bug Tracker <21@bugs.launchpad.sinasws.com>
<yaguang> To: freedomhui@gmail.com
<yaguang> Reply-To: Bug 21 <21@bugs.launchpad.sinasws.com>
<_mup_> Bug #21: "Display Settings:" "untranslated" shows translated entries <lp-translations> <Launchpad itself:Fix Released by carlos> < https://launchpad.net/bugs/21 >
<yaguang> Sender: bounces@canonical.com
<rick_h_> jcsackett: heh so spoke too soon, these tests in test_distroseriesdifferences are going to need some updates for wait()/timing issues.
<jcsackett> rick_h_: well damn.
<rick_h_> jcsackett: just a heads up since you were asking about it. I've filed an XXX and will try to get back to it, but a bit much atm.
<jcsackett> dig.
* rick_h_ changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: gmb | Firefighting: - | Critical bugs: 4.0*10^2
<sinzui> jcsackett, do you have time for a talk?
<jcsackett> sinzui: sure. give me a moment to find a google+ compatible device. :-)
<jcsackett> sinzui: had a few flubs, hangout started now
 * sinzui no see you
<jcsackett> it's telling me you're not online.
 * sinzui restarts
<sinzui> Well I see a whole lot of black screen after I joined your hangout
<jcsackett> huh.
<jcsackett> i'm not even in a hangout now.
 * jcsackett bummed that in the 21st century video chat still involves so much faffing about.
<sinzui> I just got that message
<sinzui> create another hangout and I will poll for it
<jcsackett> ok.
<wallyworld_> wgrant: sinzui: http://people.canonical.com/~ianb/enhanced-newteam-picker.png
<Bert_2> Hi, on https://dev.launchpad.net/Running/LXC it says that the usage of 12.04 is highly adviced, however the command to create the lxc container has lucid and not precise in it, is this a mistake ?
<StevenK> The host should be precise, the container should be lucid
<Bert_2> StevenK: could you tell me why the container is best to be lucid ?
<StevenK> Because the production environment of Launchpad is still lucid for the time being.
<Bert_2> I see, perfect
<StevenK> wgrant: Getting confused -- I want to check getUtility(IAccessArtifactGrant).findByArtifact([self], [user]) , or am I missing something?
<StevenK> wallyworld_: Your QA for r15542 is up.
<wgrant> StevenK: That will check artifact grants for that user, ignoring team memberships and policy grants. It is useless for access checks.
<StevenK> wgrant: I couldn't figure what the heck IBug.userCanView() was calling, I got to bugtasksearch and went cross-eyed.
<wgrant> StevenK: get_bug_privacy_filter
<StevenK> Ah
<wgrant> You can see what the job does.
<Bert_2> I am for some reason not able to login to the newly created lxc container, not with ubuntu ubuntu or my regular login (which I get from ldap), is this normal ?
<wgrant> Bert_2: lxc-create copies the system user across by default. If it has no local password that's probably not going to work well.
#launchpad-dev 2012-07-04
<Bert_2> wgrant: but loggin in as root with the root password doesn't work either
<Bert_2> or does it only copy the password of the one user it's created with ?
<wgrant> Bert_2: If asked to bind ~ (the '-b $USER' option) it will copy the relevant lines from 'getent passwd' and 'getent shadow'. If -b is omitted, it will use ubuntu:ubuntu
<Bert_2> I see
<Bert_2> so I'll have to recreate the container then I guess ?
<wgrant> You'll probably have to leave out -b, or manually 'chroot /var/lib/lxc/$CONTAINER/rootfs passwd $USER'
<Bert_2> wgrant: okey, thanks :D
<wallyworld_> StevenK: yeah, i've been otp for an hour sorry. doing it now
<wallyworld_> StevenK: is done
<StevenK> wallyworld_: Thanks
<wallyworld_> StevenK: sorry it took a while, i was otherwise engaged for a bit
<StevenK> wallyworld_: Tis all good.
<StevenK> wgrant: So I've cribbed get_bug_privacy_filter for get_branch_privacy_filter, but I'm not sure how to collapse down the policy grant filter
<StevenK> 2012-07-04 01:15:01 DEBUG2  [PopulateBranchAccessArtifactGrant] Done. 30536 items in 304 iterations, 625.594790 seconds, average size 100.449373 (48.8121224706/s)
<StevenK> wgrant: ^ DF
<wgrant> StevenK: Rather than saying access_policies && foo, you can say access_policy = ANY(foo)
<StevenK> wgrant: Does Storm export an Any, or do I need to screw around with SQL() more?
<wgrant> StevenK: I'm not sure. At worst you'll need to add a three-liner to lp.services.database.stormexpr
<lifeless> pretty sure it does
<lifeless> and please add ovbvious things like that to storm itself
<lifeless> we have reviewers in-team, no need to make the techdebt higher than it already is
<StevenK> lifeless: I've been looking and I can't see it
<lifeless> ah, probably doesn't then. its weak on scalars
<lifeless> still, as wgrant says, easy to add.
<wgrant> lifeless: You mean !scalars?
<lifeless> maybe :P
<wgrant> Bug #1020785
<_mup_> Bug #1020785: lp.services.apachelogparser.base.get_files_to_parse doesn't like gzipped files over 4GiB <ppa> <Launchpad itself:Triaged> < https://launchpad.net/bugs/1020785 >
<wgrant> Can anyone see a better solution than checking if the read size is >4GiB and gzip isize <4GiB, then ignore the file if read size % 2**32 == gzip isize?
<wgrant> lifeless: Can I have a One Patch Policy exception for http://bazaar.launchpad.net/~launchpad-pqm/launchpad/db-stable/view/head:/database/schema/patch-2209-20-1.sql and http://bazaar.launchpad.net/~launchpad-pqm/launchpad/db-stable/view/head:/database/schema/patch-2209-25-1.sql?
<lifeless> well, thats also going to fail.
<lifeless> why not keep a signature of the last bytes in the file, or the mtime, or something? What are the constraints.
<wgrant> lifeless: It'll fail in roughly 1 in a couple of billion files.
<wgrant> Which is certainly less than ideal.
<wgrant> But not fatal.
<lifeless> wgrant: +1
<wgrant> Thanks.
<lifeless> on the file thing
<lifeless> what are the constraints, why do we need to check that way?
<lifeless> would a logstash consumer be a better answer ?
<wgrant> OK
<wgrant> So
<wgrant> We need a solution this decade
<wgrant> => logstash is not an option
<wgrant> gzip seeking is expensive.
<wgrant> So we can't just seek to the end and see what is there.
<lifeless> so these are problems not constraints.
<lifeless> they are obvious, the constraints aren't.
<lifeless> when do we need it fixed.
<lifeless> Does it have cpu limits? latency limits?
<lifeless> imagine you're telling a particularly annoying dev about the problem for the first time :)
<wgrant> We need to be able to progressively parse logs
<wgrant> Ideally without gunzipping 50GB of data we've already read
<wgrant> Every time
<spm> wgrant: I've done ... similar, but if I understand this code I'm looking at correctly, basically pull in a buffer of gzipped data, and pass that to gzread to decode and give back.
<spm> of course, this is C, and doing it the hard way is expected.
<lifeless> wgrant: well, that seems unsubstantiated; really not trying to be difficult. but "why not" ?
<wgrant> notsureifserious
<lifeless> am
<lifeless> also you can fingreprint a compressed file trivially
<lifeless> so not sure why you aren't proposing that
<lifeless> as a short term stop gap at least
<wgrant> We could store and check the CRC, but that's probably only marginally better than the filesize here, and it involves a schema change.
<lifeless> is that a constraint?
<wgrant> It's a very undesirable thing, as it means I can't fix it before lunch.
<lifeless> anyhow, short term hack - just store the gzip file length.
<lifeless> if the files are append only
<wgrant> How's that better than my initial suggestion?
<wgrant> It corrupts the database
<wgrant> And requires SQL to unbreak the history.
<lifeless> this seems like a fraught discussion
<lifeless> I'd rather move it to voice, or pass; We're trying to solve the same stuff, so the contention is wholly unneeded.
<wgrant> Storing the bad size seems strictly worse than changing the check -- I can't see any benefit.
<wgrant> Can you explain?
<lifeless> you want to optimise handling of files, avoiding rereads; I'd do that by fingerprinting the primary source of data, not be depending on a documented-as-broken field.
<wgrant> Oh
<wgrant> By "gzip file length" you mean the length of the compressed file?
<lifeless> yes
<wgrant> Not the file length from gzip.
<lifeless> right
<wgrant> Can't do that, since we need to know where to seek to within the file.
<wgrant> So we know which line to start at.
<lifeless> yes, it would then be different for existing things in the db, but depending on how you handle repeated reads that might not be an issue.
<wgrant> So that requires a schema change too
<lifeless> personally, I'd do the schema change; yes it means another day or two to get the fix deployed, but its much less likely to bite you in the bum later.
<lifeless> alternatively, use memcache to store out-of-file fingerprints and have fallback code for when thats missng, but - gnnngh, I don't really like that approach except perhaps as a stopgap.
<wgrant> memcache is only sensible when it's not hideously expensive to recalculate.
<wgrant> In this case we store logs for months/years
<wgrant> Worth hundreds of gigabytes
<wgrant> So the loss of the data is a huge problem.
<lifeless> what seems odd to me is that we have code even attempting to process old logs; theres no high water mark ?
<lifeless> if there isn't we have quadratic overheads anyhow
<wgrant> It's linear over a couple of thousand files and DB rows in a single run.
<wgrant> There's no watermark because we don't assume anything about the filenames.
<lifeless> if you want to do a quickiefixie, I've no objection. But I do think its worth considering how to make it deal with a couple of OOM of growth.
<wgrant> So ops can do as they please with logrotation etc.
<lifeless> wgrant: you assume a single file can get altered though
<StevenK> lifeless: Out of memory?
<wgrant> lifeless: It can
<lifeless> StevenK: order of magnitude
<StevenK> OH! Orders of magnitude
<wgrant> lifeless: It gets appended to throughout the day.
<StevenK> Yeah, just got it
<lifeless> StevenK: cool
<wgrant> lifeless: Also, not just that it can get appended to.
<wgrant> lifeless: We parse 10 million lines per run
<wgrant> lifeless: Logs have well over 10 million lines.
<wgrant> So we'd have to only store the size of the gzip file once we reached the end.
<wgrant> Storing the physical location in the file isn't going to work, because of the trailer.
<lifeless> sure, because thats how you're handling repeated reads (and optimising for finding-what-to-process-next).
<wgrant> lifeless: How would you propose to do it?
<lifeless> I'd suck stuff off of the logs as it happens and put it in a persistent queue, process it just in time from there.
<wgrant> If I had a list of things that were good ideas, queueing up thousands of writes a second on any data store we have today would probably not be on that list.
<lifeless> wgrant: 3K per second isn't high for amqp.
<lifeless> wgrant: we've got several OOM headroom. Particularly for small things, which this is.
<lifeless> well, at least two anyhow.
<wgrant> It also prevents us from ever disabling the process, or reading historical logs.
<lifeless> An we can shard the queues (and servers) if needed.
<lifeless> it doesn't discard the logs, we can indeed do historical logs if needed, but they become a special case, not the general case.
<wgrant> Anyway
<lifeless> Disabling the process can be done a few ways; have a stateful sucker-upper; disable the consumer (and just rotate queues if we have too many 10's of GB of logs pending).
<wgrant> That rewrite is at least 10 years away
<wgrant> How would you do it this century?
<wgrant> You seem to have a strong opinion that there's a better way than storing these offsets.
<lifeless> I think that when you're re-reading append only files, storing where you got to in it is fine.
<wgrant> 12:55:13 < lifeless> sure, because thats how you're handling repeated reads (and optimising for finding-what-to-process-next).
<StevenK> Hmm
<wgrant> That suggests you think it's not fine :)
<lifeless> I think for incremental and startup performance you don't want to read the file content at all unless you have work to do, so storing a stat fingerprint along with the amount read is important.
<wgrant> lifeless: Ah, perhaps once the number of files is about 3 orders of magnitude higher, yeah.
<lifeless> I think that having linear growth of files to handle, which is going to jump by three in the short term, suggests that considering all files each run may have a shorter lifespan than you think.
<wgrant> s/files/files per parser instance/
<wgrant> lifeless: It *should* have a shorter lifespan than I think.
<wgrant> lifeless: But it won't.
<lifeless> so, right now, I'd do a schema patch, adding a string. I'd use bzrlibs stat finger print function to fingerprint files, and I'd set that for files where we processed to the end of the file as we saw it.
<lifeless> I'd nag lifeless about getting near-realtime log shipping in place and prepare a cunning plan for doing it better later this year.
<wgrant> The fingerprinting thing is completely unhelpful for the problem we've encountered here.
<wgrant> It's an unrelated optimisation.
<wgrant> The problem is that we have this file.
<wgrant> We haven't completely parsed it yet.
<wgrant> But it hasn't changed since we last tried.
<lifeless> can you explain why that makes it unhelpful ?
<wgrant> We can't skip based on the fingerprint check unless we only set the fingerprint once we hit EOF.
<wgrant> Otherwise we'll skip files that we haven't completely parsed.
<StevenK> My Any addition doesn't work. :-(
<wgrant> And we haven't hit EOF here; that's the entire problem.
<wgrant> So the fingerprint check is unrelated.
<wgrant> StevenK: Oh?
<lifeless> wgrant: I don't follow. Consider this logic:
<StevenK> wgrant: Can you prod at http://pastebin.ubuntu.com/1074183/ after you and lifeless are done. Time for lunch.
<lifeless> does the fingerprint match the file (no fp matches no file)
<lifeless> onploooooooooooooopiuuuuu
<lifeless> cat
<lifeless> if yes, open, ungzip and seek to the last recor~6~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
<lifeless> BAH
<wgrant> StevenK: +class Any(CompoundOper):
<wgrant> StevenK: ANY is not a compound operator
<lifeless> if yes, open, ungzip and seek to the last recorded offset and process from there. On halt, if we finished the file, store the fingerprint we obtained at the beginning of this.
<wgrant> StevenK: It's a function.
<StevenK> wgrant: It's a NamedFunc ?
<wgrant> Right
<wgrant> lifeless: That's precisely what I described, right.
<StevenK> wgrant: That looks better, but I think the SQL is still bong: SELECT Branch.id FROM Branch WHERE Branch.id = 77 AND Branch.information_type IN (1, 2) AND COALESCE((Branch.access_grants)&&(SELECT ARRAY_AGG(TeamParticipation.team) FROM TeamParticipation WHERE TeamParticipation.person = 243654), false) AND (Branch.access_policy) = ANY((SELECT AccessPolicyGrant.policy FROM AccessPolicyGrant JOIN TeamParticipation ON TeamParticipation.team = Acce
<wgrant> StevenK: Where'd the array_agg go from the APG bit? If you can remove that easily, as it seems, then just say access_policy.is_in(apgs). No need for ANY
<wgrant> lifeless: Oh. I missed the "and I'd set that for files where we processed to the end of the file as we saw it" you said earlier as I was dying of optimism poisoning from the following line.
<wgrant> False optimism poisoning, that is.
<lifeless> well
<lifeless> you can call it that
<lifeless> anyhow
<lifeless> whats the issue with that algorithm? How will it not work ?
<lifeless> I'd hate to be suggesting something bong
<wgrant> With the crucial "only set on EOF bit" that I missed you saying, it should be fine.
<lifeless> there are other ways to fpr files, but the bzr one is shall we say battle tested.
<wgrant> Heh
<StevenK> wgrant: http://pastebin.ubuntu.com/1074238/
<wgrant> StevenK: Indeed,
<StevenK> wgrant: Indeed?
<wgrant> StevenK: That looks roughly correct. Does it work?
<StevenK> It returns []
<StevenK> I think the public query should not be ANDed in
<wgrant> Ah
<wgrant> Yes
<wgrant> That should be an or
<wgrant> public OR artifact grant OR policy grant
<wgrant> I meant the complex bits of the query looked correct :P
<StevenK> Haha
<StevenK> wgrant: Do you think returning the branch id is okay? I'm really interested if one of those three is True for branch.id == self.id
<StevenK> Can't COALESCE against three, I guess
<wgrant> StevenK: Why would you want to COALESCE against three?
<wgrant> You basically want to SELECT 1 FROM Branch WHERE id = $BRANCH_ID AND $VISIBLE
<wgrant> In this case you're selecting branch.id, but that's just as good
<StevenK> Hm, still returns []
<StevenK> Oh, but it's supposed to. I think this is the pre-subscribe check
<wgrant> What's it meant to return?
<wgrant> Ah
<StevenK> wgrant: So my next question is I want something more elegant than "return Store.of(self).find((Branch.id,), conds) == [self.id]
<StevenK> With an ending double quote
<StevenK> And putting a list around the store, but handwave
<wgrant> StevenK: More like 'return not store.find(1, Branch.id == self.id, $VISIBLE_FILTER).is_empty()'
<wgrant> (Storm might not like the 1 very much, but it doesn't at all matter what it is)
<StevenK> (1,), even?
<StevenK> AttributeError: 'int' object has no attribute '__dict__'
<wgrant> Yeah, it's probably trying to autotable it
<wgrant> Which won't work well
<StevenK> Same error with (1,) :-(
<wgrant> Might as well just say Branch, then
<StevenK> Yeah, we don't give a toss about what it returns
<StevenK> wgrant: http://pastebin.ubuntu.com/1074270/
<wgrant> StevenK: Does it work?
<StevenK> wgrant: Nope
<StevenK> Returns 0 rows before subscription which is fine, and 0 rows after which isn't.
<wgrant> Have you checked the contents of AAG?
<StevenK> No AAG created :-(
<wgrant> That would be the issue. Missing FF?
<StevenK> IBranch.subscribe() does not use a FF to guard
<StevenK> Hm, maybe the sharing service still thinks subscription confers visibility
<StevenK> Sort of. getUtility(IAllBranches).visibleByUser() needs porting
<StevenK> Which I'm trying to work out without my brain dribbling out of my ears.
<StevenK> wgrant: http://pastebin.ubuntu.com/1074301/
<wgrant> StevenK: You'll want to remove the helper methods that you've eliminated references to, but indeed.
<wgrant> StevenK: It may be worth seeing how the query looks if you reimplement the Branch method in terms of BranchCollection, as as I did with Bug.userCanView
<StevenK> % bzr damage
<StevenK> Using submit branch /home/steven/launchpad/lp-branches/devel
<StevenK> Healing: 4 lines of code
<StevenK> I approve
<StevenK> Hm, neat. permission denied on APG
<wgrant> stub: Hi, I've got a couple of DB reviews up for you when you have some time
<huwshimi> This is the appropriate way to run a single test right? 'testr run -- -t lib/lp/bugs/tests/bug.py'
<huwshimi> It should fail, but it's passing
<stub> wgrant: k
<wgrant> huwshimi: For Python modules you want to specify something like '-t lp.bugs.tests.bug'
<wgrant> huwshimi: But lp.bugs.tests.bug doesn't have any tests in it. It's just a helper module.
<wgrant> Perhaps you mean lp.bugs.tests.test_bug
<huwshimi> wgrant: Oh, I see where I've gone horribly wrong. Thanks.
<huwshimi> I've changed a table to a list and in lib/lp/bugs/tests/bug.py the print_bugfilters_portlet_unfilled() helper wants to grab the table and print its contents with print_table(). Any suggestions on how to print it now that it's a ul?
<StevenK> print_table is probably defined somewhere in that test
<huwshimi> StevenK: It's defined in lp.testing.pages
<huwshimi> StevenK: And there doesn't seem to be print_ul or anything
<StevenK> huwshimi: BeautifulSoup?
<huwshimi> StevenK: You mean, just print out the contents?
<StevenK> huwshimi: You could use a doctest matcher, but no, I don't mean just print out the contents.
<StevenK> huwshimi: BeautifulSoup will parse the output of print_bugfilters_portlet_unfilled() and you can then inspect it with asserts.
<wgrant> huwshimi: This is in a doctest, right?
<huwshimi> wgrant: No, this is in the helper
<wgrant> huwshimi: Well, yes, but it's probably eventually used by a doctest. The function prints it out, so you don't want to directly make assertions. I'd try the usual extract_text(find_tag_by_id(...)) pattern, see if that yields usable output.
<huwshimi> I see so I just need to get it to extract the same data in the same format as the table printer was doing. I think I'm on it.
<wgrant> Right, hopefully that's easy enough.
<wgrant> If you can't get it exactly the same, get something that looks sensible and change the callsites :)
<huwshimi> Thanks :)
<huwshimi> Well, that wasn't too bad
<adeuring> good morning
<wgrant> :(
<wgrant> postgres upsets me
<wgrant> It won't use an index including 'foo IS NULL' to satisfy 'foo IS NOT NULL';
<wgrant> czajkowski: Ew
<czajkowski> wgrant: morning to you too
<wgrant> Not really. But we like OpenStack, so we might be able to arrange something with a sufficient degree of evil.
<wgrant> Indeed, morning.
<czajkowski> wgrant: wrong channel
<wgrant> IMO it belongs here, but I guess it's arguable.
<czajkowski> it's early
<lifeless> wgrant: whats the specific index its not using ?
<wgrant> lifeless: bugsummary(distribution, sourcepackagename, tag IS NULL) WHERE distribution IS NOT NULL
<wgrant> lifeless: It'll use that for a tag IS NULL query
<wgrant> And if I s/tag IS NULL/tag IS NOT NULL/ on both it'll work
<lifeless> tag is not null where distribution is null ?
<wgrant> lifeless: Confused.
<lifeless> wgrant: may be a stats thing not an index thing; e.g. many rows predicted
<wgrant> lifeless: That's pretty much disproven by the fact that inverting the last element of the index causes the query to use it.
<lifeless> oh hangon.
<lifeless> wgrant: you say'
<lifeless> 20:22 < wgrant> lifeless: It'll use that for a tag IS NULL query
<lifeless> 20:22 < wgrant> And if I s/tag IS NULL/tag IS NOT NULL/ on both it'll work
<lifeless> so both ways around work.
<lifeless> Or you mistyped somewhere.
<wgrant> Yes. Both ways work.
<jml> hello
<wgrant> But only with the matching index.
<lifeless> so
<wgrant> But both are satisfiable from either.
<lifeless> bugsummary(distribution, sourcepackagename, tag IS NULL) WHERE distribution IS NOT NULL looking for 'tag IS NOT NULL' is what doesn't work ?
<wgrant> Yes.
<lifeless> wgrant: I presume you have a distribution in the query as well ?
<wgrant> lifeless: Naturally.
<wgrant> Otherwise it wouldn't use that index :)
<lifeless> well, as it isn't using the index, I felt I should check.
<lifeless> :>
<lifeless> jml: elloh
<wgrant> jml: Re. your email, that excludes test deps, I guess?
<lifeless> wgrant: so why index tag IS NULL; looking for the specific sub summary?
<wgrant> lifeless: We currently maintain separate indices on tag IS NULL and tag IS NOT NULL
<wgrant> Was trying to avoid that.
<lifeless> hmm, I forget the detail
<lifeless> have you tried DISTINCT FROM ?
<lifeless> wgrant: and NOT tag IS NULL ?
<jml> wgrant: yes, the output. the script is just a Pythonically aware grep for 'import'
<jml> wgrant: I'm probably going to buff it up so it recurses directories (or maybe packages)
<jml> and ignores internal imports
<jml> and then maybe consults some kind of db to figure out which things are modules and which aren't
<jml> but, you know
<wgrant> lifeless: NOT (tag IS NULL) has the same behaviour as tag IS NOT NULL
<wgrant> Which doesn't really make much sense.
<stub> wgrant: It never will, because an IS NULL index contains no information about rows without a null in that column.
<wgrant> stub: Not a partial on tag IS NULL
<wgrant> stub: But an index with the boolean "tag IS NULL" as one of its elements.
<lifeless> computed indices are evil
<lifeless> wgrant: why doesn't NOT (tag is NULL) behaving the same as tag IS NOT NULL, not make sense ?
<wgrant> lifeless: While you're here, what do you think about soren's question in #launchpad? We'd need to set archive.signing_key on the new PPA fairly soon after creation.
<wgrant> lifeless: Well, it was possible that it didn't consider IS NULL and IS NOT NULL to be inversions of the same operator.
<wgrant> But it clearly does.
<lifeless> I was hoping it might :)
<lifeless> what about DISTINCT FROM ?
<lifeless> uhm, so I think doing what soren asks would be good; be better to automate it (at least to the extent of sysadmin only apis).
<lifeless> or sometihng.
<lifeless> I'm not a huge fan of manual fiddling, but this is a case where our model makes decisions more stressful for users, its largely incumbent on us to deal with the side effects.
<wgrant> Exactly.
<wgrant> Although APIing this sounds beyond perilous.
<stub> I think NOT( ) has to assume the contents of the parenthesis is true, false or null. IS NOT NULL is a single operator and it is known it returns true or false.
<stub> wgrant: You go into detail about bugsummary journaling performance, but isn't that irrelevant for writes atm?
<wgrant> stub: Not at all. It's the most expensive bit of big bug updates apart from the usual Storm n+1 query thing.
<stub> wgrant: Yes, and are big bug updates a performance problem?
<wgrant> stub: We have about 3 timeout bugs about them.
<wgrant> eg. approving Linux SRUs
<stub> Do you know what sort of number of bugs are being done in a batch?
<wgrant> Can create about 50 tasks in one transaction, because the kernel team are insane.
<wgrant> We end up with bugs with 80ish tasks, 2/3 of them created in one transaction.
<wgrant> If the bug has 8 tags, which is fairly common, that's a lot of journal rows.
<wgrant> The last task will unjournal 79*8*afew rows, journal 80*8*afew rows.
<stub> Hmm... So I'm not against performance improvements, just improving performance doesn't really solve scalability problems.
<wgrant> It stops bulk task addition from being quadratic.
<stub> We can fix it so creating 80 tasks works on our hardware in a single transaction, then some sod will try and create 81.
<stub> And things like dropping foreign key constraints mean we should have DELETE triggers too (they are there already I think?)
<wgrant> stub: Well, the FKs really shouldn't have been on these tables at all. They actually cause correctness bugs.
<stub> We normally don't bother with them in this situation. Can't recall why we had them.
<stub> I'm wondering if we rely on ON DELETE CASCADE behaviour somewhere.
<wgrant> We don't.
<wgrant> Even if we did, it would be wrong.
<wgrant> Since the deletion process would also remove the references in the original tables.
<wgrant> Which would create journal entries.
<wgrant> So you'd end up with negative counts in bugsummary after journal rollup.
<wgrant> Negative counts are bad :)
<wgrant> Though we have some :/
<wgrant> 289 bugsummaryrows with count < 0 :/
<stub> That would be a logic flaw somewhere in the triggers, wouldn't it?
<wgrant> There's a lot of corruption in Ubuntu's kernel bugs. I've identified a couple of bugs in the initial population code that cause some of them, but it doesn't explain all of it.
<wgrant> The triggers themselves weren't significantly buggy AFAICT. Just the initial population query.
<wgrant> lifeless: Given https://pastebin.canonical.com/69406/  I propose https://pastebin.canonical.com/69407/
<stub> wgrant: I wonder if we should dump doing this in triggers entirely, but I guess that would require reliable async job processing.
<jam> mgz: did you see that there is one more failing test on python-2.7? It looks like a doctest that is (again) overly sensitive.
<wgrant> stub: I came close to doing that when doing BugTaskFlat, but before we can sensibly do that we need both reliable async job processing and something resembling a sensible data access layer in the app.
<stub> r=stub
<wgrant> stub: Thanks.
<wgrant> stub: Other disclosure stuff (that doesn't touch the existing bugs model so directly) is using celery-based eventual consistency fairly extensively.
<stub> Yeah, just wondering if we are investing too much in rabbit. Complex beast, and more complexities to ensure we don't drop tasks.
<stub> Using PG as the task queue for celery could suit us much better, and I think celery supports that out of the box.
<wgrant> Indeed, although it hasn't been problematic yet, and we're not using it for absolutely critical stuff.
<stub> IIRC We haven't chosen it for some things because of this problem, or chosen more convoluted implementations.
<wgrant> stub: Right, the recent port of the classical job system to celery involves a regular task to reschedule jobs that remain pending but without a rabbitmq job.
<wgrant> We don't have pure-rabbit jobs.
<stub> Yer, that extra complexity.
<stub> Is that generic, or do we have to do this every time?
<stub> Guess you get it if you use the lp job system.
<wgrant> stub: It currently only support branch scan jobs, I believe, but it's pretty simple to genericise. I imagine it will be once it's fully deployed and tested for branch jobs.
<lifeless> wgrant: +1
<wgrant> lifeless: On the signing_key SQL?
<lifeless> yes, caveat being to follow the normal procedure :)
<wgrant> Of course, thanks.
<jam> lifeless: I thought we didn't run tests for code that wasn't launchpad proper. I'm getting a test failure on python-2.7 inside the launchpadlib test suite (in the egg)
<jam> and the plot thickens. As near as I can tell, you run the lplib tests by running "python setup.py test", but that suite doesn't load the doctests... so they are only being run by launchpad bin/test, and it is in an egg, so we need a new version of launchpadlib, a new release, and then updating the versions.cfg file
<jam> bac: I noticed you are prominent in launchpadlib development. Can I ask you some questions?
<czajkowski> jam: dont think he's on today
<jam> czajkowski: thanks
<jam> I imagine neither is benji, or anyone else in the US
<czajkowski> jam: as is most of USA
<wgrant> jam: Some launchpadlib tests only run as part of the Launchpad test suite, since they need to test against Launchpad.
<Bert_2> Hi, I'm about to do some changes to my newly rockfuel'ed launchpad, now I am to make some config changes to prevent launchpad from going into dev mode, I was hoping to use this page: https://dev.launchpad.net/WorkingWithProductionConfigs but while it says it describes editing configuration it actually doesn't seem to explain anything else apart from repo and branch stuff (which are not as important as getting out of devmode), so help ?
<jam> wgrant: so if I have a fix for a launchpadlib test, I have to get it committed into launchpadlib trunk, then officially packaged and uploaded, then update versions.cfg, right?
<jam> but I can't run that test outside of launchpad itself...
<jam> wgrant: do you just hack the code in the egg directory directly?
<wgrant> jam: You're correct on all counts.
<jam>  /cry
<wgrant> Bert_2: That page mostly documents how Canonical LP engineers should work with our production configs. The closest thing to a public production config example is the old configs from before the open-sourcing, in r8666. Are you aware of the trademark and copyright restrictions that apply to the Launchpad name and images?
<Bert_2> wgrant: I'm not fully aware of that no, I thought I should only remove the launchpad logo
<wgrant> https://dev.launchpad.net/LaunchpadLicense
<Bert_2> k, so basically if I lose the logo and icons and ask permission to use the name launchpad I'm fine ?
<jam> and the hole deepens. The specific contents of the config file is set by lazr.restful.authorize_auth.OAuthAuthorizer...
<wgrant> Bert_2: You'd be best off changing the name.
<wgrant> jam: You need to change the config file?
<jam> wgrant: we have a doc test that asserts the exact bytes of the content
<jam> and in py2.7 it changed 2 things (a blank entry now has a trailing space, and the order is changed)
<jam> I'm guessing the ordering is defined by iter(dict)
<jam> and that 2.7 just does it differently.
<jam> It was a very brittle test, waiting to break
<Bert_2> wgrant: and can that be done through the configfile or am I forced to change the name by hand in every python file ?
<wgrant> jam: Oh, lovely. Can you just change the test to be a bit more flexible?
<jam> wgrant: I was thinking to put a "sorted()" around it, yeah.
<wgrant> Bert_2: There's no config option. The one other production instance I know of basically ran sed over all the python files and templates.
<jam> I'm just struggling to even run it in isolation :). I think I at least have a plan of attack.
<wgrant> jam: Heh
<Bert_2> wgrant: alright, that can be done, and how about going from development to production mode, how's that done, cause the wikipage isn't all that clear :S
<wgrant> Bert_2: You'll see the existing public configs under configs/
<wgrant> You select a config to use with the LPCONFIG environment variable. It defaults to 'development'.
<jam> wgrant: so I think I'm going to monkey patch ConfigParser.SafeConfigParser in part of a doc test. That sounds sane, right? :)
<cjwatson> Any objection to me updating the copyright year in LaunchpadLicense?
<wgrant> cjwatson: I was going to do that myself, but feel free :)
<cjwatson> Done
<jam> (no monkey's will be patched, I swear)
<wgrant> Bert_2: I'd copy the development config, and fix up the paths and domains. In launchpad.conf of your new config, remove the 'devmode on' line. Most of the settings are in launchpad-lazr.conf.
<wgrant> jam: If it works...
<Bert_2> wgrant: alright, so if I change paths, domains, the devmode line and then the usual emailaddresses and numbers for question deprecation etc. I'm fine ?
<ivory> gmb: could you take a look at this? https://code.launchpad.net/~ivo-kracht/launchpad/bug-921901/+merge/113234
<gmb> ivory, Sure - thought the topic's out of date :)
* gmb changed the topic of #launchpad-dev to:  http://dev.launchpad.net/ | On call reviewer: - | Firefighting: - | Critical bugs: 4.0*10^2
<wgrant> Bert_2: IF you want it to send email you'll also have to reconfigure a couple of things. zcml/package-includes/mail-configure-normal.zcml redirects all email to root@localhost, and you might want to enable the immediate_mail.send_email in launchpad-lazr.conf as well. You might also want to do away with the sampledata, and instead create a clean DB using the scripts in https://code.launchpad.net/~wgrant/launchpad/bootstrap-db-from-scratch
<Bert_2> wgrant: awesome, I thought going out of devmode would give me a clean db, so thx for saving me the frustration :D
<wgrant> Bert_2: Changing an appserver configuration file fortunately won't erase your database :)
<Bert_2> wgrant: I thought the database was generated by a python script that read the config, but I didn't really take the time to read the sourcecode, I've barely been out of exams and this development platform is rather important for our student's body
<jam> wf
<jam> wgrant: do you have any experience rolling out an updated EC2 AMI?
<jam> I found EC2Test on the wiki, I'll try following that one
<wgrant> jam: Why do you need to?
<wgrant> But yes, that page is correct.
<jam> wgrant: we want to get the test suite running w/ python-2.7
<jam> so that we can get production moved to precise afterwards
<wgrant> jam: Not until we've scheduled it on production we don't.
<jam> wgrant: I'm working with flacoste to do... 1) upgrade db-devel to run py2.7 along with ec2-test running py 2.7
<jam> and then leave devel running 2.6
<jam> once we have both passing
<jam> then we can upgrade prod
<jam> but we can't upgrade prod until we know it works, and have a clean test suite.
<jam> he wants ec2-test to fail early rather than going into testfix on db-devel, and buildbot will run on devel w/ 2.6 before it goes to qastaging
<wgrant> I'm not sure it's a great idea to upgrade ec2 unless we know we're going to be upgrading production within a very few weeks, or we're going to be in trouble when people start using 2.7isms and breaking buildbot.
<jam> wgrant: so we need a migration path, which I was asked to work on. Certainly each step gets reviewed, etc.
<wgrant> Sure.
<wgrant> Let's not just actually deploy the 2.7 image to everyone until we have a schedule worked out.
<flacoste> wgrant: the plan is to upgrade quickly, really not more than a few weeks
<rick_h_> adeuring: ivory https://plus.google.com/hangouts/_/3dd64d7a6ec2e029e3e9abfc99a72a95ff0bef6c?authuser=0&hl=en-US
<jam> mgz: did you see my python-upgrade bug? I submitted a fix for it to launchpadlib
* abentley changed the topic of #launchpad-dev to:  http://dev.launchpad.net/ | On call reviewer: abentley | Firefighting: - | Critical bugs: 4.0*10^2
<mgz> I saw the mp
<jam>  it will take a bit to finish that one off, but that was the last failure after manually merging your branches.
<jam> mgz: did they all land in trunk?
<mgz> guess I can review that? not sure what the funny lp reviews system applies to
<mgz> ^jam, yup landed now and tickets moved horizontally with lp2kanban
<jam> mgz: what is the approval process for launchpadlib code?
<mgz> jam: no idea, my experience has been "bug someone after mp has been forgotten by everyone"
<mgz> or was that a lazr thing...
<jam> mgz: I think it is a more-than-one-projects thing
<jam> but everyone I've seen in launchpadlib on lp's "active in this project" is asleep or on holiday. so I might try later.
<jam> eg, tomorrow
<bac> jam: have a look at https://dev.launchpad.net/HackingLazrLibraries
<jam>  /wave bac, not doing July-4 stuff yet?
<bac> jam: it has good stuff, especially if you need to make a release
<bac> bah, our founding fathers would be embarrassed by our slothful coworkers.  :)
<mgz> we probably don't need a release, if we can just bump the lp dep to a specific revno
<jam> mgz: it is an egg, not a sourcecode
 * mgz is not sure if that's a done thing though
<jam> bac: so who actually commits stuff to launchpadlib trunk?
<bac> jam: i think the short answer is 1) make a dev branch, don't work in your copy of trunk, 2) make a MP and get it reviewed, 3)  merge to trunk and commit with a good message including [r=reviewer], 4) push.  you'll then need to decide on whether to make a release, create an egg, and bump versions.cfg in a LP branch.  probably best to just read that long wiki page, now that i think about it.
<jam> (a member of the lazr-developers team)
<jam> bac: oh, I know all about that stuff :)
<jam> I've done 1-3
<jam> "merge to trunk", you just do it directly?
<jam> or it goes through pqm (the page says pqm)
<bac> jam: yep
<bac> no bots
<bac> oh really?  ok, my memory is faulty then
 * bac defers to the wiki
<jam> I'm not sure I'm in lazr-developers
<jam> I'm in launchpad
<jam> (launchpad-dev? whatever that group is)
<jam> bac: looks like I can commit straight to lp:launchpadlib
<bac> jam, you should be in this group, indirectly: https://launchpad.net/~lazr-developers
<bac> yep
<jam> bac: it looks like 'bin/test' doesn't run the doctests either. Only running "bin/test" in launchpad itself, but not in launchpadlib
<bac> yes, iirc getting the test to run is done via a LP branch
<ivory> abentley: could you review this for me? https://code.launchpad.net/~ivo-kracht/launchpad/bug-562532/+merge/113414
<abentley> ivory: sure.
<abentley> ivory: How does that remove a page request?
<ivory> abentley: allthe other edit buttons are sprite icons, only those were <img> tags
<abentley> ivory: I'm used to sprites looking like '<span class="add sprite">'.  Why is this one different?
<rick_h_> abentley: it's removing the request to: https://launchpad.net/@@/edit via the src of the <img> tag. and referencing the sprite in the CSS.
<abentley> rick_h_: ivory already answered that.  Do you know what's up with this <button class="lazr-btn yui3-activator-act">?  I'm used to sprites looking like '<span class="add sprite">'.
<jam> bac: I cannot register with pypi, as my user doesn't have perms. I've done the upload and changes to trunk.
<jam> bac: do you have rights and can 'bin/buildout setup . register' for me?
<ivory> abentley: i just copied it, i cant really tell you why it lookslike that ...
<rick_h_> abentley: ah no, I'm guessing he copied it from somewhere.
<rick_h_> ivory: where did you grab it from?
<ivory> rick_h_: from the other implementaions of that icon
<jam> bac: I'm leaving for end-of-day now. If you get a chance, please do. Or poke benji to do so? I'll try to address it tomorrow if it hasn't happened yet.
<abentley> ivory: Okay, if there's no reason not to use "sprite edit", I think we should do it that way.
<rick_h_> abentley: adeuring heads up, afk for a sec while I pick up the family from the parade. brb
<adeuring> rick_h_: ack
<bac> jam: ok, i'll look at it
<czajkowski> bac: thought you weren't around today ?
<bac> czajkowski: i decided to work today and bank a swap day.
<beuno> how independent of you
<bac> beuno: yeppers
<czajkowski> bac: cool
<ivory> abentley: wouldn't the icon be on the other side if "sprite edit" is used?
<ivory> abentley: additionally i should mention that the <button class="lazr-btn yui3-activator-act"> is already used in this template
<abentley> ivory: I wouldn't think so.  In any case, "lazr-btn yui3-activator-act" does not mean "edit icon" to me, and I doubt it would for most LP developers.
<bac> jam: lplib 1.10.1 uploaded to pypi.  thanks for the fix!
<czajkowski> bac: abentley would one of you mind lookinat https://bugs.launchpad.net/launchpad/+bug/1020983  I don't know about receipes . please.
<_mup_> Bug #1020983: Recipe default text versioning is wrong <Launchpad itself:New> < https://launchpad.net/bugs/1020983 >
<bac> czajkowski: he may have a point in that bug.  i'd mark it triaged/low.
<czajkowski> bac: ack
<lifeless> jam: wgrant: pass --develop to buildout to point it at the launchpadlib source tree and you can edit it in situ - it won't use the egg.
<abentley> czajkowski: I know about bzr, but not about packaging.  IMO, he's wrong-- the version number is unique, can't be confused with a release version number, and that's all that's required.
<lifeless> abentley: there is another constraint you have missed, and that the bug filer hasn't called out either.
<lifeless> abentley: but I think its what motivates the filing; the packages produced by the recipe should version higher than the package they are derived from but lower than the next version of that package.
<lifeless> I'm not entirely sure his proposals are much better at that, though they will handle some cases better.
<abentley> lifeless: I think this is a distributed numbering problem that can't be completely solved.
<abentley> lifeless: It's quite possible that a point release "should" version lower, and I think it's possible for reasonable people to disagree on which should version lower.
<abentley> lifeless: Depending on the nature of the branch, of course.
<lifeless> yah
<lifeless> I guess I'm mainly advocating that we get a clear understanding of what the default intends to achieve and document that somewhere discoverable.
<lifeless> then, if the default is mismatched with our intent we can change it, and bugs that want to change our intent we can actually reason about
<abentley> lifeless: that makes sense.
<abentley> lifeless: So if we grant that we want to sort later than the version of the packaging we're based on, that suggests we should use debversion, not debupstream.
<lifeless> I agree.
<lifeless> I wonder if we wrote our original intent down, perhaps in a doctest or something.
<abentley> lifeless: It was me and james_w flailing around, IIRC.
<lifeless> hah :)
<abentley> lifeless: I believe we were using debversion originally, so there may be a related MP.
<lifeless> or something in bzr log
<lifeless> ok, time to prep for my joy of joy 3 1/2 hours of meetings.
<abentley> lifeless: bug 652109 inspired this self-reviewed proposal: lp:~thumper/launchpad/recipe-default-version into lp:launchpad and I don't think there was any discussion of whether debversion would be more appropriate as a default.
<_mup_> Bug #652109: Default Recipe text should contain {debupstream} <lp-code> <qa-ok> <recipe> <Launchpad itself:Fix Released by thumper> < https://launchpad.net/bugs/652109 >
<abentley> lifeless: sorry, proposal url: https://code.launchpad.net/~thumper/launchpad/recipe-default-version/+merge/41663
<lifeless> abentley: cool archaeology :)
<lifeless> so, we're doing a drunkards walk more or less; might be time to take another step :)
<lifeless> anyhow, I have to run - sorry.
<jam> bac, lifeless: Unfortunately, it appears that someone a
<jam> made a launchpadlib 0.10.0 which was not landed in launchpad itself
<jam> because it breaks the test suite
<jam> (issues with doctests expecting a reply_token? to be valid and not None)
<jam> so we'll need an 0.10.2 after I can sort it out. It looks like benji did the 0.10.0, I wonder if he meant to land it, but found out late it broke the doctest suite because it doesn't run via 'bin/test' in launchpadlib alone.
<jam> lifeless: i also haven't tracked down how you specify --develop to bin/buildout, since it seems to want a command first.
<abentley> lifeless: https://dev.launchpad.net/Code/RecipeBuildDebversioning
<lifeless> jam: https://dev.launchpad.net/HackingLazrLibraries?highlight=(buildout:develop)
<jam> lifeless: thanks
<lifeless> applies to LP itself too
<cjwatson> I noticed launchpadlib test failures somewhere a few days ago, which then went away.  Is it possible 0.10.0 landed briefly and then was reverted?
<cjwatson> (I'm afraid I don't remember the failures.)
<lifeless> cjwatson: is sbsign intended to be 64-bit only? what about arm...?
<cjwatson> lifeless: not this month's problem
<lifeless> cjwatson: is that 'no its not' ?
<cjwatson> the interface doesn't mandate 64-bit-only, it's just that that's all that's implemented right now because that's the urgent bit
<cjwatson> or rather amd64-only
<lifeless> gotchya
<cjwatson> (I didn't write sbsign, just did the integration - but it's all the same binary format for UEFI so I'd be fairly surprised if it weren't basically trivial to adjust)
<cjwatson> I'm told Steve has a patch to make it handle i386, at least, but I didn't have that to hand when doing the QA
<lifeless> I'm just being nosy
<lifeless> 100% curiosity
<StevenK> wgrant, wallyworld: https://code.launchpad.net/~stevenk/launchpad/drop-populate-branch-aag/+merge/113467
<StevenK> wgrant: And it seems BranchCollection makes use ClassAlias, so access_{policy,grants} might have to be added to the model.
#launchpad-dev 2012-07-05
<wgrant> StevenK: Indeed :(
<wgrant> StevenK: It's a bit unfortunate that Storm doesn't let us have lazy columns.
<wgrant> And that review is done.
<StevenK> wgrant: How do I declare an array col?
<wgrant> StevenK: List
<wgrant> List(type=Int()) is probably what you want
<huwshimi> wgrant: Did you fix the search speed?
<huwshimi> Well, whoever did, it's amazing!
<rick_h_> huwshimi: adeuring worked on that recently
<rick_h_> the fulltext search stuff?
<huwshimi> rick_h_: Ah right. It's so good!
<huwshimi> It might just be the fastest action on Launchpad...
<StevenK> wgrant: I have a bunch of failures because the test suite is assuming the owner can access private branches
<wgrant> huwshimi: Bug searches? Yeah, they're a few times faster now, it's nice.
<wgrant> The speed improvements are mine, the less obtuse text search is Abel's.
<huwshimi> wgrant: Well, thanks a lot!
<wgrant> It's still about twice as slow as it's meant to be, but I'll hopefully work that out some time this year :)
<huwshimi> wgrant: I didn't know we could load pages that quickly, already.
<wgrant> Heh
<huwshimi> Just with the roundtrip, template render etc.
<StevenK> wgrant: Considering throwing in an owner clause to get_branch_privacy_filter()
<wgrant> StevenK: Shouldn't the owner have a grant on creation, because it has a subscription automatically?
<lifeless> huwshimi: hah
<lifeless> huwshimi: its not all doom and gloom; lots of hard work tho
<wgrant> huwshimi: The minimum server-side render time for a full page with auth and template rendering is around ~200-250ms. We really need to fix our stack.
<StevenK> wgrant: Hah, I've just had a thought -- I wonder if these test failures will die due to the sampledata changes in drop-branch-populate-aag.
<wgrant> And we also need slaves in the US :)
<wgrant> StevenK: Indeed, probably!
 * StevenK wields make schema to find out
<huwshimi> lifeless: Well, it's paying off!
<StevenK> huwshimi: The sharing pages are among the quickest in LP, too
<StevenK> wgrant: Running makeBranch(owner=owner, information_type=USERDATA) in a harness gives access_grants = {}
<wgrant> StevenK: I suggest working out why.
<wgrant> StevenK: It's possibly because it transitions the information type after creating the subscription, and the transition doesn't create grants.
<StevenK> wgrant: IBranchNamespace.createBranch will only subscribe if there is a BVP
<StevenK> Which doesn't help much for the testsuite
<StevenK> We can subscribe the owner if information_type is not None in the factory
<wgrant> Ah
<wgrant> Not only does it not help for the test suite, but it also ruins all of our plans.
<StevenK> Well, it pulls teams from the BVP and subscribes them all
<wgrant> StevenK: Huh?
<wgrant> That's not the relevant bit here.
<wgrant> StevenK: I've just checked, and even public branches get a default subscription for the owner.
<wgrant> The BVP subscription stuff is for granting others access.
<wgrant>         branch.subscribe(
<wgrant>             self.owner,
<wgrant> Right after the BVP sub.
<StevenK> Ah
<StevenK> Then why is branch.access_grants = {} :-(
<wgrant>         if information_type is not None:
<wgrant>             naked_branch.transitionToInformationType(
<wgrant>                 information_type, registrant, verify_policy=False)
<wgrant> Probably because of that.
<wgrant> See the         if information_type in PRIVATE_INFORMATION_TYPES:
<wgrant> section of Bug.transitionToInformationType
<wgrant> It's probably missing from Branch.transitionToInformationType
<StevenK> wgrant: And missing_subscribers = set([who, self.owner]) ?
<wgrant> StevenK: That's a difficult question.
<wgrant> StevenK: I don't think so.
<wgrant> The owner certainly shouldn't be implicitly subscribed.
<wgrant> The actor probably should be.
<StevenK> wgrant: Adding a missing_subscribers = set([who]) and subscribing still ends up with access_grants = {}
<wgrant> StevenK: Did you add the ensureAccessGrants bits?
<StevenK> wgrant: I have now, still {}
<StevenK> Woot, access_grants = {243652}
<wgrant> StevenK: What'd you change?
<StevenK> wgrant: Moved the ensureAccessGrants() section after we set information_type and call _reconcileAccess()
<wgrant> Ah, yes, that's fairly critical :)
<StevenK> wgrant: Is test_owner_member_sees_own_branches in test_branchcollection complete crack?
<StevenK> It creates a private branch by stacking on a existing private branch, then unsubscribes the owning team and then asserts the team owner can still see the branch.
<wgrant> StevenK: Right, that test no longer makes sense, as we've changed the rules.
 * StevenK bins it
<StevenK> wgrant: The only failure left is test_initial_branches_does_not_contain_private_dev_focus_branch in code/browser/tests/test_product.py
<wgrant> StevenK: I would suggest making it not fail.
<StevenK> Obviously.
<StevenK> I just don't get why it does fail, I thought CIV() with no principal was anonymous
<lifeless> Its been replaced.
<lifeless> With CV()
<StevenK> *facepalm*
<StevenK> Oh, DUH
<StevenK> The bloody tests calls login() in the method to create the branch
<StevenK> wgrant: So it looks like this branch is creating a private development focus for a product and then assuming it can't be seen.
<StevenK> s/branch/test/
<wgrant> StevenK: I got that idea from the name, yaeh.
<StevenK> Considering replacing the login() call with 'with person_logged_in(...)' but that may have other knock-on effects.
<wgrant> The bigger question is perhaps why this only makes it break now.
<StevenK> It calls check_permission on the dev focus, which calls visibleByUser() and I have changed that method a lot
<wgrant> Right.
<wgrant> But that suggests that the login() call is not the problem.
<wgrant> Or the test was buggy before.
<StevenK> Tossing a login(ANONYMOUS) before the call to create_initialized_view() fixes the test.
<StevenK> I'm not sure how this test ever worked properly.
<wgrant> StevenK: Ah
<wgrant> I just read the code
<wgrant> Now it is clear
<wgrant> StevenK: It explicitly logs in as the product owner, not the branch owner.
<wgrant> StevenK: Recall that product owners now have an APG by default.
<StevenK> Haha, which we now allow
<StevenK> Due to the visibleByUser() changes
<wgrant> Precisely.
<wgrant> So you can either log in as another user or remove the APG
<StevenK> login(ANONYMOUS) it is then
<StevenK> wgrant, wallyworld: https://code.launchpad.net/~stevenk/launchpad/new-branch-visibility/+merge/113484
<wgrant> StevenK: Did you look into using BranchCollection to implement Branch.visibleByUser?
<StevenK> Not really, I didn't see how it could be a win
<wgrant> It removes code duplication.
<StevenK> wgrant: By calling IAllBranches.visibleByUser?
<wgrant> StevenK: BranchCollection sadly doesn't provide a means to filter by ID or URL, so it may not be doable at this stage.
<StevenK> wgrant: branch_filter_expressions=[Branch.id == self.id] ?
<wgrant> StevenK: Oh, I lie, getUtility(IAllBranches).withIds(self.id).visibleByUser(user).is_empty() should work
<StevenK> wgrant: http://pastebin.ubuntu.com/1075921/ ?
<StevenK> Hm, we can probably refactor further, and feed a list of the current branch and all branches stacked on it
<wgrant> Well
<wgrant> It's not clear whether we should preserve that behaviour at that level.
<StevenK> It's current behaviour at least
<StevenK> wgrant: http://pastebin.ubuntu.com/1075926/
<StevenK> Sigh, modulo a one character typo
<wgrant> StevenK: That doesn't make much sense.
<wgrant> The visibility of a branch might depend on the branches it's stacked on.
<wgrant> It certainly does not depend on the branches that are stacked on it.
<StevenK> That was the intent
<StevenK> wgrant: Does http://pastebin.ubuntu.com/1075932/ make you hate me?
<wgrant> StevenK: That probably won't work, since it's unlikely you can check the stacked_on of a branch you don't have access to.
<wgrant> You probably need to preserve the original behaviour.
<wgrant> Just replacing _checkBranchVisibleByUser with the branchcollection thing
<wgrant> s/behaviour/structure/
<StevenK> Yeah :-(
<StevenK> I suspect WithIds wants a list, though
<StevenK> Hmm, not sure
<StevenK> wgrant: Diff updated.
<wgrant> StevenK: r=me, but please do some data checks on prod before landing
<StevenK> wgrant: Such as checking all private branches have some AAGs? Or what did you have in mind?
<wgrant> StevenK: Actually, why not inline the branchcollection usage into visibleByUser?
<StevenK> Hm, good point.
<wgrant> _checkBranchVisibleByUser is now one statement, and it only has one callsite
<StevenK> Let's do that.
<wgrant> Might as well inline
<wgrant> Save another few lines.
<wgrant> Get you to -15 or so
<StevenK> -17
<StevenK> -18, in fact, because I'm an idiot
<StevenK> wgrant: Which is fantastic, because I've wanted _checkBranchVisibleByUser dead for a while.
<StevenK> wgrant: What data checks did you have in mind?
<wgrant> StevenK: The ones I was thinking of don't really work
<wgrant> So let's JFDI
 * StevenK tosses it at ec2
<adeuring> good morning
<StevenK> _StringException: Text attachment: garbage
<StevenK> ------------
<StevenK> [<storm.databases.postgres.PostgresConnection object at 0x168662d0>]
<StevenK> wgrant: ^ have you seen that on ec2?
<wgrant> StevenK: buildbot saw that twice last week.
<wgrant> It's spurious.
<StevenK> But it will mark the branch as failed, which is annoying
<cjwatson> Any thoughts on a nice way to export the list of custom files in a PackageUpload?  lp.soyuz.scripts.queue currently does:
<cjwatson>         for queue_custom in queue_item.customfiles:
<cjwatson>             self.display("\t | * %s Format: %s"
<cjwatson>                          % (queue_custom.libraryfilealias.filename,
<cjwatson>                             queue_custom.customformat.name))
<cjwatson> I exported URLs for the library files as PU.customFileUrls(), but maybe it should've been .customFiles() that returns a list of (URL, format) tuples or something ...
<wgrant> That might be best, indeed.
<cjwatson> (It's on devel exactly so that we can change it ;-) )
<wgrant> Yup
<cjwatson> Hmm, OTOH that would mean every 'queue info' would have to make a customFiles call just to find out whether there were any
<cjwatson> Maybe it would make more sense to fold it into getBinaryProperties
<danilos> heya, I've got 3 MPs up for different workitem-related stuff
<danilos> if anyone feels review-happy, there you go :)
<jam> gmb: for the addSkip thing
<jam> should it be falling back to addSuccess?
<gmb> Um.
<jam> AIUI that is what testtools in 2.6 is doing
<gmb> jam. A thousand botherations. Yes, it should. Good catch (and that'll teach me for self-reviewing)
<jam> otherwise, I think you break the subunit stream
<jam> in the way that it broke for 2.7
<gmb> jam, I don't - it handles the skip properly
<gmb> It just doesn't call up to the parent class.
<gmb> Actually, hmm..
<gmb> jam, They way it is now, it skips properly when using --subunit
<jam> it is possible that self.options.output.test_skip is doing the right output.
<gmb> jam, It is... it's just not incrementing the counter on unittest.TestResult, which doesn't exist.
<gmb> (For py 2.6)
<jam> gmb: ah ok. the other thing to watch out for is double-counting, but yeah, I think calling something on the super() class is useful.
<jam> gmb: we just need a # py 2.6 comment there, so we know we can nuke all that code in the future
<jam> :)
<gmb> Right :)
<gmb> Amusingly, the zope.testing test suite breaks horribly under 2.6.
<gmb> *sigh*
<gmb> But I'll go and make that change so that we fall back.
<jam> gmb: so you can't actually run the zope.testing suite in 2.6? ugh
<gmb> Exactly.
<gmb> You can run bits of it
<gmb> But not all of it.
<gmb> So "assertIn()" for example has to not be used :/
<jam> mgz: I see a bunch more of your patches have landed. kudos
<mgz> jam: I shall run the kanban update again and see if they're done-done
<jam> mgz: well, it hasn't been deployed to prod
<jam> so I think it shouldn't be
<mgz> yup, Deployment::Ready is the state
<rick_h_> jcsackett: just soyuz and translation left :) https://code.launchpad.net/~rharding/launchpad/registry_yui35/+merge/113407
<rick_h_> jcsackett: I tried to comment the actual changes vs moving as much as possible so it's a long MP, but itemized to help.
<cjwatson> wgrant: https://code.launchpad.net/~cjwatson/launchpad/queue-api-custom-files/+merge/113574 ?
<deryck> adeuring, https://plus.google.com/hangouts/_/040bd7afe73bdcaf379359f9eb707488709b7abd?authuser=0&hl=en
<deryck> rick_h_, https://plus.google.com/hangouts/_/040bd7afe73bdcaf379359f9eb707488709b7abd?authuser=0&hl=en
* jcsackett changed the topic of #launchpad-dev to:  http://dev.launchpad.net/ | On call reviewer: jcsackett | Firefighting: - | Critical bugs: 4.0*10^2
<danilos> jcsackett, hi, I've got a couple of MPs up, can you please take a look at them on +activereviews?
<danilos> jcsackett, fwiw, I am around for any questions in the next few hours
<jcsackett> danilos: dig. i'm working through the queue.
<danilos> jcsackett, cool, thanks
<nigelb> Did the bzr team get merged back into LP?
<nigelb> Or something of that sort.
<mgz> yeah, of that sort.
<rick_h_> abentley: so all three JS tests just passed in yui3.5 w/o even opening them.
<abentley> rick_h_: Good news: all passed.  Bad news: all = 3. :-)
<rick_h_> hah
<rick_h_> well three files
<rick_h_> looks like 28 actual tests
<abentley> rick_h_: Well, that's good then.
<czajkowski> mpt: where would we be without you :0
<czajkowski> :)
<rick_h_> jcsackett: quick one for you for a change when you get a chance: https://code.launchpad.net/~rharding/launchpad/fix_attr_quotes/+merge/113610
<mpt> In a world that was slightly more confusing but otherwise just the same
<czajkowski> and no spam :)
<ivory> jcsackett: i could use a rievew: https://code.launchpad.net/~ivo-kracht/launchpad/bug-728129/+merge/113612
<jcsackett> ivory: your up next as soon as i finish my current.
<ivory> jcsackett: thank you, chances are that i'm not online anymore by then since it's quite late now and it's my last day of my internship. questions go to adeuring then :)
<cjwatson> $ bzr damage
<cjwatson> Using submit branch /home/cjwatson/src/canonical/launchpad/lp-branches/devel
<cjwatson> Healing: 2007 lines of code
<cjwatson> ^- I like the look of this
<cjwatson> And I can even land that after my three queue-api-* MPs are reviewed and deployed ;-)
<rick_h_> jcsackett: ty much for the trio of reviews today
<rick_h_> cjwatson: LoC credit coming down the pipe?
<cjwatson> rick_h_: stand back
<rick_h_> cjwatson: LoC assassnator!
<rick_h_> escept with better spelling and all that
<adeuring> jcsackett: could you please do a "follow-up" review of Ivo's MP: https://code.launchpad.net/~adeuring/launchpad/bug-728129/+merge/113631
<jcsackett> adeuring: cool, can you verify if the test_decoratedresultset.py file is even needed? it look like it's to collect the doctest, but that doctest already exists and runs without thie test_*py file.
<adeuring> jcsackett: yes, we need some way to get the doc test in the tests/ directory executed. WIthout this file, the doc test is simply ignored.
<adeuring> jcsackett: note that there is another fle decoratedresultset.txt, but in the directory doc/, which is at oresent executed.
<jcsackett> adeuring: dig. r=me then.
<adeuring> jcsackett: cool, thanks, also in Ivo's name!
<abentley> lifeless: Have you seen https://dev.launchpad.net/Code/RecipeBuildDebversioning ?  The more I think about it, the more it seems like a recipe build should supersede any other packaging by default, because if you don't want the recipe version, you remove the package & PPA.
<lifeless> I saw you make it yesterday, I think.
<lifeless> Your logic makes sense to me. Its complicated by apt/dpkg behaviour but not something we can really solve.
<lifeless> by which I mean, being a slightly higher version is a good idea. But removal is more complicated (and there are special tools to use to do that).
<abentley> james_w: Could you comment on https://dev.launchpad.net/Code/RecipeBuildDebversioning ?
* jcsackett changed the topic of #launchpad-dev to:  http://dev.launchpad.net/ | On call reviewer: - | Firefighting: - | Critical bugs: 4.0*10^2
<lifeless> bbiab
<cjwatson> jcsackett: thanks for the cluster of reviews
<jcsackett> cjwatson: you're welcome. :-)
 * cjwatson runs three parallel ec2 lands, what could possibly go wrong
<jcsackett> cjwatson: generally, that all works fine. as long as your branches weren't secretly dependent on each other. :-P
<cjwatson> I don't *think* they were
<cjwatson> will be a fun QA opportunity tomorrow though
<StevenK> wgrant, wallyworld: http://pastebin.ubuntu.com/1077299/
<StevenK> FAILED (id=1, failures=662 (-3231), skips=2)
<StevenK> Progress!
#launchpad-dev 2012-07-06
<StevenK> wgrant: Looks like there are 18 or so legitimate failures, most of them related to query counts.
<StevenK> And one very strange one where the sharing service seems to generate a completly bong query.
<StevenK> wallyworld: ^ Let me get you a traceback
<wallyworld> ok
<StevenK> wallyworld: http://pastebin.ubuntu.com/1077406/
 * wallyworld looks
<wallyworld> StevenK: so, you have code that i don't, but the callto getPeopleWithoutAccess in trantitionToInfoTyoe in branch is being done with people=[]
<StevenK>             blind_subscribers = service.getPeopleWithoutAccess(
<StevenK>                 self, self.subscribers)
<wallyworld> so self.subscribers must be empty
<wallyworld> or so it seems
<StevenK>     + Subscribers: [<Person at 0x10071b50 eric (Eric the Viking)>]
<wallyworld> from the sharing service code:
<wallyworld>         # Find the people who can see the artifacts.
<wallyworld>         person_ids = [person.id for person in people]
<wallyworld> the onbly way person_ids can be [] is if people is empty
<wallyworld> so i'm not sure what's happening
<wallyworld> can you pastebin your code?
<StevenK>     + Subscribers: [<Person at 0xeed1e90 eric (Eric the Viking)>]
<StevenK>     + person_ids = [243652]
<StevenK> (It's a doctest, so pdb debugging makes me want to kill people.)
<wgrant> That's probably the wrong call.
<StevenK> Bah, so it is.
<wallyworld> ah, i may have found it
<StevenK> wallyworld: https://code.launchpad.net/~stevenk/launchpad/new-branch-visibility/+merge/113484
<wallyworld> there's also clauses in the result set In(TeamParticipation.teamID, policy_grantees)
<wallyworld> maybe policy_grantees is empty
<wallyworld> or artifact_grantees
<wgrant> That should be a subselect, shouldn't it?
<wgrant> So it won't appear as an empty sequence in the textual statement.
<wallyworld> perhaps, but it doesn't appear to be coded that way
<wgrant> It's a subselect
<wgrant>             In(Person.id, person_ids))
<wgrant> It has to be that line
<wallyworld> why not this line In(TeamParticipation.teamID, policy_grantees)
<StevenK> if not person_ids: return [] ?
<wgrant> Yeah
<wallyworld> or this one In(TeamParticipation.teamID, artifact_grantees)
<wgrant> wallyworld: No.
<wallyworld> they use the same construct
<wgrant> That is on teamparticipation.team, and it uses a subselect
<wgrant> Whereas the error is about person.id without a subselect
<StevenK> wgrant, wallyworld: http://pastebin.ubuntu.com/1077416/ is the full statement
<wgrant> launchpad_dev=# SELECT COUNT(*) FROM branchsubscription WHERE branch = (SELECT id FROM branch WHERE unique_name='~name12/firefox/main');
<wallyworld> yes, i have read the code again and person ids doesn't use the sub select
<wgrant>  count
<wgrant> -------
<wgrant>      0
<wgrant> StevenK: There is the problem
<wgrant> (1 row)
<wgrant> StevenK: So you'll want to fix the sharing service to not crash when given no people, but you might also need to fix sampledata.
<wallyworld> i don't think the service should be called with no people
<wallyworld> if no people, don't call it
<wallyworld> but StevenK said above there were people
<wallyworld> <StevenK>     + Subscribers: [<Person at 0xeed1e90 eric (Eric the Viking)>]
<wallyworld> <StevenK>     + person_ids = [243652]
<StevenK> Different call, as wgrant surmised
<wgrant> When a statement from a human is in conflict with a traceback, disregard the human statement :)
<wallyworld> ok. i believed what i was told :-)
<StevenK>         authorised_people = [self.branch.owner]
<StevenK>     Unauthorized: (<Branch u'~name12/firefox/main' (1)>, 'owner', 'launchpad.View')
<wallyworld> so, the design of getPeopleWithoutAccess was to be called with a non empty list
 * StevenK stabs things
<wallyworld> i guess i should have added an assert
<StevenK> Right, it changes the information type, reconciles access, tries to render Branch:+index, and fails hard because no one has access to the branch.
<wallyworld> that also happended at some point with bugs
<wallyworld> i think reassigning a private bug
<wallyworld> i can't recall the solution, perhaps redirect to the project page with a message i think it was
<StevenK> wgrant: The branch in question is PUBLIC, I should create a branchsubscription for the owner anyway?
<wgrant> StevenK: All new branches have a subscription created for the owner.
<wgrant> They can remove it, of course.
<wgrant> But the sample data is a little anomalous.
<StevenK> Right, so let's create a subscription
 * wallyworld -> buy coffee
 * mwhudson stabs the branch vocabulary
<lifeless> stub: hi
<lifeless> stub: can we catch up soon ? [after morning cofee :)]
<stub> lifeless: hi
<stub> lifeless: Coffeed up already, been here a while; just offline on IRC
<lifeless> cool, let me just change rooms
<lifeless> stub: skype or g+; just sent you a g+ invite but skype is running too ... whatever you prefer
<StevenK> wgrant: So, query counts?
<wgrant> StevenK: Can you paste a failure?
 * StevenK fixes another test by use of rSP
<StevenK> wgrant: http://pastebin.ubuntu.com/1077548/
<wgrant> StevenK: Oh, I guess we still need the shortcircuit for public branches
<wgrant> To avoid tripping query tests like that :)
<StevenK> wgrant: In IBranch.visibleByUser?
<StevenK> Hm, I wonder if this evil hack fixes it
<wgrant> StevenK: Yeah
<StevenK> wgrant: http://pastebin.ubuntu.com/1077551/
<wgrant> StevenK: Exactly.
<StevenK> Wow, you approving of code I wrote in anger. :-P
<StevenK> wgrant: I'm also getting a wierd failure in lib/lp/registry/browser/tests/poll-views_0.txt
<wgrant> Oh?
<StevenK> wgrant: http://pastebin.ubuntu.com/1077553/
<wgrant> StevenK: Did that fail on ec2?
<StevenK> And locally
<wgrant> StevenK: I don't care about locally. ec2 is all that's interesting :)
<StevenK> Oh, hah. It failed on ec2 with [<storm.databases.postgres.PostgresConnection object at 0x168662d0>]
<wgrant> Yeah
 * StevenK ignores it at spurious 2.7 garbage
<StevenK> s/at/as/
<wgrant> Nah
<StevenK> wgrant: And thoughts on http://pastebin.ubuntu.com/1077556/ ?
<wgrant> StevenK: It's because of the local timezone.
<StevenK> Haha
<StevenK> Polls fail
<wgrant> Test fail
<wgrant> It uses datetime.now() and assumes launchpad will interpret it the same way.
<StevenK> Polls need to die horribly anyway
<spm> StevenK: wgrant: I note you both always want features to die horribly. have you considered maybe having them be tastefully removed instead?
<StevenK> spm: They don't deserve it.
<spm> that wasn't my implication
<spm> I might have been alluding to deep and disturbing psychological problems that are currently assailing two of my close colleagues. might.
<spm> there's probably room for a "trolling" in that description too
<StevenK> spm: Polls are special. They have been removed once, and then it turned out a few community teams required them to elect new members, so they were added back.
<spm> lovely
<StevenK> spm: You're not implying that wgrant and I are closet psychopaths? :-)
<spm> sure I was
<wgrant> Closet!?
<spm> point
<spm> it could be a very BIG closet
<StevenK> wgrant: Can has look at pastebin?
<wgrant> StevenK: Something may have been preloading subscriptions before.
<wgrant> StevenK: Check the old query log to see what's different
<StevenK> wgrant: bin/test -m test_pillar_sharing -t test_view_query_count still runs four tests, can I nail it down further?
<wgrant> StevenK: -t specifies a regex
<wgrant> Multiple regexes are ORed
<wgrant> So -t test_pillar_sharing.*test_view_query_count might work
<wgrant> Otherwise -t test_pillar_sharing.TestProductSharingDetailsView.test_view_query_count should be pretty specific
<StevenK> wgrant: http://pastebin.ubuntu.com/1077582/ is the test on devel, all 2500 lines of output
<wgrant> StevenK: I was more interested in just the interesting bit.
<StevenK> Yeah, I'm just looking how to get the statement recorder to print out what it saw.
<wgrant> StevenK: Reduce the limit to 1
<StevenK> Haha, that would do it too, I suppose
<StevenK> wgrant: Interesting bit: http://pastebin.ubuntu.com/1077587/
<wgrant> StevenK: Ah, I was looking at the wrong test
<wgrant> StevenK: Given that it's viewing +sharing, it's probably logged in as the product owner
<wgrant> StevenK: And you'll never guess who owns the branches that it creates.
<StevenK> Hahah
<wgrant> So the test is buggy
<wgrant> It verifies that the query count is constant, but only in a very exceptional case.
<StevenK>     def makeArtifactGrantee(
<StevenK>             self, grantee=None, with_bug=True,
<StevenK>             with_branch=False, security=False):
 * StevenK stabs wallyworld
<wallyworld> why?
<StevenK> For crimes against PEP8
<wallyworld> could be worse
<spm> yeah. could be stabbed for not providing cake to deserving webops staff
<StevenK> spm: Hahaha
<StevenK> wgrant: So what do you think the plan is?
<wgrant> StevenK: Well, see what the query count is if the owner is someone else. I suspect it'll be the same with the new and old code.
<wgrant> StevenK: That's an important bug, but not a regression.
<StevenK> wgrant: Modulo being distracted by Sarah, http://pastebin.ubuntu.com/1077619/
<wallyworld> spm: but i gave you some cake
<spm> yeah. could be stabbed for not providing non-virtual cake to deserving webops staff
<spm> fixed.
<wallyworld> if i had one of those "beam me up scotty" things i could send you some
<stub> ð
<StevenK> wallyworld: Money can be exchanged for goods, such as cake, and services, such as cake delivery to Canberra.
<wallyworld> StevenK: ssshhh
<wgrant> StevenK: I wonder if the user might be an admin, then.
<wgrant> StevenK: It doesn't explicitly log in, AFAICT
<wgrant> So it might be running as some user tha happens to leak from the factory.
<StevenK> Hahahaha
<StevenK> TestProductSharingDetailsView.setUp calls login_person(self.owner)
<wgrant> StevenK: Right, that's what I expected. But then I wonder why the test works even after you change the branch owner.
<StevenK> Well, it's only a query count
<StevenK> I've not looked deeply into what that view is doing
<wgrant> StevenK: Well, it's somehow doing an access check in 0 queries.
<StevenK> wgrant: Shall we just bump the test from 12 to 26 and move on for the time being?
<wgrant> StevenK: No. You need to work out why there are 0 queries initially.
<StevenK> There can't be zero. Is no understand
<wgrant> StevenK: http://pastebin.ubuntu.com/1077619/ shows 0 access checking queries.
<StevenK> wgrant: What about the last one that UNIONs against BranchSubscription?
<wgrant> StevenK: Right, it only finds visible branches, same as with your changes, but then it doesn't do any subsequent checks.
<wgrant> I can't see any code to prefill the authorization cache.
<wgrant> So I don't know what's going on
<StevenK> So where is there the actual view code? I've yet to find it
<wgrant> StevenK: lp.registry.browser.pillar
<StevenK> WTF? That code doesn't help at all
<wgrant> Hm?
<wgrant> PillarPersonSharingView is what you're looking for
<StevenK> Indeed, how does this even work?
<StevenK> wallyworld: Maybe you could help? How is PillarPersonSharingView managing to do its work in 0 queries?
<wallyworld> context?
<wallyworld> when did this start happening?
<StevenK> wallyworld: http://pastebin.ubuntu.com/1077619/ and http://pastebin.ubuntu.com/1077587/ and http://pastebin.ubuntu.com/1077556/
<wallyworld> StevenK: the one with 25 queries, what changes were done for that?
<StevenK> wallyworld: That branch is my new-branch-visibility branch that uses APG and AAG to determine branch access rather than subscriptions
<StevenK> wallyworld: The other two pastebins are against devel
<wallyworld> ok
<wallyworld> so the question you want to know i think is why changing the branch owner to some other person other than the pillar owner reduces the query count
<wallyworld> StevenK: do all the other tests pass, other than the query count one?
<StevenK> wallyworld: bin/test -vvt test_pillar_sharing == Ran 40 tests with 1 failures and 0 errors in 1 minutes 4.142 seconds.
<wallyworld> StevenK: so looking at the devel branch with owner=someperson, the query count has just been reduced by 1, no?
<wallyworld> ie down to 10
<StevenK> Which makes no sense to both wgrant and I
<wallyworld> what's your concern?
<wallyworld> (save me trying to grok the sql)
<wgrant> The issue is not that changing the owner reduces the query count.
<wgrant> It's that it doesn't increase it.
<wgrant> Somehow the old code is taking 0 queries to do security proxy access checks.
<wgrant> The new code is doing one per branch.
<wallyworld> i see that the old code has an extra person select
<wallyworld> at the end
<wallyworld> so where are the SP checks you are expecting?
<wallyworld> the work to load the data is primarily done by accesspolicy stuff at the db level
<StevenK> wallyworld: Where at the DB level?
<wgrant> wallyworld: It's not the DB level that's the problem
<wgrant> It fetches the branches from the DB, then the securityproxies need to verify launchpad.View
<wgrant> In devel at present the launchpad.View checks take no queries
<wallyworld> StevenK: in the queries performed by findArtifactsByGrantee
<wallyworld> wgrant: but the current code return no branches
<wallyworld> unless i am wrong
<wallyworld> the initial tests were written only for bugs
<wallyworld> since branches didn't support info type etc then
<wgrant> wallyworld: That's actually not a crazy thought.
<wgrant> Except that this should have broken last week when we started adding AAGs.
<wallyworld> why?
<wallyworld> if the tests didn't create any branches
<wallyworld> then nothing would have broken
<wgrant> Oh, did the test change?
<wallyworld> not afaik.
<wallyworld> the test never created any branches to check
<wallyworld> i think
<wgrant> Does so.
<wallyworld> there's a bunch of test modules that need to have extra tests for branhces added
<wgrant>     def makeArtifactGrantee(
<wgrant>             self, grantee=None, with_bug=True,
<wgrant>             with_branch=False, security=False):
<wgrant>             for x in range(0, 15):
<wgrant>                 self.makeArtifactGrantee(person, True, True, False)
<StevenK> So it makes 15 bugs and branches
<StevenK> And then dies when the query count increases by 15.
<wallyworld> but are any of those branches included in the results
<wgrant> Not until last week
<wallyworld> right ok
<wgrant> But I don't see how this branch could change the set of branches that is found.
<StevenK> But it doesn't? The queries that are added are all access checks
<StevenK> If Branch and BranchSubscription are cached by earlier queries which sounds likely, the query count would be low
<wallyworld> i'm still not sure why we expect changing the owner will change the results returned
<wgrant> StevenK: BranchSubscription doesn't appear to be cached.
<wgrant> It's not even cachable.
<wgrant> StevenK: It's used as part of the query's internal access check, but it's not returned.
<wgrant> wallyworld: I was expecting that we were hitting the branch owner short-circuit.
<wgrant> But StevenK reports that the query count remains low even when a non-owner is used.
<wallyworld> well it dropped from 11 to 10
<StevenK> Yes, defined as 'remains low'
<StevenK> Rather than 26
<wallyworld> so with bugs, they are bulk loaded
<wallyworld> not sure about branches
<wgrant> Right, bug searches generally automatically populate the access cache.
<wallyworld> that's why this query count test was added
<wallyworld> since initially the query count was linear
<wallyworld> just like it appears to be now for branches
<wgrant> Exactly. The question is how that happens, and why it stops working.
<wgrant> I suspect there isn't actually any prepopulation of the access cache. We're just hitting another short-circuit somewhere.
<wgrant> So the test is buggy.
<wallyworld> why is the test buggy?
<wallyworld> it verifies that the query count is independent of nr of bugs
<wgrant> It's asserting query count linearity, but it looks like it's only linear in a special case that the test hits, rather than the general case.
<wallyworld> it used to fail though before changes were made
<wallyworld> to fix it
<wallyworld> fix the code i mean
<wallyworld> i can't recall what was done though
<wallyworld> it's not asserting linearity, it's ensuring there is no linearity
<wgrant> Er, yeah, that.
<wgrant> Sub-linearity :)
<wallyworld> well, the intent is that the query count doesn't increase as the number of artifacts increases
<wallyworld> who is to say it can't decrease by 1 or 2
<wallyworld> depending on what the exact data is
<wgrant> wallyworld: That's not the issue
<wgrant> That issue is that somehow the old code is not issuing extra queries
<wgrant> The new code is
<wgrant> The new code needs to not
<wgrant> But we can't work out how the old code manages to not
<wallyworld> so fix the new code :-)
<wgrant> So we can't make the new code not.
<wgrant> StevenK: Breaking on visibleByUser in the old code didn't reveal anything useful?
<wallyworld> wgrant: i'm not sure how did old code did its thing, too long ag
<wallyworld> ago
<wgrant> Indeed, I didn't really expect you to remember :)
<wallyworld> wgrant: bugtasks = list(getUtility(IBugTaskSet).search(param))
<wallyworld> that's what the old code does to load the bugs
<wallyworld> where param has the bug ids
<wallyworld> param = BugTaskSearchParams(user=user, bug=any(*bug_ids))
<wallyworld> iso it's whatever IBugTaskSet does
<wgrant> Yes, bugtaskset precaches authorization
<wgrant> But branchcollection appears not to
<wallyworld> yes
<wallyworld> so isn't that the answer?
<wallyworld> the old code for branches
<wallyworld>             all_branches = getUtility(IAllBranches)
<wallyworld>             wanted_branches = all_branches.visibleByUser(user).withIds(
<wallyworld>                 *branch_ids)
<wallyworld>             branches = list(wanted_branches.getBranches())
<wgrant> Hmmm?
<wgrant> The answer to how it avoids queries is not "it's not precached"
<wgrant> That's the anti-answer :)
<wallyworld> no, i meran bugtaskset precahces, so eliminates queries
<wallyworld> branch collection does not
<wallyworld> so queries increase
<StevenK> So, I think the 10th query in the devel test is IBranchCollection.visibleByUser()
<wgrant> But *something* manages to avoid an access query for the 15 branches in the old code!
<wgrant> The key thing is to identify what that is.
<StevenK> I'm not certain why it is called 15 times, but only generates one query
<StevenK> Well, I'm certain why it's called
<wallyworld> so it must be in branchcollection
<StevenK> branchcollection gives me a headache
<StevenK> It's horrible
<wallyworld> yep
<wgrant> StevenK: So how many times is the visibleByUser called in the problematic context, in devel?
<StevenK> wgrant: 15
<wgrant> StevenK: And what does stepping through it show it doing?
<StevenK> wgrant: Hold on a moment, visibleByUser returns a collection
<StevenK> So it probably returns all fifteen in one query
<wgrant> StevenK: I mean Branch.visibleByUser, not *BranchCollection.visibleByUser
<StevenK> Branch.visibleByUser is not called
<StevenK> wgrant: http://pastebin.ubuntu.com/1077708/
<wgrant> StevenK: Hm
<wgrant> StevenK: Of course, the alternate explanation is that the user can't actually see any of the branches in the devel test.
<wgrant> StevenK: So there's nothing returned to check access on.
<wgrant> StevenK: Can you check that hypothesis?
<wgrant> The project owner will be able to see them in your branch, due to the APG
<StevenK> wgrant: self.branches in PPSV is []
<StevenK> (for devel)
<wgrant> StevenK: Ha ha hahhah
<wgrant> That explains it.
<StevenK> [<Branch u'~person-name-100003/product-name-100004/branch-100010' (77)>, ...] for my branch
<StevenK> So it caches nothing and was a useless test
<wgrant> It's good for bugs.
<wgrant> Just not branches.
<wallyworld> it was designed primarily for bugs
<wallyworld> branches weren't done yet
<StevenK> So it's awesome that we've wasted a few hours bashing our head against it
<wgrant> Which is unsurprising, given that branches didn't know about the access schema when it was written
<StevenK> wgrant: So, 12 -> 26 and move on? Please?
<wgrant> StevenK: Do file a critical bug about it, but yes.
<adeuring> good morning
<StevenK> wgrant: https://bugs.launchpad.net/launchpad/+bug/1021622
<_mup_> Bug #1021622: PillarPersonSharingView uses O(n) queries for branch access <disclosure> <Launchpad itself:Triaged> < https://launchpad.net/bugs/1021622 >
<wgrant> StevenK: Thanks.
<czajkowski> morning
<cjwatson> Err, I just got http://paste.ubuntu.com/1077762/ from an ec2 land run - should I be concerned?
* adeuring changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: adeuring | Firefighting: - | Critical bugs: 4.0*10^2
<cjwatson> Oh, well, it didn't do it again on a retry ...
<danilos> cjwatson, then it does mean you should be worried :)
 * cjwatson starts to understand the layout of .../browser/ and .../templates/ and feels something else leaking out of his ears as counterbalance
<deryck> adeuring, https://plus.google.com/hangouts/_/977b4ffbaf4a8c42a0f41a913e0632048bf13e25?authuser=0&hl=en
<jam> benji: you landed some stuff in the lazr code (and launchpadlib) that seem to be dealing with httplib2 0.7+ (which enables ssl validation by default).
<jam> I'm trying to land some of that in launchpad itself (because I want a patch to launchpadlib's test suite to pass on py2.7)
<jam> however, httplib 0.7 breaks launchpad's test suite because of the ssl validation.
<jam> is it sane to just set the env var for the test suite/
<jam> ?
<benji> jam: I'm trying to remember that branch... oh, I think that is the one by jml; yeah, setting the env. variable for the test suite would work.
<jam> benji: where/when would that get set in the lp test suite?
<benji> jam: hmm, let me look
<jam> maybe in a layer?
<jam> LPFunctionalLayer is the one the specific test I'm looking at right now.
<benji> jam: that might work; if you wanted to set it globally, doing it in buildout-templates/bin/test.in would make sense
<cjwatson> What's a sensible way to write a test that attempts to open a URL on the appserver that traverses to one on the librarian?
<cjwatson> I can use setupBrowserForUser to get something that knows how to open URLs on the appserver, and urlopen to open librarian files, but I can't seem to find something that works for both
<cjwatson> (I'm in LaunchpadFunctionalLayer)
<jam> benji, bac, jcsackett: I just committed and uploaded launchpadlib-0.10.2 (as approved). Can you upload it to pypi? I can do everything but that step.
<jml> jam, benji: let me know if I can help.
<benji> jam: if you're going to be making releases, then you should have pypi access; give me your pypi user name and I'll give you access
<benji> jam: to clarify, we don't upload releases to pypi, we just register them (although, we should upload them in my opinion, but that's a seperate issue)
<jam> benji: I did mean register, and my pypi name is jameinel, let me double check, though.
<jam> benji: yeah, jameinel
<benji> jam: you're all set
<jam> jml: so the patch you requested ends up *requiring* httplib2 0.7+ because it is passing a parameter that doesn't exist in 0.6
<jam> however, using 0.7 means that the test suite aborts all over the place because by default the ssl certificate is checked and fails
<jam> I'm guessing we want to push forward, and fix that, rather than changing the patch to be backward compatible?
<jml> jam: I think so. 0.6 isn't even the version in lucid.
<jam> jml: it is the version in versions.cfg :)
<jam> jml: the problem right now is that I set the env var in bin/test, and if I drop into pdb I see it set, but the test still fails
<jml> jam: interesting. I had to make changes both to launchpadlib and to lazr.restfulclient. Perhaps only one is up-to-date?
<mpt> jml, what's "LoC neutrality"? Is it removing as many LoC as you add?
<jml> mpt: yes.
<jam> jml: something is stripping the env variable in "launchpad_for()
<jml> jam: remind me which code-base that function is in?
<jam> as in, it is set, I break on _request and then it is no longer set
<jam> lazr.restfulclient._browser.Browser._request is the chain that ends up failing
<jam> which you updated to support the flag
<jam> the test I'm running is: lp.bugs.browser.tests.test_bugattachment_file_access.TestWebserviceAccessToBugAttachmentFiles.test_user_access_to_private_bug_attachment
<jam> I'm sure I'm running updated code, as I'm getting the error: SSLHandshakeError: [Errno 1] _ssl.c:504: error:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed: perhaps set LP_DISABLE_SSL_CERTIFICATE_VALIDATION if certificate validation failed and you genuinely trust the remote server.
<jam> which means it asks me to set the variable I have set.
<jml> jam: pulling latest stable
<jml> jam: what's launchpad_for() ?
<jml> oh launchpadlib_for
<jam> jml: so... I can see that the value is being seen inside _browser
<jam> self.disable_ssl_certificate_validation is True
<jam> and I'm still getting an SSL handshake failure...
<jml> jam: perhaps I missed a place in my patch? can you paste the full stack?
<jml> jam: also, I thought you were reporting that the env var is unset at the time of error?
<jam> jml: lp:///~jameinel/launchpad/py27-introduction-1020667
<jam> jml: I thought that was the problem, as I was still getting the error.
<jml> jam: ah right.
<jam> But no, it is set, and tracking down I see RestfulHttp.__init__
<jam> sees the disabled flag correctly
<jam> and after super()
<jam> super().__init__()
<jam> self.disable_ssl_certificate_validation is True
<jml> then my bet is that there's a request I missed
<jml> in the absence of more information, like a stack trace. (which I'll get soon, once I up download-cache, make schema and run the test)
<jam> jml: http://pastebin.ubuntu.com/1078135/
<jam> jml: you'll need to update download-cache as well
<jam> once I actually commit the new packages
<jam> done
<jam> rev 498 of download-cache
<jam> anyway, I'm at my EOD here, so if you want to poke at it, that would be great.
<jam> but it can wait until monday as well.
<jam> jml: (this is all just fallout of trying to get a small 2-line patch to make introduction.txt not fail when run under python-2.7, so I'm trying to avoid fixing-the-world just to get that changed. But I want to do the 'right' thing as well)
<jml> jam: I can see how this is annoying.
<jam> anyway, enjoy your weekend
<jml> jam: you too
<cjwatson> Maybe the correct answer (to my question above) is to manually follow the first redirect to the librarian.
<jml> forgot to switch to lxc
<jml> mpt: what prompted that?
<mpt> jml, just saw you mention it in a mailing list message, and I hadn't seen it mentioned before, that's all
<jml> mpt: oh right. it's a short-hand for Launchpad's broader "no increased maintenance burden" policy
<mpt> I see
<mpt> If you want a list of features to remove to make up the numbers... ;-)
<czajkowski> mpt: only if we can submit the same amount for you to remove elsewhere on design :)
<sinzui> Speaking of LoC, I think I need to get the knives out and cut code to help my self-esteem.
<jml> mpt: heh, there's actually plenty of code that can be removed without affecting functionality one whit.
<mpt> czajkowski, there are almost certainly ways to make Launchpad prettier by reducing the amount of CSS. Would that count?
<jml> yeah.
<czajkowski> mpt: huw is working on that atm when he's not working on other areas, he's been fixing the layout of side bars and this week fixed links on the blog
<mpt> grand
<sinzui> mpt, huwshimi is landing such a branch now. I also pointed out out bug about the obsolete portletBody classes
<jml> w t f.
<jml> bzr just started thinking that my lp:divmod.org branch was a Launchpad branchh
<jelmer> jml: how do you mean, aren't all lp:... branches Launchpad branches?
<jml> jelmer: as in a branch of lp:launchpad
<jml> jelmer: the .bzr/branch/location was a URL to the branch of jam's I just checked out.
<jml> jelmer: I'm guess pebkac, but ... wow.
<jelmer> jml: that is a bit scary..
<cjwatson> I'd love a review of https://code.launchpad.net/~cjwatson/launchpad/queue-api-fix-urls/+merge/113776, if anyone's still around; nearing the end of this arc, and if I can land it today then I can QA it over the weekend and hopefully delete lots of code on Monday
#launchpad-dev 2012-07-08
<wgrant> StevenK: Ah
<wgrant> StevenK: You broke the build twice.
<StevenK> wgrant: I did notice, yes.
<StevenK> One of them I'm annoyed about, since that test failed in ec2 and then locally
<StevenK> However, it also seems we have a smoking gun for the postgres connection leak. Maybe.
<wgrant> StevenK: The postgres leak first showed up in r15502, 68 revs before this reliable failure.
<wgrant> But the immediate issue is fixed.
<StevenK> wgrant: Ah, nice.
<StevenK> How does that deal with ClassAlias? It doesn't?
<wgrant> StevenK: __storm_table__ on a ClassAlias is correct.
<wgrant> So it should be fine.
<wgrant> We'll find out for sure in 5.5 hours
<StevenK> wgrant: The Armor of Al'tair is mine, btw
<rigel_> Hello, I saw the hiring notice on the Launchpad blog, and I noticed it mentioned having an interest in Haskell. I was wondering if Canonical have plans to use Haskell for any projects in the future. Thanks!
<cjwatson> hm, qastaging looks unhappy
<cjwatson> (503 for the last hour or so)
<StevenK> That's not good, because that is roughly the time the latest revno would have hit it.
<wgrant> cjwatson, StevenK: It's probably due to a bug in the new staging restore procedure.
<wgrant> It's meant to take staging down for a day or so, but not qastaging.
<wgrant> fg
<wgrant> fg
<cjwatson> StevenK: I should've said "at least the last hour or so" - I have no information about before that
<czajkowski> aloha
<jelmer> kai ora
<nigelb> jelmer: Isn't it kia ora? :)
<jelmer> nigelb: euhm, yes, thanks :)
<nigelb> :D
<czajkowski> nigelb: pedantic :)
<czajkowski> jelmer: ello hows you
<nigelb> Hah :)
<jelmer> czajkowski: all's well :) how's your weekend?
<czajkowski> not bad, got into a tidying mode yesterday and redid the spare room so all the shuttle boxes are now labeled and I can now see the drobo box and router are off the ground
<czajkowski> am going to attempt to tackle the wiring later on that is currently glaring at me once I come back from lunch
<jelmer> czajkowski: sounds awfully well organized
<czajkowski> jelmer: the spare room was a dumping ground for new gadgets, boxes servers and bf bike gear, there needed to be some order :)
<StevenK> czajkowski: Ew, Drobo
<cjwatson> Anyone fancy reviewing https://code.launchpad.net/~cjwatson/launchpad/queue-api-fix-urls/+merge/113776?  Last piece needed to make the API queue client basically usable.
<cjwatson> (There's one bug remaining that shows up if you try to override just the section of a source upload; workaroundable in the client and easy to fix anyway.)
<wgrant> cjwatson: That won't work
<wgrant> cjwatson: It'll break the security team, since it won't work for private source.s
<wgrant> sourceFileUrls was originally introduced for generating USNs.
<cjwatson> I introduced sourceFileUrls.
<wgrant> Ah yeah, different object.
<cjwatson> What's a working way to do this?  Proxying through Archive definitely doesn't work ...
<cjwatson> And it looked to me as though .getURL() worked for private URLs, but perhaps I was misreading
<wgrant> Oh, getURL will work, yes. Thought you were still using http_url
<cjwatson> (It starts "if not self.restricted:")
<wgrant> But that's fairly evil
<cjwatson> Because it hands out private URLs that can be passed on to anyone?
<wgrant> Sort of
<wgrant> We generally try to avoid issuing TLTs unless someone explicitly asks for them.
<lifeless> (A TLT requires a DB commit)
<lifeless> TLT's are the right thing to do to hand out urls into the librarian for private objects.
<cjwatson> I guess an alternative would be to invent a thing to proxy URLs through the PackageUpload object
<lifeless> But - unless you're very sure every script calling this will download the files, better to do what e.g. bugs do
<wgrant> lifeless: In every single other case we redirect through the webapp.
<lifeless> which is what wgrant just said :P
<lifeless> wgrant: I thought, perhaps wrongly, that the API has a codepath to invoke the same thing
<wgrant> No.
<lifeless> huh
<lifeless> so its uglier than even I appreciated ;)
<cjwatson> So near and yet so far.
#launchpad-dev 2013-07-01
<StevenK> wgrant: http://pastebin.ubuntu.com/5815654/
<wgrant> StevenK: Right, that's broken
<wgrant> Also
<wgrant> Transactions
<StevenK> wgrant: Broken how?
<wgrant> StevenK: I'm not seeing how that query would always return the latest
<StevenK> The ORDER BY ?
<wgrant> Sure
<wgrant> But it doesn't pick just the latest
<wgrant> It returns all of them
<wgrant> Except if there are more than 5000, in which case it just semi-randomly won't return some of them
<StevenK> Sure, and then we loop through them, so the rejector cached property would have been set twice, and overwritten by the second one
<wgrant> Ah, because it's ascending, so that's even worse
<wgrant> Not only is it going to be slow, but it will also ignore any recent event once more than 5000 would be returned
<StevenK> wgrant: So, we want a DISTINCT ON object, operation ?
<StevenK> Well, a distinct option to /fetch/
<wgrant> Well
<wgrant> I don't particularly care about implementation
<wgrant> I'm presenting the constraints
<StevenK> wgrant: I was envisioning being able to answer "Tell me everything that happened to <object>"
<wgrant> StevenK: That's certainly a useful operation to have.
<wgrant> But it's not what we want here
<StevenK> wgrant: Given any change is going to require changes to auditor, auditorclient and lp.services.auditor.client, I want to make sure it is going to be the right call for what we want before we start.
<wgrant> Certainly
<wgrant> This is why I always say you should try to get a minimum viable implementation of infrastructure like this before you land it, before you make 4 auditorclient releases and 8 DB patches
<wgrant> in two days
<wgrant> There are also considerations that must be made in terms of transactionality.
<StevenK> wgrant: DB patches? Since when does auditor require them?
<wgrant> It hopefully won't, but it's similar in style to other large branch series that could
<wgrant> So it's the same concept
<StevenK> wgrant: In terms of DB patches, which require timing, deployment, and merging in stable, sure, I agree. auditorclient requires a pushed branch, setup.py sdist and commiting the tarball to lp-sourcedpes.
<wgrant> And getting to auditorclient 0.0.99999 before we deploy it :P
<StevenK> Django's query rubbish makes me miss storm
<StevenK> Oh, wonderful. Looks like sqlite does not support SELECT DISTINCT ON
<StevenK> wgrant: So if sqlite does not support SELECT DISTINCT ON, and GROUP BY in Django looks ... fun, since they want you to use their aggregation functions, what are my options?
<wgrant> StevenK: Possibly DISTINCT with a subquery
 * StevenK stabs QuerySet.values() being useless
<StevenK> wgrant: I'm failing to see how that buys me anything, but that could be due to Django's ORM eating my brain
<StevenK> Blah
<StevenK> q.values('object', 'operation').distinct() works, but if I call .values('id') on that, I get both
<mwhudson> StevenK: you can use custom aggregate functions in django
<mwhudson> StevenK: http://voices.canonical.com/michael.hudson/2012/09/02/using-postgres-array_agg-from-django/
<StevenK> mwhudson: This is the problem, though -- what I really want is DISTINCT ON, but I can't work out the right query when I'm forbidden from using it.
<mwhudson> heh
<lifeless> StevenK: LOL - bug 1196455
<_mup_> Bug #1196455: power manegmant discrepanty <Auditor:New> <https://launchpad.net/bugs/1196455>
<johntron> i'm trying to create a PPA for nginx that includes the SPDY and pagespeed modules. i've already built and installed from source, and everything works, so now i'm just trying to figure out how to package it. there's arleady an nginx package. i'm not sure how to specify the location of the upstream tarbal or how to build the dependencies during the build process
<johntron> can anyone help me figure out how to get the nginx tarball included correctly and how to get the dependencies `./configure`d correctly?
#launchpad-dev 2013-07-02
<StevenK> wgrant: So I hang myself with dispair over Django and its ORM, I've been poking at PackageDiff. I'm still tossing up if we write a PDJ and hook everything that way, or if I keep PackageDiff as it is and write a new class that pretends to be an IJobSource
<StevenK> Er, so I don't hang myself, even
<johntron> how do i import a github repo? i see the page that says it's possible, but it doesn't actually say how: https://help.launchpad.net/Code/Imports
<johntron> nvm
 * johntron is an idiot
<johntron> i dunno if i'm just dense or not, but i can't figure this out. i want to create a source package recipe using this as a starting point: https://launchpad.net/~chris-lea/+archive/nginx-devel
<johntron> i see that i can copy it, but don't know how to get the bzr repo checked out
<johntron> maybe i need to just download the .deb and create my own repo from that?
<jelmer> johntron: you'll need to download the source pakcage and create a branch from that
<jelmer> from the debian/ directory there specficially
<johntron> jelmer: thanks! i just found reprepo, so i'll probably just use that
<jelmer> johntron: ah, rather than a PPA?
<johntron> well, i don't want to give the impression that i'll be maintaining this...
<johntron> i was just wanting to use PPA to build and distribute for me
<johntron> so i can use puppet/chef/salt to provision easily
<StevenK> wgrant: No opinions?
<StevenK> wgrant: Oh, can't ArchivePurpose.DEBUG die horribly now?
<wgrant> StevenK: Back, sorry.
<wgrant> Ah yes, I forgot to follow up to kill the enum value
<wgrant> StevenK: I'd write a PDJ
<wgrant> You can possibly get away with just using the job table.
<StevenK> wgrant: I'm writing my thoughts
<StevenK> wgrant: http://pastebin.ubuntu.com/5835072/
<wgrant> 1) is right out, because you used the word "leverage"
<StevenK> Haha
<wgrant> PackageDiff.job to replace PackageDiff.status is an antipattern
<wgrant> It could supplement it, but it cannot replace it.
<StevenK> wgrant: Currently, ISourcePackageRelease.package_diffs will show PENDING diffs, too
<wgrant> That's correct.
<StevenK> I'm not sure if we care enough to preserve that behaviour.
<wgrant> We do
<wgrant> We don't particularly want duplicate diffs requested
<wgrant> And it will confuse eg. +localpackagediffs
<StevenK> So we want PackageDiffJob.package_diff and PackageDiffJob.job, and we create a PackageDiff when the job is created
<StevenK> And PackageDiff.status and its enum friend can die.
<wgrant> I think you can probably get away with using Job.json_data
<wgrant> Or whatever it is called
<wgrant> Referential integrity is not an issue here
<wgrant> The job should just not do anything if the PackageDiff doesn't exist.
<wgrant> So no DB changes
<StevenK> Until status dies?
<wgrant> Hm?
<wgrant> What do you mean?
<StevenK> json_data is not on Job itself, it's on the derived job classes, backing onto their own DB tables
<StevenK> wgrant: If we're linking to Job to make use of Job.status, why does PackageDiff.status need to exist?
<wgrant> StevenK: We're not linking to Job
<wgrant> The Job will be linked only be a reference in a JSON dict.
<wgrant> And even if we were linking them formally, PackageDiff.status still needs to exist unless we want to break Launchpad
<StevenK> If we create PDJs (say), we can automatically fail the job if the SPR is udev
<wgrant> No
<wgrant> Because that bug will be fixed by the time this happens.
<wgrant> I'm also not sure how that's relevant to this
<StevenK> wgrant: We can't use Job itself
<StevenK> wgrant: How does the death of PD.status break LP?
<wgrant> Why can't we use Job itself?
<wgrant> We may need to add a new JSON column to it
<wgrant> But I don't see why we can't use it
<wgrant> StevenK: Because filtering by (SPR, status) across two tables is going to be very slow
<wgrant> Think BPB, except not from 2005.
<StevenK> Then we create PDJ?
<StevenK> Bleh, derived jobs don't have status directly
<wgrant> Create PDJ for what reason?
<StevenK> I was thinking to make filtering easy but it still requires two tables
<wgrant> Even if PDJ.status existed, it still provides no benefit
<wgrant> Why do that when you can just use PD.status?
<StevenK> wgrant: So, we create Job.json_data, when a PD is requested we make a bare job that contains the PD id in json_data, and then we have a interface/model that grab the job, lookup the PD, perform the diff and set PD.status appropiately?
<wgrant> Unless you can think of a benefit to having PDJ
<StevenK> Not having PDJ means it's different to any other job via process-job-source ?
<wgrant> I mean in the database
<wgrant> process-job-source has no idea about the database layout
<wgrant> Code changes within Launchpad are cheap
<wgrant> They can easily be changed with an hour's notice
<wgrant> A PDJ table provides no benefit, apart from a foreign key reference, and I would argue that that is not a benefit.
<wgrant> And it pushes us ever closer to 300 tables
<wgrant> If you follow the "everything needs a foreign key" approach then you end up needing a separate table for every distinct set of classes that participate in a job
<StevenK> We can't destroy hwdb and kill 16 tables?
<wgrant> I haven't spoken to HWE/cert
<wgrant> lately
<wgrant> Even so
<wgrant> a PDJ table provides no benefit other than an FK
<wgrant> FK is not a notable benefit
<wgrant> => PDJ is a net negative
<wgrant> => no PDJ table
<wgrant> You still need a DB change this time.
<wgrant> But only this time.
<StevenK> So, the plan I outlined is fine?
<wgrant> Yes
<wgrant> It's effectively a schemaless rabbitmq job that happens to also be persisted in the database, because we don't have a rabbitmq HA story today.
<wgrant> Job.json_data gives us jobs that are almost as lightweight as everyone else who is using celery without the HA concerns.
<StevenK> wgrant: I accidently ArchivePurpose.DEBUG, I'm going to self-approve and land.
<wgrant> k
 * wgrant ginas saucy
<cjwatson> Has anyone had a chance to consider https://code.launchpad.net/~cjwatson/launchpad/copy-set-phase/+merge/170775 ?
#launchpad-dev 2013-07-04
<StevenK> wgrant: https://code.launchpad.net/~stevenk/launchpad/3102-by-default/+merge/172937
<wgrant> StevenK: k
<StevenK> wgrant: It's been weaned off -groups, was that your only concern?
<wgrant> StevenK: Did you consider using Job.makeDerived instead of the stuff on line 150 of the diff?
<wgrant> Also, I'd spell out SPRs in the GENERATE_PACKAGE_DIFF description, given that the initialism is ambiguous nowadays
<StevenK> wgrant: I considered it, but given we're getting back an actual IJob, I didn't know what makeDerived would actually return, or just error -- and UniversalJobSource is called from another process so is difficult to debug and I know cls(job) works.
<wgrant> I'm also not sure I understand exactly how the delegation from PackageDiffJobDerived works.
<wgrant> StevenK: I don't think makeDerived exists on the class you want it to today, but I suggest that you make it exist.
<wgrant> Make Job-only jobs as similar to non-Job-only jobs as possible
<wgrant> We're just merging the additional class into Job, effectively
<wgrant> So Job.makeDerived makes sense for Job-only jobs.
<StevenK> Then I need a mapping of type to class and circular imports?
<wgrant> A good point
<wgrant> A comment to that effect would be very helpful
<StevenK> wgrant: http://pastebin.ubuntu.com/5842427/
<wgrant> StevenK: That doesn't really explain anything. makeDerived not existing isn't a problem, because you could just add makeDerived. The problem is that Job.makeDerived would be a mess of circular imports, so it's probably cleaner to just do this.
<StevenK> wgrant: http://pastebin.ubuntu.com/5842436/
<wgrant> That's more explanatory :)
<wgrant> StevenK: I guess these can be run with run_jobs.py now?
<wgrant> Rather than process-job-source-groups.py
<wgrant> Hm
<wgrant> process-job-source.py exists as well
<wgrant> What is the different between that and run_jobs.py...
<StevenK> run_jobs takes a job name, process-job-source.py takes a source
<wgrant> Certainly
<wgrant> But what is the difference?
<wgrant> AFAICT run_jobs shouldn't exist any more
<wgrant> There are still three config sections for it
<StevenK> process-job-source.py uses the same internals as run_jobs, and the latter is still being used on ackee and loganberry
<wgrant> But I suspect they can be migrated to process-job-source.py-style entries in about 20 lines of diff
<wgrant> How odd
<StevenK> It's backing of JobCronScript can't die, however
<StevenK> % bzr grep run_jobs | cut -d/ -f8
<StevenK> run_jobs.py -vv initializedistroseries >>
<StevenK> run_jobs.py pofile_stats >>
<StevenK> run_jobs.py  packaging_translations -q --log-file=INFO:
<wgrant> Why not?
<StevenK> JobCronScript is used by process-job-source.py, scan_branches and a bunch of others.
<wgrant> scan_branches etc. are irrelevant
<wgrant> All that matters is process-job-source.py
<wgrant> And it would become the One True Runner
<wgrant> So JobCronScript would become a part of it
<wgrant> distroseriesdifference_job, merge-proposal-jobs, process-apport-blobs etc die
<StevenK> wgrant: http://pastebin.ubuntu.com/5842443/
<StevenK> Ah, you're there.
<StevenK> wgrant: Happy to start digging the graves for those scripts in a seperate branch.
<wgrant> I'm doing that :)
<StevenK> Ah
<wgrant> So we'll be down to just process-job-source.py and celery as job runners.
<StevenK> And it will be glorious
<wgrant> (process-job-source-groups just launches multiple process-job-sources)
<StevenK> wgrant: You do seem to have been distracted quite heavily from reviewing my branch, though. :-)
<wgrant> StevenK: I was hoping you'd checked that process-job-source actually manages to run the job
<StevenK> wgrant: As in IPackageDiffJob ?
<wgrant> Yes
<wgrant> Ah
<wgrant> There is a test, I see
<StevenK> Yes
<StevenK> I don't need to check, I wrote a test to check for me.
<wgrant> StevenK: Can you explain how the delegation for PackageDiffJobDerived works?
<wgrant> self.context = self is slightly smelly
<StevenK> self.context = self is for the benefit of celery who assumes that context is a job class
<wgrant> Isn't that for the delegates()?
<StevenK> No, that's self.job
<StevenK> job is for delegates, context is for celery
<wgrant> How does delegate(IPackageDiffJob) know to look at self.job rather than self.context?
<wgrant> Also, what's that __storm_table__ doing there?
<StevenK> __storm_table__ is there because celery calls IStore(db_class)
<wgrant> Right, but db_class should be the Job in this case
<StevenK> With no __storm_table__, we can't adapt
<wgrant> Sure
<StevenK> Because the IStore adapter looks up a table
<wgrant> Because it's not a DB class.
<wgrant> So celery shouldn't be trying to use it as one
<wgrant> The DB class is Job
<StevenK> Right, but it got handed a PackageDiffJob
<StevenK> It assumes all jobs it gets are DB backed
<wgrant> That's not a valid assumption
<wgrant> It could perhaps call a getDBObject method
<wgrant> We should fix the celery integration rather than faking a DB class.
<wgrant> Since the celery integration is probably actually easier to change
<StevenK> getDBObject?
<wgrant> A method on a job that returns the DB object.
<wgrant> It's an example of a way to fix this that isn't evil.
<StevenK> I didn't think __storm_table__ wasn't so evil ... it's correct in one sense
<wgrant> It's not
<StevenK> It is, we have a row in the Job table about it.,
<wgrant> That's a very low threshold for correctness :)
<wgrant> The job system has enough hacks that we haven't removed yet; we should do the small amount of work needed to integrate this nicely, rather than piling on more. Particularly when the additional hacks have the side-effect of turning a normal object into a really strange faux-Storm object with consequences that I haven't fully determined yet.
<StevenK> wgrant: Right, so we want it to be a classmethod that returns IStore(cls) for the base class, and PackageDiffJob can return IStore(Job) ?
<StevenK> getStore() or something
<wgrant> StevenK: Um, possibly, I forget exactly how this is structured at the moment.
<wgrant> StevenK: How does it work for non-Job-only jobs?
<wgrant> eg. BranchJobDerived isn't a Storm object
<wgrant> It still just delegates to BranchJob
<StevenK> db_class in this case is BranchJob
<StevenK> Or so
<wgrant> Sure
<wgrant> But how does it get there?
<wgrant> Oh
<wgrant> Because celery is given a key of BranchJob?
<wgrant> Not the concrete enumerated class?
<StevenK> Hm
<StevenK> The job classes themselves are db classes
<StevenK> I thought they had a generic base class
<StevenK> So PackageDiffJob is the odd one out
<wgrant> Not really, no
<wgrant> BranchScanJob is a BranchJobDerived
<wgrant> So BranchScanJob isn't a DB class; it just delegates to BranchJob, which is one.
<wgrant> PackageDiffJob is the same; it just delegates to a Job, which is one.
<StevenK> SharingJob for instance, is StormBase
<wgrant> SharingJob isn't equivalent to PackageDiffJob
<wgrant> SharingJob is the DB class that underlies the concrete sharing jobs
<StevenK> Right
<wgrant> lib/lp/registry/model/sharingjob.py:class RemoveArtifactSubscriptionsJob(SharingJobDerived):
<StevenK> It backs onto SharingJobDerived, which is backed onto BaseRunnableJob
<wgrant> Exactlyu
<wgrant> The actual job classes are never Storm objects
<wgrant> Which is why it's completely wrong that PackageDiffJob is
<wgrant> Which is the line in celery that expects the job to be storm?
<wgrant> I suspect it's just before it uses UniversalJobSource
<StevenK> It's in UniversalJobSource
<wgrant> The difference is the lack of makeDerived.
<wgrant> For traditional jobs, the key is ('BranchJob', id)
<wgrant> IStore(BranchJob) works
<StevenK> store = IStore(db_class) in UniversalJobSource.get
<wgrant> And the UniversalJobSource looks up the BranchJob by id
<wgrant> Oh!
<wgrant> That makes more sense :)
<wgrant> So it's not in the celery stuff
<StevenK> It's called by the celery stuff
<wgrant> Yeah
<wgrant> But I was wondering how celery could see this
<wgrant> Given that the bits that could break were in UJS
<wgrant> So
<wgrant> Easy fix
<wgrant> If db_class isn't actually a DB class, assume Job
<wgrant> So UJS can take two different things
<wgrant>  1) the traditional intermediate job DB class, like BranchJob or SharingJob, plus a Job.id
<wgrant>  2) a Job-only job class, like PackageDiffJob, plus a Job.id
<StevenK> wgrant: But __storm_table__ == 'Job
<wgrant> In 1) it uses BranchJob.makeDerived to get the concrete one
<StevenK> Is how we do return db_class(db_job)
<wgrant> StevenK: Only because you put it there to placate the code that we're replacing
<wgrant> What do you mean?
<wgrant> (also, db_class isn't a DB class in case 2, so it should probably be renamed)
<StevenK> wgrant: IOW, I'm using __storm__table__ == 'Job' to work out if it's a fake job
<wgrant> StevenK: Yes
<wgrant> And instead you're going to work out if it's a Job-only (aka. fake) job by the *lack* of __storm_table__
<StevenK> Righ
<StevenK> *Right
<StevenK> wgrant: http://pastebin.ubuntu.com/5842508/
<wgrant> StevenK: Not fake, but represented in the database by only a Job row.
<wgrant> The two different inputs that the method takes are quite confusing to distinguish, so I'd suggest a few lines of comment there describing the two cases
<StevenK> wgrant: http://pastebin.ubuntu.com/5842515/
<wgrant> A good start
<StevenK> wgrant: http://pastebin.ubuntu.com/5842531/
<wgrant> StevenK: Heh, that doesn't quite describe the extraordinary oddness
<wgrant> In the Job-only case, the class name we are given is the concrete job
<wgrant> In the non-Job-only case, the class name we are given is the intermediate abstract job class
<StevenK>         # - Jobs that have no backing, and are only represented by a row in
<StevenK>         #   the Job table, but the class name we are given is the abstract
<StevenK>         #   job class.
<wgrant> Sounds good
<StevenK> wgrant: http://pastebin.ubuntu.com/5842539/ is the diff
<wgrant> StevenK: One more thing I've just thought of... it possibly makes sense in PDJ.__init__ to assert that job.job_type matches what we expect
<wgrant> Just in case anything funny happens and there's eg. an ID mixup due to a celery issue
<StevenK> wgrant: Hm, we don't ignore FAILED diffs -- I don't think we can deal with that iterReady, so I guess we should have run() just return None if status != PENDING
<wgrant> StevenK: Didn't we do something to handle that a couple of weeks ago?
<wgrant> Not running jobs unless they are runnable
<wgrant> Or do we in this case have a WAITING job for a FAILED diff?
<StevenK> wgrant: The job *itself* will be runnable, some diffs will be created as FAILED -- eg: udev
<wgrant> That sounds like a bug
<wgrant> There should be no job in that case.
<StevenK> wgrant: http://pastebin.ubuntu.com/5842550/
<wgrant> StevenK: Right
<StevenK> I'll be happy when that diff bug is fixed and we can rip that rubbish out
<StevenK> wgrant: Any other concerns?
<wgrant> I don't think so
<StevenK> wgrant: Did you forget to remove all of the obsoleted cronscripts?
<wgrant> StevenK: In a second branch
<wgrant> StevenK: So we don't break everything ever when this rev is deployed
<StevenK> Not everything ever, just most jobs. :-)
<xnox> Why is https://launchpad.net/~costamagnagianfranco/+archive/firefox "firefox-backports" ?! building boinc on virt-arm ?!
#launchpad-dev 2013-07-05
<StevenK> wgrant: https://code.launchpad.net/~stevenk/launchpad/retry-pcjs/+merge/173116
<wgrant> StevenK: r=me
<cjwatson> wgrant,StevenK: I could do with a review of https://code.launchpad.net/~cjwatson/launchpad/copy-set-phase/+merge/170775, if you get the chance.  bdmurray is blocked on it.
<cjwatson> wgrant,StevenK: Could I have a DB patch number for bug 1198279, please?  I think "ALTER TABLE distribution ADD COLUMN development_series_alias text;" is the most sensible approach.  (I considered something more sophisticated that could deal with more aliases or be more general somehow, but I think YAGNI.)
<_mup_> Bug #1198279: Alias for development series of distributions <Launchpad itself:New> <https://launchpad.net/bugs/1198279>
<wgrant> cjwatson: Have we decided how PPAs are going to work?
#launchpad-dev 2013-07-06
<quatzal> hey, this might be a dumb question but how do I make a version higher than 2~svn###? The package is not from SVN but it is newer
<quatzal> 2.1 would not be accurate, its still 2.0, should it be 2-1~git###? I don't really get LP's versioning
<quatzal> would 2ppa1 superseed 2~svn###?
<quatzal> I mean, how can a git source superseed a svn source?
<lifeless> 2 is > 2~
<wgrant> quatzal: http://www.debian.org/doc/debian-policy/ch-controlfields.html#s-f-Version describes the interpretation of Debian package versions
<wgrant> Launchpad doesn't have its own versioning scheme
<quatzal> thanks guys, and can a git package superseed a svn package? in this case its because the svn was abandoned
<wgrant> quatzal: The ~svnXXXX style is just a convention that some people use for svn versions.
<wgrant> it's not mandated, so you can always adjust the convention such that a git version supersedes an svn version, sure
<quatzal> thanks wgrant
<quatzal> :)
<cjwatson> wgrant: in what way?  my current patch set doesn't do anything particular about PPAs - I basically made DS lookup handle the alias, and put symlink creation at the end of the publisher
<cjwatson> wgrant: I know one of Rick's goals was to make it easier for developers to stay current by default so that we stop having to separately persuade so many people to move forward, so that does seem to suggest making aliases work for PPAs too
<wgrant> cjwatson: Right, but we'll want to think about PPAs before we move anybody onto something unmaintainable, I suspect
<cjwatson> Still not totally sure what you mean
<wgrant> cjwatson: We don't want to convince everyone to move onto some primary archive devel alias, only to discover that it's not supportable for PPAs.
<cjwatson> So, my current patch will create the symlinks for PPAs too, although there's no UI for it or anything
<cjwatson> I'm certainly open to discussion about whether that's a good thing
<wgrant> Except that we don't republish each PPA every time.
<cjwatson> Hmm.
<cjwatson> Just so I have numbers (not as a proposal), roughly how long does it take to run a no-op publish on all PPAs?
<wgrant> PPAs don't get the new series cloned
<wgrant> So that doesn't even necessarily make sense at all
<wgrant> Um
<wgrant> Probably a while
<wgrant> The publisher is pretty terrible
<wgrant> Maybe an hour or two
<wgrant> But then most of them will 404
<wgrant> And apt will be sad.
<cjwatson> I suppose one possible model would be to remember whether something has been uploaded to "next" rather than "saucy", and cause those to be cloned, or something ...
<cjwatson> But I must go and help with children
<cjwatson> (And I'm not sure that makes much sense)
<wgrant> Yeah
<wgrant> But it's almost definitely not symlink-simple for PPAs.
#launchpad-dev 2014-06-30
<stub> wgrant: So I was wondering if that it isn't so much for security, but to ensure requests are not accidently processed twice, eg. when a client retries a request unnecessarily. I don't think we need to care about that, as our calls are idempotent?
<stub> wgrant: The person to ask about the security side would be the security team, since they spend all their time thinking about this stuff.
<wgrant> stub: If it was to avoid accidental double-processing it wouldn't apply to GETs.
<stub> wgrant: OAuth doesn't dictate that GETs are idempotent, does it? That is just sanity, not the spec.
<wgrant> stub: The OAuth spec says that nonces and timestamps aren't used for PLAINTEXT.
<wgrant> So it would have to be local, non-RFC reasoning
<wgrant> And our GETs are idempotent.
<stub> yeah, anyway, that is the best I can come up with.
<wgrant> Hmm
<wgrant> Oh
<wgrant> I guess some people still had that "wouldn't it be great if we let everyone not use TLS" braindeadness back then.
<wgrant> So perhaps they envisaged non-PLAINTEXT signatures in the future.
<wgrant> But it's not 1997, so that's not a concern any more.
<stub> -According to the oauth specification <http://oauth.net/core/1.0/#nonce>, for a
<stub> 181	-given client, an application should not accept a timestamp older than the most
<stub> 182	-recent timestamp received.
<stub> That is an interesting property that we lose
<stub> Only reduces the window from a security pov. I guess clients don't really care - if they resend requests, it is by choice.
<wgrant> stub: We always left a more liberal window anyway
<wgrant> A full minute, in fact
<wgrant> So unless a client is buggily retrying requests more than a minute later, nothing changes.
<stub> wgrant: Go for it from my POV. Maybe run it by the security team to see if they have any rationale for keeping it.
<wgrant> stub: Thanks.
<wgrant> cprov__: https://code.launchpad.net/~wgrant/launchpad/bug-1201984/+merge/225011
<cprov__> wgrant: on it
<cprov> wgrant: are you sure is_enabled is equivalent to lp.Append in this context ?
<cprov> wgrant: I mean, won't we list PPA alternatives for which the user cannot upload to (exception will be raised, I presume)
<wgrant> cprov: It doesn't have to be exactly equivalent. getPPAsForUser just has to approximate it; the permission check is done properly later and the copy will be rejected.
<wgrant> The permission check could also fail today, eg. if the permission was revoked between the request and the running of the job.
<cprov> wgrant: true, the permission check is delegated to the job itself.
<cprov> wgrant: I think users are already used to check copy results in the destination PPA, but that will certainly make it more importantly.
<wgrant> They can fail for heaps of different reasons.
<wgrant> This is a pretty rare one.
<wgrant> Thanks.
#launchpad-dev 2014-07-01
<wgrant> Damn
<wgrant> I wonder how many webservice query count tests I broke by dropping OAuthNonce.
<mwhudson> haha
<mwhudson> hopefully all of them i guess
<wgrant> Looks like it so far.
<wgrant> Everything loses three queries.
<wgrant> Hm, I think I broke PQM.
<wgrant> I used non-ASCII characters in my commit message, and didn't get an email back from PQM despite it successfully landing.
<wgrant> The line with non-ASCII characters is omitted from the merge commit, and the other line is not wrapped.
<lifeless> nice
<wgrant> so close
<wgrant> POFile:+translate down from 339 queries down to 175 so far even before making it not completely stupid.
<wgrant> There must be another six easy ones there somewhere.
<cjohnston> wgrant: whats the status of the postgres move?
<wgrant> cjohnston: The upgrade to 9.3? Done on production but not on buildbot. Why?
<wgrant> Also, morning.
<cjohnston> wgrant: we had discussed the whole moving to the new machines and such with the SSDs..
<cjohnston> mornin
<wgrant> Oh
<wgrant> The new hardware is still waiting on IS
<wgrant> Sitting in the DC, just out of reach...
<cjohnston> :-(
<wgrant> It's cruel, really.
<cjohnston> Swing up to the DC before you go to Germany and set it up yourself ;-)
<wgrant> I'll just ask James for the keys to GS2.
<cjohnston> I still have a master key to the world ;-)
#launchpad-dev 2014-07-02
<wgrant> cprov: https://code.launchpad.net/~wgrant/launchpad/tm-suggest-constant
<cjohnston> mornin
<wgrant> Morning cjohnston
#launchpad-dev 2014-07-03
<wogon> hi there !
<wogon> I'm running in a probably simple trouble building packages on launchpad
<wogon> I've got two of them, the first builds a libxxx and libxxx-dev
<wogon> and the second depends on the first at building time
<wogon> I've set up a "testing" repository where I successfully build anything except for the packages tyhat depend on others at build-time
<wogon> how can I set the launchpad pbuilders to import the current repository as a source for packages ?
<wgrant> wogon: That's done automatically. Are you sure the dependencies were published by the time the dependent build started?
<wogon> you're right : that's probably the problem I need to wait for the first package to be duly published
<wogon> is there a global graph of average build waiting time ? in order to schedule build during empty hours ?
<cjwatson> wogon: There's no real benefit to waiting; the build will have to be done eventually, after all
<cjwatson> We do have some internal graphs of queue size, but not public.  They don't currently show all that much of a pattern, though.  I wouldn't bother trying to do queueing externally - just upload when you're ready.
<cjwatson> There is a bit of a weekly pattern; there's a spike roughly each Monday, typically
<wgrant> The current spike (and usually the Monday spike too) are because a number of builders have failed.
<wgrant> cprov, cjwatson: https://docs.google.com/a/canonical.com/document/d/1ud9SxGCvlTYc-IwNVpjr0LosLQprDJFS6AA-M4nP-I0/edit has my analysis of the non-Ubuntu PPA naming situation. Thoughts very welcome.
<wogon> wgrant cjwatson : thanks, the build are ok. I bother with waiting for my builds only when they fail ...
<wogon> now, all my packages are ok I'll push with more confidence and won't need to wait for the result
#launchpad-dev 2014-07-05
<AlecTaylor> hi
#launchpad-dev 2015-06-29
<wgrant> blr: Landed the getAdministratedTeams timeout fix, so hopefully qastaging will be a bit happier soon.
<blr> wgrant: ah great
<maozhou> I build package of glibc on my launchpad server,then I have published it ,but the state of glibc is still Uploading ,why?
<wgrant> maozhou: What do you mean by "published it"?
<wgrant> It can't be published if it hasn't been uploaded yet.
<maozhou> I run "echo "123123" |sudo -S scripts/process-upload.py -vvv --builds -C buildd /var/tmp/builddmaster"
<maozhou> LP_DISABLE_SSL_CERTIFICATE_VALIDATION=1 /home/kylin/ubuntu-archive-tools/queue -l dev -s utopic accept
<maozhou> scripts/process-accepted.py -vv ubuntu
<maozhou> scripts/publish-distro.py -vv
<maozhou> scripts/publish-distro.py -C
<wgrant> maozhou: What's the 'echo "123123"'?
<wgrant> process-upload.py does not read from stdin.
<maozhou> the password of the user who run the script
<wgrant> That's certainly unusual.
<wgrant> You'll need to check the logs from process-upload.py to see why your build was not uploaded.
<maozhou> on another launchpad server ,  I do it by same way ,the state of the pachage is successful
<maozhou> where is the logs from process-upload.py?
<wgrant> stderr
<maozhou> wgrant: the log is http://imgur.com/mVLK4PE    http://imgur.com/25WeYTn    http://imgur.com/B8RcZg2   , but I can't find any message about "glibc"
<wgrant> maozhou: You seem to have a lot of warnings and errors there. I'd be working those out before trying to understand why another upload hasn't appeared.
<wgrant> But considering searching /var/tmp/builddmaster for a directory name ending in PACKAGEBUILD-$BUILDID, where $BUILDID is the ID of the build that is stuck Uploading.
<maozhou> And I find a pachage which bigger than 256M, can't  be uploaded succeed
<wgrant> What goes wrong when you try?
<maozhou> If a package builded succeed on my launchpad, if the binary package of it  is bigger than 256M, when run process-upload.py , It's error.
<wgrant> Error?
<wgrant> We regularly deal with packages larger than a gigabyte without a problem.
<wgrant> It's conceivable that there may be a problem with doing it on a 32-bit system, but I've not seen anything like that.
<maozhou> yes, the OS is 12.04 32bit
<wgrant> Ah, that's inadvisable for a production deployment.
<wgrant> I'd expect it to work, but it's not thoroughly tested at scale.
<maozhou> the  launchpad server  should be deployed on 64bit OS, right?
<wgrant> maozhou: All of our production systems are running on amd64, yes.
<wgrant> maozhou: The test suite works fine on i386, but it doesn't test large datasets.
<maozhou> ok, I konw
<maozhou> I know
<cjwatson> wgrant: My tentative plan for dose-builddebcheck was to see if it could be run on the builder side
<wgrant> cjwatson: What benefit does that give us?
<wgrant> cjwatson: AFAICS it just makes it less likely that we can autoretry depwaits.
<cjwatson> Actual accurate build-dep analysis rather than guesswork
<cjwatson> s/build-dep/dep-wait/
<cjwatson> The case at hand in this bug could dep-wait rather than fail.
<wgrant> In the current environment we need to be quite conservative with depwaits.
<wgrant> Though it's already imperfect because of virtual packages, I guess.
<cjwatson> One thing I was thinking of was that we could try sbuild, and if it fails to install build-deps we could then run dose-builddebcheck to find out why.
<wgrant> That is, we should avoid creating depwaits that can't be automatically retried when the dep becomes available.
<wgrant> Returning a deeper dep makes that more likely, as one of the deps in between could change such that the dep still doesn't exist, but the package's build-deps are now satisfiable.
<cjwatson> In which case you'll get a new answer from d-b
<wgrant> Only if we run it again.
<wgrant> On what condition do we trigger it to rerun?
<wgrant> There are a few depwait builds in PPAs and such.
<cjwatson> On a new build ...
<wgrant> Right, that means we lose depwait autoretry.
<wgrant> Then what value does depwait provide?
<cjwatson> Um?  No we don't
<wgrant> foo depends on bar
<wgrant> bar depends on baz, which doesn't exist.
<wgrant> baz isn't meant to exist; it's a bug in bar.
<wgrant> New bar is uploaded, not depending on baz any more.
<cjwatson> I'm proposing having launchpad-buildd run it if sbuild reports a missing dep
<wgrant> foo's build is depwait on baz, and will never be autoretried.
<cjwatson> As a replacement for the regexes
<cjwatson> Ah, right, yes, that's a problem.
<wgrant> Currently in that case we fail.
<wgrant> And everyone knows a failure is sinister.
<cjwatson> Though, the way that d-b works means that we would have control over how deep a dep-wait we choose to accept.
<wgrant> (a depwait may be sinister in the virtual case, but non-existent virtual packages are conveniently rare)
<lifeless> wgrant: is it ever dexter?
<wgrant> lifeless: Most of the time :)
<cjwatson> Hm, but d-b doesn't tell us the full chain, I clearly misremembered that
<cjwatson> So not as workable as I thought :(
<wgrant> d-b is useful, I think, if you can run it beforehand and then run it regularly until it succeeds.
<cjwatson> Right, as you say that's difficult with lots of archives and bootstrap etc. though.
<cjwatson> Although the bootstrap archive could be considered to be under manual control only, I suppose.
<cjwatson> There are two cases there: port bringup and bootstrapping cycles in normal operation.  In the latter you normally have a specific sequence of packages in mind and don't want other things to automatically pop up.
<cjwatson> Not sure about the former.
<wgrant> Yeah.
<cjwatson> If we can ignore that case then it becomes "just" a performance problem of grabbing all the Packages/Sources files to alphecca for any current dep-wait build and analysing the lot.
<cjwatson> And stopping trying to figure out the dep-wait target from sbuild output.
<wgrant> The quotes are the reason I haven't tried it.
<cjwatson> I should see how long it would actually take.
<wgrant> Unfortunately we can't exactly optimise it like we should publish-distro.
<wgrant> That is, skip archives that haven't changed.
<cjwatson> Indeed.
<wgrant> Because people occasionally do upload to the primary archive, and a few things depend on it.
<maozhou> I dputed a package to my launchpad server, and accepted it , but the queue is empty, why?
<wgrant> maozhou: Why would it be in the queue after you'd accepted it from the queue?
<maozhou> the build queue is empty, there are no build job after accepted the package.
<wgrant> maozhou: What did process-upload say about it?
<maozhou> it say "Finished checking upload" and "Setting it to ACCEPTED"
<wgrant> Did it say it created any builds?
<wgrant> Should it have created any builds?
<wgrant> Have builds previously worked on that series on that instance?
<maozhou> NO , it haven't creanted any builds.
<wgrant> Does it have appropriately DistroArchSerieses and chroots configured?
<maozhou> It's the first package on this test launchpad server.
<maozhou> the DistroArchSerieses is default , ubuntu/utopic , and I have dput the chroot to it.
<wgrant> One cannot dput a chroot.
<wgrant> And ubuntu/utopic does not exist by default.
<maozhou> I can  access the url of "launchpad.dev/ubuntu/utopic"
<wgrant> How did that get there?
<maozhou> and I run "LP_DISABLE_SSL_CERTIFICATE_VALIDATION=1   ./manage-chroot -l dev -s utopic -a i386 -f chroot-ubuntu-utopic-i386.tar.bz2 set" to dput the i386 chroot file .
<wgrant> Ah, so not dput.
<wgrant> How did you create utopic?
<wgrant> Does https://launchpad.dev/api/devel/ubuntu/utopic have a link to the chroot?
<maozhou> yes, It's exist
<wgrant> What is the full output of process-upload?
<cjwatson> E: Build-Depends dependency for sbuild-build-depends-r-bioc-biovizbase-dummy cannot be satisfied because candidate version of package r-bioc-variantannotation can't satisfy version requirements
<cjwatson> hm, that's less helpful than I might have wished for, apt
<wgrant> cjwatson: Hum,.
<cjwatson> (trying apt-get build-dep after realising that of course the way sbuild constructs the dummy Sources means that --[no-]resolve-alternatives should be fine)
<maozhou> how to verify  https://launchpad.dev/api/devel/ubuntu/utopic have a link to the chroot?
<wgrant> https://launchpad.dev/api/devel/ubuntu/utopic/i386, rather
<maozhou> oh, the chroot_url is empty.
<wgrant> The manage-chroot, then, did not succeed.
<maozhou> but I have ran "LP_DISABLE_SSL_CERTIFICATE_VALIDATION=1   ./manage-chroot -l dev -s utopic -a i386 -f chroot-ubuntu-utopic-i386.tar.bz2 set" , there is no error.
<wgrant> Consider that your hostname resolution my have issues.
<wgrant> Since you're using launchpad.dev everywhere, you must have some serious /etc/hosts entries.
<maozhou> in the file of "/etc/hosts"  ,  the ip is the ip of launchpad server
<wgrant> Does api.launchpad.dev resolve to the right IP address?
<maozhou> how to  verify  api.launchpad.dev resolve to the right IP address?
<wgrant> ping or host, for example.
<maozhou> yes, when ping launchpad.dev, the ip is right
<wgrant> Not launchpad.dev.
<wgrant> api.launchpad.dev.
<maozhou> oh, ping apt.launchpad.dev , the ip is 127.0.0.88, t's incorrect ip ?
<wgrant> I can't tell you that.
<wgrant> I know not the layout of your configuration.
<maozhou> oh, I'm sorry , I  looking in the wrong , on the machine who ran  "LP_DISABLE_SSL_CERTIFICATE_VALIDATION=1   ./manage-chroot -l dev -s utopic -a i386 -f chroot-ubuntu-utopic-i386.tar.bz2 set" , ping api.launchpad.dev, the ip is  the launchpad server
<wgrant> Can you see the relevant request in the Launchpad webapp access log?
<maozhou> where is the Launchpad webapp access log?
<wgrant> In the development configuration it is logs/launchpad-access.log
<cjwatson> wgrant: Trying to understand http://bazaar.launchpad.net/~wgrant/launchpad/product-aps-set/revision/17584 - in what way does ProductSet.getProductPrivacyFilter require a policy or artifact grant?  AFAICS it requires one of (product is public, product has no information type, policy grant)
<wgrant> cjwatson: AccessPolicyGrantFlat is a combination of AccessPolicyGrant, AccessArtifactGrant, and AccessPolicyArtifact.
<wgrant> That is, one can use it to query whether someone has a grant for a policy, or a grant for any artifact under that policy.
<cjwatson> ah I see
<wgrant> Someone with a subscription to a bug in a private project must be able to see the project, so the AAG's row in APGF is what allows them that access.
<cjwatson> So with this we'll basically stop using APGF for the project privacy filter
<wgrant> Currently implemented in getProductPrivacyFilter (buggily) and SharingService.checkPillarArtifactAccess.
<wgrant> Correct.
<wgrant> APGF was originally devised for the listing on +sharing, but it proved useful for access checks too.
<wgrant> Wait, no, not correct.
<wgrant> APGF will still be used.
<wgrant> What we're removing is the join through AccessPolicy.
 * cjwatson looks at some more branches at once.  I think I get it now.
<wgrant> We still use APGF to get the set of valid APs.
<wgrant> But we can now do that once, and compare it against the precached list on each Product.
<wgrant> Like we do on Bug/Branch/etc.
<wgrant> The only difference from the artifact case is that for Product we also allow artifact grants to count.
<cjwatson> Would it be worth using AccessPolicySource.find instead of findByPillar so that we can query for just the types we care about?
<cjwatson> (in _cacheAccessPolicies)
<cjwatson> Also, GitRepository etc. calls it access_grants instead
<wgrant> Ah, yes, find is usable now.
<wgrant> access_grants is a list of Person.ids who have AAGs for the artifact.
<wgrant> Each artifact has access_grants (a list of Person.ids), and access_policies (a list of AccessPolicy.ids)
<cjwatson> I think they mostly just have a single access_policy
<wgrant> Ah, yes, all except bugs have just one, that's true.
<wgrant> Bugs may have multiple, as they may affect multiple pillars.
<cjwatson> wgrant: We actually can make useful use of dose-builddebcheck, I've worked out (after swearing a lot at apt-get build-dep's failure to provide useful failure information).  Algorithm as follows.  First, try sbuild, which I think should be modified to use apt-get build-dep anyway but it doesn't really matter here.  If that fails and the output matches a build-dep failure of some kind, run dose-builddebcheck --explain --failures, ...
<cjwatson> ... passing all candidate Packages files and a Sources containing just the source we're building.  If it reports no reason, PACKAGEFAIL.  If it reports a reason with no depchain, then that's a broken direct build-dependency and can safely be turned into DEPWAIT.  If it reports a reason with a depchain, then that's an indirect broken dependency, and for the reasons you gave above that needs to be PACKAGEFAIL.
<cjwatson> Not as potentially elegant as using dose-builddebcheck pre-build somehow, but we know that's hard in our setup, and this deals with the problems with getting an accurate dep-wait out of apt-get build-dep.
<cjwatson> Takes 8s on my laptop to analyse something universe-sized, 1s to analyse something main-sized.
<cjwatson> We could optimise a bit by only doing this in cases where it's hard to get sensible information out of apt-get build-dep.
<cjwatson> wgrant: Any progress on webhooks, BTW?
<wgrant> cjwatson: Hm, that's extraordinarily slow.
<cjwatson> *shrug* for something at the end of an sbuild run I'm not very bothered and haven't tried to optimise it
<cjwatson> I've probably missed something
<wgrant> Yeah, just means it's completely impossible to run as something like retry-depwait eventually :/
<cjwatson> Unless I missed something blatantly obvious, which is probable :)
<cjwatson> Also it depends how that scales; if it's spending most of its time just parsing the big Packages file then that's less of a problem than if it's lots of time analysing per-package.
<blr> morning
<blr> cjwatson: ah, thanks for catching the mode edge case
<blr> this bit of code did worry me in the past... glad we have a few more tests in place.
<wgrant> cjwatson: Did you look over product-aps-use?
<cjwatson> wgrant: I got started on it, but only just
<wgrant> cjwatson: k
<cjwatson> quite a bitty day, I had to take B out to go suit shopping
<wgrant> I won't land the first bits until you've confirmed it's all good.
<wgrant> Though I guess the schema patch is safe.
<wgrant> cjwatson: The refactored diff processing is much more readable, thanks.
<cjwatson> wgrant: What do you think of asking webops for https://pastebin.canonical.com/134245/ ?  It's obviously PII, but in discussion a while back we seemed to collectively think it would be OK ...
<wgrant> cjwatson: Sounds reasonable to me.
<blr> cjwatson: did you see my comment re: comment diff email?
<blr> currently we're not preserving comments on dirty headers at all (I can't recall if we agreed on that, or that's an oversight)
<blr> bzrlib parses a modified file with properties changed as a 'dirty header'
<cjwatson> blr: Replied to your comment (indeed, you're correct)
<cjwatson> blr: I don't think it should be possible for comments to go missing from any position, so that would be a good thing to fix.
<cjwatson> Maybe just treat it much the same way you do comments on patch headers
<wgrant> cjwatson: The devel buildbot green is false. It looks like a query count test broke.
<wgrant> Though it may be spurious, as I don't see how you could have broken it.
<cjwatson> thanks.
<wgrant> Unless it was test ordering, ew.
<cjwatson> It looks like it could benefit from record_two_runs
<cjwatson> It does fail on a single run here
#launchpad-dev 2015-06-30
 * cjwatson embarks on a bit of refactoring there
<cjwatson> wgrant: I think it was actually broken by r17579 and we just didn't notice.  inferred_vcs shows up in the output.
<wgrant> cjwatson: Quite possibly. Must be test ordering.
<wgrant> Since that test has passed since then.
<cjwatson> Yeah, as I say that test doesn't seem as careful as it should be.
<cjwatson> I'm up for a while yet, so I'll clean it up
<wgrant> I'd forgotten about the JSON cache when I approved that branch. I'm tempted to revert the inferred_vcs export.
<wgrant> blr: What do you think?
<wgrant> inferred_vcs should be temporarily, anyway, as we can backfill vcs at some point.
<wgrant> And it is causing every Product view to incur Branch and GitRepository queries to render the JSON representation.
<cjwatson> I wouldn't be sad about that
<blr> wgrant: how valuable is it to expose in the api?
<wgrant> blr: I don't think it's very interesting at the moment.
<blr> agreed
<blr> we really only need it internally for now
<blr> wgrant: are you happy to revert the export or shall I?
<wgrant> blr: Could you? Most/all of the query count test fixes can probably go.
<blr> wgrant: sure
<cjwatson> Ha, after starting on this refactoring I notice that I have a branch from March with exactly the same thing in progress
<wgrant> cjwatson: I hate it when that happens.
<wgrant> Like with the product-aps thing
<wgrant> I was looking on DF and wondering why the query didn't use the Product.access_policies that clearly already existed.
<wgrant> So I tweak the query locally and the tests fail because the column doesn't exist.
<wgrant> I'd worked through most of the solution on DF in February and completely forgotten all about it.
<wgrant> blr: All the other test changes are still necessary with that reverteD?
<blr> wgrant: hmm perhaps I missed some tests, green with regex 'query_count' and 'test_product', plus the doctest
<wgrant> blr: I'd just look through your two testfix merges to see if you've missed anything.
<blr> wgrant: yes, the other querycount bumps are necessary (specification, bugtask, gitlisting)
<wgrant> blr: Weird, OK.\
<wgrant> r=me
<wgrant> Thanksw
<wgrant> Oh
<wgrant> I even filed a bug about the thing I forgot about, with details: https://bugs.launchpad.net/launchpad/+bug/1425430
<mup> Bug #1425430: Bulk project access checks are slow <privacy> <private-projects> <timeout> <Launchpad itself:Triaged> <https://launchpad.net/bugs/1425430>
<cjwatson> blr: I think your latest change will result in duplicated result lines if there is a comment in the dirty header and a comment somewhere else in the same file
<cjwatson> blr: You probably want http://paste.ubuntu.com/11797735/ ?
<cjwatson> wgrant: Any word on the Soyuz redesign docs you had sketches of?
<wgrant> cjwatson: No.
<wgrant> Hmm.
<wgrant> buildbot is very unhappy with me.
<wgrant> I have a vague recollection of Storm leaking references with List attributes.
<wgrant> Many years ago.
<cjwatson> It really is ...
<wgrant> PackageUpload.searchable_versions is a List, though.
<wgrant> And it doesn't seem to blow up quite so effectively.
<blr> cjwatson: err quite, thanks.
<wgrant> I can reproduce it locally, at least, on one test.
<blr> wgrant: shall I land this in the morning, given you and buildbot are not on good terms atm?
<wgrant> blr: Yeah, landing it now won't exactly work.
<wgrant> Ahhh, computers, how I love you.
<wgrant> Can reliably reproduce in product-aps-set
<wgrant> Cannot reproduce in current devel.
<wgrant> But buildbot can.
<cjwatson> I'm going to see if I can implement my apt-get build-dep / dose-builddebcheck suggestion today, unless anyone has serious objections.
<cjwatson> Starting with trying to backport the thing to precise ...
<wgrant> The whole pulling ocaml into everywhere thing is sort of unpleasant.
<wgrant> But worth a try, I suppose.
<cjwatson> Only into our build-dep chain.
<cjwatson> ocaml binaries are standalone once you get them built.
<wgrant> Yeah
<wgrant> But is ocaml in the interesting bootstrap set today?
<blr> ocaml? O.o
<wgrant> Oh, it's in main, so it must be.
<wgrant> blr: Yes, people are weird, sadly.
<cjwatson> This kind of problem often fits functional languages quite well.
<wgrant> Indeed.
<cjwatson> We could reduce bootstrap problems by allowing lp-buildd to degrade to failing versioned build-dep failures if it doesn't have dose.
<cjwatson> ocaml is in main because of llvm-toolchain-* b-ding on dh-ocaml.  Not sure why that is.
<blr> wgrant: yes, and then write libraries like this http://sumtypes.readthedocs.org/en/latest/
<lifeless> wgrant: people are weird w.r.t. ocaml ?
<wgrant> lifeless: There are only three projects I've ever cared about written in it, and at least one of them (sks) has no reason to be.
<blr> lifeless: I've never met a quant, but I can imagine they would be very weird.
<wgrant> I think the leak may be through Storm's MutableValueVariable._event_system, but I'm just avoiding that part of the code entirely for now.
<cjwatson> wgrant: Want me to organise a deployment this afternoon if possible, assuming that you defeat it before EOD?
<wgrant> cjwatson: Yeah, just waiting for tests to complete.
<wgrant> Then I get get -use landed and deployed tomorrow afternoon.
<wgrant> Thanks.
<wgrant> So perhaps the mere act of reading it is sufficient for doom. Bah.
<mpt> Wow bug 1853 is in progress \o/
<mup> Bug #1853: Project group "display name" is redundant with "title" <feature> <lp-registry> <projectgroups> <Launchpad itself:Triaged> <https://launchpad.net/bugs/1853>
<mpt> The 20th-oldest still-open Launchpad bug report
<rpadovani> I know you guys are super busy, but do you have 5 minutes to take a look to https://bugs.launchpad.net/loggerhead/+bug/1436483 ? The diff side by side in completely unusable, and I'm sure it's a 5 minutes fix
<mup> Bug #1436483: The diff page side by side has broken layout <loggerhead:New> <https://launchpad.net/bugs/1436483>
<blr> rpadovani: certainly not looking very side by side there.
<rpadovani> blr, :D I think it's due the  radius borders someone added back in march
<blr> hmm, that's a different issue, I'm not sure that style should be applied there.
<blr> rpadovani: I think the issue is the padding
<blr> on .code, .linenumber
<blr> rpadovani: thanks for the report, will get that fixed now. I just need to see if that style is being used elsewhere (probably how this bug was introduced)
<rpadovani> blr, thanks for the fast response, I should ping you guys more often :D
<blr> yeah, please do :)
<blr> rpadovani: wow, how long has this been broken, any idea?
<blr> bzr blame is suggesting that padding was added in 2011...
<rpadovani> blr, so it isn't, because I used to use it until I opened the bug
<blr> rpadovani: ok, something else going on in that case (although that did fix it...)
<rpadovani> blr, the width of .lineNumber
<rpadovani> https://bazaar.launchpad.net/~loggerhead-team/loggerhead/trunk-rich/revision/484
<rpadovani> that fixes https://bugs.launchpad.net/loggerhead/+bug/310255
<mup> Bug #310255: 6 digit line numbers get cutoff <loggerhead:Fix Released by cruzjbishop> <https://launchpad.net/bugs/310255>
<blr> rpadovani: looking for a diff that satisfies both
<blr> hmm I don't actually know who reviews loggerhead.. I guess we do?
<blr> rpadovani: thanks, cjwaston will probably want to look at that, but should be landed soon.
<rpadovani> blr, thanks to you :-)
<blr> maybe we need css regression testing!
<rpadovani> or a theme ex-novo, maybe responsing (just joking)
<blr> a responsive theme for LP would be great, but a fair bit of work I'd imagine.
<rpadovani> tbh after the git support the thing I really would like are webhooks (and I see you are working on them)
<blr> yep, webhooks are coming :)
<rpadovani> \o/
#launchpad-dev 2015-07-01
<maozhou> How to delete a package on launchpad?
<wgrant> maozhou: From a PPA or from a primary archive?
<maozhou> from a primary archive
<wgrant> maozhou: Use the remove-package tool from lp:ubuntu-archive-tools
<blr> wgrant: thanks, had some vague recollection that secure_codebrowse_root was wrong
<maozhou> wgrant: ok ,thank you.
<wgrant> blr: Shall we deploy?
<blr> wgrant: sounds good.
<blr> wgrant: oh, sourcedeps.cache presumably is generated?
<wgrant> blr: Yep, update sourcedeps.conf, run update-sourceode, and sourcedeps.cache will be automatically updated.
<maozhou> wgrant: What's the parameter of "-A ARCHIVE" in remove-package ?
<wgrant> maozhou: For a primary archive it is the name of the distro. See ARCHIVE_REFERENCE_TEMPLATES in lib/lp/soyuz/model/archive.py.
<maozhou> wgrant: I ran "LP_DISABLE_SSL_CERTIFICATE_VALIDATION=1 ./remove-package  -m "dput by name ubuntu,so the kord package can't dput" -d ubuntu -A ubuntu -s utopic webkitgtk" , but It's error . Error message:"lazr.restfulclient.errors.Unauthorized: HTTP Error 401: Unauthorized
<maozhou> "
<wgrant> maozhou: The user that you're authenticated as may not have permission to remove packages from the primary archive.
<wgrant> maozhou: Try `edit-acl -l dev -A ubuntu -t admin -p admins --pocket Release add` first
<wgrant> To give the ~admins team admin rights on the release pocket of the Ubuntu primary archive.
<maozhou> wgrant: After running  `edit-acl -l dev -A ubuntu -t admin -p admins --pocket Release add` ,  then delete the package, the same error message.
<wgrant> maozhou: Is the user as whom your launchpadlib is authenticated a member of ~admins?
<maozhou> wgrant: how to see my launchpadlib is authenticated a member of ~admins?
<wgrant> maozhou: Which user is your launchpadlib authenticated as?
<maozhou> wgrant: admin@canonical.dev
<wgrant> maozhou: That user doesn't exist in the sampledata.
<maozhou> wgrant: I find username is "username: System-wide: Ubuntu (LaunchpadServer)@https://api.launchpad.net/"  by running seahorse.
<wgrant> maozhou: That doesn't tell you which user it is.
<wgrant> What does "lp.me" say?
<wgrant> Where lp is a relevant launchpadlib Launchpad object.
<maozhou> where is lp.me?
<wgrant> In your favourite Launchpad API client.
<wgrant> eg. lp-shell
<maozhou> I can't find any file which named lp-shell or  lp.me
<wgrant> lp.me is a Python expression that you can evaluate inside lp-shell. lp-shell is a wrapper around launchpadlib.
<maozhou> But ,how to find the lp-shell?
<wgrant> wgrant@lamuella:~$ apt-cache search lp-shell
<wgrant> lptools - Tools for working with Launchpad
<maozhou> In folder ubuntu-archive-tools?
<wgrant> No, it's an Ubuntu package.
<maozhou> wgrant: print lp.me in lp-shell , It's output "https://api.launchpad.net/1.0/~maozhou"
<wgrant> maozhou: That's production
<wgrant> lp-shell dev devel
<maozhou> How to get that package? I installed it by running "apt-get install lptools", is it incorrect?
<wgrant> That's correct.
<wgrant> But it connects to production by default.
<wgrant> Your instance is not running at launchpad.net, as far as I am aware.
<maozhou> But how to make it to connect to dev devel ?
<wgrant> "lp-shell dev devel"
<maozhou> wgrant: running  lp-shell dev devel  can't print  lp.me,http://imgur.com/eEDwMSl
<wgrant> maozhou: If you don't have a valid certificate set up for your Launchpad instance, you'll need to use LP_DISABLE_SSL_CERTIFICATE_VALIDATION.
<maozhou> wgrant: Yes I ran it as "LP_DISABLE_SSL_CERTIFICATE_VALIDATION=1  lp-shell dev devel"
<wgrant> maozhou: Are you sure?
<maozhou> wgrant: Yes , I'm sure
<maozhou> http://imgur.com/7Q2MdMU
<wgrant> maozhou: Which version of python-lazr.restfulclient is installed where you're running lp-shell?
<maozhou> wgrant: The version of lptools is "0.0.1~bzr34-1"
<wgrant> maozhou: Oh, it's a precise host?
<maozhou> wgrant: The version of python-lazr.restfulclient is 0.12.0
<maozhou> wgrant: Yes , it's 12.04.5  64bit OS.
<wgrant> maozhou: That environment variable was only added in 0.13.0. How were you running the scripts from lp:ubuntu-archive-tools?
<wgrant> That can't have been from that host.
<maozhou> wgrant: which scripts of ubuntu-archive-tools ?  I ran most of them succeed
<wgrant> maozhou: But they also wouldn't support LP_DISABLE_SSL_CERTIFICATE_VALIDATION.
<wgrant> So I don't see how they could have connected.
<maozhou> wgrant: But I can run "LP_DISABLE_SSL_CERTIFICATE_VALIDATION=1   ./manage-chroot -l dev -s utopic -a i386 -f chroot-ubuntu-utopic-i386.tar.bz2 set" and "LP_DISABLE_SSL_CERTIFICATE_VALIDATION=1 ./queue Â -l dev -s utopic Â accept"  succeed.
<wgrant> maozhou: And your copy of ubuntu-archive-tools is also running on that precise machine, and is not patched in any way?
<maozhou> wgrant: Yes , I copyed the folder ubuntu-archive-tools   from another machine, and is not patched in any way.
<wgrant> maozhou: Does it succeed without the environment variable set?
<wgrant> On precise it should not be checking the environment variable at all.
<maozhou> wgrant: Yes, it succeed without the environment variable set.
<wgrant> maozhou: You'll need to debug this yourself.
<wgrant> maozhou: lp-shell won't accept LP_DISABLE_SSL_CERTIFICATE_VALIDATION on precise, though.
<wgrant> You need trusty for that.
<maozhou> wgrant: ok
<maozhou> wgrant:  If update python-lazr.restfulclient to 0.13.0 ,is it efficient?
<wgrant> maozhou: You may also need a new launchpadlib.
<maozhou> wgrant: ok, thank you.
<maozhou> wgrant: After running "LP_DISABLE_SSL_CERTIFICATE_VALIDATION=1 ./remove-package  -m "dput by name ubuntu,so the kord package can't dput" -d ubuntu -A ubuntu -s utopic webkitgtk", it's link to "https://launchpad.net/+authorize-token?oauth_token=KTL2Pdv4T3FsZn4l5zBT&allow_permission=DESKTOP_INTEGRATION",not link to launchpad.dev,is it the reason of "HTTP Error 401:unauthorized"?
<cjwatson> maozhou: remove-package -l dev
<cjwatson> Otherwise it defaults to launchpad.net production.
<maozhou> cjwatson:  Oh, yes , that's the reason.  the script run succedd, but the package  is still exits.
<cjwatson> Check the publishing history to see if the removal happened (https://launchpad.dev/ubuntu/+source/webkitgtk/+publishinghistory)
<cjwatson> If so, you just need to run the publisher.
<cjwatson> blr,wgrant: quick tweak to the new UI on Product:+configure-code - https://code.launchpad.net/~cjwatson/launchpad/project-default-git-location/+merge/263486
<wgrant> cjwatson: Good catch.
<cjwatson> wgrant: thanks
<cjwatson> wgrant: did you want to check that the backfilling on staging is sensible before FDT, or are you already confident in that?
<cjwatson> seems to complete in time at least
<wgrant> cjwatson: a) it's fine
<wgrant> b) i'm doing it hot :)
<wgrant> KEY SHARE ftw.
<cjwatson> oh yes I see the other channel now
<maozhou> cjwatson:  The removal happened (https://launchpad.dev/ubuntu/+source/webkitgtk/+publishinghistory) , but after running publisher , the package is still exits.
<cjwatson> Still exists in what sense?
<cjwatson> Remember that it may still be in the published archive if the same package is also published in other suites.
<wgrant> And it will continue exist in pool/ until process-death-row has been run.
<cjwatson> Also, if utopic is "Current Stable Release" or "Supported", you can't actually remove packages from it.
<cjwatson> Your removal message makes no sense to me, so I'm not actually sure why you're trying to do thi
<cjwatson> s
<maozhou> cjwatson:  It  still in published archive, but there is no same package published in other suites, and the status of utopic is "Active Development"
<cjwatson> maozhou: When you say that it's still in the archive, do you mean that it's still in the Packages and Sources index files under dists/utopic/, or that the files are still in pool/, or both?
<cjwatson> blr: Do we maybe want to restrict Product.inferred_vcs to only consider IBranchCollection(self).withLifecycleStatus(DEFAULT_BRANCH_STATUS_IN_LISTING), i.e. branches that would show up in default branch listings?
<wgrant> That's potentially perilous in terms of query performance.
<wgrant> It's likely that code.launchpad.net/launchpad's load time would double in that case.
<cjwatson> There isn't a decent index on that?
<wgrant> Most statuses are shown.
<wgrant> And active statuses dominate the table.
<wgrant> Postgres would probably judge index use unwise.
 * cjwatson tests
<wgrant> I'm also not sure it's worth it now that the default can be explicitly selected.
<wgrant> But if it's cheap, then sure.
<maozhou> cjwatson: In pool and "https://launchpad.dev/ubuntu/utopic/+builds"
<cjwatson> +builds will never go away.  If it was built, that's historical fact.
<cjwatson> As for pool, what wgrant said; that only goes away after process-death-row has run.
<cjwatson> wgrant: http://paste.ubuntu.com/11803804/
<cjwatson> (inferred_vcs is using is_empty())
<wgrant> cjwatson: Right, so it's as I suspected. In the case that there are no active branches, it will scan the entire set.
<wgrant> Most projects don't have terribly many, I will concede.
<cjwatson> Well, I'm not sure we know that from this example; the planner might have noticed that it's quickest this way for launchpad.
 * cjwatson tries to think of a project with no branches
<cjwatson> Or rather only inactive ones
<maozhou> Where is process-death-row?
<cjwatson> maozhou: scripts/
<cjwatson> Oh, there's no index on Branch.lifecycle_status
<cjwatson> Weird
<wgrant> postgres will relatively rarely use indices on large sets.
<wgrant> That is, where it would have to perform many scans.
<maozhou> cjwatson: +builds will never go away? If I delete a package, can I dput the same version of package to this test launchpad server?
<cjwatson> maozhou: No.
<cjwatson> maozhou: Change the version.  Versions are cheap.
<cjwatson> Add "build1" to the end or something.
<cjwatson> maozhou: This is why I was asking what you were trying to do, since I suspected it might be something impossible :-)
<cjwatson> It doesn't matter if you've deleted the package; Launchpad will never let you reuse the same package/version with different contents within the same archive.
<cjwatson> wgrant: Do you know why so many more projects use EMBARGOED on dogfood than on wherever you tested?  The DF count is 113
<wgrant> cjwatson: AND active
<wgrant> Nobody actually deliberately chooses it.
<cjwatson> Aha
<maozhou> cjwatson: Ok , I understand  ,thank you .
<blr> cjwatson: hmm, not something I considered but that does sound sensible - assuming we add an index to lifecycle, would it be worth it?
<cjwatson> blr: I think wgrant has persuaded me that it may not be worth the trouble
<wgrant> :)
#launchpad-dev 2015-07-02
<wgrant> blr: What's the status of the loggerhead and go get branches?
<wgrant> Oh
<wgrant> The loggerhead branch landed a minute before I asked?
<wgrant> Between loading +activereviews and the actual MP...
<wgrant> Nice timing!
<blr> wgrant: just had a look at BranchRefNavigation, so presumably I want context.development_focus.unique_name?
<wgrant> blr: Yep, I think that's right.
<wgrant> The aliases aren't resolved by the codehosting HTTP server, unlike the git case.
<wgrant> blr: Is the go branch ready now?
<blr> wgrant: yep, I think so, want to have a look?
<wgrant> blr: I'd suggest reworking the tests to verify that it actually ends up on the page.
<wgrant> blr: Create branches and repos with known paths and then check that the meta tag appears in the HTML.
<blr> wgrant: ok, worth keeping the unit tests as well?
<blr> will add a soupmatcher
<wgrant> blr: I'd probably do both. Construct the branch/repo with a known path, create the view, check the property, and then render the view and check the content to verify that the template worked.
<wgrant> Yep
<blr> wgrant: do you have a sec to look over those tests again please?
<blr> hmm perhaps would be good to have a negative test with no vcs default set.
<wgrant> Indeed, worth checking the tag isn't there at all.
<wgrant> And default VCS but no default repo or no default branch depending on the VCS.
<blr> wgrant: added those tests, thanks.
<blr> mwhudson: would you like to review the branch?
<mwhudson> blr: er, seems like i'm too late?
<mwhudson> it looks ok to me
<mwhudson> i'll have a poke at it when it hits qastaging
<mwhudson> i forget how often it updates...
<StevenK> mwhudson: Automatically, and it will try every 15 minutes
<mwhudson> ah
<mwhudson> from stable? or db-stable?
<StevenK> qastaging from stable
<mwhudson> half an hour or so then
<blr> mwhudson: great, thanks
<mwhudson> blr: hrm, shouldn't there be appropriate tags on view-source:https://qastaging.launchpad.net/pydoctor ?
<mwhudson> or is the "qastaing doesn't really have the branches" thing
<wgrant> mwhudson: You'll need to set the Product.vcs property until we backfill it. Hit the Code config link in the sidebar and hit save without making any changes.
<wgrant> That should make it appear.
<mwhudson> ah, i loaded the page and saw bzr already selected
<mwhudson> cool
<mwhudson> hm :(
<mwhudson> mwhudson@glamdring:linux_arm64_dynlink$ go get -v qastaging.launchpad.net/pydoctor
<mwhudson> Fetching https://qastaging.launchpad.net/pydoctor?go-get=1
<mwhudson> Parsing meta tags from https://qastaging.launchpad.net/pydoctor?go-get=1 (status code 200)
<mwhudson> import "qastaging.launchpad.net/pydoctor": parse https://qastaging.launchpad.net/pydoctor?go-get=1: no go-import meta tags
<mwhudson> package qastaging.launchpad.net/pydoctor: unrecognized import path "qastaging.launchpad.net/pydoctor"
<mwhudson> doesn't look happy
<wgrant> Oh
<wgrant>     <meta name="go-import"
<wgrant>           content="qastaging.launchpad.net/~mwhudson/pydoctor/github bzr http://bazaar.qastaging.launchpad.net/~mwhudson/pydoctor/github" />
<wgrant> there's the problem
<mwhudson> hm?
<wgrant> The first bit should be just qastaging.launchpad.net/pydoctor
<mwhudson> ah
<mwhudson> yeah
<wgrant> blr: ^^
<mwhudson> oh right
<mwhudson> confusing message :/
<mwhudson> hey, um, the configure code link doesn't appear after you change the default to git
<mwhudson> is that a known bug?
<blr> wgrant: hmm, so it should be, will fix.
<blr> mwhudson: sorry, check again tomorrow morning :0
<wgrant> mwhudson: Yep, for now you have to get there from the sidebar on +index
<blr> wgrant: mwhudson: fix here once the diff renders, https://code.launchpad.net/~blr/launchpad/golang-meta-import-fix/+merge/263625
<wgrant> blr: Does it fix the series case too?
<blr> wgrant: yes
<blr> wgrant: if there's a bug for the missing link, could you assign it to me (sounds like a regression from my configure-code branch?)
<blr> mwhudson: will ping you when the fix is on qastaging.
<mwhudson> blr: thanks
<wgrant> blr: Nope, I just never added the link to +git.
<wgrant> (because of the awkwardness around showing portlets for the default git repo too)
<cjwatson> Ooh, first trimmed-to-context inline comment mail I've seen.  Nice.
<cjwatson> Hm, looks slightly off though
<cjwatson> blr: Would you like me to have a look at the dodgy quoted patch headers near the top of http://paste.ubuntu.com/11809068/ ?  Must be near your EOD
<cjwatson> There's an extra "=== modified file 'lib/lp/registry/browser/productseries.py'" for some reason
<cjwatson> Oh dear, that looks like a bzrlib bug
<cjwatson> http://paste.ubuntu.com/11809104/
<wgrant> But it doesn't throw the line numbers off? Weird.
<cjwatson> Just working on setting up a bzr test case
<lifeless> wgrant: btw
<lifeless> wgrant: have you noticed the Canonical internal channel references on the sso recovery help page
<lifeless> wgrant: https://login.launchpad.net/+device-help
<wgrant> lifeless: Yes
<wgrant> We don't maintain SSO, and there is a bug reported AIUI.
<lifeless> ok cool
<lifeless> I know you don't
<lifeless> but you're closer to the issue :)
<cjwatson> blr,wgrant: https://code.launchpad.net/~cjwatson/bzr/fix-keep-dirty/+merge/263632
<cjwatson> If you're OK with that, I can stuff it into 2.6.0.lp.2 for the time being
<wgrant> cjwatson: lgtm, thanks
<cjwatson> wgrant: I don't know the bzr landing process.  Is it supposed to wait for Richard?
<cjwatson> (for trunk, that is, I'll go ahead with .lp.2 nowish)
<wgrant> cjwatson: Anyone with PQM access (eg. vila) can land it.
<cjwatson> vila: oh hai :-)
<vila> cjwatson: hey ! /me looks
<vila> cjwatson: reviewed, minor nit and it's good to land
<cjwatson> Ah, assertLength, nice.
<cjwatson> vila: Fixed, thanks
<vila> cjwatson: thanks, sent to pqm
<cjwatson> Yay, thanks
 * cjwatson gets that into LP
<cjwatson> Hopefully pqm actually works at the moment
<vila> cjwatson: err, pqm... lacks hope ?
<vila> http://pqm.bazaar-vcs.org/ is up though
<cjwatson> vila: Richard Wilbur reported that it didn't work the other day; we sent him to the public RT
<vila> cjwatson: I missed that :-/ What's the public RT ?
<cjwatson> rt.ubuntu.com, but I don't see a ticket from him there
<cjwatson> oh, https://rt.ubuntu.com/Ticket/Display.html?id=26672
<cjwatson> vila: failing that, looks like you'll need to ask webops to have a look, perhaps referencing that ticket
<vila> cjwatson: thanks ! will do
<cjwatson> Laney: I'm sure that, absent a heatwave, it would have taken me a considerably shorter time to sort out this dep-wait bug
<cjwatson> I think I have code that will do the job now, but need to figure out tests
<Laney> cjwatson: Oh nice, you were working on it?
<cjwatson> Yeah
<Laney> I saw some chatter about dose-debcheck so thought you may be going all out straight off
<cjwatson> I tried using dose but it ended up being more trouble than worth
<cjwatson> Since we explicitly only want to care about the top level of build-deps, it's just as easy to check them directly using python-debian
<cjwatson> (I'd love to have something like Debian's dose-builddebcheck setup, but it's hard to do efficiently with lots of archives)
<Laney> That's an interesting problem
<Laney> I suppose you should be able to make the overlay archive case more tractable by reusing earlier analysis
<cjwatson> In principle
<cjwatson> In practice I fear it's a fair bit of work in dose ...
<cjwatson> Anyway, not needed for this bug
<blr> cjwatson: hmm, did you happen to find the bug in bzrlib? Potentially something I introduced with the changes for preserving dirty headers
<blr> cjwatson: ah I see you did, thank you.
<blr> wgrant: one last thing with the goimport that I believe I missed, in the series case the content should be "{base}/product/series bzr branch_path"?
<blr> thanks vila
<vila> blr: ;) o/
<vila> blr: not landed on lp:bzr yet as  pqm is broken :-}
<vila> err, broken as in someone is fixing it, not broken and nobody can land in lp:bzr anymore, it's an LTS package after all :o)
<blr> hah right
<cjwatson> blr: Yeah, all sorted modulo the PQM problem, landed on qastaging
<wgrant> blr: Right, the first segment needs to match the request, and the third segment needs to be the full branch path.
<blr> wgrant: yep. that should be looking better now.
<blr> wgrant: cjwatson approved earlier, but before that fix, want to have another look?
<wgrant> blr: lgtm, thanks.
<mwhudson> woot
<mwhudson> oh yeah, i was trying to test the git support last night
<mwhudson> get "qastaging.launchpad.net/pyrepl": found meta tag main.metaImport{Prefix:"qastaging.launchpad.net/pyrepl", VCS:"git", RepoRoot:"https://git.qastaging.paddev.net/pyrepl"} at https://qastaging.launchpad.net/pyrepl?go-get=1
<mwhudson> looks good?
<mwhudson> the subsequent bits aren't going to be happy because that isn't a go project
<blr> mwhudson: yes, the git case was fine, I just mangled the bzr case :)
<mwhudson> blr: seems good to check :)
<blr> just landing the fix branch now
<mwhudson> cool
<mwhudson> so qastaging will get it in an hour or so?
<blr> mwhudson: yep!
<wgrant> Yep, about an hour.
<mwhudson> cool
<mwhudson> then what's the eta until production?
<mwhudson> want to get the special case removal change upstream asap
<wgrant> mwhudson: We should deploy today for the MP email fix anyway, so two or three hours?
<mwhudson> wgrant: AWESOME
<blr> wgrant: is that reliant on the pqm fix?
<wgrant> blr: No, we have a patched version of bzr anyway, which Colin's upgraded devel to in the latest revision.
<blr> ah good, okay.
<wgrant> bzr's PQM is broken, not ours.
<blr> right
<blr> vila: do you think there will be another bzr release?
<blr> nice to see the less verbose MP comment email coming through, modulo the wonkiness.
<wgrant> Indeed, it is a big improvement.
#launchpad-dev 2015-07-03
<mwhudson> mwhudson@glamdring:go-test-cases$ go get -v qastaging.launchpad.net/pydoctor
<mwhudson> Fetching https://qastaging.launchpad.net/pydoctor?go-get=1
<mwhudson> Parsing meta tags from https://qastaging.launchpad.net/pydoctor?go-get=1 (status code 200)
<mwhudson> import "qastaging.launchpad.net/pydoctor": parse https://qastaging.launchpad.net/pydoctor?go-get=1: no go-import meta tags
<mwhudson> package qastaging.launchpad.net/pydoctor: unrecognized import path "qastaging.launchpad.net/pydoctor"
<mwhudson> what?
<wgrant> That seems unlikely.
<wgrant> mwhudson: Oh, it's not updated yet.
<wgrant> blr: Did you actually land the branch?
<mwhudson> oh
<mwhudson> doh
<blr> wgrant: hmm I did
<blr> but no response from pqm...
<wgrant> Try again, I guess.
<blr> resubmitting
<blr> wgrant: nothing in the pqm queue afaict, yet I'm not getting an error on lp-land
<blr> mwhudson: sorry, it appears I've killed pqm.
<mwhudson> blr: congrats
<wgrant> blr: Missing QA tags
<wgrant> Jul 03 00:36:01 pqm [140255997114112] INFO: subject: [r=cjwatson,
<wgrant>  wgrant] Fix first part of golang import path for the bzr case,
<wgrant>  rendering root/product.name rather than root/branch.unique_name.
<blr> oh, doesn't it usually complain about that?
<wgrant> I'm surprised lp-land didn't complain, yeah.
<blr> wgrant: is the issue that the linked bug is already tagged 'qa-needstesting'?
<wgrant> blr: Or Fix Committed, probably, indeed.
<blr> hmm, if a second branch is linked against a fix-committed bug, should we change the status?
<blr> potentially that makes things more confusing, I don't know.
<blr> mwhudson: at any rate, that's with buildbot now, sorry for the delay.
<mwhudson> heh no worries
<mwhudson> the delay meant i had no choice but to submit an expense claim, so thanks :)
<blr> wgrant: hmm spurious buildbot failure?
<wgrant> blr: Yep, haven't seen that one in a few days.
<blr> wgrant: how do I restart the build?
<wgrant> blr: http://lpbuildbot.canonical.com/force
<blr> heh "carry on"
<blr> ah, there's a link on the index...
<wgrant> Yeah
<wgrant> The index that nobody ever goes to, because it's just an index.
<blr> I will say one thing for the UI, it actually renders ok on my phone.
 * mwhudson tries to remember if the "carry on" message was him, it sounds kinda familiar
<wgrant> 18.3.13: Michael Hudson 2009-09-03 less lame success html
<blr> hahah
<mwhudson> >:)
<blr> hmm, failure in code.model.tests.test_branchjob.TestBranchScanJob.test_branch_deleted - that seems... improbable.
<blr> passes locally
<blr> wgrant: is that one you're familiar with?
<maozhou> Package debian-install failed to build on my launchpad, a lot of package has no installation candidate.  http://chuantu.biz/t2/10/1435894813x-954497676.png
<maozhou> The Base URL is set to "http://archive.ubuntu.com/" , is it incorrect?
<blr> mwhudson: buildbot decided to behave finally, should be on qastaging soon.
<blr> let me know how you get on.
<mwhudson> will do
<wgrant> maozhou: Those are udebs.
<wgrant> debian-installer is a complicated package. You will need to debug it.
<maozhou> wgrant: But this package is succeed  to build on launchpad.net , I download the same package
<wgrant> maozhou: Yes, but your archive is not identical to the one that Launchpad has.
<wgrant> maozhou: debian-installer pulls in udebs from the archive during its build, so it's by no means as simple as other packages.
<maozhou> wgrant: Can I access the archive which Launchpad.net used?
<wgrant> maozhou: That's archive.ubuntu.com
<wgrant> Your debian-installer build is apparently not using it.
<wgrant> I do not know how debian-installer constructs its sources.list, but it's not a Launchpad problem.
<wgrant> For builds in Ubuntu itself, the entire archive is known to Launchpad.
<wgrant> I assume you don't have all of it imported into your Launchpad instance, so you are using the OEM hack to add extra sources.list lines.
<maozhou> wgrant: But in build.log which on Launchpad.net ,the archive is set to "http://ftpmaster.internal" , this it a internal archive which other people can't access?
<wgrant> maozhou: archive.ubuntu.com is a mirror of that.
<maozhou> wgrant: If set Base URL to "http://archive.ubuntu.com/" , the same error,  I don't understand why "launchpad.net" builded it succedd
<wgrant> maozhou: Neither do I. You'll need to look at the package to see what it's doing to find why it might be failing.
<maozhou> wgrant: ok , thank you.
<wgrant> mwhudson, blr: Can you look at that rev on qastaging?
<blr> wgrant: yep
<blr> wgrant: lgtm, mwhudson might want to test with go-get however.
<wgrant> Looks fine to me.
<maozhou> wgrant: I guess the source.list is not intact , need I uncompress the chroot file ,  modify it ,  and  rerun "./manage-chroot ..... " ?
<blr> wgrant: deployment unblocked now
<wgrant> maozhou: launchpad-buildd overwrites sources.list, so the one in the chroot tarball does not matter.
<maozhou> wgrant:  If modify "/usr/share/launchpad-buildd/slavebin/override-sources-list" on builder ,  the builder won't overwrites source.list.
<wgrant> maozhou: It is the case that if you disable the sources.list overwriting, the sources.list will not be overwritten, yes.
<wgrant> But I'm not sure why you'd want to do that.
<maozhou> wgrant: The archive in source.list is set to "deb archive.ubuntu.com utopic  main" as default, is it not intact ?
<wgrant> maozhou: That's not a valid sources.list line. I would expect "deb http://archive.ubuntu.com/ubuntu utopic main"
<wgrant> maozhou: But you've never explained what your actual purpose is.
<wgrant> For weeks you've been layering hacks upon hacks.
<wgrant> If you can explain what you're actually trying to do with your Launchpad instance, we may be able to advise the lowest-friction way to go about that.
<maozhou> wgrant: I want set up a local launchpad server to build packages.
<wgrant> That's not an end goal.
<wgrant> You want to build packages for a purpose.
<wgrant> Are you creating a derived distribution? Do you just want to have a PPA that is hosted on your premises?
<maozhou> wgrant: Yes, I want to build packages and creating a derived distribution.
<wgrant> maozhou: And you understand the limitations that doing that as an overlay entails?
<maozhou> wgrant: Yes
<wgrant> maozhou: OK
<wgrant> maozhou: So it sounds like you've set the archive base URL on /ubuntu/+pubconf to "archive.ubuntu.com"
<wgrant> maozhou: That's not going to work very well if you're uploading packages to ubuntu, as they won't be able to see each other as build-deps.
<wgrant> What you may want to do is set /ubuntu/+pubconf's base URL to http://archive.ubuntu.com/, then upload to an overlay derivative distribution that isn't ubuntu.
<wgrant> That distribution will reference itself in its sources.list, followed by archive.ubuntu.com.
<maozhou> wgrant: why they won't be able to see each other as build-deps?
<wgrant> maozhou: Because if the archive base URL is set to http://archive.ubuntu.com/ rather than, for example, http://archive.launchpad.dev/, your archive will not be in sources.list. Only Ubuntu will be.
<wgrant> maozhou: You can see the entries that will be in the final sources.list near the top of the build log.
<maozhou> wgrant: I konw this, I don't understand why the package debian-installer failed to build , the same archive , the same package.
<wgrant> maozhou: Examine the sources.list that debian-installer generates.
<wgrant> maozhou: IIRC it logs it to the build log
<wgrant> sources.list.udeb or something
<maozhou> wgrant: Oh, you are right , may be that's the answer.
<maozhou> but, if the source.list  is generated by debian-installer , the same package debian-installer, the  souce.list.udeb should be same.
<wgrant> It is not generated in a vacuum.
<wgrant> The sources.list in the environment is used as input.
<maozhou> Ok, I konw.
<vila> blr: re "another bzr release", I offered my help for that to the GNU maintainer but haven't received a reply.
<vila> bzr, wgrant, cjwatson : and thanks for the less verbose MP comment emails ;-)
<mwhudson> blr, wgrant: more hilarity!
<mwhudson> mwhudson@glamdring:shared-lib-stuff$ go get -v qastaging.launchpad.net/pydoctor
<mwhudson> Fetching https://qastaging.launchpad.net/pydoctor?go-get=1
<mwhudson> Parsing meta tags from https://qastaging.launchpad.net/pydoctor?go-get=1 (status code 200)
<mwhudson> get "qastaging.launchpad.net/pydoctor": found meta tag main.metaImport{Prefix:"qastaging.launchpad.net/pydoctor", VCS:"bzr", RepoRoot:"http://bazaar.qastaging.launchpad.net/~mwhudson/pydoctor/github"} at https://qastaging.launchpad.net/pydoctor?go-get=1
<mwhudson> package qastaging.launchpad.net/pydoctor: cannot download, http://bazaar.qastaging.launchpad.net/~mwhudson/pydoctor/github uses insecure protocol
<mwhudson> mwhudson@glamdring:shared-lib-stuff$
<mwhudson> blr, wgrant: this check is new in 1.5
<mwhudson> is there any anonymous way of getting the code over a secure protocol currently?
<mwhudson> well bzr branch https://launchpad.net/product i guess but that is a bit hysterical raisins
<wgrant> mwhudson: No, no anonymous secure Bazaar access today.
<wgrant> mwhudson: I guess using the webapp branch reference might confuse it enough...
<vila> wgrant, blr , cjwatson : MP comment emails... <3
<blr> mwhudson: oh fun :(
<blr> well, fair enough I suppose.
<blr> wgrant: any suggestions?
<wgrant> blr: You might be able to test it locally if you can disable its certificate validation.
<wgrant> I assume it just passes the URL down to bzr once it checks it.
<wgrant> So we can work around the security check by using https://launchpad.net/$unique_name as the path. BranchRefNavigation allows a bzr client to use that.
<wgrant> (bzr looks up https://launchpad.net/~launchpad-pqm/launchpad/devel/.bzr/branch/location and follows that URL. which go get won't see, so can't complain about)
<blr> mwhudson: is your fork on github?
<blr> https://github.com/mwhudson/go>
<blr> presumably?
<blr> wgrant: shall we add supermirror_root_https to config.codehosting?
<blr> oh I nvm, I see
<blr> so, {vhost.mainsite.hostname}/{branch.uniquename}
<wgrant> blr: It needs a protocol too.
<wgrant> eg. allvhosts.configs['mainsite'].rooturl
<blr> wgrant: thanks
<mwhudson> blr: go get only special cases launchpad.net exactly
<mwhudson> not qastaging.launchpad.net
<mwhudson> blr: so just build go tip
<wgrant> Or apt install golang-go
<wgrant> Oh, that's too old.
<mwhudson> yeah
<mwhudson> this secure insistence is pretty new
<mwhudson> <1 month i think
#launchpad-dev 2015-07-05
<blr> morning
<wgrant> Morning blr.
<blr> how was your weekend wgrant?
#launchpad-dev 2018-07-06
<maru> Hi all. I'm building an emulated environment for cyber security research. In particular, I have a thorough inventory of all applications (with versions) to be installed on a huge number of vms (mostly Ubuntu). I initially resorted to âvanillaâ ansible but the problem is with old/superseded versions of packages. In the journey I discovered the fabulous launchpad API, hence the question: can I obtain a package .deb by specifying name
<maru> and version (e.g., openssh 5.9p1) even if superseded/not in the installed OS ppa?
<cjwatson> maru: Maybe in some cases; not in general and not via apt.  If the files in question haven't been entirely purged then it would be possible to copy them (without rebuilding) into a PPA and then pin that PPA using apt.  Your question isn't quite specific enough for us to know whether that's possible in this case.
<maru> unfortunately we are talking about many different packages. My objective is having a fully automated "best-effort" solution that hopefully works on most cases (I am currently looking at: https://answers.launchpad.net/launchpad/+question/660750 which looks promising)
<cjwatson> many different packages isn't interesting.  many different source *archives* would be interesting
<maru> why?
<cjwatson> are you trying to obtain these packages in the sense of downloading them locally, or in the sense of getting them all into a single PPA that you can provision on one of the relevant systems?
<cjwatson> (or rather, on many of the relevant systems)
<cjwatson> and are they superseded from the primary Ubuntu archive, or from some random PPA?
<cjwatson> (you said "the installed OS ppa", which doesn't totally make sense because the primary Ubuntu archive isn't a PPA)
<maru> On download locally vs ppa: both directions would work in my case, as I have to download and install them on each machine. The second approach would imply building a centralized archive that then could be used as source on all machines (and updated if needed), while the second I guess would be easier/more flexible
<maru> superseded from the primary ubuntu archive
<maru> https://launchpad.net/ubuntu/+source/mysql-5.5/5.5.54-0ubuntu0.14.04.1/+publishinghistory this is a notable example
<maru> also, is anyone aware of an NVT CPE mapping to actual packages?
