#launchpad-dev 2009-07-06
<kfogel> sinzui: I really like the wording in the new "Your project isn't as free as you think it is" email.  Or did you customize that one just now?  (The one about discriminating against a field of use.)
<thumper> hi kfogel
<thumper> kfogel: thanks for that email, it really was a doozey
<ScottK> Is there a link?
<wgrant> I hope we'll see the mailing list made public - and the IRC channel properly moved here - soonish.
<ScottK> Which month is it the partially opening LP was promised?  This one, right?
<wgrant> ScottK: Late July or early August.
<ScottK> OK.  Thanks.
<thumper> wgrant: some of the devs will be moving here
<thumper> wgrant: and once we have open sourced, pretty much all conversation will be here
<wgrant> thumper: Ah, good.
<kfogel> thumper: you're welcome
<kfogel> wgrant: mailing list will become public, but the current archives won't, of course (lots of company information in them).
<thumper> kfogel: so some action on that this week maybe?
<wgrant> kfogel: Of course.
<thumper> kfogel: I'll look forward to the list being public
<thumper> someone at some stage removed my other email address from the "authorised" list
<kfogel> thumper: let's pretend we're not moving to a privchat
<thumper> and I'm always sending email from the wrong email address
<thumper> at least with the launchpad mailman, I can use either :)
<sinzui> kfogel: I did customize that email. I send it to all projects that choose Other Open Source, but discriminate against commercial or derivatives. 90% of the cases are to CC-NC or CC-ND projects.
<wgrant> A lot of people don't realise that NC and ND are non-free.
<sinzui> indeed. I think most projects would not choose them if they know the OSI or FSF reject them
<kfogel> sinzui: makes me wonder if we should anticipate this at the UI level and save ourselves some human time.
<sinzui> kfogel: most of the problem has disappeared since we added the good CC licenses
<wgrant> What fraction of projects are still doing it?
<sinzui> translations-only projects are my new top problem.
 * sinzui looks for answer
<sinzui> wgrant: kfogel: These is not easy way to know. we do not index the  the additional license info field. I think the answer is about 50 projects
<wgrant> sinzui: Not too bad.
<wgrant> But a little disturbing that people don't research their licenses.
<sinzui> cc-0 seems to have addressed the majority of users who just want their code to be public and do not want to bother with the legal issues
<wgrant> Was there a Public Domain option before CC-0 was added?
<sinzui> Well I just scanned all 128 problem projects. there are about 16 projects that are using the problem CC licenses.
<sinzui> wgrant: these was and is a Public Domain. The license comes from US law and has troubles in other countries. CC-0 tries to addresses that
<jtv> hi folks
 * wgrant welcomes jtv to the newly-populated channel.
 * jtv wais wgrant
 * kfogel says good night to wgrant, jtv
<jtv> good night Karl!
<kfogel> g'night
<wgrant> Night kfogel.
<wgrant> Damn.
<jtv> Anyone else having trouble running the Code tests?
<jtv> danilos, here's that change for the POExport view: https://pastebin.canonical.com/19361/
<thumper> jtv: what's up?
<jtv> thumper: relative humidity is waaaay up.
<thumper> jtv: I was meaning the code tests
<jtv> thumper: oh that, probably just a local problem with my system.  Never mind now.
<thumper> ok
<thumper> night
<jtv> thumper: good night
<jtv> danilos, got the paste?
<danilos> jtv: can you try again? :)
<jtv> danilos: https://pastebin.canonical.com/19361/
<danilos> jtv: thanks a lot
<jtv> np
<danilos> jtv: anyway, shall we have a call?
<jtv> danilos: definitely, I was only waiting for you to finish with stub
<danilos> jtv: or a chat, whatevah you prefer
<danilos> jtv: call me back :)
#launchpad-dev 2009-07-07
<mwhudson> wgrant: i know what causes these 500s now
<wgrant> mwhudson: What was it?
<mwhudson> wgrant: me failing to realise that bzrlib's LRUCache is not threadsafe
<wgrant> mwhudson: Aha.
<mwhudson> wgrant: it confirms that just bouncing on refresh is appropriate in this situation
<wgrant> mwhudson: Thanks for looking at that. It's a bit annoying.
<mwhudson> wgrant: i don't know why it's happening so much now, tbh
<mwhudson> it should have been happening since april, aiui
<mwhudson> https://bugs.edge.launchpad.net/loggerhead/+bug/396320 fwiw
<ubot3> Malone bug 396320 in loggerhead "loggerhead accesses the same LRUCache from multiple threads" [Critical,Triaged]
<wgrant> mwhudson: It has certainly been happening much more frequently lately.
<mwhudson> i guess i've fixed some of the problems that just make it fall over completely...
<wgrant> Heh.
<mwhudson> also more traffic will make it happen more often
<thumper> mwhudson: is it on your radar to fix soonish?
<mwhudson> thumper: i guess it should be
<thumper> mwhudson: hard to fix?
<thumper> or just a pain to test?
<mwhudson> thumper: not massively
<mwhudson> thumper: heh, this is loggerhead!
<mwhudson> i dream about tests
<thumper> ehh
<thumper> heh
<lifeless> mwhudson: put a mutex around it?
<mwhudson> lifeless: right
#launchpad-dev 2009-07-08
<thumper> just saying something so it isn't tumbleweed town here...
<lifeless> \|/-
<thumper> lifeless: is that supposed to be a spinner?
<lifeless> a tumbleweed
<wgrant> thumper: I take it that using this didn't take off? I presume #launchpad@irc.c.c is not quite this dead...
<thumper> wgrant: I think some people are still concerned about private details
<wgrant> thumper: I suppose so.
<thumper> wgrant: kfogel will be harassing people in private #launchpad-code once we open source
<thumper> but until then...
 * thumper shrugs
<thumper> we are discussing timing in the team lead meeting tomorrow morning my time
<wgrant> Aha.
<wgrant> Good news.
<wgrant> As long as the discussion doesn't postpone it even more!
 * thumper doesn't like the 6am phone calls
<thumper> wgrant: it depends on the bzr readiness
<thumper> we need to make sure that bzr 1.17 doesn't have bad format 2 bugs
<thumper> that is the most likely thing to cause delays
<wgrant> OK.
<lifeless> thumper: 2 isn't fully baked yet.
<lifeless> thumper: but must of the raw edges are on upgrade, which you won't be exposing people to
<thumper> right
<thumper> lifeless: so format 2 will still be development for 1.17?
<lifeless> its released and supported
<lifeless> we don't anticipate format changes
<lifeless> its just not polished enough to change it to default
<lifeless> (remaining critical bugs, missing network delta support, ..)
<thumper> lifeless: is the absent content factory one fixed?
<lifeless> the one with send is not yet examined
<lifeless> (ACF is hopelessly generic)
 * thumper nods
<lifeless> its like 'the segfault bug with evolution'
 * thumper wills himself to continue writing the boring branch
<wgrant> What's the boring branch?
<thumper> accessing the revision cache
<thumper> it'll give us faster counts of recent revisions and revision authors
<thumper> and make the revision feeds not time out
<thumper> but it is boring work
<wgrant> Ah, good to see an attempt at Slashdotting prevention.
<wgrant> How big will the new tree be, given the lack of history?
<lifeless> do you mean disk size?
<wgrant> Yes.
<lifeless> there's still a bunch of code:P - fairly sizable
<lifeless> I'm not sure of the exact size
<thumper> wgrant: I can't really tell you that right now :)
<thumper> bigjools: want to fix your broken build?
<bigjools> thumper: you speak in forked tongue
<thumper> :)
<thumper> bigjools: I have something I want to land :)
<bigjools> thumper: hassle IS :)
<thumper> why?
<thumper> why is it their problem?
<bigjools> what are you talking about, exactly?
<thumper> heh
<thumper> http://buildbot.devpad.info/builders/lp/builds/477/steps/shell_7/logs/summary
<bigjools> oh bollocks
<bigjools> thumper: did you notice it broken for very long?  I didn't get any emails
<thumper> I think it finished just over an hour ago
<thumper> but I've only just sat down after dinner and kids in bed
<bigjools> jeez, I landed it about 12 hours ago
<thumper> working until Chuck starts
<thumper> bigjools: the dev builder is still the slow one
<thumper> that should get fixed
<bigjools> seriously
#launchpad-dev 2009-07-09
<jtv> mthaddon, can you help me with some very easy QA?
#launchpad-dev 2010-07-12
<wgrant> spm: Around?
<spm> wgrant: yarp, sup?
<spm> it always worries me, when a fast response is followed by a long pause. implies "zomg lots of typing problem coming right up"
<wgrant> spm: I'm working on Soyuz ddeb support, and I'd like to confirm that there aren't already any in the DB, because if any are there then they are broken.
<wgrant> I am hoping that 'SELECT COUNT(*) FROM binarypackagefile WHERE filetype=4;' will return 0.
<spm> hope, springs eternal...
<spm> wgrant: sadly in this case, it does spring eternal. count == 16
<wgrant> Crap.
<spm> doing a Q&D sel *; suggests they may be related in some way; all are nearly sequential libraryfile
<wgrant> spm: SELECT person.name, archive.name, libraryfilealias.filename FROM binarypackagefile JOIN libraryfilealias ON libraryfilealias.id = binarypackagefile.libraryfile JOIN binarypackagerelease ON binarypackagefile.binarypackagerelease = binarypackagerelease.id JOIN binarypackagebuild ON binarypackagebuild.id = binarypackagerelease.build JOIN packagebuild ON packagebuild.id = binarypackagebuild.package_build JOIN archive ON archive.id = ...
<wgrant> ... packagebuild.archive JOIN person ON person.id = archive.owner WHERE binarypackagefile.filetype = 4;
<spm> wgrant: hrm. I dunno. surely you can fit a *few* more joins in there???
<wgrant> Heh, probably.
 * spm remembers fondly the 16+ join we had at #job-1
<spm> huh. yes. nicely guessed, same pacakges. will paste...
<spm> wgrant: http://paste.ubuntu.com/462294/
<wgrant> Craaaaaap primary archive.
<wgrant> At least it's only one build.
<wgrant> Thanks.
<spm> np
<spm> wgrant: so, in laymans terms - what does that actually mean/imply. that it's broken, sure - but what does that imply?
<wgrant> Oh, they were all rejected, anyway.
<wgrant> So there may not be any DB surgery required later on.
<spm> lack of DB surgery is always welcome :-)
<wgrant> spm: Basically, ddebs must always be linked to a deb, or they get lost and float around in space forever.
<spm> chewup librarian space? or worse?
<wgrant> Stay around in the archive forever, basically.
<wgrant> And chew up librarian space, I guess, yes.
<spm> this would be uncool on both fronts.
<wgrant> Now, of course, ddebs have been accepted for a year or so, but the code to link them was never written.
<spm> ahhh! I see. mucch becomes clearer.
<wgrant> But in this case they never made it into the archive, so we are reasonably OK.
<spm> kk
<lifeless> wgrant: hi
<lifeless> this sort of VWS drives me batty:
<wgrant> lifeless: Hi.
<lifeless> +        self.debug("Dominating packages...")
<lifeless> +
<lifeless> +        for name in pubs.keys():
<wgrant> Oy, 'Work in progress' :P
<lifeless> please see PEP8 :) - only do that when there is a significant difference mid-function
<lifeless> wgrant: 'proposed for merging' -> instant review for you
<wgrant> bigjools: Hi. When you have a moment, I'd like to discuss ddebs.
<elmo> wgrant: or your money back!
<wgrant> Heh/
<wgrant> Also, it'd be reallly nice if launchpad-bugs-owner would stop spamming me.
<lifeless> yes
<lifeless> uhm
<lifeless> I think matsubara was doing something about that; I'm sure he's here somewhere ;)
<matsubara> lifeless, wgrant: I found out why and described how to fix in the thread. I didn't do it because I don't have the permission
<thumper> what is VWS?
<lifeless> matsubara: what permission is needed?
<lifeless> thumper: vertical white space
<wgrant> thumper: Vertical whitespace.
<wgrant> matsubara: There was a thread?
<lifeless> wgrant: internally, I think
<wgrant> Ah, there.
<matsubara> lifeless, I think lp admin. we just need to remove ~launchpad as the default review team IIRC
<wgrant> No, it was on the public one.
<matsubara> yeah, I think it's in lp-dev
<lifeless> matsubara: on https://code.edge.launchpad.net/~launchpad-pqm/launchpad/devel ?
<lifeless> hmm, we should change the owner to a steering team too.
<lifeless> matsubara: who should it be ?
<matsubara> lifeless, ~launchpad-reviewers seems to be the correct choice
<bigjools> wgrant: ok, although I'm not sure when I will have time when you're awake
<lifeless> mthaddon: can you change the default reviewer on lp devel, db-devel, production etc etc etc to be launchpad-reviewers? thanks.
<mthaddon> sure
<lifeless> I can rt it if needed.
<mthaddon> np
<mthaddon> lifeless: just the "-devel" branches, I assume - not needed for the "-stable" branches. If so, done
<lifeless> mthaddon: thats a great start; we can deal with future glitches as needed
<mthaddon> k
<lifeless> wgrant: ^
<wgrant> lifeless, mthaddon, matsubara: Thanks.
<wgrant> bigjools: Yeah, I suspected as much.
<bigjools> it's probably best to send me an email
<Aju> Hi!
<Aju> While installation of lauchpad on my development machine, I am receiving the following error:
<Aju> * Creating database "launchpad_empty".
<Aju> * Installing PL/PythonU
<Aju> createlang: language installation failed: ERROR:  could not access file "$libdir/plpython": No such file or directory
<Aju> make[1]: *** [create] Error 1
<Aju> make[1]: Leaving directory `/root/launchpad/lp-branches/devel/database/schema'
<Aju> make: *** [schema] Error 2
<wgrant> bigjools: Sure.
<Aju> Please help.
<wgrant> Aju: Install postgresql-plpython-8.4
<Aju> It's already installed.
<Aju> I have checked it.
<wgrant> 8.4, not just 8.3?
<Aju> Yes 8.4 ver.
<wgrant> Do you have PostgreSQL 8.3 installed?
<Aju> Yes
<Aju> Do both versions conflict?
<wgrant> Maybe it's using that. Do you have both postgresql-plpython-8.3 and postgresql-plpython-8.4 installed?
<Aju> Let me check that.
<Aju> Thanks a lot, it works.
<Aju> Do I have to run the make schema script again?
<Aju> I have just fired the command:
<Aju> sudo -u postgres make create
<wgrant> You should run 'make schema' again, yes.
<Aju> Okay, thanks.
<Aju> Hi!
<Aju> I am receiving the following error, while running the 'make schema' command:
<Aju> 2010-07-12 14:09:31 WARNING No permissions specified for [u'public.openidnonce', u'public.openidauthorization']
<Aju> Traceback (most recent call last):
<Aju>   File "security.py", line 413, in <module>
<Aju>     sys.exit(main(options))
<Aju>   File "security.py", line 179, in main
<Aju>     reset_permissions(con, config, options)
<Aju>   File "security.py", line 394, in reset_permissions
<Aju>     con.commit()
<Aju> psycopg2.ProgrammingError: server closed the connection unexpectedly
<Aju>         This probably means the server terminated abnormally
<Aju>         before or while processing the request.
<Aju> make[1]: *** [create] Error 1
<Aju> make[1]: Leaving directory `/root/launchpad/lp-branches/devel/database/schema'
<Aju> make: *** [schema] Error 2
<jtv> Aju: we're all sprinting right now, so a bit busy!  Sounds like your postgres is having a problem.
<Aju> Okay, thanks
<wgrant> sinzui: Should the error message shown when a package is not found be adjusted?
<wgrant> At the moment it tells people to file bugs.
<wgrant> But they are inevitably rejected.
<sinzui> wgrant, yes in short. I ponder removing fedora and gentoo to wrong expectation
<sinzui> wgrant, yes in short. I ponder removing fedora and gentoo to avoid setting wrong expectation
<wgrant> sinzui: Removing the distributions? That doesn't seem like a good idea.
<poolie> mthaddon, is it just me (or just this network) or are we getting a lot of 500s on the librarian?
<mthaddon> poolie: let me take a look
<mthaddon> not seeing a lot of 500s
<poolie> mthaddon, https://bugs.edge.launchpad.net/launchpad-foundations/+bug/604620
<_mup_> Bug #604620: intermittent 404 errors on existing librarian files <Launchpad Foundations:New> <https://launchpad.net/bugs/604620>
<mthaddon> thx
<jml> thumper, I'm going to delete "19881 branches associated with bug reports" from the code.launchpad.net homepage. Objections?
<thumper> nope
<thumper> jml: ^^^
<jml> thumper, thanks.
<StevenK> jml: Where are you hiding?
<jml> StevenK, it's on the unsprint whiteboard
<jml> StevenK, I can't remember how to spell it.
<StevenK> jml: But I do want to talk to you about stuff you're working on
<jml> StevenK, then come in and let's talk :)
<mpt> So, there are now 12 bug statuses
<mpt> I'm still hopeful that one day we'll have only six :-)
#launchpad-dev 2010-07-13
<mtaylor> a private team can apparently not review merge-proposals
<wgrant> I hate doctests.
<spm> 3 words. so much ... emotion unsaid yet understood.
<wgrant> See, I want to fix a trivial ordering edge case (bug #604842). But the only tests for the method are doctests where I can't really fit a retry. If only they were unit tests :(
<_mup_> Bug #604842: BinaryPackageBuildSet.getBuildsByArchIds orders inconsistently <Soyuz:New> <https://launchpad.net/bugs/604842>
<mtaylor> what do I need to do to make a branch private?
<mtaylor> sorry - wrong channel
<lifeless_> wgrant: change em to unit tests ;)
<wgrant> lifeless_: I was going to ask if it was impolite to bloat a one-line branch into a few hundred lines by rewriting tests, but I presumed everyone would be asleep.
<lifeless> wgrant: I'd spin a new branch for it
<wgrant> lifeless: One branch to redo the tests, and a followup with the one-line change plus new test? Perhaps.
<lifeless> yes
<lifeless> its a little more work but - JFDI
<lifeless> its not a lot more
<wgrant> Yep.
<wgrant> I replaced several hundred lines of doctests yesterday. It seems to be becoming a bit common.
<wgrant> Nasty things.
<lifeless> the meme has caught on
<wgrant> Oh?
<lifeless> and muhaha, its now my job to start memes ;)
<lifeless> wgrant: that doctests are actively harmful
<wgrant> The anti-doctest meme?
<wgrant> Right.
<wgrant> I now hate them with a passion.
<lifeless> \o/
<wgrant> A few months ago I merely mildly disliked them.
 * cody-somerville has slowly been seeding an SOA meme.
<wgrant> Has it caught on yet?
<cody-somerville> Seem to be
<wgrant> Excellent.
<cody-somerville> Although I think lifeless might extinguish my efforts :/
<wgrant> :(
<wgrant> Why?
<cody-somerville> I remember briefly skimming minutes for a meeting I missed that noted that lifeless thought splitting launchpad up into smaller, discrete services was a distraction.
<wgrant> It is a distraction. But probably a beneficial distraction.
<cody-somerville> Indeed. Which is why I hold out hope that the comment isn't as fatal sounding as it does.
<mtaylor> cody-somerville: fwiw, I support your interest in splitting
<cody-somerville> :)
<mtaylor> of course, I have ZAROO vote
<mtaylor> but smaller discrete units is pretty much a best practice, especially when you want to scale (hi facebook)
<cody-somerville> or hello amazon
<mtaylor> or really anyone
<thumper> mtaylor: morning
<thumper> mtaylor: how's your summit going?
<spm> thumper: we have a project that has been renamed. we're trying to reset the lp:<project> alias; but that failed. now, whenever we try and re-create 'trunk' we keep getting 'you are being stacked on <junk branch as placeholder>' which is painful. suggestions?
<thumper> hmm...
<thumper> I'm just relocating, back in a few minutes
<thumper> pastebin more details please
<spm> kk
<poolie> rockstar, https://dev.launchpad.net/LEP/DynamicConfiguration
<rockstar> poolie, thanks
<mwhudson> bigjools: where did you go?
<bigjools> mwhudson: haydn
<mwhudson> bigjools: want to talk about buildd-manager now, or busy with other things?
<bigjools> we're talking about derived distros, might be useful for you?
<mwhudson> ah sure
<mwhudson> be along in a sec
<bigjools> but yes I'd like to talk about that too :)
<mwhudson> bigjools.fork()
<wgrant> jml: We still can't renew our memberships. Maybe you could reset everyone to not expire?
<wgrant> (you just changed the default for new members, it appears)
<deryck> gmb, https://code.edge.launchpad.net/~deryck/launchpad/move-bugs-js-files-again/+merge/29776
<jml> wgrant, there isn't a control for that
<wgrant> jml: It's not exposed through the API?
<jml> wgrant, I don't know.
<poolie> hi wgrant
<wgrant> Hi poolie.
<poolie> benji, leonardr: https://bugs.edge.launchpad.net/launchpadlib/+bug/604963 for this issue
<_mup_> Bug #604963: shouldn't need to close window and press enter after authenticating <launchpadlib :New> <https://launchpad.net/bugs/604963>
<jml> lifeless, noodles775: btw, if you want to see some tests that use fake objects rather than the database, lp.code.xmlrpc.tests.test_codehosting (tests for the fake & real implementations); lp.codehosting.vfs.tests (tests that use the fake); lp.codehosting.inmemory (the fake itself)
<jml> running the test_codehosting tests makes the database overhead for testing really obvious
<poolie> leonardr, http://sourcefrog.net/tmp/1y/closeme.html
<noodles775> jml: will do, thanks.
<jtv> jam: hi!  Partial work on dropping BranchRevision.id: http://paste.ubuntu.com/462977/  Still failing these tests or a subset thereof: http://paste.ubuntu.com/462978/
<jam> jtv: thanks for the headsup
<leonardr> mars, src/lazr/restful/tales.py:from epydoc.markup import DocstringLinker
<mtaylor> morning all
<lifeless> mtaylor: hai
<lifeless> mtaylor: I'd like to suggest a warm up task for you, in lp stuff,if I may.
<mtaylor> lifeless: yes!
<lifeless> make it fast
<lifeless> can you see http://people.canonical.com/~stub/ppr_test.html?
<mtaylor> lifeless: ok
<thumper> hi mtaylor
<mtaylor> lifeless: I would like lp to be fast
<mtaylor> hi thumper
<mtaylor> thumper: did you see the bug I just filed? :)
<thumper> um...
<thumper> which one?
<mtaylor> bug#605130
<_mup_> Bug #605130: subscribed team cannot view private branch <Launchpad itself:New> <https://launchpad.net/bugs/605130>
<thumper> mtaylor: probably because the branch is owned by a private team
<mtaylor> thumper: probably
<mtaylor> thumper: but bzr branching it works
<thumper> heh...
<mtaylor> thumper: I probably won't care anymore by the time you have time for it - but I thought I'd point it out :)
<lifeless> so
<lifeless> this is the same as PPPA access
<thumper> that's because the code path for accessing the branch over bzr doesn't look at the team
<lifeless> we're going to be adding a 'Traversal' permission.
<lifeless> which will be used to make sure such pages can render reasonably. granting read access to something with a url path containing private things will imply access to traverse them
<lifeless> and traversing means 'show the url and a human name for the thing'
<lifeless> mtaylor: so, can you see that page?
<mtaylor> lifeless: yup
<lifeless> cool
<lifeless> mtaylor: and see how its got blueprint bits in it ?
<lifeless> mtaylor: and see how 99% of blueprint pages complete within 200 sql statements ?!??!1
<lifeless> and 3.2 seconds ?
<lifeless> thats pretty slow ;)
<lifeless> mtaylor: and with that, gnight
<mtaylor> lifeless: mk
<mtaylor> lifeless: I will fix
<lifeless> mtaylor: mean is 50 sql statements, which is still pretty terrible.
<lifeless> mtaylor: yeah, if its fixed, be in a good position to add features.
<lifeless> gnight.
#launchpad-dev 2010-07-14
<wgrant> lifeless: Ah, I'm glad you have a better solution than granting launchpad.View on private teams just because of a PPA subscription.
<lifeless> wgrant: I try
<mwhudson> morning
<wgrant> Evening.
<lifeless> day.
<StevenK> G'Tag
<wgrant> That's not very Czech.
<StevenK> Oh, sorry. DzieÅ dobry
<poolie> leonardr, wbn to extract whatever groundcontrol uses for this and make it sit on top of a generic authenticate mechanism, and be the default gtk impl
<leonardr> poolie, i don't think it's worth it, given that the common case will be to use the credentials manager packaged with ubuntu
<poolie> jml wow ec2 test is pretty classy
<lifeless> poolie: you've seen bzr selftest --parallel=ec2 ?
<poolie> heard of it
<lifeless> :P
<poolie> i think i tried it and hit some snags
<poolie> i think for me it looked like a bad latency/throughput tradeoff :)
<poolie> maybe i should try it
<wgrant> But with LP it's different, since the test suite is damn slow?
<poolie> lifeless,  i'd actually like a variation that runs remotely on my desktop
<poolie> yeah basically
<leonardr> poolie, want to review the launchpadlib branch we talked about yesterday?
<leonardr> https://code.edge.launchpad.net/~leonardr/launchpadlib/improve-workflow/+merge/29849
<poolie> i'd be delighted
<leonardr> poolie: the reason there are no automated tests is we don't have the framework for them
<poolie> leonardr, you know when i said 'rhinos' i meant the internal mailing list about this? not the animals?
<leonardr> poolie: i thought you meant the o'reilly book (after whom the list is presumably named)
<mwhudson> the problem with launchpad for ec2 is that instance setup is quite slow of course
<mwhudson> we could have an instance with the results of 'make schema' baked into it, that would help a bit
<poolie> nice cover letter
<poolie> leonardr, reviewed, very nice but i have some tweaks
<leonardr> ok
<leonardr> poolie, maybe you can help me understand how os.open works
<poolie> leonardr, maybe i can :)
 * poolie waves
<wgrant> Is ShipIt still a Launchpad parasite?
<wgrant> Ah, lunch, I guess.
<mwhudson> wgrant: yes
<mwhudson> wgrant: to both bits
<wgrant> mwhudson: :(
<mwhudson> wgrant: although there was talk of not rolling out launchpad to shipit app servers any more
<mwhudson> gary_poster: it seems our VirtualHostRequestPublicationFactory doesn't really implement IRequestPublicationFactory, because __call__ returns a request *factory*, not a request
<mwhudson> gary_poster: does this ring any bells?
<mwhudson> or hm
<mwhudson> maybe the docs are just broken
<mwhudson> yeah, i think so
<gary_poster> mwhudson: (Sorry no reply before) That sounds likely to me, fwiw.  If it works with the rest of the Zope machinery, then that sounds like a good argument.
<mwhudson> gary_poster: np
<maxb> I appears there is one or more Git imports in the Reviewed state which somehow I am Forbidden to see
<maxb> This manifests as a Forbidden error when I try to page through the list of them, on one particular page
<maxb> It appears to be number 459 in the list
<maxb> and mumber 498 too
<mwhudson> maxb: hmm
<mwhudson> maxb: sounds worth of bug filing
<maxb> https://code.edge.launchpad.net/+code-imports/+index?field.rcs_type=GIT&field.rcs_type-empty-marker=1&field.review_status=REVIEWED&field.review_status-empty-marker=1&submit=Submit+Query&start=497&batch=1
<maxb> Can you view that?
<maxb> https://code.edge.launchpad.net/+code-imports/+index?field.rcs_type=GIT&field.rcs_type-empty-marker=1&field.review_status=REVIEWED&field.review_status-empty-marker=1&submit=Submit+Query&start=458&batch=1
<maxb> or that?
<maxb> hoping to find someone to figure out which ones are the problem, so I can say something more definite than an index in search results which will change
<mwhudson> maxb: i can't see that
<maxb> hmm. I guess we'll need a losa
<mwhudson> but i can see the branch name in the traceback
<mwhudson> i assume it's a private branch-only project
<maxb> makes sense
<james_w> leonardr: hi, is there a solution for using launchpadlib.load in (launchpad) tests that doesn't involve hardcoding the root uri or accessing a private attribute? I thought I had seen a branch from you to make load work with relative URIs.
<leonardr> james_w: yes, in the latest version of launchpadlib you can pass a relative url to load()
<leonardr> or, actually, in the latest lazr.restful
<james_w> leonardr: is that landed in launchpad yet?
<leonardr> lazr.restfulclient
<leonardr> yeah, it's lazr.restfulclient 0.9.20, revno 102
<james_w> grep restfulclient versions.cfg
<james_w> lazr.restfulclient = 0.9.14
<leonardr> oh, i see what you mean
<leonardr> is it part of the launchpad application, not is it on the launchpad site
<james_w> I'll try an upgrade branch
<leonardr> i'll do a branch upgrading the packages later this month
<leonardr> remind me to do it, if you would
<james_w> leonardr: you don't want me to do it now?
<james_w> leonardr: is there some way that I can mark the place in the code where I will put the ugliness for replacement at that point?
<leonardr> james_w: sure, if you want to take care of it i'd be very happy
<leonardr> i didn't know that was what you meant by 'upgrade branch'
<james_w> leonardr: latest versions of lazr.restfulclient, launchpadlib and lazr.restful?
<leonardr> let me double check, but yes
<james_w> i.e. should I upgrade all of them?
<leonardr> james_w: yes. send me the mp, but if the ec2 test passes, it's fine
<james_w> leonardr: any gotchas you can think of that would save me time?
<james_w> any dependencies that should be upgraded or anything?
<leonardr> james_w: the only thing i'm worried about is whether 'make' will generate a reasonable-looking apidoc
<leonardr> wait, actually it doesn't matter, because i didn't release that launchpadlib revision as a new version of launchpadlib
<leonardr> so it should be fine
<james_w> ok
<poolie> hi james_w
<james_w> hi poolie
<poolie> new post: http://blog.launchpad.net/api/three-tips-for-faster-launchpadlib-api-clients
<poolie> lifeless, one nice thing about having everything in debs would be less crap about 'please run link-external-sourcecode' etc
<james_w> I'm actually starting to like the approach taken by buildout etc. for development
<benji> poolie: utilities/link-external-sourcecode -p ~/launchpad/lp-sourcedeps/
<james_w> poolie: co-located branches would make that less of an issue too :-)
<poolie> true
<poolie> hm, though you'd still need to rebuild them etc
<mwhudson> writing unit tests for publication details makes me want to stab things
<mwhudson> ILaunchBag needs to die
<poolie> ILaunchBagBiter
<mwhudson> ILaunchPoo
<kiko> flacoste, mwhudson: who currently runs our mootbot?
<mwhudson> kiko: no idea
<leonardr> poolie, https://bugs.edge.launchpad.net/launchpadlib/+bug/605462
<mwhudson> gary_poster: this diff adds a vostok layer: http://pastebin.ubuntu.com/463574/
<gary_poster> looking
<mwhudson> gary_poster: this one adds a very simple new view for the root http://pastebin.ubuntu.com/463575/
<gary_poster> mwhudson: for first patch, I don't understand change to lib/lp/code/xmlrpc/tests/test_codehosting.py : why did it say "vostok" in the first place?
<mwhudson> gary_poster: the scripts used to be run on a machine called that, i guess i went for 'realistic' test data
<gary_poster> heh, what a coincidence
<mwhudson> yeah
<mwhudson> i changed it because i want grep -ir vostok to only find this stuff :)
<gary_poster> heh, cool, makes sense
<gary_poster> mwhudson: oooh, I like your root template
<mwhudson> gary_poster: :)
<gary_poster> mwhudson: a 50% review: looks good to me.  It is sad that it requires so much machinery, though we do this so infrequently I suppose it is not worth thinking about
<mwhudson> gary_poster: thanks
<james_w> you know, it might be useful for us to have vostok.launchpad.net as an "unsupported" interface to launchpad.net.
<mwhudson> james_w: i guess you looked at the diffs
<james_w> I did
<mwhudson> cool
<mtaylor> thumper: around?
<mwhudson> mtaylor: sorta
<mwhudson> mtaylor: if you say stuff he'll see it i imagine
<mtaylor> mwhudson: heh. something seems to be quite off with merge proposal diff
<mtaylor> diffs
<mwhudson> mtaylor: in what way?
<mtaylor> I've got three up: https://code.edge.launchpad.net/swift/+activereviews (from pipeline, depend on each other) and the unmerged revision list for each is off
<mtaylor> as is the diff that was emailed and the one that is displayed in the UI
<mtaylor> mwhudson: if you don't have super-launchpad perms, I'll have to add you to a team so you can see it
<mwhudson> mtaylor: i don't have the perms any more
<mtaylor> mwhudson: ok. lemme add you real quick
<mtaylor> mwhudson: if you look at https://code.edge.launchpad.net/~mordred/swift/add-doc-package/+merge/29896
<mtaylor> mwhudson: that merge should contain revs 16 and 17 - but even if it just contained rev 17 the diff still should not be empty
<mwhudson> mtaylor: bah, i can't see that
<mwhudson> Unauthorized: (<Branch u'~mordred/swift/add-doc-package' (361880)>, 'landing_targets', 'launchpad.View')<br />
<mtaylor> hrm. lemme add you somewhere else...
<mtaylor> mwhudson: try again
<mwhudson> success
<mtaylor> woot
<mwhudson> mtaylor: i think the unmerged revisions list being too big has been like that always
<mtaylor> mwhudson: no - that's not the problem ...
<mtaylor> mwhudson: sorry, I didn't mean unmerged revs...
<mtaylor> mwhudson: if you look at the grey list of revs under "Add a review or comment" ... it should be listing 16 as well... and then the diff should be quite different/larger
<mwhudson> mtaylor: thumper agrees that it's ****ed
<mtaylor> yay
<mwhudson> (he's now sitting next to me)
 * mtaylor cries a bit
 * mtaylor was going to teach new people how to work with merge requests and why they're wonderful today... 
<mtaylor> is it possible that is's ****ed because of all the private nonsense?
<mwhudson> mtaylor: trying to reproduce something
<mwhudson> mtaylor: very unlikely
<mtaylor> ok
<mwhudson> mtaylor: can you delete the problem proposal and try again?
<mtaylor> mwhudson: sure
<mtaylor> mwhudson: there are two problematic ones... shall I delete bothj?
<mwhudson> mtaylor: which is the other problem one?
<mwhudson> but no, just delete one for now
<mtaylor> https://code.edge.launchpad.net/~mordred/swift/rework-dh-install
<mtaylor> ok
<mwhudson> mtaylor: yeah, the diff on https://code.edge.launchpad.net/~mordred/swift/rework-dh-install/+merge/29891 is just the diff from r13 isn't it?
<thumper> mtaylor: hi
<thumper> mtaylor: I've opened my laptop again
<mwhudson> mtaylor: same empty diff again
<mtaylor> thumper: hi!
<mtaylor> mwhudson: bleh
<mwhudson> mtaylor: yes
<thumper> mtaylor: I'm sitting next to abentley discussing your diff
<thumper> or lack thereof
<mtaylor> thumper: did I break somethignm
<mtaylor> ?
<thumper> mtaylor: maybe
<thumper> mtaylor: I seem to recall something like this happening before
<thumper> but I don't remember why
<thumper> or what we did about it
<mtaylor> yay!
<mtaylor> thumper: I did push the first branch (~mordred/swift/bzr-builddeb) ... then I reconfigured to pipeline
<mtaylor> then I re-pushed it
<mtaylor> not sure if it's relevant
<mwhudson> mtaylor: you should probably remove me from those teams
<mtaylor> mwhudson: I will
<mtaylor> mwhudson: do you not need it to see things anymore?
<mwhudson> mtaylor: i'm letting thumper and abentley worry about it :-)
<mtaylor> mwhudson: cool
<thumper> mtaylor: bad news
<thumper> mtaylor: when we manually do what the script should be doing
<thumper> we get a diff
<thumper> so...
<thumper> sounds like a bug in our diff generation
<mtaylor> yay!
<thumper> please demonstrate with a different project
<mtaylor> thumper: should I try deleting an recreating all three merge props?
<mtaylor> mwhudson: done
<mwhudson> mtaylor: cool
<thumper> mtaylor: you could try adding a rev to the source branch of the one with the bad diff
<thumper> this would force a new diff to be generated
<thumper> alternatively, try merging the prerequisite into the source
<thumper> if the tip isn't already
<thumper> it may be an edge case
<james_w> no ec2test mail makes james_w a sad panda
<james_w> phew, there it is, just really slow
#launchpad-dev 2010-07-15
<james_w> I write a bazillion tests without logging in, then I add one more that accesses person.name and it fails because I haven't logged in. That seems rather arbitrary, is it because of openid?
<wgrant> james_w: What sort of stuff were you doing beforehand?
<wgrant> You weren't conveniently avoiding security proxies?
<james_w> I have no idea
<wgrant> OOPS-1656A838
<james_w> > 500 lines of tests, creating lots of objects with the factory and accessing them over the API
<james_w> then as soon as I access person.name it fails, but product.name etc. is fine
<wgrant> That is odd.
<james_w> wgrant: are you interested in what that oops is?
<wgrant> james_w: I am.
<james_w> hang on, I caused it?
<wgrant> Did you?
<james_w> yeah
<wgrant> I got it from ScottK.
<james_w> huh
<james_w> oh, hang on
<james_w> brain fail
<james_w> http://paste.ubuntu.com/463767/
<james_w> it's that taking ages
<wgrant> Seriously?
<wgrant> How long did it take?
<james_w> 20s
<wgrant> Must be a locking issue.
<james_w> yeah
<james_w> 1 repetition
<james_w> ajmitch: I hope you don't mind, but I carried on your work with https://code.edge.launchpad.net/~james-w/launchpad/expose-blueprints
<ajmitch> james_w: ah well
 * ajmitch hadn't pushed his updated stuff
<james_w> damn
<ajmitch> doesn't matter, it's best done by someone who knows what they're doing
<james_w> do you want to merge my changes in and finish it off?
<ajmitch> I can take a look, I just didn't have time to write up tests for it
<ajmitch> that'll teach me to not push changes :)
<dhastha> Need help:  I am trying to install Launchpad locally in
<dhastha> Virtual machine manager
<dhastha> But Virtual machine manager returns :  Unable to complete install: 'internal error unable to start guest: char device redirected to /dev/pts/0
<dhastha> qemu: could not open disk image /var/lib/libvirt/images/UbuntuServer.img: No such file or directory
<dhastha> How to install ubuntu server in Virtual machine manager?
<wgrant> dhastha: You may want to ask #ubuntu or perhaps #ubuntu-server
<sinzui> benji, leonardr, bug 146389 has an interesting branch
<_mup_> Bug #146389: api for blueprint tracker <api> <feature> <Launchpad Blueprints:In Progress> <https://launchpad.net/bugs/146389>
<mwhudson> morning
<lifeless> mthaddon: around? looks like we have a vandal
<mthaddon> lifeless: wassup?
<lifeless> https://edge.launchpad.net/~jmkuhn007 is making fairly random changes to bugs in the launchpad suite
<lifeless> https://bugs.edge.launchpad.net/launchpad-registry/+bug/602771
<lifeless> and
<_mup_> Bug #602771: Private Member <privacy> <Launchpad Registry:In Progress by bac> <https://launchpad.net/bugs/602771>
<lifeless> https://bugs.launchpad.net/bugs/99519
<lifeless> so far
<_mup_> Bug #99519: Team Registry - Push yourself find out how we can do this. <confusing-ui> <javascript> <story-logos> <Launchpad Registry:Triaged by jmkuhn007> <https://launchpad.net/bugs/99519>
<mthaddon> lifeless: has anyone contacted him about this?
<lifeless> not yet
<bac> mthaddon, lifeless : sinzui may have contacted him in the past
<mthaddon> probably worth checking that I would think
<lifeless> lp doesn't show that. Perhaps it should ?
<lifeless> sinzui: can you confirm - have you spoken to this account before - are they aware they are being disruptive ?
<wgrant> I recall he was doing some strange (but not particularly harmful) stuff a couple of weeks ago.
<lifeless> wgrant: thanks
<mtaylor> morning lifeless
<lifeless> hi mthaddon
<lifeless> bah
<lifeless> hi mtaylor
<mtaylor> heh
<mthaddon> :)
<mtaylor> how's prague?
<thumper> mtaylor: he is supposed to be listening, don't distract him :)
<mtaylor> do you guys run tarmac as a daemon? or do you just run it by hand?
<mtaylor> thumper: bah!
<thumper> mtaylor: we still use pqm
<thumper> others use tarmac though
<mtaylor> ah. so you're going to be no help with my tarmac questions then :)
<thumper> rockstar: tarmac question
<rockstar> mtaylor, hi
<mtaylor> hey rockstar
<rockstar> mtaylor, it's run as a cron script.
<mtaylor> rockstar: ok. so that's best-practice for it atm
<rockstar> I believe dobey wanted to run it as a daemon, but I don't see the point.
<rockstar> mtaylor, well, if by "best practice" you mean "only option," then yes.
<mtaylor> hahaha
<rockstar> mtaylor, are you setting up Tarmac?
<mtaylor> rockstar: so then ... if it keeps telling me "No approved proposals found", yet I _do_ have one, am I just stupid?
<mtaylor> rockstar: yes.
<lifeless> mtaylor: stupid, for sure.
<mtaylor> rockstar: I've got two projects starting up, and I'd like to get the teams hacking on them started not pushing to trunk themselves
<rockstar> mtaylor, so, the current criteria is that the merge proposal is approved and has a commit message set.
<mtaylor> lifeless: ok. bad question. I'm always stupid...
<rockstar> mtaylor, this will be less ass when we have merge queues.
<mtaylor> rockstar: yes. agreed
<rockstar> mtaylor, are you using trunk (please Thor say yes)
<mtaylor> rockstar: and tits... I looked at the merge prop and thought I'd set a commit message but hadn't
<mtaylor> rockstar: yes
<lifeless> I'm not 100% convinced by tim's needs-split stuff; I should talk to him today about this.
<lifeless> as in , its clearly better, but is it needed to move forward...
<rockstar> lifeless, regardless of the needs-split stuff, we need merge queues.
<lifeless> rockstar: ack
<mtaylor> rockstar: at some point I'm going to want to talk to you in more detail about tarmac... as the next step is figuring out the "right" way to integrate all of this with hudson
<rockstar> Right now, Tarmac just grabs a glob of merge proposals instead of a list of merge proposals.
<rockstar> mtaylor, ooh, I'm totally down for getting it integrated into Hudson.
 * mwhudson twitches
<rockstar> mtaylor, you are aware that Tarmac is my pet project right?
<mtaylor> rockstar: excellent... my plans are coming together
<mtaylor> mwhudson: was that a good twitch or a bad one?
<rockstar> mwhudson, your interface is undocumented, so I won't be integrating with you anytime soon.
<lifeless> rockstar: btw I'm a hudson committer, if you need info gimme a shout
<rockstar> lifeless, great.
<mtaylor> rockstar: biggest problem I've got now is duplication of effort/impedence mismatch in queue management
<mtaylor> lifeless: me too me too ... but I love help :)
 * rockstar sometimes forgets to feed his pet project, or let it out, so it shits on the floor sometimes...
<lifeless> mtaylor: I don't think you've touched the core yet have you ?
<mtaylor> lifeless: uh... no
<lifeless> mtaylor: :P
<mtaylor> lifeless: but I _do_ have commit access :)
<lifeless> mtaylor: yes, I know. Its great.
<mtaylor> rockstar: oooh. shit on floor. that should be cleaned
<lifeless> that should be on the quotes page
<rockstar> mtaylor, yeah, I think I'll get to it this weekend, instead of, you know, walking around and seeing Prague.
<mtaylor> rockstar: sweet
<mtaylor> rockstar: I've actually got a patch sitting around on my laptop ... I should push it up for you
<rockstar> mtaylor, for Tarmac?
<mtaylor> rockstar: yeah - I was trying to use it on drizzle a while back and kept hitting an exception
<rockstar> mtaylor, I'm currently thumper's review bitch, so I can do a review and merge for Tarmac.
<mtaylor> assert config.has_section(lp_branch.bzr_identity)
<mtaylor> that's unhappy
<rockstar> mtaylor, yeah, I think I fixed that, but you might have a new case.
<mtaylor> rockstar: oh, this just happened to me right now
<mtaylor> rockstar: sorry, that's not what I fixed
<rockstar> For context, Tarmac worked pretty well on 0.2.  It worked so well that I had to re-write it, because it didn't have enough bugs.
<mtaylor> fantastic
<mtaylor> rockstar: that assert seems to indicate to me that I want a section ... ah, I want a config section named after the branch
<mtaylor> rockstar: /me wags finger at the documentation...
<rockstar> mtaylor, the docs are still old.  I wrote the code first, and then I was going to document it.
<rockstar> mtaylor, if you'd stop finding bugs, I'd write some documentation, so in a way, this is your fault.
<mtaylor> Running test command: python setup.py test
<mtaylor> rockstar: hah
<mtaylor> rockstar: um, the python setup.py test command did not actually seem to actually happen
<rockstar> mtaylor, I remember talking about this recently.  There was something weird.
<mtaylor> rockstar: oh wait - pebkac
<rockstar> mtaylor, make sure to file a bug on yourself then.
<james_w> leonardr: there's a bunch of branches on https://code.edge.launchpad.net/~james-w/lazr.restfulclient/+activereviews that are going stale
<james_w> leonardr: are you interested in having any of them in trunk?
<leonardr> james_w: probably, i haven't looked recently, sorry
<leonardr> i'm about to review a launchpad branch of yours
<lifeless> mthaddon: ping
<wgrant> Yay for Vostok sanity.
<wgrant> This makes much more sense.
<lifeless> wgrant: isn't it awful-am for you?
<lifeless> wgrant: what does?
<wgrant> lifeless: Not even 1am...
<nigelb> more like awful pm
<mthaddon> lifeless: hi
<wgrant> lifeless: The whole lets-not-actually-rewrite-everything-from-scratch thing.
<lifeless> mthaddon: the daily oops reports are looking good, but the graph is a bit jerky.
<mwhudson> wgrant: yes
<wgrant> Which is rather different from the idea that was floated a few weeks ago.
<mthaddon> lifeless: daily or hourly?
<lifeless> wgrant: oh; well writing something new just gives you new bugs
<lifeless> mthaddon: hourly
<mthaddon> lifeless: yep, we've had some teething troubles getting the cronjob working properly, and if devpad goes down we miss an entry
<lifeless> urgl
<mthaddon> lifeless: the teething troubles are fixed - devpad going down shouldn't be so often as to cause too much pain, we've just been unlucky
<lifeless> ok cool
<lifeless> tomorrow then we'll lower the timeout! woo!
<leonardr> james_w, i've merged ensure-representations-are-json. ping me next week and i'll spend some time answering your other-branch questions, if i don't get to them before them
<james_w> thanks
<sinzui> jam: http://pastebin.ubuntu.com/464106/
<mwhudson> hmm
<mwhudson> ./bin/test -vvvc lp.code.model.tests took ~15 minutes
<mwhudson> slightly more than 3 minutes were in _LaunchpadObjectFactory.makePerson_
<poolie> lifeless, wget --> https://bugs.edge.launchpad.net/launchpad-foundations/+bug/605866
<_mup_> Bug #605866: anonymous readonly api requests should not require an OAuth key <Launchpad Foundations:New> <https://launchpad.net/bugs/605866>
<james_w> where do I get pocketlint?
<mwhudson> james_w: ~launchpad ppa
<james_w> oh yay, package name is different to command name, and doesn't mention the command in the description, so apt-cache search pocketlint doesn't work
<Sary> Syntax: REGISTER <#ubuntu-sa>
#launchpad-dev 2010-07-16
<lifeless> mthaddon: hai
<spm> lifeless: can I assist?
<lifeless> I want to change the prod timeouts.
<lifeless> and edge
<spm> which ones?
<lifeless> db timeout and soft timeout
<spm> db timeout being the 20secs and then we oops? I suspect stub'll know where that hides.
<lifeless> oh, I know where it is.
<lifeless> where is the prod config branch ?
<spm> oh. sorry, I misunderstood the Q.
<spm> lp:~launchpad-pqm/lp-production-configs/trunk
<lifeless> https://code.edge.launchpad.net/~lifeless/lp-production-configs/timeouts/+merge/30069
<spm> i'd grumble at the 'updating diff' message, but that'd be my problem to chase. so I won't.
<lifeless> spm: ^ I want that to be live on the prod,edge,staging appservers, please. How do we make that happen ?
<spm> lifeless: edge (I'm pretty sure on edge and config updates, but not 100%) and staging will just happen; tho if you want faster, edge can be manually done. prod is a CP.
<lifeless> spm: its a CP to change the _config_ ? if so fine, tell me what to do.
<lifeless> spm: I'd like to do this broadly in lockstep so that the oops hourly graphs are comparable
<spm> lifeless: we don't really distinguish between code vs config changes. they all follow much the same process.
<spm> lifeless: so the request will be (after approval/buildbotting etc) wind up here: https://wiki.canonical.com/InformationInfrastructure/OSA/LaunchpadProductionStatus there's a link off that to the process for getting a change thru
<lifeless> spm: please tell me you're kidding about buildbot for this?
<lifeless> buildbot doesn't use the production configs AFAIK
<lifeless> so I'm entirely unconvinced it has any use here
<spm> hrm. well I was speaking to the general case not the specific. I guess we do send all stuff thru BB as a matter of course. If only such that the appropriate branhces wind up in the appropriate spots.
<spm> In the case where a change is sufficiently ZOMGness, we can and do cowboy in.
<lifeless> well, have a look at it.
<leonardr> https://bugs.edge.launchpad.net/wadllib/+bug/274074
<_mup_> Bug #274074: Missing total_size on collections returned by named operations <api> <wadllib:Fix Released by leonardr> <https://launchpad.net/bugs/274074>
<salgado> mwhudson, you'll need the LP branch plus http://paste.ubuntu.com/464412/ and a checkout of lp:~salgado/lazr.restful/extension-interfaces on lazr-restful-dev-egg
<mwhudson> salgado: ta
<lifeless> spm: hi
<lifeless> spm: I realise its well past your off-time, but we got left half-discussed; If you don't reply I'm going to assume you're having fun and will follow up with mthaddon
<spm> heh, having fun trying to keep codebrowse alive :-)
<lifeless> spm: ouch
<lifeless> mthaddon: whats the pqm info for the prod configs branch ?
<mthaddon> lifeless: as in, which branch to submit to? bzr+ssh://bazaar.launchpad.net/~launchpad-pqm/lp-production-configs/trunk
<lifeless> and the pqm email box ?
<lifeless> mthaddon: ^
<mthaddon> lifeless: same as for all the LP branches
<lifeless> mthaddon: its been many years since I had that setup
<lifeless> mthaddon: ec2land does it magically
<lifeless> hmm, might it be pqm at pqm dot ubuntu dot com ?
<mthaddon> lifeless: there's no developer docs on that? I can find out for you, but I suspect it should be documented for developers somewhere
<StevenK> pqm_email = Launchpad PQM <launchpad@pqm.canonical.com>
<StevenK> lifeless, mthaddon: ^
<lifeless> mthaddon: the docs are a little awkward right now :P
<mthaddon> thx - not quite the developer docs I had in mind, but... :)
<lifeless> StevenK: thanks!
<deryck> sinzui, I have a simple branch needs a review and yours could qualify for code and ui...
<deryck> sinzui, I've just changed a help link that is failing.  Can you help with this?
<poolie> stub, could you test/land my flags db patch?
<poolie> i ran ec2 test but it seemed to hang
<stub> poolie: ok
<poolie> or at least the web page did not show progress
<poolie> and i suspect it couldn't authenticate to send mail
<lifeless> can we have lights please ?
<mwhudson> salgado: ICanHaveBugs is a bit of a hack, no?
<salgado> mwhudson, yes; created just so that we can have the IBugTarget views also available for things that don't directly provide that interface but can be adapted to it
<mwhudson> salgado: what's the longer term strategy there?
<lifeless> mthaddon: hi
<lifeless> [.*(?:\\[(?:[pP]=[^ \\t]+,[ ]*)?(?:[rR][sS]?=[^ \\t]+)|(?:[Tt][Rr][Ii][Vv][Ii][Aa][Ll])\\])]'
<lifeless> mthaddon: what should I put in to make both you and PQM happy, here ;)
<mthaddon> lifeless: https://code.edge.launchpad.net/~launchpad-pqm/lp-production-configs/trunk will give you ideas about ones that have worked in the past
<lifeless> lol
<mwhudson> can we have pages registered for IBugTarget and have that work for things that can only be adapted to IBugTarget ?
<lifeless> AIUI, no.
<lifeless> However, remember that I know nothing.
<mwhudson> salgado: bzr+ssh://bazaar.launchpad.net/~mwhudson/launchpad/project-to-bugs-adapter fixes the feeds tests btw
<lifeless> deryck: https://bugs.edge.launchpad.net/malone/+bug/606191
<_mup_> Bug #606191: filing a bug created two bugs <Launchpad Bugs:New> <https://launchpad.net/bugs/606191>
<poolie> spm,  can we get any debug info on why loggerhead is crashing to mkanat?
<rockstar> bigjools, can I solicit some help from you?
<bigjools> rockstar: of course, how may I assist on this fine day in Prague?
<rockstar> bigjools, take a look at the diff here: https://code.edge.launchpad.net/~rockstar/launchpad/bug-602333/+merge/29679
<rockstar> bigjools, basically, I need to make sure that distroseries that we create for source package recipe builds have a nominatedarchindep (so it doesn't use the arm builder)
<bigjools> rockstar: "ProcessorFamily.get(2)" ?
<rockstar> bigjools, however, if I try and set the processor or architecturetag on the makeDistroArchSeries, there's a constraint violated.
<rockstar> bigjools, I tried that, but it violates a unique constraint.
<rockstar> I'm assuming this means that there's possibly some sampledata somewhere that I can use instead.
<bigjools> quite probably
<bigjools> which constraint BTW?
<rockstar> bigjools, looking, one sec.
<mwhudson> salgado: Total: 4451 tests, 40 failures, 12 errors in 59 minutes 50.391 seconds.
<mwhudson> from ./bin/test -vvc lp.bugs
<mwhudson> salgado: not too bad, i guess
<lifeless> mthaddon: its landed
<lifeless> mthaddon: what happens next?
<mthaddon> lifeless: you request a rollout via the LPS wiki page - we still need to talk to flacoste about whether you can be added to the list of approvers for that
<lifeless> mthaddon: thats on wiki.canonical.com yeah?
<flacoste> mthaddon: yes, that's fine, lifeless has the responsibilities than Bjorn previously
<mthaddon> flacoste: cool, thx
<mtaylor> lifeless: how's prague treating you?
<leonardr> rockstar: https://bugs.edge.launchpad.net/launchpadlib/+bug/316694
<_mup_> Bug #316694: Add web_link property to resources <launchpadlib :Triaged> <https://launchpad.net/bugs/316694>
<lifeless> mtaylor: could be better, could be worse :)
<mtaylor> lifeless: well that's good I suppose
<mtaylor> lifeless: so, I have a thing I just hit up against which may be a bzr thing or may be a launchpad thing
<poolie> hi monty
<poolie> lp:~leonardr/launchpad/true-anonymous-access - leonardr you legend :)
<mtaylor> hi poolie
<StevenK> poolie: Laptop rescue successful?
<poolie> yes, the water-cooling project was succesful :)
<mtaylor> lifeless: I've got a branch protected by a PQM (lp:swift - using tarmac)
<mtaylor> lifeless: so nothing hits trunk there without going through a merge proposal
<mtaylor> thing is - if the only difference in the tree is that I just tagged a release
<mtaylor> that's always been weird to push, but has sort of always "just worked" ... how do I do that in this workflow?
<mtaylor> (I'm in here because the passive aggressive side of me is going to request a tag-the-branch feature be added to "make a release" ... or something)
<poolie> mtaylor, basically you need to either connect in using a key that lets you get at the bot
<poolie> or put it into a new branch that you merge
<poolie> or perhaps tarmac should get a command to do this
<poolie> leonardr, i think that's all the reviews i need to do for you?
<mtaylor> poolie: so I can do a merge request where the only branch diff is a tag? cool
<poolie> mtaylor, i think you need a new revision, but it doesn't have to have any file changes
<mtaylor> poolie: ok. so I'd want to do "bzr commit --unchanged ; bzr tag blah" and then merge that
<leonardr> poolie, i need you to look at my revisions to https://code.edge.launchpad.net/~leonardr/launchpadlib/improve-workflow/+merge/29849
<poolie> right
<mtaylor> _slight_ ugly in that that'll wind up with the tag in an internal commit
<mtaylor> but will work
<lifeless> mtaylor: I don't know whether tarmac calls the right bzr apis to propogate tags or not. Check the code.
<mtaylor> lifeless: I'm pretty sure its process is "bzr checkout trunk ; bzr merge ; bzr commit"
<lifeless> right see the bug filed last week about this
<mtaylor> lifeless: oh good
<lifeless> bzr commit in a heavyweight checkout doesn't propogate tags.
<lifeless> it should
<lifeless> please fix!
<mtaylor> rockstar: ^^^
<mtaylor> rockstar: although what I _really_ want is a way to tell tarmac to tag a version
<rockstar> mtaylor, can you file a bug about that?
<mtaylor> rather than needing to merge in a tag revision
<mtaylor> rockstar: yes
<mtaylor> lifeless: we don't have tagging done in the hudson bzr plugin do we?
<rockstar> lifeless, it's using a lightweight checkout.
<rockstar> mtaylor, how would you expect to specify a tag?
 * mtaylor goes to write a hudson param job that will tag a branch
<lifeless> rockstar: ok cool that should be fine then
<mtaylor> rockstar: not sure
<rockstar> mtaylor, I think you probably should just tag after the fact.
<mtaylor> rockstar: it's an element of trunk-protected-by-pqm workflow I've only just now thought about
<mtaylor> rockstar: totally - the question is, if my hudson is the only one with permission to do it- what's the least ridiculous way to say "yeah, this one is going to be 1.0"
<rockstar> mtaylor, well, you always want someone to be able to get to the branch that is robotically managed.
<mtaylor> what if there was a launchpad thing, similar to propose merge which was propose release...
<mtaylor> rockstar: well, I can
<rockstar> For instance, our SAs can still edit the branch.
<rockstar> mtaylor, you should also make a bug about proposing a release.
<mtaylor> rockstar: but in a general workflow/process description, "ssh in to the hudson machine as the hudson user, branch trunk, tag and push" is sort of silly
<mtaylor> rockstar: I think I will
<poolie> flacoste, that's great about "what's new", do you know there's a bug for that?
<mtaylor> rockstar: I would be a nice workflow addition, potentially
<rockstar> Also, it might be cool if you could tag a branch through the LP web interface.
<mtaylor> rockstar: yes. the github folks have already poked me about that
<flacoste> poolie: yes, it's bug 129943
<_mup_> Bug #129943: Changing home page "What's new" item shouldn't require a code rollout or cherry pick <Launchpad Foundations:Triaged> <https://launchpad.net/bugs/129943>
<mtaylor> although I do not care for github's version of making a tarball for that :)
<rockstar> mtaylor, there's an open patch to loggerhead about making a tarball, but I was concerned that it might have memory issues.
<rockstar> (because loggerhead is REALLY good with memory, and we don't want to ruin that)
<rockstar> </sarcasm>
<mtaylor> rockstar: haha
<rockstar> mtaylor, truthfully, please file a bug on EVERYTHING the github folks poke you about.  Worse thing that could happen is we could mark it "Won't fix" and actually provide a good explanation.
<rockstar> (As opposed to "No, that's stupid")
<mtaylor> rockstar: I care less about making the tarball (as I'd want to use make distcheck or python setup.py sdist in any case) as I do about the workflow around deciding, taging and creating the relase object
<mtaylor> rockstar: indeed
<mtaylor> rockstar: tarmac is totally running the show now. complete rock
<rockstar> mtaylor, WOOT!
<mtaylor> rockstar: I'm also getting a list of bugs to file (mostly feature requests)
<rockstar> mtaylor, nice.  I'll see if I can get them fixed soon.
<mtaylor> rockstar: the least silly one was someone asked if tarmac indicated anything in the commit about the reviewers/approvers
<rockstar> mtaylor, okay, so there's a way to construct a commit message template that can give that information, and the reviewers are also stored as revprops on the merge.
<mtaylor> rockstar: oh they are?
<mtaylor> rockstar: I think the revprops thing is probably fine
<mtaylor> rockstar: I just did a bzr log and didn't see anything
<rockstar> mtaylor, yeah, so I need to write a bunch of documentation this weekend.
<mtaylor> yay!
<mtaylor> rockstar: the silly bug is that if you try to merge a branch that's already merged, it doesn't update the status to merged
<mtaylor> rockstar: as in, somebody proposes for merge, there are no changes, it's approved, there are no changes - tarmac gets it ... it keeps sitting as approved in the merge queue :)
<mtaylor> rockstar: amazingly enough, this did actually happen :)
<rockstar> mtaylor, it doesn't update the status to merged because the mp should be updated on launchpad to merge.
<thumper> hey
<mtaylor> rockstar: it should - but nothing ever gets committed locally
<mtaylor> so nothing gets pushed and thus nothing gets triggered
<mtaylor> hey thumper
<rockstar> mtaylor, okay.  So the bug is "Pointless merge doesn't remove it from the queue"
<mtaylor> rockstar: yup
<mtaylor> rockstar: I'll file it and stuff... I just haven't gotten there yet
<rockstar> mtaylor, cool.
<rockstar> thumper, which room are you in?
<thumper> 424
<rockstar> thumper, coming down...
<thumper> rockstar: in that case, 425
#launchpad-dev 2010-07-17
<wgrant> rockstar: What happens if I delete or cancel an in-progress build?
<wgrant> lp:~edwin-grubbs/+junk/canonical-vim really needs to be advertised more widely.
#launchpad-dev 2010-07-18
<poolie> OOPS-1660S153
#launchpad-dev 2011-07-11
<spm> staging: make: *** No rule to make target `lib/canonical/launchpad/icing/yui/yui/yui-min.js', needed by `lib/canonical/launchpad/icing/build/launchpad.js'.  Stop. <== new problem?
<wgrant> spm: See #launchpad-ops' topic.
<wgrant> spm: I fixed it last week, but then the fix was reverted, so I refixed on Saturday, but then buildbot decided to sulk.
<lifeless> wgrant: I've filed bug 808557 about the TEMP kludge
<_mup_> Bug #808557: no way to control hand-off of temporary list files <Testrepository:Triaged> < https://launchpad.net/bugs/808557 >
<spm> wgrant: haha
<spm> right. no worries. thanks
<lifeless> wgrant: I've landed the --subunit --list fi
<lifeless> x
<wallyworld_> wgrant: there's revsions needing qa which are broken due to longpoll.js not being included in the build. the Makefile should have been updated when the longpol stuff landed. i've done a fix
<wallyworld_> wgrant: can you +1 this and i'll lp-land https://code.launchpad.net/~wallyworld/launchpad/fix-js-build-paths/+merge/67473
<poolie> hi all
<wallyworld_> hello
<wgrant> wallyworld_: '*lib/lp/*/javascript/*'
<wgrant> wallyworld_: Why the prefixed *?
<wallyworld_> wgrant: typo i think
<wallyworld_> it works but needn't be there
<wallyworld_> i'll remove it and double check
<wgrant> So, how does this fix things?
<wallyworld_> wgrant: the * is needed since the expression is in a -path parameter instead of as an argument to find itself
<wallyworld_> but it should be */lib not *lib
<wgrant> wallyworld_: lib is in the cwd.
<wallyworld_> this fixes things by being more permissive in the javascript that is included in the concat into launchpad.js
<wgrant> More permissive how?
<wgrant> It seems to be more restrictive.
<wallyworld_> wgrant: lib is in the cwd but it doesn't work without the */
<wallyworld_> i'm guessing due to how -path works compared to passing the search dir as an argument to find
<wgrant> wallyworld_: ./lib
<poolie> lifeless: i was wondering the other day if 'user notifications' would work well as a split out service too
<poolie> it seemed like they might
<wallyworld_> wgrant: yes, ./lib works too
<lifeless> could do yes
<wallyworld_> i'll use that
<wgrant> wallyworld_: So, the real change here is that -path doesn't treat / specially, I guess. So the * in the middle matches app/longpoll.
<wallyworld_> wgrant: more permissive in that lib/lp/app/longpoll/javascript/* is included whereas before it wasn't
<wallyworld_> wgrant: yes
<wallyworld_> the * in the middle matches app/longpoll
<wallyworld_> and a bucnh of other things, hence the extra exclusions
<wgrant> wallyworld_: Does lib/lp/services really need to be excluded?
<wallyworld_> wgrant: the js files in there were's being packaged previously so i didn't want to include them and risk breaking something
<wallyworld_> weren't
<wgrant> k
<wgrant> inlinehelp.js is evil anyway.
<wallyworld_> we can relax that exclusion later if required
<wgrant> So, fix those three wildcards and it looks OK.
<wallyworld_> three?
<wallyworld_> i fixed the ./lib one
<wgrant> There's 2x *lib, and a *lp
<wallyworld_> wgrant: sure, although that style matches how the other exclusions have been done. but i'll make them both ./lib/...
<wgrant> Excessive wildcards are bad.
<wgrant> Excluding */tests/* was sane.
<wgrant> Because it should match everywhere.
<wgrant> But *lp/services/* is not.
<lifeless> this ssh thing is weird
<lifeless> only affects testr
<lifeless> if I run the command being run by hand: ~/bin/lxc-start-aufs lucid-test-lp $PWD xvfb-run $PWD/bin/test --subunit --list
<lifeless> it doesn't trigger
<lifeless> s/ssh/sudo
<lifeless> ahha
<lifeless> reproduced
<lifeless> cat /dev/null |  ~/bin/lxc-start-aufs lucid-test-lp $PWD xvfb-run $PWD/bin/test --subunit --list
<lifeless> triggers it
<lifeless> as does
<lifeless> cat /dev/null | ~/bin/lxc-start-aufs lucid-test-lp $PWD true
<StevenK> ~/bin/lxc-start-aufs lucid-test-lp $PWD true < /dev/null ?
<wgrant> lifeless: I guess we want to merge today, don't we?
 * StevenK looks at QA for db-stable
<wgrant> lol
<wgrant> Well, it's actually not that bad.
<lifeless> yes we do
 * wgrant does his.
<wgrant> Oh, blah, staging is still down.
<wgrant> Hack time.
<wgrant> LOSA ping.
<spm> yo
<wgrant> spm: can has staging?
<wgrant> In particular, is there a staging restore running right now?
<StevenK> r10711 and r10766, ignoring wgrant's revision.
<lifeless> can you check the db patch application time for stubs dropping unused column patch
<spm> there is one running atm, still doing the nightly
<lifeless> wgrant: ^
<wgrant> lifeless: We are at 15 minutes in total.
<wgrant> I looked this morning.
<lifeless> ok, thats excellent
<wgrant> No, it's awful.
<wgrant> But we'll live.
<wgrant> spm: Hm, the nightly is running?
<StevenK> Haha
<lifeless> its excellent that we're below budget
<wgrant> spm: The tree build should have exploded...
<spm> is from Jul10th, so not sure what's up there
<StevenK> lifeless: Yes, but wgrant thinks the DB patch budget should be in seconds, not minutes.
<wgrant> lifeless: http://paste.ubuntu.com/641617/
<wgrant> spm: Welcome to staging.
<wgrant> spm: So, you should see approximately lots of errors.
<spm> heh, yes
<wgrant> spm: In the earlier tree build steps.
<wgrant> Right?
<wgrant> And the the appserver failing to start and stuff.
<spm> Creating bzr-version-info.py at revno 10767 <== can't see any errors
 * spm checks asuka. we may have version mismatch
<wgrant> make: *** No rule to make target `lib/canonical/launchpad/icing/yui/yui/yui-min.js', needed by `lib/canonical/launchpad/icing/build/launchpad.js'.  Stop.
<wgrant> Should be lots of that.
<spm> 10767
<spm> same there. wtf.
<wgrant> Yes.
<wgrant> But the build should have failed.
<lifeless> StevenK: I think that as well
<lifeless> StevenK: but massive downtime window - which this is - is the old style layout
<spm> wgrant: it doesn't seem to be. which is awesome.
<lifeless> StevenK: for the -new- style stuff, 15m would be untenable
<wgrant> spm: Hmm, I see errors in the logs. The fix is in devel, but it'll be 6ish hours before it hits staging. We manually applied the patch on Saturday to get the full restore working.
<spm> wgrant: no I lie. there's one.
<StevenK> lifeless: Are we using the downtime window to rip out wildcherry so we can fiddle with its disks?
<lifeless> StevenK: yes
<lifeless> we have 90 minutes to cutover, restart servers with new config and apply the patches
<StevenK> Right, I think stub's revision is qa-ok
<wgrant> spm: So, since we need to QA today, please kill the current restore, s@`pwd`@../../../..@ in sourcherry:/srv/code/db-stable/launchpad/buildout.cfg, and then staging_restore.sh quick.
<StevenK> Which only leaves us r10711. Which we need staging for.
<spm> wgrant: nasty code hack applied; rockin'. but this is r10767 fwiw.
<wgrant> spm: That's what I expected, thanks.
<wgrant> spm: How up to date is lp:lp-staging-scripts' copy of staging_restore.sh? I suspect that sourcherry has a cowboyed change to CHECK_NEW_CODE
<lifeless> wgrant: i suspect its nscd / upstart or similar cross over
<wgrant> lifeless: :(
<lifeless> wgrant: ssh has been excluded
<lifeless> can't reproduce if I comment out the container startup
<wgrant> spm: (afaict the CHECK_NEW_CODE path has not been used since early May, so maybe it was changed around then)
<spm> wgrant: they're identical afaict
<spm> both r40
<spm> no cowboys on sourcherry
<wgrant> spm: Have you diffed them? :)
<wgrant> :(
<spm> bzr di/st yeah
<wgrant> Then why is neither branch under CHECK_NEW_CODE ever executed...
<spm> logic bug maybe
<wgrant> (you'll see it logs one of two messages... neither of which have appeared since May)
<wgrant> spm: Any chance you could investigate? At present it never doesn't update, which makes updates rather slow.
<wgrant> Because the nightly jobs run every time.
<wgrant> Even if there are no changes.
<spm> hrm
<wgrant> Which then blocks the world for 7 hours.
<spm> indeed
<wgrant> Meanwhile a new rev landed in db-stable an hour into that run.
<wgrant> But is ignored for 7 hours because staging is just that sort of guy.
<spm> yeah, we know about that nice buglet in our update steps. justhaven't got around to fixing that
<wgrant> Which?
<wgrant> This all worked fine until two months ago :(
<poolie> any opinion on what content-type a tarball download should have?
<poolie> the current mp has just app/binary
<poolie> perhaps making it actually tarball is better
<StevenK> application/x-tar?
<poolie> yeah, i think so
<poolie> istr specifying the compression is a little tricky if we don't want the browser to try to decompress it
<poolie> (and we don't)
<lifeless> theres three: content type, content encoding, transfer encoding
<lifeless> ce and te are about encoding-for-transfer
<lifeless> c-t in your case should bzr application/x-tar-gz IIRC, and ce should be identity
<lifeless> (which is the default)
<lifeless> what some webservers do is ct: application/x-tar and ce: gz, which leads so the bad behaviour you want to avoid
<poolie> yep
<poolie> http://code.google.com/p/chromium/issues/detail?id=83292
<poolie> ew
<poolie> app/x-tgz perhaps
<wgrant> lifeless: I really hope no webserver is that stupid.
<poolie> it's a common problem
<poolie> thus the specific workaround for it in chromium
<poolie> and probably other places
<poolie> it's easy to misconfigure server configuration to say *.gz => ce:gz
<lifeless> wgrant: IIRC apache had it as defaults or something
<wgrant> lifeless: WTF
<lifeless> wgrant: anyhow, it was -really- widespread in the early naughties
<poolie> i think some would say app/binary is safest
<poolie> yes
<wgrant> poolie: You mean application/octet-stream?
<wgrant> That's what I would use.
<poolie> yes
<poolie> that's what i meant
<lifeless> poolie: I suspect x-tar-gz is easier to do programmatically along with x-tar-zip x-tar-lzma etc
<lifeless> I don't think application/octect-stream is any safer
<wgrant> Anything that's likely to tell the browser to keep its dirty fingers out of the file contents is a good thing...
* wgrant changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: - | Critical bugs: 215 - 0:[######=_]:256
<wgrant> http://webnumbr.com/launchpad-critical-bugs is depressing.
<lifeless> stub has landed my work on batchnav
<lifeless> which is awesome
<wgrant> Has he?
<lifeless> we can close some of the silly api things down now
<lifeless> yeah
<lifeless> rev 13392
<lifeless> wgrant: so /tmp doesn't have sudo tickets anyway
<lifeless> wgrant: how was that meant to be related ?
<wgrant> lifeless: No idea.
<wgrant> lifeless: Maybe it was upstart putting stuff there.
<lifeless> I suspect its grab-sudo-source time
<wgrant> But whatever it was, it was somewhat hiding/invalidating/obscuring/destroying/concealing any valid tickets.
<lifeless> madly
<wgrant> And not mounting /tmp fixed it.
<lifeless> if I run the lxc-start by hand nothing breaks
<wgrant> :D
<lifeless> but if I comment that + the ip grep & ssh out its fine
<lifeless> if I comment the ssh and ip out on their own it still breaks
<lifeless> and it only breaks when I cat /dev/null | lxc-start-arufs
<poolie_> ** Branch linked: lp:~stevenk/launchpad/kill-bazaar-experts
<poolie_> 8-O
<StevenK> poolie_: You don't like the branch name, or you're surprised I've done it?
<poolie_> i'm glad you did it
<poolie_> i'm scared by the branch name ;-)
<StevenK> Haha
<poolie_> i think we'll have tarball downloads from bzr soon
<poolie_> perhaps ready to be proposed to merge today
<poolie_> lifeless, do you have any performance type concerns about this?
<poolie_> i wonder if it would be worth marking these things as very cacheable
<poolie_> ... probably it should just land first
<lifeless> poolie: I think I've raised them all already
<lifeless>  - we may need additional capacity
<lifeless>  - robots.txt coverage
<lifeless>  - needs to stream to avoid both buffering, high memory use and haproxy killing active requests
<lifeless> poolie: you may have missed some discussion about bzr snapshots though
<poolie> this does stream
<lifeless> poolie: I'm somewhere betwen moderately and highly uncomfortable running non released bzr's these days
<poolie> good point about robots
<poolie> when/where was the discussion?
<lifeless> capacity isn't a reason not to land this, its just a consideration on the ops side
<lifeless> poolie: here, on friday I think
<poolie> istr you once telling me that every mainline commit is a release
<poolie> or more than once :)
<lifeless> poolie: that was before the policy change on deprecations
<poolie> hm
<poolie> that's true chronologically
<lifeless> (and associated expectations)
<poolie> anyhow
<poolie> i believed that the policy was lp would run the betas
<poolie> does that count as released or not?
<lifeless> I don't know : perhaps its best if I lay out my concerns
<poolie> i don't think our current policy is any more likely to break lp than the old one
<poolie> and i think non rigorous data from landing failures supports this
<poolie> but, it's a separate matter
<poolie> go ahead
<lifeless> there are, I think, two facets
<lifeless> one is resourcing
<lifeless> the other is likelyhood of a failure reaching production
<lifeless> on the resourcing side, monthly updates where there are API breaks changes the cost/benefit equation
<lifeless> this is more a flacoste thing to assess
<lifeless> *if* the bzr team drive landing of betas, I don't have a concern here
<poolie> hm
<poolie> well, let's work out what has the best cost benefit, rather than who pays the cost
<lifeless> the other facet is more difficult to reason about, I think, because failures can be so varied
<lifeless> I'm mainly concerned about things like the change to fetch tags (which IIUC isn't in a released bzr yet)
<lifeless> but which broke - and is still broken - svn imports
<poolie> do you want to go to voice?
<lifeless> things which I (perhaps wrongly) expect the beta process to tease out
<lifeless> sure
<lifeless> let me change rooms
* wgrant changed the topic of #launchpad-dev to: devel in RC until r13402 deployable | https://dev.launchpad.net/ | On call reviewer: - | Critical bugs: 215 - 0:[######=_]:256
<StevenK> 215 !? :-(
<StevenK>  /wrists
<spiv> danilos: you'll be happy to know I've merged your current expander-anim into my bmp-inline-diffs and resolved conflicts without much trouble
<nigelb> StevenK: I share the same sentiment :)
<nigelb> stub: Thanks to the world being a small world, I recently had a chat with someone at Mozilla who went to college with you :)
<stub> Weird. I've lost contact with nearly everybody I went to uni with :-)
<nigelb> heh, being the Ubuntu person on that network, we just compared notes on how many people we knew in common
<stub> Oh... I think I know who. Still vaguely in contact with her.
<nigelb> :)
<jtv> Hi henninge!
<henninge> Hey jtv! ;)
<jtv> henninge: if you're reviewing today then I have a job for you right off the bat.  :)  https://code.launchpad.net/~jtv/launchpad/bug-806946/+merge/67302
<henninge> jtv: wow, how thoughtful of you ... ;-)
<jtv> Well, you know me.  :)
<henninge> jtv: I have just returned to my desk for the first time since Dublin, so I am not all ready yet but I will look at your branch as soon as I am.
<jtv> henninge: oh, I forget that you've been away!
<jtv> So never mind, I'll bother someone else.
<jtv> Such as stub.
 * stub is looking
<adeuring> good morning
<jtv> hi adeuring!
<adeuring> hi jtv!
<jtv> hi mrevell
<mrevell> Hello friends
<jtv> Anyone else getting mailman build failures?
<bigjools> morning all
<wgrant> Morning bigjools.
<bigjools> halp.  email.  halp.
<wgrant> :(
<jtv> morning bigjools â drowning in email?
<jtv> thanks stub
<bigjools> jtv: not waving, so I must be
<jtv> Look at the bright side.  Your mortgage will be 3 inches longer and you'll have a nice acai-berry rolex.
<jtv> And no, unity 2d, "mumble" does not mean "remember that file utils.c?  I'd like to open it in gedit."
* henninge changed the topic of #launchpad-dev to: devel in RC until r13402 deployable | https://dev.launchpad.net/ | On call reviewer: henninge | Critical bugs: 215 - 0:[######=_]:256
<jtv> thanks henninge â that was very quick!
<gmb> lifeless: Did you get chance to take another look at https://code.launchpad.net/~gmb/launchpad/private-branches-bug-657004/+merge/65639?
<lifeless> gmb: I hadn't, no.
<gmb> lifeless: I think I've dealt with your concerns; I'll get a review from our fine, upstanding OCR and we'll bounce it back to you if we've any further worries.
<gmb> How's that sound?
<wgrant> Fail.
<wgrant> qa-bad ftw
<StevenK> wgrant: Which one?
<wgrant> wallyworld's JS build qa-bad fix.
<wgrant> It deliberately omits the YUI accordion widget that hideously haunts lib/lp/contrib
<wgrant> But it is needed.
<lifeless> gmb: I've done an actual review for you
<gmb> lifeless: Ah, thankee kindly.
<wgrant> rvba: Hi
<rvba> wgrant: Hi!
<wgrant> rvba: Your longpoll JS produces two "yui: NOT loaded: lang" warnings on load. Is that OK?
<rvba> wgrant: mm ... I don't think it's a problem but I'll look into it.
<wgrant> rvba: This is blocking a rollback to fix QA for the deployment, so it is critical-critical.
<wgrant> It seems OK, though.
<rvba> wgrant: yeah, I really think it's OK.
<wgrant> I guess lang is probably core?
<rvba> I'm not sure ...
<wgrant> http://developer.yahoo.com/yui/3/yui/ suggests it is.
<wgrant> "
<wgrant> All of this functionality is available in the YUI Core:
<wgrant> [...] lang
<wgrant> "
<rvba> Indeed.
<wgrant> Hm.
<wgrant> But they are listed as modules...
<rvba> IIRC It was a separate module in YUI 2
<rvba> Perhaps that's the reason why it's not clear ...
<rvba> wgrant: How is this blocking you?
<wgrant> rvba: The fix to fix the broken longpoll stuff is itself broken, and I don't know if my fix for that fix is good.
<wgrant> Because now it's throwing warnings, but otherwise working AFAICT.
<wgrant> (bug #808561_
<_mup_> Bug #808561: longpoll.js is not included in launchpad.js during make <bad-commit-13401> <qa-bad> <Launchpad itself:Fix Committed by wallyworld> < https://launchpad.net/bugs/808561 >
<wgrant> Ah, 13WAAFAXG!
<wallyworld> wgrant: saw the qa :-( want me to fix it? i excluded the contrib because i could have sworn the accordian was not in the original lp,js when i check the stdout :-(
<wgrant> wallyworld: I have a fix here, but now that longpoll is loading it's throwing warnings about lang not being loaded.
<wgrant> I suspect because it's core.
<rvba> wgrant: I think you're right.
<wgrant> (the only other file differences are two from */testing/*, the omission of which is harmless)
<wallyworld> yes, and we don't want those testing files
<wgrant> I'll drop lang from the requires.
<rvba> wgrant: Yes.
<wallyworld> wgrant: from requires in longpoll.js ?
<rvba> It will blow up badly if the lang module is not loaded.
<wgrant> wallyworld: Yes.
<wgrant> http://paste.ubuntu.com/641774/?
<wgrant> Just testing now...
<rvba> Testing it now too ...
<rvba> Does not seem to break the tests ...
<rvba> And the warning (I had that turned off :() is gone.
<wgrant> And the warnings are gone.
* wgrant changed the topic of #launchpad-dev to: devel in RC until r13403 deployable | https://dev.launchpad.net/ | On call reviewer: henninge | Critical bugs: 215 - 0:[######=_]:256
<wgrant> Oops.
<wgrant> Rolled back with the bug instead of rev.
<jml> *blink*
* benji changed the topic of #launchpad-dev to: devel in RC until r13403 deployable | https://dev.launchpad.net/ | On call reviewer: henninge, benji | Critical bugs: 215 - 0:[######=_]:256
<henninge> adeuring: matt asked for some lines about the web service bug we fixed in Dublin.
<henninge> adeuring: I saw that you landed it. Thank you very much for that! ;-)
<adeuring> henninge: well, it had to be reverted: caused a regression
<henninge> adeuring: oh :(
<henninge> adeuring: are we fixing that?
<henninge> is there  a  bug number?
<adeuring> henninge: not yet... for the regression, for example ooops 2013AP64
<henninge> adeuring: "not yet" about fixing it or about a bug number?
<adeuring> henninge: I haven't done any work yet
<henninge> adeuring: the traceback is not very helpful. Any idea what's causing it?
<adeuring> henninge: no... wgrant blamed our branch for the oops...
<deryck> Morning, all.
<abentley> deryck: morning.
<jtv> Did we break the makefile?
<jtv> When I "make -j2" in a fresh branch, I get
<jtv> make: *** No rule to make target `lib/canonical/launchpad/icing/yui/yui/yui.js', needed by `lib/canonical/launchpad/icing/build/launchpad.js'.  Stop.
<deryck> jtv, update download-cache maybe?
<jtv> I did a rocketfuel-get this morning.
<jtv> Repeating the "make" command gets me past the problem.
<jtv> So strikes me as a missing dependency in the makefile.
<abentley> deryck: I'm having sound difficulties.
<cr3> hi folks, I think I understand why bug cannot be assigned to a distroarchseries but I wouldn't mind hearing an explanation :)
<cr3> s/bug/bugtask/
<abentley> bac: I'd like to touch base about the json cache at some point.
<bac> abentley: sounds good
<abentley> I guess this is your first day back from vacation.  Any time good for you?
<bac> abentley: 2:00 US/East today?
<abentley> bac: Sounds good.
* henninge changed the topic of #launchpad-dev to: devel in RC until r13403 deployable | https://dev.launchpad.net/ | On call reviewer: benji | Critical bugs: 215 - 0:[######=_]:256
<bigjools> anyone got any ideas what's up here? http://pastebin.ubuntu.com/641976/
<nigelb> What tz is wallyworld?
<bigjools> +10
<nigelb> ok, this is going to be tricky.
<nigelb> I'll have to wake up early to catch him.
<cr3> is there something like a source_package in Launchpad but that also refers to the distro arch series, rather than just the distro series like source_package?
<cjwatson> Could somebody allocate a database patch number for me for lp:~cjwatson/launchpad/multiarch-translations?  A suitable description might be "DistroSeries.split_long_descriptions"
 * cjwatson is reading https://dev.launchpad.net/PolicyAndProcess/DatabaseSchemaChangesProcess with some care
<bigjools> anyone know if there's a way to determine which version of the API is being used in the method being called?
<cjwatson> wgrant: I can't remember if I asked; for backports_not_automatic, how did you go about getting it set to True for oneiric?  I don't see it in the UI or API
<wgrant> cjwatson: It was SQL, sadly.
<wgrant> cjwatson: You could probably expose this through the API, I guess.
<cjwatson> wgrant: it'll only have to be done once, so if people don't mind SQL then that's a smaller code change ...
<cjwatson> (I guess that's a question)
<wgrant> They do mind SQL.
<wgrant> And API exposure is easy.
<wgrant> So...
<cjwatson> but not for you, clearly :-)
<cjwatson> but OK
<wgrant> But permissions are a bit awkward.
<wgrant> You now have a patch number of 2208-79-0
<cjwatson> great, thanks
<cjwatson> if you have pointers to stuff I can read for permissions, that'd be good
<wgrant> cjwatson: https://dev.launchpad.net/LaunchpadSecurityPolicy is something, but it's not really what you want.
<wgrant> cjwatson: lib/canonical/launchpad/security.py defines who has launchpad.Edit/launchpad.View etc. for each interface.
<wgrant> And lib/lp/registry/configure.zcml maps from permissions to read/write of attributes.
<wgrant> Or to read/write of interfaces, in which case all of the interface's fields are permitted.
 * wgrant wanders off.
<flacoste> i'm looking for a reviewer for https://code.launchpad.net/~flacoste/launchpad/bug-801233/+merge/67353
<flacoste> soyuz-related
<flacoste> but probably not critical to have soyuz-fu
<flacoste> as it's mainly an api-export one
<flacoste> bac you might even be interested in it :-)
<flacoste> as it builds on your previous processor export
<bac> flacoste: i'll be happy to look at it in a little while.
<flacoste> bac, ok thanks
<bac> abentley: mumble? skype?
<abentley> bac: let's mumble
<bac> abentley: trying...
<abentley> bac: you don't hear me?
<bac> nope
<abentley> bac: the lips are turning red, so I think my side's okay.
<abentley> bac: yes, I can hear you okay.
<abentley> bac: Do you prefer Skype?
<bac> abentley: well, it works for me
<abentley> bac: Let's do that, then.
<bac> abentley: i am brad.crittenden
<abentley> bac: bzr+ssh://bazaar.launchpad.net/~abentley/launchpad/json-serialization
<abentley> bac: bzr+ssh://bazaar.launchpad.net/~abentley/launchpad/getnewcache
<gmb> wgrant: Do you remember how to make the librarian stop serving .tar.gzs as text/html?
<gmb> losa ping
<mbarnett> heya gmb
<gmb> mbarnett: Howdy! For some reason the Librarian is serving http://launchpadlibrarian.net/73387253/testtools-0.9.11.tar.gz as text/html; I know that wgrant mentioned that this was a known bug, but I don't remember if he told me what we need to do stop it. Do you have any ideas?
<gmb> (Incidentally, I know it's not normally text/html because it worked fine yesterday)
<mbarnett> gmb: not off the top of my head.  Unless getting cached changes how it gets served.  I will have to do some investigation to see what exactly is going on.
<gmb> mbarnett: Thanks.
<abentley> gmb: I've seen that bug.  Let me see if I can dig it up.
<gmb> abentley: Thanks.
<abentley> gmb: Is this your bug? https://bugs.launchpad.net/launchpad/+bug/703807
<_mup_> Bug #703807: launchpad sometimes serves download files as content-type text/html <regression> <Launchpad itself:Triaged> <pyOpenSSL:New> < https://launchpad.net/bugs/703807 >
 * gmb wonders why that didn't come up in his original searches.
<gmb> abentley: Thank you. Many eyes make all bugs easy to find...
<abentley> gmb: hehe.
<cr3> what's the convention in Launchpad when naming attributes? foo_bar or foobar?
<gmb> mbarnett: According to that bug ^^ it is a caching problem of some sort: https://bugs.launchpad.net/pyopenssl/+bug/703807/comments/21
<_mup_> Bug #703807: launchpad sometimes serves download files as content-type text/html <regression> <Launchpad itself:Triaged> <pyOpenSSL:New> < https://launchpad.net/bugs/703807 >
<abentley> cr3: foo_bar
<gmb> mbarnett: So I shall now endeavour to remember how the hell to use Curl and report back...
<cr3> abentley: thanks!
<abentley> np
<cr3> abentley: productseries = Choice(... what's the deal with that and lots of other attributes under IBugTask? old convention?
<mbarnett> gmb: heh
<abentley> cr3: productseries is a bit of a compound word.  And yes, I don't think we've been very strict about it.
<gmb> mbarnett, abentley: Cache-control: no-cache fixed it. Weird. I'll add a note to the bug. Also, if anyone sees wgrant around, tell him it happened. If we get him wound up enough he'll either complain until it gets fixed or fix it himself whilst I sleep.
<mbarnett> gmb: i'll make a note in my daily report as well, see if the new blood has any ideas off hand.
<gmb> Cool.
 * gmb -> afk for the evening. Nytol.
<bac> flacoste: the docstring for getBuildQueueSizes is confounding the wadl generator
<flacoste> bac: really?
<bac> flacoste: really!
 * flacoste checks
<flacoste> bac: how did you get this? make clean && make still works here?
<bac> flacoste: make is broken wrt to api generation
<flacoste> not for me :-(
<bac> flacoste: i mean, it doesn't do anything
<bac> flacoste: rm -rf lib/canonical/launchpad/apidoc ; make build
<flacoste> ah right
<flacoste> ah ok, so how did you get the error?
<flacoste> running lp-create-wadl-and-apidoc also works here
<bac> by doing the above, and looking at the output.  it won't fall-over fail but does print errors
<flacoste> bac: i only get the usual 'Unknown entry url'
<flacoste> bac: can you pastebin what you get?
<bac> yep
<bac> flacoste: http://pastebin.ubuntu.com/642108/
<flacoste> bac: thx, i guess i can fix that easily
<benji> flacoste: when you get a chance I'd appreciate a look at my 781600 branch: lp:~benji/launchpad/bug-781600
<flacoste> benji: ack
<bac> flacoste: r=bac
<lifeless> moin moin
* lifeless changed the topic of #launchpad-dev to: devel in RC until r13403 deployable | Performance Tuesday | https://dev.launchpad.net/ | On call reviewer: benji | Critical bugs: 215 - 0:[######=_]:256
<lifeless> flacoste: hi
<flacoste> hi lifeless
<flacoste> bac: thanks!
<bac> flacoste: np.  did you figure out the docstring weirdness?
<flacoste> bac: not yet
<gary_poster> wgrant, ping?
<sinzui> jcsackett, do you have time to mumble?
<gary_poster> lifeless, hi.  Do you happen to know if the code bigjools, wgrant & co did at the thunderdome has a generic LP Rabbit consumer as part of it?
<gary_poster> I'm not sure where branch would be
<jcsackett> sinzui: 15 min? Don't want to lose the thread of what I'm looking at.
<sinzui> jcsackett, okay
<lifeless> gary_poster: it doesn't, I'm pretty sure
<lifeless> gary_poster: sorry, OTP with flacoste
<gary_poster> lifeless, ok np thanks
<jcsackett> sinzui: wrapped up a bit faster than i thought. i can mumble now.
<sinzui> jcsackett, I will restart
<jcsackett> sinzui: i think it may have been me, actually.
<sinzui> jcsackett, maybe, but now thunderbird wants 50% if my cpu
<wgrant> gary_poster: Hi
<gary_poster> flacoste, danilos evaluated escalated bug 775691 and in the last comment said that he thought we ought to lower the priority, and described what needed to happen (migration, essentially, AIUI).  Should I contact David Planella and see if this addresses his concern, or should I regard the migration that danilos described as the escalated task?
<_mup_> Bug #775691: Empty translations on one side do not get translated by the other side <escalated> <not-pie-critical> <upstream-translations-sharing> <Launchpad itself:Triaged by yellow> < https://launchpad.net/bugs/775691 >
<wgrant> gary_poster: There is a rabbit client in lp.services.messaging.
<wgrant> gary_poster: Not sure how suitable it is for you.
<gary_poster> hi wgrant.  thanks for ponging.  I was wondering how your long poll work consumed rabbit events on the Launchpad side.  I saw that client and it was only a publishing queue afaict
<wgrant> gary_poster: Ah, we don't consume in LP yet.
<gary_poster> oh ok
<wgrant> gary_poster: That's done by lazr.amqp.
<gary_poster> oh?
<wgrant> It's an external twisted daemon.
<gary_poster> ah!
<gary_poster> does it have access to the LP db?
<wgrant> Hmm, but the tests should do consuming.
<wgrant> Let me see.
<wgrant> No!
<wgrant> 'tis a microservice.
<gary_poster> yeah, that's what I thought
<wgrant> gary_poster: RabbitQueue.receive
<wgrant> In lp.services.messaging
<wgrant> Tests use it.
<wgrant> It is not really production-ready.
<gary_poster> but I'm not sure how you are able to do what you need without access to the db
<wgrant> stub and bigjools couldn't work out how to do timeouts.
<wgrant> So they hacked that together.
<gary_poster> :-)
<gary_poster> k lemme look
<wgrant> Improvements welcome!
<wgrant> Yay, qastaging works now.
<wgrant> And staging is up too.
<wgrant> Good news.
<gary_poster> wgrant, cool.  ...lazr.amqp does not need db access because the only thing it needs is to convey rabbitmq messages to long poll (browser) clients?
<wgrant> gary_poster: Right. It basically bridges Apache and rabbitmq.
<gary_poster> gotcha.  ok thanks much wgrant!
<wgrant> The browser gives it a queue name to listen for, and it returns any messages that come through it.
<gary_poster> gotcha
<wgrant> lifeless: Looks like we are green, but I will poke around on qastaging a bit more, given recent happenings.
<lifeless> cool!
 * gary_poster goes to prepare cooking stuff
<gary_poster> night
* benji changed the topic of #launchpad-dev to: devel in RC until r13403 deployable | Performance Tuesday | https://dev.launchpad.net/ | On call reviewer: - | Critical bugs: 215 - 0:[######=_]:256
<wgrant> Looks OK to me.
<wgrant> lifeless: Any objections to reopening?
<lifeless> we're qa-ok ?
<lifeless> mm
<lifeless> wgrant: lets get the prelude revisions NDT deployed
<wgrant> lifeless: We're as far as we can.
<lifeless> wgrant: and wait a little for regressions from them
<wgrant> 13389 is bad.
<wgrant> 13401 fixed it.
<wgrant> But was bad.
<wgrant> 13402 is the merge.
<wgrant> 13403 fixes the bad bad fix.
<lifeless> how did 13389 be untestable ?
<wgrant> It was landed without a bug.
<lifeless> ok
<lifeless> just looking
<lifeless> trying to assess whether we should expect more cascade qa-fail
<wgrant> 13391, 13392 are of some concern to me.
<lifeless> I can see that
<lifeless> nothing could possibly go wrong with them
<lifeless> lets spend some time this morning kicking tires on those on qastaging
<wgrant> Yeah
<lifeless> has the hwdb api changes been played with ?
<wgrant> bac: Hm?
<wgrant> lifeless: No.
<wgrant> cr3: Around?
<lifeless> cr3: ping
<lifeless> hah, over to you :)
<bac> wgrant: re: the ppa question?  i thought you may be able to answer the user.
<wgrant> Probably. But I'm not maintenance any more.
<wgrant> But let me see.
<bac> wgrant: thx
<cr3> lifeless, wgrant: yep, pong, what's up?
<wgrant> cr3: Have you tested your change out on qastaging at all?
<wgrant> bac: The packages are there AFAICT :/
<cr3> wgrant: crap, totally scripped my mind. I'll do it this evening, then I set it to qa-ok, right?
<wgrant> cr3: That would be great, thanks.
<cr3> skipped my mind even :)
<cr3> wgrant: thanks for the reminder!
<wgrant> lifeless: What's changed in the new batchnav? Just addition of the memo= arg, with no functional changes for our collectioins yet?
<lifeless> wgrant: yeah
<lifeless> wgrant: infrastructure and a default adapter that makes things work much as they did before
<lifeless> wgrant: urls change slightly
<lifeless> wgrant: (consistent ordering of parameters)
<lifeless> wgrant: its possible, if someone was manually predicting the urls to use, that things will break
<lifeless> but AFAICT nothing did that
<wgrant> Well, batchnav does.
<wgrant> But it seems to work, AFAICT.
<wgrant> s/batchnav/lazr-js/
<lifeless> it does?
<lifeless> I thought lazr-js read the next_link ?
<wgrant> lazr-js pickers provide page number links.
<lifeless> yeah
<lifeless> the api for that python side generates those batches
<lifeless> and queries for their urls
<wgrant> Oh, really?
<wgrant> Anyway, we own lazr-js now, so we should stop it from doing that.
<lifeless> the new batchnav supports start-nonce=XXXX + offset=YYYY
<lifeless> so we can do that efficiently without creating the batches
<wgrant> Ahh
<lifeless> but its not all wired up to do that
<wgrant> Still, the numbers are fairly pointless.
<lifeless> yes
<lifeless> hopefully I'll get a collection converted today
<lifeless> sinzui: hi
<lifeless> sinzui: just wanted to confirm - is your team picking up the privacy-marker project (changing from ////// borders to the red thing at the top etc) ?
<nhandler> 100
<sinzui> lifeless, yes
<sinzui> wgrant, mumble>
<sinzui> StevenK, mumble?
<wgrant> sinzui: Sec
<StevenK> wgrant: It seems we are deployable?
<wgrant> StevenK: Yes, but there are a couple of big changes that lifeless and I would like to do more verification of.
* wgrant changed the topic of #launchpad-dev to: devel in RC until r13403 deployable | Performance Tuesday | https://dev.launchpad.net/ | On call reviewer: - | Critical bugs: 218 and growing all too rapidly - 0:[######=_]:256
<cjwatson> wgrant: https://code.launchpad.net/~cjwatson/launchpad/multiarch-translations/+merge/67640 is ready for review; would you be up for code-reviewing it?
<StevenK> Heh, it's even wgrant's turn to be OCR.
#launchpad-dev 2011-07-12
<wgrant> cjwatson: How are we going to produce the C translations files?
<wgrant> (also, your LongDescription interpretation is inverted, AFAICS)
<cjwatson> C as opposed to en?
<wgrant> Well, do we produce the ones it's going to look for by default?
<cjwatson> inverted> oh bugger, you're right, silly inscrutable docs
<cjwatson> apt-ftparchive defaults to writing to Translation-en, and mvo told me he'd just stop uploading -en in the i18n custom uploads.  But perhaps it ought to be C, or perhaps the client ought to be a bit smarter here ...
<wgrant> Note that it is currently not possible to create these files with apt-ftparchive.
<wgrant> From the manpage.
<cjwatson> Not in my version
<wgrant> Ahem. I was looking at carob accidentally.
<wgrant> Indeed, natty's is a bit more encouraging.
<cjwatson> hmm.  I wonder if this is going to need YA lucid backport though ...
<wgrant> That stuff should be somewhat self-contained, if we want to convince mvo to backport the patch...
<cjwatson> I'll discuss the question of using Translation-C with mvo.  At worst it would just be a matter of setting TreeDefault::Translation / Tree::blah::Translation
<cjwatson> backporting the patch will happen
<cjwatson> both mvo's manager and his tech lead want this :-)
<wgrant> Heh
<wgrant> Useful.
<cjwatson> I'll fix the inversion.  Should I rename the database column as well?  I kind of like having the new column be False by default, though
<cjwatson> so split_long_descriptions True => LongDescription "false"
<wgrant> I'm not sure. Possibly include_long_descriptions defaulting to True would work...
<wgrant> Hmmm
<cjwatson> oh, there's default= in Bool
<cjwatson> I'm OK with that if it seems less confusing
<cjwatson> anything else from a first pass that I should sort out?
<wgrant> cjwatson: Looks reasonable.
<cjwatson> Cool, thanks.  I'll work on that tomorrow then
<wgrant> Great!
<wgrant> wallyworld_: Is your last comment in bug #803996 a regresion on qastaging?
<_mup_> Bug #803996: person picker widget doesn't show link to unassign myself <disclosure> <exploratory-testing> <person-picker> <Launchpad itself:Triaged> < https://launchpad.net/bugs/803996 >
<wallyworld_> wgrant: not sure if it's a regression
<wallyworld_> but it pertains to that bug so i added it as a comment to ensure the full scope of the issue was fixed
<wgrant> StevenK: How goes the vocab?
<StevenK> wgrant: *RARGH*
<wgrant> As expected!
<lifeless> wallyworld_: should it hold up deploying?
<wallyworld_> lifeless: not imho. it's a minor usability issue (page refresh fixes it) and i'm not even sure it's a regression
<wallyworld_> i just noticed it doing some unrelated qa
<lifeless> wallyworld_: then you might like to file a new bug for it, to fit the qa process
<wallyworld_> lifeless: there's a bug already. i just augmented the bug details to provide extra info
<lifeless> wallyworld_: but its the bug that will be closed when we deploy on thursday
<lifeless> wallyworld_: if you land another incremental branch for it between now and then, we'll be unable to qa it, and the downtime deploy will get more hairy
<wallyworld_> lifeless: ? the bug is associated with a kanban card that is in the todo list
<lifeless> and... we're qa-bad for it now
<lifeless> rev 13392 is broken
<wgrant> I thought it would be.
<wgrant> But where?
<lifeless> compare https://launchpad.net/~launchpad-dev/+members and https://qastaging.launchpad.net/~launchpad-dev/+members and then click next in both on the 'active member' batch
<wallyworld_> lifeless: bug 803996 is only triaged. that's the one i added the extra info for based on by observations when playing with qas
<_mup_> Bug #803996: person picker widget doesn't show link to unassign myself <disclosure> <exploratory-testing> <person-picker> <Launchpad itself:Triaged> < https://launchpad.net/bugs/803996 >
<wgrant> lifeless: Ah, indeed.
<lifeless> wallyworld_: ah, cool
<lifeless> wallyworld_: sorry for my confusion
<wallyworld_> lifeless: np. it was well worh the question
<wallyworld_> worth
<lifeless> wgrant: the memo is not being adapted, so the memo for nav A is affecting the content for nav B
<wgrant> Yup.
<lifeless> frell
<wgrant> Revert or fix?
<lifeless> revert
<wgrant> I think revert.
<wgrant> Yeah.
<lifeless> unknown number of instances
<lifeless> wgrant: how would you feel if I asked you to do the honours ?
<lifeless> I've been chasing emails and stuff so far today - I only just got to doing qa on this
<wgrant> Like I've done that a bit too much in the last week, but okay.
 * wgrant does it.
<lifeless> I will dig up the MP and explain the situation on it
<wgrant> I have 6 reversion branches in ~/launchpad/lp-branches :(
<wgrant> :( conflicts
<lifeless> in a .txt file
<wgrant> Yeah
<wgrant> Which is merged.
<lifeless> its shallow - just keep the trunk version
<lifeless> the change was from explicit url to ...
<lifeless> but oh the pain - if it makes your eyes bleed, I will do it
<lifeless> hmm
<lifeless> the alternative is to not back this out
<lifeless> lets talk through it for a second:
<lifeless>  - for any given navigator, it will work to navigate it as long as you don't switch navigators half way through things
<lifeless>  - multi-navigator pages are the exception
<lifeless>  - and switching should be rarer still
<wgrant> But if we revert now, we can deploy it on Friday and be able to easily revert if something goes wrong.
<lifeless> so, what about filing a critical and rolling forward
<wgrant> This change is scary and hard to verify completely.
<lifeless> there may be other unintended side effects
<lifeless> or this one may go deeper than thought
<wgrant> We lose little by reverting it now.
<wgrant> We gain flexibility.
<wgrant> And, given the mess that the last 1.5 weeks have been, I think that is a good thing.
<lifeless> I'm not worried about it landing later
<lifeless> I was thinking about the headache of the reversion branch
<lifeless> thats all
<lifeless> wgrant: how fugly does the reversion look ?
<wgrant> Not very. Just running the tests now.
<lifeless> so bug batches look fine
<lifeless> though 'Last' still times out (because the order reversing stuff is not yet hooked up to any lp batches)
<wgrant> lifeless: Shall I land the revert?
<lifeless> wgrant: it was fairly clean? if so yes.
<Ursinha> !2
<wgrant> I just --take-other on the story, and it passes fine.
 * wgrant lands.
<wgrant> Bah, forgot r-c
* lifeless changed the topic of #launchpad-dev to: devel in RC until r13403 deployable | Performance Tuesday | https://dev.launchpad.net/ | On call reviewer: - | Critical bugs: 220 - 0:[######=_]:256
<nigelb> 220?
<nigelb> :(
<lifeless> jtv: this may be fallout from red's current work - https://bugs.launchpad.net/launchpad/+bug/808950
<_mup_> Bug #808950: DistroSeries:EntryResource:searchTasks AssertionError: wrong You exported name as an IChoice based on an SQLObjectVocabularyBase, you should use lazr.restful.fields.ReferenceChoice instead <api> <oops> <Launchpad itself:Triaged> < https://launchpad.net/bugs/808950 >
<lifeless> jtv: I don't know if it is|isn't
<lifeless> mpt: hi
<lifeless> mpt: how would you feel if your transcribed talk was edited - treated as a live doc? E.g. where you give the example of having the word 'should', I would like to add 'add', 'remove', 'change', and perhaps more as other 'words to be wary of'
* wgrant changed the topic of #launchpad-dev to: devel in RC until r13405 deployable | Performance Tuesday | https://dev.launchpad.net/ | On call reviewer: wgrant | Critical bugs: 220 - 0:[######=_]:256
<wgrant> wallyworld_: Around?
<wgrant> lifeless: Can I add X-Lazr-OopsId headers to webapp OOPSes too?
<wgrant> (to display less useless errors when pickers fail)
<lifeless> wgrant: can you rename it?
<wgrant> The other one is set in lazr.restful, I believe.
<lifeless> wgrant: X-LP-OOPSID everywhere would make me happy.
<wgrant> Sadly I guess clients may depend on it.
<wgrant> And it is not overridable without lazr.restful changes.
<lifeless> so there are I guess 3 q's
<lifeless> can we add a header: absolutely
<lifeless> should it be the same as the API one? I dunno
<lifeless> does the name matter? not really, but X-LP-OOPSID is or even X-OOPSID would be a lot better
<lifeless> wgrant: you could add whatever name and rename it later
<lifeless> wgrant: I don't mean to inflae the work you have to do
<wgrant> Grar
<wgrant> I hate Soyuz.
<wgrant> Stop deleting evidence.
<lifeless> no sequitur ?
<wgrant> Yes
<wgrant> Soyuz will let you delete a publication multiple times.
<wgrant> Each attempt will overwrite the last date.
<wgrant> So I can't see what this user actually did to break their archive.
<lifeless> https://dev.launchpad.net/LEP/FastDowntime for those liking 'agile'
<wgrant> yay
<poolie_> indeed
<wgrant> poolie_: I'm pretty sure both frontends are broken mimetype-wise.
<wgrant> poolie_: But not all of them have the bad data cached all the time.
<wgrant> I've seen text/html from both.
<lifeless> stub: hiya
<stub> Hi
<wgrant> poolie_: Indeed, there are logs in the bug to show it happening from both.
<lifeless> stub: I'm just filing bugs for fastdowntime at the moment
<stub> Cool.
<lifeless> stub: Thanks for landing batchnav! however we've had to roll it back - found a corner case
<lifeless> when we have two navs on one page the new variables need their name mangled
<stub> I think the pgbouncer config on prod is setup, but waiting on pgbouncer installation on sourcherry to proceed. Silly losas think testing is a good thing or something!
<spm> mad buggers
<lifeless> spm: can we get it on sourcherry then ?
<stub> We have 2 batchnavs on one page?
<lifeless> stub: yeah - e.g. ~launchpad-dev/+members
<stub> last I heard the rt was with stephan
<lifeless> so its an easy fix, I commented in the MP
<poolie_> wgrant, oh i see
<spm> stub: installed
<poolie_> that is easier to explain than having persistently different behaviour between what should be identical machines
<stub> spm: And permissions on /etc/pgbouncer contents set so I can configure it?
<spm> i assume at this stage we're not pulling pgbouncer into the lp-db-deps?
<wgrant> poolie_: Yes. :/
<wgrant> Perhaps we should go poking in squid caches or logs or something.
<stub> spm: No. Maybe one day if the rollout script is tested with out test suite in the -dev packages.
<spm> oki
<lifeless> bbiab
<spm> normally we object violently to 'install package' unless it's in the meta, but I'll let this ride for once.
<spm> where violently == 4x2.
<poolie_> wgrant, in that case it actually is a dupe
<spm> where violently == 4x2/ip
<wgrant> poolie_: I've already fixed that up :)
<poolie_> thanks
<stub> spm: I'm not fussed, but we will only need it installed on at most two machines so similar to haproxy.
<poolie_> it occurred to me it would be interesting to have a losa look directly at the backend and see what it answers
<stub> (+staging)
<poolie_> presumably always the correct thing
<wgrant> poolie_: We've never seen the backend return bad data.
<wgrant> poolie_: But it possibly only does it in response to certain request headers.
<spm> stub: have at it
<poolie_> how would you know if the backend returned bad data?
<wgrant> poolie_: With manual testing, this is.
<poolie_> iow what i said has already been tried?
<spm> stub: I'm not sure of your  meaning wrt machines similar to haproxy, wrt staging?
<wgrant> poolie_: Yes.
<stub> spm: At most, we want pgbouncer running on two production machines + staging, similar to our HAProxy dependency. So putting it in the -db meta package didn't occur to me.
<stub> Probably more a puppet thing since it one package + several customized configs
<spm> oh I see. right.
<lifeless> whats the name of the codebrowse url mapper thingy?
<stub> lifeless: Are you expecting me to sort the batch navigator branch or is it punted to a bugs team?
<wgrant> lifeless: branch-rewrite
<lifeless> stub: Its in limbo
<lifeless> stub: a bugs team working on batch related timeouts would have motivation to do it
<lifeless> stub: if you want to do it, be my guest; not expecting you to do so (or not to do so)
<stub> lifeless: It is a dependency of a bug abel is assignedl
<stub> I'll checkout the required work after breakfast.
<lifeless> stub: I think abel should do it then, it should be pretty easy.
<lifeless> but sure, EIDONTCARE :)
<stub> I'm going to pull out our reliance of transaction-persistent cursors from garbo.py so we can run pgbouncer in transaction mode rather than session mode.
<stub> Which is a bit sucky, as I thought that code was rather cool ;)
<stub> Think I'll need to replace it with two connections to remain efficient.
<lifeless> stub: it was cool, but I totally endorse that change
<lifeless> stub: could we do a brief voice call ?
<lifeless> stub: I want to check I didn't miss anything for round one of fast downtime
<stub> Sure. I'll just plug stuff in.
<lifeless> https://dev.launchpad.net/LEP/FastDowntime
<lifeless> wgrant: has lazr.amqp had its non-longpoll expunged? ready for rename ?
<wgrant> lifeless: It hasn't.
<wgrant> I might look at that this afternoon.
<lifeless> bug 809132
<_mup_> Bug #809132: lazr.amqp is overly broad <longpoll> <lazr.amqp:New> < https://launchpad.net/bugs/809132 >
<wallyworld_> wgrant: hi. i had to go out and sort out some car issues :-( back now
<wgrant> wallyworld_: Was wondering how to mock out HTTP requests in the picker for unit testing.
<wallyworld_> wgrant: we have about 3 different Y.io mocks
<wallyworld_> the bugs js uses one style
<wgrant> I saw MockIo, but it's only used in one place.
<wgrant> Seems to be from lazr-js.
<wallyworld_> there's also mickio.js brought across from lazr-js
<wallyworld_> in other places, a js object is constructed with just the method (eg patch) that is to be patched implemented
<wallyworld_> in most cases, the mock io object is passed to the js business logic
<wgrant> pickerpatcher uses Y.io directly.
<wgrant> I want to test the handling of errors.
<wallyworld_> yes, see ^^^
<wgrant> Ah
<wallyworld_> the most common approach seems to be to pass an optional io object to the business lgoic
<wallyworld_> if io===undefined, use Y.io
<wallyworld_> else use the mock
<wallyworld_> there's implementations in out code base that allow the expected api call to be chacked and a return result to be provided
<wallyworld_> if you search for IOStub you'll see some examples
<wallyworld_> but I'd rather get a good, reusable implentation into our testing framework
<wallyworld_> and have everything use that
<lifeless> wgrant: where is longpoll deployment at?
<wgrant> lifeless: Yes.
<lifeless> would a bug be helpful ?
 * lifeless makes one
<wgrant> Possibly.
<poolie> wgrant, huh, well spotted with the 304
<poolie> that's pretty perverse
<poolie> does it give you a body, or just the header?
<wgrant> poolie: lifeless did the real diagnosis :)
<wgrant> It just gives the header.
<lifeless> ok, ciao
<wgrant> Well, that lazr.amqp pruning was easy.
<sidnei> poolie, hi!
<poolie> hi sidnei
<sidnei> poolie, i had a question about branch splitting earlier, not sure you saw it
<poolie> hi
<poolie> there is 'bzr split'
<poolie> wgrant, couldn't it just not give a content-type header?
<wgrant> poolie: Possibly very true.
<sidnei> poolie, so after splitting a directory from the tree, does it get removed from the original tree or does that need to be done manually?
<wgrant> "Any HTTP/1.1 message containing an entity-body SHOULD include a Content-Type header field defining the media type of that body."
<wgrant> poolie: So, looks like we can omit it.
<wgrant> Particularly since we have no entity-body.
<wgrant> grarr
<wgrant> Just ran into this bug when trying to build lazr.amqp :(
<wgrant> Blah.
<poolie> right
<poolie> i thought it was normally omitted
<wgrant> twisted.web.server.Request.process sets it to text/html at the start of the request :/
<wgrant> So we'd need to delete it...
<poolie> sidnei, you will then have one tree with the subdirectory excised, and underneath it another checkout with that tree present
<poolie> hm that's a bit presumtuous
<poolie> i guess it depends if you want to fix this in twisted or to work around it
<wgrant> Would be nice to fix it in Twisted, but not sure how to do that in a backwards-compatible manner.
<wgrant> Or we could just declare the existing behaviour broken.
<wgrant> Perhaps.
<poolie> 304 could perhaps just squash the ct
<wgrant> Indeed.
<poolie> it seems unlikely anyone would really want that behaviour
<wgrant> Probably a good idea.
<wgrant> A self.responseHeaders.setRawHeaders("content-type", []) after the self.setResponseCode(NOT_MODIFIED) seems to work sensibly.
<wgrant> Is a little ew, though.
<sidnei> twisted was designed with POMA in mind (as opposed to POLA - http://en.wikipedia.org/wiki/Principle_of_least_astonishment)
<wgrant> Heh
<poolie> sidnei, anyhow i suggest you at least try split
<poolie> what are you going to want to do with these branches in future?
<poolie> wgrant,  it should possibly be more general than just c-t and 304 but ...
<poolie> this would be the main case i guess
<sidnei> poolie, https://bugs.launchpad.net/graphite/+bug/501828
<_mup_> Bug #501828: Create separate branches for the webapp, carbon, and whisper <Graphite:Triaged by chrismd> <Graphite 1.0:Triaged by chrismd> < https://launchpad.net/bugs/501828 >
<sidnei> poolie, which, btw, is still using pack-0.92. guess it's time for upgrade :)
 * wgrant stabs buildout in the eye.
<wgrant> FASTER
<StevenK> Haha
<wgrant> lifeless: txamqplongpoll, maybe?
<lifeless> wgrant: sure
<lifeless> though txlongpoll is a bit snappier
<wgrant> lifeless: It is, but it's also more generic.
<wgrant> Although possibly reasonable.
<lifeless> txhttpamqp ?
<wgrant> That's even more syllables.
<wgrant> Perhaps we should just do txlongpoll, as you say.
<wgrant> Hmm.
<wgrant> Maybe we should just run txlongpoll from a package.
<wgrant> Since all its runtime deps are in main.
<wgrant> Hm, no, python-txamqp is not in main, but it's in some PPA I have, but I can't think which.
<wgrant> s/have/had/
<wgrant> Must have been u1 or something.
<lifeless> wgrant: no, because we can't sanely upgrade from packages (now, if ever)
<wgrant> Bah.
<spiv> danilos: bmp-inline-diffs has the wrong cursor for the expander link... it looks to me like perhaps the CSS needs to be adjusted so that .expander-node.js-action has the cursor:pointer style, or something like that?
<spiv> danilos: or is there something my branch is missing?
<lifeless> wgrant: we need to address that eventually, but I'd rather not put 'fix python packaging upstream' in our criticalpath
<lifeless> wgrant: btw, has your revert landed ?
<wgrant> lifeless: It passed buildbot 10 minutes ago.
<lifeless> how are we on assessing 13391
<wgrant> An indeterminate point between "handwave" and "kill me now"
<wgrant> Ah.
<wgrant> It says "Remove Person", when it shouldn't be talking about removing people and should be sentence case.
<wgrant> I don't trust it any more.
<wgrant> Tempting to revert.
<lifeless> page?
<wgrant> Any person chooser on a traditional form. eg. the bug supervisor field on https://launchpad.net/launchpad/+configure-bugtracker
<wgrant> Or assignee on +filebug
<lifeless> wallyworld_: ^ is that intentional ?
<wallyworld_> i think so
<wgrant> Sentence case looks and is wrong.
<wgrant> "remove person" is a bit violent.
<wgrant> But it seems to mostly work.
<wallyworld_> it's config - it can be changed
<lifeless> Person is wrong too
<wgrant> s/Sentence/Title/
<lifeless> (teams are not people outside of LP's schema :))
<wallyworld_> it should say Remove Assignee for bug task pickers and whatever the default text is elesewhere (unless overridden)
<wallyworld_> so i guess the default text is wrong plus some overrides are required
<lifeless> oh, there is a regression
<wgrant> I've just been reading the code so far.
<lifeless> on https://qastaging.launchpad.net/launchpad/+configure-bugtracker
<wgrant> What's the regression?
<lifeless> go to the bottom - e.g. security contact
<lifeless> click on 'Choose...'
<wgrant> Hm.
<wgrant> If you reopen it goes away.
<lifeless> then click on the search widget (don't change the text)
<lifeless> now try to remove the assignee
<lifeless> however the regression is live on prod already
<lifeless> so its not a new problem
<lifeless> ah, except
<lifeless> on prod if you reopen the picker
<lifeless> you get the buttons back to remove assignee etc
<lifeless> I have to run - can you two - wallyworld_ , wgrant - decide if this is really a regression or not, and revert or file a bug accordingly? cheers,
 * lifeless goes
<wallyworld_> that issue i am looking at now - the remove/pick me buttons not reappearing after a search
<wallyworld_> it's not a regression
<wgrant> wallyworld_: They also don't appear if you reopen the picker while the textbox is empty, AFAICT.
<wgrant> It is very odd.
<wallyworld_> they won't appear if a search has been done
<wgrant> So, the UI text is pretty bad and needs fixing, but rolling back is expensive...
<wallyworld_> it's not a regression, so rolling back won't help
<wallyworld_> the issue was present in the original work john did afaict
<wgrant> Which isn't a regression?
<wallyworld_> the buttons disappearing
<wallyworld_> after a search
<wgrant> Ah.
<wallyworld_> i have a simple fix for the disappearing buttons issue but writing a test is a bitch
<wallyworld_> without any io stubs :-(
<wallyworld_> the wrong Remove Person text is also not a regression
<wallyworld_> i can'r recall when that branch landed
<wgrant> It's currently "Remove assignee" on production.
<wgrant> Which is correctly capitalised, and only questionably worded, not outright wrong :)
<wallyworld_> Removee assignee everywhere?
<wgrant> Yes.
<wgrant> https://launchpad.net/launchpad/+configure-bugtracker
<wallyworld_> ok. the branch that changed the wording was done to address a bug that said "Remove assignee" was wrong
<wgrant> It's less than ideal.
<wallyworld_> so it's wrong differently
<wgrant> But "Remove Person" is doubly wrong.
<wallyworld_> don't be a case nazi :-). we can live with "incorrect" case, no?
<wgrant> It is terribly ugly.
<wallyworld_> it's just the use of the word Person in cases where it's a team?
<wgrant> That too.
<wallyworld_> beauty is in the eye of the beholder
<wallyworld_> i honestly can't see why this is such a showstopper
<wallyworld_> it's easily fixed i'm sure
<wgrant> Because I don't trust the branch and am looking for excuses to revert it.
<wallyworld_> why don't you trust the branch?
<wallyworld_> besides cosmetics, is there anything *functionally* wrong?
<wgrant> Because I've had to revert it once already, along with 6 other things in the last week, it uses string concatenation to create HTML, the text is wrong, there are underlying bugs with the previous branch that aren't yet resolved, suggesting everything is more fragile than we can imagine...
<wallyworld_> there was one reverted due to a true/True issue but this is a different branch now afaik. the concat and etc and text don't warrent a rollback. they can be fixed
<wgrant> This is a relanding of the true/True issue branch.
<wallyworld_> not exactly - it also has other stuff in there if i am correct
<wgrant> Sure.
<wgrant> But it is a superset of the branch that was rolled back.
<wgrant> + fixes
<wallyworld_> yes, believe so
<wallyworld_> so we have 3 outward facing issues - text, wording and the disappearing buttons issue.
<wallyworld_> disappearing buttons i have a fix for but need to figure out what effort i should expend writing the %@^!$ test without a io stub
<wallyworld_> the text case is ugly but cosmetic
<wallyworld_> the wording is unfortunate and we can land a fix quickly
<wgrant> (three *known* outward facing issues, but OK)
<wallyworld_> i don't think any others have come up so far?
<wgrant> Right, so any other problems are as-yet unknown.
<wallyworld_> there's only a finite number of use caases to check
<wgrant> lol
<wallyworld_> where finite is not that large
<wallyworld_> i mean in terms of how pickers are wired up
<wallyworld_> rather than actual instances on all lp pages
<jtv> lifeless: hi.  That API bug you ran into looks like a Bugs problem, not apparently related to our derivation work.
<wgrant> jtv: You haven't been playing with distroseries vocabs at all?
<jtv> hi wgrant: you're talking about that Bugs API problem?
<wgrant> jtv: Yeah.
<wgrant> It looks like it may be a distroseries vocab change, which rvba has been touching a bit lately IIRC.
<jtv> I think so, yes
<adeuring> good morning
<mrevell> Bom dia
<wgrant> I think I almost understand buildout!
<wgrant> bigjools: Morning.
<bigjools> morning wgrant
<StevenK> wgrant: Are we deployable *now*?
<wgrant> StevenK: Has qastaging updated?
<wgrant> bigjools: lazr.amqp is to be torn apart and renamed.
<StevenK> qas is running r13405
<wgrant> As we don't use anything except for lazr.amqp.async
<wgrant> lifeless suggests deleting everything else and renaming to txlongpoll
<bigjools> dramatic
<wgrant> Yay, batchnav is no longer broken.
<wgrant> We may be deployable.
<wgrant> If we are very lucky.
<stub> Did batchnav get fixed or are you talking about the rollback?
<wgrant> stub: The latter.
<stub> wgrant: Nah. Never seemed worth the trauma or load.
<lifeless> jtv: its a bugs call, but it looks like a distroseries thing
<jtv> lifeless: yes â you could ask le stigue if his vocab work might have played a role, but he's unavailable today.
<jtv> lifeless: the method smelled like IHasBugTasks.
<lifeless> sure, I was just grabbing the closest red teamer :)
 * jtv wonders idly what connotations "red teamer" might have in English
<jtv> Still, probably safer than "scarlet brigade."  That sounds positively filthy.
<lifeless> night night
<jtv> night!
<danilos> adeuring, hi, I suppose you don't want to finish reviewing https://code.launchpad.net/~danilo/launchpad/replace-expanders-2/+merge/67314 and I should reassign back to lp-reviewers?
<adeuring> danilos: sure, I can do it
<danilos> adeuring, cool, thanks (I've fixed the problem with the focus event, should be fine now)
<adeuring> danilos: yes, I already noticed it!
<danilos> adeuring, yeah, the problem was not fixing the issue, but keeping the keyboard focus behaviour :)
<adeuring> dar=me (had to raid my desk to find my notes from friday ;)
<adeuring> danilos: r=me
<danilos> adeuring, cool, thanks
<wgrant> allenap, bigjools: Any objections to the pruning, renaming, and removal from LP?
<bigjools> wgrant: of amqp?
<wgrant> bigjools: Yes.
<bigjools> no
<wgrant> Thanks.
<mpt> lifeless, I'm not sure how I feel about others editing it, but it's quite short, so perhaps there could be a Comments/Discussion section underneath? W.r.t. "add"/"remove"/"change", really any imperative summary falls into that category ("Make this work this way"). Excellent for issues classed as to-dos, not so good for issues classed as defects.
<wgrant> I have a deployment strategy now too.
<StevenK> wgrant: For lazr.whatevernameyoupicked or LP itself?
<wgrant> StevenK: txlongpoll (nÃ©e lazr.amqp)
<bigjools> wgrant: what's the "tx" bit?
<wgrant> bigjools: Twisted extensions
<wgrant> https://launchpad.net/tx
<bigjools> ...
<bigjools> is it moving out of lazr?
<wgrant> That's apparently the plan.
<bigjools> k
<wgrant> It is de-Zoped and now entirely twisted.
<bigjools> https://launchpad.net/txamqp
<bigjools> arf
<wgrant> txamqp is the AMQP library it uses.
<bigjools> yeah
<bigjools> which begs a question ...
<wgrant> Oh?
<bigjools> WTF didn't anyone think to use tx-something in the first place!
<wgrant> Well, it had all the Zopey crap in it initially.
<bigjools> where anyone == landscape guys
<wgrant> Ah
<bigjools> anyway, all's well
<wgrant> I also now understand buildout.
<wgrant> Which is handy for this sort of thing.
<al-maisan> is anything special going on at present? I am trying to add a tag to https://bugs.launchpad.net/openquake/+bug/803419 but launchpad will not let me
<_mup_> Bug #803419: permission denied for relation geography_columns <database> <OpenQuake:Confirmed> < https://launchpad.net/bugs/803419 >
<bigjools> yes, something Is Bad
<al-maisan> also, when I refresh the bug page it will say "Log in/Register" although I *am* logged in..
<StevenK> al-maisan: You're not anymore
<StevenK> The session DB was just ... cleansed
<al-maisan> how interesting ;)
<StevenK> Ah no, we do have an issue
<StevenK> al-maisan: Wait a few?
<al-maisan> sure
<wgrant> al-maisan: Should be OK now, if you log in again.
<al-maisan> wgrant: thank you!
<wgrant> The session DB switch was a little more troublesome than expected.
<danilos> mrevell, hey, I wonder if I should say things like "they is" for the "singular they" :P
<al-maisan> I was able to add the tag FWIW .. thanks again !!
<danilos> mrevell, also, are we seriously on British English for LP?
<mrevell> danilos, Singular they is perfectly acceptable English :) As for British English ... well ... maybe this needs more discussion on the list but the company standard is British English and we are rebranding LP to fit more in line with the company brand.
<wgrant> Canonical code seems to be American English, while our text seems to be British English... I don't know why we'd use American anywhere :/
<mrevell> For a long time, I think the interface text was a case of personal choice, then we settled on a policy of American English (I forget the reasons why) but for a couple of years now our company policy has been explicitly to use British English.
<danilos> mrevell, "they is" sounds very weird to me, but then again, my English is better than yours :P
<jpds> There's.... a singular they?
<danilos> mrevell, should we make use of the scripts different en_GB teams have to switch the hood to the bonnet across the board?
<mrevell> danilos, Oh, I thought you were joking. Singular they still takes "are". It's a compromise. It's better than using "he" everywhere, it's also better than using "she" every now then as a token effort.
<mrevell> danilos, Such scripts exist?
<danilos> mrevell, right, I was joking but I didn't realise you were as well :)
<mrevell> haha
<danilos> jpds, a recent invention, I am sure :)
 * mrevell goes to find the Oxford Style Manual
<bigjools> can we please nuke the Oxford Comma from orbit?
<danilos> mrevell, good excuse to be gone for hours, good luck and enjoy the beer :)
<nigelb> On a positive note, we'll have lots of "How do I fix the color for that box?" "Add a u"
<mrevell> It was right next to me. I'm just finding the right page.
<henninge> Is there such a thing as a "webservice console"
<henninge> ?
<henninge> I mean a script that uses launchpad lib to query the webservice for testing?
<danilos> henninge, I've seen it mentioned a few times, if there is, it's probably hosted as a project on LP :)
<danilos> henninge, perhaps lpshell by poolie
 * henninge looks
<poolie> henninge, there is a gui client in wrested
<poolie> and there is lpshell in lptools
<poolie> it depends what specifically you want to do
<henninge> poolie: lpshell sounds like what I am looking for. Thanks
<henninge> poolie: only there is no lpshell in lptools
<maxb> henninge: lp-shell
<maxb> oh, and ubuntu-dev-tools
<mrevell> danilos, Basically, singular they, otherwise known as "generic they" has been used in English for hundreds of years. The alternative is to use generic "he", which is nowadays unacceptable for appearing to refer to one gender only (rather than being used in place of a neuter). From what I've read over the years, dislike of singular they appears to be an American hang-up.
<wgrant> poolie: Thanks for forwarding that.
<danilos> mrevell, oh, that's nice, I've only seen increased use of it in the last few years in the day-to-day communication (I've seen it used in literary works from way past)
<poolie> +1 to they
<poolie> s//hooray for they
<mrevell> danilos, I think it's used increasingly as a way to avoid any unintended gender bias. I think it works well for that and it's a bonus that Shakespeare used it.
<mrevell> heh
<mrevell> :)
<danilos> mrevell, anyway, what I was trying to say with the comment was that you may want to explain that "singular they" is still used with "are" for those non-English developers such as me, just so we don't have to think twice :)
<danilos> mrevell, I used to use "they" just as if it were the plural they, so the term "singular they" confused me
<mrevell> danilos, Ah, thanks, yes.
<mrevell> danilos, I shall change it to "generic they" and give a usage example.
<wgrant> jml: What is https://blueprints.launchpad.net/lazr.amqp/+spec/silly-blueprint?
<danilos> mrevell, cool, thanks
<henninge> danilos, maxb, poolie: lp:ubuntu-dev-tools/lp-shell is what I was looking for. Thanks!
<poolie> mrevell, the more examples actually taken from lp text, the better, imo
<mrevell> poolie, Cool. I'm always tempted to use "That and a pair of testicles" as an example of what we're trying to move away from.
<wgrant> Has that been removed yet?
<mrevell> But, yeah, I'll use some real world examples. Any suggestions gratefully received :)
<wgrant> I recall its abolition was considered a couple of years back.
<mrevell> wgrant, Yeah, it went quite a long while back AFAICR>
<mrevell> s/>/.
<wgrant> Good, good.
<poolie> mrevell, that is a perfect example
<poolie> i might have even excised them
<mrevell> poolie, I'm probably missing something but I'm not sure I understand the problem with "Read about uploading"-style links.
<mrevell> Can you help me understand?
<poolie> i will try
<poolie> should it be "Tell me about uploading" or "Read about uploading"?
<poolie> it's not a critical bug
<poolie> it just kind of sticks out to me as possibly inconsistent
<poolie> or should it be "Help on uploading"
<mrevell> Oh, I see.
<poolie> the question is, is it just a noun phrase describing what's at the end of the link, or is either lp or the user the subject of an imperative
<mrevell> I think that in line with mpt's guidance "As a user, you interact with the control, so it speaks as your voice", it should be "Tell me about..." The imperative seems to make the link more personal; maybe even friendlier.
<poolie> right, though perhaps "tell me about" is verging on too cute
<poolie> but i think the essential thing is that most hyperlinks, as opposed to buttons, don't have verbs at all
<poolie> (is this true?)
<mrevell> poolie, We have "Report a bug", "Ask a question" etc as hyperlinks, rather than buttons.
<mrevell> So, if there is a tendency not to have verbs in links then we don't have it as rule, I'd say.
<poolie> yeah, i was just thinking that
<poolie> we have a weak pattern of links that appear in button-like places being phrased more like actions, and those that appear in text being more like real hyperlinks
<mrevell> We tend to use a verb, I think, where there's a definite action, rather than just a request to list information.
<mrevell> s/definite action/something that changes data
<poolie> right, perhaps that's the rule
<poolie> that it shouldn't be "Tell me about blah" or "show blah" or "list blah"
<poolie> maybe there's even a rule for that?
<mrevell> I don't think it's a written rule. I'll trawl the dev wiki and elsewhere for anything else and attempt to bring it all together. I'm also going to go read up on what other people have done with the wording of help links :)
<poolie> ok
<poolie> i don't want to cause people to get stuck on that particular question
<mrevell> I think it's an interesting point though and I bet other people have handled it in ways that we can borrow.
<poolie> there are probably a lot of other style guides that cover it
<poolie> it seems like the current wording page assumes that only buttons are actions, which is not ture
<danilos> mrevell, btw, you should talk to en_GB people about the script, I am sure they have a few handy
<mrevell> danilos, Cheers. Do you have an idea of who I should speak to?
<danilos> mrevell, http://live.gnome.org/BritishEnglish is a good starting point (there's a script there as well)
* wgrant changed the topic of #launchpad-dev to: devel in RC until r13405 deployable | Performance Tuesday | https://dev.launchpad.net/ | On call reviewer: - | Critical bugs: 220 - 0:[######=_]:256
<jml> wgrant: you noticed my attempt to get top contributor :)
<wgrant> jml: Heh
<wgrant> Then I turned off blueprints.
<jml> good thought.
<benji> flacoste: I think lp:~benji/launchpad/bug-781600 is ready for you to look at; do you want to do a formal review or an informal one?
<cjwatson> OK, how do I sign files in lib/lp/archiveuploader/tests/data/suite/?  The existing ones are signed with key ID 6C64A8C5, and I *know* I've done this before but I can't remember how.  I've found lib/canonical/launchpad/ftests/gpgkeys/foo.bar@canonical.com.sec, but it has a password I don't know, and lib/canonical/launchpad/ftests/gpgkeys/foo.bar@canonical.com-passwordless.sec has a different key ID.
<cjwatson> lib/lp/archiveuploader/tests/data/secring.gpg looked plausible but doesn't contain this key.
<cjwatson> that's confusing, the last time I did this apparently I got it from lib/canonical/launchpad/ftests/gpgkeys/ - but where did I get the passphrase?
<henninge> which script do I need to run on dev to create a diff for a merge proposal?
<cjwatson> bigjools,wgrant: do either of you know the answer to my gpg question?
<bigjools> cjwatson: the password is "test"
<bigjools> IIRC
<wgrant> That is also my recollection.
<bigjools> NFI why someone decided it was a good idea to add a password
<cjwatson> aha!  thank you
<bigjools> henninge: run_jobs in the right job source
<cjwatson> (I'm fixing up dpkg-xz-support)
<bigjools> I forget the name but you have powers to find out :)
<henninge> bigjools: thanks, looking
<bigjools> henninge: there's a mergeproposaldiffpreview or similar
<bigjools> job, I mean
<wgrant> henninge: make sync_branches
<wgrant> Or cronscripts/mergeproposaljobs.py
<wgrant> Or you could run the job directly, but I don't know if that actually works these days.
<bigjools> ah that's it
<henninge> wgrant: cronscript/merge-proposal-jobs does not do it, neither does make sync_branches :(
<wgrant> henninge: We found during the sprint that some occasionally got stuck somehow... perhaps the job has failed?
<wgrant> Try resubmitting or deleting and recreating.
<bigjools> ok I can't stare at this issue any longer, I need help.  What are the usual reasons a security adapter is not called?
<wgrant> bigjools: The object is not proxied.
<wgrant> Diff?
<bigjools> it is
<bigjools> http://pastebin.ubuntu.com/642617/
<bigjools> I am fixing an existing problem I found when realising a permission test was in the zopeless layer
<bigjools> makePlainPackageCopyJob() is returning a proxied object
<wgrant> Does test_extendMetadata_is_privileged still pass?
<bigjools> not any more
<bigjools> that's the one that was in the zopeless test layer
<bigjools> it only passed because the zcml was fucked
<wgrant> Hmm. Possibly relevant is that your adapter is for IPackageCopyJobEdit. It should normally be for IPackageCopyJob, but that's hopefully not what's breaking this.
<bigjools> ah...
<wgrant> It's still odd and should be fixed.
<bigjools> let's see
<bigjools> didn't fix it, no
<wgrant> Try the adaption manually, and see if gets the right adapter?
<bigjools> I can't remember the runes
<wgrant> Let me see.
<bigjools> sleep deprivation does that
<wgrant> I've been up since 3am :)
<wgrant> (Pdb) queryAdapter(pcj, IAuthorization, name='launchpad.Edit')
<wgrant> <canonical.launchpad.security.EditPackageCopyJob object at 0xeff4c90>
<bigjools> ta
<bigjools> so that worked, but ... :/
<wgrant> It's really like it's running with PermissiveSecurityPolicy...
<bigjools> yes
<bigjools> the zcml looks ok though
<gary_poster> flacoste, https://bugs.launchpad.net/launchpad/+bug/775691 : should we contact david planella with danilos's analysis or should we treat the migration as escalated?  danilo says the migration will be pretty tough, and there are workarounds
<_mup_> Bug #775691: Empty translations on one side do not get translated by the other side <escalated> <not-pie-critical> <upstream-translations-sharing> <Launchpad itself:Triaged by yellow> < https://launchpad.net/bugs/775691 >
<wgrant> bigjools: Ahahahah
<wgrant> bigjools: It's a PlainPackageCopyJob, not a PackageCopyJob.
<bigjools> :/
<wgrant>     <!-- PlainPackageCopyJob -->
<wgrant>     <class class=".model.packagecopyjob.PlainPackageCopyJob">
<wgrant>       <allow interface=".interfaces.packagecopyjob.IPackageCopyJob" />
<wgrant>       <allow interface=".interfaces.packagecopyjob.IPlainPackageCopyJob" />
<wgrant>     </class>
<bigjools> of course - it's bloody delegating
<bigjools> gnargh
<bigjools> thanks wgrant
<cjwatson> Hmm, I'm trying to fix my dpkg-xz-support branch and getting very confused about tests.  I have a testXZDebUpload test defined right after testLZMADebUpload, and basically cargo-culted from it.  testLZMADebUpload tests for package uploads in the NEW queue, so I did the same in testXZDebUpload.  But apparently the upload ends up in ACCEPTED instead, suggesting that something isn't being torn down properly.  How do I get ...
<cjwatson> ... rid of the stale state from the previous test?
<cjwatson> this is in lib/lp/archiveuploader/tests/test_uploadprocessor.py
<bigjools> cjwatson: it's likely to be the upload policy, not test isolation
<cjwatson> wouldn't the same policy apply to both lzma and xz uploads?
<bigjools> paste a diff
<wgrant> cjwatson: You're sure you cloned the right upload?
<wgrant> All DB state should be reset between tests, unless scripts are doing something bad.
<bigjools> it's probably a package with ancestry
<cjwatson> I *think* so ...
<bigjools> so gets auto-accepted
<cjwatson> https://code.launchpad.net/~cjwatson/launchpad/dpkg-xz-support-619152/+merge/32868
<cjwatson> er, bollocks, that's stale, one moment
<cjwatson> bigjools: the MP's being slow to update, but http://paste.ubuntu.com/642636/ is my current diff against devel
<flacoste> benji: i'll do a formal one
<flacoste> gary_poster: let's leave that decision to david
<benji> flacoste: cool, in that case I'll write up an MP and ping you when it's ready
<flacoste> benji: great
<cjwatson> oh, yes, and it doesn't look like test isolation because 'bin/test --pdb -t lp.archiveuploader.tests.test_uploadprocessor.TestUploadProcessor.testXZDebUpload' shows the same thing
<bigjools> cjwatson: the one in accepted will be the source
<gary_poster> flacoste: ack will do
<bigjools> cjwatson: looks like a genuine upload failure
<bigjools> check what processUpload returns
<cjwatson> Aha.  Is the upload log left anywhere, or do I just need to pdb it?
<bigjools> it probably gets cleaned up unless you can see something hiding in /tmp
<bigjools> I'd pdb it
<cjwatson> OK, thanks
<cjwatson> SystemError: "E:This is not a valid DEB archive, it has no 'data.tar.gz', 'data.tar.bz2' or 'data.tar.lzma' member"
<cjwatson> We need an apt backport too, it seems :-(
<bigjools> cjwatson: bugger
<benji> flacoste: ok, the MP and MP-inspired polishing is done: https://code.launchpad.net/~benji/launchpad/bug-781600/+merge/67704
<bigjools> cjwatson: heh, that point was raised in April on the MP as well
<cjwatson> bigjools: that was python-apt, and I'm testing with that
<cjwatson> but there's a bit of relevant code in apt too
<bigjools> ah ok, thought you meant the former
<flacoste> benji: i'll probably only review after lunch
<benji> sounds good
<henninge> adeuring: I found the error cause.
<henninge> adeuring: it is not revision ids.
<adeuring> henninge: cool, what was the problem?
<henninge> adeuring: diffstat is defined as a Text field but it contains a dict.
<henninge> {u'__init__.py': (1, 0)}
<adeuring> henninge: argh...
<henninge> adeuring: the standard marshaller applies a blanket unicode cast to all values.
<henninge> so it becomes u"{u'__init__.py': (1, 0)}"
<adeuring> henninge: interesting... I'm wondering why nobody noticed this before in API applications...
<henninge> adeuring: as I said, it simply converted to unicode which our marshaller doesn't do
<henninge> I guess nobody ever used this particular value
<adeuring> henninge: right, seems so
<henninge> adeuring: I have to leave now, I will think about a proper fix tomorrow.
<henninge> probably just do the same thing ...
<adeuring> henninge: schÃ¶nen feierabend
<henninge> danke
<jtv> Nobody here to review https://code.launchpad.net/~jtv/launchpad/bug-809211/+merge/67731 ?
<jml> jtv: tl;dr. sorry.
<m4n1sh> is valc-0.14 actually vala git HEAD?
<m4n1sh> s/valc/valac
<m4n1sh> sorry
<m4n1sh> wrong channel
<m4n1sh> oops
<jml> cross-posted from canonical/#tech, is there a way of using virtualenv without it downloading packages from the internet all the time?
<jml> gary_poster: you know about this stuff, right?
<jcsackett> jml: as in someone wants to use virtualenv and locally install eggs themselves?
<jml> jcsackett: yeah. I've got the eggs from *previous* runs of virtualenv, and sometimes like creating new branches and hacking on them without internet access.
<gary_poster> jml, somewhat.  Making a new virtualenv without downloading packages might or might not be supported.  Once it is installed,
<gary_poster> I'm pretty sure that setuptools/distribute allows you to specify where to look for eggs
<gary_poster> buildout uses those options itself
<gary_poster> I'm not sure what those options are, but that's where I'd start
<jml> gary_poster: ok, thanks.
<gary_poster> np
<flacoste> sinzui: http://approvaltests.sourceforge.net/
<flacoste> sinzui: pycharm includes image diff support in 1.5, so would probably be possible to have good tool support there
<sinzui> :)
<lifeless> morning
<nigel> already? Unfair!
<lifeless> nigel: happens like clockwork
<nigel> heh
<lifeless> flacoste: ping
 * abentley finds it depressing that "[] !== []" in JavaScript.  Also finds !== depressing.
 * maxb blinks, and cries
<maxb> seriously, wtf?
<maxb> Oh, because Array does define sensible equality :-/
<jtv> In python, "[] is []" also evaluates to false.  Makes sense to me.
<cjwatson> !== is object identity rather than equality IIRC
<jtv> That's right.
<nigel> Isn't !==, the not of === which is object equality?
<abentley> cjwatson, jtv: No, !=== is object identity.  !== is no-coerce object inequality, != is coerce object inequality.
<nigel> s/equality/identity
<cjwatson> oh dear god how many operators
<cjwatson> and I'm speaking as somebody who likes writing perl
<flacoste> hi lifeless
<flacoste> benji: i sent you a bunch of comments on your branch
<benji> flacoste: thanks, I'm looking at them now
<flacoste> benji: let me know if there is anything confusing in them
<lifeless> flacoste: I forgot to ask you about my emeritus idea on the call yesterday
<abentley> cjwatson: I tell a lie.  You are correct.  However, [] != [] too.
<benji> flacoste: on first reading everything looked good
<flacoste> lifeless:  i liked it!
<lifeless> I'll take it to -dev ?
<lifeless> maxb: btw, I think that duping you did is incorrect
<lifeless> maxb: before I undo it, can you explain why they are dupes?
<maxb> The merged proposed team membership?
<maxb> It seemed like both bugs were complaining about being unable to operate on merged proposed team members
<lifeless> bug 204002 is about proposals after doing  amerge
<_mup_> Bug #204002: can't approve/decline a member with a merged user <404> <lp-registry> <merge-deactivate> <Launchpad itself:Triaged> < https://launchpad.net/bugs/204002 >
<lifeless> bug 393914 is about actual membership after a merge
<_mup_> Bug #393914: ~team membership of ~X-merged can not be deactivated <404> <canonical-losa-lp> <lp-registry> <merge-deactivate> <Launchpad itself:Triaged> < https://launchpad.net/bugs/393914 >
<lifeless> AFAICT that is
<lifeless> maxb: ^
<maxb> Oh. I figured they both must be talking about proposed membership, on the assumption that actual memberships would have been fixed up by the merge process
<lifeless> benji: on bug 809470, did you mean 'triaged low' ?
<_mup_> Bug #809470: Wishlist: Time and Taskplanning <gantt> <pert> <planning> <task> <time> <Launchpad itself:Won't Fix> < https://launchpad.net/bugs/809470 >
<lifeless> benji: I only ask because setting an importance seems redundant when wontfixing :)
<benji> lifeless: I meant "N/A" but that wasn't an option
<lifeless> gmb: oh hai :P
<benji> I almost used "Wishlist" but I don't think we really wish it to happen, and Undecided didn't seem to make sense.
<gmb> lifeless: Wotcher.
<lifeless> benji: it may be a linaro request;
<lifeless> gmb: since you are hanging around, want to sort out this linked_branches thing?
<gmb> lifeless: I won't be around for much longer, but we can give it a stab, sure.
<lifeless> benji: If rephrased into a symptom based report, I think that that bug would look closer to jmls timelines/todo lists/thingy concept, which we do want to do
<lifeless> gmb: so its a bait-and-switch on the implementation, the contract stays unaltered
<gmb> Right.
<lifeless> gmb: make new getter with one parameter(user); unpublished linked_branches in the API, publish the new thing as linked_branches with the user parameter auto-bound
<gmb> lifeless: Okay. I don't know enough about how the API is exposed in WADL, etc., but might that not break callsites?
<gmb> e.g:
<benji> lifeless: could be
<gmb> if it's currently:
<gmb> branches = bug.linked_branches
<lifeless> gmb: stays the same, because you're publishing it as a getter, not as a function
<gmb> Won't that break when we turn it into a getter? (i.e. wouldn't it require linked_branches())?
<gmb> Okay.
<gmb> lifeless: I think there's a cognitive gap here that I need to fill in, but I'm with you now.
<lifeless> there are examples that can be cargo culted, its the API version of 'property(get_fn)'
<gmb> lifeless: Okay. I'll dig up an example and rip it off liberally tomorrow am.
<gmb> Thanks for taking the time to explain things :)
<lifeless> I'm probably describing it slightly wonky ;)
<lifeless> benji: I might put it incomplete and follow up with some questions
<gmb> lifeless: Well, we'll see. If I get it right first thing tomorrow then I was paying attention wonkily :)
<lifeless> gmb: \o/
 * gmb goes to do evening stuff
<lifeless> benji: I'm confused by 809508 vs 809511
<benji> lifeless: how so?
<lifeless> aren't they the same thing ?
<benji> that would be confusing :)  nope, one is about viewing a bug and associating it with a branch, the other is about viewing a branch and associating it with a bug
<benji> same result, different contexts
<lifeless> ah, cool
<flacoste> lifeless: deployment-stable.html shows 13405 as QA-OK, are we good to deploy and reopen PQM?
<lifeless> cr3: have you tried your API yet ?
<flacoste> lifeless: -emeritus and asking -dev, good idea
<lifeless> flacoste: I need to touch base with wallyworld/wgrant, rev 13391 also had some questions around it
<flacoste> lifeless: ok, thanks, would have been nice to have an email to the list about the status of the queue
<flacoste> as everything was green
<flacoste> it wasn't clear without reading the IRC backscroll what was the status
<lifeless> flacoste: indeed; I asked wallyworld and wgrant to decide and sort it out last night
<lifeless> I had parenting class
<flacoste> that was one thing that having a rotating release manager eased out
<lifeless> agreed, its easier for that to be lost now
<cr3> lifeless: yep, I commented on bug #801334 and set the tag to qa-ok
<_mup_> Bug #801334: IHWSubmissionSet.search should be available in the API <qa-ok> <Launchpad itself:Fix Committed by cr3> < https://launchpad.net/bugs/801334 >
<lifeless> cr3: great, thanks
<lifeless> does anyone remember what jmls .., ah dashboards
<cr3> benji: if you happen to have a moment, might you know how a project is able to find a project_series under it when accessing /<project.name>/<project_series.name> in the api? wouldn't IProduct have to define a default collection for that to work?
<lifeless> cr3: project.active_series
<lifeless> sinzui: isn't bug 95419 in your todo list ?
<_mup_> Bug #95419: Record dependencies between bugs <lp-bugs> <oem-services> <Launchpad itself:Triaged> < https://launchpad.net/bugs/95419 >
<cr3> lifeless: aha! lazr.restful probably calls upon ProductNavigation which has: traverse(name): return self.context.getSeries(name)... that must be it
<flacoste> lifeless: btw, we have the green-light from statik on killing the lazr brand
<lifeless> flacoste: sweet
<cr3> what's this about killing lazr?
<lifeless> flacoste: I'll document creating projects on the wiki shortly
<lifeless> cr3: not killing as such, but de-emphasising its use for new projects
<lifeless> cr3: aiming for good snappy names, avoiding issues with namespace packages
<cr3> lifeless: for my edification, what's wrong with namespace packages?
<lifeless> that they exist?
<lifeless> cr3: in debian/ubuntu they can't be imported unless the package above exists, but the upstream packaging doesn't include that, or if it does include that multiple packages are then including the same file on disk.
<lifeless> cr3: its a right PITA
<cr3> lifeless: I was planning to split my launchpad-results project into namespace packages, for client and server to share a common name for example, it was a pain indeed and I'm reassured that it wasn't just me :)
<rockstar> Why are the fonts on Launchpad so small now?
<sinzui> lifeless, it is not truly in scope. I hope that a simple enum can be used so that dependencies are cheap to do (a week of work)
<lifeless> sinzui: could you unpack that for me ?
<sinzui> rockstar, the css is now Ubuntu Web Guidelines.
<sinzui> rockstar, There may be a switch back to em's which is easy to do after the normalisation I did to get it to px.
<rockstar> sinzui, so are you saying there are bugs with sizing that need to be worked out, or that they are correct (because I debate that)
<lifeless> rockstar: they are correct per the guidelines we're getting. Some users (e.g. me and you :P) find them impossible to read.
<sinzui> lifeless, we are making a is b, a duplicate that is not presented as a duplicate because we want separate conversations
<rockstar> lifeless, than the guidelines are wrong.  :) I like small fonts, and those are still painful to read on my 23" 1080p monitors.
<sinzui> rockstar, no. I *believe* that the Canonical accessibility/WAI drive requires a review of the px rule
<rockstar> sinzui, okay.  Just as long as someone is looking to correct that.
<rockstar> I first noticed it on "Propose for merging" which seems the LAST thing you want to be tiny and out of the way.
<lifeless> sinzui: hmm, I had thought we were doing 'a depends on b' which combined with privacy addresses the OEM use case as well, but is more generally useful
<sinzui> rockstar, the assumption of px was that all browsers have fixed the zooming rules for control++ and control+-. The assumption did not consider that users do not know about the feature
<sinzui> lifeless, you and I want directionality, but stakeholders have only discussed confidential conversations.
<lifeless> sinzui: concretely, I think an 'a is b' relation is something we wouldn't do if we did 'a depends on b', and so its a false economy to build it - we'll have to undo the work later
<rockstar> sinzui, well, I think there should be sensible defaults as well though.
<sinzui> lifeless, We will review and revise the requested feature in a few weeks.
<lifeless> sinzui: ok; if its ok with you I'd like to be part of that review
<sinzui> rockstar, the default happens to be the same as the Ubuntu OS defaults. The issue is that browsers always apply their own scaling rules based on 16pt.
<sinzui> lifeless, yes please
<lifeless> sinzui: \o/ - ok, ping me whenever and I'll come to you.
<sinzui> lifeless, If we can agree that the bug.duplicate column does not scale, that we need a relational table, we can solve a lot of problems once we have the migration script settled
<lifeless> sinzui: we need to know the answer to questions like 'do we need to search on the relation?' 'is it a tree (A is unique in the set of all existing (A,B) tuples), or a graph (there can be many (A,B) tuples for a given A'
<wgrant> lifeless: I believe it is not worth rolling back.
<lifeless> ok
<lifeless> so we're green
<rockstar> sinzui, I'd assert that this is a problem with CSS: http://ubuntuone.com/p/14Ad/
<lifeless> wgrant: one of us should hav emailed the list about the status
<lifeless> wgrant: not saying you or wally specifically, but as a group we dropped the ball
<rockstar> The line height is 18px, to make space for the sprite, but then the text is 12px.  At least make it fill the line-height a bit more.
<wgrant> lifeless: I had failed to notice that the downtime was a day earlier than normal.
<wgrant> lifeless: So I thought we still had >24 hours.
<lifeless> yah
<flacoste> Language:
<flacoste>     English â¼
<flacoste> Copyright Â©1999-2011 SurveyMonkey
<sinzui> rockstar, most of Lp sprite issues were solved by the CWG's normalised font/line-height rules. Lp's sizes were impossible to predict using YUI-font
<rockstar> sinzui, I guess I'm still unclear as to whether or not this is a known bug or a "this is acting as advertised and if you have a problem with it, tough"
<sinzui> rockstar, I believe there are a few places trying to make the text very small in Lp. I changed the rules to ensure they could never be less than 10px. I am not sure any text other than the footer should ever be smaller than the base font.
<sinzui> rockstar, it is not a bug. It is by design
<lifeless> wgrant: I mean a status update, not a conclusion
<lifeless> wgrant: we left everyone hanging not sure
<lifeless> wgrant: and not sure if they should be not sure
<rockstar> sinzui, so it's going to be left like that?
<lifeless> rockstar: unless the canonical design department change the rules, or we make a case for not following the rules
<flacoste> rockstar: for the moment yes
<lifeless> rockstar: we believe they are doing a review now, so there is little point making a specific case right now
<sinzui> You sample page was reviewed two weeks ago and it needs a redesign. When that page knows it is working with a temporary or permanent branch, the repetitive and odd heading, list, actions will be gone
<flacoste> rockstar: we are working on a visual refresh of Launchpad, I'll let huwshimi know about the font issue
<rockstar> Okay, so I guess the take-away is that I need to STFU for a few weeks and let things wiggle out.
<sinzui> flacoste, huwshimi discussed the WAI+font-size issue with me 4 weeks ago.
<rockstar> Am I correct here?
<flacoste> rockstar: might be a few months unfortunately :-)
<flacoste> rockstar: you have a too big monitor anyway :-)
<rockstar> flacoste, okay, as long as there's an acknowledgement that we're not "done" yet.
<flacoste> get a normal size screen like the rest of us
<flacoste> who needs more than 12"!
<wgrant> This hasn't changed in months, anyway.
<rockstar> wgrant, maybe my cache just got cleared then, because it wasn't like this yesterday when I submitted my branches for review.
<sinzui> rockstar, the fonts *are* done. The rules for the fonts *may not* be done. The refresh id the opportunity to get CWG revised
<lifeless> flacoste: my new monitor - 24" 1080p :P
<lifeless> flacoste: and I know poolie has larger :>
 * rockstar highfives lifeless 
<flacoste> you damn gamers!
<poolie> mines' 27
<rockstar> sinzui, okay, let me just register my opinion as "No call to action should ever be that small"
<flacoste> 27" that's bigger than my TV
<sinzui> rockstar, exactly!
<poolie> ah canada :)
<rockstar> poolie, I have 2 23" monitors, does that mean I win?
<poolie> i'm thinking about retiring the smallest one and going to the largest dell model
<poolie> only if you have two heads
<rockstar> sinzui, okay, you agree, so I'll STFU and go back to work.
<poolie> i can not get used to having a gap in the middle of my visual field
<poolie> though you do win from having a standing desk
<rockstar> Yeah, I was trying to think of a condescending way to let you know I'm still in the standing desk club, so obviously better than you.  :)
<lifeless> poolie: smallest one ?
<wgrant> rockstar: Are you sure you didn't zoom out or something?
<poolie> S has my old 24in
<rockstar> wgrant, positive.
<poolie> rockstar: i stood up with my laptop on a bar yesterday
<poolie> that was pretty good
<poolie> may take some getting used to
<rockstar> poolie, it certainly does.  Once you get used to it, you'll need to pick up barefoot running.  That's the next step in being condescendingly forward thinking.  :)
<wgrant> rockstar: The last font size changes were made at the start of March.
<wgrant> rockstar: And there have been none at all for at least a month.
<wgrant> s/last/last significant/
<lifeless> we did reset the session db over night
<rockstar> Yes, and I did have to log back in.
<lifeless> which also should have been announced - we've users affected by it
<wgrant> With spectacular side-effects, but this shouldn't be one of them.
<lifeless> never say shouldn't ;)
<wgrant> lifeless: It was also entirely avoidable.
<lifeless> :(
<wgrant> It would have taken roughly 120 seconds to copy the contents of the DB over.
<rockstar> I sent a branch page example out to others who all agree it's too small, so it's not just me.
<wgrant> rockstar: A page, or a screenshot?
<rockstar> wgrant, page.
<flacoste> rockstar: standing desk is so passÃ©, the future is threadmill desk!
<flacoste> which you should obviously use barefoot :-p
<rockstar> flacoste, threadmill? Like, multiple treadmills with some mutex?
<rockstar> :)
<flacoste> rockstar: ask gary for some pictures :-)
<wgrant> lifeless: Did you see my txlongpoll deployment proposal?
<lifeless> buildout.cfg? sure, I don't care particularly how its done.
<lifeless> It will still be rsync based
<wgrant> Yeah.
<wgrant> I'm wondering how we should approach OOPS integration in general.
<wgrant> Since we really don't want to force *oops into every upstream project we come across..
<lifeless> sure we do
<lifeless> catchall exception handler for most stacks is doable
<lifeless> I looked at using wsgi-oops but its *sob* a namespace package, doesn't selftest for me, and has all sorts of unrelated stuff (like storm glue) in it.
<wgrant> Yeah.
<wgrant> Hmm.
<wgrant> Maybe we can inject into sitecustomize or something.
<lifeless> no
<lifeless> its a http stack specific thing
<lifeless> wsgi being one stack, twistd anther
<lifeless> twisted another
<lifeless> java and scala containers have analagous concepts
<lifeless> its a small amount of work compared to manually checking log files
<lifeless> wallyworld_: hi
<wallyworld_> lifeless: hello
<lifeless> wallyworld_: I'm curious why in https://code.launchpad.net/~wallyworld/launchpad/private-branch-unsubscribe-283167/+merge/67651 owner members cannot unsubscribe people ?
<wallyworld_> lifeless: i thought that it would be too permissive
<wallyworld_> but i can change it
<lifeless> wallyworld_: its inconsistent with what owner means elsewhere
<wallyworld_> ok. i'll fix it
<lifeless> thanks!
<wallyworld_> np. thanks for pointing it out
<lifeless> no worries
<lifeless> its really awesome we're polishing so many privacy related things
<wallyworld_> there's a *lot* more to do
<wallyworld_> one step at a time
<lifeless> indeed
<sinzui> StevenK, mumble?
<cjwatson> phew.  finally have dpkg-xz-support working, with a combination of lucid-proposed, mvo's PPA, and a further python-apt change of my own
<cjwatson> (the last not in oneiric yet ...)
<wgrant> cjwatson: Excellent. How much work is this going to be to backport?
<cjwatson> wgrant: I've just sent mail about that
<cjwatson> we might as well deal with this and the LongDescription configuration work at the same time
<cjwatson> LP machines pull from lucid-updates, don't they?
<lifeless> possibly, same as other dc machines
<wgrant> I believe they do.
<sinzui> StevenK, http://pypi.python.org/pypi/zope.formlib
<cjwatson> the xz support will go there (min delay a week from when mvo processes my mail, though); not sure about the LongDescription change, we might ask that that go into the LP PPA instead
<cjwatson> but mvo's already backported it
<cjwatson> if we do that in a PPA, the xz support will have to be merged into it
<StevenK> wallyworld_: Seriously, your microphone sounds like a *kettle*
<wallyworld_> StevenK: i was muted for most of the time unless i wa speaking
<wgrant> No, it's a sonic screwdriver.
<wallyworld_> wgrant: did you raise a bug to fix the person picker wording issues?
<wgrant> Not yet.
<wgrant> lifeless: Are we out of RC yet?
<lifeless> ask losa; I requested it a while back
<wgrant> LOSA pingaling
#launchpad-dev 2011-07-13
<spm> yo
<spm> lifeless: we aren't yet. mb was unable to get to it; I've been fixing a few reds.
<spm> lifeless: any idea where https://dev.launchpad.net/PolicyAndProcess/ReleaseManagerRotation went? had a bunch of things we refer to on it.
<lifeless> spm: we got rid of it, no more rm
<lifeless> spm: this was announced
<spm> i see
<lifeless> what are you looking for ?
<poolie> huwshimi: also i was quite curious about your thoughts on bug 800361
<_mup_> Bug #800361: The consequences of choose an item in a picker is not obvious <javascript> <ui> <Launchpad itself:Triaged by jcsackett> < https://launchpad.net/bugs/800361 >
<poolie>  - whether you agree with me; also whether you perhaps have some more alternatives
<huwshimi> poolie: Thanks, I'll take a look and comment
<poolie> ta
<wgrant> lifeless: Ah, so we *are* going for tiny DB patches now.
<wgrant> lifeless: I was discouraged last week from optimising below a couple of minutes :(
<lifeless> wgrant: yes, this has gone from discussion to action
<wallyworld_> wgrant: bug 809651 filed
<_mup_> Bug #809651: Person picker link text is confusing <confusing-ui> <picker> <Launchpad itself:Triaged> < https://launchpad.net/bugs/809651 >
<wgrant> wallyworld_: Ah, thanks!
<wallyworld_> wgrant: any opinion on what the default "Remove" text should be. I think it needs to be two words (I don't like just "Remove"). "Remove xxx" where xxx=?
<wallyworld_> maybe huwshimi has an opinion :-)
<huwshimi> wallyworld_: Is there a problem with "Assign me" and "Remove assignee"?
<wallyworld_> huwshimi: they only really make sense in the context of assigning to bugs. what about other times were a person is chosen eg security contact for a project
<wallyworld_> or perhaps that terminology is acceptable everywhere?
<huwshimi> wallyworld_: Ah I see
<wallyworld_> the default text for "assigning" is currently "Pick me"
<wallyworld_> but the case is wrong and needs changing, it is "Pick Me" and need sto change to "Pick me"
<huwshimi> wallyworld_: Are we able to customise for each case (team/person etc.)
<wallyworld_> not at the moment - that would require extra code
<wallyworld_> hence the current "Remove person" is wrong sometimes :-(
<huwshimi> wallyworld_: Hmmm... ok
<huwshimi> wallyworld_: What's wrong with extra code?
<wallyworld_> but if the desire is to have "Remove person" and "Remove team" depending on whether the current value is a person or team we could do that i'm sure
<wallyworld_> huwshimi: nothing wrong, just saying we don't currently support it in the current implementation
<wallyworld_> huwshimi: so "Remove person" and "Remove team" would be ok?
<huwshimi> wallyworld_: To me that makes the most sense
<huwshimi> wallyworld_: I would talk to mrevell as well though as this is his area
<wallyworld_> huwshimi: ok. i think it makes sense too. i'll look at doing something assuming we will go ahead. feel free to comment on the bug if you decide something after talking
<wallyworld_> oh, i can ask him too
<wallyworld_> s/too/instead
 * wallyworld_ hates qastaging timeouts occuring *every* time a link is clicked :-(
<lifeless> wallyworld_: means there is slow code there
<wallyworld_> lifeless: but even qastaging.lp.net itself just timedout
<lifeless> exactly my point
<wallyworld_> it happens all the time, for just about any page
<wallyworld_> surely *all* of lp pages are not slow
<wallyworld_> the same pages work fine on lp.net
<lifeless> 87 queries/external actions issued in 2.10 seconds - hot qa.l.n
<lifeless> actually, medium, 0.26 hot hot
<lifeless> still -slow-
<wallyworld_> i got this after initially timing out and hitting refresh: 98 queries/external actions issued in 8.25 seconds
<lifeless> yup, slow
<lifeless> -way- too many queries
<wallyworld_> but the figures are waaay less on lp.net
<lifeless> yes, it has a 128GB disk cache
<wallyworld_> not the query count, the execution time
<wallyworld_> so you saying the only reason lp.net even runs is the huge cache?
<lifeless> yup
<wallyworld_> :-(
<lifeless> wallyworld_: sorry, I was OTP before
<lifeless> wallyworld_: LP is broadly inefficient, we are getting better.
<lifeless> wallyworld_: an efficient frontpage would be 4 queries + a memcache lookup for the blog posts
<lifeless> wallyworld_: we are doing *twenty times* that work.
<lifeless> wallyworld_: qastaging has 32GB of ram shared between *three* LP db instances, each of which is 300GB in size
<StevenK> Are we are are we not still in RC?
<lifeless> we should be out of rc
<wgrant> I don't believe we are.
<StevenK> Can't stop using 'utilities/ec2 land', but to check on my instances I run 'bin/ec2 ls'
<StevenK> I'd drop the symlink, but I suspect I'd get lynched.
<lifeless> are there docs on creating lazr projects anywhere ?
* wgrant changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: - | Critical bugs: 220 - 0:[######=_]:256
<wgrant> lifeless: Why, so you can delete them? :)
<lifeless> fold-in :)
<wgrant> wallyworld_: Did you intend to delete a couple of dozen JS test?
<wallyworld_> wgrant: yes, they were duplicated
<lifeless> flacoste: huh, we missed some lazr components in the move to lp - smptptest, publisher, authentication amongst others, I think
<wallyworld_> they were moved and a rollback put them back to the original place
<wgrant> Ah, great.
<wgrant> lifeless: At least publisher and authentication were never used, IIRC.
<lifeless> by anyone?
<wgrant> And canonicalurl and a couple of others that I don't recall.
<wgrant> They were just placeholder projects.
<wallyworld_> lifeless: sounds like a major part of the perf issue on qas is that it's running 3 db instances in 1/4 the memory
<lifeless> grah, so we need to delete them
<wgrant> Oh, they have code now.
<lifeless> wallyworld_: its part of it yes; but our pages generally touch - way - more data than needed.
<wgrant> When did that happen...
<wallyworld_> lifeless: agreed. but still, stuffing 3 db instances into so little memory....
<lifeless> wallyworld_: indeed; things will get better once we can drop staging
<lifeless> will get us down to 2 instances
<StevenK> I am amused that 32GiB is 'so little memory'
<wallyworld_> epsecially since no-one could possibly need mnore than 640kb
<wgrant> lifeless: Ah, that's right. lazr.canonicalurl and lazr.publisher have branches, but they are just empty lazr templtes.
<lifeless> wallyworld_: for now, you need to try any page on qa staging at least 3 or 4 times
<wgrant> lifeless: lazr.authentication is real.
<wallyworld_> yes :-(
<lifeless> wallyworld_: but you *cannot* assume that timeouts on [qa]staging are not a problem for prod
<lifeless> wallyworld_: there was a thread on this a couple weeks back on -dev
<wallyworld_> sure, that lesson was mentioned at the epic :-)
<poolie> lifeless, +1 to emeritii
<wallyworld_> lifeless: thanks for making me open a dictionary
<lifeless> wallyworld_: de nada!
<wallyworld_> i was hoping (expecting?) that we would do such a thing
<lifeless> look in dictionaries ?
<wallyworld_> lp is hard enough as it is without all that combined knowledge walking out the door
<wallyworld_> lifeless: no, the team thing you emailed :-)
<lifeless> :P
<StevenK> wallyworld_: dict(1) ?
<wallyworld_> StevenK: i used google
<wallyworld_> :-P
<lifeless> https://dev.launchpad.net/CreatingNewProjects could use a proof read
<wgrant> Is there a workable YUI3 calendar yet?
<wgrant> This embedded copy of YUI2 irks me.
<lifeless> wgrant: ^ thats what I wanted existing docs for
<wgrant> Looking.
<wgrant> lifeless: Looks reasonable, although you don't seem to discourage namespace packages.
<lifeless> wgrant: I figure most folk know the terrible pain
<lifeless> wgrant: perhaps I should add some notes to that effect
<wgrant> lifeless: You figure incorrectly.
<wgrant> wallyworld_: How do I get a traceback from a YUI test failure?
<wallyworld_> wgrant: not easily afaik
<wallyworld_> i normally stick in a breakpoint just before
<wallyworld_> the failure and poke around from there
<wgrant> Does anybody know where SystemErrorView is tested?
<StevenK> It doesn't look to be tested.
<StevenK> From a quick bzr grep
<lifeless> stub: hi
<lifeless> stub: I found a new bug for the fastdowntime - dealing with landed but not deployed db patches
<lifeless> stub: I think we want to avoid causing a pipeline-halt on appserver deployments
<stub> lifeless: What is the problem? launchpad/devel & launchpad/db-devel split handles that doesn't it?
<stub> We rollout launchpad/devel everywhere except the databases. We rollout db-devel to the databases, and merge it into /devel after the rollout.
<lifeless> stub: thats probably a decent day one answer, letting the bug go to back burner
<lifeless> stub: It seems to me that eventually db-devel probably wants to be either 'big downtime' patches only, or to be dropped altogether
* mthaddon changed the topic of #launchpad-dev to: Launchpad down/read-only from 08:00-09:30 UTC for a code update | https://dev.launchpad.net/ | On call reviewer: - | Critical bugs: 220 - 0:[######=_]:256
<wgrant> spm: That's not leakage.
<wgrant> It's cache.
<lifeless> 18:31 < lifeless> stub: I think we want to avoid causing a pipeline-halt on appserver deployments
<lifeless> 18:39 < stub> lifeless: What is the problem? launchpad/devel & launchpad/db-devel split handles that doesn't it?
<lifeless> 18:40 < stub> We rollout launchpad/devel everywhere except the databases. We rollout db-devel to the databases, and merge it into /devel after the rollout.
<lifeless> 18:48 < lifeless> stub: thats probably a decent day one answer, letting the bug go to back burner
<lifeless> 18:48 < lifeless> stub: It seems to me that eventually db-devel probably wants to be either 'big downtime' patches only, or to be dropped altogether
<stub> Yup. Tackle the landing workflows as a separate problem.
<stub> We can avoid the split if upgrade.py is smarter, being able to apply just the live patches or the live + heavy patches, and having buildbots run the test suite against both schemas.
<lifeless> k, how about fastdowntime2 or something as a tag for 'later' items ?
<stub> Sure, although technically how we land stuff doesn't affect downtime at all. Its just a developer time saver.
<jtv> Won't this obfuscate code changes that depend on "big downtime" patches?
<lifeless> stub: well, given that that (dev efficiency) is ultimately why we're doing this :)
<lifeless> jtv: ideally there will be no big downtime patches
<lifeless> jtv: as of today they require signoff by me / flacoste
<stub> Yer, but a nice split means smaller solvable projects. No need to pile them together in a big project.
<lifeless> stub: agreed
<lifeless> stub: pick a tag name for the $next-piece ;)
<stub> landing-workflow
<stub> Or as it is probably a standalone bug, no need for a tag.
<lifeless> stub: I'd like to be able to find them later; its tightly related to the downtime thing - I was suggesting to remove the fastdowntime tag so its not in the burndown for the inital feature
<stub> Sure
<stub> Are these milestones?
<lifeless> stub: mmm, more we have a single large theme, and stages of work
<lifeless> stub: of which right now we know 'step 1' and 'not step 1' ;)
<stub> Two moving milestones - stuff to do now, stuff to do later :)
<lifeless> sure; though milestones are heavier. Lets stay with tags for now ?
<lifeless> I've given it fastdowntime-later
<lifeless> if we find all the later stuff is landing workflow, I'll take the bullet for updating the labels
* jtv changed the topic of #launchpad-dev to: Launchpad down/read-only from 08:00-09:30 UTC for a code update | https://dev.launchpad.net/ | On call reviewer: jtv | Critical bugs: 220 - 0:[######=_]:256
<adeuring> good morning
<lifeless> morning
<LPCIBot> Project devel build #882: FAILURE in 6 hr 0 min: https://lpci.wedontsleep.org/job/devel/882/
<LPCIBot> Project devel build #883: STILL FAILING in 0.8 sec: https://lpci.wedontsleep.org/job/devel/883/
<mrevell> Hello
<jtv> hi mrevell
<lifeless> bigjools: you'll appreciate this; just got the human stopwatch trophy :)
<bigjools> lifeless: it's easy :)
<lifeless> bigjools: I figured it would be hard - and wasn't trying for it :)
<bigjools> lifeless: karts FTW :)
<lifeless> heh, i was in ? daytona superspeedway? the big O course, in a formula one car
<bigjools> ah yeah that's the other easy way
<bigjools> lifeless: tune the F1 car so it's got almost no wing on that track
<lifeless> bigjools: yeah, I was topping out at 358kph, and maintained that on the corners
* mthaddon changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: jtv | Critical bugs: 220 - 0:[######=_]:256
<jtv> thanks mthaddon, and hi :)
<mthaddon> o/
<wgrant> henninge: Isn't IDiff['diffstat'] clearly wrong, making this a trivial fix?
<henninge> wgrant: what do you mean?
<wgrant> henninge: It's a dict, not text...
<wgrant> So IText is not right.
<henninge> wgrant: what I mean
<henninge> t
<henninge> wgrant: I was just wondering if the interface can be changed like that, the whole thing about api consistency.
<henninge> but obviously nobody has been using this as text or else it would have been noticed earlier.
<wgrant> Exactly.
<henninge> ok
<henninge> easy fix then, I just hope we don't hit any more of these.
<adeuring> jtv: still around?
<jtv> yes
<jtv> got an MP for me?
<jtv> I think you've earned a review, what with all the ones you do for me.  :)
<adeuring> jtv: yes, a small one: https://code.launchpad.net/~adeuring/lazr.batchnavigator/short-read-for-backwards-batch/+merge/67823
<jtv> Coming up.
* danilos changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: jtv, danilos | Critical bugs: 220 - 0:[######=_]:256
<abentley> henninge: I don't understand why the fact that IDiff.diffstat is a serialized dict breaks the marshaller.  A serialized dict is text...
<henninge> abentley: because it is not
<henninge> it is just a dict.
<henninge> See the implementation.
<abentley> henninge: Ah, it's wrongly-exported?
<henninge> well, it is stored as a json string in the database but the model implements an attribute that de/serializes it.
<henninge> so that implementation does not match the interface anymore, hence the wrong export.
<henninge> abentley: I have to run to the doctor's now. ttyl
<abentley> henninge: ttyl
<henninge> deryck: Hi!
<henninge> deryck: I am here but on 3G so no voice ... ;(
<deryck> henninge, no worries.
<deryck> abentley, http://developer.yahoo.com/yui/3/api/ArrayAssert.html#method_isEmpty
<sinzui> Once more we require a library to makeup for the deficiencies of the language.
<jml> sinzui: one measure of a language's strength is the degree to which that is possible.
<sinzui> granted
<deryck> henninge, I've seen your mail and the card move.... looks like the obfuscated email work is on track to land now?
<bac> nice blog post mrevell!
<mrevell> bac, Thanks. I thought it was about time to announce :)
<jcsackett> sinzui: you available to chat?
<sinzui> yes. I will start mumble?
<jcsackett> sinzui: sure.
* abentley changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: jtv, danilos, abentley | Critical bugs: 220 - 0:[######=_]:256
<abentley> jtv: still OCR?
<jtv> yes, abentley, still OCR!
<jtv> adeuring: not getting a diff for your branch at all.  :(  I'll go for manual.
<abentley> jtv: I guess you're not on Thailand time right now?
<jtv> No.
<adeuring> jtv: yeah, seems that LP is a bit slow today....
<jtv> Oh, it may have got caught in the release of course.
 * deryck is switching offices, back online in 20 minutes
<abentley> adeuring: no, it seems that something's wrong with the script that does those diffs.
<jtv> adeuring: oh well, I just looked at the diff.  Done.
<adeuring>  jtv: cool, thanks
<sinzui> mrevell, sorry :( I spent my morning getting my computer to work. I can talk now if you like
<jtv> bigjools: I noticed that process-accepted commits its changes, in its inner loop, even in a dry run.  Accident?
<jam> is it supposed to be downtime right now?
<jam> I have a merge-proposal page that hasn't updated for 45 minutes +, and I tried to link a branch to a bug and it timed out after 9s.
<jtv> jam: problem with the MP diff script, apparently.
<jtv> Althoughâ¦ I just got a timeout as well.
<jam> jtv: the last status updates I see are "degraded performance" followed by "performance should be back to normal"
<jam> (from 45min ago)
<jam> I don't know if what I'm seeing is just long-term fallout
<jam> or whether something is still broken
<bigjools> jtv: there's an outer abort though right?
<jtv> bigjools: yes, but that won't help much if the inner loop's already committed.  :)
<jam> I just succeeded in linking the branch, but it took 4.27s
<bigjools> jtv: doesn't the partial commit get aborted?
<jtv> There is no partial commit â not without 2-phase, which we don't use.
<jtv> Commit is commit.
<bigjools> jtv: :/
<bigjools> then it's borked
<jtv> At least it's good to see my design decisions for libpqxx justified.
<bigjools> jtv: although we don't use dry run for anything
<jtv> Well, it's fixed now.  Up for review.
<bigjools> cool
<jtv> (I needed some cleanups before I could confidently mess with the script)
<bigjools> I hate it when I see bugs in a security adapter
<sumanah> bac, flacoste -- hi, Sumana here asking whether a blog entry about Launchpad's code review process is in the works.  flacoste I think you said you & Brad are working on it?
<sumanah> (or rather that bac  is working on it)
<mrevell> sinzui, Does 15 minutes from now suit you?
<bac> sumanah: i have it on my todo list.  :(
<sinzui> mrevell, yes
<bac> sumanah: i will try to set aside time on friday to do it.
<sumanah> bac: if I interviewed you, would that help?
<bac> sumanah: i'd like to talk to you some time!
<sumanah> thanks bac -- we in #mediawiki are looking forward to hearing about your example before a code review training on Tuesday
<sumanah> bac, Francis's email was very useful but graphs would be even better :)
<bac> sumanah: maybe we can talk tomorrow?
<sumanah> bac: ok!  and I'd also love any documents about how Canonical people train each other in code review -- if you ask reviewers to write down any lessons learned about how to teach/learn how to code review, I would be very enthusiastic to read them.
<gary_poster> flacoste, for bug 777874, can we regard that policy ("If a new bug, has >2 duplicates associated with it, or >2 people have indicated that this bug affects them, then automatically mark it confirmed.") as vetted and approved for all Launchpad users, or should addressing the bug include trying to confirm whether it is appropriate, or considering how to make it configurable/choosable?
<_mup_> Bug #777874: If multiple reports on new bug, mark it confirmed <bug-lifecycle> <bugs> <escalated> <ubuntu-platform> <Launchpad itself:Triaged> <Ubuntu:Triaged> < https://launchpad.net/bugs/777874 >
<gary_poster> all Launchpad *projects
<sinzui> matsubara, bug 803996 and bug 283167
<_mup_> Bug #803996: person picker widget doesn't show link to unassign myself <disclosure> <exploratory-testing> <person-picker> <qa-ok> <Launchpad itself:Fix Committed by wallyworld> < https://launchpad.net/bugs/803996 >
<_mup_> Bug #283167: Private branch owner cannot unsubscribe someone <disclosure> <lp-code> <oem-services> <Launchpad itself:Triaged by wallyworld> <NULL Project:Invalid> < https://launchpad.net/bugs/283167 >
<LPCIBot> Yippie, build fixed!
<LPCIBot> Project devel build #884: FIXED in 5 hr 29 min: https://lpci.wedontsleep.org/job/devel/884/
<gary_poster> mrevell, I saddled you with https://answers.launchpad.net/launchpad/+question/164474 .  If you'd rather me kick it somewhere else, let me know (and a suggestion would be lovely :-) )
 * gary_poster mixes metaphors blithely
* jtv changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: danilos, abentley | Critical bugs: 220 - 0:[######=_]:256
<jtv> abentley, while we're on the subject, care to review one of mine?  https://code.launchpad.net/~jtv/launchpad/bug-676103/+merge/67852
<mrevell> gary_poster, :) I'll take a look now.
<abentley> jtv: sure.
<jtv> thanks.
<flacoste> gary_poster: consider it vetted and approved, but you might want to feature flag it ;-)
<flacoste> gary_poster: Ubuntu wants it for sure
<flacoste> we might need some more scope controller: project:name
<nigelb> I get search warnings, if I ignore them, I can't see the failures :(
<nigelb> err, sorry
<nigelb> wrong channel
<gary_poster> mrevell, thanks :-)
<gary_poster> flacoste, ack, makes sense, thanks
<mrevell> np
<jtv> abentley: I need to go somewhere.  I expect it'll be an hour or two.
<flacoste> danilo_: you win the best commit message of the year award!
<nigelb> flacoste: linky? :)
<danilos> flacoste, heh, thanks :)
<timrc> Throwing this out here, from #launchpad.  It would be nice if, for a PPA, we could specify a URL to a downstream archive.  This archive would be consulted whenever a package is uploaded to the PPA or copy/rebuilt.  If the package with that name and version already exist downstream, the upload / copy and rebuild is rejected.  The reason is to prevent package collisions before they propagate.
<timrc> This may be a new way of thinking of a PPA in terms of how they were originally purposed, but, would like to get some thoughts anyway
<bigjools> timrc: you need Derived Distros
<bigjools> talking of which I need to catch up with you guys about that
<timrc> bigjools, yes and yes
<bigjools> timrc: I want to know how attached to your crazy custom suite names you are and whether we can work around that
<timrc> bigjools, I think what we need to do is fully understand derived distrobutions today and then develop a migration plan identifying the gaps
<timrc> distributions, too
<bigjools> timrc: right. I want to see if I can help minimise gaps by suggesting different practices for you if I think you can do better
<bigjools> timrc: we can enable DDs on staging so you can blast away, there's a couple of bugs to fix first though.
<flacoste> nigelb: http://bazaar.launchpad.net/~launchpad-pqm/launchpad/devel/revision/13419
<timrc> bigjools, I would love to push all our problems on to you, so I'm definitely supportive of moving towards a launchpad solution in that regard :)
<bigjools> timrc: I don't want your problems which is why I want to help now, rather than later :)
<timrc> bigjools, even if we could work around naming and what not, I think our biggest issues are security and responsiveness... we have to be able to react very fast if shit-hits-the-fan for a customer, so having this aspect of our commercial operations managed by launchpad adds another dimension (but that's something for you and smagoun to discuss))
<timrc> so the LEP on derived distributions did not cover privacy and last I heard there was a privacy rewrite going on?
<timrc> both of which make, at least me, nervous
<nigelb> flacoste: heh, I agree :-)
<flacoste> timrc: private distributions (and derived) is on the roadmap (~3-4 months)
<timrc> flacoste, completed and in production in 3-4 months?
<timrc> I have a date with some chicken wings, bbiab
<bigjools> flacoste: we are still having massive performance issues in production
<flacoste> bigjools: yes, i've hit a few timeouts
<cr3> hi folks, anyone have a moment for a class design question?
<abentley> cr3: sure.
<cr3> abentley: awesome! so, lets say I'm adapting Product with FooProduct which delegates(IProduct, "product"), which works well so far. now, the design problem is what if I want methods like newSeries and getSeries in FooProduct to return an adapted ProductSeries, eg FooProductSeries? might there be an alternative to overriding each of those methods explicitly in FooProduct and adapting each return value, eg def newSeries(self, name): return IFooProductSeries(se
<bac> abentley: i'm working on the JSON cache branch again.
<abentley> bac: cool.
<bac> abentley: did you open any bugs for tracking this work?
<abentley> cr3: I don't think there's an alternative to explicit overrides in that case.
<cr3> abentley: darn, where's my pattern book when I need it the most :)
<abentley> cr3: delegation is not as cool as true inheritance, because the delegated class does not have access to methods on the delegating class.
<abentley> cr3, i.e. Product.newSeries will not have access to FooProduct.asdf
<abentley> cr3: so you can't use the "template method".
<abentley> bac: I didn't open any bugs for this work.
<bac> thanks abentley
<abentley> cr3: I imagine it is technically possible if you override __getattr__() or extend delegates(), to decorate methods to adjust their return types, but it would be a pretty nasty hack.
<cr3> abentley: yeah, I'm looking more for a pattern to do this nicely otherwise suckup repetitive code rather than hack it
* danilos changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: abentley | Critical bugs: 220 - 0:[######=_]:256
<cr3> abentley: even if the pattern might be slightly overkill, my example only mentionned two methods but I actually have more which would benefit from a good pattern
<cr3> abentley: if Product.newSeries did something like getUtility(IProductSeries).create(self, name), then I was thinking that in my context I could potentially subscribe my own adapter for that interface, but then I think I puked a bit in my mouth :(
<abentley> cr3: if you were using inheritance, not delegation, you could use template method http://en.wikipedia.org/wiki/Template_method_pattern
<cr3> err, s/IProductSeries/IProductSeriesSet/
<abentley> cr3: You could add a member of Product that is a class used to retrieve the ProductSeriesSet, and then override it on your instance.  But that is gross, because Product is an ORM object.
<abentley> cr3: s/ProductSeriesSet/ProductSeries/
<cr3> abentley: I'm not sure I follow, since I use adaption rather than inheritance, it wouldn't make a difference if I override that ProductSeries class in my adapted class because, if I understand correctly, Product.newSeries would do something like self.product_series_class(*args, **kwargs)... but self would be Product, not FooProduct
<cr3> abentley: if I misunderstood, I'd appreciate clarification because, even if it might be gross, it might be worth considering :)
<abentley> cr3: In FooProduct.__init__ or similar, you'd do self.product.product_series_class = FooProductSeries.
<cr3> abentley: ah, gotcha, that would work
<cr3> abentley: is there any precedent of something like that somewhere in Launchpad where it might've made sense to do so?
<abentley> cr3: it's gross because Product is cached by Storm, so that Product with its custom product_series_class could show up somewhere else, where it wasn't expected to have that override.
<cr3> abentley: I was thinking the other way around, if it was once a FooProduct, it might also make sense for it to remain as such. since it delegates IProduct, it should still behave the same as far as callers are concerned
<abentley> cr3: No, we don't usually try to do polymorphism via delegation, so I don't know of anywhere we're doing this.
<cr3> abentley: but I can appreciate how that could be a nasty side effect :(
<abentley> cr3: re "I was thinking the other way around", the delegated-to Product was never a FooProduct, so it shouldn't remain as such.
<cr3> abentley: touche! :)
<cr3> does storm cache store.find or only store.get?
<abentley> cr3: I have no idea, but... 1) Something may *already* be using that instance 2) Something in the FooProduct.* call tree may acquire that instance.
<abentley> cr3: Can we take a step back and question the assumption of adaptation?  What's the adaptation trying to solve?
<jtv> thanks abentley â back now but glad to see my absence hasn't held things up.
<abentley> jtv: np
<cr3> abentley: it's trying to solve extending product with stats information with a single query, so FooProduct takes a product, number_of_test_runs, first_test_run_date, last_test_run_date, etc.. I have FooProductSet.create/get/search which take care of returning a FooProduct where the decorated result or resultset returns a product with all the other stats information already retrieved
<cr3> abentley: ideally, I don't want to have to pollute the registry product with that information, it shouldn't have to know about these stats
<cr3> abentley: so, the reason why these bleeds into ProductSeries is that this class also has an adapted class providing finer grained stats just for the series
<abentley> cr3: In general, we *do* pollute classes.  If you want to avoid that, perhaps a mechanism that lets you look up the stats for a given Product or ProductSeries would work.
<cr3> abentley: is it just me or did QuestionsPerson originally adapt person? I see how that it pollutes Person with QuestionsPersonMixin now :(
<cr3> I'd love to understand why...
<abentley> cr3: After all this difficulty with other options, the reason why seems clear to me :-)
 * cr3 sheds a tear for adaption :(
<lifeless> naive adaption will perform poorly in an ORM environment
<lifeless> you need fairly deep integration to avoid death by a thousand queries scenarios
<abentley> lifeless: this adaptation is being done to avoid additional queries.
<cr3> abentley: my concern about Mixins is that high level classes (like project) endup having to know about low level classes (like project series), so a person for example ends up becoming all knowing :(
<abentley> cr3: I agree with you.
<cr3> abentley: they're alright within a component, like classes within lp.answers could mixin among themselves as much as they like, but not across components
<lifeless> an alternative is to have an explicit mapping phase, and that can move away from using the ORM obtained object
<lifeless> e.g. truely map into plain old python
<cr3> lifeless: is that what you mentionned a few months ago to decouple launchpad from its database backend which would have the nice side effect of being to run tests without the database?
<lifeless> yes
<cr3> lifeless: I'm very excited about that project, especially to see the design at work, any plans to have it implemented?
<cr3> lifeless: actually, is there a LEP I could keep track of?
<lifeless> cr3: probably never going to do it - splitting LP into small services will take a year or two and after that we can reconsider
<cr3> lifeless: I'm patient :)
<benji> flacoste: I've replied to your review of https://code.launchpad.net/~benji/launchpad/bug-781600/+merge/67704 and made several changes.
<flacoste> benji: ack, will look at it shortly
<benji> k
<lifeless> cr3: why is checkbox making an OPTIONS call ?
<lifeless> actually, its someone in a browser hitting api pages.
<lifeless> we should probably block that on the api domain
<cr3> lifeless: I'm not sure I follow where this call is originating from...
<lifeless> cr3: someone is poking at the api with a browser, on the checkbox project.
<lifeless> cr3: and triggering an oops
<cr3> lifeless: weird, I wonder if that might be the results tracker integration. can I get more information?
<lifeless> https://bugs.launchpad.net/launchpad/+bug/810113
<_mup_> Bug #810113: TypeError constructing page id: cannot concatenate 'str' and 'list' objects <oops> <Launchpad itself:Triaged> < https://launchpad.net/bugs/810113 >
<jcsackett> sinzui: i will be at the meeting tonight.
<sinzui> fab
<lifeless> flacoste: do you want to look at lp.net/lazr and sort it out then ?
<cr3> lifeless: that oops doesn't seem to be related to the results tracker and I looked at some of the scripts used by my team to generate bug coverage reports, likely candidates to generates oopses against checkbox, but they all seem to use lpltk which uses launchpadlib
<cr3> lifeless: oh wait, was that oops generated a couple days ago?
<cr3> I was playing with the javascript client to make reports on one website taken from launchpad, but that was just exploratory
<lifeless> probably that
<flacoste> benji: on what ground was bug 810004 marked critical?
<_mup_> Bug #810004: +bugs page layout for bzr-hg is messed up <Launchpad itself:Triaged> < https://launchpad.net/bugs/810004 >
<benji> flacoste: I thought it was a regression before I realized the true cause (the very long bug title) and I forgot to change it.  Fixed now.
<wgrant> lifeless: :( why don't we fix OOPSes rather than blocking it?
<lifeless> wgrant: well we should as well
<lifeless> flacoste: 09:09 < lifeless> flacoste: do you want to look at lp.net/lazr and sort it out then ?
<flacoste> benji: ok, if had been a regression, please tag it appropriately
<flacoste> benji: review comments sent btw
<flacoste> and out...
<benji> flacoste: thanks
* abentley changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: - | Critical bugs: 220 - 0:[######=_]:256
<wallyworld_> wgrant: jcsackett: StevenK: sinzui: http://people.canonical.com/~ianb/picker-search-suggestion-mockup.png http://people.canonical.com/~ianb/picker-search-suggestion-mockup2.png
<StevenK> sinzui: http://pastebin.ubuntu.com/643600/
#launchpad-dev 2011-07-14
<lifeless> poolie_: whats the link you added to creatingnewprojects for ?
<poolie_> lifeless: i think people will google 'create launchpad project' end up there and be confused
<lifeless> ok, I think it needs refinement though
<poolie_> sure
<poolie_> just wanted to give people a chance to understand it is not rules about launchpad as a whole
<poolie_> typical quoting problem
* wgrant changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: - | Critical bugs: 232 - 0:[######=_]:256
<wgrant> Where did the 15 new bugs come from? :/
<wgrant> Erm
<wgrant> http://blog.launchpad.net/cool-new-stuff/better-bug-subscriptions was published yesterday, but it shows the UI from when the feature was released, which is completely different from the current one :/
<wgrant> Should we unpublish until it's fixed?
<wgrant> Oh, it's at least not frontpaged.
<lifeless> eyeballs requested: https://dev.launchpad.net/PolicyAndProcess/DatabaseSchemaChangesProcess
<wgrant> lifeless: Have we analysed recent patches to see how many of them are doable without simultaenous changes?
<lifeless> wgrant: not rigorously, but we haven't been trying to do it either, so what we've done isn't a baseline
<lifeless> wgrant: we know that all the ha services out there manage
<wgrant> lifeless: I think we should before we go much further, so we don't get ourselves into trouble.
<wgrant> Since we're throwing away read-only (which I still find fairly unwise)
<wallyworld_> lifeless: about your document ^^^: in the Deploying Patches section, "After successful QA on a patch, ...". I take it that is for hot patches. Cold patches are deployed once a month during out downtime deployment. Shoudl that text say "After QA on a hot patch, ..."?
<lifeless> wallyworld_: not any more
<lifeless> wallyworld_: you might want to read yesterdays performance tuesday mail ;)
<StevenK> wgrant: Do you think the removal of the bazaar-experts celebrity is QA-able?
<wallyworld_> lifeless: i read it but it clearly has gone in one ear and straight out the other :-(
<wgrant> StevenK: Check that you have branch access, possibly check that LOSAs do.
<StevenK> wgrant: I can change the details of an owned branch, good enough?
<wgrant> StevenK: Indeed.
<lifeless> wallyworld_: there are now no scheduled monthly patch windows
<lifeless> wallyworld_: over the next 4 weeks we'll be bringing up the process for doing short downtime multiple times a week
<lifeless> wallyworld_: (short => less than 300 seconds)
<lifeless> wallyworld_: and all patches landing from here on in need to be compatible with that process
<wallyworld_> lifeless: ah, right. thanks. for that 300 seconds, lp will be in ro mode?
<wallyworld_> this will be a fantasic change
<lifeless> no, just db connections refused
<lifeless> going into readonly mode and out again takes an hour
<wgrant> lifeless: In the current implementation.
<wgrant> There is nothing that requires that.l
<lifeless> wgrant: sure, and if someone wants to work on that they can
<wgrant> Even in the current model, where we have to rebuild the slave because slony is crap.
<wgrant> Touch file, detach slave, upgrade, remove file, rebuild slave.
<wgrant> No slower than blocking connections.
<lifeless> so no
<lifeless> we're not detaching slaves or rebuilding them.
<wgrant> Why not?
<lifeless> thats a cause of about 90% of our downtime delays.
<wgrant> We can easily do this without downtime...
<lifeless> I don't object to a great implementation of readonly mode. But readonly mode must not make the downtime longer or riskier.
<lifeless> Currently it does.
<wallyworld_> if db connections are refused for 5 minutes, the users will see page timeouts?
<lifeless> And its not at all clear to me that that can be fixed on slony.
<wgrant> Can we tell SSO to GTFO of our DB and move to a sensible replication strategy?
<lifeless> wgrant: not in the same timeframe as this project ;)
<lifeless> wallyworld_: they will see a horrible mess in the very first cut, but we'll iterate.
<wallyworld_> cool, just checking :-)
<wgrant> lifeless: "a horrible mess" meaning a maintenance page?
<lifeless> like, the appservers can show a fail-UFO page when they can't connect to the DB eventually
 * wallyworld_ was also wondering that
<wgrant> Unlike slony, reloading apache isn't risky.
<lifeless> wgrant: according to losas it is.
<wgrant> They are wrong :)
<lifeless> wgrant: they are the ones dealing with our apaches not restarting cleanly today.
<lifeless> wgrant: so, I beg to differ.
<wgrant> We have had a lot of rollout problems, and that is not one of them that I've ever heard of.
<wgrant> So we have rollout problems that are hidden from us? Yay.
<lifeless> wgrant: we don't restart apache during our rollouts.
<spm> minor correction. we graceful apache on crowberry.
<lifeless> ah, one.
<wgrant> We can't just leave the appservers silently broken for 5 minutes.
<spm> branch rewriter thingybobby
<lifeless> wgrant: 5 minutes is not the target; its the absolute outer limit
<wgrant> Perfect is the enemy of good, but terrible is the enemy of not looking like we are useless.
<lifeless> wgrant: the target is 10-20 seconds
<lifeless> wgrant: look, if you want to jump in and make the error condition look nice, that would be awesome.
<lifeless> wgrant: I was clear about what I was proposing on -stakeholders
<elmo> wait
<elmo> what?
<elmo> what's the problem with reloading apache?
<wgrant> That's what I said.
<StevenK> How do I get (and reset) the OOPS count in a test? self.getOopsCount() or so?
<elmo> re*starting* apache can be problematic
<elmo> but reloading is just fine
<elmo> we do it *all the time*
<wgrant> If we can't reload Apache, we have pretty serious problems.
<lifeless> elmo: spm was telling me a month or so back that our ones don't go cleanly occasionally
<lifeless> elmo: I didn't get details
<elmo> lifeless: restart yes - reload no
<elmo> and we wouldn't need a restart to put a maintenance page in place
<lifeless> ok, good to know. Can a script running on wildcherry drive such a change ?
<wgrant> So I may not be insane. This is good.
<elmo> lifeless: it could, sure
<lifeless> ok, cool.
<spm> lifeless: I suspect you're mixing problems there. we have a known issue where a reload won't clear a certain problem; necessitating a restart. but that's a different beastie.
<lifeless> spm: it wasn't, but its beside the point - I'm entirely happy for us to do something you guys consider low-risk-of-fail
<lifeless> spm: my criteria are: the core script must not block on niceties (because once downtime starts, users are indisposed regardless); and the script must not be able to be made unreliable due to non-core-tasks.
<spm> nod
<StevenK> wgrant: O hai, can haz review?
<wgrant> StevenK: Which?
<StevenK> wgrant: https://code.launchpad.net/~stevenk/launchpad/daily-build-oops/+merge/67915
<StevenK> wgrant: Danke
<poolie> spiv, your patch failed again the same way
<poolie> has anyone else seen "ImportError: No module named html5browser" inside importfascist?
<poolie> ah i see the mail
<poolie> nm
<poolie> spiv i think i can fix it
<spiv> Ah good :)
<StevenK> lifeless: O hai, Mr OCR. Can haz review?
<lifeless> of whut
<wgrant> StevenK: https://code.launchpad.net/~wgrant/launchpad/bug-419534/+merge/67918
<StevenK> lifeless: Sorry. https://code.launchpad.net/~stevenk/launchpad/uploaded-packages-wrong-link/+merge/67916
<StevenK> wgrant: r=me
<lifeless> stub: hiya
<lifeless> stub: after this cutover, can we do our weekly catchup ?
<bigjools> morning
<lifeless> o/~
<adeuring> good morning
<stub> lifeless: sure
<mrevell> Hallo
<jtv> Anyone mind if I update dogfood?  allenap, bigjools, StevenK, wgrant?
<bigjools> jfdi
 * jtv jfdis
<StevenK> Any one up for reviewing https://code.launchpad.net/~stevenk/launchpad/queue-no-changes ?
<StevenK> Bah, branch URL.
<StevenK> https://code.launchpad.net/~stevenk/launchpad/queue-no-changes/+merge/67913
<bigjools> StevenK: where's the bug for that?
<StevenK> bigjools: I don't think there is one.
<bigjools> StevenK: then what are you fixing?
<StevenK> bigjools: Happy to file a High bug for it.
<bigjools> just want to know more about it, 'tis all
<StevenK> bigjools: Right. When I was working with the LOSAs to reject an ACCEPTED binary upload on germanium, the queue tool didn't work. That branch should sort it.
<bigjools> ah ok
<bigjools> file bug :)
<bigjools> release the hounds - /me just made a branch that allows people to use the API to request uploads to ubuntu by copying packages from PPAs.
<LPCIBot> Project devel build #888: FAILURE in 2 hr 5 min: https://lpci.wedontsleep.org/job/devel/888/
* allenap changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: allenap | Critical bugs: 232 - 0:[######=_]:256
<bigjools> allenap: incoming branch from me if you don't mind?
<allenap> bigjools: Cool.
<bigjools> tar
<StevenK> bigjools: Can you have another look at my MP?
<StevenK> 2 hours?!
<bigjools> yarp
 * StevenK peers at Jenkins
<henninge-lunch> allenap: Hi! If you could have a look at my branch while I am away, that would be great.
<henninge-lunch> https://code.launchpad.net/~henninge/launchpad/bug-740208-ws-regression/+merge/67937
<StevenK> Sigh, the slave died
<bigjools> StevenK: that's an odd test
<StevenK> bigjools: It tests that exact code path
<StevenK> Which is what you wanted
<bigjools> StevenK: by inspecting the logger?
<bigjools> shouldn't it check no email is sent?
<StevenK> How else do I determine it did *nothing*? :-)
<StevenK> It doesn't even get that far, it bails out extremly early
<bigjools> so checking that no email is sent will be fine, right?
<wgrant> StevenK: So it's going to reject it silently?
<wgrant> That's bad.
<StevenK> wgrant: There's no changes file and no blamer
<bigjools> if there's no changes file what else can be done?
<wgrant> Ah, true.
<wgrant> Delayed copies must die.
<bigjools> nearly there!
<bigjools> my branch today does some of the PCJ attachment to API copies
 * StevenK prods bigjools :-P
<bigjools> StevenK: fix the test to check for no email, examining the debug log is gross
<bigjools> that is the intention of the change, right?  To not blow up by attempting to send email with corrupt data (no PU)
<StevenK> bigjools: Done.
<bigjools> StevenK: I approved you already
<StevenK> bigjools: Danke
<bigjools> bitte
<StevenK> Bring on longpoll
<jml> StevenK: how long do we have to wait
<jml> it's been two weeks since I saw a working demo
<bigjools> blame lifeless, he wants all this deployment gubbins done :)
<StevenK> I'm so tempted to reply with 'Oh noes'
<jml> bigjools: what sort of deployment gubbins?
<bigjools> it will be the first time we've done something with the new SOA, so the longpoll server needs its own deployment story.  We also need better metrics on it and Rabbit
<bigjools> we'll probably look at it agan when DDs are done.
<bigjools> and when I say "better", I mean "some" :)
<jml> bigjools: quite a lot of stuff then
<jml> bigjools: it'd be nice to be able to see how an external contributor could help.
<bigjools> jml: the best place would be to help us get metrics since the rest of it's internal
<jml> bigjools: got a list of the things you need & the format you need them in?
<bigjools> that's a better question for lifeless
<bigjools> let me dig up some bugs though
<bigjools> hmm no bugs
<lifeless> jml: the bugs are all linked from the LEP
<bigjools> there we go
<jml> lifeless: ok, thanks.
<LPCIBot> Project db-devel build #720: FAILURE in 5 hr 13 min: https://lpci.wedontsleep.org/job/db-devel/720/
<cjwatson> is https://dev.launchpad.net/PolicyAndProcess/DatabaseSchemaChangesProcess active as of now?  should I be refactoring my multiarch-translations branch to separate out the schema patch?
<lifeless> cjwatson: yes
<wgrant> lifeless says yes, but I doubt it.
<wgrant> But for yours it can't hurt.
<lifeless> wgrant: you also doubted making LP faster ;)
<wgrant> lifeless: Yeah, but that didn't go from 0 to implemented in three weeks :)
<lifeless> wgrant: indeed, it was faster :P
<lifeless> wgrant: I want a consult with you wearing your tz-friendly soyuz-internals-specialist about the buildmaster + db resets tomorrow, if you have time
<wgrant> lifeless: Ewww, but maybe.
<cjwatson> wgrant: seeing as I don't really want to wait for Aug 12 :-)
<lifeless> cjwatson: oh, for clarity
<lifeless> cjwatson: we need *patches* to abide by that now.
<lifeless> cjwatson: we're not live on the *deployment* side of it yet, but we're aiming to be doing that rather than a 90 minute downtime in august
<bigjools> lifeless: FWIW I don't think the buildmaster has any issues, when I was writing it I often killed it mid-cycle and there's lots of code to recover from Bad Stuff.
<lifeless> bigjools: thats a great comfort
<lifeless> bigjools: did you try taking the db out from under it ?
<wgrant> bigjools: There are a couple of consistency issues that are just about impossible to track down.
<wgrant> But they are rare.
<lifeless> bigjools: I figure we'll need some reconnection glue
<bigjools> lifeless: I didn't test that.
<cjwatson> OK; if, for the sake of argument, I posted the db part of that patch separately, might I expect it to be deployed before Aug 12?
<bigjools> all our daemons will need reconnection glue I expect
<lifeless> cjwatson: expect no. Pleasantly surprised, I hope so.
<cjwatson> (and how are you tracking dependencies between the separated-out db-devel and devel targeted branches?)
<lifeless> bigjools: https://bugs.launchpad.net/launchpad/+bugs?field.tag=fastdowntime
<bigjools> yes :)
<lifeless> cjwatson: for your branch, you'll need to land both components on db-devel (because we're not live on the incremental deploy steps yet)
<lifeless> cjwatson: but you need to land them in separate landings so we know the schema change doesn't break existing code.
<cjwatson> ok, makes sense
<bigjools> I think I am in love with bzr switch
<bigjools> how can I break this to my wife
<lifeless> bigjools: offer her preferred-courtesan status
<lifeless> bigjools: 'same job, better perks'
<bigjools> "courÂ·teÂ·san (kÃ´r t -z n, k r -). n. A woman prostitute"
<bigjools> she'll love that
<nigelb> We desperately need an Ubuntu quotes db.
<nigelb> :)
<bigjools> perhaps some bzr-backed wiki?  *cough*
<StevenK> nigelb: No, we *don't*.
<nigelb> StevenK: heh
<nigelb> StevenK: Why not? :-)
<StevenK> Because quotes pages are dangerous.
<StevenK> And productivity sinks. Ubuntu has enough of those. :-)
<nigelb> Hah
<benji> StevenK: that should go on the quotes page
* jcsackett changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: allenap, jcsackett | Critical bugs: 232 - 0:[######=_]:256
<jcsackett> henninge: ping.
<henninge> jcsackett: Hi! ;)
<jcsackett> henninge: hi, i'm looking at https://code.launchpad.net/~henninge/launchpad/bug-740208-ws-regression/+merge/67937, and had one question.
<henninge> jcsackett: ask away
<jcsackett> is this just you addressing a wart in the code you found, or was this Text/Diff thing responsible for an issue found in qa?
<henninge> It was causing an oops
<jcsackett> henninge: ok, then is it possible to create a test showing the oops doesn't happen anymore as part of this branch, to prove it's the fix?
<henninge> jcsackett: Sorry, I just realized I forgot to mention more context.
<jcsackett> henninge: no worries. just leads to me bugging you on IRC. :-)
<henninge> jcsackett: I could try but it is a test for a corner case.
<jcsackett> henninge: how "corner" are we talking? major pain to set up the testcase?
<henninge> jcsackett: I am not sure. There are tests that don't trigger the error so I think something is still missing to create the same situation as in production.
<henninge> jcsackett: But I am happy to give it a try.
<henninge> jcsackett: did you claim the review? deryck had offered to review it. Just to avoid double work.
<jcsackett> henninge: i was just looking it over. i claimed it, but can abstain in a comment and assign it to deryck, if he has more context to look at it with. :-)
 * deryck doesn't mind jcsackett taking it, but is happy to do it, too
<jcsackett> deryck, henninge: I'm OCR today, so i may as well finish it up. :-)
<deryck> jcsackett, henninge -- works for me. :)
<henninge> jcsackett: go ahead ;)
<deryck> Fresh eyes are nice as well.
<henninge> yup
<flacoste> bigjools: i need to QA https://bugs.launchpad.net/bugs/805634
<flacoste> bigjools: what do you suggest? I ask a losa to run populate-archive on qastaging and check the builds priority?
<bigjools> flacoste: yes, that should work
<bigjools> flacoste: or staging
<flacoste> bigjools: any easy way to look at an archives builds from the UI? or should I use API, or poke the DB?
<bigjools> flacoste: yes it'll appear in the UI
<bigjools> flacoste: <distro>/+archives
<bigjools> and you can examine the builds
<abentley> deryck: chat?
<jcsackett> henninge: given the corner-case issue, i think this looks fine. esp as the actual diff is fairly trivial. r=me.
<henninge> jcsackett: thank you! ;)
<deryck> abentley, sure.  let me fire up mumble.
<flacoste> danilos: can you update the description of bug 775691 to clearly describe the low-priority corner case that it is? please
<flacoste> gmb: i think i have the solution to your problem, and it's netierh #1, #2, or #3
<flacoste> gmb: i'm confirming my hunch and writing a reply
<gmb> flacoste: Cool, thanks.
<abentley> deryck: is there a way of testing whether an object is an lp.client.Entry?
<abentley> deryck: I would have thought you could do prototype === prototype, but I get false positives.
<deryck> abentley, instanceof will work.
<deryck> abentley, x instanceof y
<deryck> I don't know if yui 3 has some wrapper around this to make it nicer.
<deryck> abentley, so a docs search shows me Assert.isInstanceOf and Y.instanceOf
<abentley> deryck: cool.
<flacoste> gmb: hunch failed... writing reply :-(
<gmb> flacoste: Boo. Oh well.
<jkakar> I'm regularly getting logged out of Launchpad... is that a temporary issue or was some change made to make this happen?
<gary_poster> jkakar, temporary.  dbs were flipped around.
<gary_poster> danilos, jtv, neither of you are here, but if you had a better idea on triaging https://bugs.launchpad.net/launchpad/+bug/810309 that would be great
<_mup_> Bug #810309: Translator credits appear duplicated <Launchpad itself:Triaged> < https://launchpad.net/bugs/810309 >
<jtv> gary_poster: I am here.
<gary_poster> jtv, why for goodness sake? :-)
<jtv> gary_poster: I'm in Europe, where it's still more or less working-day hours.
<gary_poster> oh, ok, didn't know jtv
<jtv> This bug looks like some kind of nasty loop behaviour between imports and exports.
<jkakar> gary_poster: Cool, thanks.
<jtv> oh, hi jkakar!
<jkakar> jtv: Hi! :)
* allenap changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: jcsackett | Critical bugs: 232 - 0:[######=_]:256
 * deryck goes offline for lunch, back online in an hour
 * bigjools waves at jcsackett
<jcsackett> heya, bigjools.
<bigjools> howdy jcsackett.  May I push a review in your direction please?
<jcsackett> certainly.
<bigjools> https://code.launchpad.net/~julian-edwards/launchpad/async-copying-part-2/+merge/67989
<bigjools> thanks
<jcsackett> bigjools: r=me.
<bigjools> jcsackett: thanking you sir
<LPCIBot> Yippie, build fixed!
<LPCIBot> Project devel build #889: FIXED in 5 hr 59 min: https://lpci.wedontsleep.org/job/devel/889/
<LPCIBot> Yippie, build fixed!
<LPCIBot> Project db-devel build #721: FIXED in 5 hr 39 min: https://lpci.wedontsleep.org/job/db-devel/721/
<flacoste> deryck, gary_poster: a new escalated bug by ISD: https://bugs.launchpad.net/launchpad/+bug/810626
<_mup_> Bug #810626: launchpad should mark required sreg attributes  as required <escalated> <Canonical SSO provider:New> <Launchpad itself:Triaged> < https://launchpad.net/bugs/810626 >
<flacoste> should be fairly shallow
<deryck> flacoste, cool, thanks.
<gary_poster> flacoste, cool, yeah, we were asking you about that one on ops
<flacoste> but it blocks them deploying their new version as it's a potential source of problems for us
<gary_poster> cool
<deryck> gary_poster, what will it be?  orange or yellow?  rock, paper, scissors, lizard, Spock for it?
<flacoste> i've only marked the one about making the fields required Critical
<flacoste> since the other one, is only relevant when we handle multiple OPs
<flacoste> so I marked it Low
<gary_poster> deryck, whoever gets to first :-) we've got a couple of escalated in progress already
 * flacoste thinks Orange should contribute to the escalated effort ;-)
<deryck> indeed we can :)
<flacoste> Yellow has 3 potential fixes for the week
 * deryck didn't realize yellow had escalated bugs already
<gary_poster> yellow hears "escalated" and comes running!  It might be...fun, or something?!
<deryck> :-)
<deryck> flacoste, gary_poster -- we've got a card on orange next lane now for it.
<gary_poster> cool
<abentley> bac: because your getnewcache is based on my json-serialisation, merging from devel creates criss-cross merges and screws up the diffs between my branch and yours.  Could you please refrain from that in the future?
<abentley> bac: And could you please pull my ~abentley/launchpad/getnewcache, where I've fixed the criss-cross?
<abentley> deryck: a closer look at the update_cache code shows it's correctly updating the plain json objects, rather than replacing them with Entry objects.
<deryck> abentley, ok, thanks for the update.
<SpamapS> Is there an API call I can do to get this list: https://code.launchpad.net/principia/oneiric
<abentley> SpamapS: Yes: getBranches
<abentley> SpamapS: Actually, that's on DistroSourcePackage, not DistroSeries.
<SpamapS> abentley: you got me all excited.. ;-)
<abentley> SpamapS: It probably exists as a method, just not exported yet.
<gary_poster> sinzui, I'm about to file a bug about the send-person-notifications script not logging errors to the filesystem.  Am I right, or did I simply miss it?  AFAIK, one can only get the error reports from the launchpad-error-reports list, which I think is bad.
<sinzui> gary_poster, I did not find the log. report the bug
<gary_poster> thanks sinzui
<lifeless> gary_poster: well it should be oopsing
<lifeless> gary_poster: have you looked for an OOPS yet ?
<lifeless> gary_poster: changing from stderr to a log file is an RT generally, if its a Launchpad*Script
<gary_poster> lifeless, looked for OOPS: how and where would I do so?  I am only aware of a "if you know the OOPS id then you are in luck" interface
<lifeless> they all get rsynced to carob
<lifeless> flacoste: on bug 810623
<_mup_> Bug #810623: launchpad oopses when no sreg attributes are returned by SSO <Launchpad itself:Triaged> < https://launchpad.net/bugs/810623 >
<lifeless> flacoste: I think we should keep to the oops->critical, but do a real simple fix - make it a UFD or BadRequest
<lifeless> flacoste: or
<lifeless> flacoste: close it invalid
<poolie_> hi all
<flacoste> lifeless: might want to leave a comment
<flacoste> deryck's squad can sort it out when they work on the main one
<lifeless> flacoste: I will, wanted to discuss with you first ;)
<lifeless> flacoste: so we didn't get in a bidding war
<lifeless> flacoste: I favour the first case
<flacoste> lifeless: i'm easy today, so whatever :-)
<lifeless> heh, ok
<poolie_> re https://bugs.launchpad.net/launchpad/+bug/807383 in theory it ought to be possible to work out which oops a particular user hit, shouldn't it?
<poolie_> s//oughtn't it?
<poolie_> or is it just a case of, if they don't work hard enough to report it we don't have a special bug for it?
<lifeless> its not easy to find the right one (we log oops *files* for things we don't count as OOPS. [thats a bug]).
<lifeless> also, we're still swamped, so its not like we're at the point [yet] of picking up the 3 or 4 unique oopses from the daily report and driving them to zero.
<lifeless> poolie_: actually, another way to put it is: we track all oopses and well get them all eventually; if soneone wants to jump queue - manually filing a bug for us, thats fine, but then the onus is on them to help us out
<poolie_> right
<poolie_> that's pretty much what i though
<poolie_> t
<lifeless> oh wow
<lifeless> sinzui: why ?
<lifeless> New member:
<lifeless>  (Chooseâ¦)
<lifeless> You can't add a team that doesn't have any active members.
<lifeless> sinzui: adding ~canonical-launchpad-emeritus to https://launchpad.net/~launchpad-emeritus/+addmember
<poolie_> <lifeless>  The most recent 90 minute downtime was the last ever.
<poolie_> would be nice
<LPCIBot> Project devel build #890: FAILURE in 5 hr 36 min: https://lpci.wedontsleep.org/job/devel/890/
<sinzui> lifeless, I have never seen that issue show up. I do not know why Lp wont let you
<sinzui> m
<LPCIBot> Project db-devel build #722: FAILURE in 5 hr 41 min: https://lpci.wedontsleep.org/job/db-devel/722/
<sinzui> StevenK, mumble?
<lifeless> sinzui: I think its a mistake, because a team can be emptied after including in another team
<lifeless> sinzui: any objection to a jfdi fix ?
<sinzui> please jfdi
<StevenK> win 24
<nigelb> fail? :)
 * StevenK kicks nigelb :-P
 * nigelb has had a sleepless night :/
<wgrant> lifeless: When do you want to talk?
<lifeless> when I get off the phone with allison :)
<lifeless> 15?
<wgrant> Sure
<wgrant> Attack of the TAs?
<lifeless> :P
<nigelb> lol
<nigelb> wallyworld_: What time do you EOD?
<wallyworld_> nigelb: i've only just started my day
* wallyworld_ changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: wallyworld* (jtv) | Critical bugs: 232 - 0:[######=_]:256
<nigelb> wallyworld_: I'm about to head to bed on a /very/ late night thanks to work.
<wallyworld_> nigelb: do you want to ping me later?
<nigelb> wallyworld_: Yeah, that's why I was trying to figure out what's your usual EOD to plan that.
<wallyworld_> nigelb: ah, right. i'll be here for another 8-9 hours, plus i can be online tonight. there'll be a few hours in the even when i have soccer
<wallyworld_> s/even/evening
<nigelb> wallyworld_: In that case, I think I can figure out something today. I'll keep my devel up-to-date
<wallyworld_> nigelb: ok. i'll wait to hear from you
<wallyworld_> aaarrghhh. did qastaging just die?
<wgrant> wallyworld_: Looks fine to me. Maybe it just updated.
<wgrant> Shouldn't have, though.
<wallyworld_> wgrant: it just came back now
<wallyworld_> wgrant: could you do me a favour to help me qa? could you subscribe someone (not you or me) to a branch you own and then change the owner to launchpad and let me know which branch?
<wgrant> lifeless: Do you still only do Skype?
<wgrant> wallyworld_: https://code.qastaging.launchpad.net/~launchpad/launchpad/db-devel-merge-fix
<wallyworld_> wgrant: awesome thanks
 * wallyworld_ marks his bug as qa-ok
<wgrant> Thanks!
<wallyworld_> s/bug/branch
 * wgrant goes to use canonicaladmin for a while, to acclimatise himself to crap software before using Skype.
<wallyworld_> lol
<wallyworld_> skype isn't that bad. it has pretty good echo cancellation
<wallyworld_> seems to work better than mumble often times
<wgrant> The echo cancellation is the only thing that is not crap.
<wgrant> Well, and the NAT traversal.
<wallyworld_> they're pretty important features, no?
<wgrant> Yes, but the software itself is crap.
<wgrant> Crashy, slow, ugly.
<lifeless> wgrant: its easiest in a bunch of ways, if you don't mind significant feedback I can do voip
<wgrant> I have Skype not crashing now.
<wgrant> So we can use it.
<lifeless> \o/
<michaelh1> Hi there.  I'd like to import the GDB 7.3 release branch into Launchpad.  They use CVS and have a GIT mirror.  Who should I ask? (mwhudson is away)
#launchpad-dev 2011-07-15
<wgrant> lifeless: Ah, that's right. WAL won't allow us to read from slaves that are partly migrated, but it will allow us to detach a slave. Which means we could do very cheap read-only, which makes handling appserver requests much easier.
<wgrant> Doesn't help for scripts, but they're less noticable.
<wgrant> http://dev.launchpad.net/Soyuz/HowToUseSoyuzLocally has instructions on getting a buildd and buildd-manager up.
<wgrant> Best to run lp-buildd in a VM, of course.
<wgrant> But you do anyway.
<lifeless> wgrant: thanks
<lifeless> wgrant: selfreview that
<wgrant> lifeless: What?
<wgrant> Oh, the sreg thing?
<lifeless> yeah
<wgrant> Was considering it.
<wgrant> Thanks.
<wgrant> lifeless: Yay, batchnav QA.
<lifeless> well
<lifeless> it looks ok t obe
<lifeless> looks ok to me
<wgrant> It does.
<lifeless> I don't konw what abel found
<wgrant> And we are qa-bad.
<wgrant> Not on that rev, though.
<lifeless> wgrant: what on ?
<wgrant> Expander thingy.
<wgrant> Bug #807434
<_mup_> Bug #807434: Replace source package files, publishing history and similar bug expanders <bad-commit-13438> <qa-bad> <tech-debt> <Launchpad itself:Fix Committed by danilo> < https://launchpad.net/bugs/807434 >
<lifeless> missing bad-commit-XXX
<lifeless> no its not
<lifeless> just mup formatting confused me
<wgrant> Will rollback in a sec, just checking to see whether the other one needs doing too.
<wgrant> It's not pretty, but it doesn't seem to be broken.
<StevenK> spiv: O hai, you haz QA
 * wgrant rolls back.
<spiv> StevenK: for #702024 ?
<StevenK> spiv: Yup
<spiv> The only QA I can think to do replicates the automated tests, and the change is cosmetic rather than functional.
<wgrant> Bug #702024
<wgrant> Is that the Unavailable thing?
<spiv> (And would require l-osa interaction I think)
<spiv> Yeah
<wgrant> Let me poke at it.
<spiv> Thanks!
<wgrant> wgrant@mawson:~$ curl http://bazaar.qastaging.launchpad.net:8022/
<wgrant> Available
<wgrant> 0 connections:
<wgrant> Looks OK.
<spiv> wgrant: thanks!
 * spiv goes and takes some cold&flu pills to celebrate
<wgrant> StevenK: Poor amd64.
<wgrant> Hm, only 3200 OOPSes from the session incident.
<wgrant> Not so bad.
<wgrant> lifeless: Are you investigating the abel thing?
<wgrant> StevenK: Can I have DF
<wgrant> ?
<lifeless> wgrant: I've emailed him
<lifeless> wgrant: but he didn't seem to have written anything down other than the lazr.batchnavigator MP, which is approved.
<wgrant> Yes, I saw that yesterday and wondered what was going on.
<wgrant> Hoped you knew.
<lifeless> me too :>
<wgrant> StevenK: What do you know about +initseries?
<wgrant> https://dogfood.launchpad.net/deribuntu/grar/+initseries defaults to initialising from natty.
<wgrant> Despite there being existing series in that distro.
<StevenK> wgrant: "Oh God, it's full of JS!"
<wgrant> It also won't let me add any series from the current distro...
<wgrant> Oh, perhaps the parent series thing there is not for initialisation, just for DSDs.
<wgrant> This is confusing *me*. This isn't good...
<StevenK> Haha
<wgrant> Ah, yeah, the form changes subtly.
<wgrant> StevenK: Do I use run_jobs to run IDS jobs?
<StevenK> Yes
<wgrant> The logging leaves something to be desired.
<StevenK> The logging infrastructure that the job system provides is non-existant.
<StevenK> There is no way to get at the logger object inside the job's run() method
<wgrant> :)
<lifeless> wgrant: ever seen:
<lifeless> ~$ virsh console lp-builder
<lifeless> Connected to domain lp-builder
<lifeless> Escape character is ^]
<lifeless> error: internal error cannot find character device (null)
<lifeless> StevenK: ^
<wgrant> Nope.
<wgrant> But it doesn't surprise me.
<wgrant> Do you have a console-capable domain?
<wgrant> KVM normally isn't.
<lifeless> ah
<wgrant> Gar
<wgrant> WTF
<wgrant> Why is the URL of a packageset /package-sets/${distroseries.name}/${packageset.name}...
<poolie_> lifeless, i have seen that, i didn't work out why
<StevenK> Surely it should be distribution/distroseries/packageset?
<poolie_> i like the sound of "downtown deploy"
<wgrant> StevenK: You would think.
<wgrant> /ubuntu/natty/+packageset/core
<StevenK> Right
<StevenK> That sounds good
<lifeless> poolie_: where is that typo?
<poolie_> http://www.youtube.com/watch?v=FKCnHWas3HQ
<poolie_> also bug 810694
<_mup_> Bug #810694: mirror prober failed after trying to shutdown during a downtown deploy <oops> <Launchpad itself:Triaged> < https://launchpad.net/bugs/810694 >
<lifeless> poolie_: what was the initial password resulting in your ubuntu-vm-builder recipe ?
<lifeless> (the one from dev.l.n)
<spiv> lifeless: 'ubuntu' perhaps?
<lifeless> tried already :P
<lifeless> Ubuntu
<lifeless> nope
<lifeless> ><
<poolie_> i don't recall
<lifeless> ah, it totally ignored the parameter to control the user
<lifeless> presumable ubuntu-vm-builder -> vmbuilder fallout
<lifeless> also, hooray for network X
<StevenK> wgrant: Your rollback just hit buildbot
<wgrant> lifeless: We don't do backward navigation yet, do we?
<lifeless> wgrant: next, prev
<wgrant> But does that actually use the new backward navigation stuff?
<lifeless> wgrant: yes
<wgrant> :(
<lifeless> its only broken in the corner case of less than a full batch at the start
<lifeless> which I expect would occur only with changing data today
<lifeless> because all the current navigators will be offset based.
<lifeless> [or with url hacking]
<lifeless> s/which I expect would occur only with changing data today/which cannot happen today/
<wgrant> Are we OK to deploy with it, then?
<lifeless> AFAICT yes, but its friday ... figuring abel can answer what he found when he shows up
<LPCIBot> Yippie, build fixed!
<LPCIBot> Project devel build #891: FIXED in 5 hr 37 min: https://lpci.wedontsleep.org/job/devel/891/
<lifeless> wgrant: are you and free sorting out the tc* name?
<lifeless> bah
<lifeless> long poll project name.
<wgrant> I haven't discussed it since it was mentioned last night.
<lifeless> I'm going to ignore it from now on; AIUI he would like a implementation agnostic name.
<wgrant> It will probably end up as txwebnotify, but we'll see :)
<wgrant> Yeah
<lifeless> webevent
<lifeless> httpevent
<lifeless> push
<lifeless> (some arbitrary suffixes to play with)
<spiv> tenfoot
<spiv> (it's a long pole!)
<lifeless> kill me now
<lifeless> !
<wgrant> That's a very Twisted name, though.
<LPCIBot> Project db-devel build #723: STILL FAILING in 5 hr 43 min: https://lpci.wedontsleep.org/job/db-devel/723/
<lifeless> spiv: (nice one)
<lifeless> wgrant: overly cute pehraps
<spiv> lifeless: :)
<wgrant> lifeless: Yes, very Twisted.
<StevenK> Hm, how do we QA r13437?
<lifeless> its live
<lifeless> therefore its ok
<StevenK> lifeless: Marking as qa-ok, then
<StevenK> Then aside from r13413 and the pending rollback of r13438, we look in okay shape
<lifeless> cool
<lifeless> when abel gets up, someone should ask him about 413
<lifeless> I should be around
<StevenK> Right, and the only QA to do will be one rev of wgrant's, but that won't be for roughly four hours.
<StevenK> But then it's probably too late to deploy until Monday morning.
<lifeless> yup
 * StevenK wishes qas didn't take an hour to update
<spm> patches accepted
 * StevenK waits for wallyworld_ to review his branch, since he's already claimed it.
<StevenK> wallyworld_: I've just pushed a new rev that removes one line of extraneous whitespace
<wallyworld_> StevenK: thans, just about to +1 it :-)
<wallyworld_> sorry for stealing it, it's my first and only one today
<spm> wallyworld_: may I suggest a -1 due to excessive impatient impertinence?
<wallyworld_> lol
<StevenK> wallyworld_: I am of the opinion that it's too large to self-review, so it's perfectly fine.
 * spm promotes peace and understanding via the application of a nail studded club, amongst LP developers
<wallyworld_> ooh, studs. now you're talking
<spm> ... tmi.
<StevenK> Now. Look. What. You've. Done.
<spm> clearly it's all your fault StevenK. you started this.
<nigelb> I thought some bored LOSA started it :P
 * nigelb yawns and stretches
<wallyworld_> StevenK: i assume you changed the vocab because the users of the vocab wanted to pass in spn instead of dsp?
<nigelb> I wish a very painful death to telemarketeers who wake you up the day you pull an all-nighter
 * wallyworld_ wishes the same thing to *ant* telemarketer
<wallyworld_> any
<StevenK> wallyworld_: I changed it due to sinzui's e-mail of this morning.
 * wallyworld_ looks at email
<wallyworld_> makes sense now :-)
<StevenK> wallyworld_: You'll prod jtv to mentor?
<wallyworld_> StevenK: yes
<wallyworld_> StevenK: you may want to cut'n'paste a bit of the email which mentioned the need for the change into the mp description
<wallyworld_> StevenK: jtv is away today (national holiday). maybe wgrant can +1
* wallyworld_ changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: - | Critical bugs: 232 - 0:[######=_]:256
<mrevell> Hey
<adeuring> good morning
<lifeless> adeuring: hi
<bigjools> morning
<lifeless> adeuring: so batchnav
<lifeless> adeuring: you should land your patch and do a release :) - but more importantly, is lp ok at the moment with the simple offset based batches?
<lifeless> adeuring: (thinking qa for Revision 13413 )
<adeuring> lifeless: yeah... real QA is hard for such a change that affects each batched view. Any suggestions?
<StevenK> jtv: Can you please mentor wallyworld_'s review of https://code.launchpad.net/~stevenk/launchpad/dsp-vocab-use-spn/+merge/68046 ?
<lifeless> adeuring: play with as many as we can - its how I found the change needed in 1.2.5
<lifeless> adeuring: I have been playing since that hit qastaging and couldn't break it (without url hacking)
<lifeless> adeuring: I grepped for similar things too
<adeuring> lifeless: ok.
<lifeless> adeuring: how did you find that direction=forwards issue ?
<adeuring> lifeless: by reading the source code
<lifeless> adeuring: ah, fresh eyes!
<adeuring> lifeless: can you tell me how to do a release of for lazr.batchanv?
<lifeless> adeuring: https://dev.launchpad.net/ReleaseChecklist
<adeuring> thanks
<lifeless> its pretty much what I followed
<lifeless> uhm, you will need to be added to the pypi maintainer list for projects
<lifeless> I need to spend family time, so perhaps you can get flacoste to do that in ~ 4 hours
<lifeless> the pypi registration step is the last step so its not a blocker for updating LP
<bigjools> wgrant: hi - did you get a chance to look at my copyPackage branches?
<wgrant> bigjools: I think we should confirm that PCJs work first.
<bigjools> they do
<wgrant> I doubt it.
<wgrant> I don't see how some cases work.
<wgrant> eg. announcements of copies into Ubuntu.
<bigjools> so, rather than a blanket "they don't work" I think I'd appreciate it if you were more specific like that from the start
<wgrant> Well, AFAICT PCJs had a few missing bits hacked together by Steve, then he was repossessed by our squad and nobody verified that PCJs were actually complete.
<StevenK> Ah. ENOJTV
<wgrant> And this will enable copying into Ubuntu, so a thorough verification of their correctness needs to be performed.
<StevenK> Who wants to mentor wallyworld_'s review, then? :-)
<bigjools> wgrant: can you name any specific parts that you know don't work as expected?
<wgrant> bigjools: I need to leave for a while in a moment, but will return in an hour or so.
<wgrant> bigjools: But things like build notifications, copy announcements, etc. for copies into the Ubuntu primary archive are, I'm pretty sure, not going to work.
<wgrant> We've already seen that notifications are not reliable.
<wgrant> In some cases they crash.
<wgrant> They are likely to email the wrong people, because we have no way to track who the right people are.
<wgrant> The security code has not been checked since the last few rounds of improvements were made.
<bigjools> security code?
<wgrant> The interfaces are weak so it's hard to verify that everything is going to work, but we need to try.
<wgrant> This moves security into the model.
<wgrant> From the security adapters.
<wgrant> This is scary.
<StevenK> adeuring: O hai, you're fine to do OCR today?
<bigjools> the adapter can never work
<wgrant> Sure.
<wgrant> But the new approach needs to be approached with great skepticism.
<bigjools> we are now using the very tried and very tested checks that have always been there
<wgrant> Their integration is not tried, tested, nor trustworthy.
<bigjools> yes, that's exactly why I wanted your opinion
<bigjools> but this is counterproductive
<StevenK> I think you just got his opinion, in spades.
<wgrant> The recent implementations of PCJs and series initialisation need a lot of checking, because they were developed by hacking existing code to pieces and reassembling it and grafting on other refactorings until they did roughly what was required.
<wgrant> We are now trying to push them out without cleaning them up or checking that they work.
<bigjools> sorry but you are talking out of your arse
<adeuring> StevenK: sure, need a review?
<StevenK> adeuring: Sort of. A mentor of wallyworld_'s review -- https://code.launchpad.net/~stevenk/launchpad/dsp-vocab-use-spn/+merge/68046
* adeuring changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: adeuring | Critical bugs: 232 - 0:[######=_]:256
<adeuring> StevenK: ok, I'll look
<StevenK> adeuring: Thank you.
<adeuring> StevenK: +        dsp = self.context.getSourcePackage(spn)
<StevenK> adeuring: What about it?
<adeuring> StevenK: this assumes that self.context is not None, but: def __init__(self, context=None):
<StevenK> It does, yes
<adeuring> so, this breaks if no context is given
<StevenK> adeuring: This vocab is used nowhere else yet
<adeuring> StevenK: ok, so, then we can drop the "=None" from the __init__() parameters?
<StevenK> Hm, sure
<adeuring> StevenK: I mean, the =None is an invitation to create the vocab without a context, but that would explode :)
<adeuring> StevenK: r=me with this change
<StevenK> adeuring: Thanks! Changes pushed.
<adeuring> StevenK: cool
<wgrant> bigjools: Are PCJs tested with binaries at all?
<wgrant> It seems they allow binary copies, but only source overrides.
<wgrant> Not sure what is going to blow up if someone copies binaries into Ubuntu without overrides...
<wgrant> The security seems OK.
<bigjools> wgrant: this is the first 2 branches in a pipeline, I need to make some other changes yet, like feature flag checking and binary copying.  Thank you for checking the security.
<wgrant> bigjools: Oh, so this is to be behind a feature flag?
<bigjools> in fact I will probably deny binary copying for distros for now; eventually we want to use it for security uploads
<bigjools> that's the plan
 * bigjools curses code that passes logger objects around
<wgrant> If I'd known you'd not planned to land this as is, I would have been far less likely to decry it as dangerous madness. As presented, the branches suggested that you considered the functionality complete and finished and ready for widespread use, which seems somewhat premature given the complexity of the issues at hand.
<wgrant> I still contend that PackageCopyJobs are, in their present form, a hackjob with only slightly fewer special cases than delayed copies, but at least they have good promise for being cleaned up and destroying delayed copies :)
<wgrant> I also need to think about how copy/build notification recipients should work.
<wgrant> Have you considered that at all?
<wgrant> There is potential for great misdirected spammage.
<wgrant> I was hoping StevenK could look at that after the initial notification fixup, but that was not to be :(
<jtv> wgrant: I suspect that for the foreseeable future, soyuz changes will always be immature.
<jtv> Pointing it out with words like "hackjob" is a bit like pressing a "panic" button too often though; it can distract from the work that needs doing to make things better.
<jtv> Lots of stress involved.
<jtv> I find it takes a substantial mental effort to get past those terms and into constructive detail.
<jtv> wgrant: Wouldn't be an issue in low-blood-pressure situations, but as long as we're not in one, please beware of the inadvertent damage from blanket judgments!
<wgrant> jtv: Soyuz changes will always be immature, yes. But for copies, perhaps more than anything else in this project, it must be avoided. They are complex, fragile, badly understood, and most mistakes are of great consequence. Perhaps one day we will unify and clean the copiers up... but until then calling them by any positive term would be a dangerous misrepresentation of the situation.
<jtv> wgrant: not arguing any of that; I believe you.  I'm saying that in this situation of pressure and uncertainty, you need to be careful how you say these things.  Be specific about your concerns, keep them somewhere visible, state them in terms of what needs doing about them so that people can see progress towards their goals and aren't made to feel (unintentionally I know, but I'm pointing out that pressure isn't rational!) that they do a bad job, 
<jtv> Trust me, I'm middle-aged now.
<nigelb> Can't argue that ^
 * nigelb ducks
<jtv> Hey, I can say it; you are supposed to go "you don't look a day over 39Â½!"
<nigelb> haha
<wgrant> jtv: Is it worth adding the config setting?
<bigjools> jtv: you've always been middle-aged
<wgrant> jtv: Given that it is deprecated?
<jtv> It's _almost entirely_ deprecated.
<wgrant> Isn't the migrator the only thing that remains?
<jtv> Exactly.
<wgrant> Any reason not to hardcode it and delete it in three weeks?
<wgrant> Oh, tests, I guess.
<wgrant> Grar.
<jtv> The tests went across with remarkable ease, though it's nice to know that the migration itself can be tested and automated.
<LPCIBot> Yippie, build fixed!
<LPCIBot> Project db-devel build #724: FIXED in 5 hr 42 min: https://lpci.wedontsleep.org/job/db-devel/724/
<danilos> wgrant, hi, if you are still around, perhaps you'd want to review a very simple fix for DSP:+index expanders?
<danilos> wgrant, fwiw, you rolled back the wrong revision :(
<wgrant> danilos: Argh, sorry. The bug was linked, and I saw distributionsourcepackage-publishinghistory in there, and mistook it for distributionsourcepackage-index.
<danilos> wgrant, no worries, I'll reland that revision, and a fix for the problem you noticed as soon as it goes through ec2 (just to be extra careful, some tests might be expecting IMG tag in there which we don't need anymore)
<wgrant> danilos: Have you checked all the others?
<wgrant> I only checked up to the first issue.
<wgrant> And unless we're sure it's all fine we should roll it all back.
<danilos> wgrant, what do you mean? I can't QA the revision until it is on qastaging, and it's already up to your rolled-back revision
<wgrant> danilos: Well, this sort of stuff is fairly QAable locally.
<danilos> wgrant, as for the problem in previous revision, how do I look for other togglers on packages?
<danilos> wgrant, well, I did QA everything locally, but I missed it because same JS integrated in the archive-macros.pt was used on two different pages implicitely
<wgrant> Ahh, fun fun.
<danilos> wgrant, I QAd only the PPA:+packages page
<danilos> wgrant, thinking I am good
<wgrant> Yeah.
<wgrant> I am glad we are cleaning this stuff up.
<wgrant> So, I guess see if you can grep around a bit further, and if not I will just look everywhere on Monday, I guess.
<danilos> wgrant, I did grep for treeCollapsed, but I guess I missed this one case :/ (there are a few others left which I am not fixing because of the lack of time, and this one was in registry as well :/)
<wgrant> Yeah.
<wgrant> Still, I am impressed at the lack of any other breakage.
<wgrant> Considering all the cleanup.
<danilos> wgrant, heh, thanks, I did try to be very careful with QA, and you can look at QA URLs in my MPs to see the number of URLs I had to get to to test stuff out (finding proper places for that was sometimes harder than migrating the template/JS)
<wgrant> danilos: Yes. I opted to check all the expanders I knew about, because it's all so intertwined.
<wgrant> Little other choice :(
<wgrant> Still, progress!
<danilos> wgrant, indeed
<danilos> wgrant, I suppose you don't have time to review https://code.launchpad.net/~danilo/launchpad/fix-806925/+merge/68070 then? :)
<danilos> adeuring, can you please look at https://code.launchpad.net/~danilo/launchpad/fix-806925/+merge/68070 (it's very short)
<adeuring> danilos: sure
<danilos> thanks
<adeuring> danilos: r=me
<danilos> adeuring, thanks
<danilos> adeuring, I've stuck in another expander replacement in the same template fwiw, can you please glance at it and check if it's ok as well?
<adeuring> danilos: sure
<danilos> adeuring, it's in diff from line 27 onwards, QA on https://launchpad.dev/ubuntu/+source/iceweasel (because there are PPAs for that)
<adeuring> danilos: still looks good!
<adeuring> r=me
<adeuring> again
<danilos> adeuring, heh, thanks (again :))
<sumanah> hi, bac, still working on that summary and will get it to you in the next hr
<sumanah> https://dev.launchpad.net/UI/Reviews?action=show&redirect=UserInterfaceReviewNotes#Tips%20for%20reviewers will be pretty useful I think!
<maxb> stub: problems? :-)
<bigjools> adeuring: hi, can I get a review please! https://code.launchpad.net/~julian-edwards/launchpad/pcj-requester-bug-810957/+merge/68081
* bac changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: adeuring,bac | Critical bugs: 232 - 0:[######=_]:256
<bac> morning adeuring
<adeuring> hi bac
<bac> adeuring: i am grabbing ian's first branch, unless you've already started it
<adeuring> bac: no, I haven't  (working purely "Interrupt dirven" this morning )
<bac> adeuring: ok.  claimed.
<sumanah> bac: working on a summary of how Launchpad does code review & deployment -- can you give me a link about the template merge proposal that you can do via email?  I look at https://help.launchpad.net/Code/Review and it links to http://doc.bazaar-vcs.org/bzr.dev/en/user-guide/index.html#sending-changes which does not exist
 * bac looks
<sumanah> and http://doc.bazaar.canonical.com/bzr.dev/en/user-guide/sending_changes.html does not mention the template
<bac> sumanah: https://dev.launchpad.net/Code/BzrSend
<sumanah> aha! thanks bac
<sumanah> you can see how far I've gotten in Gobby
<sumanah> "The lpreview_body plugin will pre-populate the message body with our standard template, plus lint output." ooh, I'm going to look at that plugin to see what your template is!
<deryck> Morning, all.
<sumanah> http://bazaar.launchpad.net/~launchpad/lpreview-body/trunk/view/head:/body_callback.py got it.
<abentley> bac, sumanah: bzr send hasn't worked for a couple of years.
<bac> abentley: what is the replacement then?
<abentley> bac: there is no replacement.
<abentley> bac: The technical issue is that installing a bundle is incompatible with stacked branches.
<adeuring> rvba: your branch init-series-not-close-bugs looks good but I think it could have at least one more test: the new test test_multiple_parents_close_bugs should check that the bug created in the test is not closed (and... shouldn't the test method's name include a 'nit'?)  . And do we already have tests that ensure that the other callsite of the packagecopier does indeed close bugs?
<rvba> adeuring: Thanks for reviewing this branch. I think I'll add the check to the existing test.
<adeuring> rvba: thanks!
<rvba> adeuring: about the other callsites, I'm not really sure ... but I *suppose* we do have check for this.
<rvba> I'll have a look, just to make sure.
<adeuring> rvba: cool, thanks
<bac> abentley: i'm confused between 'bzr send' and lpreview/lpreview_body
<abentley> bac: bug #718723
<_mup_> Bug #718723: fetch from merge directive to stacked branch unable to fill in chk pages <oops> <regression> <Bazaar:Confirmed> <Launchpad itself:Triaged> < https://launchpad.net/bugs/718723 >
<bac> abentley: i've continued using 'bzr send --no-bundle' to create my MPs
<rvba> adeuring: BTW, the branch we are talking about depends on ~rvb/launchpad/perm-distributionjob-bug-808680/+merge/68059.
<abentley> bac: Yes, if you use --no-bundle, that probably works, but it means you have to manually upload the branch first.
<rvba> adeuring: If you have time, I'll be glad to have a review for this branch as well.
<bac> abentley: i always push first.
 * bac assumed that was standard workflow
<abentley> bac: However, lp-propose does it all over the API instead of using email, and it does the push for you, so it's more convenient and has basically replaced "send"
<abentley> bac: lp-propose also supports prerequisite branches, unlike "send".
<bac> abentley: ok.  is there a wiki page?
<sumanah> abentley: does it include a template for the merge proposal?
<abentley> sumanah: the lpreview_body plugin provides the template for lp-propose (as well as send).
<sumanah> aha
<abentley> bac: for lp-propose? No, it's a bzr command.
<abentley> bac: Part of the launchpad plugin which ships with bzr.
<adeuring> rvba: sure, I'll look at that one too
<rvba> adeuring: thanks/
<rvba> thanks!*
<bac> abentley: ok.  thanks.  we should dismantle the wiki page i posted
<abentley> bac: There is https://lists.launchpad.net/launchpad-dev/msg06026.html
<sumanah> bac, abentley - & point to http://doc.bazaar.canonical.com/plugins/en/launchpad-plugin.html ?
<abentley> sumanah: That's the standard command help.
<sumanah> abentley: sure seems like it, yes :-)
<abentley> sumanah: I would not point at it, because people can get it from the commandline, and it will be the help that is accurate for their version.
<sumanah> abentley: I'm documenting how Launchpad does its code review process, so my colleagues can use it as an example, so I need to point them to documentation of the tool support/infrastructure that you use as well
<abentley> bac: did you see my messages yesterday?
<sumanah> abentley: the people reading my summary will not have bzr installed, much less this plugin, so I would rather point them to a webpage with relevant documentation of this part of the process
<bac> abentley: i did
<abentley> sumanah: If it's your documentation, then of course it's your call.
<bac> abentley: i've worked through most of the hurdles on that branch
<sumanah> abentley: seems like you'd want reasonable documentation in the public eye -- on the web somewhere -- for the community devs, no?
<abentley> bac: excellent.
<sumanah> abentley: and point to it from the dev.launchpad wiki?
<abentley> sumanah: If you want to document it for us, please put it on the wiki, rather than pointing to it.
<sumanah> abentley: I'm surprised you didn't update https://dev.launchpad.net/Code/BzrSend when you sent that msg in Dec 2010
<abentley> sumanah: I'm not.  I was just trying to get the word out.  And at that point, I had some hope that "send" would eventually be fixed.
<sumanah> abentley: what's inaccurate about http://doc.bazaar.canonical.com/plugins/en/launchpad-plugin.html ?
<abentley> sumanah: If you're using an older version of bzr, lp-find-proposal won't exist, for example.
<abentley> I wrote that in the last year.
<sumanah> I am the least expert person in this conversation and likely to introduce errors in any documentation I write about this plugin, but .... aaand I get an error even trying to log in to the dev.launchpad wiki.
<henninge> deryck: still travelling but almost back
<sumanah> aha, it's because I have an OpenID. http://moinmo.in/MoinMoinBugs/OpenID%20login%20to%20Ubuntu%20wiki%20fails?highlight=%28\bCategoryMoinMoinBug\b%29
<deryck> henninge, ok, np
<sumanah> bac: ok, I cannot log into the dev.launchpad MoinMoin wiki (bug filed), so thanks for updating https://dev.launchpad.net/Code/BzrSend
<sumanah> or rather, even though I am logged in, I can't edit
<abentley> bac: I don't know what happened, but when you merged from me today, you didn't get three revisions that I committed yesterday.
<bac> abentley: sorry, i did see your note yesterday but haven't done the merge yet.  didn't know you'd be grabbing my branch.
<bac> i'll do it now
<abentley> bac: Oh, my bad.  The merge I'm looking at wasn't today.
<abentley> bac: If you could, that would be great, so we can get synced up.
<bac> abentley: merging from yours now.  will let you know when i've pushed
<adeuring> rvba: r=me for your init-series-not-close-bugs branch
<bac> abentley: pushed, though i've still got work to do on the branch.
<abentley> bac: I have it working (with some hacks to the old version of your code) on the translation sharing details page.
<bac> cool
<rvba> adeuring: Thanks (sorry for the delay, OTP). Instead of checking the bug's status I'll use the fakemethod trick to make sure the close_bugs_... method has not been called. It's best to do that because that is precisely what this test wants to check (as opposed to calling the method and failing with a permission denied error)
<bigjools> adeuring: hi, can I get a review please! https://code.launchpad.net/~julian-edwards/launchpad/pcj-requester-bug-810957/+merge/68081
<adeuring> rvba: sure, makes sense
<adeuring> bigjools: sure
<bigjools> thanks
<abentley> bac: Your latest changes work for me, with this patch: http://pastebin.ubuntu.com/644806/
<bac> abentley: yeah, that makes sense
<sumanah> bac: it's a bit difficult to search for information about how you deploy Launchpad -- is there an overview I am missing about your deploy/release process
<sumanah> ?
<sumanah> bac: I see https://dev.launchpad.net/QAProcessContinuousRollouts  -- aha, points to https://dev.launchpad.net/MergeWorkflow .  Is that reasonably accurate?
 * bac reads
<bac> sumanah: https://dev.launchpad.net/QAForContinuousRollouts?highlight=%28qa%5C_reports%29
<bac> er, https://dev.launchpad.net/QAForContinuousRollouts
<sumanah> thank you bac!   I should be looking for "rollout" and its variations rather than "deploy"
<sumanah> mrevell: I appreciate the wordplay & graphics in your blog posts :-)
<mrevell> thanks sumanah :)
* flacoste changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: adeuring,bac | Critical bugs: 218 - 0:[######=_]:256
<adeuring> bigjools: your branch looks good. I don't see any usage of the job.requester, but I assume that will follow in another branch?
<bigjools> adeuring: correct!
<bigjools> was just doing this separately for ease of review
<adeuring> bigjools: ok, so r=me (I appreciate that your deferred implementing the usage here -- would have made the branch even longer ;)
<bigjools> adeuring: well the other reason is that I have another branch in progress but it depends on this before I can fix it further. :)
<bigjools> adeuring: thanks for reviewing
<sinzui> deryck, I want to nuke lib/lp/scripts/utilities/lpwindmill.py and its only caller, bin/jstest. Are these oversites from our removal of windmill?
<sinzui> s/oversites/oversights/
<deryck> sinzui, yes, just oversite.  I mean to go back and remove them.  Kill at will. :)
<sinzui> :)
<flacoste> danilos: what's the story behind qa-bad of bug 806925?
<_mup_> Bug #806925: Replace bug task, ppa details and package details expanders <bad-commit-13421> <qa-bad> <tech-debt> <ui> <Launchpad itself:Fix Committed by danilo> < https://launchpad.net/bugs/806925 >
<flacoste> there are is no comment on the bug
<gary_poster> flacoste, what I know is this: a branch is landing now, and will hopefully be ready for qa later today
<gary_poster> that's according to our kanban board and morning call
<gary_poster> henninge, am I right that a question about why https://translations.launchpad.net/babelenterprise/+imports has not been approved should be transferred to the babelenterprise project?
<flacoste> gary_poster: thanks
<gary_poster> yw
<gary_poster> oh, no, this guy asking is the owner of the project
<henninge> gary_poster: No, approvals still need to be done either automatically by the script or by a lp engineer
<henninge> "the autoapprover script"
<gary_poster> henninge, so should it have shown up here for lp devs to review https://code.launchpad.net/+code-imports?field.review_status=NEW ?
<henninge> gary_poster: that's code imports, you were asking about transltion imports.
<gary_poster> and henninge, can I just approve this poor guy so he can move on with his life, or do I need to...check something first?
<gary_poster> henning, aygh!  https://translations.launchpad.net/+imports/+index?field.filter_target=[PRODUCT]&field.filter_status=NEEDS_REVIEW&field.filter_extension=pot
<gary_poster> augh I mean
<henninge> gary_poster: yes, you can approve. That is part of CHR duties (until we fix that)
<gary_poster> looks like there has been some confusion about who needs to do what since the thunderdome
<henninge> gary_poster: I was told orange did CHR last week (I was on leave)
<henninge> gary_poster: but I can work down that queue fairly quickly I think
<gary_poster> (yeah henning, I think danilos did not realize we were on CHR this week somehow, even though we talked about it :-( )
<gary_poster> thank you henninge!  I'd really appreciate it
<henninge> np
<henninge> argh ...
<henninge> bad file naming
<henninge> gary_poster: actually, I will not approve them and you shouldn't either
<gary_poster> ok henninge
<gary_poster> henninge, is there a quick reason why?
<gary_poster> (I'd like to reply to the question too, https://answers.launchpad.net/launchpad/+question/163067)
<henninge> gary_poster: I will ask the uploader to rename the files so that the template name is extracted automatically
<henninge> all the files are named translations/translations.pot
<henninge> but they should be modulename.pot
<gary_poster> ok henninge.  So I can close the question, saying that we have replied to the translation request with details on what to fix?
<henninge> gary_poster: hang on, let me look at the question first
<gary_poster> ok
<henninge> gary_poster: there is only one file in their queue and it is not approved automatically because es_es.po is not supported by Launchpad, it should be es.po.
<gary_poster> ah ok
<henninge> gary_poster: you can either approve it to spanish or ask them to re-upload with the correct name.#
* flacoste changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: adeuring,bac | Critical bugs: 242 - 0:[######=_]:256
<gary_poster> henninge, to approve I would go to https://translations.launchpad.net/+imports/5558326 and selevct "Spanish (es)" and click approve, yeah?  I can do taht and then suggest a change for the future in the reply
<henninge> yes, you can do that.
<gary_poster> ok thanks very much henninge!
<henninge> gary_poster: you also need to pick the template
<henninge> there is only one
<gary_poster> ok that makes it easy, got it :-)
<sumanah> bac, flacoste - draft sent.  Brad, I know I am getting it to you later than I said I would; can you still review today for inaccuracies?
<jcsackett> can someone point me to where LP.cache.context gets setup?
<abentley> jcsackett: We're in the process of changing that.  Right now, it's set up in lib/lp/app/templates/base-layout-macros.pt
<jcsackett> abentley: ah, thanks.
<abentley> jcsackett: In my branch, it comes out of lib/canonical/launchpad/webapp/publisher.py, getCacheJSON.
<jcsackett> abentley: that's as part of your ++cache++ stuff, right?
<abentley> jcsackett: Right.
<jcsackett> cool. i'm just trying to understand the current picture i'm working with. i look forwards to seeing your work land. :-)
<abentley> jcsackett: (though we're calling it ++model++ to try to reflect its real significance)
<jcsackett> abentley: that does seem like a better namespace.
<abentley> jcsackett: my branch is ~abentley/launchpad/json-serialization
<sumanah> bac: just checking -- do you think you'll be able to get me a critique today?  just looking for inaccuracies & gross omissions
<jcsackett> abentley: do you know of any gotchas about the json cache (aside from it not working for anonymous users) that would lead to me placing something in it only to find nothing there on the page?
<jcsackett> for context, in lib/lp/app/widgets/popup.py i'm trying to add in some global picker config options, but nothing i put in can be found once i'm on a page with a picker.
<jcsackett> (actually, right now i'm scaled back to just trying to put a random string in).
<bac> sumanah: yes
<sumanah> thanks
* adeuring changed the topic of #launchpad-dev to:  https://dev.launchpad.net/ | On call reviewer: bac | Critical bugs: 242 - 0:[######=_]:256
<abentley> jcsackett: Nothing comes to mind off the top.  Are these basic python types or exported web service entry types?
<jcsackett> abentley: basic types. i think it may have something to do with trying to do it in the widget, rather than a view's initialize method (where every other example of adding to the cache seems to do it).
<abentley> jcsackett: I'm not sure of the order of operations, but it's conceivable that the widget is rendered after the jsoncache.
<jcsackett> abentley: yes, i think that's the case.
<jcsackett> oh well, this is what i get for doing something weird. i think i'll look into other approaches.
<jcsackett> EAT!!!	
<jcsackett> me(g) is starting to have a bad day. 14:14	
<abentley> deryck[lunch]: do we have a story for where test helpers go?
<deryck> abentley, in lib/lp/app/javascript/testing/
<abentley> deryck: Ah, okay.
 * jcsackett sighs, having just noticed that somehow is empathy chats have been redirected into IRC.
<m4n1sh> what is this dependency granite for maya?
<m4n1sh> err sorry again wrong window :9
* wgrant changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: bac | Critical bugs: 239 - 0:[######=_]:256
<wgrant> Grar
<lifeless> ?
<wgrant> https://lp-oops.canonical.com/oops.py/?oopsid=OOPS-2022AU38
<wgrant> Lots of checks, looks like it's checking 44 times whether the user administers any more teams.
<wgrant> I suspect the bugsubscription stuff.
<wgrant> Which we both identified as suspect.
<wgrant> s/bugsubscription/structural subscription/
<lifeless> just cause we're paranoid doesn't mean we're right...or wrong
<wgrant> Although you put a workaround in there to stop it from executing multiple times.
<lifeless> and tuned it somewhat as well
<wgrant> Half the code says "administrated", another half "administered"
<wgrant> yay
<lifeless> that looks like either a change, or a different code path
<lifeless> perhaps late evaluation ?
<wgrant> Let's see if it renders on DF.
<wgrant> And if so, traceback time.
<lifeless> new bug time too
<lifeless> can't see a match
<wgrant> True
<wgrant>   [r=jcsackett][bug=809841] 'All-Ubuntu-derivatives' option for
<wgrant>   	process-accepted script.
<wgrant> What could go wrong.
<wgrant> OTOH, we are now below 300 lines of mochikit-using JS.
<wgrant> sinzui: Does that mean we no longer depend on the hacked spidermonkey package?
<sinzui> wgrant, correct
<wgrant> Excellent news.
<sinzui> I just removed the last vestiges of it 10 minutes ago
<wgrant> And excellent deletion of embedded code.
<wgrant> It is gone from launchpad-dependencies?
<sinzui> it is
<wgrant> Perfect.
<sinzui> update and upgrade to 94
<wgrant> Perhaps I will investigate YUI3 calendar widgets.
<wgrant> Because this YUI2 embedded in the tree irks me greatly.
<wgrant> Blah.
<wgrant> The tour embeds jQuery.
<wgrant> How unpleasant.
<sinzui> wgrant, there were non last year
<sinzui> wgrant, we paid for the tour, and jquery is the defacto standard or js libs
<wgrant> sinzui: Yes, and I like it, but I do not like embedding external dependencies in our tree :)
<wgrant> http://www.yuiblog.com/blog/2010/06/18/alloy-date-selector/
<sinzui> We use datetime sometimes, but I wonder if that need can be factored out. datetime leads to bad notices that a release it overdue.
<wgrant> sinzui: Are you sure you can remove python-profiler?
<wgrant> sinzui: It provides pstats.
<wgrant> Which the test suite uses in two places.
<sinzui> it is identical to the pstats in py2.6
<wgrant> Yes, but it's in a separate package because it's non-free.
<sinzui> wgrant, I saw only comment changes in the diff
<wgrant> It's the only part of stdlib that's in a separate package.
<wgrant> $ dpkg -L python2.6 | grep pstats | wc -l
<wgrant> 0
<sinzui> oh ?
<wgrant> python-profiler is part of the standard library, but it's a separate package in multiverse because the 'profile' and 'pstats' modules are non-free.
<sinzui> I get 1
<wgrant> I am on natty.
<wgrant> Perhaps oneiric has regressed.
<wgrant> Let me see.
<james_w> it's free now
<wgrant> Huh.
<james_w> so it's in the python packages in oneiric
<wgrant> Was declared free, or was made free?
<james_w> pass
<wgrant> sinzui: Depend on 'python2.6 (>= whatever version) | python-profiler', I guess.
<wgrant> 2.7.2-2 made the change.
<wgrant> Oh, nice.
<sinzui> wgrant, maybe I should fork lp-deps to only require python-profiler for natty/lucid
<wgrant> they got Disney to agree to relicense it.
<wgrant> sinzui: No, just use the versioned disjunction.
<sinzui> okay
<LPCIBot> Project devel build #893: FAILURE in 5 hr 40 min: https://lpci.wedontsleep.org/job/devel/893/
<wgrant> sinzui: odd, it's not in your 2.6, surely?
<wgrant> I can only see the diff in 2.7.
<wgrant> Ah, no, there it is.
<wgrant> 2.6.7-2ubuntu1
<wgrant> This also means we can remove the awful component checking stuff from rocketfuel-setup in 12 months.
<wgrant> sinzui: So, are you fixing that up, or should I?
<sinzui> wgrant, I was trying to. I do not know how to write a ored statement that essential say, don't bother
<sinzui> wgrant, can you do it. I will read your change to know how?
<sinzui> wgrant, I am guessing that we also need to state Lp requires python 2.6
<wgrant> sinzui: Just depend on 'python2.6 (>= 2.6.7-2ubuntu1 | python-profiler'
<wgrant> Er.
<wgrant> 'python2.6 (>= 2.6.7-2ubuntu1) | python-profiler'
<sinzui> ah
<wgrant> In addition to the usual python2.6 dependency.
<sinzui> There is no python2.6 dep
<wgrant> Perhaps we should fix that.
<wgrant> Let me see.
<wgrant> We should really port to 2.7 in the next month or two. But I guess adding a 2.6 dep for now wouldn't hurt, as we now only get it accidentally.
<sinzui> wgrant, Is this the line to add theN
<sinzui> python2.6, python2.6 (>= 2.6.7-2ubuntu1) | python-profiler
<sinzui> I feel like a idiot writing it.
<wgrant> sinzui: That would indeed work.
<wgrant> It feels odd, but is not uncommon.
<sinzui> I will proceed.
<wgrant> Thanks.
#launchpad-dev 2011-07-16
<wgrant> Hmm.
<wgrant> Only 583 timeouts.
<wgrant> Perhaps the lack of index bloat is useful.
<LPCIBot> Yippie, build fixed!
<LPCIBot> Project devel build #894: FIXED in 5 hr 41 min: https://lpci.wedontsleep.org/job/devel/894/
<lifeless> mtaylor: so
<lifeless> mtaylor: when are we getting those blueprints patches ? :)
<mtaylor> heh. hehehehe
<mtaylor> lifeless: which ones was I going to do again?
<lifeless> I dunno
<mtaylor> and wait - aren't you meging them with bugs?
<lifeless> you had some openstack wishlist thingies
<lifeless> yeah, and ?
<mtaylor> well, I finally have another guy working with me
<lifeless> oh oh oh
<mtaylor> who is doing some hacking - although he's heads down in jenkins and gerrit right now
<lifeless> did you see we got the bugsummary fact/dimension setup going ?
<mtaylor> BUT - we should be able to get to some of these sorts of things eventually now (bandwidth is good)
<mtaylor> I didn't - what do those words mean?
<lifeless> data warehousing approach
<lifeless> a fact table - a table with (0,1) rows per unique combination of dimensions (in this case bugtasktarget, status, importance, has-patch, fixed-upstream, viewable-by)
<mtaylor> lovely!
<lifeless> and then you slice n dice the table to pull out stats
<lifeless> it has a column containing the counter for how many tasks matched that point in the table space
<lifeless> e.g. on https://bugs.launchpad.net/ubuntu/
<lifeless> the two portlets - tags and the counts
<lifeless> used to take 10+ seconds each to render
<lifeless> now the tags one is subsecond and the count one is 1.3 seconds 99% of the time, and not over 2s ever in the stats I've looked at
<lifeless> oh, tag is another dimension
<mtaylor> excellent!
<mtaylor> nicely done
<lifeless> in the last 6 months we've dropped out 99% render time site wide
<lifeless> ... by 50%
<mtaylor> lifeless: you know what - puppet is great except when it's not
<lifeless> orly? software not being great?
<lifeless> I'm shocked
#launchpad-dev 2011-07-17
<lifeless> timeouts are looking goo
<lifeless> d
* wgrant changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: - | Critical bugs: 237 - 0:[######=_]:256
<lifeless> aieee http://pgbouncer.projects.postgresql.org/doc/faq.html#_how_to_upgrade_pgbouncer_without_dropping_connections
<wgrant> Impressive.
<lifeless> looks like a race still exists
<wgrant> Oh?
<wgrant> SUSPEND will stop it accepting connections.
<wgrant> SHOW FDS will transfer the sockets, including anything waiting on the listener.
<lifeless> yes, and somsone connecting between suspend and the new process binding will get a rejection
<lifeless> unless the listening socket is handed over
<lifeless> it might do that
<wgrant> That's what I assumed.
<lifeless> [which you said ... :P]
<wgrant> Otherwise it would be pretty useless.
<wgrant> And pretty flawed.
<wgrant> lifeless: Found any issues with batchnav/
<lifeless> none
<lifeless> abel found that glitch by code inspection
<wgrant> Great.
<wgrant> Argh.
<wgrant> People hardcoding Ubuntu.
<wgrant> grar
<lifeless> url hacking can create it, but I don't think regular use will - at least not in any volume
<lifeless> wgrant: where?
<wgrant> http://bazaar.launchpad.net/~launchpad-pqm/launchpad/stable/revision/13446
<wgrant> The idea is flawed, and it hardcodes Ubuntu too :(
<wgrant> We really need to SOA Soyuz :(
<lifeless> a funding project for it is in the pipeline
<wgrant> Ah, excellent.
<wgrant> lifeless: Shall we qa-ok bug #809719, then?
<_mup_> Bug #809719: current lazr.batchnavigator version requires high OFFSET queries <qa-needstesting> <timeout> <Launchpad itself:Fix Committed by lifeless> < https://launchpad.net/bugs/809719 >
<lifeless> what can possibly go wrong
<lifeless> [yes]
<wgrant> Now we need to check every page with an expander.
<wgrant> Hmm.
<wgrant> Bad, but very minor so far.
<wgrant> Expanded suggestions on +filebug don't have a height limit on the sprite, so the sprite below (the branch icon) leaks onto the page.
<lifeless> usable ?
<wgrant> Yes.
<wgrant> So I'm not going to qa-bad it yet.
<lifeless> I'm inclined to tolerate that w/ a regression bug
<wgrant> Yup.
<wgrant> Ah, it actually used an <img> before.
<wgrant> Fail.
<wgrant> The regression fix did not fix the regression.
<wgrant> Oh.
<wgrant> It didn't actually land.
<wgrant> https://code.launchpad.net/~danilo/launchpad/fix-806925
<wgrant> Looks like it was thrown at ec2, though :/
<wgrant> lifeless: lp-land and pray? Or roll the whole lot back again?
<wgrant> We are almost up to 50 revs :/
<lifeless> so the regression fix was landed separately to relanding what was rolled back ?
 * lifeless wants to understand a little more what went on
<wgrant> Well, there were two revs doing the same sort of thing. I rolled back one, which turned out to be the wrong one, despite modifying the relevant file. It was relanded once the fix was written, but apparently the fix was not landed.
<wgrant> I assumed they landed together.
<wgrant> Only noticing now that it's not actually fixed.
<lifeless> so
<lifeless> we're 12 hours till danilo
<lifeless> 5 hours to find out what went wrong in ec2
<lifeless> ~1 hour to fix, another 5 to bb
<lifeless> so 11 if we try to fix ourselves
<lifeless> vs 5 if we land a revert
<wgrant> We really need to fix those LXC hangs.
<lifeless> we can expect about 3 revs to land if we were to ec2 it and have it land ok
<lifeless> wgrant: the sudo thing?
<wgrant> lifeless: No, what I believe to be a reliable LibrarianLayer setup hang on one of the 6 workers.
<lifeless> I haven't seen that
<wgrant> Have you run the whole suite parallel?
<lifeless> anyhow, I'm +1 on a revert
<lifeless> wgrant: yes
<wgrant> HMm.
<lifeless> wgrant: other than the sudo thing, it seemed ok in terms of not hanging
<wgrant> I need more RAM.
<lifeless> wgrant: it helps; I have 16GB :P
 * wgrant reverts.
<wgrant> wallyworld_: Morning
<wallyworld_> hello
<wgrant> wallyworld_: Any idea what creates that horrible broken red picker error box?
<wgrant> Ah, activator.renderFailure, it seems.
<wgrant> Kill.
<wallyworld_> i looked a little at some point in the past and sort of found some generic error handling stuff but that was while ago
<wallyworld_> activator.xxx sounds right
<wallyworld_> it's pretty ugly
<wallyworld_> at least it's out of lazr-js now and we can easily fix it :-)
<wgrant> It includes the error page's HTML in a <pre>.
<wgrant> Yeah, pretty ugly.
<wallyworld_> reminds me, i need to delete the lazr-js eggs
<wgrant> I did that on Saturday.
<wgrant> Cleaned out download-cache.
<wallyworld_> oh cool
<wgrant> Except for a couple of things that are still needed by deployed code.
<wgrant> 2 revs to go.
<wgrant> One of them seems to be designed to time out.
<wallyworld_> wgrant: i was just about to qa my bug, you beat me to it. did you login as a non-priviledged user to test. if so, is there a generic user that we use for such things on qas?
<lifeless> is there a setup.py setup() parameter for test only requirements ?
<wgrant> lifeless: Sort of. Check txlongpoll
<wgrant> lifeless: You declare it in extras_require.
<wgrant> lifeless: Not sure how to install them without buildout (where you say, eg., 'txlongpoll [test]').
<wgrant> wallyworld_: I have a few test SSO accounts. I don't know of any generic one.
<wallyworld_> ok. thanks
<wgrant> LOSA ping
<wgrant> Woah
<wgrant> impressive.
<wgrant> It doesn't time out.
<wgrant> It hung chromium.
<wgrant> But it didn't time out.
<lifeless> garh
<lifeless> hth does anyone use bootstrap.py
<wgrant> (getBranchTips with no since date, so it returned the tip for every Ubuntu branch)
<wgrant> Oh?
<wgrant> It is a terrible terrible thing, but it works.
<lifeless> :!./bootstrap.py
<lifeless> Downloading http://pypi.python.org/packages/2.7/s/setuptools/setuptools-0.6c11-py2.7.egg
<lifeless> While:
<lifeless>   Initializing.
<wgrant> Ah, yes.
<lifeless> Error: The specified download cache:
<lifeless> '/home/robertc/source/launchpad/python-pgbouncer/working/download-cache'
<lifeless> Doesn't exist.
<wgrant> make
<wgrant> Most people use a global download-cache.
<lifeless> yes
<wgrant> But the makefile will create download-cache for you.
<lifeless> bringing up a new project
<wgrant> Length: 31270702 (30M) [application/json]
<wgrant> Saving to: 'ubuntu?ws.op=getBranchTips'
<wgrant> I am *really* impressed that it rendered.
<lifeless> me too
<lifeless> its a new page
<wgrant> Getting like 700KB/s, too.
<lifeless> it should have a 5 second hard timeout
<wgrant> More than I've ever got from the DC.
<wgrant> I geuss it comrpesses.
<lifeless> can you get such a timeout put on it please?
<wgrant> Not sure if we can.
<lifeless> its a api call right?
<lifeless> it will have a page id
<wgrant> Yes. I forget how pageids work there.
<wgrant> Oh, right, a third bit.
<wgrant> Distribution:EntryResource:getBranchTips or so?
 * wgrant checks.
<hloeung> wgrant: hi
<lifeless> Distribution:EntryResource:#getBranchTips I think
<wgrant> hloeung: Sorry, thought qastaging had blown up, but it survived.
<wgrant> lifeless: Yeah.
<wgrant> lifeless: Is that your +1 for the flag change?
<lifeless> yes
<lifeless> you don't need one on qastaging
<lifeless> (a +1 that is)
<wgrant> hloeung: Could you please add the feature flag 'hard_timeout pageid:Distribution:EntryResource:#getBranchTips 5 5000' to qastaging?
<hloeung> wgrant: ok done
<wgrant> Oh, last_scanned_id is stored in branch. That's cheating.
<wgrant> hloeung: Thanks.
<wgrant> lifeless: Still renders after the second try!
<lifeless> neato
<lifeless> I bet it fails the '99% < 1 second' test, but reliably under 5s is sufficient to make me happy.
<lifeless> :!bin/buildout
<lifeless> Develop: '/home/robertc/source/launchpad/python-pgbouncer/working/.'
<lifeless> /usr/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'extra_requires'
<lifeless>   warnings.warn(msg)
<wgrant> lifeless: extras_require
<wgrant> (yay)
<lifeless> so how do you trigger the test deps ?
<wgrant> With buildout you say 'eggs = some.package [test]'.
<wgrant> In the section that builds the testrunner.
<wgrant> Or, potentially, in the main section.
<wgrant> You can also do 'easy_install txlongpoll[test]', I believe.
<wgrant> Not sure if setup.py develop supports it.
<lifeless> [scripts]\neggs=pgbouncer[test]
<lifeless> seems to trigger it
<wgrant> Yeah.
<lifeless> now I need to find out why van.pg isn't findable
<wgrant> pgbouncer egg?
<lifeless> new fixture
<wgrant> What does it do that van.pg doesn't?
<lifeless> pgbouncer
<wgrant> Why do we want pgbouncer?
<lifeless> fastdowntime
<lifeless> less idle connections on db servers
<wgrant> No, why do we want it in the test suite?
<lifeless> fastdowntime behaviour checking
<wgrant> It's not sufficiently transparent?
<wgrant> So we must introduce a new layer of slowness, pain, death and destruction into the web of trouble that is our postgres fixture already? :(
<lifeless> no
<lifeless> we need 3-4 tests that use this
<wgrant> Ah
<wgrant> With a better name than python-pgbouncer, sounds good.
<lifeless> suggestions welcome; I already discounted pgbouncefixture
<wgrant> Why?
<wgrant> pgbouncer is a tad generic.
<lifeless> it is
<lifeless> but I can see us wanting a fixture, a config generator, and some glue logic to rewrite user configs and so forth
<wgrant> I have reservations around polluting the global namespace like this.
<wgrant> It's alright if it's under lazr, where it can only kill us...
<lifeless> I looked for existing modules
#launchpad-dev 2012-07-09
<huwshimi> I had an ec2 test failure on Friday, but didn't receive an email with the details. Any ideas why not?
<wallyworld> huwshimi: sometimes that happens if the attachment is too big (too many errors). a while back, there was also a config change for bzr. is this your first landing in a while?
<huwshimi> wallyworld: Yeah, in ages
<wallyworld> huwshimi: so let me see if i can recall the config change
<huwshimi> wallyworld: Thank mate!
<wallyworld> huwshimi: back in march there was an issue where the email settings in your .bazaar config file needed to be under a section header, but i believe ec2 land was fixed
<huwshimi> wallyworld: Hmmm
<huwshimi> wallyworld: So my current config should still work then?
<wallyworld> i believe so
<wallyworld> mine has the smtp server settings at the top, not under a section header
<huwshimi> wallyworld: Mine are under [DEFAULT]
<wallyworld> my email address and lp user name is under [DEFAULT] but the smtp settings are above that
<wallyworld> but i think either way works
<wallyworld> so i'm not sure what else to suggest of the top of my head
<huwshimi> wallyworld: All good, thanks anyway. I'm going to resubmit it and see if it was a once off
<wallyworld> huwshimi: there's options you can pass to keep the session open after testing has finished
<wallyworld> so you can ssh in and look around
<huwshimi> wallyworld: Oh, I've already submitted this...
<huwshimi> (22 minutes ago)
<wallyworld> see how it goes then. you can ssh in or point your browser at the logged web address while the tests run
<wallyworld> or you can run ec2 ls every so ofter
<wallyworld> it will tell youif any tests have failed
<huwshimi> wallyworld: Yeah, that's what I've been doing :)
<huwshimi> wallyworld: Thanks mate, I'll see how I go
<wallyworld> good luck!
<cinerama> hey guys, i have a launchpadlib question
<cinerama> is there a way i can get the openid identifier for a user?
<wgrant> cinerama: No -- that doesn't make sense.
<wgrant> cinerama: It makes sense to go from an OpenID identifier to a user
<wgrant> But not vice-versa
<cinerama> wgrant: for my case, it does :) but i can understand why it might not more generally
<wgrant> What is your case?
<wgrant> It's probably not actually what you think it is :)
<cinerama> wgrant: we need to provide openid urls to HR
<cinerama> wgrant: when the new users sign up
<wgrant> Ah, $HIDEOUS_HR_TOOL, my old nemesis.
<cinerama> i have a bit of shell i can use but i was hoping to do it in a tidier way. maybe that's still the go though
<wgrant> Are you using the undocumented illegal interface on https://launchpad/~username?
<wgrant> People can have more than one OpenID identifier, so that's not reliable.
<cinerama> mm
<cinerama> it might have to be good enough for the particular application to assume 1
<wgrant> Lots of applications have thought that, and then they send users after us when it doesn't work.
<wgrant> Despite the fact that we told them not to do that.
<cinerama> i know it's awful, but given the fairly limited application of what i'm doing, i think it will be OK
<wgrant> Potentially. Just don't send them after us when it breaks :)
<lifeless> cinerama: what does HR want with open id urls ?
<cinerama> lifeless: it's for one of the HR systems
<cinerama> lifeless: doing a bit of work on gathering information for new hires
<lifeless> sure
<lifeless> but, to what purpose - I mean, can we define something to cross check to identify *which* openid they need?
<wgrant> I suspect it actually has nothing to do with Launchpad.
<wgrant> It's just that SSO doesn't provide a way for a user to see their identity.
<wgrant> So they key off LP username instead for setup.
<wgrant> So they ask for LP username when they should really ask for SSO ID
<StevenK> wallyworld: Can haz QA for r15567?
<wallyworld> StevenK: i did that first thing this morning, i must have messed up settign the tag
<wallyworld> fixed now
<StevenK> Except the QA tagger won't update for ten minutes
<StevenK> Which gives me that long to figure how the heck I'm going to QA my IBranch.visibleByUser() change
<wgrant> StevenK: We need to QA danilo's stuff too.
<wgrant> I don't want to deploy the visibleByUser stuff without my two testfixes.
<StevenK> That's a point
<StevenK> At least r15577, 15576 can be ignored on prod
<wgrant> Hm?
<wgrant> <15577 is possibly a memory leak on prod
<wgrant> It was Sunday so I didn't dig into it deeply.
<StevenK> wgrant: That still doesn't answer my concern about how ot QA it :-)
<wgrant> StevenK: Check that things don't time out too much, and that branches have approximately correct access rules.
<wgrant> Nothing's going to be too broken, since Code's tests are pretty good.
<StevenK> I've hit a few branch pages on qas already
<wgrant> Listings etc. too
<StevenK> https://code.qastaging.launchpad.net/launchpad seems pretty snappy, but almost all of the branches are public
<wgrant> Look, a bug!
<StevenK> :-(
<wgrant> StevenK: Open up the page's query log
<wgrant> Search for branchsubscription
<wgrant> wtf a bit
<StevenK> Bah
<wgrant> Also, the page is hardly snappy, but the core query is like 30x faster than prod, which often times out.
<wgrant> So it's qa-not-quite-awesome-but-close-enough-for-now
<wgrant> Even better if you fix those queries or convince me to
<StevenK> My plan for this afternoon was going to staple auditor to packageupload
<wgrant> I shall fix them :)
<wgrant> Need a full rewrite, anyway
<wgrant> I'm not actually sure where they are...
<StevenK> wgrant: You can link to bug 933938, I've slammed it back to In Progress
<_mup_> Bug #933938: Migrate branch visibility rules to project sharing <disclosure> <qa-ok> <sharing> <tech-debt> <Launchpad itself:In Progress by stevenk> < https://launchpad.net/bugs/933938 >
<wgrant> Thanks.
<wgrant> Hmmm
<wgrant> BMP needs product denormed.
<StevenK> It does?
<wgrant> StevenK: Even without access checks the pending MP query is 220ms for launchpad.
<wgrant> Because it has to do a nested loop
<wgrant> To find the MPs for each branch.
<wgrant> (it could potentially do a hash join, but that wouldn't be significantly better)
<StevenK> wgrant: So, looks like r15572 is bad, it causes an OOPS.
<wgrant> StevenK: Link?
<StevenK> https://qastaging.launchpad.net/ubuntu/+milestone/ubuntu-12.04-beta-1 and https://oops.canonical.com/oops/?oopsid=OOPS-7ec83b58aab8ad14b1b194bf93116da6
<wgrant> Oh
<wgrant> I didn't think to look in the template
<StevenK> Guess deploying today was too much to hope for. :-P
<wgrant> StevenK: Please revert that rev.
<StevenK> And sinzui is *still* landing stuff
<StevenK> wgrant: r15572 has been rollbacked in r15583.
<StevenK> wgrant: What does ComponentLookupError: (<InterfaceClass lp.services.webapp.interfaces.IPlacelessAuthUtility>, '') mean?
<wgrant> StevenK: You're in the wrong layer.
<wgrant> StevenK: DatabaseLayer rather than DatabaseFunctionalLayer, most probably.
<wgrant> Or UnitTests rather than DFL
<StevenK> wgrant: Yeah, AuditorLayer inherited from BaseLayer. I moved it up the stack.
<StevenK> Sigh, I might need a commit() or to move it further up than LFL
<wgrant> StevenK: Why?
<wgrant> Why's it interacting with the auth system at all?
<StevenK> wgrant: TestCaseWithFactory
<wgrant> Ah
<wgrant> StevenK: Er
<wgrant> StevenK: What are you trying to do?
<wgrant> A test of auditorlayer probably shouldn't be using the factory.
<wgrant> And a test of something using the auditor shouldn't be in AuditorLayer.
<wgrant> It should probably be in LaunchpadFunctionalLaye
<wgrant> r
<wgrant> With the minor issue that this screws everything up.
<wgrant> Because a lot of actions are going to want to talk to the auditor.
<wgrant> So this will make just about every test require LaunchpadFunctionalLayer.
<wgrant> Which is slow and stupid.
<lifeless> why does auditor laye rneed lp functional layer ?
<wgrant> We don't run other services in DatabaseFunctionalLayer
<wgrant> So anything needing the auditor probably has to use LaunchpadFunctionalLayer
<lifeless> not what I meant
<lifeless> the layer itself should inherit baselayer only, surely?
<wgrant> Right
<wgrant> And that's fine.
<wgrant> Except StevenK apparently needs the factory.
<wgrant> So the test probably shouldn't be in AuditorLayer
<StevenK> wgrant: I'm trying to staple IPackageUpload.acceptFromQueue() to send a POST to a configured auditor instance if a feature flag is set.
<wgrant> StevenK: Right, that needs DatabaseLayer, FunctionalLayer and AuditorLayer
<wgrant> Which probably means LaunchpadFunctionalLayer
<StevenK> So I could certainly decouple this, and have a function that only requires auditor and test it with AuditorLayer, and then have a PU test that mocks out that function call
<wgrant> We don't have an existing pattern for this.
<wgrant> We probably should.
<wgrant> stub: Hi
<StevenK> Well, my thought is if everything is going to be throwing data at the auditor (and retrieving it, but meh, details), we should make it easy
<stub> yo
<wgrant> StevenK: Certainly
<StevenK> wgrant: I can hold off if you, lifeless and I want to talk about how to best to do that.
<wgrant> stub: I have three index patches up for review, if you have time today.
<StevenK> I've spent a few hours on this, but I can toss it away for a better idea when we have a plan
<lifeless> StevenK: nothing sounds alarming about what you are describing
<StevenK> lifeless: Which bit? A function call in acceptFromQueue that is mocked out?
<lifeless> StevenK: 18:13 < StevenK> wgrant: I'm trying to staple IPackageUpload.acceptFromQueue() to send a POST to a configured auditor instance if a feature flag is set.
<lifeless> 18:15 < wgrant> StevenK: Right, that needs DatabaseLayer, FunctionalLayer and AuditorLayer
<lifeless> 18:15 < wgrant> Which probably means LaunchpadFunctionalLayer
<lifeless> StevenK: I don't see much point in mocking internally as well
<wgrant> lifeless: This means basically every Launchpad test will start up the librarian etc.
<wgrant> Which is insane.
<wgrant> Because that's slow and pointless.
<lifeless> StevenK: we've made auditor's test environment lightweight precisely to avoid spurious mocking
<lifeless> wgrant: why does it mean that ?
<wgrant> Because LaunchpadFunctionalLayer means that
<lifeless> and what means LFL here ?
<wgrant> 16:20:45 < lifeless> 18:15 < wgrant> StevenK: Right, that needs DatabaseLayer, FunctionalLayer and AuditorLayer
<wgrant> 16:20:48 < lifeless> 18:15 < wgrant> Which probably means LaunchpadFunctionalLayer
<lifeless> wgrant: for the test that StevenK is writing.
<lifeless> wgrant: which is >< 'basically every Launchpad teset'.
<lifeless> wgrant: you'll need to unpack what you see.
<StevenK> wgrant is predicting the future
<wgrant> lifeless: So, I have these methods
<wgrant> On model objects
<stub> I can never find my reviews-to-do page
<wgrant> They talk to the auditor service
<wgrant> The auditor service is not there, because we don't mock it out :(
<wgrant> I die.
<wgrant> stub: http://code.launchpad.net/~stub/launchpad/+activereviews
<stub> ta
<wgrant> (you could be forgiven for not finding it; there is no link)
<lifeless> wgrant: same same for Librarian etc.
<wgrant> lifeless: Except many fewer things talk to the librarian implicitly.
<wgrant> lifeless: Basically everything will talk to auditor.
<wgrant> Implicitly.
<StevenK> lifeless: Yes, but you don't need the librarian for say, merging a person, deleting a team, copying a package ...
<stub> wgrant: Didn't we already create a trigram index somewhere, and install the extension?
<lifeless> ok, so
<lifeless> this relates back to using storm objects vs POP model objects
<lifeless> etc
<lifeless> etc
<wgrant> stub: That's this branch that we discussed but never got reviewed.
<stub> ok
<lifeless> I've -no- objection to tested fakes
<wgrant> lifeless: Sure, but it's a big regression from what we have now.
<lifeless> if you want to write one.
<wgrant> This fake can indeed be so fake that it does nothing at all, I suspect.
<lifeless> wgrant: define big; how many ms to start up the auditor test fixture, how many ms to send the reset POST to it, how many ms to audit an item
<wgrant> lifeless: +11s to start the librarian and 4s for rabbitmq
<wgrant> Since we don't have fixtures.
<lifeless> however, if you write one, please make it a tested fake. Do not use a mock
<lifeless> why is the librarian and rabbitmq +times here ?
<lifeless> - why are they not already being used ?
<lifeless> - if they aren't being used, why are they being dragged in?
<wgrant> lifeless: We don't have fixtures, and we probably don't want len(resources)! test layers.
<wgrant> librarian isn't used by much code, fortunately, so it is rarely required unless you're molesting Soyuz or using makeBugAttachment.
<lifeless> I'm thoroughly lost.
<lifeless> This seems simple to me.
<lifeless> Add the layer needed with the minimal closure of dependencies, per the existing pattern.
<lifeless> Which has problems yes, but works.
<wgrant> It's also an excellent path to a 12-hour test suite :/
<lifeless> You seem to be saying we should do something different, which we could, but it has the problems you articulate of bringing up unneeded daemons.
<lifeless> doing what we currently do of making a tailored combination of layers, is much less of a latency problem for tests.
<lifeless> -> dinner.
<wgrant> stub: btw, did you see that qastaging and staging were failing to connect to their respective DBs for >24h over the weekend? staging should have been down, but qastaging should not.
<wgrant> lifeless: Well
<wgrant> lifeless: We're going to have one layer for services that are expensive.
<stub> wgrant: yeah, that was me
<wgrant> And another layer for things like auditor that aren't expensive yet but will probably be in 6 months...
<wgrant> stub: Ah, good.
<wgrant> stub: I was a bit worried, since code+logs said it should not have been down.
<stub> wgrant: THe staging rebuild scripts work fine now apart from controlling pgbouncer, I need to fit that in somewhere
<wgrant> stub: Yeah. I guess we need to divorce them.
<stub> yeah
<stub> alas standard debian startup scripts don't support that, so need to create those and get them blessed
<wgrant> Yeah
<stub> wgrant: I'm thinking we might want a test helper that confirms a query uses an index.
<stub> but there might be false positives since we would need to disable sequential scans.
<wgrant> stub: The lack of stats makes them pretty difficult to do, even with seqscans disabled.
<stub> hmm...
<stub> so maybe a smoke test to run on staging
<wgrant> Possibly, yeah.
<wgrant> It'd be great to have *something* :/
<wgrant> stub: Yeah, the idx2s will get renamed in the patch that deletes the old ones.
<lifeless> wgrant: so your conclusion is, that StevenK should add just the services he needs to this test, not all of launchpadfunctionaltestlayer
<wgrant> lifeless: Well
<wgrant> lifeless: That solution is neither practical nor ideal.
<wgrant> So I would discourage it.
<StevenK> I've decided to write an AuditorClient so we can abstract out all of the rubbish from say, packageupload.
<StevenK> Otherwise wgrant *will* murder me.
<wgrant> Consider it done.
<wgrant> stub: So, feel like trying to apply the trigram stuff live?
<wgrant> Oh, bah, backups.
<stub> ya, dem backups
<wgrant> stub: We almost need to move the backup, actually
<stub> Might more to hot backups one day but they are harder to work with
<wgrant> stub: It's been finishing dangerously close to fdt lately.
<wgrant> nightly.sh completed Sun Jul 8 09:48:03 UTC 2012
<wgrant> nightly.sh completed Sat Jul 7 09:56:39 UTC 2012
<stub> webops can reschedule it. Sounds like moving it forward another few hours would be good.
<stub> unless we have wrapped around :P
<StevenK> Moving it forward would completely block FDT
<wgrant> StevenK: Forward != backward
<stub> forward as in earlier, the other forward
<StevenK> Time iz hard, let's go shopping.
<lifeless> wgrant: ideal involves fixtures, more or less; practical it is though.
<wgrant> lifeless: It's not particularly practical, because we don't currently have a way to specify a subset of services without drawing arbitrary lines or introducing len(services)! layers
<lifeless> there is little consequence of adding addtional layers.
<lifeless> and we don't need len(services)! layers, thats the pathological case which we are not at.
<wgrant> We'll see.
<lifeless> wgrant: I appreciate you feel strongly about this, but the data I see don't support the conclusion you're pushing.
<wgrant> lifeless: I want to avoid repeating the problem we have now where you can have this nice fast test.
<wgrant> Then you realise you need a bugattachment
<wgrant> And now your test takes 20s to start.
<lifeless> I understand. The general solution is to finish the separation of store and logic layers
<lifeless> bring in tested doubles across the board, have no ORM involved in logic tests, only in store and fetch tests.
<lifeless> and treat the orm and other services homogeneously
<wgrant> Right.
<wgrant> Although s/finish/start/
<lifeless> ian has started it :)
<lifeless> though I'm still waiting on the report IIRC.
<wgrant> In other news, I think I might try to deploy my BugSummary patch tonight.
<lifeless> my point though, is that holding new work hostage to the general solution just slows down the new work, which is only incrementally worse - and locally at that, not globally - and doesn't improve the overall situation at all.
<wgrant> lifeless: It will soon be pretty much global.
<lifeless> when in fact, the new work is*part* of the general solution, though its quite circuitous to get there solely through new work.
<lifeless> wgrant: how long does auditorfixture take to come up for you?
<wgrant> I've not dared to try.
<wgrant> But I imagine it's at least a few hundred milliseconds
<lifeless> so, get data!
<adeuring> good morning
* adeuring changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: - | Firefighting: - | Critical bugs: 4.0*10^2
<jam> jml: did you get a chance to look into the httplib thing? it is weird, I'm seeing it trying to go to an "http://" URL (api.launchpad.dev/devel) trying to get the WADL. I wonder if setting disable_ssl_certificate_validation confuses things if the URL *isn't* http
<jam> ah, ok, we get a redirect to the HTTPS side
<jam> jml: ugh.... just ugh. wsgi_intercept is screwing us
<jam> it is overriding httplib2.HTTPSConnectionWithTimeout the 'variable' with class wsgi_intercept.httplib2_intercept.HTTPS_WSGIInterceptorWithTimeout
<jam> and then this check in httplib2: issublcass(connection_type, HTTPSConnectionWithTimeout) fails
<jam> because we got the connection_type from SCHEME_TO_CONNECTION, which wsgilib did *not* override
 * jam goes in the corner and cries...
<jam> and with upgrading wsgi_intercept to 0.5.1, the tests now pass
<jam> finally
<jml> jam: no, sorry.
<jml> jam: I'm glad they're passing now. I'm sorry the world is so horrible.
<jam> jml: just so many layers of indirection
<jam> and not expecting monkey patching changing the class definitions
<jam> pdb + stepping helps a lot
<jml> jam: there's a lot of action-at-a-distance in Launchpad's test suite.
<jam> jml: yeah, and it is also encouraged by the libs. lazr.restful uses launchpadlib which wraps httplib2 which actually uses httplib, but wsgi_intercept comes in the middle, while ...
<psychognite> sir i have a project to submit on ubuntu showdown
<psychognite> i have created openpgp keys and also signed in code on coduct ...what to next
<psychognite> sir please help me out .....
<psychognite> !!
<jml> psychognite: next up you need to create a PPA, I think.
<jml> psychognite: oh right, you're on #ubuntu-app-devel, that's a much better place to get help.
<psychognite> sir i have created ppa then how to upload files their
<rick_h_> morning
<jam> cjwatson: any chance to get an update about the maverick disk-space-cleanup? Even just point me in the right direction to run the script myself. I just didn't understand what files you were using to compare against
<jam> cjwatson: also, I'd like to get a feel for what we could do in launchpad to make it easier for everyone to do this in the future
<cjwatson> jam: I'll try to do it today.  I'm comparing against all the Packages files that are actually published in the archive - you know, real data ;-)
<jam> cjwatson: sure, but what is the actual files? I think you mentioned not downloading the stuff from old-releases.ubuntu.com
<cjwatson> jam: I think this should always require Ubuntu signoff, though - I don't think it's OK for Launchpad staff to delete Ubuntu files out of the librarian without asking us first
<jam> cjwatson: basically, I'd like to at least file bugs that could be approached in the future to make things easier
<cjwatson> jam: Packages files are well-known files published in the Ubuntu archive
<cjwatson> You could get them from the dists tree on old-releases if you liked - I just used cocoplum (ftpmaster) since that's quicker for me
<jam> cjwatson: you just grab the individual package listing for each arch?
<jam> and cat them together?
* benji changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: benji | Firefighting: - | Critical bugs: 4.0*10^2
<cjwatson> It was a bit more than that.  I'll keep notes and mail you
<jam> cjwatson: thanks, along with the script you run would be good to get it all together.
<cjwatson> But this is a sanity check that ought to be performed on the Ubuntu side by somebody familiar with the archive layout
<jam> I'm happy to have Ubuntu in the loop
<cjwatson> I don't want to over-mechanise it since the point is to be a sanity check against database queries
<cjwatson> Totally scripting it so that it can be run with no understanding of the archive layout defeats the purpose slightly, I think :)
<adeuring> rick_h_: could you please review this MP: https://code.launchpad.net/~adeuring/lazr.jobrunner/bug1015667-2/+merge/113962 ?
<adeuring> rick_h_: ping
<jam> abentley: /wave . I'm not sure if you're the person to ask, but 'sendbranchmail' seems to be stuck running for 3.25hrs. Is it safe to just kill it?
<abentley> jam: otp
<abentley> jam: I think it's safe.  It runs a series of jobs, so at worst one job fails.
<jam> dpm: just a quick check about where translations are at, are we still opening it up?
<jam> abentley: so it looks like it took about 3.5hrs and 1.5GB of memory to process a mysql branch, but did, eventually, succeed. Is there any way to get visibility into what happened?
<jam> 012-07-09 13:31:39 INFO    Ran 1 RevisionMailJobs.
<jam> 012-07-09 13:31:39 DEBUG   sendbranchmail ran in 13111.023446s (excl. load & lock)
<jam> after that finally finished, the other jobs look like "ran 60 jobs in 180s" etc.
<abentley> jam: I guess you could look into the mail server logs to see whether it was actually sending lots of email.  That's all I can think of.
<jam> abentley: well, ps at the time basically said it was using 100% CPU for those 3 hours
<jam> we caught it just before it finished
<jam> https://pastebin.canonical.com/69662/ (215min of CPU usage, if I'm reading that correctyl)
<abentley> jam: It may have been walking the revision tree.  Let me refresh my memory.
<jam> abentley: it does look like the recent commits could have been big: https://code.launchpad.net/~mysql/mysql-server/5.5
<jam> a few 'empty weave merge of 5.1-security => 5.5'
<abentley> jam: So I don't see a smoking gun.  If this was a RevisionAddedJob, it could be down to find_unique_ancestors, but if it's a RevisionMailJob, I don't think it can.
<abentley> jam: Ah, it's not distinguishing between types in the log message, so it could have been a RevisionAddedJob.
<jam> abentley: yeah, they seem to just get appeneded
<jam> the log file doesn't have any "Ran ? RevisionAddedJob' so I think it is just len(queue) and queue has both types
<jam> abentley: thanks for poking into this with me. I'm at EOD now, so no need to dig much further. Though I guess making sure the process doesn't get blocked in the future would be good.
<benji> gary_poster: I'll get it
<abentley> adeuring: We do late imports of Celery-related things precisely *because* importing from Celery has the side effect of configuring Celery.
<adeuring> abentley: right. But doing an import twice -- in two tests -- does not work well: The second import is ignored, if I am not completely mistaken.
<abentley> adeuring: Yes, of course.
<adeuring> abentley: and we also need the late imports to set up the proper config module, in tests as well as in real life.
<abentley> adeuring: This is a bug in celery that was going to be fixed  in the next release.
<adeuring> abentley: ok, but for now we have to live with the problem, I'm afraid...
<abentley> adeuring: The tests were written under the assumption that there was a specific configuration used for the tests.
<abentley> adeuring: And only the celeryd configurations would vary.
<adeuring> abentley: ok, that's fine for most tests, but the clear-aqueues script should be tested with two configs, I think
<adeuring> ...like celeryd
<abentley> adeuring: is there any advantage in using the running context manager instead of using subprocess.call or something?
<adeuring> abentley: it just saves the wait() call
<abentley> adeuring: But communicate already waits for the process to terminate.
<adeuring> abentley: ouch, yes, youa are right. So I'll revert running() and use Popen  directly
<abentley> adeuring: Cool.
<adeuring> abentley: chnages pushed
<abentley> adeuring: r=me
<adeuring> abentley: thanks
<abentley> adeuring: btw, Celery 3.0 is out now.
<rick_h_> sinzui: ping, have a few min?
<sinzui> I do
<jam> benji: any chance you could look at: https://code.launchpad.net/~jameinel/launchpad/py27-introduction-1020667/+merge/113938 it is mostly just updating versions.cfg to newer packages, and then updating bin/test to pass an env var so that httplib2 doesn't try to check the ssl cert during the test suite.
<jam> (Its the last patch for python-2.7 compatibility)
<benji> jam: sure
<benji> cool
<jam> benji: thanks, what is the launchpad pqm email address again? launchpad@pqm.canonical.com?
<jam> (I ran it through ec test already, since I wanted to be safe with package updating, and I don't think I have lp-land configured correctcly)
<benji> jam: right, launchpad@pqm.canonical.com
<jam> thanks
* benji changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: - | Firefighting: - | Critical bugs: 4.0*10^2
<mwhudson> is there a js version of launchpadlib?
<wallyworld_> sinzui: mumble hates me this morning. it refuses to connect
<sinzui> Yet I see you in mumbe?
<wallyworld_> sinzui: the mumble window becomes non responsive and takes 100% cpu
<sinzui> ah
<sinzui> I had to restart when that happened. I think access to audio was broken
<StevenK> wallyworld_: I've seen that -- it seems to happen when you quit mumble and the process does not die.
<wallyworld_> StevenK: it wasn't running this time
<StevenK> wallyworld_: Not even in 'ps aux | grep mumble' ?
<mwhudson> i think mumble just decided to hate people every now and again
<wallyworld_> StevenK: nope, nothing there
<StevenK> I'd suggest 'strace', but you'll hate me.
<sinzui> wallyworld_, https://bugs.launchpad.net/launchpad/+bug/206811
<_mup_> Bug #206811: Bug feeds for BugTargets need to ensure a sufficient number of bugs are fetched <feeds> <lp-bugs> <tech-debt> <Launchpad itself:Triaged> < https://launchpad.net/bugs/206811 >
<wgrant> wallyworld_: +        return getUtility(IBugSet).getDistinctBugsForBugTasks(
<wgrant> +            self.context.searchTasks(params), self.user, limit=5)
<StevenK>   Running:
<StevenK>  lp.registry.tests.test_sharingjob.RemoveArtifactSubscriptionsJobTestCase.test_admins_retain_subscriptions^C^C^C^\zsh: quit (core dumped)  bin/test -vvt test_sharingjob
 * StevenK stabs things.
<sinzui> Finally. I can see an icon on the bug listing that does not lie or hurt my eyes
#launchpad-dev 2012-07-10
<lifeless> wgrant: thanks
<StevenK> Hmmm, looks like celery is causing the hang.
<StevenK> If I kill the celery main process the test runner actually exits at least.
<nigelb> lifeless: looks like I won't be seeing you soon afterall.
<StevenK> nigelb: No NZ for you?
<nigelb> looks like.
<nigelb> Can't say for sure though, sorting out what's next.
<mwhudson> :(
<sinzui> StevenK, wgrant, wallyworld_ the maintainers of https://launchpad.net/akiban-server cannot access branches
<wallyworld_> private branches or any?
<wgrant> They're all private
<wgrant> sinzui: How'd you discover this?
<sinzui> https://code.launchpad.net/akiban-server/+activereviews gives a 403. I can see the page, but I see nothing in it since there is no BVP that covers me
<sinzui> wgrant, arielweil reported it to me
<sinzui> the issue started in the last 3 hours
<wgrant> sinzui: Cannot access branches, or cannot access +activereviews?
<sinzui> I hope he will join us in a minute
<wgrant> The latter has various known bugs that the recent changes may have exacerbated.
<sinzui> the page will not load
<wgrant> But it's a very very different situation from not being able to access the branches.
<sinzui> it is like something we iterated over raised an Forbidden error
<wgrant> I'm trying to track down the OOPS
<wgrant> Bug #900431 is what I was thinking of
<_mup_> Bug #900431: branch visibility queries do not consider visibility of stacked on branches <403> <branches> <disclosure> <privacy> <sharing> <Launchpad itself:Triaged> < https://launchpad.net/bugs/900431 >
<sinzui> ha, I was looking at that a few minutes ago
<wgrant> https://oops.canonical.com/oops.py/?oopsid=OOPS-8ccd6bd99a7dab48e98b2f44b941fff9
<wgrant> Ah, that's actually another akiban project
<wgrant> But same issue, probably.
<sinzui> wgrant, I expect that. They have projectgroup-level rules that are applied to all the projects
<sinzui> should I set of project-specific rules for them?
 * sinzui ponders the swift death of project groups
<wgrant> https://oops.canonical.com/oops.py/?oopsid=OOPS-2c591db5bd212de7766bf7afcbcbf591
<wgrant> For akiban-server
<sinzui> I see
<sinzui> users cannot see each other's branches because the owner attr is private
<sinzui> well not sufficient permission
<wgrant> Well
<wgrant> They should hold launchpad.View
<wgrant> Note that this is a newish branch
<wgrant> Just a couple of days old.
<wgrant> Can you see who is subscribed?
<wgrant> I guess not.
<wgrant> Oh
<wgrant> Oh no
<wgrant> Hm
<wgrant> Actually, codehosting should only create branches over XML-RPC...
<wgrant> So outdated codehosting code shouldn't matter.
<wgrant> Right?
<sinzui> I don't think that is a big concern at the moment
<wgrant> Hm?
<lifeless> nigelb: :( customs?immigration? job?
<nigelb> lifeless: immigration
<lifeless> nigelb: ugh :(
<lifeless> nigelb: did they say why?
<nigelb> Mostly, they couldn't get in touch with my current or previous employer.
<nigelb> I don't even know what they tried.
<nigelb> I have to wait till Mountain View wakes up to figure out what's next.
<lifeless> ah, thats very interesting.
<lifeless> would they reevaluate if they could ?
<nigelb> I suppose they willl.
<sinzui> nigelb, Sorry to hear that. The US Government makes it very hard to hide talent. They make companies and hiring managers to a lot of work because the US doesn't believe skill is needed for any job
<nigelb> sinzui: This is NZ, not US
<lifeless> sinzui: s/hide/hire/ ?
<sinzui> NZ is better I think/
<sinzui> yes hire
<sinzui> though I did think to try and hide on H1 visa employee I had. I hate to see good people go
<ajmitch> nigelb: I hope you can reapply & get approved
<nigelb> ajmitch: yeah, I hope so too.
<wgrant> wallyworld_: There were disturbingly few failures from the trigger removal. I've sent it back after fixing them.
<StevenK> How few, out of interest?
<wgrant> 23
<StevenK> wgrant: I don't get why I get test hangs with celery :-(
<StevenK> I ran bin/test under strace after lunch and I'm still waiting for my eyes to stop bleeding.
<wgrant> Yeah, it happens. Make sure all existing test processes, rabbits, etc. are dead.
<lifeless> after 2 hours bleeding you'd hope they are already.
<StevenK> wgrant: http://pastebin.ubuntu.com/1083954/ it doesn't hang with the diff as it is, but if you uncomment the branch_invisible_filter line or the branch unsubscribe loop in run(), it does
<nigelb> Every time I'm about to expire from a team, I twich at the blatant lies the email says.
<StevenK> nigelb: You and me both.
<nigelb> StevenK: How fugly is it to fix?
<nigelb> I might be tempted to spin LP up in a VM for this :/
<StevenK> I keep getting stuck on wording
<nigelb> My problem is the "One more email" bit.
<lifeless> of course, the real irony here is that noone except a few team admins value that setup at all.
<wgrant> lifeless: Yeah, our new team creation forms go to some lengths to hide that, and we're removing the autorenew option because it's stupid.
<wgrant> We can't really remove the self-renewing, but we're going to conceal it.
<lifeless> autorenew is == nonexpiry
<lifeless> so I'm +1 on that
<nigelb> I also want to say "I know. Keep quiet".
<lifeless> self renewing, I'm fairly sure a rigorous set of interviews with stakeholders and users could remove
<lifeless> I'm glad you're going to make it less prominent in the short term
<wgrant> lifeless: Ubuntu uses it extensively to try to prevent teams from accumulating cruft.
<lifeless> yes
<lifeless> Does it work?
<nigelb> no :P
<wgrant> Apparently it does :)
<nigelb> ...
<StevenK> Haha, you told him
<nigelb> Did I give him bad news?
<nigelb> Destroy his dream?
<nigelb> :P
<StevenK> That's okay, I'm sure he can come up with a good dream, half implement it and then have wgrant rewrite it.
<nigelb> Or have you delete it.
<spm> heya nigelb
<StevenK> wgrant: Sigh, http://pastebin.ubuntu.com/1083975/ is the actual diff
<StevenK> wgrant: So, those hangs were caused by one typo, and one referencing person_id instead of personID.
<StevenK>  /wrists
<wgrant> StevenK: Yeah
<wgrant> StevenK: This is the problem with running all the tests under celery
<wgrant> When we really should have more direct tests that are about 20x faster.
<StevenK> wgrant: So, I wanted to chat about it, anyway.
<StevenK> wgrant: I wonder if would be worth trying to nail the two queries into one
<wgrant> StevenK: Bugs and branches?
<StevenK> wgrant: Yeah
<wgrant> StevenK: You like to tempt fate.
<wgrant> I would not.
<StevenK> wgrant: Heh
<StevenK> wgrant: I can put up this branch at it stands then, and you can see if you vomit?
<wgrant> Yes.
<wgrant> I shall bring my own bucket.
<StevenK> Is it a red bucket?
<StevenK> wgrant: https://code.launchpad.net/~stevenk/launchpad/teach-rasj-about-branches/+merge/114106
<wgrant> stop-using-legacy-bug-access => devel     [OK]       (up for 2:52:53) i-86ee57fe
<wgrant> remove-legacy-bug-access => devel         [OK]       (up for 2:42:03) i-9ed069e6
<wgrant> Yay
<nigelb> hey spm :)
<StevenK> wgrant: http://pastebin.ubuntu.com/1084037/ per your comments
<wgrant> StevenK: Sounds good.
<StevenK> lifeless: Last use of IMemcacheClient is about to get removed.
<StevenK> I'm not sure if we want to drop it completly or keep it around for garbo jobs.
<wgrant> Keep it for now.
<wgrant> The infrastructure which allowed people to easily abuse it is gone, so it's not particularly problematic to keep the remains around.
<StevenK> Until we have something else, I guess.
<wgrant> We'll hopefully make use of memcache soon
<wgrant> Just in ways that make sense.
<StevenK> To do what?
<wgrant> Make pages lightning fast, perhaps.
<wgrant> Previously we've used memcache to take pages from timing out to near-acceptably glacial.
<wgrant> With no invalidation
<wgrant> Which is a completely inappropriate way to use it
<wgrant> memcached it great at taking pages from fast to lightning fast.
<wgrant> s/it/is/
 * StevenK races ec2
<StevenK> I wish AWS could start up an instance in a few seconds rather than ~ 180
<lifeless> StevenK: e.g. load in json blobs for entire pages, then do cheap business logic validation and render
<StevenK> lifeless: Right, so your plans to rip it out completly are on ice?
<lifeless> StevenK: I never put forward such a plan
<lifeless> StevenK: I ripped out inappropriate uses
<lifeless> StevenK: and said we definitely couldn't remove it entirely while it was used
<lifeless> but also that there are valid uses for it still
<lifeless> I guess that that is ambiguous at best
<lifeless> sinzui has expressed interest in entirely destroying it ;)
<adeuring> good morning
<stub> wgrant: The problem we found with memcache is there isn't much we can actually cache in the webapp.
<cjwatson> wgrant: what did you mean by "it'd be nice if we could fix BPB/PU permissions around copies"?
<cjwatson> BPBs seem to already have code to check whether SPRs have been copied to a visible archive, much as I just added to PU
<wgrant> stub: There's *tonnes*. Just not in the way we were doing it before.
<wgrant> cjwatson: Right, what you've done with PU matches BPB
<wgrant> cjwatson: So it'll do for now. But neither is correct.
* gmb changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: gmb | Firefighting: - | Critical bugs: 4.0*10^2
<danilos> gmb, hi, I've got a simple MP up (that's fixing qa-bad landing and thus includes the full patch that's already been reviewed): https://code.launchpad.net/~danilo/launchpad/bug-1021196/+merge/114157
<gmb> danilos, Looking.
<danilos> gmb, thanks, fwiw, new changes up at https://pastebin.canonical.com/69731/
<gmb> danilos, Approved.
<danilos> gmb, woohoo, thanks
<jam> dpm: did anything happen with opening Q translations?
<danilos> gmb, do I need to use any special flags to mark the rollback as cleaned up now? (I assume not, but just checking)
<gmb> danilos, Not that I know of, no.
<danilos> cool, thanks
<dpm> hi jam, sorry for not having come back to you earlier. I'm working on the Ubuntu app showdown right now, and I won't be able to do any translations work until the end of the week. Let me come back to you then.
<jam> dpm: is there another Ubuntu-translations person? or is it just you?
<jam> (*I* don't need them open, but I figure Ubuntu wants to open them pretty soon)
<jam> Also, I was wondering if we would want to re-start the copy in case there were new P translations since we did the copy
<dpm> jam, anyone in the ubuntu-translations-coordinators team can open translations, but there are still some pending tasks, such as manually approving/blocking templates in the Q translations imports queue that should probably be done. I listed them on the e-mail I sent to the ubuntu-translations-coordinators list and to you before going on holiday
<dpm> but I'll get onto it by the end of the week, it's just that the app showdown is a higher priority right now
<jam> cjwatson: any chance you got to the filenames list yet?
<cjwatson> sorry, not quite yet
<Laney> how hard is the LoC delta policy? :-)
* rick_h_ changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: gmb, rick_h | Firefighting: - | Critical bugs: 4.0*10^2
<rick_h_> Laney: pretty solid, no credit atm?
<Laney> rick_h_: nah, this is only my second change. But it's only 30 lines. I'll find it.
<rick_h_> Laney: ok thanks. Looks ok otherwise, left a comment on your question on the test adjustment
<Laney> ty
<nigelb> Laney: ONE OF US, ONE OF US!
<nigelb> :D
<nigelb> (though I haven't written LP code in a long time:( )
<Laney> DO IT.
<cjwatson> kill off another doctest, that's usually a good candidate :)
<cjwatson> (where by "kill off" I mean "rewrite as unit tests" of course)
<cjwatson> wgrant: I think it should be safe to look at https://code.launchpad.net/~cjwatson/launchpad/queue-api-fix-urls/+merge/113776 again now; the PU security changes have landed and I fixed it to redirect everything through the webapp
<jcsackett> rick_h_, gmb: can one of you look at https://code.launchpad.net/~jcsackett/launchpad/hidden-comment-count-error/+merge/114219
<rick_h_> jcsackett: loading up
<jcsackett> thanks.
* gmb changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: rick_h | Firefighting: - | Critical bugs: 4.0*10^2
<rick_h_> jcsackett: so this /home/jc/wtf.html file?
<rick_h_> just for sanity checking hte tests?
<jcsackett> rick_h_: that should have been deleted. :-P
<jcsackett> one sec.
<jcsackett> ok, that line is gone and change is being pushed. sorry, rick_h_.
<rick_h_> jcsackett: k, also, not following the out of order logic. Shouldn't it be adding more time to the hidden comment so that the last one comes before it?
<jcsackett> no, because i need a visible comment before and after the hidden, but i need to move the hidden out of it's usual spot.
<jcsackett> so, i push the last one way out and push the hidden one slightly less out in time.
<rick_h_> ah ok, so the moving of the last assures there's something after, so this should drop -5 to -2 and the last stays last
<jcsackett> yup.
<rick_h_> ok gotcha.
<rick_h_> jcsackett: ok r=me
<jcsackett> thanks, rick_h_.
<jcsackett> sinzui: free to talk a bit?
<sinzui> jcsackett, in a few minutes yes, and only for a few minutes
 * sinzui has meeting at 2:00
<jcsackett> dig. i'll get on google+ post haste.
<jcsackett> sinzui: set up. you get the invite?
 * cjwatson wonders what the record is for number of branches attached to a single-task bug
* rick_h_ changed the topic of #launchpad-dev to:  http://dev.launchpad.net/ | On call reviewer:  | Firefighting: - | Critical bugs: 4.0*10^2
* rick_h_ changed the topic of #launchpad-dev to:  http://dev.launchpad.net/ | On call reviewer: -  | Firefighting: - | Critical bugs: 4.0*10^2
<lifeless> cjwatson: what ever it is, its too many.
<maxb> Could a member of ~launchpad nuke ~registry's bugmail subscription here, please: https://bugs.launchpad.net/unity-foundations/+subscriptions
 * sinzui loos
<sinzui> looks
<maxb> ugh, actually, there are lots
<maxb> https://bugs.launchpad.net/~registry/+structural-subscriptions
<sinzui> holy #!!!
<sinzui> someone deleted a team without cleanup
<sinzui> The subscriptions are gone.
<sinzui> I am checking branches
<sinzui> http://people.canonical.com/~curtis/lp-milestone/report.html
<sinzui> ^ StevenK, wgrant, wallyworld_
<wgrant> StevenK: You'll want to confirm that with jtv.
<StevenK> Yeah
<StevenK> wgrant: I'm looking at https://code.launchpad.net/~cjwatson/launchpad/queue-api-fix-urls/+merge/113776 , and I'm concerned by lines 215-219
<wgrant> StevenK: I haven't looked at the recent changes in that branch yet. Let me see.
<wgrant> win 45
<wgrant> StevenK: What worries you?
<wgrant> StevenK: It looks pretty good to me.
<wgrant> The lack of traversal to binaries is odd, but understandable.
<StevenK> wgrant: SPRF.one()
<wgrant> StevenK: It's restricting by filename
<wgrant> We have bigger problems if there's multiple SPRFs in one SPR with the same filename.
<StevenK> Ah
#launchpad-dev 2012-07-11
<huwshimi> I have some odd failures that have started appearing in my branches. http://pastebin.ubuntu.com/1085503/
<StevenK> huwshimi: Merge devel, wgrant fixed them last night.
<huwshimi> StevenK: Ah, I tried that yesterday afternoon, I guess I was a little too early
<huwshimi> Also, if I have a bzr pipe do I need to do anything special to 'ec2 land' that pipe. I'm sure I used to just make sure I had switched to the pipe and submit, but it doesn't seem to be working anymore
<StevenK> huwshimi: ec2 landing one pipe is easier if you run 'ec2 land <MP URL>'
<huwshimi> StevenK: Ah right, I'll give that a go. Cheers mate.
<StevenK> wgrant: So, should I look at that bug you assigned me?
<StevenK> wgrant: teach-rasj-about-branches is landing
<StevenK> Bah, wallyworld_ got r15600.
 * StevenK stabs him
<wgrant> StevenK: Great
<wgrant> wallyworld_: Huh, what's with the circular import thing you added to lib/lp/bugs/interfaces/bugtask.py?
<wgrant> wallyworld_: It's at the top level, so it can't be for circular import avoidance.
<StevenK> A mail from LP making the MP as merged before I get a success mail from PQM? :-(
<StevenK> wgrant: Oh, hah. You assigned that bug to me and then closed it.
<wgrant_> StevenK: Which bug? I think the freenode node I was on has died.
<StevenK> wgrant_: bug 1014625
<_mup_> Bug #1014625: pkgme should have one license <pkgme:Fix Released> < https://launchpad.net/bugs/1014625 >
<StevenK> BAH
<StevenK> wgrant_: bug 1014624
<_mup_> Bug #1014624: Sharing details page does not show branches <disclosure> <sharing> <ui> <Launchpad itself:Fix Released by stevenk> < https://launchpad.net/bugs/1014624 >
<wgrant> StevenK: Right, what about it?
<wgrant> Last I saw was:
<wgrant> 11:32:59 < wgrant> wallyworld_: It's at the top level, so it can't be for circular import avoidance.
<StevenK> [11:31] < StevenK> wgrant: So, should I look at that bug you assigned me?
<StevenK> wgrant: There's little point looking at it -- you closed it, I just didn't notice that bit.
<wallyworld_> wgrant: if i put it at the top of the file the zcml won't load
<wgrant> Oh, I got that but missed it. Oops
<wgrant> wallyworld_: Lies.
 * wgrant tries.
 * StevenK waits for "Hmmmm."
<wgrant> Unless it's an import order issue, which we haven't really seen since c.l.i was destroyed.
<wgrant> But if enums is importing other stuff then enums is buggy.
<wallyworld_> the vocab imports stuff
<wgrant> wallyworld_: But lp.registry.enums doesn't import anything else in LP, so it can't be circular.
<wgrant> I just moved it to its rightful location and it works.
 * wgrant lands.
<sinzui> +1
<wallyworld_> not sure what happened for me then
<StevenK> PyCharm caused it
<wgrant> PyCharm
<StevenK> Or something
<wgrant> Because it's written in Java
<StevenK> Hahahaha
<wgrant> And Java is Java
<wallyworld_> huh?
<wgrant> :)
<wallyworld_> i ran it from command line
<wallyworld_> what's pycharm got to do with it?
<wgrant> Don't you go trying to use logic on us!
<wgrant> sinzui, wallyworld_: I'm wondering about private team junk branches... we could subscribe the team, or we could create an immutable non-pillar access policy with a single grant for the team and make the branches use that.
<wgrant> That would prevent the team from losing control of their branches through unsubscription.
<wgrant> In a project the maintainer can always regain control.
<sinzui> I favour the latter
<wallyworld_> don't we want to avoid subscribe and visibility conflayion?
<wgrant> But in the private team case they may not be able to.
<wgrant> wallyworld_: Unfortunately scope was cut, so it will still be slightly conflated.
<sinzui> I would like to think sharing will be adapted to teams one day
<wallyworld_> the latter works for me too
<wgrant> The AP model is deliberately flexible to this sort of thing.
<wgrant> The core bits don't care about information type at all
<wgrant> This also means we have a well-defined owner for all branches in the system.
<wgrant> Finally.
<wgrant> Non-owner owner, that is.
<StevenK> Non-owner owner?
<sinzui> no maintainer that co-owns the branch
<wallyworld_> registrant vs owner perhaps?
<wgrant> Well, project maintainers own project branches
<wgrant> Branch owners don't own their branches
<wgrant> The project maintainer does.
<StevenK> Right
<wgrant> In terms of data ownership
<StevenK> Branches are hard, let's go shopping.
<sinzui> Well the registrant is another cock-up by Lp. Lp is not a home for Mr Cock-up, but we seem to have prepared a bed for him
<spm> that's just begging to be quotes paged...
<wgrant> Agreed
<StevenK> sinzui: Just a bed? He turns up so often he has a wardrobe and a toothbrush in LP>
<StevenK> s/>/./
<sinzui> StevenK, well noted. we also have given him several email addresses
<wallyworld_> sinzui: there's a sendTeamEmailAddressValidationEmail() method that's only called if contact email is entered when creating a team. it doesn't seem to be called from the team contact address view. should it be?
<StevenK> sinzui: Haha
<sinzui> what?
<nigelb> spm: Srsly.
<nigelb> I could scroll though #launchpad-dev.log every day and just find amazing quotes.
<sinzui> wallyworld_, all email addresses have to be validated. Maybe Lp is using the user email confirmation process by mistake
 * sinzui welcomes the deletion on unused code though
<wallyworld_> sinzui: the the team contact form, a different method is used, validate_new_team_email()
<wallyworld_> sinzui: the team creation actually sends an email
<wallyworld_> whereas the validate_new_team_email() seems to simple parse the email
<sinzui> ug
<sinzui> wallyworld_, I have more faith is the  validate_new_team_email() method. I expect barry's work it be 'canonical' approach. You can remove code I think.
<wallyworld_> ok, thanks. it seems silly that we do it 2 ways
<wallyworld_> sinzui: although you could enter an email that parses but is not valid
<wallyworld_> so the sendTeamEmailAddressValidationEmail(0 does at least check that bit
<sinzui> wallyworld_, an email address validation RE is more than 1024 chars long and will topple Lp trying to use in pages
 * sinzui remembers that sad incident
<sinzui> wallyworld_, So we care about the email being sent to the the address, and a human being using the link to confirm to Lp that the address is owned.
<sinzui> wallyworld_, I will tell the squad about the email address RE tomorrow.
<wallyworld_> sinzui: yes, that's my point. so we would want to keep the sendTeamEmailAddressValidationEmail() method and ditch the RE?
<sinzui> though the beer in me wants to tell the bogus yarn now
<wallyworld_> the RE is used during form submission of the contact address edit form
<wallyworld_> the send email and human clicks link is used for team creation
<wallyworld_> but we are removing the latter
<wallyworld_> so the question is - do we want to augment the RE check with the send email check on the edit contact adress form
<sinzui> wallyworld_,  I think we do want to keep it.
<sinzui> sendTeamEmailAddressValidationEmail() is the proper method
<wallyworld_> ok, thanks. will do that
<wallyworld_> confusing that we diverged
<sinzui> wallyworld_, yes. I just read the code. That is the proper method. The one we send to users would be a lie
<wallyworld_> it would be good to know why we only used that method on team creation
<wallyworld_> and not when editing the external contact address later
<wallyworld_> oh, i lied, we do send it
<StevenK> sinzui, wgrant: Bug 933935 can be closed now?
<_mup_> Bug #933935: Remove bug and branch subscription checks from visibility checks <disclosure> <sharing> <tech-debt> <Launchpad itself:Triaged> < https://launchpad.net/bugs/933935 >
<sinzui> I don't know, sorry
<wgrant> StevenK: Indeed, I've closed it
<wgrant> The last subscription check was removed from production an hour ago :)
<wgrant> I think that the BVP replacement I'm working on and the release of writable +sharing will close at least 25 bugs.
<StevenK> wgrant: Can I start on that? I'm looking for something to do.
<wgrant> StevenK: How much do you know about bug notifications?
<StevenK> Enough to know that I don't want to know more.
<StevenK> wgrant: Whyfor?
<wgrant> Bug #933934 is a blocker with a significant amount of remaining work.
<_mup_> Bug #933934: Remove the bug supervisor/security contact subscriptions rules <disclosure> <sharing> <tech-debt> <Launchpad itself:Triaged> < https://launchpad.net/bugs/933934 >
<wgrant> sinzui: ^^
<StevenK> Hello minefield, please blow my foot off.
<wgrant> We need structsub filters by information-type, and we need structsubs to work for private bugs.
<wgrant> The rules here are very well-defined, fortunately.
<sinzui> I think that will have to happen when sharing is out of beta
<sinzui> That will be fun code to cut
<StevenK> wgrant: bug 297756 is close to being closeable, too
<_mup_> Bug #297756: Cannot manage access to a project's private bugs and branches <disclosure> <lp-bugs> <oem-services> <sharing> <Launchpad itself:Triaged> < https://launchpad.net/bugs/297756 >
<wgrant> StevenK: Right, that's one of the >20 :)
<StevenK> sinzui: Why? structsub work seems out of the way
<wgrant> sinzui: It needs to be done before sharing can be really useful, and it has no dependency on sharing being complete.
<wgrant> sinzui: We can't remove the default bugsupervisor sub until we've sorted out structural subs
<wgrant> Otherwise nobody will be notified about new private bugs
<wgrant> == bad
 * StevenK tosses a card from Backlog to Deployment-Ready in wgrant's name
<wgrant> ITYM Done-Done
<StevenK> wgrant: I wasn't sure if the NDT was done because it's still Fix Committed
<wgrant> It's been on all the appservers for more than an hour. I think hloeung just hasn't noticed it's finished yet.
<StevenK> Hahaha
<sinzui> wgrant, okay, I think we have a card for that. I think there is a bug suggesting that we create structural subscriptions for security contacts and bug supervisors when we enter beta so that there is no apparent change in behaviour
<StevenK> sinzui, wgrant: So I will look at making structsubs work with private bugs.
<StevenK> Or I can do filter by information_type first
<wgrant> StevenK: Right. There's two largely separate pieces of work: firstly, structsubs have to work for private bugs. Secondly, they have to be able to filter by information type
<wgrant> You can't do the filtering by information_type first, since most of the other information types are private :)
<sinzui> StevenK,  Bug #1008526 and Bug #1008538 roughly describe the forthcoming train wreck that we want to avert
<wgrant> So it's going to be a bit hard to test
<_mup_> Bug #1008526: Security contacts are not notified when the project is not shared <bugs> <disclosure> <sharing> <Launchpad itself:Triaged> < https://launchpad.net/bugs/1008526 >
<_mup_> Bug #1008538: Bug Supervisors are not notified when the project is not shared <bugs> <disclosure> <sharing> <Launchpad itself:Triaged> < https://launchpad.net/bugs/1008538 >
<wgrant> sinzui: Bug #1008541 too
<_mup_> Bug #1008541: Sharing policies unconfigure existing projects <bugs> <disclosure> <sharing> <Launchpad itself:Triaged> < https://launchpad.net/bugs/1008541 >
<sinzui> yep
<StevenK> wgrant: Is there a bug about structsubs on private bugs?
<wgrant> StevenK: Bug #376186 is the main bug for that
<_mup_> Bug #376186: implicit (e.g.structural) subscriptions to private bugs don't work even when the subscriber has visibility on the bug <disclosure> <lp-bugs> <privacy> <sharing> <story-better-bug-notification> <ubuntu-qa> <Apport:Won't Fix> <Launchpad itself:Triaged> < https://launchpad.net/bugs/376186 >
<sinzui> StevenK, nothing old that I am aware of. We know that these three bugs assume SS work with privacy
<StevenK> wgrant: Right
<sinzui> So I tagged it but could not remember. maybe I need to switch to ice cream
<wgrant> StevenK: Bug #901548 should be fixed implicitly by that, but it's something to keep in mind.
<_mup_> Bug #901548: launchpad does not send emails to the assignee of private bugs <bugs> <email> <privacy> <sharing> <Launchpad itself:Triaged> < https://launchpad.net/bugs/901548 >
 * StevenK looks for a card for that bug
<StevenK> Searching Kanban is as bas as searching LP bugs
<StevenK> So I'm glad it's not just us
<wgrant> StevenK: I meant to create one yesterday, but then code privacy burnt down.
<wgrant> I don't think there is one.
<StevenK> wgrant: But then we teamed up and rebuilt it stronger, faster and better.
<sinzui> wgrant, I updated https://wiki.canonical.com/InformationInfrastructure/OSA/LP/DeploymentIssues/2012-07-10-revert-NDT
<wgrant> StevenK: I hope this series of tasks is a bit less fraught with inclarity and terror than your last one with branch privacy...
<StevenK> Haha
<wgrant> sinzui: Thanks, looking.
<StevenK> sinzui: ArtifactAccessGrants
<wgrant> AccessArtifactGrants
<StevenK> Artefact is something much older
<StevenK> And I don't think it was 2500 branches
<wgrant> sinzui: If you're still editing, it might be worth noting that 99% of the 2500 were old misconfigured branches.
<wgrant> StevenK: Slightly under 2400.
<wgrant> But 2500 is close enough :)
<StevenK> We only fixed like 140?
<sinzui> I will add that in another footnote. I don't want to brag about the period were projects were setup correctly
<wgrant> StevenK: Oh, you must have been busy last night.
<wgrant> StevenK: There was a secondary disaster after the one sinzui notified us of.
<StevenK> Yeah, I'm reading it now
<StevenK> I was off IRC
<sinzui> ah 2400
<StevenK> wgrant: Can I have a starting thread to tug at?
<wgrant> StevenK: Bug.getBugNotificationRecipients
<wgrant> And get_also_notified_subscribers
<wgrant>     if bug.private:
<wgrant>         return []
<StevenK> I was just laughing at the assertion in getBugNotificationRecipients
<StevenK> wgrant: So I guess the plan would be to have the functions do the usual things (rather than bugging out early for private bugs) and then filtering the results against get_bug_privacy_filter or something like it?
<lifeless> shouldn
<lifeless> bah
<lifeless> ignore tht
<wgrant> StevenK: Right
<StevenK> wgrant: I just hate the idea of looping over get_bug_privacy_filter
<wgrant> StevenK: There's one significant question that's unanswered.
<StevenK> wgrant: Private dupes?
<wgrant> StevenK: We should possibly do the filtering when we're calculating the recipients
<wgrant> But we don't expand teams at that stage
<wgrant> We only expand teams when we're sending mail
<wgrant> Well
<wgrant> I guess it's not clear that we want to do the filtering at the time of the action.
<wgrant> lifeless: What do you think?
<wgrant> You have opinions on this matter
<StevenK> s/ on this matter//
<lifeless> I would like to offer some constraints
<wgrant> They're all wrong.
<lifeless> don't do something incompatible with a notification service
<lifeless> which can do team expansion itself, but can't callback for individual visibility rules.
<wgrant> It's not at all clear that that constraint is necessary or supportable.
<wgrant> But it might be nice.
<StevenK> We certainly need it.
<lifeless> I have every intention of landing my stuff at some point; if its incompatible with that constraint, it will get dropped/redefined/changed to be compatible at that time,.
<lifeless> I'd rather not rework something you're reworking Right Now.
<lifeless> what else
<wgrant> I guess team structural subscriptions are evil anyway, so we can probably disregard them.
<lifeless> uhm, yes, duplicates and notifications are a pain.
<wgrant> Do the filtering at the time of the action
<wgrant> So BugNotificationRecipient will continue to have only permissible recipients.
<lifeless> How would it have others? [were you thinking to check subscriptions without cross referencing grants?]
<wgrant> lifeless: Well, the other option is to insert everything into BugNotificationRecipient and filter after team expansion in the sending cronscript. That would mean members could get mail through a team subscription as long as they had a grant, even if the team did not.
<wgrant> But team subscriptions are evil and should probably be disallowed, so it's not a major loss to break that.
<lifeless> wgrant: I'm not clear how team subscriptions would be broken
<lifeless> wgrant: grants -> subscriptions -> TP -> persons.
<lifeless> its one query, no ?
<wgrant> lifeless: I believe BugNotificationRecipient is the un-team-expanded set.
<wgrant> Yay, bugsummaryrebuild works
<wgrant> And the result is even correct
<lifeless> wgrant: I don't see how checking access at the front end prohibits teams working in the backend
<wgrant> lifeless: If we only insert into BugNotificationRecipient those Persons that have access, then team members that have access won't get email if their team doesn't have access.
<wgrant> The entire team will be mailed, or none of it will.
<lifeless> wgrant: if you dedup before inserting ?
<lifeless> wgrant: but that implies you are deduping before checking access
<lifeless> which is precisely not checking access at the front end of the process
<wgrant> The current pipeline is: event -> BugNotificationRecipient -> team expansion -> email
<wgrant> The question is whether we put the access check after event or after team expansion.
<wgrant> Deduping isn't relevant here
<lifeless> oh, I see
<lifeless> you're postulating the following scenario:
<lifeless> structural subscription for team A
<lifeless> no grant for team A
<lifeless> person X, a member of team A, has a grant.
<wgrant> Right.
<lifeless> either via team B, or via a direct subscription and a person-X direct grant.
<lifeless> given that we filter such teams from the list of who will be notified today, I would say that X should not expect a notification, except and unless A is given a grant.
<lifeless> Its confusing and inconsistent to have such metaconfiguration possible. IMNSHO.
<lifeless> is that a useful opinion ?
<wgrant> Indeed, it supports my feeling, so it's useful :)
<wgrant> StevenK: So it's nice and easy
<StevenK> wgrant: It is?
<wgrant> StevenK: Yeah, I think you can just remove the special-case which excludes indirect subscriptions, and add a filter on the end of getIndirectSubscribers to only return those people who satsify get_bug_privacy_filter
<StevenK> wgrant: As well as get_also_notified_subscribers?
<wgrant> StevenK: getIndirectSubscribers calls get_also_notified_subscribers
<wgrant> StevenK: As well as getting dupe subs
<wgrant> Both of which need to be filtered, I suspect.
<wgrant> Unless we don't want dupe subs to count at all, but that would seem odd
<StevenK> wgrant: Right, so now I'm concerned about how to do the filtering. Since get_bug_privacy_filter only takes one user, I don't want to loop over it.
<wgrant> StevenK: Ye of little faith
<StevenK> wgrant: I know they're storm fragments, so it can't get that bad, but ...
<wgrant> StevenK: RemoveArtifactSubscriptionsJob.run()
<wgrant> StevenK: It uses get_bug_privacy_filter_terms manually to do a bulk access check
<wgrant> In a tasteful fashion.
<wgrant> StevenK: Making any sense at all?
<StevenK> gary_poster: I hope you mean 'July 6'
<adeuring> good morning
* frankban changed the topic of #launchpad-dev to:  http://dev.launchpad.net/ | On call reviewer: frankban* (rvba)  | Firefighting: - | Critical bugs: 4.0*10^2
<StevenK> jtv: Hai
<jtv> Hi StevenK
<StevenK> jtv: Could I get your opinion on https://code.launchpad.net/~stevenk/launchpad/kill-rpsd/+merge/114300 ?
<jtv> StevenK: people keep saying that it's been replaced by a job, but every time I check we're not quite there yet.  Where does that stand?
<StevenK> jtv: I have no idea, TBH
<jtv> That might be worth checking.  The bit that was always missing was a background scrubber that fires off jobs for all POFiles, not just ones that we know need their stats updated.
<StevenK> benji: ^ Anything light you can shed on that? I *think* it was you that wrote the job.
<jml> if it's not being run at all, then why does it matter?
<StevenK> jml: Because the job may end up relying on infrastructure I'm proposing to remove.
<jml> StevenK: so then you can restore the code if that turns out to be the case.
<gary_poster> StevenK, heh, whoops, yes :-)
* abentley changed the topic of #launchpad-dev to:  http://dev.launchpad.net/ | On call reviewer: frankban* (rvba), abentley  | Firefighting: - | Critical bugs: 4.0*10^2
<czajkowski> gary_poster: did you get your juju mp sorted yet?
<gary_poster> czajkowski, no, but thank you for mentioning it on G+ :-) We need juju charmers to do the work
<czajkowski> ok
<czajkowski> let me go and poke
<gary_poster> heh, thanks czajkowski
* frankban changed the topic of #launchpad-dev to:  http://dev.launchpad.net/ | On call reviewer: abentley  | Firefighting: - | Critical bugs: 4.0*10^2
<czajkowski> bkerensa_: hey
<bkerensa_> hi
<czajkowski> bkerensa_: gary_poster is the one who is looking for juju help
<bkerensa> hi gary_poster
<gary_poster> bkerensa, hi
<bkerensa> gary_poster: I hear you are running into some issues with your charm? Perhaps I might be of assistance
<gary_poster> bkerensa, what we need is some packaging help.
<gary_poster> bkerensa, not exactly
<bkerensa> packaging?
<gary_poster> let me get you a mp
<bkerensa> k
<gary_poster> it is charm-helpers
<gary_poster> bkerensa, we've talked to mark_mims and SpamapS about this before; we're just hoping to get it resolved sooner rather than later.  Here's one MP... https://code.launchpad.net/~gmb/charm-tools/add-charm-helpers/+merge/96204
<gary_poster> That depends on packaing python-shelltoolbox
<bkerensa> gary_poster: Were mark_mims and Spamaps not able to help? They are perhaps the best Juju Hackers :)
<bkerensa> ahh I see
<bkerensa> so this is in regards to the juju trunk and not a actual juju charm?
<bkerensa> charm-tools specifically
<gary_poster> bkerensa, they want to help.  we just haven't been able to get their help.  Here is the most recent branch/MP, actualy.
<gary_poster> https://code.launchpad.net/~yellow/charm-tools/trunk/+merge/101554
<gary_poster> They have been too busy to help
<gary_poster> and this has gone on for months and months now
<gary_poster> and we would like to get this resolved
<bkerensa> ahh well I'm unfortunately not the right person... I just author charms and do not work on the juju or charm-tools branches at all.... I will ping mark and Spaamaps but also have you perhaps talked to jcastro since he is the Cloud Liason?
<bkerensa> gary_poster:  bkerensa: yeah, I really need to just sit down and do this. Perhaps today
<bkerensa> so it looks like Clint is going to get working on this
<gary_poster> bkerensa, fantastic, thank you
<bkerensa> hopefully today but at least very soon
<gary_poster> cool
<czajkowski> bkerensa: thanks
<bkerensa> might wanna join #juju to follow up with him as neccesary or prod him a couple times
<bkerensa> ;)
<bkerensa> czajkowski: no problem
<bac> gary_poster: cool
<gary_poster> thanks very much czajkowski !
<czajkowski> gary_poster: see it pays to blog :)
<gary_poster> czajkowski, :-)
<sinzui> jcsackett, wgrant, wallyworld_, StevenK, This is my draft of a report stating what we have been doing over 13 months: http://people.canonical.com/~curtis/lp-milestone/report.html
<StevenK> wgrant: Total: 120 tests, 3 failures, 0 errors in 2 minutes 17.362 seconds.
<wgrant> StevenK: Not bad
<StevenK> wgrant: Those 3 failures are all TestNotificationRecipientsOfPrivateBugs
<StevenK> wgrant: http://pastebin.ubuntu.com/1087043/
<jcsackett> sinzui: Mumble crashed just as you suggested we adjourn. So the tech spirits agreed with you. :-p
<StevenK> wgrant: I've had to comment out the issuperset assert too
<wgrant> Indeed
<StevenK> wgrant: I'm just wondering why that assert is there
<StevenK> wgrant: Total: 120 tests, 0 failures, 0 errors in 2 minutes 17.914 seconds.
<StevenK> % bzr damage
<StevenK> Using submit branch /home/steven/launchpad/lp-branches/devel
<StevenK> Damage: 0 lines of code
<wgrant> StevenK: I don't really know
#launchpad-dev 2012-07-12
<rick_h_> huwshimi: https://github.com/yui/yui3-gallery/tree/master/build/gallery-anim-morph woot!
<rick_h_> https://github.com/yui/yui3-gallery/tree/master/src/gallery-anim-morph for the source with the api docs
<huwshimi> rick_h_: Fantastic!
<rick_h_> so first gallery module done, working on a couple more, but good to get the ball rolling
<huwshimi> rick_h_: Yeah, feels like we're doing this properly now.
<StevenK> wgrant: Clearly, AccessArtifactGrantSource.grant([artifact, subscriber]) is not the right way to call it, and I can't work out the right way from the code.
<wgrant> StevenK: AccessArtifactGrantSource.grant([(artifact, grantee, grantor), ...]), IIRC, but let me check the interface
<wgrant>     def grant(grants):
<wgrant>         """Create `IAccessArtifactGrant`s.
<wgrant>         :param grants: a collection of
<wgrant>             (`IAccessArtifact`, grantee `IPerson`, grantor `IPerson`) triples
<wgrant>             to grant.
<wgrant>         """
<StevenK> Right
<StevenK> That pesky subscribed_by thing
<wgrant> grantor
 * StevenK blinks.
<wgrant> StevenK: Hmm hmm?
<StevenK>         # Only include subscribers who can see the bug, if it's private.
<StevenK>         print list(indirect_subscribers)
<StevenK>         if self.private:
<StevenK>             print list(indirect_subscribers)
<StevenK> [<Person at 0xf29fdd0 product-subscriber (Product-subscriber)>]
<StevenK> []
<wgrant> StevenK: It's probably a resultset
<wgrant> Or some other sort of generator
<wgrant> That you're consuming with the first list()
<StevenK> It's a chain()
<wgrant> Right, a generator
<StevenK> Sigh, so I can't print it
<StevenK> Bah, the query returns [], so it's fine, but product-subscriber is coming from somewhere.
<wgrant> wallyworld: Did you consider precache_permission_for_objects?
<wgrant> branchlisting.py uses it
<wallyworld> yes but decided against it
<wallyworld> since it's not the http caller needing to be cached
<wallyworld> and it's exactly the same problem as for bug searching
<wgrant> Isn't it?
<wallyworld> ah, it is, yes
<wgrant> Ah, you do it in BranchCollection, so this is a more general solution.
<wallyworld> but i wanted to stick to the bug search pattern
<wgrant> But you could equally do it in the view.
<wgrant> Right.
<wallyworld> i don't want to do it in the view
<wallyworld> wrong place
<wgrant> wallyworld: r=me, thanks
<wallyworld> np, thanks for reviewing :-)
<wallyworld> i really wanted to get that one sorted asap
<wgrant> Yeah
<wgrant> https://code.launchpad.net/~wgrant/launchpad/more-branchnamespace/+merge/114563 is pretty simple if you have a few mins.
<wallyworld> sure
<StevenK> wgrant: That does USERDATA, not PROPRIETARY
<wallyworld> wgrant: 30	return False
<wallyworld> should be return None i think
<StevenK> I guess it will migrate when everything else does
<wgrant> wallyworld: Good point. Doesn't actually matter, but it's wrong :)
<wallyworld> wgrant: i agree with StevenK. we can cater for Proprietary in the if statement and it will work later
<wallyworld> 42	+ if (information_type == InformationType.USERDATA and
<wallyworld> 43	+ getFeatureFlag('disclosure.display_userdata_as_private.enabled')):
<wallyworld> 44	+ return 'Private'
<wallyworld> 45	+ return information_type.title
<wallyworld> can add an == propprietary bit
<wgrant> Where?
<wgrant> That "return 'Private'" is just for the userdata_as_private
<wallyworld> ah yes, sorry. misread it
<wgrant> StevenK: Do you refer to getDefaultInformationType?
<StevenK> That feature flag is getting used everywhere :-(
<StevenK> wgrant: Yah
<wgrant> StevenK: Well, the default information type is never proprietary yet
<wgrant> So making it sometimes proprietary when it's never proprietary would be sort of wrong :)
<wgrant> StevenK: I'll be adding a new feature-flagged codepath tomorrow which will return proprietary if the enum is set to return proprietary
<StevenK> Ah
<StevenK> wgrant: Yeah, you could use disclosure.proprietaryblahblah.disabled
<wgrant> But returning proprietary before anything should be proprietary would seem to be a Bad Ideaâ¢
<wgrant> StevenK: No
<wgrant> disclosure.bvp_must_die or so
<StevenK> Haha
<StevenK> Curtis will be unhappy if you add another FF :-)
<wallyworld> we can use the existing ff
<StevenK> No, wgrant is right, we can't.
<wgrant> We can't
<wallyworld> disclosure.proprietary_information_type.disabled
<wgrant> It's entirely different.
<wgrant> This engages the BVP replacement
<wgrant> Which we'll want to do before we enable proprietary
<wallyworld> if the branch is associated with a proprietary project, shouldn't new branches be propprietary?
<wgrant> Proprietary projects don't exist yet.
<wallyworld> regardless / instead of bvp
<wgrant> I suspect that a proprietary project has an enforced PROPRIETARY_ONLY branch config setting.
<wallyworld> if a projject is has a current commercial subscription it can be
<wallyworld> so i guess yes it may be too early
<wgrant> Commercial subscriptions will enable the project owner to enable proprietary branches.
<wgrant> They don't enforce all branches to be private.
<wallyworld> wgrant: r=me, looks great
<wgrant> Thanks
<wallyworld> welcom
<wgrant> The plan is to enable writable sharing RSN, then once that's out and working we can do the migration away from BVP to the new enumcol. The first stage of that work will still create User Data branches, like BVPs do now.
<wgrant> Once BVP is dead, we can sensibly turn on Proprietary
<wallyworld> ok
<StevenK> wgrant: I guess we want IProject.default_propietary_branches
<wgrant> At which point the BVP replacement can start doing Proprietary instead
<wallyworld> so close yet so far
<wgrant> StevenK: It's a four-state enum, remember.
<wgrant> â£ Public: Branches are public unless they contain sensitive security information.
<wgrant> â£ Public, can be private: New branches are public, but can be made proprietary later.
<wgrant> â£ Proprietary, can be public: New branches are proprietary, but can be made public later. Only people who can see the project's proprietary information can create new branches.
<wgrant> â£ Proprietary: Branches are always proprietary. Only people who can see the project's proprietary information can create new branches.
<wgrant> Or so
<StevenK> wgrant: Ah yes. I'm distracted by ACB
* jelmer changed the topic of #launchpad-dev to:  http://dev.launchpad.net/ | On call reviewer: jelmer | Firefighting: - | Critical bugs: 4.0*10^2
<adeuring> good morning
<czajkowski> morning
 * cjwatson wonders if there's any chance of deploying up to r15608 today ...
<jelmer> hi adeuring, czajkowski, cjwatson
<adeuring> hi jelmer!
<wgrant> cjwatson: We have no UK ops today, but jjo might be able to do it in a few hours if QA gets done
<cjwatson> Yeah, I suppose I was hinting at those with pending QA :)
<wgrant> StevenK, jcsackett ^^
<wgrant> Ah, I think jcsackett might have done his, but forgotten one of the bugs
<wgrant> So it's just StevenK, on something that doesn't really matter hugely at this point.
<cjwatson> (That deployment would let me update all our documentation to refer to the API client and decommission the queue script, since the API is now feature-complete.)
<wgrant> It would seem wise to keep the queue script around for a couple of weeks, just in case...
<wgrant> Since we have no other way out.
<cjwatson> Yeah, perhaps
<cjwatson> I wonder if SRU handling is enough to stress-test things, or if we need a freeze period of quantal
<cjwatson> The latter are a bit difficult to come by these days, since there's considerable top-down pressure not to freeze for alpha milestones
<wgrant> I think routine SRU+security stuff is probably more likely to shake out issues than freezing a dev series.
<wgrant> Freezes tend to often be handled through the web UI anyway, right?
<cjwatson> Only because we felt we ought to and didn't have an API.
<wgrant> Right, but they weren't traditionally done through queue
<cjwatson> They've traditionally been a mix
<wgrant> So removing queue is less likely to impact them.
<wgrant> True
<cjwatson> I'm not especially worried about routine NEW handling; the things I'm less sure about are timeouts on accepting uploads that close lots of bugs
<cjwatson> Which are more likely in the development series, apart from kernel SRUs which happen through a PPA anyway so don't count
<wgrant> Hm, indeed
<wgrant> Perhaps we can accidentally freeze before the next unity upload...
<cjwatson> I could certainly ask for an experimental freeze where we in fact just accept everything for a few days
<cjwatson> And I've tweaked my pending announcement mail to remind people that I need to actually hear about bugs rather than quietly working around them
<wgrant> :)
<jtv> Hello jml!  Julian had an interesting suggestion: we found Launchpad's FakeMethod a tough thing to do without in MAAS, so maybe it would be an interesting thing to have in testtools?
<jml> jtv: I've forgotten what FakeMethod does. Looking now.
<jml> jtv: tbh, I think not. I think what you want is mfoord's mock: http://pypi.python.org/pypi/mock
<jtv> jml: it's used for st/ubbing (mis-spelling to avoid bothering Stu) functions and methods.
<jtv> Yes, this is a lot more lightweight.
<jtv> \
<jml> jtv: what does that mean?
<jtv> It does a lot less.
<jtv> It does:
<jml> jtv: and that's a good thing?
<jtv> Can be, if you don't want to climb the learning curve.
<jtv> That was the problem with Mock when we first ran into it, I think: there was too much new stuff to take in, when all we really wanted is patch out some method to return a predetermined value, or raise a predetermined exception.
<jml> it's pretty easy
<jml> @patch("foo.bar")
<jml> def f(mock_foo_bar):
<jml>   mock_foo_bar.returnvalue = 42
<jtv> Still:
<jtv> self.patch(foo, "bar", FakeMethod(42))
<jml> I riposte:
<jml> self.patch(foo, 'bar', lambda *args: 42)
<jtv> Don't forget the **kwargs.
<jml> my test will fail if that matters :)
<jtv> But sometimes we find that afterwards you want to know about what calls have been made to your fake, and that's also built into FakeMethod.  (Mock has something nicer for that, but again FakeMethod keeps it very simple)
<jml> jtv: ok
<jtv> Anyway, not trying to force it on you.
<jml> jtv: No worries. I'm not interested in putting it into testtools. I would strongly suggest using mock and then having less code to maintain. Alternatively, create a library for FakeMethod.
<jtv> Seems a bit silly to have something so small stand alone as a libraryâ¦ I think that's one reason why Julian suggested it might have a place in testtools.
<jml> jtv: perhaps it might have a place in mock
<jtv> What we did for now is just copy it.  Doesn't really matter if it diverges or not.
<jml> well, maintaining way too much code _is_ the Launchpad Way.
<jtv> :)
<jml> no point in making your job too easy
<jml> sorry. I shouldn't snark :(
<jtv> That's okayâ¦ I can see the truth in it.
<jtv> In this case, it's stable enough that I don't see it mattering much.  It's not like we ever find bugs in it.
<jtv> We'd _like_ to have it in a shared place, of course, but it's not hugely important.  I think it'd probably fit poorly with Mock; maybe Mock has a good replacement for it.
<cjwatson> OK, zope is melting my brain and I need some advice.  What might I have forgotten in http://paste.ubuntu.com/1087847/ that would cause "KeyError: 'editchroot'"?
<cjwatson> It looks like it's failing to look up overview_menu.
<cjwatson> But I might be misguessing as it's kind of hard to keep my head straight in 20-odd stack frames.
<cjwatson> The expression it's trying to evaluate at the time is Expression: <PathExpr standard:u'overview_menu/editchroot/fmt:icon'>
<cjwatson> And I think the menu is None.
<jam1> hey jelmer, how's it going today?
<cjwatson> http://paste.ubuntu.com/1087861/ is the full traceback I'm staring at.
<cjwatson> Ah, maybe I have the NavigationMenu attached to the wrong interface
<rick_h_> sorry cjwatson you're on my kryptonite
<rick_h_> maybe adeuring knows a bit more
<adeuring> cjwatson: right; without looking further I'd guess that overview_menu does not know about "editchroot"
<jelmer> jam: hey
<jelmer> jam: alright, reviewing and wrapped up the bzr upload to quantal
<jelmer> jam: are you still working on the python2.7 transition?
<mgz> with a ribbon?
<rick_h_> sinzui: do you know where YUI comes from? It's in lp-sourcedeps and symlinked into the app root, but I can't figure out where that lp-sourcedeps directory came from
<sinzui> I do not
<sinzui> StevenK, do you know ^
<wgrant> rick_h_: buildout
<wgrant> rick_h_: There's a yui tarball in download-cache
<wgrant> A buildout rule extracts it into yui/
<rick_h_> is there? I was checking out and missed it
<wgrant> Which is a symlink to lp-sourcedeps
<rick_h_> ok, looking thanks
<wgrant> $ ls download-cache/dist/yui-3.*
<wgrant> download-cache/dist/yui-3.3.0.tar.gz  download-cache/dist/yui-3.3.tar.gz  download-cache/dist/yui-3.4.1.tar.gz  download-cache/dist/yui-3.5.0pr1.tar.gz
<rick_h_> heh, lovely so just blind
<rick_h_> oh heck, scrolling ftw thanks wgrant
<wgrant> Heh
<cjwatson> adeuring: mm, well, I now have http://paste.ubuntu.com/1087894/ which I think is less incorrect (it now actually has a NavigationMenu with usedfor = IDistroArchSeries), but I'm still getting exactly the same failure :(
<cjwatson> queryAdapter(<a DAS object>, INavigationMenu, 'overview') -> None
<wgrant> cjwatson: Have you registered it?
<wgrant> You want the browser:menus ZCML directive
<cjwatson> Ah, yes, I just got there by grepping for another case
<cjwatson> Excellent!  Now I have a different failure, but a comprehensible one.  Thanks
<rick_h_> abentley: http://blog.singly.com/2012/07/09/from-itch-to-scratched-google-hangout-permalink/
<cjwatson> Is there a limit (practical or otherwise) to the size of file that can go in a multipart/form-data upload to LP?
<abentley> rick_h_: Cool.  Sounds like what smoser did.
<czajkowski> sinzui: good day. do you happen to know if ther eis a way to automate via a terminal milestones for a project ?
<sinzui> czajkowski, do you mean create them, release them?
<czajkowski> sinzui: https://answers.launchpad.net/launchpad/+question/202937
<czajkowski> is why I am asking
<sinzui> There  many launchpadlib scripts that do that
<sinzui> https://launchpad.net/+apidoc/devel.html
<sinzui> I have one that U use to release milestones, but nothing that creates the milestone, or adds a file to a project_release
 * sinzui looks for example
<sinzui> czajkowski, point the user to lp-project-upload in https://launchpad.net/ubuntu-dev-tools which will create a milestone, release it, then upload a file
<czajkowski> sinzui: great, thank you/
<sinzui> czajkowski, , actually, I don't see that script anymore
 * sinzui looks for rename
<Laney> it's probably in lptools now
<sinzui> czajkowski, lp-project-upload moved to https://launchpad.net/lptools
<czajkowski> sinzui: cheers
* jcsackett changed the topic of #launchpad-dev to:  http://dev.launchpad.net/ | On call reviewer: jelmer, jcsackett | Firefighting: - | Critical bugs: 4.0*10^2
<jam> jelmer: yes, I'm still working on the python2.7 stuff. I'm getting a segfault now in the lucid container, which is being problematic to track down.
<mgz> a py2.7 specific segfault? because you had lucid vanilla working right?
<jelmer> jam: do you have a gdb backtrace that's hinting at what might be happening?
<rick_h_> jcsackett: ping, have a review for you if you get a sec. I *think* this is right and works locally but please sanity check me https://code.launchpad.net/~rharding/launchpad/lpyui-dep/+merge/114430
<jam> jelmer: that's the best part, 'ulimit -c unlimited' and I can't find a core file
<jam> mgz: yes
<jam> jelmer: It fails while bootstrapping 'bin/py' I can at least run 'py -vv' and see that it is failing during "import code_..." ?
<jam> however, just importing that module isn't enough to get it to crash
<jam> I'm guessing there is something that is breaking the import machinery, or something along those lines.
<jam> mgz, jelmer: I'm trying to do a fresh bootstrap (somehow, not sure if 'make clean' is actually clean enough)
<jam> now that is weird...
<jam> I did 'make schema' and it crashes at bin/py, but the outer OS seems to think gnome-screensaver just died
<jelmer> hmm
<jelmer> pid reuse?
<jcsackett> rick_h_: it looks alright to me, but my Make/buildout knowledge is bare bones. i believe gary_poster has some expertise there iirc. if he has time, might be worth having him take a look too.
<rick_h_> jcsackett: ok, I tested it out locally and tried to make sure make check/tests run, turning on/off the combo loader FF works, and then setting and unsetting the yui verison
<jcsackett> rick_h_: given all that and a second pair of eyes, i think it's probably fine. if it's not, ec2 will choke on it anyway. so, like i said, r=me.
 * jcsackett realizes he didn't actually say that.
<gary_poster> jcsackett, hiya.  on calls. lemme know if you need me later
<jcsackett> gary_poster: will do.
<jam> jelmer: well this time it is crashing trying to import _ctypes which seems much more likely to be a problem.
<jam> jelmer: so right now I'm trying to make sure it isn't re-using .pyc files or something along those lines, with the 2.6 install. But I accidentally nuked my download-cache and have to redownload it, etc.
<jelmer> jam: fun :(
<jelmer> jam: what did you have to do to get make to run, btw? for me it still fails (silently) during the mailman step
<jam> jelmer: well right now 'make schema' is segfaulting, so I don't know that 'make' is working
<jelmer> jam: have you tried "bzr grep python2.6" ?
<jelmer> jam: I think one of the database scripts uses its own shebang line
<jam> jelmer: bin/py was specifically segfaulting while running 'import site', so it is happening pretty early.
<jelmer> ah, ok
<mgz> jam: what's your recipe roughly? just add the lucid py2.7 ppa to a normal lucid setup of launchpad?
<jam> mgz: and have a branch of launchpad that changes the PYTHON:= line in the makefile to not do crazy version checks, but just 'PYTHON:=python2.7'
<jam> add-apt-repository ppa:pythoneers/lts
<jam> then sudo apt-get install python2.7
<mgz> ace. can probably repo here reasonably easily then, unless rocketfuel borks things too badly (then it's extra hacks in the branch)
<jam> mgz: note that on startup I'm getting: http://paste.ubuntu.com/1088093/
<jam> but everything was working with python-2.6, so I thought that was just noise
<jam> I did run out of disk space at one point, and had to expose more space for the VM. maybe something got horrbly borked and I should just restart from scratch
<mgz> fun.
<czajkowski> jelmer: can you investigate please. https://support.one.ubuntu.com//Ticket/Display.html?id=19323
<jelmer> czajkowski: on it
<czajkowski> jelmer: thanks
<abentley> jcsackett: Could you please review https://code.launchpad.net/~abentley/launchpad/mp-by-revision-id/+merge/114693 ?
<jcsackett> abentley: r=me. looks good, and thanks--i've been wanting that to work. :-)
<abentley> jcsackett: Thanks.  It'll also require some bzr-side fixing before lp-find-proposal can do it.
<magcius> ev, ping
<sinzui> jcsackett, wgrant, StevenK, wallyworld: http://people.canonical.com/~curtis/lp-milestone/report.html
<lifeless> sinzui: did you do that yourself ?
<lifeless> sinzui: or is it some stock tool we have hanging around ?
<sinzui> Yes
<lifeless> its nice
<sinzui> I wrote it. It thinks it is talking to Lp, it is actually using json I pulled and removed private data
<rick_h_> lifeless: replied in -dev for the combo loader notes. Let me knwo if there's something else I need to do.
<sinzui> A lot could be reused in Lp or other tools
<lifeless> sinzui: are the burn down charts totally LP sourced?
<sinzui> yes
<lifeless> sinzui: perhaps lpkanban would be a good place for the chart logic to live
<sinzui> Most of the code is tested.
<sinzui> I was thinking of pulling data from kanban, but I need s break from this report
<lifeless> sinzui: lpkanban shows LP bug data for a team as a kanban board
<lifeless> sinzui: its not leankitkanban related
<lifeless> sinzui: and sure, not trying to give you work, just speculating about structure :)
* jcsackett changed the topic of #launchpad-dev to:  http://dev.launchpad.net/ | On call reviewer: - | Firefighting: - | Critical bugs: 4.0*10^2
<StevenK> wgrant: http://pastebin.ubuntu.com/1088958/  test_add_comment_no_access fails since product-subscriber is in the list, but it should have no access to the bug, so WTF ...
<wgrant> StevenK: That's pretty odd. Let me have a look
<wgrant> StevenK: Oh
<wgrant> StevenK: The return is a lie.
<wgrant> getBugNotificationRecipients doesn't actually use the return value. It uses the contents of BugNotificationRecipients.
<wgrant> So you need to filter what you add :(
<wgrant> Or get BugNotificationRecipients to filter
<StevenK> Oh, bleh
<wgrant> Or *possibly* filter in getBugNotificationRecipients
<StevenK> Let's teach BugNotificationRecipients how to filter
<StevenK> Which looks ... fun
<wgrant> StevenK: btw, qa?
<StevenK> Oh, bleh.
<rick_h_> ccccccbgjgvclbkvdbvchdknlnkhgutbkgbcurerrjvb
<wgrant> I didn't know rick_h_ had a cat.
<spm> possibly he's moving to Aus and this is the town he's moving to?
<wgrant> True
<StevenK> spm: That must be the short name.
#launchpad-dev 2012-07-13
<rick_h_> wgrant: heh, ubikey
<rick_h_> once in a while bump the dippy thing
<StevenK> wgrant: Twitch.
<StevenK> % grep -c 'pdb.set_trace' lib/lp/bugs/mail/bugnotificationrecipients.py
<StevenK> 1
<wgrant> StevenK: Bug #1024148
<_mup_> Bug #1024148: Branch.transitionToInformationType breaks when making a subscriberless branch private <disclosure> <regression> <sharing> <trivial> <Launchpad itself:Triaged> < https://launchpad.net/bugs/1024148 >
<wgrant> StevenK: Oh wow, that's nice.
<rick_h_> StevenK: got a sec? I think I blew up buildbot, but my ec2 test run passed so wonder if you can help me figure out what the difference is and what I can do to fix
<rick_h_> my understanding is that this runs a make check so if I make clean_buildout and make check locally and it works I should have been ok?
<StevenK> rick_h_: make check will take ~ 5 hours
<StevenK> rick_h_: Right, the buildbot slaves don't have convoy
<rick_h_> StevenK: right but what I mean is that the buildbot fail is about convoy, but it's in the download cache
<rick_h_> ah bah that's right...I forgot it's packaged
<StevenK> Assuming convoy is available everywhere is a recipe for disaster
<rick_h_> crap...ok...looking
<wgrant> +jsbuild: $(PY) combobuild jsbuild_widget_css $(JS_OUT)
<wgrant> jsbuild very deliberately didn't depend on combobuild before :)
<rick_h_> right, but combo-rootdir drops the YUI code into the build dir now. Since it worked locally with make check I thought I was clear
<rick_h_> I'll move the meta.js to make run and should fix it
<wgrant> rick_h_: Why did you need to move combo-rootdir?
<StevenK> rick_h_: So local has convoy, as does ec2. buildbot does not.
<rick_h_> StevenK: right, gotcha.
<rick_h_> wgrant: so with the policy of 'all things come from system packages or python packages' YUI is served from a python package now. I set the combo-rootdir to extract that out when it runs
<rick_h_> and it needs to run before the launchpad.js generation now
<wgrant> That policy is not sensible for non-Python things like YUI
<rick_h_> where before yui was put into place by buildout
<wgrant> Right
<wgrant> Which is probably correct.
<wgrant> lifeless: opinion pls
<lifeless> hi
<lifeless> 'sup ?
<wgrant> Repackaging yui tarballs as a launchpad-yui egg seems pretty insane.
<rick_h_> my understanding in talking with sinzui deryck and company that this was the preferred path.
<wgrant> They might think they prefer it, but they don't.
<rick_h_> that anything like the sourcedeps is to be deprecated and that all things much come from system or python packages
<rick_h_> and since I need to be able to run multiple yui versions at once for testing, I setup a python package that dumps any contained YUI versions to the build dir
<lifeless> sorry, whats the full story?
<wgrant> Dumping multiple versions of yui into a custom tarball just to make it an egg is entirely crazy.
<StevenK> rick_h_: source*code* is deprecated, sourcedeps is not
<wgrant> lifeless: Yesterday we had a custom buildout rule which took a normal upstream YUI tarball and turned it into something Launchpadish
<wgrant> lifeless: Today we instead have a Launchpad-specific egg with two versions of YUI unpacked within it
<StevenK> Change utilities/sourcecode.conf == bad, change versions.cfg ==okay
<wgrant> I don't see this as an improvement.
<StevenK> Bah, I can't call BugNotificationRecipients.update() with a resultset. :-(
<rick_h_> wgrant: the improvement is that this package has make commands to help auto extract only the build bits from the YUI download, etc
<rick_h_> and then it dumps all versions it contains into the build dir without needing to update any list/config for them
<wgrant> Right, a helper like that is sensible
<rick_h_> and the only thing that matters is that build/js/yui is symlinked to the version in versions.cfg
<wgrant> Embedding the YUI codebase is is not
<rick_h_> well based on conversations in pre-impl it was sort of based on the lazr.js model and some other 3rd party modules I found
<lifeless> rick_h_: so, we want to reduce the number of dependency systems in use, that is correct.
<rick_h_> http://pypi.python.org/pypi/launchpad_yui/0.1
<wgrant> This doesn't reduce the number of dependency systems
<wgrant> It adds a new one: embedding large third-party codebases in a Launchpad-specific public egg
<lifeless> rick_h_: however, with current deployment logic and packaging tech, that means we'll end up with a) debian package for firefox, rabbit, pgsql etc; b) buildout versions.cfg entries for python packages and anything else that buildout can deliver.
<wgrant> It happens to sit behind something like looks like one of our other dependency systems
<wgrant> But it isn't.
<rick_h_> right, well because JS doens't have a good dependency system in LP, so I shoe-horned my way into an existing one
<StevenK> Which is terrible
<lifeless> rick_h_: We've had some friction in the past using 'make YUI into an egg' approaches; can I ask what problem you were trying to solve ?
<wgrant> We already had it fairly cleanly shoehorned, didn't we?
<wgrant> I don't understand what was wrong with the old way.
<lifeless> StevenK: wgrant: Can you guys chillax a little so we can understand whats going on?
<rick_h_> lifeless: so the problem was 'how to more easily maintain/manage multiple yui versions in LP'
<lifeless> rick_h_: can you expand on that a little? what was giving you grief, what do you want to be able to do (and how does this do it)
<lifeless> rick_h_: I realise you've probably discusse that with deryck & sinzui already, but I can't really give an opinion till I get a handle on all the friction :)
<rick_h_> ok so it's a couple parts of me not getting it initially (I realize now) and wanting to make installing/testing newer versions of yui easier.
<rick_h_> the idea was that we need to be able to drop several versions of YUI into the build/js dir and use the feature flag to switch which one you use
<rick_h_> we kind of had that, there's some buildout stuff to install 3 versions that are in the download-cache of yui which I didn't quite get until today
<lifeless> we'll need that for production too
<lifeless> otherwise race conditions galore happen.
<rick_h_> and in discussions the 'idea' seems to be that things must either be a python package or a system package going forward
<rick_h_> that seems to have been taking the idea too far from what I'm hearing now though. download-cache is still ok with non-python packages
<rick_h_> so the yui zips in there would have been ok
<lifeless> download-cache is just a holding place; the key thing is to get away from the 4! we have today.
<rick_h_> so with the understanding that YUI has to come in via python package or system package, I thought a python package was easier to deliver multiple versions and setup http://pypi.python.org/pypi/launchpad_yui/0.1
<lifeless> while we use buildout, and buildout can do more than just eggs, we're fine with zips in download-cache, yes.
<rick_h_> today I landed changes to extract YUI from that package vs the buildout download cache, I did that in the current combo-rootdir bin script, and that needed to run now before launchpad.js is built for non-combo loader users
<rick_h_> so I moved around the make deps so that jsbuild required combo biuld, which required convoy and the buildbot slaves don't have that, so I broke buildbot
<lifeless> That pypi project is probably a great place for scripts to do things like download the latest zip and put it in download-cache, to live.
<lifeless> rick_h_: thanks.
<lifeless> so heres what I think: shovelling one package into another as a transport mechanism is necessary sometimes but usually at best somewhat unpleassant
<rick_h_> so I'm trying to see 1) what to do to unfubar buildbot and 2) if I should be rethinking the changes all together.
<lifeless> that strategy tends to run into several classes of headache: versioning is tricky if you want it to match at all; authentication of the transported thing is often hard or at the very least nontrivial
<lifeless> and lastly its usually surprising for everyone that encounters it.
<lifeless> rick_h_: do you have any ideas about how we manage syncing and using the gallery widgets going forward ?
<lifeless> (this may seem like a non sequitor, but its very related :))
<rick_h_> so that's a seperate issue at the moment. This is only for the base YUI library. I've started conversations with others on the syncing, but there's not currently an acceptable answer yet
<rick_h_> honestly, my pitch to deryck was to run our own gallery.yui.com but js.canonical.com that could be https and shared modules for all teams. Since modules are versioned, they could be kept and updated in a single place so all could follow dev/what's available.
<lifeless> It seems like the same problem to me: we have code in a different *space (language/name/dependency - everything), that we want to consume easily and reliably and upgrade from time to time, and possibly fork from time to time.
<rick_h_> however, that's a BIG undertaking
<rick_h_> since it required infrastructure and someone to shepard/manage things
<lifeless> it also doesn't give any assistance towards offline development, offline testing and local forking
<rick_h_> lifeless: you're right in that it is, but as I said, there's no 'great' answer yet so I admit that this python package setup was hackish, but kind of an the immediate means to the end
<lifeless> sure, I'm not critiquing it [yet] :P :)
<rick_h_> lifeless: well, the way the yui gallery works is that it's a single repo, anyone can checkout and serve
<rick_h_> much as you checkout yui itself, and put it into a combo loader server
<lifeless> by checkout, do you mean 'grab a tar' or 'git clone' ?
<rick_h_> but yes, it's not perfectly worked through for sure. There's a balance on duplication of effort and visiblity of a central location vs the distribution of things to offline/individual running
<rick_h_> git clone
<lifeless> ok
<rick_h_> https://github.com/yui/yui3 you clone that and get it all
<rick_h_> and updates are done by forking their repo, adding your new module, and requesting they merge it back
<lifeless> so, with all that in mind
<lifeless> do you think we should handle widgets the same as base YUI ?
<rick_h_> ideally, it would be nice
<lifeless> are there any reasons to handle them differently ?
<rick_h_> just beacuse the yui libary is not apt to change as often, and is a single part vs many modules
<rick_h_> there's a manner of scale when you consider managing a couple of versions of one dep, vs couple of versions of many deps
<rick_h_> and I don't have the same issues of making existing code visible to teams, updates sync'd, etc with YUI base as I do any modules
<lifeless> I don't follow those issues; sorry to keep doing this - but can you expand on them please?
<rick_h_> sure, so let's agree that everyone is using YUI and knows where it is. If they need JS, they know what to get and where to go get it
<rick_h_> and it's a pull only use
<rick_h_> if a new version comes out, they pull it down
<lifeless> the base YUI ?
<rick_h_> right
<lifeless> why is it pull only? Is it bug free?
<rick_h_> well, sure there are bugs, and you go through their system, get them updated, pull the new release. I've not seen us keep a patch against YUI anywhere so far
<rick_h_> not saying it's not possible, but not something experienced in work so far
<rick_h_> so for the base YUI it's "usually" a pull only of maybe 2 or 3 versions (current running, next version to test with, keeping an eye on a future preview release)
<rick_h_> while wigets/gallery code needs to be made visible to teams. "Is there a module for this?", they need to be able to update those, push changes without breaking backward compatiblity for other users, and there's many more as teams develop new JS modules
<lifeless> I don't get the difference
<lifeless> when you say visible to teams, is YUI base not visible to teams?
<rick_h_> and then there's infrastructure, we don't test YUI downloads, but we'd have to test and document gallery items/etc
<rick_h_> YUI base is because they all know where to look
<rick_h_> how many canonical teams know of the various YUI modules other teams have written
<rick_h_> ?
<rick_h_> this is the main reason I hope to one day get together a centralized area for YUI modules done like the kitchen code on the -tech mailing list recently
<rick_h_> landscape mentioned a nice mock tool they use in testing, but I've got no idea that exists to be able to share/reuse that
<wgrant> wallyworld: Could you find some time to review https://code.launchpad.net/~wgrant/launchpad/edit-stacked-information-type/+merge/114767?
<wallyworld> sure
<lifeless> rick_h_: I don't understand why modules we'd handle ourselves but base we'd handle by collaborating closely with upstream
<lifeless> rick_h_: I'm sure there is a key thing I'm missing :(
<rick_h_> I'd think it was purely that we're more hands on with modules, while 99% of the time hands off on YUI base
<rick_h_> 99% of the time YUI base is r/o while modules we dev are very write heavy with docs/tests/sharing
<lifeless> so you're saying that we can block a project waiting for a fix to yui base, but we want to run a local fork for changes to a module (whether we created the module or not)
<rick_h_> you'd use memcache all the time for very read heavy uses, but would you stick your writes in there? :)
<lifeless> rick_h_: thats called 'redis' :P
<StevenK> wgrant: Bah, I keep distracted by my QA
<rick_h_> well, I think we'd look at a system to carry a temporarity patch or else build a module that patched it for us
<rick_h_> gallery-patch-bug1234 and then use that until the next YUI version came out with the fix
<rick_h_> and then all of our teams would gain access to that patch module while we worked with upstream
<wallyworld> wgrant: can you add a screenshot to the mp?
<lifeless> rick_h_: so while I accept that it may happen *more often*, that seems like a reason to run the same system as yui base to me: so that there is no surprise for dealing with the time when yui base is the thing that needs fixing.
<lifeless> rick_h_: by run the same system I mean the dep/deploy/dev story around yui modules/yui base.
<rick_h_> lifeless: right, and that's fine. However, from my conversations, the dream of a shared JS area is so far away, I'm working around that for the time being
<rick_h_> and trying to work within the current dev environment to get LP caught up to the latest YUI, make it easier to test new versions, and keep the gallery modules we've subsumed where they are for the moment
<lifeless> sure
<wgrant> wallyworld: It looks like https://code.qastaging.launchpad.net/~canonical-launchpad-branches/lp-production-configs/lalala/+edit but with fewer information type choices
<lifeless> note that nothing you've said above speaks 'shared JS area' to me.
<rick_h_> now, when we get to a good answer for the gallery code, I'm all for revisiting the YUI base story into that lifeless
<wgrant> wallyworld: The UI hasn't changed. Just the set of choices
<lifeless> so theres still at least some causal link you need to help me understand :)
<wgrant> wallyworld: In general the only private stacked-on branches will be Proprietary in Proprietary projects, so there'll only ever be one choice.
<rick_h_> lifeless: sure, so I don't see a good way to manage and expose modules built by different teams than having a shred JS setup of some sort. Right now I'm told we're to try to push stuff up to the YUI gallery, but as we can't use that to pull/download from at the moment, it's a push only setup
<wallyworld> wgrant: ah right thanks. i had just read the covering letter and didn't quite grok the change
<lifeless> rick_h_: why can't we use that to pull or download from ?
<rick_h_> lifeless: well currently LP uses I think 2 gallery components, and we don't want to pull down all of hte gallery (size/complexity), and we need to be able to get whatever code into our download-cache, so it would be a pain to keep that up to date/in sync
<lifeless> how big is all of the gallery (size on disk, git clone time)
<rick_h_> lifeless: we also only want the build files, not the src/docs/etc.
<wgrant> wallyworld: It both reduces the amount of code we have for this unlikely case, and allows users to fix conflicts if they arise (eg. because the stacked-on branch changes type)
<rick_h_> lifeless: the git clone is approx 64MB
<rick_h_> lifeless: the build directory of only production files is 21M
<lifeless> rick_h_: you proposed cloning it al for a shared area, but that will also require devs to have a full copy etc, so I don't understand why you'd say we *don't want to pull...*
<wallyworld> wgrant: i think though we decided to disallow a stacked on branch changing type to private if a public branch was stacked on it?
<rick_h_> lifeless: right, as I said, we've not come to a satisfactory answer yet on that front
<wgrant> wallyworld: Ah, yes, so that particular case probably can't happen.
<rick_h_> lifeless: and while a dev can take a download hit, doing it for every build/etc seems something to avoid
<lifeless> sure, but why would we do it on every build/etc ?
<lifeless>  we don't bzr pull on every build/etc.
<rick_h_> lifeless: but agreed, there's work to be done on that front which is why I'm putting it off :)
<lifeless> I mean.
<rick_h_> lifeless: true, I guess download-cache is updated, but not redownloaded
<lifeless> You'd have to *write code* to pull etc on every build.
<rick_h_> lifeless: so we'd have to have something like that for the gallery/JS code
<wallyworld> wgrant: with the hidden types code, will the order of the radio buttons be messed up ie the (un)embargoed security choices come at the end
<lifeless> here are the computed (vs root cause) constraints I know of that are imposed on us in this area:
<wallyworld> wgrant: which would be inconsistent with elsewhere
<lifeless>  - be able to build and deploy LP without internet access
<lifeless>  - be able to do buildbot tests likewise
<lifeless>  - be able to inspect all changes happening to code that we execute in trusted zones - e.g. js that runs in our browser windows, python on the sever etc.
<wgrant> wallyworld: getInformationTypesToShow returns a set of allowed. setUpWidgets then iterates through the vocab, picking out items that are in the allowed set. So order is retained.
<lifeless> I think pretty much everything else is up for grabs
<wallyworld> wgrant: cool, thanks. just wanted to double check as i couldn't recall the exact implementation detail
<rick_h_> lifeless: ok
<wgrant> That implementation detail only landed three hours ago, so you're forgiven :)
<wallyworld> wgrant: and the diff didn't really show the full picture
<wallyworld> wgrant: are we missing a test?
<wgrant> wallyworld: I think so. I couldn't find one for the checkbox override.
<wgrant> ec2 will tell me for sure
<wallyworld> to check that security is allowed branches linked to security bugs
<wgrant> Oh
<wgrant> There's one for that
 * wgrant hunts
<lifeless> rick_h_: how big is a tar of all of yui3 gallery (not base) ?
<wallyworld> regardless of the stacked on type
<rick_h_> lifeless: I'll pull together some notes on ideas and why don't we schedule a time to sit down and look it over.
<wgrant>     def test_private_branch_with_security_bug(self):
<wgrant>         # Branches on projects that allow private branches can use the
<wgrant>         # Embargoed Security information type if they have a security
<wgrant>         # bug linked.
<rick_h_> lifeless: just the build or the whole thing?
<lifeless> rick_h_: something suitable for devs to have, buildbot to have, and prod to have.
<wallyworld> wgrant: ok, thanks. r=me
<wgrant> wallyworld: Marvellous. Thanks.
<lifeless> rick_h_: and is there a programmatic interface to do releases of individual yui modules? (are they separately versioned etc) ?
<rick_h_> lifeless: yes, they use an ant based build system and do weekly releases that are served
<rick_h_> so you can specify "load the gallery from date 2012-07-12"
<rick_h_> in your yui config
<lifeless> rick_h_: what granularity does that have ?
<rick_h_> lifeless: only a weekly build
<lifeless> not frequency, granularity :)
<lifeless> is it all of gallery, or per-module ?
<rick_h_> lifeless: it's all the repo
<rick_h_> lifeless: all the gallery
<rick_h_> lifeless: I'm not sure if you can specify it per module, you might be able to but not tried it
<lifeless> ok
<lifeless> so, here are some questions we might want to answer
<lifeless> rick_h_: oh, how big was that allofgallery thing ? Are you finding out ?
<lifeless>  - how big is allofgallery
<rick_h_> lifeless: ah sorry, got distracted
<lifeless>  -> would e.g. daily snapshots into the download cache be a significant burden?
<lifeless> rick_h_: you said something earlier that each module has multiple versions, is that right? so you can ask for a specific one that is known to work ?
<rick_h_> lifeless: so build tar/gz is 3.4M
<rick_h_> lifeless: yes, when you write a module you specify a version string, but we've not used it so far
<lifeless> so the weekly snapshot includes an unversioned tip of the module + all the prior still supported releases?
<lifeless> (that yui host)
<rick_h_> lifeless: so the weekly snapshot builds the tip of all modules, and serves it via a new url specific to that build date. Each build is a new url.
<lifeless> ok, so you *can't* stick to an arbitrary old version trivially ?
<rick_h_> I know and have used that to go back in time for the whole gallery, but not seen how to use that on a per-module basis
<lifeless> righto.
<rick_h_> lifeless: I don't know, will look into it
<lifeless> indeed thats a question too
<lifeless> I mean, if you only get snapshots-of-tip granularity, a shared JS area for all of Canonical would likely be not an improvement over working directly upstream
<rick_h_> lifeless: true
<lifeless> because you'd still have folk sitting on a known-good, and then upgrading many bits at once and dealing with the fallout.
<lifeless> unless we engineer something special using introspection etc; but at that point we could do that upstream and then work upstream with less friction.
<lifeless> I don't think we have quite enough knowledge yet to progress this further
<lifeless> what other questions should we queue up
<rick_h_> lifeless: right
<lifeless>  - how do other YUI users deal with this?
<rick_h_> well, the big questions are is this feasible? will teams work together and want to do so. How will it be managed, and how to get this setup for local dev.
<lifeless>  - how often would we really be updating the gallery cop(ies) we have - assume we update to a) get bugfixes b) to work with a new YUI and c) to have newer versions of modules we're contributing to.
<rick_h_> lifeless: honestly I'm not sure. While I know teams using it, I've not seen places doing a lot of cross-team work like we're looking at
<lifeless> rick_h_: I haven't seen any need for cross-team stuff yet, in all that you've discussed.
<lifeless> rick_h_: I've seen you propose it :)
<lifeless> rick_h_: cross team work is super hard
<rick_h_> lifeless: well that's the point. Is that with all of this, we're worried about a single gallery build breaking a team using an older version that missed something that changed
<rick_h_> which would mean another team submitted a fix/update for a module that caused it to change
 * rick_h_ is missing how this isn't cross team 
<lifeless> rick_h_: I think you have a hidden assumption somewhere.
<lifeless> :)
<rick_h_> lifeless: more than likely
<lifeless> uhm, its not cross team because you can (naive, may need tuning): drop yui-gallery-XYZ.zip into download cache, pull that out like we were yui base-XYZ.zip, drop in additional single-module zips for things we fork (so they are visible)
<lifeless> setup and document how to do it a way for folk like e.g. sinzui or wgrant or $anyone to do a new snapshot as needed.
<lifeless> none of that implies shared cross team stuff; the place for collaboration is yui gallery upstream.
<rick_h_> lifeless: sorry, but I really need to wrap up. It's late here and I'm hoping to unblock buildbot before bed
<rick_h_> lifeless: ok, I see
<lifeless> Ok, short term suggestion: revert it back to what it was before.
<StevenK> A revert is the easiest.
<rick_h_> lifeless: ok, I'm hoping to get the ok to push it with http://paste.mitechie.com/show/735/ as that moves the conflicting point out to where it really belongs
<StevenK> rick_h_: I'm happy to revert your revision out if you want to crash.
<rick_h_> StevenK: ok, if that's easiest will do then thanks. I'll revisit the buildout setup and adjust from there then and see if I can come up with a place for the helper code to assist in getting updates going
<rick_h_> s/will do/to do
<rick_h_> ugh
<lifeless> rick_h_: From what I've heard so far the canonical specific shared area is entirely orthogonal to handling yui + gallery + odd forks of gallery modules effeciently for LP
<rick_h_> thanks StevenK and wgrant for the help and lifeless for the discussion. I've got some tweaking to my long term vision to work on
<rick_h_> lifeless: right, I'm just trying to start with the goal of updating YUI in LP and leaving the gallery/etc for a later date
<lifeless> rick_h_: if LP can do it efficiently with a direct relationship with upstream, that provides a template for other Canonical projects to also work with it efficiently and collaborate with upstream
<rick_h_> lifeless: agreed, and we've started that process. We have our first module inthe gallery yesterday, and I've been working to facilitate discussion with the PES team and their modules and upstream
<lifeless> I think - I'm sure - that its easier to replicate a template of working with an established group that setting up a new group which would amongst other things have to broker with upstream.
<rick_h_> lifeless: lifeless but as working with upstreeam doesn't help us at all in our deploy/serving area, I held out a vision of a single 'canonical-gallery' at some point in the future (loooong future)
<lifeless> rick_h_: (discussion) - anytime. sorry I had to ask so many questions :)
<lifeless> rick_h_: from the sounds of it though, a canonical gallery wouldn't help either.
<rick_h_> lifeless: no it's good, it's going to focus me on thinking more of the tar/offline mode than I originally was
<lifeless> rick_h_: also note that anything on a new domain name will make first-view loads for LP slower.
<rick_h_> lifeless: right, it has weak spots as well
<lifeless> rick_h_: about 3 seconds slower.
<lifeless> rick_h_: (well, 3-10, but definitely 3).
<rick_h_> ok, you know better than I on that front
<lifeless> SSL :)
<StevenK> rick_h_: Revert is landing.
<rick_h_> 3s for a SSL handshake to a second domain?
<lifeless> rick_h_: you can test - add a 300ms latency to your outbound packets, see the world the way the non-Europe folk do :)
<rick_h_> StevenK: ty much sir
<StevenK> rick_h_: Easily.
<rick_h_> ok, I'm spoiled from my network/location :)
<lifeless> rick_h_: http://wiki.bazaar.canonical.com/SmartPushAnalysis1.4#Network_connection
<StevenK> rick_h_: Move to Sydney/Melbourne/Christchurch and use LP.
<rick_h_> ok, so as I said, I need to refocus the long term goal for sure, but it is a bit different from the current issue of 3.5 on LP and making sure we don't spend a year getting to 3.6 after it's out in a month/two
<lifeless> yah
<lifeless> totally.
<StevenK> BAH
 * StevenK stabs PQM, and adds [testfix]
<rick_h_> StevenK: heh, I asked deryck if anyone's complained on the combo loader and we decided if the aussies didn't raise a fuss it must be working out ok :)
<lifeless> FWIW I'd use the current approach which handled the 3.4->3.5 OK AIUI, and look at rearranging as part of doing the gallery.
<rick_h_> lifeless: rgr
<lifeless> rick_h_: I haven't looked up your mail about combo loader stuff yet either, I've been sick this week
<StevenK> rick_h_: I think the combo-loader needs to actually cache the files
<lifeless> am only just on deck now.
<rick_h_> lifeless: not a problem, I figure with it being Friday for me tomorrow I'd not turn on a FF then until next week anyway
<lifeless> cool, I'll look for it monday
<lifeless> rick_h_: is the combo loader oops enabled, and do we get perf information out of it ?
<rick_h_> StevenK: oh, I thought webops copied over the way the apache cache/etc is setup from U1/Landscape. Not peeked at it tbh
<StevenK> rick_h_: Revert landed as r15619
<rick_h_> StevenK: ty
<rick_h_> lifeless: no, convoy isn't oops enabled currently. I'd assume any perf info would be normal apache request/logging/etc
<wgrant> StevenK: It's pretty aggressively cached already, I believe
<wgrant> Yeah
<wgrant> Cache-Control: public,max-age=5184000
<StevenK> wgrant: Oh? I thought the caching was non-existant.
<rick_h_> the only issue is that each deploy breaks the cache
<StevenK> Like it should.
<rick_h_> since the url changes for deploy purposes
<wgrant> StevenK: It's probably done by the Apache config, as rick_h_ suggests
<rick_h_> right, just saying that might make it seen less cached with frequent deploys going on
<rick_h_> s/seen/seem
<lifeless> btw
<lifeless> this is a reason to have things like yui not part of lp's build area at all
<lifeless> version them entirely independently
<lifeless> may require some assembly.
<rick_h_> lifeless: agree, but definitely some assembly
<rick_h_> but again, perfect is the enemy of good
<rick_h_> step by step
<StevenK> wgrant: Join added. Now how to fix the other bug.
<lifeless> totally ;)
<rick_h_> ok, night all, thanks again and sorry to fubar buildbot on you guys
<lifeless> man, this is ripe: http://arstechnica.com/tech-policy/2012/07/verizon-net-neutrality-violates-our-free-speech-rights/
<lifeless> "Broadband networks are the modern-day microphone by which their owners [e.g. Verizon] engage in First Amendment speech"
 * StevenK peers at buildbot
<lifeless> uhm, nothing that my ISPs have *ever* said to me has been even slightly protected speech. Or would have been had I lived in the US.
<StevenK> bzrlib.errors.InvalidHttpResponse: Invalid http response for https://xmlrpc.launchpad.net/bazaar/: Unable to handle http code 502: Bad Gateway
<StevenK> Hmmm, at least its not my fault.
<StevenK> lifeless: At least it isn't RIPE.
<StevenK> wgrant: I can't say Branch.id IS IN () without toys being depramed, so what was your thought there?
<lifeless> Branch.id.is_in(...)
<StevenK> lifeless: Obviously. You're missing some context.
<wgrant> StevenK: Hm, possibly just exclude the branch query entirely in that case, then.
<StevenK> wgrant: But wasn't the bug query written in such a way that it would deal with no bug ids being passed, or am I on crack?
<wgrant> StevenK: Yes.
<wgrant> StevenK: Because in the case that no bug IDs were passed, it was correct behaviour to just operate on every bug.
<wgrant> But that's not correct any more.
<wgrant> Since you pass in a list of artifacts that you want to operate on.
<wgrant> Not just a list of bugs.
<wgrant> So if I pass in only bugs, it should only work on those bugs.
<wgrant> If I pass in only branches, it should only work on those branches.
<wgrant> If I pass in a few of each, it should work on those bugs and those branches.
<wgrant> If I don't pass in a list of artifacts, it should work on all bugs and branches that satisfy the other filters.
<StevenK> wgrant: But run() had if self.bug_ids <filter on bugs> else: every other filter
<wgrant> StevenK: Yes. Is that odd?
<wgrant> Because it didn't support branches before this, the logic was:
<wgrant> If I pass in only bugs, it should only work on those bugs.
<wgrant> If I don't pass in a list of artifacts, it should work on all bugs that satisfy the other filters.
<wgrant> fin
<StevenK> wgrant: Then just the join is needed
<wgrant> StevenK: Howso?
<wgrant> StevenK: Currently if I pass in a list of bugs, it will work on those bugs and *every* branch.
<wgrant> Won't it?
<StevenK> wgrant: http://pastebin.ubuntu.com/1089151/ but it's a bit messy
<wgrant> StevenK: I'd invert those flags, but something like that, yeah
<wgrant> StevenK: Alternatively, start bug_filters and branch_filters off empty, rather than with the visibility filter
<wgrant> StevenK: Keep the branch_ids/bug_ids/else code the same.
<wgrant> StevenK: Then wrap the two queries at the end in an 'if bug_filters' and 'if branch_filters', and insert the visibility clause there
<StevenK> wgrant: http://pastebin.ubuntu.com/1089155/
<wgrant> StevenK: You should probably put *_invisible_filter inside the conditional blocks, and maybe even inline them if they fit nicely.
<StevenK> wgrant: http://pastebin.ubuntu.com/1089206/
<wgrant> StevenK: Looks reasonable. Does it work?
<StevenK> That's a seperate problem.
<wgrant> Heh
<StevenK> wgrant: Spot the glaring error in the branch query.
<wgrant> StevenK: Hah
<wgrant> Indeed
<wgrant> findspec is in the wrong place
<StevenK> And no .using()
<wgrant> Yeah, but all the important bits are there :)
<wgrant> Just not the syntax...
<StevenK> wgrant: As an added bonus, http://pastebin.ubuntu.com/1089218/ passes tests.
<wgrant> StevenK: Did you write new tests?
<wgrant> To catch the issues?
<StevenK> Nope
<StevenK> Well, not yet.
<StevenK> wgrant: A great deal of the tests just pass in bugs, so I'm not sure why they don't tickle what afflicted qas.
<wgrant> StevenK: Because they don't check that the branch subscriptions survive
<wgrant> Because they only care about bugs.
<StevenK> wgrant: But the query was running for *ages*
<StevenK> So why don't the bug tests show that ...
<wgrant> StevenK: Because sampledata doesn't have hundreds of thousands of branches and millions of subscriptions
<wgrant> So the query will take milliseconds.
<StevenK> Oh. Right.
<StevenK> wgrant: So we want to sprinkle in StormStatementRecorder or what do you want to do?
<wgrant> StevenK: We want a test that tries to remove subscriptions from a set of bug IDs, and checks that those subscriptions are gone but other bug and branch subscriptions remain.
<wgrant> And the same but for branch IDs.
<wgrant> There are probably existing tests to whichyou can add a couple of lines to do this.
<StevenK> wgrant: Right, http://pastebin.ubuntu.com/1089239/
<StevenK> wgrant: I get two failures if I shelve the changes in sharingjob
<wgrant> Great
<StevenK> wgrant: Commit, push and force wallyworld to review it since we effectively wrote it together?
<wgrant> Plus he's OCR
<StevenK> % bzr log | head -n 8 | tail -n 2
<StevenK>   Fix RASJ to deal with branches correctly without spawning a query that will
<StevenK>   take approximately three eons to complete.
<wgrant> (and do the wrong thing)
 * StevenK stabs Firefox for crashing
<wgrant> StevenK: Crashing, or the DnD hang that sprung up in the last major release?
<StevenK> That looked like a crash, the window disappeared followed by the process
<wgrant> Ah
<StevenK> wallyworld: O HAI. https://code.launchpad.net/~stevenk/launchpad/fix-rasj-and-branches/+merge/114779
<StevenK> wgrant: http://pastebin.ubuntu.com/1089253/ is that other thing I'm working on.
<wgrant> Ah, I;ve solved commercial subscription expiry too.
<StevenK> wgrant: Do you think I'm missing something from my branch?
<wgrant> StevenK: Which one?
<StevenK> wgrant: [15:08] < StevenK> wgrant: http://pastebin.ubuntu.com/1089253/ is that other thing I'm working on.
<wgrant> StevenK: Rather hard to say. Does it work?
<StevenK> wgrant: test_bugnotification works at least
<wallyworld> StevenK: hi, just got back from school pickup
<wallyworld> bad traffic today in the rain :-(
<StevenK> Bright, sunny and 23degC here today
<wallyworld> StevenK: we will conflict. i also have removed those unused imports, but yours will land first since i'm just completing the covering letter for mine
<wgrant> Dreary, dismal and 12degC or so :)
<StevenK> wgrant: So normal Melbourne weather then.
<wgrant> Hey, from what I've heard SE QLD has been pretty similar for the last week...
<wgrant> Although I guess warmer :)
<StevenK> wgrant: Did the Victorian goverment write a stern letter asking for Melbourne's weather back?
<wgrant> StevenK: No, that wouldn't involve building more jails or decomissioning educational institutions :)
<StevenK> Haha
<StevenK> Maybe your government needs to turn the educational institutions into jails.
<wgrant> That approximates their current policy.
<StevenK> Haha
<wallyworld> StevenK: looks nice, r=me
<StevenK> wallyworld: At least you weren't sobbing saying "RBSJ looked nice until you touched it inappropiately and renamed it." :-P
<StevenK> steven@undermined:~% uptime
<StevenK>  16:00:06 up 16:50,  4 users,  load average: 3.71, 3.23, 2.15
<StevenK> Sigh
<wallyworld> StevenK: nah, all good. i started out with a generic job but at the time it was messy so just got bugs working.
<StevenK> test_sharingjob needs to good spring clean at some point
<wgrant> Everything we've touched does :)
<wgrant> A lot can be cleaned up once sharing is fully deployed
<wallyworld> and some of the other sharing tests need XXXs removed and (likely) some branch tests added
<StevenK> Apparently, reading from USB is *hard* or something.
 * cjwatson celebrates his 100th LP branch landing
<wgrant> :)
<wgrant> Pre-LP me has competition :(
<cjwatson> Yeah.  Little bit of a way to go yet.
<nigelb> dammit
<nigelb> wait
<nigelb> cjwatson gets on a different list.
<nigelb> wgrant: nah, you don't.
<nigelb> unless I boot up my vm and start contributing.
<wgrant> Heh
<adeuring> good morning
<jml> cjwatson: congratulations!
<jml> cjwatson: how are you celebrating?
<cjwatson> jml: coffee
<cjwatson> and more branches :)
<jml> cjwatson: coffee is fantastic, wonderful and slightly bitter. What better way to celebrate landing a hundred Launchpad branches?
* adeuring changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: adeuring | Firefighting: - | Critical bugs: 4.0*10^2
<jam> anyone have familiarity with buildout? I'm trying to do a bootstrap using python-2.7, and
<jam>  it is telling me 'could not install pyinotify-0.9.3' but there is no other error, and I can see the .tar.gz available
<mgz> ...the windows installers use it!
<jam> mgz: yes, and we are super happy about it there :)
 * mgz doesn't have familiarity of it either :)
<czajkowski> wgrant: have you ever seen an error message like this when a user tries to log in. https://answers.launchpad.net/launchpad/+question/203032
<wgrant> czajkowski: I reassigned that to SSO a couple of minutes back. It usually means they refuse to accept cookies
<czajkowski> bah that will teach me to open my work in tabs and come back to things
<czajkowski> wgrant: cheers
<Tribaal> hi all
* bac changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: adeuring, bac | Firefighting: - | Critical bugs: 4.0*10^2
<jam> jelmer: ping
<jelmer> jam: pong
<jam> jelmer: sorry for the delay, otp, but I'd like to have you pick up some of the 'get launchpad running on python-2.7 on Lucid'
<jelmer> jam: sure, which things in particular?
<jam> right now python-2.7 is segfaulting, and we probably need to bring in barry
<jelmer> jam: okay
<jam> jelmer: I guess the other option is doku, but I think barry has done more on that particular ppa
<jelmer> jam: still no idea on what's causing the segfault?
<jam> jelmer: 'import ctypes' seems to be a pretty common trigger
<jam> it is failing right now trying to build pyinotify
<jam> but I notice that there is a 'import ctypes' in there.
<jam> I'm pinging barry in #canonical
<jelmer> what do we use pyinotify for anyway?
<jam> also, jelmer, mgz, vila: You probably want to keep in touch with eachother while I'm gone. I imagine at least doing the standup. (I realize vila is gone next week)
<jam> jelmer: it doesn't really matter, ctypes gets imported at other times.
<jam> bzrlib probably improts it, etc.
<jam> jelmer: not sure if we have to have ctypes, but segfaulting regardless is bad :)
<jelmer> jam: does that mean that even "python -c 'import ctypes'" segfaults?
<jam> jelmer: yes
<jam> I did just try that directly
<mgz> we can do a jelmer/mgz standup on tuesday
<mgz> perhaps at 9pm
<jam> mgz: heh, whatever works for you :)
<jam> you guys work all day long anyway, right?
<mgz> at least some days I stop at 6 :)
<mgz> I have lp on lucid up and working and the ctypes crash from the ppa,
<mgz> so should be able to continue from where you are.
<mgz> is there a neat way of getting gdb to look in the right place for src?
<mgz> given it has paths like the following for ppas:
<mgz> /build/buildd/python2.7-2.7.2/Modules/_ctypes/libffi/src/closures.c
<cjwatson> based on the similar python3.2 failure a while back I suspect it's a pretty easy build fix.
<mgz> sure, I'm leaving that to barry, just following along at home :)
<czajkowski> jelmer: can you please follow up on the RT I gave you yesterday when you get a chance being poked for an update on the ppa
<jelmer> czajkowski: ah, yes, sorry
 * jelmer is having an unproductive day
<czajkowski> :(
<czajkowski> jelmer: need the link ?
<jelmer> czajkowski: no, I still have it open here; thanks though
 * mgz sends jelmer some cuddles
<jelmer> thanks  mgz
* adeuring changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: bac | Firefighting: - | Critical bugs: 4.0*10^2
<bac> jcsackett: darn, someone beat me to it:  https://launchpad.net/jujube
#launchpad-dev 2012-07-15
<jml> "Report a bug" does not work for me.
<wgrant> jml: The version on Product:+index works, Product:+bugs appears to have broken.
<wgrant> https://answers.launchpad.net/launchpad/+question/203216 and https://bugs.launchpad.net/bugs/1024866
<_mup_> Bug #1024866: "report a bug" link is not working <Launchpad itself:New> < https://launchpad.net/bugs/1024866 >
<jml> wgrant: thanks.
#launchpad-dev 2013-07-08
<wgrant> StevenK: Why shouldn't bug #941926 crash?
<_mup_> Bug #941926: InvalidTransition: Transition from Completed to Waiting is invalid. <oops> <package-copies> <queue-page> <Launchpad itself:Triaged> <https://launchpad.net/bugs/941926>
<StevenK> wgrant: I just thought it should actually give a nice error message rather than an OOPS
<wgrant> StevenK: Why shouldn't it OOPS?
<StevenK> Because we have a policy
<wgrant> Sure
<StevenK> WHERE "auditor_auditor"."object" IN (foo) GROUP BY (object), (operation), "auditor_auditor"."object", "auditor_auditor"."operation", "auditor_auditor"."date"
<StevenK> Dear Django, I HATE you
<wgrant> But I could fulfill the word of that policy with an unconditional except:
<wgrant> That displays an error page
<wgrant> Turning data corruption into an error message rather than OOPS is the opposite of progress
<StevenK> wgrant: It can happen when data isn't corrupted, though
<wgrant> StevenK: How?
<wgrant> Complete job with unaccepted queue entry sounds like data corruption to me
<StevenK> acceptFromQueue() doesn't check, it just forces it to be accepted
<StevenK> wgrant: Archive Admin A and B both load +queue, A accepts the sync, while B goes to make a coffee. B comes back, the job has completed and submits the form -- OOPS.
<wgrant> Right, so you're papering over a real and dangerous bug
<wgrant> Surely the problem is the double accept
<wgrant> If acceptFromQueue can today be called on a DONE or ACCEPTED entry, we have a data corruption bug
<wgrant> That is the bug here, not the InvalidTransition OOPS
<StevenK> Oh, screw you Django
<StevenK> wgrant: Hm, we already check
<StevenK> wgrant: Hm, it will croak on ACCEPTED, but not DONE
<cjohnston> Ursinha: ping
<Ursinha> cjohnston, pong
<cjohnston> hey Ursinha.. Was there any further progress ever on exporting all subscribers for blueprints?
<Ursinha> cjohnston, not really, I guess we need to start a discussion somewhere to figure out the impact and what to do (I guess that's what was said, iirc)
<cjohnston> hrm.
#launchpad-dev 2013-07-10
<StevenK> wgrant: http://pastebin.ubuntu.com/5860489/
#launchpad-dev 2013-07-12
<achiang> hello, is there a way to prevent LP from sending email to a team every time there is a merge proposal? i can't seem to find the email setting that controls that
<lifeless> achiang: the team subscription
<achiang> lifeless: i can manage subscriptions on a branch level, but can't find a setting for an entire team
<lifeless> the team will have a subscription on the target branch
<achiang> lifeless: ok, right. so we have like 90+ branches...
<lifeless> achiang: a script may be useful.
<achiang> lifeless: yeah, that's what we're doing now
<achiang> thanks
#launchpad-dev 2014-07-08
<cjwatson> wgrant: Can you think of any faster way to do the RTM "which SPPHs to copy" calculation than to basically do ubuntu.main_archive.getPublishedSources(distro_series=utopic, pocket="Release") and walk through the whole collection?  There are about 35000 elements in that right now, and I guess maybe a couple of thousand more by August.  I'm sure that's faster than doing lots of individual getPublishedSources calls, but wondering if I ...
<cjwatson> ... should be adding new API first
<wgrant> cjwatson: grep-dctrl?
<cjwatson> On what?  I'm not necessarily forking from today's state
<wgrant> xnox has an app for that.
<cjwatson> And that still leaves me with querying for all the SPPHs anyway
<wgrant> But I wouldn't be averse to enabling filtering on datepublished > X and (datesuperseded IS NULL OR datesuperseded > Y)
<wgrant> You don't need the SPPHs, just the versions.
<cjwatson> Oh, true
<cjwatson> xnox: Remind me where your archive wayback machine is?
<cjwatson> The datepublished > X component of that wouldn't be very useful, incidentally.  Some of the SPPHs in question might well just have been published when utopic was created.
<wgrant> Er yeah.
<wgrant> datepublished < X
<cjwatson> Ah yes
 * cjwatson tests materialising the whole gPS collection to see whether this is worth optimising in the first place
<wgrant> My condolences.
<wgrant> though sources might not be so bad, I guess.
<wgrant> Possibly only a thousand requests.
<cjwatson> That terminal window wasn't doing anything else anyway
<wgrant> SPPHs scoped to series and archive might be doable without any special indices, but we might need to investigate GiST over a tsrange to get adequate performance.
<cjwatson> Hopefully we can get it from already-published Sources.
<wgrant> That's the ideal.
<cjwatson> Failing xnox's wayback machine, I could hack archive-reports to stash copies for a while
<wgrant> Exactly.
<xnox> cjwatson: i have one locally, what dates are you interested in?
<cjwatson> xnox: Roughly August 1-15
<wgrant> Argh, I need to sort out overrides this week.
<cjwatson> I do not expect you to have this yet :-)
<xnox> cjwatson: utopic?
<cjwatson> Yes
<cjwatson> xnox: This is for forking ubuntu-rtm in about a month
<cjwatson> xnox: If you don't have it somewhere public already, maybe it's easier for me to just start stashing Sources files now
<xnox> cjwatson: i'm like, hm, which year =))) ah. right. there is github.com:xnox/apt-mirror.git
<xnox> cjwatson: or, i need a machine which at times uses up to 8GB of ram (efficient git repack requires to store the largest blob in RAM and thus not use too much disk-space)
<xnox> cjwatson: i could run it on e.g. snakefruit.
<cjwatson> Hum.  Maybe this is overkill.
<xnox> otherwise it eats up disk-space quickly
<wgrant> xnox: Huh, what's the big blob?
<xnox> well this is arching *all* pockets though.
<wgrant> Unless you're storing gz/bz2, this should compress well and easily.
 * xnox should measure how much it is to archive just one series.
<xnox> at the moment my .git is 3.3GB + 4.6GB current tree
<cjwatson> dists/utopic/*/source/Sources.bz2 is 8M total, snakefruit has 356G free
<xnox> it's all dists/ for all ubuntu suites, and only uncompressed files are commited into history.
<cjwatson> I could just stash them all
 * jpds wonders if xnox has heard of git-annex.
<xnox> wgrant: .gpg do not compress at all, as they are full re-writes on each publish cycle.
<cjwatson> wgrant: customs maybe?
<wgrant> customs would be much bigger than that, surely.
<wgrant> Though I guess the isos might compress well.
<xnox> cjwatson: i believe the right solution is to do round-robin type of thing somehow, with e.g. rsync /rsnapshots / hardlinks?! Cause it doesn't make sense to store per 15minute resolution indefinately.
<xnox> and that would keep disk/memory usage constant.
<cjwatson> We don't have to store indefinitely; for this purpose we're interested in a fairly narrow window, we just don't know exactly when in that window.
<wgrant> If I were doing this I'd just store the non-custom, non-compressed bits in a git repo forever.
<cjwatson> I'd have to get git installed on snakefruit, but we could run apt-mirror-snapshot out of archive-reports for a shortish period of time.
<wgrant> Apart from the small OpenPGP sigs they should compress very well.
<cjwatson> Or indeed forever if it works well enough, yeah.
<xnox> with my silly git thing, I do essently 2x rsyncs (archive & ports), verify all .gpg to have consistent tree, commit *.gpg Packages Release, and have a mini front-end to query timestamps and generate .gz .bz2 on the fly.
<xnox> or one can check them out.
<cjwatson> Doing it from archive-reports guarantees the right granularity.
<xnox> (frontend is separate script, from the snapshotter)
<cjwatson> And we could discard the first two steps of that.
<xnox> well, all you need then is just $ git init .; git add -A; git commit -m 'auto'. In that directory. And then repack/rewrite to discard useless stuff.
<xnox> and a proper .gitignore to skip useless things.
<xnox> (that can be recreated)
<xnox> (*.gz *.bz2)
<cjwatson> Materialising gPS for utopic release takes about 20 minutes on my ADSL, BTW.
<xnox> if we have proper dists/ for the right publisher cycle, we are done. Or I can bring up canonistack instances and run them from now till september. And stash copies somewhere e.g. people.canonical.com
<xnox> jpds: i haven't used git-annex, as it's typically never installable in devel releases =)))))) </heretic>
<cjwatson> It's typically installable in devel, just not in devel-proposed :-)
<xnox> i know :-P
<cjwatson> OK, so it sounds like I just want to get git on snakefruit and then do roughly as you suggest above
 * cjwatson files an RT for the former
<xnox> cjwatson: and if you make that .git repository clonable to me, I can pull it to my servers & provide nice public frontends from my servers to query it on per timestamp basis et.al.
<xnox> reliable snapshotting which doesn't get OOMed, is the thing i'm missing to make snapshotter interface public.
<cjwatson> snakefruit has 6G of RAM; if this requires a ton of RAM I can't guarantee that
<xnox> cjwatson: so, git commit will always succeed. (it only needs RAM to hash the largest file), But git repack may fail, thus .git may be growning in size. If you don't go $ git repack -A -d --window 9999 --depth 9999 you should be fine.
<wgrant> Heh
<wgrant> That's going to OOM on just about any repo.
<xnox> if disk-space becomes an issue, and you get OOM to repack it to safe disk-space then we'd need to do something, e.g. split/graft/offload history.
 * xnox should think of a round robin solution and estimate required disk-space there. And that will have little memory requirements.
<cjwatson> wgrant: Do PackageBuildFormatterAPI and ArchiveFormatterAPI perhaps want to gain the distribution name?
<cjwatson> wgrant: Mind if I take the "Optimise publish-distro phase A" Asana task?  I think I understand what shape things ought to be
<wgrant> cjwatson: Lovely, that's exactly the first step I was going to do.
<wgrant> cjwatson: re. the formatter APIs, they'll all use the new Archive.reference that I'm about to land.
#launchpad-dev 2014-07-09
<cjwatson> Huh, sqlobject doesn't have GROUP BY?
<wgrant> cjwatson: SQLObject has just about nothing.
<wgrant> cjwatson: Stormify that stuff.
<wgrant> But you probably want DISTINCT distroseries, pocket anyway.
<cjwatson> Stormifying xPPH seems like it might take a while
<wgrant> Hm, what are you doing?
<cjwatson> Looking at the next stage of optimising publisher phase A, so changing getPending*Publications to return everything; seemed like it'd be easiest to group it by series to get roughly the same ordering (hence logging) as before
<wgrant> That's ORDER BY, not GROUP BY.
<cjwatson> Oh, er, cough.  I speak SQL good, I learn him from a book.
<wgrant> Heh
<cjwatson> And SQLObject has ORDER BY, good.
<wgrant> Also, getPendingPublications doesn't use any of the other xPPH methods; you can Stormify it alone without a problem.
<cjwatson> Oh, I guess that's true, yes
<wgrant> Hm
<wgrant> Tempting to fix all the suite references while I'm touching almost the same code for archive references.
<wgrant> Also not sure whether to leave the reference for primary archives as 'ubuntu', or to make it 'ubuntu/primary' like the rest.
<cjwatson> I always found ubuntu/primary weird.  I think just ubuntu is an improvement.
<cjwatson> Marginally.
 * wgrant tries to track down all of the ancestry implementations.
<wgrant> There's NascentUpload.get(Source|Binary)Ancestry and lp.soyuz.adapters.overrides, which I think are the two that matter, but then I'm pretty sure there's at least another two elsewhere that are used for varying purposes.
<wgrant> PackageUploadSource.getSourceAncestryForDiffs
<wgrant> And check_copy_permissions
<wgrant> I think all except getSourceAncestryForDiffs are relevant for ubuntu-rtm, as check_copy_permissions assumes a package will require manual approval if no overrides are found.
<wgrant> cjwatson: Oh, https://code.launchpad.net/~wgrant/launchpad/archive-references-everywhere/+merge/226061 basically includes your pcj-repr changes.
<wgrant> I can back those bits out, I guess.
<cjwatson> wgrant: Oh, I can just withdraw my branch
<wgrant> That's probably easier. I had tests running overnight after I basically grepped through and replaced almost every reference to archive.owner.name and archive.name.
<wgrant> So it wasn't pushed yet, sorry.
<cjwatson> Not a problem.
<wgrant> cjwatson: For the ubuntu-rtm needing to inherit ubuntu's overrides problem, I'm thinking we might just want a flag on DistroSeriesParent which lets overrides fall back to a parent series' primary archive.
<wgrant> Also, did we decide that they wanted to skip NEW in the fallback case?
<cjwatson> A flag would seem sensible.  I thought we wanted to just do NEW as normal in the fallback case though.
<cjwatson> I believe we agreed it was OK for #ubuntu-release to manage that.
<wgrant> I couldn't remember, either works.
<wgrant> getBinaryAncestry and getSourceAncestry die tonight.
<wgrant> http://paste.ubuntu.com/7769369/ is my notes for the various cases atm.
<cjwatson> We only want to force component to universe/multiverse in some limited cases.
<wgrant> cjwatson: Don't we always want to do that if there's no ancestry in either archive?
<cjwatson> New binaries uploaded to Ubuntu with source in main default to main.
<cjwatson> For source, yes.
<wgrant> Er yeah, forgot that binary override complexity, good point.
<Eisbrecher_xnox> is it still the case that backports can't build-depend on other backports? or has that been fixed?
<cjwatson> That's been fixed.
<Eisbrecher_xnox> cool.
<cjwatson> https://bugs.launchpad.net/launchpad/+bug/888665
<_mup_> Bug #888665: Backports can't build-depend on other backports <soyuz-build> <Launchpad itself:Fix Released by adconrad> <https://launchpad.net/bugs/888665>
<wgrant> cprov: Thanks.
<cprov> wgrant: no problem, I am still owning you few reviews.
#launchpad-dev 2014-07-10
<wgrant> cjohnston: Do you want me to have a look at the queries in your publisher branch?
<wgrant> Bah
<wgrant> cjwatson: ^^
<cjwatson> wgrant: Oh, if you can, yes please.  I'm not going to have time before tomorrow morning, and possibly not tomorrow in general as I'm on landing team duty.
<wgrant> doing
<cjwatson> wgrant: I did wonder whether it would be better to just leave the release-pocket-updates checks out of the query and just exclude those after the fact
<cjwatson> Since hopefully they're fairly uncommon
<wgrant> cjwatson: Not just fairly uncommon, but probably something that should be noticed.
<cjwatson> (Do you use EXPLAIN ANALYSE for this kind of thing?)
<wgrant> Z, but yes
<cjwatson> ANALYZE, sorry.  Bah, Americans
<wgrant> Well, EXPLAIN (ANALYZE ON, BUFFERS ON) nowadays.
<wgrant> BUFFERS tells you exactly how many pages are read, so you can see if it's just working because of hot caches etc.
<cjwatson> Oh, nice
#launchpad-dev 2014-07-11
<wgrant> cjwatson: I guess if we log OOPSes for pubs that can't be published (which the error you have now will do), the queue of pending pubs should never exceed a single run, so the (archive, status) index will be fine.
<wgrant> Will never exceed that a single run, excluding the cases that OOPS because there are new pubs in invalid suites, that is.
<wgrant> So I'd probably just remove the allowUpdatesToReleasePocket check from getPending*Publications and land it.
<xnox> Can recipes that target devirt ppa, be build on devirt builders?
<xnox> at the moment 12 ppa builders are disabled as well, which doesn't help.
<cjwatson> Recipes are only ever built on virtualised builders
<cjwatson> <cjwatson@amber ~/src/canonical/launchpad/lp-branches/devel>$ grep -H is_virtualized lib/lp/code/model/sourcepackagerecipebuild.py
<cjwatson> lib/lp/code/model/sourcepackagerecipebuild.py:    is_virtualized = True
#launchpad-dev 2015-07-06
<blr> wgrant: am I right in understanding that https is disabled for vhosts in the testrunner, but allvhosts.configs['mainsite'].rooturl renders using 'https
<blr> 'https' on staging/production?
<wgrant> blr: Ah yes, that's a point I'd forgotten.
<wgrant> HTTPS is enabled on pretty much everywhere except the testrunner.
<blr> no obvious way to disable cert validation for `go get`... reasonably confident this is fine, but would like to test.
<wgrant> Knowing them, they probably statically link the CA database into the compiler :)
<blr> looks like I'll have to write some go! tls.Config{InsecureSkipVerify : true}
<blr> hahah
<blr> mwhudson: wgrant: no apparent way to disable a cert check in https://code.google.com/p/go/source/browse/src/cmd/go/http.go
<wgrant> blr: You could also use a variant of rf-setup-certs from lp-dev-utils to get a launchpad.dev cert that's valid on your local machine. go appears to respect /etc/ssl/certs/ca-certificates.crt like it should.
<blr> wgrant: thanks, will try that
<mwhudson> i'm sorry this is working out to be such a pain!
<blr> mwhudson: not a pain, I'm just still unfamiliar with parts of LP. It would help if go get had a flag though :)
 * mpt resists the urge to redesign Launchpadâs team page
<cjwatson> wgrant: rf-setup-certs!  Thanks, I'd wondered how people had handled that in the past
<cjwatson> I hacked things out brutally when I was testing bazaar codebrowse changes recently
<blr> morning
#launchpad-dev 2015-07-07
<wgrant> blr: Can you have a look at https://bugs.launchpad.net/launchpad/+bug/1471426?
<mup> Bug #1471426: AttributeError: 'BinaryPatch' object has no attribute 'get_header' <code-review> <email> <fallout> <oops> <regression> <Launchpad itself:Triaged> <https://launchpad.net/bugs/1471426>
<blr> wgrant: sure
<wgrant> mwhudson: Is go get happy with qastaging now?
<blr> wgrant: seems to be, although it complains about no go source for the repos I've tested
<blr> mwhudson may want to confirm with an actual golang repo
<blr> wgrant: trying to write a test for this.. in what context do we have binary patches in the diff text?
<cjwatson> blr: You can see a couple in https://code.launchpad.net/~blr/launchpad/project-meta-go-import/+merge/262144
<blr> cjwatson: sorry, which patches are you referring to?
<wgrant> blr: gitbranch*.png
<wgrant> blr: "Binary files [...] differ"
<blr> wgrant: ah I see thanks
<blr> given a binarypatch has no header, do we throw them out? there could be a comment there
<cjwatson> we can't throw out anything, a comment might go anywhere
<mwhudson> blr: seems ok to me, fighting the fun that is qastaging codehosting
<blr> mwhudson: great, glad something is working today >.<
<mwhudson>  OOPS-ec59fa242226c1c084ff47bddab04e86
<mwhudson> on trying to register a new series
<mwhudson> well, actually on trying to view the new series
<mwhudson> https://qastaging.launchpad.net/pydoctor/series
<mwhudson> blr: known?
<blr> mwhudson: hmm no, I wonder why I didn't see that on turnip's series page
<wgrant> blr: Probably because therer's no default branch on that new series.
<mwhudson> i was about to add one :)
<wgrant> Oh, there's also a correctness bug in that path.
<wgrant> Ah, the bugs are one and the same.
<blr> wgrant: in https://qastaging.launchpad.net/~mwhudson/pydoctor/new-trunk ?
<wgrant> blr: ProductSeriesView.golang_import_spec is buggy.
<wgrant> A template crashing with a LocationError usually means that the expression raised an AttributeError.
<wgrant> The TALES interrpreter helpfully swallows the AttributeError for you so you can't see what it actually is.
<blr> right, have seen that before.
 * mwhudson twitches
<blr> wgrant: ah, development_focus can be None
<wgrant> blr: It can, but that's not the issue here.
<wgrant> Actually, can it? I forget.
<blr> hmm ok
<wgrant> It's not the cause of this crash, though, as it is set for pydoctor.
<blr> ok have the binary patch handling sorted, just tidying up this test.
<blr> wgrant: ah, that should just be self.context.branch
<blr> waiting for the binary patch branch to scan
 * blr rounds up the goats
<blr> wgrant: could you have a look please https://code.launchpad.net/~blr/launchpad/golang-meta-import-buggy-productseries/+merge/263994
<blr> made the product series test clearer as well hopefully.
<wgrant> blr: You pushed the test change to the wrong branch.
<wgrant> golang-meta-import-https-fix rather than golang-meta-import-buggy-productseries
 * blr sighs
<blr> thanks
<blr> should have made a new branch
<blr> pushed
<wgrant> blr: Both reviewed, thanks.
<blr> wgrant: thanks, I believe a BinaryPatch will always have a dirty header
<wgrant> blr: It's still rather confusing for it to be inside that block. It doesn't need to be.
<blr> yes, agreed.
<blr> wgrant: unblocking buildbot, then will get those two branches landed.
<blr> and hopefully can now figure out why elmo's diff was so mangled.
<blr> wgrant: hmm I wonder if blank newlines are not being stripped, bzrlib won't parse subsequent patches otherwise.
<wgrant> blr: Have you got a test that reproduces it?
<blr> wgrant: I have a failing test yep
<wgrant> Great.
<blr> will dig into bzrlib again in the morning.
<mwhudson> huh, https://bugs.launchpad.net/launchpad/+bug/1465467 is deployed now, but i don'
<mup> Bug #1465467: put <meta name="go-import"> tags on project, series pages <git> <qa-ok> <trivial> <Launchpad itself:Fix Released by blr> <https://launchpad.net/bugs/1465467>
<mwhudson> t see the meta tags ? :(
<cjwatson> mwhudson: I see one on https://launchpad.net/germinate
<cjwatson> not on code. though
<mwhudson> cjwatson: hmm
<mwhudson> cjwatson: i was looking at https://launchpad.net/pydoctor
<mwhudson> bzr vs git?
<cjwatson> Yeah, I looked at a git example
<cjwatson> blr: ^- looks like you have more to do here :-/
<mwhudson> this simple sounding feature seems to be cursed :)
<blr> mwhudson: did you set a default vcs?
<mwhudson> blr: oh right, no
<mwhudson> blr: is there going to be a garbo job to set that to bzr when trunk is already bzr?
<blr> it will only render if you've one into code and set bzr (no need to change any of the form fields)
<blr> mwhudson: good question, that would certainly be nice.
<mwhudson> blr: i think its essential to be able to remove the special case code from the go tool
<blr> mwhudson: ok will discuss that with william and colin in 5m (we have our LP meeting)
<mwhudson> or we'd have to ask the maintainers of every go package on lp to visit the code config page...
<mwhudson> blr: ta
<blr> mwhudson: looks like the tag is rendering now at least :)
<mwhudson> blr: can confirm that that fixes it, yeah
<mwhudson> (forgot about that part from qastaging)
<blr> well, less than obvious I think. Hopefully we can run a job (we have an 'inferred_vcs' that we could potentially use to set product.vcs globally)
<cjwatson> ah, you decided not to make it inferred_vcs?  I guess that would have performance problems.
<cjwatson> we'd talked about backfilling, yes.
<blr> cjwatson: yes, I think wgrant was concerned about performance there.
<cjwatson> makes sense.
<blr> mwhudson: we'll look at garboing that in the next few days.
<mwhudson> blr: thanks
#launchpad-dev 2015-07-08
<blr> wgrant: one problem (possibly not the only problem), is the the semantics of dirty headers are still weird (I could have implemented this better in bzrlib)... each group of dirty headers is only associated with the first patch, where really all of the following patches before a blankline should be associated.
<blr> so currently if the diff looks like [dirty, dirty, patch, patch, patch, dirty, ...etc] we get [{dirty: foo, patch: <patch0>}, patch1, patch2]
<blr> where really we should probably have [{dirty: foo, patches: [<patch0>, <patch1>, etc]}]
<blr> bzrlib of course only sees dirty lines for the first patch it parses
<blr> I think I do have an interim fix though which should demangle the mail diffs.
<wgrant> blr: Why wouldn't it be [{dirty: foo, patch: patch0}, patch1, patch2, {dirty: bar, patch: patch3}]?
<blr> wgrant: if the first 3 patches are from the same file?
<wgrant> blr: Three patches can't be from the same file, but the hunks could be.
<blr> hmm, sorry you're correct, I think the diff_text in the test is actually in an impossible state.
<blr> wgrant: would you mind looking at test_codereviewcomment.py
<blr> presumably the 'baz' and 'foo' patch headers would always have a dirty header
<blr> and in practice would always be preceded with a newline
<wgrant> blr: In which branch?
<wgrant> I don't see a baz in test_codereviewcomment in devel.
<blr> wgrant: hmm second patch in the diff_text
<blr> wgrant: http://pastebin.ubuntu.com/11840023/
<blr> wgrant: that first dirty header has always been in the sample data, but presumably it is incorrect?
<wgrant> blr: What's wrong with it?
<blr> ah they're supposed to represent directories
<blr> in which case it should be bar.py and baz and foo should have their own dirty headers?
<blr> I don't think we generate diffs with patches in succession like that without headers.
<wgrant> That diff looks believable to me.
<wgrant> A directory was added, which is only indicated by a === line.
<wgrant> In our parsing, that will appear as a dirty line on the patch that follows it.
<wgrant> Now, technically it has no relationship with that patch, but it doesn't particularly matter if we associate them.
<blr> wgrant: in what case would the second and third patches not have dirty headers (or a blankline)?
<wgrant> blr: I don't know if we'd generate them like that (though git might), but there's no reason we can't parse them AFAICS.
<blr> wgrant: sure, we are parsing them like that now, it just didn't seem to align with reality
<wgrant> blr: Parsing them like what?
<blr> I've added another patch preceeded by a blankline and a new dirty header in this branch
<wgrant> Ah, it was misinterpreting the blank line case?
<blr> wgrant: I mean, we can parse a diff in that state.
<blr> well, we didn't have test data with a blankline/new header prior to this, which accounts for some of the failings in the tests.
<wgrant> Right.
<blr> wgrant: just wanted to ascertain if there were circumstances where we might generate a diff like that.
<wgrant> Yep, I understand now.
<wgrant> git may, not sure.
<wgrant> We should test the various cases.
<blr> sorry my brain is a little fried after looking at this all day
<wgrant> No gap, blank gap, dirty gap, dirty and blank gap.
<wgrant> Oh I know the feeling,
<blr> I'm probably not making much sense :)
<blr> right
<blr> wgrant: anyhow, I'm parsing elmo's big diff and it seems to be fine now
<wgrant> blr: Great, small bzrlib fix?
<blr> no, I think bzrlib is actually fine.. the problem there was a malformed hunk header in my test which made me erroneously think the problem was in bzr
<blr> wgrant: our comment count includes the blanklines before dirty headers which bzrlib throws away
<wgrant> blr: But if bzrlib throws them away then how can we know if they were there?
<wgrant> (unless you do what Colin suggested, and keep track of which lines bzr emits yourself)
<blr> wgrant: we consistently render a blankline before a dirty header
<blr> well, all but the first
<blr> Perhaps this should all be refactored, but this fix does appear to work, so perhaps we should land it and revisit it.
<wgrant> blr: So you just assume there's always a blank line between patches?
<blr> wgrant: I've written two tests using elmo's diff, not sure I should commit them however, that logic is basically tested in other tests, but I did want to parse a real (large) diff
<blr> well, that does appear to be the case.
<wgrant> blr: Yeah, no point having such a huge diff (of proprietary code, no less)
<cjwatson> blr: There's always a blank line between bzrlib-generated patches, but not between git-generated patches.
<cjwatson> blr: So we can't assume there's a blank line there.
<cjwatson> blr: And we probably shouldn't insert one into git patches either.
<cjwatson> wgrant: Shall I take it from your review that you would like me to fix IPersonSet.getByEmail(None) to return None?
<cjwatson> AttributeError: 'NoneType' object has no attribute 'lower'
<wgrant> cjwatson: Oh, hm, nevermind then.
<cjwatson> OK.  That was why I left the wrapper in place.  Though I'll have another look since we seem to be unnecessarily running things through safe_fix_maintainer again when we could pass in an unadorned e-mail address instead.
<cjwatson> unnecessarily running things through safe_fix_maintainer again> ignore that, looking at the wrong branch.  So I'll just land this.
<blr> morning
<blr> cjwatson: then perhaps we should not count them when generating the comment dict
<blr> cjwatson: we also have no test using a git generated diff
<blr> so afaict, we either need to change the behaviour of the caller, or add some heuristics to detect a bzr (unified) vs git (coimbined) diff.
<blr> not certain if bzrlib.patches can parse a combined diff actually.. I suppose I should test that :)
<wgrant> blr: The comment dict is a map of physical diff lines to comments.
<wgrant> blr: We can't not count them.
<wgrant> The parser which is used to generate the email must be able to cope with the line being there or the line not being there.
<blr> wgrant: I think it will have to be special cased for git anyway.
<wgrant> blr: Why?
<blr> git diffs are not unified diffs?
<wgrant> They conform to every definition of that term that I know of.
<wgrant> How do they differ?
<blr> wgrant: hunk headers differ
<wgrant> blr: In what way? The only difference I know of is that the --- and +++ lines don't carry timestamps.
<blr> git combined diffs can have hunk headers comparing two or more files
<blr> I'm not sure bzrlib is going to parse those
<blr> I have yet to check though, it may happily.
<blr> have a look at http://git-scm.com/docs/git-diff under "combined diff format"
<wgrant> blr: Does the way we use pygit2 actually give those to us?
<blr> not certain
<wgrant> It would be pretty hostile for it to just give them to us without us asking.
<blr> wgrant: how would feel about some code then which checks for the presence of a blank line preceding a hunk header before passing the diff to bzrlib?
<cjwatson> bzrlib parses git patches justfine.
<cjwatson> tested that many times.
<blr> great, that simplifies things.
<cjwatson> all we need to do is to be able to cope with arbitrary stuff in between the bits bzrlib gives us.
<cjwatson> which is easy because bzrlib preserves the order
<cjwatson> at worst it skips some lines
<cjwatson> anyway, back a bit later, gaming
<wgrant> blr: If LP parses the patch before passing it to bzrlib then it might as well do the whole thing itself.
<wgrant> bzrlib needs to preserve the blank line(s), or we need to take Colin's approach.
<blr> yep, okay, I'll have a look.
<blr> wgrant: "if line.startswith('=== ') or not line.strip():" should do it? (or clause is new)
<wgrant> blr: That won't work.
<wgrant> blr: It will, for example, consider any blank line within a hunk to be part of the next dirty head.
<blr> hmm, in the context of this iterator, not certain how I can determine if the blank is followed by a dirtyheader
<cjwatson> also with git diffs I think there's a line starting with "diff"
<blr> cjwatson: and also "index"?
<wgrant> Yes.
<wgrant> There is an arbitrary set of junk between the end of one patch and the start of the next.
<blr> unfortunately there's nothing denoting the end of a patch, so presumably we will always have to regex the line
<wgrant> Is the junk just any line that doesn't start with '+', '-', '@' or ' '?
<wgrant> (or you could count the lines described in the hunk header. not sure which is messier)
<blr> junk is anything preceding '--- '
<blr> begiunning is set when the first patch line is encountered
<blr> so this should work: if beginning and line.startswith(dirty_headers) or not line.strip()
<blr> where dirty_headers is a tuple of things we care about
<blr> or maybe it should just all be preserved
<wgrant> "or not line.strip()" will catch lines in the middle of a hunk.
<blr> not if beginning is False?
<wgrant> Junk can't just be anything preceding '--- ', because the entire previous hunk precedes it.
#launchpad-dev 2015-07-09
<wgrant> cjwatson: That's quite a nice diff...
<cjwatson> blr: BTW in one of your test cases I noticed that you had " \n" as an inter-patch junk line, which seems inaccurate - normally in bzr it would just be an empty line, "\n"
<cjwatson> I begin to wonder whether just copying the code from bzrlib.patches and customising it to our particular needs mightn't be more sensible at this point.
<cjwatson> wgrant: Quite :-)  I ran a subset of tests to my satisfaction, though not the whole suite
<cjwatson> Will land tomorrow, don't want to have to testfix tonight
<wgrant> cjwatson: Heh, good plan.
<blr> cjwatson: wouldn't have mattered in that case, it just incremented the linecount for every dirty header other than the first
<cjwatson> Right, but in general a line starting with a space is part of a patch.
<cjwatson> So it probably wouldn't have actually been considered dirty.
<cjwatson> Hence why line.strip() is inaccurate - line.rstrip("\n") would be better, although it still only catches a restrictive subset of non-patch lines where really we should be open-ended.
<blr> as far as I can tell, bzrlib patches would need to be re-written, it has no notion of intra-patch junk
<blr> it only parses junk until the first patch header
<blr> it only works currently as we're explicitly matching against "=== "
<cjwatson> So, there are two basic approaches.
<cjwatson> Instead of throwing away lines that don't fit into any of its categories, it could remember them to attach as dirty headers to the next patch (maybe some slightly special handling of "=== ")
<cjwatson> Or, given that we have a full list of the lines in Launchpad and that bzrlib is currently returning a subset of them in monotonic order, we could spot when it's skipped lines
<blr> sorry called to lunch, but please continue
<cjwatson> The second could be done while continuing to use bzrlib.patches as it stands today
<cjwatson> But it involves at least one extra string comparison per line, so is less efficient
<cjwatson> The first could perhaps be done in bzrlib.patches, but rather than continuing to hack that about and fiddle with its semantics, it's not that much code, maybe it's better at this point to just clone-and-hack it and make it handle unrecognised lines in some more useful way for us than "continue"
<blr> ls
<blr> well, that could have been worse
<blr> cjwatson: I'll try that latter
<blr> will give me opportunity to remove the semicolons at least.
<blr> wgrant: cjwatson: shall we land this in it's current state to fix prod? https://code.launchpad.net/~blr/launchpad/bug-1472045-demangle-inlinecomment-mail/+merge/264105
<blr> will continue to work on a more general solution
<wgrant> blr: Have you tested whether that makes the git situation worse?
<blr> wgrant: no, there are no git specific tests.
<blr> hmm cgit renders a space between patches
<blr> wgrant: have a test with a simple git diff, could you have a look?
<wgrant> blr: That has a blank line between patches, unlike real git diffs.
<blr> wgrant: I've added a space in a hunk in the test, and fixed the spaces between patches.
<blr> it does appear to be working, aware that it could be better.
<blr> wgrant: where best to document the origin of patches.py, before or after the copyright notice?
<wgrant> blr: Perhaps the top.
<blr> wgrant: see you at some ungodly hour :)
<wgrant> blr: Heh, yep.
<wgrant> blr: Hm, test failure.
<wgrant> Ah, small.diff is buggy, fixing.
<wgrant> It has a blank line when it should contain a single space.
<wgrant> Trust the least thorough test to be the one that breaks :)
<blr> gah sorry, only ran the TestInlineCommentsSection tests.
<blr> wgrant: want me to get that, or are you already on to it?
<wgrant> blr: Just running the rest to ensure it doesn't break anything.
<blr> thanks
<wgrant> blr: Looks like it was just that. Will submit when it fails in a minute.
<blr> wgrant: great thanks
<wgrant> Ahhh
<wgrant> I know why elmo's diff was weird now.
<wgrant> bzrlib is quite cunning.
<wgrant> '\ No newline at end of file' ends up tacked onto the end of the preceding line.
<wgrant> So there's a line with two \ns.
<blr> wgrant: hmm, well that and the blank line handling
<wgrant> blr: Right, but we worked out that that didn't explain all of it.
<blr> hmm, any ideas on resolving that?
<wgrant> I have a fix which isn't too bad, writing a test atm.
<blr> ah great. although, I _did_ reparse elmo's diff and it seemed to align?
<wgrant> Oh, indeed, it doesn't suffer from this case.
<wgrant> I just threw other big diffs at it until it went wrong.
<wgrant> cjwatson: https://code.launchpad.net/~wgrant/launchpad/diff-no-newline/+merge/264254
<blr> wgrant: huh, that is surprising.
<wgrant> blr: Yes, not exactly what I'd expected.
<blr> thanks for persisting with that wgrant
<wgrant> blr: Thanks for the review. Will land once Colin fixes the weird xx-person-edit.txt failure.
<wgrant> That is also not the sort of thing I'd expect...
<cjwatson> wgrant: How entertaining, although the "line is a ContextLine/ReplaceLine" comment can no longer be true.
<cjwatson> wgrant: Annoyingly, I spotted the problem by code inspection just before buildbot did.
<wgrant> cjwatson: Ah, fair point.
<wgrant> Heh
<cjwatson> Misplaced (and unnecessary) classImplements transformation
 * wgrant out for a bit.
<blr> see you in a few hours
<wgrant> Huh
<wgrant> cjwatson: Interesting failures now...
<cjwatson> the hell?
<wgrant> I was going to go a bit stronger than that, but yes.
<cjwatson> I mean, sure, I touched announcement, but only rather trivially.
<wgrant> It also didn't fail last time, did it?
<cjwatson> Nope.
<wgrant> And the test count was at least roughly corrrect.
<wgrant> It doesn't look plausibly spurious, but it must be.
<cjwatson> Not in stdio from the last run either.
<wgrant> Unless a testrunner died, but I don't see that either.
<wgrant> Force and pray.
<cjwatson> And you'd expect more missing tests from that.
<cjwatson> I'll just run those ones locally a few times to make sure
<wgrant> Unless you caught the end of a small layerr.
<wgrant> Indeed.
<wgrant> gah, bad keyboard.
<wgrant> cjwatson: No luck reproducing the failures?
<cjwatson> wgrant: Sorry, that container was busy with other tests at the time, I'm running them now
<cjwatson> Can't reproduce so far.  Forcing and praying.
<wgrant> Yep...
<wgrant> postgres was probably just having a bad day.
<wgrant> Looks like buildbot is happy, good.
<cjwatson> Don't jinx it!
<wgrant> Come at me, buildbot.
<wgrant> cjwatson: Not even a thousand lines?
<wgrant> 79 +@adapter(IBugBranch, IWebServiceClientRequest)
<wgrant> 80  @implementer(Interface)
<wgrant> Does that actually make sense?
<wgrant> Hm, something must be referencing it. How odd.
<wgrant> Oh, the registration is named, right.
<blr> night
<wgrant> Night blr
<cjwatson> night, thanks
<cjwatson> wgrant,blr: If you could get QA done and a deployment sorted out over my night, that would be very helpful to the phone folks
<cjwatson> Then I could turn on ubuntu-rtm/15.04 translation imports
<blr> cjwatson: ack
<pepee> hi. how come comments in launchpad don't have anchor tags?
<blr> pepee: if that would be useful to you, by all means file a bug and tag it 'feature ui'
<pepee> ... I'm guessing it would be useful for me and everyone else. I haven't found a way of showing a comment in its context
<pepee> I doubt no one has suggested that
<blr> pepee: yep, appears there is a bug
<pepee> which one?
<blr> https://bugs.launchpad.net/launchpad/+bug/124166
<mup> Bug #124166: browsing to bug comment permalink should show comment in context <lp-bugs> <ui> <Launchpad itself:Triaged> <https://launchpad.net/bugs/124166>
<blr> https://bugs.launchpad.net/launchpad/+bug/627252
<mup> Bug #627252: no comment permalinks for issue comments <lp-answers> <ui> <Launchpad itself:Triaged> <https://launchpad.net/bugs/627252>
<pepee> hehe
<blr> I'll see if I can get to those next week, shouldn't be too hard.
<pepee> thanks!
<wgrant> blr: Yeah, I'd much prefer that the comment-specific URLs redirected to an anchor on the bug page. The dynamic loading JS complicates that, however.
<blr> wgrant: yep, the single comment view is of questionable value.
<blr> wgrant: we could check the location on comment load for an anchor and scrollTo
<wgrant> blr: But we'd have to also force that chunk of comments to load.
<pepee> can there be both things? like a URL parameter to select between a single comment and the whol context
<pepee> like in forums...
<blr> wgrant: true, that is a little tricky. could we always pre-render a map of ids to comment chunks?
<blr> err s/render/generate/
<blr> on load, check the location for an achor, match to the map, load the appropriate chunks
<blr> no idea how feasible that is, I need to look at the code again
<wgrant> pepee: Why?
<wgrant> What's the use of seeing just the comment?
<pepee> wgrant, same reason people do both things in forums, some may want to show only one comment, some may want to show the whole thread, and one specific comment in context
#launchpad-dev 2015-07-10
<blr> I think having two views is unnecessary and possibly even confusing.
<wgrant> Indeed, people linking to single comments in forums is just annoying.
<pepee> some comments give you more than enough info for what you are trying to show. for example, a workaround or specific info
<pepee> that way, when linking you'd say "do what this link says"
<pepee> I'm not a webdev, but... I'd leave it as it is, and simply add anchors
<wgrant> But then you need two links to each comment.
<pepee> people already know that lp links to comments anyway
<wgrant> How do you distinguish them?
<pepee> yes, which is what forums do
<pepee> you click on the date, and get a link. you click on the number, and get an anchor
<pepee> ... IIRC
<wgrant> That doesn't make sense to me (and the number goes to the out-of-context page today)
<pepee> yes, it would be the inverse, but... why not?
<wgrant> I don't think "date goes to anchor, number goes out of context" is particularly understandable.
<pepee> I'm just giving an example of how I think it could work
<pepee> for the dual link/anchor thing
<wgrant> Right, but I still really don't see a compelling argument for the single-comment view.
<wgrant> We either have to have two links that non-obviously go to slightly different pages, or a single in-context view.
<pepee> I mean, that's the way it works now. it doesn't make sense, but it's already working that way
<pepee> instead of changing something, add something
<wgrant> No, a link to a comment is always out-of-context.
<wgrant> There aren't two slightly different links today.
<pepee> yes, which I why I asked if someone could add anchors to LP
<pepee> I don't even use forums that much, but sometimes context is not necessary, most of the time it is... depends on your needs
<pepee> other reason would be to, say, reduce data usage
<pepee> some bug reports are lengthy
<wgrant> Right, and we only load 100 comments by default for that reason.
<wgrant> In the anchor case, we'd load the 100 comments around that comment.
<pepee> oh well, I'm not sure if I'm misreading or just not convincing you... in any case, I won't argue anymore
<pepee> thanks for your work blr
<pepee> also, please make sure it works if you disable JS...
<wgrant> That's the only issue I see with the lack of out-of-context view.
<wgrant> I'm happy to be convinced that the out-of-context view is useful, but I haven't seen a strong argument yet.
<pepee> I'm saying that you could keep both, I have no clue about webdev, even less about webdesign
<wgrant> It is certainly technically possible to keep both, but it is undoubtedly confusing.
<pepee> in fact, apparently the "link to a specific post" feature in forums is kinda hidden...
<wgrant> You seem to think we should keep the out-of-context view, and I'm interested to know why.
<pepee> because it can be useful
<pepee> imagine you are trying to show someone some specific command that was posted in LP, there is no need to show the whole bug report, you just link to that comment
<wgrant> But is there a need to not show the whole bug report?
<pepee> or you are talking about the bug report, but only want to show what someone said
<pepee> no, there is no need to hide the context, but... there is also no need to load so many resources
<wgrant> For the server, the overhead of having a separate view is greater than just rendering the bug.
<wgrant> And the extra download size wouldn't be significant.
<pepee> $ curl https://bugs.launchpad.net/xserver-xorg-video-intel/+bug/1432194/?comments=all | wc -c   352923
<mup> Bug #1432194: Graphics unstable on Ubuntu 14.04 and 14.10 using Intel HD Graphics 5500 <amd64> <trusty> <verification-done> <OEM Priority Project:New> <xf86-video-intel:Unknown> <xserver-xorg-video-intel (Ubuntu):Fix Released> <xserver-xorg-video-intel-lts-utopic (Ubuntu):Invalid>
<mup> <xserver-xorg-video-intel-lts-utopic (Ubuntu Trusty):Fix Committed> <xserver-xorg-video-intel (Ubuntu Utopic):Fix Committed by tjaalton> <https://launchpad.net/bugs/1432194>
<pepee> $ curl https://bugs.launch  pad.net/xserver-xorg-video-intel/+bug/1432194/comments/85 | wc -c  14160
<blr> github shoes the entire thread, and scrolls to the relevant comment fwiw.
<blr> shows
<wgrant> cjwatson: You said that the new lazr.delegates broke lazr.restful. Does that only affect the test suite, or are we likely to run into problems with your branch that upgrades LP?
<blr> although there's no notion of a comment view there
<pepee> 14160 vs 352923 bytes, > 20x, and that's only the page
<pepee> and as I said, my guess is that most people will want show the comments in context anyway
<wgrant> Right, and the question is whether the very few people who want them out of context justify the overhead and confusion of keeping the extra page around
<wgrant> (and comments=all is cheating!)
<pepee> I'm using firefox + noscript, and it's the only way I could see all the comments, but you can imagine that some threads are as lengthy as that one...
<pepee> thing is, there must be a way to show them that the old "link-to-specific-post" feature exists
<pepee> my way is simply looking at the url by moving the mouse pointer, now I'm not even sure how forums do that
<pepee> I was wrong when I said that forums have links in the date
<pepee> ahh, vbulletin removed this 5 years ago, heh
<pepee> oh well, don't mind me...
<wgrant> pepee: Hm, so they only provide the in-context link now?
<wgrant> It's slightly more awkward for us, because we don't have a batched view, I admit.
<pepee> http://www.vbulletin.com/forum/forum/vbulletin-4/vbulletin-4-suggestions/362540-open-single-post
<pepee> "showpost.php has been removed for SEO reasons and I don't think it will come back."
<pepee> I just don't get why it wasn't done that way from the start
<pepee> could have copied functionality in forums...
<wgrant> blr: How's the email testing going?
<blr> wgrant: ok, although I had forgotten how awful evolution is. Did you want to deploy soon?
<wgrant> blr: Yep, my testing seems fine as well. We can deploy whenever you want
<blr> wgrant: diffs are all looking healthy
<blr> wgrant: is there a binding for forcing a refresh in offlineimap?
<blr> going to grab some lunch. 17614 should be green in a sec.
<wgrant> blr: I've never used offlineimap.
<blr> pleased I can test mail on qastaging, less painful than I thought it would be.
<blr> ok, back in a bit.
<cjwatson> wgrant: Barry fixed that in lazr.delegates 2.0.3, which is the version I upgraded to.
<wgrant> cjwatson: Ahh
<wgrant> Heh
<wgrant> That didn't go well...
<wgrant> Or the new lazr.delegates is really, really fast.
<wgrant> Hum
<wgrant> cjwatson: Error: Couldn't find a distribution for 'nose==1.1.2'.
<wgrant> Ah, it's in ztk-versions.cfg and lazr.delegates newly requires it.
<wgrant> I guess you added it to lp-source-dependencies but didn't commit?
<wgrant> Fixed.
<cjwatson> Oh whoops, probably had junk in my download-cache and didn't notice
<cjwatson> Thanks
<ev> is https://app.asana.com/0/26780198332250/ up to date with yesterday's stakeholder meeting?
<beuno> ev, I didn't add my "split out gpg and ssh storage to a separate service", could you throw that in?
<ev> yup
<rpadovani> https://code.launchpad.net/~wgrant/launchpad/webhook-api/+merge/264005
<rpadovani> Guys, I swear you're making me want to learn well python to help you
<rpadovani> (and if you start to work also on the UI of lp I definitely switch from helping ubuntu touch / oxide to lp
<cjwatson> rpadovani: we are totally interested in taking patches for it even if we aren't currently focused on it :)
<cjwatson> and will be happy to help mentor people who're motivated
<rpadovani> I'm really tempted... A couple of weeks I finish uni exams and then I think I'll ping you with a couple of questions ;-)
<cjwatson> sure thing
#launchpad-dev 2015-07-11
<xnox> cjwatson: wgrant: will merge / release lazr.* & launchpadlib things soon.
<cjwatson> xnox: I was going to do a release of several of them fairly shortly after making sure I have a good python3 test harness for them
#launchpad-dev 2015-07-12
<blr> morning
<blr> wgrant: have a garbo class for backfilling product.vcs, but not certain how it is run. garbo-daily?
<wgrant> blr: I'd go with garbo-hourly or garbo-daily. It should only take one run, either way.
<blr> wgrant: thanks. how was your weekend?
<wgrant> blr: No power yesterday, for what I think was the coldest day of the year, but otherwise not too bad.
<wgrant> Yourself?
<blr> Yes I saw there was a bit of snow there. Good, had power fortunately!
<blr> you're going to need to invest in a generator at this rate.
<wgrant> That's hopefully the last of it.
<wgrant> They were replacing a whole lot of power poles on the nearest major road.
<wgrant> I guess they did it on a weekend because they had to close 2/3 of it.
<wgrant> But still, worst day of the year for it.
<blr> wgrant: do you have a log burner or open fire?
<wgrant> Nope.
<blr> oh dear, gather that was a bit miserable then :/
#launchpad-dev 2016-07-15
<sparkiegeek> hi, I have a MP which has generated a rather long diff, enough that LP is truncating it. Is there a way of persuading LP to show me the full diff? (via URL hackery or similar)
<cjwatson> sparkiegeek: I'm afraid the only option is to check out the branches and generate it locally
<cjwatson> I think ...
<cjwatson> sparkiegeek: URL to the MP?
<sparkiegeek> cjwatson: fair enough, I discussed with reviewers and agreed a different approach
<sparkiegeek> cjwatson: I actually have a follow-up question - can I tell bzr/LP that a file is "binary" even if technically it isn't? the large file in question is generated code, which shouldn't be reviewed, it's just an artifact that's version controlled for REASONSâ¢
<cjwatson> sparkiegeek: so you should be able to use the "Download diff" link to get the whole thing, albeit not syntax-highlighted
<sparkiegeek> cjwatson: right, we really were hoping for inline comments on chunks past the first "page"
<sparkiegeek> don't worry, problem has been worked around (for now)
<cjwatson> Yeah, this is basically "if we do that then we time out due to spending too much time rendering"
<sparkiegeek> ack
<cjwatson> bzr's logic for detecting a binary file is just "is there a NUL byte in the first 1024 bytes"
<cjwatson> which is crude but apparently mostly pretty effective
<cjwatson> and indeed that's pretty much diff's logic too
<cjwatson> don't see a way to override that
<sparkiegeek> right, was hoping for a way to hint to LP that it should treat /this/ file as a blob
<cjwatson> in subversion I'd suggest setting the content-type or something but bzr doesn't have that degree of per-file properties
 * sparkiegeek nods
<cjwatson> it would have to be a facility in the underlying VCS rather than in LP, really
<cjwatson> now, git *does* have this facility
<cjwatson> https://git-scm.com/docs/gitattributes
<cjwatson> so you can have this if you switch to git :)
<sparkiegeek> haha
<cjwatson> (in LP, obviously ...)
<sparkiegeek> and a series of headaches to go along with it :D
<sparkiegeek> thanks though
<cjwatson> best I can do, sorry
<sparkiegeek> np, it's good to have confirmation either way!
#launchpad-dev 2017-07-10
<Terry_> How to use codehosting locally with git?
#launchpad-dev 2017-07-16
<frecel> hello
#launchpad-dev 2018-07-12
<cjwatson> wgrant: Would you mind re-reviewing https://code.launchpad.net/~cjwatson/launchpad/git-ref-remote-blob/+merge/348089 ?
#launchpad-dev 2018-07-13
<wgrant> cjwatson: Done
<cjwatson> Tested https://code.launchpad.net/~cjwatson/wadllib/fix-mime-encoding/+merge/348955 and it seems to be behaving itself fine
