#launchpad-dev 2010-05-10
<jelmer_> thumper: I'm surprised you're seeing errors on subversion.conf rather than Sqlite locking errors
<thumper> jelmer_: yeah, no idea really
<jelmer_> thumper: anyway, it shouldn't be all that hard to fix, somebody just needs to take a couple of hours and JFDI
 * thumper nods
 * jelmer_ -> sleep
<wgrant> thumper: Why can't you expose IBranchTarget?
<wgrant> You can expose an interface without a URL -- the object's URL is used.
<wgrant> Oh, unless objects don't really implement IBranchTarget, but adapt to it...
<thumper> wgrant: branch targets are adapted to the branch target objects
<jtv> danilos: yes, about permissions and flag-stealing policy...
<danilos> jtv, go ahead
<jtv> Did I remember correctly that the flag-stealing policy (how do I type that "O/+"!?) allows flag-stealing if (1) you're translating upstream or (2) you're translating in Ubuntu but with upstream privileges?
<jtv> We documented what the policy does, but didn't go into much detail (tired as we both were after the sprint I guess!) about where it came from.
<jtv> So I thought I'd better check.
<jtv> The reason I'm checking is that translating Ubuntu with upstream privileges will probably AIUI still produce different effects from translating directly upstream, and I haven't given much thought at all to whether that's what we want.
<danilos> jtv, so
<danilos> jtv, it comes from "external policy flag" which we assume is set to "prefer upstream" right now
<jtv> Plus your privileges, right?
<danilos> jtv, plus privileges, of course
<jtv> Ah yes, and then as a sort of optional extra in the design we talked about kyleN's scenario where upstream prefers Ubuntu rather than the other way around.
<danilos> jtv, right, that bit was kind of symmetric as well except that we don't have a way to set that option yet
<jtv> Someday we'll get around to thinking about how close we get to symmetry when you set both, but not today I hope :-)
<danilos> "set both"? what do you mean?
<jtv> When you make the Ubuntu translations follow upstream and vice versa.
<danilos> jtv, so, privileges here have a bigger "weight" if the option to prefer "current context" is set
<danilos> jtv, well, that's not really an option because well, you can't prefer ubuntu over upstream and upstream over ubuntu at the same time :)
<jtv> Not at the same time, but you could prefer the current context when you're editing Ubuntu and then again prefer the current context when you're working upstream.  :)
<jtv> But anyway
<danilos> jtv, it'd be nice to investigate if this policy can include translation focus handling, but I'd definitely be a bit worried about that without going through the same exercise we went through in Recife
<jtv> The sketch API I pushed Friday expresses the "weight" by passing a boolean for "translator is privileged upstream" to the Ubuntu translation method but not the converse to the upstream one.
<jtv> Ah, translation focus, that's the other thing I wanted to ask about.
<jtv> I've got some XXX's up on the wiki page.  Today I'm planning to outline how the share-upstream-focus-imports change affects things.
<jtv> Unless you have it all figured out already and just need some time to write it down, I'll just update the wiki page and ask for feedback.
<bigjools> wgrant: do you have any thoughts on extending the file copy check to binaries?
<bigjools> oh and I just spotted where you're sitting :)
<jtv> wgrant: duck!
<wgrant> bigjools: Where are you? I've tracked down StevenK.
<bigjools> you just looked at me
<wgrant> Bah.
<bigjools> haha
<wgrant> It's too dark back there.
<StevenK> Bwahaha
<bigjools> I can almost hear you laughing StevenK
<wgrant> Heh.
 * StevenK resolves to not laugh
<wgrant> bigjools: I don't think the file copy check is useful there.
<bigjools> I mostly agree
<wgrant> bigjools: Except for epochs, binaries don't conflict.
<wgrant> Only (component)orig tarballs do.
<wgrant> (epochs are a concern, but they're evil, so screw them)
<bigjools> heh
<StevenK> ARGH!
<wgrant> Uhoh.
<StevenK> HE SAID THE NAUGHTY WORD
<wgrant> Heh.
<wgrant> bigjools: So, our file data model sucks, and we have lots of bad data in production. What do we do?
<bigjools> fix it (tm)
 * StevenK tries to make a big destination archive to break copying
<wgrant> mozilla-daily?
<StevenK> I can't copy into that, though
<StevenK> Well, not without hand-waving
<bigjools> I just read: StevenK tries to ... break copying
<wgrant> INSERT INTO TeamParticipation [...]...
<bigjools> or go admin on DF :)
<wgrant> That was my intention, yeah.
<StevenK> I am wondering if it's a large source archive, or destination archive, or both
<wgrant> Destination.
<wgrant> The source archive doesn't matter at all.
<wgrant> The key metric for the old code is number of SPPHs with the target name in the target archive.
<wgrant> For the new code, it's the number of SPPHs in the target archive, but it's encapsulated in a DB query so it's not comparable.
 * wgrant vanishes.
<mwhudson> so ... why am i getting bucketloads of mail aimed at ~registry
<thumper> blame someone on bugs I guess
<thumper> I've had to create a mail filter
<thumper> X-Launchpad-Message-Rationale contains @registry -> delete
 * mwhudson tries to remember how thunderbirds message filters work
<bigjools> jml: are you free by any chance?
<jml> bigjools, I'm available.
<jml> bigjools, as in speech
<jml> bigjools, in Pomegranite downstairs.
<bigjools> jml: could you spare me some time in Kawi or are you ensconsced down there?
<jml> bigjools, sure. in a couple of minutes, poolie is laying out the bzr sprint
<bigjools> it's about the sftp server
<bigjools> jml: great, thanks
<jml> bigjools, https://lpstats.canonical.com/graphs/CodehostingCrowberryConnections/ -- this graph shows the connections to the codehosting ssh server
<jml> bigjools, you'll easily be able to see where we added the timeout :)
<bigjools> jml: it's not that obvious actually!
<jml> bigjools, https://lpstats.canonical.com/graphs/CodehostingCrowberryConnections/20081101/20100511/
<bigjools> jml: ok that one is :D
<jml> StevenK, what's your email address?
<jml> :(
<jml> mail server fail
<StevenK> jml: stevenk@{ubuntu,canonical}.com ?
<jml> bigjools, anyway, one hour timeout seems to work out just fine. I don't remember why it was so big though -- mwhudson might
<wgrant> StevenK: Have you actually pushed your latest changes?
<wgrant> Also, you should check out how it works on the primary archive .
<wgrant> With multiple SHA1s.
<StevenK> wgrant: Yes, I have.
<StevenK> wgrant: Oh?
<wgrant> StevenK: Because there is bad data.
<wgrant> And you need to be able to deal with it.
<wgrant> (or we need to fix it, but that may be impossible.)
<StevenK> Personally, I think we should fix it.
<StevenK> wgrant: Right, so we key off the filename, so one of the SHA1s will win.
<StevenK> Just thinking about it
<wgrant> Yes.
<wgrant> I guess we should run a query and find out how much bad data there is.
<wgrant> Someone with dogfood access could do that easily.
<allenap> sinzui: Hi there. I see that there are 4 identical download links on https://edge.launchpad.net/lazr.config. Is that a known bug?
<wgrant> allenap: That's correct. The file has been added four times.
 * sinzui looks
<allenap> wgrant: Ah, okay. Is that a bug in LP, or just a mistake?
<sinzui> allenap, that is not a bug, just incompance
<sinzui> ence
<wgrant> allenap: A mistake.
<sinzui> allenap, I deleted the duplicates
<allenap> sinzui, wgrant: Cool :)
<mwhudson> oh the irony
<mwhudson> jelmer: did you see that the dulwich import is failing?
<jelmer> mwhudson: yeah, bzr-git bug I need to fix
<jelmer> mwhudson: the wonders of dogfooding ;-)
<deryck> gmb, I've volunteered you for something at UDS. :-)
<gmb> deryck, Oh gawd.
<gmb> deryck, It's not inline upstream dupe searchign is it?
<deryck> gmb, it's not, but nothing major.  Just scope and estimate a way to show what external trackers use our plugin(s).
<gmb> deryck, Ah, cool.
<mars> sinzui, ping, is Edwin around this week?
<sinzui> mars, Edwin will be available tomorrow
<mars> sinzui, ok, thanks.
<wgrant> StevenK: Once you've finished fighting Twisted... did you end up checking what will happen if there are multiple SHA1s? It's probably going to non-deterministically fail. That is probably bad.
<jml> StevenK, https://code.edge.launchpad.net/~jml/launchpad/poppy-sftp
#launchpad-dev 2010-05-11
 * thumper lunches
<wgrant> noodles775: Are you guys lurking somewhere?
<lifeless> wgrant:
<lifeless> Branched 10725 revision(s).
<lifeless> real    9m29.912s
<lifeless> user    5m47.220s
<lifeless> sys     0m2.050s
<lifeless> we're looking at it
<wgrant> lifeless: Is that devel?
<wgrant> That's a fair bit better than I thought it was, but that may be because the latency here is tiny, I guess.
<noodles775> wgrant: In Flamboyant room atm.
<wgrant> bigjools: Why are gutsy and intrepid still accepting PPA uploads?
<bigjools> wgrant: forgetfulness?
<cody-somerville> Does anybody know how I can get postgresql to stop writing to pgstats.tmp?
 * bigjools thinks about making the das:+admin page lp,commercial
<wgrant> bigjools: Eww.
<wgrant> Aren't there enough ~admins here?
<bigjools> wgrant: ?
<bigjools> lp.admin is too restrictive
<wgrant> Possibly.
<bigjools> it's not ad admin function, it should be a distro manager's job really
<bigjools> s/ad/an/
<wgrant> Possibly.
<wgrant> But distro permissions are broken.
<bigjools> quite
<maxb> EdwinGrubbs: pong from friday
<mwhudson> danilos: https://bugs.edge.launchpad.net/launchpad-code/+bug/578331/comments/3
<mup> Bug #578331: exporting to bzr seems broken since a few days <Launchpad Bazaar Integration:New> <Launchpad Translations:Triaged> <https://launchpad.net/bugs/578331>
<danilos> mwhudson, thanks, how soon can we get this in? I think this is a CP candidate for us
<danilos> mwhudson, do you think we should change anything on the translations side as well, or should we just close the rosetta bug there?
<mwhudson> danilos: i'd hope to be able to hack something up today
<danilos> mwhudson, cool, thanks
<mwhudson> danilos: i _think_ it's probably a code task, not a rosetta one
<danilos> mwhudson, I guess there's not much sense in fixing all the other branches until the fix is in production
<mwhudson> danilos: i guess not
<mwhudson> the lock/unlock the branch trick should free up a branch
<mwhudson> but that can be done case by case
<mwhudson> danilos: have to go do uds stuff now, back in an hour
<danilos> mwhudson, right, it'd be nice to have a workaround for users wanting to unblock their translation exports listed there
<danilos> mwhudson, sure, thanks
<mars> gary_poster, can you see any problem with setting haltOnFailure=True if buildbot fails on shell_6 or compile_2?
<mars> gary_poster, I really don't see why we should run the test suite if either of those steps die.
<gary_poster> mars: on call; sounds ok on face of it
<danilos> jtv, regarding traits approach in your setCurrentTranslation branch; I really don't see value in constructing this parameter object, especially since you do it with a separate function on every call
<danilos> jtv, also, you are changing getCurrentTranslation to be specific as well, whereas it should be a mirror of setCurrentTranslation and get you a relevant one
<danilos> jtv, so, it's "incompatible" change
<jtv> danilos: that's where I didn't want to change what we already had, ironically.  :)
<danilos> jtv, but you end up doing it by not changing it :)
<danilos> jtv, since we are changing the underlying model, we have to change the "public" APIs to work around that change
<jtv> I do want that to change, but avoided doing it in this branch.
<danilos> jtv, that's why you shouldn't have touched those methods, because they are changed in my branch :)
<jtv> As for the traits object: it moved the getattr/setattr dirty work out of setCurrentTranslation, making it easier to read.  Otherwise it's a bunch of variables like "callable to read the other flag on a message"
<jtv> danilos: It's no problem to keep those as they are;
<jtv> that was just an afterthought since things became simpler
<danilos> jtv, right, so please do away with that; if you are trying to use getCurrentTranslation for testing, you probably shouldn't, but instead just check properties on the created/returned translationmessage
<danilos> jtv, and with that, you also won't need the 'getIncumbentMessage' on TranslationSideMessageTraits class, since that's what getCurrentTranslationMessage will do
<jtv> danilos: that's great... obviously a lot of overlap between what we're doing
<danilos> jtv, so, the only thing I see as meaningful in traits is get/setFlag and translations side, up to you if that still makes sense for a separate parameter object class
<jtv> danilos: yes, it still does afaic
<danilos> jtv, sure, then keep it in :)
 * danilos gets some lunch
 * jtv considers the same
<gary_poster> mars, off call, and looked at your question more carefully.  haltOnFailure=True is the default, I believe.  jml tried to fix this for shell 6 already.  I looked at what he did and it looked correct, but whatever it was didn't actually work.  Please tell me when you find out the error.
<gary_poster> I'm surprised that compile 2 is haltOnFailure=False now; that seems more patently obvious that it needs t succeed.  (shell 6 doesn't have to succeed if the revisions are already up-to-date).
<mars> gary_poster, any idea where I may find jml's shell 6 fix?
<cody-somerville> how can I stop someone spamming me with retrying a code import that fails?
<cody-somerville> ah, I know
<mars> gary_poster, regarding shell 6, it looks like it will succeed even if there are no updates.  {halt,flunk}OnFailure would still work fine.  The step would only stop on real errors.
<mars> gary_poster, and consider that stale sourcecode often causes test failures, then I would argue that sourcecode *must* be updated for a particular test run to be considered valid.  Stale sourcecode and passing tests /may/ lead to testcase false passes.
<jml> remind me what I did?
<jml> StevenK, hello. where are you?
<allenap> bac: Hello there, glad you made it :)
<gmb> BjornT, can I have your approval on https://code.edge.launchpad.net/~gmb/launchpad/halp/+merge/25058 please? This is a branch that landed on db-devel at the end of last week. I want to merge those changes into devel (they were db-devel specific last cycle but not any more).
<gmb> I've merged the revision in which the changes landed from db-devel into devel for this branch to avoid conflicts.
<gary_poster> jml, you made a change to lpbuildbot to make a failed update of sourcecode actually fail the build.  It didn't work, for a reason neither of us could figure out.
<jml> gary_poster, ahh, ok.
<wgrant> bigjools: We may need a further branch to fix some copies. http://paste.ubuntu.com/431607/ will tell us the problematic files, and how much we need to worry.
<bac> hi allenap ... thanks
<manish> gary_poster, Hey Gary. You have a few minutes to spare?
<gary_poster> manish: hey!  honestly, I'm dealing with a couple of high priority issues right now, but I don't want to send you away.  I'm also skeptical of actually being a help to you. ;-) But ask away, and I'll get back to you as quickly as I can.
<manish> gary_poster, I think it would be better that I ask you the doubts on the launchpad-dev list. You can answer the doubts in your free time and also those mails would be publicly archived.
<manish> gary_poster, You can go ahead with your work. Not something blocking as of now.
<gary_poster> manish: ok, great.  thank you again!
<BjornT> gmb: if it's just changes that have already landed in db-devel, that's ok. just be prepared to fix any conflicts after merge merge
<gmb> BjornT, Thanks, will do.
<EdwinGrubbs> maxb, ping again
<maxb> hi
<EdwinGrubbs> maxb, hi, I'm having a weird problem with the python-tickcount package in the ~launchpad ppa on lucid. If I run "dpkg -c" on it, it clearly contains a tickcount.so for python2.5, but when I install it, only the tickcount.so for python2.6 gets installed.
<maxb> aha
<EdwinGrubbs> maxb, btw, I'm running lucid 64bit
<maxb> I believe you are experiencing bug 576159
<mup> Bug #576159: launchpad-dependencies / rocketfuel-setup does not ensure all launchpad PPA dependencies have been installed  <Launchpad Foundations:Triaged by maxb> <https://launchpad.net/bugs/576159>
<maxb> Short version: update all your packages to latest *after* enabling the LP PPA and installing launchpad-developer-dependencies
<EdwinGrubbs> maxb, but I already have it all installed, and I've updated all my packages since then. Should I downgrade something and then uninstall/reinstall launchpad-developer-dependencies?
<maxb> You are saying 'apt-get dist-upgrade' does nothign?
<EdwinGrubbs> maxb, apt-get dist-upgrade installed some other packages, but it didn't fix tickcount.
<maxb> hmm
<maxb> Please: dpkg -l python python-minimal python-support, and pastebin
<EdwinGrubbs> maxb, https://pastebin.canonical.com/32165/
 * maxb != canonical
<EdwinGrubbs> I also just noticed that "aptitude update" says "Ign" next to the ppa urls.
<EdwinGrubbs> maxb, oops, http://paste.ubuntu.com/431781/
<maxb> ugh, the size of the columns has removed relevant info. Can you run dpkg -l .... | cat, which will cause it not to size itself to your terminal?
<EdwinGrubbs> sorry, here it is. http://paste.ubuntu.com/431782/
<maxb> OK, now I am fairly confused
<maxb> the versions all look file
<maxb> *fine
<EdwinGrubbs> btw, the /usr/lib/pymodules/python2.5 directory didn't even exist. I symlinked it over to /usr/lib/pyshared/python2.5, but it only has .py files and .so files.
<maxb> erm
<maxb> I would suggest that adhoc symlinking is liable to confuse matters
<EdwinGrubbs> maxb, I downloaded and reinstalled python-support and python-tickcount, and now the .so shows up.
<EdwinGrubbs> thanks for your help
<maxb> ok then
<maxb> A bit weird, I recently tested installing into a clean lucid chroot, and it worked fine there
<EdwinGrubbs> and pymodules/python2.5 also got generated correctly.
<thumper> morning
<Ursinha> thumper, morning
<Ursinha> thumper, let me ask something: is this a known problem? https://lp-oops.canonical.com/oops.py/?oopsid=OOPS-1591S1000
 * thumper looks
<thumper> WTF?
<thumper> ah staging
<thumper> we need to fix the staging puller script
<thumper> Ursinha: not known (is now)
<Ursinha> thumper, want me to file a bug ?
<thumper> Ursinha: yeah, sure
<thumper> it is just a config change in the cron scripts on staging
#launchpad-dev 2010-05-12
<maxb> thumper: Did you notice that ~vcs-imports have odd extra privileges when registering projects, too?
<maxb> ~vcs-imports members can set the owner and license_reviewed flag of projects when registering a new project, specifically
<thumper> maxb: I don't think so...
<thumper> maxb: that is registry experts
<sinzui> I can and do set every project I register or change as reviewed. And anyone in Registry Admins can too
<thumper> sinzui: ping
<thumper> sinzui: emailed you
<lifeless> hi thumper
<thumper> lifeless: hi
<thumper> lifeless: aren't you at UDS?
<lifeless> yes
<thumper> lifeless: early start then?
<lifeless> jetlag
<lifeless> had stomach ache style crookness yesterday afternoon, crashed out, woke at 4am
<mtaylor> lifeless: hey! we had similar days then ... I've been up since 2am after having gone to sleep at 6:$0
<mtaylor> 6:30
<mtaylor> thumper: you know what would be great - if merge proposals that were dependent on each other were sorted so that the one that includes all of the other ones was really easy to find...
<stub> Where is the mailinglist unsubscribe link hidden?
<wgrant> stub: It's pretty obvious on the team page for me.
<wgrant> In the 'Mailing list' section, an actual button labelled 'Unsubscribe'.
<stub> My eyes must have skipped the button looking for icons. duh.
<wgrant> Yeah, it is just about the only button.
<jelmer> mwhudson: were you publishing runs of pydoctor on launchpad anywhere?
<mwhudson> jelmer: http://people.canonical.com/~mwh/canonicalapi/
<jelmer> mwhudson: thanks!
<mwhudson> also http://people.canonical.com/~mwh/bzrlibapi/ fwiw
<jelmer> mwhudson: I've added a local rule for pydoctor as well, but for some reason it doesn't seem to include the css stuff
<jelmer> (https://code.edge.launchpad.net/~jelmer/launchpad/pydoctor, couldn't find anything existing in the launchpad sourcecode)
<mwhudson> jelmer: css stuff?
<jelmer> mwhudson: I'm not getting any of the colors that the URL you just mentioned has
<mwhudson> jelmer: i can't remember how that works :-)
<jelmer> mwhudson: hmm, I'll have a look then
<mwhudson> jelmer: but i'm pretty sure it should just be copying the css file into the directory
<mwhudson> jelmer: yes, it should copy apidocs.css to the target directory
<jelmer> mwhudson: what command are you running to generate the html docs at that location?
<wgrant> bigjools: Bug #527551
<mup> Bug #527551: Intra-archive copying of a source with a failed build may leave that source uncopyable <soyuz-core> <Soyuz:Triaged> <https://launchpad.net/bugs/527551>
<mwhudson> ~/pydoctor-2a/bin/pydoctor ~/db-devel/lib/lp ~/db-devel/lib/canonical/ --make-html --html-output ~/public_html/canonicalapi/ -q -q --verbose-about epytext-summary --project-name=Launchpad --docformat restructuredtext
<mwhudson> jelmer: ^
<jelmer> mwhudson: thanks
<jelmer> mwhudson: whoa, that's spitting out a lot of warnings in the new version
<mwhudson> yeah probably
<mwhudson> about a billion launchpad docstrings aren't actually valid rest
<mwhudson> for one thing
<thumper> jelmer: how goes the bzr-git, dulwich updates?
<jelmer> thumper: oh, right. I've deferred any bzr stuff so far, but I'd like to spend some time on bzr things this UDS.
 * thumper nods
<jelmer> mwhudson: If you have a chance, can you review the trivial branch at https://code.edge.launchpad.net/~jelmer/launchpad/pydoctor/+merge/25133 ?
 * jelmer goes back to the more complicated stuff
<jelmer> mwhudson: thanks
<jml> bigjools, wgrant: where are we at wrt getting visible stats for ppa usage?
<wgrant> jml: The script tried  to murder germanium.
<StevenK> It nearly did, too
<jml> wgrant, after which rollout?
<wgrant> The first run of the librarian log parser would have run into this too, but I don't know how it was solved then.
<wgrant> jml: It was run a couple of weeks after 10.03, IIRC.
<wgrant> I'm thinking I'll just tell it to stop after processing some x lines of log, where x is yet to be defined.
<wgrant> So it will take lots of runs to process the existing logs, but it won't eat all of the RAM.
<wgrant> The API stuff is all landed, but there's no UI at all yet.
<jml> you mean there's a manual process to get the historical data in?
 * jml has to go to a GTG thing
<maxb> Is there an ETA on a CP for the copy-packages bug?
<StevenK> It will be submitted to PQM in a few minutes. So, soon.
<jtv> Anyone else seeing rocketfuel-get fail on the CSS build?  The problem seems to be some extra dots between classes in l/c/l/javascript/test.css
<jtv> mars: do you see rocketfuel-get failing on the CSS build?  See above.
<mars> jtv, haven't seen it.  CHR today.  sinzui or noodles785, ^ any idea what's going on?
<jtv> mars: it went away when I fixed l/c/l/javascript/test.css
<BjornT> gary_poster: ping?
 * sinzui knows nothing
<jtv> mars: it had lines of the form ".class1..class2.class3 {"
<gary_poster> BjornT: pong
<mars> jtv, "bzr praise"?
<jtv> ooh, didn't know about that one... do we have a secret backchannels for stats on blame/praise?
<manish> gary_poster, have a few minutes to spare ?
<BjornT> gary_poster: when running 'make', i get a lot of: 'import site' failed; use -v for traceback
<BjornT> gary_poster: what's that about, and to what should i pass -v?
<jtv> mars: sinzui.
<mars> jtv, ?  blame/praise/annotate are the same.  If you see a funky line like that, I would say check the source of the build chain.  Something changed, and the machine didn't do it :)
<mars> ha!
<jtv> mars: the machine didn't do it?  there's a novel concept...  we meatbags usually help, but...
<gary_poster> BjornT: you can typically pass -v to the interpreter, so bin/py -v should tell you what is going on
<gary_poster> BjornT: and generally, I don't know what that's about, but it's not good
<gary_poster> When you gimme the -v output that will tell me
<gary_poster> manish, I saw your email.  I sympathisize but I don't know the answers to your questions.  leonardr will not be here until next week, as I said.  flacoste could possibly know.  If this is just a matter of interpreting the wadl spec then I can try to help, but you've read the spec much more than I, I'm sure.  I'll reread your email now to make sure I don't see anything more concrete I can offer...
<manish> gary_poster, Thanks. I have read the spec, but the spec is just too flexible and extensible ( a bit too much). I can try to contact the people who created WADL spec, but I hardly expect any reply from them.
<manish> gary_poster, anyway leonardr created the LP WADL? or flacoste ?
<gary_poster> manish, I reread your email.  if I had to guess, I would say that you will want to be able to send parameters and representation_type together
<manish> gary_poster, Nope. I just wanted to know if both can be together?
<manish> but as the spec says multiple representation_type means multiple ways to pas data which means overloaded method ? right?
<gary_poster> manish, I think so.  I'd give it 90% probability yes.
<gary_poster> manish: leonardr is responsible for this infrastructure, and is the WADL expert
<manish> gary_poster, If both can go together then it makes the situation more complex :(
<gary_poster> manish, yes.  however...
<manish> till now i was working thinking both can mutually exclusive
<manish> s/can/are
<gary_poster> manish: I think launchpadlib itself only ever requests JSON
<manish> gary_poster, sorry I didn't get it. Can you rephrase it please?
<BjornT> gary_poster: ok, found the problem: ImportError: Bad magic number in /devel/launchpad/rocketfuel/lib/site.pyc
<BjornT> gary_poster: this makes the problems go away: http://pastebin.ubuntu.com/432258/
<gary_poster> BjornT: typically that means that your pyc was from a different version of Python
<gary_poster> BjornT: +1 on that patch
<BjornT> gary_poster: cool, i'll merge it then, thanks
<gary_poster> thank you BjornT
<gary_poster> manish, rephrasing...
<jml> StevenK, do you know what's pretty hard?
<jml> StevenK, getting it right the first time :(
<StevenK> jml: I bet
<wgrant> What broke?
<jtv> mars: it's getting late here and I'm putting out a fire... any chance you could pick up that css issue?
<StevenK> jml: Does that mean there is further stuff to be abstracted?
<jml> StevenK, ish. I made a couple of mistakes when I did it.
<gary_poster> manish, I believe it is theoretically and practically possible to ask for different representation types.  We do this in Launchpad, in fact: in our browser API calls, sometimes we ask for HTML back instead of JSON.
<gary_poster> *However*, in launchpadib, I believe we only ever ask for one response type: JSON.  Therefore, for your case, I believe the following is reasonable advice:
<gary_poster> 1) your C# code should generate code that can accept parameters.
<gary_poster> 2) it should never have to accept a representation type, because you always want JSON back, (which your code will process transparently.)
<gary_poster> 3) if you ever want to make code that used a different representation type, you could have it specify a representation type in a different place.  I do not think you should implement this now.  My personal idea would be that you could generate a root object that sends a different representation type be default.  You also could change it on the fly: a Python version might be launchpadlib.representation('application/xml').foo(
<mars> jtv, please file a bug and assign it to me.  I'm a bit buried myself, and doing CHR.
<jtv> mars: oh yes forgot about that... maybe we should just bully sinzui into fixing it?
<manish> gary_poster, a minute. I need time to read and understand
<jtv> *That* is what I forgot.  Breakfast, lunch & dinner.
<jtv> stub: just heard some pretty rousing words from Dr. Weng...  this ain't over yet.
<StevenK> jml: Should we do another branch that sorts out more abstraction issues?
<gary_poster> manish, of course.  when you have digested what I wrote: did that make sense, and does that seem helpful?  If so, I'll put it on the list so that you and others can comment on it, and leonardr and flaocste can see it later.
<sinzui> jtv, what exactly is broken?
<jtv> sinzui: lib/canonical/launchpad/javascript/test.css
<jtv> and as a consequence, rocketfuel-get
<jml> StevenK, yeah, probably the best thing to do is use pipelines or whatever to split out that level of work from the poppy level of work
<manish> gary_poster, it did make some sense. Well I know that I should expect only JSON
<jtv> sinzui: it's got some "class1..class2" (note double dot)
<jml> StevenK, because the only sane way to drive the abstraction stuff now is by actually doing poppy
<sinzui> jtv r=me to remove it
<jml> (now that all the mentions of "codehosting" and "bazaar" have been factored out)
<manish> gary_poster, till now what I was doing was that - either param[] or represenation_type[] can be present
<StevenK> jml: Pipelines are one of those that I saw and went "That's cool", but I have no idea how to drive
<jtv> sinzui: late night & putting out fire
<StevenK> one of those things, even
<jtv> sinzui: I was hoping I could guilt or bully you into it.
<jml> StevenK, me either, I just do it the hard way by having two branches and merging one into the otther
<sinzui> jtv: I will do it during my lunch
<manish> gary_poster, when param[] is present, i create a method with returnType foo (param1,param2... paramn)
<jtv> sinzui: wayyy to bounce the guilt back.  Well played.  :-)  And thanks!
<manish> gary_poster, and when represenation_type[] I create n methods and in each method I expanded a  represenation_type and that becomes it's parameter
<StevenK> jml: That's sounds like work. Let me investigate pipelines
<jml> StevenK, :D
<jml> bigjools, does poppy have a config section?
<gary_poster> manish: ok.  I'm personally thinking that you should just ignore representation_type for now.  It will make using your library in (what I believe will be) the common way easier.  Later...if you want to support it I'd suspect you'd want to do it a different way.  it strikes me that we could be looking at lazr.restfulclient and seeing what leonardr does with representation types.  Let me see if I can find that...
<manish> gary_poster, thanks a lot
<gary_poster> sure
<manish> gary_poster, I have one more issue in this WADL
<gary_poster> manish, ok
<manish> have a look at https://edge.launchpad.net/+apidoc/1.0.html#branches
<manish> getByUrls method
<manish> there is no return type specified
<manish> even the WADL doesnt have anything
<bigjools> jml: no
<StevenK> jml: pipeline looks full of goodness
<manish> gary_poster, http://pastebin.com/kqZtj9zW is the section in WADL for branches.getByUrls
<manish> gary_poster, so the C# code generated is "public void branches_getByUrls(string urls)"
<jml> bigjools, oh, I see. it just hardcodes strings
<gary_poster> manish: I think it returns an application/json representation of multiple branches (a sequence)
<manish> gary_poster, difference between think and present. From the name I can make out, but it is not present in the WADL
<manish> which mean I need to hand modify the WADL or the generated code which is BAD :(
<manish> from name I was sure it returned a branch list
<StevenK> jml: Where are you heading?
<manish> but being absent in WADL and the documentation makes the case weak as every update to the app will break since every time the WADL has to be hand edited
<gary_poster> manish, I understand your concerns.  I don't know enough to say the cause or the best remedy.  My suggestion is that you open a but against launchpad-foundations (https://bugs.edge.launchpad.net/launchpad-foundations/+filebug) about the WADL.  I'm hopeful that we can generate better WADL for you.  Either way, we can record the conversation there.
<manish> gary_poster, sure. I will file. I just wanted to bring it to notice to some LP dev
<manish> gary_poster, done. should I subscribe Leonard too?
<gary_poster> manish, that's ok. thank you!
<manish> https://bugs.edge.launchpad.net/launchpad-foundations/+bug/579516
<mup> Bug #579516: Launchpad WADL's "branches" section with method getByUrls doesn't have a return type <Launchpad Foundations:New> <https://launchpad.net/bugs/579516>
<gary_poster> got it
<gary_poster> mars, sinzui: IRT https://bugs.edge.launchpad.net/launchpad-foundations/+bug/537298 and https://bugs.edge.launchpad.net/launchpad-foundations/+bug/575254 , do we know conclusively if the trigger is repeating background images?  Do either of you know of someone able to dupe?  If that is the case, we now have five reports.  Can we trivially fix the CSS to possibly make this go away?
<mup> Bug #537298: Lucid firefox renders Launchpad very slowly <Launchpad Foundations:Invalid by mars> <firefox (Ubuntu):New> <https://launchpad.net/bugs/537298>
<mup> Bug #575254: Switching to a tab containing Launchpad is extremely slow in Firefox <Launchpad Foundations:Confirmed> <firefox (Ubuntu):Incomplete> <https://launchpad.net/bugs/575254>
<sinzui> gary_poster, I do not know anything about the issue. I could speculate the size of compressed CSS and JS frustrates FF.
 * sinzui uses Epiphany and is very happy
<gary_poster> heh
<sinzui> gary_poster, I wonder most engineers are not Chrome users...they do not see the problem
<mars> gary_poster, no one could conclusively narrow down the issue.  Major sites are also affected.  It really is Firefox' problem, not the Internet's.
<sinzui> mars, Thanks for the sober summary. WONTFIX
<gary_poster> ok thanks, guys,
<mars> sinzui, social puzzle for you: what the heck do I do with this? https://answers.edge.launchpad.net/launchpad/+question/110662
<mwhudson> mars: wtf
<mars> mwhudson, excellent summary :)
<gary_poster> lol
<mwhudson> mars: i hadn't read the comment
<gary_poster> that makes me happy the way Dali sometimes makes me happy
<gmb> mars: That is possibly the best thing I've read in *days*
<lifeless> jml: #python-testing may interest you
<lifeless> jml: I plan to make testtools only emit one outcome per startTest/stopTest pair
<lifeless> jml: any objection?
<jml> lifeless, I'd like to see an email explaining why, if that's ok
<lifeless> jml: I blogged a while back :)
<lifeless> and I'm sure there is a bug; digging
<lifeless> bug 335816
<jtv> BjornT, danilos, your reviews kindly requested: https://code.launchpad.net/~jtv/launchpad/bug-578331/+merge/25168
<mup> Bug #335816: tests report multiple results <testtools:Confirmed> <https://launchpad.net/bugs/335816>
<lifeless> jml: ^
<jml> lifeless, thanks.
<lifeless> adding details there, now.
<jml> ok thanks.
<jml> lifeless, I'm doing some twisted hacking w/ StevenK right now so my brain is a bit full
<lifeless> sure
<lifeless> I want 'u' in launchpad bugs
<lifeless> kfogel: so, rob [gnash] is pinging me, face2face
<cperrin88> Hey Peaople
<cperrin88> I'm not sure if I'm in the right channel but I will ask
<cperrin88> Can anyone help me with the Apache HTTP Commons lib and the OAuth signing process of launcpad
<mars> gary_poster, are you able to help cperrin88 with the OAuth process?  Or is that something for Leonard when he gets back?
<cperrin88> The problem is driving me nuts because I don't see the mistake
<cperrin88> and I can't sniff a damn https connection >.<
<gary_poster> mars, cperrin88, that's probably leonardr I'm afraid, next week.  If someone else can help, they are probably doing UDS things
<cperrin88> *sighs*
<gary_poster> cperrin88: I'm not familiar with that lib.  Does this mean that you want to use the REST webservice without launchpadlib?
<cperrin88> Yep
<cperrin88> I mean
<cperrin88> I have no other choice
<cperrin88> Android is Java
<gary_poster> cperrin88: right, got it.  flacoste (who is not here) would be able to help; salgado (who is not here) might be able to help.  If you give focussed questions there's a chance I can dig up answers for you, but that's all I can offer now, I'm sorry.
<cperrin88> That's a start
<cperrin88> well I'm trying to get a request token as described here https://help.launchpad.net/API/SigningRequests
<gary_poster> ack
<gary_poster> mm, sorry, I meant, got it
<cperrin88> ack ;)
<gary_poster> :-)
<cperrin88> I'm using the Apache HTTP commons for this request as they are preinstalled
<gary_poster> ok
<cperrin88> I have tried to send this requst to non SSL server and it looks like this http://pastebin.com/E8SrwLjv
<cperrin88> of course i send it to edge.launchpad.net over https and to the path /+request-token
<cperrin88> If I send this as a raw HTTP request over a tool named Fiddler, I get what I want
<cperrin88> If i send this with my code I get a 401 Unauthorized
<gary_poster> cperrin88: (probably where you are going, but to join along...) so that seems to suggest the HTTP Commons lib is doing something a bit different than the vanilla request?
<cperrin88> probably
<mars> cperrin88, you said you can't sniff the connection: does Apache HTTP commons have debug logging output that you could inspect (assuming your application has logging)?
<gary_poster> Have you gotten a look at the actual request over the wire? (assuming the answer is "I can't" for some reason :-) )
<mars> log4j or Apache commons logging, I don't remember which, been too long
<cperrin88> I can log some things but not what I need to get a clue
<cperrin88> What I can log seems correct
<cperrin88> It would be cool the see what the server really gets
<cperrin88> but I don't have a SSL cert
<cperrin88> so I can't write a logging server
<gary_poster> cperrin88: wireshark an option?  It seems to have SSL support, based on googling around
<cperrin88> nope
<gary_poster> for my own knowledge, why not?
<cperrin88> I'd need the private key of the launchpad cert
<gary_poster> ah right
<gary_poster> ...
<cperrin88> If I had that first S in SSL would be in vain for launchpad
<gary_poster> :-)
<cperrin88> I think i got an idea
<gary_poster> cperrin88: an off-chance, but does the body of the 401 response seem to have anything I could do something with on my side?
<cperrin88> nope
<cperrin88> it's empty
 * gary_poster is happy cperrin88 has an idea
<gary_poster> ok
<mars> cperrin88, maybe try stunnel as a man-in-the-middle, as described here: http://codepainters.wordpress.com/2010/01/23/androids-https-implementation-slow-under-debugger/
<cperrin88> I will try to write a logging app on app engine
<cperrin88> that has htttps
<gary_poster> fwiw, in Python something else I'd try is actually hacking the underlying library to tell me what the heck is going on, but that can be problematic.  That said, *some* way of seeing what you are actually sending us does seem like a pretty reasonable diagnostic tool to have.
<gary_poster> so sure, the https app engine logging engine sounds easyish
<gary_poster> I like mars' idea too
<cperrin88> I was reading that
<gary_poster> cperrin88: no idea what I'm talking about, but http://hc.apache.org/httpclient-3.x/logging.html ?
<gary_poster> about to go on call; back in a bit
<cperrin88> gary_poster: I don't have that lib in Android SDK
<cperrin88> but I will try it another way
<gary_poster> ok
<manish> cperrin88, Can you tell your problem again clearly. This is nearly the same problem I had when I started with LP APi
<cperrin88> I build my requst with APche HTTP commons for a requst token and everythime i send it I get a 401 Unauthorized back
<cperrin88> a vanilla request, using a tool named fiddler, works
<cperrin88> It drives me nuts
<manish> cperrin88, I have used Fiddler. That was what I too used to figure out my problem
<manish> you get a 401 even in step 1?
<manish> oauth_consumer_key=LaunchDroid&oauth_signature_method=PLAINTEXT&oauth_signature=%26
<manish> cperrin88, ^
<cperrin88> yep
<cperrin88> but fiddler works
<manish> cperrin88, you sending it to? https://edge.launchpad.net/+request-token
<manish> https or http?
<cperrin88> https
<cperrin88> but http woudl refirect my request
<cperrin88> *would redirect
<manish> oh. crap. This is even weirder. What is the content of the exception you get?
<manish> means the exception description
<manish> cperrin88, you using this? http://developer.netflix.com/docs/Security#0_18325
<cperrin88> nope
<manish> your HTTP request shows this?
<cperrin88> I just send my request somewhere that doesn't use https to sniff it once
<manish> Host: api.netflix.com at http://pastebin.com/E8SrwLjv
<cperrin88> I know
<manish> you can use HTTPS withj fiddler too
<manish> *with
<manish> cperrin88, using fiddler with https is a bit tricky, but easy http://www.fiddler2.com/Fiddler/help/httpsdecryption.asp
<cperrin88> I know but https commons is a bit tricky with selfsigned certs
<cperrin88> I'm jsut writing a logging server
<cperrin88> I'm nearly don
<cperrin88> e
<manish> that's good
<manish> cperrin88, but  I *strongly* using Fiddler for HTTPS since it saves a lot of headaches. Even I had this problem. Fiddler solved it
<manish> and yeah with the help of wgrant
<gary_poster> manish, btw, I'm replying to your email with a summary of what we discussed on IRC.  I doublechecked and lazr.restfulclient does in fact control the representation type itself (that is, you can't request another representation type, at least in the common cases that I saw).  It specifies the JSON, as you'd expect.
<gary_poster> The approach wadllib and lazr.restfulclient take are that wadllib simply is responsible for parsing the wadl--you can ask wadllib resources (like nodes in the WADL) questions like WADL_RESOURCE.parameter_names(media_type) to get a list of the parameter names for that call and parameter.  Then lazr.restfulclient uses that library to ask for the JSON parts of things.
<gary_poster> This led me to the thought that you might find it interesting and useful to try porting wadllib and lazr.restfulclient, rather than figuring out your own architecture.
<manish> gary_poster, Thanks. I would look into wadllib and lazr. restfulclient
<manish> getting them now
<cperrin88> gary_poster: *sigh* I'm don't know more now
<cperrin88> *I
<gary_poster> cperrin88: so, you got the logging server, you have the full request, and it is pretty much identical to what you had before (http://pastebin.com/E8SrwLjv with the replacements)?
<gary_poster> and that same request works with Fiddler?
<cperrin88> Yep
<gary_poster> ...
<cperrin88> that's pretty depressing -_-
<cperrin88> Searching an error that doesn't seem to be existent .....
<cperrin88> an error taht should not be there
<gary_poster> cperrin88:
<gary_poster> (1) I'm happy to look at a pastebin of the what you got from the logging server, if you want another pair of eyes.
<gary_poster> (2) maybe try to dupe outside of android but with the same library in Java?  Then conceivably you could turn on the logging we discussed earlier, if you can dupe?  (If you can't dupe that way....!)
<gary_poster> (3) in case there's some horrible fundamental bug in our code, you've duped the *exact* same output you saw in the logging server (for instance, all the same headers all in the same order, and whatever else you can think of) with Fiddler and still gotten success?
<cperrin88> gary_poster: I'm double chacking everything now
<cperrin88> *checking
<gary_poster> ok
<Ursinha> thumper, already there? good morning :)
<Ursinha> there's an oops that I've seen for a while, a ProcessTerminated one, like https://lp-oops.canonical.com/oops.py/?oopsid=OOPS-1557CIW15
<Ursinha> I was pretty sure you replied the list asking about it, but I can't find the message, I guess I'm (or was) on crack
<thumper> Ursinha: morning
<thumper> Ursinha: I'm about to get a coffee
<Ursinha> thumper, go for it :)
<thumper> Ursinha: that is a job being killed
<thumper> an import worker
<Ursinha> thumper, is that a real problem?
<thumper> not sure
<Ursinha> thumper, well, any investigation would be appreciated :)
<thumper> ack
<thumper> rockstar: ping
#launchpad-dev 2010-05-13
 * thumper goes to make coffee
<rockstar> thumper, pong
<thumper> rockstar: nm
<rockstar> thumper, :(
<rockstar> Dude, this ec2 crap is getting out of hand...
<rockstar> I'm 0/3 on ec2 attempts today...
<thumper> rockstar: still around?
<rockstar> thumper, yeash.
<thumper> rockstar: wanna talk?
<rockstar> thumper, sure, one sec.
 * thumper EODs
<jml> bigjools, has anyone talked to the losas about the upcoming poppy changes yet?
<mthaddon> nope
<bigjools> lol
<jml> mthaddon, oh hi :)
 * jml runs
<bigjools> mthaddon: we're adding an sftp server for cocoplum/germanium
<bigjools> it'll obviously need some admin input for ports/configs/security etc
<mthaddon> bigjools: what will these sftp servers do?
<bigjools> mthaddon: same as poppy, except over sftp
<mthaddon> bigjools: which reminds me - we need to plan a day for you to come to millbank and tell me all about soyuz...
<bigjools> indeede
<StevenK> And not using Zope's FTP service, thank $DEITY
<mthaddon> bigjools: any chance of that happening next week? would be good to do it before the LOSA sprint so I can spread the love
 * bigjools thanks beer
<bigjools> mthaddon: +1
<bigjools> I know how you like to spread love
<mthaddon> you do indeed
<lifeless> jml: I want 5 minutes with you before next session, if possible
<StevenK> Including a heavy, blunt object?
<lifeless> no
<lifeless> need to flip a coin
<mwhudson> jml: he's in flamboyant
<lifeless> mwhudson: was that for me?
<mwhudson> ah
<mwhudson> lifeless: yes :)
<lifeless> where is flamboyant ;)
<mwhudson> lifeless: one floor up from you
<mwhudson> lifeless: there's a session going on, but you may be able to ambush him at the end
<lifeless> On the ceiling ?
<mwhudson> lifeless: now he's running over to ebene
<lifeless> jml: ping
<jml> StevenK, http://docs.python.org/release/2.5.4/lib/module-os.html
<lifeless> jml: I would _love_ a review of https://code.edge.launchpad.net/~lifeless/testtools/bug-335816-one-outcome-per-test/+merge/25226 - this will improve bzr's CI story.
<danilos> bigjools, twistd rotates logs in a *very* weird way... some bits disappear for a certain time, or at least it seems so
<bigjools> danilos: it renames all the files
<bigjools> welcome to twisted
<danilos> bigjools, right, but not just that... what used to be generic log is neither .1 or .2 anymore
<wgrant> It's not still in the middle of a rename?
<bigjools> danilos: what do you mean by "generic log" ?
<danilos> bigjools, unnumbered buildd-manager.log
<danilos> bigjools, look at this: https://pastebin.canonical.com/32246/
<danilos> bigjools, a bunch of "Starting template..." entries were in buildd-manager.log, then it got rotated, and they are neither in .1 nor .2 anymore
<bigjools> wot wgrant said - and also remember that devpad has synced logs
<danilos> bigjools, I guess it just takes ages for it to rotate all files?
<bigjools> so it might have synced in the middle of a rotation
<danilos> wgrant, bigjools: could be, yes
<danilos> well, it syncs every few minutes and I had something very similar yesterday for over 30 mins
<bigjools> every 5 mins
<bigjools> you can always get a losa to grep on cesium to confirm
<danilos> well, you can't _always_ get a losa, but sure :)
<danilos> bigjools, anyway, not a big deal, probably takes a while to rename ~3k files
<bigjools> we should dump those old logs :/
<bigjools> where dump == trash
<danilos> where trash == recycle bin ;)
<bigjools> eww
<jtv> noodles775: I'm pushing a fix for our traceback... would you mind reviewing it?
<jtv> Meanwhile it seems the fighting's started here, so I'll be a bit distracted.  :(
<jtv> Crap, killed by a sniper...
<jtv> Last taxi driver I talked to said this guy was his hero.
<jtv> Pissing off a few million more people.. great way to make peace.  TV shows nothing but game shows, ads & documentaries, which usually means it's serious.
<noodles775> :/
<jtv> Just got message from the Happy Smiley Superior Firepower committee: they're going to let children and the elderly out alive.
<jtv> That leaves the women and very few others.
<danilos> jtv, and you? :)
<jtv> danilos: no, but it's beginning to look like they got my number because I've been in that area.
<jtv> Nobody else I know is getting them.
<danilos> jtv, good luck surving the next few hours then
<jtv> Oh, I'll be fine.
<jtv> If I had my camera I might go
<jtv> noodles775: https://code.edge.launchpad.net/~jtv/launchpad/production-579914/+merge/25230
<noodles775> wgrant: the db-scheme in its current state: https://code.edge.launchpad.net/~michael.nelson/launchpad/db-build-generalisation-db-changes/+merge/22527
<cperrin88> It's pretty tricky to crawl through all the laucnhpadlib imports to finally understand what's happening to port it to java -_-
<gary_poster> cperrin88: ack. :-/   fwiw, I know many of the imports better than launchpadlib itself, so I can potentialy help with specific questions.  I understand that doesn't help much with getting the big picture though.
<cperrin88> gary_poster: at least I found my error with the token
<gary_poster> cperrin88: great!  something for us to share on the help page, to help others?
<cperrin88> nope
<cperrin88> to stupid :D
<gary_poster> heh, ok
<cperrin88> I sent my request to  +access-token instead of +request-token >.<
<gary_poster> ah, suck
<gary_poster> well, glad you're past that at least
<danilos> bigjools, got a minute to discuss what we need to do to get translation jobs not to molest arm builders?
<bigjools> danilos: yes I was just talking to jtv
<bigjools> I have a solution
<bigjools> was about to implement it but since you're around ... ;)
<danilos> bigjools, who is around? :)
<danilos> bigjools, nah, go ahead, go ahead, I am all good :)
<wgrant> Which of the many possibilities are being considered?
<bigjools> danilos: the easiest and sanest way right now is to change TranslationTemplatesBuildJob.create() to set buildqueue.processor to  the nominatedarchindep
<bigjools> it restricts the jobs to i386 for now
<bigjools> when we get builder pools sorted out, it will solve this issue in a better way
<jtv> I did it that way once, but IIRC was told leaving it null was fine... how's that for irony?
<bigjools> go figure
<danilos> bigjools, so, we'll want this CPed as well?
<bigjools> hel;l yes
<danilos> bigjools, mthaddon told me there is a workaround in place right now
<bigjools> this is the cycle of yeehaw and CP
<jtv> danilos: I have to go now.  Requested CP for yesterday's fix; got the one for the buildd-manager traceback approved conditionally on dogfood Q/A.  (It's in EC2).
<danilos> bigjools, is there?
<wgrant> Well, nominatedarchindep sucks. It just sucks less for the moment.
<bigjools> danilos: very temporarily, the builder was moved to the nonvirtual pool
<bigjools> danilos: but that was to stop the manager from bailing
<danilos> bigjools, ah, ok, so the sooner we get this done, the better, and then we can get that builder back to the virtual pool
<bigjools> danilos: we need to put it back at some point
<bigjools> yep
<danilos> bigjools, ok, got the point
<bigjools> g'night jtv
<danilos> jtv-zzz, good night, safe walk home :)
<bigjools> mind the snipers
<kfogel> Can anyone confirm that the 'lp_teamparticipation' table in the DB schema is obsolete and the 'teamparticipation' table should always be used instead?
<jtv-zzz> danilos: home how?
<jtv-zzz> I'm not going home
<danilos> jtv-zzz, ah, ok, enjoy the night at the office :)
<jtv-zzz> thanks
<bigjools> danilos: ok cool - if you need some more guidance let me know, I don't think I'm going out tonight since it's a holiday in Belgium (!)
<danilos> bigjools, oh, it's a family holiday so you are spending the day at home? :)
<bigjools> haha
<danilos> bigjools, anyway, sure, will probably ping you since I totally have no idea what half of these words meant (like nominatedarchindep :)
<wgrant> kfogel: You should use teamparticipation -- lp_teamparticipation is used in the replication mess for SSO. don't go near it.
<mars> gary_poster, is there a new bootstrap.py that I can using for the new zc.buildout beta?
<kfogel> wgrant: ah.  Thanks.
<bigjools> danilos: ok :)
<wgrant> lp_* are for the use of SSO.
<gary_poster> mars, yes.  Use 1.5.0b1, and use http://svn.zope.org/repos/main/zc.buildout/trunk/bootstrap/newbootstrap.py (that URL is transient, btw; will become http://svn.zope.org/repos/main/zc.buildout/trunk/bootstrap/bootstrap.py)
<mars> gary_poster, my buildout keeps picking 1.4.3.  According to the bootstrap.py --help, it should be going to PyPI and fetching your version.  I am using your newbootstrap.py.  I don't have a zc.buildout version locked down anywhere.
<mars> gary_poster,   Perhaps it is using the local zc.buildout egg first, rather than going online?
<gary_poster> mars, yes
<mars> gary_poster, thanks.
 * mars fixes
<wgrant> gary_poster: The bug #329178 that you just referred a user to is private.
<wgrant> (and, well, I'm interested in the proposed solution)
<gary_poster> wgrant: ack. on call.  a bug in our openid provider, so not in my (or Launchpad's) domain directly.  um...I'll ask if I can add you in as a subscriber, I guess.
<wgrant> gary_poster: Well, c-i-p is now open source, so it probably needn't be private any more.
<gary_poster> it's considered security, wgrant.
<wgrant> Ah, I see.
<mars> gary_poster, you were right.  I killed the download-cache versions, and it picked my ~/.buildout/download-cache/ version instead, 1.5.0b2.
<gary_poster> mars, you want b1
<mars> gary_poster, ok
<mars> gary_poster, lots of warnings like this: /home/mars/canonical/lazr-js/1.0/eggs/distribute-0.6.10-py2.6.egg/setuptools/command/easy_install.py:198: UserWarning: Unbuilt egg for setuptools [unknown version] (/usr/lib/python2.6/dist-packages)
<mars> gary_poster, probably because your new version does "-S", then imports /usr/lib/python2.6/dist-packages/site.py.  That system-specific site.py contains setuptools yet again.  Something in the old easy_install code doesn't like that.
<kfogel> wgrant: two questions, one is a meta-question: I'm trying figure out the difference between teammembership and teamparticipation tables (particularly since the latter appears to do what I would have expected the former to do).  And, I'm wondering where I'm failing to dig that would have answered this question for me (if only SQL tables could have descriptive comments attached *in the database*... or can they?  We don't appear to, anyway.)
<wgrant> kfogel: teamparticipation is the flattened version of teammembership.
<wgrant> The latter only contains direct memberships.
<wgrant> The former takes membership's transitive nature into account.
<gary_poster> mars, ack, have not seen.  upgrade setuptools to c11 and everything is fine?
<wgrant> So teamparticipation is calculated from teammembership.
<danilos> kfogel, \d+ for comments on tables in psql
<wgrant> kfogel: There is comments.sql, but I'm not sure how to see that.
<danilos> kfogel, and comments.sql
<wgrant> Ahh, I see.
<mars> gary_poster, this is entirely using distribute
<kfogel> danilos: awesome!  thank you
<kfogel> wgrant: all clear now, thank you.
<gary_poster> mars, may just be broken setuptools locally.  I use distribute sometimes and is fine
<kfogel> wgrant: so what I'm trying to do here is track growth in the ~contributor-agreement-canonical team's membership; in this case, I think there are only ever direct memberships, but it won't hurt to just query teamparticipation, because if there ever were indirect memberships, we'd want to count them.
<kfogel> wgrant: talking out loud only for sanity check; no question needing answering there.
<wgrant> (Well, the first issue is that that team is wrong.)
<kfogel> wgrant: ?
<wgrant> It's not complete. I'm not in it, for one.
<danilos> kfogel, btw, there's also lib/lp/registry/doc/teammembership.txt (first paragraph seems to explain the difference between the two), so checking doc/<model-class>.txt can sometimes be a good start (ideally, it would always be a good start, but ok)
<kfogel> wgrant: ah, I see what you mean.  The team is right, but it's got a bug :-)
<kfogel> danilos: good hint, thx
<mars> gary_poster, looks like it doesn't like the setuptools compatability package installed on Ubuntu.  "python-setuptools" is a versionless compatability package for setuptools 0.6c9
<Ursinha> Chex, gary_poster, rockstar, bigjools, danilos, sinzui, allenap: production meeting on #launchpad-meeting @ Freenode in 8 minutes
<allenap> Ursinha: Someone else will have to represent Bugs; I'm a treacherous Landscaper now.
<Ursinha> allenap, oh, cool :)
<Ursinha> allenap, I'll find someone else, thanks!
<allenap> Ursinha: I'm asking now.
<Ursinha> Chex, gary_poster, rockstar, bigjools, danilos, sinzui, someone in bugs: production meeting on #launchpad-meeting @ Freenode now
<mars> gary_poster, do you think it is safe to update lazr-source-dependencies to use zc.buildout 1.5.0b1?  It currently uses zc.buildout-1.5.0dev-gary-r110073.tar.gz
<gary_poster> mars, yes
<poolie> jml, if you're still here, https://dev.launchpad.net/LEP/DKIMAuthenticatedMail is now ready for review
<jml> poolie, I am. thanks.
<kfogel> wgrant: you're on the contrib agreement team now
<wgrant> kfogel: Yep, I noticed. Thaks.
<maxb> StevenK: Are you sure about marking bug 576193 as a dup of bug 568745? I admit to not understanding the issues fully, but it looks a bit non-obvious.
<mup> Bug #576193: Debian package metadata importer getting the wrong versions <gina> <Soyuz:Triaged> <https://launchpad.net/bugs/576193>
<mup> Bug #568745: Debian unstable record of gpt missing <gina> <Soyuz:Triaged by stevenk> <Debian:Fix Released> <https://launchpad.net/bugs/568745>
#launchpad-dev 2010-05-14
<thumper> rockstar: still there?
<rockstar> thumper, yes.
<rockstar> (for various definitions of the word "there")
<thumper> did you work out the upgrade branches issue?
<rockstar> thumper, no.  I started looking through logs but then got distracted by an actual ec2 email (!)
<thumper> ok
<rockstar> ...and now I seem to have lost my connection to devpad and cannot reconnect.
<rockstar> ssh: Could not resolve hostname chinstrap.canonical.com: Temporary failure in name resolution
<rockstar> wtf?
<thumper> we need a losa query to work out which upgrade branch is in progress
<thumper> because it isn't being logged
<thumper> sinzui: I don't suppose you can take a 5 minutes call?
<sinzui> thumper, I do
<lifeless> moinish
<thumper> lifeless: hi
<thumper> anyone remember how to get a vocabulary from a name?
<StevenK> maxb: They are caused by the same issue.
 * thumper is hanging out for the day to end
<thumper> almost got this bug fixed
<thumper> StevenK: how are you enjoying soyuz?
<StevenK> thumper: It's challenging, but I'm enjoying it
<stub> thumper: They are named utilities IIRC, so getUtility(IVocabulary, 'foo') (not sure if that is the real interface to use)
<thumper> stub: thanks, but I've got it now
<thumper> stub: I've avoided the need :)
<stub> IVocabularyFactory looks like... anyway.
<thumper> stub: I don't suppose there is a way to replace the vocabulary being used in form_fields easily is there?
 * thumper fires up pdb again
<stub> thumper: I think you create a new interface and override that attribute. Not sure.
<thumper> hmm
<thumper> I have to do it somewhat dynamically
<thumper> it seems I have to do the form_fields.omit, then add the new one
<thumper> I was just wondering if there was a short circuit
<thumper> although I feel that the underlying fields that are being referenced are exactly that, references
<thumper> and changing the vocabulary could be hazardous
<jtv> stub: thanks, your help got me a lot further.  Added comments to bug 388997.
<mup> Bug #388997: +imports page should handle URL "hacks" more gracefully, instead of OOPS with LookupError <oops> <Launchpad Translations:Triaged> <https://launchpad.net/bugs/388997>
 * thumper EODs
<jtv> g'night thumper
<jtv> danilos: that branch for setting BuildQueue.processor looks good to me... what work does it still need?
<danilos> jtv, QA :)
<jtv> pah
<danilos> jtv, well, I'd like to QA it before I get it landed :)
<jtv> sissy
<jtv> danilos: two separate issues there afaics...  is the branch good, and does it solve the problem.  As part of the former, I can give you a review now.
<danilos> jtv, heh, certainly then :)
<wgrant> bigjools: Can I get a full publisher log at some point? The last one was truncated early on.
<bigjools> wgrant: ppa or ubuntu?
<wgrant> bigjools: The latter.
<wgrant> (the former I can't really see, since it is full of private stuff)
<wgrant> (and the latter is the one that is really really really slow)
<wgrant> bigjools: I know how that build go corrupted yesterday.
<wgrant> The one where the build was 'Needs building', but the job was FAILED.
<wgrant> There's a bug in the abort handler.
<ghostoyo> hi
<ghostoyo> i would like have some help about installing launchpad
<mars> Hi ghostoyo.  I might be able to help.  What is the problem?
<ghostoyo> i have install bzr
<ghostoyo> then i launch the script rocketfuel-setup
<ghostoyo> i have an error : ERROR: The user root has not registered any SSH keys with Launchpad.
<ghostoyo> then i have a question i must to give some keys but i donnu which
<mars> ghostoyo, what do you see when you type "bzr whoami"?
<ghostoyo> root <root@lxc-vz2>
<mars> ghostoyo, and have you set up a launchpad account?
<ghostoyo> yes i have a launchpad account
<mars> ghostoyo, and "bzr launchpad-login" shows the correct user?
<ghostoyo> no it's not the same
<mars> well, it should be your launchpad username
<mars> one is the account that will be signing the commits
<mars> the other is the account that communicates with launchpad
<ghostoyo> No Launchpad user ID configured.
<ghostoyo> how i can configure a account ?
<mars> ghostoyo, if you have a launchpad account set up, just type "bzr launchpad-login whatevermylpaccountnameis"
<ghostoyo> it say me that i am not registered
<mars> ghostoyo, could you tell me your Launchpad account name, so I can look it up?
<ghostoyo> Ghostoyo :)
<ghostoyo> did you found me?
<mars> ghostoyo, I'm reading the source for rocketfuel-setup, and it says that the error can be ignored.  When you ran it, did the script continue despite the error?
<ghostoyo> yes but then i have a ask about som keys
<wgrant> You probably shouldn't be running rocketfuel-setup as root.
<wgrant> It's not designed for that, and probably hasn't been tested.
<ghostoyo> have u succed in install it?
<wgrant> Yes, as my own user.
<mars> ghostoyo, I just checked on launchpad for "https://edge.launchpad.net/~Ghostoyo", and there is not account there.
<ghostoyo> i launch the script and give you exactly the message
<wgrant> Does it cause an error?
<wgrant> It should work even without keys.
<mars> correct
<ghostoyo> ok i will try
<ghostoyo> https://launchpad.net/~milot-jean
<maxb> ghostoyo: Please confirm you're not running any of this as root any more
<ghostoyo> yes i'm root
<maxb> Well, that is incorrect
<ghostoyo> sorry ?
<maxb> You should NOT be setting up Launchpad as root
<ghostoyo> ok
<maxb> Also, when you say you want to "install" launchpad, you do realize that what you are doing is setting up a Launchpad *development* environment, *not* installing Launchpad for production usage?
<ghostoyo> yes
<mars> ghostoyo, when you run the rocketfuel-setup script (as a user account, your own or a new one like /home/launchpad), the script will ask you for your Launchpad username.  Type 'milot-jean' when it does.
<ghostoyo> yes i do
<mars> ghostoyo, it should work then.  You can try adding "set -x" to the top of the script, and it will print out each command as it is executed.  You can then tell us the line with errors if it happens again.
<mars> gary_poster, should zc.buildout projects still have the "import ez_setup" line at the top of their setup.py file?
<mars> gary_poster, I'm noticing that lazr-js uses "import distribute_setup" instead.  That dies horribly when you then try to run "bin/py setup.py"
<gary_poster> mars, that and other changes I now regard as mistakes.  I tried to make changes so a non-buildout workflow would still be possible  It made things messier than necessary for little to no benefit.  So IOW, "no" is fine.
<mars> good!
<mars> that is a nice answer :)
<mars> gary_poster, fwiw, I noticed that 1.5.0b1 also fails in isolating lazr-js from the base system when you set install-from-cache with the default setting.
<mars> gary_poster, it picks zope.interface 3.5.1 from dist-packages, then dies.  What it /should/ do is look on PyPI for 3.5.3 and download it into the global .buildout/ cache.
<mars> gary_poster, I suspect that there is a new "buildout way" for 1.5.0, one that uses nice things like the global cache, and that lazr-js does not follow it.  Pain ensues.
<gary_poster> mars, ah, yes!
<gary_poster> mars, is there a branch I can see?
<gary_poster> the easy answer is this:
<mars> gary_poster, lp:lazr-js/1.0
<gary_poster> for scripts, instead of using the zc.recipe.egg recipe in buildout.cfg, use z3c.recipe.scripts
<gary_poster> that should be it
<mars> subtle
<gary_poster> Some recipes still need to use the new approach, so they still may be problematic
<gary_poster> mars, I guess yes.  I felt it was important to provide full backwards compatibility
<mars> I agree
<mars> it is subtle because it is not obvious that I need to change the buildout.cfg recipe in order to use the new buildout cache, and to fix the isolation problem.
<mars> gary_poster, given that there is a new "buildout way", and upgrade guide or cheatsheet may be in order.
<gary_poster> mars, agreed about the lack of obviousness. :-/  in the same vein as the discussion we had aboutan "opinionated" buildout, I thought it might be nice to have a bootstrap (or some other script) that would give you a buildout directory set up the opinionatedly-correct way.  Something much lighter weight than the lazy.yourpackage thing, but a similar way to get started.
<gary_poster> s/discussion we had aboutan "opinionated" buildout/discussion we had about an "opinionated" bootstrap/
<mars> right
<ghostoyo> the message : Fetching revisions:Inserting missing keys:
<ghostoyo> what i must to enter ?
<cody-somerville> sinzui, https://launchpad.net/live-installer
<cody-somerville> sinzui, Why doesn't launchpad suggest the 'live-installer' package under 'Packages in Ubuntu' section?
<sinzui> Usually because live-installer is not published in Maverick
<sinzui> cody-somerville, it may have matched, but we limit the matches to 8
<cody-somerville> It is published.
<sinzui> cody-somerville, I think the latter is the case here
<cody-somerville> me too
<cody-somerville> Should I file a bug?
<sinzui> Lp is search DistributionSourcePackageCache. It looks like the method does not give precedence to exact matches
 * cody-somerville nods.
<cody-somerville> sinzui, Also, can I have maintainership of the live-installer project on launchpad? I maintain the package in Ubuntu and participate upstream as well.
<sinzui> cody-somerville, you need to ask Mantas KriauÄiÅ«nas, if he never replied a Launchpad admin can give it you
 * sinzui has a plan to let registry administrators do this too
<mars> EdwinGrubbs, ping, were we supposed to nuke the apidoc dirctory in r10865 ?
<mars> EdwinGrubbs, the $(API_INDEX) target in the Makefile still depends on that directory existing in source control.  If you want, I can make it into a generated target instead of a checked-in one if the deletion was intentional.
<EdwinGrubbs> mars, I'm looking at that revision now.
<pabelanger> I'm testing out launchpad from ~/launchpad/lp-branches/devel, but can't locate etc/zope.conf to disable Developer mode
<maxb> Why would you want to disable developer mode?
<pabelanger> to see how much faster it is
<mars> flacoste, ping
<mars> flacoste, unping
<mars> gary_poster, do you know where in the test suite an expensive functional test for test_on_merge.py might go?
<gary_poster> mars on call
<mars> k
<sinzui> gary_poster, ping
<gary_poster> sinzui on call
<sinzui> notes
<sinzui> noted
<gary_poster> mars, um....canonical.launchpad.scripts.tests ?
<gary_poster> sinzui, pong
<mars> gary_poster, I was looking for something under lib/lp/ actually.  I think lib/lp/scripts/tests/ or lib/lp/testing/tests/ may work.
<mars> gary_poster, it is slightly complicated by the fact we do not distinguish between unit tests and functional/integration tests, but oh well.
<gary_poster> mars, sounds fine.  "expensive" should give the build engineer pause, I would think, but I expect it's a tricky balance?
<mars> I prefer not to touch lib/canonical/... for new code.  There are 6? scattered directories under there with some sort of 'test' in the name.
<mars> gary_poster, expensive because it involves IPC and a one second timeout.  IPC means it is not a unit test.
<gary_poster> mars, ah ok
<creatix> http://tinychat.com/pugquit
<thumper> creatix: irc spam?  please don't
#launchpad-dev 2010-05-16
<cperrin88> Hey
<cperrin88> Am I the only one who gets a 503, Service Unavailable on staging.launchpad.net/+request-token ?
<adiroiban> cperrin88: nope
<adiroiban> I can not access staging.launchpad.net
<adiroiban> at all
<cperrin88> is it down?
<adiroiban> cperrin88: no idea. the error message says that you can check that on #launchpad
<maxb> which is pretty much a lie outside UK office hours
<adiroiban> maxb: :)
<ofirk> hello
<ofirk> I installed launchpad using the instructions on the wiki but I am unable to push to bazaar
<ofirk> I keep getting Connection Refused errors
<adiroiban> ofirk: have you done âmake run_codehostingâ
<adiroiban> ?
<adiroiban> https://dev.launchpad.net/Code/HowToUseCodehostingLocally
<ofirk> no, I did make run_all though
<thumper> ofirk: have you loaded an ssh key into the local launchpad?
<thumper> ofirk: the utilities/make-lp-user does this
<ofirk> thumper: I did, and I remember that the output said the key was loaded
<ofirk> I just get 503 error when trying to connect via bzr
<ofirk> I followed the instructions on https://dev.launchpad.net/Code/HowToUseCodehostingLocally and I got http://paste.ubuntu.com/434549/
<ofirk> I know that it means push can only be done via ssh
<ofirk> When I run bzr launchpad-login ofir it says that the username ofir is not registered
<ofirk> I registered my lp username with the bzr and now I get:
<ofirk> ssh: connect to host bazaar.launchpad.dev port 22: Connection refused
<ofirk> Do I need to install openssh-server?
<jpds> ofirk: No; you need your SSH key on LP.
<thumper> ofirk: the launchpad login needs to match your local created account
<thumper> what I do is use make-lp-user script with my public LP name
<thumper> so the launchpad login is the same
<ofirk> I did so
<thumper> not according to your pastebin above
<thumper> it is saying you haven't done a launchpad login
<thumper> so lp://dev/ is getting resolved to http://bazaar.launchpad.dev
<thumper> and that is why it is getting the mkdir error over http
 * thumper dropping kids at school
<ofirk> I done all the instructions again and know I have a username which is similar to my LP account
<adiroiban> ofirk: have you defined âbzr whoamiâ ?
<ofirk> no, what does it suppose to say?
<ofirk> I changed it to my LP account
<ofirk> but I still get the connection refused error
<ofirk> http://paste.ubuntu.com/434555/
<adiroiban> does âbzr launchpad-loginâ returns your LP account?
<ofirk> it returns klinger-ofir which is my LP username
<ofirk> maybe I need to connect from other port?
<adiroiban> the port should be 5022
<thumper> ofirk: did you go 'make-lp-user klinger-ofir' ?
 * thumper relocates
<ofirk> thumper: yes
<ofirk> I can login to the account via launchpad.dev
<adiroiban> ofirk: I am not sure why you got the  âport 22â error
<ofirk> can I browse the branch through http://bazaar.launchpad.dev ?
<ofirk> I mean, using a browser
<adiroiban> ofirk: you should have the SSH server on 5022 http://paste.ubuntu.com/434567/
<ofirk> adiroiban: I see exactly that
<adiroiban> and you still get the âport 22â error?
<adiroiban> ofirk: ah... not I get the same error
<ofirk> adiroiban: I get the same error...
<adiroiban> it looks like bazaar.launchpad.dev should be started on 127.0.0.99
<adiroiban> ofirk: you can start your own ssh server
<ofirk> adiroiban: can you give me instructions?
<adiroiban> ofirk: nope... there should be a better way for starting the LP sftp server
<adiroiban> for bazaar.lp.dev
<ofirk> adiroiban: if the ssh server is running on port 5022, shouldn't bzr also connect through that port?
<adiroiban> ofirk: I don't know the exact architecture of LP components and what is the purpose of that SSH server on 127.0.0.88:5022
<adiroiban> bazaar.launchpad.dev should be on 127.0.0.99
<adiroiban> ofirk: I think it is better to ask on the mailing list
<adiroiban> or durring working hours
<ofirk> ok, thanks I will try there or in another time
#launchpad-dev 2011-05-09
<lifeless> wgrant: what techniques do you use to debug fails-in-ec2 branches?
<wgrant> lifeless: A test or two that only fail in ec2?
<lifeless> a fixture
<lifeless> rabbitmq
<wgrant> Ah, that one.
<wgrant> I've previously used ec2 demo and run the relevant tests manually.
<wgrant> lifeless: r12987
<wgrant> It seems to now touch the config dir in twistedsftp, but it's twistedftp that uses GPGHandler.
<wgrant> Am I missing something, or is that wrong?
<lifeless> jml: if you reopen something, please triage it - avoids double-work
<lifeless> wgrant: lib/lp/poppy/twistedsftp.py isn't the new poppy?
<wgrant> lifeless: I guess it will still work, since they run in the same process.
<wgrant> But twistedftp and twistedsftp are two fairly independent things that live in the same host.
<wgrant> twistedsftp doesn't use GPGHandler.
<wgrant> (yet)
<LPCIBot> Project db-devel build #528: FAILURE in 5 hr 12 min: https://lpci.wedontsleep.org/job/db-devel/528/
<wgrant> Hmm.
<wgrant> So it's not good :(
<wgrant> lifeless: I think this is going to leak. A SFTPServer is instantiated for each authenticated SFTP session, so: A) it doesn't touch until the first SFTP user authenticates, despite it being required for FTP, and B) each new session will create the LoopingCall, and they won't be destroyed until the end.
<wgrant> Right?
<lifeless> yes, thats faulty
<wgrant> I think it should be moved into the tac.
<wgrant> I guess wallyworld's not back yet.
<lifeless> that might work better, yes
<jtv> salutations fellow carbon-based lifeforms
<spm> yo jtv
<jtv> yo indeed, spm!
<StevenK> jtv: Still in Cali?
<jtv> StevenK: no, back but pushing the timezones hard.  Got home around 8 hours ago.
<StevenK> jtv: Nice!
<jtv> I'm not sure I'll think so in a few hours.  :)
<StevenK> Haha. :-)
<jtv> By the way, travel tip: if you're going to find out that your bookings have been all screwed up, checkin around 04:00 (yes, AM) on a super-security-conscious flight is a bad idea.
<jtv> Sorry, a bad idea *for a time to find out* that you're not actually booked on your "confirmed" flight.
<jtv> Also, if you do manage to get past that and out on a flight that actually leaves slightly earlier and you get to your hotel on the other side of the world, do not joke that your hotel reservation may be equally messed up because it just might turn true.
<jtv> Speaking of super-security-conscious, both flights happened after the OBL thing yet I was only run through the give-us-a-look-under-those-clothes-then on the way _out_.
<jtv> Which earned me a detailed patdown because my shorts' lower-left pocket was designed with slightly too much cloth, and thus showed up as a suspicious Something Potentially Hidden on the scan.  Come on.
<jtv> And I stood in line for half an hour for _that_ ride?
<jtv> Also, do not believe the helpful signs about the security checks when changing planes in Tokyo.
<jtv> A quiet connection still dies regularly, but at least I seem to get better bandwidth than in Silicon Valley.
<lifeless> jtv: so did I spot a bug then?
<jtv> lifeless: I think so, yes.  Arguably a "soft" one in that it may still be nice to know about the TMs that aren't filtered out, but only arguably.
<jtv> (Note by the way that, alack, we do not expect any useful selectivity from this extra filter)
<lifeless> that was my next question :)
<jtv> "No, sorry, you can't use the extra clause for optimization."  :-)
<wgrant> jtv: Hm, why does Julian want them all generated separately?
<wgrant> There is no reason to do that at present.
<wgrant> It may perhaps be useful when we can add extra suites.
<jtv> I suppose, yes.  Or when the script fails.
<jtv> Doesn't _solve_ anything as such, but gives us a more fine-grained do-over.
<jtv> wgrant: I'm slightly distracted atm (unpleasant family news) but let me just look up what exactly Julian said.
<jtv> wgrant: I can't find any trace of the discussion about doing the indexes per suite.  May have been otp.
<jtv> wgrant: btw have you been able to run the new (python) publish-ftpmaster through its paces recently?  It should be in pretty good shape now and I'm eager to sign off on the basic changeover.
<wgrant> jtv: OK. Was just wondering if I was missing something, but I guess the extra flexibility is useful.
<wgrant> I haven't run it in a couple of weeks, but it hasn't changed much lately, right?
<jtv> Not really, apart from the built-in index creation.
<wgrant> Right.
<jtv> It all hinges on the magnitude of "a couple" of weeks.
<wgrant> We are limited to partial fake runs now.
<wgrant> Since DF only has 120GB free with the new DB.
<wgrant> Which might just be enough to squeeze one series in.
<wgrant> If we are lucky.
<jtv> So what is it that happened to the DB?
<wgrant> We restored it over the weekend.
<wgrant> So it is now fresh and clean.
<wgrant> But some dozens of gigabytes larger.
<wgrant> We are considering purging branchrevision.
<wgrant> To save a hundred gigabytes or so.
<jtv> I wonder how stub's been getting on with the id-dropping.
<wgrant> That will only save like 25GB, but at least it's something.
<wgrant> Hmmm, plus indices.
<wgrant> So maybe more.
<wgrant> In fact it's involved in two indices.
<wgrant> So it's not an unsubstantial saving.
<lifeless> spm: yo
<lifeless> spm: we have a spammer
<spm> yo
<spm> probably
<lifeless> bugs 423094,5,6
<lifeless> possibly more
<wgrant> That's no longer LOSAish.
<lifeless> oh
<wgrant> Suspended.
<lifeless> ok, so wgrant, we have a spammer ;>
<lifeless> but some of the bugs are u1, you may not be able to see them all
<wgrant> Indeed, there are eight tasks involved, but I can only see 7.
<wgrant> We need a good solution to this.
<lifeless> indeed
<wgrant> I can see 3,4,5,6,7. 2 and 8 haven't been spammed.
<wgrant> Maybe there's a deactivated project here somewhere.
<wgrant> spm: How many tasks do you see at the URL in https://pastebin.canonical.com/47251/?
<spm> 7
<wgrant> Hmm. Why are there 8 karma entries, then :(
<wgrant> Thanks.
 * wgrant cleans them up.
<lifeless> wgrant: 2 on one bug
<wgrant> They're not normally that stupid.
<lifeless> hey, I got the mails
<LPCIBot> Project windmill-devel build #50: STILL FAILING in 1 hr 7 min: https://lpci.wedontsleep.org/job/windmill-devel/50/
<wgrant> Anyway, all gone now.
<wgrant> lifeless: Does Storm do DISTINCT ON?
<wgrant> I can't see any code for it.
<lifeless> wgrant: yes
<lifeless> wgrant: well, we can make it do it.
<lifeless> its -ffffugly- but works
<lifeless> wgrant: grep for 'DISTINCT ON'
<wgrant> No references. I'm thinking that SQL() in a Select's columns attribute should do it.
<StevenK>                 # XXX: GavinPanella 2010-09-17 bug=374777: This SQL(...) is a
<StevenK>                 # hack; it does not seem to be possible to express DISTINCT ON
<StevenK>                 # with Storm.
<wgrant> Oh, in our tree. RIght.
<wgrant> Indeed, that's how it's done. That is indeed fugly, but it seems to work.
<lifeless> wgrant: your grep foo is weak
<wgrant> lifeless: I thought you meant storm.
<lifeless> heh no sorry
<wgrant> It seems like this is a fairly easy Storm fix, though.
<wgrant> The syntax is even obvious: .config(distinct=(Foo.bar,))
<lifeless> :( bug triage
<wgrant> We're on it this week, but I haven't gone there yet.
<wgrant> StevenK: Bug #779406 is yours.
<_mup_> Bug #779406: Staging rollout for revno 10513 failed <canonical-losa-lp> <Launchpad itself:New> < https://launchpad.net/bugs/779406 >
<StevenK> Oh damn it
<wgrant> DSD parent stuff doesn't handle existing DSDs.
<wgrant> Won't affect production yet, but will probably tomorrow.
<StevenK> We don't have any DSDs in production?
<wgrant> Julian wants some this week.
<wgrant> For UDS.
<wgrant> I believe.
<StevenK> Hmmmmm, this is un-good
<wgrant> Yes.
<wgrant> Perhaps we can coerce him to use staging instead.
<lifeless> jml: so when shall we talk ?
<lifeless> poolie: -very- early draft, and incomplete because we started this call - https://dev.launchpad.net/ArchitectureGuide/Services
<poolie> jml: ha ha
<lifeless> jml: I am back, and at your service
<bigjools> morning all
<wgrant> Morning bigjools.
<wgrant> Dogfood is alive!
<bigjools> hmmn?
<bigjools> I hope you didn't start it
<bigjools> you've bloody started it
<wgrant> I did, after fixing permissions. I needed it for QA to fix poppy. But it turned out it was bad so it didn't matter :/
<wgrant> Should I not have?
<bigjools> well my screen is full of errors
<wgrant> Right.
<wgrant> The slony fix script is fucked.
<bigjools> and various bits of data need fixing
<wgrant> But rerunning with -U postgres worked fine.
<bigjools> like the builders
<wgrant> Then ran security.py/upgrade.py and ran the builder fixer.
<wgrant> Yeah.
<bigjools> you fixed the b iuolders?
<wgrant> Of course.
<wgrant> I don't like destroying production much :)
<wgrant> I followed the wiki page.
<bigjools> ok now we need production chroots re-uploading to DF
<wgrant> It's OK for now, but yes, we should do that at some point before lamont replaces them.
<wgrant> I tested it and builds and uploads all work fine.
<wgrant> And don't demolish prod.
<bigjools> ok thanks
<bigjools> did you happen to notice when it finished updating?
<wgrant> It spent most of Sunday turning the foreign keys back on.
<wgrant> (yes, that took more than 12 hours)
<wgrant> But let's see...
<StevenK> Sunday night, when wgrant pinged me with excitment
<StevenK> Well, .au Sunday night
<wgrant> Yeah. It finished some time Sunday afternoon.
<wgrant> When I checked on Sunday morning it was very slowly ALTER TABLEing.
<wgrant> And then it became clear that it hadn't gone further than pg_restore :/
<LPCIBot> Project windmill-db-devel build #252: STILL FAILING in 1 hr 7 min: https://lpci.wedontsleep.org/job/windmill-db-devel/252/
 * bigjools chuckles at jml's response on bug 778437
<_mup_> Bug #778437: Recipe build success sends emails, please stop doing that <email> <recipe> <Launchpad itself:Triaged> < https://launchpad.net/bugs/778437 >
<adeuring> good morning
<mrevell> Morning
<lifeless> jml: is now good? Its getting late is all
<lifeless> bah, hes probably not actually connected... /me starts phone calls instead
<beuno> lifeless, we're still in the plenary
<lifeless> beuno: oh
<beuno> and kiko just saked if you where here
<beuno> *asked
<lifeless> beuno: I thought it was over already
<beuno> something about a pie in the face  :)
<lifeless> beuno: hah :)
<beuno> lifeless, yeah, it was suppose to
<lifeless> beuno: he wants another? :P
<beuno> lifeless, he just challenged everyone in Linaro
<beuno> so eems so
<wgrant> Which naysayer shall be pied this time?
<lifeless> beuno: and what was the challenge?
* henninge changed the topic of #launchpad-dev to:  https://dev.launchpad.net/ | On call reviewer: henninge | https://code.launchpad.net/launchpad-project/+activereviews
<stub> Ahh... so I suspect if a request creates a notification, and the request needs to be retried due to a conflict, the retried request will create a second notification and both will be displayed. Because the session machinery isn't tied to the transaction machinery (by design).
<lifeless> stub: sounds plausible
<lifeless> poolie: hi; I was trying to locate jml before it gets late late here
<jtv> henninge, can I trouble you for a review?  https://code.launchpad.net/~jtv/launchpad/bug-779701/+merge/60346
<henninge> jtv: Hi! I don't like being troubled ... ;-)
<henninge> jtv:  ... but maybe it's fun?
<lifeless> beuno: the keynote is over now, right ?
<beuno> lifeless, it is
<beuno> everybody is running around to sessions
<henninge> jtv: just in case I was unclear: I am taking it. ;)
<jtv> henninge: thanks!
 * mwhudson blinks
<mwhudson> is lib/contrib/oauth.py in the launchpad tree used ?
<mwhudson> there is also oauth = 1.0 in versions.cfg
<mwhudson> ok, the answer is yes
<wgrant> Yes, contrib.oauth is used explicitly.
<lifeless> poolie tried to remove it
<wgrant> poolie tried to fix that.
<lifeless> it went boom
<wgrant> But there were test failures.
<lifeless> shakalaka
<wgrant> Curse you lifeless.
<lifeless> boom
<lifeless> shakalaka
<lifeless> boom
<lifeless> :)
<mwhudson> wtf
<bigjools> shake the room
<mwhudson> this whole area is littered with unmaintained forks
<mwhudson> afaicty
<StevenK> lifeless: If you keep that up, I'm flying to NZ to silence you
<wgrant> BeautifulSoup?
<wgrant> mwhudson: Most of them are gone.
<lifeless> StevenK: bring it :)
<wgrant> It used to be much worse.
<mwhudson> wgrant: do you mean in our stuff, or in general?
<wgrant> Oh, BeautifulSoup isn't in contrib.
<wgrant> It's straight in lib/
<lifeless> -win-
<wgrant> Our stuff.
<mwhudson> wgrant: right, i meant in general
<wgrant> mwhudson: I like to ignore the general Python situation.
<wgrant> It is too upsetting.
<stub> I'm just amazed all the people who know the one-hit wonder from 1987.
<jml> lifeless: hi
<mwhudson> wgrant: i don't think it's just python is it?  there are effectively unmaintained forks of the oauth *spec* aiui
<wgrant> mwhudson: Hmm, I've not heard about that.
<wgrant> And I mean Python in general, not just OAuth :)
<mwhudson> maybe i'm exaggerating
<mwhudson> certainly, lib/contrib/oauth.py does not implement http://tools.ietf.org/html/rfc5849
<wgrant> mwhudson: It's not OAuth 2.0?
<wgrant> I don't know.
<mwhudson> wgrant: no, i don't think so, i think it's an older version of oauth 1
<wgrant> Awesome.
<henninge> jtv: why only FROZEN distroseries?
<jtv> henninge: this is an initialization step, and initialization of distroseries happens in FROZEN state.
<jtv> (It's a bit odd, and the series will also go into that state later on prior to release)
<henninge> indeed
<jtv> But this way we'll only have to fake those files for one Ubuntu release, and the rest will just continue as they were.
<henninge> jtv: I'd probably replace "series = self.factory.makeDistroSeries(status=SeriesStatus.FROZEN)" with a small factory method on the test case which could then have a comment explaining why.
<jtv> henninge: come to think of it, I may already have one... let me check
<wgrant> What's with the status restriction?
<henninge> jtv: is this removing duplicates?
<henninge> self.assertEqual(sorted(markers), sorted(list(set(markers))))
<jtv> Yes
<henninge> jtv: how would that test fail?
<jtv> Duplicates or missing items.
<henninge> this test is to show that it "_uses_separate_files_per_suite".
<henninge> jtv: I don't understand how it show that.
<gmb> adeuring: I made the changes we talked about on Friday, could you take another look at https://code.launchpad.net/~gmb/launchpad/bug-772609/+merge/60176 for me?
<adeuring> gmb: sure
<gmb> Thanks
<jtv> henninge: the test looks up the filenames for the respective suites, then shows that there are no duplicates.
<adeuring> gmb: r=me
<gmb> Thanks
<adeuring> gmb: just one minor question: I assume that the "import BugNotificationRecipients" in personIsAlsoNotifiedSubscriber is necessary to avoid a circular import?
<gmb> adeuring: Ah, yes. I'll add a comment to that effect.
<adeuring> gmb: great., thanks!
<bigjools> wgrant: can you remember when the fix went out to unembargo changelogs properly?
<wgrant> bigjools: A while ago, but I've had production data fixed since then... why?
<wgrant> I can dig up dates if it's important.
<bigjools> wgrant: I've got one that was created 2011-04-21 but is marked restricted
<bigjools> it sounds close to the date but ...
<wgrant> bigjools: I last fixed the data on April 25, around this time.
<wgrant> bigjools: Can you see when the pub was created?
<bigjools> wgrant: https://launchpad.net/ubuntu/+source/gmp/2:5.0.1+dfsg-7ubuntu2
<wgrant> Reading logs I now remember that I misremembered.
<bigjools> so uploaded to the P3A on 21st, unembargoed on 28th
<wgrant> 20:26 < wgrant> (I guess cocoplum is still running the bad code, so more bad data is being created :()
<wgrant> So we need to fix it up one more time.
<wgrant> LOSA ping.
<bigjools> staging has the bad data too ...
<bigjools> wgrant: wrong channel
<bigjools> :)
<wgrant> Oh good, staging unbroke.
<jtv> bigjools: I should test index creation on staging now, not df, right?
<bigjools> jtv: no, DF is still the place unfortunately.
<jtv> Is it in a shape where I can create a series & run publish-ftpmaster?
<bigjools> jtv: yes, should be ok
<jtv> There I go then.  It's been too long since I wrecked a server.
<lifeless> bigjools: wgrant: fyi - https://dev.launchpad.net/ArchitectureGuide/Services
<lifeless> its about 2/3rds written up
<wgrant> There are not enough \o/s in the universe.
<bigjools> lifeless: sweet
* benji changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: henninge, benji | https://code.launchpad.net/launchpad-project/+activereviews
<LPCIBot> Yippie, build fixed!
<LPCIBot> Project db-devel build #529: FIXED in 5 hr 30 min: https://lpci.wedontsleep.org/job/db-devel/529/
<poolie> StevenK: hi
<lifeless> gary_poster: stub: statik: https://dev.launchpad.net/ArchitectureGuide/Services may interest you
<lifeless> gary_poster: I haven't discussed this with you at all as you were away last week :) - but if you can spare time near the end of your day (after I've slept) I've love to do so
<poolie> bigjools: hi, thanks for commenting on the bug
<gary_poster> lifeless, very cool!  I'd love to talk about it.  I'm actually taking today and tomorrow off as well--I'm just here to make sure my squad is all set for the next two days, then I'm disappearing.  I'll take a more thorough glance at the page later.  It would be an exciting initiative.
<bigjools> poolie: np - I am responsible for a lot of the last changes to the email code, hence its hideousness but at the same time I understand what is supposed to happen :)
<poolie> i'm not sure if you mean "so don't just cut it out"
<poolie> or if it's just purely a reminder for the future
<lifeless> gary_poster: kk; well - I am looking for critical analysis and for problems/benefits on both current and possible layout
<gary_poster> lifeless, ok, I'll think in that direction.
<lifeless> gary_poster: this is a synthesis of many ideas that have been floated by many folk : I want to get the best possible chance of fixing real issues and not adding terrible new ones :)
<StevenK> poolie: Sorry, I was having dinner.
<gary_poster> lifeless, I hear you :-)
<lifeless> gary_poster: the page is only about 2/3 written - I have another page of notes and concepts to turn into prose
<poolie> np
<lifeless>  - details about how we might do it, possible migration strategy, making each stage pay for itself sort of thing
<StevenK> poolie: Just doing a quick grep turns up test_handleStatusNotifies in lib/lp/code/model/tests/test_sourcepackagerecipebuild.py -- I can certainly see that one failing.
<poolie> StevenK: that test looks a lot to me like it's asserting there are 0 notifications from ok builds
<StevenK> poolie: Which is, in fact, backward. :-(
<poolie> well, i don't understand why it was passing already
<poolie>  i wondered if it was checking for some other kind of notification
<poolie> is there any?
<wgrant> I haven't read the test, but it's possibly checking for the upload notification.
<wgrant> (you're suppressing the build notification)
<poolie> that seems likely
<poolie> makeSourcePackageRecipeBuildJob
<wgrant> Hm, no, that's a build notification.
<wgrant> Interesting.
<poolie> perhaps it doesn't actually reach the point it would be fired
<poolie> i can trace into it and try to get it to fail with the old code
<wgrant> poolie: I guess it's too asynchronous now.
<wgrant> poolie: handleStatus is meant to be called by Twisted.
<wgrant> poolie: It returns a deferred.
<deryck> Morning, all.
<StevenK> wgrant, poolie: In either case, the test is wrong, and should be removed, no?
<poolie> wgrant: oh, that could interact quite badly with a test that's looking for side effects
<wgrant> poolie: Just a bit.
<wgrant> Well, looking for absence of side-effects without checking that it actually finished.
<poolie> stevenk if it's almost right, perhaps it should be updated to fail, then updated to pass again
<deryck> henninge-lunch, ping for standup
<LPCIBot> Project windmill-db-devel build #253: STILL FAILING in 1 hr 9 min: https://lpci.wedontsleep.org/job/windmill-db-devel/253/
<henninge> deryck: sorry, forgot the time ... and my notebook
<deryck> henninge, no worries.  we're done now though.
<henninge> deryck: ok
<deryck> henninge, the card for bug 196679 is still in progress on the board, but that's ready for deployment, yes?
<_mup_> Bug #196679: ValueError on +language-packs <lp-translations> <oops> <qa-untestable> <Launchpad itself:Fix Committed by henninge> < https://launchpad.net/bugs/196679 >
<henninge> deryck: I was actually able to reproduce the bug on Friday so I am able to provide an actual fix for the bug.
<henninge> deryck: that's straight forward but I have not yet returned to it today.
<henninge> deryck: will be finished today, though.
<deryck> henninge, ok.  you're OCR today, so no expectations from me about finishing it.
<deryck> henninge, just wanted to sort out that board
<jml> do we support binonlynmu uploads?
<jml> istr talking about it a while ago
<bigjools> jml: no we don't
<jml> bigjools: thanks. Do you know if there's a bug report for it?
<bigjools> it's BinNMU if you want to get the terminology right :)
<bigjools> ohhhh yes
<bigjools> there's bugs all right
<jml> bigjools: hmm, I'm sure I heard people hear say "only" :)
<bigjools> https://bugs.launchpad.net/launchpad/+bug/245594
<_mup_> Bug #245594: Rebuilds of binary packages without source changes <feature> <lp-soyuz> <motu> <Launchpad itself:Triaged> < https://launchpad.net/bugs/245594 >
<jml> bigjools: thanks.
<bigjools> jml: the crux is that people want to re-generate binaries for the already-uploaded source
<bigjools> which implies an auto version bump for the binaries
<jml> bigjools: makes sense.
<sinzui> henninge: benji, will either of you have time to review https://code.launchpad.net/~sinzui/launchpad/anwers-api-0/+merge/60238
<benji> sinzui: I can.
<wgrant> sinzui: I have a couple of Answers API bugs that I plan to fix tomorrow, after finding them just now when hiding spam. Mostly fixing IQuestionSet['getByID']'s return type, exposing messages, and fixing setCommentVisibility's comment numbers so they match the ones shown in the UI.
<wgrant> I'm not going to conflict with yours, am I>?
<sinzui> fab
<sinzui> wgrant: I do not think so. I exported answer contact management in this branch
<wgrant> Great.
<henninge> benji: Hi! I'll pick something else from activereviews.
<wgrant> sinzui: I shall speak with you in 9 hours, I suppose.
<wgrant> For the first time in a while.
<benji> henninge: sounds good
<henninge> rvba_: is either of your branches more urgent?
<sinzui> wgrant: yes. I am sorry about last week. I fell ill late in my day and could not get off the sofa
<henninge> rvba_: oh, one is a db patch. Not for me ;)
<wgrant> sinzui: :( Better now?
<sinzui> Yes, thanks
<rvba_> henninge: the other is really not urgent ... but if you want to review it, thanks fine by me :)
<rvba_> s/thanks/that's/
<adeuring> deryck: could you please run these two queries on staging: https://pastebin.canonical.com/47268/ ?
<deryck> adeuring, sure
<adeuring> thanks!
<deryck> first one still going .... ;)
<benji> sinzui: I'm done with https://code.edge.launchpad.net/~sinzui/launchpad/anwers-api-0/+merge/60238
<sinzui> thanks benji
<benji> my pleasure
<benji> henninge: are you doing any reviews right now?  I'm doing the unclamed ones and want to be sure I don't double-up.
<henninge> benji: I was but I forgot to claim it ...:(
<henninge> benji: pick anyone you like.
<benji> k, thanks
<rvba_> benji: thanks for the review!
<benji> my pleasure
<benji> henninge: an idea: to keep us from stepping on each other's toes (something I seem to be good at), should we have a gentleman's agreement to both claim the review and tell the other that we claimed it?
<poolie> thanks for the review benji!
<benji> my pleasure
* henninge changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: benji | https://code.launchpad.net/launchpad-project/+activereviews
<henninge> benji: I am outta here.
<poolie> review_s_
<poolie> remarkable
<poolie> bigjools: can you fix the syntax beore you land the flag?
<poolie> or do a followup?
<bigjools> poolie: ?
<poolie> flags should have underscores not dashes, like python
<poolie> names
<bigjools> oh, didn;t know that.  I was basing it on an existing similar flag
<poolie> at the moment they're inconsistent which seems kind of gross and likely to cause mistakes
<poolie> :)
 * bigjools fixorates
<poolie> thanks
<poolie> i guess i should fix them all to be one way
<poolie> ust wanted to stop it getting worse
<bigjools> I hope hitting ctrl-c terminates the ec2 instance :)
<jml> bigjools: './utilities/ec2 list' should tell you
<poolie> if it's already gone it's not a big deal
<bigjools> yeah waiting for it to finish ...
<poolie> you can kill it from the ec2 console
<bigjools> yeah
<bigjools> jeepers it's been going for 4 minutes now
<bigjools> right, sorted, thanks for pointing that out poolie, I am now wiser than I was yesterday
<poolie> np
<poolie> generally i am not a big fan of pedantic conventions but here it seems worth while
<poolie> since they'll be communicated to losas etc
<bigjools> poolie: I am in total agreement with everything you just wrote :)
<LPCIBot> Project windmill-devel build #51: STILL FAILING in 1 hr 14 min: https://lpci.wedontsleep.org/job/windmill-devel/51/
<bigjools> night everyone
<LPCIBot> Project windmill-devel build #52: STILL FAILING in 47 min: https://lpci.wedontsleep.org/job/windmill-devel/52/
<sinzui> jcsackett: do you have time to mumble?
<jcsackett> sure, give me one moment.
<sinzui> jcsackett: I think I need to restart mumble again
<jcsackett> sinzui: i am on mumble, i am not sure i have voice.
<jcsackett> sinzui: righto.
<sinzui> jcsackett: I do not see an indication you are speaking
<jcsackett> sinzui: i have just restarted. i too see no indication of you speaking.
<jcsackett> sinzui: i heard you.
<jcsackett> sinzui: i think perhaps my mike is not working.
 * jcsackett is checking audio settings.
<jcsackett> sinzui: i think mumble just does not work for me today. i just had a full crash trying to fix it.
<sinzui> ouch
<jcsackett> sinzui: and now a unity crash trying to start mumble from terminal.
 * jcsackett tries mumble one last time.
<jcsackett> sinzui: i gather you still do not here me?
<jcsackett> s/here/hear/
<sinzui> jcsackett: No. Can I call you Patsy, my mute servant
<jcsackett> sinzui: i would prefer not. :-P
<jcsackett> skype? ekiga?
<sinzui> I will start skype
<LPCIBot> Project windmill-devel build #53: STILL FAILING in 1 hr 7 min: https://lpci.wedontsleep.org/job/windmill-devel/53/
<sinzui> benji: I think your hide notification link script wires itself to every link in the notifcation: https://bugs.launchpad.net/launchpad/+bug/779538
<_mup_> Bug #779538: Link from superseded blueprint to its successor disappears when you click it <javascript> <regression> <Launchpad itself:Triaged> < https://launchpad.net/bugs/779538 >
<benji> sinzui: ooh, that's not nice; I'll take a look at it.  Do you have an example?
<sinzui>  https://blueprints.edge.launchpad.net/ubuntu/+spec/desktop-o-auto-rotate
<sinzui> benji: it will also affect an deactivated project or bug converted to a question
<abentley> benji: Could you please review https://code.launchpad.net/~abentley/launchpad/person-product-merges/+merge/60417 ?
<benji> abentley: sure
<benji> sinzui: for me it hides the message box but doesn't prevent the browser from navigating to the link, is that the same behavior you see?
<benji> if so, I think I can fix that
<sinzui> benji: yes. many user want to open the link in a new tab and see the link they used in the previous tab
<benji> absolutely, I wasn't arguing for not considering it a bug, just verifying what you were seeing
 * benji hates web apps that break middle-click/control-click to open a new tab.
<benji> abentley: I'm done with https://code.launchpad.net/~abentley/launchpad/person-product-merges/+merge/60417
<abentley> benji: I prefer to write single-line imports as multi-line imports, so that we don't have to change the formatting later.
<benji> abentley: I'm not stingy with vertical whitespace, but that's a bit too much even for me. ;)  It's a good thing we're trying to not be so up tight about style issues. :)
<benji> sinzui: is there a bug for the overly agressive hiding?  If not, I'll file one.
<sinzui> I pasted you the bug, so I thought
<sinzui> https://bugs.launchpad.net/launchpad/+bug/779538
<_mup_> Bug #779538: notifications disappear when you click *any* link in the text <javascript> <regression> <Launchpad itself:Triaged> < https://launchpad.net/bugs/779538 >
<sinzui> ^ benji
<benji> thanks
<abentley> deryck: I'm looking at fixing https://bugs.launchpad.net/launchpad/+bug/777958 and I'd like to do a pre-imp call.
<_mup_> Bug #777958: branch upgrade jobs keep transaction open <oops> <Launchpad itself:Triaged> < https://launchpad.net/bugs/777958 >
<lifeless> sinzui: deryck: https://dev.launchpad.net/ArchitectureGuide/Services - incomplete, but the product of the discussions so far
<sinzui> thanks
<lifeless> jml: !! http://pypi.python.org/pypi/van.pg
<lifeless> benji: hi; please triage-as-you-file, it saves double work (bug 779958)
<_mup_> Bug #779958: lazr.restful.error.expose isn't documented <lazr.restful:New> < https://launchpad.net/bugs/779958 >
<benji> lifeless: will do
<lifeless> benji: thanks! I've triaged that particular one; Its just a kindness to the maintenance squads really :)
<benji> thanks
<lifeless> statik: hi
<abentley> deryck: ping
<deryck> hi abentley
<deryck> abentley, i can chat here in just a sec....
<deryck> just got back
<abentley> deryck: ah, cool.
<deryck> school run took much longer than I expected
<deryck> lifeless, thanks for the link.  our call is top of this hour tomorrow, yes?
<lifeless> deryck: or today if you like, statik has timeshifted for uds, and flacoste is @ uds
<deryck> lifeless, abentley is waiting on a call now.  so tomorrow would be better.
<lifeless> deryck: I'm easy
<abentley> deryck: okay.
<deryck> abentley, firing up mumble now
<abentley> deryck: I'm looking at fixing https://bugs.launchpad.net/launchpad/+bug/777958 and I'd like to do a pre-imp call.
<_mup_> Bug #777958: branch upgrade jobs keep transaction open <oops> <Launchpad itself:Triaged> < https://launchpad.net/bugs/777958 >
<LPCIBot> Project windmill-db-devel build #254: STILL FAILING in 1 hr 7 min: https://lpci.wedontsleep.org/job/windmill-db-devel/254/
<sinzui> Is there a method on storm objects that can be called to reinstantiate the object after I call reconnect_stores()?
<lifeless> sinzui: *blink*
<lifeless> sinzui: I tihnk you need to reload the objects
<lifeless> sinzui: looking at existing tests
<sinzui> lifeless: I was regetting the objects (IPersonSet.get()) but that clutters a lot of tests. review.reload() would be nicer to read
<lifeless> sinzui: this is in a layer that can't use switchDbUser ?
<sinzui> of well. I will write a helper. for the test harness
<LPCIBot> Project windmill-devel build #54: STILL FAILING in 1 hr 7 min: https://lpci.wedontsleep.org/job/windmill-devel/54/
<deryck> catch you all later on
<lifeless> how do you get a js console on chromium?
<benji> lifeless: shift-control-J on mine
<lifeless> benji: thanks
<benji> (or navigate into the deep, deep, dark menus)
<benji> np
* benji changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: - | https://code.launchpad.net/launchpad-project/+activereviews
<rvba_> Hey guys I have a traversal problem I can't wrap my head against.
<rvba_> I want this kind of url to publish a DSD: /derived-distro/derived-series/+source/package-name/+difference/parent-distro/parent-series
<rvba_> My problem is that I can't see how to use @stepthrough because I need *2* parameters (parent-distro *and* parent-series)
<rvba_> Any advice?
<lifeless> its fugly but you probably want an adapted parent distro as a place holder
<rvba_> you mean an object that would be accessible at  /derived-distro/derived-series/+source/package-name/+difference/parent-distro ?
<lifeless> yes
<lifeless> representing the diff of derived-distro/derived-series/+source/package-name to parent-distro
<lifeless> which is noddy
<lifeless> so you wouldn't make a fat interface
<lifeless> you'd only offer enough to traverse
<rvba_> I see
<lifeless> basically currying a factory for the thing you do want
<rvba_> right
<lifeless> which is the difff from  derived-distro/derived-series to parent-distro/parent-series of package-name
<rvba_> makes sense ... I kinda hoped to get away with something simpler ... but I suppose I don't really have a choice here :)
<LPCIBot> Project windmill-devel build #55: STILL FAILING in 43 min: https://lpci.wedontsleep.org/job/windmill-devel/55/
<rvba_> lifeless: thanks a lot, I'll sleep on it and get this done tomorrow.
<rvba_> 'night everyone.
<lifeless> rvba-afk: gnight
#launchpad-dev 2011-05-10
<wgrant> lifeless: It's actually much easier than that.
<wgrant> Use self.request.stepstogo.consume()
<sinzui> wgrant: mumble?
<wgrant> sinzui: I'm there.
<wgrant> ... but in the wrong room.
<sinzui> I think my audio is buggered again. I will leave and return
<wgrant> I can hear you.
<wgrant> lifeless: QuestionMessage doesn't have indices in the DB.
<lifeless> wgrant: well then.
<lifeless> wgrant: it needs them or spam marking is racy
<jtv> wgrant: any idea why the df app server wasn't running?
<wgrant> jtv: None.
<jtv> Oh well, I started it.
<wgrant> Well, unless "bigjools" is an idea.
<wgrant> Also, it's 9:30am here, you should not be here.
<lifeless> jetlag
<jtv> 'zcly
<jtv> Plus, my UPS can't sleep.
<wgrant> lifeless: Try to assign/unassign a private bug on qastaging.
<lifeless> wgrant: why?
<wgrant> lifeless: Because it crashes for me.
<lifeless> what bug were you trying on
<wgrant> 1234
<lifeless> Indirect subscribers found on private bug. A private bug should never have implicit subscribers!
<wgrant> Yes.
<wgrant> Can you change status/importance?
<lifeless> win
<wgrant> They OOPS for me without a traceback.
<wgrant> But it's possibly just a timeout.
<lifeless> OOPS-1955QASTAGING103
<lifeless> ID:OOPS-1955QASTAGING104
<wgrant> Yeah, been waiting for a sync for a while :(
<lifeless> 3 seconds, not a timeout
<wgrant> Thanks.
<lifeless> uhm
<lifeless> is ~launchpad in malone-alpha
<lifeless> no
<wgrant> But I am.
<lifeless> so i'm getting this with the 'normal' notification stuff
<wgrant> On qas.
<lifeless> yes
<wgrant> It works fine on dev.
<wgrant> I wonder if a custom filter has to exist.
<wgrant> I also wonder if it's related to the fix for bug creation subscribers.
<wgrant> If I can find it...
<wgrant> No.
<lifeless> now, is indirect 'via a team' or 'via a pillock' ?
<wgrant> Pillar.
<wgrant> Not team.
<wgrant> fuuu
<lifeless> ECONFUSINGTERMINOLOGY
<wgrant> lazr.restful suppresses the traceback.
<wgrant> Let's use non-AJAX...
<lifeless> theres a bug on that
<wgrant>  OOPS-1955QASTAGING107
<wgrant> Module lp.bugs.subscribers.bug, line 151, in add_bug_change_notifications
<wgrant> level=BugNotificationLevel.METADATA)
<wgrant> Module lp.bugs.model.bug, line 1050, in getBugNotificationRecipients
<wgrant> "Indirect subscribers found on private bug. "
<wgrant> AssertionError: Indirect subscribers found on private bug. A private bug should never have implicit subscribers!<br />
<wgrant> Not too helpful.
<wgrant> But it's something.
<wgrant> Surely not.
<wgrant> This would have been broken for weeks.
<lifeless> I envy you; you're getting to code.
<wgrant> You're not?
<lifeless> I'm architecting at the moment
<lifeless> https://dev.launchpad.net/ArchitectureGuide/Services
<lifeless> which is harder than you might think
<wgrant> Ah, yes.
 * wgrant comes dangerously close to deleting bug heat.
<lifeless> \o/
<lifeless> which reminds me
<lifeless> to do an analyse of bug heat top-20 or whatever with and without decay
<lifeless> I'm totally convinced that in successful projects the sort order is identical
<lifeless> but ENEEDSDATA
<LPCIBot> Project windmill-devel build #56: STILL FAILING in 1 hr 15 min: https://lpci.wedontsleep.org/job/windmill-devel/56/
<jtv> StevenK, perhaps you would care to review a small fix for automatic index creation in publish-ftpmaster?  https://code.launchpad.net/~jtv/launchpad/bug-780214/+merge/60444
<LPCIBot> Project db-devel build #531: FAILURE in 5 hr 20 min: https://lpci.wedontsleep.org/job/db-devel/531/
<jtv> Or wgrant maybe?  Need a small fix reviewed.
<jtv> No antipodeans in the review market today?
 * wgrant returns.
<wgrant> jtv: What creates that normally?
<jtv> wgrant: I don't even know tbh
<wgrant> Also, why always PRIMARY, never PARTNER?
<wgrant> generateConfigForPocket does this.
<wgrant> As long as it's given some components.
<wgrant> I presume this failed on DF?
<jtv> There are no separate PARTNER runs for this, as far as this script is concerned, so it uses PRIMARY as the place to keep track.
<LPCIBot> Project windmill-db-devel build #255: STILL FAILING in 1 hr 7 min: https://lpci.wedontsleep.org/job/windmill-db-devel/255/
<jtv> Yes.
<wgrant> Do you still have the traceback?
<jtv> I can make one.  Hang on.
<jtv> wgrant: http://librarian.dogfood.launchpad.net/71105515/hyYRRP1nMvTD9b3OKTaVtUZy789.txt
<wgrant> jtv: querulous hasn't been initialised, has it?
<jtv> Nope
<wgrant> It has no components.
<wgrant> There's a bug for that.
<wgrant> But why are you writing out indices before initialisation?
<jtv> Skipping a few steps in testing, I guess.
<wgrant> So this won't be the case in production?
<wgrant> what triggers the index generation?
<jtv> publish-ftpmaster.
<wgrant> That's what runs it.
<wgrant> In what case does it run?
<jtv> wgrant: little accident with chat window here...  index generation is triggered by publish-ftpmaster being run on the distro and seeing that one or more suites in a frozen series do not have indexes yet.
<wgrant> jtv: Hmm, that's too early.
<wgrant> It needs to wait until it's been populated.
<wgrant> Bug #675042 covers the bug that you've fixed here, but it reveals that it's being run too early.
<_mup_> Bug #675042: Release file generation fails for series without components <lp-soyuz> <soyuz-publish> <Launchpad itself:Triaged> < https://launchpad.net/bugs/675042 >
<lifeless> tsekci times
<wgrant> Pardon?
<lifeless> just being silly
<lifeless> a riff on other interpretations of 'populated'
<jtv> wgrant: you are taking into account that publish-ftpmaster will not be running around the time a new series is created?
<wgrant> jtv: Hm, why not?
<wgrant> He's good at that.
<wgrant> 11:02:06 < wgrant> jtv: Hm, why not?
<wgrant> We currently stop the publisher so the manual index generation can be run.
<wgrant> If we're doing the manual index generation automatically, there's not much point stopping the publisher.
<jtv> My understanding is that stopping the various cron jobs was still needed for other reasons as well.
<wgrant> That sounds like a temporary situation that we don't want to add more reasons to.
 * wgrant invokes StevenK.
<jtv> wgrant: when we get close to the point where the cron jobs no longer need to be disabled, we'll also have a better view of what else is needed to manage the various jobs that need to be done.
<StevenK> wgrant: Oh?
<wgrant> StevenK: Why do we still have the publisher off during initialisation?
<StevenK> I hadn't done any work on it -- but we don't want to publish a half-initialised series
<wgrant> jtv: Regardless, this fix should probably be in ftparchive.py. And it's almost tempting to keep the crasher to indicate that something has gone wrong.
<wgrant> StevenK: Sure, but that's probably better handled by IDS signaling that it's finished and enabling publication of the series.
<StevenK> Agreed
<jtv> How do you enable publication of a series?  I thought that was per-distro.
<wgrant> Perfect may be the enemy of good and all that, but perfect is not difficult here, and making it easier to silently do things wrong is also the enemy of good.
<wgrant> jtv: Indeed, but that doesn't make much sense.
<jtv> What is "that" in this case?
<wgrant> jtv: DistroSeries needs an "initialised" flag or so. At the moment we have series like p-series sitting around and only not being published because they have nothing to publish.
<wgrant> The publisher should only consider initialised series.
<wgrant> And that solves this problem.
<jtv> This is separate from the "initialised" flag on DSP, I take it.
<wgrant> I don't think that flag should exist.
<wgrant> But perhaps it should.
<jtv> That's actually a recurring problem here, I think.
<wgrant> Oh?
<jtv> A proliferation of status flags for what is really a process flow.
<StevenK> That's why I think it should be a DBEnum
<wgrant> The larger problem is that it is apparent nobody has though about how this is all going to work.
<wgrant> s/though/thought/
<jtv> Exactly.
<lifeless> thinking is overrated
 * lifeless goes back to thinking
<StevenK> I don't think it belongs on IDistroSeries, but only because it's enormous.
<wgrant> I have thought about fixing DS.status.
<wgrant> It is possible taht we could reuse that.
<wgrant> We need to think how it will interact with Debian.
<wgrant> But it seems reasonable.
<lifeless> StevenK: I don't think thats a good reason not to do it
<lifeless> StevenK: thats a good reason to make it smaller, but if you hold of on genuine improvements because its not [yet] small, you hamstring yourself.
<lifeless> StevenK: you could say 'we won't make it bigger' and go yak shaving to make it smaller to let you put this field in and keep it hte same size
<wgrant> jtv: I am really worried that there is no design for any of this.
<wgrant> jtv: It's just a set of cards.
<wgrant> jtv: With people working it out as they go.
<lifeless> but that seems like saying 'making the class smaller is more important than making this part of the process reliable' - which I wouldn't agree with.
<jtv> wgrant: same here, but I'm not a big fan of chucking in an enum right away either.
<wgrant> jtv: Sure, we should actually talk about this.
<wgrant> With Julian and stuff.
<jtv> Yes.
<jtv> I think we're sort of racing past an underlying problem,
<wgrant> Rather than just piling more hacks onto the pile of hacks that is already a significantly larger pile than before DDs.
<jtv> which is that we don't have a clear map of the dependencies between actions.
<jtv> We have the current Ubuntu process, which is a serialization of actions--out of some set of possible ones.
<wgrant> Riht.
<jtv> So I'm thinking: the first thing we ought to do is figure out what the dependencies are.
<jtv> Which is hard, and involves goat sacrifice.
<wgrant> That sounds dangerously close to planning ahead of time...
<jtv> Or some other way of seeing into the nether realm of soyuz.
<jtv> Planning what ahead of what time?
<wgrant> A feature.
<wgrant> Or development thereof.
<jtv> ?
<jtv> I'm saying we should understand what it is that we're already doing.  That doesn't strike me as premature feature development planning.
<StevenK> jtv: wgrant is showing his bitterness again.
<jtv> He's too young to be bitter.
<jtv> (For a non-Windows user, that is)
<wgrant> It was a derogatory implication that much of this work lately has been quickly patching over immediate problems without considering the underlying problems.
<wgrant> Some post-release updates need to go to -updates? Hardcode -updates when datereleased is set! We need to create distroseries indices after a series has been initialised? We turn the publisher off during initialisation because we haven't quite fixed that yet, so we can just assume the publisher will be off!
<wgrant> etc.
<wgrant> The feature was initially planned well, but the issues that have cropped up during development have had no planning at all.
<wgrant> And this is how Soyuz got as bad as it is now.
<jtv> I think these are two separate issues though.
<jtv> The new requirements were thrown into the design in a hurry.  Very bad.
<jtv> But what we were discussing is old design, and there a few hackish small steps may actually bring us to a better ultimate situation.
<wgrant> But there has been no consideration of the ultimate solution before implementing the hackish small steps.
<wgrant> So we don't know if we're going in the right direction.
<wgrant> Or if we're going the direction soyuz did for its first three or so years.
<jtv> Part of the problem is that we don't know what the ultimate initialisation procedure for a new series will look like.
<wgrant> (which was possibly not unreasonable, due to time constraints, but that's not the case here)
<wgrant> jtv: We need to sit down and work that out, I guess.
<jtv> But I suspect trying to design that up front may lead us to decisions that turn out to be wrong at a later stage.  In which case it's more effective to let a pattern emerge first, and then design for the pattern.
<wgrant> What is in question?
<lifeless> wgrant: this isn't how soyuz got to its current state
<wgrant> lifeless: Oh?
<wgrant> Parts of it are very clear layers of hacks.
<lifeless> yes
<wgrant> Much moreso than the rest of LP.
<lifeless> wgrant: there was a lot of BDUF done early on
<lifeless> wgrant: and it was wrong and never corrected
<wgrant> Sure.
<wgrant> I'm not asking for BDUF on that scale.
<wgrant> I am asking for a minimal amount of design.
<lifeless> wgrant: I'm arguing that that is the root cause of many of the hacks
<wgrant> Not 5-year speculative design like Soyuz.
<wgrant> Hmm.
<wgrant> Partly, yes.
<wgrant> But not entirely.
<lifeless> now, subsequent work hasn't fixed the root issues, and has yes papered over stuff
<lifeless> I'm all for aiming for a good design and root cause fixes
<lifeless> I don't think you're wrong to argue that some more analysis is needed
<lifeless> I think its false to say that incremental development got us here
<lifeless> what got us here was overly focused designs that didn't address the root causes which /made/ the basic facilities hard to reuse.
<lifeless> for instance, picking something arbitrary we publish archives in two different ways for no good reason
<StevenK> That's because Celso got cold feet about NMAF.
<wgrant> No.
<wgrant> PPAs need to be done, NMAF wasn't fast or complete enough for primary.
<wgrant> Completion never happened.
<wgrant> Nothing about getting cold feet.
<StevenK> Ah. And since it worked fine for PPAs, it was good enough.
<lifeless> or we could have used apt-ftp for ppas [with different tradeoffs]
<wgrant> It turns out it didn't work fine, but the minor additional fixes were made.
<wgrant> But yes.
<lifeless> but the choice to bring in a different stack was done with sprints and design; the underlying issue wasn't, and hasn't been addressed.
<wgrant> lifeless: For big stuff like that, yes, incremental development is not at fault. But there are so many special-cases and other gross hacks throughout Soyuz which are labelled as temporary hacks that will be discussed at $nextsprint, dated 2005 or 2006.
<lifeless> wgrant: I think that shallower stuff is due to the overall 'we don't get to finish things' issue.
<lifeless> wgrant: I give you e.g. blueprints
<lifeless> :)
<wgrant> lifeless: And we still don't get to finish things.
<wgrant> Even with feature squads.
<lifeless> thats interesting
<lifeless> I thought we were doing a lot better
<wgrant> We set a deadline for feature completion weeks beforehand.
<wgrant> Then the last couple of weeks are very conservative.
<lifeless> I think you should raise this on one of the two lp dev lists
<wgrant> Feature completion is not based on completion of the feature.
<lifeless> we have a glass display case for well, glasses
<lifeless> it has a bottom draw thats wood
<lifeless> so the glass starts about 25/30cm from the floor
<lifeless> our newest kitten tries to jump into the cabinet
<lifeless> so there is this patter patter patter ... THUMP
<wgrant> Ahaha.
<lifeless> as he propels himself up that 30cm gap aiming at where he can see stuff... and whacks into the glass front
<wgrant> Smaaart.
<wgrant> Wait, newest kitten? You have acquired multiple now?
<lifeless> we have 2 (the desired number) now
<StevenK> And then shakes himself off and goes to lay in the sun?
<wgrant> Sun in NZ? lol.
<StevenK> Oh. Right.
<StevenK> s/sun/slightly warmer bit/
<lifeless> sun, no. chases the other cat, then tries again a bit later
<wgrant> (although it was around 0 here this morning, so I guess I can't really talk)
<StevenK> It was 10 degrees when I went to bed
<StevenK> That's cold enough
<wgrant> Yay, bug heat timeouts on production now.
<wgrant> OOPS-1956BA33
 * StevenK waits the 30 seconds that OpenID seems to take
<StevenK> Sigh, searching for oops makes me sigh
<jtv> wgrant: some prolonged disconnectivity there...  What is in question?  Well currently, how we trigger and track the various steps in setting up a new series.
<wgrant> jtv: Right, but why can this not be determined now?
<jtv> I'm on thin ice there, but perhaps because so many things are shifting: we're generalizing beyond Ubuntu, IFP is becoming a job, we're introducing multiple parent relationships.
<wgrant> Right.
<wgrant> And the LEP basically only specifies +localpackagediffs UI.
<jtv> You'll hear no argument from me there...
<wgrant> Oh!
<jtv> ?
<wgrant> Someone agrees that the design is insufficient :)
<jtv> I've been saying that for quite a while now I think.  You may not have heard all of it because it was on team calls.
<wgrant> I have seen nothing other than a continuous stream of branches that reveal woefully inadequate design.
<wgrant> I am glad that there has been some discussion or at least realisation that this is the case :()
<jtv> But again, I see different considerations for the design we inherited and the one we're trying to implement.
<wgrant> *:)
<jtv> I don't think anyone particularly disagrees that we have a hurried design and it's not exactly a work of fine art.  But what we _do_ about it is another kettle of fish altogether.
 * wgrant sighs and lunches.
<wgrant> 3 rollbacks in 24 hours :(
<spm> new record?
<wgrant> I think so, yes.
<wgrant> We've had two a couple of times. But not three.
<spm> silver lining - is new record in our QA process to ensure that quality code lands on the LP prod branch. We're not afraid to rollback bad versions on discovery!
<wgrant> Heh
<lifeless> waheeee
<lifeless>  548 KeyError: 56589
<jtv> that's expressing just plain disdain for humans
<wgrant> lifeless: That's the send-bug-notifications crap.
<wgrant> You might have noticed there were tens of thousands of them yesterday.
<wgrant> It is happy now.
<lifeless> kk
<lifeless> thanks
<lifeless> is wallyworld doing uds?
<wgrant> lifeless: The first couple of days, I think.
<wgrant> Ian Booth
<wgrant> 2011-05-09 till 2011-05-10
<lifeless> thanks
<wgrant> jtv: Hi.
<wgrant> https://answers.launchpad.net/launchpad/+questions?field.search_text=&field.sort=RELEVANCY&field.sort-empty-marker=1&field.actions.search=Search&field.language=en&field.language-empty-marker=1&field.status=OPEN&field.status=NEEDSINFO&field.status-empty-marker=1 has four questions assigned to Launchpad Translations Administrators/Coordinators. Who is meant to handle that sort of thing now?
<lifeless> aren't we all rosetta admins now?
<jtv> Yes
<jtv> hi wgrant... my life is so nice and interruption-free now that I'm no longer notified of messages
<wgrant> Some of these are policy.
<wgrant> Heh.
 * jtv looks
<jtv> By the way, Launchpad Translations Coordinators are a very different team.  They run our reference translation group.
<jtv> Of the ones for rosetta admins, one is in Needs Information.
<jtv> Another actually should have been un-assigned.
<wgrant> jtv: Right, I know they're a different group, which is why I was asking about them :)
<wgrant> Because I doubt they watch LP questions.
<jtv> Good point.
<lifeless> I need a proof reader
<lifeless> Someone that will read a doc and make a list of wtf moments to give to me
<lifeless> wgrant: tag, you're it ^, if you have time.
<wgrant> I'm not going to get any useful code done today, so I might as well.
 * wgrant looks.
<jtv> wgrant: you were right by the way--IFP fixed my directories problem.
<wgrant> jtv: Great.
<wgrant> jtv: Are bug #55211 and bug #777941 deployable?
<_mup_> Bug #55211: shouldn't require careful publisher run when cloning new released distrorelease <derivation> <lp-soyuz> <new-release-cycle-process> <qa-needstesting> <soyuz-publish> <Launchpad itself:Fix Committed by jtv> < https://launchpad.net/bugs/55211 >
<_mup_> Bug #777941: Create distroseries indexes from regular publisher run <derivation> <qa-needstesting> <Launchpad itself:Fix Committed by jtv> < https://launchpad.net/bugs/777941 >
<jtv> wgrant: yes, because we're not using that script yet anyway.
<wgrant> That's what I thought. Thanks.
<jtv> Also, I just got it working on df.  :)
<wgrant> Excellent!
<jtv> Now to fake those marker files for O.
<jtv> Or rather, figure out whether we need to.
<wgrant> Well, what's it going to do? A no-op index publication when we set it to frozen?
<LPCIBot> Project windmill-devel build #57: STILL FAILING in 1 hr 7 min: https://lpci.wedontsleep.org/job/windmill-devel/57/
<jtv> wgrant: I was told that doing a publish-distro -A on the suites at such a later stage would be harmful.
<jtv> StevenK: where is IDerivedDistro as mentioned in bug 643369?
<_mup_> Bug #643369: IDistroSeries.deriveDistroSeries() should use the security adapter <derivation> <lp-registry> <Launchpad itself:Triaged> < https://launchpad.net/bugs/643369 >
<wgrant> jtv: I don't believe that to be the case.
<wgrant> It's not ideal.
<wgrant> But it shouldn't break anything.
<wgrant> Undesirable, but not harmful beyond possibly wasting a few minutes.
 * jtv suffers a mild gust of despair from hearing these contradictory requirements
<jtv> It's safe to run publish-distro -A on a series (and suites) that has already been published?
<wgrant> Yes. It will do bad things if it's released, but frozen is not release.
<wgrant> +d
<wgrant> All it does it forces the pockets to be dirty.
<jtv> wgrant: that doesn't sound so bad, for a one-off migration issue
<jtv> Hmmm this looks useful: http://justinhalpern.tumblr.com/post/5135237870/office-e-mail-translator
<wgrant> jtv: I hoped it would be something to de-Outlook HTML.
<jtv> Alas.
<jtv> wgrant: I'll ignore the marker files issue then, but leave it on the table for a double-check with jools.
<wgrant> jtv: Sounds good.
<jtv> Meanwhile, trying to make sense of bug 643369, which is marked as a high priority.  What is this IDerivedDistro it speaks of, and why should Ubuntu not count as one once it's marked as derived from Debian?
<_mup_> Bug #643369: IDistroSeries.deriveDistroSeries() should use the security adapter <derivation> <lp-registry> <Launchpad itself:Triaged> < https://launchpad.net/bugs/643369 >
<wgrant> jtv: Because 2005, that's why.
<jtv> Ah yes, 2005, the explanation for everything.  :)
<jtv> wgrant: still, what would determine whether a distro is an IDerivedDistro?
<wgrant> I think the owner is a more sensible check, but using IDerivedDistro is stupid and wrong.
<wgrant> IDerivedDistro itself is stupid and wrong.
<wgrant> In its current definition, at least.
<wgrant> I believe it's currently != ubuntu.
<wgrant> Which is obviously crap.
<jtv> So... debian would be an IDerivedDistro!?
<wgrant> Yes.
<wgrant> When it is in reality the only non-derived distro.
<wgrant>         if self.name == 'ubuntu':
<wgrant>             alsoProvides(self, IBaseDistribution)
<wgrant>         else:
<wgrant>             alsoProvides(self, IDerivativeDistribution)
<StevenK> IDerivedDistro is effectively "Any distro which isn't Ubuntu"
<wgrant> (From Distribution._init)
<StevenK> jtv: I think that bug is hard to fix, too
<jtv> Ah, "Derivative."  That's why I got no hits for "Derived."
<wgrant> Ah, IDerivativeDistribution is, in its current form, used for Ubuntu-specific permissions.
<wgrant> So you might want to rename it to IHaveBadlyDefinedUbuntuDrivers.
<jtv> Heh
<jtv> Maybe I shouldn't be working on this bug then, especially since it doesn't say why the change is needed.
<wgrant> It's certainly not pressing.
<jtv> It's marked as a high priority.
<jtv> Unlike all other pending tasks.
<StevenK> It's needed because currently you need to be a member of soyuz-team or an admin to derive a series. You should be a driver or owner of the destination d
<jtv> Oh, that
<StevenK> *distribution (since the series may not exist)
<jtv> That's much clearer, thanks.
<jtv> StevenK, wgrant: how weird to have deriveDistroSeries on DistroSeries...  I would have expected it to be on the derivative distribution.
<LPCIBot> Project windmill-devel build #58: STILL FAILING in 43 min: https://lpci.wedontsleep.org/job/windmill-devel/58/
<StevenK> jtv: The DistroSeries you call it from is the one you're deriving from?
<jtv> Yes.
<jtv> But ISTM you only need read permissions on that.  The derivative distro is what you're appending to.
<StevenK> Yes ...
<jtv> And then it checks if the derived series you want to create may already exist, and if it does, initialize it.
<StevenK> jtv: Completly seperate.
<StevenK> jtv: We use the object were calling deriveDistroSeries() on as the parent series -- so it doesn't need to be fed into the argument list.
<StevenK> And deriveDistroSeries() is happy enough to create the series if it can't find it.
<jtv> Frightening.
 * StevenK goes back to fixing tests
<jtv> ISTM this wants to be a method on the parent Distribution plus a method on the derived DistroSeries, rather than a single method on the parent DistroSeries.
<StevenK> What?! You're making no sense, and I'm getting cross.
<wallyworld> wgrant: ping
<wgrant> wallyworld: Hi.
<wgrant> jtv: Possibly, but it may be sensible to do them in one operation sometimes.
<jtv> Sensible or easy?
<wallyworld> wgrant: i have had a go at moving the gpghandler stuff to the tac file but am unsure a) if it's correct and b) how to test. could you have a look?
<wallyworld> wgrant: i've not seen or used the tac stuff before. here's a draft mp https://code.launchpad.net/~wallyworld/launchpad/poppy-sftp-gpgconf/+merge/60454
<wgrant> wallyworld: I didn't do it myself precisely because I couldn't think of a quick way to test it.
<wgrant> Twisted people and/or lifeless may have ideas.
<wallyworld> wgrant: ok. what's the bit of code that event loads and runs the tac files?
<wallyworld> even
<wgrant> wallyworld: They're run by twistd.
 * StevenK reads the MP and vomits
<StevenK> Surely there HAS to be a better way
<wgrant> wallyworld: See utilities/start-dev-soyuz.sh
<wallyworld> wgrant: ok.
<wallyworld> StevenK: you're welcome to tell me a better way
<StevenK> Extending gpg handler that didn't use directory that gets reaped would be nice
<wallyworld> StevenK: the bug report had a bit of discussion and what's been done seems to be the best that could be though of
<wallyworld> StevenK: apparently there's issues with that approach
<wgrant> We could possibly use /var/tmp. But this isn't *that* bad.
<StevenK> I meant a IDaemonGPGHandler or so -- unless there is an easy way to overwrite methods
<StevenK> wgrant: It's a hacky kludge, and it makes me sad.
<wallyworld> StevenK: makes me sad too. but it seems to be the "accepted" approach according to the bug report comments
<wgrant> wallyworld: I'd rename GPGHandler job to something more obvious, but apart from that it looks OK. Still not sure how to test.
<lifeless> I don't think an integration test is worth doing
<lifeless> tac files are astonishingly hard to test and I have never understood why they are way they are
<lifeless> I always write regular functions
<lifeless> and then make my tac files
<lifeless> from foo import bar
<lifeless> bar()
<wallyworld> lifeless: ok. so doing it that way you just unit test bar separately
<lifeless> right
<lifeless> and take it on faith that the tac calls bar
<lifeless> but you keep it to those literal two lines
<lifeless> wallyworld: what was the architecture of the stuff @ caterpillar
<lifeless> wallyworld: I mean, did you have one big tree like we do, or a bunch of small trees; did you write micro services or have a single big DB
<wallyworld> lifeless: it was one big tree (project started over 10 years ago before EJB spec had even hit 1.0) but it's slow evolving to a more modular approach using stuff like OSGi
<wallyworld> lifeless: there is currently a single Oracle instance but separate schemas for BI reporting etc
<lifeless> wallyworld: I'm starting the socialisation process of this - https://dev.launchpad.net/ArchitectureGuide/Services - and given how recently you've joined the team, I think you'd make a good early-draft reader of it... if you have the time
<lifeless> wgrant: did you have any feedback ?
<wallyworld> lifeless: cool. will look.
<wgrant> lifeless: I have no real suggestions for improvement. It's not yet really at a stage where I can tell you you're wrong.
<wgrant> It looks very good so far.
<lifeless> wgrant: s/you/if/?
<wgrant> No.
<lifeless> oh, right.
<wgrant> +that, potentially.
<lifeless> do you think I'm wrong and waiting for evidence, or hope I'm not but didn't find any (due to lack of concrete data)
<wgrant> I believe you're right, and that page gives me no doubts.
<LPCIBot> Project windmill-db-devel build #256: STILL FAILING in 1 hr 17 min: https://lpci.wedontsleep.org/job/windmill-db-devel/256/
<lifeless> I think I need more pessimistic colleagues :)
<wallyworld> lifeless: *very* brief look before going to breakfast - the proposal also needs to mention domain modelling changes - the sort of stuff we've discussed previously wrt business objects vs persistent objects. also inter-service apis which exchange object references need to do so using external/business key references etc. in short, this aspect is a whole topic worthy of discussion in the lep
 * wallyworld off to breakfast
<lifeless> wallyworld: those are good things to talk about
<wgrant> But I don't think this is the place.
<wgrant>  /stage
<lifeless> wallyworld: on the latter I was assuming we'd use a locally unique key - its a well known and solved issue. On the former the template/api service would be sane, the current core would not change (but would lose all view code).
<lifeless> wallyworld: so I concur with wgrant; I think those things are truely orthogonal to the business case
<jtv> wgrant: why did you mark the bug for creating ds indexes from publish-ftpmaster as qa-untestable?
<wgrant> jtv: Because it was about 30 seconds before you told me you'd tested it, and I wanted to clear up the QA state.
<wgrant> Which is currently an unprecedented mess.
<jtv> I thought our current procedure for "safe to deploy even if not actually Q/A'ed" was to mark the bug as qa-ok.
<jtv> I'm asking because I could easily have missed a change.
<lifeless> either is fine
<lifeless> untestable is what --no-qa sets when you pass it to ec2 land
<wgrant> I've been told off for using qa-ok for OK-to-deploy before.
<lifeless> using qa-ok will add confusion if folk want to assess 'fixes bug' vs 'ok to deploy'
<lifeless> when I say either is fine, I mean I don't care :)
 * jtv wishes we had deploy-ok and so on
<lifeless> someone just needs to do it - patch up qatagger
<lifeless> wgrant: have you landed that backout?
<wgrant> lifeless: Yes.
<wgrant> Will be out of buildbot in a couple of hours.
<lifeless> cool
<wgrant> Thinking about it again I should have had buildbot killed.
<lifeless> I'm just looking for a good rev to test this rabbit branch
<wgrant> Since it landed soon after a new run had started.
<LPCIBot> Yippie, build fixed!
<LPCIBot> Project db-devel build #532: FIXED in 5 hr 52 min: https://lpci.wedontsleep.org/job/db-devel/532/
<lifeless> man the pypy bug tracker is -fail- ui
<lifeless> filing a new bug asks for a 'change note'
<StevenK> Sounds like two bugs, then
<lifeless> StevenK: well, they use roundup
<lifeless> https://codespeak.net/issue/pypy-dev/issue718?@ok_message=msg%202507%20created%3Cbr%3Eissue%20718%20created&@template=item
<wgrant> lifeless: ah, so you are trying lmirror on pypy?
<StevenK> Ewwwww
<wgrant> I was going to suggest that over rewriting it in Go.
<lifeless> wgrant: I wanted to evaluate it yes
<lifeless> wgrant: real    0m7.188s
<lifeless> user    0m6.760s
<lifeless> sys     0m0.390s
<lifeless> vs
<lifeless> real    0m9.227s
<lifeless> user    0m8.880s
<lifeless> sys     0m0.340s
<wgrant> Which is which?
<lifeless> to capture a delta of a mirror push on a 500K file tree whose shape was given by the ls -lr from archive.u.c
<lifeless> wgrant: the faster one is pypy
<wgrant> Right.
<wgrant> Less good than I would have expected.
<lifeless> wgrant: even with the extra 0.8 seconds from the parsing split
<wgrant> Oh.
<wgrant> Hmm.
<wgrant> you probably really want an itersplit, anyway.
<lifeless> yes
<lifeless> I've rewritten the parser to use a generator, which was ~0 saving in cpython
<lifeless> so I shelved it
<wgrant> But could be massive in PyPy..
<lifeless> yeah
<StevenK> lifeless: Your bug report title says string.plit, by the way\
<StevenK> s/\\//
<lifeless> heh
<jml> lifeless: awesome!
<lifeless> jml: I know! and now we have a user we can't change the api!? :(
<jml> lifeless: uhhh, I reckon we could work with him
<jml> lifeless: also, we could just make something better & make it part of fixtures
<lifeless> jml: yes, I was teasing about the API
<lifeless> I'm sure he'd adapt to a nicer API if we get it togethe
<lifeless> r
<lifeless> I got mail from him out of the blue
<lifeless> saying that this was nicer than the layer he had before
<jml> :)
<NCommander> What's the best way to create test packages for Soyuz? I need to implement a test suite which causes an arch-any/arch-all skew
<NCommander> s/suite/case/g
<wgrant> What are you trying to do?
<lifeless> wgrant: with ec2 demo, should it appear to hang on buildout ?
<lifeless> wgrant: or is it just -darn- slow?
<wgrant> lifeless: No, but it will take several minutes.
<wgrant> Unless you are asuka, then 30 minutes is common.
<NCommander> wgrant: cause publisher to retain older binaries during an arch any/arch all skew which has been the primary cause of image breakage on armel
<wgrant> NCommander: You should talk to the Soyuz developers about that first oh wait.
<NCommander> wgrant: well I want the test case so I can show the problem before any solution is implemented
<lifeless> wgrant: what should it do ?
<lifeless> wgrant: end up in a shell? return to the prompt and wait for manual shutdown?
<wgrant> NCommander: You don't want to use real packages for that. Use SoyuzTestPublisher.
<wgrant> lifeless: It will tell you that the webapp is running now and then give you a Python REPL.
<lifeless> wgrant: apparently not
<wgrant> Lies.
<NCommander> wgrant: I disagree, since the test validation should be if APT can still install packages from LP
<lifeless> utilities/shhh.py PYTHONPATH= ./bin/buildout \
<wgrant> NCommander: That's a very high-level complex integration test at a scale we do not currently have.
<lifeless>                 configuration:instance_name=development -c buildout.cfg
<lifeless> was the last I have seen from it
<wgrant> lifeless: Is buildout eating CPU?
<NCommander> wgrant: *grumble* :-/.
<lifeless> NCommander: doing that is fine to demonstrate the problem
<wgrant> NCommander: Can you imagine how slow the test suite would be if we did that for everything?
<lifeless> NCommander: but its at a huge remove from the source of the issue
<wgrant> But we are well aware of that problem.
<wgrant> It does not require demonstration.
<lifeless> NCommander: and there is a bug for this case already :P
<NCommander> lifeless: was not aware of this, I was simply poking LP with a big stick to try and work towards this
<lifeless> NCommander: cool
<NCommander> lifeless: My initial idea was to simply implement the publishing algrothmin used by dak to retain old binaries until the arch any/all skew is resolved in unstable
<NCommander> (and I poked cjwatson on it who pointed me to the original Debian spec on this subject)
<wgrant> For us it's far easier than that.
<NCommander> ?
<wgrant> To fix it is basically a matter of deleting code.
<wgrant> Since we explicitly supersede them atomically now.
<lifeless> we do this deliberately you see :>
<NCommander> lifeless: it causes a lot of issue with ARM ports.
<lifeless> \o/ bug search timeouts
<NCommander> (and to a lesser extent PPC)
<lifeless>  OOPS-1956EB151
<rvba_> stub: Hi and thanks for the review!
<wgrant> rvba_: Hi. Did you get anywhere with your traversal?
<NCommander> lifeless: wgrant: where's the existing LP bug?
<lifeless> wgrant: is it bug 402935 ?
<_mup_> Bug #402935: Domination of architecture independent binaries is not restricted to the source publication boundaries <boobytrap> <lp-soyuz> <ppa> <Launchpad itself:Triaged> < https://launchpad.net/bugs/402935 >
<wgrant> lifeless: No, but that would be fixed by this.
<wgrant> lifeless: Because it's a bug in the code that would be deleted.
<rvba_> wgrant: I'll start with that just now.
<wgrant> rvba_: grep for stepstogo.consume
<wgrant> rvba_: No need for an intermediate object.
<rvba_> wgrant: ah ... I'll take a look at  stepstogo.consume, thanks!
<lifeless> wgrant: do you know the bug fo rthis?
<wgrant> Bug #34086
<_mup_> Bug #34086: morguing arch-all packages can result in uninstallable binaries <feature> <lp-soyuz> <soyuz-publish> <Launchpad itself:Triaged> < https://launchpad.net/bugs/34086 >
<wgrant> Mmm, I guess it does have to be aware of dependencies, so it is a bit of work.
 * NCommander notes that dak handles this now */comment*
<lifeless> NCommander: thats nice dear :)
<wgrant> Yes, but dak doesn't suck.
<wgrant> And is improving.
<wgrant> So it is doubly much better than LP.
 * NCommander helped rewrite much of dak long ago :-/
<lifeless> wgrant: hey, LP is improving
<lifeless> NCommander: we'd love a patch for this for lp
<NCommander> lifeless: I'd be glad to do the foot work, but I'm not a LP guru
<StevenK> 'leg work' :-P
<wgrant> lifeless: Yes, but glacially.
<NCommander> this is the dak spec for how Debian fixed it http://ftp-master.debian.org/wiki/projects/arch-all-tracking/
<NCommander> I think its probably pretty sane if we simply implement in a similar fashion
<wgrant> "We remove all arch all packages that are not referenced by existing source packages but to be safe we always keep the newest version similar to the source removal case. The process is called 'old_and_unreferenced', too."
<wgrant> That's not an entirely useful description of how referencedness is calculated.
<lifeless> jml: the ec2 postmortem console is interesting
<lifeless> jml: but it lies:
<NCommander> wgrant: a lot about dak is simply looking at the code :-P
<lifeless> EC2Instance is available as `instance`.
<NCommander> er, is only explainable by looking at the code
<lifeless> >>> instance
<lifeless> Traceback (most recent call last):
<lifeless>   File "<console>", line 1, in <module>
<lifeless> NameError: name 'instance' is not defined
<jml> lifeless: the postmortem console isn't my feature (although I probably broke it).
<lifeless> jml: and here I was giving you undue credit :)
<jml> lifeless: :)
<NCommander> wgrant: I'm guessing the way to start is drafting a LP spec which describes how dak does it and how we want to implement their braindamage^W process
<adeuring> good morning
<lifeless> NCommander: uhm, probably not.
<lifeless> NCommander: a LEP /may/ be useful, but probably isn't.
<lifeless> NCommander: I'd refine the bug until its implementable, make sure you have input from experienced folk like Julian, and then start doing it.
 * NCommander is beginning to under Soyuz's design ...
<NCommander> *understand
<wgrant> I think we probably want to undo ArchiveRemovalRedesign :(
<NCommander> wgrant: what is ArchiveRemovalRedesign?
 * NCommander is writting a comment on the bug
<wgrant> NCommander: 2006 stuff.
<lifeless> I've bumped the bug
<wgrant> Which is mildly relevant here.
<wgrant> I am pondering how this will work.
<NCommander> lifeless: I'm also putting some comments on why this is important now
<wgrant> We need to only drop things from the indices when all binaries from a source on a particular arch are superseded.
<wgrant> Which probably means we need an intermediate status.
<lifeless> wgrant: superseded-or-not-built-anymore
<lifeless> wgrant: the dependency check is more accurate
<wgrant> Which dependency check?
<NCommander> Wouldn't the solution simply be keeping the old binaries published until superseeded?
<StevenK> We do
<NCommander> (i.e., change when the superseeded status is applied to a binary record)
<wgrant> NCommander: They are superseded already.
<lifeless> wgrant: modelling this as 'no version locked rdeps exist'
<lifeless> NCommander: https://blueprints.launchpad.net/launchpad/+spec/archive-removal-redesign
<wgrant> lifeless: That's taking it to another level entirely.
<NCommander> lifeless: that's like making testing exist in Soyuz
<StevenK> NCommander: You can make that joke only after fixing the testsuite
<NCommander> StevenK: Debian testing
<NCommander> not testing in general
<StevenK> Testing does exist?
<StevenK> Does it not?
<lifeless> zomg
<lifeless> celso wtf
<NCommander> ... its like implementing britney's functionality in Soyuz
<NCommander> Grumble
<NCommander> lifeless: ?
<wgrant> lifeless: Where?
<lifeless> https://launchpad.canonical.com/SoyuzArchiveRemovalRedesign/ - see the explain analyze and assertions about fineness
<StevenK> NCommander: We also run britney
<wgrant> lifeless: Oooooh I can see the spec now.
<NCommander> StevenK: yes, but we don't split out a specialize archive of packages that are only installable
 * NCommander can't see that wiki page ...
<StevenK> NCommander: Testing doesn't mean the package is installable, just that it built
<StevenK> And it isn't a seperate archive, it's a seperate *series*
<NCommander> StevenK: britney doesn't allow a package to migrate from unstable->testing unless its installable on all release-candidate architectures
<lifeless> NCommander: click on 'login'
<lifeless> NCommander: it should let you see it
<NCommander> lifeless: yeah, figured that bit out
<wgrant> lifeless: What about that assertion is particularly faulty?
<wallyworld> wgrant: review? :-) https://code.launchpad.net/~wallyworld/launchpad/poppy-sftp-gpgconf/+merge/60454
<lifeless> wgrant: seqscan == seqscan
<lifeless> wgrant: both are horribly inefficient
<lifeless> wgrant: as an assessment for its utility on production data (even of the time) its utterly useless
<wgrant> lifeless: I didn't even consider the possibility that it was a seq scan, because that would just be stupid.
<lifeless> wgrant: Aggregate  (cost=5909.43..5909.44 rows=1 width=0) (actual time=543.342..543.346 rows=1 loops=1)
<lifeless>    ->  Seq Scan on securesourcepackagepublishinghistory  (cost=0.00..5754.15 rows=62112
<wgrant> So I just looked at the filter and time.
<wgrant> Impressive.
<bigjools> securesourcepackagepublishinghistory?
<lifeless> bigjools: I'm lamenting a previous performance analysis which I happened to run into fallout from just last week
<lifeless> as in 'things are slow because of this'
 * NCommander is getting in the feeling he's stepped into something old and crufty
<wgrant> NCommander: What, you mean Launchpad? :)
<NCommander> wgrant: heh
<stub> Is it possible to reorder a bzr pipeline?
<mrevell> Hello all
<lifeless> wgrant: jml: ec2 demo seems reproducibly broken for me
<wgrant> lifeless: Probably.
<wgrant> It's not used very much.
<wgrant> at all.
<jtv> allenap, got a mo' to chat about PackageCopyJob?
<allenap> jtv: Sure.
<jtv> Thanks.  This is about the "only sync up to 100 packages" work.
<allenap> jtv: Mumble or MS Skype?
<jtv> If you put it like that, then mumble please.  :)
<bigjools> MS Skype, lol
<lifeless> bigjools: you've seen the news, right?
<bigjools> yeah
<bigjools> trying to work out what I can use instead
<wgrant> Yeah, this could be interesting.
<lifeless> its a damn shame
<lifeless> their echo detection is awesome
<bigjools> yes, it is
<bigjools> the biggest problem for me is finding a skypeout replacement
<lifeless> dial 9 for an outside line?
<bigjools> jml!
<bigjools> lifeless: if I can use that for personal calls as well, sure :)
 * bigjools considers remaining critical bugs as just high on June 26th
<StevenK> bigjools: Host asterisk on a VPS?
<lifeless> bigjools: speaking of criticals
<bigjools> so that jml gets a pie
<lifeless> bigjools: I want to nuke that soyuz specific oops report
<allenap> I have three Netgear land-line phones with Skype built-in. MS have pwned my base.
<bigjools> lifeless: is there a way of including it in the prod report?
<lifeless> bigjools: you've had a month or so of low oops counts in the main report to see if that wil lcause you grief or not
<StevenK> allenap: :-(
<lifeless> bigjools: the oopses are there
<bigjools> lifeless: that's not the issue
<bigjools> lifeless: the issue is that occasionally there's a ZOMG problem that is only apparent from that report
<lifeless> bigjools: that applies to all services though
<lifeless> bigjools: a spike of oopses is a problem
<bigjools> lifeless: yes, but the single oops that we need to see is not visible enough buried in the other report
<lifeless> bigjools: if you still need it, we'll leave it, but I fail to understand what makes it special
<bigjools> visibility
<bigjools> if you can guarantee that we'll see that single oops in the main report, I am very happy to lose one more email from my inbox
<lifeless> we'd get -one- oops and it would be a zomger?
<bigjools> the report is based around volume of oopses as opposed to their criticality
<bigjools> yes
<lifeless> how does that work?
 * bigjools racks brain
<lifeless> also, relatedly -  285 OperationalError: FATAL: Ident authentication failed for user "sync_packages" FATAL: Ident authentication failed for user "sync_packages"
<lifeless> -really- needs fixing.
<bigjools> allenap: ^
<lifeless> like really really.
<lifeless> it needs a config dir setup, the cronscript set to use that dir, with a new OOPS prefix, and the db user created.
<lifeless> about 15 minutes work.
<allenap> lifeless, bigjools: Rats, I meant to fix that last week. Sorry. I'll do that today.
<lifeless> allenap: thank you
<lifeless> bigjools: thanks!
<bigjools> cheers allenap
<bigjools> lifeless: I can't remember the specific instance now, sorry :(  But there have been times in the past where a single or few oopses would have been buried in a sea of timeouts in the main report.
<lifeless> bigjools: kk, thanks for trying :)
<bigjools> lifeless: what I'd like to see is all those routine oopses removed in the current reports, they should not be oopses, and then we could consider putting soyuz oopses at the top of the prod report.
<bigjools> lifeless: the publisher has the odd problem where corrupt data is caught by an assertion once an hour
<bigjools> lifeless: but believe me, fewer emails is good, I am all for that :)
<lifeless> so lets leave the status quo for now
<NCommander> wgrant: lifeless: so where do we go from here on re:arch-all/any?
<lifeless> NCommander: code
<lifeless> NCommander: and talk to julian
<NCommander> lifeless: I guess I need to start with writing a test case ...
<wgrant> check_permission in model code is still verboten, right?
<wgrant> rvba_, bigjools: checkCopy calls check_permission... this is probably not legal.
<wgrant> In fact, yeah, that's buggy.
<bigjools> I can't remember why it's verboten
<wgrant> It always checks the primary archiev.
<wgrant> Fourth rollback in 24 hours woo.
<bigjools> what are you looking at?
<bigjools> rvba_ is no around this morning BTW
<wgrant> bigjools: CopyChecker.checkCopy
<wgrant> if reason is not None
<wgrant> It checks launchpad.Append on the primary archive.
<wgrant> Bad for at least two reasons:
<bigjools> :/
<wgrant>  1) check_permission in model is bad.
<wgrant>  2) main_archive is mostly not the right thing.
<wgrant> bigjools: What do you think?
<bigjools> wgrant: main_archive is clearly wrong
<bigjools> when we have self.archive
<LPCIBot> Project windmill-db-devel build #257: STILL FAILING in 1 hr 7 min: https://lpci.wedontsleep.org/job/windmill-db-devel/257/
<wgrant> bigjools: And I think using check_permission is a bad idea, given that the method takes a Person already.
<bigjools> wgrant: agreed
<wgrant> bigjools: I really don't want confusing security in this sort of code :)
<bigjools> wgrant: agreed
<wgrant> But rather than roll it back for a second time, I shall fix it.
<wgrant> Or at least try to quickly.
<wgrant> Should be a matter of deleting that block and adding check_permissions=False to the do_copy call in syncSource.
<bigjools> wgrant: thanks
<bigjools> wgrant: assuming the security adapter already DTRT
<wgrant> bigjools: The security on syncSource hasn't changed.
<bigjools> wgrant: where was the lp.append for security enforced, I can't remember
<wgrant> bigjools: It's enforced by the Archive securityproxy.
<wgrant> Archive.syncSource(s) is hidden behind launchpad.Append
<bigjools> right
<wgrant> They're on IArchiveAppend.
<wgrant> Still.
<bigjools> wgrant: goes to show, we need experienced soyuz people carefully checking soyuz branches :/
<wgrant> bigjools: Surely not!
<wgrant> There's a reason I'm not unhappy that most deploys happen during my day where I can watch :)
<bigjools> heh
<bigjools> wgrant: IIRC this was added because you said to rvb that security would lose lp.append without it
<wgrant> bigjools: Yeah. But I intended that syncSource would call do_copy with check_permissions=False, to limit the security hack as far as possible.
<wgrant> Not that check_permissions=True would handle the hack itself.
<bigjools> ok
<wgrant> (check_permissions=False turns off all the new code, so we just get the old behaviour)
<poolie> hi all
<bigjools> wfm
<wgrant> Great, thanks.
<bigjools> wgrant: we only want the new behaviour in the job
<bigjools> hi poolie
<bigjools> nice blue hair :)
<lifeless> bigjools: so I think this is 'new to lp coding' not 'new to soyuz' - I've seen other similar thinkos
<bigjools> lifeless: the reviewer missed it
<lifeless> bigjools: yes, yes they did
<bigjools> wgrant: we need to talk about build dependencies for multiple parents
<bigjools> and our current schema
<wgrant> bigjools: So we do.
<wgrant> Do you want to mumble tonight?
<bigjools> I was hoping to get jtv and StevenK involved too
<bigjools> but yes
<adeuring> stub: still around?
<stub> adeuring: yes
<adeuring> stub: I'm trying to figure out how to avoid the timeouts from bug 739075. AS I understand it, most time is spent in joining QuestionMessage and Message in order to find messages from given person
<_mup_> Bug #739075: Person:+questions timeouts <timeout> <Launchpad itself:In Progress by adeuring> < https://launchpad.net/bugs/739075 >
<adeuring> stub: I'm wondering if we should add a column questionmessage.owner, simmilar to bugmessage owner
<wgrant> adeuring: We fixed that for bugs recently by denormalising Message.owner onto BugMessage.owner
<wgrant> That.
<stub> adeuring: The way it was solved for BugMessage was to add an owner column
<adeuring> right
<wgrant> "recently" meaning "last week"
<stub> adeuring: I'd work from that template. The garbo.py job is still there for you to cargo cult.
<adeuring> ok, so I'll try that too for questions
<stub> Although for questions, we can probably just do all the data migration in the db patch since it is much smaller.
<stub> That will save the overhead of multiple db patches etc.
<adeuring> ok, sound more conveniet :)
<bigjools> wgrant: can you remember the exact reason for not using check_permission in model code?
<wgrant> In this particular case I would forbid it anyway, since a user is taken explicitly. But check_permission is pretty terrible in model code in general because it's executed from scripts and other situations without a user context, so it's fairly meaningless.
<bigjools> ah that was it
* gmb changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: gmb | https://code.launchpad.net/launchpad-project/+activereviews
<bigjools> I am moving an exported method from IDistroSeriesPublic to IDistroSeriesEditRestricted.  Unfortunately the copy_field directives in the @operation_paramters now refer to fields in the other class.  Is there a way to make it work or do I need to repeat the actual field declaration again?
<wgrant> bigjools: You can't just refer to the other class?
<wgrant> Or are they in the wrong order?
<bigjools> wgrant: wrong order, but that's easily fixed
<wgrant> Otherwise use one of the usual patchers.
<bigjools> nope, doesn't work
<wgrant> !doesn't work
<wgrant> Bah, no ubottu
 * bigjools scratches head
<wgrant> 21:01:09 <ubottu> Doesn't work is a strong statement. Does it sit on the couch all day? Does it want more money? Is it on IRC all the time? Please be  specific! Examples of what doesn't work tend to help too.
 * bigjools slaps wgrant with a large haddock
<bigjools> I'm doing something so obviously wrong I can't see the wood for the trees.
<jpds> Hmm, work from couch.
<bigjools> I've got name=copy_field(IDistroSeriesPublic.name ....
<spiv> bigjools: slapping people with Tintin characters is a bit extreme!
<bigjools> which errors out with AttributeError: 'InterfaceClass' object has no attribute 'name'
<wgrant> bigjools: Oh.
<wgrant> IDistroSeriesPublic['name']?
<bigjools> spiv: Python reference!
<bigjools> wgrant: EUARGH
<spiv> bigjools: a weakref :P
<bigjools> spiv: not counted properly
<wgrant> bigjools: An interface is not a class!
<bigjools> yes it is
<bigjools> it says "class" :)
<henninge> gmb: Here is a nice little branch for you, if you'd be so nice to take it, please. ;-)
<wgrant> That's just because Python sucks.
<henninge> https://code.launchpad.net/~henninge/launchpad/bug-196679-language-packs-value-error/+merge/60487
<bigjools> sometimes, yes it does
<bigjools> and coming from a C++ background it confuses the shit out of me sometimes
<gmb> henninge: Ah, the perfect size to review just before lunch.
<henninge> gmb: great, thanks
<spiv> bigjools: also, wikipedia, the source of all true and indisputible knowledge, says the fish-slapping dance involved pilchards and a halibut.
<bigjools> I bow to wikipadeoia
<gmb> henninge: r=me.
 * gmb -> Vittles
<wgrant> If there is something bad in r13009 or r13011, I am going to be very sad.
<spiv> wgrant: Python's class statement doesn't suck, it's great.  It allows stuff like class Ten(1,2,3,4): __metaclass__ = type('', (type,), {'__new__': lambda *a: sum(a[2])})
<spiv> How could that be a bad thing? ;)
<wgrant> spiv: Did you pull that off the quotes page?
<wgrant> I'm sure I've seen it there.
<spiv> No, but it's probably due to me that a version of that is already there.
 * bigjools vomits
<wgrant> Indeed, https://wiki.canonical.com/Quotes/2005
<wgrant> "Evil Python"
<spiv> I think last time I made that remark I didn't condense it into a one-liner though!
<henninge> gmb: thanks
<LPCIBot> Project db-devel build #533: FAILURE in 5 hr 2 min: https://lpci.wedontsleep.org/job/db-devel/533/
<wgrant> bigjools: I can talk DSPs for a while now if you still want, and the others are around.
<bigjools> wgrant: sure
<bigjools> -> mumble
<bigjools> jtv, StevenK, are you free to mumble with wgrant and me for a little while?
<jtv> coming
<jtv> oh no not this problem again
<bigjools> 'fraid so
<danilos> gmb, hey-hey, I've got a review for you when you find some time :) https://code.launchpad.net/~danilo/launchpad/bug-771204/+merge/60493
<gmb> danilos: I'll take a look at that now.
<danilos> gmb, thanks
<gmb> danilos: Looks good. r=me
<danilos> gmb, ta (sorry, lunch :)
<danilos> hum, is it just me or has all the ajax stuff stopped working on the MP page?
 * danilos tries in firefox
<wgrant> Worked for me a few hours ago.
<danilos> works in firefox and chromium, a problem with bare webkit JS engine?
<StevenK> bigjools: Sorry, I was dinner-ing.
<bigjools> StevenK: np, we're just finishing :)
<danilos> hum, false alarm: launchpad.js was misloaded, some garbage at the beginning â I suppose some of my hardware is acting up
<StevenK> bigjools: Anything earth-shattering?
<bigjools> haha, who did the tweet about jml's pie being meaty?
<bigjools> StevenK: I'kk summarise in an  email
<bigjools> I'll, even
<StevenK> bigjools: Will I'kk help?
 * henninge reloacates
 * bigjools -> food
<deryck> Morning, all.
<rvba_> wgrant: (I just got back ...) thanks for taking care of the fix ;).
<LPCIBot> Project windmill-devel build #59: STILL FAILING in 1 hr 7 min: https://lpci.wedontsleep.org/job/windmill-devel/59/
<wgrant> rvba_: It all looks good. Sorry, I should have been more clear when I rolled it back the first time.
<jtv> allenap: do you know what would be a simple way for a test to verify whether a package had been copied?
<rvba_> wgrant: and I think I should have been more careful before modifying the callsites. Thanks anyway.
<allenap> jtv: I think I know the answer to that...
<allenap> jtv: See PackageCopyJobTests.test_smoke.
<jtv> ah thanks!
<allenap> jtv: The last 3 lines actually. It's quite simple.
<jtv> Archive.getPublishedSources?
<wgrant> I tend to check that getPublishedSources(name=whatever) returns nothing before, and one row after.
<henninge> Do either staging or qastaing process incoming mail? i.e. MP discussions
<mwhudson> henninge: they both do i think?
<mwhudson> process-mail.py may not be run very often though
<henninge> ok, I'll try it out once qastaging stops timing out on me ...
<henninge> oh, it's not a timeout
<wgrant> MPs don't work.
<wgrant> Because they need restricted librarian access.
<wgrant> You might want to create a new MP.
<wgrant> MPs synced from production don't work, that is.
<henninge> wgrant: I just got a LookupError for an LFA AFAICT. Is that what you mean?
<wgrant> henninge: yes.
<henninge> aha
<wgrant> They try to look up the LFA in the staging librarian. That would normally forward to the production librarian, but that can't happen for restricted files.
<henninge> how do I simulate mail processing locally?
<wgrant> lib/canonical/launchpad/doc/mailbox.txt
<wgrant> But I'd try creating a fresh MP on qastaging first.
<henninge> wgrant: cheers
<wgrant> It won't have a diff.
<wgrant> but it will receive emails OK.
<henninge> oh, ok
<henninge> thanks
<henninge> code update in progress ...
<wgrant> henninge: Nice catch.
<wgrant> henninge: I hadn't noticed that the . had wrapped in both cases.
<henninge> ;-)
<wgrant> It may still be a problem in the MTA stack in front of LP, but now we'll be able to find out :)
<henninge> righit, that's what I figure.
<henninge> wgrant: but I think that MTAs and MUAs watch for things like that and escape single dots.
<henninge> just like "From"
<wgrant> henninge: Yes, but our MTAs do all sorts of strange crap.
<henninge> :(
<henninge> ok, I'll see if I find something
<wgrant> To add extra headers like X-Launchpad-Original-To
<wgrant> Night all.
<henninge> wgrant: night
<mwhudson> oh ****, is that why mails to mps sometimes get truncated?  because they contain '\n.\n.?
<henninge> mwhudson: yup
<mwhudson> henninge: wow
<henninge> mwhudson: although it looks like we create these by wrapping lines.
<bigjools> ha, that's the SMTP end of data line
<mwhudson> eeeeeeeeeeeeeeeeeeeeeeeeeee
<mwhudson> that's just so messed up
<mwhudson> so how does the wrapping happen?  presumably it must be before lp gets to see it, if smtp-level corruption is what's causing the problem
<henninge> I don't know yet
<mwhudson> good luck finding out :)
<LPCIBot> Project windmill-db-devel build #258: STILL FAILING in 1 hr 6 min: https://lpci.wedontsleep.org/job/windmill-db-devel/258/
<bigjools> gmb: afternoon m'lad, I have a small branch for your perusal if you're still reviewing?
<gmb> bigjools: Sure. Diff me.
<bigjools> gmb: https://code.launchpad.net/~julian-edwards/launchpad/derive-permissions-bug-643369/+merge/60515
<bigjools> cheers
 * gmb looks
<jml> wallyworld_: https://bugs.launchpad.net/launchpad/+bug/184737 is relevant to the thing you just said
<_mup_> Bug #184737: No tag autocomplete in +filebug extra options <bugtag> <filebug> <lp-bugs> <Launchpad itself:Triaged> < https://launchpad.net/bugs/184737 >
<jml> wallyworld_: about tag completion on bug subscription forms.
<wallyworld_> jml: wow. that's an old bug :-)
<jml> wallyworld_: yeppers.
<jml> wallyworld_: classic case of a bug that's not too hard to fix, very useful, but no one getting around to it
<bigjools> gmb: I wish diff generators could be more useful sometimes, the one on that MP looks like a dog's breakfast
<wallyworld_> jml: i hear you. hopefully *this* particular one will now be fixed since it's tied to a feature sprint :-)
<bigjools> hey wallyworld_ are you at UDS?
<gmb> bigjools: It's okay, I'm used to it these days.
<gmb> But yer not wrong
<bigjools> gmb: it needs a "moved but not changed" marker
<gmb> +1
<wallyworld_> bigjools: yep :-)
<bigjools> wallyworld_: will see you Thursday arvo then
<wallyworld_> bigjools: sadly no. i'm leaving tomorrow :-(
<bigjools> wallyworld_: ah bugger
<wallyworld_> yep. bollocks all right
<wallyworld_> bigjools: brad c is doing a session on the new bug subscription stuff. looks good
 * gmb is listening in.
<wallyworld_> hi gmb
<gmb> Hey wallyworld_. Glad you like :)
<wallyworld_> gmb: to quote borat - "niiiiiiice"
<gmb> Indeed. You should ask sabdfl to do his Borat.
<wallyworld_> gmb: so long as there's no mankini involved :-)
<bigjools> I have the vid on my laptop....
 * gmb goes to bleach his minds eye
<wallyworld_> lol :-)
 * StevenK has it here somewhere too
<gmb> I'm guessing jml is sat near the mic, based on the typing and the deep sound of his voice.
<jml> gmb: heh heh
<jml> what's the bug for seeing all subscriptions across LP?
<wallyworld_> don't know
<bigjools> wallyworld_: which room?
<gmb> jml: Looking now for you
<wallyworld_> bigjools: krudy
<wallyworld_> room 6
<bigjools> ta
<jml> gmb: thanks.
<jml> also in topic of #launchpad
<gmb> jml: https://bugs.launchpad.net/launchpad/+bug/110953 seems to be the one being referenced in the most recent thread about this.
<_mup_> Bug #110953: Can't easily see everything I'm subscribed to <infrastructure> <lp-registry> <story-better-bug-notification> <ui> <Launchpad itself:Triaged> < https://launchpad.net/bugs/110953 >
<jml> gmb: thanks.
<bigjools> is everyone sat nowhere near the mics as usual? :)
<jml> bigjools: bac, mordred & I
<jml> bigjools: everyone else is in the back row.
<bigjools> jml: figures
<jml> bigjools: including flacoste
<bigjools> !
<wallyworld_> bigjools: so we can heckle easier :-)
<gmb> If we didn't draw the line somewhere on this feature I was going to have to self-trepanate in order to make 8 months' worth of brain juice drain away.
<StevenK> No definitions found for "trepanate"
<gmb> StevenK: Try trepanation or Trepanning
<StevenK> Oh, using a trepan on yourself
<StevenK> Sounds gruesome. And hard to get right.
<jelmer> is gmb using difficult words again ?
<gmb> Be thankful I aten't prattling in Lanky.
<jelmer> garr. now I really have to spend some quality time with my dictionary.
<StevenK> jelmer: s/\(is gmb\) \(using difficult words\)/\1 still \2/
<StevenK> Oh bah, that won't drop the again. Oh well
<bigjools> spectacular backfire
<gmb> bdmurray: No, that doesn't work right now, but there probably should be some way to do it - we just haven't figured it out yet. I don't think there's a bug for it though.
<StevenK> It's nearly 1am, and I'm trying to fix Jenkins
<StevenK> Right, new build slave finally started
<wallyworld_> jml: you still have etherpad open?
<jml> wallyworld_: yes.
<wallyworld_> jmL: 780568
<jml> wallyworld_: thanks
<NCommander> Is there a way to determine why a PPA upload is dropping into the void?
<bigjools> NCommander: https://answers.edge.launchpad.net/launchpad/+faq/227
<adeuring> cody-somerville: could you please try to access https://qastaging.launchpad.net/%7Ereprepro/+archivesubscriptions ? the timeout should be gone there -- but I can't test it myself -- permission denied
<cody-somerville> adeuring, qastaging doesn't have enough subscriptions to cause it
<adeuring> cody-somerville: really? It should use the same DB as the production server
<cody-somerville> adeuring, really
<cody-somerville> :(
 * cody-somerville is in a session, can chat more about this later.
<LPCIBot> Project db-devel build #534: STILL FAILING in 1 hr 6 min: https://lpci.wedontsleep.org/job/db-devel/534/
<abentley> deryck: Am I high on context managers? http://pastebin.ubuntu.com/605743/
<benji> gmb: I have a small branch ready for review (no particular rush): https://code.launchpad.net/~benji/launchpad/bug-778689/+merge/60520
<deryck> abentley, looking....
<gmb> benji: I'll take a look shortly
<deryck> abentley, high as in too many?  Or high as in floating wild and free in the joy that is context managers? :-)
<deryck> abentley, I like the code, though.  nice approach to the problem.
<abentley> deryck: Well, really, is there a better way to do this?
<deryck> abentley, I can't think of one.  I think this is spot on.
<abentley> deryck: Okay, cool.
<gmb> bigjools: Your branch is r=me. Has been for ages but I got distracted by the UDS session.
<bigjools> gmb: ok ta :)
<gmb> benji: Your test needs a comment stating what it's testing for so that lazy gits like me don't have to go and look up the bug report.
<gmb> Other than that, r=me.
<benji> gmb: heh, will do
<sinzui> jcsackett: mumble?
* gmb changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: - | https://code.launchpad.net/launchpad-project/+activereviews
<gmb> And that's quite enough of that for today.
<NCommander> are there known issues with LP on natty? postgresql is puking when I try to make schema
<jcsackett> sinzui: when you're available, https://code.launchpad.net/~jcsackett/launchpad/spam-button-ui-bugs/+merge/60437
<jcsackett> i assume the one you need is https://code.launchpad.net/~sinzui/launchpad/person-merge-oopses-0/+merge/60521 ?
<sinzui> jcsackett: I will start it now, and yes
<jcsackett> cool. i will start yours momentarily.
<sinzui> I may be disappointed when developers stop cargo culting my "// Lock, stock, and two smoking barrels." comment
<jcsackett> :-)
<jcsackett> it's a fantastic way to wrap up a test.
<sinzui> It is one of my favourite films
<jcsackett> it is pretty fantastic.
<jcsackett> i clearly do not actually understand IStore/storm stores. I would have thought that creating a store of IPerson would only let you get person objects.
<jcsackett> sinzui: r=me. as i mention in the comment, there may be some value to moving the reload utilities to something accessible for other tests as it could be handy, but the argument of YAGNI may very well apply.
<sinzui> jcsackett: I will move it into lp.testing
<jcsackett> sinzui: cool. :-)
<mrevell> Night!
<jcsackett> sinzui: thanks for the notes, i'll make those changes.
<jcsackett> though i may do the test changes as part of my tech-debt branch.
<LPCIBot> Project windmill-devel build #60: STILL FAILING in 1 hr 7 min: https://lpci.wedontsleep.org/job/windmill-devel/60/
 * deryck looks around the room for yellow squad....
<deryck> benji or gmb maybe?  I have fixed a critical js bug related to the subscriptions story.
<deryck> maybe one of you would want to review it?
<benji> deryck: sure
<deryck> benji, thanks!  https://code.launchpad.net/~deryck/launchpad/dupe-unsub-no-remove-name-775335/+merge/60538
<benji> deryck: looks good
<deryck> benji, thanks!
<benji> np
<LPCIBot> Project windmill-db-devel build #259: STILL FAILING in 1 hr 6 min: https://lpci.wedontsleep.org/job/windmill-db-devel/259/
<deryck> mbarnett, hey.  any time to look at that disappearing merge yet?
<abentley> deryck: To determine the correct timeout for branch upgrades, I decided to look at the logs.
<deryck> abentley, ah, cool.  what did you end up with then?
<abentley> deryck: These are the durations I was able to find: http://pastebin.ubuntu.com/605825/
 * deryck looks
<abentley> deryck: It doesn't feel like there's really enough data to have a good guess.
<deryck> abentley, yeah.  based on that, most everything finishes in under 10 minutes, right?
<abentley> deryck: No, many things are +30
<deryck> abentley, is it:  HH:MM:SS ?
<deryck> the log format?
<abentley> Yes, it's HH:MM:SS.
<abentley> I think the ~0 second durations don't count.  They're prolly cases where the upgrade was already performed.
<deryck> abentley, so I see 36 lines, with 8 items > 10 minutes.  so roughly 20%.
<deryck> ah
<deryck> so somewhere around 8/18.  closer to 50% over 10 minutes.
<deryck> abentley, yeah, that's a hard call what to set then.
<abentley> deryck: On the plus side, we can't be wasting a lot of resources if upgrades are this rare.
<deryck> true
<deryck> abentley, maybe we don't need the timeout then?  Or else a high number, i.e. 1 or 2 hour cap?
<abentley> Yes, I think we need a timeout for outliers, but 1 hour seems reasonable.
<deryck> abentley, cool. agreed.
<abentley> deryck: thanks.
<deryck> np!
<sinzui> jcsackett:  can you put some thoughts about the effort needed to hide merge comments in the UI: https://bugs.launchpad.net/launchpad/+bug/697495
<_mup_> Bug #697495: need SQL to hide/edit comments from merge proposals <canonical-losa-lp> <code-review> <comments> <Launchpad itself:Triaged> < https://launchpad.net/bugs/697495 >
<sinzui> jcsackett: I can see that ICodeReviewComment is exported, we may only need to add the setVisibility() method and extend the js to work with the page
<LPCIBot> Project windmill-db-devel build #260: STILL FAILING in 43 min: https://lpci.wedontsleep.org/job/windmill-db-devel/260/
<jkakar> FYI, a bug that appeared randomly a few days ago is preventing us (Fluidinfo) from seeing milestone views. :(
<jkakar> A sample OOPS: OOPS-1956CO39
<jkakar> Should I file a bug about this or is the presence of the OOPS enough that someone will get around to looking at it?
<jcsackett> sinzui: i believe you are correct. since the interface is already exported, we should only need to copycat the setCommentVisibility + js work.
<jcsackett> i have just discovered there is a comment.js thing as well; i may explore putting the hide-comment code in there and then we can extend to each comment type.
<sinzui> jcsackett: I am tempted to commit to this if we could land it by Monday
<jcsackett> sinzui: i think monday is possible.
<jcsackett> did you just come across this bug, or has it come up as a problem recently?
<sinzui> It is a bug linked to a question we cannot resolve
<jcsackett> ah.
<jcsackett> so, let me finish my tech-debt branch, and i'll commit to 4 hours to explore MP comments?
<deryck> jkakar, hi.  I would recommend filing the bug to be safe.
<jcsackett> if i don't have a good feeling of scope at the end of that (which will probably be tomorrow) we won't commit to doing the work?
<jcsackett> sinzui ^
<jkakar> deryck: Awesome, I'll do that, thanks!
<sinzui> jcsackett: I want to be sure that we are not getting bug reports when we switch to features. I image we want to be able to land a branch to fix any issued after Monday. I
<jcsackett> sinzui: next week is theoretically our last week before switching, or week after next?
<sinzui> We are in our last two weeks of maintenance.
<jcsackett> ok.
<lifeless> benji: hi
<benji> hi
<lifeless> https://bugs.launchpad.net/launchpad/+780703
<lifeless> benji: I *think* thats a better-notification-regression
<lifeless> benji: its definitely a regression, the regressor is what I'm unsure of
<lifeless> benji: (they are a paying customer FWIW, but I suspect its much more widespread)
<benji> it's tangentially related to the better-notification story; I'm working on it as we speak
<lifeless> oh cool - you were already looking at the regression ?
<benji> more specifically, it's working it's way through ec2 as we sppeak
<lifeless> wowsers, even better
<benji> yep, there is another bug; let me find it
<benji> here it is: https://bugs.launchpad.net/launchpad/+bug/778689
<_mup_> Bug #778689: ProjectMilestone:+global-actions: oops rendering the structural subscription link <oops> <story-better-bug-notification> <Launchpad itself:In Progress by benji> < https://launchpad.net/bugs/778689 >
<lifeless> thanks!
<benji> I'll mark the new one a duplicate of the one I was working on
<lifeless> I've just done that
<benji> cool
<lifeless> deryck: ready when you are
<deryck> lifeless: ok, firing up skype now
<deryck> lifeless: call when you're ready
<LPCIBot> Project windmill-devel build #61: STILL FAILING in 1 hr 6 min: https://lpci.wedontsleep.org/job/windmill-devel/61/
<benji> sinzui: do you have a minut to do a teeny, tiny JS review? (of a branch that fixes a bug that you reported (notifications close if you click on a link inside them))
<sinzui> I can do it now
<benji> sinzui: great! https://code.launchpad.net/~benji/launchpad/bug-779538/+merge/60552
<sinzui> benji: I select text in the notifications sometimes.I think the function will never let me copy the text to report a bug
<benji> hmm, good one; let me look at it real quick
<benji> sinzui: yep, that's a problem; back to the drawing board
<benji> I may well revert the feature if I can't figure something sane out quickly.
<sinzui> benji: can the action be wired to a the links id?
<sinzui> We know the text of the link is Hide x
<benji> well, that's not really a link; that's style; there's no element there
<sinzui> ah, right
<sinzui> I see your point about reverting.
<LPCIBot> Project windmill-devel build #62: STILL FAILING in 43 min: https://lpci.wedontsleep.org/job/windmill-devel/62/
<jcsackett> sinzui: if you had very similar testcases in questions and bugs, and were creating a base class for them to share, where would you put that class? lp.testing doesn't feel right, since the base test case won't be *that* generally useable.
<sinzui> jcsackett: lp/coop/bugquestion
<jcsackett> excuse the muddy thinking; a shared mixin is what i'm talking about.
<jcsackett> sinzui: thanks!
<james_w> benji, could you add the hide link with js? That would allow you to create an actual element with a onClick behaviour?
<sinzui> jcsackett: lib/lp/coop/answersbugs/ actually. but may lib/lp/coop/comments if we add merge proposals
<benji> james_w: I'm trying to avoid that because I want on-page JS to be able to add message boxes after page load time
<jcsackett> sinzui: i like lib/lp/coop/comments, since some day we want that all to be shared properly.
<james_w> benji, ah, ok
<benji> the only way I can think of doing that right now is useing setInterval, but that feels a bit gross
<lifeless> jcsackett: I would avoid creating a base class :)
<lifeless> jcsackett: what does the code you want to share do
<jcsackett> lifeless: base class was the wrong statement. mixin is what i'm looking at.
<lifeless> jcsackett: same dealio
<jcsackett> i have *very* similar tests that deal with question messages and bug comments. i would like to not have repeating code.
<lifeless> jcsackett: indeed!
<jcsackett> lifeless: so, a mixin that provides those shared things seemed like a good idea to me. what's the argument against?
<lifeless> http://rbtcollins.wordpress.com/2010/09/18/maintainable-pyunit-test-suites-fixtures/ and http://rbtcollins.wordpress.com/2010/05/10/maintainable-pyunit-test-suites/ may interest you
<lifeless> jcsackett: what are the shared things
<jcsackett> lifeless: aside from slight diffs in setup and the context being thrown at a test browser, the tests are looking at the same series of things.
<lifeless> jcsackett: ah, so you want to test their adherence to the contract?
<lifeless> pypi.python.org/pypi/testscenarios for that :>
<lifeless> jcsackett: a mixin is fine, but its not all that maintainable
<lifeless> jcsackett: you don't need to do anything different, I'm just gathering data about the things we do and why
<jcsackett> lifeless: well, if what i'm doing is going to perpetuate problems we have, i'd rather not.
<jcsackett> i'll take a look at testscenarios. i may have descibed my issue wrong.
<lifeless> jcsackett: cool
<jcsackett> in essense both cases have "test_thing_one", "test_other_thing", "test_thing_one_with_edge_case"
<lifeless> so, I *love* test-by-contract
 * jcsackett is not really sure what is meant by that.
<lifeless> uhm
<lifeless> testing that two objects that implement interface I obey the behaviour of the interface
<lifeless> so bugmessage and questionmessage are two classes that need the same behaviour
<lifeless> I wrote http://people.canonical.com/~robertc/interfaceverification.txt back in 2005 about doing this for persistence layers :)
<jcsackett> ah, okay.
<lifeless> jcsackett: anyhow it seems to me that you have some common do-stuff code, a different factory (or fixture) for each class and perhaps variant 'did it do the right thing' for each class
<lifeless> I haven't integrated testscenarios into launchpad yet, but using a per-file test_suite hook should let it integrate pretty easily on a case-by-case basis.
<lifeless> I would be inclined to structure these tests as fixtures for the bugmessage/questionmessage construction, matchers for the things to check and a single test class in the message area which is parameterised by the fixtures and matchers
<lifeless> *but*
<lifeless> this is sucking away your activation energy bringing in different ways of doing stuff
<lifeless> so I really do think its fine to do a mixin
<lifeless> like I said, I meant just to do data gathering
<jcsackett> matchers are something i am using, per a suggestion you had on an earlier branch.
<lifeless> yeah, they are pretty cool :)
<lifeless> and already integrated in
<lifeless> fixtures are already integrated in too
<jcsackett> yeah, i used matchers on some snapshotting work.
<deryck> Later on, everyone.
<LPCIBot> Project windmill-db-devel build #261: STILL FAILING in 43 min: https://lpci.wedontsleep.org/job/windmill-db-devel/261/
<wgrant> sinzui: Hi.
<sinzui> hello
<wgrant> sinzui: Can you QA your answer contact API stuff?
<sinzui> I can
<wgrant> It looks like we may finally not have any bad revs.
<lifeless> statik: yo
#launchpad-dev 2011-05-11
<sinzui> wgrant: mumble?
<wgrant> sinzui: You can't hear me?
<sinzui> Once again I think I need to restart to hear
<lifeless> I'm amazed our test suite works at all
<lifeless> os.environ['HOME'] = self.root
<lifeless> in the buildd tactestsetup affects all subsequent tests.
<lifeless> but
<lifeless> it helpfully deletes that directory
<wgrant> Heh
<lifeless> so when rabbit tries to start up
<lifeless> it uses that as home
<lifeless> which doesn't exist
<wgrant> Ahh
<lifeless> two bugs: rabbit should isolate itself.
<lifeless> and we should restore global state we futz with
<lifeless> \o/
<lifeless> bug 780794 if you're interested
<_mup_> Bug #780794: tests of/using BuilddSlaveTestSetup mangle global state for other tests <build-infrastructure> <Launchpad itself:Triaged> < https://launchpad.net/bugs/780794 >
<sinzui> wgrant: I am perplexed. My test script only passes the person changes. None of the question_target methods work
<wgrant> sinzui: Did you change the inheritance hierarchy?
<wgrant> sinzui: Each object has a single interface exported.
<wgrant> So eg. IProduct needs to inherit IQuestionTarget.
<sinzui> No, But I am wondering I I need to
<wgrant> Product cannot implement IProduct and IQuestionTarget directly.
<sinzui> okay, that is my mistake
<sinzui> wgrant The branch is okay to release, but I need to followup with another branch to pass my test
<wgrant> sinzui: Great, thanks.
<LPCIBot> Yippie, build fixed!
<LPCIBot> Project db-devel build #535: FIXED in 5 hr 31 min: https://lpci.wedontsleep.org/job/db-devel/535/
<lifeless> does Failure in test testEpochNonInteger (lp.archivepublisher.tests.test_debversion.VersionTests)
<lifeless> happen for anyone else?
<wgrant> Have you run update-sourcecode lately?
<wgrant> That test is new.
<wgrant> 30 rev deploy time :(
<lifeless> no, I haven't
<wgrant> lifeless: It was a bug in python-debian.
<wgrant> So update-sourcecode and try again.
<wgrant> Greeeeeeeen
<wgrant> zomg
<wgrant> Assignee picker on +filebug works on qastaging.
<LPCIBot> Project windmill-devel build #63: STILL FAILING in 44 min: https://lpci.wedontsleep.org/job/windmill-devel/63/
<wgrant> Hmm.
<wgrant> I wonder how many users the API export of archive.checkUpload has.
 * wgrant checks the PPR.
<lifeless> \o/\o/\o/\o/\o/\o/
<wgrant> ?
<wgrant> Rabbit working?
<lifeless> yup
<lifeless> lp-land in a second
<wgrant> !!!
<wgrant> Did you decrapify the U1 harness a bit?
<lifeless> a tad more
<wgrant> Great.
<wgrant> Next stop: wildcherry.
<lifeless> but I want the darn thing in tree
<wgrant> Yeah.
<lifeless> so that the next 'if we had queues' comment I see I can drop the bomb on
<wgrant> Yes yes yse.
<lifeless> of course, we have 1 person (jtv) with srs queue experience
<lifeless> so its not at cargo cult point yet
<wgrant> :( 127 checkUpload calls last month.
<wgrant> Maybe I can grep logs and destroy whoever is doing that.
<jtv> Stereotactic RadioSurgery?
<wgrant> Because it's a shit API.
<lifeless> jtv: serious
<jtv> lifeless: ah... so, exciting rabbit things happening?  Gimme!
<lifeless> :)
<jtv> oh damn crappy shitty damn bad connection I smegging hate this
<lifeless> :>
<lifeless> so I answered you with
<lifeless> :)
<jtv> lifeless: got the smileys, and would be joining in the smiles if it weren't for the broken connections!
<spm> o/ jtv
<jtv> Hi spm!
<jtv> And the good thing about bad connections is you get to keep saying hello to your friends.  :)
<lifeless> jtv: the next step is a Layer for rabbit, which should be trivial with the fixture
<jtv> Aigh!  Not a layer!  We want resources and fixtures!
<lifeless> jtv: and after that we probably need some test support - things to say 'a message was sent' and so on
<lifeless> jtv: we have a fixture
<lifeless> jtv: but it needs to fit in the current test layout
<jtv> Which we all hate :)
<lifeless> sure
<lifeless> orthogonal problem
<jtv> True.
<lifeless> anyhow
<lifeless> pull devel
<lifeless> its there
<jtv> Great
<jtv> Meanwhile, I thought I was being clever but am running into problems that you'd be able to help with.
<lifeless> shoot
<jtv> My test takes 6s or so.  I thought I could speed it up by re-using test SPPH objects across tests, because creating those seems to be taking most of the time.
<jtv> I had a really nice idea for that: a generator that caches a list of SPPHs.
<jtv> You request one, it gives you the next one from its cached list.
<lifeless> the db layer rolls back the db
<jtv> Yes.  :(
<lifeless> either have one test, or accept the cost, or test narrower layers that don't need the db (oops the last one is [for now] science fiction)
<jtv> I suppose I might produce fakes, but it gets a bit hackey.
<jtv> (Thank you, alarm clock, but I've been at work for hours)
<jtv> This particular one is pretty painful.  It'd have been nice to avoid it.
<jtv> I do have a plan B:
<jtv> see if somewhere in the factory we do commits just for the Librarian's sake.
<jtv> If so, use FakeLibrarian and replace the commit with the fake.
<jtv> It's really ridiculous: at one point I'm spending 0.4s on creating 2 test objects!
<lifeless> yeah
<lifeless> thats about 10 tests in bzr :>
<jtv> Hmm not a lot of committing going on in the factory these days.  That's good, but it means the remaining fruit hangs higher.
<wgrant> jtv: Have you profiled?
<wgrant> That's hideous.
<wgrant> Alternatively use makeSourcePackagePublishingHistory, which doesn't create any files AFAIK.
<jtv> Only print-profiling so far.
<wgrant> At least I was able to use it in DatabaseFunctionalLayer without a problem.
<jtv> It _is_ makeSourcePackagePublishingHistory that's taking up that time.
<jtv> It seems to be a bit irregular somehow.
<wgrant> Some apparently trivial utility operations can be really slow.
<wgrant> NFI why.
<jtv> I'm going to try it in "make harness" with SQL logging on.
<jtv> See if it's doing any outrageous DB work.
<wgrant> Yeah.
<wgrant> There's lots of SQL.
<wgrant> Huh.
<wgrant> makeSPPH repeatably takes 0.3s
<wgrant> That's awful :/
<jtv> Quite.
<jtv> Imagine what we could save if we had an easy way around that.
<wgrant> I guess it creates lots of people.
<wgrant> Which is slow.
<wgrant> Because of Account.
<jtv> I was going to ask about that: how does that currently work?
<wgrant> Which?
<wgrant> Account?
<jtv> Account.  I half expected to see commits to synchronize the account/LP dbs.
<wgrant> Right, that was removed a while ago.
<wgrant> Since we no longer have a HA auth store.
<wgrant> Because SSO is a separate DB.
<wgrant> So you no longer need to commit after creating people.
<lifeless> and session is also not replicated
<lifeless> if we fail over its trashed
<wgrant> Looks like creating a person takes 15 queries :(
<wgrant> Still should be quick, though.
 * wgrant fires up a profiler.
<wgrant> :( no context manager profiler
<jtv> Can't start one of the regular python profilers in "make harness"?
<jtv> Looks like makeArchive() takes up a significant portion of the time of a makeSPPH().
<wgrant> Yes. But a context manager would probably be easier.
<lifeless> bzrlib.lsprof.profile(...) is your friend
<lifeless> and yes, it works in harness
<wgrant> Can we just merge bzrlib into the standard library or something?
<lifeless> *blink*
<lifeless> lp.testing.storm
 * lifeless sobs
<wgrant> Does not exist.
<lifeless> lib/lp/testing/storm.py
<lifeless> pull devel
<lifeless> its quite buggy for new generic code
<wgrant> I pull in trepidation.
<lifeless> sinzui: when you said hack, I didn't imagine this
<lifeless> sinzui: (sorry for being negative; it is a bit of a tricky problem)
<wgrant> 28% in the DB, 20% clearing lazr.restful memcache...
<lifeless> the representation cache?
<wgrant> Yes.
<lifeless> turn the f*er off
<wgrant> 10% on top of the 28% in the storm tracer.
<lifeless> I thought the representation cache was off in prod anyhow
<wgrant> It is.
<wgrant> Killing the representation cache only shaved off a bit under 10%.
<wgrant> Now 50% in storm, 40% in the DB.
<lifeless> OOPS-1957DY36
<lifeless> actually filing a bug should not lookup similar bugs ><
<lifeless> right, bug 780835
<_mup_> Bug #780835: representation cache a pessimism <lazr.restful:New> < https://launchpad.net/bugs/780835 >
<jtv> Pessimism or pessimization?
<lifeless> bah
<lifeless> yes
<lifeless> ESLEEPFFFFAIL
<jtv> Hmm I got some savings of a few tens of seconds by eliding Person constructions.
<lifeless> fixed
<wgrant> jtv: createPersonAndEmail seems to take 33% of the time.
<wgrant> And a third of that is EmailAddress.new...
<wgrant> How.
<lifeless>                                                                                                                        
<lifeless>         if self.getByEmail(email) is not None:
<lifeless>             raise EmailAddressAlreadyTaken(
<lifeless>                 "The email address '%s' is already registered." % email)
<lifeless> LBYL :(
<wgrant> Yeah.
<lifeless>         assert status in EmailAddressStatus.items
<lifeless> is inefficient (should be a type check)
<wgrant> Still trivial.
<lifeless> valid_email looks sane enough
<lifeless> how many persons are you creating?
<wgrant> 5, looks like.
<lifeless> oh
<lifeless> that LBYL isn't indexed.
<wgrant> Aha.
<lifeless> possibly
<wgrant> It must be, surely.
<lifeless> need to check one more thing
<lifeless>     "emailaddress__lower_email__key" UNIQUE, btree (lower(email))
<lifeless> ah, it is
<wgrant> 6% in PersonSet.getByName :(
<jtv> At <100 persons in the sample data, would it matter anyway?
<lifeless> jtv: psosibly ;>
<jtv> I nearly halved my test time just by eliminating makePerson calls.
<lifeless> but
<lifeless> its stupid to have an index on a function
<wgrant> lifeless:
<lifeless> when we can apply the function on data insertion
<wgrant>  Sort  (cost=6.65..6.66 rows=1 width=39) (actual time=0.146..0.146 rows=0 loops=1)
<wgrant>    Sort Key: email
<wgrant>    Sort Method:  quicksort  Memory: 25kB
<wgrant> But should still be fast, given how small it is :/
<wgrant>    ->  Seq Scan on emailaddress  (cost=0.00..6.64 rows=1 width=39) (actual time=0.096..0.096 rows=0 loops=1)
<wgrant>          Filter: (lower(email) = 'email755122@example.com'::text)
<wgrant> That's the plan.
<wgrant> So WTF is it so slow.
<wgrant> It's sub-ms.
<jtv> lifeless: absolutely... especially because last I heard, function indexes weren't quite as effective as I expected.
<lifeless> specifically, all queries on email come from our appservers
<lifeless> joins are on id
<lifeless> so we should be able to lower() in the appserver before querying rather than in the db
<lifeless> anyhoo
<lifeless> wgrant: how many ms does your profiling think it is?
<lifeless> wgrant: and what profiler did you use?
<wgrant> 5% of roughly 300ms.
<wgrant> So 15ms.
<wgrant> I used bzrlib.lsprof.
<lifeless> ok
<wgrant> The query is 200Âµs hot.
<lifeless> so @ 15ms that suggtests 14ms in python
<jtv> wgrant: what exactly was slow?
<lifeless> wgrant: how long does the profiler think the cursor get took ?
<wgrant> jtv: EmailAddressSet.getByEmail
<jtv> Weirdness.
<jtv> Unexpected ASCII/Unicode conversion issue?
<jtv> (Since this is one of the few cases where ASCII would probably be fine)
<wgrant> kcachegrind confuses me.
<wgrant> Or maybe it's just lying to me. It's not showing any callees beyond selectOne.
<lifeless> you've turned min-node down ?
<wgrant> As low as it goes, yes. But it's still showing some nodes without children. Hrm.
<lifeless> C modules add some fun
<jtv> Stripped?
<jtv> Or otherwise without debug info?
<lifeless> they show as a monolithic block in the profiler
<lifeless> because the profiler works by running code between every opcode
<lifeless> (thats a lie, its every line, shrug)
<lifeless> and when C modules call back into python the callgraph can be fun
<spiv> 'perf' can sometimes show you where C modules are consuming time.
<spiv> (Not tied into the Python callstack at all, of course)
<wgrant> I think kcachegrind might just be broken.
<wgrant> gprof2dot sees calls into EmailAddress.new as expected. kcachegrind sees 0.
<lifeless> win
<wgrant> Hmm. kcachegrind shows two createPersonAndEmail functions.
<wgrant> I wonder if some C in the middle is breaking everything.
<wgrant> One of them has no calls.
<wgrant> And the one with no calls calls the uncalled functions!
<wgrant> That's not very nice.
<lifeless> \o/
<lifeless> that might be fixed by jams recent lsprof tweak in bzrlib actually
<wgrant> Ahh, and the harness will be using ancient bzr...
<wgrant> 2.2.2 :/
<wgrant> We should really fix that.
<lifeless> yes!
<lifeless> bzr team have been doing that
<wgrant> Have they?
<lifeless> thats my understanding
<lifeless> IMBW
<wgrant> I don't think it's been updated since Code was disbanded and then everyone ran away,.
<wgrant> It was last changed 2011-02-09, when I applied a patch for a stacking bug.
<wgrant> So it hasn't been upgraded since Code evaporated.
<wgrant> => bzr probably doesn't do it.
<wgrant> lifeless: Ahh, that's much better.
<wgrant> The tree is no longer lots of trees.
<wgrant> (after a quick port to 2.4b2)
<wgrant> Perhaps I will toss it at ec2 and see what breaks.
<wgrant> After I work out how to use weave_fmt properly as a plugin.
<wgrant> Ahh
<wgrant> 2/3 of the getByEmail time is flushing writes.
<wgrant> Before it does the find.
<wgrant> It would be more illuminating if we could turn off the write cache...
<LPCIBot> Project windmill-devel build #64: STILL FAILING in 1 hr 8 min: https://lpci.wedontsleep.org/job/windmill-devel/64/
<LPCIBot> Project windmill-db-devel build #262: STILL FAILING in 1 hr 7 min: https://lpci.wedontsleep.org/job/windmill-db-devel/262/
<lifeless> wgrant: which write cache
<wgrant> lifeless: Storm's.
<lifeless> the same one that broke commits on the archive thingy the other week ?
<wgrant> Hm?
<wgrant> Anyway, I emulated disabling it by flushing after adding each new object.
<lifeless> when I eager loaded source packages
<lifeless> commits crawled to a halt because of O(cache size) behaviour in commit
<wgrant> Ah.
<wgrant> No, this is the dirty object cache.
<lifeless> I'm trying to decide between
<lifeless> fix a timeout
<lifeless> write a rabbit layer
<lifeless> $other
<wgrant> Rabbit.
<wgrant> Rabbit will fix lots of timeouts.
<wgrant> lifeless: staging would like to have http://paste.ubuntu.com/605983/ run on it with an EXPLAIN ANALYZE.
<lifeless> [qa]?
<wgrant> Either.
<lifeless> http://paste.ubuntu.com/605985/
<wgrant> lifeless: And hot?
<lifeless> 340.216
<wgrant> Slower than DF :(
<wgrant> 205ms hot.
<lifeless>                ->  Hash  (cost=13594.89..13594.89 rows=5168 width=8) (actual time=112.326..112.326 rows=5189 loops=1)
<wgrant> But more like what I was expecting.
<lifeless>                ->  Index Scan using securesourcepackagepublishinghistory__distroseries__idx on sourcepackagepublishinghistory  (cost=0.00..19355.35 rows=16746 width=24) (actual time=0.259..214.622 rows=17420 loops=1)
<StevenK> Hold on, my world is crashing down.
<jtv> We'll have to revise the term "dog [food] slow"
<lifeless> wgrant: you just restored; less bloat
<StevenK> staging should have just done that, too?
<lifeless> hahahahaahhahahahahaha
<lifeless> no, restore failed
<StevenK> :-(
<lifeless> also this is qastaging
<StevenK> Ah
<lifeless> which is better to test on for things that aren't coupled to schema changes
<StevenK> But still only restores on demand :-(
<lifeless> (same hardware, longer lifetime - it gets its own bloat going, and its where we qa upcoming deploys)
<jtv> wgrant: did you eliminate that silliness in packagecopier where a permissions error was suppressed if the user had Append privileges?
<jtv> "if reason is not None: raise CannotCopy(reason)" instead of a nested "if" to give it another chance?
* jtv changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: jtv | https://code.launchpad.net/launchpad-project/+activereviews
<jtv> Or maybe it was Gavin.
<jtv> And by the way wgrant, would you care to review my "copy asynchronously if there's more than 100 packages to copy" branch?  https://code.launchpad.net/~jtv/launchpad/bug-780319/+merge/60571
<lifeless> jtv: 100?!
<lifeless> jtv: we struggle to copy 2 in the webapp
<jtv> That's what's been specced.
<lifeless> well, reality is going to intervene
<jtv> Then feel free to lower the threshold to 1.  :)
<StevenK> Er, doesn't it make use of PackageCopyJob?
<jtv> Yes.
<StevenK> Then that's completly different.
<StevenK> lifeless: ^
<lifeless> so if its using a job
<lifeless> its already asynchronous
<lifeless> isn't it?
<lifeless> jtv: the time budget for backend tasks that hold transactions is effectively 7-8 seconds (allowing time for webapp changes after they start actually doing writes
<jtv> This implements a choice between synchronous and asynchronous.
<lifeless> does synchronous mean in-webapp
<jtv> Yes.
<lifeless> if that is set to 1, it might work sometimes.
<jtv> Is someone planning to speed that up massively, perhaps?
<lifeless> yes, by making it async
<lifeless> 964  OOPS-1955CE140  Archive:+copy-packages
<jtv> I mean the actual copying.
<lifeless> 11 /    8  Archive:+copy-packages
<wgrant> Copying is very fast.
<wgrant> Copy checking is still slow sometimes.
<lifeless> and when its slow its glacial
<lifeless> 11 failures on 416 copies yesterday
<lifeless> 2.5% failure rate
<wgrant> Lots of those were probably delayed copies.
<lifeless> 99th percentile is 8.3 seonds
<lifeless> mean is 3 seconds
<wgrant> Non-sql time: 4333 ms
<lifeless> when you say 'fast'
<wgrant> lifeless: +copy-packages or syncSource?
<lifeless> Archive:+copy-packages
<wgrant> That's got UI crap.
<wgrant> Which is unoptimised and constant.
<lifeless> Archive:EntryResource:sync has its 99th percentile 11 seconds
<lifeless> mean 2.15 seconds
<wgrant> Right, because it's used a lot for delayed copies, or copies from -proposed to -updates.
<wgrant> Which close bugs.
<wgrant> None of which is optimised.
<lifeless> 99th percentil of db time is 6.5 seconds
<wgrant> Actual copying is fast. Delayed copies are not. Bug closing is not.
<wgrant> Checking when the source already exists in the target archive is not.
<lifeless> sure, but does 'do a copy' trigger these other htings?
<wgrant> Yes. They can and should be optimised.
<lifeless> if it does, then a discussion about copying has to assume that copying is slow [and optimisable perhaps]
<jtv> In any case, my branch isolates a policy choice for when to do which.
<lifeless> which is great
<lifeless> I guess I'm saying that right now, I would put the policy at 0
<jtv> I figure improving and tuning that choice is a separate step.
<lifeless> jtv: in fact, there is a good request: please make sure its possible to disable synchronous copying entirely.
<wgrant> As long as these both use pretty much the same code.
<wgrant> And it isn't like delayed copies.
<wgrant> If it is anything like delayed copies I will rs it out now.
<jtv> rs?
<wgrant> Delete without review, because delayed copies are just that hideous.
<jtv> Where do delayed copies come into the picture?
<wgrant> They are the old asynchronous copy mechanism.
<jtv> I think I saw bits of that, deeper down in the call tree.
<wgrant> Which is not even slightly like the synchronous copy mechanism.
<wgrant> If both of these use the direct copy function, I am happy.
<jtv> When do we use which?
<jtv> They use do_copy.
<wgrant> Delayed copies are used for copies from private to public archives.
<jtv> So that's a policy decision, not some kind of time management?
<wgrant> Half and half and half of some more hacks.
<StevenK> Delayed copies are *hideous*, and I'm going to remove them
<wgrant> It was initially because those copies needed to reupload files to the public librarian.
<wgrant> Which was slow.
<wgrant> Then more hacks were layered on top that were unrelated.
<wgrant> And then it became clear that the initial rationale was wrong: you can just toggle the restricted flag in the DB.
<wgrant> But it all remains.
<jtv> So... it might make sense for the sync/async choice to check whether any of the source archives are private and the dest archive is public?
<wgrant> This UI cannot go anywhere near production until delayed copies are dead and buried.
<wgrant> Or we'll have doubly async copies.
<wgrant> And badness.
<jtv> If you're worried about doubly async copies, then I guess that would be a blocker for using PackageCopyJobs.
<wgrant> Doubly async is stupid, and the behaviour is inconsistent. a PackageCopyJob should never use a delayed copy.
<jtv> Well, both it and the view code call do_copy.
<wgrant> And do_copy will create a delayed copy if the stars are aligned and it deems the witchcraft justfieid.
<jtv> If do_copy may then defer yet again to another async kind of job, then that needs fixing but it's independent of my branch.
<jtv> Anyway, I need to go out and find food and coffee.  In that order: I'm that desperate.
<wgrant> As long as this is all heavily flagged.
<jtv> With wgrant telling me the safe choice is to go synchronous and lifeless telling me I should probably always go asynchronous.  Whee!
<wgrant> I didn't say to go synchronous.
<wgrant> I said that synchronous can be fairly fast, and that you are to under no circumstances go *doubly* asynchronous by bringing delayed copies into the mix.
<jtv> In other words, that "the safe choice is to go synchronous"
<wgrant> Synchronous will do delayed copies in the same cases as asynchronous will.
<jtv> And so delayed copies can't be handled synchronously as per robert and can't be handled asynchronously as per william?
<wgrant> Delayed copies must not be handled at all.
<wgrant> If you are creating them in this new workflow you are doing it wrong.
<wgrant> But you may be doing it accidentally.
<jtv> How would I know whether I'm creating them?
<jtv> This still seems orthogonal to me.
<wgrant> If you're not aware of them then it's not orthogonal.
<wgrant> You need to be aware of the horrors that lurk within packagecopier.
<wgrant> So, the reason I like synchronous copies for now is that our async UI story is completely hopeless.
<wgrant> You don't get a response in a reasonable amount of time.
<wgrant> Because a) the job probably won't run for more than a minute, and b) the client has to poll.
<wgrant> So you end up with crap like the apport refreshing-every-10-seconds-don't-mind-me thing.
<jtv> Which I want no part of.
<wgrant> I think until we can fix both we should try to keep things synchronous.
<spiv> wgrant: skimming this conversation, it strikes me you are warning loudly about the lurking horrors, without explaining how to actually find out about them.
<wgrant> spiv: Because there's no way to solve this yet :)
<spiv> (But I'm only skimming, so apologies if I've missed an important bit)
<jtv> Welcome to R'lyeh.
<lifeless> jtv: hi, I didn't mean to cause confusion
<lifeless> jtv: have you ate?
<jtv> No.
<jtv> It's getting urgent.
<lifeless> jtv: so go do that
<lifeless> jtv: This is a years old mess
<jtv> But wgrant keeps talking about the horrors that haunt his nightmares.
<spiv> Um, I'm not talking about a solution.  I'm just saying you're saying "must not handle delayed copies" without giving jtv any pointers on how to ensure that's the case.
<lifeless> jtv: one meal won't make any difference.
<wgrant> You hadn't skipped any solution :( CopyChecker takes an allow_delayed_copies argument. If it's True it might create delayed copies, which is the wrong thing. If it's False it won't create delayed copies, so they will be copied directly which then misses the extra delayed copy functionality so is also the wrong thing.
<wgrant> spiv: There is no solution to ensure that's the cae.
<wgrant> +s
<spiv> (Other than by not implementing things at all)
<wgrant> Right.
<wgrant> My advice would be to not try to extend the copy mechanism until the copy mechanism is not a nightmare.
<wgrant> In more general terms, plan!
<lifeless> mmm
<lifeless> I agree that nuking tech debt makes things easier to do.
<lifeless> This new functionality will be doing maaasive copies we don't do at all
<lifeless> It may need to fix things up to move forward.
<lifeless> now, all that said doing double-async is gross but would work.
<wgrant> Sure. But it's currently *not possible* to implement this correctly.
<wgrant> Unless delayed copies are nuked.
<lifeless> wgrant: other than grossness, where am I wrong?
<wgrant> You would have to use delayed copies for all copies.
<wgrant> Which means making everything doubly-async and inconsistent.
<lifeless> why?
<wgrant> Because only delayed copies respect overrides and announce.
<lifeless> ok, and this patch needs that ?
<wgrant> That's why delayed copy elimination is part of this work.
<wgrant> Because DDs cannot operate without overrides and announcements.
<lifeless> hang on
<lifeless> we have a narrow patch up
<lifeless> to do async if > some N
<lifeless> how will that *break*
<wgrant> It won't break, it's just the first time I've looked at this additional mess in depth.
<lifeless> ok
<lifeless> so, if it won't break, we don't need to block jtv on this, as long as the overall arc will fix shit up.
<wgrant> This is Launchpad.
<wgrant> The overall arc will not.
<lifeless> well, we can talk to julian about that
<wgrant> Julian is under the impression that the feature squad is over in just a few weeks.
<wgrant> I don't know how accurate that is.
<wgrant> But it does not bode well.
<lifeless> julian and flacoste :)
<lifeless> we may need to tweak our size estimation process
<lifeless> we definitely need to get faster
<wgrant> We do.
<lifeless> so, *personally* I would:
<lifeless>  - make it always async
<wgrant> That would be no problem if the UI didn't suck.
<wgrant> But the UI will suck.
<wgrant> So we should avoid it.
<lifeless> I think you need to consider more variables in assessing this
<lifeless> do you agree that not all copies can be done in <1 second, and that in fact most will need > 1 second ?
<wgrant>  Sure.
<lifeless> well, s/most/more than 1%/
<lifeless> ok
<lifeless> so we need the async case to exist; we need it to have a reasonable UI.
<wgrant> Launchpad has no facilities for reasonable async UI.
<wgrant> We are not ready.
<lifeless> we have sufficient facilities to get a tolerable UI that will be robust and amenable to optimisation.
<wgrant> lol
<lifeless> and we will save /significant/ complexity having one code path.
<wgrant> But perhaps you consider "This page will refresh every 10 seconds." to be tolerable UI.
<wgrant> It may be,.
<wgrant> But it seems pretty crap to put that on a new feature.
<lifeless> wgrant: vs a timeout, I think its a marvelous improvement.
<wgrant> No, it's vs immediate feedback for small copies and frequent refreshes for large copies. And people probably don't care much about immediate feedback for large copies,.
<lifeless> wgrant: we should refresh more often than that anyway, for a polled approach.
<wgrant> lifeless: I was quoting from our existing async page refresher.
<lifeless> wgrant: s/refresh/ajax-check/
<lifeless> wgrant: so thats an argument for having two UI's.
<lifeless> wgrant: on a purely technical basis i think that that is suboptimal and harder for us to do (I don't mean tougher, I mean 'more total effort')
<lifeless> wgrant: you have expressed a desire for us to finish things more completely; part of that is making things cheaper to do. That means less effort output to achieve stated goals.
<wgrant> lifeless: Once we can quickly switch to a page probably with rows for each copy and a progress indicator, sure.
<lifeless> wgrant: all the bits are in place to do that.
<wgrant> But if it's a tradeoff between a little additional complexity and a completely terrible UI, we should go for that minimal extra complexity.
<lifeless> thats not the tradeoff though
<wgrant> We may well be able to quickly whip up an AJAX UI for this. If so, great! We can make everything async and it will be excellent.
<lifeless> its between having 4 code paths and 2 (sync, delayed, job-sync, job-delayed) vs (job-sync, job-delayed)
<wgrant> But I don't think there's time, given the huge amount of unscheduled trainwreck that's still to come.
<wgrant> lifeless: Well, delayed copies probably don't have much life left (hi StevenK)
<StevenK> lifeless: delayed copies are still a pox and need to die
<lifeless> right, I agree, which is why we shold be making this as *simple* as possible to let work proceed on the bits that matter!
<wgrant> Oh, it turns out that you get more meaningful results from the PPR if you're looking at the recent one in ~lpqateam, not the ancient one in ~stub.
<wgrant> Oops.
<wgrant> StevenK: With a new index I have the source query down to 80ms.
<wgrant> 70ms...
<StevenK> Whoa
<StevenK> What index?
<wgrant> sourcepackagepublishinghistory(archive, distroseries, pocket, status);
<wgrant> That's 70ms.
<wgrant> Without pocket is 80ms.
<wgrant> Need to check how that works with a pocketless query, though.
<wgrant> Because i need that fast for permission checks.
<wgrant> (it's currently >300ms per source)
<StevenK> We don't need a DB patch for the index?
<wgrant> We'll do a live one. Just working out what's best.
<wgrant> (archive, distroseries, status, pocket) is quick for both pocketful and pocketless queries.
<wgrant> But I hope I can get even better.
<lifeless> jtv: back?
<lifeless> jtv: ah, just your irc client autoconnecting after network -fail-
<wgrant> StevenK: Do you have a binary version of that query>?
<wgrant> StevenK: Preferably with some common binaries.
<StevenK> I do, but only for 'dpkg'
<StevenK> Pasted in PM
<wgrant> That is a bit slow cold :(
<StevenK> Tis, yes :-(
<StevenK> It also isn't as simple as the source query
<wgrant> Hot it's 100ms for a few common binaries.
<wgrant> In a single query.
<wgrant> The plan is the opposite of the source one, though.
<StevenK> Strange
<lifeless> many more binaries than sources
<StevenK> Sure, but sometimes the query planner gets strange ideas
<wgrant> It always finds the BPRs first.
<wgrant> Whereas it tries the SPPHs first.
<StevenK> wgrant: What does the query look like for multiple binaries?
<wgrant> StevenK: Well, I just did a binarypackagename IN (blah), so I'm not doing multiple DASes yet.
<StevenK> Ah
<StevenK> So, you're cheating
<wgrant> That doesn't change the plan, though.
<wgrant> Because it finds all BPRs with the right name, then indexes into BPPH on those IDs, then filters for DAS.
<wgrant> Only thing it will change is the volume of input to the sort.
<StevenK> wgrant: As you say, it's interesting it picks BPR first, whereas the source query starts with SPPH.
<wgrant> Well, the source query actually grabs both and does a hash join.
<wgrant> It's hard to say which is more efficient, because of the differing row counts.
<wgrant> StevenK: Heh, I just did the ultimate source test.
<wgrant> Finding overrides in maverick for every sourcepackagename in Ubuntu.
<wgrant> 480ms.
<wgrant> Not too bad.
<wgrant> Better than the publisher's attempt at that, at least.
<StevenK> Haha
<StevenK> \o/
<StevenK> wgrant: And how evil was that query?
<wgrant> Oh, I just selected all the SPNs into a temp table and used that.
<StevenK> If we could get the publisher to use this, it would be awesome
<StevenK> I'm not sure it fits
<wgrant> It fits.
<wgrant> Mmm, but it's already only a couple of seconds.
<wgrant> Except for binaries.
<wgrant> Which are like 20s.
<wgrant> Need to work out why the planner does that.
<StevenK> Doesn't it drop a bit of code duplication?
<StevenK> (And so is a win from that perspective)
<wgrant> Yes.
<wgrant> But once we have the generic overrides stuff a lot gets simpler.
<wgrant> Not just that :)
<wgrant> Including two things I was going to work on this week, but have deferred until this work is complete.
<StevenK> \o/
<StevenK> I think I need to format the query better and write docstrings and I'm down with Existing
<StevenK> How to split the DISTINCT over mutiple lines, for instance
<jtv> lifeless: back now... thanks for stepping in earlier btw
<wgrant> lifeless: What do you think of http://bazaar.launchpad.net/~wgrant/storm/distinct-on/revision/391?
<lifeless> hash joins are generally poor :(
<lifeless> at least in our scale
<wgrant> It's fast enough here.
<wgrant> But yes, I would normally expect them to be bad.
<wgrant> Not sure why it picks it, but it works.
<lifeless> wgrant: thats lovely and minimal
<wgrant> But...
<lifeless> I don't know what raw=True implies
<lifeless> no buts
<wgrant> Oh.
<wgrant> It lets you pass in strings directly. Not sure if it's desirable, but it's what's used for group_by and similar things.
<wgrant> That was what I was going to check after you didn't reject this idea.
<lifeless> thank you for having a low activation energy threshold
<wgrant> Given it's so simple it seems better than perpetuating the string hack everywhere.
<lifeless> god yes
<lifeless> if I was a theist.
<wgrant> Haha
<StevenK> Ooh, when can I use it, then?
<wgrant> With a bit of luck once I merge it into the LP branch in a few minutes.
<wgrant> Then I'll write an LP branch to use it everywhere, and you can possibly merge that before it lands.
<StevenK> \o/
<wgrant> I guess I should file a storm bug.
<wgrant> Oh look, there's a storm bug already.
<lifeless> there is one
<wgrant> Bug #374777
<_mup_> Bug #374777: DISTINCT ON queries <Storm:New> < https://launchpad.net/bugs/374777 >
<wgrant> That's almost half a Launchpad ago.
<wgrant> More than half a Launchpad ago.
<wgrant> lifeless: If raw=True, .config(distinct=['something']) will work like .config(distinct=[SQL('something')])
<wgrant> Like order_by/group_by do: a string is automatically treated as an SQLRaw.
<jtv> lifeless, wgrant: was there a conclusion to the question of synchronous/asynchronous immediate/delayed package copies?
<wgrant> jtv: Do it all async and get at least a minimally unawful UI on it later.
<wgrant> I think that was the result.
<jtv> What happened to "on no account should you..."?
<lifeless> jtv: there were a few specific things
<wgrant> As long as you are aware of delayed copies and do not run this code in production before they are removed, it's OK.
<lifeless>  - what you have doesn't imply blowing up awfully; its fugly if you do delayed copied in an alyread async job, but fugly != broken.
<lifeless>  - you /will/ need async for probably a majority of your copies, even with the logic fixed, copying as a whole has a lot of work to do.
<wgrant> lifeless: It will give inconsistent results. And inconsistency is brokenness when dealing with a primary archive.
<lifeless> wgrant: what form of inconsistency
<lifeless> wgrant: vs triggering the same delayed copy in the webapp
<wgrant> lifeless: Delayed copies and direct copies perform difference overrides, copy different set of binaries, announce differently.'
<wgrant> s/difference/different/
<lifeless> wgrant: I want to separate the issues in *this patch* vs those in the existing code
<wgrant> OK.
<wgrant> Sure.
<lifeless> wgrant: the issues in the existing code are a topic for julian and work scheduling; the issues in or introduced by this patch are a topic for jtv & code review
<jtv> Exactly.
<lifeless> wgrant: so, are there any actual correctness, (vs fugliness) issues in this patch ?
<lifeless> wgrant: my understanding is that there are not
<jtv> lifeless: thank you
<wgrant> Apart from building further on a shaky foundation without stabilisation scheduled, no, it's fine.
<lifeless> wgrant: (pending a detailed code review for thinkos etc)
<lifeless> ok
<lifeless> so back to the things I think we concluded
<lifeless>  - because async will be needed for most copy operations, the async UI will need a bunch of work
<lifeless> Now, *I* have a recommendation that you (jtv) just do async and put time into making the experience nice
<lifeless> because that benefits all cases.
<lifeless> Its only a recommendation
<jtv> By the way, what exactly does "the async UI" mean?  AIUI there don't need to be separate UIs for sync and async _requests_, but it'd be nice to have some _visibility_ into pending async requests.
<lifeless> but I think it would save a bunch of duplicate code.
<lifeless> jtv: minimally a spinner while the copy happens.
<lifeless> Perhaps a progress bar showing x/Y completed.
<lifeless> the details are a topic for you/julian/jml/sinzui/huwshimi :)
<jtv> Some interesting design questions in those details; generally you'll probably have just one or two jobs for the whole thing.
<jtv> Also makes it hard to see which requests are underway.
<wgrant> Hmm, so there's a single job for all 10000 copies?
<lifeless> Finally, the underlying mechanics of copies badly badly need rectification to be more robust and sane, given the level of current pitfalls
<jtv> Could be, yes.  That's the way those jobs work.
<wgrant> OK.
<jtv> I fully buy into the notion that soyuz needs a firm broom.
<lifeless> now, where to go from here. I think you should discuss with julian - and I'm delighted to participate - about whether its worth having a sync+async api.
<lifeless> s/api/ui/
<jtv> In fact just a very simple change with basically no UI implications almost hit the 800-line point because of the simple cleanups needed.
<lifeless> but I'm totally serious when I say that the threshold to avoid timeouts is about 0.
<jtv> That brings me to something that came up earlier.
<jtv> Somebody mentioned that _checking_ the copies took up most of the time..?
<jtv> Because the permissions check at least (part of the checking work) happens synchronously either way.
<wgrant> Apart from one case (closing bugs when copying to -updates or -security), the actual copy is very fast.
<lifeless> the soyuz datamodel is not enforced (nor can it be -today-) by DRI
<lifeless> the mechanics of adding the new rows is fast
<wgrant> jtv: Permission checking we can do in a few hundred milliseconds for the whole archive.
<wgrant> jtv: Does it do the full check?
<jtv> Just the permissions check.
<lifeless> jtv: also async ui - error reporting is a component too
<wgrant> jtv: (at the moment it will be ~400ms per source, but with StevenK's work and some refactoring I have planned for after that, it becomes 600ms for the whole archive)
<jtv> wgrant: that's for the full copy including checks?
<wgrant> jtv: Just permission checking, sorry.
<wgrant> At the moment doing permission checks for more than a few packages at a time is infeasible.
<jtv> Ah, so that's relatively costly and I can't even avoid it in the async case.
<lifeless> per package you're doing ?
<jtv> Yes.
<lifeless> sorry, that was to wgrant
<lifeless> is that 600/400ms 'for all the packages in a copy' or 'per'
<wgrant> jtv: I thought we were just checking for any permission on the relevant archive, and leaving the details to syncpackagejob.
<jtv> lifeless: AIUI 400ms is per package.
<wgrant> lifeless: 400ms is per source. 600ms is for every source in one query.
<wgrant> For a reasonable selection of sources is more like 100ms.
<lifeless> wgrant: so 600ms is 'copy all sources from archive a to b and make sure you have perms to do that' ?
<wgrant> lifeless: Just permission checking.
<jtv> wgrant: from what I can see, the PackageCopyJob (or CopyPackageJob, I forget) does do_copy without a permissions check.  Which makes sense semantically: why let people create jobs to sync packages they're not allowed to?
<lifeless> wgrant: yes, I phrased it badly
<wgrant> jtv: Well, do_copy only gained permission checking yesterday.
<wgrant> jtv: So it's unsurprising that CopyPackageCopyJobCopyPackage doesn't use it. But it could.
<jtv> wgrant: and when you say the 600ms is for "the" archive, you mean the destination archive, right?  I'm asking since AIUI there can be multiple source archives in the same request and they may also be involved.
<wgrant> jtv: But I saw an "any permission on the archive at all" check in the view
<jtv> wgrant: did you get my question about that weird check earlier?
<wgrant> Which weird check?
<lifeless> grah, my graphs, my pretty pretty graphs are bust in daily chrom
<lifeless> but I need daily chromium for speedtool thingy
<wgrant> jtv: The expensive bit of permission checking is determining the destination component of each package.
<lifeless> *fuckers*
<wgrant> lifeless: :(
<jtv> wgrant: the check for "if there is a reason to reject this, check again for Append permission and if the user has it, don't raise an error"
<wgrant> jtv: You mean the thing I reverted last night?
<jtv> wgrant: I guess--but that is exactly what I've been asking for the past hours.  :)
<jtv> I guess my damn connection trouble has been stopping that from getting through.  :( :( :(
<wgrant> I don't see how it's related.
<wgrant> The launchpad.Append special case is now restricted to the Archive.syncSource(s) API methods.
<wgrant> Nowhere near do_copy any more.
<jtv> I was going to explain how it was related, but I needed to start somewhere.
<jtv> (I wonder if my power problems are affecting my connections--UPS is beeping many times a minute)
<wgrant> Hmm, that's not ideal.
<jtv> Anyway:
<jtv> I extracted the permissions check, because I needed to but also because it made the code a bit easier to work with.
<wgrant> You extracted it from CopyChecker?
<jtv> Yes.
<wgrant> To where?
<jtv> To a function right above it.
<wgrant> ie. not a method?
<jtv> No, but it can be if you want it to.
<jtv> Ah no it can't
<wgrant> No, no, just getting a better picture.
<wgrant> Don't worry, my background is not Java :)
<jtv> The problem was that the async case shouldn't be doing the full checks (right?) because it sort of defeats the purpose.
<wgrant> Right.
<jtv> Phew.  :)
<jtv> But I needed the permissions check because of how the jobs worked (and how the jobs worked seemed reasonable to me, apart from potential performance concerns)
<jtv> So I needed it out of the CopyChecker, into a reusable function.
<jtv> (Which by the way would also be unit-testable, hint hint :)
<wgrant> CopyChecker isn't tooooo badly tested.
<jtv> It's not that
<wgrant> I think we may want to keep it on CopyChecker, in a partial check mode, but I guess either works.
<wgrant> Depends on if we might want other pre-creation checks.
<jtv> It's that it's better to have a "check permissions" method with its own unit tests, than to have unit tests on the surrounding method that exercises the crooks and nannies of the permissions-checking part.
<jtv> Nooks and crannies, I mean.
<StevenK> Criminals and grandmothers?
<StevenK> :-)
<jtv> Thank you.
<lifeless> bankers and politicians?
<wgrant> jtv: Sure.
<jtv> Go wash your mouth.
<wgrant> jtv: So that takes a list of sources?
<jtv> wgrant: I believe it's currently one per SPN, but not sure.
 * jtv checks
<jtv> Yes, currently one call per.
<wgrant> It makes no difference at the moment, but we'll need it to take a set of SPNs/SPPHs if we want it to not take forever eventually.
<jtv> But that's easily fixed.
<wgrant> OK.
<wgrant> What are the arguments it takes?
<jtv> Yes, sounds like a solid plan.  It's a small loop anyway, shouldn't be hard.
<lifeless> jtv: oh btw
<lifeless> have you seen load_related?
<lifeless> jtv: it would make one of your helper functions smaller
<jtv> wgrant: person, destination archive/pocket/..., and spn.  But again, it's not set in stone.
<jtv> lifeless: yes, I've seen it thanks.
<jtv> I'm looking forward to applying it.
<wgrant> jtv: The emerging pattern is that all underlying functions/methods like this will take something like (archive, distroseries, pocket, (spns))
<wgrant> jtv: So it deals with SPNs directly, not SPRs or SPPHs?
<jtv> wgrant: then that's a very nice fit indeed.
<lifeless> http://www.zdnet.com.au/telstra-scores-patent-win-over-amazon-339314845.htm
<jtv> wgrant: right, it seemed cleaner to do it that way.
<jtv> Eliminated some unneeded stuff.
<jtv> (spph, spr)
<wgrant> jtv: Great, because that's also the emerging pattern, although it goes against everything we've done for years.
<jtv> It does?
<wgrant> Sets of names works well for most stuff.
<wgrant> But traditional we've passed around single SPRs or SPPHs instead.
<wgrant> +ly
<wgrant> jtv: The idea is that the copier and uploader will both share most of these functions, which will be set-of-name based and very fast.
<wgrant> So we can delete tonnes of code and it will all be much faster too.
<jtv> Great, great.
<jtv> That shouldn't differ between the sync & async cases anyway.
<wgrant> Yeah.
<wgrant> jtv: For now I guess your function will just loop over the SPNs.
<jtv> (I don't usually get upset about this distinction: easier to see where it's a problem and fix it without social upheaval :)
<wgrant> Until we have a verifyUpload that takes a set of names.
<wgrant> Right?
<jtv> Sure, I could make it take a list-of-SPN right away.
<jtv> (set-of-names would be particularly nice for easy testing... I had to write some fakes to get acceptable speed on my tests)
<wgrant> Yeah.
<wgrant> It also works well with lifeless grand plan.
<wgrant> '
<jtv> Given all this, will there still ever be a case where we want to work synchronously in the web request?
<wgrant> If we can get a basic UI scheduled soonish, I don't see why it would ever be required.
<wgrant> That will fix +copy-packages, too.
<jtv> Then maybe the sanest thing for me to do right now would be to get to work on that UI, and later we can rip the synchronous code out of my current branch.
<jtv> (And by "get to work" I mean "start worrying about what the UI should actually do")
<wgrant> I think get your branch fairly sensible now, and discuss the UI and delayed copy murder implementation and scheduling with Julian tonight?
<wgrant> Not much consideration has been given to any of this, but it's not *that* much work.
<StevenK> wgrant: The delayed copy murder implementation is already known.
<wgrant> StevenK: I know you know it.
<wgrant> I don't know that Julian knows all about it.
<wgrant> Does he?
<StevenK> Indeed he does
<wgrant> Excellent.
<jtv> And for me, who is struggling to understand the directly relevant parts, it's an undesirable side track.
<StevenK> We spoke about it at length and he agrees
<StevenK> wgrant: And you know the plan, so I don't need to re-iterate.
<wgrant> Yup.
<jtv> So... what's needed to get my branch fairly sensible now?
<wgrant> It taking a bit of an unfortunate side-track with the overrides generalisation, but that should be better in the long run.
<wgrant> jtv: Well, given that we can't use this in production yet anyway, just make it always async if that's quick?
<jtv> Quickest & most flexible way to do that is to tweak the policy choice.
<NCommander> wgrant: on writing soyuz tests, should I have it use breezy-autotest as the series? (if so, how do I add a new architecture?)
<StevenK> NCommander: Do not use sampledata if you can at all help it
<wgrant> NCommander: You would ideally create a new series using LaunchpadObjectFactory... but if you're using SoyuzTestPublisher then breezy-autotest is a good option.
<StevenK> STP can be taught to respect a new series
<StevenK> It just involves 20 lines of vomit
<NCommander> wgrant: I was planning on recycling STP instead of writing an entire new test_.py file
<wgrant> NCommander: Ideally try to use LaunchpadObjectFactory for everything, though. Can you outline the specifics of the test?
<StevenK> NCommander: Don't recycle STP, either
<NCommander> wgrant: I need a series with two architectures. Upload source package that has an arch all and an arch any bit which has a ${binary:Version} relationship on the Depends. Upload binaries for both architectures. Upload a 1.0-2, only upload arch all, and i386 bits. Test should attempt to see if the arch all packages from 1.0-1 are still published
<StevenK> NCommander: distroseries = self.factory.makeDistroSeries()
<StevenK> das1 = self.factory.makeDistroArchSeries(distroseries=distroseries)
<StevenK> das2 = self.factory.makeDistroArchSeries(distroseries=distroseries)
<StevenK> distroseries.nominatedarchindep = das1
<wgrant> Indeed, I suspect the factory will be better for this.
<jtv> wgrant: of course one misgiving about async is we have no reporting mechanism (definitely not in the UI; don't know what support there is in the data model)
<wgrant> NCommander: You can then use makeBinaryPackageBuild() to create a build, and makeBinaryPackageRelease(build=yourbuild) to create the binaries. Then publish them with makeBinaryPackagePublishingHistory.
<NCommander> wgrant: the existing test_publishing.py uses SoyuzTestFactory, do I need to write a new test py?
<wgrant> jtv: Right, that's my sole misgiving, and the reason I like synchronous for now.
<StevenK> NCommander: Have a look at lib/lp/soyuz/tests/test_initialisedistroseriesjob.py, sub _create_child
<wgrant> NCommander: It probably doesn't belong in test_publishing. But the location doesn't really matter for now.
<StevenK> NCommander: That creates a new series and sets STP up to use it
<wgrant> NCommander: Just write a new test class with a single test case.
<wgrant> Then we can move that to wherever.
<NCommander> StevenK: holy crap, my eyes, they bleed!
<jtv> wgrant: so I was thinking the numeric threshold ain't so bad: we can tweak it to be ridiculously high (always synchronous) or ridiculously low (always asynchronous) and maintain our freedom of choice while we may possibly still want it.
<NCommander> (thanks)
<StevenK> NCommander: Frankly, that test is one of the better ones
<wgrant> jtv: I guess. But once we have at least a basic async status reporting UI (hopefully very soon!) we can destroy sync.
<NCommander> StevenK: no, just Soyuz's API in general
<StevenK> NCommander: Most of that test is using LaunchpadObjectFactory
<StevenK> So, uh, no.
 * NCommander eats shoe
<NCommander> k
<StevenK> s/test/function/
<jtv> wgrant: absolutely.  Given though that it's the -- what comes below known-good?
<jtv> wgrant: Given though that it's the devil we know, and that we have code that accommodates it, keeping that as an option seems an easy choice.
<jtv> For now.
<StevenK> NCommander: Hold on, I found a better example.
<jtv> wgrant: I guess the ideal migration path would be: go async only for cases that currently don't work at all, then implement async UI, then drop sync.
<StevenK> NCommander: It sets up two DASes and then the STP. lib/lp/soyuz/tests/test_build_set.py , function setUp()
<wgrant> jtv: That's the path that I originally intended. But lifeless says to drop sync then implement async UI.
<wgrant> Which is the best solution, if the UI is coming before the feature finishes.
<jtv> wgrant: Also sensible, but at the moment, "drop sync" is work and would merely delay the UI work.  So I'd say: let's get this branch reviewed, and start defining the UI.
<wgrant> +1
<NCommander> StevenK: looks straightforward enough
<StevenK> NCommander: Yes, it's an awesome amount of boilerplate
<jtv> Thanks wgrant!  Maybe lifeless can help me figure out the UI reqs.
<jtv> Uh-oh.
<jtv> wgrant: I don't suppose a do_copy that fails for whatever reason _beyond_ CannotCopy leaves any kind of paper trail?
<wgrant> jtv: In the DB?
<wgrant> No.
<jtv> That is sad news, because neither does the job type.
<wgrant> Hmm, that sounds suboptimal.
<wgrant> We may need to fix that.
<lifeless> 'may' :>
<jtv> So the only way I see to report failure or indeed progress is to look for jobs that would apply to a given package (there may be several, and finding them is nontrivial) and inspect their status.
<wgrant> jtv: So, I'd imagined that there would be a job for each package, so they'd be somewhat queryable.
<wgrant> For the UI case, it can remember the job ID, perhaps?
<jtv> Eh eh eh such dreams
<jtv> I suppose Ajax-y UI could remember job IDs, yes.
<jtv> (AIUI there may be multiple jobs, for copies from different source archives)
<wgrant> Ah, those are separate jobs?
<jtv> Yes.
<wgrant> I thought the UI would just get a job ID back for each source, and then request the status for all those IDs regularly.
<wgrant> Hard to do progress or failure counts if it's all one or two jobs :(
<wgrant> Hard to do failure info, too.
<jtv> Progress is in some ways the easier one.
<jtv> The jobs should benefit, speed-wise, from doing multiple packages in one go I guess.  Progress we can measure by how many jobs there are ahead of us in the queue.
<wgrant> They will benefit significantly, yes.
<wgrant> But it seems like that's more of something for the job runner to do.
<wgrant> Grab jobs that can be done together, and do them together.
<NCommander> setUp(self) is called by the test harnass automatically, correct?
<jtv> wgrant: tempting discussion, but rabbit hole.  :)
<StevenK> NCommander: Yes
<jtv> wgrant: Fairness, starvation, pathological cases, db cost vs. actual job work...  what we have now ain't so bad I guess.  :)
<jml> lifeless: hi
<jml> lifeless: want to /join #ubuntu -uds-mikszath?
<wgrant> jtv: Yeah, but UI will be interesting, and just returning failure info at all :/
<lifeless> jml: coming
<jml> lifeless: thanks.
<jelmer> wallyworld: hi :)
<jelmer> wallyworld: I think you still have my badge :)
<wallyworld> jelmer: hello
<wallyworld> jelmer: oh yeah. where are you now?
<wgrant> Down to exactly three pages of criticals!
<jtv> StevenK: maybe you would like to review, or at least comment on, my sync/async branch?  Yes, it needs more.  But I believe it's a step forward.
<jtv> wgrant: ctrl-+
<StevenK> jtv: Do I have to? I'd like to finish this policy before EOD
<jtv> StevenK: no, just asking if you'd like to.
<StevenK> jtv: Then I'll respectly decline
<jtv> No worries.
<jelmer_> wallyworld, I'm in the bug triaging session, mikszath
<jelmer_> wallyworld, there's no hurry, I can find you afterwards. I was just worried you would fly back to Brisbane with it. :)
<wallyworld> jelmer: np. i'll find you
<wgrant> lifeless: http://webnumbr.com/launchpad-critical-bugs and http://webnumbr.com/launchpad-oops-bugs need restarting. Looks like they failed like a month ago.
<wgrant> lifeless: https://lp-oops.canonical.com/oops.py/?oopsid=OOPS-1956CH112
<wgrant> Copy checked and source copied in 300ms, then it goes and spends the next 9 seconds dealing with strucsubs :(
<StevenK> Oh, bleh
<wgrant> StevenK: ?
<StevenK> wgrant: Your last comment
<wgrant> Oh, right.
<wgrant> Only eight bugs. Hm.
<wgrant> Six, in fact. One is referenced three times...
<adeuring> good morning
<jtv> hi adeuring
<adeuring> hi jtv!
<wgrant> lifeless: BugMessageSet.createMessage currently calculates the index by counting the number of messages... any reason we can't take the max index and add one?
<lifeless> +1
<wgrant> It's much faster.
<wgrant> The existing one can take tens of ms :/
<lifeless> sorry, in 'redo lp bugs entirely for ubuntu'
<wgrant> Oh *awesome*.
<wgrant> Interestingly it's actually slightly faster still to ORDER BY index DESC and take the index of the first row.
<wgrant> Chromium, why are you using 4GiB of RAM?
<StevenK> Hah
<lifeless> wgrant: also bug 424671
<_mup_> Bug #424671: attachments: accessing attachment's message is very slow <api> <lp-foundations> <performance> <Launchpad itself:Triaged by leonardr> < https://launchpad.net/bugs/424671 >
<lifeless> wgrant: the numbrs are broken
<lifeless> spiv: do you have working webnumbr voodoo for lp bug searches?
<spiv> lifeless: http://webnumbr.com/profile?name=https%3A%2F%2Flaunchpad.net%2F~spiv is what I've got
<lifeless> wgrant: its xpat - substring-after(//div[@id='maincontent']/div/div[2]/div/div/div/div[1]/table//tr/td[1], 'of')
<lifeless> substring-after(//div[@id='maincontent']/div/div[3]/div/div[1]/div/div[1]/table//tr/td[1], 'of')
<lifeless> so, 2->3
<lifeless> no, deeper
<lifeless> copying
<bigjools> morning
<jtv> moaning bigjools!
<lifeless> wgrant: http://webnumbr.com/.join(launchpad-oops-bugs.all,launchpad-timeout-bugs.all,launchpad-critical-bugs.all)
<lifeless> ok, I fail at search
<lifeless> I'm trying to find the (long) analysis I wrote about our bug task UI/model
<lifeless> cake credits to anyone that finds it
<mrevell> Hello!
<bigjools> wassup jtv
<jtv> bigjools: where do I start :)
<jtv> Hi mrevell
<jtv> bigjools: currently trying to figure out what UI is needed (and reasonably implementable) for async package copying.
<LPCIBot> Project devel build #709: FAILURE in 5 hr 33 min: https://lpci.wedontsleep.org/job/devel/709/
<StevenK> Hmph
<bigjools> jtv: ah yes we need to talk about that
<lifeless> wgrant: it used the count because the indices hadn't been populated
<StevenK> lifeless: ^ Rabbit fail
<bigjools> jtv: I previously chatted to allenap about the relevant page polling for copy jobs
<jtv> Ah, that ties right in
<bigjools> jtv: then the notifications can work mostly as they do now, just asynchronously
<bigjools> but we need the addition of a "job in progress" one
<lifeless> StevenK: win ip-172-56-124-252: nxdomain
<jtv> bigjools: my mental image of this atm involves JS on the page knowing what job ids it has in progress, and seeing where the last of those is in the queue.
<bigjools> jtv: well it can poll for all copy jobs for that series, there may be more than one
<bigjools> but yes
<jtv> Yes, but probably not many.
<jtv> One per source archive, basically.
<StevenK> lifeless: If rabbit is attempting to look up it's local hostname, then that's just stupid
<bigjools> jtv: well it depends how many archive admins are using it at once
<lifeless> StevenK: rabbit attempts to look up its local hostname
<jtv> StevenK: I believe that's what broke shutdown on maverick.
<lifeless> StevenK: thats what made maverick fail to shutdown, because not-manager went away too fast
<jtv> lifeless, StevenK: fwiw my filesystem hasn't corrupted itself once since I upgraded to natty!
<lifeless> jtv: congrats
<lifeless> !
<jtv> bigjools: there's another thing... it's hard to find other jobs that may be pending for the same copies.  I'm hoping repeated copies will be no-ops.
<bigjools> jtv: yes, the copy checker will fail the second one
<StevenK> lifeless: So that's why it fails to start? Er, can we not have it do that.
<bigjools> so not a no-op
<jtv> bigjools: fail it?  That seems a bit harsh, esp. if it's perfectly reasonable for multiple admins to request the same copy concurrently.
<wgrant> lifeless: That's remarkably constant progress we've made over the last month!
<lifeless> wgrant: it is!
<lifeless> StevenK: I don't know. patches appreciated, or perhaps put localhost first in the hosts file?
<bigjools> jtv: well possibly in the future, but right now it'll fail
<jtv> Which won't do much except generate an oops, I guess, after which a maintenance squad will pick it up very quickly.
<StevenK> ubuntu@ip-172-56-124-250:~$ cat /etc/hosts
<StevenK> 127.0.0.1 localhost
<StevenK> lifeless: ^
<jtv> bigjools: right now my main concern is that it be fast, regardless of whether it ends in front of the restaurant or partway through its outside wall.  :)
<jtv> Car analogy ^
<bigjools> jtv: it'll be fast!
<StevenK> lifeless: (There's more, but ... :-)
<jtv> bigjools: then sod the brakes :)
<bigjools> jtv: no oops, just a failed job, which we need to show on the page
<lifeless> StevenK: ><
<StevenK> lifeless: Hm?
<lifeless> StevenK: I'm unhappy that this -long- overdue patch has broken jenkins
<StevenK> lifeless: I can run hostname localhost on builder bring-up, but ew
* danilos changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: jtv, danilos | https://code.launchpad.net/launchpad-project/+activereviews
<jtv> hi danilos
<danilos> jtv, hi :)
<danilos> jtv, and the answer is "no"
<jtv> damn, beat me to it!
<jtv> The other OCR sure ain't gonna do it
<danilos> jtv, sure thing, send it my way
<jtv> :)
<danilos> 780319?
<jtv> danilos: thanks... it's largish: https://code.launchpad.net/~jtv/launchpad/bug-780319/+merge/60571
<jtv> Yup
<wgrant> bigjools: Is DF's appserver meant to be not running?
<jtv> Again!?
<bigjools> wgrant: last I checked it was ok
 * bigjools looks at log
 * bigjools fixes
<bigjools> nohup: cannot run command `bin/run': No such file or directory
<bigjools> awsum :)
<wgrant> Handy.
<bigjools> grar, how do I stop this PoS mumble from starting the config wizard every time I start it
<stub> jtv: Is the only usage of multitablecopy copying translations into a newly opened distro?
<jtv> stub: afaics yes
<jtv> (did a quick grep)
<stub> jtv: I guess we can always recover from that job if things explode - we have done it a few times already.
<jtv> And actually it's no longer even copying the translations themselves.
<jtv> At the time it was a huge enough problem that the overhead of making this big a generic module wasn't a big deal.  But I guess it hasn't been a smash success.
<jtv> otp
<StevenK> lifeless: So starting rabbit gives: Crash dump was written to: erl_crash.dump
<NCommander> Grumble, I'm getting a ComponentLookupError when I try to do         self.admin = getUtility(IPersonSet).getByEmail(ADMIN_EMAIL)
<wgrant> NCommander: Does your test class have a layer set?
<wgrant> NCommander: Try LaunchpadFunctionalLayer.
<danilos> jtv, fwiw, you do already have conflicts :)
<jtv> again!?
<jtv> danilos: otp now, will fix soonest
<danilos> jtv, oh, perhaps they are just in the email
<danilos> can't see them on the MP, all good then
<jtv> Phew!
<StevenK> lifeless: So help to fix rabbit would be awesome, but I personally am *utterly* unimpressed with it.
<NCommander> wgrant: ah, didn't know that was used by anything
<NCommander> wgrant: now I've gotten librarian traceback errors. Obviously something is unhappy with my testing setup
<adeuring> stub, lifeless: could you please have a look at my mp: https://code.launchpad.net/~adeuring/launchpad/bug-739075/+merge/60516 ?
<wgrant> NCommander: Which layer did you use?
<wgrant> And what's the traceback?
<lifeless> jml: https://bugs.launchpad.net/launchpad/+bug/655385/comments/14
<_mup_> Bug #655385: Allow bug status change from Triaged only for bug supervisor <accesscontrol> <acl> <bugs> <lp-bugs> <Launchpad itself:Opinion> < https://launchpad.net/bugs/655385 >
<lifeless> jml: is the analysis I was looking for
<lifeless> jml: I segue from the bug itself quite sharply
<jml> lifeless: right, thanks.
<lifeless> jml: I've mailed you and skaet
<lifeless> should I have cc'd ali/
<lifeless> ?
<jml> lifeless: probably ok for just skaet & I atm. Have linked it from IssueTracker as well.
<lifeless> cool
<lifeless> jml: do you need me for anything? I might go remind my self what my wife looks like...
<jml> lifeless: no, I think we're good.
<jml> lifeless: thanks for staying around
<lifeless> kk, my pleasure
<danilos> jtv, r=me with some minor comments; general impression is that it's mostly refactorings, and that's even easier to review as two separate branches :)
<danilos> oh well
<LPCIBot> Project windmill-db-devel build #263: STILL FAILING in 1 hr 7 min: https://lpci.wedontsleep.org/job/windmill-db-devel/263/
<danilos> <-- jtv has quit (Read error: Connection reset by peer)
<danilos> <danilos> jtv, r=me with some minor comments; general impression is that it's mostly refactorings, and that's even easier to review as two separate branches :)
<stub> adeuring: reviewed
<adeuring> stub: thanks!
<stub> was on phone
<stub> jtv: https://code.launchpad.net/~stub/launchpad/bug-179821-lowercase-tablenames/+merge/60511 should be easy. https://code.launchpad.net/~stub/launchpad/bug-778338-transient-schema/+merge/60512 is a further uglification of your code.
<jtv> stub: are you asking for reviews?
<stub> jtv: If you want to. Otherwise I can grab an ocr.
<jtv> stub: check again.
<stub> oic. Yes, you can review it then :)
<stub> I was just giving you first right of refusal since this was originally your work.
<stub> The second one went ugly, as "foo.bar" is a table in the public schema, unlike "foo"."bar"
<stub> Either that, or refactor most of canonical.database.postgresql
<jtv> stub: as far as the first one is concerned, there's a quandary: we're probably supposed to quote variable table names, which is exactly what leads to the case sensitivity.  If only there were a way to escape, but not quote them!
<stub> Indeed. I think at one stage I had an assert to ensure the name didn't need to be quoted. Think I lost that somewhere.
<jtv> stub: I'm not sure it's worth migrating existing tables...  with single-master and just a few production-like environments, wouldn't it be easier to check for any existing copying tables?
<stub> Oh... we are still quoting table names. I'm just forcing them to lowercase.
<jtv> Yes.
<jtv> Which should work fine, as long as you don't run anything in a Turkish locale.
<stub> jtv: I don't want the rollout aborted because there was a job running at the time.
<jtv> It's a manual job, and the one for Oneiric has already been run.
<stub> ok. so if we promise not to run another one in the near future, I can land this on launchpad/devel then.
<stub> Although there is little point landing this code until before the next time one of these jobs is run... so... whatever :)
<wgrant> stub: The less we have in db-devel the better.
<stub> jtv: I think lower() is fine and we can assume nobody is going to stick Turkish or worse in here. I think it is either lower(), or generate the tablename using punycode or sha1 hash.
<jtv> stub: sure, I was just being flippant.
<stub> Its a valid concern. At the moment, no callsites of this code will get non-ascii user input.
<jtv> (And since I'm still feeling a _little bit_ flippant, let me point out that "I".lower() != "i" in Turkish.  :)
<stub> I think myself and all the losas are all native english speakers so that is fine :)
<jtv> I think it's fine to have strong requirements for this.  I've forgotten most of the details but maybe you could simply require lower-case strings and fail if that requirement is not met?
<stub> sure. I'll have callsites to modify then though? Many?
<NCommander> wg  /win 24
<NCommander> bah
<NCommander> wgrant: http://paste.ubuntu.com/606078/
<stub> The multitable tests are all lowercase
<wgrant> NCommander: You're not using LP_PERSISTENT_TEST_SERVICES?
<wgrant> (that's an environment variable... it should be unset these days)
<NCommander> wgrant: its not set
<wgrant> :(
<wgrant> No pids floating around in /var/tmp?
<NCommander> wgrant: no love
<jtv> stub: sorry, food arrived...  no, I don't think there'll be many callsites.
<wgrant> NCommander: Can you start a dev instance OK?
<NCommander> wgrant: works fine.
<NCommander> correction
<NCommander> I'm OOPSIng
<stub> I suspect the Librarian won't be starting.
<NCommander> now the question is why isn't it
* jtv changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: danilos | https://code.launchpad.net/launchpad-project/+activereviews
<stub> NCommander: Got a librarian log file in logs directory?
<NCommander> yeah, looks like librarian doesn't want to start, but I can't figure out why
<stub> Might have a juicy traceback for you
<NCommander> where do I find that?
<stub> I think logs directory, top of the lp tree.
<wgrant> Otherwise there could be something interesting in /var/tmp
<NCommander> don't have any librarian logs
<stub> Try bin/start_librarian
<NCommander> bah, I'm going to reboot
<wgrant> bigjools: You (or you squad) have a lucid_db_lp failure
<wgrant> (<DistroSeries u'distroseries344950'>, 'getDifferencesTo', 'launchpad.Edit')
<NCommander> Ok, I rebooted
<NCommander> launchpad.dev works, my test still breaks (Connection refused is the error now)
<wgrant> That's more encouraging.
<NCommander> wgrant: http://paste.ubuntu.com/606086/
<wgrant> Can you try attaching adding a bug attachment on launchpad.dev, just to test that it's really working?
<NCommander> can't login
<NCommander> grumble
<wgrant> Your /etc/hosts might be old.
<wgrant> Does testopenid.dev resolve?
<NCommander> 2011-05-11 03:25:07-0700 [-] twisted.web.server.Site starting on 58085
<NCommander> 2011-05-11 03:25:07-0700 [-] Starting factory <twisted.web.server.Site instance at 0x383c3b0>
<NCommander> 2011-05-11 03:25:07-0700 [-] Not using upstream librarian
<NCommander> 2011-05-11 03:25:07-0700 [-] daemon ready!
<NCommander> wgrant: yes, it sresolves
<NCommander> I can' tlogin cause it oops'ed
<NCommander>  3745 pts/1    Sl+    0:09 /usr/bin/python2.6 -S /home/mcasadevall/launchpad/lp-branches/arch-indep-skew-fix/bin/twistd --no_save --nodaemon --python /home/mcasadevall/launchpad/lp-branches/arch-indep-skew-fix/daemons/librarian.tac --pidfile /var/tmp/development-librarian.pid --prefix Librarian --logfile librarian.log
<NCommander> I've got librarian running, nothing interesting in the log
<wgrant> What was the OOPS?
<NCommander> OOPS-1957X7
<wgrant> That refers to /var/tmp/lperr/2011-05-11/*X7, so it's not very helpful for me :(
<NCommander> http://paste.ubuntu.com/606088/
<NCommander> wgrant: ^
<wgrant> What.
<NCommander> wgrant: that's what that file has
<wgrant> Do you have a broken proxy set or something?
<NCommander> don't think so, this was workign fine ...
<wgrant> Or a broken apache or something.
<NCommander> grumble
<wgrant> NCommander: AFAICT that should be grabbing a page from blog.launchpad.net.
 * NCommander make stop/make clean/make run's
<jtv> lifeless: it seems great minds think alike.
<NCommander> this is really irritating me :-/
 * NCommander probably should throw LP into a VM and save myself some frustation
<LPCIBot> Project windmill-db-devel build #264: STILL FAILING in 1 hr 8 min: https://lpci.wedontsleep.org/job/windmill-db-devel/264/
<gary_poster> hi jml.  When you get a chance (no rush), I'd like your review of my non-high triaging of bug 780568.  All the other new high/critical bugs I agree need to be addressed before we move on.
<_mup_> Bug #780568: bug structural subscription auto tag complete <story-better-bug-notification> <Launchpad itself:Triaged> < https://launchpad.net/bugs/780568 >
<jml> gary_poster: yeah. that's fair enough.
<gary_poster> coolcool thanks
<jml> how do I demo the unsubscribe-in-anger feature?
<jml> all I need is a screenshot, actually
<deryck> Morning, all.
<danilos> jml, click on "Edit bug mail" link on any bug page
<bigjools> wgrant:  can you remember if someone fixed a conflict on db-devel just before buildbot whinged?
<danilos> jml, (that page is going to be linked from all emails, but it was a mess to make the link conditional in the email template so we left it for when we go fully public)
<wgrant> bigjools: There was no conflict.
<bigjools> hmm
<bigjools> wgrant: it's rvb's API change, not quite sure why it put it in EditRestricted.... and how it passed ec2
<wgrant> bigjools: It's not related to the deriveDistroSeries permission change?
<wgrant> In devel?
<wgrant> The API changes landed days ago.
<bigjools> db-devel
<wgrant> In db-devel, yes.
<wgrant> Then deriveDistroSeries permission changes landed yesterday in devel.
<bigjools> why would my change affect that?  I only moved deriveDistroSeries
<wgrant> And would have been merged into db-devel today...
<wgrant> Hmm.
<wgrant> Have you bisected?
 * bigjools looks at self
 * bigjools is still in once piece
<wgrant> Sounds painful.
<wgrant> It is 11pm, so it's your problem :)
<bigjools> gee thanks
<bigjools> :)
<danilos> bigjools, I can help bisect you ;)
<bigjools> danilos: no I am not rooming with you again
<danilos> bigjools, a shame, a shame :)
<bigjools> :)
<deryck> henninge, ping for standup
<wgrant> bigjools: Any luck with the conflict?
<bigjools> what conflict?
<wgrant> Test failure, whatever it is these days.
<bigjools> just sent a fix in
<wgrant> Great.
<bigjools> I have *no* idea how that *ever* passed *any* buildbot/ec2 run
* danilos changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: - | https://code.launchpad.net/launchpad-project/+activereviews
<jml> gmb: I'm doing a lightning talk tomorrow on the bug subscription stuff. I'm trying to get a screenshot of the unsubscribe-in-anger stuff
<danilos> jml, did you see my suggestions above?
<jml> danilos: no, I didn't.
<jml> sorry
<jml> my IRC backlog spew is not working: (
<gmb> jml: $bug/+subscriptions will give you the UIA page for any bug.
<danilos> <danilos> jml, click on "Edit bug mail" link on any bug page
<danilos> <danilos> jml, (that page is going to be linked from all emails, but it was a mess to make the link conditional in the email template so we left it for when we go fully public)
<gmb> (Or that)
<jml> Thanks :)
<jml> also, how do I remove the product team from the yellow team
<danilos> jml, you may want to have a suitable set of subscriptions through teams, to duplicates and structural subscriptions
<jml> this Launchpad thing can be very confusing at times
<danilos> jml, yeah, you should talk to our strategist about that :)
<jml> danilos: he says they know about it and would love to work on it but don't have the capacity right now.
<danilos> jml, anyway, deactivated the product team in ~yellow
<jml> danilos: thanks.
<danilos> :)
<jml> https://launchpad.net/~yellow/+mugshots still shows my mug :(
<danilos> jml, sounds like a different bug :(
<wgrant> jml: That page is memcached. We can turn that off with an FF.
<gmb> Jeez, I still have that potato photo up.
<gmb> Mind you, could be worse
<mwhudson> it also says, results 1-5 of 5 and then shows 11 photos
<gmb> Heh. Nice.
<danilos> gmb, heh, I had photo set by kiko originally to something weird... forcing me to change it something like 18 months later
<gmb> "There are probably at least five people in this team"
<danilos> mwhudson, so obviously, a portlet is memcached only
<danilos> or a page snippet, making it all the more fun
<mwhudson> danilos: yeah
<henninge> abentley: two gestions about API use:
<wgrant> https://launchpad.net/~yellow/+mugshots?nocachethanks is a bit more sensible :)
<henninge> abentley: how do I switch launchpadlib to use 'devel' instead of 1.0?
<jml> wgrant: thanks. I didn't know about that trick.
<henninge> abentley: how do I get to the HTML representation of an object, as mentioned in the bug description?
<abentley> henninge: You specify the root URL to your initial client call.
<wgrant> Of course now *that*'s cached.
<jml> also, anyone got spare time for gravatar integration?
<gmb> Heh. The three big problems, eh.
<wgrant> henninge, abentley: Give Launchpad.login_with the kwarg version='devel'.
<gmb> jml: +many
<abentley> henninge: my bad.
<henninge> abentley: no worries
<henninge> wgrant: thanks
<wgrant> Hacking the URL was right until Lucid.
<abentley> henninge: The HTML representation is just the web service representation, as a list of javascript variables.
<wgrant> Objects can define custom HTML representations.
<henninge> abentley: what I get when I call the URL in a browser, right?
<abentley> wgrant: you mean, the JSON representation provided by the LP.cache can be overridden?
<abentley> henninge: No, I don't know what you mean.
<wgrant> abentley: The JSON representation is distinct from the HTML representation.
<abentley> wgrant: So this is an additional representation provided by the web service on top of JSON and XML representations?
<wgrant> abentley: There are two types of representation that lazr.restful can give you: JSON and XHTML. You can also ask it to give you the XHTML inside the rest of the JSON.
<wgrant> abentley: The default XHTML representation is just a list of attributes and their values, but interfaces can declare adapters to give custom representations.
<wgrant> https://api.launchpad.net/devel/launchpad is the default.
<wgrant> https://api.launchpad.net/devel/~wgrant is a custom one.
<abentley> wgrant: Okay.  So I guess henninge needs to know how you request that representation using launchpadlib.
<wgrant> NFI
<wgrant> You can override it with ws.accept
<wgrant> But not sure how to tell launchpadlib about that.
<henninge> abentley, wgrant: that's fine for now. Thanks
<wgrant> eg. https://api.launchpad.net/devel/~wgrant?ws.accept=application/json
<abentley> henninge: The JSON representation ends up in the LP.cache through lib/lp/app/templates/base-layout-macros.pt:174
<henninge> abentley: thanks
<wgrant> You add stuff to that using IJSONRequestCache.
<wgrant> Do not attempt to serialise JSON manually.
<wgrant> Or I will come and get you :)
<abentley> wgrant: he's fixing bug 740208
<wgrant> Ah.
<wgrant> That's a fairly difficult one.
<henninge> I always pick those it seems ...
<wgrant> I have very little idea of how to fix it sanely.
<wgrant> Apart from slapping people who think it's necessary.
<wgrant> But that's less than productive :(
<bigjools> wgrant, the voice of reason
<mwhudson> i like wgrant's proposed solution
<aalex-sat> hello
<aalex-sat> eh I could work for Canonical, since I know Twisted. :) I didn't know that you guys were using it a lot.
<aalex-sat> I'm currently working on something much similar to Ubuntu One. Maybe I shouldn't reinvent the wheel.
* abentley changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: abentley | https://code.launchpad.net/launchpad-project/+activereviews
<abentley> deryck: I'm OCR, so could you please review https://code.launchpad.net/~abentley/launchpad/safe-branch-upgrade/+merge/60634 ?
<deryck> abentley, definitely.
<abentley> deryck: thanks.'
<sinzui> jcsackett: any commercial admin can increase an archive size. I am one
 * sinzui looks at question
<bigjools> sinzui: we need to fix that.  commercial admins have far too many powers over PPAs
<wgrant> bigjools: I guess we probably want to grant the OEM namespace an entitlement for unlimited private PPAs and quotas.
<wgrant> Then we can remove commercial admins.
<wgrant> I suppose.
<bigjools> not quite
<sinzui> bigjools: I think ~registry needs to change the archive size. That is all I want to support for the whole team
<bigjools> we need a PPA admin celeb
<wgrant> Why?
<bigjools> the celeb will be able to do what commercial can currently do to PPAs and we restrict it better.  We then need to come up with a way for certain people to self-service within some boundary
<deryck> abentley, looks great.  r=me.
<abentley> deryck: thanks.
<deryck> abentley, np!
<deryck> benji, I'm trying to qa my fix from yesterday that you reviewed... the click unsubscribe link doesn't remove name issue....
<deryck> benji, but I wrote this, not being mindful of the feature flag, and can't qa against the unsubscribe link myself...
<deryck> benji, but I notice the "Edit subscription" goes to loading when you're subscribed via a dupe and never completes.
<deryck> benji, wondering if this is known or something I introduced?
<benji> deryck: that doesn't ring a bell, let me look at the full bug list real quick
<benji> deryck: I don't see anything on https://bugs.edge.launchpad.net/launchpad-project/+bugs?field.tag=story-better-bug-notification that sounds like that, so either we haven't noticed or you did it ;)
<deryck> benji, ok, thanks
<benji> sinzui: since you've been on this click-to-close message box journey with me from the begining, would you like to look at the final (I hope) revision? https://code.launchpad.net/~benji/launchpad/bug-779538/+merge/60650
<deryck> benji, but is the expected behavior that I should be subscribed via a duplicate and click "edit subscription" and be able to drop my subscription via a dupe that way?
<benji> deryck: that I don't know
<deryck> benji, ok, thanks again
<deryck> benji, I filed bug 781245 about what we talked about.  I'll poke a bit more to see if its from me or not.
<_mup_> Bug #781245: Clicking "Edit subscription" when subscribed via a duplicate gets stuck at "Loading..." <story-better-bug-notification> <Launchpad itself:Triaged> < https://launchpad.net/bugs/781245 >
<LPCIBot> Project db-devel build #538: FAILURE in 5 hr 11 min: https://lpci.wedontsleep.org/job/db-devel/538/
<benji> abentley: do you have a minute to review https://code.launchpad.net/~benji/launchpad/bug-779538/+merge/60650 for me?
<benji> I think the MP has all the background you need, but feel free to ask if you have any questions.
<abentley> benji: okay.
<abentley> benji: Why not just tweak the delegate callback to only affect the correct link?
<benji> abentley: in the former approach there wasn't a link, the "Hide X" "link" was just CSS style, the on-click actually happened on the message box itself, that meant that clicking on a link, or selecting text, or control-/shift-clicking on a link would trigger the on-click
<benji> this way there's a real DOM element to hook an on-click up to so we don't react to all those stray events
<abentley> benji: So normally, we'd just hook this up when the DOM is ready.  I assume the notifications are created after DOMReady?
<benji> abentley: they can be; for example the new structural subscription bits create message boxes after doing AJAX requests
<abentley> benji: So could the javascript that creates the notification also attach the handler?
<benji> abentley: it could, but that would mean we have to remember every time; sounds easy to forget
<abentley> benji: Really?  There's no common code to create a notification?
<benji> not that I'm aware of; all the ones I know of are done server-side; if there is, then the fact that we created ours directly (Y.Node.create) suggests to me that others will do the same
<abentley> benji: Having everyone create their notifications using Y.Node.create sounds like something we should discourage.
<abentley> benji: instead of polling, what about a "contentready" event handler?
<benji> abentley: I wasn't aware that contentready would fire when new elements were added to the DOM
<abentley> benji: I don't know for sure, but it seems likely there's a way to detect new elements being added to the DOM.
<benji> re. Y.Node.create: <shrug> It seems like embracing our medium to me.
<abentley> benji: It seems like it would lead to discrepancies and confusion to me.
<abentley> benji: Notifications have a higher-level meaning to us.  They're not just HTML.
<benji> threre's the DOMNodeInserted event, but it's still in draft and slightly older Firefoxes don't support it, and I'd be afraid of an event storm on our larger pages
<benji> well, at this point I've exhausted my timebox, so I'll instead remove the current, broken click-to-close behavior
<sinzui> benji: I agree they are not HTML, but we have several bug open that indicate that developers think they are HTML. Many pages show object state as a notification: bug-converted-to-question, product-deactivated, package-not-published.
<sinzui> benji: I think reverting is a fair decision. The only case I want to close a notification is the broken cases above. I know the notification will be gone if I reload
<LPCIBot> Project windmill-db-devel build #265: STILL FAILING in 44 min: https://lpci.wedontsleep.org/job/windmill-db-devel/265/
<benji> abentley: here's the remove-click-to-close MP: https://code.launchpad.net/~benji/launchpad/remove-click-to-close/+merge/60673
<abentley> benji: It's not true that it was deemed unsuitable in review.  You terminated review before I reached a conclusion.
<benji> abentley: I was thinking more of my conversation with sinzui.
<benji> I didn't intend to cut the review short, but I don't see how we would have ended up landing the branch (more or less) as-is.
<abentley> benji: r=me.
<benji> thanks
<abentley> benji: It's important to avoid polling unless there is no reasonable alternative.  I was trying to find out whether the alternatives had been exhausted.
<benji> yep, absolutely understandable
<abentley> If I believed that there was no reasonable alternative, I would have been willing to approve a polling-based solution.
<deryck> abentley, hi.  I have an easy one for review, if you're available.
<abentley> deryck: sure.
<deryck> abentley, thanks!  https://code.launchpad.net/~deryck/launchpad/edit-subscription-always-loading-781245/+merge/60683
<abentley> deryck: r=me
<deryck> abentley, awesome, thanks!
<lifeless> gary_poster: hi
<gary_poster> lifeless, hey
<abentley> deryck: chat?
<deryck> abentley: sure.
<lifeless> gary_poster: wondering if you wanted to chat about the services draft
<deryck> abentley: mumble or here?
<abentley> deryck: mumble
<deryck> ok
<gary_poster> lifeless, it looks like it has grown further since my last skim.  Lemme finish what I'm up to, then I'll read again and more carefully.  I'm out in 45 min (and trying to be strict about it because of the baby).  I'll touch base with you in 30 min and let you know if I have any immediate thoughts, and if I think I have anything worth talking about over a longer time.
<lifeless> gary_poster: thats great
<lifeless> sinzui: hi
<lifeless> sinzui: I'm trying to do what bug 781326 asks for but
<_mup_> Bug #781326: Please remove erroneous LibreOffice Launchpad project website https://launchpad.net/libreoffice <Launchpad itself:New> < https://launchpad.net/bugs/781326 >
<lifeless> it had a series linked to sid (the df-libreoffice has its series linked to natty and oneiric
<lifeless> so I unlinked and relinked
<lifeless> the https://launchpad.net/libreoffice/+admin form though is whinging that there is a series
<lifeless> sorry, a package link
<lifeless> which I just deleted :(
<lifeless> sinzui: any thoughts on how to address ?
* abentley changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: - | https://code.launchpad.net/launchpad-project/+activereviews
<lifeless> gary_poster: hi
<lifeless> gary_poster: you're about to run; thought i would ping a touch-base before then
<gary_poster> lifeless: hi, I didn't forget/ :-) I'm reading and making notes as I go.  I think I do have some thoughts, though I don't know how helpful they are, since they are so casual
<lifeless> gary_poster: cool! I had no doubts :)
<lifeless> gary_poster: if you can email me what you have when you're done; I'll incorporate them, and then start wider distribution of this for discussion
<gary_poster> lifeless, will do.  they are of this sort--questions, casual thoughts, and so on.  http://pastebin.ubuntu.com/606283/  Maybe they will be more substantial when done. :-)
<lifeless> thats good stuff
<gary_poster> ok, I'll keep going tomorrow and send it your way then lifeless.
<gary_poster> have a great day
<NCommander> wgrant: so I've been starring at Soyuz, and I'm kinda lost on using the factory, I get that SoyuzTestPublisher does things like setup BreezyAutoTest, and it seems its got enough methods to at least get my initial uploads in, publish them, etc.
<NCommander> I'm very confused on where the Factory comes in, or why test_build_set has a lot of things that seem ignored by SoyuzTestPublisher ...
#launchpad-dev 2011-05-12
<LPCIBot> Yippie, build fixed!
<LPCIBot> Project db-devel build #539: FIXED in 5 hr 15 min: https://lpci.wedontsleep.org/job/db-devel/539/
<LPCIBot> Project windmill-db-devel build #266: STILL FAILING in 2 min 0 sec: https://lpci.wedontsleep.org/job/windmill-db-devel/266/
<wgrant> NCommander: SoyuzTestPublisher is deprecated in favour of the factory.
<LPCIBot> Project db-devel build #540: FAILURE in 2 min 1 sec: https://lpci.wedontsleep.org/job/db-devel/540/
<wgrant> Hmm, db-devel is fairly borked.
<lifeless> 417 /   66  CodeImportSchedulerApplication:CodeImportSchedulerAPI
<lifeless> hmm
<wgrant> Possibly related to the network glitches last night.
<wgrant> Not sure if we know exactly what happened yet.
<lifeless> network glitches?
<lifeless> grrr tsearch2 indices are painful
<wgrant> We had unexplained high queue depth and 502s, and other rather suspicious unexplained OOPSes. There was suggestion that unknown network issues were the culprit.
<lifeless> thats the second time in a fortnight
<lifeless> 4 second queries on codeimportjob
<StevenK> Right, rabbit sucks.
<StevenK> I can't get it to start on the Jenkins build slaves
<wgrant> It should be happy enough as long as it can resolve its local node name.
<wgrant> Possibly both ways.
<LPCIBot> Project windmill-devel build #65: STILL FAILING in 1 hr 7 min: https://lpci.wedontsleep.org/job/windmill-devel/65/
<StevenK> Forcing the hostname to localhost:
<StevenK> root@localhost:~# /etc/init.d/rabbitmq-server start
<StevenK> Starting rabbitmq-server: FAILED - check /var/log/rabbitmq/startup_log, _err
<StevenK> rabbitmq-server.
<StevenK> wgrant: Did your distinct on branch of awesome land?
<wgrant> StevenK: Not yet reviewed by Stormers.
<StevenK> I thought lifeless self-reviewed his last few storm branches, TBH
<LPCIBot> Project windmill-db-devel build #267: STILL FAILING in 1 hr 7 min: https://lpci.wedontsleep.org/job/windmill-db-devel/267/
<lifeless> StevenK: no
<lifeless> StevenK: I'm not in the review group for storm
<lifeless> StevenK: we're running a fork because the patch for WITH is kindof stalled
<StevenK> Bah, my evil plan of using sets isn't working
<wgrant> Oh?
<StevenK> Oh, bah!
<StevenK> The overrides return the DAS, but the input is the archtag
<wgrant> That may or may not be correct.
<wgrant> You'll note that publishBinaries calculates the DAS itself from the old archtag at the moment. Consider how you are going to refactor that area when you decide what you need to return from the overrides.
<LPCIBot> Project devel build #710: STILL FAILING in 5 hr 28 min: https://lpci.wedontsleep.org/job/devel/710/
<jtv> Does a feature flag's default value have to be a string?  Or am I dangerously close to suggesting a full type system for feature flags?
<lifeless> its a string
<lifeless> they have not defaults
<lifeless> if not present in the system, you will get back None
<lifeless> they can be set explicitly to ''
<jtv> They have not defaults?
<jtv> It'd be nice if I could make more sense of the code + comments.
<jtv> There's a suggestion that there are defaults.
<jtv> Is the "default value" part of flag_info just documentation of what the code will assume you mean if no value is set then?
<lifeless> yes
<lifeless> default behaviour
<lifeless> not default value
<jtv> I see.  I'll drop that into the comment then.
<lifeless> all you need to do is
<lifeless> value = get_feature_flag('...')
<lifeless> if not value: value = '100'
<lifeless> try:
<lifeless>     limit = int(value)
<lifeless> except ValueError:
<lifeless>     limit = 100
<LPCIBot> Project windmill-db-devel build #268: STILL FAILING in 1 hr 7 min: https://lpci.wedontsleep.org/job/windmill-db-devel/268/
<lifeless> hmm, can just drop the if not value: value = '100' from my sketch, its not needed.
<jtv> lifeless: ironically, I think that is _to the letter_ what I have in my code.
<jtv> Ah no, apart from the ValueError because I wasn't sure about None vs. empty string.
<jtv> But variable name: check.  Flag type: check.  Default value: check.  All as in my branch.  Had you been reading it?
<lifeless> nope :)
<jtv> Heh
<jtv> By the way, I _think_ there's no firm guarantee about whether an unset value is an empty string or a None.  Or at least, that seems to have been a driving consideration in how boolean checks work.
<jtv> int(None) -> TypeError, in fact
<wgrant> An unset value is None. An empty value is ''
<jtv> whereas int('') -> ValueError
<jtv> Argh.
<wgrant> Yes
<jtv> I am sorely tempted to make these things typed and build defaults into the descriptor mechanism.
<jtv> (By the way, I'm adding an "int" domain)
<lifeless> I'd rather we didn't.
<lifeless> this is currently very lean; I'd hate for it to become less lean.
<lifeless> I'd be fine with helper functions to do 'get me an int with a default for missing or invalid of 100'
<jtv> Does the default behaviour document only what the code will do if the flag is not specified?  Or what the code will do if the flag is either not specified or given an empty value?
<lifeless> jtv: not specified
<lifeless> an empty value is a value
<jtv> OK... FWIW: just trying to get the assumptions for my comments right, not mapping out a type system. :)
<jtv> lifeless: would you concur with these documentation updates?  https://dev.launchpad.net/FeatureFlags?action=diff
<lifeless> s/"not specified" / None
<lifeless> and drop the then redundant Reading the flag's value will then return None.  sentence
<lifeless> jtv: other than that its great
<jtv> I wanted to avoid implying that A Flag Has A Value, when actually the relationship is more complex.
<jtv> Q: "Are you Johnny (The Fingers) Carlsted" -- A: "Who wants to know?"
<wgrant> Two tiny reviews, if anyone is interested: https://code.launchpad.net/~wgrant/launchpad/question-api-fixes/+merge/60465 and https://code.launchpad.net/~wgrant/launchpad/bug-348996/+merge/60612
<jtv> wgrant: I'll take the first
<wgrant> Thanks jtv.
<jtv> lifeless: that's right, isn't it?  A flag doesn't so much have a value as produce a value from the rules that apply at the moment you're reading it?
<jtv> wgrant: by the way there is (was?) a fast way of testing launchpadlib, but it sort of broke down where the type system was concerned.  Or rather, it became object-oriented where the WADL prefers the rigid, compiled-language-style class orientation that the world has moved away from.
<lifeless> jtv: the value in a particular request is a result of a lookup yes
<lifeless> jtv: but at any point, get_...('..') does evaluate to one and only one value
<lifeless> which might be None
<jtv> lifeless: yes, I'm sure that's correct but I didn't want to imply something that will set the wrong expectations once things get more advanced.
<wgrant> Thanks lifeless, jtv.
<lifeless> jtv: I don't see this changing TBH
<jtv> lifeless: what changing?
<lifeless> 'once things get more advanced.'
<lifeless> the only change I expect to happen vis-a-vis flags is exposing them inernally for codehosting etc to lookup
<lifeless> (without db access)
<jtv> You misunderstand me.
<jtv> I mean once the discussion of the topic gets more advanced.
<lifeless> oh
<lifeless> I'd aim for directness
<jtv> I don't want to sort-of-imply that there is a simple 1:1 relationship between features and flags if you then need to overcome that misleading impression later on.
<lifeless> flags and values perhaps
<jtv> Sorry, yes.
<lifeless> features and flags is a whole other discussion
<jtv> I meant flags and values.
<jtv> lifeless: hope I didn't miss anything.  Anyway, if I were to say "if the value is not specified," that would slightly confuse the matter of "a value may be specified but not apply."  I'm wide open to more direct phrasings, but I like to avoid that sort of subtle accidental misdirection.
<LPCIBot> Project db-devel build #541: STILL FAILING in 5 hr 26 min: https://lpci.wedontsleep.org/job/db-devel/541/
<LPCIBot> Project windmill-devel build #66: STILL FAILING in 1 hr 10 min: https://lpci.wedontsleep.org/job/windmill-devel/66/
<lifeless> jtv: I don't think thats misdirection
<lifeless> implicit in any discussion of value is the flag->rules->value mapping
<lifeless> which should happen first
<jtv> lifeless: you're right--I'm describing values right after you introduce that concept, so that's not an issue here.
<NCommander> wgrant: Ahhh!, now it makes sense, although I'm not seeing a test that uses the factory method
<jtv> wgrant: you up for a similarly small review?  https://code.launchpad.net/~jtv/launchpad/ff-780319/+merge/60731
<jtv> lifeless: I'll update it.
<wgrant> jtv: So the flag already exists and has a default of 100?
<jtv> wgrant: well... in a branch I'm currently landing.
<wgrant> Sure.
<wgrant> jtv: Done. StevenK, could you mentor?
<jtv> wgrant: ah, didn't realize you weren't fully hatched yet.  Thanks though.
<StevenK> wgrant: Sure.
 * wgrant is yet to evolve.
<StevenK> wgrant, jtv: Done
<wgrant> Thankyou sir.
<jtv> Thanks both!
<jtv> My comment: "I considered assimilating them, but I don't believe the display of these domain descriptions gives much of a clue so I thought it'd be nice to have just one verbose description in there.  Two would have been overkill though."
<jtv> Is that acceptable?
<wgrant> jtv: I wasn't favouring one over the other. Your reasoning is sound, too.
<jtv> Thanks.
<bigjools> good morning from Luton airport.  Good job I didn't pack razor blades, I might need to use them on my wrists.
<wgrant> Oh?
<bigjools> Luton is the armpit of the world
<LPCIBot> Project windmill-devel build #67: STILL FAILING in 45 min: https://lpci.wedontsleep.org/job/windmill-devel/67/
<adeuring> good morning
<henninge> Hi adeuring!
<henninge> adeuring: there is the UDS session about translation sharing starting in 5. Do you mind to join?
<henninge> adeuring: #ubuntu-uds-krudy
<henninge> adeuring: audio at http://icecast.ubuntu.com:8000/
<mrevell> Goodly morning
<adeuring> henninge: sorry, missed your message...
<lifeless> stub: can we catch up tomorrow?
<stub> lifeless: sure
<lifeless> cool
<lifeless> nn folks
<stub> sweet dreams
<NCommander> afternoon all
<lifeless> stub: https://dev.launchpad.net/ArchitectureGuide/Services#preview is coming together
 * NCommander is really feeling pretty lost w.r.t. to LaunchpadObjectFactory ...
<NCommander> Can I do LPCONFIG=testrunner make run to load the database used by tests? I'd like to see the state of the LP database after running through my test
<lifeless> NCommander: the test db is reset at the end of the test
<NCommander> lifeless: would sticking stop() in work?
<NCommander> or preventing cleanup? :-/
 * NCommander was readinghttps://dev.launchpad.net/Debugging but thats mostly for page tests :-/
<stub> NCommander: make harness PGDATABASE=launchpad_ftest_playground works. make run PGDATABASE=launchpad_ftest_playground likely works too.
<NCommander> stub: I managed to get it working bybreaking into PDB, then running with the same LPCONFIG environment variable used by the test
<stub> (abusing features of libpq)
 * NCommander has a wonderfully generic Distroseries90786 :-)
<NCommander> stub: thanks though, that's handy for future refernece
<NCommander> lifeless: wgrant: does this look sane so far? http://paste.ubuntu.com/606461/
<NCommander> (it looks correct in the webui)
<LPCIBot> Project windmill-devel build #68: STILL FAILING in 1 hr 18 min: https://lpci.wedontsleep.org/job/windmill-devel/68/
<jkakar> Yay, we (Fluidinfo) can view milestone pages again.  Thanks! :)
<wgrant> jkakar: You use projectgroup milestones?
<jkakar> wgrant: Yes.
<jkakar> wgrant: It's probably the most useful bug-related view for me in Launchpad.
<jkakar> Actually, I use an lp:kanban based view much more than I use milestones now... in fact, I rarely look at milestone views now.
<wgrant> I find project milestone views to be very handy.
<wgrant> But we don't use the project group much any more.
<jkakar> lp:kanban doesn't provide as much detail as the milestone view in some ways, like bug counts by status, milestone due date and some other bits.
<wgrant> Since the merge.
<jkakar> Yeah, makes sense.
<jkakar> We use milestones as a way to have a view of work we are doing (or want to do) during our 4 week iterations.
<wgrant> It's just about the closest thing LP has to a dashboard.
<jkakar> We have 5 or 6 projects in a project group, so the project group milestone view is important for us.
<wgrant> Right.
<jkakar> wgrant: Yep, exactly.
<jkakar> wgrant: I've thought about adding the missing details from the milestone view to lp:kanban but I try to be very careful about what gets added to the board... I want it to be as simple as possible and to avoid clutter.
<wgrant> NCommander: I'm not sure why you're explicitly creating ProcessorFamilies, but that looks like a good start.
<NCommander> wgrant: That's how it was handled in some of the other examples, but I've been ripping apart things so I suspect those can die
<NCommander> wgrant: my next challenge is figuring out how to use NascentUpload to put files into my archive
<wgrant> NCommander: You don't want to use NascentUpload.
<wgrant> NCommander: The factory provides methods to create the objects directly.
<wgrant> archiveuploader is some of the oldest and least pretty code in LP. Avoid it if at all possible -- and it is completely possible here.
<NCommander> wgrant: don't tihnk that's going to work the way I want to because I want to have properly arch-all/any dependency info and single source/multiple binaries
<wgrant> NCommander: NascentUpload is Launchpad code, and it achieves that. Therefore you can do the same thing in a test.
<wgrant> Without the abomination.
 * NCommander feels the descent into madness
<wgrant> NCommander: You probably want to makeBinaryPackageBuild, then use makeBinaryPackageRelease to attach your various binaries to it.
<NCommander> I'm thinking I need to start with a source package in there somewhere
<wgrant> makeBinaryPackageBuild will create a SourcePackageRelease on which to base the build.
<wgrant> I'm not sure you care about the source, do you?
<NCommander> i do
<wgrant> What about it do you care about?
<NCommander> I'm caring about dependency relationships here, which is why I want to upload actual packages
<NCommander> Making sure there's a way to retain the arch-any/arch-all bits when half a package gets superseede
<wgrant> Why do you think uploading actual packages would be better?
<wgrant> What does that achieve that you cannot do in Python?
<NCommander> It means that I can simply get the data from the package itself instead of feeding out the entire relationships in LP functions. ideally, my test case woudl actually parse the resulting binary-* files to make sure teh skew is avoided, but I've been told thats too resource intesive
<wgrant> End-to-end tests like that are ugly, really slow, difficult to write, difficult to read, difficult to change, unobvious, make the rest of the system inflexible, and provide negligible benefit.
<NCommander> wgrant: looking at the makeBinary stuff, I don't have a way to make multiple binaries off a source record sanely
 * NCommander is still reading
<wgrant> NCommander: You can pass one BinaryPackageBuild into multiple makeBinaryPackageRelease calls.
<NCommander> If you say its difficult to do, I'll default to you, but I generally consider a test invalid unless it tests end to end IMHO
<wgrant> There need to be some end-to-end tests, but for edge cases like this they are not appropriate.
<NCommander> fair enough
 * NCommander is not looking forward to setting up soyuz fully locally so I can manually examine the edge case
<NCommander> er, output
<NCommander> brain == goo
<NCommander> lunch time
<wgrant> NCommander: There's fairly good instructions for that now :)
<LPCIBot> Project devel build #711: STILL FAILING in 5 hr 32 min: https://lpci.wedontsleep.org/job/devel/711/
<StevenK> allenap: O hai. Are you fine to OCR?
<allenap> StevenK: Yes, but I have to go in a few minutes. I can do it when I get back.
<StevenK> allenap: https://code.launchpad.net/~stevenk/launchpad/generic-overrides/+merge/60730 when you're ready
<allenap> StevenK: Cool.
<henninge-lunch> autTnf-2
<LPCIBot> Project db-devel build #542: STILL FAILING in 5 hr 29 min: https://lpci.wedontsleep.org/job/db-devel/542/
<LPCIBot> Project windmill-db-devel build #269: STILL FAILING in 1 hr 15 min: https://lpci.wedontsleep.org/job/windmill-db-devel/269/
<benji> jcsackett: are you OCR today?
<LPCIBot> Project windmill-devel build #69: STILL FAILING in 1 hr 7 min: https://lpci.wedontsleep.org/job/windmill-devel/69/
<benji> wow, I won a Macbook Air: https://bugs.launchpad.net/launchpad/+bug/1234
<_mup_> Bug #1234: Gina is an unmaintainable mess of command line options, environment variables and shell scripts <lp-foundations> <Launchpad itself:Fix Released by debonzi> < https://launchpad.net/bugs/1234 >
<deryck> Morning, all.
* jcsackett changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: jcsackett | https://code.launchpad.net/launchpad-project/+activereviews
<jcsackett> benji: just went on shift. :-)
<benji> jcsackett: cool, here's a small one to get warmed up: https://code.launchpad.net/~benji/launchpad/fix-generated-html/+merge/60697
<jcsackett> benji: r=me. shame we need the hack, but at least it's a well done hack. :-)
<LPCIBot> Project windmill-devel build #70: STILL FAILING in 45 min: https://lpci.wedontsleep.org/job/windmill-devel/70/
<adeuring> allenap: are ou doing reviews today?
<allenap> adeuring: Yes, I'll fix the topic.
* allenap changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: jcsackett, allenap | https://code.launchpad.net/launchpad-project/+activereviews
<adeuring> allenap: cool, I have a small mp: https://code.launchpad.net/~adeuring/launchpad/bug-739075-2/+merge/60768 could you habe a look at it?
<allenap> adeuring: Okay, cool.
<adeuring> thanks!
<adeuring> allenap: thanks for your review!
<allenap> adeuring: Thank you for an easy one :)
<LPCIBot> Project windmill-devel build #71: STILL FAILING in 44 min: https://lpci.wedontsleep.org/job/windmill-devel/71/
<deryck> gah.  try to fix one critical js bug and I'm sucked into qa blackhole for subscriptions.
<jcsackett> allenap: can you take a look at https://code.launchpad.net/~jcsackett/launchpad/spam-tech-debt/+merge/60794 ?
<jcsackett> it's quite a bit shorter than the diff implies--mostly code deletes and replaced in one spot.
<benji> deryck: anything I can do to help?
<deryck> benji, no, it's my own stupidity.  thanks, though.
<deryck> benji, if you see bugs against the story about the subscription overlay, it's just me cleaning up my mess.
<benji> ok :)
<deryck> I'm really not trying to nosy my way into the Yellow Squad work :-)
<allenap> jcsackett: Sure.
<allenap> jcsackett: I'm doing a review for StevenK right now, but after that.
<jcsackett> allenap: no rush. it's just simplifying some tests. meant to get it in for review yesterday but completely forgot about it.
<poolie> bigjools: hi?
<bigjools> poolie: hey
<poolie> hi
<poolie> i'm trying to fix bug 778437
<_mup_> Bug #778437: Recipe build success sends emails, please stop doing that <email> <recipe> <Launchpad itself:In Progress by mbp> < https://launchpad.net/bugs/778437 >
<bigjools> ok
<poolie> i'm trying to work out how mail gets sent after a successful sprb
<poolie> there is an existing test that seems to assert that mail is not sent
<poolie> in fact it is sent
<bigjools> !
<bigjools> is the mail as a result of the upload, or the build?
<poolie> SourcePackageRecipeBuild._handleStatus_OK
<bigjools> IYSWIM
 * bigjools looking
<poolie> tries to send mail on status=FULLYBUILT
<poolie> however
<poolie> afaics that method is only ever reached with the packgae in UPLOADING
<poolie> istm the mail ought to be sent after the build not the upload
<poolie> hm
<poolie> no, the example on the bug shows mail is sent on the source upload too
<bigjools> you could remove the self.notify() in _handleStatus_OK and see what tests break
<bigjools> I also remember reviewing some code that sent email from the uploads
<poolie> nothing breaks :)
<poolie> i'd be happy to just delete it,
<poolie> but it does seem like there's a gap in test coverage
<poolie> and perhaps that won't actually suppress the mail
<bigjools> hmmm
<poolie> but nothing else seems to create that mail template
<poolie> the odd thing in the mail this sends is that
<poolie> it says "state: successfully built" but the log file is about uploading a source pacakge
<LPCIBot> Project windmill-db-devel build #270: STILL FAILING in 45 min: https://lpci.wedontsleep.org/job/windmill-db-devel/270/
<rvba> jcsackett: allenap: Could one of you guys have a look at this MP https://code.launchpad.net/~rvb/launchpad/multi-parent-diff-pages2/+merge/60799 ?
<jcsackett> rvba: sure.
<rvba> jcsackett: thanks!
<poolie> bigjools: can you give me any clus?
<poolie> i'm really puzzled how that code gets hit in production
<sinzui> jcsackett: mumble?
<henninge> Hi, general Python question.
<jcsackett> sinzui: sure.
<henninge> I have the feeling I am missing something very obvious or more pyhtonic here. This implementation looks very C-ish.
<henninge> http://paste.ubuntu.com/606529/
<jcsackett> henninge: assuming haystack is a sequence, i think haystack.count(needle) works.
<jcsackett> sinzui: can you hear me?
<henninge> jcsackett: a string
<henninge> search a string within a string
 * henninge tries that
<henninge> jcsackett: thank you, I was not aware of that (obviously)
<jcsackett> henninge: yw.
<LPCIBot> Project windmill-devel build #72: STILL FAILING in 45 min: https://lpci.wedontsleep.org/job/windmill-devel/72/
<poolie> jelmer: are you still around?
<jelmer> poolie, hey
<jelmer> poolie, yep
<poolie> hi, can i find you somewhere?
<poolie> i think i need to update your soyuz code
<jelmer> I'm in Elod, or I can come find you
<jelmer> Where are you?
<poolie> i'm upstairs, i'll work out where elod is
<poolie> i guess there's a lesson that things that look trivial in soyuz, aren't
<jelmer> :)
<jelmer> I'm going to move towards the area near the front desk
<rvba> jcsackett: I don't know if you need me for the review ... but FYI I'll have to EOD in ~20 min.
<jcsackett> rvba: yeah, sorry this is taking so long--pretty big diff. i'll leave all comments on the MP.
<rvba> jcsackett: sure, I know the diff is kinda big.
<jcsackett> sinzui: i am finally looking at your project-review MP.
<sinzui> \o/
<jcsackett> one question, i see both a license-reviewed and license-approved. shouldn't the first be a project-reviewed widget?
<jcsackett> sinzui ^
<sinzui> jcsackett: The field is license_reviewed. We can rename the field to be project_reviewed
<sinzui> We changed the meaning of the field in 2008
<jcsackett> sinzui: ah, dig. i don't know that it's necessary, i was just somewhat confused.
<sinzui> Indeed. engineers think they are approving projects, when they are approving licenses (hence the errors we see when approving a proprietary and I don't know project)
<jcsackett> sinzui: that makes sense.
<jcsackett> r=me, sinzui. and i look forwards to seeing this deployed. :-)
<sinzui> jcsackett: I can change the field name do you want me to do that?
<sinzui> I think it is a trivial mechanical change
<jcsackett> sinzui: actually, i think it's good as is. as you said, we're trying to prevent people thinking they're reviewing/judging the project.
<sinzui> jcsackett: well we do mean review_project
<jcsackett> ah. so i was still confused. :-P
<jcsackett> ok, then yeah, we probably should update the field, if that's not a pain. i would rather get the improved project review in place then quibble about fields tho. :-)
<sinzui> okay. I will make this last change ans land it. Thank you very much
<jcsackett> thank you, sinzui.
<jcsackett> i must go grab some food, and then i will look at your other branch.
<LPCIBot> Project db-devel build #543: STILL FAILING in 6 hr 37 min: https://lpci.wedontsleep.org/job/db-devel/543/
<LPCIBot> Project devel build #712: STILL FAILING in 6 hr 38 min: https://lpci.wedontsleep.org/job/devel/712/
 * lifeless stretches
<lifeless> morning gary_poster
<LPCIBot> Project windmill-devel build #73: STILL FAILING in 1 hr 8 min: https://lpci.wedontsleep.org/job/windmill-devel/73/
<LPCIBot> Project windmill-db-devel build #271: STILL FAILING in 51 min: https://lpci.wedontsleep.org/job/windmill-db-devel/271/
 * gary_poster winces at not finishing doc review for lifeless, then runs out the door, to be back soon
<lifeless> gary_poster: :P
<lifeless> allenap: hi
<lifeless> allenap: why is bug 776283 untestable ?
<_mup_> Bug #776283: PackageCopyJob needs separate archive and series columns for query purposes (db patch) <derivation> <qa-needstesting> <Launchpad itself:Fix Committed by allenap> < https://launchpad.net/bugs/776283 >
<lifeless> _mup_: you are stale!
<lifeless> ah, I meant bug 779949
<_mup_> Bug #779949: PackageCopyJob needs separate archive and series columns for query purposes <derivation> <qa-untestable> <Launchpad itself:Fix Committed by allenap> < https://launchpad.net/bugs/779949 >
<lifeless> nvm
<LPCIBot> Project windmill-db-devel build #272: STILL FAILING in 51 min: https://lpci.wedontsleep.org/job/windmill-db-devel/272/
<abentley> jcsackett or allenap: could you please review https://code.launchpad.net/~abentley/launchpad/translation-packaging-resource/+merge/60832 ?
<jcsackett> abentley: sure.
<abentley> jcsackett: thanks.
<jcsackett> abentley: that was very to the point. r=me.
<abentley> jcsackett: cool, thanks.
<jcsackett> you get today's coveted "shortest diff" award. :-)
* jcsackett changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: - | https://code.launchpad.net/launchpad-project/+activereviews
<LPCIBot> Project windmill-devel build #74: STILL FAILING in 49 min: https://lpci.wedontsleep.org/job/windmill-devel/74/
#launchpad-dev 2011-05-13
<sinzui> wgrant: mumble?
 * sinzui wanders off to feed children
<wgrant> sinzui: Sorry, here now.
<sinzui> Hello wgrant. Do you have time to mumble?
<sinzui> wgrant: I also want a review of this script to remove projects without any license: http://pastebin.ubuntu.com/606665/
<jtv> Has anyone else seen EC2 fail notfound-traversals.txt spuriously?   http://paste.ubuntu.com/606695/
<wgrant> jtv: It was not spurious. I broke trunk for a few hours last night.
<wgrant> jtv: How did the moldavian deletion go?
<wgrant> Ah, the question is gone. That is a good sign.
<wgrant> Thanks.
<LPCIBot> Project db-devel build #544: STILL FAILING in 5 hr 30 min: https://lpci.wedontsleep.org/job/db-devel/544/
<lifeless> -> airport to pick up family; bbiab
<jtv> wgrant: broke trunk, eh?  well done!  I was getting nervous because my branch failed twice with different test failures.
<wgrant> lifeless: rabbit failed lucid_db_lp
<LPCIBot> Project windmill-db-devel build #273: STILL FAILING in 1 hr 10 min: https://lpci.wedontsleep.org/job/windmill-db-devel/273/
<LPCIBot> Project devel build #713: STILL FAILING in 5 hr 34 min: https://lpci.wedontsleep.org/job/devel/713/
<LPCIBot> Project windmill-devel build #75: STILL FAILING in 1 hr 9 min: https://lpci.wedontsleep.org/job/windmill-devel/75/
<jtv> lifeless, back?
<wgrant> Hmm, only 541 timeouts yesterday?
<wgrant> I guess UDS might be reducing it a bit... but 50%?
<jtv> wgrant: how does it compare to previous UDSes?
<LPCIBot> Project windmill-devel build #76: STILL FAILING in 45 min: https://lpci.wedontsleep.org/job/windmill-devel/76/
<jtv> wgrant: I see you added a qa-untestable tag to my bug just around the time the tagger was supposed to add a qa-needstesting to it.
<jtv> It's actually qa-ok, but I wanted to avoid concurrency issues.
<wgrant> jtv: Well, you removed the needstesting tag which upsets qa-tagger.
<wgrant> You can't remove QA tags until the revision is deployed.
<wgrant> Also, why was the tagger meant to add qa-needstesting?
<wgrant> It had already, but you removed it.
<wgrant> And the next buildbot run won't finish for half an hour, and then an hour later it will tag the two involved bugs, neither of which are yours.
<wgrant> So, I am confuse!
<wgrant> But I am also hungry, so I shall lunch to resolve my confusion.
<wgrant> Or at least defer it.
<jtv> wgrant: the first branch was deployed, and the second was going to be deployed.
<jtv> Both on the same bug.
<lifeless> jtv: yes
<jtv> lifeless: didn't know how far a drive it was etc. :)  Trust all is well.
<jtv> I wanted to ask for feed back
<lifeless> yup
<jtv> *feedback
<jtv> about the UI work I'm sketching out for async package syncs.
<lifeless> sure, I'll be passing by the keybard for a bit, and once folk are settled, fully here.
<jtv> OK, I'll start typing.  :)
<jtv> We have a Job for sync'ing packages into a derived distro.
<wgrant> jtv: I still don't understand why the qa-needstesting tag was removed. But it seems all OK  now.
<jtv> There's a UI that shows differences between a derived distroseries and its parent(s).
<jtv> That's where you can request that some of these differences be synchronized, in various ways.
<jtv> Some or all of these requests will be asynchronous.
<jtv> At first I tried to sketch out a way where the UI remembers "you requested async sync of packages [x, y, z]; these went into jobs {a: [x], b: [y, z]} (one job per source archive, but potentially multiple packages per job)
<jtv> "
<jtv> But it's costly and awkward and ignores other pending requests.
<jtv> If you try to repeat a sync, you get an error.
<jtv> (I'm not too enthusiastic about that notion, but there may be good reasons just outside my current field of view).
<jtv> I want to break this down into stages that I/we can implement in realistic time: basics first, shiny later.
<jtv> My current plan: have the static page always (request or no request) show which of the visible distroseries differences have sync jobs pending.
<jtv> (Polling and error reporting come later)
<jtv> Since it makes no sense to repeat a pending sync request, I was thinking to _replace_ the checkbox next to a difference that's waiting to be synced away with a little watch as used elsewhere.
<wgrant> Is that feasible in the current model?
<jtv> I don't know.
<wgrant> (it's the right thing to do, but I don't think it's efficiently doable without a separate jo)
<wgrant> +b
<jtv> I think you're talking about finding out which jobs are pending.  I'm talking about the UI.
<wgrant> We unfortunately already have a model to work within, and that needs to influence the UI decisions :(
<jtv> Yes yes yes I know.  But there is _some_ specificity to the search and I'm assured the queue won't be very large.
<wgrant> It will fairly often have thousands of packages.
<wgrant> But not terribly many jobs, perhaps.
<jtv> Packages aren't where the cost is.
<jtv> Other ways to find this out quickly lead to extra client/server interactions, which I expect to be much more costly.
<jtv> We know which jobs and which destination archive we're interested in when we render the page, and that helps; no need to send an API request from the client after rendering, listing the packages you're interested in.
<jtv> It may also be slightly jarring to have your view changed too long after the page has rendered, possibly with consequences for the placement of widgets you're about to click on.
<lifeless> so
<lifeless> with queues, in general, we shouldn't assume we can tell whats in the queue
<lifeless> this is a rabbit limitation AIUI, and though we don't use rabbit today, I'm leery of depending on things we know it doesn't know
<jtv> Sensible, but I have a feeling that a lot will go overboard when we do that anyway.
<jtv> Most of the wait, for one.
<lifeless> well
<lifeless> the latency is 60 seconds for a job runner
<lifeless> nevertheless
<lifeless> lets talk concurrency for a second
<lifeless> two users may both submit a request to sync overlapping packages at the same time
<lifeless> unless we are -very- careful in our model, we won't trigger unique constraints
<lifeless> not sanely
<lifeless> I think if someone wants package version X synced from A to B
<lifeless> and its queued twice
<lifeless> shrug
<lifeless> let the second sync say 'noop, DONE'
<lifeless> IMNSHO
<jtv> Sure, but...
<lifeless> in other words, make syncing an idempotent operation
<jtv> shrug
<jtv> Been over that for about as much as I feel we need to right now.
<wgrant> It's not quite that simple.
<lifeless> jtv: ok
<wgrant> Because binaries are fun.
<jtv> I'm saying it doesn't make much sense to request a repeated sync; not that I want to avoid it at all costs.
<jtv> I was just thinking it might be a minimally invasive, yet effective way of expressing the situation: don't bother requesting a sync here, it's already pending.
<lifeless> so, internet fail
<jtv> Thank you, Natty, for leaving a loophole that gives me access to the "reconnect" button on my chat client that you think I don't want to access ever.
<lifeless> :>
<jtv> To repeat then:
<jtv> In the current pre-rabbit situation, this way of doing it has the advantage of making the first step in providing async visibility very easy to implement.
<jtv> The main questions for that are, I think: (1) does this display help the user at all?  (Actually I'd also want some kind of "sync pending" notice in there, obviously) and (2) does it form any kind of usable basis for the later steps (tracking progress, reporting errors) if we should decide to drop the jobs and use rabbit instead?
<jtv> If we later find that it becomes too difficult to mark pending syncs in the static page, we can always drop that bit.  But I would hope that unless and until that happens, the work will have helped us move closer to a "nice" UI.
<jtv> Trying to do a "nice" UI right away would probably involve making the whole sync request Ajaxy to track the jobs it created (if any), which goes at the cost of getting the first bit of visibility into pending requests usable later than with this approach.
<lifeless> I'm agnostic here
<lifeless> I can see merits in both approachs
<lifeless> I agree that they are to some degree at cross purposes
<lifeless> I think from an adminny perspective we want to know about things stuck in the queue
<lifeless> and to be able to fix (e.g. in the current code do a cancel) them
<jtv> Well if a job fails, it fails.
<jtv> If it's stuck, the whole queue is stuck.
<jtv> (Which should show up as oopses, I guess, though my UI would also show it as an increasing amount of pending differences in the display)
<jtv> We'd lose that advantage with rabbit I guess, but maybe at that point we'd just accept having to replace that visibility with something as a cost of moving ahead, and my current idea will have served its purpose as a stepping stone.
<jtv> I hate being under the weather on a holiday.
<lifeless> its a holiday?
<jtv> Well my girlfriend says it is.
<jtv> Can't find it online, but here you're never sure about those things.
<jtv> We do have something different next week, but my web searches put it at up to 3 different days this year, at a length of either 1 or 2 days.
<jtv> Nondeterministic calendars are a great job-security scheme for religious establishments.
<thumper> I think Thailand has more holidays than any other place I know
<jtv> I try not even to register the slightly iffy ones ("government holiday" etc.) but when you get right down to it, there's a vast sea of iffy holidays.
<jtv> I believe in rare cases the official word comes down literally the day before.
<jtv> Not to mention curfews and such.
<lifeless> bah
<lifeless> where is that bug about the 404 for pages depending on vhost
<jtv> Damn, can't search for "404"!
<jtv> bug 422960?
<_mup_> Bug #422960: appear to be failing to record oops for all +translate HTTP 503 errors <canonical-losa-lp> <lp-foundations> <Launchpad itself:Triaged> < https://launchpad.net/bugs/422960 >
<jtv> bug 626878?
<_mup_> Bug #626878: pages only on subdomains cause user confusion (e.g. launchpad.net/project/+activereviews is a 404) <lp-foundations> <Launchpad itself:Triaged> < https://launchpad.net/bugs/626878 >
<jtv> bug 574697?
<_mup_> Bug #574697: Launchpad strips incoming TE header <lp-foundations> <Launchpad itself:Triaged> < https://launchpad.net/bugs/574697 >
<lifeless> #626878:
<lifeless> thanks
<_mup_> Bug #626878: pages only on subdomains cause user confusion (e.g. launchpad.net/project/+activereviews is a 404) <lp-foundations> <Launchpad itself:Triaged> < https://launchpad.net/bugs/626878 >
<jtv> Used google with site:
<lifeless> 'None' is a subscriber to this bug apparently
<wgrant> lifeless: I think your Chromium is bad :(
<lifeless> wgrant: probably; how so?
<wgrant> Well, BjornT is not None.
<lifeless> wgrant: different bugs
<wgrant> And the recipe owner picker works for everybody else.
<lifeless> bug 735972
<_mup_> Bug #735972: Person:+related-software timeouts <timeout> <Launchpad itself:Triaged> < https://launchpad.net/bugs/735972 >
<wgrant> (can I close that picker bug now?)
<lifeless> wgrant: give me a chance to dup
<lifeless> it takes a while to reopen 70 or so tabs
<lifeless> so I've been being productive in the meantime
<wgrant> Heh
<lifeless> openid soooooo slow
<lifeless> (well, our sso is)
<lifeless> oh man +related-software has lots of room to improve
<lifeless> ughhhh
<lifeless> I can't find this darn bug
<lifeless> the one about bzr push not being able to create a spn
<wgrant> Bug #386596
<_mup_> Bug #386596: pushing to a packaging branch can't create a new package <codehosting-ssh> <lp-code> <package-branches> <Launchpad itself:Triaged> < https://launchpad.net/bugs/386596 >
<lifeless> hth
<lifeless> thanks
<wgrant> Found using Launchpad's search, even.
 * wgrant stabs asuka.
<wgrant> Update faster.
<lifeless> thanks!
 * lifeless escalates
<lifeless> whats a sensible fs for an archive mirror?
<lifeless> stub: so, yo
<stub> lifeless: yo
<lifeless> shall we try skynet?
<stub> I'm going to boot my laptop to try and narrow down issues - there is some other weirdness happening with that network card and natty.
<lifeless> ok
<LPCIBot> Project devel build #714: STILL FAILING in 5 hr 11 min: https://lpci.wedontsleep.org/job/devel/714/
<StevenK> Is that more rabbit failure?
<StevenK> Why, yes, it is.
<wgrant> StevenK: That happened on lucid_db_lp earlier.
<StevenK> I've come to the conclusion that rabbit sucks, and I can't get it to start on the Jenkins build slaves. Not sure why that afflicts buildbot.
<wgrant> lifeless: "Ubuntu currently has a problem with things that haven't built yet being unable to have bugs filed on them"
<wgrant> Huh?
<wgrant> It's got nothing to do with builds.
<lifeless> StevenK: https://bugs.launchpad.net/launchpad/+bug/655385/comments/14
<_mup_> Bug #655385: Allow bug status change from Triaged only for bug supervisor <accesscontrol> <acl> <bugs> <lp-bugs> <Launchpad itself:Opinion> < https://launchpad.net/bugs/655385 >
<lifeless> bah
<lifeless> stub: https://bugs.launchpad.net/launchpad/+bug/655385/comments/14
<_mup_> Bug #655385: Allow bug status change from Triaged only for bug supervisor <accesscontrol> <acl> <bugs> <lp-bugs> <Launchpad itself:Opinion> < https://launchpad.net/bugs/655385 >
<lifeless> wgrant: orly?
<lifeless> wgrant: whats it got to do with?
<wgrant> lifeless: It's all source-based.
<wgrant> Doesn't go near binaries.
<lifeless> wgrant: source publication then ?
<wgrant> Yes.
<lifeless> fine :>
<lifeless> wgrant: so i didn't say builds in that bug
<wgrant> Ubuntu currently has a problem with things that haven't built yet being unable to have bugs filed on them
<lifeless> wgrant: I said 'published packages', what was wrong with that ?
<wgrant> "things that haven't built yet"
<lifeless> ah, fixorating
<lifeless> thanks
<LPCIBot> Project db-devel build #545: STILL FAILING in 5 hr 44 min: https://lpci.wedontsleep.org/job/db-devel/545/
<wgrant> Curse you, lifeless! Escalating bugs :(
<wgrant> Still, 215 criticals.
<wgrant> Well, plus loggerhead, but we don't talk about him any more.
<jml> 241
<jml> launchpad-project
<wgrant> Hm. I see 237.
<wgrant> Are there private bugs I cannot see? :(
<wgrant> lpstatus also sees 10-15 more than I can.
<jml> I see 234 now
<jml> yeah, lpstats counts by category
<wgrant> Still 237 for me.
<wgrant> Ah, so if something is in more than one category it will show it twice?
<jml> http://paste.ubuntu.com/606803/
<jml> wgrant: exactly
<wgrant> :(
<jml> (space bar broken, slow typing)
<wgrant> jml: Is that anonymous?
<wgrant> I know of two private bugs, maybe there is a third.
<jml> wgrant: it's not anonymous
<LPCIBot> Project windmill-db-devel build #274: STILL FAILING in 1 hr 10 min: https://lpci.wedontsleep.org/job/windmill-db-devel/274/
<wgrant> Then why do I see 237 at https://bugs.launchpad.net/launchpad-project/+bugs?field.importance=Critical :(
<jml> hmm
<wgrant> Possibly multi-task bugs.
<jml> I see that too
<jml> and 234 on the portlet
<wgrant> Indeed.
<wgrant> Possibly 237 tasks, 234 bugs.
<jml> yeah, makes sense
<jml> http://pastebin.ubuntu.com/606805/
<wgrant> Anyhow, minor progress.
<jml> that's my script
<wgrant> We may finally be getting somewhere.
<wgrant> Thanks.
<jml> wgrant: -44 is awesome
<wgrant> I also have four or five other fixes in review.
<wgrant> So we nearly made it to 50 for the week.
<jml> also, for pie purposes, I'm not counting newly escalated bugs
<wgrant> Ah, excellent.
<wgrant> They always seemed a bit disenheartening.
<lifeless> wgrant: sorry
<adeuring> good morning
<wgrant> Hmm, isn't hardy desktop EOL'd now?
<wgrant> Yes, yesterday.
<wgrant> ubuntu-mozilla-daily still builds for it :(
<wgrant> And keeps lpia alive.
<wgrant> Damn you lpia, damn you!
<StevenK> It was announced?
<wgrant> A month ago.
<StevenK> I think I have a little more hate for lpia than you :-P
<wgrant> At least I didn't mention Hildon.
<wgrant> Because that would just be mean.
<StevenK> HULK SMASH
<mrevell> Hello
<StevenK> allenap: Morning! Can you look at https://code.launchpad.net/~stevenk/launchpad/generic-overrides/+merge/60730 again?
<StevenK> allenap: I think I've addressed your concerns, and have even written you a note.
<allenap> Morning StevenK, sure I'll look at that now.
<wgrant> Bah.
<StevenK> wgrant?
<wgrant> allenap, jtv: Your packagecopyjob changes have conflicted in stable->db-devel.
<wgrant> Could be a bit of a mess.
<jtv> Thanks for the notice.
<wgrant> StevenK: I'm reasonably happy with your branch now.
<jtv> What does tradition dictate?  Gloves or no gloves?
<allenap> wgrant: Woohoo. Okay, I'll look in a minute, thanks for letting me know.
<jtv> allenap: I'm checking it out right now
<allenap> jtv: Awesome.
<StevenK> Which gives allenap more time to read my horrible diffs.
<wgrant> Bah, there's no functools.compose.
<wgrant> How stupid.
<jtv> I haven't had to deal with this in the current devel/db-devel configuration... can I still branch off db-devel, merge in devel, resolve conflicts, and land into db-devel?
<wgrant> jtv: Merge in *stable*.
<wgrant> jtv: But yes.
<jtv> ahhh
<wgrant> Otherwise you'll preempt buildbot.
<wgrant> And make us all sad.
<jtv> Hmmm conflicts don't look very bad from here.
<wgrant> There is an apparent argument reordering.
<jtv> Oh that
<wgrant> The visible conflict is easily resolvable.
<wgrant> But there may be deeper things.
<wgrant> The change suggests that arguments have changed.
<wgrant> Or that there may be additional IPackageCopyJobSource callsites.
<wgrant> When it's now IPlainPackageCopyJobSource, and in a different module.
<jtv> I switched two params around in the interface to make them match the implementation order.
<jtv> Should have fairly minimal impact but oh, the trouble I had once with this kind of inconsistent parameter ordering between interface and model...
<wgrant> StevenK: Hm, why do you eager load BPN and SPN?
<wgrant> StevenK: Given that you get the IDs from the objects, you presumably have them already.
<StevenK> "lol"
<StevenK> I can just return the source object directly
<jtv> Oh blast I may lose connectivity.
<wgrant> I also question the way you are given archtags and return DASes, but that may possibly be reasonable.
<StevenK> wgrant: Because FromExistingOverridePolicy does it too
<wgrant> But you wrote that as well, so you can't reason against it :P
<StevenK> Oh, clearly
<StevenK> Sigh. Now calculateSourceOverrides() for UnknownOverridePolicy is 2 lines.
<wgrant> Why so many?
<wgrant> Oh, for default_component?
<StevenK> Yes
<wgrant> cronscripts/foaf-update-karma-cache.py upsets me.
<StevenK> +4, -16. I am an idiot
* adeuring changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: adeuring | https://code.launchpad.net/launchpad-project/+activereviews
<wgrant> adeuring: Hi!
<adeuring> hi wgrant!
<StevenK> Oh look, an OCR. *pounce*
<wgrant> adeuring: https://code.launchpad.net/~wgrant/launchpad/bug-307269/+merge/60721 and https://code.launchpad.net/~wgrant/launchpad/bug-449561/+merge/60729 both need a review, if you could.
<adeuring> wgrant: sure
<StevenK> allenap: Are you blind yet? I have yet more changes, but I can hold off if you wish
<allenap> StevenK: I've just approved it.
<StevenK> allenap: Just glance at http://pastebin.ubuntu.com/606829/ then?
<allenap> StevenK: Haha, oh yes :) +1
<jtv> allenap: few questions about the jobs!  I guess the version in source_packages would correspond to the parent_source_version in the DSD?
 * StevenK attempts to not swear about himself or wgrant in the commit message
<allenap> jtv: Yes, I think so, but I'll check.
<wgrant> StevenK: I didn't write those!
<wgrant> Much of the rest, sure...
<jtv> allenap: it's also not always clear to me which lists of source-packages tuples are (name, version) and which have the third entry.
<StevenK> wgrant: But you pointed it out, so pffft
<wgrant> Ah, I see.
<allenap> jtv: Yes, I should probably land a branch to remove the ambiguity.
<jtv> Heh... you may want to give my existing conflict branch time to settle.  :)
<jtv> Any idea what time jools's demo is?
<allenap> jtv: Yes, parent_source_version.
<jtv> Thanks.
<allenap> No, no idea.
<jtv> May be useful to be at the ready just prior & during.  ;-)
<jtv> Argh!  Natty, stop switching my desktops around!
<StevenK> jtv: I was planning on denying everything.
<jtv> Good start.
<jtv> But you don't know how crafty he is.  He may lead with "Steve implemented this great feature here..."
<jtv> <kaboom!>
<allenap> wgrant: Btw, what would functools.compose do?
<wgrant> allenap: functools.compose(f, g) would basically do lambda *args, **kwargs: f(g(*args, **kwargs))
<wgrant> But it was vetoed.
<wgrant> Because you can do it with a lambda.
<wgrant> Despite the fact that you can do functools.partial even more cleanly with a lambda.
<allenap> wgrant: Yes, odd decision.
<adeuring> wgrant: one MPP approved.
<adeuring> wgrant: your other MP looks good too. just one question: Bug 449561 says not only that anonymous users get the edit icon for the assignee, but also that anon users have the option to use "mark as duplicate". What about this problem? And did you check for other possible edit options presented to anon users, as mentioned in the bug report?
<_mup_> Bug #449561: Bug task assignee picker shown to unauthenticated users <api> <lp-bugs> <oops> <Launchpad itself:In Progress by wgrant> < https://launchpad.net/bugs/449561 >
<wgrant> adeuring: That was fixed at some indeterminable time in the past.
<wgrant> adeuring: I can't see any other unguarded JS picker activations on the page.
<adeuring> wgrant: ah. cool. so, r=me
<wgrant> adeuring: Thanks!
<poolie> francis just said he thought question comments were added to the api and curtis blogged about this
<poolie> but that doesn't seem to be true...
<jml> which bit?
<jml> did sinzui not blog?
<jml> or are they not in the API?
<poolie> both
<poolie> neither seems to have happened
<poolie> but i might just be missing it
<wgrant> jtv: How goes the conflict? My inbox is upset with it.
<wgrant> I can have a look if you're busy/gone.
<stub> I'm just looking at it
<stub> wgrant: Or have you beat me?
<jtv> I've had my fix reviewed.
<poolie> jml, well, https://bugs.launchpad.net/launchpad/+bug/782093 it can be duped if appropriate
<_mup_> Bug #782093: question/answer comments not available in api <api> <questions> <Launchpad itself:Triaged> < https://launchpad.net/bugs/782093 >
<lifeless> poolie: https://launchpad.net/+apidoc/devel.html#question
<stub> jtv: The merge conflict or something else?
<jtv> stub: the merge conflict.
 * stub uncommits and leaves it to jtv
<lifeless> poolie: ah, the questions collection isn't exposed.
<lifeless> its controllable, but not exposed.
<lifeless> questions are exposed
<wgrant> The collection is exposed but not iterable.
<lifeless> and the spam control knob is exposed
<lifeless> wgrant: https://launchpad.net/+apidoc/devel.html#question has no comments_link
<lifeless> wgrant: is that what you mean ?
<wgrant> Oh, you said questions collection.
<wgrant> The comments collection isn't, no.
<lifeless> questions comment collection, I meant
<wgrant> Right.
<poolie> lifeless: right, the comments
<poolie> i thought you closed a bug about this the other day
<poolie> but perhaps we were confused baout the comments vs the questions
<poolie> anyhow, it would be nice for ISD but is not urgent
<wgrant> What does ISD want them for?
<wgrant> I thought they had other solutions for RnR.
<poolie> apparently they get questions across a lot of projects and they want to summarize them
<wgrant> Ahh, so something more sensible this time :)
<wgrant> They shouldn't be too difficult to export.
<wgrant> Just no point if nobody was going to use them.
<poolie> lifeless: in https://bugs.launchpad.net/launchpad/+bug/373078/comments/6 aaron says 'Needs fixing is meant to mean "This is approved but I'd like you to fix X"
<_mup_> Bug #373078: code review merge status and docs need clarity, particularly for 'tweak' cases <code-review> <lp-code> <Launchpad itself:Triaged> < https://launchpad.net/bugs/373078 >
<poolie> it's a shame launchpad's own use of them isn't consistent with that
<poolie> </bignose>
<wgrant> I thought Approve with comments meant that :/
<poolie> but perhaps that will get fixed
<poolie> it seems quite ambiguous and inconsistent across projects
<poolie> anyhow i shall try to remember that in future
<jtv> Do I land conflict fixes with release-critical then?
<wgrant> jtv: No.
<wgrant> Does it want it?
<wgrant> I '[rs=wgrant][no-qa] Whatever'
<jtv> Seems to, yes.
<wgrant> I think you are probably misreading it.
<jtv> Could well be.
<jtv> I hate those messages.
<wgrant> It will probably accept release-critical=, but does not require it.
<wgrant> What was your commit message?
<jtv> And I also hate $#%@ unity hiding my windows from me.
<jtv> Hang on
<jtv> Manual submission notes: your submission message needs at least one of [r=...] [rs=...] or [release-critical=...] AND at least one of
<jtv> Ah, it was the other one that's missing.
<jtv> no-qa.
<wgrant> Right.
<jtv> The next attempt failed for yet other reasons.
<wgrant> Even more confusingly, testfix is exempt from no-qa
<lifeless> poolie: the labels are a bit arbitrary
<lifeless> poolie: aside from that may /mean/, I think a label with 'needs' in it does not communicate 'should' or 'might'
<lifeless> poolie: thanks for trying to remember in future
<jtv> Well, that seems to be in.
<poolie> well, i'm not an authoritative reviewer so semantically i can't say "must" anyhow
<poolie> i guess it means "please consider this before merging"
<lifeless> mmm, you're in the review team, so the ui shows you as authoritative
<lifeless> even if it showed you as community, it seems a courtesy not to send conflicting signals ;)
<lifeless> poolie: how has uds been?
<poolie> sorry, i just set the status the way i would in bzr
<poolie> just wanted to save him needing to hmake a second submission
<poolie> good
<poolie> sneezy
<poolie> ubuflu is in fine form
<lifeless> heh, I'm very glad I'm not there then ;)
<poolie> interesting session right now about python versions
<poolie> mm
<jtv> I've been doing this work for too long.  My first reaction to the word "sneezy" was "oh they're planning Ubuntu releases up to S."
<poolie> it seems like a lot of things are internal teem meetings
<lifeless> poolie: you got me curious, doc/developers/code-review.txt doesn't say how bzr uses votes
<lifeless> and points at https://help.launchpad.net/Code/Review
<lifeless> anyhow, enough of that
<lifeless> 10:30 on friday :)
<poolie> the informational or seminar type sessions are not clearly distinguished which means some of them turn out to be less interesting thatn you might think
<poolie> up until wednesday the schedule churned a great deal
<poolie> it was crazy
<poolie> apparently the scheduler in a couple of cases thought it would be clever to schedule things into sessions that had already passed
<poolie> look at all that wasted space last monday!
<poolie> have a good weekend
<LPCIBot> Project windmill-devel build #77: STILL FAILING in 1 hr 9 min: https://lpci.wedontsleep.org/job/windmill-devel/77/
<LPCIBot> Project db-devel build #546: STILL FAILING in 5 hr 17 min: https://lpci.wedontsleep.org/job/db-devel/546/
<gmb> adeuring: Could you take a look at https://code.launchpad.net/~gmb/launchpad/bug-777783/+merge/60910 for me?
<adeuring> gmb: sure
<gmb> Thanks
<wgrant> gmb: Hi.
<gmb> wgrant: Wotcher.
<wgrant> gmb: I'm gardening our various queues, and noticed that you have two approved, unlanded branches from about 6 months ago.
<gmb> Orly?
<gmb> wgrant: I'll take a look, thanks for the heads-up
<wgrant> First section of https://code.launchpad.net/launchpad/+activereviews
<gmb> wgrant: I've marked them both merged (I think they were subsumed by other branches, IIRC). Thanks again.
<wgrant> Thanks!
<wgrant> jelmer: https://code.launchpad.net/~jelmer/launchpad/publisher-use-debian-2/+merge/45011 appears to have been approved just days prior to your defection to bzr -- should I try to land it, or do you recall why it didn't?
<jelmer> wgrant: feel free to land it
<jelmer> wgrant, I didn't because I was worried I wasn't going to be able to deal with any fallout
<adeuring> gmb: r=me
<wgrant> jelmer: Ah, sure. I'll throw it at ec2.
<wgrant> Thanks.
<sinzui> poolie: jml: I have not made questions usable over the API yet. That is probably 2 branches away from being complete. jcsackett landed feature to hide question comments.
<wgrant> sinzui: Hi. I was wondering if you could have a look at the tests in https://code.launchpad.net/~wgrant/launchpad/bug-449561/+merge/60729
<sinzui> okay
<sinzui> wgrant: I am a little apprehensive about the "client.waits.sleep(milliseconds=1000)" line. Is there an element that we could use "client.waits.forElement" instead?
<wgrant> sinzui: That was my question.
<wgrant> sinzui: I am checking for the absence of an element that is created asynchronously after page load.
<wgrant> I don't know if there is some post-load event that I can hook into.
<LPCIBot> Project windmill-db-devel build #275: STILL FAILING in 1 hr 8 min: https://lpci.wedontsleep.org/job/windmill-db-devel/275/
<wgrant> sinzui: This is why I waited for you/deryck.
<wgrant> 'cause I don't really know how to avoid it.
<sinzui> wgrant: can't you use  forElement(HIDDEN_ASSIGN_BUTTON)? forElement is an assert
<wgrant> sinzui: That
<wgrant> That's how the button is in the HTMl.
<wgrant> The JS hook changes it.
<wgrant> I am asserting that it does not execute.
<sinzui> wgrant: We should at least use lp.testing.windmill.constants.SLEEP so that the test is consistent with other tests
<wgrant> I had hoped to catch a deryck of some variety today, but that seems unlikely at this point :(
<gmb> adeuring: Thanks for the review.
<sinzui> derycks now come in variety packs? I would like a green one please
<wgrant> https://lpbuildbot.canonical.com/builders/lucid_lp/builds/942/steps/shell_6/logs/summary is interesting.
<wgrant> Is something pruning /tmp a bit aggressively?
<wgrant> Both rabbit and lp-serve complain about similar things.
<sinzui> wgrant: deryck is listed as on leave in canonicaladmin
<wgrant> This is unfortunate.
<wgrant> I guess it will have to wait until next week.
<sinzui> How can bac be off twice on the same day. If he is time traveling, I would advice not meeting himself. That always causes embarrassment.
<sinzui> wgrant: I don't think you need to wait given the large number of SLEEPs I see in windmill tests. Use the constant. That would be my only requirement to land.
<wgrant> sinzui: Thanks. I know deryck hates it when people use sleep, but I don't see a clear way around it here.
<LPCIBot> Project windmill-devel build #78: STILL FAILING in 46 min: https://lpci.wedontsleep.org/job/windmill-devel/78/
<bigjools> man, LP is fast.
<LPCIBot> Project devel build #715: STILL FAILING in 5 hr 30 min: https://lpci.wedontsleep.org/job/devel/715/
<StevenK> bigjools: You're not being sarcastic
<StevenK> ?
<bigjools> StevenK: a first for me, I know
<benji> jcsackett: the new comment hide/unhide functionality is nice.  I wonder if we should make the hidden comments collapsed or something so they aren't quite so visible to those of us blessed with the ability to see them.
<wgrant> I also think the "Mark as spam" button should probably turn into an icon somewhere near the comment number. But those are just tweaks -- I'm really glad I can remove my scripts now :)
<jcsackett> benji: that's a good idea.
<jcsackett> wgrant: basically you mean move it from the footer to the header?
<wgrant> jcsackett: And make it smaller.
<wgrant> It currently stands out a lot.
<wgrant> Possibly it should be an edit icon next to the number that brings up the AJAX boolean picker.
<wgrant> Like the one used for affects-me-too.
<jcsackett> wgrant: not sure about the latter, i'm not a huge fan of multiclick steps for simple actions. but that's a personal preference.
<jcsackett> you may be right that it fits lp's flow better that way.
<wgrant> Yeah, fair point.
<wgrant> It may be OK to just move it to the header.
<wgrant> But it's a bit obtrusive at the momnet.
<jcsackett> wgrant: yeah, it's a little on the big side now.
 * jcsackett makes note to file bugs on both those points.
<wgrant> Thanks for all your work on this.
<benji> Is the yellow background on "hidden" comments like the one on https://bugs.launchpad.net/launchpad/+bug/1234 common?  It kind of makes the spam stand out, which is ironic. :)
<_mup_> Bug #1234: Gina is an unmaintainable mess of command line options, environment variables and shell scripts <lp-foundations> <Launchpad itself:Fix Released by debonzi> < https://launchpad.net/bugs/1234 >
<wgrant> I'm not sure why a highlight was chosen :/
<jcsackett> benji, wgrant: i'm not sure about that either, but the hidden comment style was already established, and i didn't want to mess with css much at the time.
<benji> sensible
<jcsackett> i think it should be within a branch of work to turn the hidden comment style into a collapsible tho.
<jcsackett> bac referred to the style as "eye-gouging" which isn't too far off the mark.:-P
<benji> lol
<wgrant> Heh
<wgrant> Maybe it's there to convince us to stop the spam from arriving in the first place :)
<benji> good motivation indeed
<jcsackett> i like that theory. it's not ugly, it's "motivational"
<jcsackett> :-P
 * benji is good at creating "motivational" UI.
 * jcsackett laughs.
<benji> I guess that gives whole new meaning to "motivational speaker".
<jcsackett> by this standard, that would be someone throwing things at you from the stage until you made something of yourself, right? :-P
<benji> I think it would be a very unattractive person speaking to you.  I might now qualify to be a motivational speaker.
<LPCIBot> Project windmill-devel build #79: STILL FAILING in 47 min: https://lpci.wedontsleep.org/job/windmill-devel/79/
<poolie> hi sinzui
<poolie> i wasn't trying to rush you
<poolie> it was just something that selene raised in a session here at UDS
<sinzui> poolie. Okay. I have not announced answers of the API because it is clearly not usable. When I can use it for my projects, and  I will blog about it
<jcsackett> sinzui: are you, or do you know, a good person to grab for help on a blueprints question in #launchpad?
<sinzui> No is a good person since no-one who wrote it or uses it works on Lp
<sinzui> jcsackett: I will see if I can help
<poolie> sinzui, i was misinformed and thought they were public
<poolie> anyhow, she thanks you for adding them
<sinzui> poolie: I think many people misunderstood what was exported to allow admins to hide comments. It was a single method on question, not the actual objects
<poolie> yes i misunderstood that
<poolie> well,
<poolie> i think it's reasonable to want it regardless of your work
<poolie> you don't have to do it right now
<jcsackett> wgrant, benji, poolie: just wanted to let you know the various feedback has been captured as filed bugs.
<benji> great! thanks
<jcsackett> poolie: come to think of it, i *think* you provided feedback, i may have mixed up emails. :-P
<ToyKeeper> poolie: Thanks for getting that bug in the queue...
<ToyKeeper> sinzui: Not trying to rush you; I just mentioned it in a session today and poolie wanted to help.  :)
<ToyKeeper> (I'm very happy that questions are being added and I don't have to do it myself!)
<sinzui> I am waiting the QA the change that allows me to add and remove my teams as answer contacts. I am starting the branch that lets you search for question in a couple hours
<ToyKeeper> My goal is to generate a list of open questions for a given team's projects, without having to check each one manually.
<ToyKeeper> ... just making it easier to do support for dozens of projects.
<sinzui> ToyKeeper: then we are in  agreement. That is the actual test I am writing. When the test is complete, then I can blog that I have something useful
<ToyKeeper> Cool.  :)
* adeuring changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: - | https://code.launchpad.net/launchpad-project/+activereviews
<LPCIBot> Project db-devel build #547: STILL FAILING in 5 hr 21 min: https://lpci.wedontsleep.org/job/db-devel/547/
<jcsackett> sinzui: could you, if you get the time, look at https://code.launchpad.net/~jcsackett/launchpad/apocalyptic-messages-0/+merge/60953?
<jcsackett> it's somewhat terrifying, but all mechanical, i believe, and you have a lot of experience with the apocalypse.
<sinzui> jcsackett: I am reviewing it now
<sinzui> jcsackett: 1. line 1008 shows the addition of the services.message package. I think that list is alphabetical.
<sinzui> jcsackett: 2. You create a doc/ dir and moved tests there, but I do not see the test_doc.py that loads the the doctests. I was expecting something like lp.services.mail.tests.test_doc.py
<sinzui> jcsackett: did you forget to commit test_doc?
<jcsackett> sinzui: i think i may have.
<jcsackett> one sec.
<jcsackett> sinzui: yup, forgot the all important 'bzr add'.
<jcsackett> it's pushed up now.
 * sinzui waits
<jcsackett> sorry about that, sinzui.
<sinzui> jcsackett: message.txt does not look special. It does not have a special setup, teardown or layer. Did you try the generic test_doc.py that is used in mail. I think it will work
<jcsackett> sinzui: i don't think so, i just cribbed it from one of the services modules. i'll try out mail's version.
<sinzui> also, that test may play DatabaseFunctionalLayer. The layer it is using starts the librarian, which may not be needed
 * sinzui reads the test
<jcsackett> sinzui: the layer is what it originally had in canonical; that said it may very well work with something else and i'm happy to change it.
<sinzui> Almost everything in canonical.launchpad runs on the wrong layer. Always question why a test runs on the LaunchpadFunctionalLayer
<poolie> thanks jcsackett, that's a good feature to have
<poolie> i'm sure it will save time
<jcsackett> thanks, poolie.
<jcsackett> sinzui: got it.
<sinzui> faboo
<jcsackett> Sickn3ss
<jcsackett> freaking unity lock up....
<LPCIBot> Project windmill-devel build #80: STILL FAILING in 1 hr 9 min: https://lpci.wedontsleep.org/job/windmill-devel/80/
<jcsackett> sinzui: the test needs the librarian; it's doing uploads on msgs. i changed the layer and get all sort of upload errors.
<sinzui> okay
<sinzui> but it does not need a special setup/teardown does it?
<jcsackett> nope.
<jcsackett> cribbing the test_doc in mail works fine but for the layer.
<sinzui> great. r=me the land with that test
<jcsackett> sinzui: cool, thanks.
 * jcsackett ssh's to reboot his dev machine. again.
<LPCIBot> Project windmill-db-devel build #276: STILL FAILING in 49 min: https://lpci.wedontsleep.org/job/windmill-db-devel/276/
<LPCIBot> Project devel build #716: STILL FAILING in 5 hr 27 min: https://lpci.wedontsleep.org/job/devel/716/
<LPCIBot> Project windmill-devel build #81: STILL FAILING in 46 min: https://lpci.wedontsleep.org/job/windmill-devel/81/
<jcsackett> sinzui: i have enabled thumb-click drag on ubuntu on my macbook.
<sinzui> \o/
<jcsackett> sinzui: follow the instructions here, even tho it is not for the 5-1 macbook.
<jcsackett> https://help.ubuntu.com/community/MacBookPro7-1/Maverick#Touchpad
 * sinzui reads
<jcsackett> you will then need to muck about with sensitivity/acceleration some in the mouse settings.
<jcsackett> b/c it becomes VERY speedy moving the cursor around.
#launchpad-dev 2011-05-14
<LPCIBot> Project db-devel build #548: STILL FAILING in 6 hr 21 min: https://lpci.wedontsleep.org/job/db-devel/548/
<LPCIBot> Project devel build #717: STILL FAILING in 5 hr 50 min: https://lpci.wedontsleep.org/job/devel/717/
 * al-maisan reads up on shape files, http://en.wikipedia.org/wiki/Shapefile
<al-maisan> ECHAN
<al-maisan> sorry
<jtv> Any reviewers here to help out with a [testfix]?  https://code.launchpad.net/~jtv/launchpad/testfix-post-conflict/+merge/60981
<wgrant> jtv: Self-review it.
<wgrant> It's only changing tests.
<wgrant> And is pretty much trivial.
<wgrant> However, can't you leave makeDistroSeriesParent to create parent_series itself?
<jtv> wgrant: nothing is trivial when it comes to multi-parent.
<wgrant> Meaning you just have to add a self.factory.makeDistroSeriesParent(derived_series=series) line?
<wgrant> Heh
<jtv> This was that test where I elided makePerson to get under the 2-second bar.
<jtv> So I thought I might as well keep that.
<wgrant> Ah.
<wgrant> Anyway, you have my half-vote.
<jtv> Thanks.  :)
<jtv> I suppose as a compromise, I could mentor your review.  :)
<jtv> By the way, why don't you move to a country where you can legally start reviewing at 18?
<wgrant> Heh.
<jtv> I know, I know, technically it's the drinking that's not allowed.  But one leads to the other.
<wgrant> AU isn't the US :)
<jtv> Good for you.  But then why..?
<StevenK> That test can be made much simpler
<StevenK> dsp = self.factory.makeDistroSeriesParent()
<StevenK> And then use dsp.derived_series rather than derived_series
<LPCIBot> Project windmill-db-devel build #277: STILL FAILING in 1 hr 9 min: https://lpci.wedontsleep.org/job/windmill-db-devel/277/
<LPCIBot> Project windmill-db-devel build #278: STILL FAILING in 1 hr 9 min: https://lpci.wedontsleep.org/job/windmill-db-devel/278/
<LPCIBot> Project db-devel build #549: STILL FAILING in 5 hr 37 min: https://lpci.wedontsleep.org/job/db-devel/549/
<LPCIBot> Project windmill-devel build #82: STILL FAILING in 1 hr 8 min: https://lpci.wedontsleep.org/job/windmill-devel/82/
<LPCIBot> Project devel build #718: STILL FAILING in 5 hr 30 min: https://lpci.wedontsleep.org/job/devel/718/
<LPCIBot> Project windmill-db-devel build #279: STILL FAILING in 1 hr 9 min: https://lpci.wedontsleep.org/job/windmill-db-devel/279/
<LPCIBot> Project db-devel build #550: STILL FAILING in 5 hr 13 min: https://lpci.wedontsleep.org/job/db-devel/550/
#launchpad-dev 2011-05-15
<LPCIBot> Project db-devel build #551: STILL FAILING in 5 hr 28 min: https://lpci.wedontsleep.org/job/db-devel/551/
<lifeless> zomg
<lifeless> who comments on bug 4, really?
<_mup_> Bug #4: Importing finished po doesn't change progressbar <lp-translations> <Launchpad itself:Fix Released by carlos> <Ubuntu:Invalid> < https://launchpad.net/bugs/4 >
<wgrant> Yeah :(
<lifeless> his only comment on lp
<lifeless> poor dude, going to be annoyed noone replies ...
<StevenK> lifeless: Re https://code.launchpad.net/~stevenk/launchpad/karma-store/+merge/60987 , I wasn't comfortable enough to self-review -- however, since you like the change, I'll do so and toss it at ec2.
<lifeless> StevenK: cool
<StevenK> And rabbit fails on ec2.
 * StevenK lp-land's the branch
<wgrant> StevenK: We're in testfix.
<wgrant> Also because of rabbit.
<StevenK> Oh, for the love of!
<wgrant> Failed twice on lucid_db_lp, I haven't bothered to try again.
<wgrant> May back it out tomorrow.
<wgrant> If lifeless doesn't kill me.
<StevenK> Personally, I think that's a wise decision -- it's broken on buildbot, ec2 and jenkins.
 * StevenK grumbles at lp-land
<StevenK> Stop adding [r=..] when the commit message already has them!
 * StevenK tries to figure out how to run bzr-pqm tests.
<lifeless> wgrant: whats the fail ?
<nigelb> launchpad people --> YOU ALL ROCK :)
<nigelb> I would have said this earlier, but my IRC wasn't very stable
<nigelb> Awesome work on reducing timeouts and fixing bug subscriptions :)
<wgrant> lifeless: It fails to start, apparently.
<wgrant> lifeless: See the last $FORGOTTEN failures.
<lifeless> grah
<lifeless> wgrant: possibly regressed
<wgrant> Possibly.
<lifeless> before backing out, run the unit test layer - if that fails its reproducable
<lifeless> [locally, I mean]
<lifeless> if that works, its buildbot specific in most probability
<wgrant> lifeless: it breaks here.
<wgrant> Let me try again, though.
<NCommander> morning all
<NCommander> wgrant: I did some work on LP on my flight ot the US, but I'm a bit confused which API is the current perfered one for creating packages (
<wgrant> NCommander: "packages"? "creating"?
<NCommander> wgrant: fake packages via the testing API
<wgrant> What kind of packages? For what purpose?
<NCommander> wgrant: for the archive any/all skew
<NCommander> brb
<wgrant> NCommander: Source? Binary?
<NCommander> wgrant: both, but I have only gotten as far as source
<NCommander> my code is currently using makeSourcePackagePublishingHistory from the factory and setting the status directly to published.
<NCommander> http://paste.ubuntu.com/607760/
<wgrant> NCommander: I'm not sure that setting dsc_binaries will do anything at all, but that looks reasonable so far.
<NCommander> wgrant: yeah, I'm just finding it a bit hard to figure this out, most of the test cases use the older Soyuz testing code it seems, and I'm coming up short on finding documentation :-/
<wgrant> You really need to know how our model is structured. Do you know it?
<NCommander> wgrant: to an extent, but there are significant gaps in places. I didn't see a useful diagram on the wiki when I looked
<wgrant> https://dev.launchpad.net/Soyuz/TechnicalDetails
<wgrant> You shouldn't have to go near PackageUpload*
<NCommander> I agree I should be able to avoid going near PackageUpload (I just need to create some dummy packages via the testing API). The trick I haven't figure is how to run the publisher. I'd like to avoid it if ipossible, but since we'r edealing with a corner case that needs to be processed when a package is superseded, I think its needed for the test case
<wgrant> You probably should run the publisher's full domination step, yes.
<wgrant> Grep for B_dominate
<wgrant> Don't bother with the other steps.
<wgrant> Step A is just setting everything to Published, C is index generation, D is Release generation.
<wgrant> So only B is relevant to you.
<NCommander> Right. that will caused packages to go from PENDING->PUBLISHED, and the previous published packages to go to SUPERSEDED, right?
<wgrant> A does PENDING->PUBLISHED.
<wgrant> B does PUBLISHED->SUPERSEDED
<wgrant> A also writes the package files to disk, so you want to avoid it if possible.
<NCommander> so for a brief peroid, both the old package, and the new package are published
<wgrant> That's right.
<wgrant> It is rather odd.
<NCommander> Ok, so then I can just create the new binaries with SUPERSEDED
<wgrant> PUBLISHED, you mean?
<NCommander> I was under the impression things would break if I had multiple packages set in the PUBLISHED state
<NCommander> er
<NCommander> yeah
<NCommander> sorry, my caffiene level is only starting to reach critical mass
<wgrant> Stuff shouldn't break.
<wgrant> Heh.
<wgrant> Are you also jetlagged?
<wgrant> I presume so.
<NCommander> wgrant: I'm still in transit
<wgrant> Ah!
<wgrant> That is unfortunate.
 * NCommander had a 22 hour layover in NYC
<NCommander> I'm now on my way to SFO, then another hop to PDX
<wgrant> 22h domestic layoverâ½
<NCommander> international->domestic
<NCommander> Dropped in to see my mom
<wgrant> Ah, I see.
 * NCommander writes the fake binary bit
<NCommander> wgrant: it seems to me that BPPHs are not (directly) linked to their SPPH. As such, I'm confused on how LP tied source and binaries together
<wgrant> NCommander: Yeah, they're not directly linked because they can be overridden etc. separately.
<wgrant> NCommander: So it goes through the BPB and SPR.
<wgrant> And finds publications in the same series and archive at the other end.
<NCommander> wgrant: makes sense in an odd sorta way. Reading on that page you linked me and a few comments, I get the impression SUPERSEDED packages are still published until they are removed (either by archive admin or automated processes)
<wgrant> NCommander: They are removed from the indices immediately. The files are not removed until they are unreferenced and process-death-row is run.
<wgrant> NCommander: The indices are generation from all publications with status PUBLISHED.
<NCommander> Ah. and the difference between SUPERSEDED and PUBLISHED becomes clear
<NCommander> Thanks
 * NCommander vaguely wonders if we need a SUPERSEDED_BUT_STILL_PUBLISHED state :-)
 * NCommander ducks
<wgrant> There is some argument that we should include superseded packages in the indices until they are removed.
<wgrant> That is easy to do, and possibly beneficial for a couple of reasons.
<wgrant> But it would require much thought.
<NCommander> plus it makes this trivial to solve
<wgrant> Right.
<wgrant> Well, not trivial.
<NCommander> well, mor estraightforward
<wgrant> But much easier: you just have to prevent judgeSuperseded from setting scheduleddeletiondate for the binaries that you want to keep.
<NCommander> wgrant: ugh, I just realized that since I need to make BPBs to tie my sources together, I also need to desginate the arch-all architecture. I didn't see an obvious way to do that (or will it magicially get set to 'i386')
<wgrant> NCommander: Are you calling makeDistroArchSeries manually?
<StevenK> NCommander: distroseries.nominatedarchindep =
<NCommander> yes
<wgrant> What StevenK said.
<StevenK> If you aren't making a distroseries, you can fetch it by distroarchseries.distroseries
<NCommander> Unauthorized: (<DistroSeries u'lucid'>, 'nominatedarchindep', 'launchpad.Moderate')
<NCommander> Something tells me I can't just set that property directly
<wgrant> Ah. Use removeSecurityProxy
<StevenK> I've been able to in tests ...
<wgrant> StevenK: In ZopelessLayer, maybe.
<StevenK> Ah, yes
<StevenK> NCommander: The other thing you can do is fetch the admin from IPersonSet and run that as the admin
<NCommander> StevenK: is there a preference in test cases?
<StevenK> NCommander: I personally prefer logging in as a admin, since I find removeSecurityProxy distateful, but not really -- as you see fit -- but your reviewer might prefer you don't use it.
<NCommander> StevenK, wgrant: as a thought, can we trust the depends field in a BPR on a way to determine if a package is skewed? it would also pave the way to keep development releases in an always installable state ...
<StevenK> NCommander: Now that I've thought about it -- why?
<StevenK> NCommander: And I think it's dangerous -- think about the case of a transition. If we can't remove the binaries of a package since it would make the archive uninstallable, how are we able to proceed?
<NCommander> StevenK: it was an idea that simply occured to me since I've always liked the 'always-installable' aspect of Debian testing. As for transitions, we don't remove NBS'ed binaries until the transition is complete anyway
<StevenK> NCommander: No, but we have the option to.
<StevenK> NCommander: However, yes, the BPR.depends field is to be trusted.
<NCommander> k
<NCommander> What's the proper way to get self.logger to exist? A lot of tests use it, but I don't see an obvious way to make it be defined
 * NCommander feels that STP sets up a lot of this ...
<cjohnston> jml: bug 255024 - if you remember we talked about that.. I've pushed a fix and just wanted to get your opinion.. And I'll take anyone elses opinion as well
<_mup_> Bug #255024: "Participation essential" is "Required" when subscribing <lp-blueprints> <trivial> <ui> <Launchpad itself:Triaged> < https://launchpad.net/bugs/255024 >
<maxb> Hrm. What could cause a commented bug not to appear in a user's commented bugs page?
<maxb> Case in point is the spam comment in https://bugs.launchpad.net/ubuntu/+source/radeontool/+bug/601539
<_mup_> Bug #601539: Please sync radeontool 1.6.1-1 (main) from Debian unstable (main). <workflow> <radeontool (Ubuntu):Fix Released> < https://launchpad.net/bugs/601539 >
<lifeless> wgrant: hi
<lifeless> wgrant: interesting
<cjohnston> hey lifeless
<lifeless> maxb: wow, I'm blind now :)
<lifeless> cjohnston: hi
<lifeless> maxb: if message.owner is not them, though that would on the face of it not make much sense
<lifeless> or their message index is nonzero
<lifeless> there is a denormalised field in there
<lifeless> wgrant: rabbit test passes for me (but I haven't merged up with trunk yet)
<lifeless> wgrant: who should get ppa build notifications (see bug 782924)
<_mup_> Bug #782924: Send e-mail notification when PPA builds are finished <Launchpad itself:Incomplete> < https://launchpad.net/bugs/782924 >
#launchpad-dev 2012-05-07
<StevenK> wgrant: http://pastebin.ubuntu.com/972558/
<wgrant> What's the generated SQL?
<wgrant> Need to ensure that there's no ORDER BY etc.
<StevenK> 0-21@SQL-main-master UPDATE Branch SET information_type=CASE WHEN transitively_private THEN 4 ELSE 1 END WHERE Branch.id IN (77)
<wgrant> Huh, interesting that it evaluates that early.
<wgrant> I would have expected it to use a subselect.
<wgrant> Still, okay either way
<StevenK> Just checking what happens with bigger chunks
<StevenK> 0-4@SQL-main-master UPDATE Branch SET information_type=CASE WHEN transitively_private THEN 4 ELSE 1 END WHERE Branch.id IN (7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 75, 76)
<wgrant> Mm, might as well turn it into an explicit subselect.
<wgrant> Oh, but it's wrapped in a method, isn't it.
<wgrant> So you'd have to duplicate.
<wgrant> Not worth it, just go with this way
<StevenK> wgrant: Yup, let me commit, push and pqm-submit
<wgrant> StevenK: How goes QA?
<StevenK> wgrant: Trying to get access to the staging mailbox
<wgrant> StevenK: Don't you only need inbound mail?
<StevenK> wgrant: I thought new@bugs ... would reply back?
<wgrant> StevenK: Sure, but why do you care about that?
<wgrant> You didn't change the response AFAIK
<StevenK> wgrant: Because I want to see if ' informationtype proprietary' dies horribly.
<wgrant> Ah
<StevenK> wgrant: Or you can rummage inside the staging mailbox and see if the malonehandler yelled at me.
<wgrant> I'm currently wondering if stub will hate me for adding more indices to bugtaskflat :)
<wgrant> Well
<wgrant> I think Thunderbird hung before it even hit the network to retrieve the mailbox
<wgrant> How nice of it
<StevenK> Haha
<StevenK> wgrant: Thunderbird still hates you?
<wgrant> It's still restarting.
<wgrant> I think I might need to vacuum some SQLite.
<wgrant> Let me delete indices and see what happens./
<lifeless> wgrant: got time for a brief call ?
<wgrant> lifeless: Sure
<lifeless> hows your skype install these days ?
<wgrant> Calling...
<jtv> I wonder who knows about PQM these days?  It won't let me land branches.
<wgrant> jtv: What's it saying?
<jtv> 'Failed to verify signature: gpgv exited with error code 2'
<wgrant> Sure you're using the right key and email address?
<jtv> I don't see any sign that I'm not.
<jtv> I registered a key for it, which is the same key that I get prompted for to sign my revisions, and for the same email address that the email is addressed to and that bzr is set up to use.
<StevenK> jtv: Do you know which key PQM is using, since PQM is entirely seperate.
<jtv> It's not basing it on the email address I'm using?
<wgrant> jtv: It has a keyring.
<jtv> So I need to have this key registered somewhere besides Launchpad?
<StevenK> jtv: Yup. If it's a new key, you need to file an RT to get the webops to add it to PQM's keyring.
<wgrant> jtv: Yes. PQM doesn't know about Launchpad.
<jtv> Arrrh.  Thanks.
<jtv> (It used to let me land branches with this key, although ec2 wouldn't)
<StevenK> jtv: I have one key, it saves thinking about it.
<StevenK> wgrant: Thunderbird still hates the world?
<jtv> So did I â but then the trouble started.  First ec2 stopped accepting it, and now apparently pqm as well.
<jtv> Sorry, actually it was more complicated than that.
<jtv> One key is not necessarily simpler.
<StevenK> jtv: I just transistioned from a 1024 DSA key to a 4096 RSA key, and I know that ec2 and pqm-submit/lp-land use completly different codepaths to get the signing key.
<jtv> Yeah, what happened is that one of the earlier changes forced me to add a new key, and then ec2 had trouble with that.  But now that trouble seems to have extended to pqm.
<StevenK> jtv: How do you get forced to generate a new key?
<jtv> My old one was no longer accepted.
<wgrant> Howso?
<spm> jtv: that error message is also what you'll get if your email address order in the 'right' key changes.
<spm> but seeing as that requires one of us to change your key on pqm, probably not that.
<jtv> spm: sorry, I don't followâ¦ if my email address order in the "right" key changes?  What does that mean?
<spm> jtv: eg mine: http://keyserver.ubuntu.com:11371/pks/lookup?search=0x19273D1D1CC65F5BBC798AA93D8EAB2DEBE3BC06&op=index <== if I changed the primary contact email in that, pqm will not recognise it as valid.
<jtv> Ah.  No, that can't have happened here.
<spm> the flip side, is if the address you send as changes; it has a similar effect too
<lifeless> jtv: did the key expored ?
<lifeless> expire
<jtv> No.  It was a while ago but if I recall correctly, the problem was that it wasn't attached to my canonical address.
<StevenK> You can edit the key and 'adduid'
<wgrant> StevenK: Error message:
<wgrant> Proprietary bugs are forbidden to be filed via the mail interface.
<StevenK> wgrant: Win!
<StevenK> wgrant: There should be another message in there
<wgrant> What should it look like?
<wgrant> There's about 20000 other messages in there.
<wgrant> And that's after I deleted 100000
<StevenK> wgrant: It was exactly the same as the Proprietary mail, but set it to Unembargoed Security.
<wgrant> StevenK: That's a bugnotification.
<cjwatson> jtv: It's allowed to have a PQM key whose primary UID isn't your Canonical address, but you do need to get ops to set your primary UID as permitted.
<StevenK> wgrant: What's the bug number?
<wgrant> StevenK: We don't send bug notifications automatically.
<StevenK> Ahh.
<cjwatson> (I deduce that it's allowed because my key is that way.)
<wgrant> So I'd look for it in the web UI
<jtv> cjwatson: I think ec2 forced me to use my canonical address.
<wgrant> It doesn't.
<StevenK> wgrant: Yeah, I'm doing so.
<wgrant> My primary isn't canonical.com
<cjwatson> DOesn't for me.
<StevenK> Depends on the mail setup.
<wgrant> My canonical.com address isn't primary for anything anywhere.
<wgrant> And it works for me :)
<jtv> wgrant, cjwatson: I didn't say "as primary"
<cjwatson> It's trivial to add a Canonical UID to an existing key.
<StevenK> wgrant: qa-ok, finally.
<wgrant> StevenK: Great.
<StevenK> wgrant: Thanks for the assist.
<jtv> cjwatson: maybe it is, but for other reasons I did not want to do that.
<lifeless> http://www.slideshare.net/dings/dont-play-games-with-me-promises-and-pitfalls-of-gameful-design
<lifeless> worth a read
<StevenK> wgrant: We just need jml's QA now.
<wgrant> StevenK: Unfortunately I believe achuni is likely to be at UDS.
<wgrant> But hopefully not.
<StevenK> We can deploy r15204 if we want.
<StevenK> In terms of my stuff, that's the mail command and adding information_type to IBranch.
<danhg> hey all
<rick_h_> howdy danhg
<nigelb> StevenK: ping, around?
<StevenK> nigelb: O hai
<nigelb> Heya, got a few mins for a PM?
<StevenK> nigelb: Sure.
* bac changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: - | Firefighting: - | Critical bugs: 3.47*10^2
* benji changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: benji | Firefighting: - | Critical bugs: 3.47*10^2
<abentley> rick_h_: per deryck's email, he and adeuring aren't standupping today.  Would you like to stand up anyhow?
<rick_h_> abentley: sure
<czajkowski> morning
<rick_h_> party czajkowski
<czajkowski> sleepy czajkowski this timezone lark is most confusing :)
<rick_h_> hah, timezones do suck. My wife says melatonin ftw (she's a doc) and helped when I hit budapest I think
<lifeless> morning
<czajkowski> morning
<lifeless> flacoste: how is UDS ?
<czajkowski> I've gottent to mee huw and rvba :)
<czajkowski> niec to put the face to the names
<czajkowski> *gotten
<lifeless> cool
<czajkowski> lifeless: flacoste isnt here this week
<flacoste> lifeless: on the phone with deryck
<lifeless> czajkowski: oh, I thought he was ;)
<lifeless> flacoste: o/
<lifeless> gary_poster: around ?
<gary_poster> lifeless, yes.  hey.  I got your email.  I don't have any other branches to land, but I wanted to see if the trunks would all work for us if we put them in a PPA before I replied.  You could go ahead and release optimistically: that will make it easier for me to test
<lifeless> gary_poster: you read my mind :)
<gary_poster> :-) ok cool
<lifeless> gary_poster: testtools 0.9.15 is up
<gary_poster> ok cool thanks lifeless will try soon
<lifeless> gary_poster: did you end up trying an HVM AMI?
<gary_poster> lifeless, no we still have a card for it.  Could well be this week.
<flacoste> lifeless: do you want to catch-up?
<lifeless> flacoste: I'd love to
<lifeless> flacoste: when suits ?
<flacoste> lifeless: now?
<lifeless> sure
<lifeless> hangout? skype?
<lifeless> flacoste: ^
<nigelb> 6/ws 27
<nigelb> grrr
<abentley> deryck: btw, listing queued jobs is working, and the next step, finding "ready" jobs that aren't queued is also working.
<deryck> abentley, nice!  good news then.
<abentley> deryck: Also, bug filed: https://github.com/ask/kombu/issues/126
<deryck> abentley, man, that's a nice bug report!  :)  Code and everything! :)
<gmb> Has anyone ever seen the following error whilst trying to connect to launchpad.dev:
<gmb> An error occurred during a connection to launchpad.dev.
<gmb> SSL received a record that exceeded the maximum permissible length.
<gmb> (Error code: ssl_error_rx_record_too_long)
<gmb> Hmm. Could be a proxy problem...
<lifeless> .dev? no.
<lifeless> or rather, not I
<mwhudson> gmb: that can mean one side thinks it's doing ssl and the other side doesn
<mwhudson> 't
<mwhudson> gmb: does http://launchpad.dev:443/ "work" ?
<gmb> mwhudson, Checking now. It works from within the host machine (this is on ec2 btw)
<gmb> mwhudson, Ahh, I see what's happened.
<gmb> rf-setup clobbered my apache config.
<gmb> Thanks for the hint.
<lifeless> orly ? :P
<lifeless> gmb: having a fun time at UDS ?
<gmb> lifeless, So far :)
<gmb> Ask me again after the clinic tomorrow.
<wgrant> gmb: What broke about the config?
<wgrant> gmb: `make LISTEN_ADDRESS=* install` should now do all the remote access stuff for you
<sinzui> jcsackett, mumble?
<czajkowski> aloha
<czajkowski> wgrant: a ticket has been assinged to me in RT from toykeeper you were involved with https://support.one.ubuntu.com/Ticket/Display.html?id=13727 user still having issues and has followed your instructions.
<wgrant> czajkowski: Looking
<czajkowski> wgrant: thanks
<wgrant> czajkowski: Huh, how did that site for 3 weeks without being bounced back to us :(
<wgrant> s/site/sit/
<czajkowski> wgrant: it only got assigned to me yesterday
<czajkowski> and has been sitting in the other IS queue till then
<wgrant> Dammit
<wgrant> That OOPS has expired
<wgrant> czajkowski: Anyway, looks like the usual Active/Deactivated thing. He was Active, needed to be Deactivated to get the email address reactivation behaviour.
<czajkowski> ah ok
<czajkowski> wgrant: confused as to which account https://launchpad.net/~nsates  or https://launchpad.net/~enesates/
<wgrant> czajkowski: nsates was merged into enesates
<czajkowski> ah
 * StevenK staples wallyworld to the Internet.
<lifeless> I think you may want to upgrade to a rivet gun.
<StevenK> I might hit vital bits.
<bigjools> it won't make any difference
<lifeless> jelmer: any chance you could look at bug 987938 ?
<_mup_> Bug #987938: subunit trunk packaging breaks subunit-* commands <subunit:Incomplete> < https://launchpad.net/bugs/987938 >
<lifeless> gary_poster: subunit releases are up; sorry for the delay.
<gary_poster> cool thanks lifeless
<lifeless> cutting testrepository in a second
#launchpad-dev 2012-05-08
<lifeless> gary_poster: testrepository is up
<wgrant> Huh
<wgrant> Native postgres synchronous replication can be opt-in at the transaction level. Didn't realise that.
<lifeless> wgrant: as in 'how much risk to take' ?
<wgrant> lifeless: Yeah. You can say that a particular commit should or should not wait for replicas.
<wgrant> I assumed it was all-or-nothing.
<StevenK> wgrant: O HAI, Mr. OCR
<StevenK> wgrant: https://code.launchpad.net/~stevenk/launchpad/information_type-notification-flag/+merge/105014
<wgrant> StevenK: I don't appreciate your lies.
* wgrant changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: wgrant | Firefighting: - | Critical bugs: 3.47*10^2
<StevenK> Which lie in particular?
<wgrant> The one about me being OCR
 * wallyworld_ sighs. 18 minutes and still no mp diff
<nigelb> wallyworld_: what happened to using rabbitmq for that?
<wallyworld_> nigelb: ?
<wallyworld_> for what?
<nigelb> wasn't there something to use it for diff generation?
<wallyworld_> yes. the branch first needs to be scanned
<wallyworld_> and that job is timing out
<wallyworld_> there's something in the branch it doesn't like
<nigelb> ah.
<StevenK> wgrant: I had a test failure, so I've made sure the notifications respect the flag no matter the setting of the UI flag: http://pastebin.ubuntu.com/975101/
<wgrant> StevenK: LGTM
<adeuring> good morning
<jtv> benji: I don't mean to seem ungrateful for your fantastic reviews, but would you mind having one more look at this one?  I made some changes that didn't quite make sense as a separate branch.  https://code.launchpad.net/~jtv/launchpad/bug-994650-scrub-pofiletranslator/+merge/104866
<jtv> (Look for âLate-breaking change:â)
<StevenK> jtv: Why experimental loop?
<jtv> StevenK: isn't that appropriate?
<jtv> I still need to test it on staging.
<StevenK> jtv: If you're not going to land it until you've tested it on staging, then I would recommend you add it to the 'normal' loop.
<jtv> Oh.  OK.
<StevenK> If you're still worried, you can control execution with a feature flag.
<jtv> (Have we passed bug 1 million yet?)
<_mup_> Bug #1: Microsoft has a majority market share <ubuntu> <Clubdistro:Confirmed> <Computer Science Ubuntu:Confirmed for compscibuntu-bugs> <LibreOffice Productivity Suite:New> <dylan.NET.Reflection:Invalid> <dylan.NET:Invalid> <EasyPeasy Overview:Invalid by ramvi> <elementary OS:Confirmed> <Ichthux:Invalid by raphink> <JAK LINUX:Invalid> <LibreOffice:In Progress by bjoern-michaelsen> <Linux:New for brunovam> <Linux Mint:In Progress> <The Linux OS
<StevenK> The workitems tunable loop does that.
<jtv> No mup, not what I meant.
<StevenK> bug 1000000
<wgrant> Yeah, I added feature flag support a few months back, so it's probably easier now to use that than --experimental
<wgrant> Because if it isn't broken, you would need a subsequent landing and $LOTSOF hours to get it working on prod.
<wgrant> With a feature flag you just set the feature flag.
<jtv> Well I don't think there's too much point to landing this unless I know the garbo job can do a decent job first anywayâ¦
<StevenK> Then shift it and get the webops to run garbo-daily with -vv goodness
<jtv> Shift?
<StevenK> jtv: Sorry. What I mean is move it from experimental, and then have the webops run garbo-daily by hand with -vv.
<jtv> webops, would you be available to try that now?
<jtv> (I've already pushed the branch)
<gnuoy> jtv, sure
<jtv> Let me check that I've really pushed it to all the right branches...
<jtv> gnuoy: I guess the change where POFileTranslator.latest_message is allowed to be null is still in place?
<gnuoy> it is
<jtv> Great.  Then if you would, please merge lp:~jtv/launchpad/combined-async-pofiletranslator and run garbo-daily with -vv!
<rick_h_> wgrant: you EOD? updating OCR
* rick_h_ changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: rick_h | Firefighting: - | Critical bugs: 3.47*10^2
<wgrant> rick_h_: Ah, yeah, a few hours ago, sorry.
<rick_h_> wgrant: np, just didn't want to delete you unnecessarily
<wgrant> jml: Hi, are you around?
<wgrant> jml: Your QA is now blocking everything, and achuni seems perfectly aligned to avoid my timezone.
<wgrant> jml: If I hear nothing by my morning I'll revert both -- I guess retrying after UDS would work better.
<benji> jtv: if you could still use a review, I can do it here in a bit
<jtv> benji: wonderful, thanks.  I'd feel more confident.  The actual code is the same, but it's all moved, mostly towards the left-hand margin.  Methods are now functions.
<benji> jtv: cool, I'll look at it in the next half hour or so
<jtv> Thanks.  I won't be here for too much longer, unfortunately, but I may be able to check back in later.
<noodles775> wgrant: achuni was trying to find someone at UDS to ask about that.
<rick_h_> jcsackett: morning, around for ?
<noodles775> wgrant: afaik, we're ready to QA it, it's just that the QA person was in London (public holiday yesterday)
<noodles775> wgrant: I'll chase it up now. When you say both, what's the other branch? (I'm only aware of jml's one branch?)
<wgrant> noodles775: Ah, forgot about the public holiday.
<noodles775> wgrant: yeah, so did I when I did the developer QA and handed over.
<wgrant> noodles775: jml has a second branch which doesn't have special QA requirements, but it renames Archive.commercial
<wgrant> So a straight revert of the first branch won't work, as code it removes relies on .commercial
<noodles775> wgrant: can you point me at the second branch so I can make sure it was on staging.lp.net when I QA'd the first one? Thanks!
<wgrant> http://bazaar.launchpad.net/~launchpad-pqm/launchpad/stable/revision/15209
<wgrant> db-devel r11573
<noodles775> wgrant: excellent, that's what staging was running when I QAd.
<wgrant> noodles775: (sorry, wasn't aware you were a SCA QA victim these days, or I would have pinged you directly)
<wgrant> Great.
<noodles775> wgrant: no problem - I just wanted to make sure our QA person would be able to do their QA straight away. I should have put my notes on the bug rather than email, just added them now.
<wgrant> noodles775: Thanks, those emails clarify everything.
<noodles775> wgrant: np
<wallyworld> abentley: hi. what do i need to do to have celery run correctly for lp.dev? i have code where tests using CeleryLayer work but I want lp.dev to run the jobs also. i have added the jobs to jobs.celery.enabled_classes
<abentley> wallyworld: You would run bin/celeryd --config  lp.services.job.celeryconfig
<wallyworld> abentley: ah, i ran celeryd but didn;t know the correct config arg. thanks
<wallyworld> abentley: should we make that part of make run now?
<abentley> wallyworld: no, we're still working on getting it ready for production use.
<wallyworld> abentley:  ok. np. i have a new job which remove subscriptions for inaccessible bugs and branches which is celery only. when do we think lp.net will support that?
<abentley> wallyworld: I hope we can do it this week.
<wallyworld> \o/ sounds good. thanks
<StevenK> abentley: It's in use and working on qas/staging?
<wallyworld> StevenK: btw memtest86+ ran a few passes and no errors. but still lots of crashes :-( bollocks and all that
<abentley> StevenK: We're using it to do branch scans on staging only, and because fast lane/slow lane is only partially implemented, I cannot recommend using that setup in production.
<StevenK> wallyworld: Hmmm.
<wallyworld> yeah :-(
<wallyworld> very frustrating
<wallyworld> the only change i can se is adding the new memory
<wallyworld> might have to revert to the old to see if things improve
<deryck> Morning, everyone.
<abentley> deryck: Morning.
<StevenK> wallyworld: Try folding@home to see if that dies horribly?
<wallyworld> StevenK: what do you mean?
<StevenK> wallyworld: folding@home is an application you can run to do protein folding for the project. Extremly CPU intensive.
<StevenK> wallyworld: It's not packaged, but they have Linux binaries for download
<wallyworld> StevenK: ok. i will look at that. right now, i am writing some code and it's not too bad so will not look a gift horse in the mouth. will do as much as i can while the sun shines
<StevenK> Heh
<StevenK> wallyworld: Sounds better than what I'm doing.
<wallyworld> what you doing?
<StevenK> wallyworld: Installing Windows.
<wallyworld> StevenK: WTF? hahahahahaha. why?
<wallyworld> galaxy s2?
<StevenK> wallyworld: Steam
<wallyworld> StevenK: can't wait for the linux port? :-P
<StevenK> wallyworld: Well, the Linux port is only likely to bring Source games.
<StevenK> At least in the short term.
<wallyworld> yeah i know, just trolling
<jcsackett> rick_h_: around now.
<StevenK> wallyworld: It's better than what I heard before which was Valve going "Linux? HTF do you spell that?"
<wallyworld> lol
<rick_h_> jcsackett: ok, I did your review, wasn't sure on a couple of points so tried to note my assumptions, have 10min to do a call if you need now
<rick_h_> jcsackett: or if you need to chat we can do so after out stand up
<jcsackett> rick_h_: ping me after your standup, i'm reading your comments now.
<rick_h_> jcsackett: ok, will do
<rick_h_> jcsackett: ping
<jcsackett> rick_h_: pong.
<rick_h_> jcsackett: howdy, quick hangout?
<jcsackett> rick_h_: yup. lemme go grab some coffee real quick and i'll be on g+ in a minute.
<rick_h_> k
<rick_h_> jcsackett: just one heads up on the visible thing, you need to rely on that css class, but you have to implement it http://yuilibrary.com/yui/docs/widget/#attributes see the 'visible' notes there
<rick_h_> jcsackett: you can see we implemented it in the indicator-core.css file
<benji> rick_h_: one for your list-o-reviews: https://code.launchpad.net/~benji/launchpad/bug-994694/+merge/105090
<rick_h_> benji: k, on call will grab it shortly
<benji> rick_h_: perfect
<rick_h_> benji: r=me
<gmb> czajkowski: Where y'at?
<sinzui> rick_h_, I award you a gold star â for demolishing doc/bugtask.txt
<czajkowski> gmb: ballroom G
<gmb> rick_h_: Also, if what sinzui just said is true, I owe you beer.
<sinzui> My cat is bring me her toy so that I can throw it. I have never had a cat that behaves like a dog
<czajkowski> sinzui: strange thins happen with you :)
<rick_h_> gmb: huh?
<gmb> rick_h_, I have a long-running hate-hate relationship with doc/bugtask.txt
<rick_h_> sinzui: oh, well it's not done yet...deryck is making sure I didn't fubar it yet
<rick_h_> gmb: no freaking kidding...thank the LoC policy for making me do it to submit my bugfix
<rick_h_> and deryck for being awesome and accepting the abnormal giant pita review for me
<sinzui> rick_h_, I doubt that, though there is a small chance that the new tests duplicate the tests in lib/lp/bugs/model/tests/test_bugtask.py
<rick_h_> sinzui: there is no bugtask, that's why they were in bugtaskset. I couldn't find a clean line to split bugtaskset/bugtask very well
<rick_h_> well I didn't think there was one, I shouldn't make blanket statments
<rick_h_> oh yea...so there is stuff there :/
<rick_h_> crap, totally missed that because the files aren't test_XXX when I did my searches
<rick_h_> *sigh*
<sinzui> I think there is overlap with conjoined bugtasks. I see your tests and the existing tests both check that attributes are synced for example
<jcsackett> cool.
<jcsackett> ...and window manager fail.
<rick_h_> doh?!
<rick_h_> jcsackett: awesome not liking you?
<jcsackett> rick_h_: awesome and focus-follows-mouse playing together poorly.
<rick_h_> hmm, focus follows my mouse, is that a special setting?
<sinzui> rick_h_, I totally missed the whole module. for a year!
<jcsackett> rick_h_: no, no special setting. awesome is supposed to do focus-follows-mouse automatically. and now it's not.
<jcsackett> and i don't know why. :-p
<rick_h_> ugh, sucky. wfm but that won't help
<jcsackett> rick_h_: contemplating hopping back to the dark side and going with xmonad.
<rick_h_> yea, buddy moved there and is happy
<rick_h_> if awesome breaks for me that's where I'm heading nexet
<lifeless> gary_poster: hi there, any news on the testtools/subunit/testrepository releases goodness?
<gary_poster> lifeless, well, progress:
<gary_poster> I have copied over the testing cabals packages over to yellow's ppa (since that's what we currently use for buildbot integration).  I established locally that the Python 3 build issue still stands, but I think you already knew that.  I am trying to gather some information to file two weird bugs (one of which might have a chance of being fixed by those three packages) and then I was planning on taking today's buildbo
<gary_poster> t master down and installing the new packages and giving them a spin there.  So, I should have something for you be my EoD
<lifeless> ok cool
<lifeless> jelmer: hi thar
<lifeless> jelmer: did you see my ping about the python3 issue gary_poster refers to above, above ?
<rvba> Laney: lib/lp/bugs/browser/tests/test_bugtask.py
<lifeless> allenap: hey, did you still want to talk? I didn't see the threatened email :)
<czajkowski> cjohnston: aloha
<cjohnston> howdy
<czajkowski> StevenK: you about ?
<cjwatson> Could I have a DB patch number for bug 990219?  The substantive change is "ALTER TABLE Packageset ADD COLUMN score INTEGER DEFAULT 0 NOT NULL;"
<_mup_> Bug #990219: Reprioritize package build scores based on packageset <feature> <packagesets> <soyuz-build> <Launchpad itself:Triaged by cjwatson> < https://launchpad.net/bugs/990219 >
<lifeless> let me just check the size of the table
<cjwatson> Wow, shiny extra text on bug status changes
<cjohnston> isnt it though cjwatson
<cjohnston> takes up my whole screen almost
<cjwatson> lifeless: I think it's fairly tiny
<lifeless> 2209-18-0
<cjwatson> Thanks
<lifeless> cjwatson: it is, but anything with a default has to be special cased.
<lifeless> because its a table rewrite to apply it.
<cjwatson> Right, I guessed
<cjwatson> It'll be fast enough?
<lifeless> yeah
<lifeless> 150 rows
<lifeless> doubt you could measure the difference
<lifeless> its likely a single page :)
<gary_poster> lifeless, I have a mildly interesting subunit story for you.  Briefly, the subunit stream in buildbot stopped processing tests in one run after about 3000 tests.  After investigation, it turns out that processing stopped right before this test: http://pastebin.ubuntu.com/976408/ .  Clearly this is a problem in the stream: the "No handlers...\n" log message polluted the line and subunit did not recover.  A question f
<gary_poster> or me (and you if you want) is whether we can get those log messages out of the stream.  A question more squarely for you is if subunit could be more robust in the face of this kind of mess.  We're in heuristic land, but AIUI subunit already has a bit of mess-cleaning code in it already.  Thoughts?
<gary_poster> That was brief for an email, but not so brief for IRC. :-) oh well
<lifeless> :P
<lifeless> so, this is a recurring problem
<gary_poster> it is for us, but rare
<gary_poster> this is #2 in all of my runs
<lifeless> subunit users encounter it from time to time
<lifeless> uhm
<lifeless> I'll skip the backstory for now
<gary_poster> :-) k
<lifeless> the short answer is: either stop things spewing to stdout at random times (e.g. during __del__ handlers triggered during full gc mid-stream-command-output)
<lifeless> or move the stream to write to !stdout
<lifeless> or move sys.stdout to !stdout
<gary_poster> right
<lifeless> I don't think you can reliably resync the stream; I haven't tried very hard to do so though
<gary_poster> moving the steam to !stdout seems the easiest to make robust
<gary_poster> though still not trivial
<lifeless> moving the stream to !stdout has the downside that 'test.py | subunit-thing' becomes harder to construct
<gary_poster> right
<lifeless> moving sys.stdout to be e.g. stderr will preserve messages sent to it, keep the pipe construction easy, and should be robust - few if any modules will take a reference to sys.stdout implicitly
<gary_poster> yeah, I was thinking along those lines
<gary_poster> so bin/test --subunit could do the switch
<gary_poster> and we'd only ( :-/ ) have to mess with our zope.testing fork
<lifeless> shouldn't need to mess with z.testing for this
<lifeless> well, maybe
<gary_poster> no?
<lifeless> I forget where the exact bits are buried.
<gary_poster> the subunit flag is processed there I think
<lifeless> if we have to, we should let the pipe to write to be customised in the API, so that we don't have to fiddle again later.
<lifeless> erm
<lifeless> pipe to write to isn't hte thing being customised.
<lifeless> I claim 0704.
<gary_poster> heh
<lifeless> so yeah, we probably want to add an option or something though, rather than just changing the behaviour fullstop.
<gary_poster> You don't think that --subunit itself is enough of a flag?  if it isn't, then this would be a ./bin/test --subunit --do-not-make-the-subunit-stream-broken
<lifeless> gary_poster: the No handlers...\n thing is probably due to an object with a __del__ method calling into logging
<gary_poster> makes sense, but I'd really rather not have to have the squad find all possible cases of this
<lifeless> I'm with you there
<lifeless> I'm suggesting an API option, ./test.py does command line processing already
<gary_poster> API where?
<lifeless> one sec
<gary_poster> np
<lifeless> lib/lp/services/testing/parallel.py uses subunit.run directly
<lifeless> but I think we can delete that module entirely now
<lifeless> digging
<cjohnston> rick_h_: ping
<lifeless> gary_poster: I guess on testrunner.run
<lifeless> gary_poster: the zope.testing API is one I find a bit hard to reason about
<lifeless> gary_poster: (zope.testing.testrunner.run)
<gary_poster> lifeless, ok cool.  So...we still expose this through a command line though.  Do you agree that the commandline --subunit flag will automatically switch to stdout to stderr?
<lifeless> gary_poster: ultimately, its up to you whether its unconditional or an optional
<gary_poster> ok
<allenap> lifeless: I'm about to send that email. I was lacking wind-in-sails last week. I can't stay around here long enough to have a discussion, but hopefully later in the week.
 * gary_poster hopes that the change won't cause too many nasty bugs
<lifeless> gary_poster: what I was mainly thinking was that being able to switch this behaviour off easily would be a good thing for e.g. dealing with a module that does silly buggers with sys.stdout
<gary_poster> in test runs
<lifeless> where 'easily' could mean 'edit bin/test'
<gary_poster> ah ok
<lifeless> or it could mean 'set an env variable'
<lifeless> or 'pass an option'
<gary_poster> oh, I was with you then not with you :-P
<lifeless> allenap: ok
<gary_poster> if I had a silly sys.stdout test...
<lifeless> one that breaks if sys.stdout==sys.stderr, for instance.
<gary_poster> right
<lifeless> you'd want to be able to switch things around to find out whats going on
<gary_poster> then I would still need that test to pass within a fill test run
<lifeless> without editing the contents of an egg
<gary_poster> full
<lifeless> sure, this may be a YAGNI
 * lifeless is happy to wear the paranoid hat
<lifeless> gary_poster: which is why I say its up to you :) - just trying to articulate the reasons I was proposing it be an option
<gary_poster> ok, so a full implementation of that direction, combined with my belief that --subunit should default to the "non-intermittently-broken" version, would yield...
<lifeless> tl;dr - an API with side effects that might be a problem is easier to poke at and play with if the side effects are controllable
<gary_poster> --subunit makes stdout ==stderr
<Ursinha> gmb, hey :)
<gary_poster> --disable-stdout-switch disables that (I don't like negative flags usually either, but sometimes incilinations fight), and
<Ursinha> gmb, I'm looking at the bugactivity interface, and I see it references a bug using a BugField. Do I need to create a BlueprintField as well? What is that for?
<gary_poster> the ability to temporarily turn off the stderr/stdout switch for a single test?  So that tests can be allowed to be silly?  Of course, every test that does this allows another chink in the armor
<lifeless> I wouldn't bother with one-test support
<gary_poster> the log messages might slip in at that time
<gary_poster> yeah
<lifeless> the way I see the switch being used would be:
<lifeless> someone has a problem
<gary_poster> you just have to fix the test
<lifeless> they are told 'toggle this and try the test'
<lifeless> if that works, they dig in to fix the test
<gary_poster> +1
<lifeless> if it doesn't, we know its something else.
<gary_poster> OK, cool thanks lifeless.  I'll go back to filing bugs. :-)
<lifeless> anytime
<Ursinha> is the blueprint name an unique field? like a bug number?
<cjohnston> Ursinha: I don't believe so
<cjohnston> I believe you can have the same name on different projects
<cjohnston> but I'm not 100%
<lifeless> sinzui: \o/
<Ursinha> cjohnston, yes, you can have that in different projects, just checked on staging...
<Ursinha> cjohnston, thanks :)
<cjohnston> Ursinha: we had issues with Summit with same named BPs, that's why we require the cycle letter to be in BPs
<Ursinha> cjohnston, makes sense
<cjohnston> gmb: ping
<Laney> Given an SPPH, how do I get to a page like https://launchpad.net/ubuntu/+source/geg/1.0.2-6build1 (what are these called?) in a unit test?
<Laney> an object whose canonical_url is that I suppose I mean
<lifeless> That should do it
<lifeless> its an SSPH
<lifeless> bah
<lifeless> SPPH
<Laney> lifeless: thought so, but "ComponentLookupError: ((<SourcePackagePublishingHistory at 0xee32c10> â¦"
<Laney> should this work? I'm using create_initialized_view here
<lifeless> IIRC you need a layer/site there
<Laney> hrm
<lifeless> cjwatson: do you want me to listen in on the archive-admin session ?
<czajkowski> cjohnston: gmb is doing headshots so may not be on irc
<czajkowski> cjohnston: better to drop him a mail
<wgrant> Laney, lifeless: That's a DSPR
<wgrant> Not an SPPHJ
<wgrant> Bah, SPPH
<wgrant> SPPHs don't exist in the traversal hierarchy
<wallyworld_> sinzui: my laptop died totally last night. i'm on one of the kid's but mumble is refusing to play nicely as it does at times
<sinzui> wallyworld_, hangout?
<sinzui> I will take that answer as no
<wallyworld__> sinzui: we can try. i have never logged into google+ before
<wgrant> noodles775, jml: I see no QA, and no news in 8 hours. I'll revert and we can retry in a less awkward week.
<sinzui> wallyworld__, then I can assume you do not have the google app on your phone
<wallyworld__> sinzui: i have a dumb phone
<wallyworld__> it makes phone calls
<wallyworld__> and nothing else
<wgrant> wallyworld__: What's mumble not doing?
<wgrant> I haven't had a single problem with it in >6months
<wallyworld__> wgrant: i starts, says it connects, and then consumes 100% cpu
<Laney> wgrant: so how can I get at that?
<sinzui> wallyworld__, maybe it is killing the sound server
<lifeless> wgrant: lets try #ca-internal first
<wallyworld__> sinzui: maybe. this issue used to happen a lot but of late it's been good
<wgrant> Ah, so that's where they lurk.
<wgrant> Laney: spph.meta_sourcepackagerelease
<Laney> ta
<sinzui> wallyworld__, does that computer have skype or the googletalk plugin?
<wallyworld__> sinzui: yes, skype is available
<wallyworld__> and also plugin i think
<wallyworld__> sinzui: skype is open now
<sinzui> wallyworld__, wgrant, StevenK. I will try to remember how to conference call on skype
<sinzui> wallyworld__, The skype app does not appear to support conference calls :(
<wallyworld__> sinzui: i'm sure one time i had a conference call using skype, let me see if i can find out how
<sinzui> wallyworld__, The desktop app does. I do not have it anymore. I never had it on this computer actually
<cjohnston> I am looking at this review: https://code.launchpad.net/~vorlon/launchpad/lp.994110/+merge/105078  I don't see anything in the file that is mentioned that would be effected by the code change.. Could someone give me a little bit of help please?
<czajkowski> cjohnston: rick_h_ is the on call reviewer
<wgrant> cjohnston: Have you run the test?
<wgrant> cjohnston: It returns 2 specs instead of 1, presumably because of the new relaxed filtering.
<cjohnston> czajkowski: I pinged him hours ago.. wgrant, no.. I don't have it setup to be able to run tests.. I was trying to figure out the problem though.
<cjwatson> lifeless: sorry, too late :-)
<wgrant> cjohnston: http://paste.ubuntu.com/976901/ is the failure
<cjwatson> lifeless: there wasn't too much extra, I'll integrate it into the wiki page
<cjwatson> (I didn't look into IRC until the end of the session ...)
<cjohnston> thanks wgrant
<rick_h_> czajkowski: heh, sorry EOD I missed updating that
<rick_h_> cjohnston: running the test, it just failed when I did a broad sweep of tests for that stuff
<cjohnston> rick_h_: thanks.. wgrant pasted it for me..
<rick_h_> cjohnston: https://pastebin.canonical.com/65709/ ah behind sorry
 * cjohnston == ~not-canonical
* rick_h_ changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: - | Firefighting: - | Critical bugs: 3.47*10^2
<rick_h_> cjohnston: ah sorry
<cjohnston> np
<wgrant> cjohnston: Have you considered following https://dev.launchpad.net/Running/LXC to non-invasively get LP running locally?
<rick_h_> wgrant: work out for you?
<rick_h_> sorry, meant wgrant 's past work out then?
<cjohnston> rick_h_: yes, it gave me the errors
<cjohnston> wgrant: I was trying not to get that deep into this bug on the LP side :-)
<cjohnston> hehe
<wgrant> Ah, right, you're just on the summit side
<cjohnston> wgrant: I will probably bring this up on Thursday in the Clinic if slangasek doesn't get to it before that
<wgrant> cjohnston: Sure. It's very easy to fix.
<cjohnston> I'm not familiar enough with testing to do it on my own. :-)
<lifeless> gary_poster: Another idea
<lifeless> gary_poster: make the replacement sys.stdout grab a traceback and report the object triggering the write
<lifeless> e.g. a wrapper around sys.stderr with a custom write() + writelines() methods
<lifeless> afk for a bit
#launchpad-dev 2012-05-09
<gary_poster> lifeless, fun idea.  I added it to the bug
<gary_poster> night all
<wallyworld__> StevenK: i can get a asus with i7-2670QM and 16GB RAM and 1920x1080 15.6" display for $1300. sounds ok i think
<wgrant> That's huge.
<wgrant> But nice res.
<wallyworld__> wgrant: huge? you mean screen size?
<wgrant> Yeah
<wallyworld__> i wouldn't go smaller
<wallyworld__> 15.6" is the smallest i've had ever
<wgrant> I guess if you're using it as a desktop without an external monitor it might make sense.
<wallyworld__> i have an external monitor too
<wgrant> OK, so it doesn't make sense :)
<wallyworld__> i like to move about a bit sometimes
<gary_poster> lifeless, I was shutting down my juju instances and realized I forgot to give the new subunit/testtools/testrepository packages a whirl.  I just replaced everything on the master and slave and started a new run.  Everything seems great.  Once that packaging snafu is dealt with for subunit, everything looks good for a release from what I can tell.
<gary_poster> thank you!
 * gary_poster disappears again
<StevenK> Bah
<StevenK> Disappeared before I could tease him.
<wallyworld__> StevenK: are you going to update factory methods to use info_type for branches?
<StevenK> Down the line, yes.
<StevenK> It's a bit premature right now.
<wallyworld__> StevenK: ok. i've got some tests with placeholders for when that happens
<StevenK> wallyworld__: I'm happy to sprinkle in information_type into makeBranch() in the current branch I'm doing.
<StevenK> I just won't remove private yet
<wallyworld__> StevenK: that would be nice, thanks
<StevenK> wallyworld__: Aye, I shall.
<wallyworld__> you ocr?
<StevenK> I should be.
* StevenK changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: StevenK | Firefighting: - | Critical bugs: 3.47*10^2
<wallyworld__> i will have a mp soon :-)
<StevenK> I'll be sure to give it the appropiate amount of attention.
<StevenK> (IE, none at all.)
<StevenK> cjwatson: Re your db branch, don't worry about the +8, it's a DB patch, and I don't think it should count.
<lifeless> wgrant: have you read the browserid spec?
<wgrant> lifeless: Not in depth, but I know roughly how it works.
<wgrant> lifeless: Assuming it becomes semi-standardised and gains some widespread support, I see no technical reason to forbid it.
<lifeless> the thing I wonder, quite apart from whether its a good idea
<lifeless> is whether someone can create an openid thunk
<lifeless> and we can avoid supporting it entirely
<lifeless> wgrant: overhead is overhead
<wgrant> Sure, one could pretty easily implement an OpenID provider which effectively proxies BrowserID.
<lifeless> wgrant: I skimmed it about 6 months ago
<lifeless> but the details are paged out
<lifeless> so, if you can do that, one wonders my mozilla didn't just do that instead.
<wgrant> lifeless: Well
<wgrant> lifeless: Part of the idea of BrowserID is that browsers will implement it.
<wgrant> So there'll be no bouncing around.
<wgrant> The browser will authenticate directly with its keypair and a time-limited certificate signed by the provider.
<wgrant> One could implement an OpenID gateway, but it would still involving hideous redirects and ugliness.
<lifeless> wgrant: yes, I'm aware of that
<lifeless> wgrant: my HTTP nazi hate just loooves that
<wgrant> It does mean that you have to trust the browser, and you can't really seamlessly integrate server-side 2FA.
<wgrant> But it in all otherwise a tonne less hideous, sick and wrong.
 * wallyworld__ -> computer shop @#$%%$#!
<StevenK> Hahaahaha
<wallyworld__> bastard :-(
<jtv> StevenK, wgrant: just had another silly ideaâ¦ one of the more expensive parts of publishing is now the âls -lR.â  Could we maybe kick that off earlier, using the MF?
<wgrant> We might be able to, yeah.
<jtv> It's getting hard to single out culprits now, but IIRC this was a relatively big one.
<StevenK> wallyworld__: Are you back yet?
<wallyworld__> StevenK: back. bloody traffic accident. took 1 hour for a 15 minute trip :-(
<bigjools> haha
<jtv> wgrant: interested in reviewing the scrubber change?  https://code.launchpad.net/~jtv/launchpad/bug-994650-scrub-faster/+merge/105169
<wgrant> jtv: It would be my pleasure.
<jtv> \o/
<jtv> I think I'll grab some food, meanwhile.
<wgrant> jtv: Looks good.
<wgrant> jtv: I'm not sure if avoiding loading the POFiles is a useful optimisation at this stage, but it can't hurt.
<wgrant> StevenK: Have you run the garbo job over dev and test sampledata?
<StevenK> wgrant: Nope, I was going to do that with the db patch
<wgrant> StevenK: Sounds reasonable.
<wgrant> Just means you have to practice a bit of necromancy.
<StevenK> Meh, the garbo job is still in db-devel ... :-)
<wgrant> True
<stub> Should garbo tasks become celery tasks, or should we keep the garbo infrastructure?
<wgrant> I think it'll all become celery once that's stable.
<jtv> wgrant: are you well?
<jtv> You approved my branch without comments.  Naturally I'm concerned.
<wgrant> Heh
<nigelb> hahahahah
<nigelb> jtv: remember to make some pep8 violations so you both can breathe easy :D
<jtv> Thanks.  I'll try to remember that.
<jtv> And if I don't, then wgrant can point out that I forgot.
<nigelb> Heh.
<StevenK> Oh, sure. "You forgot to make some silly mistakes in this branch so I can comment. Oh wait, I just did comment. Damn it!"
<adeuring> good morning
<rick_h_> morning adeuring
<adeuring> morning rick_h_
<jtv> Any reviewers about?  If so, have an MP!  https://code.launchpad.net/~jtv/launchpad/bug-994650-scrub-in-batches
<czajkowski> morning
<rick_h_> morning czajkowski early out west isn't it?
<czajkowski> yes
<czajkowski> 6:35
* abentley changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: abentley | Firefighting: - | Critical bugs: 3.47*10^2
<czajkowski> any advice folks https://answers.launchpad.net/launchpad/+question/196568
<abentley> adeuring: is "name.find('/eggs') >= 0" equivalent to "'/eggs' in name" ?
<adeuring> abentley: ah, sure...
<abentley> adeuring: I find the latter more intuitive.
<adeuring> right
<abentley> adeuring: So most of this code is about determining whether the src directory is sys.path, and adding it if it's not?
<adeuring> abentley: yes
<abentley> adeuring: Have you tested this with lazr.jobrunner installed as an egg?  Seems like this kind of thing could easily break in that environment.
<adeuring> abentley: I haven't tested it yet. But why should it break?
<abentley> adeuring: I find the whole eggs/buildout thing magical, and I never know what's going to break it, so I would want to test it.
<abentley> adeuring: In particular, I'd worry that /usr/bin/python -m celery.bin.celeryd_detach is not going to pick up all the other eggs that Launchpad installs.
<adeuring> abentley: Have a look at the scripts in <lp-branch>/bin. Most of them are thin wrappers around a real script and just configure sys.path. Take celeryd as an example
<adeuring> abentley: celeryd starts fine
<abentley> adeuring: Sure, but most of them specify a giant load of paths.  I thought yours was just about adding one path.
<adeuring> abentley: no: extended_path = [name for name in sys.path if '/eggs' in name]
<adeuring> that adds the whole bunch
<adeuring> abentley: the extra steps with this_path is about the development environment
<adeuring> where lazr.jobrunner is not W"eggified"
<abentley> adeuring: Is there any need to configure CELERYD_PREFETCH_MULTIPLIER now that we're using CELERY_ACKS_LATE?
<adeuring> abentley: I am not 100% sure but don't want to test it either. I must admit that I did not understand the related docs fully...
<abentley> adeuring: r=me
<adeuring> abentley: cool, thanks
<cjwatson> https://code.launchpad.net/~cjwatson/launchpad/db-packageset-score/+merge/105113 looks like it has sufficient review now - could somebody mark the MP as Approved for me and then I can land it?
<adeuring> abentley: what were the jobs again which we wanted to run via celery on stanging/qastaging? i.e., which classs names should appear in jobs.celery.enabled_classes ?
<rick_h_> cjwatson: done
<abentley> adeuring: BranchScanJob.
<adeuring> abentley: thanks
<cjwatson> rick_h_: ta
<rick_h_> abentley: https://code.launchpad.net/~rharding/launchpad/ga_combo/+merge/105219 for you when you get a sec pls
<rick_h_> abentley: note that there's a lot of qualifications and such with it
<rick_h_> looks worse than it is because of the file move/copy of the ga.js around :/
<abentley> rick_h_: "a diff'd version"?
<rick_h_> abentley: in the google-analytics directory is a diff to the raw google provided js file so that it strips the things against our 'outside js/css' policy
<abentley> rick_h_: I don't think that copyright statement is correct.  They don't nest like that.
<rick_h_> abentley: yea, wasn't sure about that. I tried to state the "Code below" to help clarify
<abentley> rick_h_: I believe that including Google's code makes the whole file a derived work, so the header should be "Copyright 2012 Canonical Ltd, Copyright Google Inc." or similar.
<rick_h_> abentley: ok, sounds reasonable to me. Is there someone I should run this by/check officially you know of?
<abentley> rick_h_: You could try Amanda Brock.  The dept is https://wiki.canonical.com/LegalServices
<rick_h_> abentley: ty
<abentley> rick_h_: Where was this file before now?
<rick_h_> abentley: icing/google-analytics/ga.js
<abentley> rick_h_: So with this change, do we have it in three places?
<rick_h_> abentley: yes, for the moment
<rick_h_> we have to support both combo loader/non-combo loader users
<rick_h_> I guess we could get rid of the copy in the app/js/google-analytics directory, only the .diff file is *required*
<abentley> rick_h_: If we can get rid of a copy, that would be good.
<rick_h_> abentley: yea, I don't think we'd lose anything. Will do.
<gmb> Ursinha, czajkowski said you were looking for me.
<Ursinha> gmb, yeah, I have a lot of questions about launchpad code... but will enter a session in a few minutes :/
<gmb> Ursinha, Okay, let's try and catch up later on then. Some time after lunch if you're free.
<lifeless> morning
<lifeless> allenap: hi
<lifeless> sinzui: \o/ \o/
<sinzui> lifeless, I am glad you are happy
<lifeless> sinzui: contact-this-team
<sinzui> I was sure you were tracking that
<lifeless> :)
<czajkowski> sinzui: nice blog post have posted in all the places.
<lifeless> I'm sure we'll still get admins of casual 'groups' making the same mistake, but hopefully at a much lower incident rate
<sinzui> I wish I could have solved the Cc sender issue too, but the the objects need restructuring. I reverted aft 3 hours so that I could get your bug fixed
<czajkowski> nods I know tis fine
<lifeless> sinzui: thats a shame
<sinzui> yes. barry and I over-specialised the recipient set object. I need to make it behave more like a normal object.
<allenap> lifeless: Hi there.
<lifeless> allenap: hi, if you'd like some voice time, I'd be delighted to do so.
<allenap> lifeless: Cool, that sounds good. I've just seen your reply, so I'll read that.
<lifeless> allenap: skype ?
<allenap> lifeless: Is G+ okay? I don't have Skype installed right now.
<lifeless> tis fine
<lifeless> it has nearly as good echo cancellation
<lifeless> which is, for high latency, esssssential
<allenap> lifeless: Okay, invited.
<rick_h_> lifeless: ping, when you get a sec
<lifeless> rick_h_: pong; afk for a sec, but write here and I'll reply in a few
<rick_h_> lifeless: working on wrapping the ga.js code into a YUI module so I can combo load it and drop the request/icing/etc.
<rick_h_> lifeless: in code review, dealing with the copyright notice stuff came up and trying to figure out how to handle documenting the "This is a LP YUI module, all this code inside here is Copyright Google" as it is now
<gmb> czajkowski, about?
<rick_h_> lifeless: so came up to ping legal on it and legal is asking me " How do we get permission from Google to modify and redistribute their code?" and can't find anything atm
<rick_h_> lifeless: think you were involved and wondered if you 1) know how to handle it or 2) know the info I can give legal about how we got/get permission from google to mod/distribute the ga.js script
<rick_h_> lifeless: see https://code.launchpad.net/~rharding/launchpad/ga_combo
<lifeless> interesting
 * lifeless runs screaming
* abentley changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: - | Firefighting: - | Critical bugs: 3.47*10^2
<rick_h_> lifeless: so in particular looking at lib/lp/app/javascript/google-analytics/google-analytics.js commenting
 * rick_h_ is waiting for bzr.lp.net to finish thinking
<rick_h_> http://bazaar.launchpad.net/~rharding/launchpad/ga_combo/view/head:/lib/lp/app/javascript/google-analytics/google-analytics.js
<lifeless> rick_h_:
<lifeless> http://support.google.com/analytics/bin/answer.py?hl=en-GB&answer=1032389
<lifeless> I believe flacoste checked with Google when we first put it in the tree
<rick_h_> lifeless: rgr, I understood there was some conversation but wasn't sure if it was you someone else and didn't find any reference in the docs/code
<lifeless> rick_h_: so why do you need to convert it to yui ? is it a combo loader limitation ?
<czajkowski> gmb: looking for me ?
<lifeless> rick_h_: or is it because you're going to batch it with all the other modules needed? Could we not not do that and just reference that one file via a combo loader url ?
<rick_h_> lifeless: well, it was blocking/hanging when I was doing combo loader testing. If I make it a YUI module I can combo load it in the same request with the other JS on every page
<lifeless> rick_h_: keeping the unaltered google file and unaltered async-loading snippet from google
<lifeless> that would keep the delta lower too
<rick_h_> so it's benifit of -1 request per page done, and servied via our combo loader removing one more thing in icing
<lifeless> http://support.google.com/analytics/bin/answer.py?hl=en&answer=1008080
<lifeless> - goes in the end of <head>
<lifeless> you can serve via the combo loader just by using a combo loader url
<lifeless> so its solely -1 request, per the very first page they land on
<lifeless> rick_h_: or am I misunderstanding something about combo loader internals ?
<rick_h_> lifeless: well in order for YUI to know how to request it from the combo loader we parse the YUI modules (make jsbuild) and it's listed
<rick_h_> lifeless: but we could manually muck with/modify that I think
<lifeless> rick_h_: nono, *not* yui
<rick_h_> right, I gotcha
<lifeless> if we use http://support.google.com/analytics/bin/answer.py?hl=en&answer=1008080
<lifeless> which we probably do today
<lifeless> and replace the url in it with one pointing at the combo loader
<lifeless> we'll get back unaltered content.
<lifeless> right ?
<rick_h_> yes
<lifeless> so just moving the url - trivial. No delta vs google's authoratitive copy.
<lifeless> and still async, nonblocking on the page.
<rick_h_> ok, so we can work with the async version, our old one just wasn't using that one?
<lifeless> I don't know, you're the one doing the debugging :)
<rick_h_> k, I'll pull down and compare tomorrow
<lifeless> rick_h_: I gotta run, TL meeting; will pick this up after that.
<rick_h_> rgr
<lifeless> if we're blocking today, I'd say keep blocking, do the minimal work
<lifeless> move the url to the combo loader, add a bug saying we should be async, move on :)
<rick_h_> jcsackett: hey, have you been testing out/using the auto reloader for the JS?
 * rick_h_ is curious
<jcsackett> rick_h_: i have not.
<jcsackett> how does one use that?
<rick_h_> make jsbuild_watch
<rick_h_> then just edit the js files and on save it'll auto jsbuild them
<jcsackett> i will have to check that out.
<rick_h_> that's the idea at least
<jcsackett> that would make somethings *way* easier.
<rick_h_> yea, I got wondering, I've not done a ton of JS since adding it, but seems you are
<jcsackett> and the more i do, the more it seems i need to do. :-P
<rick_h_> ruh roh
<jcsackett> ah, the life of cleaning up old bad code.
<jcsackett> :-P
<rick_h_> that's the truth
<rick_h_> give it a try next time and let me know if you hit issues
<jcsackett> so, by "next time" you mean tomorrow? :-P
<rick_h_> :)
<rick_h_> I didn't want to presume
<rick_h_> just deeper than you thuoght it was or issues you're hitting?
<jcsackett> more things making use of it than i thought.
<rick_h_> going to be hacking at the coffee house tonight if you need a second set of eyes on anything
 * jcsackett nods.
<rick_h_> ah, gotcha k
<jcsackett> quick thought? know any reason that a BuiltClass from Y.Base.create might barf when it's constructor.NAME is grabbed?
<jcsackett> in that, it's constructor has no .NAME.
<rick_h_> yea, you didn't create the formal constructor. You'd have to inspect it and try to see if it even has a NAME property
<rick_h_> I think you'd have to get it from your instance?
<jcsackett> hm. wonder why Banner was fine but something derived from Banner dies ...
<rick_h_> not sure, honestly, never used the NAME bit :/
<jcsackett> i'm not either, YUI throws the error.
<jcsackett> i'll keep poking.
<rick_h_> well that NAME should change to the Base.create('ISNAME'... right?
<rick_h_> make sure your requires is correct
<rick_h_> the error I got today on that was waaay out of left field
<jcsackett> already checked requires. would that it was that easy.
<jcsackett> i'll keep poking. :-)
<rick_h_> k
<cjwatson> I forgot to add a COMMENT in db-devel r11582.  I guess since stub didn't comment on that during review it's not the end of the world; but is there anything I should do?
<cjwatson> As in is it worth landing a follow-up commit?
<jcsackett> rick_h_: just realized. no `new` call. :-P
<jcsackett> it's always the little things.
<rick_h_> jcsackett: ah, yea that one bites me all the time. I've learned that error hard core
<sinzui> wallyworld, how goes the morning? What is your preferred means of voice today?
<sinzui> jcsackett, mumble?
<jcsackett> sinzui: yup. signing on.
<sinzui> wallyworld, wgrant_ look at https://bugs.launchpad.net/launchpad/+bug/702429
<_mup_> Bug #702429: Pillar owners and private non-security bug visibility <bugs> <disclosure> <qa-ok> <Launchpad itself:In Progress by wallyworld> < https://launchpad.net/bugs/702429 >
<wgrant_> cjwatson: Not really, but you can if you want.
<lifeless> cjwatson: it is worth it, jfdi it, and it can land on devel
<lifeless> the comments are only used for dev machines anyhow
<jcsackett> damn thunderstorms...
<StevenK> wallyworld_: wgrant_ is stealing your thing!
 * wallyworld_ wants it back
#launchpad-dev 2012-05-10
<cjwatson> lifeless: OK, I'll just roll it into the code change corresponding to that db change, then
<StevenK> Test failures due to a DB patch can be fixed in the DB branch, or should be landed first?
<lifeless> land first if at all possible
<lifeless> as we can't make code changes when the DB patch deploys, we want to be sure things will be ok when it deploys
<lifeless> (and land on devel of course)
<wgrant> StevenK: What's the issue?
<StevenK> wgrant: test_branch_privacy_triggers does manual INSERTs into Branch which blows up when it's NOT NULL.
<StevenK> When information_type is set NOT NULL, sorry.
<wgrant> Even if it's just test changes, I'd still do it in devel.
<wgrant> If it's not just test changes, it has no choice but to go to devel.
<StevenK> Yes, I'm putting together a branch now
<wallyworld_> wgrant: i've done a branch behind a feature flag to revoke bug access on unsubscribe but of course the triggers do that and so the test fails. one option is to have the test remove the trigger. any other ideas?
<StevenK> wgrant: https://code.launchpad.net/~stevenk/launchpad/information_type-branch-privacy-trigger/+merge/105291
<lifeless> wallyworld_: wait, what? Do you mean unsubscribe after bug access is revoked?
<wallyworld_> lifeless: no. other way around. we already have a job to unsubscribe after access is revoked
<wgrant> lifeless: Someone hasn't been reading launchpad-dev :)
<lifeless> wgrant: thread name ?
<wgrant> [Launchpad-dev] Next steps for Better Privacy
<lifeless> wallyworld_: you should be able to unsubscribe and retain access in the new world order, surely ?
<wgrant> lifeless: This is the Transitional World Order
<wallyworld_> lifeless: eventually yes
<wgrant> Not yet New World Order
<wgrant> More specifically, this is the Transitional While We Have No UI Design But Need To Sort The Model Out World Order
<wallyworld_> lifeless: we initially want to mimic current behaviour
<lifeless> wgrant: I have no such email
<wgrant> wtf
<StevenK> lifeless: When did you unsubscribe from -dev? :-P
<lifeless> StevenK: probably when I subscribed to a different list; +editemails is error prone
<wallyworld_> wgrant: i could also make the trigger look at the feature flag table. i think that's the best option
<lifeless> yup, there we go
<lifeless> no wonder it was quiet
<wallyworld_> lol
<wgrant> wallyworld_: Ew
<wallyworld_> why ew?
<StevenK> lifeless: Would you like me to forward it?
<wgrant> wallyworld_: It's unprecedented and there must be a better way
<lifeless> I'm reading the list archives
<wallyworld_> what is wrong with business logic checking feature flags?
<wgrant> wallyworld_: It's a second implementation of the feature flag logic.
<wallyworld_> shouldn't matter that the business logic is in a trigger
<wgrant> This time in PL/pgSQL
<StevenK> wallyworld_: The flag table contains a scope. How are you going to get a trigger to work THAT out?
<wallyworld_> the only other way that i can see is to have the test disable the tirgger
<StevenK> As wgrant says, Ew.
<wallyworld_> detail, details
<wallyworld_> forgot about the scope
<wallyworld_> but i was thinking this would be on or off for all
<wgrant> Right, scope is the main problem.
<wgrant> You could just always use 'default';
<wallyworld_> so could ignore scope
<StevenK> If you implement flag scoping in PL/pgSQL, wgrant may murder you.
<wgrant> But it's still a bit ew, which is why I'm trying to think of a better way
<wallyworld_> so i could just ignore scope is what i was thinking
<lifeless> ok
<wgrant> wallyworld_: This is the test for when the flag is disabled?
<lifeless> so I can see this transition
<lifeless> I have two thoughts
<lifeless> firstly, have you considered just doing the final thing, directly.
<wgrant> lifeless: I was surprised when you didn't reply.
<wgrant> lifeless: I guess this explains it :)
<wallyworld_> wgrant: yes
<wgrant> lifeless: I guess checking the flag, explicitly in the 'default' scope, might be best. Otherwise removing the trigger later without illegally hacking tests might be awkward.
<lifeless> wgrant: so, AIUI the remove-grants-on-removed-subscriptions is being written to cope with...
<lifeless> 'there is no UI for showing who has access but not a subscription'
<lifeless> so if you wrote that UI today (and define it as crudely as possible - e.g. just show 'everyone with access')
<lifeless> you wouldn't need a job to remove access when a subscription is removed
<wallyworld_> it wouldn't be a job
<wallyworld_> but yes
<lifeless> job task code function helper etc
<lifeless> :P
<wgrant> lifeless: Sure.
<wallyworld_> i was being a pedant :-P
<lifeless> wallyworld_: orly ? :)
<wgrant> lifeless: But the UI would be very duplicatastic with the subscriber list, and we would have to introduce a whole lot of new sharing management widgets on the bug page.
<lifeless> wgrant: so, you're not doing htat.. why not ?
<wgrant> None of which we have design for, and all our UI people are off doing other things.
<lifeless> are you planning on doing that anyway ?
<wgrant> Mostly MAAS
<wgrant> Yes.
<lifeless> we don't have to get the presentation perfect first time around
<wgrant> We have to have it not be utter shit.
<lifeless> the key elements of design in this are already committed too - this is just the ramifications
<wgrant> And it has to not be confusing.
<wgrant> Because this involves privacy.
<lifeless> I agree.
<wgrant> So, we can implement this transitional thing in the next roughly two weeks.
<lifeless> is there a page showing who can see a given bug, today ?
<wgrant> We cannot get design tested before then.
<wgrant> lifeless: The subscriber list.
<wallyworld_> lifeless: wgrant: should we move the subscribers portlet off the default bugtask view and replace with 'who has access'?
<wallyworld_> that makes more sense to me
<wgrant> wallyworld_: That's for UI people and testing to answer :)
<wallyworld_> since i want to know who can see my stuff
<wgrant> We'll be lucky to get an hour of UI time this month.
<lifeless> wallyworld_: we could; some folk want to know who will be notified. Arguably the contents of both portlets should be expandable.
<wallyworld_> and till now, who can see = who is subscribed
<lifeless> one way would be to expand the subscribers portlet
<lifeless> it currently says 'you get xxx, these people get xxx, these people may get xxx', adding 'these other people are able to view the bug but do not get xxx'
<wallyworld_> we would need to use expanders i think if we do that, but could be ok
<lifeless> would make a lot of sense to me
<lifeless> it would be totally empty for a bug today.
<wgrant> lifeless: The transitional phase lets us add a single simple extra bit of UI (the policy grantee list) and a couple of extra model bits, and it lets us sort of the model so the new UI can be implemented easily once it's designed.
<lifeless> and if its the same portlet, there is ~= no perf overhead.
<wgrant> s/sort of/sort out/
<lifeless> w.r.t. removing the triggers, my suggestion would be:
<lifeless>  - have tests that use a feature flag to indicate whether the triggers exist or not.
<lifeless> and have a DB patch that removes the triggers and sets the feature flag ob.
<lifeless> s/tests that use.../code that uses.../
<wgrant> That's a possibility.
<lifeless> a small test fixture to give you no triggers + feature rule, will let you write tests now
<lifeless> you won't be able to go live except in a foul swoop, so you'd want to tightly limit how much code lived under that flag.
<lifeless> (OTOH we could rollback relatively easily)
<wgrant> Heh
<wallyworld_> lifeless: you mean the fixturewould disable the triggers in its setup? and enable after?
<wgrant> Foul swoop? :)
<wgrant> Sounds reasonable, though.
<lifeless> wallyworld_: yeah.
<lifeless> ideally you'd write code that doesn't care about the triggers or not.
<lifeless> Which is why I am also questioning why you're doing that.
<wallyworld_> lifeless: i was thinking the same thing when i initially posed this issue above
<lifeless> The lack of design-team seems like a poor reason not to use your own judgement
<wallyworld_> lifeless: well the code is to replace the trigger eventually
<wallyworld_> i think it better that the ui be done so the triggers can be removed
<lifeless> do something tolerable and quick (show them at the end of the subscribers list), and revisit when design have bandwidth.
<wallyworld_> +1
<wgrant> lifeless: It's not just showing them.
<wgrant> lifeless: We have to be able to remove them and possibly add them too...
<lifeless> wgrant: with a (-) button to remove them.
<wgrant> That's why showing policy grants is light.
<wgrant> Because you can't remove them from that page.
<wallyworld_> wgrant: that's what the +sharing page is for
<wgrant> lifeless: Who has permission to remove them?
<wallyworld_> or not?
<wgrant> wallyworld_: You really need to be able to revoke from both ends.
<wallyworld_> not   initially
<wgrant> wallyworld_: Particularly since only the project owner can remove people on +sharing
<lifeless> lets separate out needed and wanted
<wgrant> This is needed.
<wallyworld_> sure?
<wallyworld_> why?
<wallyworld_> we could just do read only portlet on bugtask page
<wgrant> We can't block on the CC to remove people from bugs...
<lifeless> wgrant: wallyworld_ is not suggesting that
<wallyworld_> be we can remoe people via the sharing page
<wgrant> wallyworld_: Only the pillar owner can use +sharing
<lifeless> ah yeah
<wallyworld_> sure, so?
<lifeless> wgrant: so, you can add (-) with the same rules from removing subscriptions, initially.
<lifeless> wgrant: that is clearly ok, because it works for subscriptions.
<wgrant> Perhaps.
<wallyworld_> why should anyone besides the pillar owner be able to revoke access?
<wgrant> I can see danhg and huwshimi coming to strangle us in our sleep, though.
<lifeless> wgrant: you could broaden it to allow the pillar owner to always remove, to be in line with +sharing.
<wallyworld_> regardless of if it's done from bugtask or +sharing
<wgrant> wallyworld_: I've filed a bug containing user data that's private to me.
<wgrant> wallyworld_: But I can't prevent people from accessing it, because Launchpad is stupid :(
<lifeless> wgrant: on what basis? That you did /something/, with communication with them about what you're doing (to give them the opportunity to go nononnonoon if needed), and behind a ff, so that its got no UI impact until you're happy with it?
<wallyworld_> ok so we can allow bug owner + pillar owner
<wgrant> lifeless: Anyway
<lifeless> wallyworld_: there is no 'bug owner' :P
<wgrant> I still think we should do what I proposed.
<wgrant> It's one little workaround to remove grants on unsubscriptions
<wgrant> And preserves the current UI
<lifeless> wgrant: its up to you guys as a group, you're doing the work.
<wallyworld_> bug task owner(s) i think i meant
<lifeless> myself, I always try to go as directly as possible.
<lifeless> I don't see preserving the current UI as a goal, at any stage.
<wgrant> Sure
<wallyworld_> in this case i'm +1 with lifeless
<wgrant> Evolving the privacy rules gradually is a recipe for disaster.
<wallyworld_> especially if it's behind a flag
<wgrant> wallyworld_: It can't be behind a feature flag.
<wgrant> Not completely.
<wallyworld_> the ui i mean
<wgrant> It has to be shown to everyone the moment anyone can edit grants directly.
<wallyworld_> and then when we are happy with it, we can remove the triggers to revoke on unsubscribe
<wgrant> Or access becomes opaque
<lifeless> so what wgrant means, I think, is that when 'add without subscription' becomes available to *anyone*, then that access can't be hidden by the ff
<lifeless> so the change to the subscriber portlet, while it can be ff'd to start with
<lifeless> has to go live as soon as any manual grant - pillar or artifact scoped - is addable to the system
<wallyworld_> yes
<wgrant> So we cannot do gradual rollout.
<wallyworld_> so long as the portlet looks ok, what's the issue?
<lifeless> but again, this is a tiny, shallow UI change, consistent with current layout, and if we're committing to reviewing it to fix issues huwshimi/dan/mrevell identify, I don't see the issue.
<lifeless> you don't need it globally visible *until* you FF enable the 'add manual grant' feature
<lifeless> and that feature, is one that you don't need to enable until very late in the piece.
<lifeless> folk may want it sooner, but its up to you when its delivered.
<lifeless> So you can have as much design/UI review as you like before flipping the switch.
<lifeless> You can gradual-rollout the new subscriber portlet info in advance of gradual-rollout the ability to add grants
<lifeless> totally doable.
<wgrant> And totally another two weeks.
<wallyworld_> if we roll out the new ui, it will always be empty anyway atm due to the triggers
<wgrant> Indeed, the new section of the porlet will not be seen until manual grants happen.
<wgrant> So a gradual rollout there is not useful.
<lifeless> less useful
<lifeless> certainly
<lifeless> totally unuseful? don't think so. Missing table perms etc would be caught
<lifeless> poor query performance likewise.
<lifeless> wgrant: why another two weeks?
<wgrant> lifeless: Because that's how Launchpad works.
<wallyworld_> and i could hack in data for a screenshot
<lifeless> I can't usefully comment on such hyperbole
<wgrant> It's not hyperbole.
<wgrant> Attempting to iterate on interlocked UI and model changes *does not work*.
<wgrant> In Launchpad.
<lifeless> what model changes are expected here?
<lifeless> other than the trigger discard
<wallyworld_> wgrant: i call bullshit on that one. i did it fine for the +sharing page
<wgrant> lifeless: Moving access management into the application and out of the database, for one.
<wallyworld_> lifeless: we will only likely one one additional service method on the sharing service
<wallyworld_> for me to do the ui
<lifeless> wgrant: how is that different to the trigger being discarded?
<wgrant> wallyworld_: Permissions are going to be very awkward, but we'll see.
<wallyworld_> permissions for editing maybe
<wallyworld_> but that's a separate issue
<wallyworld_> the ui can still be done incrementally, first read only, the editable
<wallyworld_> and feedback can still be gotten
<wgrant> I am very wary about gradually evolving the privacy UI like that.
<wgrant> It's going to confuse people, and everyone already makes enough mistakes.
<wallyworld_> confuse who?
<wallyworld_> it will only be visible to select people
<wallyworld_> initially
<wallyworld_> us and product team perhaps
<wgrant> Sure, but we have to eventually roll this transitional edit UI out to everyone.
<wgrant> Only to change it again later.
<wallyworld_> or we could deploy it in one go like we said above
<lifeless> we change UI all the time, after we have data about how well it works.
<wgrant> wallyworld_: This UI we've discussed here won't be the final one.
<lifeless> wgrant: why not ?
<lifeless> well, let me rephase
<lifeless> beyond design requested changes
<lifeless> which isn't a guaranteed thing, because you might be doing it well enough first pass.
<wgrant> ... because there will be design requested changes.
<wallyworld_> so? we have to start somewhgere
<wallyworld_> and so long as it's functional and not too ugly
<wallyworld_> then it will be fine surely
<lifeless> retitle 'subscribers portlet' to 'sharing and notifications' - |your status | other people with different sorts of status| status2 | status3\
<lifeless> (rotated 90'
<lifeless> )
<wgrant> wallyworld_: I can see the blog post now
<lifeless> 'LP developers delivered on time' ?
<wallyworld_> lol
<wgrant> "Now, there are some people that have access but aren't subscribed. You can't add them, and they'll only appear sometimes, when something has happened. In a week this will change and everything will be like this"
<wallyworld_> that's not very helpful. i don't think it will happen like that at all
<lifeless> So, keep your current job :)
<lifeless> I don't think marketing is your forte.
<lifeless> hmm, that sounded meaner than I meant it. Sorry.
<wgrant> Heh
<lifeless> But seriously, I would say something like this:
<lifeless> 'The disclosure project has now enabled direct access control for the beta users of Launchpad. Users of that team can now grant access to private and security bugs without needing to create subscriptions. Less mail!.
<wgrant> So now we need the grant access UI too!
<wgrant> Awesome.
<wallyworld_> hmm. if only there were someone around who could write code
<wgrant> Code isn't the problem.
<wgrant> Code is easy.
<lifeless> *All users* of Launchpad will be able to see who has been granted access to a private or security bug. Once we've shaken down the system with our beta team, we will permit all users to use this facility."
<wgrant> UI when all our UI people have been stolen is the problem.
<lifeless> wgrant: hangon, take a step back please.
<lifeless> and a good long breath.
<wgrant> We won't be allowed to do this without mockups and user testing, and that will take at least 6 weeks.
<lifeless> My understanding is that noone will know this change is around until the grant access UI is enabled.
<lifeless> True or False.
<wgrant> The sharing management UI in general, yes/
<wgrant> .
<lifeless> So True.
<lifeless> And, thats feature flagged.
<lifeless> Right?
<wgrant> Yes.
<lifeless> So, there is no need for the blog until that flag is toggled.
<lifeless> And the sketch I gave for a blog is sufficient when the flag is toggled - if the 'grant access' UI is the sharing management UI, thats sufficient.
<wgrant> Sure.
<wgrant> We this still means we can't finish the model until the new UI is designed.
<lifeless> so why did you wax sarcastic?
<lifeless> wgrant: so do the design for the UI.
<wgrant> Heh
<wgrant> Have you ever designed UI using the new process?
<lifeless> There is a difference between design and signoff.
<wallyworld_> we don't need the entire process to get started
<wallyworld_> or to have enough to do the model
<wgrant> We do.
<lifeless> I feel like you have a genuine point somewhere, but its getting hidden behind absolute statements and general hysteria.
<lifeless> This is exhausting trying to work through.
<wgrant> We cannot evolve the model until the UI enforces its display limitations.
<wallyworld_> sigh. i'm going to write code. see y'all
<wgrant> The point of my plan is to allow us to evolve the model before we have UI
<wgrant> In lifeless' plan, we cannot evolve the model until the new UI is turned on.
<wgrant> Which requires signoff.
<wgrant> turned on for everyone, that is
<lifeless> Your plan requires that where the model is implemented move around, but htat the model (in the general sense of 'rules of the system') stay the same for the same period of time.
<wgrant> Crucially, it allows us to completely stabilise the model code and work on more project privacy.
<lifeless> I was hoping to unblock you on a technically hairy transition, but if you believe you can't do it differently, thats your call. I'm not here to tell you how to do it.
<wgrant> Meanwhile the UI can be bikeshedded without blocking the rest of the world.
<lifeless> What I would do is nab dan or huw on skype tomorrow, talk them through the key things (show the non-subscribed grants, show a (-) if you can remove them), get them to ack that as a provisional thing, and move on.
<lifeless> LP UI will be bikeshedded for the next 50 years
<lifeless> (thats hyperbole :P)
<cody-somerville> Yea. We'll be using github by then.
 * cody-somerville ducks.
<StevenK> wgrant: Does any of the bugsearch stuff still reference Bug.private ?
<wgrant> StevenK: Not the main stuff, but there are some stragglers.
<wgrant> Most were eliminated in a branch that landed a few hours ago.
<StevenK> wgrant: So we can't quite rip out Bug._private and friends yet
<wgrant> Sadly not.
<StevenK> Do you think it's worth putting together a silly branch that drops them just to see how much fallout is involved, or not yet?
<wgrant> Not yet.
 * StevenK will continue to bash his head against IBranch.
<wgrant> Not until the legacy half of get_bug_privacy_filter is gone and we've grepped for [Bb]ug.private
<StevenK> jtv: O HAI
<StevenK> wgrant: Bug.private isn't dying, ._private is.
<wgrant> I mean DB [Bb]ug.private
<StevenK> Hm, I think Bug._private might actually be dead.
<wgrant> I believe it was meant to be unused except for compatibility with DB queries.
<StevenK> Right. I just can't see any others in the tree.
 * StevenK reads the triggers related to Branch.transitively_private and prepares for a 12 hour drive.
<StevenK> wgrant: Hm, you were involved with jtv's testing on staging, does that qualify as QA for the purposes of r15218 and r15220 ?
<wgrant> StevenK: No, there have been some slight changes since.
<wgrant> He might be able to judge it sufficient, but I don't particularly want to.
<StevenK> wgrant: Yah, okay
<czajkowski> aloha
<lifeless> czajkowski: I thought you were @ UDS ?
<czajkowski> lifeless: yup I am indeed
<czajkowski> lifeless: and I work on lp stuff in and around sessions and stuff so working now for a bit
<czajkowski> but need jtv :/
<lifeless> czajkowski: isn't it 11pm ?
<czajkowski> lifeless: yup
<jtv> StevenK: no, it's not quite the same thing.  I've been trying to Q/A, but running into qastaging trouble
<lifeless> czajkowski: btw, feature work defects are never critical, unless they are oopsing
<lifeless> czajkowski: (because its planned work, its not an interrupt)
<czajkowski> lifeless: ah ok thought they were as they're part of a new feature not working as the feature intended
<czajkowski> sorry
<lifeless> czajkowski: no worries; also minor things like typo are at most high
<lifeless> (they don't stop people getting their work done)
<lifeless> czajkowski: 997346 should indeed be critical, because its a regression - but it should have the tag added
<lifeless> czajkowski: there is I think (and if not there should be), in the bug triage wiki docs a comment to the effect of 'all critical bugs have to have a tag explaining why'
<lifeless> e.g. timeout, or oops, or escalated, or regression
<lifeless> mmm, and looking closer
<lifeless> stub: hi thar
<stub> urgh
<lifeless> good morning :P
<lifeless> stub: ready for a catch up?
<stub> Ha. Can do, but not ready :)
<lifeless> skype or g+ ?
<stub> Hang on, need to steal my mike back.
<stub> I still haven't tried G+.
<lifeless> skype then :)
<wallyworld_> stub: hi. for some tests, is there an 'admin' user i can use like "with dbuser('admin')" which allows me to avoid 'you do not own this relation' errors. i have got it to work with 'ian' but obviously i can't commit that
<stub> wallyworld_: There is a 'testadmin' user
<wallyworld_> stub: it still complains when i try that
<stub> Hmm... 'you do not own this relation'. What are you trying to do?
<wallyworld_> stub: disable a trigger
<stub> oic. Need to be the real owner then.
<wallyworld_> as in 'ian' or whoever?
 * stub checks to confirm if 'launchpad' or other
<stub> wallyworld_: os.environ['USER'] would be best for now
<wallyworld_> stub: cool. thanks
<stub> wallyworld_: Ideally, the 'testadmin' user should be a superuser if we are trying to do this sort of thing. Open a bug if you like.
<wallyworld_> stub: will do. i was hoping there'd be a symbolic admin user i could use
<stub> Currently it must just have full permissions on every object, which isn't quite a superuser.
<wallyworld_> ok
<stub> Yer. Didn't think of tests issuing DDL.
<wallyworld_> well, most don't :-)
<wallyworld_> i'll file a bug and add an XXX to my branch
<stub> No need for the XXX really - just a normal comment that os.environ['USER'] is a superuser (or our database setup scripts will completely fail)
<wallyworld_> stub: i was going to add the XXX so that when the bug i file is fixed, we can replace the os.environ with 'testadmin'
<stub> Sure.
<wgrant> wallyworld_, stub: Why not use postgres?
<wgrant> In test scenarious the current user should be able to auth as postgres, and postgres is both a superuser and the real owner.
<stub> wgrant: If we know that works for every dev, sure.
<wgrant> They have be able to auth as dozens of other users for the test suite to work.
<wgrant> So unless they've gone and named them all explicitly...
<stub> More that pg_hba.conf often starts with restrictive settings for connecting as postgresql (to ensure autovacuum etc. can work), followed by user settings. But our setup instructions say don't do that, so...
<stub> wallyworld_: Use 'postgres', no need for the bug or XXXX
<wallyworld_> ok, thanks
<stub> wallyworld_: We already switch to that user in other tests
<stub> wgrant: ta
<wallyworld_> i tried to find something but there's so many... :-)
<StevenK> wallyworld_: Can you review https://code.launchpad.net/~stevenk/launchpad/information_type-branch-privacy-trigger/+merge/105291 so I can land it?
<StevenK> stub: A review of https://code.launchpad.net/~stevenk/launchpad/db-branch-information_type-not-null/+merge/105282 would be lovely.
 * stub looks
<wallyworld_> StevenK: ok, give me a minute
<StevenK> wallyworld_: It's tiny, only test changes.
<stub> StevenK: Are you using PG 9.1 yet?
<wgrant> StevenK: That col is indexed now, right?
<StevenK> stub: I'm not, but neither was the sampledata in db-devel
<stub> StevenK: Oh, ok. Thought that the 9.1 version had landed already.
<wgrant> cjwatson didn't regen sampledata
<StevenK> It looks to have in devel, so I will probably have fun with a merge
<wgrant> I did in devel a few hours back, with 8.4, because I knew there was more coming
<stub> diff didn't look that huge - was just checking
<stub> r=stub anyway
<StevenK> wgrant: stub did the index the day before the garbo job landed.
<wgrant> Should be deployable tomorrow.
<wgrant> Yeah, I thought I remembered that, but best to check.
<StevenK> wgrant: Except that I can't land it yet, since it requires the test fix that I wanted wallyworld_ to review.
 * wallyworld_ is looking
<wgrant> StevenK: That's true. You may have to be lucky.
<wallyworld_> StevenK: r=me
<StevenK> Yes, rather.
<StevenK> wallyworld_: I considered changing it to the factory, but decided against it.
<StevenK> Why it does INSERT directly is beyond me,
<adeuring> good morning
<wallyworld_> StevenK: i can't recall why. i think it was because it was just testing the triggers and wanted to keep it simple
<StevenK> wallyworld_: Right, so there is a reason.
<lifeless> www.pretotyping.org -
<jtv> Any reviewers in the house?  I have 2 incremental MPs â https://code.launchpad.net/~jtv/launchpad/bug-994650-scrub-in-batches/+merge/105189 & https://code.launchpad.net/~jtv/launchpad/bug-994650-cache-potmsgsets/+merge/105193
<jtv> Does anyone have any tips on how to find, given a job class, what command line we use to run that job?
<jtv> Ahhh, the trail leads to the culprit.  Wow, we can't standardize this stuff soon enough.
<wgrant> jtv: There's only around 4 ways to run jobs now.
<wgrant> Celery is a 5th, but will subsume the rest.
<wgrant> Eventuall.
<wgrant> y
<jtv> like any One New Standard To Replace All Legacy Standards.
<wgrant> This one actually has major benefits over all the existing ones, and is much easier to set up, though.
<wgrant> And is nearly ready.
<nigelb> wgrant / jtv https://www.xkcd.com/927/
<wgrant> Indeed
<wgrant> I don't even need to click the link :)
<nigelb> haha
<nigelb> I didn't have to actually open the url.
<nigelb> It's like 386 is Duty Calls, 705 is Devotion to Duty, etc.
<deryck> Morning, everyone.
<nigelb> Hey deryck
* jcsackett changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: jcsackett | Firefighting: - | Critical bugs: 3.47*10^2
<deryck> adeuring, I'm free again.  want to meet back in the standup hangout?
<adeuring> deryck: sure
<abentley> adeuring: Could you please review https://code.launchpad.net/~abentley/lazr.jobrunner/rollback-29/+merge/105335 ?
<adeuring> aabesure
<adeuring> abentley:  sure
<adeuring> abentley: r=me
<rick_h_> jcsackett: if you get a sec pls https://code.launchpad.net/~rharding/launchpad/bugnom_874250/+merge/105317
<jcsackett> rick_h_: it's in my queue.
<rick_h_> jcsackett: ty
<jcsackett> jtv: you around?
<cjwatson> wgrant: sampledata> Oops.  Should I have done?
<stub> cjwatson: Doesn't matter.
<cjwatson> OK
<stub> cjwatson: Just me reconciling what I thought had occurred with reality
<jcsackett> rick_h_: r=me, with nitpicks.
<rick_h_> jcsackett: ty, will go check them out.
<rick_h_> jcsackett: ah, heh I didn't clean that up. Had pdb break points in there so having the middle var allowed stuff
<jcsackett> rick_h_: dig.
<lifeless> morning
<sinzui> jcsackett, ping
<jcsackett> sinzui: pong.
<sinzui> jcsackett, have time to hangout>
<sinzui> ?
<jcsackett> sure, be on g+ in a minute.
<sinzui> jcsackett, sorry, lost my phone. I am calling it now
<jcsackett> ok.
<sinzui> bugger, g+ changed it's UI again
<jcsackett> sinzui: i'm still seeing you as offline.
<sinzui> oh, I think I am in a hangout with you in g messenger
<sinzui> New UI, but same crap
<jcsackett> yeah, i'm messaging you via g messenger, but still not seeing you online.
<lifeless> bbiab, shopping run
<james_w> jcsackett, hi, would you take a look at https://code.launchpad.net/~james-w/launchpad/no-subscription-cancellation-email/+merge/105400 please?
<jcsackett> james_w: sure.
* jcsackett changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: - | Firefighting: - | Critical bugs: 3.47*10^2
<jcsackett> james_w: looks good. if you'll set the commit message i can send it off to ec2 and land for you.
 * jcsackett likes short, direct MPs
<james_w> jcsackett, ace, thanks
<james_w> jcsackett, done
<jcsackett> james_w: may have lied to you, i can't seem to get ec2 to workout properly. sinzui, can you land james_w's branch? i keep getting silent failure on the last step of ec2 land.
<sinzui> I can land
<rick_h_> lifeless: ping
<jcsackett> sinzui: thanks. MP is https://code.launchpad.net/~james-w/launchpad/no-subscription-cancellation-email/+merge/105400
 * jcsackett makes note to poke at ec2 stuff tomorrow.
<lifeless> rick_h_: ola!
<rick_h_> lifeless: just commenting on your comment
<rick_h_> https://code.launchpad.net/~rharding/launchpad/bugnom_874250/+merge/105317
<rick_h_> lifeless: bath/bed time for the boy, but will check back in in 30ish. If you want it changed I can, but it was that way with some reason for consistancy. Let me know what you want and I'll update it in the morning
<lifeless> rick_h_: replies
* wallyworld changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: wallyworld | Firefighting: - | Critical bugs: 3.47*10^2
#launchpad-dev 2012-05-11
<wgrant> wallyworld: https://code.launchpad.net/~wgrant/launchpad/better-createManyTasks/+merge/105422 should be pretty quick
<wgrant> When you have a sec
 * wallyworld looks
<wgrant> Thanks.
<czajkowski> gmb: ping
<gmb> czajkowski, Hi
<czajkowski> gmb: coming int san fran @ 6:30
<czajkowski> shopping n dinner
<wgrant> How'd the clinic go?
<gmb> czajkowski, Can't; too much to get done. Thanks for the invite, though.
<czajkowski> 2 people asked for help to fix bugs today
<czajkowski> gmb: np
<czajkowski> gmb: too much retouching to do to my pic :)
<czajkowski> wgrant: how tied are ppas into builds?
<wgrant> czajkowski: How do you mean?
<czajkowski> wgrant: someone was sayign debian would like to have ppas
<czajkowski> but mentioned that they are tied into builds
<wgrant> I believe that's Launchpad's bug with the most me-toos...
<wgrant> Bug #188564
<_mup_> Bug #188564: Build also packages for Debian in PPA's <feature> <lp-soyuz> <ppa> <soyuz-core> <Launchpad itself:Triaged> < https://launchpad.net/bugs/188564 >
<wgrant> Technically it's easy.
<wgrant> The resources and social aspects are less so.
<czajkowski> well its been brought up todayin session
<bigjools> one of my goals is to allow people to use their own machines to build packages in their own ppas
<bigjools> which would open up the possibility of debian ppas
<rick_h_> wgrant: so the flush() business on the createTask just isn't necessary to maintain?
<czajkowski> bigjools: ah good to know
<bigjools> czajkowski: but that's totally unofficial :)
<bigjools> czajkowski: but it's a place where maas could shine
<wgrant> rick_h_: createManyTasks uses create(), which does that implicitly -- it runs a manual explicit Insert and then loads the tasks from the DB.
<wgrant> rick_h_: Creating with BugTask() just adds it to the list of pending add operations.
<rick_h_> wgrant: ah, ok gotcha
<wgrant> We also probably should rewrite validate_new_target and validate_target to be set-based, as they can be somewhat expensive, executing a few queries per target.
<wgrant> Eventually.
<rick_h_> wgrant: while you're in there with this can you hit lifeless's suggestoin on removing the BugTask middleman?
<wgrant> rick_h_: I didn't see that... is it a comment on the MP?
<rick_h_> https://code.launchpad.net/~rharding/launchpad/bugnom_874250/+merge/105317
<rick_h_> yea, after it was merged, so was going to do a follow up branch in the moring, but while you've got changes going
<rick_h_> it'll be more LoC reduction
<wgrant> Ah, it being post-merge would explain why I didn't see it.
<wgrant> Will fix, thanks for the pointer.
<rick_h_> ty much for the work, looking it over for education purposes
<wgrant> It can seem nice to have helpers on Bug like that, but then you end up with multi-thousand line classes like Person :/
<wgrant> np
<wgrant> Thanks for working on these damn timeouts.
<rick_h_> yea, well consistant vs "right" so oh well, guessed wriong
<wgrant> I might land a followup to drop Bug.addTask too
<wgrant> There's only a few dozen callsites.
<wgrant> Mostly tests.
<rick_h_> heh, this is awesome, so when I looked at removing the _init_ didn't want to dupe some of the code, but by doing that you get to remove it all
<rick_h_> wgrant: yea, that seems like it'd be the best plan, but trying to wrap this up by EOD torrow
<wgrant> Exactly :)
<rick_h_> so wasn't goint to go all the way through the clenaup atm since we've got a full plate starting next week
<wgrant> rick_h_: Eventually we probably want to move the nomination stuff into createManyTasks as well, but that opens a bit of a can of worms which I didn't really want just yet.
<rick_h_> wgrant: so the next timeout issue there is on the notification subscriber stuff
<rick_h_> it queries person 100's of times for 100's of subscribers, that's the next point after this one lands
<wgrant> BugNotificationRecipient?
<wgrant> Oh
<wgrant> That, right.
<rick_h_> sec, wonder if I still have the paste
<wgrant> That's going to be a fun one -- I'll be surprised if you get it done this week. But worth a try.
<rick_h_> yea, this is all for now
<rick_h_> I've got to finish the bugtask.txt killing tomorrow
<wgrant> Yep
<wgrant> Good riddance :)
<wgrant> Death to doctests, and all that
<rick_h_> damn straight
<rick_h_> http://paste.mitechie.com/show/650/ is a sucky paste of the oops post this change
<rick_h_> the select Person seems to be the next target
<wgrant> Inserting 242 bugtasks may still be too slow, given the exponential behaviour of bugsummary. But this has to be better than before.
<wgrant> And I'm rewriting bugsummary today to not be exponentially complex.
<rick_h_> cool
<wgrant> Ah, I misread the OOPS, there's not 242 in that case.
<wgrant> But we do have insane numbers like that sometimes.
<wgrant> The Ubuntu kernel team always manages to find new ways to break LP :(
<rick_h_> right it was taking 240ms per insert, now it's one inesrt to hopefully this helps a ton
<wgrant> Most of that's likely to be bugsummary, but we'll see.
<wgrant> wallyworld: Thanks.
<wallyworld> np
<wgrant> rick_h_: Ah, can't actually remove addTask, since it's exposed on the webservice.
<StevenK> return None!
<StevenK> WCPGW
<rick_h_> wgrant: booo, oh well
<wallyworld__> wgrant: did you have a pre-impl with anyone re: the sample data change? it sounds reasonable
<wgrant> wallyworld__: I polled lifeless and StevenK in -ops and nobody complained.
<wgrant> 15:05:17 < StevenK> wgrant: When I last used it, which was yesterday, I expected it to write to current{,-dev}.sql.
<wgrant> etc
<wallyworld__> wgrant: ok, thanks. r=me
<wgrant> wallyworld__: Thanks
<lifeless> wgrant: I suggested a larger sample size...
<wgrant> My sample size covered a third of the active team.
<wgrant> That's sufficient.
<StevenK> Haha
<lifeless> wgrant: so, I've just mailed the list in reply to curtis; I may be confused on some stuff... feel free to unconfuse me :)
<wgrant> lifeless: I can't disagree with anything you said.
* wallyworld__ changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: - | Firefighting: - | Critical bugs: 3.47*10^2
<lifeless> kk
<adeuring> gmb: would something explode if I increase the version number for testresources in versions.txt to 0.2.5?
<stub> adeuring: I think everything using testresources is using the package or buildout, so bumping the version is pretty much a requirement before anything can see your changes.
<adeuring> stub: right
* adeuring changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: adeuring | Firefighting: - | Critical bugs: 3.47*10^2
<stub> wgrant: We should be able to write a script to generate the bugtaskflat indexes by now :)
<wgrant> stub: Indeed.
<wgrant> I'll hopefully drop 2/3 of bug and bugtask's indices next week.
<StevenK> And hopefully DROP COLUMN bug.private soon after
<stub> wgrant: r=stub. Does this go live now?
<stub> Or staging first?
<wgrant> stub: I've tested them on DF, so live would be nice. Nothing will use them until Monday.
<stub> I'll apply it live now, assuming everything is quiet.
<wgrant> Should be.
<wgrant> Given that we killed everything 20 minutes ago
<wgrant> I'm surprised BugTaskFlat inserts are still performing OK, with 63 indices.
<wgrant> Although I guess most of them are very partial.
<stub> Launchpad is very low write.
<stub> And PG has been optimized over decades :)
<wgrant> Indeed.
<wgrant> Speaking of which, when can we upgrade to 9.2? :)
<wgrant> Index-only scans ftw.
<stub> It will take a while with librariangc and apache-log-parser running
<stub> wgrant: Sometime after it is released I think.
<wgrant> Bah
<wgrant> Live a little.
<gary_poster> wgrant, hey.  it looks like the sampledata change breaks a lot of tests.  The parallel runner shows 197 errors (http://ec2-23-20-37-133.compute-1.amazonaws.com:8010/waterfall) and the standard one shows 6 so far (https://lpbuildbot.canonical.com/waterfall)
<gary_poster> actually the errors seem very different
<gary_poster> http://ec2-23-20-37-133.compute-1.amazonaws.com:8010/builders/lucid_lp/builds/4/steps/shell_8/logs/summary
<gary_poster> vs.
<gary_poster> https://lpbuildbot.canonical.com/builders/lucid_lp/builds/2054/steps/shell_6/logs/summary (which is, wow, hard to read)
<gary_poster> lp.codehosting.puller.tests.test_scheduler.TestPullerMasterIntegration causes everything to fall over in the official buildbot
<wgrant> gary_poster: Odd, I've had two ec2 runs with those revs without a problem.
<wgrant> The production thing looks unrelated.
<wgrant> It's just normal flakiness
<wgrant> Looking at parallel
<wgrant> That looks unrelated as well
<gary_poster> wgrant, agreed.  sorry for bothering you; I saw the seeming correlation and assumed.
<wgrant> Quite a reasonable assumption... it's a more spectacular failure than we normally see spuriously :/
<gary_poster> heh, yeah
<wgrant> Worrying.
<gary_poster> we'll look into the ones we encounter on parallel.  As you probably saw, I forced a build on parallel, so hopefully that will show a green build
<wgrant> Well
<wgrant> TBH I hope it fails the same way :)
<wgrant> Having 200 spurious failures like that is ungood.
<gary_poster> Well, we generally file a bug and try to fix them anyway, if we can.  Sometimes it's harder to dupe than others.
<wgrant> Yeah.
<wgrant> stub: Thanks.
* bac changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: adeuring,bac | Firefighting: - | Critical bugs: 3.47*10^2
<bac> morning adeuring
<bac> adeuring: i'm going to start on wgrant's review
<adeuring> bac: ok (and good morning :)
<wgrant> Thanks bac.
<rick_h_> deryck: ping for stand up party time
<deryck> rick_h_, yup, on the way, thanks
<noodles785> Hi! Do the production LP updates happen at specific times during the day?
<rick_h_> noodles775: no, they occur as they're ready to go usually
<noodles775> k, thanks rick_h_
<rick_h_> noodles775: has some info https://dev.launchpad.net/LEP/FastDowntime
<abentley> rick_h_: I'm getting "ImportError: No module named convoy.meta" http://pastebin.ubuntu.com/981754/
<rick_h_> abentley: your download cache is up to date?
<abentley> rick_h_: yes.
<rick_h_> sorry, mean your deps, it's in the lp ppa
<abentley> rick_h_: I'll check.
<rick_h_> abentley: yea, looking for the -deps
<rick_h_> abentley: looks like it's in apt-cache show launchpad-developer-dependencies
<abentley> rick_h_: Not seeing convoy listed by apt-get upgrade.  I do see other LP deps.
<rick_h_> abentley: it's listed as python-convoy if that helps at all
<czajkowski> morning
<abentley> rick_h_: looks like I had uninstalled launchpad-developer-dependencies.
<rick_h_> abentley: ah ok. Yea, debugging here, I've got some many copies floating around with dev version, pypi versions, etc
<abentley> allenap: How does with-xvfb compare to xvfb-run?
* adeuring changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: bac | Firefighting: - | Critical bugs: 3.47*10^2
<rick_h_> jcsackett: ping, so is the issue with the dupe banners a matter of cleaning up the creators of banners is a pita?
<rick_h_> jcsackett: one other way to do the single method is to stick a variable in the module file and to store the $node so you don't have to query again.
<jcsackett> rick_h_: oh, i like the idea of just storing the node.
<jcsackett> although, come to think of it, that won't actually work.
<rick_h_> jcsackett: yea, I started thinkering in a branch but it's a bit messier
<rick_h_> you'd have to store a dict key'd by the class.NAME
<rick_h_> but then it looks funky when you pull it out and .set(text)
<rick_h_> and I don't have a full view of the scope so I'll just shush
<jcsackett> rick_h_: well, and there's still the problem of what happens if a banner object is created in another module.
<jcsackett> like, there's a banner created in the js in base-layout-macros on any private page.
<rick_h_> right, but this is tracked in the YUI.add() so there's only one
<rick_h_> you can stick 'private' vars inside that .add() code
<rick_h_> just how var ns = Y.namespace... each module has that ns. and it's a single value for all of that module definition
<rick_h_> no matter where it's used/messed with
<jcsackett> rick_h_: ah, got it.
<jcsackett> i think when i read "in module" i thought "in class".
<rick_h_> but yea nvm...forget me. Ugh on the need to deal with it, but understand
<rick_h_> right, no I mean in module
<jcsackett> rick_h_: yeah, it's still not perfect, but it's better.
<jcsackett> sometimes i'll take better. :-)
<rick_h_> a module can contain private stuff, mutliple classes, etc
 * jcsackett nods
<rick_h_> anyway, sorry I stuck my nose in it, my curiosity got peaked
<jcsackett> all good. you didn't happen to read enough of the MP to vote "approve" on it, did ya? :-P
<rick_h_> yea, I guess I'm there might as well eh
<jcsackett> *i* certainly wouldn't mind. :-)
<jcsackett> and i'm sure bac would be happy to let someone else help clear out active reviews. :-P
<rick_h_> k, sec, will do
<rick_h_> jcsackett: ok, approved with comments
<jcsackett> thanks, rick_h_.
<sinzui> bac ping
<bac> hi sinzui
<sinzui> bac , have you looked at https://code.launchpad.net/~dooferlad/launchpad/upcoming_view_show_all_work_items/+merge/105495 yet
<bac> sinzui: no
<sinzui> bac: I just investigated it. I will take it
<bac> sinzui: ok.  do you know what happened to wgrant's branch?
<sinzui> The one replaced methods to create fast queries of new bugs?
<bac> sinzui: ah, i see you got to it.
<bac> i got tied up before lunch.  thanks for handling it
<sinzui> I did. He made some changes for me and merged it 30 minutes later
<bac> cool
<lifeless> morning
<rick_h_> jcsackett: thanks for giving that a shot, sucky it didn't help
<lifeless> flacoste: ping; any chance of a quick call ?
* bac changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: - | Firefighting: - | Critical bugs: 3.47*10^2
#launchpad-dev 2012-05-12
<wgrant> sinzui: Oh, was the display vs visibility thing why the overlay buttons were showing through?
<sinzui> wgrant, yep
<sinzui> ms invented display, netscape invented visibility. MS took about 12 year to implement their rival's rules
<wgrant> sinzui: Yay
<wgrant> sinzui: Thanks for sorting all that out.
<sinzui> Monday I will finally work on choicesource
<wgrant> sinzui: If you have a sec I have a small branch to finish off the BugTaskFlat v2 stuff
<sinzui> okay
<wgrant> https://code.launchpad.net/~wgrant/launchpad/bugtaskflat-v2-use/+merge/105545
<sinzui> already approved
<wgrant> Thanks.
<sinzui> wgrant, you are right about that hack to suppress the bug tag links
<wgrant> Great. What about the duplicated domready in bugtask_index or whereever it was?
<sinzui> I had it disabled for a day using IE and chromium without any issue
<wgrant> Excellent.
<sinzui> I just removed it.
<sinzui> I have a lot of spaces and semi-colons to fix before I can land
#launchpad-dev 2012-05-13
<wgrant> 9.1's GiST trigram indices are nice.
#launchpad-dev 2013-05-07
<StevenK> wgrant: https://code.launchpad.net/~stevenk/launchpad/override-love-for-ddebs/+merge/162453
<wgrant> StevenK: k
#launchpad-dev 2013-05-08
<StevenK> wgrant: The removal of self.log(self._output.output) is needed?
<StevenK> (From lp-dev-utils)
<wgrant> StevenK: I'm not sure what the real fix is, I just quickly deleted the line in the traceback to get ec2 to work.
<StevenK> I have another change to make, so I'll ignore that removal for now
<StevenK> [2013-05-08 00:58:14,724: INFO/PoolWorker-3] Job resulted in OOPS: OOPS-f707f73e2b097961847648306d155654
<StevenK> InternalError: current transaction is aborted, commands ignored until end of transaction block
<wgrant> There was an earlier OOPS
<StevenK> Not that I can see in the log
<StevenK> wgrant: So, unscan-branch or delete and repush?
<wgrant> flacoste: Are you using Chromium to get those librarian 404s?
<wgrant> StevenK: What's the new DB MP?
<StevenK> I had to delete them because sync-pipeline completly lost its mind
<StevenK> wgrant: Right, all six MPs are up, all with diffs.
<wgrant> StevenK: Only six? I'm disappointed.
<wgrant> Will look soon
<wgrant> StevenK: 350	+ def findBranchMergeProposalIDs(self):
<wgrant> 351	+ return self.store.find(
<wgrant> 352	+ (BranchMergeProposal.id),
<wgrant> That doesn't do what you think it does
<wgrant> () is an empty tuple, but (foo) is just foo
<StevenK> wgrant: The garbo job actually works, though ...
<wgrant> StevenK: But the code still doesn't mean what you intended, so probably best to drop the parens.
<StevenK> wgrant: Right.
<StevenK> wgrant: Killing yui_3.3.0.zip from sourcedeps, should yui_3.5.1.zip also meet its demise?
<wgrant> StevenK: Might as well
<StevenK> wgrant: Fix for http://pastebin.ubuntu.com/5643581/ landing/landed.
<StevenK> wgrant: Your comment about removing merge_proposal_id is on the right MP?
<StevenK> Oh
<StevenK> Why?
<wgrant> StevenK: Why not?
<wgrant> It was added just for garbo
<wgrant> And now garbo is gone
<StevenK> It was not.
<wgrant> Wasn't it?
<StevenK> You should read switch-bmp-to-previewdiff-merge_proposal carefully.
<wgrant> Ah, so you're leaving it for a reference that only appears in a later branch?
<StevenK> Pretty much
<wgrant> That's OK, then.
<StevenK> We really can't use it in the code until it's populated
<StevenK> 371 + @property
<StevenK> 372 def branch_merge_proposal(self):
<StevenK> Why does this still exist?
<StevenK> wgrant: Because it's exported :-(
<wgrant> StevenK: Hm, can you just call the property that, then?
<wgrant> Rather than having merge_proposal separate
<wgrant> Just call it branch_merge_proposal
<StevenK> So the merge_diff madness continues
<wgrant> Hm?
<StevenK> merge_proposal_id / branch_merge_proposal probably isn't going to make you like me
<wgrant> No, that's illegal.
<StevenK> Which means we have branch_merge_proposal = Int(name='merge_proposal')
<wgrant> Or you rename the DB column
<wgrant> This sort of thing is why you don't land DB patches early. Makes them easy to revise in context :)
 * StevenK twitches at having to rename it in six branches
<StevenK> Well, five
<wgrant> StevenK: Also, have you considered pruning?
<wgrant> Until we have pruning, bmp.preview_diffs may grow without bound
<StevenK> I was thinking about that yesterday, but I'm not sure about the prune condition, since PreviewDiff.branch_merge_proposal is NOT NULL
<wgrant> We want to prune unreferenced ones. At the moment that probably just means everything except the latest.
<wgrant> But that will change once we start tying comments to a particular diff version
<StevenK> wgrant: Compound index?
<wgrant> StevenK: So we can efficiently answer "what is the current diff for this merge proposal?"
<StevenK> wgrant: Sure, I'm not sure what that index looks like, and if we need the current index in the patch
<wgrant> StevenK: (branch_merge_proposal, date_created)
<wgrant> (branch_merge_proposal) lets me quickly identify the diffs for the merge proposal, but without the additional sort of date_created I'd have to examine all of them to find the newest.
<stub> wgrant: how many diffs are you expecting to be attached to a MP?
<stub> If it is normally one or two and we never have thousands, I don't think the extra maintenance of the index is worth it.
<wgrant> stub: date_created will never change, and the index will always be appended to, so is that significant?
<wgrant> It should rarely be more than a dozen, but in pathological cases it may be.
<wgrant> And I don't see the expense of the additional index column as being particularly significant.
<stub> An extra index to keep hot, extra writes on insert, fatter index vs thin index and extra row loading...
<stub> And the overhead of all the extra typing in the db patch ;)
<wgrant> It's not an extra index, just a wider one.
<stub> oh right, new column.
<wgrant> We need one with branch_merge_proposal at the start either way
<stub> yup. I'm not fussed one way or t'other.
<wgrant> If it were a larger table then I would be concerned about the wider index, but I don't think it's an issue here.
<wgrant> StevenK: https://code.launchpad.net/~wgrant/launchpad/dont-overrideFromAncestry/+merge/162933
<StevenK> wgrant: r=me
<wgrant> StevenK: Thanks.
<StevenK> wgrant: Glance at the six again?
<StevenK> wgrant: When you're done with <redacted>, can you peer at the six MPs again?
<wgrant> StevenK: Sure
#launchpad-dev 2013-05-09
<mwhudson> wgrant, StevenK: do you guys keep track of how many lines of source code you've killed from lp over the last little while?
<cjwatson> I have a graph I update every now and again
<cjwatson> http://people.canonical.com/~cjwatson/tmp/loc-cum.png
<cjwatson> Data per revision in http://people.canonical.com/~cjwatson/tmp/loc
<mwhudson> it started around 500k loc ?
<mwhudson> so down about 15%
<cjwatson> About a million, I think
<mwhudson> ah
<mwhudson> still a nice big number
<wgrant> Depends on how you count, but by most metrics is 700KLOC-1MLOC
<StevenK> mwhudson: I've killed close to 40k
<StevenK> That last big drop on http://people.canonical.com/~cjwatson/tmp/loc-cum.png was me killing YUI 2.
<lifeless> woo
<lifeless> killing the source code gently
<wgrant> StevenK: In https://code.launchpad.net/~stevenk/launchpad/index-previewdiff-merge_proposal/+merge/162915 do you also want to make date_created NOT NULL?
<wgrant> And you need to rename the column in the ALTER TABLE to branch_merge_proposal
<wgrant> You should do both NOT NULLs in the same statement to get them done in a single table scan.
<StevenK> wgrant: Do I need DESC in the index too?
<wgrant> StevenK: It only makes a very marginal difference in this case, so I wouldn't bother.
<wgrant> It's important for the bug indices where we have a compound sort in opposing directions.
<wgrant> StevenK: https://code.launchpad.net/~stevenk/launchpad/switch-bmp-to-previewdiff-merge_proposal/+merge/162916 still pulls out all of the preview diffs just to get the last one
<wgrant> I'd do
<wgrant> @cachedproperty
<wgrant> def preview_diff(self):
<wgrant>     return self._preview_diffs.first()
<StevenK> wgrant: Ah, but isn't first the earliest diff?
<wgrant> I can think of an easy solution to that.
<StevenK> Asc(date_created) ?
<wgrant> (use .last(), or reverse the sort)
<StevenK> Ah, so .last() does exist
<StevenK> wgrant: Index fixed
<wgrant> StevenK: You also probably need to alter the preloading to populate preview_diff rather than preview_diffs
<wgrant> Anything that wants multiple BMPs isn't going to care about their historical diffs, just their current ones.
<StevenK> wgrant: Except that I'm pulling all previewdiffs
<wgrant> That there is precisely the problem.
<wgrant> It's unnecessarily pulling an unbounded volume of data.
<wgrant> It's potentially a large set, particularly before we have pruning implemented.
<StevenK> wgrant: I agree, but at the time I couldn't think of a way to make certain we only return the latest diff
<wgrant> StevenK: DISTINCT ON
<wgrant> SELECT DISTINCT ON (branch_merge_proposal) * FROM previewdiff WHERE branch_merge_proposal IN (1, 2, 3, 4) ORDER BY date_created;
<wgrant> ish
<wgrant> That'll return the oldest for each MP, but you get the idea
<wgrant> I believe I added the .config(distinct=PreviewDiff.branch_merge_proposal) syntax to storm a while ago, so that should be easy.
<StevenK> wgrant: Both index and switch-bmp have been pushed, and the diff has regenerated.
<wgrant> StevenK: ITYM Desc
<wgrant> In the previewdiff preloading
<wgrant> Asc is the default
<wgrant> I believe that what you have there will grab the oldest, not the newest
<wgrant> Which perhaps suggests that your preloading test is incomplete.
<StevenK> Preloading test? :-)
<StevenK> There's a preloading test?
<wgrant> Hopefully :)
<wgrant> If not then it's not difficult to add.
<StevenK> wgrant: You tell funny jokes. I'll add one.
<wgrant> StevenK, lifeless: Do you recall why ~canonical-launchpad exists?
<lifeless> No
<lifeless> is it private?
<wgrant> No
<wgrant> AFAICT it was only used to own production-auditor, which I've fixed.
<lifeless> it says '
<lifeless> Canonical employees working directly on Launchpad.
<lifeless> '
<wgrant> Which is roughly the definition of ~launchpad
<lifeless> yeah
<lifeless> there are specific branch owning teams already
<lifeless> which the new project policy page references
<wgrant> Exactly
<wgrant> And this isn't a member of any teams, and only had privileges in one project
<wgrant> So I'm not sure why it exists
<wgrant> Doesn't own any branches, not subscribed to any bugs...
<StevenK> wgrant: switch-bmp is ready for you again.
<wgrant> StevenK: Oh, that preloading sort should probably actually be on (bmp, date_created)
<wgrant> It'll work as it is now, but potentially slowly and it doesn't really make sense
<StevenK> wgrant: .order_by(PreviewDiff.branch_merge_proposal, Desc(PreviewDiff.date_created) ?
<wgrant> StevenK: Right
<StevenK> wgrant: Is that your only remaining concern?
<wgrant> StevenK: I thiiiiiink so, apart from the pruning thing
<wgrant> Which we at least need a plan for before this lands
<StevenK> wgrant: Given your idea that we want only one previewdiff per MP in the short term, I'm happy to put a pipe at the end that adds it
<wgrant> StevenK: Sounds good
<wgrant> I'll go over the MPs again for a final check in a sec
<StevenK> wgrant: I want GROUP BY for this query to only show old previewdiffs?
<wgrant> StevenK: DISTINCT ON and GROUP BY are usually mutually exclusive
<wgrant> They perform a similar role, except that GROUP BY gives you aggregates.
<StevenK> I don't want DISTINCT branch_merge_proposals, though
<wgrant> StevenK: DISTINCT ON branch_merge_proposal
<wgrant> Not DISTINCT branch_merge_proposal
<StevenK> To handle the case that we have 2 old previewdiffs to be pruned.
<wgrant> Oh
<wgrant> To show old ones
<StevenK> Yes
<wgrant> You want neither
<wgrant> You'll need window functions or a subquery
<StevenK> I can see how a window function can help, how does a subquery?
<wgrant> You can use a subquery to select the latest previewdiff for the MP and then use NOT IN
<wgrant> Or !=
<StevenK> Bah
<StevenK> ERROR: window function requires an OVER clause
<StevenK> And so I add one:
<StevenK> ERORR: window functions not allowed in WHERE clause
<wgrant> Sure, you usually have to use a subquery
<wgrant> So you return the window function result in the select list of a subquery, and filter on that in the outer query
<StevenK> wgrant: http://paste.ubuntu.com/5646915/
<wgrant> StevenK: Right
<StevenK> It even works
<StevenK> wgrant: .config(distinct=PreviewDiff.branch_merge_proposal) is not making use of DISTINCT ON
<wgrant> StevenK: Hm, it possibly needs to be a tuple
<wgrant> I forget
<wgrant> I only wrote it
<wgrant> 39	+ if isinstance(select.distinct, (tuple, list)):
<wgrant> 40	+ tokens.append(
<wgrant> 41	+ "ON (%s) " % compile(select.distinct, state, raw=True))
<StevenK> Excellent
<cjwatson> StevenK: YUI 2> Yeah, I went and looked at that commit :)
<StevenK> cjwatson: Haha, as a "What the hell just happened?" sort of check?
<cjwatson> Not really, I just tend to look at big negative commits out of interest
#launchpad-dev 2013-05-10
<StevenK> wgrant: Pushed a pruner branch up, I fits after switch-bmp and before drop-branchmergeproposal
<StevenK> wgrant: I'm waiting to see if the scanner actually loves the branches
<wgrant> StevenK: I pushed a couple of branches this morning, so it should be fine.
<wgrant> StevenK: How well does BulkPruner handle long queries like the one in PreviewDiffPruner?
<wgrant> StevenK: Can you push -r-2 and repush https://code.launchpad.net/~stevenk/launchpad/switch-bmp-to-previewdiff-merge_proposal to get the diff updated?
<StevenK> wgrant: From BulkPruner's docstring: "This is designed for the case where calculating the list of items is expensive, and this list may be huge. For this use case, it is impractical to calculate a batch of ids to remove each iteration."
<wgrant> StevenK: Aha
<StevenK> wgrant: I can't really push from a pipe'd branch
<wgrant> StevenK: Why not?
<wgrant> It's not that magical.
<wgrant> Switch to the relevant pipe
<wgrant> Then push
<StevenK> The push branch in bzr info still includes .bzr/branches
<wgrant> StevenK: Sure, that automatic push branch will be wrong
<wgrant> So don't use the automatic push branch :)
<StevenK> wgrant: I need --overwrite with -r -2?
<wgrant> StevenK: Yes
<wgrant> StevenK: Only a push of a superset will work without --overwrite
<StevenK> wgrant: Hopefully that fixed switch-bmp
<wgrant> StevenK: Indeed
<StevenK> wgrant: I've re-sync'd the pipeline to fix your concern with 44-1
<wgrant> StevenK: So, apart from the DB branch I think that's fine. Normally now I'd do a full trial migration on DF just to catch any gotchas, given that it won't get a full test run on qastaging until most of it's already deployed to production (and the DB)
<wgrant> StevenK: Are you going to move the NOT NULLs into 44-2?
<StevenK> wgrant: That was your concern with 44-1
<wgrant> StevenK: 44-2 is an fdt patch
<wgrant> 44-1 is live
<wgrant> a NOT NULL on a non-trivial table must be fdt
<StevenK> And yeah, they'll be in 44-2
<wgrant> Right
 * StevenK reaches for mawson
<StevenK> wgrant: Anything surprising in ZTK 1.1.6?
<wgrant> StevenK: No, just a few minor upgrades
<wgrant> Our stuff also mostly works with ZTK 2.0a1, once I hacked buildout 2 to be sane.
<StevenK> Hah
<StevenK> Bleh
<wgrant> (by "mostly works" I mean RootObject:index.html renders)
<StevenK> The pruner stuff is infecting drop-branchmergeproposal-merge_diff
<StevenK> Now I have to resubmit it
<wgrant> Or you could just pretend that drop-branchmergeproposal-merge_diff isn't really part of the pipe any more
<wgrant> Given you're basically at the end anyway
<wgrant> uncommit the merge?
<StevenK> Not that simple
<wgrant> It's not the last branch?
<StevenK> Given r16611 is the last revision which changes the DB patch
<StevenK> And r16609-r16610 are merges from the prune branch
 * StevenK waits for mawson to finish WADLing
<StevenK> wgrant: Do you still require the lib/lp/soyuz/scripts/publishdistro.py hack on DF?
<wgrant> StevenK: The one to publish just my PPA?
<StevenK> wgrant: Yup
<wgrant> StevenK: If it's just that, then kill it
<StevenK> wgrant: DF populated, 108020 previewdiffs with NULL branch_merge_proposal and date_created
<wgrant> StevenK: And how many not null?
<StevenK> Oh, sorry, that is NOT NULL
<StevenK> 178059 are both NULL
<wgrant> StevenK: Does 108020 roughly equal the number of merge proposals?
<StevenK> 114550
<StevenK> Close enough
<wgrant> Right
<StevenK> wgrant: Let me slay 178059 previewdiffs
<wgrant> StevenK: Perhaps not
<StevenK> Oh?
<wgrant> StevenK: That part of the testing is difficult to revert
<wgrant> Everything else is easy to redo later
<wgrant> Deleting them is not :)
<wgrant> So I'd skip that piece of testing until it's unavoidable.
 * StevenK drags DF up to switch-bmp
<nigelb> Hrm, is there any way I can get the api I made for bug status in comments made more public?
<wgrant> nigelb: What do you mean "more public"?
<nigelb> wgrant: Um, can somethign that's not LP hit it?
<nigelb> It's one of those +link urls.
<wgrant> +check-links?
<wgrant> I don't see why not.
<nigelb> Oh.
<nigelb> Stupid question - how do I hit it?
<nigelb> launchpad.net/+check-links?
<wgrant> +check-links should exist on every object, I think
<wgrant> https://launchpad.net/+check-links?link_hrefs={%22branch_links%22:%20[%22/+branch/foo%22,%20%22/+branch/launchpad%22]}
<nigelb> <p>Launchpad requires a <code>REFERER</code> header to perform this action. There is no <code>REFERER</code> header present. This can be caused by configuring your browser to block <code>REFERER</code> headers.</p>
<nigelb> (hitting it with requests)
<wgrant> nigelb: GET should work
<wgrant> For POST you'll need to forge a referer
<nigelb> aha
<StevenK>     raise LookupError(aliasID)
<StevenK> LookupError: 51928797<br />
<StevenK> BAH
<StevenK> wgrant: I guess I need to create a BMP and then run the BMPJ to get a real previewdiff on DF?
<wgrant> StevenK: Or create a BMP and fake the BMPJ
<wgrant> I'd fake the BMPJ
<StevenK> I have a bunch of BMPs
<wgrant> DF can in theory talk to parts of prod codehosting, but probably not directly to the branch store
<wgrant> So create the PD manually with fake content, however the BMP does it.
<StevenK> wgrant: /srv/launchpad.net/dogfood-logs/2013-05-10/OOPS-e4a155a1aec93e2714a3a720edf626f7
<wgrant> StevenK: That's why I said not to use a BMPJ
<StevenK> Bleh
<czajkowski> morning
<StevenK> wgrant: https://code.dogfood.launchpad.net/~stevenk/launchpad/skip-private-dev-focus-mishmash/+merge/131124
<StevenK> wgrant: http://pastebin.ubuntu.com/5650261/ is relevant too
<wgrant> StevenK: And date_created is set?
<StevenK> 2013-05-10 06:40:50.0436
<wgrant> Excellent
<StevenK> wgrant: So I can land 40-0?
<wgrant> StevenK: What's your full deployment plan?
<wgrant> The sequence of landings, deployments and migrations.
<StevenK> wgrant: http://pastebin.ubuntu.com/5650283/
<wgrant> StevenK: Sounds good
<wgrant> (plus the pruner after that)
<wgrant> StevenK: https://code.launchpad.net/~wgrant/launchpad/bug-1124352/+merge/163270
<StevenK> wgrant: Pruner is step 5
<wgrant> Er yeah
<wgrant> That
<StevenK> wgrant: You allow setting of buildd_secret in the ZCML
<StevenK> For IArchiveAdmin
<wgrant> StevenK: That's necessary because setting private sets buildd_secret
<wgrant> But no forms exposes it, and it's not on the API
<StevenK> Right
<wgrant> Locking it down further in ZCML will cause lots of test failures
<wgrant> But we should do it eventually
<StevenK> wgrant: +98/-97 disappoints me
<StevenK> But r=me
<wgrant> Huh
<wgrant> I assumed it'd be net +100 or so
<wgrant> What did I do...
<wgrant> All the buildd_secret stuff, I guess
<wgrant> This wasn't meant to be even close to negative :)
<wgrant> A convenient accident
<StevenK> Haha
<wgrant> I'll create the celebrities on all four instances.
<wgrant> StevenK: Can you QA your changeOverride thing?
<wgrant> I hope to deploy tonight
#launchpad-dev 2014-05-05
<haobug> hi hi, got problem again. http://pastebin.com/XxaqxmbJ after about 3 days.
<cjwatson> haobug: I answered you yesterday.
<cjwatson> 15:41 <cjwatson> haobug: You don't need to redo it all from scratch, at least, but there's some manual work to do to recover from a failure there without re-downloading - it should be possible to look at /usr/share/lxc/templates/lxc-ubuntu and search for
<cjwatson>                  "Installing updates" to find where it got to, but you'll have to trace the function call sequence up to there for yourself
<haobug> strangely, it is not there, "ls: cannot access /usr/share/lxc/templates/lxc-ubuntu: No such file or directory"
<cjwatson> What Ubuntu release are you using?
<cjwatson> It's apparently in /usr/lib/lxc/templates/lxc-ubuntu in 12.04 and earlier.
<haobug> 12.04.1 (Linux dell990 3.5.0-23-generic #35~precise1-Ubuntu SMP Fri Jan 25 17:13:26 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux)
<haobug> /usr/lib/lxc/templates is not there either.
<cjwatson> That's surprising given that it's in the lxc package.
<haobug> i rebooted my computer last night. i can can not found the command typed.  but i remembered follow the manual `sudo lxc-create -t ubuntu -n lpdev -- -r precise -a amd64 -b $USER` <--replaced i386 to amd64
<cjwatson> /usr/lib/lxc/templates/lxc-ubuntu is in the same package as lxc-create is.
<cjwatson> Anyway, can't help further, it's a holiday here and you're going to have to apply some creativity to this ...
<haobug> are there some *dirty* way to configure all that up on my current Ubuntu installation?
<haobug> cjwatson: i must be the most careless reader. "AlternativelyYou can also run in aÂ chrootÂ environment or aÂ VM." i have a vm with ubuntu installed in it.
<cjwatson> I'm afraid I can't help further, sorry.
<haobug> cjwatson: thank you all the way alone. the VM way struck me a bit, chroot way looks okay to me. i am going to run into that alone ;)
<cjwatson> Containers are generally quicker and less memory-hungry than VMs, but sure, if you already have one set up then it should work fine.  You might like to take a snapshot of it before starting.
<haobug> cjwatson: the  dos tells to install KVM, aha, that remind me about the lxc story.
<barry> xnox: hey, i'm going to look at your branch, but it's been a jillion years since i built and ran lp locally ;)
#launchpad-dev 2014-05-06
<xnox> barry: the latest & greatest way is via lxc container these days =)
<wgrant> https://dev.launchpad.net/Running/LXC
<wgrant> But which branch?
<xnox> barry: see launchpad dev wiki. I had it running in the container quite quickly without destroying anything.
<xnox> wgrant: the one were i add support for header auth on the +access-page.
<wgrant> Oh, right.
<xnox> lp:~xnox/launchpad/devel or some such
<wgrant> Yeah, worst branch name ever :)
<xnox> wgrant: well i had a normal one, but after the whole branch setup machinery it started to force devel name for me.
<xnox> wgrant: anyway, for some reason just by changing the form to either be from headers or from body, results in request token to getting self destroyed when going via header auth.
<xnox> thus one can exchange it multiple times.... which is bad.
<wgrant> xnox: Perhaps the transaction is not being committed in some cases?
<wgrant> I'd turn on full statement logging in postgresql.conf to see what's going on.
<xnox> interesting. i'll explore that. is there a way to enforce committing transaction either in the view code or in the "story" document?
<wgrant> A POST should always be committed unless it crashes.
<xnox> i see. i do wonder if it is crashing.
<cjwatson> wgrant: Your livefs copy_field/export_factory_operation comment about my error in IPersonViewRestricted.createLiveFS: wouldn't doing either of those entail importing ILiveFS into lp.registry.interfaces.person, which is probably suboptimal?
<wgrant> cjwatson: That's a good point. You could use contributes_to, or better still use something like lp.livefses.new
<wgrant> We should avoid bloating IPerson even if it doesn't require layer-violating imports.
<cjwatson> Mm, right
<wgrant> Also, I've realised it's probably a terrible idea to allow random people to request builds of someone's LiveFS. It means a build is half owned by the requester, half by the LiveFS owner, and half by the archive owner.
<wgrant> Requiring that the requester must be able to operate on behalf of the LiveFS owner solves most of that problem, as the only impact the archive owner can have is deleting their PPA.
<Kagetsuki> hello, I'm trying to set up a PPA and I'm a little confused on some things. First off I have a github repository which builds two debs with cmake - a lib and a command line client. Is there some way to specify this in the control file? should I have two control files? should I have two debian directories?
<wgrant> If we eventually open up builds to the world, we probably want some way to easily clone LiveFSes. But the free-for-all nature of projects and recipes isn't ideal, and we can easily avoid introducing the same problems here.
<cjwatson> The use case I was thinking of was somebody in the touch team requesting a build of one of their LiveFSes against a particular PPA, without them having to be authorised to build the primary ones.
<wgrant> Kagetsuki: This channel is for the development of the code that runs Launchpad.net. You probably want #ubuntu-packaging.
<cjwatson> Perhaps being able to copy a LiveFS easily is better, indeed.
<wgrant> cjwatson: Yeah, I see the use cases.
<Kagetsuki> @wgrant OH! Sorry! Keep up the awesome work!
<cjwatson> (For the time being that could be an API script without much trouble.)
<wgrant> Exactly.
<wgrant> The consumers are so few early on that we can see what happens.
<wgrant> And if it doesn't work out, we can find some way to relax the restriction without introducing the foreseen problems.
<wgrant> But it's difficult to reign it in later.
<wgrant> rein
<wgrant> Given that you haven't implemented authorisation yet, I guess the main change this yields is that the paths become something like /~ubuntu-cdimage/+livefs/ubuntu/utopic/desktop/+build/1234, rather than /ubuntu/+archive/primary/+livefsbuild/1234.
<cjwatson> That's better anyway.
<wgrant> Yeah.
<cjwatson> Well.  Somebody in ~ubuntu-cdimage (say) could still in theory request a build for a different PPA
<wgrant> They could, yes.
<cjwatson> But maybe that's not worth supporting
<wgrant> Why is that a problem?
<cjwatson> The per-archive URLs would still make sense in that case
<cjwatson> If we're relying on LiveFS copying to support the touch use cases, I wonder if we shouldn't fold a bunch of stuff from LiveFSBuild back into LiveFS
<wgrant> LiveFSBuild has to have archive on either way, it's mostly a matter of whether requestBuild takes the arg or not.
<cjwatson> Though I think we still want to have the same LiveFS for a pre-release build against the release pocket and a post-release build against -proposed
<wgrant> Hm, I guess it doesn't have to, actually, if we decide that LiveFSBuilds are solely owned by the LiveFS owner.
<wgrant> Right, we should support that. That's a common case that doesn't cross any trust boundaries.
<wgrant> Changing the archive also doesn't really cross any trust boundaries. It's mostly a matter of whether we want to support it.
<cjwatson> And of course the DAS should still be on LiveFSBuild.
<wgrant> I believe the only thing in question is Archive.
<cjwatson> So I guess we don't gain much, because LiveFS would still be a partial specification.
<cjwatson> I would like to avoid the whole question of how privacy of livefs vs. archive owner interact, if possible.
<cjwatson> So saying that if you want a different archive then you must copy the livefs is somewhat appealing.
<wgrant> That problem will always exist. We just have to minimise the impact.
<wgrant> The LiveFS owner could lose access to the P3A either way.
<cjwatson> Oh, and other members of the LiveFS-owning team might not be able to see a private Archive even if the member of the team who created the LiveFS can.
<wgrant> Having LiveFSBuilds live in a hierarchy under the LiveFS means that the owner could still traverse to them, they just wouldn't be able to see the archive field.
<cjwatson> Yeah.  In that case I'm inclined to leave the DB layout the way it is
<wgrant> The only way you could totally avoid it is by requiring archive.owner == livefs.owner, which isn't going to happen
<cjwatson> Being able to request a build into some other archive is convenient for one-off tests
<cjwatson> Sorry, I should stop saying into
<cjwatson> against some other archive
<wgrant> Right, I think the only change we want from the old design is that the LiveFSBuild requester must participate in the LiveFS owner at the time of the request.
<cjwatson> And the hierarchy change
<wgrant> Er yes, that too.
<cjwatson> Sounds doable then.
<cjwatson> /livefses is a messy plural, but whatever
<wgrant> Indeed, and it's unfortunate having a collection with one method, but it's better than having every create method ever on IPerson or IProduct. Really makes things difficult to understand, with all the layering violations.
<barry> xnox: cool.  i have a few other things to do this morning, but will take another look later today
<cjwatson> wgrant: FWIW I've thought about it and I think we have to keep LiveFSBuilds against private Archives themselves private.  Otherwise you can run a livefsbuild of your choice against a private archive and find out what's in it even after you've lost access to it.  And if you don't have access to the archive then none of the artifacts on livefsbuild are going to be any use to you, so there isn't much point in being able to traverse to it ...
<cjwatson> ... either.
<wgrant> Yeah, I realised that a couple of hours ago as well.
<cjwatson> Ugh, the build system isn't happy with this /livefses top-level collection
<wgrant> cjwatson: Oh, howso?
<cjwatson> http://paste.ubuntu.com/7404885/
<cjwatson> Something to do with there being no non-devel methods, maybe?
<cjwatson> The traceback is not wholly transparent
<wgrant> Yeah, TALES is great. Suppresses everything into a LocationError.
<wgrant> Try exporting LiveFS and LiveFSBuild on >=beta, rather than >=devel
<cjwatson> Ah, https://bugs.launchpad.net/lazr.restful/+bug/760849
<_mup_> Bug #760849: No way to restrict export_as_webservice_collection to a given API version <api> <lazr.restful:Triaged> <https://launchpad.net/bugs/760849>
<wgrant> Ah, yeah, I remember that now.
<cjwatson> wgrant: Thanks, better now.
<cjwatson> xnox: devel> rocketfuel-{setup,get} etc. force devel for the main LP devel branch, but you aren't expected to work there; you're expected to "bzr branch devel some-proper-name" and do your work in there.
<cjwatson> (And run "utilities/link-external-sourcecode && make" in the new branch in an appropriate container.)
<xnox> ok.
#launchpad-dev 2014-05-07
<cjwatson> wgrant: If it's OK by you, I'm going to defer LiveFS deletion to a later branch; deleting a LiveFSBuild is easy enough, but it's not totally obvious how to delete a LiveFS in a way that doesn't scale terribly to ones with lots of builds
<wgrant> cjwatson: Sure.
<wgrant> This should have been five branches already anyway :P
<cjwatson> FWIW my date_last_modified ZCML hack is also present in the SourcePackageRecipe ZCML, so somebody else had the same problem
<wgrant> cjwatson: Why not put it in the Edit interface?
<cjwatson> It is in the Edit interface
<cjwatson> ILiveFSEditableAttributes that is
<wgrant> Hrmph.
<wgrant> Zooooope
<cjwatson> Oh, it works if I make date_last_modified readonly=False, but then it's writeable over the webservice
<wgrant> Ah, of course, set_schema respects readonly
<cjwatson> Don't suppose there's a gadget to export a writeable attribute as readonly?
<wgrant> The pattern I've usually seen is that the form that needs it to be writable will override the field in its schema.
<wgrant> So I don't know of one.
<cjwatson> I guess what I have is the least bad hack then.
<wgrant> So it seems.
#launchpad-dev 2014-05-08
<bigjools> wgrant: longpoll doesn't seem to be working lately
<wgrant> I thought it was just me.
<haobug> rabbitmq-server problem: http://bpaste.net/show/we7EQafgxt46BZJZEAel/, here is /var/log/rabbitmq/startup_err: http://bpaste.net/show/gFQEpF1qG2Vprp0HYtRR/ and the /var/log/rabbitmq/startup_log is empty
<haobug> i am now trying to use the chroot way to run launchpad. because i failed the lxc way and struck by the vm way.
<haobug> why does the launchpad use such many dependencies?
<haobug> er... http://bpaste.net/show/jdMsJi6SADj54kPTtJ5n/
<cjwatson> wgrant: Are you OK with my suggestion in my livefs MP to rename the livefs/livefsbuild columns to distro_series/distro_arch_series, as a way to resolve the naming inconsistency?
<cjwatson> If so I might as well do that sooner rather than later
<wgrant> cjwatson: That would be my suggestion too.
<cjwatson> OK, will do.
<haobug> are you talking about my problem? i don't get you idea.
<cjwatson> I was not talking about your problem since I don't know the answer to that.
<wgrant> haobug: You really should be using LXC or a VM. Running it in a chroot is going to lead to weird port conflicts and all manner of other madness.
<haobug> wgrant: my vm is configured use NAT, the bridge way cannot access to internet.
<wgrant> Is NAT a problem?
<haobug> yes i can use port mapping, but the vm guide told me to install the kvm bla bla.
<haobug1> I have tried to config launchpad in guest vm run on top of virtualbox, fialed again.
<haobug1> How about release preconfiged vmdisk file. That someone like me can get start easily.
<wgrant> Failed how? The documented setup processes work fine; I tested both just last week.
<haobug1> I am now returned from office. I can feed back to you tomorrow.
<haobug1> I remembered a little: rocket-setup says, can not create local rocketfeul blabla
<haobug1> How about to pack you working env up as a tarball or vmdisk or any other preconfiged way.  |wgrant
<wgrant> That gets out of date easily.
<wgrant> We provide easy scripts and instructions to set up a development environment.
<haobug1> I am newbie to linux. But not that new. I have used linux for about 3 years.  I can't recover form the errors.
<haobug1> Say, there is a dist-upgrade step in the instruction. This step itself will fail sometime.
<haobug1> And all steps are bond to ubuntu. I use gentoo at home, I can't try it out after i returned from work.
<haobug1> Think about release a working(even out of date) bundle for newbies like me please.
<cjwatson> wgrant: Renames done now.  (I need to update my browser branch, but that's not pushed anywhere yet anyway.)
#launchpad-dev 2014-05-09
<haobug> virtualbox ubuntu 12.04 fail: error part1: http://bpaste.net/show/SkCeEcWaCt9X5B85AJqd/. i have restarted the process, the second error have not come up.
<lifeless> mpt: your unfailing politeness on old bugs is amazing
<mpt> lifeless, hah, if I had to describe my comment in one word âpoliteâ would not have been my choice
<mpt> But thanks :)
<lifeless> mpt: it was politer than what was in my head :)
#launchpad-dev 2015-05-04
<blr> wgrant: can BRANCH_TYPE_VOCABULARY in productseries return a more generic message?
<blr> i.e. "Link to a branch already on Launchpad" (removing the explicit mention of Bazaar)
<wgrant> blr: The form needs a bit of a redesign, I think, but we can't make the optimal changes until later on (eg. once we have Git mirroring).
<wgrant> It's all a bit weird, since you can (and, during migration, want to) have defaults for both Git and Bazaar.
<wgrant> So we can't just add an additional radio button for Git.
<wgrant> We need to have a Git section and a Bazaar section.
<wgrant> And the Git section can't currently allow an import.
<blr> wgrant: is there some voodoo required for importing yui styles for a given yui widget?
<blr> wgrant: assuming I need to customise those styles in style.css?
<blr> would prefer to overriding specific styles rather than copying them entirely into styles.css
<blr> wgrant: nvm, figured it out, appears everything under components is built
<wgrant> blr: Which widget are you using?
<blr> wgrant: tabview
<blr> which doesn't appear to be used anywhere
<wgrant> Indeed, I think it is not.
<wgrant> Where are you applying it?
<blr> wgrant: making bzr/git instructions more compact
<blr> a tab for each - think it will work, but need to tidy up the styles, the yui defaults are not particularly nice.
<wgrant> blr: Hmmmm.
<wgrant> blr: Except for Person:+branches, we should probably have separate pages for bzr and git listings, since only one's going to be active at a time.
<blr> context here is push instructions on +setbranch
<wgrant> And when there are separate pages there are obvious places for both sets of instructions.
<wgrant> Ah, right.
<wgrant> Worth a try for that, indeed.
<blr> but right, better if we don't generally have to have both!
<blr> wgrant: was pleasantly surprised how easy it was to add a new sprite btw, thanks
<blr> black magic.
<wgrant> Yeah, it's pretty well integrated.
<wgrant> So well integrated that I haven't bothered to look at how exactly it works.
<blr> the git logo is still recognisable at 14x14 thankfully
<wgrant> Where are you using it?
<blr> on the tabs e.g. | (logo) git | (logo) bzr |
<blr> I'll include a screenshot in the bmp
<wgrant> blr: Ah, hm, that looks great, except I've just realised it's not so simple, of course.
<wgrant> blr: There is no concept of a series-default git repository, because a series is a single branch, while the git repository is target-global.
<wgrant> We probably need to rethink the setup UI before we can do anything even vaguely sensible.
<blr> wgrant: argh, ok :(
<blr> wgrant: I've set it back to wip, perhaps some of it can be re-used later.
<wgrant> blr: I suspect we'll want to convert that into a project-wide default code setting page, which does what works for most projects (sets the project default for git, and the development series default for bzr).
<wgrant> So it can definitely be reused.
<blr> wgrant: not sure how things were left with cjwatson re: remaining UI tasks - any suggestions for tomorrow?
<blr> think I generally have a better grasp on the frontend now
<wgrant> blr: I don't know where Colin got up to with the repo listings, if anywhere.
<wgrant> Nobody knows exactly how they should be done.
<wgrant> It's mostly a matter of experimenting at this point
<blr> hopefully I'll have a window to catch up with him in the morning
<wgrant> Indeed.
<wgrant> (catching up is much easier if you can arrange to have your IRC client always connected, too)
<cjwatson> morning> fwiw I'm off today, bank holiday
<blr> cjwatson: ping me if you're about please :)
<blr> interesting to see people asking for webhooks on that reddit thread.
<blr> cjwatson: for the UI work, are you finding it sufficient to use factories, or are you running turnip in dev as well?
<cjwatson> blr: factories for unit testing, a dev turnip instance for in-browser experimentation
<blr> cjwatson: are you running turnip in a container?
<cjwatson> blr: have done in the past, right now I'm using a juju-deployed instance
<cjwatson> either works
<cjwatson> well, that's deploying to a container
<cjwatson> so "yes"
<blr> hah right
<cjwatson> easiest is to set git.launchpad.dev to the appropriate IP address in /etc/hosts in your host system, the individual containers fall back to that by way of (I guess) dnsmasq
#launchpad-dev 2015-05-05
<wgrant> blr: Morning. What're you experimenting with atm?
<blr> wgrant: looking at colin's branch/patch. Are there any views you can think of that need attention?
<blr> wgrant: how difficult do you think setting a default on project creation will be?
<wgrant> blr: We can't set a default repo on project creation, since the project can't have any repos yet.
<wgrant> Oh, or a default VCS type?
<wgrant> That's not terribly difficult, but it requires some reshuffling so we shouldn't do it right now. Let's proceed with the default trunk series, and integrate git into the Code setup steps.
<blr> right, default cvs.
<blr> err vcs even :)
<blr> well, that would be a great way to lose all the free will git has bought us, more cvs integration.
<wgrant> Haha
<blr> err good will even
<blr> ugh, juju having an absolute conniption.
<wgrant> What now?
<blr> old environment from utopic, it appear to be happier after a clean bootstrap.
<wgrant> Ah, yes.
<wgrant> If you're on vivid now, make sure you're running at least juju 1.23.1
<wgrant> Which isn't in vivid.
<blr> yep on 1.23.2
<blr> wgrant: trying to diagnose "fatal: could not read from remote repo" locally, lpdev and turnip instances can see one another, and I've set my pubkey on the test user. Product exists
<blr> any ideas?
<wgrant> blr: There should have been an error before that.
<blr> wgrant: oh, proxying
<blr> no route to 22
<blr> are you running haproxy locally?
<wgrant> blr: You need to use 9422 locally.
<wgrant> the 22 -> 9422 rewrite is done at the firewall in prodstack.
<wgrant> 22 will get you openssh on a local juju deployment.
<blr> wgrant: ah, turnip isn't running - Couldn't listen on any:/srv/turnip/data/repos/hookrpc_sock_19418
<wgrant> blr: Huh, how old is that?
<wgrant> Or does the directory not exist?
<wgrant> It's meant to unlink the socket on startup if it already exists for some reason.
<blr> /srv/turnip/data does data/repos does not
<blr> that's rev 155
<wgrant> blr: You're not deploying turnip with use_storage=true, are you?
<blr> wgrant: deploying from the makefile
<wgrant> Hrm
<blr> wgrant: are you able to deploy 155 locally without issue?
<wgrant> blr: I just did a fresh run on a fresh environment and it worked fine.
<blr> wgrant: yep, trying a clean env, I've clearly done something insane.
<blr> wgrant: on push [KeepAliveSettingSSHServerTransport,11,10.0.3.1] Got remote error, code 11
<wgrant> blr: That's not the first error message.
<wgrant> blr: You set virtinfo_endpoint and authentication_endpoint correctly, and they are connectable from turnip/0?
<blr> wgrant: ah I think virtinfo_endpoint is incorrect
<blr> wgrant: currently localhost:6543
<wgrant> Ah, that would be an issue.
<blr> wgrant: ok all is well. that's enough gross stupidity for one day! thanks
<wgrant> Heh
<blr> wgrant: where would I find an OOPs logged locally?
<wgrant> blr: In the depths of rabbitmq. Easiest thing to do is pkill beam.smp, then retrigger the OOPS and look in /var/tmp/lperr
<blr> wgrant: cheers
<cjwatson> wgrant,blr: https://code.launchpad.net/~cjwatson/turnip/enable-reflog/+merge/258084 should be an easy win if you get a minute.
<blr> cjwatson: any idea why mojo is trying to unpack libpam-systemd_204-5ubuntu20.11_amd64.deb on trusty?
<blr> cjwatson: hmm could be my mojo version
<blr> nope... https://pastebin.canonical.com/130761/ libpam-systemd shouldn't be selected surely.
<cjwatson> blr: damnit stay on IRC :)
<cjwatson> wgrant: BTW not sure I'll make tonight's meeting - Kirsten and I have a date
<wgrant> cjwatson: Sure.
#launchpad-dev 2015-05-06
<cjwatson> wgrant: Do you have the new sbuild stuff nearly ready to go?
<wgrant> cjwatson: I deferred it after the whole ddeb debacle, and I'd like to give it a little more time, since it completely removes our ability to back out.
<cjwatson> Not if we only deploy it to scalingstack.
<wgrant> True.
<cjwatson> But OK ...
<wgrant> The items I have on my list still are verifying correct depwait parsing, logtail encoding, and working out how on earth dpkg-buildpackage managed to still have LANG set despite it being unset by the two layers above it.
#launchpad-dev 2015-05-07
<blr> the MenuAPI is making my head hurt.
<blr> wgrant: have you ever considered moving the permission checks on views to decorators?
<blr> might be nice
<wgrant> blr: MenuAPI is indeed a little weird, and parts of it are not used in the current UI design.
<wgrant> The way views are defined in ZCML is inherited from Zope 3/ZTK. Grok has an alternate syntax that uses the same infrastructure, but doesn't use class decorators: http://grok.zope.org/documentation/tutorial/permissions/tutorial-all-pages
<blr> huh, this is the first I've heard about Grok.
<wgrant> It was really Zope 3's dying breath.
<wgrant> We use parts of its underlying libraries (martian) in lazr.restful to generate the API from decorators.
<wgrant> But it does away with ZCML almost entirely.
<wgrant> repoze.bfg (the original name for pyramid) used to use martian, but I think it's moved to internal stuff.
<blr> huh, interesting.
<cjwatson> wgrant: What do you mean by your comment on https://code.launchpad.net/~cjwatson/launchpad/git-target-inline-default-repo/+merge/258033 ?  Just break up the elif into two parts so that future changes can handle the imported case, or do I need to introduce a repository_type enum?
<wgrant> cjwatson: My comment is missing an "eventually", I think.
<wgrant> I'd just XXX it for now.
<wgrant> Any default git repo now causes the project to state it's using LP officially for code, even when that's a lie.
<cjwatson> OK.  We probably do want to do a repository type thing quite soon, because it'll be needed for recipes.
<cjwatson> And better to do it when it's easy to widen the table.
<wgrant> Why's it needed for recipes?
<cjwatson> Well, not strictly needed, but in practice a lot of them are going to want to be imports from elsewhere.
<wgrant> Ah, sure, we'll want mirrors for recipes.
<wgrant> But that's a bit more than a simple repository_type :)
<cjwatson> Well, sure, but it's the start.
<cjwatson> Also probably not that much more :-)
<cjwatson> At least for git->git
<wgrant> Indeed, it's not hugely difficult.
<cjwatson> Did we agree to just deal with log extraction by opening up rsync from carob, BTW?  I think we did and can deal with that but wanted to check.
<wgrant> I think so, since ELK isn't ready yet.
<wgrant> I wonder if there's an existing rsyncd subordinate.
<cjwatson> basenode does it
<wgrant> Oh, handy.
<cjwatson> And we only need this in prod (and qas for symmetry) anyway
<cjwatson> wgrant,blr: I self-reviewed https://code.launchpad.net/~cjwatson/launchpad/testfix-git-target-inline-default-repo/+merge/258528 to fix buildbot, but it could do with a post-hoc review.
#launchpad-dev 2015-05-08
<bigjools> wgrant: congrats on the preliminary git support
<wgrant> bigjools: Thanks. It's great to finally be there.
<wgrant> We'll be moving lp:launchpad across soonish to dogfood it.
<bigjools> is anyone using it yet?
<wgrant> The kernel team are the first users with it.
<bigjools> figures :)
<wgrant> And they're not complaining about performance at all, so I think we're good.
<bigjools> I think you can claim to be the largest open source Git repo already :)
<danilos> wgrant, the usual problem with git smart server is memory consumption, so if you can take 50 simultaneous git clones of the kernel on slow connections, you are good :)
<wgrant> danilos: Yeah, if someone tries that we're in trouble, but we have a good amount of RAM.
<wgrant> And are working on horizontal scaling.
<danilos> wgrant, cool :)
<danilos> wgrant, (on git.linaro.org, we had that happen with CI or when AOSP and git.kernel.org went down, when everybody wanted to use git.l.o)
 * bigjools waves to danilos
 * danilos waves back to bigjools (went to sleep right after :))
<bigjools> danilos: I tend to do that to people
<danilos> hehe
<StevenK> bigjools: O Ha*snore*
<cjwatson> I think I have GitRef:+activereviews more or less done; still one query count test that's slightly off, but I've found most of the necessary preloading.  Will finish it on Monday.
<qtuser> hello!
<qtuser> I am having trouble putting a project on launchpad, is anyone around to help?
#launchpad-dev 2016-05-09
<sil2100> Hello! I have a question about a possible bug in LP API, maybe I'm just doing something wrong
<sil2100> I'm doing such a thing:
<sil2100> Having a binary package name, I do a getPublishedBinaries() on the archive to get the binary_package_publishing_history and then accessing its source package through getLatestSourcePublication() - all works fine here so far
<sil2100> So I have now the source package that generated the selected binary package
<sil2100> But now, on that source_package_publishing_history when I call getPublishedBinaries() to fetch ALL the binaries it generates (besides the one I queried for), it returns an empty set
<sil2100> No binaries
<sil2100> Even though if I do a query directly for the source package in the archive, I see all the binaries there
<sil2100> Just not when the source_package_publishing_history object is returned through getLatestSourcePublication()
<sil2100> Does anyone know why?
<cjwatson> SPPH.getPublishedBinaries only gives you active binary publications
<cjwatson> So if they've been e.g. superseded it won't return them
<cjwatson> In fact this is documented: https://launchpad.net/+apidoc/devel.html#source_package_publishing_history-getPublishedBinaries
<cjwatson> This is certainly an annoyance.  It might be nice if there were an option to pass active_binaries_only=False down through SPPH.getPublishedBinaries -> PublishingSet.getBinaryPublicationsForSources -> PublishingSet._getSourceBinaryJoinForSources
<sil2100> Ok, yes, but the strange thing is that when I fetch the SPPH for the selected source and do getPublishedBinaries I get all the requested binaries, but when I do the very same thing on the SPPH that's returned by getLatestSourcePublication() I get 0
<cjwatson> Are you sure it's the same SPPH?  It must not be.
<cjwatson> Because getPublishedBinaries doesn't know how you got the SPPH in question.
<cjwatson> Compare .self_link
<sil2100> Here's what I did in an lp-shell session (I cut out the unnecessary bits)
<sil2100> http://paste.ubuntu.com/16316937/
<sil2100> So I make sure I get the SPPH for the source that has the same name and version as the one returned through getLatestSourcePublication()
<sil2100> If they're not the same SPPH then I guess they should? As both queries should lead to the same thing I suppose?
<cjwatson> sil2100: and what's src2[0]?
<sil2100> It's the result of ppa.getPublishedSources() with the same source name and version as the one I got from going through BPPH->SPPH
<sil2100> ppa.getPublishedSources(source_name="account-plugins", version="0.12+15.04.20160126-0ubuntu1", exact_match=True) <- both the name and version are exactly the same as what I got from the SPPH in the earlier steps, the one that had 0 binaries published
<sil2100> And suddenly it has 96 binaries
<cjwatson> sil2100: I'm trying to get you to look at src2[0]
<cjwatson> sil2100: Compare it with src
<sil2100> Oh
<cjwatson> sil2100: The actual link, not its contents
<sil2100> Ooooh
<sil2100> Ah, I get it now
<cjwatson> sil2100: So it matters here whether you're looking at the archive where it was originally built, or the one it's now published in
<sil2100> Damn, the SPPH one is the first build that resulted in the binaries, which is already deleted from the silo PPA
<sil2100> Yeah, all makes sense now
<sil2100> Thanks, this does complicate the logic of my scripts, but at least I know why I was getting these different results
<sil2100> :)
<sinzui> help.launchpad.net is being spammed. Someone also took my edit away so I cannot delete the spam https://help.launchpad.net/Norton%20care%20@@!1844-8O1-3966*!!!!**Norton%20phone%20Number%20uSa%20HELP%20desk%20360%20customer%20care%20number,Norton
#launchpad-dev 2016-05-11
<eliasps> Any Launchpad administrator here to have a look at this when convenient? https://answers.launchpad.net/launchpad/+question/293510 It's about downloading a few mailing list archives and renaming of one team and its mailing list. Thanks.
#launchpad-dev 2016-05-12
<eliasps> Hello, any launchpad administrators here to have a look at this when convenient? https://answers.launchpad.net/launchpad/+question/293510. It's about renaming a team and it's mailing list, and downloading some launchpad teams' mailing list archives. Thank you.
#launchpad-dev 2017-05-11
<cjwatson> RIP buildout-templates
#launchpad-dev 2018-05-08
<cjwatson> lifeless: Would you mind either doing a timeline release (I added py3 support) or granting me PyPI upload access?
#launchpad-dev 2018-05-09
<lifeless> cjwatson: done. May I suggest twitter @'s as a faster way to grab me.
<cjwatson> lifeless: Ah, for some reason I almost never think of those while working.  Will try to remember, thanks.
<cjwatson> I'll push that up tomorrow
<lifeless> I think in this case it would have been equivalent, but I hadn't actually looked at IRC in a week...
<lifeless> so that was chance
<cjwatson> reminds me, I must see if I can find some way to get in touch with James Westby about soupmatchers, since neither an MP nor IRC seem to have worked
<lifeless> he's on steam :)
<cjwatson> hm, list of places it would have occurred to me to bother someone about a package on PyPI: 1) email 2) IRC 3) Twitter 4) IDK maybe G+ or something ... 98) taking out advertisement space on the side of a cow 99) Steam
<cjwatson> but I will consider it, thanks :)
#launchpad-dev 2020-05-04
<tomwardill> zope configs are hard and weird
<cjwatson> What's up?
<tomwardill> trying to work out if you can inherit zope.conf
<tomwardill> doesn't look like you can, so might need to do a bit of shuffling.
<tomwardill> But also trying to work out if I need the settings that I'm trying to create (the <server> ones) at all
<cjwatson> Oh, those things, right
<cjwatson> https://code.launchpad.net/~cjwatson/launchpad-buildd/+git/launchpad-buildd/+merge/383320 (Ensure that regex patterns with \-escapes are raw strings)
<ilasc> cjwatson: MP 383320 looks good to me at first glance, I'm tempted to clone and run unit tests, would u recommend, or would that turn into a time sync ?
<ilasc> *sink
<cjwatson> Feel free
<cjwatson> Though it's not very necessary since Jenkins will do that on merge
<cjwatson> https://code.launchpad.net/~cjwatson/launchpad-buildd/+git/launchpad-buildd/+merge/383341 (Adjust X-LXD-mode header construction for Python 3)
<cjwatson> Also I have a couple of backlogged code-gardening MPs that it would be good if people could look at: https://code.launchpad.net/~cjwatson/launchpad/+git/launchpad/+merge/382875 and https://code.launchpad.net/~cjwatson/launchpad/+git/launchpad/+merge/383046
<pappacena> I'll take a look
<cjwatson> (The latter has a very bulky follow-up MP, but I was planning to self-approve that if/when the prereq lands since it doesn't make sense to have people spend time on >4000 lines of more or less mechanical changes)
<tomwardill> woo, this version works without having zope.app.server installed at all. Although tracelog then stops working, so we might need to have a replacement for that
<cjwatson> tracelog is occasionally handy though I don't know how vital it is.  If I had to trade tracelog for statsd (or similar) I'd certainly take statsd
<tomwardill> might be possible to reintroduce it, atm the easiest way to smoke test stuff that's relying on ZServer functionality is to just uninstall it, but it's a dep for tracelog
<cjwatson> Maybe.  We don't necessarily have to emulate everything
<tomwardill> so tracelog stops working at the same time
<cjwatson> It's also probably a pretty simple piece of middleware
<cjwatson> The page performance report in lp:lp-dev-utils does use the trace files
<cjwatson> (which is visible at the bottom of http://lpqateam.canonical.com/)
<tomwardill> yeah, I think it's safe to re-enable it eventually
<tomwardill> well, less 'eventually' more, almost immeadiately, once I've fixed servers.py to stop importing and using zserver components
<SpecialK|Canon> er
<cjwatson> Right, possibly just reimplemented as simple middleware
<SpecialK|Canon> I don't think I knew that site existed - how might I have discovered that site, because it's neat
<cjwatson> Good question, not sure.  I knew because the deployment report used to be there
<SpecialK|Canon> How/where is this site generated/hosted/etc.?
<cjwatson> I'd search wikis but my ISP is having a wobbly day
<cjwatson> It's on carob, old-school ~lpqateam/public_html
<SpecialK|Canon> Fab, thanks
<cjwatson> Some is hand-rolled HTML, some is generated from bits of lp-dev-utils and similar, I think it's very much symbiosisware
<cjwatson> But handy
<SpecialK|Canon> Hm, not sure it's just you - I couldn't reach wiki.c.c!
<cjwatson> It may well not be just me, I just knew of https://aastatus.net/34840 and there wasn't much point investigating further when DNS was timing out
<SpecialK|Canon> Ah
<SpecialK|Canon> ISP-buddy-five o/
 * SpecialK|Canon prods bits
<cjwatson> https://code.launchpad.net/~cjwatson/launchpad-buildd/+git/launchpad-buildd/+merge/383344 (Treat build logs as binary files)
<tomwardill> cjwatson: +1
<cjwatson> Thanks.  Getting there!
<cjwatson> About 500 lines of actual porting left and then some packaging stuff on top
<SpecialK|Canon> Nice
<cjwatson> pappacena: Your most recent comment on https://code.launchpad.net/~pappacena/launchpad/+git/launchpad/+merge/382779 mentions "with comments", but I don't see them - did you forget to save them?
<pappacena> uhm. I think there was only one comment, about the `if exact_match: return`, in the previously pushed revision.
<cjwatson> Right, I was wondering about whether just removing the 'return rs.one()' was quite correct
<pappacena> This is the comment: https://www.irccloud.com/pastebin/zoiCdnDe/
<cjwatson> But just trying to think through it ...
<pappacena> Strange... the diff comments were actually not saved when I added the other comment. I just saved it now
<cjwatson> And yes, I think you're right
<cjwatson> r=me, just one test suggestion but go ahead after that
<pappacena> Ah, right! Indeed, a good suggestion. I'll do that, and land the MP
<tomwardill> well, that was a bunch of test explosions...
<cjwatson> https://code.launchpad.net/~cjwatson/launchpad-buildd/+git/launchpad-buildd/+merge/383345 (Treat build output files as binary files)
 * pappacena reviewing
<cjwatson> https://code.launchpad.net/~cjwatson/launchpad-buildd/+git/launchpad-buildd/+merge/383349 (Treat intltool-related files as binary files)
<tomwardill> Total: 47 tests, 0 failures, 2 errors, 0 skipped in 3.310 seconds. (from test_servers)
<tomwardill> that... went better than I expected
<cjwatson> pappacena: Couple of tiny things on https://code.launchpad.net/~pappacena/turnip/+git/turnip/+merge/380451 (we should really get going on isort or equivalent ...) but otherwise good to land
 * tomwardill launches a full test run
<cjwatson> How long do they take for you?
<tomwardill> not sure, not measured one for a while
<tomwardill> although it just segfaulted...
<tomwardill> (test:760044): Gtk-CRITICAL **: gtk_settings_get_for_screen: assertion 'GDK_IS_SCREEN (scree
<tomwardill> n)' failed
<tomwardill> ah, forgot to run xvfb
<pappacena> cjwatson, any known issue on Jenkins / Turnip test suite? Jenkins is failing some tests running git commands, and I cannot reproduce it locally.
<cjwatson> pappacena: I don't recognise that one, and it's in code you changed, so hmm
<cjwatson> pappacena: Ah, I see, look for the "Please tell me who you are" errors.  The test container doesn't have a git identity configured
<pappacena> Yes, it is. I'll try to debug it. Sad that those tests passes locally (both on python2 and python3)
<pappacena> ahhm
<pappacena> Strange that it started failing now...
<cjwatson> pappacena: Because your change switches to using git tag rather than pygit2 to create the tag
<cjwatson> So rules for the git command line apply
<pappacena> Ah, yes! Right! I forgot about this change
<cjwatson> pappacena: I think perhaps it'd be simplest to have RepoFactory.add_tag add '-c', 'author.name={}'.format(self.author.name), '-c', 'author.email={}'.format(self.author.email), '-c', 'committer.name={}'.format(self.committer.name), '-c', 'committer.email={}'.format(self.committer.email)
<cjwatson> Since it's just that one command at the moment, and could be generalised later if we start using the git command line for more things in the test suite
<pappacena> I was checking the code, and got to the same conclusion. I'll just put those extra parameters in another method, to make it reusable when needed...
<cjwatson> Sounds good, thanks
<pappacena> Thanks for the help!
<cjwatson> It's a shame that the test runner there doesn't gather the stdout/stderr with the ERROR: output
<cjwatson> Since then it would've been a lot more obvious
<cjwatson> I suppose that's what we get for just using unittest discover though
<pappacena> Let me see if I can do something about it too.
<cjwatson> Don't feel obliged, but it would be a good opportunity :)
<pappacena> cjwatson: quick review (for tomorrow?) https://code.launchpad.net/~pappacena/turnip/+git/turnip/+merge/380451
<pappacena> Added some logging when the command fails...
<cjwatson> pappacena: Looks OK if it works, go ahead.  (Though I normally prefer self.addDetail for this sort of thing - compare turnip.pack.tests.test_functional)
#launchpad-dev 2020-05-05
<tomwardill> sigh test run failed and my terminal crashed
<tomwardill> that's... useful
<tomwardill> oh no, i ran it in byobu!
<tomwardill> past me is sensible
<tomwardill> okay, missing python-keystone client
<tomwardill> I think I've fixed all the dependencies now
<tomwardill> cjwatson: Total: 21271 tests, 2 failures, 19 errors, 2 skipped in 177 minutes 15.970 seconds.
<tomwardill> full (non-parallel) test run
<cjwatson> OK, about twice as fast as my laptop then
#launchpad-dev 2020-05-06
<cjwatson> https://code.launchpad.net/~cjwatson/launchpad-buildd/+activereviews now includes everything needed to get the test suite passing with PYTHON=python3 (the py3-* branches; there are also some older unrelated ones at the start).  After that the plan is to adjust the packaging to use python2 on xenial and python3 on bionic.  I'd appreciate reviews of what's there so far
<tomwardill> multi-arch docker is hard
<tomwardill> I think I'm not going to worry about it
<tomwardill> cjwatson: mostly reviewd
<cjwatson> Thanks
<cjwatson> Anyone feel like reviewing https://code.launchpad.net/~cjwatson/launchpad/+git/launchpad/+merge/383158 as well?  That disables new submissions to the hardware database; I've checked that the certification teams no longer use that.
<jelmer> Still looking for a review on https://code.launchpad.net/~jelmer/loggerhead/breezy-compat-1/+merge/377809 if somebody has a spare moment :)
<cjwatson> Ugh, sorry, looking now
<cjwatson> Merged
<jelmer> Thanks!
#launchpad-dev 2020-05-07
<tomwardill> as handy as it is, the weird string concatination 'feature' of python  catches me out more often than not
<tomwardill> https://code.launchpad.net/~twom/launchpad-buildd/include-base-in-digests/+merge/383584 extract the base from OCI images.
<cjwatson> tomwardill: Could you have a look at https://code.launchpad.net/~cjwatson/launchpad-buildd/+git/launchpad-buildd/+merge/383594 ?  It sort of partially reverts one of your OCI-related changes from a while back, which I think doesn't cause any problems there but you should probably have a look
<tomwardill> yep, looking
<tomwardill> `CarefulFakeProcessFixture`
<tomwardill> this is a good name
<cjwatson> :)
<tomwardill> cjwatson, pappacena: Tidied up https://code.launchpad.net/~twom/launchpad-buildd/include-base-in-digests/+merge/383584 and found some other bits that were doing silly things with exceptions, if you could check it over again
<cjwatson> tomwardill: re-reviewed
<tomwardill> ta
<tomwardill> cjwatson: the KeyError is actually the same cause as the LXDException
<tomwardill> it's the error the FackBackend raises on a missing file
<cjwatson> Oh right
<cjwatson> Still, my point was that the exceptions raised by copy_out vs. the exceptions raised by everything else are logically different
<tomwardill> yeah, fair, that block should be shorter
<tomwardill> all fixed
<teward> just going to share things here, but RE: DMARC compliance I'm still going to iron out the kinks locally once LP catches up to what I'm after (a branch in my userspace for the LP code) but it looks like DMARC compliance for bug mails might be easier to implement than previously thought heh.  fixing up the tests will be the next Big Problem for that but from the functionality perspective it shouldn't be too hard.
<teward> thank you cjwatson for pointers at where to look in the codebase
<teward> once i get that working you'll all ahve a merge to review for those changes :)
<cjwatson> Trying to work out why your fork is timing out - LP has four refs so it really shouldn't be an issue
<cjwatson> Guess I'll try repacking the repository
<teward> cjwatson: unless something else is going wonky?
<cjwatson> No, it's clearly an API timeout on the repository create call
<teward> cool.  more work for you then cjwatson xD  :P
<cjwatson> teward: Can you retry your push now?
<teward> 1 moment
<teward> didn't time out and is 'writing objects' right now
<teward> (*waits for the eternity to pass*)
<teward> i'm assuming that the dev launchpad env attempts to use localhost:25 for email sending?
<teward> (just checking)
<cjwatson> teward: An improperly configured Launchpad instance could very easily spam a whole bunch of innocent people, so by default it's either configured not to send email at all or to drop them into an mbox somewhere, I forget which.  See doc/email.txt
<teward> Yep.  I also configure a postfix locally that literally just delivers to an mbox xD  So far though i caught two things that needed altered in the tests :P
<teward> You also werent lying the test suite takes an eternity to run
<cjwatson> teward: To give you an idea, the dogfood instance has a mail-configure-normal.zcml that looks like https://paste.ubuntu.com/p/dq5P36X95D/, which just drops everything into an mbox
<cjwatson> You probably don't want to run the entire test suite.
<cjwatson> lp.bugs should be enough - maybe lp.coop.answersbugs too, though it's tiny.
<teward> running the bugs testsuite with the command you stated in #launchpad earlier.  Which is fine 'cause I want to make sure that works :)
<teward> yeah i am running lp.bugs :)
<cjwatson> The whole thing takes six hours on my laptop.  Fortunately most changes don't require running it all
<teward> indeed.  but because I want to make sure nothing with my minor changes in the LP codebase explode all of bugs I"m running lp.bugs entirely.  Found a docs discrepancy too because of the email From generations being different, that was an easy thing to fix, but i'm waiting to see what else triggers
<teward> 'course APache just exploded but that i think might've been my fault (oops)
<teward> (like literally seized up the container)
<teward> i only can blame myself there :)
<teward> cjwatson: i think one of the tests is just 'busted' to some extent
<teward> but it might be on my end... *retests with a clean setup*
<cjwatson> teward: We have some flaky tests, but they do all pass
<cjwatson> On a correct setup
<cjwatson> And most of the flakiness doesn't manifest when you're just running the tests serially locally rather than in more complicated parallel setups
<teward> the flakiness i'm seeing is in... lib/lp/bugs/doc/bugnotification-sending.txt
<teward> where Subject lines with ... fail because it's got an actual numeric in there
<teward> ellipsized text not matching
<teward> similar for link behavior
<teward> and the From addresses but it *seems* to be more a case of the docs not matching the expected outputs (hence failures)
<teward> i'm quadruple tasking at the moment heh
<teward> let me run a 'pure' test with LP pure just to make sure this isn't my code changes causing the errors (if it reproduces in clean master I'm assuming the issue is due to flakyTest)
<wgrant> That test is not known to be flaky, and it's not of the kind that is likely to be flaky.
<teward> wgrant: true, but i'm curious why it's triggering here but not elsewhere hence testing a clean deploy atm
<teward> to rule out my changes (which were only to the from address generator) as the cause
<teward> (i'm testing local heh)
<teward> wgrant: the *expected* failures with those changes are, as I stated, expected, but the specific ones that *aren't* expected are with handling of the Subject line matches
<teward> let me get the traceback into a file that I can share (and then delete after)
<cjwatson> Beware that doctest mismatch output can be extremely confusing and can suggest mismatches that don't exist if you aren't completely sure of how to read them
<teward> cjwatson: yeah those're the tests that failed.  This said, those're the only failures in the entire bugs test suite
<teward> (with my changes).  Trying to get full output but E:SLOW
#launchpad-dev 2020-05-08
<teward> cjwatson: this is the test output for the DMARC-changed code.  There was early tests also which were also doctests erroring because the from addresses did actually change and didn't match - but when those were revised the remaining errors are the only ones I see failing AFAICT.  https://people.ubuntu.com/~teward/lpdev-dmarc.txt
<teward> (Chrome exploded when I tried to paste it into a pastebin)
<teward> failed errors are towards the end
<wgrant> teward: That looks fine
<wgrant>     - Subject: [Bug ...] subject
<wgrant>     ?               ^^^
<wgrant>     + Subject: [Bug 18] subject
<wgrant>     ?               ^^
<mup> Bug #18: RFE: I'd like to be CC:d automatically to bugs I report <feature> <iso-testing> <lp-bugs> <Launchpad itself:Fix Released by bradb> <https://launchpad.net/bugs/18>
<cjwatson> teward: Yep, as noted, this is doctest output being surprising to the uninitiated
<wgrant> That's the full diff, not necessarily the diff that caused the failure.
<wgrant> It doesn't include whitespace normalisation, and it doesn't include ellipsis.
<teward> ð thanks cjwatson
<teward> the only other failure cases are the From addresses
<teward> which *are* what changed in my diff
<cjwatson> You should find that if you fix just the From: bits then the rest will match
<teward> that's the tricky part in *that* doctest
<teward> unless it's always using 18
<cjwatson> <...@bugs.launchpad.net>
<teward> ack
<cjwatson> That's what ellipses are for
<teward> those doctest errors are confuzling to me :)
<wgrant> Sometimes I wonder if that was one of the design goals
<wgrant> They can be remarkably obtuse.
<teward> that i don't doubt xD
<teward> but ODDLY no other bug notification tests errored heh
<cjwatson> That is a little remarkable to me, but OK
<cjwatson> I guess we don't *have* to have a bazillion tests that interact with everything.  Usually there's more fallout from a change like this though
<teward> cjwatson: indeed.  yeah the lack of other things exploding here was also something that made me headscratch.  Unless we don't *actually* test bug delivery?
<cjwatson> Well, it evidently is being tested in that doctest
<teward> yep
<cjwatson> Unexpectedly light coverage, but we are actually testing it
<cjwatson> Albeit slightly in passing
<cjwatson> Though, I don't understand how your changes didn't cause bugnotification-email.txt to fail
<teward> it did
<teward> i fixed those
<cjwatson> Oh, I see
<teward> those were the straightforward fixes
<cjwatson> OK, good
<cjwatson> That's where get_bugmail_from_address is unit-tested (albeit in a ridiculous format, but they're conceptually unit tests)
<teward> yeah and those (as expected) failed
<teward> my git tree has two documentation test fixes - one for the latest errors (Ellipses are evil) and one with the earlier two from address mismatches (which were very easy fixes by replacing the email addresses)
<cjwatson> And hopefully the surrounding prose commentary
<teward> wanted to get the tests passing before I rewrite the commentary :)
<teward> literally just filed a short time ago - https://code.launchpad.net/~teward/launchpad/+git/launchpad/+merge/383643 - but this should handle the DMARC compliant bug mail issue.  The documentation needs vetted because I basically rewrote a whole portion of it.  Kept all four test cases for From addresses, but the function is much less verbose and quite straightforward - I kept the test cases to show behavior.
<teward> tests pass here, which was... surprising heh.
<teward> if this goes into deployment however it would help having a nice document posted somewhere about the change as people might start headscratching when things change majorly.
<teward> but that's a given :(
<teward> :) *
<cjwatson> Will look at it next week, thanks (public holiday here today)
<teward> yep just putting it on the radar :)
<teward> (wish I had a public holiday today... I don't want to be doing Work from Home today - could use an extra 4 hours sleep xD)
<teward> cjwatson: do you know where Merge Proposal emails are constructed in the codebase?
<teward> i see some in docs but I need to find the constructor (they're also DMARC-noncompliant)
<cjwatson> teward: Somewhere in lib/lp/code/mail/
<teward> thanks i'll dig.  that's going to be the next thing I start poking at.  What's the testsuite for just merge proposals?  (or merges) - lp.code ?
<teward> (instead of lp.bugs like before)
<cjwatson> Yes.  The -t option takes a regex match applied to test case IDs - you can be more specific if you like, and I'd probably be inclined to do something like -t merge.\*proposal -t code.\*review
<cjwatson> Should be rather faster
<teward> +1
<teward> i'll dig there next, but i'm thinking a more phased approach to DMARC implementation - make sure there's no regressions in one part before implementing the rest.  Unless we want to one-and-done it all but that has more regression risks.
<cjwatson> We try to name tests at least somewhat regularly so that things like that have a better chance of matching
<cjwatson> Doing this in pieces is absolutely correct and appropriate
<cjwatson> No need to rush it
<teward> yep
<teward> i'm bored again though right now so :P  Need something to pass time :P
