[00:28] <james_w> jml: https://code.edge.launchpad.net/~james-w/launchpad/fix-publishing-getbyid/+merge/28359
[07:12] <adeuring> ood moring
[07:28] <lifeless> I wonder
[07:29] <lifeless> how much work would be involved to make 'mark x as dup of y' move all of x's dups to be dups of Y
[07:29] <lifeless> its awefully manual at the moment
[07:29] <poolie> wbn
[07:30] <poolie> even just when i try to make x a dupe of y and y is already a dupe of z
[07:30] <poolie> it could just do it
[07:30] <poolie> rather than giving me an ugly error
[07:30] <poolie> lifeless: oh also i was contemplating a command to attach a local file to a bug
[07:30] <poolie> do you know of anything at the moment?
[07:32] <wgrant> lifeless: I think deryck implemented that recently.
[07:32] <lifeless> no, that would be nice. Bughugger probably has a gui one.
[07:32]  * wgrant hunts.
[07:33] <lifeless> wgrant: wabbits?
[07:33] <wgrant> https://code.edge.launchpad.net/~deryck/launchpad/do-the-right-thing-dupe-move-78596/+merge/27144
[07:33] <wgrant> Not merged yet.
[07:33] <StevenK> poolie: apport hooks probably do that
[07:33] <lifeless> StevenK: no, they do something quite different.
[07:33] <lifeless> StevenK: it uploads to a staging area, gets a magic prefix, etc
[07:34] <StevenK> Ah
[07:34] <poolie> StevenK I want: lsusb | lp-attach 123123 --name lsusb.txt
[07:35] <lifeless> oh
[07:35] <lifeless> there is an apport thing to *add* to a bug
[07:35] <lifeless> under the ubuntu-bug banner
[07:35] <wgrant> apport-collect?
[07:35] <lifeless> ubuntu-bug -u bug
[07:44] <poolie> mm i want something with less policy
[07:44] <poolie> i may try one sometime
[07:45] <lifeless> bug.addAttachment
[07:45] <lifeless> its in the API
[08:16] <poolie> i know
[08:16] <poolie> but thanks
[08:16] <poolie> i anticipate it should be a pretty small bit of code
[08:17] <lifeless> hmm
[08:17] <lifeless> I don't like the question spam
[08:17] <lifeless> when there is a linked bug
[08:17] <poolie> not my fault :)
[08:17] <poolie> or is it?
[08:17] <lifeless> no
[08:17] <lifeless> well maybe you linked two things
[08:17] <lifeless> but it appears to be that questions mails me directly
[08:18] <lifeless> when a bug status attached to a Question is altered.
[08:18] <poolie> ah yes
[08:18] <lifeless> and I'm subscribed indirectly to the bug.
[08:18] <poolie> that bugs me too
[08:18] <poolie> also it's not obvious why the mail is being sent
[08:18] <lifeless> I'm enbugginating
[08:19] <lifeless> and.... bug dup suggestions are timing out again
[08:21] <lifeless> https://bugs.edge.launchpad.net/launchpad/+bug/597981
[08:21] <_mup_> Bug #597981: launchpad tells me twice about changes to bugs that are linked to questions <Launchpad itself:New> <https://launchpad.net/bugs/597981>
[08:29] <lifeless> poolie: also https://bugs.edge.launchpad.net/launchpad/+bug/597982
[08:29] <_mup_> Bug #597982: emails about 'questions' have many different subject lines <Launchpad itself:New> <https://launchpad.net/bugs/597982>
[08:52] <poolie> cute
[08:53] <poolie> if  you create an attachment and don't specify the mime type
[08:53] <poolie> launchpad assumes it is chemical/x-mopac-input
[08:53] <poolie> i think that's pretty reasonable, don't you? :)
[08:56] <lifeless> rotfl
[08:56] <lifeless> thats awesome
[08:56] <poolie> i guess it's alphabetically first or something
[09:00] <poolie> might be behind bug 204560
[09:00] <_mup_> Bug #204560: Incorrect MIME type when uploading an audio/x-musepack (*.mpc) file <Launchpad Bugs:Triaged> <https://launchpad.net/bugs/204560>
[09:03] <mrevell> Guten morgen
[10:58] <jml> lifeless, wrt to your recent comment on the factory... I wonder how much an LP factory could be broken up.
[10:59] <lifeless> jml: I'm doing some experiments in the space
[10:59] <lifeless> ask me at the epic, I hope to have better answers
[10:59] <lifeless> and I may have had opportunity to look at the LP factory by then.
[10:59] <lifeless> jml: trivially it could be made into a Facade
[10:59] <lifeless> or given .foo.bar subfactories
[11:04] <deryck> Morning, all.
[11:59] <deryck> jml, I'm reviewing your branch to add search tasks modified_since parameter... I'm curious why you wrapped the factory makeProduct call in IBugTarget.  We usually just pass the made product straight to searchTasks.
[12:03] <jml> deryck, because the test doesn't care about it being a product, I guess.
[12:03] <jml> deryck, it just wants a BugTarget and doesn't really care about the implementation
[12:03] <jml> deryck, wrapping means that the test reads the way I think about it.
[12:05] <deryck> jml, ok, fair enough.
[12:07] <deryck> jml, and I'm curious too about your addition of the login method, rather than passing something like user='test@canonical.com' to the setUp of the test.  Is this to completely avoid sample data?
[12:07] <jml> deryck, basically, yes.
[12:07] <jml> deryck, I hate login with a particular fixed email string. I can never remember what privileges it's supposed to have.
[12:08] <jml> deryck, again, the idea is that the code communicates "login in as someone, anyone, it doesn't matter"
[12:08] <jml> deryck, because that's actually what we want, right?
[12:08] <lifeless> \o/
[12:08] <lifeless> clarity, its a shocker
[12:09] <deryck> jml, yup, that's fine.  I think the branch fine as is, then.  r=me.  I'll add some comments to the MP for posterity's sake.
[12:09] <jml> deryck, thanks.
[12:18] <didrocks> jml: FYI, I fw you the email sent to the ML about LP and Quickly (hope you will receive it this time :))
[12:18] <jml> didrocks, I just forwarded it to launchpad-dev
[12:18] <jml> didrocks, that's definitely the first time I've seen that email.
[12:18] <didrocks> jml: ok, what's wrong with the adress I used? I got no email about rejection or anything else
[12:19] <didrocks> jml: and my @ubuntu.com adress is registered on LP, I thought that you can send to a LP ML even not being member of the team
[12:19] <jml> didrocks, I don't know.
[12:20] <didrocks> jml: weird… if you receive any answer, can you check that I'm in CC :)
[12:20] <jml> didrocks, ahh, fwiw, your original emails were in the moderation queue
[12:20] <didrocks> jml: oh ok, so make sense now :-)
[12:21] <didrocks> thanks jml
[12:21] <jml> I don't know who gets those emails.
[12:21] <jml> as in, who is told about pending moderation.
[12:21] <didrocks> maybe nobody, hence the pending issue :-)
[12:53] <Mez> trying to setup a local LP install on lucid - cant start up postgrsql again
[12:54] <Mez> "Invalid data directory"
[13:00] <maxb> Mez: That's really not enough info to enable anyone to help you
[13:00] <Mez> maxb: No worries, I'm cleaning up, getting rid of postgres, and reinstalling ;)
[13:01] <Mez> trying a manual DB setup, rather than the automated one
[13:01] <Mez> seems postgresql8.4 had it's /etc stuff wiped somehow
[13:02] <Mez> ah, does LP support postgres 8.4 ?
[13:04] <maxb> Yes
[13:04] <Mez> it's not in the docs ;)
[13:05] <Mez> (which just mentions 8.3, leafing me to ask the Q)
[13:05] <maxb> Although, launchpad-database-dependencies won't install all the supplementary packages for 8.4, so you'll have to do it yourself
[13:06] <Mez> like?
[13:06] <maxb> postgresql-plpython-8.4 postgresql-contrib-8.4
[13:07] <Mez> that may have been where it went wrong then
[13:17] <Mez> ok, now I've got my DB running, however - make schema doesnt work
[13:17] <Mez> http://pastebin.com/wNmNYmti
[13:23] <maxb> Mez: 'make SHHH= schema' to turn off the silly output hiding thing so the output is more informative
[13:25] <Mez> maxb: pretty much the same
[13:25] <Mez> http://pastebin.com/q3JSfNWR
[13:25] <Mez> I removed the download-cache and reran rocketfuel-get
[13:52] <henninge> Hi adiroiban ! ;)
[13:53] <Mez> maxb: any siggestions ?
[13:54] <Mez> ah, get a copy from the python site, that works
[14:03] <maxb> oh, is a file missing from download-cache?
[14:05] <maxb> Mez: I have a dist/setuptools-0.6c9-py2.6.egg in my download-cache
[14:14] <Mez> maxb: no, the file was there, but the file wasnt matching it's MD5 sum
[14:14] <maxb> huh. weird
[14:14] <Mez> I re-downloaded it directly from the site, and it seems to have fixed it.
[14:14] <Mez> *sigh*
[14:14] <Mez> now even more issues
[14:15] <Mez> no build-essential (gcc)
[14:21] <james_w> so "ec2 land" is running all the tests, and they all pass, but it either isn't doing anything, or the mails aren't getting out
[14:22] <jml> james_w, hmm.
[14:22] <jml> james_w, you've double checked spam filters I suppose
[14:23] <james_w> The code seems to be hiding the request to send to pqm if successful, but seeing as I don't get a success email I'm suspecting the latter
[14:25] <jml> hmm.
[14:25] <jml> james_w, actually, I don't understand that last sentence of yours.
[14:26] <james_w> I'm reading the code to try and work out what it is trying to do, but I can't see where "cmd_land" sets things up such that a request will be made to pqm
[14:27] <jml> james_w, it generates a command line for 'ec2 test' and then runs that.
[14:27] <james_w> yes
[14:28] <jml> james_w, the "-s" option in _get_landing_command
[14:28] <jml> which is submit_pqm_message
[14:28] <james_w> ah
[14:28] <james_w> thanks
[14:28] <james_w> it was hiding :-)
[14:29] <jml> np
[14:29] <james_w> ah
[14:29] <jml> ultimately it all gets passed as options to remote.py
[14:29] <james_w> I changed my bzr whoami to have my linaro address, I wonder if that's it
[14:29] <jml> which is what actually does the interesting work
[14:29] <jml> james_w, I bet you that is it.
[17:27] <rockstar> sinzui, do you have some time for mumble really quick?  I need a UI sanity check.
[17:27] <sinzui> I do
[17:28] <rockstar> sinzui, what channel are you in?
[17:29] <sinzui> i do not know?
[17:32]  * rockstar thinks pre-imp calls with sinzui are awesome because they are so short.
[17:35] <sinzui> rockstar, that is odd. I talk to much
[17:35] <sinzui> I have no social life
[17:35] <sinzui> bac and EdwinGrubbs can attest that I do not know when to shut up
[17:38] <rockstar> sinzui, that may be teamlead-itis.  thumper has the same disease
[17:39] <sinzui> :)
[17:49] <james_w> what was the name of the effort to restructure launchpad in to the lp.app layout we have now? Was it called the apocalypse?
[18:02] <jml> james_w, yes.
[18:03] <james_w> thanks
[18:03] <jml> james_w, progress is charted here: https://lpstats.canonical.com/graphs/CodeBaseFileCount/20090101/20100625/
[18:10] <james_w> jml: can you confirm I have the right idea on: https://dev.launchpad.net/APPocalypse
[18:10] <jml> james_w, punned!
[18:10] <james_w> salgado's idea
[18:11] <james_w> I was thinking branchpocalypse
[18:11] <jml> james_w, that seems the right direction
[18:11] <jml> james_w, the way I'd do it is split things by service/process, rather than by app.
[18:11] <jml> at least to start with.
[18:12] <jml> james
[18:12] <james_w> jml: example?
[18:12] <jml> james_w, buildmaster, codehosting-ssh, etc.
[18:12] <james_w> ok
[18:12] <james_w> I think starting with lp.whotsits would be good
[18:13] <jml> james_w, yeah, but the ones of those I'd start with would be the ones that correspond to near-standalone services
[18:13] <james_w> I don't want us to go too far beyond the already established lines, as we only need to split a certain amount for our aims.
[18:13] <jml> james_w, also, I have a .dot file for you.
[18:13] <james_w> right
[18:14] <james_w> jml: is that the one that will break my eyes, if not my poor little laptop?
[18:14] <jml> james_w, yes. http://people.canonical.com/~jml/lp-clustered.dot
[18:15] <james_w> merci
[18:16] <jml> james_w, no mercy.
[19:54] <lifeless> jml: poolie said something, and I may be misremembering the details /completely/ about a resolution at a recent previous epic
[19:54] <lifeless> jml: to squash the timeout oops vigorously
[19:55] <jml> lifeless, there was, but it died.
[19:55] <lifeless> oh
[19:55] <jml> lifeless, and it was subtly different.
[19:55] <lifeless> is there a mail trace about that? I would like to read it
[19:56] <jml> lifeless, depends on which initiative you mean
[19:56] <jml> lifeless, there was one to reduce / eliminate timeouts
[19:56] <jml> lifeless, there was another to reduce / eliminate all oopses.
[19:59] <lifeless> ok
[19:59] <lifeless> do you know what killed it ?
[19:59] <lifeless> tools/size-of-work+time-allocated/motivation/playing-mallet-on-the-metric ?
[20:00] <jml> it dissipated, basically.
[20:01] <jml> lifeless, I'll grep my email to see if there was a closure email, but probably your best bet is to talk w/ flacoste.
[20:01] <lifeless> whose awl today
[20:01] <lifeless> ok
[20:03] <jml> lifeless, nope, no email
[20:03] <jml> gotta go. dinner time.
[20:03] <lifeless> thanks, ciao
[20:28] <lifeless> hi bac
[20:29] <bac> hi lifeless
[20:45] <lifeless> bac: did my subunit/testr notes make sense to you?
[20:45] <lifeless> bac: and if they didn't can I debug that; if they did, did they work for you? [and if not...]
[20:45] <bac> lifeless: yes it did, thanks.
[20:45] <bac> lifeless: i haven't actually tried it yet, though
[20:49] <bac> lifeless: trying now...
[20:49] <lifeless> brb
[20:53] <lifeless> mmm, in the court of the crimson king
[20:53] <lifeless> rhythmbox plays this to me first, every time :P
[20:54] <lifeless> bac: we could add an import command
[20:54] <lifeless> testr import attachment.gz
[20:54] <lifeless> which would:
[20:54] <lifeless>  - sniff the file type
[20:54] <lifeless>  - 'do the right thing'
[20:54] <lifeless> but I'd like to keep the core building blocks really dumb
[20:55] <bac> lifeless: that would be nice...but the reason i raised the issue was really an appeal for human readable summary of failures in the email message.  re-running them is secondary.
[20:56] <lifeless> bac: right, I addressed that too
[20:56] <bac> lifeless: and i think the second part of your email recipe does that
[20:56] <bac> lifeless: yep
[20:56] <bac> so many thanks on both counts
[20:56] <lifeless> for instance, if I grab a pqm failure from the bzr project
[20:57] <lifeless> we have the make output
[20:57] <lifeless> and the tests
[20:58] <lifeless> its all quite customisable
[20:58] <lifeless> https://pastebin.canonical.com/33967/
[20:58] <lifeless> is the end of a recent failure mail
[20:58] <lifeless> it uses a very similar bit of code to what I supplied
[22:53] <sinzui> lifeless, ping
[22:57] <lifeless> hi
[22:58] <lifeless> sinzui: ^
[22:59] <sinzui> lifeless, the milestone has been a real problem for me for a year because of timeouts. There are many enhancements I cannot consider until I can make that page reliably load...
[23:00] <lifeless> sinzui: yes, I'm just curious is all
[23:00] <sinzui> I can see in a timeout that 4.5 seconds was sql time, everything else was python maybe
[23:00] <lifeless> sinzui: I meant absolutely no impediment to your landing it
[23:00] <sinzui> We have had several developers look into it.
[23:00] <sinzui> No, I am hoping your brains can help me really solve the problem.
[23:01] <lifeless> sinzui: I'm really sorry if I wasn't clear about that, I was asking totally from curiosity :)
[23:01] <lifeless> oh, ok cool
[23:01] <lifeless> uhm
[23:01] <lifeless> did you happen to get an lsprof trace of a typical render from a laptop ?
[23:02] <lifeless> if its mainly python that sort of thing can be really enlightening
[23:02] <sinzui> I discovered recently the the diversity of information seem to be a greater factor than number of bugs/blueprints. lots of private object or private secondary objects lost the page down, to does having 100+ assignees and a diversity of statues
[23:03] <sinzui> lifeless, I think edwin or bac looked into lsproof. but we could not produce the issue in dev until I wrote a script that makes the diversity I saw in ubuntu timeouts
[23:04] <lifeless> so that raises a couple of background tasks - get more data on oops to better reproduce (but gary/stub are working on that already I believe, which is great)
[23:04] <lifeless> I think I'd approach it thusly:
[23:05] <lifeless> which ever is greater, sql/python; focus on that (you've said python)
[23:05] <lifeless> for the thing being focused on, get a profile trace of a page that would have timed out.
[23:05] <lifeless> that is - on staging or something
[23:05] <lifeless> * turn off timeouts
[23:05] <lifeless> * install a profile middleware
[23:06] <lifeless> * run the page
[23:06] <lifeless> turning off the timeouts is useful, because that way we can see the total places tha time goes
[23:06] <lifeless> otherwise we'd be optimising on only (say) 20 seconds out of 40.
[23:06] <lifeless> then, locally, as you have done already, setup something to trigger badnesses
[23:07] <lifeless> optimise that locally until (at minimum) you've reduced the local time by the same fraction needed to make the total time the profile took be acceptable
[23:07] <lifeless> e.g.
[23:07] <sinzui> staging was setup with lsprof. I need to see if that is true since we changed packaging and python versions
[23:08] <lifeless> if its 16 seconds in python now to timeout, and 32 seconds in python to profile, and we want to make the python time give a total page render < 20 seconds, we need to take 1/2 the time it does
[23:08] <lifeless> iterate on that locally
[23:10] <lifeless> sinzui: does this add anything to what you where doing?
[23:10] <lifeless> sinzui: I don't have any immediate stuff beyond 'gather some more data' though, I'm sorry :(
[23:10] <sinzui> yes. It is a sound outline to find the root cause(s)
[23:11] <lifeless> you know the ++oops++ magic decorator thingy
[23:11] <lifeless> ^ technical term
[23:11] <sinzui> I do
[23:11] <lifeless> what might be really nice
[23:11] <lifeless> is a magical ++profile++ decorator enabled only on staging (and for non-devs it turns itself off once the user is determined)
[23:12] <mars> ooo
[23:12] <lifeless> it could write the profile to a directory that gets rsynced to devpad
[23:12] <mars> +1
[23:12] <lifeless> and put the filename in the timing block at the end of the page
[23:12] <lifeless> so you could just do this
[23:12] <lifeless> wait a couple of mintes
[23:12] <lifeless> and have a kcachegrind ready analysis
[23:13] <sinzui> thanks
[23:13] <lifeless> (I can show you the exact functions in bzrlib to call to use its lsprof glue; though we may want a tweaked version or a lock around the profiling to kick other threads off, or something)
[23:14] <lifeless> mars: launchpad-foundations would be a good place to record that idea, yes ?
[23:14] <mars> lifeless, yep!
[23:14] <lifeless> I shall file a bug
[23:14] <mars> lifeless, did you get kcachegrind/lsprof to work for bzrlib?
[23:14] <lifeless> mars: years ago :)
[23:14] <mars> My kcachegrind bin/test patch causes a cachegrind memory explosion
[23:15] <lifeless> mars: run this: bzr --lsprof-file foo.callgrind st
[23:15] <lifeless> mars: kcachegrind foo.callgrind
[23:15] <mars> I wonder if you run something different than I do
[23:15] <lifeless> oh
[23:15] <lifeless> for tests we do it per-test
[23:15] <mars> I just grabed lsprof from... somewhere
[23:15] <lifeless> not for the whole test run
[23:15] <mars> ah, ok
[23:15] <lifeless> the whole test run is *freaking huge*
[23:15] <mars> I tried the whole run.
[23:15] <lifeless> GB's of arcs
[23:16] <mars> ... well that would explain a few things...
[23:16] <lifeless> and thats in bzr which is considerable leaner than lp test-time wise
[23:16] <lifeless> mars: have a look in bzrlib.lsprof
[23:16] <lifeless> mars: it has shinies
[23:16] <mars> So my patch is sane then
[23:16] <mars> where'd it go...
[23:16] <lifeless> mars: (use it directly, we do fix it)
[23:17] <lifeless> I quite like the chromium speed-tracer
[23:17] <mars> lifeless, the zope testrunner takes a --profiler= argument.  I came up with a monkey-patch for zope.testing that enables one to cachegrind
[23:17] <lifeless> mars: how does it compare to the performance tools you've been working with and posting about
[23:17] <mars> speed-tracer - is that the one with the zooming timeline across the top?
[23:18] <lifeless> mars: yeah, built into the browser
[23:18] <lifeless> per-item timelines etc
[23:18] <lifeless> things like
[23:18] <lifeless> Request Timing	@21093ms for 18897ms
[23:18] <lifeless> Response Timing	@39991ms for 661ms
[23:18] <lifeless> Total Timing	@21093ms for 19559ms
[23:19] <lifeless> (which is for the html in https://edge.launchpad.net/bzr btw)
[23:19] <mars> From the tutorial it looks like they may have improved it
[23:20] <mars> they used to not distinguish between DOM events
[23:20] <mars> I really wanted to know "How much blocking was JS execution, and how much was waiting for the load event"
[23:20] <mars> And it was near impossible to answer that
[23:21] <mars> So I could not (before) answer a question like "the page has a one second gap between visible and the UJS being active - why?"
[23:22] <mars> What I really want to see is this: http://developer.android.com/guide/developing/tools/traceview.html
[23:23] <mars> with the whole zooming timelines and such
[23:23] <mars> hey, they've improved it!
[23:24] <mars> they actually documented the file format - that means you could write a Python traceview profile formatter
[23:24] <lifeless> ok, bug https://bugs.edge.launchpad.net/launchpad-foundations/+bug/598289 filed
[23:24] <_mup_> Bug #598289: support a profile decorator for use in staging and development environments <Launchpad Foundations:New> <https://launchpad.net/bugs/598289>
[23:25] <lifeless> I've captured what we discussed in brief, I think.
[23:25] <mars> Now I just have to find some sleep I can sacrifice to get 100 points of time.  Where to look...
[23:26] <mars> lifeless, looks good
[23:27] <mars> lifeless, would be nice for the epic.  Heck, any profiling would be nice for the epic.
[23:27] <lifeless> we could do this before the epic
[23:27] <lifeless> so that its available
[23:43] <lifeless> mars: what do you think?
[23:53] <mars> lifeless, I have no idea if that is possible
[23:54] <mars> sorry
[23:54] <mars> and I'm already working on another performance tool for the epic