[00:14] <wgrant> Can someone please throw https://code.launchpad.net/~wgrant/launchpad/bug-522800-getFileByName/+merge/43159 at EC2? Julian ran it overnight with one failure, which I've fixed.
[00:56] <LPCIBot> Project devel build (286): STILL FAILING in 3 hr 12 min: https://hudson.wedontsleep.org/job/devel/286/
[00:56] <LPCIBot> Launchpad Patch Queue Manager: [bug=620458,
[00:56] <LPCIBot> 629804] [r=gmb][ui=none][bug=629804] Back out hack to serve
[00:56] <LPCIBot> restricted files directly from the internal restricted librarian port.
[02:12] <lifeless> [02:12] <lifeless>     Hard / Soft  Page ID
[02:12] <lifeless>       68 / 4938  Archive:+index
[02:12] <lifeless>       44 /    0  Person:EntryResource:retractTeamMembership
[02:12] <lifeless>       42 /  190  BugTask:+index
[02:12] <lifeless>       37 /  307  Distribution:+bugs
[02:12] <lifeless>       24 /  263  POFile:+translate
[02:12] <lifeless>       14 /   18  DistroSeries:+queue
[02:12] <lifeless>       11 /  367  Distribution:+bugtarget-portlet-bugfilters-stats
[02:12] <lifeless>       11 /   39  Milestone:+index
[02:12] <lifeless>        9 /   79  ProjectGroupSet:CollectionResource:#project_groups
[02:12] <lifeless>        9 /   33  RootObject:+login
[02:30] <wgrant> lifeless: Hmm, that's very odd.
[02:59] <poolie> i hope that's a bot not really robert :)
[03:01] <mwhudson> you only think there's a difference
[03:02] <poolie> :)
[03:03] <lifeless> should I feel complemented or insulted
[03:03] <lifeless> wgrant: whats odd abou tit
[03:03] <wgrant> lifeless: So few Archive:+index timeouts.
[03:04] <lifeless> wgrant: we've had the odd 'good' day
[03:04] <lifeless> wgrant: the soft timeouts are pretty consistent - many K
[03:04] <wgrant> lifeless: True.
[03:04]  * wgrant still needs someone to EC2 that regression-fix branch.
[03:10] <lifeless> is it all gtg? commit message set etc etc etc
[03:13] <poolie> does "number of affected users" now total up across dupes?
[08:39] <adeuring> good morning
[09:19] <mrevell> Hola
[09:33] <wgrant> ... and the award for the revision with the most reviewers goes to r10047.
[11:29] <bigjools> jml: hi, I'll be ready for our call in 5 minutes
[11:30] <jml> bigjools: np
[11:30] <jml> bigjools: I'm on the strategerie on mumble
[11:30] <bigjools> jml: ok, I only just got back from my hospital appt.  I now need drugs and coffee.
[11:30] <jml> bigjools: understood.
[12:11] <deryck> Morning, all.
[12:16] <LPCIBot> Yippie, build fixed!
[12:16] <LPCIBot> Project devel build (287): FIXED in 3 hr 36 min: https://hudson.wedontsleep.org/job/devel/287/
[14:15]  * beuno waves at deryck 
[14:15] <beuno> are but attachments on private bugs, private?
[14:15] <beuno> IIRC, there is such a thing as a private librarian
[14:17] <deryck> hi beuno.  was on call, sorry.
[14:17] <deryck> beuno, they are indeed truly private now.  but that's a source of great pain at the moment. ;) :)
[14:18] <beuno> deryck, pain for whom?  :)
[14:18] <deryck> beuno, private attachments aren't available over the api at the moment.  accept for an exception made for apport and the retracers.
[14:18] <deryck> except, rather
[14:18] <beuno> deryck, ah, gotcha
[14:19] <beuno> I don't think we need to get them through the api
[14:19] <deryck> beuno, btw, I'm very supportive of the cdn idea and getting a yui loader going. willing to help rockstar as he works on it.
[14:19] <deryck> beuno, we need it desperately on lp
[14:20] <beuno> deryck, awesome, I'm going to push for something canoni-wide if it gets traction, otherwise build something for u1 and slowly push it out for other teams
[14:21] <deryck> beuno, yeah, we'll definitely take it up.  the 1.3Mb js file is killing us now.
[14:21] <beuno> deryck, still carrying mochikit?
[14:23] <deryck> beuno, yes, we are.  but I don't think it adds much.  yui itself is 1Mb.  lazr-js is the other largest bit.
[14:23] <beuno> deryck, so, FWIW, I manually build a YUI combo file and brought it down to ~500kb
[14:24] <beuno> instead of bringing in all of the -min files in YUI
[14:24] <deryck> beuno, I had a little play at that and didn't have much luck.  but I could be including something we really don't need.
[14:24] <deryck> I'm working on this over the next several weeks, so I look at that again.
[14:24] <beuno> deryck, well, what I did, is use their online combo generator
[14:25] <beuno> http://yui.yahooapis.com/combo?3.2.0/build/yui/yui-min.js&3.2.0/build/oop/oop-min.js&3.2.0/build/event-custom/event-custom-min.js&3.2.0/build/attribute/attribute-min.js&3.2.0/build/pluginhost/pluginhost-min.js&3.2.0/build/base/base-min.js&3.2.0/build/classnamemanager/classnamemanager-min.js&3.2.0/build/dom/dom-min.js&3.2.0/build/dom/dom-style-ie-min.js&3.2.0/build/event/event-min.js&3.2.0/build/node/node-min.js&3.2.0/build/widget/widget-min.j
[14:25] <beuno> oooooops!
[14:25] <beuno> sorry
[14:25] <beuno> that covered 95% of yui
[14:26] <beuno> 476kb
[14:26] <beuno> so I use that as the YUI file, and tack on the rest of our js
[14:26] <beuno> I have *no* idea why it differs so much from compressing it from the tree
[14:28] <beuno> you can play with it here: http://developer.yahoo.com/yui/3/configurator/
[14:30] <rockstar> deryck, could you pipe in on the list about the CDN as well then?
[14:33] <beuno> rockstar, I was thinking we could also just proxy yui's cdn
[14:33] <beuno> with https
[14:34] <rockstar> beuno, maybe, but then we can't get to lazr-js or our own deps.
[14:34] <beuno> right
[14:36] <deryck> rockstar, yeah, I was planning to.
[14:43] <EdwinGrubbs> deryck: how do I attach a bug to a series as opposed to a milestone in the UI?
[14:43] <deryck> EdwinGrubbs, target to release link.
[14:43] <sinzui> help. I created a new error type and provides a useful message for users, but the page shows the errors docstring. the TALES is just "structure error"
[14:43] <EdwinGrubbs> thanks
[14:43] <deryck> EdwinGrubbs, np
[15:58] <jcsackett> salgado: ping.
[16:00]  * deryck is getting excited about the BugJam next week
[16:01] <Ursinha> abentley_, hi
[16:01] <Ursinha> or abentley, hi
[16:01] <abentley> Ursinha, hi.
[16:01] <sinzui> jml, mumble?
[16:02] <jml> sinzui: yes
[16:02] <Ursinha> abentley, about bug 688092, do you have an example of how qa-tagger failed when you tried running that locally?
[16:02] <_mup_> Bug #688092: It's too hard to start hacking on qa-tagger <qa-tagger:New> <https://launchpad.net/bugs/688092>
[16:03] <abentley> Ursinha, not offhand, but I can get one for you.
[16:03] <Ursinha> thanks abentley, that would be useful
[16:03] <Ursinha> when you create something it's hard to identify things you should explain to others
[16:04] <abentley> Ursinha, sure.  I've been there.
[16:06] <salgado> hi jcsackett
[16:06] <jcsackett> hi salgado. i'm working on the distribution mirror prober, and it looks like you've been the busiest in it.
[16:07] <jcsackett> i'm wondering if you can help me figure out some issues i've introduced in trying to fix a bug.
[16:07] <jcsackett> bug 681034
[16:07] <_mup_> Bug #681034: UnknownURLScheme raised running distributionmirror-prober script <oops> <Launchpad Registry:In Progress by jcsackett> <https://launchpad.net/bugs/681034>
[16:07] <salgado> jcsackett, sure, I can try.  although I haven't touched it in a while.  can it be in 15'?
[16:08] <jcsackett> salgado: fifteen works just fine. gives me time to get stuff together so i can ask better questions. :-)
[16:08] <salgado> cool. :)
[16:21] <pcjc2> hen running a local launchpad instance (for bug import testing), I want to wipe it and start again. make schema has wiped most things, but I can't run the bug import script a second time - it claims the bugs have already been imported
[16:21] <pcjc2> http://launchpad.dev:58080/93/sB5xaCAHZcRjyaqsZrDH2sV7pCj.txt (the bug has already been imported)
[16:23] <salgado> jcsackett, I'm ready
[16:23] <jcsackett> salgado, awesome.
[16:23] <jcsackett> so, the problem in the bug is that on the machine being probed, there is a redirect, which we can't really do anything about. but it's causing an oops because unsupported url's trigger an exception.
[16:24] <jcsackett> i'm trying to change the prober to simply log the bad redirect so a human can look at it, and move on with triggering the oops.
[16:24] <jcsackett> salgado: this was my naive attempt: https://pastebin.canonical.com/40796/
[16:24] <jcsackett> the problem is that the prober system is more complex than i expected, and i can't seem to find a way to do this that escapes the redirect method without having a timeout across the tests for the prober. :-(
[16:25] <jcsackett> so: short question: do you have any ideas of how to tackle this? sinzui suggested refactoring the prober to test redirection before actually doing it. i tried that as well, and simply caused further timeouts.
[16:25] <salgado> jcsackett, can I see the diff as well?
[16:25] <jcsackett> ah, yes, one sec salgado.
[16:26] <jelmer> deryck: ^
[16:27] <deryck> pcjc2, hey, there is a .pickle file somewhere in the tree you need to remove.  I think it's called bugmap.pickle.
[16:27] <deryck> jelmer, thanks! :-)
[16:27] <pcjc2> ok, cool
[16:28] <pcjc2> bug-map.pickle
[16:28] <pcjc2> thanks so much
[16:28] <deryck> bzr st should turn it up.
[16:28] <deryck> right, that's it
[16:28] <jcsackett> salgado: https://pastebin.canonical.com/40800/
[16:28] <pcjc2> need to ask some questions at some point about the possibility of munging inline "bug 1423312" from the old SF bug reports to match new LP numbers
[16:29] <pcjc2> (Would require changes of code on the LP import scripts), or possibly just have the format converter pick up on "Bug 412312", and turn it into a string which won't get auto-linked to a random LP bug)
[16:29] <_mup_> Bug #412312: Implement Songbird support <daemon> <songbird> <Panflute:Fix Released by kuliniew> <https://launchpad.net/bugs/412312>
[16:29] <jcsackett> salgado: my sense tells me the issue i'm running into is the callback stuff and the interaction between the prober and all the twisted machinery behind it. but that may just be suspicion born of ignorance.
[16:31] <deryck> pcjc2, you'll have to get any changes to the importer back upstream to us, since we'll actually run the import.  Or sed your export file to change the form of the old link.
[16:31] <deryck> or old text, rather
[16:31] <pcjc2> I understand
[16:32] <pcjc2> Any changes which require mapping to new LP bug numbers will have to be within the import scripts run on the LP production servers. I'm just playing locally here to tweak the SF -> LP conversion as best I can without creating too much work for the LP admins
[16:34] <salgado> jcsackett, I haven't touched this code in a while, but I suspect the timeout is because upon an UnknownURLScheme exception you don't do anything that cancels the delayed call that triggers the timeout (e.g. connect() or fail())
[16:35] <salgado> jcsackett, can you do a self._cancelTimeout(None) before your logger.error() call?
[16:36] <jcsackett> salgado: i think i tried that, but let me give it another go.
[16:39] <salgado> also, self.redirection_count will always be greater than 0 there
[16:39] <salgado> so you don't need the if/else
[16:41] <jcsackett> salgado: good to know. the tests, however, still show a timeout error.
[16:42] <salgado> jcsackett, ok, have you pushed your changes?  that way I can merge and see what the failure looks like
[16:42] <jcsackett> salgado: sure, i'll push it up now.
[16:47] <jcsackett> salgado: https://code.launchpad.net/~jcsackett/launchpad/redirects-681034
[16:47] <jcsackett> salgado: the test in question is the test in question is test_redirectawareprober_fail_on_unknown_scheme; right now that asserts that an unknown schema url exception is raised, and instead a timeout is raised. win condition is that fails because NO exception is raised. then i will figure out testing the logging.
[16:49] <salgado> jcsackett, I think I know what's going on
[16:50] <jcsackett> salgado: lay it on me. do not spare my ego. :-P
[16:50] <salgado> unlike I previously thought, just cancelling the timeout is not enough
[16:50] <salgado> you have to callback() on the deferred (or errback())
[16:51] <salgado> so you do have to fail(e) on UnknownURLScheme
[16:51] <salgado> and on a layer above that you can catch the exception and not raise an OOPS
[16:51] <jcsackett> hm.
[16:52] <salgado> jcsackett, but we already have support for that.  see ArchiveMirrorProberCallbacks
[16:52] <salgado> it does a failure.trap(*self.expected_failures)
[16:52] <jcsackett> salgado: ah yes, i see.
[16:53] <salgado> at first I thought you could add UnknownURLScheme to expected_failures
[16:53] <salgado> but that's not a good idea
[16:53] <jcsackett> no, i would need to create a new excpetion.
[16:53] <salgado> because I think we want an OOPS raised when that exception is raised on the first connect() but not when it's raised in a redirect()
[16:54] <salgado> so, yes, a new exception which you pass on to self.fail() on redirect()
[16:54] <jcsackett> salgado: it doesn't look like ArchiveMirror uses the RedirectAware prober...the only thing using redirect is MirrorCDImageProberCallbacks
[16:54] <jcsackett> salgado: that doesn't seem right to me, does it make sense to you?
[16:55] <salgado> that is right
[16:55] <salgado>         # According to http://lists.debian.org/deity/2001/10/msg00046.html,
[16:55] <salgado>         # apt intentionally handles only '200 OK' responses, so we do the
[16:55] <salgado>         # same here.
[16:56] <salgado> I think that has changed recently, but it was intentional that the archive prober doesn't use the redirect-aware factory
[16:56] <salgado> s/that/apt
[16:57] <jcsackett> salgado: ah. so then the assumption in the bug about it being redirect isn't necessarily accurate, because the RedirectAware prober wouldn't have been involved.
[16:57] <salgado> it was not an archive mirror that caused the OOPS
[16:57] <salgado> https://server4.bremer-it.de/ubuntu///maverick/ubuntu-10.10-server-i386.iso
[16:57] <salgado> that's a CD image mirror
[16:58] <salgado> and the traceback shows it was on redirect()
[16:58]  * jcsackett didn't understand what CD image mirror was about. i get it now.
[16:59] <jcsackett> salgado: okay, so basically i can crib from ArchiveMirror to handle expected errors, and add that to CDImageMirror stuff.
[16:59] <salgado> oh, of course.  now I see how I mislead you
[16:59] <salgado> yes, that's correct
[17:00] <salgado> MirrorCDImageProberCallbacks is the one you want to change
[17:00]  * jcsackett nods.
[17:00] <jcsackett> salgado: thanks so much!
[17:01] <salgado> jcsackett, don't thank me before confirming it actually works. ;)
[17:01] <jcsackett> salgado: fair. :-)
[17:01] <jml> flacoste: ping
[17:01] <flacoste> hi flacoste
[17:02]  * flacoste talks to himself
[17:02] <flacoste> jml: hi
[17:02] <jml> flacoste: mumble!
[17:19] <EdwinGrubbs> benji: do you know if there is a way to override devmode without changing the launchpad.conf or creating a new config directory specified by the LPCONFIG env var?
[17:20] <benji> EdwinGrubbs: hmm, let me see
[17:33] <benji> EdwinGrubbs: I don't see a way.  We could fairly easily add command-line config overrides to bin/run if needed.
[17:34] <EdwinGrubbs> benji: do you want me to open a bug for that?
[17:35] <benji> EdwinGrubbs: if it's something you need, sure!
[18:06] <jml> g'bye all. See you again on the 20th.
[18:08] <EdwinGrubbs> deryck: ping
[18:11] <deryck> hi EdwinGrubbs
[18:15] <EdwinGrubbs> deryck: I'm working on the milestone +index page for distros, projects, and project groups. These pages list the bugs assigned to the milestone that do not have a conjoined master. For project groups, it's possible that a bug is on a milestone one of the projects and on the devfocus series for another project? Would this count as having a conjoined master? The old code did not count it, but my optimization started doing it as a
[18:15] <EdwinGrubbs> side effect.
[18:16]  * deryck is reading, processing that....
[18:16] <deryck> EdwinGrubbs, I honestly don't know if that counts as a conjoined master.  It shouldn't as I understand conjoined bugtasks, but there may be a bug there.
[18:17] <deryck> EdwinGrubbs, or do you mean "is it okay if they're conjoined after this work?"
[18:18] <EdwinGrubbs> deryck: I really don't understand the workflow around this to know why we don't want bugs attached to the devfocus series to show up on the milestone page. yes, I mean, "is it ok?"
[18:18] <salgado> jcsackett, did it work?
[18:19] <jcsackett> salgado: seems to. i'm refactoring some code to properly trap things and cleanup some other details, but this seems to be the right path.
[18:19] <salgado> cool!
[18:19] <jcsackett> sorry i didn't ping you earlier; got wrapped up in pursuit of the solution. :-)
[18:19] <jcsackett> thanks again!
[18:20] <salgado> np; glad it worked
[18:20] <deryck> EdwinGrubbs, I don't know why we wouldn't want it either.  I think the bug should appear if a task has linked the milestone....
[18:21] <deryck> EdwinGrubbs, as to your issue, are the two projects on different tasks on the same bug?
[18:21] <EdwinGrubbs> deryck: yes, so the bug would show up on the milestone page for one of the projects but not for the other one.
[18:22] <deryck> EdwinGrubbs, ah, that's fine with me.
[18:24] <EdwinGrubbs> deryck: so, you don't think anyone will complain that milestone X on one of the projects has the bug, but milestone X on the project group doesn't because a differ project has it linked to the devfocus series?
[18:25] <deryck> EdwinGrubbs, ah, right.  yes, that would be wrong.  hard to get my head around it.
[18:26] <EdwinGrubbs> deryck: ok, that's fine.
[18:28]  * deryck will brb, rebooting
[23:26] <flacoste> mbarnett: still around?