[01:46] <StevenK> wallyworld__, wgrant: https://code.launchpad.net/~stevenk/launchpad/fix-branch-subscribe-open-team/+merge/111326
[01:47] <wallyworld__> StevenK: otp, can look soon
[02:05] <wallyworld_> StevenK: r=me
[02:05] <wallyworld_> StevenK: i assume the existing tests look for the field error
[02:14] <StevenK> wallyworld_: Yeah, that was added in branch-subscribe-aag
[02:15] <wallyworld_> cool
[02:28] <wgrant> cjwatson: I'll run gina with commits enabled in a sec to revert the DB archive to the state we have on disk.
[02:39] <wgrant> Oh
[02:39] <wgrant> gina might be even slower than I thought it was.
[02:39]  * wgrant fixes.
[02:40] <wgrant> Ah, no.
[02:42] <wgrant>    ->  Seq Scan on distributionjob  (cost=0.00..3711.70 rows=1 width=70) (actual time=69.665..129.033 rows=4 loops=1)
[02:42] <wgrant>          Filter: ((job_type = 3) AND (distroseries = 107) AND (json_data = '{"sourcepackagename": 72909, "parent_series": 50}'::text))
[02:42] <wgrant> That seems... fragile
[02:42] <wgrant> StevenK: wtf
[02:42] <wgrant> ?
[02:42]  * wgrant blames bigjools too.
[02:43] <wgrant> Do either of you recall why we do that, and what will break when I make it stop doing that?
[02:43] <StevenK> Context?
[02:43] <bigjools> errr someone is filtering on json_data
[02:43] <bigjools> ?
[02:43] <wgrant> DSDJ creation seems to first check if there are any existing jobs
[02:44] <wgrant> By byte-matching the JSON
[02:44] <bigjools> sweet fuck
[02:44] <StevenK> Twitch, there are columns for thatr
[02:44] <StevenK> s/r$//
[02:44] <wgrant> There aren't columns for that.
[02:44] <wgrant> But it's also probably not necessary.
[02:44] <wgrant> And it's slow and fragile.
[02:44] <wgrant> So I think I might delete it and see what happens.
[02:44] <bigjools> who is bzr blaming?
[02:45] <bigjools> do not delete
[02:45] <wgrant> No idea, I haven't found the code yet.
[02:45] <bigjools> check the commit msg please
[02:46] <wgrant> You ruin all my fun.
[02:46] <bigjools> why the heck is lp-propose pushing a target branch if it doesn't exist?!
[02:46] <bigjools> wgrant: your delete and revert reflex is well known
[02:48] <wgrant> Oh, sneaky, DSDJ is in Soyuz while DSD itself is in Registry.
[02:48] <StevenK> I think it was jtv
[02:48] <StevenK> find_waiting_jobs in DSDJ's model
[02:48] <wgrant>     # Look for identical pending jobs.  This compares directly on
[02:48] <wgrant>     # the metadata string.  It's fragile, but this is only an
[02:48] <wgrant>     # optimization.  It's not actually disastrous to create
[02:48] <wgrant>     # redundant jobs occasionally.
[02:48] <wgrant> Yeah
[02:48] <wgrant> So it can be deleted without peril.
[02:49] <StevenK> I can recall being a little dubious when that code landed.
[02:54] <jtv> StevenK, wgrant: yes that was me.  Feel free to improve it.
[02:54] <spm> o/ jtv
[02:54] <jtv> Hi there spm!
[02:56] <jtv> It's just a filter for curbing duplication of jobs.  The thinking was that with 1 item in the JSON, it's a pretty reliable match; with 2 items in the JSON, there's only going to be at most 2 jobs even if the keys are in random order; and with 3 items in the JSON, it's time to actually think what should be done.
[02:58] <wgrant> Heh, true.
[03:04] <jtv> If the filter is too costly, it can be removed — but first I'd assess the risk of duplicating DSDJs.  That was part of the work I was hoping to avoid with this simple check.
[03:05] <jtv> Then again, 50ms doesn't seem like a huge cost.
[03:05] <jtv> An index could be another way to go.
[03:06] <lifeless> well
[03:06] <lifeless> its racy
[03:06] <lifeless> unless we up our isolation level
[03:06] <wgrant> Only slightly, but yes.
[03:06] <wgrant> Upping the isolation doesn't help, I don't think
[03:07] <wgrant> Oh, during the running I guess it would, indeed.
[03:10] <lifeless> yah
[03:11] <lifeless> it would make read-its-there-don't-add-commit conflict.
[03:18] <lifeless> unless of couse a specific DSDJ is idempotent
[03:18] <lifeless> in which case, the race isn't important, as it cannot be stale, but OTOH a duplicate should be approx free because the worker can check whether the output is there already.
[03:21] <wgrant> The jobs are fairly expensive due to librarian access.
[03:21] <wgrant> But it's otherwise not a problem for them to be duplicated.
[03:25] <lifeless> can they not determine that they completely finished and shortcut (assuming idempotency)
[03:29] <wgrant> Indeed, they probably could avoid recalculating the base version if neither series' version has changed.
[04:42] <czajkowski> morning
[05:17] <czajkowski> jtv: hiya, heading offline for a few days, left message in pm
[05:17] <czajkowski> see you all Monday
[06:12] <cjwatson> wgrant: thanks for QAing that, I wasn't looking forward to it
[06:13] <wgrant> It took about 3 hours for the first run, IIRC.
[06:13] <wgrant> Subsequent runs take less than a minute.
[07:52] <adeuring> food morning
[09:34] <ivory> jelmer: could you review this? https://code.launchpad.net/~ivo-kracht/launchpad/bug-897571/+merge/111367
[10:04] <jelmer> ivory: sure
[10:06] <jam> wgrant: are you still around
[10:07] <jam> hi jelmer
[10:12] <jelmer> hellÃ¸ :)
[11:10] <jml> is there an http equiv of 'bzr branch bzr+ssh://bazaar.launchpad.net/+branch/pkgme' (i.e. a fixed string that I can put before a project name to fetch its trunk?)
[11:11] <jml> I don't want 'lp:' because it generates warnings about launchpad-login and this is an automated process.
[11:13] <rick_h_> morning
[11:14] <rick_h_> jml, so what I've done is branch with the lp: and bzr info to get the full url for that project
[11:14] <rick_h_> is it for one or many projects?
[11:14] <jml> rick_h_: thanks. that's a bit of a shame, because I'd rather depend on the slightly more stable lp:foo url equivalent.
[11:16] <rick_h_> jml: it could be there for sure, I'm just so far from a bzr expert to say for sure. abently will be around in a couple of hours and he'd be able to tell for sure
[11:16] <jml> rick_h_: many. thanks.
[11:33] <mgz> jml: nope, there's no simple mapping, you need xmlrpc
[11:37] <mgz> don't do this but...
[11:37] <mgz> >>> from bzrlib.plugins.launchpad import lp_directory
[11:37] <mgz> >>> lp_directory.get_lp_login = lambda: None
[11:37] <mgz> >>> lp_directory.LaunchpadDirectory().look_up("er...", "lp:bzr")
[11:37] <mgz> 'http://bazaar.launchpad.net/~bzr-pqm/bzr/bzr.dev'
[11:41]  * jml would rather patch LP to have a proper rewrite map for http
[11:42] <jml> but there are only so many patches to LP that I can justify.
[11:42] <mgz> would be nice, wouldn't it. xmlrpc is an arse.
[11:54] <jelmer> doesn't just http://launchpad.net/PROJECT work?
[11:54] <jelmer> it seems to here
[11:54] <jelmer> jml: ^
[14:13] <jam> dpm: how's the translation stuff coming?
[14:16] <jam> jtv, dpm: from what I can see the summary statistics pages between P and Q are virtually identical. So it looks like all the async jobs have done their work, and got Q into pretty good shape.
[14:16] <jam> guess jtv isn't here now.
[14:26] <jcsackett> is anyone else getting sporadically failing javascript errors on ec2? I keep getting hatemail with a ton of failures but when i run all YUI tests locally they all pass.
[14:33]  * jcsackett realizes it's thursday.
[14:42] <rick_h_> jcsackett: been that way all week. Was so sad when the wife corrected me today that it's not friday
[14:44] <dpm> jam, I'll talk to the ubuntu-translations-coordinators team, and unless they see anything, I'm aiming to open translations tomorrow
[14:49] <sinzui> jcsackett, can you review https://code.launchpad.net/~sinzui/launchpad/bug-tags-edit-0/+merge/111410
[14:49] <jcsackett> sinzui: just opened it, actually. :-)
[14:50] <frankban> abentley: ping
[14:50] <abentley> frankban: pong
[14:51] <frankban> abentley: I am having troubles trying to fix a test isolation bug; if 3 tests are run in a predefined order I get a db DisconnectionError
[14:52] <frankban> abentley: these are the 3 tests: http://pastebin.ubuntu.com/1052752/
[14:53] <abentley> frankban: Okay, is that the order they must be run in?
[14:53] <frankban> and the last raises "DisconnectionError: terminating connection due to administrator command. SSL connection has been closed unexpectedly."
[14:53] <frankban> abentley: yes
[14:53] <abentley> frankban: Can you post the traceback, please?
[14:54] <jcsackett> sinzui: r=me.
[14:54] <sinzui> faboo
[14:55] <jcsackett> sinzui: so, given that a) i'm not that far in the new team form work, b) wallyworld_ seems to have all of that *well* in hand, and c) there's a critical in our next lane that's been sitting for awhile, i'm thinking of dropping my curent work and doing the critical.
[14:55] <jcsackett> thoughts?
[14:56] <sinzui> jcsackett. I do want it worked on...but do we know how to solve it at this moment?
[14:56] <jcsackett> sinzui: i *think* so. want to chat about it?
[14:57] <sinzui> We want to unsubscribe users that project does not share with, and are not in the bug-supervisor or security contact roles, but not break ubuntu
[14:57] <jcsackett> right, but we had to put a flag in for the ubuntu case anyway. which means we can structure what happens around that.
[14:58] <sinzui> jcsackett,  when the ubuntu community make a bug public, they expect all the subscribers to remain because they and Lp do not know who should have access.
[14:58] <frankban> abentley: http://pastebin.ubuntu.com/1052761/
[14:59] <jcsackett> sinzui: hrm; fair. i had not thought about the "return to public" part.
[14:59] <sinzui> jcsackett, so solving the issue for everyone except ubuntu fixes 99% of the problem
[15:00] <sinzui> jcsackett, community might toggle between public and private by accident. The ubuntu bug supervisor does not get subscribed (we know that for certain) so we just have to hope the community can mange the subscriptions themselves
[15:00] <abentley> frankban: So, that exception is actually being raised on the Celery side, I believe.
[15:01] <jcsackett> sinzui: so, on switch to a private type, do the remove all subscriptions except special roles thing, and just hope everyone sorts it out when it switches back?
[15:01] <frankban> abentley: I agree, it is raised by a transaction.commit called by pop_remote_notifications
[15:01] <abentley> frankban: Which probably indicates a bug in the Celery stuff.
[15:01] <sinzui> jcsackett, not for ubuntu. their supervisor is never subscribed
[15:02] <jcsackett> sinzui: right.
[15:02] <jcsackett> but otherwise.
[15:02] <sinzui> jcsackett, lets just have a guard for ubuntu...no unsubscribing. They continue in the compromised fashion
[15:02] <jcsackett> sinzui: ok. so you're good with me tackling that and dropping the (now unecessary) new team form work?
[15:03] <sinzui> every other project can be secure because Lp will unsubscribe everyone who is not in the legacy role or the info type is not shared
[15:03] <sinzui> jcsackett, yep
[15:03] <abentley> frankban: So, the Celery tasks do some hacky stuff to switch database users.  I'd assume that's the cause of the problem.
[15:03] <jcsackett> cool stuff. :-)
[15:03] <abentley> frankban: Are multiple tests doing pop_remote_notifications?
[15:05] <sinzui> jcsackett, If my qa goes well today, I could return to bug tag manager. It requires a lot of markup that it could generate as progressive enhancement...this would be an incremental step to adding tag completion to +filebug and search which do not have the cumbersome markup.
[15:05] <abentley> frankban: Also, you know that you can get the celeryd-side traceback from the result object, right?
[15:06] <frankban> each test in that TestCase. and all the tests in that testcase call switch_dbuser(config.branchscanner.dbuser). But if I replace the last test with another one in the same TestCase everything is ok.
[15:06] <frankban> abentley: yes
[15:07] <jcsackett> sinzui: that would be awesome.
[15:07] <jcsackett> now, i just have to figure out why js tests are failing on ec2 but passing locally for me.
[15:09] <sinzui> jcsackett, what really?
[15:11] <sinzui> jcsackett, lucid uses an older version of webkit that knows that ids are unique. It return null if you try to locate it. The webkit on out computers returns the first one
[15:29] <jcsackett> sinzui: so, the require-id flag thing maybe the issue?
[15:31] <sinzui> jcsackett, I am not sure. Recall we stopped the line a few months ago when wallyworld___ had a branch that always failed on lucid. The failure was legitimate because the page was creating bad ids, and the js only worked with the first item in a listing
[15:32] <jcsackett> sinzui: i do recall that; but these are modules i haven't touched and that don't rely on pickers, so i feel like something else might be happening than the new picker stuff generating bad ids.
[15:32] <sinzui> I saw the same problem in workitems a few weeks ago too
[15:32] <sinzui> oh, which modules
[15:34] <jcsackett> sinzui: http://pastebin.ubuntu.com/1052825/. all with JS timeout.
[15:34] <jcsackett> can't get it to happen locally though. :-/
[15:34] <sinzui> whoa
[15:34] <jcsackett> i've sent of ec2 test -o '--layer=YUI' to see if it's a "whole suite" vs "yui suite" issue.
[15:35] <sinzui> jcsackett, I am very surprised. gary_poster added incremental reporting to html5-browser that also resolved timeout issues where tests grew too large.
[15:36] <jcsackett> sinzui: forget everything, i just figured it out, and i am an idiot.
[15:36] <gary_poster> :-)
[15:36] <sinzui> yui_xhr are slow even on my computer.
[15:36] <gary_poster> yeah
[15:36] <gary_poster> they are
[15:37] <jcsackett> when i resent out the branch, not everything was pushed--it tested the old version, which had syntax issues.
[15:37] <jcsackett> so, if this ec2 test run comes back green i'll lp-land, since it's all just YUI changes.
[15:37] <sinzui> well done.
[15:37] <jcsackett> yeah. i am smooth.
[15:38] <jcsackett> note to self: always double check revid.
[15:38] <gary_poster> sinzui, Easter Division?
[15:38] <sinzui> jcsackett, I still use ec2 test and bzr pqm-submit in my working tree...they always abort if the revs are different
[15:39] <jcsackett> sinzui: yeah. i'm not actually sure why ec2 land doesn't.
[15:39] <jcsackett> perhaps that's something to change.
[15:39] <cjwatson> Huh, I thought it did.  Doesn't it claim that it does?
[15:39] <cjwatson> Certainly seems surprising for it not to
[15:39] <sinzui> gary_poster, In the Five Star Stories manga, the heroes are members of the Easter division. I think it is a Japanese corruption on East
[15:39] <gary_poster> it uses pqm-submits code to do that sort of thing
[15:40] <gary_poster> :-) cool sinzui
[15:40] <gary_poster> oh
[15:40] <gary_poster> and ec2 land is just a wrapper around ec2 test
[15:41] <gary_poster> at least that's what I saw when I looked at it last :-P
[15:41] <sinzui> but you can run ec2 land from any directory. I think its goal it to land the approved version, not the current version
[15:42] <gary_poster> oh
[15:42] <gary_poster> maybe it changed then
[15:46] <frankban> abentley: here is the traceback attached to the job: http://pastebin.ubuntu.com/1052839/
[15:49] <abentley> frankban: So my guess is that it's committing using the old connection.  That seems to be the only way past operations could have an influence.
[15:52] <frankban> abentley: I'll investigate it, thanks for the hint.
[15:56] <replaceafill> hello everybody, i'm new to launchpad development, but i'd like to contribute fixing trivial bugs
[15:56] <jelmer_> hi replaceafill
[15:56] <replaceafill> i set up my environment with rocketfuel
[15:56] <replaceafill> hi jelmer_
[15:57] <replaceafill> and chose LP: #210821
[15:57] <_mup_> Bug #210821: bug tracker list shows inactive projects <404> <bugwatch> <lp-bugs> <oops> <trivial> <Launchpad itself:Triaged> < https://launchpad.net/bugs/210821 >
[15:57] <jelmer_> replaceafill: great! How is it going?
[15:57] <replaceafill> jelmer_, http://pastebin.ubuntu.com/1052858/
[15:57] <replaceafill> that's my approach
[15:58] <replaceafill> but i have a few doubts
[15:58] <replaceafill> 1. it changes part of a model
[15:58] <replaceafill> i'm not sure if it's better doing it at the "view level"
[15:58] <replaceafill> (assuming using .active checks is ok)
[15:59] <replaceafill> btw, sorry for my english, is not my first language :)
[15:59] <jelmer_> replaceafill: using .active seems reasonable to me
[15:59] <replaceafill> jelmer_, ah ok
[15:59] <sinzui> replaceafill, that wont fix the bug
[15:59] <sinzui> replaceafill, projects can be toggled from active to inactive, then back to active
[16:00] <sinzui> It is okay for an inactive project to have a remote bug tracker
[16:00] <replaceafill> sinzui, i tried it in launchpad.dev, marking the gnome terminal project as inactive, and it gets filtered in the bugtrackers view
[16:00] <replaceafill> sinzui, ah
[16:00] <jelmer_> sinzui: this is just about what projects are listed in the remote bug tracker's info page I think
[16:01] <jelmer_> sinzui: wouldn't it make sense in that case to hide those projects that are inactive?
[16:01] <sinzui> replaceafill, the problem is that queries to get the bugtacker/bugtask queries often forget to add Product.active == True
[16:02] <sinzui> jelmer_, we do want to hide deactivate projects. That is either a query or a .active check in code iterating over a dumb result set.
[16:03] <jelmer_> sinzui: isn't that the code replaceafill is changing though?
[16:08] <sinzui> okay. I am mistaken. That works, but it is iterating over unused data. Both queries in the preceding lines could be fixed to make LP faster: http://pastebin.ubuntu.com/1052872/
[16:09] <sinzui> jelmer_, replaceafill ^ is less work for Lp
[16:09] <replaceafill> ah, i had the sense that it could be fixed that way :)
[16:09] <replaceafill> but i don't know enough yet to change queries :(
[16:09] <jelmer_> sinzui: that makes sense
[16:09] <jelmer_> (does that really need raw SQL, btw?)
[16:12] <sinzui> jelmer_, it should be storm, but this is very old code.
[16:13] <replaceafill> sinzui, could i help writing a test for your solution?
[16:14] <replaceafill> or maybe that's easy enough to write for you and i should move on to something else?
[16:14] <sinzui> replaceafill, BugAlsoAffectsProductWithProductCreationView. _loadProductsUsingBugTracker is the last remaining origin of 404's for this case. BugTracker.product contains inactive products, and that view coverts the result set to a list without a filter
[16:16] <sinzui> replaceafill, I think the test for your solution would be identical for either your solution of mine. We want to add a bugtracker to two products, make one inactive, then assert that .getPillarsForBugtrackers() returns one product
[16:17] <replaceafill> sinzui, ah ok, can i help with that?
[16:18] <sinzui> sure. I want to have lunch, so you can continue work on the bug. I did not intend to steal it from you.
[16:18] <replaceafill> sinzui, :) np i understand
[16:18]  * sinzui needs to do QA
[16:19] <replaceafill> i'll dig into tests looking for examples to follow
[16:19] <replaceafill> and will report back
[16:19] <replaceafill> thanks sinzui and jelmer_
[16:55] <rick_h_> ivory: the link in your email came up bad, can you dbl check it please?
[21:55] <jcsackett> sinzui: just found out i'll need to bail around 6:30 or 6:45 tonight. :-/
[23:59] <lifeless> doh, no sinzui