[00:00] <wgrant> I was hoping to go and eat something first.
[00:00] <wgrant> So feel free.
[00:01] <thumper> ok
[00:01] <thumper> I'll take it
[00:01] <thumper> wgrant: standup?
[00:01] <thumper> wgrant: or do you want to do it after food?
[00:02] <wgrant> thumper: Well, there's no Curtis, but I guess we could do it anyway.
[00:03] <thumper> wallyworld_: mumble?
[00:03] <wallyworld_> thumper: just untangling my cords
[00:24] <wallyworld_> thumper: https://bugs.launchpad.net/launchpad/+bug/698138
[00:24] <_mup_> Bug #698138: Assigning a non-contributor to a bugtask should show a warning <regression> <Launchpad itself:In Progress by wallyworld> < https://launchpad.net/bugs/698138 >
[00:52] <thumper> I'm busy upgrading the laptop in a non-x session as compiz / unity / whatever has become unresponsive
[00:52] <thumper> all praise to quassel for being awesome
[01:23] <rockstar> gmb, are you around-ish?
[01:30] <lifeless> can has review? https://code.launchpad.net/~lifeless/launchpad/bug-421901/+merge/56861
[01:35] <mwhudson> lifeless: the change to pgsql is just refactoring right?
[01:35] <lifeless> yes
[01:35] <lifeless> reuse ftw
[01:35] <mwhudson> just checking my eyes
[01:35] <mwhudson> :)
[01:35] <lifeless> :)
[01:36] <mwhudson> lifeless: this test will fail if there's more than 5000 bugmessages in sampledata!!
[01:36] <lifeless> mwhudson: oh hoes
[01:36] <lifeless> *noes* heh
[01:37] <lifeless> if someone puts 5K messages in sampledata, I'll ask mark for a company credit card to pay for my flight to their house...
[01:37] <mwhudson> yeah
[01:37] <lifeless> mwhudson: though actually its not 5K it would fail at
[01:37] <lifeless> thats the *chunk* size
[01:37] <mwhudson> and actually even then, it might not fail, i admit i don't know the details of tunableloop
[01:37] <mwhudson> lifeless: right
[01:38] <lifeless> the default window it gets is something like 20 minutes, or more on a many-core machine
[01:38] <lifeless> its about 500 bm updates/second
[01:38] <mwhudson> lifeless: there is a limit there, but it's comically high i agree
[01:38] <lifeless> 60*20*500 -> something big
[01:38] <lifeless> 0.0M
[01:38] <lifeless> bah
[01:38] <lifeless> 0.6M
[01:39] <mwhudson> i guess i'm a little surprised that the launchpad dbuser has the right to disable triggers
[01:39] <lifeless> it doesn't
[01:40] <lifeless> connecting with None as the user
[01:40] <lifeless> tells postgresql to use the unix uid
[01:40] <mwhudson> aah ok
[01:40] <lifeless> which on all out test environments has such access
[01:40] <lifeless> I head-desked on this for about an hour digging around to get it right
[01:40] <lifeless> its *already* commented in the place I found, so i didn't copy the comment.
[01:40] <lifeless> but I did name the method 'root_connection'
[01:41] <lifeless> to hint at this :)
[01:41] <lifeless> I can add more prose, though I'm not sure it will save anyone time
[01:41] <mwhudson> oh right
[01:42] <mwhudson> lifeless: would superuser_connection be acceptable as a name?
[01:42] <lifeless> sure
[01:42] <mwhudson> i didn't grasp the sense that root_ was intended in
[01:42] <lifeless> ok
[01:42] <lifeless> changing
[01:43] <mwhudson> thanks
[01:43]  * mwhudson grimaces at notNull=False
[01:43] <lifeless> double negatives ftw
[01:43] <mwhudson> it's almost triple, in some sense :)
[01:44] <lifeless> its a bit [perhaps a lot] annoying that storm doesn't have the concept of 'can query on this but do not retrieve it ever'
[01:44] <mwhudson> you mean, stuff that shouldn't be retrieved when the object is materialized?
[01:45] <lifeless> yeah
[01:45] <mwhudson> to make the data sent back smaller?
[01:45] <mwhudson> hm yeah
[01:45] <lifeless> nor used when inserting the object
[01:45] <lifeless> this is a field we need in constraints
[01:45] <wgrant> That seems like it should be pretty easy...
[01:45] <lifeless> not in memory, not in inserts and not in select clauses.
[01:45] <wgrant> huwshimi: Hi.
[01:45] <huwshimi> wgrant: Hello
[01:45] <wgrant> huwshimi: Seen bug #753423? We turned it off on prod last night.
[01:45] <lifeless> we'd also need to expose it in subselects etc
[01:45] <_mup_> Bug #753423: Privacy notification overlay broken in Firefox <javascript> <ui> <Launchpad itself:Triaged> < https://launchpad.net/bugs/753423 >
[01:45] <lifeless> so its actually a bit complex.
[01:46] <huwshimi> wgrant: huh. What version of Firefox was that?
[01:46] <mwhudson> lifeless: basically it's something the query compiler should see, but nothing else?
[01:46] <mwhudson> lifeless: holy pee administered btw
[01:47] <mwhudson> ... and back to performance reviews
[01:47] <lifeless> mwhudson: yeah
[01:47] <mwhudson> (procrastination driven development anyone)
[01:47] <lifeless> zigactly
[01:47] <wgrant> huwshimi: Seen in 4 and 3.6
[01:48] <huwshimi> wgrant: Ok thanks.
[02:14]  * mwhudson giggles
[02:14] <mwhudson> Conflict adding file versions.cfg.THIS.moved.moved.moved.moved.moved.  Moved existing file to versions.cfg.THIS.moved.moved.moved.moved.moved.moved.
[02:14] <lifeless> \o/
[02:15] <lifeless> procrastinating again ?
[02:15] <mwhudson> how did you guess!
[02:15] <lifeless> no idea
[02:15] <lifeless> none at all
[02:16] <lifeless> wgrant: want the good news or the bad news
[02:17] <thumper> https://code.launchpad.net/~thumper/launchpad/fix-unicode-path-oops/+merge/56688 needs review
[02:17] <lifeless> mwhudson:   5 https://edge.launchpad.net/bazaar/resolve_lp_path (BazaarApplication:PublicCodehostingAPI)
[02:17] <StevenK> lifeless: We have to use Buildbot for the next 15 years?
[02:17] <lifeless>        OOPS-1923EA1350, OOPS-1923EE1204, OOPS-1923EE1205, OOPS-1923EE1211, OOPS-1923EE1212
[02:17]  * thumper closed twitter tabs to avoid twitter induced procrastination
[02:17] <wgrant> lifeless: BugTask:+index didn't go down at all, and searchTasks is even higher?
[02:17] <thumper> about to close facebook soon too
[02:18] <lifeless> wgrant: it did go down
[02:18] <lifeless> wgrant: it was over a hundred
[02:18] <lifeless> wgrant: I think we've done the low hanging fruit
[02:18] <wgrant> WTF, +login timing out?
[02:19] <wgrant> I wonder if they were around the DC breakage time.
[02:19] <lifeless> possibly
[02:19] <mwhudson> lifeless: is that timeouts?
[02:19] <wgrant> Yes, they were.
[02:19] <wgrant> So our results from today could be a little off, I guess :(
[02:20] <lifeless> mwhudson: yes, probably from the inter-DC fail
[02:20] <mwhudson> oh ok
[02:20] <lifeless> will need to reevaluate on Monday
[02:21] <wgrant> It would be nice to see the distribution over time of each OOPS clsuter.
[02:21] <lifeless> I is
[02:21] <lifeless>   1 AssertionError: Build 434541 lacks a corresponding source publication.
[02:21] <lifeless>    GET: 1 Robots: 1  Local: 0
[02:21] <wgrant> I suspect a lot of the spiky OOPSes are just people hitting refresh lots.
[02:22] <lifeless>       1 https://launchpad.net/ubuntu/+source/sugarcrm/4.5.0h-0ubuntu2/+buildjob/434541/+index (BinaryPackageBuild:+index)
[02:22] <lifeless>        OOPS-1923K942
[02:22] <lifeless> db inconsistency ?
[02:22] <lifeless> wgrant: I'm positive they are
[02:22] <wgrant> lifeless: Sort of.
[02:22] <wgrant> lifeless: There are two causes.
[02:22] <wgrant> lifeless: 1) gina sucked when Ubuntu was imported, created some builds where there should not have been any.
[02:22] <wgrant> 2) Incorrect builds were created when hppa was reintroduced.
[02:23] <lifeless> wgrant: and we assert on this *6 years* later?
[02:23] <wgrant> lifeless: The assertion was commented out for years, because the sampledata was broken.
[02:23] <wgrant> lifeless: it was reeneabled just a few months ago, after the sampledata was fixed.
[02:23] <wgrant> It was not realised that production data was broken too.
[02:23] <wgrant> (the brokenness is not a significant issue, so we should probably just recomment the assertion)
[02:24] <wgrant> lifeless: How are we going with the new appservers?
[02:25] <wgrant> SQL time: 995 ms
[02:25] <wgrant> Non-sql time: 10592 ms
[02:25] <wgrant> Not. Cool.
[02:26] <wgrant> And this is just going to get worse as we drop the timeout, because average CPU:SQL ratio will increase.
[02:29] <StevenK> wgrant: Rargh! It took me WEEKS to uncommet that assertion
[02:29] <lifeless> wgrant: exactly
[02:29] <wgrant> StevenK: But you destroyed tonnes of doctests.
[02:29] <lifeless> wgrant: they are coming
[02:29] <StevenK> wgrant: WEEEEEEEEEEEEEEEEEEEEKS!
[02:29] <StevenK> Still, I suppose it was 6,000 lines of doctests
[02:29] <lifeless> wgrant: 2 of the 4 have memory in place, we have the name of the 3rd, and the 4th has had its memory order
[02:30] <wgrant> lifeless: wampee and soybean are 24GB now?
[02:31] <StevenK> lifeless: Can share name? Just curious what it is.
[02:31] <lifeless> https://lpstats.canonical.com/graphs/WampeeMemory/ https://lpstats.canonical.com/graphs/SoybeanMemory/
[02:31] <lifeless> StevenK: I'd need to look up the rt
[02:31] <lifeless> StevenK: tom pastebinned a template generation command line that faceplanted
[02:33] <StevenK> chaenomeles. I can't even pronounce that.
[02:37] <wgrant> lifeless: Hm, soybean seems to have a bit of headroom :)
[02:37] <lifeless> yeah
[02:37] <lifeless> the issue is shuffling ids around
[02:37] <wgrant> lifeless: What sort of CPU expansion capabilities do these guys have?
[02:37] <lifeless> its just a pita
[02:37] <lifeless> wgrant: we can add another 8 cores + another 24GB of RAM AIUI
[02:37] <lifeless> wgrant: /possibly/ more beyond that
[02:37] <wgrant> Ah, nice.
[02:37] <lifeless> dell 'G6' if that means anything to you
[02:38] <wgrant> Since it would be nice to continue avoiding scaling horizontally.
[02:38] <lifeless> right
[02:38] <StevenK> Er, I thought they were DL380 G6s?
[02:38] <lifeless> StevenK: sounds right
[02:38] <StevenK> Which is *HP*
[02:38] <lifeless> erm
[02:38] <lifeless> talk to ISD :)
[02:40] <spm> wgrant: why do you want to avoid scaling horizontally? genuinely curious, as horizontally scaling has always been the holy grail to aim for in every system I've designed
[02:40] <lifeless> spm: we need to scale horizontally as needed, but:
[02:40] <wgrant> We need to be able to scale both ways.
[02:40] <lifeless>  - there is pressure to reduce datacentre rackspace footprint
[02:40] <wgrant> Vertical is probably cheaper.
[02:41] <wgrant> At least to an extent.
[02:41] <wgrant> CPU + RAM probably cheaper than new machine, if we can fit it.
[02:41] <lifeless>  - the base cost for a machine is fairly high mgmt wise at the moment (logging etc overhead)
[02:42] <mwhudson> according to hp.com you can fit 192 gigs of ram in a G6 :-)
[02:42] <spm> at what cost....
[02:42] <mwhudson> i imagine that's howlingly expensive though
[02:42] <lifeless> spm: anyhow, the choice of new machine or upgraded machine is ultimately down to TCO
[02:43] <lifeless> spm: elmo has expressly indicated a preference for scaling of the *capacity* of individual appserver machines in the medium term
[02:44] <lifeless> spm: we will always keep the ability to scale horizontally (and continue working on being able to do that at the DB, archive, codehosting etc)
[02:44]  * spm recalls get a very blank look from the senior architect at $job-1 when asked what the ROI on her new exciting vision for revamping the entire departments systems was. it was like the question hadn't even arisen. ho hum.
[02:44] <lifeless> spm: the lowest configuration I'd be truely happy with is N+1 redundancy across the board; then scale in whatever way is cheapest.
[02:44] <spm> fair enough. was curious, thanks!
[02:45] <StevenK>  4 files changed, 24 insertions(+), 57 deletions(-)
[02:45] <StevenK> Excellent.
[02:46] <mwhudson> huh, i don't really understand the configurator, but it looks like 144 gigs of ram can be got for about $6k
[02:46] <spm> $job-2 we used to aim in all our designs to ROI in 6-9 months. any longer than that was invariably excessively optimistic. 3 months was preferable.
[02:46] <mwhudson> which is a lot less than i expected
[02:48] <wgrant> I guess the larger DIMMs are a lot more expensive than smaller ones, but it probably has a few slots...
[02:48] <spm> which was a hilarious experience to bring into the public service at $job-1. when in a single 1 month project you can save your client enough that they essentially get you "free" for the next 11 months, your contract renewal is a doddle. :-D
[04:08]  * wgrant waves to Windmill.
[04:08] <wgrant> Bye.
[04:09] <mwhudson> heh
[04:09] <mwhudson> are you killing it, or is it dying without help?
[04:09] <wgrant> I am killing it, since it's still failing like before.
[04:09] <wgrant> Except now there's a timeout set, so it at least doesn't hang and break the world.
[04:10] <StevenK> wgrant: You're disabling it again?
[04:10] <wgrant> Yes.
[04:10] <StevenK> Okay, re-enabling the windmill job on Jenkins. :-)
[04:10] <wgrant> Heh.
[04:11] <StevenK> Last failure on Jenkins was Windmill, too.
[04:11] <wgrant> Yeah.
[04:17] <wallyworld_> huwshimi: did you have a chance to look at the screencast?
[04:17] <huwshimi> wallyworld_: I did briefly. One sec.
[04:18] <wgrant> Yay, fast test suite again.
[04:19] <lifeless> wgrant: you nuked w/m ?
[04:20] <wgrant> Yeah.
[04:20] <wgrant> :(
[04:20] <wgrant> At least it doesn't crash the testrunner any more, though!
[04:21] <wallyworld_> wgrant: it was just the one test that failed. surely it wouldn't be too hard to fix it.
[04:21] <wallyworld_> at the least, it would be good if we could still run the javascript tests
[04:21] <wgrant> wallyworld_: No, it was the same breakage as last time.
[04:21] <StevenK> wallyworld_: Like we said before, we do.
[04:21] <wgrant> Except that it doesn't hang any more, since there was a timeout added.
[04:21] <StevenK> wallyworld_: On Jenkins.
[04:22] <wgrant> wallyworld_: It's not specific to any one test.
[04:22] <huwshimi> wallyworld_: It would be nice if it did the fade back to the search when you click 'no'. Also there's a little design work that needs to be done on the confirmation screen, but I'm happy to have a look at that once your branch is ready.
[04:22] <wgrant> wallyworld_: It is an infrastructure issue.
[04:22] <huwshimi> wallyworld_: But other than that it looks great. Really nice work
[04:23] <wallyworld_> huwshimi: cool. there was no ready made css that looked suitable for the confirmation screen so i just used whatever. i'm open to suggestions. i'm writing some tests now - there's no exiting ones for our lp picker sadly, just the kazr one
[04:24] <huwshimi> wallyworld_: OK no problems. If you want I can push a branch with some changes once it's ready to do so.
[04:24] <wallyworld_> wgrant: StevenK: i really wish the windmill ones were run as part of the build process, else they will bitrot. in any case, could we not run the javascript ones?
[04:25] <wgrant> wallyworld_: They still rely on Windmill.
[04:25] <wgrant> wallyworld_: And I believe it's the Windmill infrastructure that is at fault here.
[04:25] <wgrant> wallyworld_: I really wish we ran them too.
[04:25] <wgrant> But I wish we sucked less more.
[04:25] <wgrant> And frequent test breakage prevents us from sucking less.
[04:26] <wallyworld_> wgrant: yes but given they are mich lighter weight i'm hoping they won't hang or whatever
[04:26] <wallyworld_> huwshimi: my current work branch (sans tests) is https://code.launchpad.net/~wallyworld/launchpad/assign-non-contributor
[04:26] <lifeless> wallyworld_: if its an infrastructure issue, then it doesn't matter which tests are run within that set, its all/nothing.
[04:26] <wallyworld_> huwshimi: if you flick me some css i can add it in
[04:26] <huwshimi> wallyworld_: Cool thanks.
[04:27] <lifeless> wallyworld_: if its a specific test issue, we can fix that test specifically as needed; I havent studied wgrants analysis, but if he thinks it infrastructure, I'm inclined to trust it
[04:27] <wallyworld_> lifeless: sure. i'm thinking though that the js tests are much lighter weight and perhaps less prone to the issues
[04:27] <wallyworld_> lifeless: likely is infrastructure since i *think* different tests fail
[04:27] <lifeless> wallyworld_: that assumes that its a test issue, not infrastructure, no?
[04:27] <wgrant> It's possible (probable, even) that the JS tests are safer to run.
[04:28] <wgrant> Because they're less likely to expose infrastructure issues.
[04:28] <wgrant> If someone wants to try putting them in a different layer and reenable them, by all means do so.
[04:28] <wallyworld_> lifeless: the full windmill tests (ie not he js ones) are more likely to fail from what i've seen, even though windmill is used as the harness for the js tests
[04:29] <wgrant> It is, as usual, the client.open call failing.
[04:29] <wallyworld_> wgrant: if i get a chance, i'll look to do that. i'd like as much testing done "in band" as we can
[04:29] <wgrant> We previously couldn't get that from failed test runs, but had determined it experimentally.
[04:29] <wgrant> (since the testrunner hang and was killed without a traceback)
[04:29] <wgrant> We at least have tracebacks now.
[04:30] <wallyworld_> yes, true
[04:30] <wgrant> I'll file a bug.
[04:30] <wallyworld_> wgrant: i'll mention it to deryck tonight. he will be inetrested i'm sure
[04:32] <wgrant> wallyworld_: Indeed.
[04:32] <wgrant> Bug #754256
[04:32] <_mup_> Bug #754256: WindmillLayer tests occasionally hit a timeout in client.open <spurious-test-failure> <Launchpad itself:Triaged> < https://launchpad.net/bugs/754256 >
[04:33] <wallyworld_> wgrant: thanks. and unless a few remaining issues with onkeypress events are sorted, windmill tests won't work with natty (ff4) anyway :-(
[04:33] <wgrant> We may want to include extra debugging info and switch it back on until it fails.
[04:33] <wgrant> Now that we can get useful info out of it.
[04:34] <wgrant> Or even run buildbot with -D for a while, or something like that.
[04:34] <wallyworld_> yeah. i'll see what deryck suggests since he's more across this than i am
[04:34] <wgrant> wallyworld_: It fails to start with a profile error for me :/
[04:34] <wgrant> Maybe I should wipe some of my ~/.*
[04:34] <wallyworld_> wgrant: ah. i know how to fix that
[04:35] <wallyworld_> wgrant: you need to symlink a couple of things
[04:35]  * wallyworld_ looks
[04:35] <wgrant> Oh, new unity today.
[04:35]  * wgrant prepares to reboot.
[04:40] <lifeless> wgrant: you're familiar with the bfj stuff right ?
[04:40] <wgrant> lifeless: Yeah.
[04:40] <lifeless> what would prevent us removing BFJ
[04:40] <wgrant> The table, or the class?
[04:40] <lifeless> lets assume I'm right that the table should be folded into the two adjacent tables (packagebuild and translationsbuild)
[04:41] <wgrant> I'd prefer to keep the current *object* model. Database model I'm not fussed about.
[04:41] <lifeless> sorry, translationtemplatesbuild
[04:42] <wallyworld_> wgrant: you need to symlink /usr/lib/firefox/defaults/profile -> /etc/firefox/profile
[04:42] <wgrant> wallyworld_: Ah, thanks!.
[04:42] <lifeless> wgrant: well, I don't care for or against the object model. the db model is killing performance.
[04:42] <wallyworld_> wgrant: or lib64
[04:42] <wgrant> lifeless: Right.
[04:42] <lifeless> wgrant: table scans on million row tables - bad
[04:42] <wallyworld_> whatever is there
[04:43] <wgrant> wallyworld_: None of those three paths exist.
[04:43] <lifeless> wgrant: and binarypackagebuild + sourcepackagerecipebuild need to be rolledup too
[04:43] <lifeless> wgrant: BFJ + PB + BPB -> one table
[04:43] <lifeless> wgrant: BFJ + PB + SPRB -> one table
[04:44] <lifeless> wgrant: BFJ + TTB -> one table
[04:44] <wallyworld_> wgrant: you should have a /usr/lib/firefox ? if so mkdir defaults and then do the symlink
[04:44] <wallyworld_> wgrant: or you saying /etc/firefox/profile doesn;t exist?
[04:45] <wgrant> wallyworld_: /etc/firefox/profile doesn't exist.
[04:45] <wgrant> lifeless: So, we need to keep BFJ around, at least a bit.
[04:46] <lifeless> wgrant: hy
[04:46] <lifeless> why
[04:46] <wallyworld_> wgrant: ok. i think what i did was start ff3 and created a new empty profile. then copied that to /etc/firefox/profile. not sure if a ff3 install creates /etc/filefox/profile or not
[04:46] <wgrant> lifeless: Unless you want to do a big UNION.
[04:46] <wgrant> lifeless: eg. for builder history.
[04:46] <lifeless> wgrant: do we care about builder history?
[04:46] <wallyworld_> wgrant: we really need windmill to support ff4's directory layout :-)
[04:46] <lifeless> wgrant: beyond a couple of days
[04:46] <wgrant> wallyworld_: Yeah :(
[04:47] <lifeless> ff4 writes its profiles to /etc/firefox?
[04:52] <wallyworld_> lifeless: ah that could be it. i couldn't recall exactly why i symlinked to that directory. but that i think is the correct answer
[04:53] <wgrant> I think it's more that Windmill looks for the default profile in one of those directories, to make a copy of its own.
[04:53] <wgrant> And Firefox 4 doesn't really have a default profile, AFAICT...
[04:53] <wallyworld_> lifeless: it's a one line change to windmill. but i would like to solve the other ff4 specific test failures first
[04:53] <wgrant> or it's very well hidden.
[05:00] <LPCIBot> Yippie, build fixed!
[05:00] <LPCIBot> Project devel build #619: FIXED in 5 hr 46 min: https://lpci.wedontsleep.org/job/devel/619/
[05:02] <wgrant> lifeless: So, it should be fairly simple to alter BuildFarmJob/PackageBuild to direct inheritance, and flatten the tables.
[05:02] <wgrant> The only difficult bits will be the things that query for BFJs or PBs directly, then adapt them to the real objects.
[05:05] <wgrant> Basically... flatten tables, merge *Derived into *, change SPRB and BFB and TTB to inherit from * instead of *Derived... fix queries.
[05:05] <lifeless> yeah
[05:06] <lifeless> this may be 'undo the refactoring that was done', which if true is sad in a way, but the performance measurements are pretty clear about the status quo.
[05:07] <wgrant> lifeless: The old one wasn't ideal, either. The table split (which this is undoing, and then going further) was not the primary goal -- it was removing duplicated code and cleaning up too.
[05:07] <lifeless> a big question
[05:08] <lifeless> which I don't know enough to reason about
[05:08] <lifeless> is is it easy enough to Just Do
[05:08] <lifeless> or is it big enough to need a feature squad
[05:08] <wgrant> It's not small enough to Just Do, but not big enough for a feature squad.
[05:08] <lifeless> or perhaps a few carefully chosen denormalised columns (like BugMessage.owner is)
[05:08] <wgrant> I guess it depends what Just Do means.
[05:09] <wgrant> It's a few days of work.
[05:09] <lifeless> oh
[05:09] <lifeless> thats small enough to JustDo
[05:09] <lifeless> it will fix about 10 timeouts
[05:09] <lifeless> thats *cheap*
[05:11] <wgrant> The hairy bits are things like getBuildsForBuilder and findBuildCandidate.
[05:11] <wgrant> Everything else is actually pretty simple.
[05:11] <huwshimi> Woah! I have crazy scrollbars!
[05:11] <lifeless> so fBC should be easy enough.
[05:11] <wgrant> Since it's not all duplicatastic any more.
[05:12] <lifeless> wgrant: gBFB - how deep does that need to go, realistically ?
[05:12] <lifeless> wgrant: like, why do we care about what was built on $builder-N 4 years ago ?
[05:12] <wgrant> lifeless: Probably only a few days, yeah.
[05:13] <wgrant> lifeless: Other non-trivialities include URLs... We could share a sequence, but that sounds mildly evil.
[05:14] <lifeless> wgrant: urls for ?
[05:14] <wgrant> lifeless: BPBs and SPRBs.
[05:14] <lifeless> composite key
[05:17] <lifeless>  /bpb-1234
[05:17] <lifeless>  /sprb-1234
[05:17] <lifeless> or /+bpb/1234
[05:17] <lifeless>    /+sprb/1234
[05:20] <huwshimi> A review (fixes privacy notifications in Firefox): https://code.launchpad.net/~huwshimi/launchpad/privacy-notification-firefox-753423/+merge/56872
[05:21] <huwshimi> wgrant: If you're interested ^
[05:21] <wgrant> huwshimi: Just saw the email. Looking.
[05:21] <wgrant> huwshimi: Have you tried other browsers?
[05:21] <wgrant> Well, I guess all that's left is Opera and IE.
[05:23] <huwshimi> wgrant: It doesn't work in IE because our javascript is broken in IE. I'm just double checking Opera.
[05:23] <huwshimi> wgrant: brb
[05:24] <wgrant> huwshimi: Heh.
[05:40] <wgrant> huwshimi: You can't have the content margin there from the start?
[05:40] <wgrant> huwshimi: It's not ideal that the page jumps once the JS loads.
[05:41] <huwshimi> wgrant: Right, sorry back.
[05:42] <huwshimi> wgrant: It's not ideal.
[05:43] <huwshimi> wgrant: I'm not sure how else to modify the body classes and add html to the main template without JS.
[05:44] <wgrant> huwshimi: Can't you add a class in the template if the FF is activated?
[05:44] <wgrant> (and the object is private)
[05:44] <huwshimi> wgrant: To the <body>?
[05:45] <wgrant> Something like that.
[05:49] <huwshimi> wgrant: Oh and yes, it works in Opera (whatever version I have on my computer)
[05:50] <StevenK> wgrant: Do you remember where ppa-run and such are hiding on DF?
[05:52] <wgrant> huwshimi: Excellent.
[05:52] <wgrant> StevenK: ppa-run?
[05:53] <StevenK> wgrant: There's a script on DF that I used as a cheat-sheet for how to accept uploads
[05:53] <wgrant> StevenK: Upload, run process-upload.
[05:53] <wgrant> If you don't remember all the incantations, maybe follow my HowToUseSoyuzLocally.
[05:54] <wgrant> I didn't know there was a script... I just do it all manually.
[05:54] <wgrant> There's not that much to it, particularly with --builds in cron.
[05:58] <huwshimi> wgrant: Do you actually know if it's possible to modify the body classes and add new divs before content or do you just imagine that it should be possible? :)
[05:59] <huwshimi> wgrant: Cause from what I'm looking at it doesn't look very possible with the current setup
[05:59] <jtv> Can someone walk me through the process to upload a debian package on dogfood?
[06:01] <wgrant> jtv: Have you uploaded it? That's the first step.
[06:01] <jtv> Yes
[06:01] <wgrant> Well, I guess the first step is obtaining a package.
[06:01] <wgrant> Do you have a package?
[06:01] <jtv> Yes, I've uploaded it.
[06:01] <huwshimi> wgrant: If you take a look at base-layout.pt you'll get some clues about what's currently possible.
[06:01] <wgrant> Great, so it's in /srv/launchpad.net/ubuntu-queue/incoming?
[06:01] <wgrant> huwshimi: Let's see.
[06:02] <jtv> wgrant: trying to figure that out
[06:02] <wgrant> huwshimi: Could you put something in view/context/fmt:public-private-css?
[06:02] <wgrant> huwshimi: That's already injecting classes into body.
[06:03] <jtv> wgrant: yup, it's there
[06:03] <wgrant> jtv: ^Rprocess-upload^R^R or so until you see an invocation that includes /srv/launchpad.net/ubuntu-queue
[06:04] <wgrant> https://dev.launchpad.net/Soyuz/HowToUseSoyuzLocally is probably a useful guide to have open.
[06:04] <jtv> wgrant: by the way, why "ubuntu-queue" when it's a debian package?
[06:05] <wgrant> jtv: Er.
[06:05] <jtv> Oh.
[06:05] <wgrant> You're trying to upload a package to *Debian*, rather than a Debian package?
[06:05] <jtv> Oh dear, there's a difference?
[06:05] <wgrant> Well, Ubuntu uses Debian packages.
[06:05] <StevenK> Yes
[06:05] <huwshimi> wgrant: I guess. It doesn't seem like an appropriate place to do it though
[06:05] <wgrant> Uploading *to Debian* on something production-like *may* work, but is untested.
[06:06] <jtv> Well yes, but it'd be a bit meaningless for me to say "Debian" when I just mean "not RPM," right?
[06:06] <wgrant> It certainly won't create any builds or anything.
[06:06] <jtv> Oh
[06:06] <jtv> That could make it rather hard for me to Q/A DSD packageset filtering for the debian case by "just uploading a package."
[06:07] <wgrant> jtv: Well, you could try.
[06:07] <wgrant> jtv: It won't create any builds, but the primary archive shouldn't care about that.
[06:07] <wgrant> I'll run the processor so I can watch it blow up, unless you have already.
[06:08] <jtv> If it doesn't work, will I know whether I caused the problem or it failed somewhere else?
[06:08] <wgrant> I'll hopefully be able to work that out.
[06:08] <jtv> I suppose I don't really need any builds, as long as it generates a DSDJ.
[06:08] <wgrant> Right.
[06:08] <StevenK> And an SPPH
[06:08] <StevenK> wgrant: I'm about to IDS up DF
[06:08] <wgrant> Keep in mind that this ranks very high on the sick, wrong, and abuse of -C almost-anything scales.
[06:08] <StevenK> wgrant: Or I can wait until you're done, just in case
[06:09] <wgrant> StevenK: Are you trying to kill Nafallo?
[06:09] <StevenK> Pffft, IDS only takes 7 minutes.
[06:09] <wgrant> ... on prod.
[06:09] <StevenK> No, on DF
[06:09] <wgrant> Huh.
[06:09] <StevenK> I ran it yesterday
[06:10] <StevenK> I'm expecting it to be quicker on qas
[06:10] <wgrant> jtv: You ran it and it got rejected?
[06:10] <wgrant> /srv/launchpad.net/ubuntu-queue/rejected/upload-ftp-20110408-035934-000012/~/ubuntu/libpqxx_3.2-1sid1.dsc
[06:10] <wgrant> Not a terribly helpful upload path; you probably wanted /debian instead of /~/ubuntu
[06:11] <jtv> wgrant: ran what exactly?
[06:11] <wgrant> The upload has been processed.
[06:11] <wgrant> and rejected.
[06:11] <wgrant> Someone ran process-upload.py.
[06:11] <jtv> Yay.
[06:11] <jtv> Ah, yes.
[06:11] <jtv> But yes, I wanted to upload to debian.
[06:11] <wgrant> We'll need -C absolutely-anything, I suspect, which is why I want to run it myself.
[06:12] <jtv> Ahhh, ubuntu is set in "incoming" in my .dput.cf.
[06:12] <wgrant> Since Debian has no ComponentSelections or SourcePackageFormatSelections or ArchivePermissions.
[06:12] <wgrant> Or even a SectionSelection, for that matter.
[06:15] <wgrant> StevenK, jtv: https://code.launchpad.net/~wgrant/launchpad/remove-archivepublisher-config/+merge/56873 would like a review.
[06:15] <jtv> wgrant: I'll take it.  Meanwhile, I replaced "ubuntu" with "debian" in my .dput-cf (well, I cloned the "dogfood" section) but that doesn't seem to be right.
[06:15] <wgrant> jtv: Do you still have a ~/ in there?
[06:15] <StevenK> wgrant: r=me
[06:15] <wgrant> StevenK: Thanks.
[06:15] <jtv> That was quick :)
[06:16] <wgrant> It's not a very difficult branch.
[06:16] <StevenK> It's +0/-6
[06:16] <StevenK> So a no-brainer
[06:16] <jtv> wgrant: yes, I do… just copied what I had for ubuntu.
[06:16] <wgrant> jtv: That's for PPAs.
[06:16] <jtv> ~%(dogfood)s/ubuntu
[06:16] <wgrant> Please don't make me hack non-Ubuntu PPA support onto mawson.
[06:17] <wgrant> [df]
[06:17] <wgrant> method = ftp
[06:17] <wgrant> fqdn = upload.dogfood.launchpad.net:21
[06:17] <wgrant> incoming = %(df)s
[06:17] <wgrant> login = anonymous
[06:17]  * jtv backs off slowly, keeping both hands in view
[06:17] <wgrant> That's what I use.
[06:17] <wgrant> dput df:debian whatever_source.changes
[06:17] <jtv> And that'll do both ubuntu and debian?
[06:17] <jtv> ah!
[06:17] <StevenK> wgrant: Bug 753249 makes me sad.
[06:17] <_mup_> Bug #753249: +initseries calls deriveDistroSeries() with incorrect arguments <derivation> <Launchpad itself:Triaged> < https://launchpad.net/bugs/753249 >
[06:17] <jtv> And the %(df)s is the section name?
[06:17] <wgrant> jtv: Right.
[06:18] <wgrant> jtv: It is replaced with the argument after the colon.
[06:18] <jtv> Great.  Trying the upload again.
[06:18] <wgrant> I'm not sure if absolutely-anything will be sufficient, but we'll see.
[06:19] <jtv> Grrr package has already been uploaded
[06:19] <wgrant> -f
[06:20] <jtv> Oh, I just edited the changelog and uploaded again.
[06:20] <jtv> It's almost done I think.
[06:20] <jtv> Almost there…
[06:20] <jtv> Done!
[06:20] <wgrant> There it is.
[06:22] <wgrant> 2011-04-08 05:21:47 DEBUG   Unable to find distroseries: unstable
[06:22] <wgrant> Overriding...
[06:25] <wgrant> Nearly there.
[06:26] <wgrant> Bah.
[06:26] <jtv> Should I say "sid" instead of "unstable"?
[06:26] <wgrant> P-a-s crashing due to no nai.
[06:27] <wgrant> 2011-04-08 05:27:19 DEBUG   Thank you for your contribution to Debian GNU/Linux.
[06:27] <wgrant> jtv: ^^
[06:28] <jtv> wgrant: what is that in English?
[06:28] <jtv> Something crashed, something succeeded?
[06:28] <wgrant> 2011-04-08 05:27:19 DEBUG   Sent a mail:
[06:28] <wgrant> 2011-04-08 05:27:19 DEBUG       Subject: [debian/sid] libpqxx 3.2-1sid2 (Accepted)
[06:28] <wgrant> 2011-04-08 05:27:19 DEBUG       Recipients: "Jeroen T. Vermeulen" <jtv@canonical.com>
[06:28] <StevenK> It broke, and he fixed it
[06:28] <jtv> Ah.  Thanks!
[06:28] <jtv> And there's my DSDJ!
[06:29] <jtv> Where's the DSDJ runner?
[06:29] <wgrant> (FTR, I changed the upload path to debian/sid, added a ComponentSelection for main in sid, amputated pas.py, then process-upload -C absolutely-anything)
[06:30] <jtv> Wow, this stuff isn't easy is it?
[06:30] <wgrant> That was actually a lot easier than I expected.
[06:30] <jtv> What is pas.py by the way?
[06:30] <StevenK> jtv: Don't ask
[06:30] <wgrant> Do not go there.
[06:30] <StevenK> You don't want to know
[06:30] <wgrant> Heh.
[06:30] <jtv> StevenK: gee thanks, I already asked didn't I?
[06:30] <jtv> Oh well, I'm quite happy for the monster to stay under the bed.
[06:31] <wgrant> In this case the monster *is* the bed.
[06:31] <lifeless> hmm
[06:31]  * jtv has a brief, swimming vision of Ripley and the child hiding under the bed and the facehugger on top squeaking "oh dear oh dear there's monsters under the bed"
[06:31] <lifeless> are bugs 739144 and 701525 dups
[06:31] <wgrant> Bug #739144, bug #701525
[06:31] <_mup_> Bug #739144: Code review comment via email truncated, most content lost! <email> <regression> <Launchpad itself:Triaged> < https://launchpad.net/bugs/739144 >
[06:31] <_mup_> Bug #701525: Merge-proposal reply truncated <Launchpad itself:Incomplete> < https://launchpad.net/bugs/701525 >
[06:31] <wgrant> I think so.
[06:32] <wgrant> I thought someone already duped them.
[06:32] <jtv> Is that the same truncation that also happens on the web page?
[06:32] <wgrant> .. oh.
[06:32] <wgrant> A third instance.
[06:32] <wgrant> That is odd.
[06:32] <lifeless> I was looking for a dup for 754303
[06:32] <lifeless> which I could *swear* I saw
[06:32] <lifeless> but the search eludes me
[06:33] <wgrant> I've seen something like it before, yeah... but I can't think of the summary.
[06:33] <poolie> i guess it's not news if staging is timing out?
[06:33] <lifeless> nope
[06:33] <lifeless> I mean
[06:33] <poolie> k
[06:33] <lifeless> if it doesn't work how ever many times you try
[06:34] <lifeless> and it works on prod
[06:34] <lifeless> it may be an issue
[06:34] <lifeless> but considering you may need 40 or 50 attempts to seed the cache
[06:34] <poolie> sure, but intermittent is ok
[06:34] <poolie> np
[06:34] <lifeless> you need rather a lot of patience
[06:51] <lifeless> wgrant: is bug 51007 really still present ?
[06:51] <_mup_> Bug #51007: unnecessary truncation in queue output <lp-soyuz> <soyuz-ftpmaster-tools> <Launchpad itself:Triaged> < https://launchpad.net/bugs/51007 >
[06:52] <wgrant> lifeless: I think so. It should be gone in 6 months, though.
[06:52] <wgrant> (queue is dying)
[06:52] <lifeless> wontfix ?
[06:52] <wgrant> Not yet.
[06:52] <wgrant> Maybe eventually.
[06:52] <lifeless> well
[06:52] <lifeless> if its going to be deleted
[06:52] <wgrant> It hasn't been decided exactly what will happen yet.
[06:52] <lifeless> and we're not going to fix in the interim
[06:56] <lifeless> wgrant: so queue isn't dying ?
[06:57] <wgrant> lifeless: It has to at least change significantly for DDs. But we don't know quite how yet.
[06:57] <lifeless> wgrant: so, we'll provide a webUI, one that works ?
[06:57] <wgrant> It will probably die, given that shell access is hopefully going to vanish.
[06:57] <wgrant> Right.
[06:57] <lifeless> so, lets wontfix.
[06:58] <lifeless> its been *7* years
[06:58] <lifeless> for a trivial show-wider-columns problem.
[06:58] <wgrant> 5 years, but OK.
[06:59] <lifeless> bah, yes.
[07:04] <jtv> StevenK: having some weird trouble with the dsdj runner.  It doesn't seem to clean up its jobs.
[07:05]  * lifeless closes a 4-digit bug
[07:07] <jtv> StevenK: I always forget whether job runners are supposed to do that or not.
[07:07] <StevenK> It is not
[07:07] <lifeless> I think bug 674759 might be fixed
[07:07] <_mup_> Bug #674759: hide-comments.py is hiding the wrong bug <canonical-losa-lp> <lp-bugs> <Launchpad itself:Triaged> < https://launchpad.net/bugs/674759 >
[07:08] <jtv> StevenK: Ah sorry.  Just seeing some aborts in the debug output that mystify me a bit.  It's hard to Q/A "things shouldn't happen."  :-)
[07:08] <StevenK> jtv: Oh, like what?
[07:08] <jtv> I'm Q/A'ing the packageset filtering of DSDs.
[07:08] <StevenK> Yes, I know, I'm interested in the output
[07:09] <jtv> Jesus if these neighbours could just be a _little_ bit quiet…  I got the guy with the truck to turn off the radio on the external speakers, so now the baby kicks in.
[07:09] <jtv> Unfortunately I can't snip anything right now; trackpad stopped working.
[07:10] <StevenK> jtv: The next door apartment is having their bathroom remodelled, so I got woken up at 8:00am by them removing tiles with a very noisy power tool.
[07:10] <jtv> StevenK: kill!
[07:10] <StevenK> jtv: Thankfully, they stopped using it at 2:00pm
[07:10] <jtv> Due to successful kill?
[07:10] <StevenK> Sadly, no.
[07:11] <StevenK> And I got to miss 30 minutes of it while I walked to get lunch.
[07:12] <jtv> Personal tip: if you miss the comforting sound of a power tool being used near you at an ungodly hour, play some Ronald Keating.  Different sound, same feeling.
[07:13] <lifeless> hmm
[07:13] <lifeless> 7 seconds to add a bug comment
[07:13] <lifeless> I think the structural filters are slowing things down somewhat.
[07:16] <spiv> StevenK: I had the pleasure of being one of those residents that inflicted power tool noise on our neighbours, finally.
[07:16] <StevenK> Haha
[07:16] <StevenK> spiv: Vincent got "My First Power Drill" as a present?
[07:16] <spiv> StevenK: we had a leaking pipe in the shower wall, so there wasn't really any choice, so I didn't even have to feel too bad about it :)
[07:17] <spiv> Although the week and a half without a functional shower was tiresome (fortunately we also have a bathtub).
[07:18] <spiv> StevenK: Vincent is quite capable of making noise without power tools…
[07:18] <StevenK> Haha
[07:18] <spiv> Such as by banging things together that aren't meant for banging, and then he makes even more noise when you take the things away from him!
[07:30] <lifeless> bam bam
[08:33] <wgrant> I'm on the conflict.
[08:36] <wgrant> Non-trivial JS refactors on both branches :(
[08:36] <wgrant> Ah, maybe it's the one that gary referred to.
[08:40] <jtv> Anybody up for a review involving shell quoting?  https://code.launchpad.net/~jtv/launchpad/db-bug-752178/+merge/56719
[08:42] <jtv> hi henninge!
[08:42] <henninge> Hi jtv! ;)
[08:55] <danilos> does YUI3 have anything regarding string formatting (ala sprintf)?
[08:56] <wgrant> danilos: Y.Lang.substitute may help.
[08:56] <wgrant> lib/lp/code/javascript/requestbuild_overlay.js uses it.
[08:58] <danilos> wgrant, exactly what I needed, thanks
[08:59] <jtv> wgrant: should GNUPGHOME really be unset in the 10-sign-releases script?
[08:59] <jtv> Or is that just an example?
[09:00] <wgrant> jtv: It needs to be set to some special value on prod.
[09:00] <wgrant> On DF I just unset it to make it work.
[09:01] <jtv> wgrant: thanks.  The scripts I implemented were meant for production, so we could just hard-code the production value there if we know what it is.  But what is it?
[09:02] <jtv> Luckily I made it easy to use in-tree or out-of-tree scripts, so we could just let IS copy and edit the scripts.
[09:05] <wgrant> He always drops out as I try to answer :(
[09:06] <wgrant> jtv: Check cron.publish-ftpmaster for the prod value.
[09:07] <jtv> Thanks!
[09:07] <jtv> (So much for saving my IRC connection by setting my TCP keepalive wait to 1 minute… I'll try with a shorter interval)
[09:09] <adeuring> good morning
[09:10] <jtv> hi adeuring!
[09:10] <jtv> Will you be reviewing today?
[09:10] <adeuring> hi jtv!
[09:11] <jtv> adeuring: if you'll be reviewing, then https://code.launchpad.net/~jtv/launchpad/db-bug-752178/+merge/56719  please.  ☺
[09:11] <adeuring> jtv: sure, I'll look
[09:12]  * adeuring just needs a bit more coffee
[09:13] <mrevell> Hello
[09:19] <jtv> adeuring: thanks, and enjoy your coffee.  :)
[09:21] <bigjools> good morning world
[09:21] <jtv> world julian morning good
[09:22] <bigjools> wgrant: are people really so thick as to repeatedly try to upload using the same unregistered GPG key despite the warning they'll get now?
[09:23] <lifeless> bigjools: you are nowhere near pessimistic enough
[09:23] <bigjools> lifeless: ha
[09:24] <bigjools> the sun is shining, I am feeling the opposite today :)
[09:25] <wgrant> bigjools: Do we know that the "Signing key %s is not registered in launchpad." error is returned properly to the client?
[09:26] <bigjools> yes
[09:26] <wgrant> I know the "no public key" one is, but that is an actual gpg error.
[09:26] <lifeless> wgrant: I know something we can do
[09:26] <lifeless> wgrant: about tags
[09:26] <bigjools> unless something broke since I tested it
[09:26] <lifeless> wgrant: use the fast query, and do an exists subquery to find *a* bug that is [visible clause]
[09:29] <wgrant> lifeless: Hmm. That's not really correct. But something to think about.
[09:30] <lifeless> wgrant: http://paste.ubuntu.com/591153/
[09:30] <lifeless> wgrant: how is it incorrect?
[09:32] <wgrant> lifeless: I can file a bug in any project, tag it with some tag, and magically that tag shows up in lots of projects' portlets despite the bugs all being invisible to me.
[09:33] <lifeless> why would that happen ?
[09:33] <wgrant> Oh, I misread.
[09:34] <wgrant> That's not a bad idea.
[09:35] <lifeless> needs testing and result cross referencing
[09:35] <lifeless> but it seems snappy
[09:52] <bigjools> wgrant: so, we need arbitrary pockets
[10:03] <wgrant> bigjools: No, we need no pockets.
[10:03] <wgrant> bigjools: We will have suites instead.
[10:03] <bigjools> wgrant: that won't work very well with our distroseries model
[10:03] <wgrant> I guess :(
[10:04] <bigjools> oh fuck sake, natty updates this morning are hanging on update-initramgs
[10:04] <bigjools> update-initramfs even
[10:04] <wgrant> Suite = (DistroSeries, suffix)
[10:04] <bigjools> yes, which is basically a Pocket table :)
[10:05] <wgrant> Maybe.
[10:06] <adeuring> jtv: r=me
[10:06] <wgrant> Bah.
[10:06] <jtv> thanks adeuring!  And got more on the way.  :)
[10:06] <henninge> wgrant: Hi!
[10:06] <wgrant> henninge: Hi.
[10:07] <henninge> wgrant: Looks like I deleted you mail on xss attacks. Where did you post that so I can go to the archives?
[10:07] <lifeless> stub: probably want to unlink that old rosetta bug from your garbo branch :)
[10:07] <lifeless> henninge: canonical-lp
[10:07] <wgrant> henninge: It was canonical-launchpad
[10:07] <henninge> cheers
[10:08] <stub> lifeless: Yeah, that old annoying bug.
[10:09]  * stub throws away some historical data because MPs are too thick to realize bugs that have been fixed are unlikely to be fixed again.
[10:10] <wgrant> bigjools: I guess the table will also need to encompass policies that are hardcoded now :(
[10:10] <lifeless> stub: there is a bug open on that too; we should capture bugs against the mp not infer from the branch
[10:10] <lifeless> or something like that
[10:12] <bigjools> wgrant: well we need a pocketpolicy table
[10:13] <wgrant> bigjools: And SuiteDependency.
[10:13] <wgrant> Like ComponentDependency :(
[10:13] <bigjools> yeah
[10:13] <bigjools> man this is a deep rabbit hole
[10:13] <wgrant> But the branches are fairly shallow.
[10:13] <bigjools> talking of rabbit, can we please get rid of rabbit-mq from the lp dependencies
[10:13] <wgrant> There's just an awful lot of them.
[10:14] <bigjools> its package is a royal pita
[10:14] <lifeless> bigjools: how so? [and no, I have a branch enabling it, just needs a little debugging to land]
[10:15] <bigjools> lifeless: I do not want server daemons running on my laptop
[10:15] <bigjools> so this applies to apache and all the other crap we need
[10:15] <bigjools> it should start up on demand
[10:15] <lifeless> bigjools: disable it
[10:15] <bigjools> the rewrite daemon that apache starts uses huge gobs of ram as well
[10:15] <lifeless> edit /etc/defaults/rabbitmq-server and set the start command to false; IIRC
[10:15] <bigjools> lifeless: you think I haven't already? :)
[10:16] <lifeless> the test suite will spin up its own rabbit
[10:17] <jtv> adeuring: got a very very brief one: https://code.launchpad.net/~jtv/launchpad/db-bug-752179/+merge/56896
[10:17] <adeuring> jtv: sure
[10:17] <jtv> thanks
[10:17] <jtv> I'm also about to propose another small one.
[10:18] <wgrant> bigjools: Is there a list of this sort of thing on the LEP?
[10:19] <bigjools> wgrant: I've amended it at a high level, no implementation details at all
[10:20] <adeuring> jtr=me
[10:27] <bigjools> wgrant: are you getting problems upgrading rabbitmq-server on natty?
[10:27] <wgrant> bigjools: Nope.
[10:27] <wgrant> Works fine.
[10:28] <bigjools> I get dpk errors every time because the packaging is a PoS
[10:28] <wgrant> I haven't disabled it, though.
[10:28] <bigjools> either way it belly-aches
[10:28] <lifeless> bigjools: details
[10:28] <bigjools> Starting rabbitmq-server: TIMEOUT - check /var/log/rabbitmq/startup_{log,err}
[10:28] <bigjools> the logs are empty because I disabled it
[10:29] <bigjools> but it still bitches the same if I enable it
[10:29] <lifeless> SpamapS: ^
[10:30] <bigjools> the package installation also fails if there's no internet connection
[10:39] <bigjools> lifeless: I get that error when doing a manual start so something's fairly hosed!
[10:55] <lifeless> bigjools: sure sounds it
[10:58] <bigjools> lifeless: re-installing, and it still failed.... :/
[10:58] <bigjools> it now sits there as an unremovable package that blocks apt-get dist-upgrade.  Sigh.
[10:59] <wgrant> Even after a purge?
[10:59] <lifeless> buggy maintainer script
[10:59] <lifeless> file a bug
[10:59] <lifeless> and edit /var/lib/dpkg/info/rabbit*
[11:00] <bigjools> wgrant: can't purge, had to do dpkg remove --force-depends to get rid of it
[11:01] <wgrant> Now purge it.
[11:01] <bigjools> E: Unmet dependencies. Try 'apt-get -f install' with no packages (or specify a solution).
[11:03] <wgrant> dpkg --purge --force-depends rabbitmq-server
[11:03] <wgrant> Otherwise we may have to hack the cache.
[11:03] <bigjools> ok done
[11:04] <bigjools> re-instally?
[11:04] <bigjools> re-install? ...
[11:04] <wgrant> Yeah
[11:04] <wgrant> Hopefully --purge deleted the database.
[11:05] <bigjools> Starting rabbitmq-server: SUCCESS
[11:06] <bigjools> so now I just need to disable it again :)
[11:06] <bigjools> thanks for the help
[11:06] <jtv> adeuring: would you mind another one?  Also quite small…  https://code.launchpad.net/~jtv/launchpad/db-bug-752181/+merge/56903
[11:06] <adeuring> jtv: sure
[11:07] <jtv> thanks
[11:09] <bigjools> wgrant: so I am trying to come up with a plan to phase in some pocket/pocketpolicy schema
[11:10] <bigjools> wgrant: I think it can be done by making the schema/model and then converting little bits of soyuz one at a time.  When they're all done, then we can have other distros start to use it.
[11:17] <wgrant> bigjools: Gradually move bits of policy onto a table which uses the enum, then once they're done replace the enum with an fkey everywhere?
[11:19] <bigjools> wgrant: so first I want to have a pocket table.  Then we need some policies defined, probably enums which map to some code.  Then we have a pocketpolicy table (pocket, policyenum) which gets looked up instead of hardcoding the rules, whereever the rules are currently applied.  They can be gradually moved.
[11:20] <bigjools> but this may need thinking over a bit more, we might need to apply policies in order for example
[11:20] <StevenK> bigjools: I'd suggest a better name than 'pocket'
[11:20] <wgrant> Well, the policies are pretty simple at the moment.
[11:20] <bigjools> StevenK: pocket is a very strong name in Soyuz
[11:20] <wgrant> Suite = (DistroSeries, suffix)
[11:20] <wgrant> We will emulate Ubuntu's pockets using suites.
[11:21] <wgrant> We have to fix a whole lot of stuff, we might as well stamp out pocket.
[11:21] <bigjools> I'm not convinced suite is the right way
[11:21] <wgrant> Why not?
[11:22] <bigjools> I can't articulate it right now but I have a feeling that it'll be hard to make it work with the existing code
[11:22] <wgrant> Most stuff treats (archive, distroseries, pocket) opaquely.
[11:22] <bigjools> OEM still uses pockets, they just don't call them that
[11:22] <wgrant> Except for some views.
[11:23] <bigjools> if we keep the notion of a pocket it will fit in nicely
[11:23] <wgrant> That satisfies all of OEM's use-cases?
[11:24] <bigjools> wgrant: pretty much - they have a devel, staging and "production" pocket
[11:27] <wgrant> But they do insane things with series, right? Or are we going to model that with separate Distributions?
[11:27] <StevenK> bigjools: You already said on the call that it's going to involve touching a lot of code -- to my mind, I still like the idea of wgrant's Suite table.
[11:27] <StevenK> We already call it that anyway
[11:28] <adeuring> jtv: r=me again
[11:28] <jtv> and thanks again
[11:30] <bigjools> yes so a suite table would be (distroseries(FK), pocket(text))
[11:30] <wgrant> For now, yes.
[11:30] <wgrant> It gives us the flexibility to change it in future without changing EVERYTHING.
[11:31] <bigjools> well we need to change everything to use it :)
[11:31] <wgrant> Right, but we need to change just about everything anyway.
[11:31] <bigjools> but the most important thing is to encapsulate this stuff in *one damn place*
[11:31] <wgrant> Yes.
[11:31] <bigjools> then we can gradually refactor existing code
[11:31] <wgrant> Ideally most stuff would just deal with an opaque (archive, suite) object, but that's going to be a bit awkward to model well.
[11:32] <bigjools> very awkward
[11:32] <StevenK> bigjools: So now you agree?
[11:32] <bigjools> this needs to be an evolution not a revolution
[11:32]  * StevenK blinks
[11:32] <bigjools> StevenK: agree with what?
[11:32] <StevenK> The idea of a suite table
[11:33] <wgrant> I guess the DB layout doesn't really matter.
[11:33] <wgrant> As long as the code only sees suites.
[11:33] <bigjools> StevenK: naming is irrelevant, the idea we had was the same
[11:34] <StevenK> bigjools: I was thinking about the multi-inheriance thing while walking to meet Sarah, too. It's fairly easy to model, if we have an id in Suite as well
[11:34] <StevenK> Then the parent/child is a Suite, which solves the pocket crap too
[11:34] <bigjools> StevenK: mmm not sure about that
[11:35] <bigjools> remember that suites are inherited too
[11:38] <StevenK> bigjools: If it's just distroseries as it is now, we
[11:39] <StevenK> just have awkwardness in regards to pockets
[11:39] <bigjools> StevenK: I need to think this through some more, it might do the trick
[11:40] <bigjools> we need pocket/suite blacklisting I think
[11:42] <StevenK> bigjools: This way, that particular suite is not involved in the dependency chain, so it's elegant
[11:43] <wgrant> I think we probably need to do it StevenK's way.
[11:43] <wgrant> Or derived series will have to have a matching set of suites.
[11:43] <wgrant> Or no derivation will happen.
[11:44] <bigjools> I don't see how it would need to match
[11:44] <StevenK> wgrant: So it sounds fine to you?
[11:47] <wgrant> bigjools: What happens if it doesn't match?
[11:47] <bigjools> wgrant: you tell me
[11:48] <wgrant> bigjools: Say OEM wants to have stuff inherited from natty-updates into $CRACKFUL-staging, to test updates before pushing them out to customers.
[11:48] <StevenK> At the moment, they can only dervie all of natty
[11:49] <wgrant> They probably want to inherit natty-release into $CRACKFUL-release.
[11:49] <wgrant> But not natty-updates into $CRACKFUL-updates.
[11:49] <wgrant> Since copying those blindly would be a really bad idea.
[11:50] <bigjools> you've still not explained why they need to match
[11:50] <bigjools> using words like "probably" don't fill me with confidence :)
[11:51] <wgrant> bigjools: I manage a derivative distro. I want to test natty's updates before I push them out to my millions of users.
[11:51] <wgrant> So I have natty-updates inherited by my crackful-proposed.
[11:51] <wgrant> I test them there, then copy to crackful-updates.
[11:52] <wgrant> If there is no explicit mapping, how does LP know to  inherit from natty-updates to crackful-proposed?
[11:52] <wgrant> It will try to inherit from natty-updates to crackful-updates, and all my users will die.
[11:52] <bigjools> this is all based on the assumption that you're inheriting from suites
[11:52] <wgrant> I would hope you would be.
[11:52] <wgrant> Because OEM doesn't want -backports.
[11:53] <bigjools> there are other ways
[11:53] <wgrant> Oh?
[11:53] <bigjools> such as listing all the versions in each suite for a distroseries
[11:54] <wgrant> But I don't care about -backports; I don't want it to show up so I have to dismiss it every time.
[11:54] <bigjools> unless we blacklist it
[11:55] <jtv> StevenK, you probably have better things to do right now but if you don't, got some time to explain something to me?
[11:55] <StevenK> Right, so like I said, in this case the backports suite isn't in the chain, so it doesn't show up
[11:55] <StevenK> jtv: I'm trying to not think about work, but bigjools seems to have solved that already.
[11:56] <jtv> Maybe bigjools can then—it'd be fairer but I saw him in another discussion.
[11:57] <bigjools> wgrant: we don't need mappings for this stuff, we need a mechanism to say put that source (or sources) from that suite into this suite.  Job done.
[11:57] <bigjools> but, I would like to explore all ideas
[11:57] <bigjools> having many options is rarely bad
[11:58] <wgrant> bigjools: I think we probably need defaults, and definitely need to be able to exclude some suites.
[11:58] <bigjools> exclusion, definitely
[11:58] <wgrant> Launchpad knows it's a security update, but it still makes me explicitly put it in -security :(
[11:58] <StevenK> jtv: So, need what explained?
[11:59] <bigjools> wgrant: we need to get more use cases, I'm not convinced (yet)
[11:59] <jtv> StevenK: thanks.  The question is what publish-ftpmaster should do when it's moving $distscopyroot/dists to $archiveroot/dists.new and there's an existing dists.new in the way.
[11:59] <jtv> Or rather, that's what triggered the question.
[12:00] <StevenK> I don't think that should happen.
[12:00] <jtv> Probably not, no.
[12:00] <jtv> But weird things can happen.
[12:00] <bigjools> jtv: doesn't the cleanup prevent that?
[12:01] <jtv> In principle, yes.  But what if the cleanup itself fails?
[12:01] <jtv> Yet with strange aeons, even Death may die.
[12:01] <bigjools> then we need a cleanup cleanup
[12:01] <bigjools> :)
[12:01] <bigjools> at some point we have to rely on shit working
[12:01] <bigjools> I suspect that overwriting dists.new is never wrong
[12:02] <jtv> That is exactly what I'd like to know.
[12:02] <jtv> Plus, what exactly is the role of distscopyroot?
[12:02] <bigjools> provided there's transactional integrity
[12:02] <jtv> (There isn't, but we try to approximate it.)
[12:02] <wgrant> jtv: The dists copy needs to be outside archiveroot when the mirror sync is triggered.
[12:02] <bigjools> well I mean, the publisher may set the source's status to PUBLISHED, it's there in dists.new, commit() and then boom....
[12:02] <bigjools> we're fucked
[12:03] <wgrant> jtv: And we need to keep a copy of dists or we're copying it every time, and that's slow.
[12:03] <jtv> wgrant: okay, but what is it _for_?  What does it hold?
[12:04] <LPCIBot> Project devel build #621: FAILURE in 32 min: https://lpci.wedontsleep.org/job/devel/621/
[12:04] <StevenK> Oh dear
[12:05] <StevenK> Ah. Slave went bang
[12:06] <jtv> At this time of day, the domain name almost looks like a gentle reminder to StevenK not to bite off more than he can chew.  :-)
[12:06] <wgrant> jtv: There are two copies of the dists tree. One lives in the archive root and is used by clients, the other lives in distscopyroot.
[12:06] <jtv> "Sleep?  You thought you were getting time off?"
[12:06] <wgrant> jtv: The publisher shuffles them around to do atomic updates.
[12:07] <jtv> And the dists directories dance back and forth between these two?
[12:08] <deryck> Morning, all.
[12:08] <jtv> hi deryck
[12:08] <bigjools> wgrant, StevenK: the other problem we have is that we want to treat some derived distros as nothing but overlays on the parent.  (like PPAs) which means we need to only publish its changes from the parent
[12:09] <wgrant> bigjools: Oh, that's in scope for the initial implementation? :.
[12:09] <wgrant> *:/
[12:09] <wgrant> I didn't realise that.
[12:09] <bigjools> wgrant: it might be, it depends on what jml decides as chief strategerist :)
[12:10] <wgrant> I think that's probably not going to be doable, but we'll see.
[12:10] <wgrant> Maybe with excessive pinning.
[12:10] <bigjools> no, no pinning
[12:10] <bigjools> stabby stabby
[12:10] <bigjools> we need 2 ways of deriving basically
[12:10] <bigjools> copy, or overlay
[12:10] <wgrant> An overlay isn't safe without pinning.
[12:11] <bigjools> why?
[12:11] <wgrant> You apply an important patch to $package.
[12:11] <rvba> bigjools: by pinning you mean stick with a specific version?
[12:11] <wgrant> Ubuntu applies a security update.
[12:11] <bigjools> (I can guess but..)
[12:11] <wgrant> Your users get Ubuntu's version instead.
[12:11] <bigjools> that won't happen
[12:11] <bigjools> because we're overlaying only on the release pocket
[12:12] <bigjools> updates need to go via the overlay
[12:12] <StevenK> Which we can only do if we have Suites :-)
[12:12] <bigjools> rvba: yes, it's an apt config thing
[12:12] <bigjools> StevenK: not necessarily, there are many ways of implemeting it
[12:13] <bigjools> keep an open mind :)
[12:13] <StevenK> But if we do it for the other ...
[12:15] <wgrant> bigjools: Ah, I see.
[12:16] <wgrant> So it relies on having a frozen release pocket. I guess that's a reasonable constraint.
[12:16] <bigjools> wgrant: ah reading the notes from yesterday, what I said can happen and also your way.  I need to ask cody-somerville how they cope with that
[12:18] <bigjools> wgrant: it seems they rely on reporting
[12:18] <bigjools> hmm
[12:19] <bigjools> wgrant: they also create snapshots of release plus some -updates and work off that
[12:22] <wgrant> I was about to ask for a forward.
[12:22] <wgrant> Thanks.
[12:23] <bigjools> :)
[12:23] <bigjools> not sure why I didn't do that already ...
[12:26]  * maxb is intrigued to see how https://answers.launchpad.net/launchpad/+question/152080 gets answered
[12:26] <jtv> bigjools: I may be way behind the discussion on this but… is "multiple distroseries inheritance" really what's needed?  The notes and diagrams I've seen so far seem to be more about a separate "pull PPA packages into distroseries" feature.
[12:27] <bigjools> jtv: it's both
[12:28] <bigjools> jtv: they manage composite archives
[12:29] <wgrant> maxb: "Where is the documentation targeted to people who're willing to run their own instance of lp?" suggests a misunderstanding of the situation :(
[12:35] <jtv> bigjools: I'll have to read in more detail later then…  It's sad that evidently we thought we were ready-to-code but we weren't.  I wonder if we could reduce this kind of risk with closer "customer" involvement once coding begins.  Or would that just open the bikeshed?
[12:39] <wgrant> jtv: Ready to code what?
[12:39] <wgrant> jtv: The stuff that is already done?
[12:40] <jtv> No, the LEP that we're implementing.
[12:40] <jtv> As a whole.
[12:40] <bigjools> we are ready to code the LEP
[12:41] <bigjools> and were ready
[12:41] <wgrant> But the LEP needs extension.
[12:41] <bigjools> yes
[12:41] <jtv> But these are new requirements, right?
[12:41] <wgrant> Perhaps an extension.
[12:41] <bigjools> the changes don't affect existing plans
[12:44] <wgrant> Yeah, I think the existing Linaro plan is reasonably compatible.
[12:45] <jtv> I must be missing something.  What I understood was happening was that we were supposed to include these extra requirements into the initial delivery of the functionality in the LEP.
[12:46] <jtv> If that's not the case, and existing plans are not affected, then that sets my mind at ease.
[12:49] <bigjools> jtv: right, we're not changing the immediate deliverables, just extending the scope
[12:49] <jtv> So I can think of it as an extra LEP building on the current one?
[12:51]  * jtv was traumatized by an extended scope as a child and still flinches when he sees them in the street
[12:51] <bigjools> jtv: O_o
[12:52] <jtv> Okay, okay, I made that up.
[12:52] <bigjools> jtv: yes, I may even do a sidebar LEP :)
[12:52]  * jtv pictures one of those old bibles with columns of notes about the notes
[12:54] <wgrant> I'd always understood that this LEP was just the first phase, to get Linaro and possibly Ubuntu onto the new system.
[12:54] <wgrant> With OEM requiring more complex rules and privacy, in at least two later phases.
[12:54] <wgrant> Oh seriously.
[12:54] <wgrant> More conflicts.
[12:54]  * wgrant fixes.
[12:55] <bigjools> did we land any schema patches yet? if not just merge the damn thing to devel
[12:56] <wgrant> Yeah, one was landed a few hours ago :(
[12:56] <wgrant> I may land the rev before, though.
[12:56] <bigjools> yes
[12:56] <wgrant> I meant to do that, but too many distractions this week :/
[12:56] <bigjools> the conflicts I fixed were very odd
[12:56] <wgrant> There's a lot of strucsub-related ones.
[12:56] <wgrant> But this one is me+jtv.
[12:57] <jtv> whawhat?
[12:57] <bigjools> the one I fixed was in the distroseries stuff we've been changing but there were no actual conflicts to speak of, it looked like bzr getting it wrong
[12:57] <wgrant> archivepublisher config schema.
[12:57] <jtv> ah that
[12:57] <jtv> yes, sorry, no way around it—I kept mine as far away as I could, which was nowhere near far enough.
[12:57] <wgrant> Yup.
[12:57] <wgrant> Just pqm-submitting now.
[13:00] <wgrant> Yay, I beat buildbot-poller.
[13:00]  * jtv goes looking for a whiteboard to map out the dists directory dance
[13:01] <wgrant> jtv: Is 10383 still qa-bad?
[13:01] <wgrant> I presume not, since you reverted cron.publish-ftpmaster.
[13:01] <wgrant> Ah, not tagged properly.
[13:02]  * wgrant fixes.
[13:03] <bac> hi adeuring
[13:03] <bigjools> I'd really like a simpler page explaining how to go about tagging when we have blocking qa failures
[13:04] <wgrant> bigjools: Meh, I missed a table addition. Can only merge one rev, due to a qa-bad interlocked with the new table.
[13:04] <jtv> wgrant: I'm not sure since it doesn't appear to be documented, but I think the qa-bad tag is appropriate.  I landed the fix with the --rollback option.
[13:04] <bigjools> balls
[13:04] <jtv> wgrant: that also means that that revision is still qa-bad, but that the problem goes away if you also merge the later revision.
[13:04] <wgrant> jtv: Need to tag with bad-commit-NNNNNN as well.
[13:04] <jtv> Argh
[13:05] <wgrant> jtv: This is a bug tag, not a commit tag.
[13:05] <jtv> You mean the bug that was qa-bad
[13:05] <jtv> ?
[13:05] <wgrant> Land with --rollback=NNNNNN, tag original qa-bad bug with bad-commit-NNNNNN
[13:05] <wgrant> (is this documented? not sure. does it work? sort of, most of the time)
[13:06] <jtv> There's a lot of documentation but all it seems to say is to, er, fix it the usual way or something.
[13:06] <adeuring> hi bac
[13:08] <bigjools> wgrant: hmmm I wonder if we could generalise PPAs into "overlay" distributions.
[13:09] <wgrant> bigjools: What does that do?
[13:09] <bigjools> wgrant: the difference is only in the sources.list we use to build with, really
[13:10] <bigjools> (ignoring the components thing for now)
[13:10] <wgrant> bigjools: Right, so we need to improve archive deps, I guess.
[13:10] <bigjools> yeah
[13:10] <bigjools> that's reasonably well encapsulated
[13:11] <wgrant> We really need to gather all the Ubuntu/PPA/OEM/Linaro/whatever use-cases and work out the sorts of archives we need...
[13:11] <wgrant> The post-Linaro LEP is probably going to have to be fairly revolutionary.
[13:11] <bigjools> we'll never finish :)
[13:12] <jtv> mv red-squad soyuz
[13:25]  * henninge-lunch relocates
[13:40] <jtv> bigjools, wgrant: here's the migration pattern of the dists/dists.new/dists.old directories: http://people.canonical.com/~jtv/publish-ftparchive-dists.png
[13:41] <jtv> Oh, that's still without the cleanup.
[13:42] <wgrant> jtv: That looks fine for a normal run.
[13:47] <jtv> Hmm… I wonder if this isn't a bug in the original script.  If DONE_PUB, then all that the cleanup method does is rename {archiveroot}/dists.old to {distscopyroot}/dists.
[13:48] <jtv> But the next step after setting DONE_PUB, in this simplified diagram, is to move {archiveroot}/dists.old away.
[13:48] <jtv> In fact, that _is_ the next step.
[13:50] <jtv> I can't come up with any set of circumstances where that would make sense.
[13:50] <wgrant> jtv: Indeed, that is a bit odd.
[13:50] <wgrant> Perhaps it is just for consistency.
[13:50] <jtv> (Not including situations like "the disk is flapping" where it _might_ make sense but there's no real way of getting a grip anyway)
[13:51] <jtv> I think the script just grew out of hand.
[13:51]  * jtv goes to update the online version of that diagram
[13:53] <jtv> wgrant: if you refresh now, you'll see a version with the beginning of the cleanup routine marked.
[13:53] <jtv> This is all based on the old shell version, to avoid clay feet.
[13:58] <jtv> wgrant: I think this also means that the publish-distro run-parts scripts shouldn't need to know about $DISTSROOT.new; the DISTSROOT parameter should include the .new part.
[14:00] <jtv> It's nice to know that ultimately, $DISTSCOPYROOT/dists and $ARCHIVEROOT/dists change places.
[14:00] <jtv> That was not very obvious from the script.
[14:01] <deryck> adeuring, abentley -- I'm on, just mic not working, I guess.
[14:03] <deryck> adeuring, abentley -- I'm guessing you guys can't hear me?
[14:03] <jtv> wgrant: does this mean that $DISTSCOPYROOT/dists is really just a backup of the previous state?
[14:04] <deryck> adeuring, abentley -- let me relog.
[14:06] <wgrant> jtv: Right, the idea is to have two copies for speed.
[14:06]  * wgrant grumbles.
[14:08] <benji> PQM is unhappy and I can't tell if I'm the cause.  At one point (about 5 hours ago) it was complaining about a conflict in lib/lp/registry/javascript/tests/test_structural_subscription.js but it stopped and is now complaining about lib/canonical/config/schema-lazr.conf.
[14:08] <benji> the former is mine, the latter is not
[14:08] <wgrant> 23:06:38 < wgrant> jtv: Right, the idea is to have two copies for speed.
[14:08] <jtv> ah thanks
[14:08] <wgrant> Since it can be pretty big.
[14:09] <wgrant> (it has some CD images and stuff)
[14:10] <jtv> Ah, now I see how the rsync -H works.
[14:10] <jtv> It's an Ouroboros.  The only copying that ever happens is the rsync (with hard-linking) back to another instance of itself.
[14:11] <jtv> Now,
[14:11] <jtv> where did all you zombies come from?
[14:12] <jtv> (If the reference wasn't obvious: Robert Anson Heinlein, "All You Zombies," short story)
[14:14] <LPCIBot> Project windmill build #150: FAILURE in 1 hr 2 min: https://lpci.wedontsleep.org/job/windmill/150/
[14:25] <gary_poster> benji, thanks for pqm conflict update.  why don't we see complaints on canonical-launchpad?
[14:26] <benji> gary_poster: tell me what "canonical-launchpad" is and I'll find out ;)
[14:27] <StevenK> benji: The mailing list
[14:27] <gary_poster> right
[14:28] <wgrant> PQM does spam canonical-launchpad@ if there's a conflict.
[14:28] <wgrant> Lots.
[14:28] <gary_poster> :-)
[14:28] <benji> I got a message about it an hour ago to launchpad@lists.canonical.com
[14:28] <wgrant> I resolved that one
[14:29] <wgrant> I possibly should have replied, but it should be reasonably obvious that it's resolved if your inbox isn't flooding every 5 minutes :/
[14:29] <gary_poster> heh
[14:29] <benji> wgrant: awesome, thanks
[14:29] <gary_poster> the only way it is not obvious is if you are not familiar with that aspect of the current landing machinery
[14:29] <gary_poster> which seems conceivable
[14:29] <wgrant> I resolved two conflicts this evening: the earlier was strucsub JS tests, which I'm not sure I did correctly, since I don't have Windmill working locally. You might want to check that out.
[14:29] <wgrant> Heh.
[14:29] <wgrant> Yeah, true.
[14:30] <gary_poster> :-)
[14:30] <gary_poster> ok thanks wgrant
[14:30] <benji> then my question becomes, why isn't my revision in devel or listed on https://devpad.canonical.com/~lpqateam/qa_reports/deployment-stable.html
[14:30] <wgrant> I'd EOD'd and mostly wanted the spam to stop, but it seemed OK.
[14:30] <wgrant> benji: Which revision?
[14:30] <wgrant> benji: deployment-stable.html should update within 1.5 hours of your rev passing buildbot, after it's on qas.
[14:31] <gary_poster> benji, since those test additions are yours, could you take a glance as he suggests?  I'm going to run make an mp before Graham leaves
[14:31] <benji> bzr+ssh://bazaar.launchpad.net/~benji/launchpad/add-edit-tests-2 r12775
[14:31] <wgrant> It wasn't testfixed out?
[14:31] <wgrant> Or conflicted out?

[14:32] <wgrant> Check for PQM failure emails that are to you, not launchpad@
[14:32] <jtv> bigjools, may I invite you to look at how the dists directories shuffle about in publish-ftparchive?  http://people.canonical.com/~jtv/publish-ftparchive-dists.png
[14:32] <benji> I got a direct email about the conflict in lib/lp/registry/javascript/tests/test_structural_subscription.js
[14:33] <wgrant> benji: That was probably rejecting your branch.
[14:34] <benji> ok, so I need to freshen my branch from devel so I can resolve the conflict and then use bzr lp-land to get it in? (or do I need to do the whole ec2 land dance again?)
[14:34] <wgrant> No point ec2ing it, since Windmill is turned off now anyway.
[14:34] <wgrant> Is the branch targetted at devel or db-devel?
[14:36] <benji> devel
[14:36] <adeuring> abentley: r0me
[14:37] <wgrant> benji: Merge, hopefully you'll see the conflict. But it will probably conflict when it tries to merge to db-devel.
[14:37] <abentley> adeuring: marse1lles
[14:37] <wgrant> If you're just adding tests, should be easy enough to resolve.
[14:40] <benji> wgrant: got and fixed the conflict
[14:41] <wgrant> Great.
[14:41] <wgrant> Commit, push, lp-land, hopefully PQM won't hate you too much.
[14:41] <wgrant> I shall expect a conflict in the morning.
[14:41] <rvba> adeuring: Hi! Could you take a look at this MP? https://code.launchpad.net/~rvb/launchpad/dds-link-to-derivedseries/+merge/56931
[14:41] <adeuring> rvba: sure
[14:41] <rvba> adeuring: thx.
[14:48] <henninge> abentley: did you run the one-off merging script? I can't remember.
[14:50] <abentley> henninge: No.  I included it in the RT: https://portal.admin.canonical.com/45152
[14:51] <henninge> abentley: ah no, that's not what I meant.
[14:51] <abentley> henninge: what did you mean?
[14:52] <henninge> abentley: wasn't there a merge run of already existing package links before you started coding the jobs?
[14:52] <abentley> No.  Jobs were the first thing I worked on.
[14:52] <abentley> Then I worked on a one-off script.
[14:53] <abentley> I requested that one-off script to be run in RT 45152.
[14:54] <henninge> abentley: ok, thanks
[14:54] <LPCIBot> Project windmill build #151: STILL FAILING in 40 min: https://lpci.wedontsleep.org/job/windmill/151/
[14:57] <deryck> adeuring, r=me, with some minor comments.
[14:57] <adeuring> deryck: thanks!
[14:58] <abentley> henninge: I determined that it didn't make sense to do the merge until the jobs were being created (or we would miss some Packagings).  And since the release, I've focused on JS instead of getting these scripts run.
[14:59] <henninge> abentley: but the jobs only get created when new link are created. But what about the links that already existed when message sharing was introduced?
[14:59] <abentley> henninge: That's what the one-off script fixes.
[15:00] <henninge> ah
[15:00] <henninge> sorry, misunderstood that.
[15:01] <abentley> henninge: np
[15:01] <henninge> abentley: so as of now we have virtually no upstream message sharing going on.
[15:01] <henninge> until that script is run
[15:01] <abentley> henninge: correct.
[15:01] <henninge> dpm: ^^^
[15:02] <henninge> dpm: we are in a transitional stage here. So some things might not yet work as expected because the data has not been migrated yet.
[15:03] <henninge> dpm: but that would not explain what happened to synapatic.
[15:03] <wgrant> henninge: What's happened? It's not importing POs from the package?
[15:04] <henninge> wgrant: no, it somehow reverted to old (wrong) translations it seems.
[15:04] <wgrant> Huh.
[15:04] <henninge> bug 740153
[15:04] <_mup_> Bug #740153: Translations skewed for synaptic <synaptic> <ubuntu> <upgrade> <synaptic:Triaged> <synaptic (Ubuntu):Invalid> < https://launchpad.net/bugs/740153 >
[15:05] <bigjools> jtv: looked at the png.
[15:05] <jtv> Enough colours?
[15:06] <adeuring> rvba: could you please add unit tests for Distribution.derivatives and DistroSeries.detDerivedSeries()?
[15:06] <rvba> adeuring: ok, will do.
[15:07] <jtv> bigjools: This should help a lot with figuring out the various error paths.  One question I still have though: is $DISTSCOPYROOT/dists used for anything besides this flow?  Because if not, I'd favour rsync'ing in the new stuff _before_ starting to shuffle directories.
[15:07] <adeuring> rvba: Especially, I'd be curious about the new clause "DistroSeries.distributionID != self.id" and the related clause in the distroseries class
[15:07] <adeuring> rvba: other than that, your branch looks goos
[15:07] <adeuring> good...
[15:08] <rvba> adeuring: a series is a derivation if and only if it has a parent_series and the distribution of the parent_series is not the same as it's own distribution
[15:08] <dpm> henninge, thanks.  And the other question: on https://translations.launchpad.net/synaptic/main/+pots/synaptic/ca/5/+translate - why did the "In Ubuntu" suggestion did not just get accepted if permissions are correct? Is this how it is supposed to work? I.e. translations done downstream will need to be reviewed and accepted again before they make it to upstream? - I understand that some things might not yet work as expected, I'm just trying to understan
[15:08] <dpm> d the expected behaviour
[15:08] <rvba> adeuring: hence the clause.
[15:08] <adeuring> rvba: right. what I mean is: A test that this clause is necessary would be great
[15:09] <bigjools> jtv: NFI.  :(
[15:09] <rvba> adeuring: ok, got it.
[15:09] <bigjools> jtv: this stuff is seriously crusty
[15:09] <jtv> bigjools: the diagram also reveals that the cleanup routine really doesn't do anything once the done_pub point has been passed.
[15:09] <bigjools> no
[15:09] <wgrant> jtv: Do what you want with DISTSCOPYROOT.
[15:10] <jtv> wgrant: thanks
[15:10] <henninge> dpm: the expected behavior is: If you, being a translator for Catalan in both the Ubuntu package and the upstream project, then any translation you do in Ubuntu will be mirrored to upstream *if* they are identical already.
[15:10] <henninge> dpm: if they are different, they will stay different.
[15:10] <jtv> bigjools: "no it doesn't do anything" or "no you're totally wrong, you simpleton"?
[15:10] <bigjools> jtv: sorry, was agreeing but in a terse way :)
[15:10] <henninge> dpm: in that case "different" is that upstream is untranslated.
[15:10] <bigjools> E_TOOMANYSIMULTANEOUSCONVERSATIONS
[15:11] <jtv> bigjools: It's okay to say "simpleton"
[15:11] <jtv> Ah, OK
[15:11]  * jtv trundles off to apply his newfound insight
[15:11] <henninge> dpm: Although that depends on what the Ubuntu translation was before that.
[15:12] <henninge> dpm: do you know if it was empty, too?
[15:12] <dpm> henninge, what's the rationale on having to review untranslated strings with translations coming from downstream? I thought that they'd flow in transparently to automate the process. This is no different than having global suggestions and having to click them
[15:13] <dpm> henninge, I'm not sure what you mean by empty, it's untranslated upstream:
[15:13] <dpm> https://translations.launchpad.net/synaptic/main/+pots/synaptic/ca/5/+translate
[15:13] <henninge> empty = untranslated.
[15:13] <henninge> dpm: that translation was done long before this mechanism came into place.
[15:14] <henninge> dpm: I think that they are not linked yet.
[15:14]  * henninge tries on qastaging
[15:17] <henninge> dpm: they seem to be linked. I could change both sides by entering a translation upstream.
[15:18] <henninge> dpm: which is the expected behavior when upstream is untranslated -> the first translation overides an existing Ubuntu translation
[15:22] <dpm> henninge, yeah, I can understand that, but what's up with the untranslated behaviour? Should ubuntu translations not flow to upstream if the message is linked, upstream is empty and permissions are right?
[15:24] <henninge> dpm: they should but it may be a bug. OTOH this is done when the translation is entered and this one was entered long before upstream message sharing existed.
[15:27] <dpm> henninge, ok, yeah, that helps, I was just trying to understand the expected behaviour. Do you want me to file a bug on that one?
[15:27] <henninge> dpm: well ...
[15:27] <henninge> I am not sure.
[15:28] <henninge> dpm: The situation is this: normally both the sourcepackage and the uspstream translation would start out untranslated.
[15:28] <henninge> dpm: so they are identical
[15:29] <henninge> dpm: someone with permission translates in Ubuntu and that gets mirrored to upstream.
[15:29] <henninge> dpm: let me try that out real quick
[15:29] <dpm> ok
[15:29]  * henninge translate on some obscure untranslated language
[15:29] <henninge> eo?
[15:29] <henninge> :)
[15:30] <henninge> nah, that has translations.
[15:30] <henninge> nds?
[15:30] <henninge> De snakt platt!
[15:31] <henninge> ksh
[15:31] <henninge> Yeah, nobody speaks Kölsch ... ;)
[15:33] <henninge> dpm: nope, not working :(
[15:34] <henninge> dpm: uh oh, tracking does not work ...
[15:34] <LPCIBot> Yippie, build fixed!
[15:34] <LPCIBot> Project windmill build #152: FIXED in 39 min: https://lpci.wedontsleep.org/job/windmill/152/
[15:35] <dpm> henninge, oops
[15:36] <henninge> dpm: yeah, it only works from upstrem to downstream.
[15:36] <henninge> dpm: when upstream and Ubuntu are identical, an change in upstream will be reflected in Ubuntu.
[15:37] <henninge> but not the other way round although I have permissions on both sides by virtue of being a rosetta admin.
[15:37] <henninge> dpm: but it may well be that the check for admin priviliges is just missing.
[15:37] <henninge> dpm: please file a bug so that we look at that more closely.
[15:38] <dpm> henninge, yessir!
[15:38] <dpm> thanks for your help
[15:38] <henninge> including the behavior when untranslated message are involved.
[15:38] <henninge> dpm: BUT
[15:39] <henninge> dpm: if upstream is untranslated and Ubuntu is translated then we could assume that upstream chose no to accept the Ubuntu translation.
[15:39] <henninge> dpm: we might want to keep them different then.
[15:40] <henninge> dpm: OTOTH that situation would also arise if somebody who is only an Ubuntu translator entered the first translation in Ubuntu.
[15:41] <henninge> dpm: In that case an override by the might be advised. But we can't tell from the situation.
[15:41] <henninge> In general it might be a better idea to be mare accepting then rejecting about translations.
[15:47] <dpm> henninge, yeah, I'd second that in this particular case (i.e. being more accepting)
[16:18] <henninge> How do I run javascript unit tests?
[16:19] <sinzui> henninge:  --layer=Windmill to explicitly add that layer to the testrunner
[16:20] <henninge> hm, RegistryWindmillLayer just tried to came up but died horribly ...
[16:20] <wgrant> henninge: Doesn't work on Natty at the moment.
[16:20] <wgrant> (our mozrunner doesn't like Firefox 4)
[16:20] <henninge> wgrant: ah yes, that's what it looked like
[16:21] <henninge> thanks
[16:21]  * henninge will have to let ec2 sort it out.
[16:21] <wgrant> henninge: ec2 doesn't run Windmill any more. But Hudson will run them in a separate job once it lands.
[16:21] <henninge> argh
[16:21] <henninge> wgrant: thanks
[16:22] <sinzui> That reminds me that I want to get pocket-lint to use seed since most users will have that installed
[16:26] <henninge> sinzui: can you still run windmill tests?
[16:26] <henninge> i.e. no natty yet?
[16:26] <sinzui> I have been on natty since 2010
[16:27] <sinzui> There was a very painful day with no menus
[16:27] <henninge> ouch
[16:28] <henninge> oh well, I don't expect to have broken anything anyway.
[16:28]  * henninge goes lp-propose
[16:28] <sinzui> henninge: do you need to test windmill page integration or do you want to test yui unittests
[16:29] <henninge> sinzui: yui unittests AFAICT
[16:29] <sinzui> henninge: yui unit tests work so I just open that page to verify they pass
[16:29] <henninge> sinzui: the html page that goes with the test?
[16:29] <sinzui> henninge: you can open the test page (the actually harness) in any browser
[16:29] <sinzui> henninge: that is right
[16:29] <henninge> cool
[16:29] <sinzui> it is faster than the test suite startup too
[16:31] <henninge> passed!
[16:31] <henninge> thanks sinzui
[16:32] <sinzui> henninge: I am glad to be of service. I write js with that test page open. That lets me do test-driven development
[16:42] <benji> bac: I have some structural subscription JS test sprucing up for review: https://code.edge.launchpad.net/~benji/launchpad/fix-ss-test-lint/+merge/56974
[16:42] <bac> benji: on it
[16:42] <benji> thanks
[16:50] <bac> benji: approved with my condolences.  thanks.
[16:50] <rvba> adeuring: I spoke with bigjools about this ... and I've added a test (the test for getDerivedSeries existed already) ... but I deliberately did not check for the condition (;-)) ... because of 754750. We need to refactor the whole parent/child modelling and deal with existing wrong data. So you're right, the condition should not be needed, but it is for now because of this wrong data (i.e. parent/child relationship where none should b
[16:50] <rvba> e there).
[16:51] <benji> bac: heh :)
[16:52] <rvba> adeuring: I'll have to EOD soon (and so do you I guess) so maybe you can check it out next week if it's ok with you.
[16:52] <adeuring> rvba: np Ii understand the issue with bug 754750. Let me just have a look at the diff again
[16:52] <_mup_> Bug #754750: Distroseries' parent_series attribute is misleading. <derivation> <Launchpad itself:New> < https://launchpad.net/bugs/754750 >
[16:54] <adeuring> rvba: r=me. thanks for the additional test and the comments!
[16:55] <rvba> adeuring: great, thanks for spotting this. Have a nice we!
[16:55] <adeuring> rvba: thanks! have a nice weekend too!
[17:04] <henninge> adeuring, bac: Is either of you up for a JS review?
[17:05] <adeuring> henninge, bac: I am a bit tired (worked quite long yesterday...) so... bac?
[17:05] <henninge> adeuring: time for the weekend, I guess. ;)
[17:05] <adeuring> henninge: yeah :)
[17:16] <sinzui> bac: will you have time to review https://code.launchpad.net/~sinzui/launchpad/mlist-sync-0/+merge/56468
[17:17] <sinzui> bac: do not hesitate to ask questions if you do. It took me a few days to understand what the original test was trying do
[17:35] <SpamapS> lifeless: interesting about rabbit. I'm starting to think the only sane way to run it on a roaming machine is inside a vm/container
[18:43] <bac> sinzui: yes, i'll look now
[18:50] <LPCIBot> Yippie, build fixed!
[18:51] <LPCIBot> Project devel build #622: FIXED in 5 hr 23 min: https://lpci.wedontsleep.org/job/devel/622/
[19:35] <bac> hi sinzui can you explain what test_staging_sync_list_without_team is meant to do?
[19:35] <sinzui> The team was deleted, the mlist and archive are still on the system
[19:36] <sinzui> the mlist+archive do not create an oops
[19:37] <renata> hello everybody! I am new at working with launchpad.. I am starting to work with launchpadlib and the API. I was wondering if anyone could tell me if there is a specific API function to access distribution's packages.
[19:37] <sinzui> ^ bac
[19:37]  * bac looks
[19:37] <renata> for example, I would like to obtain all ubuntu packages..
[19:37] <renata> is it possible to do so only by giving a release name like "natty" ?
[19:38] <bac> sinzui: so the fact we only see team-1 means the other is not there...therefore the test passed
[19:40] <sinzui> renata: https://launchpad.net/+apidoc/devel.html distribution has two methods to get source packages,distro series has some too
[19:41] <sinzui> bac: correct
[19:41] <sinzui> bac: I can update the test to state that mlists+archives without a team are ignored
[19:42] <bac> sinzui: yeah, some sort of explanation would be good.  most people reading that test aren't going to have a clue, assuming someone besides you ever reads it again.
[19:42] <bac> sinzui: and you have the most interesting typo on the copyright year!
[19:43] <bac> usually it is an off by one error, not an order of magnitude
[19:43] <sinzui> renata: source package releases and binary package releases are not supported. You can get source_package_publishing_history from the source_package to learn about versions
[19:44] <sinzui> bac: I claim the cough syrup made me do it
[19:45] <bac> sinzui: in other tests on ec2 we've seen spurious failures with shutil.rmtree
[19:46] <sinzui> hmm
[19:46] <renata> sinzui tank you! I was just trying to find a list of packages to a certain distribution. I think that may do it :)
[19:46] <bac> sinzui: would it be safe to add the 'ignore_errors' option
[19:46] <bac> sinzui: the other tests fail when the dir is not empty
[19:48] <sinzui> bac: I will look into it. The other tests will not fail of data is left behind. since each is in their own tmp dir
[19:49] <sinzui> oh, well the archive setup is not in tmp, so I will add the 'ignore_errors option
[20:11] <renata> sinzui, thanks a lot. your information was very useful!
[20:13] <sinzui> np
[20:17] <benji> bac: I have another JS MP for you, this one a bit shorter and with slightly more meat: https://code.edge.launchpad.net/~benji/launchpad/bug-750573-move-overlay/+merge/56999
[20:17] <bac> benji: ok
[20:26] <lifeless> benji: hi
[20:26] <lifeless> benji: I'd like pointers at how the api server side glue goes about making a batchnavigator for a collection
[20:27] <lifeless> benji: I want to glue lazr.batchnavigator 1.2.3's shiny new range_factory in
[20:28] <benji> lifeless: hmm, let me see if I can come up with some pointers
[20:28] <bac> benji: thanks for the fixes.  however, when i click on the 'submit' i still see a two-stage disappear on the overlay
[20:28] <bac> benji: it is working for you w/ff4?
[20:30] <lifeless> benji: if what I'm talking about sounds mysterious, I can point you at the BN changes, or describe the concept directly
[20:31] <benji> bac: yeah, it looks fine in FF4 and Chromium; are you seeing the overlay dissapear and then the filter controls or the other way around?
[20:31] <benji> unfortunately I think we have to choose one or the other, and I /think/ the former is slightly better
[20:32] <bac> benji: other way.  when fully opened, the accordion, et al, go away leaving a smaller overlay which then disappears
[20:33] <benji> lifeless: I believe I understand the root of your request.  You've been working on removing the explicit start indices from batching requests.
[20:33] <bac> benji: i see the same with FF4 on os x.
[20:33] <benji> bac: darn, that's what I was trying to eliminate; I may need to find a way to run this on FF3 so I can reproduce
[20:33] <bac> the cancel button causes it to go away cleanly but submit shows as above
[20:34] <lifeless> benji: yes, well, adding data so the query doesn't use OFFSET; the start index still stays for cosmetics
[20:34] <benji> bac: oh! I can fix that, one second
[20:37] <benji> bac: ok, it'll take more than a second, I'll let you know
[20:39] <benji> lifeless: I think lazr/restful/_resource.py line 640 is what you're looking for
[20:41] <lifeless> great
[20:41] <lifeless> benji: now, leonard said that in url generation
[20:41] <lifeless> just changing the next/prev links should be enough
[20:41] <lifeless> or do I need to change some compilation metadata too ?
[20:43] <benji> lifeless: That's a good question; I don't know off the top of my head.  Let me see if I can find anything out real quick.
[20:48] <benji> lifeless: It looks like just changeing the batch links should work.
[21:01] <lifeless> benji: wicked
[21:10] <benji> bac: I believe I've fixed the two-step overlay hide problem, the diff is up to date on the MP
[21:11] <bac> benji: cool, grabbing it now
[21:14] <bac> benji: works great!
[21:14] <benji> cool!
[22:09] <cr3> hi folks, what's the launchpad practice for naming classes containing acronyms? HTTPFoo or HttpFoo?
[22:09] <lifeless> yes
[22:09] <lifeless> I suspect we have both
[22:09] <lifeless> If I was guessing - e.g. to grep, I'd expect HTTP
[22:10] <cr3> lifeless: grepping around has indeed returned both practices, which is understandable for such a large code base, any reason to use one practice over the other?
[22:11] <cr3> it just occured to me, which is strange since I've been doing this for a while, that two acronyms in a row would suck with all caps: APIWSGIWTF
[22:11] <lifeless> I think changing the case on acronyms is more surprising than not
[22:13] <lifeless> OTOH theres little excuse for more than ~ 2 classes with the name of a given standard in them
[22:13] <lifeless> [outside of the actual implementation of said standard]
[23:00] <lifeless> sinzui: still here?