[00:01] <thumper> lifeless: what have you done to edge?
[00:09] <wgrant> It exploded yesterday.
[00:09] <wgrant> Quite spectacularly.
[00:32] <lifeless> wgrant: please try to get a broken wadl now
[00:39] <wgrant> lifeless: I'm not sure this explains everything.
[00:39] <lifeless> wgrant: I'm not sure it explains anything
[00:39] <lifeless> wgrant: but, are the symptoms gone
[00:40] <wgrant> the last run succeeded. Doing it again to confirm...
[00:47] <poolie> lifeless: oh, did you land those flag scopes? or someone else?
[00:47] <poolie> either way, that's great
[00:48] <lifeless> I'm been extending it a little, yes.
[00:48] <lifeless> the pageid one will land today
[00:48] <lifeless> once we climb out of this downtime hole
[00:49] <lifeless> wgrant: ?
[00:52] <wgrant> lifeless: Seems happy now.
[00:52] <lifeless> great
[00:52] <wgrant> I can't say for sure, but three runs succeeded.
[00:52] <lifeless> so running edge on 2 appservers with a backlog of 160 requests is a bad idea
[00:52] <wgrant> Heh.
[00:52] <wgrant> Remarkable.
[01:06] <poolie> that's great
[01:06] <poolie> i hope to lower my queue enough to do the edit gui
[01:07] <poolie> it is in a branch though, if i don't get to it
[01:10] <lifeless> poolie: I filed a bug to track it; if you're not actively on it I can unassign you
[01:10] <lifeless> I needed a reference point for a code comment IIRC
[01:10] <poolie> i'm not actively on it but i feel i ought to close a few bzr bugs first
[01:11] <poolie> i'm hoping/aiming for quite a productive week here
[01:16] <wgrant> lifeless: I threw some example error messages on bug #636713.
[01:18] <lifeless> poolie: could you perhaps link your branch to the bug?
[01:18] <poolie> perhaps i could
[01:19] <lifeless> thanks:)
[01:21] <poolie> nice summary :)
[01:21] <poolie> i mean description
[01:21] <poolie> bug 636192
[01:22] <lifeless> poolie: ;)
[01:51] <poolie> no bot?
[01:53] <lifeless> _mup_: yo
[01:53] <lifeless> tumbleweeds
[02:26] <lifeless> wgrant: 41 /    9  Distribution:+search third top oops in prod
[02:29] <wgrant> lifeless: Is that hard/soft?
[02:30] <wgrant> So the third top OOPS only occurs 41 times a day? That's not bad.
[02:34] <lifeless> on sunday
[02:40] <wgrant> Ah.
[02:42] <lifeless> still, edge is ~ 13 seconds, time to shrink lpnet a little
[03:21] <lifeless> thumper: https://code.edge.launchpad.net/~lifeless/lp-production-configs/timeouts/+merge/35239
[03:41] <lifeless> thumper: ^ when you have a sec ;)
[03:48] <stub> thumper: So both the existing indexes *might* be useful - you can look up BranchRevision indexes by revision using the (branch, revision) index, but I think it is more efficient to look them up using the (revision, branch) index.
[03:49] <stub> thumper: It would be nice if we can drop both unique indexes and use just the primary key index though, as inserts will be faster and we save disk space. And maybe it will even be faster as we are more likely to have the index in PG shared memory.
[03:50] <stub> thumper: So if we drop both now, we can try adding one back later if there are read performance issues. Or we can leave things with two indexes for now, and try dropping one later.
[03:51] <stub> Do we ever need to see which branches a particular revision is in?
[03:55] <thumper> yeah...
[03:55] <thumper> stub: but this is one of the things I'm wanting to either factor out
[03:55] <thumper> stub: or find another way
[03:56] <stub> I'm thinking we should keep the patch as you have it, and just be aware that if Revision -> Branch traversal is slow we might be able to fix it with a new index.
[03:56] <thumper> stub: I hadn't thought of that reason for the constraint
[03:57] <stub> I can't really guess which will be fastest for reads, and no index is faster for inserts and saves disk
[03:57] <thumper> stub: *most* of the time we are going from the branch to the revisions
[04:24] <lifeless> \o/ less bare excepts
[05:05] <lifeless> stub: I just remembered I wanted to ask you about layer teardowns.
[05:05] <lifeless> stub: looks like zope.app.testing.functional/FunctionalTestSetup.tearDownCompletely() should let use teardown some more layers?
[05:07] <stub> Sounds like it would, yes. That didn't exist before.
[05:07] <lifeless> ok
[05:07] <lifeless> rs= you to do it ?
[05:07] <stub> Maybe make the whole fork-a-new-process thing unnecessary too
[05:07] <lifeless> yes
[05:07] <lifeless> thats a good 2 or more minutes per test run
[05:07] <lifeless> on its own
[05:07] <stub> rs=stub. If the tests pass, great.
[05:22] <wgrant> lifeless: Anything more to add to the bug?
[05:30] <lifeless> stub: could you do me a small favour
[05:31] <lifeless> ttps://devpad.canonical.com/~stub/dbr/last-3-hours.txt suggests to me that we've reduced load on the master DB post-rollout.
[05:31] <lifeless> or is it within the variation you've seen anyway
[05:42] <stub> lifeless: load seems less
[05:42]  * stub looks for the graph
[05:42] <lifeless> \o/
[05:47] <lifeless> brb
[05:53] <stub> lifeless: https://lpstats.canonical.com/graphs/DBCpuLoadAppServers/20100907/20100914/ -- doesn't seem much changed according to that
[05:58] <thumper> lifeless: have you filed a bug about the code import xmlrpc timeouts?
[06:07] <lifeless> yes
[06:07] <lifeless> search on the page id
[06:07] <lifeless> hmm, I thought I had
[06:11] <lifeless> stub: you did the session db schema change?
[06:11] <stub> lifeless: yes
[06:11] <lifeless> cool
[06:11] <lifeless> I doubt spm has the cycles
[06:12] <lifeless> but I'll see if Tom does, to move the cert deployment forward.
[06:12] <spm> me? doubt it. (cycles)
[06:31] <lifeless> how hard is it to batch a page ?
[06:31] <lifeless> is there a howto ?
[06:59] <lifeless> so
[06:59] <lifeless> batching
[06:59] <lifeless> I guess I need to dig myself.
[07:06] <stub> lifeless: Have you seen anything to suggest we should change our Storm cache sizes? Its available as a config parameter in launchpad-lazr.conf but we have never tuned it - just an initial guess.
[07:06] <lifeless> stub: I haven't seen anything saying we're thrashing in swap
[07:07] <lifeless> stub: and from a page layout perspective the storm cache is in principle unneeded anyway, once we finish structuring things right
[07:07] <lifeless> stub: but even so, no, I haven't seen any [obvious] cache thrashing
[07:07] <lifeless> once we get down to a sensible 10-20 queries per page that would be very obvious
[07:07] <stub> I wonder if we would benefit from a larger cache then? I don't think flushes are instrumented, so I don't know if we are needlessly reloading objects that could have been pulled from cache
[07:08] <lifeless> stub: I doubt we'd benefit - we flush the cache on transaction boundaries anyway
[07:08] <lifeless> stub: flushes aren't recorded AFAIK; but we'd see the loads as a series of single object loads at the end of the apge.
[07:09] <stub> I'm thinking about prejoin type stuff which by its design relies on objects remaining in the cache until they are needed.
[07:09] <stub> (as nothing is holding references to them)
[07:10] <stub> But I guess loading that many objects is the real problem, not how to ensure that we can access too_many objects efficiently.
[07:10] <lifeless> yeah
[07:10] <lifeless> I think theres a few angles to it
[07:10] <lifeless> firstly prejoin is not a complete enough eager-loading for us
[07:10] <lifeless> what we're starting to do in e.g. Person is sufficient
[07:11] <lifeless> secondly, if we are hitting a cache threshold, I think we'll see it very clearly when we look at an OOPS.
[07:11] <lifeless> stub: what is the cache size set to ?
[07:13] <stub> Oh - for the appserver it is 10k so I guess no problem there. Most batch jobs have it set to 500.
[07:13] <lifeless> ah
[07:13] <stub> All in schema-lazr.conf - nobody has ever overridden the defaults in the instance configs.
[07:13] <lifeless> batch jobs I'm not really scrutinising yet.
[07:14] <lifeless> 500 is possibly too small hyes.
[07:14] <lifeless> but OTOH
[07:14] <lifeless> batch jobs should be pretty darn precise with what they do compared with appserver pages which are all over the shop
[07:14] <lifeless> Intrinsically I mean; like 'show me all specs' is an appserver use case, for a batch job it would be 'email changes to specs' and thats time-bounded.
[07:20] <StevenK> Okay, fine. SQLObject I hate you.
[07:20] <lifeless> StevenK: you can return a storm ResultSet even while the class uses SQLBase, its easy and lets callers start depending on it.
[07:20] <StevenK> lifeless: That sounds like a good plan
[07:21] <lifeless> I did this to searchBugs, for instance.
[07:21] <lifeless> oh
[07:21] <lifeless> and for a relatively clean one - hahhaha -
[07:21] <lifeless> see Distribution.specifications
[07:23] <StevenK> lifeless: Right, which ends up using Store.of anyway
[07:23] <wgrant> Store.of is Storm...
[07:25] <lifeless> StevenK: thats the point, the guts of the function is still shitty hand-string-joined-sql
[07:25] <StevenK> Then I may as well just use IStore(IDistroSeries) ...
[07:25] <lifeless> StevenK: but it returns a perfectly cromulent storm result set
[07:26] <lifeless> StevenK: Store.of if you have an object is *important*
[07:26] <lifeless> StevenK: You want to match the store that the object came from
[07:27] <StevenK> lifeless: Or can I just call methods on the SQLObjectResultSet I get back?
[07:27] <lifeless> of course you can
[07:27] <lifeless> converting that to a storm ResultSet would be a good idea though.
[07:28] <lifeless> StevenK: perhaps you should explain whats up
[07:28] <StevenK> lifeless: My hidden question is "Will that involve more database roundtripping" ?
[07:28] <lifeless> StevenK: will whatr
[07:28] <StevenK> lifeless: Will calling methods on the SQLObjectResultSet result in more database roundtripping?
[07:29] <lifeless> if you call methods that do round trips.
[07:29] <lifeless> (sorry that thats the answer, but thats the situation)
[07:29] <lifeless> resultsets will do roundtrips when you call methods on them
[07:30] <lifeless> which methods cause roundtrips depends on the resultset type
[07:30] <StevenK> lifeless: To explain what's up: I'm writing a new method on IDistroSeries called deriveDistroSeries, and one of the arguments is the name of the distroseries. The method will look up if the distroseries exists, and if it does, use it, and if it doesn't, use it.
[07:30] <StevenK> I'm having trouble looking up if the distroseries exists.
[07:30] <mwhudson_> StevenK: i get the feeling that you are worrying about something that essentially shouldn't be worried about, but slightly over specific questions are masking the real issue
[07:31] <lifeless> StevenK: so
[07:31] <lifeless> StevenK: firstly, write the function taking a distroseries object not a name.
[07:31] <StevenK> mwhudson_: TBH, I suspect my problem is that getUtility(IDistroSeriesSet).findByName(name) doesn't return what I think it should
[07:31] <lifeless> StevenK: if you intend it to work in that way.
[07:32] <lifeless> StevenK: secondly, if you want a name-based version, add that as a trivial wrapper.
[07:32] <lifeless> don't conflate the two.
[07:32] <lifeless> you'll find it clearer and easier to test if you avoid magic like that.
[07:32] <wgrant> I can't see a reason to have it take a name.
[07:33] <lifeless> thirdly, if you are concerned about db round trips, *test it* - write tests, like those I've mentioned in perf tuesday emails, that check the number of db calls that go on.
[07:33] <wgrant> That's only good for things like components and sections.
[07:33] <lifeless> there is /nothing/ like tests for catching gotchas; I can help you ensure that the test is valid, once you have a sketch of it up.
[07:33] <lifeless> tests and bastard users AKA 'QA'
[07:34] <lifeless> wgrant: mwhudson_: any tips on how to 'batch a page'
[07:34] <lifeless> is there a howto
[07:34] <lifeless> or docs
[07:34] <wgrant> lifeless: I don't know of any.
[07:34] <stub> I tend to avoid the old crufty SQLObject era APIs on the FooSet utilities and use native Storm syntax. IStore(DistroSeries).find(DistroSeries, name='myname').one()
[07:34] <wgrant> I just grep.
[07:34] <lifeless> wgrant: do you know how to batch a page? if so, please be writing it up!
[07:35] <mwhudson_> lifeless: don't know of anything either
[07:35] <stub> But I'm not usually in browser code, which is supposed to go via the *Set for security issues.
[07:35] <StevenK> stub: This is API code
[07:35] <lifeless> the store objects we have are security proxied
[07:35] <lifeless> so they will wrap any objects returned
[07:35] <lifeless> the Sets don't add anything security wise.
[07:36] <StevenK> Just pain, misery and SQLObject
[07:36] <lifeless> unless I'm deeply confused.
[07:37] <stub> I didn't think our Store objects were security wrapped. Under the hood, they come from the IZStorm utility which is not registered as a secureutility IIRC.
[07:38] <lifeless> hmm
[07:38] <lifeless> then we should security wrap them
[07:38] <lifeless> because API's also need the security enforcement, and model code what APIs expose.
[07:39] <lifeless> I'll file a bug
[07:39] <stub> Just include 'may' in there - I'm not 100% on this.
[07:39] <wgrant> The security stuff here is a bit complicated.
[07:40] <wgrant> Even if those stores are proxied, Store.of isn't.
[07:40] <wgrant> I don't think IStoreSelector is secured.
[07:40] <lifeless> all in the big
[07:40] <lifeless> *bug*
[07:41] <lifeless> ok
[07:41] <lifeless> so whats a simple batched page
[07:41] <lifeless> branches?
[07:42] <wgrant> Bug listings, or possibly /people
[07:42] <wgrant>  /people used to be non-standard, but I think Curtis fixed that recently, so it might be a nice example.
[07:44] <StevenK> steven@liquified:~% du -ch /tmp/launchpadlib-cache-* | tail -n 1
[07:44] <StevenK> 225M	total
[07:44] <StevenK> For want of a test suite that doesn't do evil things to /tmp
[07:48] <lifeless> we need to fix up zope.app.testing first
[07:48] <lifeless> here, a blindness pull - zope.app.testing.functional.FunctionalTestSetup.__init__
[07:48] <lifeless> the first line.
[07:50] <StevenK> lifeless: TBH, I'm happy to accept it's a bug in Zope
[07:51] <lifeless> its not per se
[07:51] <lifeless> its a stack that needs cleaning up is all
[07:59] <lifeless> poolie: those scopes have now landed.
[08:02] <wgrant> Oh, nice.
[08:02] <lifeless> wgrant: ?
[08:03] <wgrant> pageid-based FF.
[08:03] <lifeless> yes
[08:03] <lifeless> in e2 atm is a FF memcache control
[08:03] <wgrant> So we can turn caching on and off with a flag?
[08:03] <lifeless> yes
[08:03] <lifeless> per pae
[08:03] <lifeless> per page
[08:05] <lifeless> next up after that is per pageid timeout control, the prerequisite for that should land overnight too
[08:18] <poolie> oh, great
[08:45] <adeuring> good morning
[08:58] <henninge> Is LP production running on python 2.6 now?
[08:58] <lifeless> partly
[08:59] <lifeless> AIUI rolling upgrades are still going on - check the wiki page if you care
[09:05] <mrevell> Hello
[09:05] <lifeless> hi
[09:16]  * mwhudson_ wonders how crazy running launchpad on arm would be
[09:16] <mwhudson_> i guess the lack of ram would be the killer
[09:16] <lifeless> 1TB should be fine
[09:16] <StevenK> Our current servers don't have that ...
[09:17] <mwhudson_> 256 megs might be enough to have an appserver limp along, i guess
[09:17] <elmo> 1000     21319 50.2 10.9 1022248 669056 ?      Sl   Sep09 2859:33 /usr/bin/python -S bin/run -i lpnet7
[09:17] <elmo> mwhudson: ^--
[09:17] <wgrant> You can just run a dev appserver and librarian in 512.
[09:17] <wgrant> But it's, er, painful.
[09:18] <lifeless> we should fix that
[09:18] <wgrant> There's also the issue of the PPA not building on armel.
[09:18] <wgrant> But apart from that it should work.
[09:19] <mwhudson_> elmo: i wasn't suggesting it for prod :)
[09:46] <lifeless> jml: welcome back
[09:47] <jml> lifeless, thank you.
[09:47] <mwhudson_> hello jml
[09:47] <jml> mwhudson_: hi
[10:11] <jtv> noodles775: hi there!  I'm working on displaying TranslationTemplatesBuild.  Ironically, it involves adding a view for TranslationTemplatesBuildJob so the builder displays can link to the TranslationTemplatesBuild.
[10:12] <noodles775> Huh, a view for the TTBuildJob? Do you mean for the TTBuild?
[10:13] <jtv> noodles775: no, that's the thing
[10:13] <wgrant> Along the lines of BuildFarmBuildJob?
[10:13] <jtv> I need the *-current.pt template to link to the build.
[10:13] <jtv> wgrant: exactly
[10:14] <jtv> and hi ;)
[10:14] <wgrant> hi.
[10:14] <jtv> But to stop me from getting it all wrong: how do I find the right BuildFarmJob for a given BuildQueue?
[10:14] <jtv> (I realize it's all going to change)
[10:15] <wgrant> You shouldn't need to know that.
[10:15] <wgrant> (and I forget)
[10:15]  * noodles775 would need to look at the code and see what soyuz/code does too.
[10:15] <jtv> Okay, then can I just replace the display of the current BuildQueue for a builder with a display of the ongoing build?
[10:15] <jtv> Or BuildFarmJob?
[10:16] <jtv> I guess I'll just steal from there then.  :)
[10:16] <wgrant> Stealing ideas from elsewhere is almost always the best idea.
[10:16] <wgrant> Since that code probably works...
[10:18]  * bigjools asks jtv to consider performance when writing this code
[10:18]  * jtv is happy consider performance, but getting the smegging thing working at all does come into play as well
[10:19] <jtv> Now, the soyuz builds seem to put all this into a link formatter.  Which one?
[10:19] <jtv> PackageBuildFormatterAPI?
[10:20] <wgrant> Watch out -- there are unobvious formatter specialisations which have caught me before.
[10:20] <wgrant> I don't remember details. But search for subclasses first.
[10:20] <wgrant> Or it gets very confusing.
[10:21] <jtv> Hmm what _is_ the context for soyuz's view there?  The build or the job?
[10:21] <wgrant> The +current view?
[10:21] <jtv> Yes, on that one?
[10:22] <jtv> Looks like it's still an IBuildFarmJob as before.
[10:23] <wgrant> BuildFarmBuildJob wraps the Build.
[10:23] <wgrant> Which is itself an IBuildFarmJob.
[10:23] <wgrant> But not an IBuildFarmJobOld.
[10:26] <jtv> So chances are that we'll want the same on BuildFarmBranchJob.
[10:27] <jtv> And perhaps that'll allow us to get rid of BuildFarmBuildJob and BuildFarmBranchJob, which after all were really only there to cover for the absence of a Build.
[10:28] <wgrant> Once Translations complies, we can do all sorts of mass deletions.
[10:29] <jtv> True—no particular need to fix that up now.
[10:32] <thumper> jml: https://dev.launchpad.net/Code/BranchRevisions
[11:09] <lifeless> \o/ memcache branch passed ec2
[11:27] <jtv> bigjools, noodles775: lib/lp/soyuz/browser/configure.zcml defines a browser:page (with name="+current") for lp.buildmaster.interfaces.buildqueue.IBuildFarmJob
[11:28] <jtv> I'm surprised that doesn't blow up, given that I don't see IBuildFarmJob in lp.buildmaster.interfaces.build*queue*
[11:31] <noodles775> jtv: it seems to have been added (or at least moved there) by you? I'm not sure why it was added (or perhaps just moved there from elsewhere).
[11:31] <jtv> noodles775: I guess it's just a dead page and I can delete it.  But WTF doesn't anything complain about it?
[11:32] <noodles775> Removing it and running the tests might show why... otherwise, not sure.
[11:33] <jtv> I think I know why it's there and it should indeed be a dead page.
[11:40] <jtv> wgrant: afaics I do need to know how to get from a BuildQueue to a BuildFarmJob because the UI seems to find my build as buildqueue.specific_job.build.  :/
[11:40] <jtv> The specific_job is TTBJ.
[11:41] <jtv> So the TTBJ needs to find the BuildFarmJob
[11:42] <jtv> (I get here because Builder.currentjob is the starting point for the rendering of builds and buildjobs, and that's a BuildQueue.)
[11:43] <jtv> noodles775: do you have any idea how I can find my BuildFarmJob given a BuildQueue?
[11:44] <noodles775> jtv: Do you mean other than how the soyuz/code parts do it?
[11:44] <jtv> (I also need to look at build histories, of course)
[11:44] <jtv> noodles775: they keep references to build objects in the db, don't they?
[11:45] <noodles775> Yep, they have linking tables (ie. BuildPackageJob)
[11:46] <noodles775> I don't know what you do and don't have in the DB... can you remind me what's different for translations?
[11:46] <jtv> That's hardly something I can do.  I guess I could move TTBJ over as some kind of view on TTB, but… yuck.
[11:46] <jtv> noodles775: there was never any build object.
[11:47] <jtv> And now the new infrastructure is at a point where it seems to require a build object in the obsolescent old-model classes.
[11:50] <noodles775> jtv: what is build_queue.job_type in your case? TRANSLATIONTEMPLATESBUILD?
[11:50] <jtv> yes
[11:55] <noodles775> jtv: and can you get a hold of your TTBJ based on the queue (ie. based on the job.id I assume)?
[11:55] <jtv> noodles775: yes, that's all linked together using the old model—BuildQueue.job == TTBJ.job == Job.id
[11:57] <jtv> The problem is getting from anywhere in the old model (BuildQueue, TTBJ, Job) to the new model (BuildFarmJob, TTB)
[11:58] <noodles775> jtv: I was going to ask why buildqueue.specific_job doesn't just work (or at least, why it can't just work if you hook up the utility), but then saw that TTBJ.getByJob() is returning a BranchJob.
[11:58] <jtv> Well TTBJ lives in the BranchJob table.
[11:59] <jtv> buildqueue.specific_job does just work, but it returns a TTBJ not a TTB.
[11:59] <jtv> If it needs to return a TTB, same problem: I have a BuildQueue and I need to find the corresponding TTB.
[11:59] <jtv> Or BJF.
[11:59] <jtv> *BFJ
[12:00] <wgrant> jtv: isn't the TTB accessible from the TTBJ?
[12:00] <jtv> wgrant: no—that's _exactly_ the problem I've been trying to solve.  :)
[12:01] <wgrant> In the other cases, the job exists to link the build and the buildqueue.
[12:01] <jtv> Yes, and the big problem we were going to solve with this new model was that TTBJ was not related to any build object.
[12:02] <wgrant> But it exists now.
[12:02] <wgrant> So TTBJ could link to it.
[12:02] <jtv> Exactly.  It's great, but I can't find it.
[12:02] <wgrant> Oh, right, it's a BranchJob arrgh.
[12:02] <noodles775> wgrant: TTBJ isn't a DB model...
[12:02] <noodles775> yep.
[12:02] <jtv> TTBJ could link to it _if_ I move the entire class to a new table, which will be lots and lots of work for an obsolescent class.
[12:03] <noodles775> jtv, wgrant: I wonder if this would be a good opportunity to start the transition. ie. jtv's branch could add the job attribute to BFJ. Then his TTBJ class could have a build attribute that just looks it up based on the job?
[12:04] <wgrant> Depending on how large the branch already is, that could be reasonable.
[12:04] <jtv> Will they be the same Job though?
[12:04] <wgrant> They probably could be.
[12:04] <noodles775> jtv: you'll need to make sure they are - no one else is using it yet (so it will be NULL for code/soyuz).
[12:04] <wgrant> I see no reason why the BuildQueue Job can't be made persistent.
[12:05] <wgrant> It shouldn't break anything.
[12:05] <jtv> flw :)
[12:05] <wgrant> You don't even need to start using its attributes yet, I guess.
[12:07] <jtv> I'm not going to be able to make this kind of change in my ongoing branch.  How about this:
[12:07] <wgrant> Adding and populating a simple BFJ.job FK shouldn't be troublesome.
[12:07] <wgrant> You don't have to do anything with it.
[12:07] <jtv> I make sure build histories can link to and render TTBs (they already render).
[12:08] <jtv> I get that change reviewed.
[12:08] <wgrant> Linking isn't that important. Not crashing is.
[12:08] <jtv> My branch can already render TTBs.
[12:08] <jtv> So I make sure build histories can link to and render TTBs without crashing,
[12:09] <jtv> get that change in together with a bunch of other ones I've already accumulated
[12:09] <jtv> (they're blocked on this branch),
[12:09] <noodles775> Sounds good.
[12:09] <jtv> land the lot of 'em,
[12:09] <jtv> then we twiddle the model further so that Builder.currentjob.specific_job.build works,
[12:09] <deryck> Morning, all.
[12:10] <jtv> and then (afaic in yet another branch) retire BuildFarmBranchJob and basically anoint BuildFarmBuildJob as the buildmaster-standard implementation for BuildFarmJobs.
[12:10] <wgrant> And then delete it.
[12:10] <noodles775> Yay.
[12:10] <jtv> Yes.
[12:10] <jtv> Kill.
[12:10] <jtv> Kill.
[12:10] <jtv> Kill.
[12:12] <jtv> This means that for now, the "Building [...]" display for a builder will _not_ link to the TTB yet, only to the branch.
[12:12] <jtv> That change comes in part (3).
[12:12] <jtv> Which leaves me free right now to worry about functional display of TTBs in builder _histories_
[12:13] <jtv> As an Orwell character might have put it, to fix the present we must first fix history.
[12:14] <jtv> Now, how _does_ the builder history get rendered?  Retrieve all BuildFarmJobs and for each, obtain & render the specific job?
[12:28] <jtv> noodles775: does BuildFarmJob play any role in rendering build histories yet?
[12:39] <jtv> danilos: my plan is this:
[12:39] <jtv>  * Finish & land my UI branch.
[12:39]  * danilos is all ears (well, /me is all ears anyway, but now especially so)
[12:40]  * jtv holds back nasty remark about huge nose
[12:41] <danilos> :)
[12:41] <jtv>  * Adjust the model so that we can find the new buildfarmjob class (the historic record, basically) where we need to from the old TTBJ class in the same way that the other old-model classes can.
[12:42] <jtv>  * Clean out the classes, interfaces & templates that we currently have to paper over this problem, using the new tie between the two models instead.
[12:43] <jtv> That last step eliminates our "special" position in the build-farm model afaics.
[12:43] <danilos> jtv, can we not have a simple adapter for TTBJ to IBuild if that's what we need? what exactly is missing?
[12:44] <jtv> What's missing is any kind of link between the objects in the old model and the objects in the new model.
[12:44] <jtv> Because the two are tied together only through the build objects that we never had.
[12:45] <danilos> jtv, right, so is it not simpler to add a link to TTBJ on the Build object then? (I assume Build object is part of the "new" model)
[12:45] <jtv> You assume right (although it's confusingly called BuildFarmJob)
[12:46] <danilos> jtv, ok, so that's what you mean with "adjust the model" step?
[12:47] <jtv> Doing it that way would still be very ugly.  But more to the point, that's all for the next step—which we should hammer out in detail with the Wellington crew.  There is a plan.
[12:48] <jtv> For the more distant future, the plan was to re-introduce Job which is so far completely missing from the new model.
[12:48] <jtv> In the old model, Job was what tied all of it together.
[12:48] <danilos> jtv, hum, so what kind of "adjusting" is needed in the first step? uglifying and then cleaning up later is ok by me :)
[12:49] <jtv> The first step is just making sure nothing blows up with what I already implemented, and landing it all.
[12:49] <danilos> jtv, right, then I mean second step
[12:50] <danilos> jtv, also, would what you implemented allow us to eg. generate stats based on the data in the DB and interrogate DB directly if we want to know more about any particular build?
[12:50] <jtv> The second step _as it looks now_ involves moving the re-introduction of Job forward.  We add a FK to Job to BuildFarmJob, and then I can use that to find our new build class ("derived" from BuildFarmJob, though in a separate table) from our old buildjob class.
[12:52] <jtv> danilos: it should yes, since it introduces creation of BuildFarmJobs for our job type.
[12:53] <danilos> jtv, right, so step 1 is golden for us, step 2 and 3 are basically interdependent as well?
[12:54] <danilos> jtv, also, what happens with TTBJ, where do we move the functionality it provides to?
[12:55] <jtv> danilos: step 1 should be a small one from where I'm standing (cross fingers), which would be great for reporting on our end.  And again as you say, 2 and 3 are interdependent in that 2 moves forward design and implementation work that step 3 is currently stuck on.
[12:56] <danilos> jtv, right, so let's do that, and then we can try to figure out the minimal stuff we need to do to unblock everybody else
[12:56] <jtv> danilos: TTBJ provides very little.  It gets replaced with TranslationTemplatesBuild, which fits neatly into the new model.
[12:57] <jtv> I think step 1 would unblock things for the others.  Which is one reason among many why I want to phase things this way.
[12:57] <danilos> jtv, right, if that's the case, let's get it done, and not worry about other things yet :)
[12:58] <danilos> jtv, though, I am a bit confused: wouldn't we need to introduce the link between the TTBJ and BFJ before we'd really unblock others?
[12:59] <danilos> jtv, also, why can't we simply introduce a new field on TTBJ that links directly to BFJ (TTBJ is going away anyway)
[12:59] <jtv> TTBJ lives in the BranchJob table, so we can't do that.
[13:00] <danilos> jtv, uhm, why not? (yes, it might be ugly, but wouldn't it be dirt cheap?)
[13:00] <jtv> I don't think we need to introduce a link in order to unblock others; what matters is that each of our jobs will have a representation in the new model.  Where the UI starts out in the old model and tries to navigate to the new model, our jobs simply don't exist just as things stand now.
[13:01] <danilos> jtv, ok, fair enough
[13:01] <danilos> jtv, but, how will the jobs be executed? will there be an interim phase where such link is needed for execution as well?
[13:02] <jtv> Exactly _how_ we link the two is part of step 2, and that's something to be discussed with the Wellington team.  How the jobs will be executed won't matter much, since our jobs will be present in both models simultaneously.
[13:03] <jtv> AIUI as soon as we start executing based on the new model, we also stop generating the old-model objects.
[13:03] <danilos> jtv, it does matter because I can easily think of a migration approach that would leave our jobs not running
[13:03] <danilos> jtv, but, since you seem confident, I'll trust you on that ;)
[13:04] <jtv> We're generating the same jobs in both the old and the new model.  They both contain the same information.
[13:05] <jml> I'm off to grab some lunch. Back soon.
[13:05] <danilos> jtv, ok, imagine a migration path like this: "we want to start using the new code, but from within the old model, just like UI: 1. get old model jobs, 2. get a link to new model jobs [just like UI, right], 3. if found, run the job"
[13:05] <jtv> Yes, we could do it wrong if we _really_ wanted to, but I don't really see the need.
[13:06] <danilos> jtv, it's not about a need, I actually find the above approach quite sane, especially considering that that's how UI migration seems to be getting done
[13:06] <jtv> Well what would happen if you tried it that way, things would blow up.
[13:06] <danilos> jtv, how come the UI doesn't blow up, yet it uses new model for all the other builds?
[13:06] <danilos> (or does it?)
[13:06] <jtv> Because that's builds, not jobs.
[13:06] <jtv> We execute jobs, not builds.
[13:07]  * danilos is now thoroughly confused
[13:07] <jtv> danilos: if you're thoroughly confused then at last you fully understand the model.
[13:07] <jtv> congratulations!
[13:07] <danilos> jtv, thank you :)
[13:08] <danilos> jtv, ok, so let's watch out for any weird migrations, I'll go watch out for some food, and not worry about much else right now other than landing the code you've got
[13:09] <jtv> I'm also past eod now, not just temporally but especially mentally.  :)
[13:09] <jtv> But I did want to appraise you of this before the heart attacks started.
[13:14] <danilos> jtv, sure, tty tomorrow
[13:15] <gmb> rockstar, Is it fair to assume you're not around for the next hour or so?
[13:16] <gmb> Heh. s/hour/3 hours/
[13:17] <gmb> deryck, Do you know if rockstar got anywhere with his wigety wizard wonderousness? ISTR seeing something fly across my screen about it late last week, but I didn't really pay it much heed at the time due to being RM-frazzled.
[13:17] <deryck> gmb, yeah, I worked with him on Friday to get his event bug fixed and unblocked him.
[13:18] <deryck> gmb, I suspect he may still be today or tomorrow wrapping that up still.
[13:19] <gmb> deryck, Okay. I'll find something smallish to do in the meantime.
[13:20] <gmb> Oh, I've got that bug import to do first. Win.
[13:24] <deryck> gmb, and there's really no bug in the story left that doesn't require the config widget, is there?
[13:25] <gmb> deryck, One or two, but no, not many.
[13:25] <gmb> It is something of a blocker.
[13:25] <deryck> yeah, I told rockstar to please ping you or I if he needed help, since we really need this.
[13:27] <gmb> deryck, Cool.
[13:28] <deryck> gmb, he did mention he was stopping at a two-step widget, since that's all he needed.  Do you need more steps?
[13:32] <deryck> gmb, maybe you should take a peak at lp:~~rockstar/lazr-js/wizard-widget to get an idea of what's coming to.
[13:32] <gmb> deryck, Well, yes and no. There are potentially up to 4 forms that could be show in one overlay in a given sequence, but they could be logically split across two widgets. I'll take a look at rockstar's branch this afternoon and see what the deal is.
[13:33] <gmb> (Also, there's always the chance that I could hack it to work the way I want, but that might not be TRTTD)
[13:33]  * jml back
[13:33] <jml> and my inbox is empty!
[13:34] <gmb> jml, Select all, delete?
[13:34] <jml> no. inbox zero style, instead of 1100 unread conversations in my inbox I know have about 100 flagged as @READ or @ANSWER
[13:35] <jml> s/know/now/
[13:52] <bac> welcome back jml
[13:52] <jml> bac, thanks.
[13:58] <sinzui> mrevell, ping
[13:59] <mrevell> hello sinzui
[13:59] <sinzui> mrevell, you may be interested in this question: https://answers.edge.launchpad.net/launchpad/+question/125165
[13:59]  * mrevell looks
[14:00] <mrevell> Thanks sinzui. Yes, that's right up my street. I shall deal with it.
[14:00] <sinzui> thanks
[14:00] <mrevell> In that I'll answer it and also write a blog post inviting nominations
[14:23] <matsubara> hey bigjools
[14:23] <bigjools> matsubara: hey dude
[14:24] <matsubara> bigjools, could you help me with a upload failure from a recipe? I created a recipe to create packages for bzr-pqm and put them in the ~launchpad PPA. It seems the source build step worked correctly but the upload failed
[14:25] <bigjools> matsubara: build URL?
[14:25] <matsubara> bigjools, this is the recipe URL: https://code.edge.launchpad.net/~matsubara/+recipe/bzr-pqm-launchpad-ppa/+build/2465
[14:25] <matsubara> I'll paste the failure email I got, just a sec
[14:25] <bigjools> matsubara: did you read the log? :)
[14:26] <bigjools> all will become clear when you do
[14:26] <matsubara> bigjools, yep. it says the versions already existed there but this was my first try AFAIK
[14:26] <matsubara> bigjools, normally I'd have to bump the version, right?
[14:26] <bigjools> yes
[14:27] <bigjools> it's the source that already exists, it was probably uploaded as a regular package already
[14:27] <matsubara> bigjools, and when I look here: https://edge.launchpad.net/~launchpad/+archive/ppa/+packages, looks like the package is there and published and everything seems to be alright
[14:27] <bigjools> there you go then
[14:28] <bigjools> matsubara: oh it's never been uploaded at all before?
[14:29] <matsubara> bigjools, not with that version number. there was a older bzr-pqm package in the ~launchpad PPA  uploaded by Gary some time ago
[14:29] <bigjools> yeah bzr72
[14:30] <matsubara> bigjools, btw, when I apt-get update && apt-get upgrade I get the new package (bzr 73) that I uploaded.
[14:30] <bigjools> matsubara: hmmm I'd check with one of the Code guys then, seems like it tried to upload itself twice
[14:30] <matsubara> so I'm confused by the email saying it failed to upload
[14:30] <matsubara> hmm
[14:31] <matsubara> bigjools, don't know if it helps, but here is the failure email: https://pastebin.canonical.com/37090/
[14:31] <bigjools> ok
[14:32] <bigjools> matsubara: no it doesn't help :(
[14:32] <matsubara> and the recipe page says (https://code.edge.launchpad.net/~matsubara/+recipe/bzr-pqm-launchpad-ppa/+build/2465) still says it failed to upload
[14:33] <matsubara> hmmm
[14:33] <matsubara> looking here: https://code.edge.launchpad.net/~matsubara/+recipe/bzr-pqm-launchpad-ppa I see there are two build records there
[14:35] <bigjools> matsubara: so 2 builds got created, which explains the error
[14:36] <bigjools> matsubara: I don't know what could cause that to happen
[14:36] <bigjools> but abentley might
[14:37] <matsubara> bigjools, maybe I set the daily build option and requested a build manually through the "request builds" link
[14:37] <matsubara> not sure, but looks like PEBKAC heh
[14:37] <bigjools> yeah that could be it
[14:37] <bigjools> :)
[14:38] <matsubara> thanks for your help!
[14:38] <bigjools> matsubara: it's a bug that it allows 2 builds for the same version though
[14:38] <matsubara> yep, I'll search/file a bug
[14:38] <bigjools> that should be caught earlier so that it doesn't waste build farm time
[14:38] <matsubara> bigjools, launchpad-code or soyuz?
[14:38] <bigjools> code
[14:38] <matsubara> okie. ta!
[14:39] <cr3> hi folks, I'm mapping some launchpad concepts externally for referential integrity purposes, so that my web app can refer to launchpad projects for example. my question is: should I name that particular concept as a "product" to remain true to launchpad or "project" because that's how it's exposed externally?
[14:40] <matsubara> bigjools, aha! https://bugs.edge.launchpad.net/launchpad-code/+bug/620248 the description and tim's comment describes exactly my situation
[14:40]  * cr3 is leaning towards remaining as close as possible to the launchpad core for consistency purposes
[14:41] <gmb> cr3, Project. The "product" moniker is archaic and should be taken out and shot, but it's got its tentacles into a lot of code.
[14:41] <gmb> (That and "Project Groups" are called "projects" in code, which is just weird)
[14:41] <cr3> gmb: that's the feeling I got from the launchpad code, but it doesn't seem like it's ever going to happen
[14:41] <gmb> cr3, I think it's a JFDI-when-we-have-breathing-room thing.
[14:41] <gmb> Because it has the potential to break tonnes of stuff.
[14:42] <cr3> gmb: breathing what? not in my vocabulary :)
[14:42] <bigjools> matsubara: yep!
[14:42] <gmb> Maybe we'll do it in the week before Xmas, when we have nothing major scheduled, just lots of code cleanup.
[14:42] <abentley> matsubara, bigjools,  it is expected behaviour that when you enable daily builds, it will always attempt a build, even if you have previously done a build.  One has no way of knowing whether the result will fail to upload without doing a build.
[14:42] <bigjools> abentley: I have deja vu about this fact :)
[14:42] <gmb> cr3, Breathing room... er... space to make the change and clean up all the breakages without making everything far to stressful.
[14:42] <cr3> gmb: that's much sooner than I had expected, I'll code for the future then. thanks for the advice!
[14:43] <gmb> cr3, np. But note: "Maybe" from me != "We will" from flacoste or thumper :)
[14:43] <cr3> gmb: as long as the intention is there, at least I can justify my naming decision. I will document that my concept of project maps to launchpad's concept of product though
[14:44] <matsubara> gmb, cr3: https://bugs.edge.launchpad.net/launchpad-code/+bug/109153
[14:45] <abentley> matsubara, bigjools: upload failures usually come from debversion, and some substituted values (e.g. {time} will always be different.
[14:45] <gmb> matsubara, Hurrah, saved me a job; thanks :)
[14:46] <matsubara> abentley, right. my case is that I think I set the daily build option and requested a manual build at the same time
[14:46] <matsubara> then I was confused by the failure email
[14:46] <matsubara> while the first request was successfully built
[14:46] <abentley> matsubara, I understand that.
[14:48] <abentley> matsubara, if you had requested the same build twice, that would have been rejected.  But the daily build would have been scheduled after your manual build completed.
[14:51] <matsubara> abentley, even if it completed successfully, hence bug 620248?
[14:52] <abentley> matsubara, even if it completed successfully, because there's no way to know that it wouldn't complete successfully again without attempting a build.  See above.
[14:52] <matsubara> right, makes sense
[14:53] <rockstar> gmb, skype?
[14:55] <gmb> rockstar, Sure. Let me get mah Skype rolling.
[14:57] <rockstar> gmb, I don't have your skype id already here, and apparently your name is not original enough.
[14:57] <gmb> rockstar, binnsgm. :)
[15:35] <rockstar> gmb, you'll want to bzr pull.
[15:35] <gmb> rockstar, Okay, doing so now ,thanks.
[15:36] <rockstar> gmb, you should have revno 187 (a bad omen)
[15:37] <gmb> rockstar, yep :)
[16:00] <jml> sinzui, we have a call now.
[16:00] <jml> sinzui, but I need a cup of tea first.
[16:04] <sinzui> jml, I am ready on mumble
[16:09] <jml> sinzui, me too!
[16:22] <bigjools> sinzui: I'm not sure, but I think this bug is registry?  https://bugs.edge.launchpad.net/launchpad/+bug/636151
[16:26] <sinzui> bigjools, yeah. That has no home. give it to registry. I think most of the objects are registry related.
[16:27] <bigjools> that's what I figured, thanks
[16:28] <bigjools> that question about featured projects probably belongs there too
[16:32] <bigjools> mrevell: did I see earlier that you were dealing with this? https://answers.edge.launchpad.net/launchpad/+question/125165
[16:33] <mrevell> bigjools, Yeah, I'll be on it.
[16:33] <bigjools> mrevell: ok mind if I assign to you?
[16:33] <mrevell> go for it
[16:34] <bigjools> ahhh questions needs some ajax love
[16:36] <sinzui> jml, UbuntuBeta, Ubuntu, 'Bitstream Vera Sans', 'DejaVu Sans', Tahoma, sans-serif;
[16:42] <gmb> bigjools, Questions needs some love from the Loving Hammer of Rectification, never mind AJAX.
[16:44] <gmb> Oh, no, wait, I'm thinking of blueprint.
[17:09] <james_w> some help with diagnosing bug 637323 would be welcome, as it is causing problems for the apport retracers in particular
[17:19] <EdwinGrubbs> Is there something I need to do besides put /++oops++ in the url to trigger an oops on launchpad.dev?
[17:43]  * bigjools EoDs, good evening all
[18:04] <mrevell> night all
[18:54]  * rockstar lunches
[19:11] <jelmer> rockstar: hi
[19:26] <lifeless> moin
[19:36] <lifeless> james_w: so
[19:37] <lifeless> james_w: bug:
[19:37] <lifeless>  - appserver revno differences cause PATCH error
[19:37] <lifeless>  - launchpadlib in stable Ubuntu points to edge
[19:37] <james_w> that's intended design in the current code
[19:38] <james_w> well, not exactly, but the revno changing the etag is
[19:38] <lifeless> so, we have to fix that.
[19:38] <lifeless> what else
[19:38] <lifeless> (if we don't fix it we cannot have a smooth upgrade of the server farm). Its a designed in error.
[19:39] <lifeless> oh, and the appserver 'name' should be present in responses so we can debug things remotely.
[19:55] <bdmurray> I've running into an error when running make run - "cannot import name SAFE_INSTRUCTIONS" from bzrlib.plugins.builder.recipe
[19:56] <lifeless> update your sourcecode
[19:56] <lifeless> utitilies/update-sourcecode
[20:02] <bdmurray> Updating dulwich to revision 424 (No change)2kB/s | Fetching revisions:Finishing stream
[20:02] <bdmurray> Exception AttributeError: "'NoneType' object has no attribute 'close'" in <bound method SmartSSHClientMedium.__del__ of SmartSSHClientMedium(bzr+ssh://brian-murray@bazaar.launchpad.net/)> ignored
[20:03] <lifeless> and cd to the download-cache and run bzr update
[20:06] <bdmurray> it is up to date at revision 183
[20:10] <lifeless> james_w: can we move the retracer to prod please?
[20:11] <james_w> lifeless: that's not what I was playing with, and I have no control over it
[20:11] <lifeless> ah ok
[20:18]  * maxb raises two eyebrows at finding an executing code import for a deactivated project
[20:21] <maxb> (lp:libs)
[20:24] <jelmer> maxb: There is more than one..
[20:24] <maxb> yes, my point is there shouldn't be :-)
[20:25] <jml> it's not that surprising when you think about it.
[20:25] <maxb> LP does at least does not forbid visibility of these
[20:47] <bdmurray> lifeless: did you have anymore ideas?
[20:47] <lifeless> sorry otp
[20:55] <jml> why do we need to compile templates to run tests?
[20:59] <jml> lifeless, are we having our scheduled call?
[21:04] <cr3> jml: welcome back! fyi, I've spent the last week or so implementing the results tracker: https://launchpad.net/launchpad-results
[21:04] <jml> cr3, thanks. I'm glad to hear it.
[21:04] <cr3> jml: following discussions with lifeless, we decided to impelement this new feature as a standalone web app which could eventually be folded into the core once it becomes stable
[21:04] <jml> a sound plan.
[21:05] <cr3> jml: that way, we can reach stability more quickly than if I implemented directly in the core, and we can make mistakes much more easily too
[21:05]  * cr3 usually gets things right on the third try on average :(
[21:05] <lifeless> jml: yes
[21:06] <cr3> jml: I already have a pretty neat web app pushed into a branch which integrates django, zope, storm and lazr. seems to work well so far
[21:06] <lifeless> jml: ran slightly over with statik
[22:02] <jml> g'night all
[22:02] <mwhudson> jml: g'night
[22:11] <lifeless> night
[22:37] <lifeless> sinzui: hi
[22:37] <lifeless> sinzui: I would like to learn aboot batching!
[22:38] <sinzui> hi lifeless
[22:39] <lifeless> sinzui: is now a good time for you?
[22:39] <sinzui> it is
[22:39] <lifeless> great
[22:39] <lifeless> so I was thinking mailing list moderation pages would be a good one for me to learn on
[22:40] <sinzui> Good choice
[22:40]  * sinzui looks at browser code
[22:40] <wallyworld_> morning
[22:42] <sinzui> lifeless I think we want to start by making held_messages a cachedproperty because we may reference it 3 times
[22:43] <lifeless> sinzui: ok. I'm just pushing current devel up to lp:~lifeless/launchpad/registry which is where I'll put this work
[22:43] <sinzui> okay
[22:43] <lifeless> so, I can do the cachedproperty change easily; can you tell me a little about how batching hangs together while I do that?
[22:45] <sinzui> lifeless: lets talk about this: http://pastebin.ubuntu.com/493322/
[22:45] <lifeless> sinzui: so held_messages won't benefit much by being a cachedproperty. we can come back to that
[22:46] <lifeless> ah, interesting
[22:46] <lifeless> so we wrap the result set
[22:46] <sinzui> lifeless, the BN holds the record set and does not realise it until we iterate over it. BEware of indexing it though. I recently discovered that that does not realise the data, we just call the DB 75 times
[22:48] <sinzui> lifeless our BN sets the default batch size to what we have in config.launchapd.default_batch_size (75?) you can pass size=20 if you believe we want a different size
[22:48] <lifeless> to the constructor ?
[22:49] <sinzui> lifeless correct. Subclasses of the BN may set different size rules or singular/plural headings so that we do not need to set the info in __init__
[22:49] <lifeless> http://paste.ubuntu.com/493324/
[22:50] <lifeless> thats obviously mostly your code
[22:50] <sinzui> lifeless, the request is very important. The BN will look for start and batch (size) params to control the slice. We will raise an BatchSizeError (?) if the user tries to exceed our hard limit
[22:51] <sinzui> ouch we were calling getReviewableMessages() twice for every page load?
[22:51] <lifeless> yes
[22:52] <lifeless> which is old school sqlobject in fact
[22:52] <lifeless> note that creating a resultset is pretty cheap in general because they are lazy: they don't do sql immediately
[22:52] <lifeless> brb guy at door
[22:55] <sinzui> lifeless: We often have navigation links before and after the batch we are iterating over: This is template code: http://pastebin.ubuntu.com/493327/
[22:55]  * sinzui looks for real template
[22:57] <sinzui> ah, that is nice, we are adapting the message in the template. This might be an easy addition
[22:59] <rockstar> mwhudson, is this better?
[22:59] <mwhudson> rockstar: i think so
[23:00] <mwhudson> rockstar: obviously there's no sql that can find the affected recipes
[23:00] <bdmurray> sinzui: did you have an idea about how to fix bug 636412?
[23:00] <rockstar> mwhudson, on the contrary.
[23:00] <mwhudson> rockstar: tbh, i'm tempted to suggest a bzr-builder change
[23:00] <sinzui> lifeless: in team-mailing-list-moderate.pt, I think we only require the navigation headers because the table is doing the right thing. This might be the final code for the template: http://pastebin.ubuntu.com/493328/
[23:00] <rockstar> http://pastebin.ubuntu.com/493314/
[23:00] <rockstar> mwhudson, ^^
[23:01] <mwhudson> rockstar: how do you know that there are no recipes with the same problem?
[23:01] <rockstar> mwhudson, a better query would be to find out where "." is the value of sourcepackagerecipedatainstruction.directory
[23:01] <mwhudson> yeah, that would work here i guess
[23:01] <rockstar> mwhudson, what would you bzr-builder change be?
[23:01] <mwhudson> rockstar: i'm tempted to say that bzr-builder should change so that it can return an invalid recipe from stringification, but tell you about it
[23:01] <mwhudson> so you can display a warning on the recipe's page
[23:01] <lifeless> sinzui: sorry for the interrupt
[23:01] <sinzui> bdmurray, since I cannot see how the user got to that page, I think we should consider that the view should handle the case of URL hacking...
[23:03] <bdmurray> sinzui: right, I haven't seen something like that before
[23:03] <rockstar> mwhudson, that would probably work longterm, yes.
[23:04] <rockstar> mwhudson, for the short term, to restore access to the recipe, does the SQL route sound fine?
[23:04] <mwhudson> rockstar: this would mean changing bzr-builder to not use __str__ to stringify recipes i guess, something i always hate a bit :)
[23:04] <mwhudson> rockstar: yeah
[23:04] <sinzui> bdmurray, the template could guard the form with a new view attr. Maybe view/can_subscribe. That view property does something like:
[23:04] <sinzui> return self.context is not getUtility(ILaunchpadCelebrities).ubuntu
[23:04] <mwhudson> rockstar: is it clear what the intent of the recipe is?
[23:04] <rockstar> mwhudson, maybe we could just modify __str__ to be a bit smarter?
[23:04] <mwhudson> that'd be possible too
[23:05] <rockstar> mwhudson, the recipe has two lines: the base branch and the nest command
[23:05] <sinzui> bdmurray, I suppose the form should also have a sentence for the ubuntu to explain we do not want the user to shoot himself in the foot
[23:05] <lifeless> sinzui: that seems very easy
[23:05] <lifeless> sinzui: whats the key thing in the tal:repeat that you looked for to make sure it wrks ?
[23:05] <sinzui> lifeless yes indeed
[23:06] <sinzui> That widget hack is rather convenient I think
[23:06] <lifeless> sinzui: I don't undertand whats going on there
[23:07] <lifeless> sinzui: what tests are normally added in a case like this?
[23:10] <sinzui> lifeless: I like to see a test to verify the view is returning a BN. Since the headers are set, we should test the bn.default_singular_heading and default_plural_heading
[23:11] <sinzui> lifeless: The template will have upper and lower batch ids to confirm that both navs are rendered. We can test by calling render() on the view or with a TestBrowser
[23:11]  * sinzui looks for existing tests
[23:11] <lifeless> sinzui: can you expand on what the 'widget hack' does?
[23:12] <sinzui> lifeless the adapted message is converted to a HTML fragment in lp/registry/templates/message-moderation.pt
[23:12] <lifeless> sinzui: is that the message/@@+moderation bit ?
[23:13] <sinzui> lifeless yes, but I now think there is one more change needed to the table...
[23:14] <sinzui> lifeless: I think we need to change the line before @@_moderation to
[23:14] <sinzui> tal:repeat="message view/held_messages/currentBatch"
[23:16]  * sinzui is still looking for existing tests for held_messages
[23:18] <sinzui> lifeless: I do not see any tests for the view that is generating the form. We really want one and I really think we have once because I recall reading a test to explain how to put messages into the expected state
[23:19] <lifeless> I'm adding one to test_team
[23:19] <lifeless> can consolidate later
[23:20] <lifeless> browser/tests/test_team, I mean
[23:20] <sinzui> lifeless look at stories/mailing-lists/moderation.txt which puts messages into the state
[23:20] <lifeless> thanks
[23:20] <lifeless> most of these tests don't need that, just the batch ids do, right?
[23:22] <sinzui> lifeless, yes. and we could check the batch ids in the moderation test since we have the template rendered. Lifeless I am a bit pedantic about verifying the batch ids because I discovered we had two upper navs before, or that one never rendered at the bottom of a very long list
[23:22]  * sinzui also added the singular/plural message support
[23:23] <lifeless> ok
[23:23] <lifeless> looking in moderation.txt for the batch ids
[23:23] <lifeless> Would this be a good point to check ?
[23:23] <lifeless> Foo Bar sees that there is one message waiting for approval.
[23:23] <sinzui> yes
[23:23] <lifeless> is there an example of checking this elsewhere?
[23:27] <sinzui> me thinks
[23:28] <thumper> sinzui: are you free for a call?
[23:29] <lifeless> thumper: sinzui is training me just now, should be finished soon.
[23:30] <thumper> lifeless: training you to do what?
[23:30] <lifeless> batch things
[23:30] <lifeless> just bringing all the bits together quickly
[23:30] <lifeless> I couldn't find a clear guide to batching-pages when I asked around yesterday.
[23:32] <sinzui> lifeless print find_tag_by_id(admin_browser.content, 'batchnav_first')['class'] should print first
[23:33] <sinzui> that does not look right, the upper class is missing from this template
[23:33] <lifeless> will it error if someone has doubled things up or something?
[23:33] <sinzui> yes
[23:35] <sinzui> lifeless, I think my days of checking upper and lower batches are over. The markup is rendered with html classes now: upper-batch-nav and lower-batch-nav
[23:36] <sinzui> lifeless, and the nav is only rendered if there is a batch! we need to exceed 5 messages (5 is the default batch size in test)
[23:38] <mwhudson> jelmer: around? http://launchpadlibrarian.net/55539932/vcs-imports-r-project-trunk.log is very odd
[23:38] <sinzui> I have just confirmed that the top and bottom navigators are rendering duplicate ids for the links :(
[23:39] <sinzui> Well That is why i wanted a test ;)
[23:39] <jelmer> mwhudson: It's happened on at least one other cscvs import as well
[23:39] <mwhudson> jelmer: but not all?
[23:39] <mwhudson> jelmer: could it be per-importd perhaps?
[23:39] <jelmer> mwhudson, it doesn't appear to occur that often for that
[23:39] <lifeless> sinzui:     self.assertEqual('message', navigator.default_singular_heading)
[23:39] <lifeless> AssertionError: not equal:
[23:39] <lifeless> a = 'message'
[23:39] <lifeless> b = 'result'
[23:40] <jelmer> mwhudson: actually, that reminds me
[23:40] <lifeless> current diff
[23:40] <lifeless> http://paste.ubuntu.com/493338/
[23:40]  * sinzui rechecks code
[23:40] <mwhudson> jelmer: well it will only happen when there is a new rev to import i guess?
[23:41] <sinzui> lifeless ._singular_heading and ._plural_heading.
[23:41] <jelmer> mwhudson: that's true I guess, but we've only seen 4 or so failures since the rollout. Are there really that few working and active CSCVS imports left?
[23:41] <sinzui> lifeless ^ did you see my discovery that we are generating duplicate ids for links in the upper and lower nav?
[23:42] <mwhudson> jelmer: don't know
[23:42] <lifeless> sinzui: hah!
[23:42] <lifeless> sinzui: fun.
[23:42] <mwhudson> jelmer: but if it was just pear, it would have to run on pear 5 times in a row
[23:42] <mwhudson> jelmer: that's fairly unusual
[23:43] <jelmer> mwhudson: ah, I hadn't considered that. it seems likely that's the cause then
[23:43] <mwhudson> losa ping time i guess
[23:43] <sinzui> The fix looks trivial...concatenate the view class for the nav to the like id in the template. The view/css_class will make them distinct by upper and lower.
[23:44]  * sinzui reports bug to explain
[23:44] <mbarnett> mwhudson: heya.
[23:44] <mwhudson> mbarnett: can you log into pear as importd and run "bzr whoami" pls?
[23:44] <mbarnett> importd@pear:~$ bzr whoami
[23:44] <mbarnett> Import Daemon <importd@pear>
[23:44] <mbarnett> importd@pear:~$
[23:45] <mwhudson> mbarnett: bzr --version?
[23:45] <mbarnett> http://pastebin.ubuntu.com/493339/
[23:46] <lifeless> BatchNavigator.count() is wrong
[23:46] <lifeless> need to find the right way to do that
[23:46] <jelmer> mwhudson: Perhaps not pear but one of the other ones?
[23:46] <mwhudson> mbarnett: can you try python /srv/importd.launchpad.net/production/launchpad/eggs/bzr-2.2.0-py2.6-linux-x86_64.egg/EGG-INFO/scripts/bzr whoami ?
[23:47] <mwhudson> mbarnett: pls modify path until it exists :)
[23:47] <mwhudson> jelmer: https://code.edge.launchpad.net/~vcs-imports/r-project/trunk <- looks like pear is the bad apple
[23:48] <jelmer> mwhudson: Ah, right
[23:48] <jelmer> mwhudson: also, I hadn't considered the bzr path
[23:48] <mwhudson> me neither until i saw that :)
[23:49]  * thumper upgrades to maverick
[23:50] <maxb> jelmer: I think I've figured out what's up with those slow KDE imports.... the bzr-svn logwalker doesn't incrementally commit the cachedb
[23:50] <mbarnett> mwhudson: it is fighting with me: http://pastebin.ubuntu.com/493348/
[23:50] <mbarnett> chmoding
[23:51] <mwhudson> mbarnett: that's why i put python first :)
[23:51] <lifeless> sinzui: so should we not test that here, for now ?
[23:51] <mwhudson> sorry for not using quotes, that would have been easier
[23:51] <mbarnett> mwhudson: ah, i missed the python
[23:52] <mwhudson> mbarnett: "/srv/importd.launchpad.net/production/launchpad/bin/py /srv/importd.launchpad.net/production/launchpad/eggs/bzr-2.2.0-py2.6-linux-x86_64.egg/EGG-INFO/scripts/bzr whoami" would be even more correct i guess
[23:52] <mbarnett> importd@pear:~$ /srv/importd.launchpad.net/production/launchpad/bin/py /srv/importd.launchpad.net/production/launchpad/eggs/bzr-2.2.0-py2.6-linux-x86_64.egg/EGG-INFO/scripts/bzr whoami
[23:52] <mbarnett> bzr: ERROR: Unable to determine your name.
[23:52] <mbarnett> Please, set your name with the 'whoami' command.
[23:52] <mbarnett> E.g. bzr whoami "Your Name <name@example.com>"
[23:53] <lifeless> hmm, getting a LocationError from hold_count now
[23:53]  * lifeless puts it back as 'future'
[23:54] <sinzui> lifeless the fix is pretty easy, and I am sure there is no test for th default BN since I can see any attempt will fail.
[23:54] <sinzui> .me makes quick change
[23:55] <mwhudson> mbarnett: ok, that's good (ish, it confirms my expectation)
[23:55] <mwhudson> mbarnett: can you try the same on the other importds?
[23:55] <mbarnett> heh
[23:59] <lifeless> sinzui: yeah the test you gave blows up with NoneType in unsibscritable
[23:59] <lifeless> there are also some later failures
[23:59] <lifeless> like the Moderate button is missing?
[23:59] <mbarnett> mwhudson: same