[12:04] <sabdfl> BradB|away: yup
[12:11] <BradB|away> cool
[01:12] !lilo:*! 1097536034  <+lilo> anyone want to donate a book on nonprofit accounting practices to PDPC? it sure would speed up this process
[01:13] !lilo:*! (message me for specifics 8)
[01:30] <sabdfl> daf: around?
[02:01] <kiko> ddaa, have you been adding to interfaces the methods you need to call?
[02:01] <kiko> hey stub
[02:01] <stub> yo
[02:02] <ddaa> kiko: you must me talkng to the wrong person...
[02:02] <kiko> how fares it?
[02:02] <kiko> ddaa, you've been activity-frustrated about zope security issues.
[02:02] <ddaa> Ha... it's unrelated. Not my code.
[02:03] <kiko> ddaa, you might just be calling methods that aren't available because they aren't listed in the relevant interfaces.
[02:03] <kiko> is there such a thing as "not my code" in the big plan of things, though? 
[02:03] <ddaa> My mission is to get a test env for importd.
[02:03] <kiko> it could just be you're requiring stuff that hasn't been exposed externally before.
[02:03] <ddaa> I have not touched launchpad code in any significant way since the last arch team sprint before oxford.
[02:04] <kiko> just from a passers-by POV I mean -- I don't actually know what your problem is, though you might want to write to launchpad-list in hope of answers.
[02:04] <ddaa> I am just using the existing launchpad stuff, doing what I remember of lifeless instructions....
[02:04] <kiko> you should write to launchpad for help, usually you get prompt answers and it's nicer than waiting around on IRC :-)
[02:05] <ddaa> Yeah... I guess so... I expect to be able to annoy lifeless enough so he will consider seriously the issue of setting up a test env for importd...
[02:06] <ddaa> The problem is deeper. I'm not quite sure of what I am supposed to do launchpad-wise... just that I must test the imports with buildbots...
[02:06] <kiko> #launchpad should be responsible for launchpad IMHO, not anyone in particular (though particular experts do exist of course), so complain away and we should be able to help.
[02:06] <kiko> I see
[02:07] <ddaa> quite a frustrating situation...
[02:07] <kiko> can't you get someone to spec it for you? rob I suppose?
[02:07] <ddaa> I'm going bed now. Thanks for the help though.
[02:07] <kiko> wish I could have provided more -- night
[02:07] <ddaa> I asked him last time, he did not seem to think anything more than getting the right config was troublesome...
[02:08] !lilo:*! And, in further news, we've found a place with a used copy of that accounting book.  It's really much cheaper.  Really.  Someone want to help out?  Message me. 8)
[02:08] <kiko> apparently defining the problem well would be a good pre-requisite to explore :)
[02:08] <ddaa> The problem is well defined:
[02:09] <ddaa> Run imports for chosen projects, as defined from info files (or some other sources, but that can come later).
[02:09] <ddaa> And check that the result are sane.
[02:09] <ddaa> Well... actually that's just words for me...
[02:09] <ddaa> No idea what are the prerequisites and outputs of the import process...
[02:09] <kiko> that's what I mean.
[02:10] <ddaa> *shrugs*
[02:10] <ddaa> right... apparently it seemed to be something obvious, so I though I would be able to figure out. Lifeless seemed to expect that much.
[02:11] <ddaa> I try to live up to his expectations generally.
[02:13] <kiko> do you know how the import process works?
[02:14] <ddaa> something involving cscvs to create arch archives...
[02:14] <ddaa> with inputs from sourcesource
[02:14] <kiko> that's as much as I know as well.
[02:14] <ddaa> the sourcesouce must be moved to a project and enabled
[02:14] <kiko> do you know what "sanity" means in the context of a cscvs import?
[02:15] <ddaa> then when running a botmaster/slave pair, the slave should get the job and start doing things.
[02:15] <ddaa> I'm stuck at the "enable job" point.
[02:16] <ddaa> The sanity check is looking at the generated changelog. Lifeless promises that failures cause obviously wrong changelogs.
[02:16] <kiko> I see
[02:16] <ddaa> buildbot is deep voodoo...
[02:16] <kiko> that it may be
[02:17] <kiko> but it appears you're stumbling inside lp, not bb :)
[02:17] <ddaa> yup...
[02:18] <ddaa> there's a zillion things that can go wrong there for someone like me who do not know the land. The error messages do not help....
[02:19] <ddaa> Once it gets to buildbot, I should be able to figure out problems, since I expect most failures to come from tla...
[02:19] <kiko> yes.
[02:19] <ddaa> (except conversion problems...)
[02:19] <kiko> what exactly is triggering the Unauthorized errors?
[02:20] <kiko> can you show me a code snippet?
[02:20] <ddaa> The first thing I try to do is move a souresource from the do-not-sync project to another project.
[02:20] <ddaa> Via the web interface.
[02:20] <ddaa> No code.
[02:21] <kiko> it could just be broken from the Great Renaming. 
[02:21] <kiko> there is some code, it just might not be written by you :)
[02:21] <ddaa> I'm logged in as Foo Bar. I tried importing the sources with owner=Foo Bar, instead of Robert Collins, but then I believe I got some other error...
[02:21] <kiko> what line triggers the error, I mean?
[02:22] <ddaa> I do not quite remember...
[02:22] <ddaa> The launchpad stuff is hermetic to me...
[02:22] <ddaa> I really appreciate the help, but I need to sleep now.
[02:22] <kiko> you can browse the source code -- we'll let you look at it for free <wink>.
[02:22] <kiko> sure, we all do eventually.
[02:23] <kiko> night.
[02:35] <mdz> spiv: #1922-ping?
[03:24] -dmwaters(dmwaters@dmwaters-gentoo.staff.freenode)- {global notice} Hi all! In about 4.5 hours we are going to take a major outage for some server upgrades. Please see 'http://freenode.net/news.shtml' for more info. Thank you for your patience, and thank you for using freenode!
[05:43] <daf> sabdfl: pong
[07:34] -dmwaters(dmwaters@dmwaters-gentoo.staff.freenode)- {global notice}  Hi all! in about half an hour I'm going to start rebooting a few servers for some upgrades. Please see: 'http://freenode.net/news.shtml' for more info.
[07:59] -dmwaters(dmwaters@dmwaters-gentoo.staff.freenode)- {global notice} Alright guys, Here we go with  the updates This shouldn't take long
[08:40] <stub> Can someone please tell me that teams cannot contain teams as members?
[09:27] <lifeless> why shouldn't they? Canonical team includes sysadmin + developers + business dev + distro teams ?
[09:35] <sabdfl> morning all
[09:43] <sabdfl> hey mdz
[09:43] <mdz> morning
[09:44] <sabdfl> 11 RC bugs?
[09:44] <mdz> more or less
[09:44] <sabdfl> daf: around?
[09:44] <mdz> a few of them are debatable and may end up being downgraded
[09:44] <sabdfl> ok. you get  to say when she ships.
[09:45] <mdz> 2 of them we have fixes ready to upload, 2 are branding stuff (one of which is blocked on receipt of the artwork)
[09:45] <mdz> one live-cd-only bug
[09:45] <mdz> we're actually in fairly good shape
[09:45] <sabdfl> i will have the plain and plain+logo images today, who should i send them to?
[09:46] <sabdfl> calendar image will be in today or tomorrow
[09:46] <mdz> spiv reported this bug where his machine catches fire, but it's waiting for yet more information from him
[09:46] <mdz> sabdfl: jdub
[09:46] <sabdfl> ok
[09:47] <sabdfl> tech board meeting this evening / afternoon / morning / 1700 UTC?
[09:47] <sabdfl> or it it 1600 UTC?
[09:47] <mdz> 8 hours from now?
[09:47] <mdz> it was 1600 last time
[09:47] <sabdfl> yes
[09:48] <sabdfl> oh, actually, would be cc meeting, right?
[09:48] <mdz> tech board meeting is going to be very fast unless you have something to bring up
[09:49] <mdz> "release tomorrow lots to do kthxbye"
[09:53] <sabdfl> i think the schedule is actually community council meeting this week
[09:53] <sabdfl> was going to invite tech board to participate and have you sign off on the release process with the cc
[09:53] <sabdfl> sound ok?
[09:53] <sabdfl> will have a release process drafted for the meeting
[09:55] <mdz> I thought tech board was going to be every two weeks
[09:55] <mdz> which would make it today
[09:55] <mdz> but it's perfectly OK if I can sleep instead
[09:55] <mdz> I can attend a CC meeting if needed
[10:09] <sabdfl> morning limi
[10:10] <limi> morning sabdfl :)
[10:15] <stub> yo
[10:27] <Kinnison> Morning
[10:30] <sabdfl> stub: any chance you can do a regular mapping of tables + comments to HTML and commit it? 
[10:30] <sabdfl> there's a useful piece of software to do it
[10:30] <stub> I'll do it every schema update, or do you mean more regularly in case people actually update commentssql?
[10:31] <sabdfl> i guess every schema update would be fine
[10:31] <sabdfl> updates to comments.sql would best be handled through the pending queue too, i think
[10:33] <stub> Hmm.... I suspect that is better handled through cooperation. ie. don't change the meaning of something without discussion, but feel free to comment anything and everything that has no or incomplete definitions.
[10:46] <limi> where are the valid severity and priority values listed?
[10:47] <sabdfl> limi: canonical.lp.dbschema
[10:48] <limi> thanks
[10:49] <sabdfl> stub: doing code review, if those were your vocab patches, then thanks!
[10:50] <stub> np
[10:51] <sabdfl> hmm... except maybe for the zcml locations
[10:51] <sabdfl> you don't think it's better to keep the vocabulary definition for object foo together with everything else about object foo?
[10:52] <stub> Nope - the zcml should live with the code since they need to be edited in tandem. 
[10:55] <stub> I'm unsure if canonical.widgets should stay where it is or move to canonical.launchpad.widgets
[10:57] <limi> why do *I* always have to find all the Mozilla rendering bugs? ;)
[11:00] <sabdfl> limi: it's a special privilege
[11:00] <limi> where did I sign up? :] 
[11:01] <Kinnison> It was part of your contract
[11:01] <limi> damn
[11:02] <Kinnison> sabdfl: I'm currently doing the dbschema.py changes to support the new meaning of {Sourcep,P}ackagePublishing.state. Were the notes I put on the wiki okay? If so; I'll carry on.
[11:02] <sabdfl> Kinnison: rebooting, pls send url when i'm back
[11:32] <ddaa> Hello.
[11:32] <Kinnison> Morning ddaa
[11:34] <spiv> mdz: I'm working on gathering information for #1922 as fast as I can, but it triggers pretty much at random. :)
[11:35] <spiv> mdz: Sometimes it happens even before booting has finished, sometimes I've gone a whole day without it occuring.
[11:44] <limi> ah, gotta love bugzilla.mozilla - the bug regarding Moz not supporting multi-line tooltips (title attribute) evolves to the following after 141(!) comments:
[11:44] <limi> > One of the questions is should we using \uFFFD if JS/DOM is only UTF-16, if not
[11:44] <limi> > then should we go back to using "?"
[11:57] <limi> is there any way to say to arch "get rid of all conflicts and get the stuff from rocketfuel as the authoritative versions?"
[11:57] <limi> I have an insane amount of conflicts here in files I haven't touched
[12:00] <ddaa> limi: let's serialize the issues.
[12:00] <limi> let's not :] 
[12:01] <limi> if it isn't a simple command to do it, I'll ignore it for now
[12:01] <ddaa> Okay. All your stuff is merged into rocketfuel and you just want your tree to be set to the latest rocketfuel?
[12:01] <limi> just wanted to commit a simple spelling error fix in dbschema
[12:01] <limi> well, most of it is - I can afford to lose what is there
[12:01] <limi> just minor edits there now, I believe
[12:02] <Kinnison> limi: I'm currently prodding at dbschema -- if you want to punt me a diff I'll apply it here while you tidy up
[12:02] <sabdfl> limi: i have the subrosasoft cd waiting here for when you come by to install warty. much fast tla performance :-)
[12:02] <ddaa> Are there revisions in your branch which were not merged into rocketfuel?
[12:02] <limi> Kinnison: in BugSeverities, there's a status that says "but" instead of "bug" - I think it was in Severity Major
[12:03] <limi> ddaa: no idea, if I can just get what's currently in RF, that's fine
[12:03] <Kinnison> limi: found that
[12:03] <sabdfl> SteveA: is the password reset supposed to be working?
[12:03] <limi> ok
[12:03] <sabdfl> i am trying to rset the passwd for mark@hbd.com
[12:03] <Kinnison> limi: Any others?
[12:03] <Kinnison> sabdfl: https://wiki.canonical.com/Lucille_2fPublishingNotes
[12:03] <Kinnison> sabdfl: https://wiki.canonical.com/Lucille_2fQueueNotes
[12:03] <limi> nope, that was it
[12:04] <Kinnison> limi: that'll go into my next pqm request htne
[12:04] <limi> thanks
[12:04] <Kinnison> s/htne/then/
[12:04] <ddaa> Well, in any case: % tla undo $(tla revisions --full --reverse rocketfuel@canonical.com/launchpad--devel--0 | head -1) && tla sync-tree $(tla tree-version)
[12:05] <ddaa> Will give you a tree which is latest rocketfuel plus any additional patchlog in the latest revision of your branch.
[12:05] <spiv> ddaa: I'm seeing conflicts too, and my tree was previsouly up-to-date with rocketfuel, I though...
[12:05] <ddaa> Dunno if that qualifies as "one simple command" :-)
[12:06] <Kinnison> ddaa: For a value of simple known only to tla developers :-)
[12:06] <spiv> ddaa: also, I've discovered another downside to changing the tagging-method for .rej/.orig -- tla undo now refused to work in the prescnece of conflicts :(
[12:06] <ddaa> It's a bit like photo-touching programs. You have words with which you can make sentences.
[12:07] <limi> if you make your sentences in photo programs, you have a problem anyway ;)
[12:08] <ddaa> spiv: which makes some sense. You can do "tla inventory -s | grep -e '\.(orig|reg)$' | xargs rm" (untested) to remove the conflict files.
[12:08] <ddaa> oops
[12:08] <ddaa> tla inventory -j | grep -e '\.(orig|reg)$' | xargs rm
[12:08] <limi> ah, why didn't I think of that :] 
[12:09] <ddaa> or more simply
[12:09] <ddaa> tla inventory -j | xargs rm
[12:09] <ddaa> Which _should_ be obvious...
[12:09] <ddaa> Hu...
[12:09] <spiv> ddaa: Ah, that's nicer than grepping tla tree-lint's output ;)
[12:10] <ddaa> tla inventory -u
[12:10] <Kinnison> ddaa: since when has tla been obvious? </troll>
[12:10] <ddaa> !!!
[12:10] <spiv> ddaa: I'd prefer it if undo just undid anyway, or had a --yes-really-undo optoin, or something.
[12:10] <ddaa> -j is "junk", here the rejects are "unrecognized", thus -u
[12:10] <ddaa> spiv: your request makes perfect sense.
[12:10] <ddaa> From a user perspective.
[12:11] <ddaa> Tla makes sense from a tomlord perspective.
[12:11] <ddaa> That tend to be slightly different things...
[12:11] <spiv> How should I put his...
[12:11] <spiv> s/his/this/
[12:11] <spiv> I don't care about Tom Lord ;)
[12:12] <ddaa> the hct folks supposedly are working towards easing your pain.
[12:12] <spiv> Seriously, what's the best thing to do here, wrt to getting undo to "do the right thing" for users?
[12:12] <limi> sabdfl: (SubRosaSoft) yeah, looking forward to see how Warty works on my Powerbook - I believe there is a problem with the WiFi card (Broadcom?)
[12:12] <spiv> Pester g-a-u?  Use raw?
[12:12] <ddaa> spiv: you should define "the right thing" a bit more accurately...
[12:13] <spiv> ddaa: Ok, let's start with my naive user's view of what "undo" actually means :)
[12:13] <spiv> ddaa: Then look at where "tla undo" falls short :)
[12:14] <ddaa> In my opinion, the path of least resistence is to implement stuff in a wrapper until we hit the point where we need to reimplement some major tla functionality.
[12:14] <sabdfl> Kinnison: w.r.t. scheduleddeletiondate please check that the date is never too far in the future
[12:14] <spiv> I expect undo to mean, essentially, "revert".  Restore the working tree to the same state as if I'd just done a few tla get, modulo files tla doesn't care about (,,*, and nested working trees, etc).
[12:14] <sabdfl> assert (scheduleddeletiondate<now+2 days) sort of thing
[12:14] <spiv> s/few/fresh/
[12:15] <Kinnison> sabdfl: do you want that done when it's set; each time it's checked for having expired, or as part of a cruft checker?
[12:15] <sabdfl> Kinnison: your call
[12:15] <sabdfl> SteveA and i did discuss have an "oscar the grouch" process, which basically runs a set of tests on the db
[12:15] <sabdfl> sort of higher-level sanity checking than db constraints
[12:16] <sabdfl> and one test might be that.
[12:16] <spiv> ddaa: my gripe is that occasionally tla undo refuses to undo for what seems to this naive user to be no good reason.  I'm not sure where the poor conflict handling fits in this picture :)
[12:16] <ddaa> are backup and unrecognized, "files tla do not care about". What should happen to them?
[12:16] <Kinnison> So even if there isn't a generic one; there'll be a lucille one
[12:16] <spiv> ddaa: I would expect them to be untouched.
[12:17] <ddaa> So if you undo after conflicts, you'll end up with same conflict files. Which, per your other wish, will prevent commit.
[12:17] <spiv> ddaa: Hence my comment about not being sure where poor conflict handling fits in this ;)
[12:18] <spiv> ddaa: Can I take teh discussion on a tangent for a moment? :)
[12:18] <ddaa> No pb. You are the one asking :)
[12:18] <spiv> Thanks :)
[12:18] <Kinnison> sabdfl: I'll put a note in the wiki that the scheduleddeletiondate should be sanity-checked in 'oscar'
[12:19] <spiv> The use of unrecognised for .rej and .orig is a pretty crude hack that I found suggested on g-a-u.  The undo problem highlights just how crude it is :)
[12:19] <ddaa> PS: I'm not trying to prove that what want to do is Bad, I'm just trying to make you spell out clearly what you want so I can help more effectively.
[12:19] <spiv> ddaa: Understood -- and appreciated. . I haven't thought these ideas out fully yet, and you're helping me do that :)
[12:19] <ddaa> spiv: yup, crude hack is an accurate qualification.
[12:20] <spiv> What I think I'd prefer is something more like SVN's model here.
[12:20] <spiv> i.e. in the metadata about that file in the working tree, flag it as conflicted (in the inventory?)
[12:20] <spiv> So that you'd have to say tla resolved-conflict filename
[12:21] <ddaa> I think the inventory classification concept is basically a good idea, but they way it is done get very much in the way now.
[12:21] <spiv> Until then, tla would keep reporting that file as conflicted, regardless of .rej/.orig files, or evil CVS <<<<< markers, or even if the file is removed :)
[12:22] <spiv> Then the crdue tagging-method hack I'd use would probably be to mark .rej and .orig as source, sinstead :)
[12:23] <spiv> (or maybe I wouldn't use any...)
[12:23] <spiv> (aside: I don't know how .rej/.orig cope with multiple conflicts on the same file from multiple merges, because I've never tried doing anything that hard :)
[12:24] <spiv> So, back to undo.
[12:24] <ddaa> metadata is one of those words which reach far into what does not exist yet in Arch, and what is very polemic. For example it touches issues of transcoding, magic tree data munging and so on, which are popular among new users but quite impopular among devel afaitt.
[12:24] <spiv> You must have a different definition of metadata to me :)
[12:25] <ddaa> What is and what is not metadata is not very clear to start with...
[12:25] <spiv> I'm not looking for a generic "associate value foo to file bar with key x" stuff.  I'm just talking in the sense of what {arch} and .arch-ids already are.
[12:26] <spiv> i.e. forom the point of view of a user working with their source code, the source is the data, and everything else arch records other than the actual contents of the files is metadata.
[12:27] <ddaa> You are speaking of associating a flag to a file. You have to think about how it works in terms of file ids. For example tagline-tagged file have no associated id file, yet you want to associate a conflict flag to them. How do you do that in a way which is rename-resilient?
[12:28] <spiv> ddaa: By using the file's id -- each file has a unique ID, right?
[12:28] <ddaa> right
[12:29] <spiv> (there's an edge-case when there's a newly-created file locally not commited, then a conflicting merge with the same file name added, but I don't carea bout that case ;)
[12:29] <ddaa> btw, multiple conflicts for the same file are not coped with. Older rejects are moved in a precious directory with some random crappy name.
[12:30] <spiv> So, with no knowledge of how tla's internals work, I'd expect that, e.g. a {arch}/conflicts directory would do the job.
[12:30] <spiv> With a file in that dir per fiel that is conflicted.
[12:31] <spiv> keyed by some fs-safe encoding of file-id.
[12:31] <ddaa> Would have to be something like {arch}/++conflicts or something.
[12:32] <spiv> Sure.
[12:32] <spiv> Does that sound workable and reasonable to you, or am I missing something?
[12:33] <limi> today's Ubuntu misspelling: "Bununtu"
[12:33] <spiv> Bananatutu!
[12:33] <ddaa> spiv: that starts to sound like a sane idea. I suggest you poke jblack with the keyword "manifesto" with my recommendation about this idea.
[12:34] <limi> Collect them all!
[12:34] <spiv> makes me want to dress a banana up in a tutu ;)
[12:34] <spiv> ddaa: Great.  I'll do that.  Thank you :)
[12:35] <ddaa> considering the way things are currently not evolving in the gnu arch community, I fear that we will have to do our own sauce sooner or later...
[12:38] <ddaa> What's great with that name, is that it's recognisable even when horribly misspelt.
[12:42] <ddaa> spiv: the idea is to have you champion this 
[12:43] <ddaa> feature, design a complete, peer-reviewed specification
[12:43] <spiv> ddaa: That's fine by me.
[12:43] <ddaa> Then when it's ready put it on the community. That should help save a lot of discussion noise.
[12:43] <spiv> :)
[12:43] <SteveA> sabdfl: there's a problem with signing up on the ubuntulinux site -- actually two problems.  1. the join form is not linked from the front page.  2. the join form doesn't work because the xml-rpc authentication server hasn't been updated to know about Person.name (the nickname)
[12:44] <spiv> ddaa: So, another issue. :)
[12:44] <sabdfl> i have an existing account i think in the db
[12:44] <ddaa> And that will let some time for the devel process situation to get sorted...
[12:45] <spiv> ddaa: why is it I'm getting conflicts in files I've never touched when star-mergeing from rocketfuel?
[12:45] <spiv> ddaa: Even though I don't think I've got any local unmerged changes in my version?
[12:45] <ddaa> spiv: probably because you send merge request, then merged before it was completed.
[12:46] <spiv> ddaa: It's possible, but I'd be surprised.  I've been very careful about not doing that :)
[12:46] <spiv> Of course, I'm only human :)
[12:46] <spiv> (But it sounds like what limi saw too)
[12:47] <ddaa> I'll look at your tree.
[12:47] <spiv> ddaa: thanks :)
[12:48] <spiv> ddaa: If I have no unmerged changes, then I can wrokaround this with tla tag -S rocketfuel@... andrew.bennetts@canonical.com/launchpad--devel--0, I guess?
[12:48] <ddaa> No need for -S here.
[12:49] <spiv> (purely hypothetically at this stage, I'll wait for your diagnosis)
[12:49] <spiv> Oh, of course :)
[12:49] <ddaa> spiv: yup that could work.
[12:49] <spiv> (plus I'll want to double-check I don't have any unmerged changes)
[12:50] <ddaa> but bear with me a minute, I was in the process of typing the name of your version....
[12:50] <ddaa> 197 patches, that's going to take a while...
[12:52] <spiv> Maybe now would be a good opportunity to point me at some good docs on cacherevs ;)
[12:53] <ddaa> There was a thread on gau just yesterday about the issue of cachedrevs and mirrors.
[12:53] <spiv> Ok, I'll take a look...
[12:54] <ddaa> Well... I think so... maybe that was on #arch.... but there definitely is some cachedrev related talk on gau...
[12:55] <spiv> There was, but it didn't seem to involve mirrors.
[12:56] <lifeless> raw should autocache rev
[12:56] <lifeless> we need to get everyone using raw.
[12:56] <lifeless> thats more and more evident.
[12:56] <ddaa> Basically, mirrors do not get cachedrevs after mirroring the revision the first time.
[12:56] <lifeless> ddaa: integration allows this.
[12:57] <ddaa> integration gives you a way to force updating the mirror for a specific _revision_.
[12:57] <lifeless> ddaa: yes - which you would then do when you cacherev locally
[12:57] <ddaa> That would work well in your situation since you are the one pushing the mirror.
[12:58] <ddaa> Of course that would not help mirrors maintained by other people.
[12:58] <lifeless> well they can mirror locally, its only ever a problem when mirror-name != canonical name
[12:58] <lifeless> hows the test-environment ?
[12:58] <ddaa> The archive format is quite stupid in the way it stores cachedrevs, and makes it very expensive to check for what cachedrevs are present.
[01:00] <ddaa> lifeless: I decided to shortcut lp entirely. Actually when looking at master.cfg and wrapping the sql limit into something human readable things make a lot more sense.
[01:00] <lifeless> ddaa: are you able to run test-imports yet? thats the crux.
[01:00] <ddaa> Probably I'll end up with a handful of scripts and a specific filter.
[01:00] <ddaa> lifeless: no
[01:00] <ddaa> spend waaaay too much time fighting with launchpad.
[01:01] <ddaa> I should not have gone down the "keep production filter" road to start with.
[01:02] <ddaa> lifeless: I'd like if you could bless a "botmaster config for testing" version.
[01:02] <ddaa> Prolly on rocketfuel, so I could put a configuration useful to everyone there.
[01:02] <lifeless> ddaa: for this testing, we use the production config.
[01:02] <ddaa> The production dists points to lifelesslap.
[01:03] <lifeless> which I don't recall making any changes to that aren't in tla already.
[01:03] <lifeless> yes
[01:03] <ddaa> Which is not really a version where I can put changes.
[01:03] <lifeless> ddaa: exactly, its production, you're not meant to put changes.
[01:03] <ddaa> And whose up-to-dateness i question.
[01:03] <ddaa> Mh...
[01:03] <lifeless> 'which I don't recall making any changes to that aren't in tla already'
[01:03] <ddaa> Yes...
[01:03] <ddaa> (13:02:43) lifeless: ddaa: for this testing, we use the production config.
[01:04] <ddaa> You realise that is wishful thinking?
[01:04] <ddaa> I'm not on the production machine, not with the production data, the config has to be different.
[01:05] <ddaa> Mhh... sounds like there is confusion between "arch config" and "botmaster config"...
[01:05] <lifeless> ddaa: the database name is imported from canonical.lp, so the botmaster config doesn't need to change at all other than the locations to create archives.
[01:06] <ddaa> and private.py stuff, and the name and location of slaves
[01:07] <lifeless> true, and those are already documented.
[01:07] <ddaa> Well, if you insist that I use the production filter I can do, but since I'm going to poke the db w/o using launchpad, I may as well use some hand made filter to make life easire.
[01:07] <ddaa> For some value of "documented"...
[01:07] <ddaa> The ImportProcess page has grown almost completely stale.
[01:07] <lifeless> you can just disable the production filter - set that parameter to ''
[01:07] <lifeless> or None
[01:08] <lifeless> ddaa: the ImportProcess page shouldn't be stale at all, almost nothing has changed.
[01:08] <lifeless> now, these deadlocks.
[01:08] <ddaa> Then why was I unable to run buildbot? Either something has changed or i was being stupid.
[01:09] <lifeless> ddaa: I don't know, I'm happy to help you debug whats going on.
[01:09] <lifeless> s/happy/keen to/
[01:09] <ddaa> Okay, let's talk about this "deadlock" problem.
[01:09] <ddaa> Since I was unable to run imports, I was unable to test that.
[01:10] <lifeless> I suspect pyarch, because it didn't happy before, and pyarch is the only buidlbot component in the code path that has changed.
[01:10] <lifeless> the symptoms are:
[01:10] <lifeless> we get an extra python process, not in the group, blocked on a futex.
[01:11] <lifeless> the import process stops cold for one job.
[01:11] <lifeless> if I kill -9 that extra process, the import process resumes.
[01:11] <lifeless> I wanted to ask, why are you using a Queue ? Its not needed for twistd.
[01:12] <lifeless> or at least, I don't think its needed.
[01:12] <ddaa> The queue is used to translate the asynchronous flow of events in the reactor into a synchronous sequence used in the thread using the pyarch API.
[01:13] <lifeless> you should use a deferred for that.
[01:13] <ddaa> Since the pyarch API is blocking, the queue is used to block the thread when no data is present.
[01:14] <ddaa> I do not see how deferred would help here. I have talked about it with spiv, and he agreed that a synch-queue was the least bad solution there. But then maybe I did not explain the problem properly.
[01:14] <lifeless> hmm.
[01:16] <ddaa> Besides, if python forks, and your are using TwistedSpawningStrategy, it's all happening in twisted stuff. There should not be _any_ low-level process handling left.
[01:18] <lifeless> well something is deadlocking, and the only thing I can think of is a dropped message
[01:18] <ddaa> "dropped message"?
[01:19] <lifeless> lets get you able to test this.
[01:19] <lifeless> what is the current status ?
[01:19] <ddaa> Been talking with lifeless since the last status.
[01:20] <ddaa> I'm about to set the filter to the empty string.
[01:20] <lifeless> try that, I'll be here
[01:20] <ddaa> And import only one info file.
[01:20] <ddaa> So I'm currently resetting the database.
[01:20] <lifeless> huh
[01:20] <lifeless> don't reset the database
[01:20] <ddaa> too late.
[01:20] <lifeless> that would waste time and lose all your changes
[01:21] <ddaa> it's running
[01:21] <lifeless> such as imported archives etc etc etc
[01:21] <ddaa> My changes are unimportant.
[01:21] <ddaa> ?
[01:21] <lifeless> the database has important persistent data in it.
[01:21] <ddaa> why would imported archives be important? (ignoring for a moment that there are currently none)
[01:22] <lifeless> when imports complete for instance, they get 'taxied' into the database so that launchpad can see everything about them.
[01:22] <ddaa> How is that useful for testing imports. I'll try to stay away from launchpad for this tests. It's way too much trouble.
[01:23] <lifeless> its important because its part of the end to end test.
[01:24] <ddaa> a52dec is known to work, right?
[01:24] <lifeless> yes
[01:25] <lifeless> you'll need this patch - about to commmit is
[01:25] <lifeless> --- orig/lib/canonical/arch/broker.py
[01:25] <lifeless> +++ mod/lib/canonical/arch/broker.py
[01:25] <lifeless> @@ -598,8 +598,7 @@
[01:25] <lifeless> 
[01:25] <lifeless>      def set_patchlog(self, patchlog):
[01:25] <lifeless>          self._patchlog = patchlog
[01:25] <lifeless> -        from canonical.database.launchpad import RevisionMapper
[01:25] <lifeless> -        mapper = RevisionMapper()
[01:25] <lifeless> +        mapper = database.RevisionMapper()
[01:25] <lifeless>          mapper.update_log(self, patchlog.summary)
[01:25] <lifeless> 
[01:25] <lifeless>      def patchlog(self):
[01:25] <ddaa> I have to leave to feed the old witch.
[01:26] <lifeless> ok
[01:26] <limi> sabdfl: sent you the first version of the portlet/priority/severity scheme, ping me when you have time to give some feedback
[01:26] <lifeless> is a53dec running ?
[01:26] <ddaa> okay, I'll give a try before leaving :)
[01:26] <lifeless> we've got 30 minutes to the meeting.. you will be back right ?
[01:26] <ddaa> That's the point of leaving now.
[01:27] <lifeless> ok.
[01:29] <ddaa> apparently the empty clause causes an error
[01:30] <lifeless> try None when you return please.
[01:35] <lifeless> ddaa: I found the problem I think
[01:46] <lifeless> yeah, wasn't actually using the twisted strategy.
[01:46] <lifeless> can you do the following for me please ?
[01:46] <lifeless> In pyarch's __init__, if there is a top level symbol twisted when you are imported, set the use of twisted automatically ?
[02:00] <stub> SteveA: I can do a quick fix for ubuntulinux.org by adding a DEFAULT to Person.name.
[02:01] <stub> SteveA: (On the production database)
[02:06] <SteveA> stub: I think it is easier if spiv fixes the auth server to do the right thing.
[02:07] <stub> yup
[02:07] <SteveA> stub: there seems to some strange interaction between plone and the auth server, like plone is cacheing authentication information for a while.
[02:07] <stub> Just offering an alternative ;)
[02:07] <SteveA> the launchpad change-password stuff is working, and putting the new password in the database 
[02:07] <stub> exUserFolder has a lot of caching if I remember. It should be tunable (?)
[02:08] <SteveA> sometimes, you can't log in to plone with the new password immediately
[02:08] <SteveA> hmm
[02:08] <SteveA> so, we should perhaps turn that off, or maybe flush the caches on a new login 
[02:08] <stub> Flush the caches, preferably just the affected user.
[02:09] <SteveA> right
[02:09] <stub> Hitting the xml-rpc server every request is a bit mad
[02:11] <SteveA> BradB: ping
[02:17] <spiv> BradB: ping
[02:18] <BradB> pong
[02:18] <BradB> pong
[02:18] <spiv> :)
[02:18] <BradB> spiv: I can try changing my password and relogging in.
[02:18] <spiv> BradB: Ok.
[02:19] <spiv> I've gotten watching the logfile on the authserver down to a fine art ;)
[02:19] <BradB> Worked fine.
[02:20] <spiv> Intriguing.
[02:20] <BradB> Maybe SteveA somehow tried authenticating via http://www.ubuntulinux.org/login_form, which won't work.
[02:20] <spiv> But unlike SteveA, you changed it via plone... what about via the production launchpad?
[02:21] <spiv> (I can tell, because I saw a changePassword call in the authserverlogs)
[02:22] <spiv> BradB: btw, your SQLObjectGuide changes looked good to me.
[02:22] <BradB> spiv: Do you have anyway of verifying the login_form he was hitting when the login failed for him?
[02:22] <spiv> No, unfortunately.
[02:22] <BradB> hm
[02:23] <spiv> He's aware of the http vs. https issue, though, so I presume he was careful to avoid that.
[02:23] <BradB> okay, so how about if you try changing it in the prod db?
[02:23] <spiv> Ok.
[02:25] <BradB> ouch, AttributeError
[02:26] <BradB> second time through worked fine though
[02:26] <spiv> Ouch.
[02:26] <BradB> first time through i was logged in, just got that error message though
[02:26] <spiv> Hmm.
[02:28] <spiv> I'm stumped... any thoughts?
[02:29] <spiv> I need to do lunch.
[02:30] <spiv> Anything you want from me immediately before I disappear? :)
[02:31] <BradB> no don't think so :)
[02:32] <spiv> Ok.
[02:34] <lifeless> spiv: I could use some help
[02:34] <lifeless> before you dissapear
[02:36] <spiv> lifeless: Ok.
[02:36] <spiv> You're just in time ;)
[02:36] <ddaa> lifeless: couple of questions in my queue
[02:37] <lifeless> remember the canonical.arch.database module ?
[02:37] <spiv> Yep.
[02:37] <lifeless> it had a commit method, and a connect method ?
[02:37] <lifeless> I need replacements for those  for buildbot..
[02:37] <spiv> Right.  You need that for taxi?
[02:37] <lifeless> BINGO.
[02:37] <ddaa> 1. okay for archive_mirror_dir, now what is th "slave_home" argument to processDB for?
[02:37] <spiv> lifeless: Ok, that will be done today, for real this time :)
[02:38] <spiv> lifeless: Is taht soon enough?
[02:38] <spiv> i.e. can I have lunch :)
[02:38] <lifeless> ddaa: that is where the slaves home dir is - which is used to create the .gnupg check scripts and so forth.
[02:38] <lifeless> spiv: ok, I'll crash early, and get up early... so I can test it before you go to bed tonight
[02:38] <lifeless> is that ok - will you have net at my 8am ?
[02:38] <spiv> lifeless: I have net 24hrs here :)
[02:38] <lifeless> spiv: woohoo.
[02:39] <lifeless> and is my 8am ok for you ?
[02:39] <spiv> 9.5 hrs from now?
[02:39] <ddaa> 2. about pyarch/twisted you suggest that, when arch._tla is imported, if sys.modules has a key which is "twisted" or starts with "twisted." then spawn is initialized to an instance of TwistedSpawningStrategy instead of PyarchSpawningStrategy. Right?
[02:40] <spiv> lifeless: Yeah, just :)
[02:40] <lifeless> spiv: lynne wakes my @ 7:30 tomorrow, guaranteed.
[02:40] <lifeless> I'm pushing my buildbot changes to rf now, so you'll have the latest to play^Wfix.
[02:40] <lifeless> ddaa: right.
[02:41] <spiv> lifeless: Great.  Thanks!
[02:41] <ddaa> [importd]  slave_home has something to do with os.env['HOME']  should it be the same? Is it used to set the env variable? Should there be a .arch-params directory there?
[02:41] <lifeless> ddaa: it may differ from os.env['HOME] .
[02:42] <lifeless> ddaa: you should ahve a directory in the directory slave_home points at called 'gpg'
[02:42] <lifeless> in there you need the arch@canonical.com.pub and arch@canonical.com.secret gpg files
[02:42] <lifeless> ls
[02:42] <lifeless> bah
[02:43] <lifeless> also you need a dir called 'archives', its in there the imported archives will be created
[02:44] <ddaa> Those are the first-stage, undoable archives, right?
[02:44] <lifeless> ddaa: right.
[02:44] <ddaa> Which have corresponding mirrors in archive_mirror_dir.
[02:45] <lifeless> right
[02:47] <ddaa> I probably need to get it out... my parents are out so I must attend things like that.
[02:48] <ddaa> The last forseeable issue is the clause argument. I'll try with None.
[02:48] <ddaa> lifeless: I release you. I'll report back in one hour.
[02:48] <lifeless> ddaa: I'm going to bed early, to get up early for spiv
[02:48] <lifeless> so, if you can, lets burn through now and get it going minimally.
[02:49] <lifeless> then you have all the time you need to 'walk the dog'
[02:50] <ddaa> Works better with clause=None. I get job in the slave!
[02:51] <lifeless> cool
[02:51] <lifeless> now try to run the job
[02:51] <ddaa> Sure. How?
[02:51] <ddaa> Nothing seems to happen here.
[02:51] <lifeless> click on the word 'waterfall'
[02:51] <lifeless> then on the job the job title
[02:52] <lifeless> that should give you a form with two push buttns
[02:52] <lifeless> push the top button
[02:52] <lifeless> which should take you back to the waterfall 
[02:52] <lifeless> what is the top of the waterfall now ?
[02:53] <ddaa> There's a zillion occurences of the same job here... not that slave_home and archive_mirror_dir are not set to something sane yet.
[02:54] <lifeless> what do you mean ?
[02:54] <ddaa> a52dec-HEAD-import.job occurs on the waterfall page something like 100 times.
[02:55] <lifeless> whats the URL
[02:55] <ddaa> After forcing the build, they all turned to yellow-building
[02:55] <lifeless> ahha
[02:55] <ddaa> http://localhost:8008/, for what it's worth
[02:55] <lifeless> thats the site, not the full url unless you've changed something
[02:55] <lifeless> it should be /importd/status/ ...
[02:56] <ddaa> http://localhost:8008/importd/status => No such Builder 'importd'
[02:56] <lifeless> ok, you've changed master.cfg.
[02:56] <ddaa> sure
[02:57] <lifeless> ok, the url / for you is the summary view, not the waterfall.
[02:57] <ddaa> my slave is called slave, that's what is in the Makefile, and is needed for all the useful targets there.
[02:57] <ddaa> Anyway, the slave failed because the slave_home is not set properly.
[02:59] <lifeless> as for the bazjillion jobs, try visiting /reload and see if that helps.
[02:59] <limi> sabdfl: there?
[02:59] <sabdfl> limi: yes, back
[02:59] <limi> great
[02:59] <lifeless> if that doesn't, then your database has a bazjillion jobs in it :)
[02:59] <limi> let me wrap up the latest changes
[02:59] <limi> and send them to you
[03:00] <Kinnison> spiv: ping?
[03:00] <sabdfl> limi: thanks, will review, commit and comment on irc
[03:01] <BradB> SteveA: ping
[03:01] <limi> probably not yet in committable state, will fix that shortly
[03:01] <limi> focused on the visual representation so far
[03:02] <Kinnison> Has anyone encountered an issue with dbconnection not reusing connections?
[03:02] <ddaa> lifeless: should the gpg keys provide a arch@canonical.com uid, or can they just be everyone's favourite snakeoil signatures?
[03:02] <ddaa> s/signatures/keys/
[03:03] <lifeless> ddaa: thyey must, or the signing script will fail
[03:03] <sabdfl> limi: are you at least doing a regular star-merge from rocketfuel so that what you send me includes everyone elses updates?
[03:03] <limi> my arch is not working at the moment, same problems as spiv had
[03:04] <limi> these portlets are not in arch at the moment anyway, I believe?
[03:05] <lifeless> ddaa: gpg --no-default-keyring --keying arch@canonical.com.pub --secret-keyring arch@canonical.com.secret  --gen-key
[03:05] <lifeless> IIRC
[03:05] <ddaa> lifeless: thx, looks sane...
[03:05] <lifeless> may need an extra --no-default-secret-keyring or something, dunno
[03:05] <limi> sabdfl: sent
[03:06] <Kinnison> Erm
[03:06] <Kinnison> You should use ./arch@....
[03:06] <Kinnison> otherwise gpg will create them in ~/.gnupg
[03:06] <lifeless> Kinnison: err, yea.
[03:06] <sabdfl> limi: they are in arch. will revert shortly, thank you!
[03:06] <limi> ok
[03:07] <limi> I should probably try to sort out my local repo then
[03:07] <limi> what I sent you contains everything in one file
[03:07] <limi> will componentify them shortly
[03:08] <limi> just need some feedback on the general direction + some guidance on what is most important and placement of things inside Malon
[03:08] <limi> e
[03:09] <lifeless> spiv: found a regression in buildbot, can't merge till I fix it. please do what you were going to do anyway, I'll cherry pick the buildbot bits of the fix from you.
[03:09] <lifeless> thanks!
[03:10] <Kinnison> Anyone here an sqlobject/sqlconnection/whatever guru?
[03:12] <ddaa> lifeless: /reload does not help the job duplication
[03:13] <lifeless> run up psql against launchpad_test
[03:13] <lifeless> select count(*) from sourcesource;
[03:14] <ddaa> one row only
[03:16] <lifeless> now that is bizarre.
[03:16] <lifeless> it suggests something wrong in the query that sqlobject is making
[03:16] <ddaa> It's just the same job being displayed multiple times, it seems.
[03:16] <lifeless> yah, theres dedup code in the guts.
[03:17] <ddaa> Wow, getting somewhere: _sqlite.DatabaseError: no such table: revision
[03:17] <lifeless> hrm.
[03:17] <lifeless> thats a cscvs failure of some sort.
[03:17] <lifeless> well, more than some sort.
[03:17] <lifeless> its a corrupt cscvs cache.
[03:17] <ddaa> File "/home/david/home/devel/canonical/dists/launchpad/lib/CVS/StorageLayer.py", line 191, in __init__
[03:17] <ddaa> 	    self.getCursor().execute("delete from changeset where csnum in ( select distinct csnum from changeset except select distinct csnum from revision )")
[03:18] <lifeless> find -name 'Catalog.sqlite' | xargs rm
[03:18] <ddaa> That's not the deepest call, but that seems to be the significant one
[03:18] <lifeless> then run the job again.
[03:19] <ddaa> CVS.InvalidVersionError: Could not determine CVS version from CVS version string ""
[03:19] <limi> I get lots of this when trying to bring my tree up to date:
[03:19] <limi> Duplicated ids among each group of files listed here: 
[03:19] <limi> database/schema/.arch-ids/patch-2-07-0.sql.id
[03:19] <limi> E_Stuart_Bishop_<stuart.bishop@canonical.com>_Tue_Oct__5_19:06:48_2004_21645.0 
[03:19] <limi> database/schema/archive/.arch-ids/patch-2-07-0.sql.id 
[03:19] <limi> lib/canonical/launchpad/templates/.arch-ids/bugsystem-edit.pt.id
[03:19] <limi> E_Mark_Shuttleworth_<mark.shuttleworth@canonical.com>_Mon_Oct__4_23:39:57_2004_1
[03:20] <limi> 0357.1 
[03:20] <limi> lib/canonical/launchpad/templates/.arch-ids/bugtracker-edit.pt.id 
[03:20] <lifeless> limi: do a tla tree-lint
[03:21] <limi> I did, doesn't help
[03:21] <lifeless> that will list the problem files.
[03:21] <kiko> wtf is that, .arch-ids/bugsystem-edit.pt.id
[03:21] <lifeless> did you have any uncommitted changes ?
[03:21] <lifeless> kiko: its the equivalent of a CVS Entries file line.
[03:22] <limi> lifeless: not really, I just want my tree to be the same as current RF
[03:22] <lifeless> limi: ok.
[03:22] <kiko> a latin-1 filename with spaces in it?
[03:22] <kiko> ?
[03:22] <lifeless> kiko: no, I don't se that.
[03:23] <ddaa> I see it too.
[03:23] <limi> kiko: I run UTF-8 on IRC
[03:23] <lifeless> limi: ok, so first we need to fix the lint, then we can undo and correct it.
[03:23] <limi> lifeless: ok, the lint lists the same files
[03:23] <lifeless> limi: tla tree-lint -d
[03:23] <kiko> hmmm. odd.
[03:23] <lifeless> that should list just a bunch of filenames IIRC.
[03:23] <limi> nope
[03:23] <limi> lists everything
[03:24] <lifeless> if it does, great, if it doesn't paste a line or two.
[03:24] <limi> lib/canonical/launchpad/templates/bugsystems-index.pt   x_Mark_Shuttleworth_<mark.shuttleworth@canonical.com>_Mon_Oct__4_21:56:28_2004_8303.0
[03:24] <limi> lib/canonical/launchpad/templates/bugtrackers-index.pt
[03:25] <lifeless> tla tree-lint -d | awk '{ print $1}' | xargs rm
[03:25] <lifeless> now
[03:25] <lifeless> tla tree-lint
[03:27] <lifeless> there will be other errors now because I'm brute-forcing the fix.
[03:27] <limi> These files would be source but lack inventory ids (`tla add' or a tagline perhaps?):
[03:27] <limi> lib/canonical/soyuz/sql.zcml.save
[03:27] <lifeless> thats ok
[03:27] <limi> I assume that is a temp file
[03:27] <limi> ok, so tree-lint is empty
[03:27] <lifeless> any other headings ? (like 'missing files for id') ?
[03:27] <lifeless> tree-lint is empty? cool.
[03:27] <lifeless> now tla undo -n
[03:28] <limi> ok
[03:28] <lifeless> which will undo the local changes that a partial merge may have caused.
[03:28] <limi> and sync with RF?
[03:28] <lifeless> now you should be able to star-merge
[03:28] <limi> aha
[03:28] <limi> whoa, it's busy :] 
[03:29] <limi> *waits for the star-merge*
[03:30] <kiko> if you don't have a revlib, wait for 70 minutes :)
[03:30] <lifeless> kiko: there are cached revs, should never be anywhere near that long.
[03:31] <kiko> yeah, I'm just making a snide remark because I blew away mine yesterday and was amazed at how long it took.
[03:31] <BradB> spiv: pign
[03:31] <BradB> spiv: ping :)
[03:31] <lifeless> kiko: oh right, yeah revlibs make order of magnitude differences
[03:32] <BradB> that's a scary thought
[03:32] <BradB> considering that even with revlibs it takes an hour to star-merge on my machine
[03:32] <lifeless> BradB: what ?!?
[03:32] <limi> arch is an order of magnitude different in any case :] 
[03:32] <lifeless> its < 10 seconds for me.
[03:33] <ddaa> Still invalidversionerror... I bet it's something with the duplication of jobs...
[03:33] <BradB> watching tla do its thing with fs_usage is pretty frightening
[03:33] <BradB> makes me glad i'm not a hard drive
[03:33] <ddaa> BradB: yeah, same thing here. The initial revlib population is a bit painful, but then it's painless and fast.
[03:34] <BradB> ddaa: it never gets painless and fast on os x.
[03:34] <ddaa> BradB: Use a greedy non-sparse revlib to avoid all worst-case scenarios.
[03:34] <ddaa> BradB: are you using HFS+ or something?
[03:34] <lifeless> ddaa: macosX's disk io sucks compared to linux on the same hw. something weird in there.
[03:34] <BradB> ddaa: yep
[03:35] <ddaa> BradB: maybe you should use UFS for tla work. It's quite demanding on fs performance.
[03:35] <kiko> BradB, but only use a gnsprl IF you have 2G free for it.
[03:35] <BradB> ddaa: i was thinking of that
[03:35] <ddaa> And HFS+ sorta sucks big time.
[03:36] <BradB> kiko: hehe.
[03:36] <SteveA> BradB, spiv: see anything interested in the exuserfolder setup ?
[03:36] <BradB> SteveA: I need to know what you meant by "it doesn't work." :)
[03:36] <lifeless> limi: did that work ?
[03:37] <BradB> SteveA: er, first off, is this a use case we need to support
[03:37] <limi> still waiting for the star-merge
[03:37] <BradB> ?
[03:39] <limi> there we go
[03:40] <ddaa> kiko: my revlib is a mere 1.2 gig
[03:40] <kiko> mine is 1.7
[03:40] <limi> just 10 minutes :] 
[03:40] <limi> and I have about 300 conflicts
[03:40] <SteveA> BradB: did you and spiv talk?
[03:40] <limi> yay
[03:40] <lifeless> kiko: unique or shared data ?
[03:40] <BradB> SteveA: Yes. Is this a use case we need to support?
[03:40] <kiko> unique I think 
[03:40] <SteveA> is what a use-case we need to support?
[03:40] <lifeless> kiko: du by default gives you the shared result
[03:41] <lifeless> its smart
[03:41] <BradB> SteveA: Changing p/w in the db instead of via the Plone interface.
[03:41] <SteveA> yes, of course
[03:41] <lifeless> ddaa: where is it at for you ?
[03:41] <Kinnison> SteveA: When is __del__ invoked?
[03:41] <ddaa> always the same exception
[03:41] <lifeless> paste it ?
[03:41] <ddaa> I have imported all info files to try other jobs
[03:41] <BradB> SteveA: Okay, so secondly, what did you mean by "doesn't work"? What error message did you see?
[03:41] <BradB> I saw an AttributeError, but was still logged in.
[03:41] <ddaa> CVS.InvalidVersionError: Could not determine CVS version from CVS version string ""
[03:42] <SteveA> BradB: I think we are talking at cross-purposes
[03:42] <limi> lifeless: any easy way I can ditch the conflicts and just get what RF has?
[03:42] <lifeless> limi: sure.
[03:42] <lifeless> do this.
[03:42] <lifeless> tla undo -n
[03:43] <lifeless> tla get rocketfuel@canonical.com/launchpad--devel--0 ,,temp-dir
[03:43] <SteveA> right now, I am not talking about any specific errors.  I am talking about resetting a password using the "lost password" interface, and then being unable to use that password to log into plone for a while, sometimes.
[03:43] <lifeless> cd ,,temp-dir
[03:43] <SteveA> BradB: the behaviour is as if the password is the wrong password.
[03:43] <lifeless> tla set-tree version alexander.limi@canonical.com/launchpad--devel--0
[03:43] <SteveA> BradB: but, I expected you and spiv to have talked about this.
[03:43] <BradB> And we did. :)
[03:44] <BradB> He had to leave shortly after that.
[03:44] <lifeless> tla sync-tree $(tla revisions -rf alexander.limi@canonical.com/launchpad--devel--0 | head -n 1)
[03:44] <stub> SteveA: Was the exUserFolder caching tunable?
[03:44] <lifeless> tla commit -s 'revert to rocketfuel code base'
[03:44] <lifeless> cd ..
[03:44] <lifeless> rm -rf ,,temp-dir
[03:44] <lifeless> tla update
[03:44] <lifeless> ok whew, done.
[03:44] <lifeless> I know thats a little long, its *yet another* thing raw should encapsulate.
[03:45] <lifeless> what that does is check out rocketfuel, tell it it is your branch, and then commits it
[03:45] <SteveA> Kinnison: http://docs.python.org/ref/customization.html#l2h-174
[03:45] <BradB> SteveA: Which "lost password" interface are you referring to?
[03:46] <limi> so you can't just blow away what you currently have and get a clean copy from RF?
[03:46] <ddaa> lifeless: firefox is going made trying to render the summary page... probably a consequence of job duplication. 236MB VM and growing...
[03:46] <lifeless> ddaa: lol
[03:46] <kiko> limi, apparently no.
[03:47] <ddaa> Bah. I guess I'm going to spend the rest of the day trying to debug that problem.
[03:47] <ddaa> lifeless: want me to try anything else before I take care of my pets?
[03:47] <lifeless> ddaa: show me the exception
[03:48] <SteveA> stub: I do not know.  I asked Brad and spiv to continue looking into it.
[03:48] <ddaa> lifeless: already did, twice, I'll flood you the backtrace in private, then.
[03:48] <SteveA> BradB: let's talk this through off channel -- too noisy here :-)
[03:48] <kiko> flood!
[03:48] <lifeless> ddaa: I haven't seen it, for whatever reason
[03:48] <limi> lifeless: $ tla sync-tree $(tla revisions -rf alexander.limi@canonical.com/launchpad--devel--0 | head -n 1)
[03:48] <limi> archive not registered: alexander.limi@canonical.com
[03:49] <kiko> oink
[03:49] <lifeless> oh, what is your archive name ?
[03:49] <sabdfl> bradb: what's wrong with this:
[03:49] <limi> maybe it has a --2004?
[03:49] <sabdfl> ProductSeries.selectBy(product=self.product)
[03:49] <sabdfl> where self.product is a Product object
[03:49] <sabdfl> ?
[03:49] <BradB> sabdfl: nothing, but sqlobject doesn't support that yet :)
[03:50] <stub> A quick solution might be to set the cache times to 30 seconds, and stick a sleep(30) between changing the password in the database and sending the email
[03:50] <lifeless> limi: ?
[03:50] <sabdfl> >>> for x in ProductSeries.selectBy(product=Product.get(4)): print x
[03:50] <sabdfl> ...
[03:50] <sabdfl> Traceback (most recent call last):
[03:50] <sabdfl>   File "<stdin>", line 1, in ?
[03:50] <sabdfl>   File "/home/mark/projects/ubuntu/launchpad/lib/sqlobject/main.py", line 958, in selectBy
[03:50] <sabdfl>     cls._connection._SO_columnClause(cls, kw),
[03:50] <sabdfl>   File "/home/mark/projects/ubuntu/launchpad/lib/sqlobject/dbconnection.py", line 428, in _SO_columnClause
[03:50] <sabdfl>     return ' AND '.join(['%s = %s' %
[03:50] <sabdfl> KeyError: 'product'
[03:50] <sabdfl> >>>
[03:50] <lifeless> oh, let me checkf or you
[03:50] <limi> lifeless: what is the command to find out?
[03:50] <lifeless> limi: tla archives
[03:50] <limi> tried tla my-archives - aha
[03:51] <limi> yup, alexander.limi@canonical.com--2004
[03:51] <lifeless> limi: that would be consistent wouldn't it. damn we got that wrong
[03:51] <BradB> sabdfl: you'll have to do productID=self.product.id, I think (until that is implemented to work the way you expect.)
[03:51] <lifeless> ok, you need to redo the set-tree-version too.
[03:51] <BradB> there's a lot of work i could do on sqlobject :)
[03:51] <sabdfl> BradB: i tried that but get a forbiddenattribute on "id"
[03:52] <BradB> you need to add it to the schema
[03:52] <sabdfl> BradB: ok, will fund some time for you to qork on sqlobject
[03:52] <BradB> cool
[03:52] <sabdfl> let's get malone, rosetta, soyuz out the door then the lp team gets a bit of itch scratchin' time
[03:52] <BradB> sounds like a plan
[03:52] <kiko> qorking for funds!
[03:52] <sabdfl> BradB: would prefer not to expose id's in the schemas
[03:53] <sabdfl> nobody qorks like kiko qorks
[03:53] <kiko> national holidays included
[03:53] <lifeless> and one qork said to the other
[03:53] <sabdfl> spanishness day in brazil too? wow
[03:54] <jblack> kiko, mind answering a random thought about launchpad ?
[03:54] <limi> portugueseness
[03:54] <kiko> for some reason the first holiday in 3 months it rains.
[03:54] <kiko> jblack, sure
[03:54] <sabdfl> sun is on holiday too, y'know
[03:54] <sabdfl> hey jblack
[03:55] <jblack> kiko: Great. Thanks. I'm thinking about the star-merge bug that you guys hit so often. That got me to wondering why you guys merge so frequently. Is it because everybody's highly dependant upon everybody else for day to day work, or is it just some fear that people will get out of date irt the center of the star? 
[03:56] <jblack> i.e. does everybody need everybody's new work, or are people acting as a conflict avoidance system by staying as close to center as possible? 
[03:56] <SteveA> there's lots of moving around of code happening right now
[03:56] <kiko> jblack, there's been at least some refactoring going on in common areas, but mostly it's because the codebase is small enough and people work together in the projects.
[03:57] <kiko> so celso and daniel and I are bound to step on each other's feet unless we merge frequently.
[03:57] <SteveA> also, many people find conflicts very awkward to deal with
[03:57] <kiko> this is aggravated by the very unreasonable handling of conflicts
[03:57] <SteveA> so they prefer to avoid them 
[03:57] <kiko> oh, SteveA beat me to it.
[03:57] <jblack> Ok. so its conflict avoidance.
[03:57] <jblack> For the most part. I had a hunch. :) 
[03:58] <kiko> jblack, note that this is something that nicer conflicts and a more modular codebase could help, but you can't expect the latter in projects that are only a few months old.
[03:58] <kiko> no software starts out nicely modular the first time it's written..
[03:58] <jblack> Oh, actually, rf was more modular, remember? 
[03:58] <SteveA> it often starts off wrongly modular
[03:58] <jblack> oh, I know what you mean. Yeah.
[03:58] <kiko> modularity is not an objective measurement
[03:59] <kiko> it has to do with grouping things that *should* be coupled
[03:59] <kiko> and that's usually a mix of domain and design-specific knowledge that, well, requires experience.
[03:59] <jblack> Oh. That's a minihowto I should write. Conflict resolution.
[03:59] <kiko> since us people in the software business are always building newfangled things..
[04:00] <lifeless> night all
[04:00] <kiko> jblack, you could just make tla not commit if unresolved conflicts were present (and clearly indicate them) and I would buy you a house in heaven
[04:00] <lifeless> ddaa: oh, there is a buildbot test failure you could look at once you get back.
[04:00] <lifeless> just make check in the buildbot dir should show it. its reusing a checked out copy of the revision its reverting,,,
[04:00] <kiko> I would much rather have sane behaviour in tla than a howto :)
[04:00] <jblack> kiko: Yeah. I'd like that too.
[04:01] <jblack> kiko: I know how you could do it. :) 
[04:01] <jblack> kiko: make *.rej as unrecognized. :) 
[04:02] <kiko> that's a hack
[04:02] <jblack> Then if you try and commit, arch will abort.
[04:02] <BradB> spiv: so n/m, we fixed it. it was the http vs https thing that i mentioned first off
[04:02] <jblack> That's what the unrecognized method is for. 
[04:02] <kiko> it shouldn't require local configuration; conflicts are an essential part of SCS! 
[04:03] <jblack> Yes, I agree. It should be in the boilerplate =tagging-method as well.
[04:03] <jblack> sorry. Didn't mean to distract you from lp for so long. 
[04:04] <kiko> that's not distracting, it's an essential part of lp
[04:05] <jblack> Once arch development is moving again, I'll submit the tagging-method boilerplate change. You'll have to do it for existing branches though.
[04:05] <jblack> (users wouldn't like us if we hacked their tagging-methods behind their back!)
[04:05] <kiko> yeah, no need to add the complexity of that
[04:12] <sabdfl> stub: still around?
[04:13] <SteveA> Kinnison: some things you need to know about __del__: 1. it is called when the last reference to the object with the __del__ method is deleted.  It can be difficult to know when this will be.  2. __del__ is not necessarily called at all, if the interpreter exits.  3. exceptions raised in a __del__ are logged to stderr and swallowed.  4. objects with a __del__ are not collected by the cyclic garbage collector, and so any cycles that involve s
[04:13] <SteveA> uch an object will not be collected.  The test runner notices this, and prints a warning.
[04:13] <stub> Yup
[04:13] <sabdfl> need to add a productseries field to ProductRelease which references ProductSeries
[04:13] <sabdfl> can be NULL
[04:14] <sabdfl> though i'm tempted (since we have no release data in the db) to make it NOT NULL
[04:14] <sabdfl> but that means creating a Series for every Product
[04:14] <stub> I like NULL's :-)
[04:15] <sabdfl> so i've noticed :-)
[04:15] <sabdfl> with hindsight you've generally been smarter than me on that front, so I'll put my fascist whip aside for the moment
[04:15] <sabdfl> soo...
[04:16] <sabdfl> this is going into mark-patch-series
[04:16] <sabdfl> ALTER TABLE ProductRelease ADD COLUMN productseries INT;
[04:16] <jblack> Good idea. safdfl is probably taken anyways. :) 
[04:16] <sabdfl> now what's the voodoo to make it point at ProductSeries?
[04:16] <sabdfl> jblack: those guys have all the fun then tanks roll in and their statues get toppled, so it's a short term gig, y'know
[04:17] <Kinnison> SteveA: Thanks. I'm not considering creating __del__ methods; I was just trying to work out how to cleanly release this transaction
[04:18] <kiko> Kinnison, SteveA just means to say, don't trust __del__ to do critical work.
[04:18] <stub> sabdfl: ALTER TABLE ProductRelease ADD COLUMN productseries INT REFERENCES ProductSeries(id);
[04:18] <sabdfl> ah... found an example in patch-3.04
[04:18] <sabdfl> ok, works too. thanks!
[04:18] <limi> lifeless: is this an error message or just "FYI"?
[04:18] <limi> $ tla commit -s 'revert to rocketfuel code base'
[04:18] <limi> commit: tree has no patch log for version
[04:18] <limi>     tree: /Users/limi/Work/Canonical/launchpad/launchpad/,,temp-dir
[04:18] <limi>     version: alexander.limi@canonical.com/launchpad--devel--0
[04:18] <Kinnison> stub: Guess what... I have a couple more tables which need id columns. Fancy adding id columns to binarypackagefile and sourcepackagereleasefile ?
[04:19] <lifeless> limi: you need to re run the set-tree-version
[04:19] <lifeless> with the right archive bit
[04:20] <limi> lifeless: does not compute, sorry :] 
[04:20] <limi> re-run the earlier command?
[04:20] <limi> inside the temp dir?
[04:25] <sabdfl> kiko: cprov around today?
[04:26] <sabdfl> elmo: client cert auth?
[04:26] <sabdfl> SteveA: is there a workaround for the "cannot log in to debugging port" for lp?
[04:27] <SteveA> you should be able to log into port 8086
[04:28] <SteveA> I need to push a change upstream before you'll be able to log into 8089
[04:29] <spiv> BradB: cool :)
[04:30] <elmo> sabdfl: I'm working on it - apache2 is breaking me tho
[04:31] <sabdfl> elmo: turru turru turrrrrrruuuuuu!!!!! (cavalry sirens)
[04:31] <sabdfl> got a ca cert, and client cert issued by it?
[04:33] <elmo> sabdfl: yeah
[04:33] <elmo> well, signed by it
[04:34] <sabdfl> elmo: msg me the ca cert and client cert
[04:37] <BradB> Presumably "urgency" of a sourcepackagerelease is a column whose value describes how important it is to upgrade to that release? (Or is it the result of a Saturday night codathon?)
[04:38] <sabdfl> BradB: maps to a field in the debian package toolset
[04:38] <BradB> ah
[04:38] <sabdfl> elmo: thanks
[04:40] <elmo> sabdfl: these are just mockups btw, for local testing, the details aren't final
[04:40] <Kinnison> BradB: It's related to an upload
[04:41] <Kinnison> bradb: uploads have urgencies associated with them to allow for ordering of builds by buildds etc by priority
[04:41] <Kinnison> Debian also uses this to help with the 'testing' infrastructure
[04:42] <Kinnison> (ICBW, e&oe, etc)
[04:42] <elmo> [Debian uses it for testing, but not buildds] 
[04:43] <Kinnison> elmo: Aah; right; ta
[04:43] <jblack> Ok spiv, what's up? 
[05:02] <limi> finally arch is back to a usable state, now to check in my changes :)
[05:03] <SteveA> elmo: the redirect from http -> https of http://www.ubuntulinux.org/login_form is not working
[05:03] <spiv> ddaa: Btw, any luck figuring out why my tree conflicts with rocketfuel?
[05:04] <limi>     ZopeXMLConfigurationError: File "/Users/limi/Work/Canonical/launchpad/launchpad/lib/canonical/lp/configure.zcml", line 32.2-37.8
[05:04] <limi>     ConfigurationError: ('Invalid value for', 'factory', "Couldn't import canonical.lp.tales, cannot import name IPerson in canonical.lp.tales.RequestAPI")
[05:04] <Kinnison> sabdfl: Are we expecting all derivative distributions to share the same set of package priorities? (base,essential,standard,optional,extra) ?
[05:04] <limi> make: *** [run]  Error 1
[05:04] <limi> anything I need to change or update to get rid of that error?
[05:04] <sabdfl> Kinnison: ultimately i guess not
[05:04] <Kinnison> limi: that's clearly a spelling mistake, s/tales/tables/
[05:04] <limi> aha
[05:04] <sabdfl> no, not a speling mistake
[05:04] <spiv> Kinnison: Eh?  
[05:05] <limi> what needs updating?
[05:05] <sabdfl> that looks like a circular import
[05:05] <sabdfl> limi are you up to date?
[05:05] <limi> yes
[05:05] <limi> 2 mins ago
[05:05] <Kinnison> sabdfl: Currently the priority in packagepublishing is just an integer which I assume will have a corresponding dbschema.py class.
[05:05] <limi> star-merged the launchpad part
[05:05] <limi> any changes in Zope or similar?
[05:06] <elmo> SteveA: looking
[05:06] <sabdfl> limi: yes, there's been an sqlobject rev
[05:06] <limi> aha
[05:06] <sabdfl> need to update your config to 0.6
[05:06] <sabdfl> delete that dir and re-get it
[05:06] <sabdfl> lifeless promises not to do that to us again
[05:06] <limi> hehe
[05:06] <limi> what's the line to re-get it?
[05:09] <ddaa> spiv: I thought you were sleeping
[05:10] <ddaa> Up to not very long ago I was on importd things. Checking your tree will be good for me to relax,
[05:10] <spiv> ddaa: lunching, which turned out to be more effort than expected ;)
[05:13] <ddaa> spiv: you've got my token
[05:13] <Kinnison> sabdfl: Is it okay for now for me to assume priority is limited to required,standard,important,optional,extra ?
[05:13] <Kinnison> sabdfl: or would you prefer a table for them?
[05:14] <sabdfl> Kinnison: put them in dbschema with values of 10, 20, 30... etc with required being the highest please
[05:14] <Kinnison> sabdfl: Sure
[05:14] <sabdfl> we can remap them to a table if derivatives really want this
[05:15] <sabdfl> but I suspect the toolset has this hardcoded in a lot of places
[05:15] <Kinnison> It's certainly plausible that it may
[05:16] <ddaa> spiv: what's the revision of your version you have a problem with?
[05:17] <spiv> ddaa: Whatever the latest revision of andrew.bennetts@canonical.com/launchpad--devel--0 is :)
[05:17] <ddaa> i have patch-197
[05:18] <spiv> It seeems patch-198 needs mirroring.
[05:18] <spiv> (it's taking a long time for some reason..)
[05:19] <ddaa> no problem merging that with rocketfuel's patch-581. It just appears massively out of date. It does a honest-to-god cross-version apply-delta, in effect making the tree identical to rocketfuel.
[05:19] <ddaa> ha, something new in the pipe....
[05:21] <spiv> ddaa: mirrored.
[05:23] <ddaa> unrelated, I just found out about "get --non-sparse". A sparse revlib anh using --non-sparse for initial gets should be a good compromise between performance and disk usage.
[05:25] <jblack> ddaa: Um, actually, its --sparse that minimizes disk usage.
[05:25] <ddaa> okay, i have conflicts
[05:25] <jblack> --nonsparse means "grab all of them on the way", which is definitely a drive hog. :) 
[05:25] <ddaa> jblack: yes, but when you have a big version, you want the initial get to be non-sparse because you want the full revlib there for the next star-merge.
[05:25] <elmo> SteveA: should be working now
[05:26] <ddaa> jblack: then, further commands can populate sparsely.
[05:27] <jblack> Whatever works best for you. :) 
[05:27] <jblack> I use greedy nonsparse libraries myself.
[05:27] <spiv> jblack: troublemaker ;)
[05:27] <jblack> Then again, it takes 20 minutes for me to check the size of my revision library
[05:27] <SteveA> elmo: thanks
[05:27] <spiv> BradB: SQLObject's SVN repo seems to be broken, btw.
[05:28] <SteveA> sabdfl: the "change password" facilities should work for you now
[05:28] <jblack> dvd+rw media costs too much
[05:28] <ddaa> spiv: okay I have the conflicts
[05:28] <sabdfl> on the production machine?
[05:28] <BradB> spiv: Indeed.
[05:28] <sabdfl> SteveA: ^?
[05:28] <limi> can anybody give me the magic command to get the new SQLObject?
[05:29] <spiv> ddaa: Ok, can you tell if it's because I crossed the streams, or something else silly? :)
[05:29] <spiv> limi: Hmm, I thought lifeless sent a mail about that to the list a few days ago?
[05:29] <sabdfl> limi: i think you need to rm -rf the sqlobject dir in sourcecode dir
[05:29] <limi> did he? I must have missed that
[05:29] <sabdfl> then you need to make sure your config refers to the new version as per lifeless email
[05:29] <limi> can someone forward it if they have it handy?
[05:30] <limi> sorry to be a pain :)
[05:30] <spiv> limi: It should be rm -rf sourcecode/sqlobject; tla get rocketfuel@canonical.com/sqlobejct--test--0.6 sourcecode/sqlobject
[05:30] <carlos> limi: tla get rocketfuel@canonical.com/sqlobject--test--0.6 sqlobject
[05:30] <limi> great, thanks
[05:30] <sabdfl> ./launchpad/sourcecode/sqlobject        rocketfuel@canonical.com/sqlobject--test--0.6
[05:30] <ddaa> spiv: checking
[05:30] <sabdfl> then do as carlos says :-)
[05:30] <spiv> limi: Unless you rely on the config for anything other than the initial setup.
[05:30] <limi> not to my knowledge
[05:30] <carlos> spiv's command is better 
[05:30] <spiv> From what you said eariler, it doesn't sound like you do :)
[05:31] <spiv> limi: Beware the typo in my spelling of sqlobject in that command, btw ;)
[05:31] <spiv> carlos: Shouldn't you be holidaying, or something? :)
[05:32] <carlos> spiv: yes, something like that :-P
[05:32] <limi> spiv: yes, noticed ;)
[05:37] <ddaa> spiv: does not look like a cross merge...
[05:38] <SteveA> sabdfl: yes, on the ubuntulinux website, that is, the plone server and the "forgotten password" application part of launchpad running on macquarie
[05:38] <spiv> ddaa: Interesting.  But merging from my patch-197 worked... but patch-198 didn't touch the conflicting files.
[05:39] <sabdfl> SteveA: cool, thanks
[05:39] <ddaa> spiv: trying to figure out what is the cause of the duplicate ids, looks very fishy.
[05:39] <spiv> ddaa: Indeed!
[05:40] <spiv> ddaa: Although not all the conflicts were due to that?
[05:40] <spiv> ddaa: All the files that conflicted were touched by Kinnison, from what I can see.
[05:42] <ddaa> maybe it's still a cross-merge, I always have a hard time figuring it out.
[05:44] <Kinnison> kiko: ping?
[05:46] <ddaa> spiv's patch-198 merged with rocketfuel's 559
[05:47] <ddaa> and rocketfuel's patch-160 merged with spiv's patch-197
[05:48] <ddaa> Cross merge...
[05:48] <spiv> Ah.
[05:48] <ddaa> I'll 
[05:48] <ddaa> fix it and send you a changeset
[05:48] <spiv> I guess I must have been careless then :/
[05:48] <spiv> Well, I'm happy to just retag.
[05:49] <spiv> If that's sufficient and simpler.
[05:49] <limi> ok, dinner time here - back later
[05:50] <ddaa> spiv: okay. Does this technique has been validated by lifeless? I'm concerned that it might interact poorly with 
[05:50] <ddaa> the broken log-for-merge behaviour.
[05:51] <ddaa> It's really exasperating to see the amount of brokeness we have to work around...
[05:52] <BradB> sabdfl: Just curious (now that I'm editing something for which this issue comes up): why do you not want to expose id in the schema, given that there are times when we need to access the id of an SQLObject?
[05:52] <spiv> ddaa: I'm happy to do whatever you tell me is safest :)
[05:52] <ddaa> Mh... methink it should work okay.
[05:53] <spiv> ddaa: you don't sound certain :)
[05:53] <spiv> Just send me the changeset ;)
[05:54] <ddaa> I remember that the log-for-merge issue was caused by the removal of a patchlog. That should not happen with tag because the revision you are skipping was never added to rocketfuel.
[05:54] <ddaa> Nah. Tag from rocketfuel.
[05:55] <spiv> Ok, I'll do that.
[05:55] <sabdfl> BradB: just prefer to keep things like that out of UI, and if its in the schema it usually shows up in UI somewhere
[05:55] <spiv> Thanks for the diagnosis.
[05:55] <sabdfl> sometimes its necessary but we avoid it as much as possible with name's
[05:55] <ddaa> star-merge really ought to provide sensible diagnosis for those cases.
[05:56] <ddaa> But somehow it seemed we did not manage to get the idea through to tom :-(
[05:56] <spiv> Oh :(
[05:57] <ddaa> abentley purportedly implemented that diagnosis in his "improved" merge operator in fai.
[05:57] <BradB> sabdfl: Okay, well, I'm thinking I need to have it in there for things like IBugPackageInfestation, because I need to get at ID's when I'm building the little table with edit links to each i9n
[05:57] <ddaa> hct definitely ought to do it too.
[05:58] <sabdfl> BradB: understood. highlevel design goal we can defer and come back to once we have our head wrapped around the basics
[05:58] <BradB> sure
[05:58] <spiv> ddaa: I guess tagging will solve my cacherev issue ;)
[05:58] <spiv> So it's not all bad ;)
[05:58] <ddaa> spiv: right, that's another good side effect.
[05:59] <ddaa> Actually, methink tagging could be subsituted for star-merging in several cases and might lead to less pain.
[06:00] <ddaa> Only problem: tag patchlogs list all patchlogs present in the tagged revision.
[06:00] <ddaa> Which are a few thousands in rocketfuel...
[06:00] <ddaa> could very well end up being a disk hog.
[06:01] <spiv> ddaa: Oh, while I've got your token..
[06:01] <spiv> ddaa: Our pqm submit script sends a message saying 'star-merge "$(tla tree-version)" "$(cat {arch}/+upstream)"'
[06:01] <ddaa> Yeah.
[06:02] <spiv> Would it be a better idea to at the revision on to that?
[06:02] <spiv> To catch e.g. trying to merge unmirrored commits.
[06:02] <ddaa> Wow.
[06:03] <spiv> s/at/add/
[06:03] <ddaa> Actually my custom scripts does the mirroring.
[06:03] <ddaa> but you have a very good point.
[06:03] <spiv> Also, would this help the cross-merge problem? :)
[06:03] <ddaa> That would not help cross-merge.
[06:03] <spiv> Drat.  :)
[06:04] <spiv> I doubted that it would (it'd be just too easy), but I thought I'd ask ;)
[06:04] <ddaa> The cross-merge problem is caused (among other things) by the finite speed of the light...
[06:04] <spiv> So, the right way to spell that is :)
[06:04] <spiv> Gah, weird lag issues.
[06:05] <spiv> So, adding "--$(tla logs | tail -1)" would do the right thing?
[06:06] <ddaa> spiv: generally we do $(tla logs --reverse | head -n 1)
[06:06] <spiv> Heh.  Fair enough.
[06:06] <ddaa> might me a bit faster...
[06:06] <ddaa> also
[06:07] <ddaa> you could just do: star-merge "$(tla logs --full --reverse | head -n 1)" "$(cat {arch}/+upstream)"
[06:08] <ddaa> probably you should request confirmation from lifeless before merging such a change into rocketfuel.
[06:09] <ddaa> The pqm toolset is a very critical infrastructure, I do not see how that could cause a problem (I got his blessing for sending a full revision name as an option) but if I were him I would be quite anal with that.
[06:09] <spiv> Sure :)
[06:10] <ddaa> I hope you do not get tired by my bouncing you around to other team members :-)
[06:10] <spiv> Not at all :)
[06:17] <SteveA> elmo: where can I read about the ssh configuration to connect to other machines via chinstrap ?
[06:17] <spiv> SteveA: It's on teh wiki somewehre, iirc...
[06:18] <SteveA> I've just been looking...
[06:18] <elmo> Host chinstrap.warthogs.hbd.com
[06:18] <elmo>      ProxyCommand none
[06:18] <elmo> Host *.warthogs.hbd.com
[06:18] <elmo>      ProxyCommand ssh YOURUSERNAME@chinstrap.warthogs.hbd.com nc -q0 %h %p
[06:18] <elmo> in ~/.ssh/config and fix YOURUSERNAME
[06:18] <elmo> it may not be on the wiki - it's in the ml archives - someone could fix the wiki if they want - it's (low down) on my TODO list
[06:23] <SteveA> thanks
[06:49] <BradB> Thought of the day: why bother saying "no class registered for SourcePackageRelease", when you can simply say "AttributeError: 'BugPackageInfestation' object has no attribute '_SO_class_SourcePackageRelease'"?
[06:55] <Kinnison> BradB: Feel free to add classes for all those tables
[06:56] <BradB> I have to :)
[07:07] <sabdfl> Kinnison: schweet
[07:09] [mako(mako@micha.hampshire.edu)]  help
[07:09] [mako(mako@micha.hampshire.edu)]  hello
[07:09] [mako(mako@micha.hampshire.edu)]  log
[07:09] [mako(mako@micha.hampshire.edu)]  #ubuntu-meeting
[07:10] <Kinnison> sabdfl: Aye. I'm currently doing a main-only import to get the final feel of what I need to tweak.
[07:10] <Kinnison> sabdfl: I'm *slowly* getting the hang of all this
[07:10] <sabdfl> sqlobject?
[07:10] <sabdfl> has potential
[07:10] <Kinnison> sqlobject, the dbschema stuff etc.
[07:11] <Kinnison> It's very interesting stuff.
[07:11] <Kinnison> It doesn't have an externally callable method to relinquish a connection to the db which breaks pooling connections
[07:12] <Kinnison> At the moment I have a hack in to call txn._makeObsolete() -- personally I'd be happy to docstring the method and remove the underscore; but I need to talk to someone who knows the code better
[07:13] <spiv> Hmm, where is this?
[07:14] <Kinnison> lib/sqlobject/dbconnection.py:class Transaction(object):
[07:14] <spiv> Kinnison: Where are you using the Transaction class?  Inside gina/
[07:14] <Kinnison> librarian
[07:15] <spiv> The version in rf doesn't seem to?
[07:15] <spiv> Or perhaps I need to update.
[07:15] <Kinnison> I've not merged
[07:15] <Kinnison> It's all local until I work out the nicest way of doing it
[07:19] <spiv> Grr, lag.
[07:20] <spiv> (the DHCP server here has 30s leases!)
[07:21] <spiv> Kinnison: Hmm, is this on a branch I can see anywhere?
[07:21] <Kinnison> spiv: It's in my homedir on zhongshan
[07:21] <spiv> I'm quite curious about how you're doing it... 
[07:21] <Kinnison> spiv: It's all random hacking I've been doing today to try and build it up
[07:21] <spiv> I don't think I have an account there...
[07:23] <spiv> I see.
[07:23] <spiv> Ok, the real problem here isn't SQLObject, I think.
[07:23] <spiv> Or, perhaps not.
[07:24] <spiv> Where's the connection pool?
[07:25] <spiv> Oh, ugh. SQLObject's "Connection" object is actually a ConnectionPool when you use Transaction objects.  Ick.
[07:25] <spiv> Stupid naming.
[07:26] <spiv> BradB|lunch: When you get back from lunch, please hurt Ian for me ;)
[07:27] <Kinnison> spiv: I'm off to listen to the radio for half an hour, I'll be back then.
[07:27] <spiv> Kinnison: Oh, right :)
[07:28] <Kinnison> s'hitchhikers time :-)
[07:28] <spiv> Enjoy HHGTTG :)
[07:29] <spiv> I think I already knew this part of sqlobject was... bizarre, but the fullness of it only just hit me.
[07:30] <spiv> Kinnison: What you're doing is the simplest and sanest option for now, I think.
[07:31] <spiv> But we should be able to do better at some point.
[07:37] <SteveA> spiv: what is the issue?
[07:45] <daf> SteveA: did you get the emails I sent you last night?
[07:47] <sabdfl> daf: rosetta.warthogs was barfing earlier
[07:47] <sabdfl> could you nudge it into action please?
[07:49] <daf> sabdfl: sure
[07:50] <daf> oh, right, I didn't bring it back up after the reboot
[07:54] <daf> ok, it's back up now
[07:54] <ddaa> yeeeeeeeeeeeeeeeeha!
[07:55] <ddaa> first import running here
[07:55] <ddaa> that has cost me a lot in goats...
[07:56] <sabdfl> ddaa great!
[07:56] <sabdfl> what was broken?
[07:57] <ddaa> Dunno, something prevented the a52dec import here, related to cvs version detection. But it worked with aalib. Then it needed pyarch to automagically choose between classical and twisted process handling (that should have been done in buildbot
[07:58] <ddaa> but lifeless did not do it and requested that feature anyway)
[07:58] <ddaa> Now it seems to work...
[07:58] <SteveA> daf: yes, it's great, thanks
[07:58] <ddaa> Still a lot of various mysterious breakage around...
[07:59] <daf> SteveA: any idea about the inconsistencies I was seeing?
[08:11] <elmo> so what do we want client certed?  both the sites on rosetta and the launchpad instance on macquarie - anything else, immediately?
[08:12] <SteveA> what's rosetta's new name?
[08:12] <elmo> mawson
[08:12] <elmo> .ubuntu.com
[08:13] <SteveA> I think only the devel server ought to require certificates, not the rosetta alpha server.  But, daf can confirm that
[08:13] <daf> it depends whether we care if people stumble upon the alpha or not
[08:14] <SteveA> do you think all of the alpha users will be competent and willing to add a client certificate?
[08:15] <SteveA> I suppose it will stop it getting accidentally slashdotted
[08:15] <SteveA> elmo: what certificates do we have?
[08:15] <SteveA> is there one for canonical staff, and one for others ?
[08:16] <elmo> as many as we want, it's trivial to create them
[08:16] <elmo> (on a once off basis)
[08:16] <elmo> I'm certainly planning on having at least a 'launchpad' and a 'warthogs' one
[08:16] <elmo> if you want more granularity than that, go for it, just let me know what you want to do
[08:17] <SteveA> if daf wants one for the rosetta alpha, then we want a "launchpad testers" one too
[08:23] <BradB> spiv: Sounds nasty. I hope to make things better once Malone's out the door.
[08:36] <BradB> What's a "format" of a source package release? (I'm wanting to describe srcpackageformat in a schema). I'm assuming it's things like "bzipped tarball" or "gzipped tarball".
[08:36] <Kinnison> Bugger knows; that's not something I've worked out yet
[08:37] <Kinnison> gina only ever puts '1' in that column
[08:40] <sabdfl> SteveA: resetting password...
[08:40] <sabdfl> A system error occurred
[08:40] <sabdfl> it might be that my mark@hbd.com account details are in an unusual state because they were created some time ago
[08:40] <sabdfl> BradB: deb vs rpm
[08:41] <sabdfl> it should be documented in dbschema.py
[08:41] <spiv> BradB: dbschema has a SourcePackageFormat
[08:41] <sabdfl> BradB: is there not a SoyuzSourcePackageRelease that should be ported, rather than a new implementation?
[08:42] <sabdfl> i've been trying to port Soyuz classes (probably breaking Soyuz horribly - bring on the page tests :-)
[08:42] <SteveA> sabdfl: what is the exact URL you are using to try to reset your password?
[08:43] <sabdfl> dude, it's on my windows PC you want me to type the hash?
[08:43] <sabdfl> will mail it to ya :-)
[08:43] <SteveA> ok
[08:43] <SteveA> although, I'm more interested in the first part
[08:43] <sabdfl> on the wire
[08:44] <SteveA> I'm sure at thawte you regularly typed in whole certificates ;-)
[08:48] <sabdfl> don't freakin joke :-)
[08:48] <BradB> sabdfl: Not that I saw.
[08:48] <BradB> I'll check again.
[08:48] <BradB> I would have though deb vs. rpm is something that a sourcepackage is aware of, rather that all the individual releases.
[08:49] <sabdfl> mark@slinky ~/projects/ubuntu/launchpad/lib $ grep -ir "class.*sourcepackagerelease" *
[08:49] <sabdfl> canonical/launchpad/ikiko.py:class ISourcePackageRelease(Interface):
[08:49] <sabdfl> canonical/launchpad/scripts/gina/classes.py:class SourcePackageRelease(AbstractPackageRelease):
[08:49] <sabdfl> canonical/launchpad/scripts/gina/grabber.py:from classes import SourcePackageRelease, BinaryPackageRelease
[08:49] <sabdfl> canonical/launchpad/database/package.py:class SoyuzSourcePackageRelease(SQLBase):
[08:49] <sabdfl> canonical/sourcerer/soyuz/__init__.py:class SourcePackageRelease(ManifestRecordObject):
[08:49] <sabdfl> canonical/sourcerer/soyuzwrapper.py:class SourcePackageRelease(object):
[08:49] <sabdfl> canonical/soyuz/sql.zcml:  <content class="canonical.soyuz.sql.SoyuzSourcePackageRelease">
[08:49] <BradB> argh, I was grepping for "class ..."
[08:50] <BradB> no Soyuz
[08:50] <sabdfl> I.... think there's a little inheritance crack there, haven't looked to see if it's gooooood stuf
[08:50] <sabdfl> there are a ton of them
[08:50] <BradB> Only one's an SQLObject though.
[08:50] <BradB> I found the non-SQLObject one's already
[08:50] <BradB> s/one's/ones/
[08:52] <daf> spiv: the authserver (a) seems to have failing unit tests and (b) seems to be accessing the database inside the unit tests
[08:54] <spiv> Ok, I didn't break that test ;)
[08:55] <daf> :)
[08:55] <spiv> I'm open to ideas on how to better segregate the database-dependent tests without nasty contortions.
[08:56] <elmo> SteveA: am I okay to put this on the rosetta devel server now then?
[08:57] <spiv> daf: just to confirm, you're only seeing one failing test (test_createUserUnicode)?
[08:57] <daf> only one from authserver, I think, yes
[08:58] <spiv> Ok, good :)
[08:58] <daf> oh, also: "Failed doctest test for canonical.database.sqlbase.quote"
[09:03] <BradB> sabdfl: So does it make sense that an spr knows it's spf? Shouldn't it be the sp that knows that?
[09:04] <BradB> s/it's/its/
[09:04] <sabdfl> bradb: good point
[09:04] <sabdfl> hmm...
[09:04] <sabdfl> yes, i think that's a winner
[09:04] <sabdfl> thinko from me on the db model
[09:05] <sabdfl> could you move it please?
[09:05] <BradB> sure
[09:05] <sabdfl> also, mail celso, he'll need to update some code in soyuz
[09:05] <sabdfl> though i doubt anything depends on that
[09:06] <BradB> ok
[09:45] <BradB> SteveA, daf: What was still needed before you guys could document on the wiki the process by which we create functional doctests?
[09:46] <daf> BradB: I've written a document
[09:46] <daf> BradB: Steve wants to make some changes before I send it to the list
[09:46] <daf> I don't know what these changes are
[09:48] <BradB> ok
[09:51] <daf> you could look in lib/canonical/ftests/{test_pages.py,page-tests/}
[09:51] <SteveA> BradB: I'll be checking in the first proper ones shortly
[09:53] <SteveA> BradB: if you want to do some ad-hoc ones while I'm doing these, ask daf for a quick hint on using the script.  You can move yours into the new location after I've send those instructions.
[09:53] <daf> BradB: the procedure is basically this:
[09:54] <daf> ./utilities/page-test-helper > lib/canonical/ftests/page-tests/blah-blah-blah.txt
[09:54] <daf> go to http://localhost:9000/some/page
[09:54] <daf> ^C
[09:54] <daf> $EDITOR lib/canonical/ftests/test_pages.py
[10:48] <SteveA> daf: I'm confused as to how ftesting.zcml runs at all :-/
[10:49] <SteveA> daf: It is going to take me a bit longer to look into this.  I'll be checking in the full pagetests stuff tomorrow, not tonight.  If you want to start writing some page tests, do it the way you've been doing it, and I'll move them to the new system tomorrow.
[10:49] <SteveA> same goes for you, BradB
[10:50] <BradB> o
[10:50] <BradB> er, ok
[10:54] <daf> SteveA: ok
[10:54] <daf> SteveA: I think I can get by without them for another day
[10:55] <daf> SteveA: string.encode('ascii', 'replace') seems to allow non-ASCII characters through, which is unexpected
[10:56] <spiv> daf: That's what 'replace' means.
[10:57] <spiv> file:///usr/share/doc/python2.3/html/lib/module-codecs.html
[10:57] <spiv> Perhaps you mean 'strict'?
[10:57] <daf> hmm
[10:58] <spiv> (the docs aren't particuarly clear, admittedly)
[10:58] <daf> ok, I wasn't sure about the behaviour of 'replace' when it doesn't have a replacement
[10:58] <daf> yes, the docs could be better
[10:58] <spiv> Actually, file:///usr/share/doc/python2.3/html/lib/string-methods.html is the right docs to read here :)
[10:58] <daf> right, I found that one
[10:58] <daf> which wasn't easy :)
[10:59] <spiv> Although the precise meaning of 'replace' isn't clear unless you read the other doc, too...
[10:59] <spiv> ...and in fact, it wrongly claims 'strict', 'replace' and 'ignore' are the only possible values...
[11:00] <spiv> ...when xmlcharrefreplace is also valid.
[11:00] <lifeless_> spiv: yoyoyo
[11:01] <spiv> lifeless_: Yo.
[11:01] <spiv> Hmm, you're an hour earlier than I expected. :)
[11:01] <spiv> Maybe my arithmetic was wrong.
[11:02] <daf> spiv: you are clearly the man to update the docs :)
[11:03] <spiv> lifeless: I take it the importd wants to be able to have multiple db transactions open at once?
[11:03] <spiv> daf: I'll add a note about it to my todo list... :)
[11:03] <lifeless> spiv: erm, shouldn't
[11:04] <spiv> lifeless: Oh, really?  That'll make my life easier :)
[11:05] <daf> spiv: :)
[11:06] <spiv> daf: I've added it, but I'm afraid it's at the bottom :)
[11:06] <daf> spiv: and it's reaaally long, right? :)
[11:07] <spiv> daf: Not yet, but growing ;)
[11:07] <lifeless> spiv: there are 3 sets of transactions.
[11:07] <spiv> lifeless: Oh, sorry, I'm getting myself confused for no good reason :)
[11:07] <ddaa> (23:06:37) ddaa: lifeless: I see that... a bit wary of that... there are a modules called library, log, patch, there... quite common names...
[11:08] <lifeless> ddaa: yeah, its a todo to tidy up how we get at that code, I didn't want cscvs to specify anything in the canonical namespace.
[11:08] <lifeless> as cscvs is -> public soon.
[11:08] <ddaa> I'm all with you on that.
[11:09] <lifeless> and, it works, so yeah, just add it dude ;)
[11:09] <spiv> lifeless: I'm rather concerned that taxi appears to want to call canonical.arch.database.nuke()!
[11:09] <lifeless> spiv: comment that out.
[11:10] <lifeless> spiv: it is a test only method, no need for you to fix/replace/be concerned.
[11:10] <spiv> lifeless: Ok.  *sidelong glance*
[11:10] <spiv> Hah.
[11:10] <spiv> A comment reassuring me that it's test only would be appreciated.
[11:10] <spiv> But anyway :)
[11:11] <ddaa> Comment to note that this should be removed because it's too dangerous even as a comment?
[11:11] <lifeless> you know, I was going to do that, then I go the merge regression-check-failure, so I didn't.
[11:11] <lifeless> just nuke it if you like as you go.
[11:11] <lifeless> nuke the call I mean.
[11:11] <spiv> Right :)
[11:12] <spiv> So, taxi is a module that's called in a single-threaded process?
[11:12] <lifeless> spiv: nope
[11:12] <lifeless> taxi is called from a thread of buildbot.
[11:12] <spiv> lifeless: Drat.
[11:12] <lifeless> possibly concurrently.
[11:12] <spiv> Ok :)
[11:13] <lifeless> 'm sure there are concurrency bug in there. overhauling that code is on my 'as soon as possible' short-list.
[11:13] <spiv> Heh.
[11:13] <lifeless> which is distinguished from my long list in some unfathomable manner.
[11:13] <spiv> :)
[11:14] <spiv> Ok, I think I know what to do now.
[11:14] <lifeless> ddaa: what hack are you looking for ?
[11:14] <ddaa> Not looking for, providing.
[11:14] <spiv> The threadedness make me suspectc that this code was horribly broken before, though :)
[11:14] <lifeless> ddaa: all the same, 'what one'
[11:14] <ddaa> Automagic choice of spawning strategy.
[11:15] <spiv> Because our zopeless sqlobject support hasn't had any support for that.  Ill add it now, because you obviously need it :)
[11:16] <lifeless> right, you want to check if there is a twisted module already imported, and if there is, do
[11:16] <lifeless>         import arch._tla
[11:16] <lifeless>         import arch._twisted
[11:16] <lifeless>         arch._tla.spawn = arch._twisted.TwistedSpawningStrategy('tla')
[11:17] <ddaa> lifeless: that's something much more manly I did.
[11:21] <lifeless> spiv: eek. eek. eek. 
[11:22] <spiv> lifeless: quite.
[11:32] <lifeless> elmo: so, am I lucky yet?
[11:36] <lifeless> ddaa: am I right in saying that _package_revision_param should awlays return a value?
[11:36] <ddaa> yes
[11:36] <lifeless> heheheh then look at the second half of it
[11:36] <lifeless> spot the pyarch bug
[11:37] <lifeless> return p.fullname ?
[11:38] <ddaa> return pkg_rev
[11:38] <ddaa> compare with _version_revision_param
[11:40] <sabdfl> incoming pqm message makes productseries core complete (create, view, edit)
[11:44] <lifeless> spiv: ok, those buildbot tweaks are on the wire now that bug is fixed (it was always there, tla improvements just showed it up)
[11:45] <lifeless>  I'm gonna get some breakfast, if you've no immediate queries for me?
[11:45] <ddaa> incoming pqm message brings manly pyarch spawning strategy that makes buildbot happy.
[11:45] <lifeless> ddaa: YAY!
[11:47] <spiv> lifeless: enjoy your breakfast :)