[12:16] <dilys> Merge to rocketfuel@canonical.com/launchpad--devel--0: [trivial]  PackageVersionVerification (patch-2174)
[12:36] <dilys> New Malone bug 1591 filed on Bazaar by Matthieu Moy: baz rm non-existant-file should give a clean error message.
[12:36] <dilys> https://launchpad.ubuntu.com/malone/bugs/1591
[12:42] <dilys> Merge to rocketfuel@canonical.com/launchpad--devel--0: Add Soyuz legalese. r=sabdfl (patch-2175: daniel.silverstone@canonical.com)
[01:09] <dilys> Merge to rocketfuel@canonical.com/launchpad--devel--0: [trivial]  fixes to various page titles (patch-2176: mpt@canonical.com)
[02:22] <dilys> Merge to rocketfuel@canonical.com/launchpad--devel--0: Fix checkwatches.py, take 2: s/logger/log and other randomness, r=stub (patch-2177: christian.reis@canonical.com)
[10:40] <carlos> morning
[10:49] <elmo> ok, wiki migrated - until DNS propogates (30 mins or so), an editable copy is reachable via https://wiki2.launchpad.canonical.com/ (NB: https till DNS propogates)
[10:49] <elmo> the old one is uneditable, so don't worry about editing the wrong one
[10:50] <carlos> elmo, what changes with that migration?
[10:50] <elmo> oh, y'all have to use your launchpad login now
[10:50] <elmo> carlos: I've no idea, I just got "MOIN 1.3 NOW. SCHNELLE, SCHNELLE, BITTE" orders from brazil
[10:51] <carlos> ok, so it's a version update
[10:51] <carlos> + launchpad integration
[10:51] <jordi> hello
[10:52] <carlos> jordi, !
[10:52] <carlos> jordi, tonight acs, Kal, xavi de blast and Ismael will be in my house having dinner, do you want to come?
[10:59] <jordi> carlos: I can't, I have another one :(
[10:59] <carlos> ok
[11:36] <dilys> Merge to rocketfuel@canonical.com/launchpad--devel--0: [trivial]  Improve descriptions in registry interfaces (patch-2178: morgan.collett@canonical.com)
[01:55] <kiko> ahoy
[01:56] <Kinnison> ahoyhoy kiko
[01:56] <kiko> the pillowman
[02:17] <carlos> SteveA, hi, around?
[02:54] <SteveA> hi carlos 
[02:54] <SteveA> i'm around
[02:55] <carlos> SteveA, launchpad.Append does not exists 
[02:56] <carlos> also, https://launchpad.ubuntu.com/malone/bugs/1564 -> a bug I think it's related with the sqlobject garbage collector, that bug is a bit critical for Rosetta
[02:57] <daf> carlos: have you tried using row.expire()?
[02:58] <carlos> daf, no, I didn't 
[02:58] <carlos> the problem is that I need to change it, commit and ask stub to run it
[02:58] <SteveA> carlos: I said:
[02:58] <SteveA>   Add a launchpad.Append permission to permissions.zcml.  Make this
[02:58] <SteveA> permission be used for the +addpotemplate form.
[02:58] <carlos> daf, anyway, it's still a bug
[02:58] <SteveA> 
[02:58] <carlos> oh, ok, I missed that part
[02:58] <carlos> SteveA, thank you
[03:02] <Kinnison> fortunately he then ran off
[03:16] <dilys> Merge to rocketfuel@canonical.com/launchpad--devel--0: [trivial, rs=sabdfl]  Tighten comment spacing in bug report pages (patch-2179: mpt@canonical.com)
[03:19] <SteveA> carlos: hello
[03:19] <SteveA> carlos: stub and i have been looking at the memory issues when running that script
[03:20] <SteveA> carlos: stu will try another run of it in an hour or so
[03:23] <carlos> ok
[03:23] <carlos> SteveA, is it related to the way I implemented it?
[03:23] <carlos> or it's a problem with sqlobject?
[03:23] <SteveA> you're doing a commit every so often
[03:23] <SteveA> the caches should be cleared out on a commit
[03:23] <carlos> once per row
[03:23] <SteveA> although, you may need to explicitly del and run gc some things.  we'll see how the next run goes.
[03:24] <SteveA> running one commit per row may end up being very slow, but it's a good enough start.
[03:24] <SteveA> holding onto objects across a commit is a bit dodgy
[03:24] <SteveA> so, you should probably get a list of ids
[03:24] <SteveA> and then for each id, get the object of that id, do the work, commit
[03:25] <SteveA> that avoids the problem of hanging onto objects across a commit
[03:32] <carlos> SteveA, ok, will take into account it for the next script of that kind I develop
[03:32] <SteveA> please rewrite this one
[03:32] <carlos> in case next run works....
[03:32] <SteveA> to use a list of ids
[03:32] <SteveA> even if it works, it will be a source of bugs
[03:32] <SteveA> because we shouldn't be holding onto objects across a commit
[03:32] <carlos> SteveA, once it's executed the script will not be needed anymore
[03:32] <SteveA> also, stub says that the latest sqlobject fixes don't fix it
[03:32] <carlos> it's a migration script
[03:33] <SteveA> 1. if it will remain in RF, it needs to be right.  2. it still doesn't work.
[03:33] <carlos> ok
[03:33] <SteveA> for point 1, other people will be using it as an example of how to write this kind of code.
[03:34] <carlos> SteveA, If it will stay in RF after the migration is done, we should find a better place to store those kind of scripts than the pending db directory
[03:34] <SteveA> ok
[03:35] <daf> see also database/schema/pending/daf-migrate-pre-po-template-rawfile-2.py
[03:38] <carlos> ok
[03:40] <daf> (it's been there for months and months)
[03:43] <carlos> daf, anyway the amount of potemplates and pofiles is lower than the amount of potranslations
[03:43] <carlos> daf, so perhaps you don't have the memory problem because that
[03:43] <daf> sure
[03:44] <daf> I was just pointing it out as another example of a Python data migration script
[03:44] <daf> we were having problems with the query to get PO files pending import running out of memory
[03:44] <daf> you remember?
[03:47] <carlos> daf, but that was because the attachments being huge, right?
[03:47] <daf> yes
[03:48] <daf> that script had that problem
[03:48] <daf> because it needed to get all the attachments
[03:48] <daf> and put them in the Librarian
[04:10] <dilys> Merge to rocketfuel@canonical.com/launchpad--devel--0: [trivial, rs=sabdfl]  Tighten comment spacing in bug report pages (take 2) (patch-2180)
[04:46] <mdke> what infrastructure does launchpad use for authentication? is there any information about it anywhere that I can have a look at?
[04:46] <daf> infrastructure?
[04:47] <mdke> i've found a page on the wiki... it refers to ZODB
[04:48] <mdke> that is where the account information is stored i guess?
[04:48] <daf> no
[04:48] <daf> account information is stored in Postgres
[04:48] <daf> the ZODB is used for session information
[04:49] <mdke> ok...
[04:50] <mdke> i don't know what either of them are anyhow :)
[04:50] <bob2> ZODB = Z Object database
[04:50] <bob2> which i a python object persistence deal
[04:51] <daf> yep
[04:51] <daf> whereas Postgres is a relational database
[04:51] <mdke> so the authentication server is the latter?
[04:52] <daf> it's a bit more complicated than that
[04:52] <mdke> damn
[04:52] <daf> there is a daemon called the authserver
[04:53] <daf> when somebody logs in to Launchpad, it asks the authserver if the account is valid
[04:53] <daf> the authserver looks in Postgres
[04:53] <daf> Launchpad uses the ZODB to keep track of which users are logged in
[04:53] <mdke> ok
[04:53] <mdke> so on something like www.ubuntu.com, ZODB isn't used at all, only the authserver
[04:53] <daf> other things like wikis use the authserver for authentication
[04:54] <bob2> why doesn't LP hit the db itself?
[04:54] <daf> bob2: so that authentication code for all our services is in one place, I'd imagine
[04:54] <mdke> daf, is the authserver like an ldap server?
[04:54] <daf> mdke: I'm not sure how one does authenticatoin with LDAP, so I can't say
[04:55] <mdke> ah ok
[04:55] <daf> I don't think www.ubuntu.com uses the ZODB, no
[04:55] <mdke> so in theory, anything can use the auth server?
[04:55] <mdke> if for example ubuntuforums wanted to unify their login, they could do it?
[04:57] <daf> hmm
[04:57] <daf> good question
[04:57] <daf> in principle, yes
[04:57] <ddaa> graaaagrl... recent update broken my firefox :(
[04:57] <daf> there may be some problems with the authserver being firewalled off
[04:58] <bob2> ddaa: I need to ask you about pybaz and debian at some point
[04:58] <bob2> ie is the public version api-stable enough
[04:58] <mdke> daf, sure, but in principle yes.
[04:58] <ddaa> bob2: hehe
[04:58] <mdke> daf, is there anything I can read on how the authserver works?
[04:58] <ddaa> FYI. the public version == the canonical version, I merge into rocketfuel from the ddaa.net devo branch
[04:59] <bob2> I'd hope that wouldn't expose any account data to whoever ran ubuntuforums
[04:59] <bob2> ddaa: ah, cool
[04:59] <ddaa> Stability wise...
[04:59] <daf> mdke: dunno
[04:59] <ddaa> well... it's not really clear... There are some big changes right now because archive registration is being phased out of baz
[04:59] <daf> mdke: it's spiv who wrote it
[04:59] <mdke> daf, i'll search around on the wiki
[05:00] <ddaa> I mean, "registered names" are being phased out
[05:00] <bob2> ah, right
[05:00] <daf> it's a fairly simple thing that uses Twisted, as far as I know
[05:01] <ddaa> But generally, lack of time to work on it, and generally lack of future for all this code base means I just do not have the time to fix the broken API, so it's likely as stable as it will ever be.
[05:01] <ddaa> (though I hope to remove all the deprecated stuff soon)
[05:01] <ddaa> When I'm reasonably satisfied of pybaz, I release it on ddaa.net
[05:01] <bob2> is deprecated stuff marked with warnings?
[05:02] <mdke> daf, thanks for your help :D
[05:02] <ddaa> bob2: yes it is
[05:02] <daf> mdke: no worries
[05:02] <bob2> rock
[05:02] <carlos> hmm
[05:02] <carlos> lifeless, did you changed "baz diff" recently?
[05:03] <carlos> lifeless, https://chinstrap.ubuntu.com/~dsilvers/paste/fileiZSVkG.html
[05:05] <jamesh> carlos: s/launchpa--devel--0/launchpad--devel--0/ ?
[05:05] <carlos> yeah, I just saw it
[05:05] <carlos> jamesh, thanks anyway
[05:05] <ddaa> bob2: I have learn a thing with pybaz...
[05:05] <carlos> btw, bazaar error messages are a bit cryptic, aren't they?
[05:06] <ddaa> When you have an API that is broken in a way that cannot be fixed with a deprecation path (e.g. WorkingTree deriving from str), just fix it NOW.
[05:07] <ddaa> because you'd rather have such a fix behind you than in front of you...
[05:10] <bob2> haha
[05:10] <bob2> yeah
[05:29] <jblack> Morning guys
[05:29] <carlos> jblack, morning
[05:38] <dilys> Merge to rocketfuel@canonical.com/launchpad--devel--0: [trivial, rs=sabdfl]  Tighten comment spacing in bug report pages (take 3) (patch-2181)
[05:38] <jblack> ddaa: I took a look at your docs yesterday. What about % of the api did you deprecate? 
[05:40] <ddaa> mh... I guess about 20%... certainly more if you weight by use...
[05:40] <jblack> Heh. Yeah. Looks to me like I'm about 80% affected. =) 
[05:40] <jblack> You're getting rid of the lists. What should I be using instead? (The deprecations don't say what they're deprecated by) 
[05:41] <ddaa> Yeah, the bits that were deprecated tend to be frequently used in client code.
[05:42] <ddaa> sorry, but providing lists of that can be _long_, take a long time to produce due to network delays, and that are mutable by changes outside of the system, was very much a bad idea.
[05:43] <jblack> I know you think very carefully about how your choices affect people, so I know that you had good reasons. 
[05:44] <jblack> So I'm looking at it from the perspective of how to get this code back in line with the direction you're moving. 
[05:44] <ddaa> I think the ChangeLog should give you the patterns you need to grep for.
[05:45] <jblack> For categories, branches and versions, there's NameIterables. What's the archive equivilant? 
[05:45] <ddaa> Hu?
[05:45] <ddaa> NameIterables?
[05:45] <jblack> CategoryIterable, BranchIterable, VersionIterable
[05:46] <ddaa> Exposing this stuff was sooo crackful... it's all about reducing duplication of code and making pychecker happy...
[05:46] <ddaa> Archive equivalent: "arch.archives()" -> "pybaz.iter_archives()"
[05:47] <ddaa> that's one of those ones where the renaming does not really make sense by itself, only for consistence with other changes.
[05:47] <jblack> Aha. Here it is. (Finally found the archive summary) 
[05:47] <jblack> pardon, the function summary
[05:49] <jblack>  I should be fine now. 
[06:03] <ddaa> lifeless: looks like that old bug with missing branch name
[06:04] <ddaa>   File "/home/importd/dists/launchpad/lib/CVS/CacheGenerator.py", line 92, in cleanUpDefaultFillers
[06:04] <ddaa>      endFileRev = self.findLastFileRevision(rev.branch, rev.filename)
[06:04] <ddaa>    File "/home/importd/dists/launchpad/lib/CVS/CacheGenerator.py", line 124, in findLastFileRevision
[06:04] <ddaa>      return self.fileInChangeset(self._catalog.getBranch(branchName).getChangesetList()[-1] , fileName)
[06:04] <ddaa>  IndexError: list index out of range
[06:04] <ddaa> Would take me _days_ to figure out how to fix it.
[06:06] <lifeless> ddaa: try nuking all state and resetting it back to enabled, nbot syncing. I bet it will come good.
[06:06] <lifeless> ddaa: I won't be able to look at it for 3-4 weeks :[
[06:06] <ddaa> mh... has not been published... should be legal to do that...
[06:12] <jblack> bleh.
[06:19] <carlos> lifeless, jblack, ddaa:  https://chinstrap.ubuntu.com/~dsilvers/paste/filesWFX6W.html
[06:20] <kiko-fud> salgado, BjornT: http://www.squarefree.com/bugkarma/bugkarma.html
[06:28] <ddaa> mh... IIRC, that one looks like a different failure
[06:30] <carlos> lifeless, ddaa jblack any suggestion to be able to commit? I'm blocked on that problem with bazaar
[06:31] <ddaa> carlos: try using 1.4.N
[06:31] <ddaa> (where N is the latest release, whatever it is ATM)
[06:32] <ddaa> looks like the kind of resource-handling issues that are bound to pop up with the changes lifeless is doing in 1.5. Extra point for making it reproducible.
[06:34] <carlos> ddaa, I can give you my current tree
[06:35] <carlos> ddaa, ok, I will revert to hoary's bazaar
[06:35] <lifeless> carlos: what is the error ? 
[06:35] <ddaa> https://chinstrap.ubuntu.com/~dsilvers/paste/filesWFX6W.html
[06:35] <lifeless> carlos: I'm at the end of a long thin pipe, web pages DoNotHelp
[06:35] <ddaa> carlos: current tree would certainly help
[06:37] <carlos> lifeless, /pqm/build_dir/thelove@canonical.com/thelove@canonical.com---hackerlab--devo--1.5/src/hackerlab/vu/vfdbuf.c:1681:botched invariant
[06:37] <carlos>     avail >= count
[06:37] <carlos> baz: uncaught exception: -1:(exiting on botched invariant)
[06:37] <carlos>   please report this as a bug to bazaar@lists.canonical.com
[06:37] <carlos> lifeless, I get that with a commit
[06:37] <ddaa> just after the changeset generation
[06:37] <carlos> ddaa, ok, I suppose a tar.gz would be enough, right?
[06:37] <lifeless> carlos: thanks. That is new and interesting :[
[06:38] <ddaa> carlos: bz2 preferred, considered the size of a launchpad tree nowadays
[06:38] <lifeless> carlos: I'm fixing now, rollback a few baz's to get it fixed in the interim
[06:38] <lifeless> carlos: no need to send the tree around
[06:39] <carlos> ok
[06:41] <lifeless> fixed, testing
[06:42] <carlos> is it too bad if I go back to 1.2?
[06:42] <carlos> it's the version available with hoary
[06:43] <dilys> Merge to rocketfuel@canonical.com/launchpad--devel--0: [trivial]  Fixed the permissions for the addpotemplate form + test r=Steve (it's not really a review but implemented the way he asked) (patch-2182: carlos.perello@canonical.com)
[06:44] <carlos> stub, lifeless could you cherrypick that patch? ^^^
[06:47] <lifeless> carlos: fixed, uploading
[06:57] <carlos> lifeless, cool, thanks
[07:04] <lifeless> sprint guys : kiko, me, stevea and mark are in a meeting for the next 60 minutes odd - please take care of yourselves/next break etc.
[07:04] <lifeless> Kinnison: ^^
[07:05] <SteveA> well, i was going to announce it in 5 mins
[07:08] <dilys> New Malone bug 1597 filed on Bazaar by David Allouche: baz switch to fail with unrelated revision
[07:08] <dilys> https://launchpad.ubuntu.com/malone/bugs/1597
[07:10] <kiko-fud> ddaa!
[07:10] <ddaa> kiko-fud: ?
[07:11] <kiko-fud> I didn't get a chance to say bye, how was your trip home?
[07:12] <ddaa> unexpectedly lengthy (as I lost my password on the bus to Sao Paulo) but otherwise reasonably pleasant (as Varig changed my ticket for free for the day after, and I got a business class seat).
[07:13] <kiko-fud> ddaa, password?
[07:13] <ddaa> hu... passport
[07:13] <ddaa> same difference :)
[07:13] <kiko-fud> you're joking?
[07:13] <ddaa> no, I'm not
[07:14] <ddaa> It was reasonably easy to get a right-of-the-way at the french consulate, and a helpful .br policeman at the airport put me on the track.
[07:17] <kiko-fud> where did you sleep?
[07:18] <ddaa> NH hotel near the consulate, they had a week-end price (about R$150)
[07:18] <lifeless> thats like 2francs right ?
[07:18] <ddaa> I'll try to expense it though
[07:19] <ddaa> lifeless: no, that's more like 300FF
[07:19] <ddaa> or 60 eurs
[07:21] <ddaa> I could have found cheaper, but I decided it was too much to ask at midnight, with my luggage, in a foreign country whose language I do not speak :)
[07:22] <lifeless> yah
[07:22] <lifeless> should be expensable I think
[07:26] <jblack> lifeless: porting to pybaz looks easy enough. Got reminded in a bad way that elmo hasn't fixed pydavlib.
[07:29] <dilys> Merge to thelove@canonical.com/dists--bazaar--1.5: new build (patch-57)
[07:29] <dilys> Merge to thelove@canonical.com/hackerlab--devo--1.5: unbreak vfd buffers where a write is smaller than the current buffer size increase (patch-12: robert.collins@canonical.com)
[07:33] <lifeless> carlos: ^^ the fix.
[07:33] <carlos> lifeless, thanks!
[07:46] <ddaa> lifeless: cannot run importd locally... looks like your "reconnection" hack broke something.
[07:46] <ddaa> haha...
[07:47] <lifeless> ddaa - /my/ reconnection hack ?
[07:47] <ddaa> Must be the removal of implicit begin...
[07:47] <lifeless> oh, db reconnection.
[07:47] <lifeless> command line runner you mean ?
[07:47] <ddaa> nah buildbot... but actually it's not your hack
[07:48] <ddaa> well, it's the "spiv who is reviewing this suggested this XXX abstraction violation. RBC 20050608"
[07:50] <ddaa> yeah, working now
[07:50] <lifeless> do not be doing that in production
[07:50] <lifeless> I'd like to know why its not working locally for you
[07:51] <ddaa> that's because it's doing txn.abort() before the first txn.begin() in jobsFromDB when the service was not setup (i.e. pristine tap).
[07:54] <ddaa> mh... probably the sqlos reconnection broke the assuption that SQLBase._connection was not None when no transaction was begun.
[07:54] <dilys> Merge to rocketfuel@canonical.com/launchpad--production--1.26: Cherry pick patch-2177 into production (patch-4: rocketfuel@canonical.com, christian.reis@canonical.com)
[08:00] <lifeless> jobsFromDB should do what hte test I did does and check for a transaction then
[08:01] <ddaa> lifeless: the issue is almost certainly the fact that production launchpad is based on 1.22, then
[08:02] <ddaa> I'm being called
[08:03] <ddaa> I'll look into upgrading importd's launchpad soon.
[08:05] <Kinnison> Can the launchpad webapp launch a long-running process to run in the background?
[08:06] <SteveA> um
[08:06] <SteveA> sounds yucky
[08:06] <Kinnison> I want to trigger a germinate run on a web request, but not hold the request until the germinate completes
[08:06] <Kinnison> It could be a thread, I don't mind that
[08:06] <SteveA> we need to talk about this
[08:07] <SteveA> there are many complexities here
[08:07] <Kinnison> okay
[08:07] <lifeless> rosettas export process is a good example of a resasonable approach
[08:07] <SteveA> for example, we'll have many server machines with many launchpad webapp servers on them
[08:07] <SteveA> ideally, you'd use the database to signal to the germinate box
[08:08] <SteveA> the database is our central point of coordination
[08:08] <lifeless> i think there is a clear line to these things - either they are near-instant or they get queued and processed asynchronsouly
[08:08] <Kinnison> Okay, so I have to queue them? Hmm
[08:08] <Kinnison> That's fine
[08:08] <SteveA> okay
[08:08] <SteveA> if you need it more responsive, we can do something about that.  but i'd rather not.
[08:09] <Kinnison> naah, a cronscript will be fine
[08:09] <SteveA> good
[08:22] <dilys> Merge to rocketfuel@canonical.com/launchpad--production--1.26: Cherry pick patch-2182 into production (patch-5: carlos.perello@canonical.com, rocketfuel@canonical.com)
[08:23] <carlos> stub, lifeless thanks
[08:58] <carlos> mpt, ping
[10:00] <carlos> Kinnison, hi, around?