[12:03] <zyga> heh, then I guess someone needs to write delayfs
[12:03] <ddaa> sounds like a good idea actually
[12:04] <zyga> delay underlying fs operations untill timeout ;-)
[12:04] <Kinnison> baz is fook-fast in a ramdisc
[12:04] <zyga> that was an intelligent joke I guess...
[12:04] <ddaa> "benchmark how your code behaves with incrementally worse filesystem performance".
[12:05] <Kinnison> huzzah, gina run on dogfood almost finished. only 13 failed binarypackage imports
[12:05] <Kinnison> and some of those are due to sources which couldn't be imported rather than couldn't be found
[12:05] <ddaa> Kinnison: I guess the failures include the kernel, OOo, Mozilla, and gcc?
[12:06] <ddaa> like the bits that nobody uses anyway...
[12:06] <zyga> heh
[12:06] <zyga> fixing those packages is left as an exercise for the reader ;
[12:07] <ddaa> Enough talk, I need to get more fixes in for the samba import.
[12:07] <Kinnison> ddaa: zope3 (for zope3-lib)
[12:07] <ddaa> SVN BUGS ARE FUTILE, YOU WILL BE IMPORTED!
[12:08] <ddaa> (actually, I'm unfair, these are cscvs and pysvn bugs, not svn's fault)
[12:08] <Kinnison> ddaa: gcc-2.95 (oddly)
[12:09] <Kinnison> ddaa: some ruby crud, and a d-i manual
[12:09] <ddaa> pretty good
[12:09] <Kinnison> and a few whose SPs won't import due to version number h0rkage
[12:12] <ddaa> Kinnison: I propose the upstream theorem
[12:12] <ddaa> For any value of upstream, upstream does stupid things.
[12:12] <Kinnison> Heh
[12:14] <Kinnison> 20:49:08 WARNING Sourcepackage openoffice.org2 (1.9.125-1ubuntu2) (1.9.125+2.0beta2-1ubuntu2) not found for openoffice.org2 (1.9.125+2.0beta2-1ubuntu2)
[12:14] <Kinnison> that's the annoying one
[12:14] <Kinnison> I can see no reason why that failed
[12:14] <ddaa> I knew it :)
[12:14] <Kinnison> especially since earlier in that run it imported 1.9.125-1ubuntu2
[12:34] <dilys> Merge to rocketfuel@canonical.com/launchpad--devel--0: [r=Keybuk]  fix #2361. make bugmail From: be the address of the person who made the change, Reply-To: be the bug address, and Sender: be the LP-wide bounce address. (patch-2460: brad.bollenbach@canonical.com)
[12:48] <Kinnison> bloody thing is suspended
[01:13] <sivang> good night all
[01:24] <elmo> 1000     27048 24.1 27.3 1216676 1080420 ?     Ssl  Sep21 165:39      \_ python /srv/launchpad.net/production/launchpad/cronscripts/rosetta-poimport.py -q
[01:24] <elmo> you guys know about that, right? :P
[01:31] <spiv> elmo: That's exciting.
[01:32] <spiv> Good thing hardware is so cheap ;)
[01:35] <Kinnison> Woohoo, another six SPs imported okay
[01:35] <Kinnison> only 7 left unimportable
[01:35] <Kinnison> at least one of which truly is missing
[01:37] <Kinnison> two...
[01:38] <Kinnison> dunno wtf to do about one of 'em
[01:38] <Kinnison> so two are missing
[01:38] <Kinnison> one is bizarre
[01:38] <Kinnison> and the other four are because of version numbers
[01:38] <Kinnison> much better
[01:39] <Kinnison> elmo: zope3 and qdvdauthor (both universe) both lack source
[01:40] <elmo> qdvdauthor | 0.0.10-0.1 | breezy/multiverse | source
[01:40] <elmo> qdvdauthor | 0.0.10-0.1 | breezy/universe | amd64, i386, powerpc
[01:40] <elmo>      zope3 |   3.0.93-1 |        breezy | source, amd64, i386, ia64, powerpc, sparc
[01:40] <elmo> no they don't?
[01:41] <Kinnison> Why is the source in multiverse but the binaries in universe? (for qdvdauthor)?
[01:41] <elmo> they shouldn't be; I've fixed that
[01:41] <elmo> but source in one component, binaries in another is generally valid
[01:41] <elmo> it's very common for source to be in main and binaries to be in universe, f.e.
[01:41] <Kinnison> aye, but we weren't importing multiverse at all
[01:42] <Kinnison> which was the snag there I guess
[01:42] <Kinnison> launchpad@mawson:/srv/launchpad.ubuntu.com/dogfood/launchpad$ ls /srv/archive.ubuntu.com/ubuntu/pool/universe/z/zope3/*
[01:42] <Kinnison> /srv/archive.ubuntu.com/ubuntu/pool/universe/z/zope3/zope3-doc_3.0.93-1_all.deb
[01:42] <Kinnison> /srv/archive.ubuntu.com/ubuntu/pool/universe/z/zope3/zope3-lib_3.0.91-1ubuntu1_amd64.deb
[01:42] <Kinnison> /srv/archive.ubuntu.com/ubuntu/pool/universe/z/zope3/zope3-lib_3.0.91-1ubuntu1_i386.deb
[01:42] <Kinnison> /srv/archive.ubuntu.com/ubuntu/pool/universe/z/zope3/zope3-lib_3.0.91-1ubuntu1_powerpc.deb
[01:42] <elmo> Kinnison: pool/universe/
[01:42] <elmo> Kinnison: try pool/main/
[01:42] <Kinnison> aye, they're different versions
[01:42] <Kinnison> that which was in main has been imported fine
[01:42] <Kinnison> so 3.0.93-1 is fine
[01:43] <Kinnison> 3.0.91-1ubuntu1 however (zope3-lib) is failing to import
[01:43] <Kinnison> I'm guessing it's a dangling binary
[01:43] <elmo> /srv/ftp.no-name-yet.com/ftp/pool/main/z/zope3/zope3_3.0.91-1ubuntu1.diff.gz
[01:43] <elmo> /srv/ftp.no-name-yet.com/ftp/pool/main/z/zope3/zope3_3.0.91-1ubuntu1.dsc
[01:43] <elmo> looks fine to me?
[01:43] <Kinnison> bizarre
[01:43] <elmo> seriously - i know there are flaky parts of the katie, but the source reference counting is NOT one of them
[01:43] <Kinnison> I believe you
[01:43] <Kinnison> oh one last one which is confusing me:
[01:44] <Kinnison> 23:23:56 WARNING Sourcepackage debian-installer (20050317ubuntu15.0.200509200) not found for debian-installer-manual (20050317ubuntu15.0.200509200)
[01:44] <elmo> heh
[01:44] <elmo> blame lamont
[01:44] <Kinnison> is that a bizarro bin-nmu or something?
[01:44] <elmo> they're exploiting the lax bin-only NMU checks to do daily d-i builds
[01:44] <elmo> yes, basically
[01:45] <Kinnison> why is the Source: line wrong then?
[01:45] <elmo> is it?  currently we don't have a bin-only NMU version but a real one in the archive
[01:46] <Kinnison> dunno because this is an archive from 4am
[01:46] <Kinnison> Package: debian-installer-manual
[01:46] <Kinnison> Source: debian-installer
[01:46] <Kinnison> Version: 20050317ubuntu15.0.200509200
[01:46] <elmo> yeah, ok the source is wrong
[01:46] <elmo> I guess it's an artifiact of how lamont's script works
[01:46] <Kinnison> okay
[01:47] <elmo> you may be able to get him and/or infinity to fix it
[01:47] <Kinnison> right
[02:36] <salgado> stub!
[02:36] <stub> yo
[02:36] <salgado> what patch number should I use for my basic-voting branch?
[02:39] <stub> I havn't seen the patch yet, have I?
[02:40] <salgado> stub: that patch to drop the unique constraint in Vote.token 
[02:40] <stub> If that is the only change, I'll pre-approve it and give you patch number 25-29-0
[02:41] <salgado> stub: this is the patch: https://chinstrap.ubuntu.com/~dsilvers/paste/fileSwV5Uo.html
[02:45] <stub> salgado: That looks fine. Approved - 25-29-0
[02:45] <salgado> cool. thank you
[02:58] <sladen> can somebody kick launchpad.  It's still b0rken and hanging on page load
[03:36] <dilys> Merge to rocketfuel@canonical.com/launchpad--devel--0: Simple and Condorcet polls (although the latter is disabled for now) for teams. r=BjornT (patch-2461: guilherme.salgado@canonical.com)
[04:55] <zakame> hello all!
[04:55] <zakame> is there a prob?  I'm getting a 502 when posting translations at rosetta
[05:41] <stub> spiv: Librarian connections appear to be working differently than they used to. It used to keep about 10 connections open, and now it only opens connections when it needs them. 
[05:42] <stub> Is this fallout from the transaction debugging, and does it mean the Librarian should no longer need bouncing after a database outage?
[05:44] <spiv> stub: Yes and maybe. :)
[06:36] <dilys> Merge to rocketfuel@canonical.com/launchpad--devel--0: [trivial]  Ensure specialized log handlers work for the root logger too and merge in production db patch (patch-2462: stuart.bishop@canonical.com)
[08:40] <Burgundavia> should I be getting forbidden on this page? https://launchpad.net/malone
[08:41] <SteveA> nope
[08:42] <SteveA> it is because that page is trying to display a bug that is private to you
[08:42] <SteveA> it's a bug
[08:42] <SteveA> BjornT: around?
[08:43] <Burgundavia> I get when I click on the bug tab on one of my bugs
[08:44] <SteveA> we'll get it fixed shortly
[08:48] <Burgundavia> cheers
[08:55] <stub> Burgundavia: We already have a fix. It will be rolled out next Tuesday.
[08:55] <SteveA> i don't think that's good enough.  i'm cooking up an instant fix.
[09:03] <SteveA> stub: i have a minimal patch to production to fix the malone front page
[09:04] <stub> SteveA: Branch, commit and email me and lifeless the branch
[09:04] <SteveA> it's two lines in one .pt file.  can't i just log in and surgically change the file? ;-p
[09:05] <stub> SteveA: Only if you want the fix nuked the next cherry pick
[09:05] <SteveA> BORING
[09:07] <SteveA> https://chinstrap.ubuntu.com/~dsilvers/paste/fileoOQjDF.html
[09:07] <SteveA> there's the fix.  i'm making a branch...
[09:08] <stub> You can do both if you want
[09:10] <SteveA> Burgundavia: try malone now please.
[09:10] <Burgundavia> works now
[09:11] <Burgundavia> thanks for the quick fix
[09:11] <SteveA> cool, thanks for reporting it.
[09:14] <SteveA> stub: my archive is still mirroring, but i've mailed you and lifeless 
[09:14] <stub> ok.
[09:18] <SteveA> stub: do you know why when i run the launchpad tests, i get various lines of stuff on stderr?
[09:18] <SteveA> is this the root logger sending these things out?
[09:19] <SteveA> this happens when i run a subset of tests, like  python test.py canonical
[09:19] <stub> I would need to see an example
[09:19] <SteveA> 18:44:10 WARNING The plural form header has an unknown error. Using the default value...
[09:20] <SteveA> 18:43:01 CRITICAL ***** Missed Dependency build needs manual intervention due MANUALDEPWAIT *****
[09:20] <stub> Not necessarily the root logger - could be any logger
[09:20] <SteveA> 18:40:49 ERROR   http://localhost:9000/cchangelog.html
[09:20] <SteveA>  -> http://localhost:58000/36/36/tHq2ia1dvmO00LvrJ3M4seoSBnh.txt
[09:20] <SteveA> it kinda hoses my dream of having a test suite with clean output ;-)
[09:21] <stub> it would be quite possible to install a handler I think to catch tests that emit log messages and raise an exception.
[09:21] <SteveA> okay, i'll ask for culprits in the meeting
[09:22] <stub> That last one is interesting, as it indicates a subprocess is being run without redirecting stdout/stderr (and monitering it to ensure the output is what is expected)
[09:22] <SteveA> i don't think the last one is for a subprocess
[09:23] <SteveA> that's a just a page not found, a page test of a page that doesn't exist
[09:24] <SteveA> i want to write some kind of test for the "shut down the socket if there are too many pending tasks" stuff
[09:24] <SteveA> i found that in the example i sent you, for some reason, it isn't really running with multiple task threads
[09:24] <stub> It is from a subprocess, as the only thing that installs the 'log exceptions to the librarian' is canonical.launchpad.scripts.logger
[09:25] <stub> Unless there is something really screwy going on
[09:26] <SteveA> i want to check we're running launchpad with the proper number of concurrent tasks, and also shutting down the socket and re-opening it as appropriate
[09:26] <SteveA> a pagetest won't do, because this needs to be running outside of launchpad
[09:27] <SteveA> so i thought of the following: have two pages on the debug layer; one that when rendered hangs until some global changes value, another that changes the value.  have these pages log progress to some file.
[09:29] <SteveA> then, have the tester clear the file, run launchpad, get several of the "first page" pages with a second delay between gets, until the connection is refused.  check the connection is refused a few times.  then, get the second page.
[09:29] <SteveA> aha.. here's where my logic fails
[09:29] <SteveA> because of course the second page isn't accessible
[09:29] <SteveA> so instead, have the first pages look at a file on the filesystem
[09:30] <SteveA> have the file absent, and when it appears, have them continue
[09:30] <stub> Can we just have a page that interrogates the Zope internals and reports? That might be good enough (although it doesn't prove that Z3 is working as it says it is, that should really be fixed by adding tests upstream)
[09:30] <SteveA> well...
[09:31] <SteveA> the problem is that we're testing the case when we want zope to not respond to queries
[09:31] <SteveA> we could query the page on the production port, and test it on the debug port, i guess
[09:32] <SteveA> i don't want to put a "wait indefinitely" page up on production, for obvious reasons
[09:32] <SteveA> the ++etc++process stuff does something like this, i think
[09:32] <stub> I don't see a problem if it is protected. Anyway, your test could install it by adding a view.
[09:33] <stub> Unless you need to run a real Z3 instance
[09:33] <SteveA> i need to run a real z3 instance
[09:33] <SteveA> i could install these pages as overrides
[09:33] <SteveA> so they'd exist only for the purposes of this test
[09:33] <stub> Indeed, or even pages that are only loaded in the test environment
[09:34] <SteveA> it wouldn't run in the test environment
[09:34] <stub> The config machinery supports that right now
[09:34] <SteveA> it would be a special "test the behaviour of the whole server" test environment
[09:34] <SteveA> so, not a regular test environment
[09:38] <stub> You should be able to test the 'don't accept more than n simultanous connections' code without running an entire Z3 instance, by just configuring and starting up a minimal HTTP server. I'm not sure why you need to worry about multiple threads when the only thing that needs its behavior tested is the HTTP server.
[09:39] <stub> I'm worried that a test fixture that fires up an entire Z3 instance for poking at will make for flakey tests (the page tests are bad enough at that already)
[09:39] <SteveA> not intending to run this test in the test suite
[09:39] <SteveA> more for special diagnosis
[09:40] <SteveA> as no application code will affect it
[09:40] <SteveA> i'm concerned that we aren't really running multiple threads in production
[09:40] <SteveA> i guess i could give it a one off manual test
[09:44] <stub> I'm pretty certain we are running multiple threads because a bad rosetta page can lock up all four of them (and I can see what they are trying to do on emperor)
[09:45] <SteveA> okay
[09:45] <SteveA> does (FALSE OR TRUE) make any sense in sql ?
[09:49] <lifeless> SteveA: most definately running multiple threads, we've seen them when debugging hangs
[09:51] <SteveA> ok
[09:52] <stub> SteveA: No - that only makes sense in Python
[09:52] <SteveA> ok
[09:54] <SteveA> is the "abort lengthy requests" code running on staging?
[09:57] <stub> SteveA: The code might be there, but it isn't turned on.
[09:57] <SteveA> can we turn it on for staging soon please
[09:57] <stub> I'll refresh it
[09:58] <SteveA> what timeout do you recommend?
[09:59] <SteveA> hmm... got a major 500 from https://launchpad.net/malone/bugs/2491 just now
[09:59] <SteveA> looks like an apache page
[09:59] <SteveA> so maybe pound is down
[10:03] <stub> SteveA: No - launchpad is accepting connections and not doing anything with them again. So pound is passing the requests through and never returning, so Apache gives up eventually
[10:03] <SteveA> arse
[10:03] <SteveA> can you cherrypick the "max requst length" stuff?
[10:03] <stub> I think I should cherry pick that code into staging?
[10:03] <stub> heh
[10:04] <carlos> morning
[10:04] <SteveA> let's bang on it in staging now, and get it to production asap
[10:04] <SteveA> i was about to start work on getting the "socket shutdown" stuff into lp
[10:04] <SteveA> i can aim it at production, if that'll help things
[10:04] <SteveA> probably very little in this code, though
[10:26] <mvo> hm, malone gives me a 502 proxy error when I try to update some bug info in malone. is that a known issue?
[10:28] <ivoks> nice note guys :)
[10:28] <ivoks>  Launchpad is offline at the moment for maintenance. It should be back, better than ever, soon.
[10:40] <ivoks> is it only me or lp runs on speed of light now?
[10:44] <stub> ivoks: Enjoy it while it lasts - it is having issues :-/
[10:44] <stub> SteveA: Latest code is running on staging
[10:45] <SteveA> stub: what's the timeout?
[10:45] <stub> 2 seconds
[10:49] <SteveA> ok
[10:49] <SteveA> can you run a linkchecker over it and see which pages fail?
[10:52] <ddaa> Hey SteveA, read the message I left for you in the backlog yesterday about FewerBazConflicts and today's agenda?
[10:56] <SteveA> it can be on the agenda
[10:56] <ddaa> Thanks.
[10:58] <stub> SteveA: I've had linkchecker off for a while. It just can't cope with the number of URLs on launchpad at the moment and leaks RAM so it never completes
[11:09] <SteveA> how about just running wget on it in mirror mode?
[11:17] <Kinnison> Morning
[11:29] <stub> SteveA: I could do that. No idea if it will ever finish.
[11:29] <SteveA> doesn't matter, provided we get a log with pages that time out in it
[11:41] <stub> Bah - I can't drive wget
[11:41] <BjornT> SteveA: hi. i'm back now
[11:42] <SteveA> BjornT: hi.  i was going to ask about the malone front page.  but i fixed it anyway.
[11:42] <stub> https://chinstrap.ubuntu.com/~dsilvers/paste/filebhDT1c.html
[11:43] <zyga> WaterSevenUb: morning
[11:43] <BjornT> SteveA: ok, thanks
[11:43] <WaterSevenUb> zyga, 'morning ;-) though..still sleeping
[11:44] <SteveA> stub: that is lame.  try pointing it at +index
[11:44] <stub> That gives me a 404 :-(
[11:46] <stub> 127.0.0.1 - Anonymous [22/Sep/2005:10:44:30 +0100]  "HEAD /++vh++https:staging.ubuntu.com:443/++/+index HTTP/1.1" 404 214 "" "Wget/1.9.1"
[11:48] <carlos> will be back in about one or two hours
[11:48] <stub> Oh - I think the issue is wget trying to create local files.
[11:48] <stub> Despite me not wanting it too :-/
[11:57] <SteveA> stub: i have a diff i want you to review.  it is small.
[11:58] <SteveA> it is adding a facility for a "launchpad will be going down for maintenance in X minutes" message
[11:58] <stub> wget -o spider.out -S -O /dev/null -r https://staging.ubuntu.com/ seems to do the trick, although the output isn't particularly parsable it should do
[12:09] <SteveA> salgado can do a version of the template on the shipit layer to say "shipit" rather than launchpad
[12:10] <SteveA> stub: mailed you a small diff
[12:17] <stub> SteveA: You say 'root', but all the code looks for the file in the current working directory. What provides the guarantees that this will always be the launchpad root?
[12:18] <SteveA> launchpad doesn't run if you don't run it from the root
[12:18] <SteveA> it would be better to make this control file a config option, though
[12:18] <SteveA> that way, you can have different control files for different processes on the same codebase
[12:18] <stub> There is already config.root if you want to use it
[12:19] <SteveA> having the filename explcicitly in the config would make it more discoverable
[12:20] <SteveA> what do you want me to do?
[12:20] <Kinnison> is it resultset.count() to get the COUNT(*) ?
[12:23] <stub> SteveA: That all looks fine, although you should attempt to remove os.system from your brain and embrase subprocess.call instead.
[12:23] <stub> SteveA: So merge it.
[12:23] <stub> So do we have someone maintaining the production systems who actually keeps to a schedule now?
[12:24] <SteveA> okay.  i'll set up the shipit version, and then merge it.
[12:24] <SteveA> dunno, but even given 10 mins notice will be a great improvement
[12:24] <SteveA> it will stop people from entering stuff into forms, and then have it fail
[12:25] <spiv> Kinnison: Yes.
[12:25] <Kinnison> spiv: cool
[12:25] <Kinnison> spiv: I've put a gina branch on your list for review
[12:25] <spiv> Ok, I'll take a look.
[12:25] <Kinnison> spiv: and I'll have a small dominator fix for you to look at soon too
[12:25] <spiv> Sure.
[12:35] <stub> I've found a +translations page that locks staging solid :-/
[12:35] <SteveA> wahey
[12:35] <SteveA> so, it doesn't trigger the abort DB stuff?
[12:36] <stub> Nope - I'll debug it a bit more to determine if it is a single query or lots of little ones
[12:36] <SteveA> kinda rude that the abort db stuff isn't working
[12:38] <ddaa> lifeless: yay! while working on the samba problems, I isolate the quake3 problem :)
[12:38] <lifeless> ddaa: lol. nice
[12:38] <ddaa> Here's the quake3 problem: PatchedFile.apply does noop when trying to apply a diff between binary files.
[12:39] <ddaa> -> need to detect "binary files differ" kind of diffs and fallback to overwriting in those cases.
[12:39] <SteveA> eeeew... the new slashdot CSS removes the underlining on links you hover over
[12:40] <Kinnison> bizarre
[12:40] <ddaa> lifeless: BTW, I have been bothering some #svn folks yesterday, and they say that the python subversion bindings in the next release will be MUCH improved.
[12:41] <ddaa> I'm not sure how relevent that is to us, but I think that in the long term we should be moving the svn_oo stuff to something vaguely sane...
[12:42] <ddaa> a big part of my work recently is working around bugs and limitations of pysvn
[12:48] <lifeless> ddaa: indeed, thats why svn_oo exists, I tried to just use pysvn.. HAH
[12:48] <ddaa> lifeless: you know that pysvn is _not_ the official bindings
[12:49] <lifeless> ddaa: one of the two we use is
[12:49] <lifeless> ddaa: I started with the official one, which didn't have a 'commit' that worked.
[12:49] <ddaa> well, yes, but in the code I saw pysvn is very much predominant...
[12:49] <ddaa> ha, no commit... that sounds annoying.
[12:49] <lifeless> theres a reason for that.
[12:50] <Kinnison> Anyone know why:
[12:50] <Kinnison> ProgrammingError: ERROR:  deadlock detected
[12:50] <Kinnison> DETAIL:  Process 29282 waits for ShareLock on transaction 12797011; blocked by process 2222.
[12:50] <Kinnison> Process 2222 waits for ShareLock on transaction 12794766; blocked by process 29282.
[12:50] <Kinnison> would happen in a single-threaded app?
[12:51] <dilys> Merge to rocketfuel@canonical.com/launchpad--devel--0: [trivial]  Update database documentation and diagrams (patch-2463: stuart.bishop@canonical.com)
[12:52] <lifeless> Kinnison: single threaded != single connection
[12:52] <lifeless> Kinnison: i.e. librarian
[12:54] <Kinnison> lifeless: right
[01:06] <stub> Yay - not only does this page execute an SQL command that takes 164 seconds to execute, it does it (at least) twice.
[01:07] <Kinnison> woo
[01:07] <Kinnison> missing indexes? obvious brokenness? or just plain knackering SQL?
[01:07] <SteveA> stub: and the system doesn't abort it?
[01:07] <SteveA> Kinnison: the system should be aborting such queries
[01:07] <stub> Nope. I'm not seeing it issue the 'stop long running queries' to the postgresql backend (although I have seen it when running locally).
[01:08] <stub> I'll need to step through it with a debugger to see wtf the option isn't being triggered or executed.
[01:08] <segfault> no way to connect to LP today?
[01:08] <stub> But I'm looking into the dud query at the moment
[01:09] <Kinnison> SteveA: I see. I hope that's only something the webapp turns on?
[01:09] <SteveA> yes
[01:09] <SteveA> only the webapp
[01:10] <SteveA> we'll turn it on for xmlrpc too
[01:10] <Kinnison> right
[01:10] <SteveA> but not yet
[01:10] <Kinnison> 'cos the publisher sometimes runs queries which can take up to 2 minutes to return
[01:10] <Kinnison> although that may be assisted with judicious indexing
[01:30] <stub> Kinnison: I thought you were the only person left using dogfood?
[01:30] <Kinnison> I may be
[01:30] <Kinnison> but it's always worth checking
[01:30] <Kinnison> If I get a changeset with 'baz undo'
[01:31] <Kinnison> sorry, with 'baz diff -o ,,blah'
[01:31] <Kinnison> can I apply it (in reverse) to another tree with baz apply-changeset --reverse ,,blah ?
[01:33] <SteveA> launchpad developers meeting in about half an hour.  /msg me agenda items
[01:40] <SteveA> cprov: what
[01:40] <SteveA> cprov: what's PBR?
[01:41] <cprov> SteveA: PVR, sorry (Personal Video Recorder)
[01:41] <SteveA> i see
[01:42] <niemeyer> Good morning!
[01:48] <dilys> Merge to rocketfuel@canonical.com/launchpad--devel--0: r=stub add 'launchpad (or shipit) is going down for maintenance' facility. (patch-2464: steve.alexander@canonical.com)
[01:53] <lifeless> bombs away
[01:54] <Kinnison> pardon?
[01:55] <lifeless> sent off my 81K patch for bzr-twisted for thoughts and feedback
[01:56] <spiv> lifeless: Whee!
[01:56] <Kinnison> lifeless: is that a shiny twisted xmlrpc server for bzr?
[01:56] <lifeless> Kinnison: hell no
[01:56] <Kinnison> what is then?
[01:56] <lifeless> Kinnison: its twistified bzr, which is 15 minutes to pull, not 93.
[01:57] <SteveA> meeting in 3.5 or so
[01:57] <Kinnison> lifeless: cool
[01:57] <Kinnison> lifeless: is there a bzr development mailing list?
[01:57] <lifeless> bazaar-ng@l.c.c
[01:57] <Kinnison> righty
[01:58] <SteveA> how much traffic?
[01:59] <spiv> This much traffic: http://gmane.org/details.php?group=gmane.comp.version-control.bazaar-ng.general ;)
[01:59] <spiv> (look!  pretty graphs!  ooh!)
[01:59] <SteveA> mpt, salgado: i just landed a "shipit (or launchpad) is going down for maintenance" page
[01:59] <SteveA> could do with UI love
[01:59] <mpt_> SteveA: Such a page already exists
[01:59] <SteveA> s/page/message on main template/
[01:59] <mpt_> SteveA: launchpad/offline.html
[02:00] <SteveA> this is a message about goin down soon
[02:00] <Kinnison> meeting time
[02:00] <mpt_> oh, goING
[02:00] <mpt_> ok
[02:00] <SteveA> MEETING TIME
[02:00] <SteveA> who is here?
[02:00] <salgado> me
[02:00] <BjornT> me
[02:00] <spiv> me
[02:00] <mpt_> I'm here, but I won't be for much longer
[02:00] <SteveA> kiko-zzz: ?
[02:00] <lifeless> I appear to be
[02:00] <mpt_> I'll probably need to leave early
[02:01] <jblack> me
[02:01] <jamesh> me
[02:01] <bradb> me
[02:01] <SteveA> mpt_: /msg me 3 sentences, activity report status, any points to raise
[02:01] <mpt_> yep
[02:01] <SteveA> carlos: ?
[02:01] <SteveA> jordi: ?
[02:01] <SteveA> mpool? (lifeless...)
[02:02] <lifeless> SteveA: I've pung
[02:02] <SteveA> ta
[02:02] <SteveA> cprov: ?
[02:02] <SteveA> hi gustavo
[02:02] <SteveA> salgado: what about gnueman and diogo?
[02:03] <SteveA> hi kiko
[02:03] <kiko> SteveA, they're still not aware of the 9am meeting I believe
[02:03] <kiko> but should be around shortly
[02:03] <niemeyer> Hello
[02:03] <cprov> sorry, found another wireless network
[02:03] <kiko> we'll set them up for next week
[02:03] <kiko> hello hello
[02:03] <niemeyer> I have some network issues here..
[02:03] <SteveA> stub: ?
[02:03] <mpt_> gneuman's just arrived
[02:03] <mpt_> and there's diogo
[02:03] <kiko> and so have diogo
[02:03] <SteveA> hi matsubara 
[02:03] <stub> yo
[02:03] <kiko> heya stub 
[02:03] <matsubara> hi steve
[02:04] <SteveA> okay, let's go.  /msg me other items for the agenda, which so far is...
[02:04] <mpool> oh
[02:04] <mpool> not -meeting
[02:04] <SteveA> == Agenda ==
[02:04] <SteveA>  - roll call
[02:04] <SteveA>  - agenda
[02:04] <SteveA>  - activity reports
[02:04] <SteveA>  - production and staging (stub)
[02:04] <SteveA>  - ddaa's item (ddaa)
[02:04] <SteveA>  - launchpad breezy archive (kinnison)
[02:04] <SteveA>  - test suite spew (stevea)
[02:04] <SteveA>  - three sentences
[02:04] <SteveA> 
[02:04] <SteveA> activity reports: who's in with the in crowd?
[02:04] <mpool> me
[02:04] <lifeless> IN
[02:05] <BjornT> me
[02:05] <spiv> I missed Friday, but otherwise I'm IN.
[02:05] <stub> Except for monday, which is sitting in gtimelog waiting for me to extract it
[02:05] <cprov> in
[02:05] <ddaa> hey
[02:05] <jblack> in
[02:05] <ddaa> hu, I mean uptodate
[02:05] <kiko> I am up to date
[02:05] <mpt_> up to yesterday
[02:06] <SteveA> jamesh: so, you and me need to improve
[02:06] <SteveA> jamesh: can you send one for today?
[02:06] <jamesh> okay
[02:06] <kiko> just keep gtimelog open and keep scribbling in it
[02:06] <SteveA>  - production and staging (stub)
[02:06] <SteveA> ta
[02:06] <kiko> it's easy
[02:07] <SteveA> stub: status and updates to production and staging please
[02:07] <stub> production and staging are locking up. I've tracked down the dud pages - there are some queries that look like they simply will not work for some of our more verbose translators.
[02:07] <stub> Also the 'kill queries that take too long' code is not being activated on staging or production for some reason I havn't had a chance to look into yet.
[02:08] <stub> So production has been locking up regularly.
[02:08] <SteveA> also also... pound doesn't seem to be doing its balancing act
[02:08] <kiko> yesterday evening was a crackup
[02:08] <stub> Yes - which doesn't help either.
[02:08] <stub> I need elmo to look into that since I can't see the Pound logs.
[02:09] <SteveA> hi sivang 
[02:09] <sivang> hey SteveA  :)
[02:09] <carlos> SteveA, I'm up to date with activity reports
[02:09] <SteveA> so, what's the plan stub?
[02:09] <SteveA> can the queries be optimized?
[02:09] <stub> Otherwise, staging is back onto daily syncs and the whitespace migration script running right now (over a days runtime to go)
[02:10] <carlos> kiko, finally!
[02:10] <stub> SteveA: I don't think so. There are just too many rows involved for some people - creating special indexes and forcing postgresql to use them isn't helping. There are a few tricks to try, but I don't hold out hope.
[02:11] <kiko> stub, do you have a list of problematic pages?
[02:11] <stub> So we might need to remove some stuff from the dud pages as simply can't be done.
[02:11] <kiko> also, jamesh' request time-outer may help
[02:11] <SteveA> kiko: it isn't be activated in production / staging
[02:11] <SteveA> kiko: for unknown reasons
[02:11] <stub> kiko: I have 1 confirmed, and can guess a handful of others that will exhibit the same behavior.
[02:12] <kiko> SteveA, isn't be?
[02:12] <stub> But I' not going to mention them in a public channel since accessing them currently locks launchpad.
[02:12] <kiko> stub, file the bugs and I'll look at them
[02:12] <kiko> might just disable bits and pieces
[02:12] <SteveA> kiko: the timeout stuff isn't being activated in production/staging
[02:12] <kiko> stub, can you hold onto the production cutoff till tomorrow morning?
[02:12] <jamesh> kiko: it needs publication changes to properly time out multiple queries
[02:12] <SteveA> kiko: for unknown reasons
[02:12] <SteveA> jamesh: i made the publication changes
[02:12] <kiko> wb niemeyer 
[02:12] <stub> sure - I can tag production whenever.
[02:12] <SteveA> jamesh: works  on dev boxes
[02:13] <jamesh> kiko: cool
[02:13] <kiko> stub, I would like some shipit changes and carlos' code to land
[02:13] <carlos> kiko, but I need first the review...
[02:13] <niemeyer> Hi again.. hope to stay here for some time now.
[02:14] <SteveA> okay... any other production / staging issues?
[02:14] <stub> Isn't that enough?
[02:14] <kiko> yeah
[02:14] <lifeless> what issues would you like ?
[02:15] <SteveA> i'll be doing the pound-failover-socket-shutdown stuff this afternoon
[02:15] <SteveA> so it would be nice to make pound actually failover
[02:15] <SteveA>  - ddaa's item (ddaa)
[02:15] <SteveA> ddaa: speak!
[02:15] <ddaa> I hear a lot of grumbling about baz. I am under the impression the biggest problems fall in two broad categories:
[02:15] <ddaa> 1. way too slow: at this point you _need_ to use hardlinked trees to get any decent performance. You _should_ use fl-cow with hardlinked trees to avoid risking corrupting your revision library. Ask lifeless for details about fl-cow. Ask any bazaar guy for details about hardlinked trees.
[02:15] <stub> SteveA: are you able to chace the 'why pound is no longer load balancing' stuff with elmo or Znarl today?
[02:15] <dilys> Merge to rocketfuel@canonical.com/launchpad--production--1.33: Cherry pick FixMaloneFrontPageHack into production 1.33 (patch-7: steve.alexander@canonical.com)
[02:15] <SteveA> stub: sure willdo
[02:15] <ddaa> 2. way too many conflicts: if you think you have chronically more conflicts than you should have, it's probably because you have a workflow problem. And the pain will persist regardless of the tool. We want to make a FewerBazConflicts BOF to diagnose the problems and propose fixes.
[02:15] <Kinnison> ddaa: fl-cow no worky on breezy (currently)
[02:16] <ddaa> But for that we need direct participation from the boys who have those problems. My perception might be wrong, so I want know who will attend this BOF to provide input before having it scheduled. Who will want to attend and provide input to a FewerBazConflicts BOF?
[02:16] <ddaa> lifeless: fl-cow needs fixing on breezy apparently
[02:16] <lifeless> ddaa: I plan to have a peek this weekend
[02:16] <lifeless> its probably a libc issue
[02:17] <ddaa> So, who think he has more conflicts than he should and will provide input to this BOF?
[02:17] <SteveA> i have the correct number of conflicts
[02:17] <mpool> ddaa: tree shape conflicts or text conflicts?
[02:17] <SteveA> although i did have one odd conflict the other day
[02:18] <SteveA> which on reflection wasn't so odd -- text moved from one file to another
[02:18] <cprov> ddaa: I'm in, even fl-cow helps me, still in hoary and would be glad to expose the problems I have in the nasty "builddUI" branch
[02:18] <ddaa> anyboby else thinks this BOF is worthwhile?
[02:18] <ddaa> ... apparently no
[02:18] <ddaa> My perception must have been incorrect.
[02:18] <mpool> i'd be interested
[02:19] <mpool> at least as far as things to avoid
[02:19] <ddaa> Kinnison: we cannot do it unless we have concrete input.
[02:19] <Kinnison> ddaa: hence I was quiet
[02:19] <SteveA> so, i propose...
[02:19] <stub> I'm happy with conflict stuff. I get them when I expect. 
[02:19] <Kinnison> ddaa: I'm interested but cannot provide concrete input yet
[02:19] <SteveA> that we all take care to notice odd conflicts over the next couple of weeks
[02:19] <cprov> ddaa: I'm sure they were my mistake somepoint, just want to figure out where ...
[02:19] <SteveA> and bring such to the attention of ddaa
[02:20] <SteveA> agreed, ddaa?
[02:20] <ddaa> I'm mostly thinking about "1234 conflicts in this tree" cases.
[02:20] <ddaa> merely odd conflicts can be caused by mere diff3 suckiness.
[02:20] <ddaa> move
[02:20] <mpool> ok
[02:20] <SteveA>  - new trainees (kiko)
[02:20] <kiko> so we have two trainees starting this week
[02:20] <kiko> they are matsubara and gneuman 
[02:21] <kiko> you may have noticed bugspam generated by them these past days
[02:21] <matsubara> hello all
[02:21] <kiko> gneuman is being trained for a QA and fixer position
[02:21] <carlos> hi
[02:21] <kiko> matsubara for a full-time developer position
[02:21] <cprov> matsubara: hi
[02:21] <stub> fixer? official Canonical dealer or something?
[02:22] <kiko> stub, precisely. we've got their local working environment set up, using an rsynced rocketfuel for now
[02:22] <kiko> lifeless, just file the bug :-P
[02:22] <lifeless> kiko: ppfft
[02:22] <sivang> LOL
[02:22] <kiko> so help them out, and nudge them towards a greater understanding of our applications
[02:23] <kiko> that is all from me
[02:23] <SteveA> welcome matsubara and gneuman 
[02:23] <SteveA>  - who is using breezy? (stevea)
[02:23] <mpool> welcome
[02:23] <SteveA> all using breezy, say "aye!"
[02:23] <mpool> nay
[02:23] <spiv> aye!
[02:23] <jamesh> aye
[02:23] <sivang> aye!
[02:23] <niemeyer> aye!
[02:23] <mpt> aye, but not at work
[02:23] <jamesh> (upgraded my main machine today)
[02:23] <ddaa> nope
[02:23] <jblack> aye, on 10 systems, nay on 1
[02:24] <lifeless> aye
[02:24] <mpt> bradb: "noe"
[02:24] <SteveA> BjornT: is someone on the distro team helping you?
[02:24] <gneuman> hwllo SteveA 
[02:24] <mpool> i have it installed, but something broke and i gave up
[02:24] <mpool> will try again
[02:24] <kiko> stub, btw, you haven't disabled the https proxy yet
[02:24] <SteveA> carlos: get support from the distro team
[02:24] <stub> did too! (?)
[02:24] <SteveA> mpool: distro team will help you
[02:24] <carlos> SteveA, I'm scared with the launchpad development tools
[02:24] <carlos> the distro itself is easy
[02:24] <BjornT> SteveA: well, two bugs are filed about it. i haven't had time to look into it more, but i might try to get some help this weekend
[02:25] <SteveA> BjornT: what are the bugs?
[02:25] <jamesh> took a little time getting postgres-8.0 set up after upgrading, but I didn't run into any big issues
[02:25] <SteveA> the distro team are very keen on us using breezy early
[02:25] <SteveA> but the other side of that is that they can help us out to get it running
[02:25] <carlos> ok
[02:25] <SteveA> jamesh: we don't need postgres8 though, surely
[02:26] <BjornT> SteveA: basically the same bug, neither grub nor lilo can be installed. they don't like my partition table
[02:26] <stub> kiko: I updated one of the cronjobs,  but missed the important one ;)
[02:26] <SteveA> BjornT: get me the number, and i'll hassle the distro team
[02:26] <lifeless> SteveA: thats whats in breezy
[02:26] <kiko> stub, you're terrible :)
[02:26] <SteveA> everyone who hasn't upgraded to breezy, please do so soon
[02:26] <kiko> SteveA, this weekend for the async diskless and me
[02:27] <SteveA>  - launchpad breezy archive (kinnison)
[02:27] <spiv> lifeless: I'm using postgres 7.4 from breezy just fine...
[02:27] <Kinnison> Right
[02:27] <BjornT> SteveA: #15743 and #7506
[02:27] <Kinnison> as per my email to launchpad@ last night/this morning, I have managed to get breezy imported and re-published from launchpad well enough to upgrade my laptop to breezy
[02:28] <Kinnison> it's not signed, so you have to put up with apt whinging
[02:28] <Kinnison> the deb line you'll need is on http://wiki.launchpad.ubuntu.com/BreezyDogfooding
[02:28] <sivang> Kinnison: is it polling baz archive of all of the distro and repackas for use?
[02:28] <Kinnison> sorry, https://wiki.launchpad.canonical.com/BreezyDogfooding
[02:29] <Kinnison> This is currently a direct import of the main sources/binaries
[02:29] <Kinnison> and it's only source+i386
[02:29] <carlos> Kinnison, are those packages built with launchpad's tools?
[02:29] <SteveA> sivang: this is breezy, but built using the brand new build infrastructure
[02:29] <carlos> cool
[02:29] <sivang> SteveA, Kinnison : wow :)
[02:30] <SteveA> many many hours of work from Kinnison and cprov
[02:30] <Kinnison> These binaries are not rebuilt
[02:30] <Kinnison> There is another archive which contains binaries being rebuilt
[02:30] <Kinnison> but we've not got that automated yet
[02:30] <SteveA> ok
[02:30] <Kinnison> the rebuilt archive is called foodix instead of ubuntu
[02:30] <SteveA> this is important dogfooding stuff for launchpad and ubuntu
[02:30] <Kinnison> and the distrorelease is called breezyfood
[02:31] <SteveA> those who haven't upgraded to breezy, seriously consider using this to upgrade
[02:31] <Kinnison> cprov will mail the list once breezyfood is rebuilding stuff
[02:31] <SteveA> gotta move on, still two more items
[02:31] <SteveA>  - announcement (mpool)
[02:31] <kiko> (should I hold off upgrading until you guys are building, Kinnison, cprov?)
[02:31] <cprov> yes, we still working on chroot for breezyfood
[02:32] <sivang> Kinnison: would it be worhwhile if I tried that upgrade just for sake of testing launchpad build infra ?
[02:32] <lifeless> Kinnison: random thought, have you done a diff against archive.ubuntu .. ?
[02:32] <SteveA> mpool: ?
[02:33] <Kinnison> sivang: If you would, that would be helpful
[02:33] <cprov> kiko: I'll try the shadow today in one desktop 
[02:33] <Kinnison> lifeless: there are a few small changes needed to the publisher before we can do that
[02:33] <sivang> Kinnison: let's discuss the details afterwards.
[02:33] <Kinnison> sivang: sure
[02:33] <Kinnison> sivang: jabber me to chat
[02:33] <SteveA> thanks sivang.
[02:33] <sivang> Kinnison: cool
[02:34] <kiko> rock
[02:34] <lifeless> mpool: ping
[02:34] <SteveA> as mpool seems to be afk, 
[02:34] <SteveA>  - test suite spew (stevea)
[02:34] <mpool> oh, i just wanted to announce that bzr's weave-based storage is now working
[02:34] <mpool> no just typing
[02:34] <mpool> i'm very pleased
[02:34] <mpool> it took longer than i wanted
[02:34] <mpool> software being what it is
[02:34] <SteveA> that's great news
[02:34] <mpool> but it considerably rocks: smaller, therefore should be faster on teh network, especially with robert's stuff
[02:34] <mpool> quite fast
[02:35] <lifeless> When this hits mainline, I'll be redoing the demo bzr launchpad on chinstrap
[02:35] <niemeyer> Great work Martin
[02:35] <mpool> and a good position for better annotate/merge and so on
[02:35] <lifeless> which will make that time(<lifetime) to download ;0
[02:35] <mpool> yep
[02:35] <kiko> mpool, a branch of bzr itself appeared to take a long time to run
[02:35] <mpool> yep 
[02:35] <mpool> that's putting it politely :)
[02:35] <kiko> will that improve now?
[02:35] <lifeless> kiko: YES
[02:35] <mpool> these two things will make it much better
[02:36] <niemeyer> mpool: When do you plan to merge that with mainline?
[02:36] <kiko> rock and roll
[02:36] <lifeless> kiko: I'm guessing ~1 minute once both land.
[02:36] <kiko> wow
[02:36] <kiko> fun
[02:36] <lifeless> kiko: but I may be overly optimistic.
[02:36] <kiko> (lifeless, can you approve matsubara and gneuman's list requests, btw?)
[02:36] <mpool> i plan to make a 0.0.9 bugfix, then merge from mailine into the weave branch, then declare that mainline
[02:36] <lifeless> kiko: what requests ?
[02:36] <spiv> lifeless: Don't forget to factor in Brazilian bandwidth in your calculations ;)
[02:36] <kiko> lifeless, don't you read email?
[02:36] <kiko> lifeless, subscriptions to arch-commits
[02:36] <lifeless> kiko: sure I do
[02:36] <mpool> in other words will start reconciling tomorrow
[02:37] <kiko> spiv, very important remark
[02:37] <mpool> spiv: it's pretty shitty for us too, since it's going all the way to the uk
[02:37] <niemeyer> mpool: Very nice!
[02:37] <mpool> anyhow
[02:37] <mpool> dumb server support is definitely a popular feature
[02:37] <mpool> but we have to do the work to make it tolerably fast
[02:37] <mpool> that's all from me
[02:37] <SteveA> cool
[02:37] <SteveA>  - test suite spew (stevea)
[02:38] <mpool> bring your pie-eating shoes, kiko :)
[02:38] <SteveA> when i ran the tests recently, i got a lot of curous stuff on stderr
[02:38] <SteveA> who added it?  what's it for?
[02:38] <mpt> ok, time for me to go
[02:38] <kiko> mpool, even when I lose, I win
[02:38] <mpt> SteveA has my sentences
[02:38] <SteveA> 18:40:17 ERROR   http://localhost:9000/codeofconduct/donkey/index.html
[02:38] <SteveA>  -> http://localhost:58000/35/35/jbhAtOEr8s2wTQ7Uyceb8trZ4UH.txt
[02:38] <SteveA> i don't have mpt's sentences
[02:38] <lifeless> thats not my donkey
[02:38] <SteveA> because he didn't authenticate with NickServ
[02:38] <kiko> SteveA, don't worry, he'll be back on
[02:38] <kiko> lifeless?
[02:39] <SteveA> 18:41:44 WARNING PO file header entry has no content-type field
[02:39] <SteveA> there's another example
[02:39] <SteveA> 18:43:01 WARNING Disabling: No CHROOT found for pocket 'Release'
[02:39] <spiv> SteveA: /msg nickserv set unfiltered on
[02:39] <SteveA> and another
[02:39] <carlos> SteveA, I saw them also, but I didn't add them ....
[02:39] <carlos> (the ones related to Rosetta)
[02:40] <SteveA> so, who's been doing stuff with logging recently?
[02:40] <stub> me
[02:40] <kiko> not me
[02:40] <SteveA> i want the test runner not to spew stuff when tests run okay
[02:40] <kiko> SteveA, even if there are warnings?
[02:40] <SteveA> and to spew minimally when something goes wrong
[02:40] <kiko> these are warnings that are pertinent, you know..
[02:40] <SteveA> warnings are collected and categorized at the end of the test run
[02:40] <lifeless> kiko:  if the warning matters the tesst should be grabbing it and testing for it
[02:40] <SteveA> these are not all warnings that are pertinent
[02:40] <lifeless> kiko: theres a log collector you can use to do that.
[02:40] <kiko> lifeless, the point is that the test isn't
[02:41] <SteveA> these are spew about things like 404 pages
[02:41] <jamesh> possibly the tests in question should be using a dummy logger
[02:41] <kiko> lifeless, so the test should be fixed
[02:41] <SteveA> that are being explicitly tested for
[02:41] <lifeless> kiko: and if it doesn't matter, it should be hidden.
[02:41] <kiko> but unless we see the warnings they are hidden
[02:41] <jamesh> if the intent is to test that a particular input generates a log message
[02:41] <SteveA> the test running infrastructure collects in-process warnings and logs them
[02:41] <SteveA> i don't want random spew
[02:41] <kiko> jamesh, agreed. I agree that not all warnings are pertinent either.
[02:41] <SteveA> so, who added this?
[02:41] <kiko> but I think that hiding warnings is not a good idea
[02:42] <lifeless> but you don't mind deliberate spew? Ignore me, I'm still thinking about that poor donkey.
[02:42] <SteveA> NO SPEW!
[02:42] <SteveA> right, let's talk about this later, and work out exactly why it is spewing
[02:42] <jamesh> kiko: not hiding warnings -- collecting them and testing that the messages generated match what you expected
[02:42] <SteveA> now, it's time for the tree sentences
[02:42] <SteveA> um, three sentences
[02:42] <kiko> oh-oh
[02:42] <SteveA> please go ahead
[02:42] <lifeless> DONE: asyncification of bzr, MUCH performance and ui love resulted.
[02:42] <lifeless> TODO: gpg, face-pie avoidance
[02:42] <lifeless> BLOCKED: de nada
[02:42] <mpool> "i think that i will never see"
[02:42] <ddaa> DONE: some love to python and samba imports
[02:42] <ddaa> TODO: python and samba imports, importd-archivelocation, land london sprint, upgrade to breezy
[02:42] <ddaa> BLOCKED: no
[02:42] <salgado> DONE: Fixed a lot of small bugs, finally merged basic-voting--1, more work on ShipItNG
[02:42] <salgado> TODO: Get shipit exports working, and some other things Marilize requested
[02:42] <salgado> BLOCKED: No
[02:43] <carlos> DONE: Holidays, languagepacks, poimports
[02:43] <spiv> DONE: Reviews, Librarian and authserver updates, helped Robert with his bzr/twisted stuff.
[02:43] <spiv> TODO: Reviews, AuthserverCaching, Supermirror SFTP work.
[02:43] <spiv> BLOCKED: No.
[02:43] <Kinnison> DONE: buildd work, publisher work, dominator work, gina work. Got laptop upgraded to breezy from dogfood re-published breezy archive *wooyay*
[02:43] <Kinnison> TODO: support bin-nmu in gina, fix SP refcounting in dominator, get dogfood updating on cron, actually get down to working on uploader again, and sort out suspend again on laptop :-)
[02:43] <niemeyer> DONE: 2 more days of London sprint, binary deltas for bzr weaves, executable bit tracking on bzr, more research.
[02:43] <niemeyer> TODO: Continue work on bzr.
[02:43] <niemeyer> BLOCKED: Nope
[02:43] <jblack> DONE: advocacy work
[02:43] <mpool> DONE: weave storage and conversion working!
[02:43] <cprov> DONE: buildUI and build-scoring merge in RF
[02:43] <cprov> TODO: missed UI pages, scoring according build-deps, foodix/breezyfood building
[02:43] <cprov> BLOCKED: myself ;(
[02:43] <jblack> FUTURE: advocacy work
[02:43] <Kinnison> BLOCKED: Slowed in general by mawson's speed, hence long days recently. Otherwise able to plod along.
[02:43] <kiko> DONE: set up trainees, work out shipit, random QA, lots of other things I won't remember
[02:43] <jblack> BLOCKED: None
[02:43] <bradb> DONE: Landed Malone URL changes. Landed bugmail headers Reply-To/From/Sender fix. Landed various other bugfixes.
[02:43] <carlos> TODO: Final languagepacks checks and kill all errors from poimport script
[02:43] <mpt> DONE: landed deactionizing, LaunchpadMenus work, gnome-screensaver design, bug fixes
[02:43] <kiko> FUTURE: try out the spec tracker, more of this
[02:43] <bradb> TODO: Make sure Malone menus are working superbly. Feedback/error/success message desuckification. Beg that Launchpad not lock up because that just sucks.
[02:43] <BjornT> DONE: converted bug search listings to new format. started to spec out email interface for the ticket tracker. some work on pre-defined bug reports. reviews.
[02:43] <bradb> BLOCKED: No.
[02:43] <SteveA> DONE: menus, availability work, reviews
[02:43] <SteveA> TODO: menus, availability work, reviews
[02:43] <SteveA> BLOCKED: no
[02:43] <BjornT> TODO: pre-defined bug reports. outgoing emails for the ticket tracker.  fix email wrapping bug.
[02:43] <kiko> BLOCKED: no
[02:43] <BjornT> BLOCKED: no
[02:43] <carlos> BLOCKED: None
[02:43] <mpool> TODO: merge mainline into it, take outstanding patches, cut over
[02:43] <mpt> TODO: polish up LaunchpadMenus, finish support tracker cleanup, more bug fixes
[02:43] <mpool> BLCOKED: no
[02:43] <mpt> BLOCKED: no
[02:43] <kiko> SteveA, sorry, fucked up FUTURE with TODO for some reason :-(
[02:44] <Kinnison> SteveA: I have to go now :-(
[02:44] <SteveA> okay
[02:44] <jblack> Whoops
[02:44] <SteveA> cheers
[02:44] <jamesh> DONE: reviews, request timeout branch, upgrade to breezy
[02:44] <jamesh> TODO: scheduler thing, calendar code, reviews
[02:44] <jamesh> BLOCKED: no
[02:44] <jblack> TODO: advocacy work
[02:44] <Kinnison> thanks steve
[02:44] <sivang> bye Kinnison 
[02:44] <SteveA> no one is blocked then?
[02:44] <SteveA> anyone lack reasonable TODO items?
[02:44] <SteveA> (kiko and i can check through later)
[02:44] <stub> DONE: BrowserNotificationMessages
[02:44] <stub> TODO: Polish off BrowserNotificationMessages, with example page to give to mpt to tart up
[02:44] <stub> BLOCKED: Production errors
[02:45] <SteveA> stub: i'll talk with the admins about pound
[02:45] <SteveA> okay, that's it.
[02:45] <SteveA> MEETING ENDS
[02:45] <SteveA> cheers folks
[02:45] <SteveA> oh, same time next week...
[02:45] <lifeless> k
[02:45] <carlos> ok
[02:45] <SteveA> um
[02:45] <ddaa> mpool: basically the things to avoid are departing too much from star-topology
[02:46] <kiko> thanks SteveA 
[02:46] <kiko> rock rock
[02:46] <ddaa> lifeless: I need to talk to you about PatchedFile
[02:46] <lifeless> ok
[02:46] <lifeless> talk
[02:46] <ddaa> lifeless: how important is it to use diff/patch?
[02:46] <ddaa> that's the source of many problems
[02:46] <lifeless> I don't care a rats
[02:47] <lifeless> it just seemed the only sane way to get stuff out of svn
[02:47] <ddaa> lifeless: so it's okay to do a svn cat instead and overwrite contents, all the time?
[02:47] <lifeless> sure
[02:47] <lifeless> I guess.
[02:48] <ddaa> I think it would work. It just annoys me I put in some much plumbing on the assumption we avoided that.
[02:48] <lifeless> ok, sounds good
[02:48] <ddaa> lifeless: also, the #svn guy has been talking was quite critic on the not using the taylor approach.
[02:48] <lifeless> mmm
[02:49] <ddaa> I do think that would have saved a LOT of trouble with SVN.
[02:49] <lifeless> 'tailor approach' ? sequential copy over a whole tree ?
[02:49] <ddaa> basically, yes
[02:49] <ddaa> and letting svn do the update voodoo
[02:49] <lifeless> very hard to tell what is actually happening if you do that
[02:50] <ddaa> bah... I'm too tired to argue about it again.
[02:50] <ddaa> I really hate cscvs, should just focus on making it work somehow and be done with it :(
[02:51] <kiko> ddaa, AIUI cscvs will be around forever
[02:51] <lifeless> If we want to do it different for every different system, I guess we can
[02:51] <ddaa> "focus on making it work somehow, be done with it", rinse, repeat
[02:52] <ddaa> I need fud.
[02:52] <lifeless> its come a hell of a long way since we started, is -much- cleaner.
[02:52] <lifeless> I think its just a hard problem to do well.
[02:52] <lifeless> you do know that tailor's svn thing is bust right ?
[02:52] <ddaa> One thing I do hate about it, is that every time I go to a foreign chan to ask questions, people thing I'm stupid and doing stupidly wrong things without realising it.
[02:53] <ddaa> While the situation I'm in is entirely not my doing.
[02:53] <kiko> lifeless, I think you're right that it's a hard problem to do well. very hard.
[02:53] <ddaa> And that pisses the hell out of me.
[02:53] <lifeless> ddaa: sure, I get that too. the svn guys are suggesting something that cannot work for rcs or cvs. How much hair would it add to do different repos fundamentally differently ?
[02:54] <lifeless> so, they have their opinions, and where they add useful info, lets put that into our future plans list
[02:54] <lifeless> but don't let them get you down - they ain't accomplished over 500 conversions
[02:55] <ddaa> whatever, their stuff does not go to eat gigabytes of memory while converting gcc
[02:55] <lifeless> it also gets it wrong
[02:55] <ddaa> so you say
[02:56] <lifeless> so gcc is a heavy user of aliases last I checked. and cvs2svn doesn't do them at all
[02:56] <ddaa> and our infrastructure just does not support configs
[02:56] <ddaa> it's just filters stuff out
[02:57] <lifeless> which is a marked improvement imo. anyway, you sound plain frustrated.
[02:57] <ddaa> not even mentioning the dozens production failures I do not even have the time to start looking at
[02:57] <ddaa> I am.
[02:58] <lifeless> I think that that is reasonable. You have a challenging role.
[02:58] <ddaa> I really need lunch, I'm starting to feel really bad.
[02:58] <lifeless> Please remember though, that the folk you are talking with, particularly cvs2svn guys are solving a different problem
[02:58] <lifeless> enjoy lunch.
[02:59] <lifeless> for when you get back, the problem they are solving is one shot, convert and dump. we're not, we're solving long term syncronisation. seriously different stuff.
[03:37] <cprov> lifeless:   mako@deseo.yukidoke.org still in activity@list.ubuntu.com and delivery is failing, could you remove this address from the maillisting ? 
[03:51] <Treenaks> brokenness is known, right?
[03:54] <kiko> Treenaks, server down again?
[03:55] <Treenaks> kiko: launchpad access has been flaky all day
[03:55] <Treenaks> kiko: sometimes it works, sometimes it doesn't
[03:55] <Treenaks> kiko: + all the timeouts I had yesterday _if_ it works
[03:56] <kiko> Treenaks, we've been having problems with database performance, unfortunately
[03:56] <kiko> stub's looking into it
[03:56] <Treenaks> kiko: hmm.. ok
[04:00] <salgado> SteveA, around?
[04:01] <Kinnison> kiko: fancy doing a quick review of two small publisher fixes?
[04:01] <Kinnison> one is trivial really, the other is large because I moved a block of code as well as fixing it
[04:01] <kiko> Kinnison, hmmm, I'm not a good pick today, but I'll do it if nobody else can
[04:02] <Kinnison> kiko: okay, I'll try someone else first
[04:02] <Kinnison> kiko: https://chinstrap.ubuntu.com/~dsilvers/paste/fileFX53yA.html is the larger one
[04:05] <salgado> stub, ping
[04:05] <stub> salgado: pong
[04:06] <void_main_void> opa...
[04:07] <salgado> stub, I just saw that steve cherrypicked one of /his/ branches into production. and I want to know what's our policy here; should I always branch from production and ask you to cherrypick my branch or this is only a fallback when it's impossible to cherrypick the fix from rocketfuel's --devel?
[04:09] <stub> salgado: Normally stuff can be cherry picked directly from launchpad--devel--0. We only need to bother with branches of the production branch if the changes cannot be applied due to conflicts or similar.
[04:12] <salgado> stub, this is what I thought. anyway, this merge that dilys will announce soon (hopefully) should be cherrypickable. should I ping you or are you going to sleep soon?
[04:13] <stub> Try pinging me
[04:22] <SteveA> salgado: ping
[04:29] <salgado> SteveA, nm. already sorted with stub
[04:29] <SteveA> ok
[04:42] <kiko> stub?
[04:42] <carlos> stub, spiv around?
[04:42] <stub> eh?
[04:42] <spiv> carlos: Hmm?
[04:43] <carlos> spiv, stub are we mirroring librarian into staging?
[04:44] <stub> staging librarian should be configured to pull stuff from the production librarian that it doesn't have locally
[04:45] <carlos> stub, cool, thanks
[04:45] <carlos> stub, and do I have access to staging librarian from mawson?
[04:45] <carlos> stub, could I get it?
[04:45] <stub> staging is world accessible
[04:45] <stub> librarian.staging.ubuntu.com
[04:46] <carlos> ok, thanks
[04:46] <salgado> stub, here it comes
[04:47] <dilys> Merge to rocketfuel@canonical.com/launchpad--devel--0: [trivial]  Highlight high priority orders and display the order id in the list of orders. (patch-2465: guilherme.salgado@canonical.com)
[04:48] <stub> SteveA: I've got the statement timeout code running on staging (there was a zcml slug that was overriding the launchpad da with the standard psycopgda)
[04:51] <kiko> rock stub 
[04:52] <SteveA> stub: cool
[04:52] <SteveA> i'm hooking up the "shutdown the socket" code
[04:53] <stub> And on production now. So db statements taking longer than 4 seconds will die and the page will show a system error. Which will make the rest of the system stable.
[04:53] <SteveA> salgado: suggest you write a braindump spec on different authentication redirects
[04:53] <SteveA> stub: have you seen it happen on production? ;-)
[04:53] <stub> Yes
[04:54] <SteveA> what do you think is a good value for the max pending tasks?
[04:54] <stub> https://launchpad.net/errors/showEntry.html?id=1127400721.970.643347247388
[04:54] <stub> I'd got for 20 (5 per thread)
[04:54] <SteveA> ok
[04:56] <salgado> stub, have you seen my pseudo-request for a cherrypick review?
[04:56] <kiko> SteveA, I think we need this ASAP for shipit, TBH
[04:56] <salgado> s/review//
[04:59] <stub> eh?
[04:59] <SteveA> kiko: should be much improved already.
[04:59] <salgado> stub, ok, you haven't. patch-2465 is the one I need cherrypicked. can you do that? :)
[04:59] <stub> yup
[04:59] <SteveA> we need to get pound sorted properly before my current work will have any effect anyway
[05:00] <kiko> SteveA, I mean the redurect
[05:00] <kiko> SteveA, the redirect :)
[05:00] <SteveA> why can't salgado just do a shipit-specific registration template?
[05:06] <salgado> SteveA, IIRC I decided for not doing so because the view we use for the +login page is not easily extandable, and I need to extend it
[05:06] <salgado> so, that would mean a considerable amount of work
[05:06] <SteveA> how do you mean "not easily extendable" ?
[05:07] <SteveA> write me something explaining exactly what you need, and i'll help you do it
[05:07] <SteveA> (not on irc though)
[05:07] <salgado> not extendable at all. I'd need to do a lot of work on it
[05:07] <SteveA> (so i can do it asyncronously)
[05:07] <SteveA> write me your requirements
[05:07] <SteveA> and i'll sort it ou
[05:07] <SteveA> t
[05:08] <salgado> okay, that sounds good
[05:10] <carlos> stub, before you leave, could you tell me when is supposed to finish the whitespace fix script?
[05:11] <Kinnison> SteveA: fancy casting your eye over https://chinstrap.ubuntu.com/~dsilvers/paste/fileFX53yA.html ?
[05:12] <stub> carlos: 38.3778 done (1750000 of 4559926). eta 20:33:20.007313
[05:13] <carlos> stub, so tomorrow noon it should be done.
[05:13] <carlos> stub, I will need a staging db refresh after it finish
[05:13] <carlos> stub, is it possible?
[05:13] <stub> carlos: Sure.
[05:13] <carlos> stub, cool thansk
[05:17] <stub> salgado: I just fast tracked that cherry pick - its running in production now
[05:18] <kiko> rock!
[05:31] <carlos> jblack, kiko Jordi just called me
[05:31] <kiko> and?
[05:31] <carlos> he said that his network is down because they are moving to another office and he didn't know it would happen
[05:32] <carlos> kiko, jblack he will mail us tonight to set another meeting next week
[05:33] <dilys> Merge to rocketfuel@canonical.com/launchpad--devel--0: [trivial]  Make staging and production use Launchpad variant of psycopgda (patch-2466: stuart.bishop@canonical.com)
[05:36] <SteveA> stub: how should i do logging in launchpad code?
[05:37] <stub> In the webapp? Dunno. Hopefully Z3 is setup so you just grab a Python logger and log to it and the launchpad.conf configuration sorts out where it goes.
[05:52] <Kinnison> can pqm be asked to merge two branches into RF at the same time?
[05:53] <SteveA> no
[05:53] <SteveA> you need to ask lifeless to arrange such a thing
[05:53] <Kinnison> right
[05:59] <SteveA> stub: argh
[05:59] <SteveA> stub: ZServer is kinda wank.  Many of the adjustments just don't work.
[05:59] <SteveA> they've never been used by anyone.
[06:00] <dilys> Merge to rocketfuel@canonical.com/launchpad--production--1.33: Cherry pick production config fix (patch-8: stuart.bishop@canonical.com)
[06:00] <stub> SteveA: Maybe we should leave it until twisted support lands in the Z3 trunk. With the timeout stuff we should be fine.
[06:01] <SteveA> nah, i've done it now
[06:01] <SteveA> would like to try it on staging, or somewere
[06:01] <SteveA> works for me, with random manual banging on it
[06:01] <SteveA> logs a message to the app log when it shuts down or reopens the socket
[06:02] <SteveA> there's a request in RT for looking at the pound config and getting the logs
[06:03] <stub> heh.... rate of new users has gone up significantly since shipit rolled out. So far 60,000 Breezy CDs ordered in 7000 requests.
[06:03] <\sh> shipit is open to order breezy cds?
[06:04] <\sh> hmm...i should order some of them ;) to spread ubuntu again in our company
[06:13] <bradb> heh
[06:14] <Kinnison> aah, it's my own fault :-)
[06:27] <dilys> Merge to rocketfuel@canonical.com/launchpad--production--1.33: Cherry pick patch-2465 into production 1.33 (patch-9: guilherme.salgado@canonical.com, rocketfuel@canonical.com)
[06:38] <Kinnison> ddaa: I've sent you some conflict amusement stuff
[06:38] <ddaa> okay, I'll try to look at it tomorrow
[06:48] <zyga> this is probably a FAQ but what is this and why it doesn't work: https://launchpad.net/people/zkrynicki/+edithackergotchi
[06:57] <kiko> DO IT
[07:00] <zyga> SteveA: what is hackergotchi anyway??
[07:00] <zyga> it sounds like tamagotchi
[07:00] <zyga> 'breed your own hacker'
[07:00] <SteveA> see planet.ubuntu.com
[07:00] <zyga> 'your hacker is houngr, throw some code at it'
[07:00] <SteveA> those floating heads are "hackergotchi"
[07:00] <zyga> ah :)
[07:01] <zyga> cool
[07:02] <zyga> hmm
[07:02] <zyga> where can I send bugreports about planget ubuntu?
[07:02] <SteveA> i don't know
[07:03] <zyga> the links are totally b0rked and pretty much indistingushable from regular text ;P
[07:31] <SteveA> hello latvians
[07:31] <Kinnison> ciao dudes
[07:32] <SteveA> Virtuall: do you ever go to vilnius?
[07:32] <Virtuall> SteveA, no, should I?
[07:32] <Virtuall> ;)
[07:33] <SteveA> depends
[07:33] <Virtuall> hello to you too
[07:33] <Virtuall> do yo go to Riga?
[07:33] <SteveA> haven't been yet
[07:33] <SteveA> it's on my todo list
[07:33] <Virtuall> :)
[07:34] <SteveA> if you're interested in launchpad, and are in vilnius sometime, there are two launchpad developers here.
[07:34] <SteveA> it would be good to get real-life feedback on your using launchpad
[07:50] <Virtuall> is your lithuanian beer good
[07:50] <Virtuall> ?
[07:50] <Virtuall> :))
[07:56] <SteveA> if it wasn't there would be some kind of national revolution
[07:58] <dilys> Merge to rocketfuel@canonical.com/launchpad--devel--0: SP refcounting and publish-distro trivial fix. r=stevea (patch-2467: daniel.silverstone@canonical.com)
[08:23] <salgado> carlos, have you seen https://launchpad.net/errors/showEntry.html?id=1127412456.020.90591756315 ?
[08:24] <bradb> BjornT: is it expected that, in a doctest, if i call:
[08:24] <bradb> login('foo')
[08:24] <bradb> the launchbag.user will be foo
[08:24] <bradb> and if, right after, i call:
[08:25] <bradb> login('bar')
[08:25] <bradb> the launchbag.user will be bar?
[08:25] <BjornT> bradb: if 'foo' and 'bar' are valid email addresses, then yes
[08:32] <BjornT> (of course that is, launchbag.user will be the user with given login)
[08:35] <bradb> BjornT: It appears to not work in that way. As an example, line 404, bugtask.txt:
[08:35] <bradb> if I do:
[08:35] <bradb>     >>> login('spiv@jabber.org')
[08:35] <bradb>     >>> getUtility(ILaunchBag).user.id
[08:35] <bradb>     >>> upstream_task.status = STATUS_ACCEPTED
[08:36] <bradb> my test raises an error, saying NoneType has no attribute 'id'
[08:36] <bradb> (this code will look different that what you'd have, because I just added that code locally.)
[08:37] <bradb> this makes me think that it's possible that login() is broken, and bugtask.txt was accidentally passing anyway
[08:37] <bradb> ACH
[08:37] <bradb> it's also possible that i'm a retard
[08:38] <BjornT> yeah, i was going to say that it doesn't handle errors well.
[08:38] <bradb> but, i tried this in other parts of the file as well, and saw a similar error. it's not yet clear if this is a bug or not. i'll let you know more in a few mins.
[08:39] <BjornT> bradb: feel free to put 'assert principal is not None, "Invalid login"' after 'principal = authutil.getPrincipalByLogin(email)' in the login function
[08:40] <bradb> right, that's a pretty serious bug, IMHO
[08:40] <bradb> i'll fix it here
[08:40] <BjornT> cool
[08:44] <salgado> what's up with all these "ProgrammingError: ERROR: canceling query due to user request ..." on production?
[08:45] <bradb> salgado: sounds like it might be stub's query timeout thing maybe?
[08:45] <salgado> oh, right. that's beeing triggered all the time
[08:45] <salgado> even on some inserts
[08:46] <salgado> this is bad, I guess
[08:48] <SteveA> we can have a different timeout for POSTs if we need that
[08:48] <SteveA> i think it is acceptable for POSTs to take longer than GETs
[08:49] <kiko> 3 seconds sounds a bit too little, though
[08:49] <SteveA> 3 seconds of processing seems okay for most GETs
[08:49] <SteveA> note that it will take longer to be actually returned to the user
[08:50] <SteveA> also...
[08:50] <SteveA> we can change the code to have a "warn" level and a "kill" level
[08:50] <kiko> we have queries that currently take 3 minutes
[08:50] <SteveA> so, requests between 3 and 6 seconds get a warning
[08:50] <SteveA> and requests 6 seconds and over get killed
[08:50] <SteveA> we don't have queries that take 3 minutes
[08:50] <SteveA> they get killed
[08:52] <kiko> we used to up to this morning!
[08:53] <kiko> SteveA, we're getting deadlocks running the test suite
[08:53] <bradb> kiko: Are you interested in doing a drive-by review of the admin-awareness patch once I'm done make linting/bazzing?
[08:53] <kiko> bradb, probably not today :-(
[08:53] <kiko> SteveA, any clue as to where to start debugging?
[08:54] <SteveA> find what tests are causing the problem
[08:54] <bradb> ok. i can trivial it, i think. it's really simple. 2 lines of security checker changes and then some test adjustments.
[08:54] <SteveA> maybe ones that run external processes?
[08:54] <kiko> SteveA, it's funny. one process is waiting to write on a pipe, and the other end of the pipe is waitpid()ing 
[08:55] <kiko> on the first process
[08:55] <SteveA> o, right
[08:55] <SteveA> thought you meant database deadlocks
[08:55] <kiko> who does our test suite run baz?
[08:55] <bradb> somebody once emailed the list about how to debug a hanging test suite. i wonder if they ever put that doc on the wiki, or if it just got swallowed up in the archive.
[08:56] <kiko> this is fresh rocketfuel though
[08:56] <kiko> it's something in the local setup
[08:57] <SteveA> kiko: probably crappy reading on the out vs error streams vs input -- often code that invokes processes will makes assumptions about that.
[08:57] <SteveA> the new subprocess should handle it better 
[08:57] <SteveA> so, suspect code that uses the old APIs
[08:58] <salgado> I think I found the problem
[08:58] <SteveA> running the suite with -vv etc. will help show the hanging tests
[08:58] <kiko> salgado?
[08:59] <salgado> no, false alarm
[08:59] <salgado> I was running with -vv and saw an error message
[08:59] <kiko> it's running test_on_merge
[09:11] <SteveA> kiko: try running a subset of tests, rather than test_on_merge
[09:12] <kiko> SteveA, yeah, I'll start looking into this :-(
[09:12] <salgado> I'm running with -vv and everything is passing up to now
[09:14] <kiko> rock
[09:23] <koke> carlos: if I deactivate a user from a team, and fill the comment field, does he receive the comment?
[09:24] <koke> there's one who has just desubscribed from the list
[09:24] <koke> unsubscribed :)
[09:24] <carlos> koke, no idea
[09:24] <carlos> salgado, ?
[09:24] <salgado> koke, no, he'll not receive anything
[09:25] <koke> hmm, but will the comment be visible or it will be just removed from the list of team members?
[09:25] <salgado> but maybe it'd be a good idea to have it
[09:26] <salgado> he'll be removed and the comment will be stored. right now we're not displaying this comment anywhere (apart from the deactivation page)
[09:33] <carlos> salgado, so only admins will see it?
[09:34] <salgado> carlos, yes
[09:34] <carlos> ok
[09:39] <kiko> salgado, and the tests?
[09:39] <salgado> still running
[09:39] <salgado> I ran them on another tree, using anthem's postgres and it already finished
[09:40] <salgado> but using the local postgres is taking ages
[09:40] <kiko> mmmkay
[09:46] <salgado> SteveA, I braindumped a little in https://wiki.launchpad.canonical.com/ShipItLogin. please let me know if something is not clear or if there's anything missing
[09:46] <SteveA> salgado: i've almost gone home -- can you mail me, and i'll look tomorrow morning
[09:47] <salgado> SteveA, sure
[10:00] <kiko> SteveA, bug queries are broken in production
[10:00] <kiko> this is horrible
[10:01] <kiko> they are getting killed prematurely
[10:01] <bradb> argh
[10:01] <kiko> works on staging
[10:03] <kiko> oh here's a fun one
[10:03] <kiko> bradb, you can no longer query for a bug # on staging
[10:03] <kiko> it doesn't redirect you anymore
[10:03] <kiko> bradb, also, I can't seem to find bug 2331 by searching..
[10:04] <bradb> BjornT: are you available for a quick code review? it's a patch to make Malone admin aware, and fix an upstream task related privacy bug (that I don't believe is currently exploitable.)
[10:04] <bradb> kiko: redirection seems to work fine here.
[10:05] <kiko> oh
[10:05] <kiko> bug 2331 doesn't exist in staging
[10:05] <bradb> in the case of 2331, it appears to simple not exist. in that case, it doesn't redirect you anywhere. instead, it shows you a 404, like it would now.
[10:05] <kiko> sorry.
[10:05] <bradb> no worries
[10:05] <kiko> it doesn't show you a 404 though
[10:06] <kiko> it redisplays the list apparently
[10:06] <bradb> i'm seeing 404's
[10:06] <kiko> on staging?
[10:06] <bradb> yeah
[10:06] <kiko> wait, I'll give you a URL
[10:06] <bradb> 2331 takes me to https://staging.ubuntu.com/malone/bugs/2331
[10:07] <kiko> https://staging.ubuntu.com/products/launchpad/+bugs
[10:07] <kiko> type in 2331
[10:07] <kiko> do search
[10:07] <kiko> see what I mean?
[10:07] <salgado> kiko, tests completed when running with test.py -vv
[10:07] <kiko> you must be talking about the malone homepage
[10:07] <kiko> salgado, race conditions are fun
[10:08] <kiko> I'm looking into the configurable database names.
[10:08] <bradb> ah, i guess you're talking about a search from the bug listing
[10:09] <bradb> yes, that would be a bug; it should jump you to the correct place
[10:09] <kiko> yeah, regressed on staging
[10:10] <kiko> bradb, where's the diff
[10:10] <bradb> being bazzed right now
[10:11] <kiko> then stop bazzing
[10:11] <kiko> or rather
[10:11] <kiko> stop bugging me :)
[10:11] <bradb> kiko: no point doing the diff if nobody's going to volunteer to review it.
[10:12] <kiko> why am the only one that cares about other people's problems?
[10:15] <bradb> kiko: btw, the bug listing/bug # search though, though a bug, is not a regression
[10:15] <bradb> s/though/thing/
[10:16] <kiko> there aren't tests for it, which is pretty bad, but still, it works in production, but not on staging
[10:16] <kiko> actually
[10:16] <bradb> kiko: it behaves precisely the same on both :)
[10:16] <kiko> yeah, whatever
[10:16] <kiko> it's corner-case behaviour
[10:16] <kiko> you could argue it's correct
[10:17] <kiko> lifeless, autocacherev isn't working on chinstrap, and that kills us
[10:18] <bradb> kiko: i can send you the malone admin awareness patch now, if you're ready for it
[10:19] <kiko> yeah
[10:19] <kiko> sure
[10:19] <kiko> I have to fix the most horrible bug ever today
[10:19] <kiko> gah
[10:20] <bradb> sent
[10:21] <bradb> kiko: the insanely-short timeout, you mean?
[10:23] <kiko> no, hardcoded database names
[10:23] <kiko> running local databases on diskless is slower than molasses
[10:26] <bradb> Seeing what state the error message derobotization branch is in now that the URL changes (off which I branched for it) have landed
[10:40] <mpt> Good mooooooooooooooooooooooooooorn- ... afternoon, Launchpadders
[10:45] <GoRoDeK> hi mpt :)
[10:46] <sivang> good aftermorning mpt!
[10:46] <sivang> :)
[10:47] <mpt> What's new and improved?
[10:48] <kiko> nothing
[10:48] <mpt> really?
[10:48] <mpt> Has PQM gone on strike?
[10:58] <bradb> kiko: how's the admin patch looking?
[10:58] <kiko> nice
[10:59] <kiko> almost there
[11:03] <mpt> brrrrm!
[11:05] <bradb> good thing i happened to have 1.2G of diskspace to throw away to get me to this point in the merge
[11:09] <shawarma> Is searching b0rken or is it just me?
[11:09] <shawarma> er.. in Malone, that is.
[11:10] <shawarma> \sh: And you actually get a result from that URL and not just a "Sorry, a system error occurred"?
[11:10] <\sh> yep
[11:10] <\sh> https://launchpad.net/malone/distros/ubuntu?field.searchtext=vpnc&search=Search&advanced=&status=10&status=20&assignee=all
[11:11] <\sh> shawarma: ah..no..it's the error message
[11:11] <shawarma> \sh: heheh..
[11:12] <\sh> just confused..switching between 3 projects ;)
[11:12] <shawarma> It appears that searching only works when you're not logged in.
[11:12] <shawarma> Weirdness.
[11:15] <salgado> shawarma, \sh, launchpad is having some problems with long-running queries. this is not a problem in the search itself, it happens because we're imposing some heavily low limits on how long a database query can run
[11:16] <\sh> salgado: k...so it's ok for now...and will be changed somehow in the future
[11:16] <\sh> oh missing a ?
[11:17] <salgado> yes, it'll definitely be fixed as soon as we identify what was causing the breakage we had without these limits
[11:19] <asmodai> kiko: oi, around?
[11:19] <kiko> fala asmodai 
[11:19] <asmodai> kiko: you interested in more jobs?
[11:22] <kiko> always :)
[11:22] <asmodai> Heh, I must've scared him.
[11:25] <\sh> salgado: rock :) and we could need a good search for fixed/rejected reports as well, sorted by teams or LP members....;)
[11:25] <bradb> ********************************************************
[11:25] <bradb> *  27 conflicted items in this tree. Please            *
[11:25] <bradb> * resolve each conflict with "baz resolved 'filename'" *
[11:25] <bradb> ********************************************************
[11:25] <bradb> i give up for today
[11:25] <mpt> bradb: How does one customize the order of fields in an addform or editform?
[11:26] <bradb> mpt: they appear in the order named in ZCML, IIRC
[11:26] <bradb> i.e. in the "fields" attribute of that form's declaration
[11:27] <mpt> ah, brilliant, thanks
[11:27] <bradb> np
[11:32] <asmodai> is hackergothi supposed to work if you logged in?
[11:35] <kiko> I have no clue, asmodai -- we're disabling it till the author reappears from vacation :-(
[11:35] <asmodai> oh ok
[11:35] <asmodai> lol
[11:35] <asmodai> I clicked and I was surprised about no access, that's all
[11:36] <asmodai> mmm
[11:36] <asmodai> on my user page
[11:36] <asmodai> I click Add Specification
[11:36] <asmodai> and get a 404
[11:37] <kiko> asmodai, could you file a bug please?
[11:38] <asmodai> will do
[11:38] <asmodai> just verifying ;)
[11:39] <asmodai> btw, site design improved a lot, I like it
[11:40] <asmodai> ok
[11:40] <asmodai> so I have the home page open
[11:40] <asmodai> spot malone for tracking bugs
[11:40] <asmodai> cool
[11:41] <asmodai> come to the malone page
[11:41] <asmodai> see file a bug on a package and read that's only for rpms, debs and so
[11:41] <asmodai> but why does the upstream not have such a convenient quick link?
[11:41] <asmodai> instead you have:
[11:41] <asmodai> Locate Product and View Bugs - locate a product to view, search or file bugs
[11:41] <bradb> kiko: reply sent!
[11:42] <asmodai> which allows to file, but is a bit counter intuitive?
[11:42] <asmodai> ah well, file that as well
[11:45] <asmodai> 17
[11:48] <kiko> asmodai, we're getting there -- bradb just landed one big requisite
[11:49] <asmodai> Well, it's a huge improvement yea
[11:49] <asmodai> :D
[11:49] <asmodai> from a few months ago
[11:49] <asmodai> kudos
[11:51] <bradb> When I get to start working on searching again, the power will shift to the users.
[11:51] <asmodai> heh
[11:51] <bradb> That will happen in November, around time of UBZ
[11:52] <asmodai> UBZ?
[11:52] <bradb> UbuntuBelowZero, developer summit in Montreal
[11:52] <asmodai> ah
[11:52] <bradb> ********************************************************
[11:52] <bradb> *  27 conflicted items in this tree. Please            *
[11:52] <bradb> * resolve each conflict with "baz resolved 'filename'" *
[11:52] <bradb> ********************************************************
[11:52] <bradb> oops
[11:52] <asmodai> mmm, yea, canuck country starts to get colder soon
[11:52] <bradb> https://wiki.ubuntu.com/UbuntuBelowZero
[11:53] <asmodai> where?
[11:53] <bradb> Tokyo
[11:54] <asmodai> Ah nice
[11:54] <asmodai> I will go to Narita, spend 1-3 days there with the gf
[11:54] <asmodai> and then we move to Tochigi prefecture to her parents place
[11:54] <bradb> sweet
[11:54] <bradb> i love tokyo. i could totally live there.
[11:55] <asmodai> yea :)
[11:55] <asmodai> Even with the humidity?
[11:55] <asmodai> err
[11:55] <asmodai>  Sorry, a system error occurred
[11:55] <asmodai> crap
[11:55] <asmodai> after editing some translations
[11:55] <bradb> *humidity*? i'd be too distracted by the earthquakes to worry about how much water's in the air.
[11:55] <asmodai> errr
[11:56] <asmodai> how about a taifuu?
[11:56] <asmodai> nope, getting consistent system errors for editing this translation =\
[11:57] <carlos> see you tomorrow
[11:59] <kiko> asmodai, likely because we're killing long-running queries. sucks, but performance is killing us there
[12:01] <dilys> Merge to rocketfuel@canonical.com/launchpad--devel--0: [trivial]  Fixing bug 2004:  IRosettaApplication has a lot of duplicated methods and methods with a 'self' argument. Clean up interface. (patch-2468: christian.reis@canonical.com)