bdmurraysinzui: the +subscriptions mp is ready for rereview now - https://code.edge.launchpad.net/~brian-murray/launchpad/limited-subscriptions-page/+merge/3517700:05
jmlI'm so behind on email.00:14
mwhudsonjml: i suggest putting a small weight on your 'y' key00:15
jmlmwhudson: I much prefer '['00:15
jmlmwhudson: even so, I've still got 50+ threads marked as "actually, I should read this" and another 20+ as "should answer"00:16
mwhudsonah ok, that's a harder problem00:16
jmlI think the only answer is to never concentrate ever again00:17
jml(unless an email demands that I do so)00:18
jkakarjml: Remove the label from all the emails.  Important things that you should have paid attention to will either (a) get taken care of naturally or (b) spiral out of control and explode.00:29
jkakarjml: It'll be easy to see which things need attention. :)00:29
jkakarjml: I've been enjoying the labelling system you described to me... I find it really hard to be disciplined to actually do the "and get back to deal with labelled items" part, though.00:30
jkakarMuch of the time I procrastinate and then, when I get back to the items, they're no longer relevant.  It's quite an effective technique.  I'm very efficient.00:31
thumperlunch calls00:36
wgrantbuildd-manager seems broken.00:46
wgrantCan someone check its logs?00:47
wgrantDid the CP end up happening?00:47
wgrantStevenK: ^^00:53
pooliethumper: i have a deal for you00:53
pooliethumper: get someone from your team to review and sponsor jam's lp-serve branch, and then he can move on to commit to stacked branches00:55
wgrantsinzui: Does the move of tales to lp.app imply that we're not going to split it?01:12
wgrantAh, good.01:12
sinzui25% belongs in lp.app, 50% registry01:13
thumperpoolie: ok01:15
thumperpoolie: deal01:15
rockstarthumper, skype?01:20
thumperrockstar: sure, let me finish my roll first01:20
thumperrockstar: ok, now01:25
wgrantsinzui: Are you sure that users should be granted access to see *everything*? Even embargoed security bugs?01:31
sinzuiwgrant, I am not sure01:32
lifelessIn the context of an already private project?01:37
lifeless / distro01:37
sinzuilifeless I think the issue can be generalised to all projects and distros.01:38
sinzuiIf I grant view on a public project with private bugs and branches I assume most cases I mean to let the user see everything01:38
sinzuibut in the case of security (which is security contact, not bug supervisor) we may want to require subscription01:39
wallyworld__thumper: ping02:47
rockstarwallyworld__, I understand you're wanting to do some YUI stuff.02:50
wallyworld__rockstar: yeah. do you want to take a peek at https://code.edge.launchpad.net/~wallyworld/launchpad/improved-broken-link-handling02:51
wallyworld__and tell me i'm doing it all wrong02:51
wallyworld__we can skype after you've had a look?02:51
rockstarwallyworld__, okay, I'll take a look.  I was actually going to say "we can skype tomorrow and chat about it"02:52
wallyworld__rockstar: tomorrow is fine02:53
rockstarwallyworld__, as it stands, this is my wife's night off this week, and if I don't watch Madmen with her, I will probably find the locks are changed the next time I leave the house.02:53
wallyworld__rockstar: np. we can chat tomorrow. i forgot it was way past eod for you, sorry02:53
rockstarwallyworld__, my EOD is kinda fuzzy some days.  :)02:54
wallyworld__rockstar: me too :-) i was up late last night figuring out some of the YUI stuff that i'm playing with now02:54
wallyworld__hopefully i'm not totally on the wrong track. :-)02:55
rockstarwallyworld__, I'm still trying to find the right track with YUI.  :)02:55
wallyworld__well, i just tried to see how other stuff had been done and go with the flow. anyway, have a good time watching madmen. talk tomorrow02:56
wgrantmwhudson: Does Shotwell realise that it doesn't own your photos?02:56
mwhudsonwgrant: what do you mean?02:56
wgrantmwhudson: F-Spot likes to copy the photos into its local library and store all metadata somewhere other than the EXIF tags.02:57
wgrantWhich makes it impossible to use across multiple machines.02:57
mwhudsonwgrant: i think it's better about the copying (when you import it asks if you want to copy them)02:57
mwhudsonwgrant: dunno about tags02:57
thumperwallyworld__: hey03:00
wallyworld__thumper: just wanted to skype at some point about the new work. when you are free03:02
thumperwallyworld__: I'm never free :)03:02
thumpercheap maybe, but not free03:02
thumperwallyworld__: let me get a small bit of work done03:02
thumperthen we can talk03:02
wallyworld__ack. i'll have lunch03:02
thumperProgrammingError: operator does not exist: text = bytea03:08
thumperhas anyone been fixing these on trunk yet?03:08
marsthumper, bug 63101003:08
_mup_Bug #631010: ProgrammingError: operator does not exist: text = bytea <database> <maverick> <storm> <Launchpad Foundations:In Progress by mars> <https://launchpad.net/bugs/631010>03:08
marsthumper, working on it right now03:08
lifelessthat reminds me to talk to doko or simila rabout it03:08
thumperis that the package downgrade hack?03:08
* thumper sighs03:09
lifelessmars: if the fix is 'scatter u around the place' then its going to make the code base really ugly.03:09
lifelessmars: so please don't do that ;)03:09
marslifeless, https://bugs.edge.launchpad.net/launchpad-foundations/+bug/631010/comments/403:09
_mup_Bug #631010: ProgrammingError: operator does not exist: text = bytea <database> <maverick> <storm> <Launchpad Foundations:In Progress by mars> <https://launchpad.net/bugs/631010>03:09
lifelessmars: do you mean to say you're putting the older version in the ppa ?03:10
lifelessmars: if so cool, but can you please file a bug on maverick at the same time?03:10
thumpermars: do you have the downgrade command handy?03:10
marsthumper, no, sorry, in the thread, look for "aptitude hold python-psycopg2", maybe "apt-get install python-psycopg2=2.0.13-2ubuntu2"03:11
marsjust a guess03:11
marslifeless, what do you mean "file a bug against maverick"?  Like bug 585704?03:13
_mup_Bug #585704: Storm test suite fails when using psycopg2 2.2 <Storm:Fix Committed by jamesh> <https://launchpad.net/bugs/585704>03:13
lifelessI mean that psycopg2 in maverick is buggy03:13
lifelessthis change (presumably from upstream) is -not safe- to make in python 2.x modules.03:14
lifelesspython 2.x is hopeless confused about bytes vs text, and I can pretty much guarantee that every python webapp on pgsql will be fucked by maverick.03:14
marslifeless, may I suggest you file the bug then, if you know the implications and the people involved?03:14
lifelessI don't know the people.03:14
lifelessbut sure03:14
wgrantOTOH we could just fix LP.03:16
thumpersudo dpkg -i --force-downgrade python-psycopg2_2.0.13-2ubuntu2_amd64.deb03:17
thumperthat's what I did (for future reference)03:17
lifelesswgrant: it will, I guarantee, bit us for months.03:17
lifelesswgrant: its unbelievably hard to be 'correct' in this regard in python 2.03:17
lifelesseven if you just use the core built in things like 'Exception'03:17
wgrantlifeless: It is hard, but Storm already forces us to be pretty much correct.03:18
wgrantThere are only a few places where we're not.03:18
lifelesswgrant: really?03:18
marsso does anyone know if simply copying the package from Lucid to Maverick in the LP PPA would work?03:18
lifelesswgrant: so < 2 hours work?03:18
wgrantlifeless: Storm will refuse str vs unicode mismatches.03:18
wgrantlifeless: So it's only where we're using string-based SQL that there are problems.03:18
lifelesswgrant: thats only one form of incorrectness03:18
lifelesswgrant: if its so transparent, presumably storm can fix it transparently?03:19
wgrantlifeless: Storm is pedantic in the way you say psycopg2 shouldn't be.03:19
wgrantAnd has been since day 1.03:19
marswgrant, maybe you know if the package can simply be copied?03:19
lifelesswgrant: then how can this issue exist? we pass 99% of everything through storm03:20
wgrantlifeless: Pillar name retrievals don't go through Storm.03:20
lifelessmars: if wgrant is correct just fixing LP would be better?03:20
wgranteg. getUtility(IDistributionSet)['ubuntu'] doesn't.03:20
lifelesswgrant: they don't?03:20
wgrantBecause of the pillar name mess.03:20
lifelesswhat does it go through ?03:20
marslifeless, I think henning looked at that, but he "stopped doing this when I hit the first decoding errors. :("03:21
marsHe was putting u'' or unicode() around the query parameters in the test suite03:21
wgrantlifeless: IPillarNameSet, which constructs parameterised SQL manually.03:21
lifelesswgrant: but that still goes through storm03:21
lifelesswe ask storm to run it and storm asks psycopg to run it03:22
lifelessso I repeat, why can't storm fix this?03:22
pooliedoes "fix released locking" mean that once a bug is fixreleased, it can't leave that state?03:22
lifelessactually lets back off03:22
lifelesspoolie: I think so03:22
wgrantBecause Storm is even more pedantic than psycopg2. It would go against its normal policies to fix it.03:22
lifelesswgrant: We have two competing theories.03:22
lifelesswgrant: one is that its shallow.03:22
lifelesswgrant: the other is that its complex and contains surprises.03:23
wgrantmars: You should be able to get away with copying the Lucid *primary archive* version of psycopg2 to Maverick in the PPA.03:23
lifelesswgrant: The data we have supporting it being shallow is nonexistent.03:23
lifelesswgrant: its speculation based on the changes storm needed itself to work.03:23
marswgrant, ok, that would solve the Py2.5 dep problem by being for Py2.6.  I'll give it a shot03:23
wgrantI was able to fix lots of it just by hacking the celebrity descriptors to unicode() their names.03:24
wgrantmars: We shouldn't have psycopg2 for Lucid in the PPA at all.03:24
lifelesswgrant: the data we have suggesting it being complex is - decode errors being thrown when putting u before strings was attempted.03:24
lifelessplus how freaking hard it was to get bzr roughly sane, and the testtools stack similarly.03:24
lifelesswgrant: I humbly submit that you're being optimistic, unless you have more data you haven't presented.03:25
wgrantThe thing is that LP is already very close, because Storm and Zope force it to be.03:25
marspoolie, I wouldn't be surprised.  There must be a fair bit of thought behind a request like that.03:25
lifelesswgrant: but they don't03:25
lifelessFoo.name=='thing' <- clearly not 'correct'03:25
lifelessFoo.name=u'thing' <- clearly ugly. Vive la Python tres.03:25
wgrantWell, let's get Zope running on 3.2 :P03:26
lifelesswgrant: and 'thing' works at the moment in lucid.03:26
lifelesswgrant: whyever its not working in maverick is a regression in the stack.03:26
lifelessin python 3 it would be correct, and historically in python 2 it is also correct.03:26
wgrantHas anybody talked to the psycopg2 folks about this?03:27
lifelessmeh, mere facts.03:29
_mup_Bug #650777: operator does not exist: text = bytea <amd64> <apport-bug> <maverick> <psycopg2 (Ubuntu):New> <https://launchpad.net/bugs/650777>03:29
lifelessits also partly storm03:31
lifelesstreating str as a bytea03:31
lifelesswhich is arguable either way as 'correct' in Python2 - but clearly isn't pleasant to interoperate with.03:32
poolieis there any systematic preference towards having one feature rules page that appears either editable or not, vs having a separate edit page to which only some people have access?03:34
poolielaunchpad tends towards the latter but the former seems to have less dry03:34
poolieless repetition, i mean03:34
mtaylorCan't open average time db /var/debbuild/avg-build-times03:37
mtaylorCan't open average space db /var/debbuild/avg-build-space03:37
wgrantmtaylor: That's normal.03:37
mtaylorwgrant: then why did this fail03:46
mtaylorwgrant: http://launchpadlibrarian.net/56710656/buildlog_ubuntu-lucid-amd64.drizzle_2010.09.1797-1.1~lucid01_BUILDING.txt.gz03:46
wgrantThat looks like an upload failure.03:47
wgrantCan you link me to the build?03:47
wgrantAh, there.03:48
mtaylorwgrant: ah yes03:48
mtaylor -> http://launchpadlibrarian.net/56710652/57D507LpzoCnzjO9QXav5M64BhB.txt (duplicate key value violates unique constraint "binarypackagerelease_binarypackagename_key"03:48
wgrantWhich means a bug has resurfaced.03:48
wgrantYour build is fine; just ignore the failure.03:48
wgrantIt already finished once, then magically got illegally retried.03:49
mtaylorok. cool03:53
mtaylorwell, glad I could help find a bug :)03:53
lifelesspoolie: I don't care either way.03:55
wgrantmtaylor: With any luck it will be fixed after tonight's cherrypick.03:56
wgrantIt replaces what I suspect to be the problematic code, for others reasons.03:56
StevenKlifeless: Ping04:30
StevenKlifeless: Re: your mail about poppy, is there anything you need me to fix?04:33
lifelessit was meant to educate04:34
lifelessI knew you and jml collaborated on that stuff04:34
lifelessand this got through your pair programming sessions.04:34
StevenKlifeless: I'm happy for education and the chance to clean up after my mistakes04:35
lifelessStevenK: my 'test' branch has fixes.04:36
lifelessStevenK: I appreciate the offer to clean up, but there's no need to handoff here.04:37
lifelessI'll be back around later04:41
stublifeless: Are you hacking layers today?05:18
lifelessstub: lp:~lifeless/launchpad/test has layer stuff in it and is in ec2land atm05:33
lifelessstub: got a request?05:33
stubI didn't want to conflict.05:33
lifelessif you check that you don't conflict with my branch, it will be fine.05:34
lifelesswhat are you doing ?05:34
stubI think I might have pushed too early - pqm is currently in process of landing by branch, and I pushed new revisions. Have I borked it or will it only merge revisions pushed when it started processing?05:34
stubMaking the thread check more robust - there is a codehosting test that occasionally fails, so I'm giving garbage threads a few seconds to shut down and gc.collect() to help them along05:35
stubUncommit and push overwrite before PQM finishes processing?05:36
wgrantDoesn't the signed request include a rev ID?05:37
stubNo it doesn't05:38
stublifeless: ^^05:38
wgrantWell. That's a bit strange.05:39
* stub tries the push overwrite before it is too late05:39
stubbah - something else has it locked05:39
lifelessit will only merge what it found when it started.05:39
stubok. So I can get this changed reviewed and check for conflicts :)05:40
lifelessstub: any reason we can't just remove the thread check? what does it actually help us with? Or make it like the bzr one (reports at the end on tests that leaked)05:40
stublifeless: We can drop the thread check if we don't care about tests leaving threads lying around. I've mentioned that option when that has come up in the past, but people have always opted for fixing the source of the issue.05:41
lifelessstub: I quite like what bzr does:05:41
lifeless - gathers threads05:41
lifeless - doesn't fail, but reports on the tests that leaked.05:41
lifelessits a mix of both worlds.05:41
lifelessI very much like fixing the source of issues, but a thread not being gone by the time a test finishes isn't prima facie an issue05:42
stubA test can turn the fail into a warn if it wants, or we could set it globally by changing the flag in BaseLayer.05:42
poolieany guesses why my validate method is getting an empty 'data' dict?05:50
pooliei guess the field is not correctly seen as a widget05:52
pooliejtv i realized i can use validate() to give a better syntax error message06:02
jtvpoolie: yes, that's a good place for it.06:02
lifelessstub: so rather than adding multi second delays06:02
lifelessstub: lets make it a warning06:02
lifelessstub: globally06:03
stublifeless: Tests rarely hit this - we will be delaying maybe one in 10 test runs a few seconds06:03
jtvpoolie: I see no excuse for the field not to be in the data dict… does the form use GET instead of POST or something?06:03
stubI'd like to land it as is because the revision did get through PQM :-/06:03
lifelessstub: thats fine, but perhaps you could follow up with more stuff?06:04
stubSo the sleep is still valuable as it is 1) unreasonable to expect tests to wait on every thread spawed, particularly if they were not responsible for spawning it 2) sometimes threads can take a while to shut down.06:05
stubAre both those points reasonable?06:05
lifelessso we had a period in bzr where we tried a few different things06:06
stubWe can of course make the sleep less - maybe 100 checks at 0.1 second intervals rather than 10 at 1 second?06:07
stubIf it is just the thread didn't get allocated enough cpu to shutdown, it is likely sorted in milliseconds.06:08
lifelessso, to cleanup threads, they *have* to be joined06:08
pooliejtv i've solved it now06:09
jtvpoolie: what was it?06:09
pooliei had handcrafted html for the textarea and i should have let launchpad generate it06:10
pooliewell, zope06:10
pooliethen the field name would have matched what it expected06:10
jtvah ah ah06:10
jtvSo your data wasn't empty—you just weren't looking for the right key!06:10
stublifeless: I looked for a way to do that and couldn't find it. A second check now and it looks like I *can* iterate over the new threads and join them with a timeout.06:11
lifelessstub: well, more importantly, I think this invalidates 1)06:12
lifelesstest code is either directly responsible for spawning the thread, or its using an api which must have a facility to join its own threads06:12
stubYes. I think we can make BaseLayer wait for the threads to terminate for a period of time. I think it is worth failing the test if they don't die, as it is a leak.06:13
lifelessor its something like a thread pool which reasonably wants to stay active.06:13
pooliejtv, it was empty, but that's because zope code doesn't pass through unrecognized input parametres06:13
pooliemy god it's a bit disgusting how many http requests there are per page in test mode06:14
stubWe have a flag to turn the fail into a warn. I think it is worth keeping it as a fail until we actually have threads that should remain active across threads.06:14
pooliei realize in production some things are rolled up into larger resources06:14
lifelessif we want to catch incidental spawned threads that leak, we either need an API to say that some threads *are* expected, or we have to ensure that all tests clean up everytime06:14
jtvpoolie: a lot of those will be for icing that isn't served by LP06:15
lifelesssetting up a thread pool is both reasonably cheap and also reasonably expressed as a fixture for sharing across tests.06:15
lifelessso I think we can say 1) it is reasonable to require tests to join all threads they cause (in)directly to be spawned.06:15
jtvpoolie: but it'd be great if we could reduce the requests and db queries we see on dev systems, yes06:15
lifelessstub: with that assertion, 2) becomes irrelevant,06:16
lifelessstub: what do you think?06:16
stubYes, 2 becomes irrelevant.06:17
stub1) is still relevant though, as threads could be spawned by third party libraries like bzrlib.06:18
lifelessdoes my argument for revising (1) make sense to you?06:18
stubIf bzrlib starts spawning a new thread where previously it didn't, it is unreasonable to update all the LP tests to check for this new thread to terminate.06:18
stubor zope or psycopg or whatever06:19
lifelessBut a test fail is a test fail.06:19
jtvpoolie: perhaps you can help me with the traceback you see near the end of this "make harness" session: https://pastebin.canonical.com/37820/06:19
lifelessstub: I mean, I agree, thats why I was suggesting warning only initially;06:19
stubThe layer will wait for the thread to terminate if it takes time (the spurious failures I'm trying to address), and if it doesn't terminate in a few seconds, then it fails.06:20
jtvpoolie: that's on the staging db.  Why the lp-internal URL scheme, and why does convert_path_to_url choke on it?06:20
stubwarnings just get ignored, and we are left with a test isolation leak06:20
lifelessI think warnings that are very clear are less likely to be ignored06:21
lifeless'test foo leaked 4 tests' is pretty clear.06:22
lifelessstub: waiting for e.g. a thread pool thread to terminate is likely to just timeout, only a few threads are ones that will naturally go away.06:22
stubSo what is the advantage of warning? I see absolutely none, except a potential problem has leaked into the codebase. Its not like we expect this to happen - we have never had a thread leak that wasn't a) spurious b) an error06:23
lifelessstub: I don't think its a strong argument for the layer waiting at all given that being able to wait successfully is itself a special case.06:23
stub(a or b)06:23
pooliejtv, this is just spm randomly typing stuff?06:23
pooliejtv, that transport is registered by the lp-serve plugin06:23
lifelesssigh, openid grrr grr grr grr06:24
poolieso for some reason that's not loaded in this process06:24
jtvpoolie: no, this is spm pasting a script I wrote up for this purpose into make harness.06:24
poolie1- you didn't ask for plugins to be loaded in this interpreter06:24
poolie2- that plugin wasn't found, probably because it wasn't on your path06:24
poolie3- other06:24
lifelessstub: so there are three approaches: a) wait and fail; b) warn [no point waiting if it can't fail], c) fail [make it the tests responsibility]06:25
stubFor b), we should wait. We don't want spurious warnings as people just filter them out as noise.06:27
lifelessstub: I don't see the value of waiting06:27
jtvpoolie: well I didn't ask for them to be loaded in this script, but it's not unthinkable that the branch scanner (which goes through similar steps) doesn't either.  How do I ask for that?06:27
lifelessstub: we've spent enough time on it - do what makes sense to you.06:28
lifelessstub: I can say that I wouldn't wait :)06:28
stublifeless: warn and not wait is the solution everyone has rejected so far.06:28
pooliejtv bzrlib.plugin.load_plugins06:32
pooliejtv importing lp.codehosting seems to do that as a side effect06:32
jtvpoolie: yup, see it in the __init__06:33
jtvSo that's my script, not the branch scanner code06:33
poolietry running bzrlib.plugin.plugins()06:33
jtvpoolie: locally that gives me a bunch, including lpserve.06:34
poolieperhaps the paths are being set up wrong so it's not loaded06:34
jtvpoolie: well in that script I think I simply didn't do anything that loads it.  Trying a corrected version now.06:35
lifelessstub: sure, thats doesn't mean its wrong :P06:39
lifelessstub: anyhow, as I say, do what makes sense to you06:39
wgrantlifeless: What does the bad-commit-11566 tag do?07:23
pooliehi jtv? all ok?07:24
jtvhi poolie!  No reply from spm yet07:24
lifelesswgrant: https://dev.launchpad.net/QAProcessContinuousRollouts07:28
lifelessStevenK: I would like an answer on my concurrent poppy question07:33
StevenKlifeless: I've had a few stack dumps since then, can you please re-ask?07:34
lifelessStevenK: look in the mailing list :)07:35
StevenKlifeless: And no, we don't07:36
StevenKlifeless: Both cocoplum and germanium run the FTP and SFTP servers07:37
StevenKAnd I have no idea about how either of those services would cope or behave in a load-balancing situation07:37
noodles775Morning ppl07:38
lifelessStevenK: we don't what ?07:39
StevenKlifeless: We don't run a single copy of both the FTP and SFTP servers.07:40
lifelessfor ppas, how many copies do we run ?07:42
StevenKOne, on germanium07:43
wgrantBut cocoplum's works perfectly well.07:43
StevenKlifeless: Notice I'm not disagreeing it's a SPOF, I'm disagreeing that we only run one copy.07:44
lifelessStevenK: I don't know what you're saying at all07:44
lifelessStevenK: so i'm trying to pin it down07:44
lifelesswgrant: will a ppa upload to *either* work for PPAs' ?07:44
wgrantlifeless: Yes.07:44
lifelessso all we hve to do is shove ha proxy in front and make all the dns names point at the same place?07:45
StevenKThat will then cause havoc07:45
wgrantI'd prefer to resolve a germanium SPOF in a way that doesn't involve cocoplum.07:45
lifelesswhat havoc?07:45
StevenKSince people download and upload from ppa.launchpad.net07:45
lifelesslet me explain what I need.07:45
lifelessI need:07:45
wgrantlifeless: Does haproxy do FTP and SFTP?07:45
lifeless - to know whether the software is safe to run concurrently for a given 'queue' (or whatever abstraction it uses - sounds like it is)07:46
wgrantpoppy can safely run concurrently, yes.07:46
lifeless - to provide a detailed request for config changes to the losas to eliminate the current downtime during upgrades.07:46
wgrantIt's not safe to run multiple upload processors, but we do anyway.07:46
lifelessupload processors are less concerning07:46
lifelessbecause its not listening on the network07:47
wgrantBut we can't safely run multiple upload processors from different hosts.07:47
lifelessso, which machine is the ppa one07:48
lifelessand what is cocoplum known as?07:48
wgrantcocoplum is the other instance -- it is ftpmaster.07:48
lifelessdoes ftpmaster provide an HTTPS interface?07:48
lifeless and HTTP ?07:48
wgrantNot externally.07:48
lifelessso, how does this sound.07:48
StevenKHTTP only, and only internally07:48
lifelessis that http apache?07:49
lifelesshere is what I propose07:49
lifelessha proxy with ppa and ftpmaster pointing at it07:49
lifelessha proxy directs http requests to germanium07:50
lifelessha proxy or germanium's apache sends ftpmaster internal requests to cocoplum07:50
lifelessha proxy directs ftp and sftp to either machine depending on whats running07:50
wgrantThis sounds like two SPOFs.07:50
wgrantAlso, will FTP be much of a fan of being run through haproxy?07:51
lifelesswgrant: SEP07:51
StevenKOr SFTP, for that matter07:51
wgrantSFTP should be fine.07:51
wgrantIt's a single TCP stream.07:51
lifelessso, elmo has said there is a tool (it might not be haproxy) for doing this stuff.07:51
StevenKI'm worried about host keys07:51
lifelessif there isn't we'll have to go find or write one.07:51
lifelessStevenK: ok, thats a good point. We can do one of two things.07:52
wgrantStevenK: They'd have to be the same on both.07:52
wgrantEasy enough.07:52
lifelesswe can say 'these should really have been one service', announce and JFDI it.07:52
StevenKI'm also worried about user-confusion07:52
wgrantlifeless: Why should ftpmaster.internal requests go through germanium?07:53
lifelessor we can say 'these are meant to be different', and we then bring up new instances of these services on other machines and tell the twisted daemon to use an appropriate host key.07:53
lifelessStevenK: getting rid of downtime can go a long way :)07:53
lifelessnote that openssh host keys are not involved here07:53
lifelessbecause its twisted doing ssh07:53
wgrantlifeless: The two hostnames used for FTP and SFTP are upload.ubuntu.com and ppa.launchpad.net.07:53
lifelesswgrant: so what are the two spofs you see?07:54
lifelesswgrant: I'm suggesting only that ftpmaster things on the hostname that poppy is on get redirected.07:54
lifelesswgrant: if there is no http/https on 'upload.ubuntu.com' then its irrelevant07:54
wgrantlifeless: ftpmaster.internal should still point at cocoplum. upload.ubuntu.com and ppa.launchpad.net will point at haproxy.07:55
wgranthaproxy forwards HTTP(S) to germanium, and sends ftp/sftp to germanium or cocoplum.07:55
wgrantThis leaves germanium as only a SPOF for PPA, not ftpmaster too.07:56
lifelessanything else we need to tweak, or can I shoot this off?07:57
wgrantWell, it's not safe at the moment.07:57
lifelesswgrant: is it less safe than what we have now?07:57
wgrantUploading to both can result in rather corrupt archives.07:58
wgrantIf I upload a package twice to one host, it will be rejected the second time.07:58
lifelesswhat do you mean?07:58
wgrantIf the second one lands on another host, and is processed simultaneously, both uploads may succeed.07:58
wgrantThis is not theoretical -- this happened to me a couple of months ago.07:58
lifelesssounds like a bug07:58
wgrantIt is, yes.07:58
lifelesswhen will you fix it?07:58
wgrantWe need better locking. But neither Julian nor I have much idea of how to do it.07:59
lifeless[did you see what I did there?]07:59
lifelesswgrant: how did that happen? or were you uploading to the other host deliberately?07:59
wgrantlifeless: I'd waited an hour for an upload to be processed on germanium, and had no response. So I uploaded to cocoplum. Also no response.08:04
wgrantHalf an hour later, both uploads were processed simultaneously.08:04
wgrantAnd my PPA got rather unhappy.08:04
lifelesswgrant: could we just put a unique index on the archive?08:06
wgrantlifeless: Over about 6 tables, sure.08:06
wgrantie. "no"08:07
wgrantThat too.08:08
wgrantAnd SPRF.08:08
stubI'm fixing the stable->db-devel merge conflict08:08
StevenKSo it's Archive-BPPH-BPR-BPF-SPPH-SPR-SPRF-LFA08:09
wgrantWe could put a constraint in, I suppose.08:09
StevenKTasty, 8 table unique index08:09
lifelessmmm, that says you have a buggy model.08:11
lifelesswe should look at that.08:11
wgrantThe model is horrifyingly complex.08:11
wgrantI'm not sure it's *buggy*.08:11
lifelessits simple to fix this08:11
lifelessrather than active active08:11
lifelessis the load on the system low enough that that would work08:12
StevenKNot for cocoplum08:12
StevenKThe publisher on cocoplum is mean08:12
stubwgrant: If you need to enforce uniqueness over that many tables, it is buggy (well - unsupported by RDB model). The data that needs to be unique needs to be broken out into a separate table.08:13
lifelessStevenK: why are we talking about the publisher?08:13
StevenKBecause it's involved as well, it runs on cocoplum for the Ubuntu archive08:13
lifelessStevenK: It wasn't listed in the changes proposed above, can you describe how its connected?08:14
StevenKlifeless: You asked if the load on the system is low enough08:15
lifelessso if we make germanium the normally active master08:15
lifelessfor poppy08:15
lifelesscan cocoplum handle the load for say 30 minutes during a germanium services reboot ?08:16
StevenKIt will suffer, and I have no idea about germanium's nominal load, since I don't have access to it08:16
lifelessthe question is whether the uploads during that period would feel degraded to users.08:17
lifelessbut if the main load is the archive, it would presumably feel just like upload.ubuntu.com does today.08:17
lifelessso, we have:08:17
lifelessupload & ppa -> haproxy08:17
lifelesshttp -> germanium08:17
lifelesssftp and ftp -> germanium unless its down, then to cocoplum08:18
lifelessannounce and consolidate a single hostkey for both services08:18
StevenKThey may already share one, but I'm unsure08:19
lifelessok, go/nogo ?08:19
lifelessI'm going to cc bigjools08:19
lifelessbut I want to be sure we're happy with the plan first08:19
StevenKTo be perfectly honest, I have my doubts, but nothing I can put my finger on.08:20
StevenKCall it a gut feeling08:20
lifelessWill let Julian mull on it.08:20
StevenKThat bit sounds good08:20
wgrantstub: I'm not sure there's a significantly better way, unfortunately.08:22
StevenKIn an utterly unrelated question, can I have an __init__ function take arguments if it's being called from getUtility(Interface) ?08:23
lifelessno, but you can register an instance rather than a factory08:23
stubStevenK: utilities are singletons and even shared between threads.08:24
lifelesswgrant: whats the bug number ?08:24
poolieit seems like zope, if a field is '', omits it from the data dict?08:25
poolieis that true or am i doing something weird?08:25
lifelessI would not be surprised. Displeased, but not surprised.08:26
poolieit's kind of unfortunate because it conflates that case with "i made a mistake and didn't get the form i expected"08:26
lifelesswgrant: does that race condition exist with ftp + sftp as well ?08:28
wgrantlifeless: I don't think so.08:31
wgrantBut I can't see the configs.08:31
wgrantDo they use the same upload queue directory?08:31
StevenKNo, they don't08:33
StevenKOh, queue directory. Yes.08:33
wgrantThen they're safe.08:34
pooliejtv, lifeless, i think i'm finally almost there08:40
adeuringgood morning08:42
pooliehi abel08:46
wgrantlifeless: Do you have a plan to remove the main germanium SPOF?08:46
wgrantThat is HTTP.08:47
wgrantIt probably has more users than any other LP service.08:47
wgrantIdeally, yes.08:50
wgrantDo you have one?08:50
wgrantWill it be used for the librarian too?08:52
lifelessthough running something like ceph might be even better08:54
wgrantHmm. Maybe.08:54
wgrantBut SANs are easier.08:54
lifelessnot really08:55
lifelessgrrr ec2land fail09:04
wgrantCan we rename it to ec2reject?09:04
poolieit's kind of russian-roulette landing isn't it09:05
wgrantIt varies.09:05
wgrantFor a while it was almost always one-shot.09:05
wgrantThen it... wasn't.09:05
StevenKec2 landifyourelucky09:06
lifelessthis is changing fundamental assumptions09:06
lifelessso its not surprising to be playing 'tanks'09:07
lifelessits just slow.09:07
=== almaisan-away is now known as al-maisan
jmllifeless: the ec2 land fail... what was the bug?10:03
stubDoes the failure at https://lpbuildbot.canonical.com/builders/lucid_db_lp/builds/272/steps/compile/logs/stdio make any sense to anyone?10:19
stubFailure looks genuine, and the landing is the stable -> db-devel merge10:20
stubhttps://lpbuildbot.canonical.com/changes/13 - tales stuff being modified by someone.10:21
wgrantUm. "Dissolve cornflakes"?10:22
wgrantAh, I see.10:22
stubI'd guess https://lpbuildbot.canonical.com/changes/910:22
stubnope - that is production...10:23
lifelessjml: interrupted subunit stream, probably due to the layer code - perhaps even the thing I fixed in zope.testrunner that isn't merged yet because it has no tests because buildout was not working for it10:23
lifelessstub: sinzui moved tales to a new place10:24
bigjoolsjelmer: there he is :)10:24
stubhttps://lpbuildbot.canonical.com/changes/12 from jcsackett, but that landed on lp/devel too recently to be affecting db-devel?10:25
stuboic. so moving it broke some new code on db-devel10:27
lifelessstub: https://bugs.edge.launchpad.net/launchpad-foundations/+bug/607391 needs qa10:29
_mup_Bug #607391: upgrade robustness for cronscripts <cron> <qa-needstesting> <Launchpad Foundations:Fix Committed by stub> <https://launchpad.net/bugs/607391>10:29
lifelessstub: and if it needs special deployments stuff, writing up in the normal fashion.10:29
lifelessstub: its next in the https://devpad.canonical.com/~lpqateam/qa_reports/launchpad-stable-deployment.txt queue10:29
stubI'm sending a [testfix] with the corrected import10:31
jmllifeless: ahh, ok10:33
jmllifeless: I guess we ought to upgrade to z.testrunner if we can, just to make sending things upstream easier (or, as you say, abandon it altogether)10:33
lifelessone less thing to maintain.10:34
jmllifeless: well, first port of call is stop using layers :)10:42
jmlalternatively, stop needing z.testrunner to do layers10:43
lifelessworking on that, this branch keeps bouncing with more glitches10:43
lifelessbecause the tolerance for bad environment goes way deep10:43
jmllifeless: good luck!10:44
stubIf we stop using layers, I don't think there is any reason to use z.testrunner.10:57
jmlstub: yeah, that's my impression too10:59
stubmthaddon: Can you confirm that staging cronscripts are happily running? (This is for QA).11:01
mthaddonstub: erm, all of them? not quite sure what you mean11:02
stubmthaddon: Any of them11:02
mthaddonstub: process-mail.py seems to be working fine11:03
stubmthaddon: Assuming they are running, can you please create http://paste.ubuntu.com/502544/ as cronscripts.ini in the root of the staging tree and run an arbitrary cronscript?11:03
stubIt should give you a log message about being disabled, and I can qa-ok by bug.11:03
stub(INFO level message)11:06
stubWee! The daily storm is starting. Love the show this time of year ;)11:07
mthaddonstub: https://pastebin.canonical.com/37829/11:07
stubmthaddon: Ta. That's everything I need.11:08
stubmthaddon: So two changes you should like have landed. The --log-file argument has changed. Now you can run cronscripts as 'foo.py -q --log-file=debug:/srv/logs/foo.log' and have WARNING and above go to stderr and DEBUG and above go to the log file. -qq if you only want to see ERROR and above as you would expect.11:11
stubmthaddon: logrotate will work fine on these logs - the module will notice and cope with the file being rotated under its feet.11:12
stubmthaddon: The other one is cronscripts.ini being available at a configurable URL and enabling/disabling scripts individually or in bulk. I suspect the first use of this will be to disable the cronscripts while running upgrade.py etc. during the staging restore.11:13
stubmthaddon: How should we go about documenting these?11:13
mthaddonstub: I think probably an email to the losas is the way to start - we can then figure out what to do from there11:15
stubmthaddon: ok. Will do.11:15
gmbWell now, this is special...11:41
gmbDoes anyone have any idea why I've suddenly started having this problem after merging devel: http://pastebin.ubuntu.com/502558/11:41
gmbstub: Is this ^^ something to do with the logging stuff you were working on last week?11:41
gmbOr am I just seeing "logger" and jumping to conclusions?11:42
* gmb make cleans11:42
stubgmb: I think it is fallout11:42
gmbstub: Okay. We'll see what happens when I've cleaned and rebuilt.11:43
stubgmb: We were previously throwing away a lot of log messages by accident. We should fix that noise in the Librarian log file. Ideally by someone who understands how twisted does its logging.11:44
stubI think we currently have some frankenstein Python/twisted hybrid?11:44
stubMight just involve adding a NullHandler to getLogger('librarian') in lp_sitecustomize.py or in the librarian startup gumph.11:45
stubgmb: This won't stop the librarian from starting though, so either the real error is further down or the real error isn't being output because librarian logging has been screwed.11:46
gmbHmm, interesting.11:46
stubgmb: Suspect rogue librarian process (aka. the usual suspects)11:46
gmbEntirely possible.11:46
* gmb goes to poke around11:46
stubjml might know since I saw his XXX somewhere in there...11:48
stubjml: I broke librarian logging. please fix it.11:49
jmlstub: OK. I'll need some coffee and to finish with this email thread first though.11:50
gmbstub, jml: FTR the problem went away after make clean / make. No stale processes hanging around.11:52
jmlgmb: I guess there's a deeper bug about the shitty failure mode11:52
gmbI do like the librarian's strategy of just shouting about the problem twenty or thirty times and then falling over. That might be the new way forward for error handling.11:53
jmlgmb: in particular, shouting about the wrong problem11:57
gmbYes. That was somewhat unhelpful.11:58
stubWe could make our logs more readable by dropping the LEVEL prefixes entirely and use repetition to indicate severity.11:59
deryckMorning, all.12:00
jmlstub: I disagree. We should use ANSI colors to indicate severity.12:01
stub--log-file=debug:/var/tmp/log.html --log-format=html12:02
stub<blink>No handlers could be found for loggNo handlers could be found for logger "librarian"No handlers could be found for logger "librarian"er "librarian"</blink>12:02
wgrantjml: Critical errors can blink!12:03
wgrantDamn, I was beaten.12:03
deryckallenap, morning.  Do you think we need a card on the kanban board for bug 650991?  Or does the current card cover it?12:04
_mup_Bug #650991: Add getSubscriptionsForBug to IStructuralSubscriptionTarget <Launchpad Bugs:In Progress by allenap> <https://launchpad.net/bugs/650991>12:04
allenapderyck: I've forgotten how I should model these things. So, that bug is going to end up having 2 or more branches. I've completed the first, am working on the second. Should I have a card for each branch or bug?12:05
deryckallenap, I think generally we want a card for each branch.  Thinking being that the branch is the unit of work.12:07
allenapderyck: Okay, I'll make it so.12:07
jmlI'm going to gently ambiate feelings of warm encouragement about fixing the librarian's bad failure mode12:07
deryckallenap, thanks!  Just trying to get us conscious about moving work forward clearly.12:08
deryckallenap, you can use the incremental flag as you land these and just do qa on the final branch/card that closed the bug, though.12:10
deryckso the other cards can land and flow straight into done-done.12:11
allenapderyck: Okay, cool.12:11
adeuringderyck: could you please run an EXPLAIN ANALYSE for this query on staging: https://pastebin.canonical.com/37832/ ?12:39
bigjoolsjelmer: congrats on the new buildd-manager code, it looks to be FREAKING ROCK :-)12:40
deryckadeuring, sure.  Doing so now....12:40
deryckwow, that's a query, adeuring :-)12:40
jmlbigjools: that's in stable now?12:40
bigjoolsjml: it's gone live12:40
bigjoolsdb-stable I think12:41
* wgrant watches build farm latency drop to zero.12:41
jmlbigjools: ahh, ok.12:41
jmlbigjools: just thinking about merging it into the twisted work12:41
wgrantLogtails updating a couple of times a minute... I approve.12:43
wgrantbetter than every 20 :)12:43
bigjoolsjml: yes, I'd merge it12:44
bigjoolsjml: although thinking about it, I think we already have it12:44
bigjoolsit landed a while ago12:44
jmlbigjools: yeah, but we've been developing from devel/stable, not db-devel12:44
jmlbigjools: which revision is jelmer's change?12:44
wgrantIt shouldn't be in db-12:44
jmlor what branch...12:45
bigjoolsgood point well made12:45
wgrant11566 or so?12:45
* wgrant checks.12:45
* bigjools is busy rescuing the librarian12:45
wgrantjml: 1156612:45
jmlwgrant: you know, that's just a little eerie.12:45
jmlwgrant: but thank you.12:45
bigjoolsthere was another branch12:45
wgrantOh, right, 1157912:46
wgrantThat fixes recipe builds.12:46
bigjoolslifeless: ping12:48
bigjoolsoptimistic ping....12:48
wgrantIt's 1am...12:48
bigjoolsthey're on DST already?12:48
wgrantLast Sunday.12:48
bigjoolshuh, weird :)12:49
wgrantWe're not until this weekend, though.12:49
wgrantThey are special.12:49
deryckadeuring, it still hasn't completed.  I think I should kill it.12:49
bigjoolsand another month for us12:49
adeuringderyck: gahhh...12:49
bigjoolsto go back that is12:49
bigjoolsit's going to make my standups interesting with Steve12:49
=== mrevell is now known as mrevell-lunch
adeuringderyck: could you try this one: https://pastebin.canonical.com/37835/ ?12:54
deryckadeuring, sure12:58
deryckadeuring, still going.  I'll kill it now.13:03
adeuringderyck: yeah, sure13:03
=== matsubara-afk is now known as matsubara
wgrantbigjools: Why don't we run the upload processor every minute?13:15
bigjoolswhich upload processor?13:15
wgrantThe one behind popp.13:16
bigjoolshysterical raisins13:16
wgrantI presume we now run the buildd-manager one every minute.13:16
bigjools30 seconds actually13:16
wgrantEven better.13:16
jmldear pycon, please stop calling for papers when I am phenomenally busy on non-programming stuff, love jml13:18
bigjoolswgrant: oh I lie, it's every 513:19
wgrantbigjools: Uh, really?13:19
bigjoolsit should be every minute... jelmer?13:19
wgrantThat's going to break date_finished pretty badly.13:19
wgrantAlthough not as badly as it was broken before, I guess.13:19
jelmerbigjools: the upload processor runs every 5 minutes13:21
=== mrevell-lunch is now known as mrevell
jmljust to confirm...13:30
jmlIBuilder.active = True does not imply that there's an active build on that builder13:30
wgrantactive just controls whether it's shown on /builders.13:30
wgrantSo, no, it's unrelated to whether there's a build on it.13:31
wgrantYes, we like confusing names.13:31
jmlwgrant: ta13:47
wgrantbigjools: Um.14:17
wgrantbigjools: You didn't run that SQL, did you?14:17
wgrantThat's rather overbroad :/14:18
bigjoolsI did14:24
wgrantbigjools: I think you just deleted most of hardy.14:26
bigjoolswgrant: huh?14:26
bigjoolswgrant: actually I see what you mean14:28
wgrantbigjools: You found all binaries in intrepid that weren't published or pending, and deleted their files.14:28
wgrantMost of those are inherited from dapper.14:28
wgrantAnd will be in jaunty and co too.14:28
wgrantWe need to move this expiration to something more like the librarian GC.14:31
wgrant(excluding active records, rather than trying to select inactive ones)14:32
bigjoolsyes, reference counting would be nce14:33
wgrantI forget how librariangc does it.14:35
wgrantBut it's reasonably not too bad.14:35
wgrantBut anyway, I hope the GC hasn't run yet, or we are screwed if we want to be able to initialise natty.14:36
bigjoolsit has not14:39
bacabentley, adeuring, allenap , bac, danilo, sinzui, deryck, EdwinGrubbs, flacoste, gary, gmb, henninge, jelmer, jtv, bigjools, leonard, mars, noodles77514:59
bac: reviewers meeting starting soonish14:59
bigjoolsbac: I will be late14:59
marsbac, gary is out today, he sends his apologies15:02
maxbMy email "[Launchpad-dev] RFC: Cleaning Launchpad Lucid PPA" is lonely. Anyone feel like replying? :-)15:02
=== Ursinha-afk is now known as Ursinha
jmlmaxb: I feel totally unqualified to reply in any way other than "Gosh I'm glad someone else is taking care of this"15:09
salgadosinzui, we seem to have some spurious TeamParticipation entries which might be caused by those changes Edwin did to the code which maintains that table: https://bugs.edge.launchpad.net/launchpad-foundations/+bug/597208 (last comment is the relevant one)15:34
_mup_Bug #597208: Run cronscripts/check-teamparticipation.py on production and make its output more visible <Launchpad Foundations:Triaged> <https://launchpad.net/bugs/597208>15:34
sinzuisalgado, there are all merged teams. I think they were merged using the delete action which is a special form of merge15:37
salgadosinzui, they're not all merged. for instance, https://edge.launchpad.net/~ubuntuone-users is on that list15:38
sinzuisalgado, I delete mailing-list-beta-testers myself. This is a bug in the delete rules. I assumed something like this could not happen since delete is a subclass of merge where the destination team is ~registry15:38
salgadooh, nm me15:39
salgadoI see that the entry is for another team that merged into ubuntuone-users15:39
sinzuisalgado, Do we have a script to clean up the mess that is in TeamParticipation?15:40
salgadosinzui, nope15:40
gmbCan someone remind me what I need to do to enable a feature flag in my dev environment? I.e. how do I get features.getFeatureFlag('malone.foo-bar.enabled') to return True?15:41
sinzuiokay, I will arrange a code fix and a cleanup15:41
wgrantCan someone poke ISD about the login.launchpad.net issue that has been reported in #launchpad?15:44
wgrantI can confirm; it seems fairly broken.15:44
deryckgmb, I believe you're looking for noodles775 reply to the email thread "how to use feature flags".  And subsequent emails.15:44
deryckgmb, unless you mean doing the db insert locally to enable the flag?15:44
deryckIf I'm understanding this right, not having used it myself.15:45
noodles775gmb: http://pastebin.ubuntu.com/494656/ is what I'm using (just line 3).15:47
gmbnoodles775: Thanks.15:47
noodles775wgrant: I've pinged the ISD guys.15:49
wgrantnoodles775: Thanks.15:49
flacostegmb: poolie landed a UI for this yesterday15:49
bdmurraysinzui: the +subscriptions mp is ready for rereview now - https://code.edge.launchpad.net/~brian-murray/launchpad/limited-subscriptions-page/+merge/3517715:57
sinzuibdmurray, salgado should go first15:58
salgadobdmurray, sinzui, I've just started. :)15:58
bdmurraysinzui: okay15:58
lifelessbigjools: yo?16:07
lifelessflacoste: For simpler rollouts I think qastaging is needed, but not edge gone.16:07
lifelessflacoste: discuss.16:07
bigjoolslifeless: jeebus, it's what, 4am there?!16:07
lifelessyes, early plane flight to sydney. brb16:08
gmbI would like it noted that the fact that +subscribe is handled by BugTaskView instead of BugSubscriptionAddView (or similar) is a) odd and b) annoying.16:14
bigjoolslifeless: anyway unping16:15
bdmurrayI concur16:15
bdmurraysalgado: what bug didn't send you back to +subscriptions on clicking cancel?  it works for me16:21
salgadobdmurray, I think I tried a couple different ones.  let me try again16:22
lifelessbigjools: hah, ok16:24
salgadobdmurray, indeed, reproduced with bug 13 and bug 416:24
_mup_Bug #13: empty signing rules lead to invalid checksums <Baz (deprecated):Fix Released by lifeless> <https://launchpad.net/bugs/13>16:24
_mup_Bug #4: Importing finished po doesn't change progressbar <Launchpad Translations:Fix Released by carlos> <Ubuntu:Invalid> <https://launchpad.net/bugs/4>16:24
bdmurraysalgado: as name12?16:24
salgadobdmurray, yes16:24
deryckgmb, indeed, it's ridiculous.  And we're likely doing extra work for the regular view that we don't need.16:24
bdmurraysalgado: okay, thanks I'll poke at it some more16:25
gmbderyck: Indeed. So, new bug to file, then :)16:26
deryckgmb, yes, indeed.16:26
gmb(I'll fix that before doing bug 65110816:26
_mup_Bug #651108: Update the bug +subscribe view to include the options for bug_notification_level <story-better-bug-notification> <Launchpad Bugs:In Progress by gmb> <https://launchpad.net/bugs/651108>16:26
salgadobdmurray, I see at http://bazaar.launchpad.net/~brian-murray/launchpad/limited-subscriptions-page/revision/11486 that it will redirect to the referrer when you hit the Continue button but not the Cancel link.  for the link you need to set cancel_url, IIRC16:27
bdmurraysalgado: http://bazaar.launchpad.net/~brian-murray/launchpad/limited-subscriptions-page/annotate/head%3A/lib/lp/bugs/templates/bug-subscription.pt the cancel href is set to the _return_url there16:31
bdmurraysalgado: so I'm not quite certain how that is happening for you.  Does the branch also require sinzui's approval?16:40
salgadobdmurray, I think I know what's going on... ReturnToReferrerMixin must be mixed into a LaunchpadFormView, but you've mixed into a LaunchpadView16:40
salgadobdmurray, does the Cancel link work for you?16:40
salgadobdmurray, yes, it needs sinzui's approval as well16:41
bdmurraysalgado: yes and the test in lib/lp/registry/stories/person/xx-person-subscriptions.txt works too16:42
salgadobdmurray, I thought that maybe I could have a plugin which was causing chromium to not send the referrer, but I see the same behaviour on epiphany, which has no extensions here16:46
salgadobdmurray, the last rev on your branch is r11487, correct?16:47
bdmurraysalgado: I just pushed 11488 which fixed the Continue test in the previously mentioned test16:48
lifelesssee you on the flip side16:54
bigjoolsjml: I thought of another thing we need to sort out in the async world of buildd-manager code - the log messages are going to look *weird* out of order, so they need vastly improving with more context :)16:55
flacostegmb: who is the release manager for 10.10?17:09
gmbflacoste: Edwin.17:10
flacostegmb: thanks! and hi!17:11
gmbflacoste: Hi! Welcome back :)17:11
=== Ursinha is now known as Ursinha-lunch
henningedanilos: can have a look at this, please? http://paste.ubuntu.com/502711/17:21
daniloshenninge, I don't like the orange colour on the web page17:22
henningedanilos: This error comes straight from gettextpo.17:22
=== benji is now known as benji-lunch
henningeThe difference is "msgid_plural"17:22
daniloshenninge, sounds like what jtv has seen recently with upgrade to maverick as well17:22
henningeah, I was expecting some upgrade in gettext17:23
daniloshenninge, we should probably normalize these into exceptions in pygettextpo and not worry about the text17:23
daniloshenninge, though, not sure how doable it is... where do you see this?17:23
henningein a pagetest17:23
daniloshenninge, locally or? I am assuming it's maverick?17:23
henningeyes, locally on maverick17:24
henningeerror display in pagetests sucks17:24
=== salgado is now known as salgado-lunch
henningeI'll just add some ... to get the test passing on both maverick and older ...17:31
henningedanilos: ^17:31
daniloshenninge, yeah, it sucks, though I've been fighting my test runner today as well17:31
daniloshenninge, it'd be very useful to try it out on both (you can run just a single test on ec2)17:32
=== deryck is now known as deryck[lunch]
henningedanilos: I want a full run now, anyway.17:40
bigjoolshenninge: it sucks when people do "print browser.contents" and then match with an ellipsis ... :(17:46
danilosbigjools, oh, you should have seen most of our pagetests from back in the day then, you'd love it17:51
bigjoolsdanilos: dude, I work in Soyuz .... :)17:54
=== Ursinha-lunch is now known as Ursinha
henningebigjools: I replaced it with "extract_text" in this instance ;-)18:07
bigjoolshenninge: did you get the page fragment too? :)18:07
henningeOh, it was already just a tag being tested.18:08
henninge  >>> for tag in find_tags_by_class(user_browser.contents, 'error'):18:08
henninge  ...     print extract_text(tag)18:08
bigjoolsheh :)18:08
=== matsubara is now known as matsubara-lunch
=== benji-lunch is now known as benji
=== salgado-lunch is now known as salgado
jmlheh heh18:24
jmllifeless: lounge?18:24
lifelessdownloading my failed stream so i can try to fix the errors18:25
sinzuijcsackett, you may be partly responsible for the sampledata conflict. Do you want to try to fix it my merging stable into db-devel, then submit the resolution to pqm?18:34
jcsackettsinzui: actually, i just saw those errors, did a clean merge of devel and db-devel on my machine with resolution, and was about to ask you about it.18:34
sinzuistable to db-devel18:34
sinzuijcsackett, we have a step them ensure that db-devel will also merge into stable for CP and rollouts. The failures are saying we cannot do either18:35
jcsackettsinzui: yeah, i follow that; just didn't realize which branch was involved. i'm pulling down stable now to merge--i suspect the resolution step will be the same.18:36
sinzuijcsackett, pull db-devel, merge in production-stable, resolve, push, send to pqm18:36
* lifeless waves18:37
sinzuijcsackett, remember that this is developer data, not test/app data so sending to to ec2 is pointless18:37
jcsackettsinzui: got it. does using lp-land work for that, or is there some special invocation?18:37
abentleyjcsackett: you're not supposed to land database changes to devel/stable.  The one exception seems to be security.cfg18:37
abentleyjcsackett: In this case, you haven't changed the database schema, so you shouldn't have needed to change the sampledata.18:38
sinzuijcsackett, bzr pqm-submit -m "[testfix][r=<review>][ui=none] Resolve conflicts."18:38
=== deryck[lunch] is now known as deryck
jcsackettabentley: i was under the (errant) belief that referred only to schema changes (like adding columns); i had made a change for sample data to illustrate some cases per salgado's request.18:38
jcsackettthis won't happen again. sorry all.18:38
jmljcsackett: sorry our process isn't simpler or at least more self-documenting18:40
sinzuiabentley, I am not sure jcsackett broke rules since his change was for engineers. The data is not used by the test runner18:40
abentleysinzui: If there's not a rule, there definitely should be, because what just happened is what will always happen.18:41
=== flacoste changed the topic of #launchpad-dev to: Launchpad Development Channel | Week 2 of 10.10 | PQM is open for business | firefighting: - | https:/​/​dev.launchpad.net/​ | Get the code: https:/​/​dev.launchpad.net/​Getting
jmlyay more rules18:41
jmloh wait I mean boo.18:41
abentleyjml: simpler rules: "don't change anything in "database/" except security.cfg and current-dev" becomes "don't change anything in "database/" except security.cfg."18:44
jmlabentley: I like simpler.18:44
jcsackettsinzui: okay, i have resolved conflict; you want to take a look at it beforehand?18:45
jcsackettsinzui: hm; diff looks ridiculous. fixing the merge error was just a case of deciding which block of blank lines to keep.18:48
jcsackettsinzui: https://pastebin.canonical.com/37863/18:50
* jcsackett hides in cave.18:52
jcsackettshould i update the wiki describing editing sampledata to include the fact that changes must be submitted to db-devel, not devel?18:55
abentleyjcsackett: please do.18:59
=== al-maisan is now known as almaisan-away
=== matsubara-lunch is now known as matsubara
jcsackettsinzui: https://pastebin.canonical.com/37865/19:23
cr3hi folks, anyone happen to know where/how the "webservice" part of "${context/webservice}" is defined? I looked throughout interfaces and zcmls, but couldn't quite find where/how that attribute or method is defined.19:36
jcsackettabentley, can you take a look at this? https://pastebin.canonical.com/37865/19:37
jcsacketti gather that pqm rejected my merge b/c it found conflict markers, but there appear to be none in my branch or in a local checkout of db-devel if i merge my branch into it.19:38
jcsackett(according to the same test).19:38
abentleyjcsackett: the rejection would have been an attempt to merge stable, not your branch, so that's the closest thing to try.19:39
cr3nevermind folks, found it under lazr.restful stuff19:39
jcsackettabentley: why would it be trying to merge stable if it's trying to merge my testfix?19:40
cr3by the way, what does "LAZR" stand for? Launchpad And Zope REST?19:40
abentleyjcsackett: Oh, I thought you were referring to the original failure.19:40
jcsackettabentley: no, this was a response to my testfix.19:40
jcsacketti know what the conflict was in merging stable; that's what i resolved.19:41
jcsackettwas hoping error code in the paste might be more meaningful to someone else, and give me something to chase.19:41
abentleyjcsackett: What you pastebinned is not what PQM does if there's a merge conflict.19:44
bachi abentley, i used 'bzr lp-land' this morning and it sent my branch off to pqm even though i had uncommitted changes.  most of the other tools check for that, should lp-land too?19:44
abentleyjcsackett: That's a rule that's checking for unwanted database changes.19:44
jcsackettabentley: dig. i'm confused since the error appears to be in make check_merge.19:44
abentleyjcsackett: We don't check for conflict markers, we check the status code of the merge command.19:45
jcsackettabentley: ah, i see. the test i see when running make check_merge is test_no_conflict_markers, which is where i made the guess.19:46
abentleyjcsackett: this is checking that the outcome of the merge is reasonable.19:46
abentleyjcsackett: not checking whether the merge succeeded.19:46
abentleyjcsackett: Okay, maybe we *also* check for conflict markers, but those would be markers that got there because someone committed them.19:47
jcsackettabentley: okay. so then, is my supposition that what is failing in the paste happens after pqm is merging the testfix into db-devel?19:48
jcsackettabentley: i'm tryin to chase down the error so i can figure out what in the testfix is unpalatable and fix it.19:48
abentleyjcsackett: yes.  After the merge, it's running make check_merge19:48
jcsackettso, theoretically, if i have a clean branch of db-devel, merge my testfix into it, and run make check_merge, if all is well locally i shouldn't be seeing that error?19:49
abentleyjcsackett: You should try doing the merge locally and then running "make check_merge" locally.19:49
abentleyjcsackett: yes.19:49
jcsackettabentley: excellent. that's what i did, and the make check_merge passes locally.19:49
jcsackettabentley: i suppose not 'excellent', as that removes the most obvious path for a fix.19:50
sinzuijcsackett, sorry. I was in a meeting you are landing in db-devel: pqm-submit --submit-branch=bzr+ssh://bazaar.launchpad.net/~launchpad-pqm/launchpad/db-devel19:50
sinzuijcsackett, bzr before that command and -m "message" after that command19:50
abentleyjcsackett: okay, make sure your db-devel is up-to-date.19:50
sinzuijcsackett, did you branch devel or db-devel as your base?19:52
jcsackettsinzui: db-devel.19:52
abentleybac: I guess that would make sense, to be in line with pqm-submit.19:52
sinzuiokay, then I think you are fine19:52
sinzuijcsackett, if I gave you my bazaar aliases, then dbsubmit  does the pqm-submit --submit-branch=bzr+ssh://bazaar.launchpad.net/~launchpad-pqm/launchpad/db-devel19:53
bacabentley: yeah, it would be nice.  i just put us in [testfix] b/c i made a change to a failed test but then sent it off to PQM without commiting the change.  :(19:53
bacabentley: i'll open a bug19:53
jcsackettsinzui: thanks.19:54
jcsackettabentley: thanks as well.19:55
jmlg'night all20:06
rockstarHas the upload server for PPAs changed?  I can't ping upload.launchpad.net20:25
rockstarUgh, it's ppa.launchpad.net20:27
brycehin a configure.zcml, when should I make my classes class="lp.bugs.model.*" vs. class="canonical.launchpad.database" ?20:31
salgadobryceh, always the former. canonical.launchpad.database is deprecated and at some point everything there will have been moved somewhere else20:34
brycehsalgado, ok thanks20:34
* rockstar goes to find lunch20:45
=== salgado is now known as salgado-brb
mwhudsonwhat does 'dissolve cornflakes' mean as a commit message?21:41
thumperI saw that too21:41
bachi thumper21:42
thumperI think it was a merge conflict resolution21:42
thumperhi bac21:42
bacthumper: the branch i landed redesigned the a project's code page to show whether the project uses launchpad, and if it does not to show where the code is hosted21:43
thumperbac: so what change the capitalisation?21:43
bacthumper: part of that design was to add side portlets to the page to bring it into the 3.0 UI world21:43
bacthose statements went from in-line to being in a portlet21:43
* thumper is not happy21:44
thumperwho reviewed it?21:44
bacthumper: salgado, allenap, sinzui, and mrevell21:44
bacthumper: look at screenshots at http://people.canonical.com/~bac/code_usage/21:44
maxbuhoh, we have code imports failing with NoWhoami again21:45
thumperbac: what about personal branches? distro branches?21:47
thumperbac: the style of those will now be inconsistent?21:47
thumperbac: also, I don't see the point of saying that branches will be public initially21:48
thumperbac: no-one can make public branches private21:48
mwhudsonmaxb: eek21:49
mwhudsonmaxb: particularly as it's failed on each importd :/21:49
=== salgado-brb is now known as salgado
mwhudsonlosa ping21:49
bacthumper: look at curtis' last comment on https://code.edge.launchpad.net/~bac/launchpad/bug-643538-code/+merge/3637721:50
bacthumper: the one from jml specifically asking for consistency across tabs for projects21:50
mbarnettheya mwhudson21:51
mwhudsonmbarnett: has anything happened to the ~importd/.bazaar/bazaar.conf files on the import slaves recently?21:52
=== Ursinha is now known as Ursinha-bbl
* thumper sighs21:52
thumperbac: so... back to the consistency thing then21:52
thumperbac: have the other branch listings been changed?21:52
bacthumper: no21:52
maxbmwhudson: Yes, I think *ALL* cscvs imports are currently failing, but only when the upstream repository has a new commit. And this is why the entire collection of CSCVS imports haven't migrated to failed already21:53
* thumper agrees with maxb21:53
mwhudsonmaxb: this is strange, we definitely had them working at some point since the nowhoami behaviour got rolled out21:53
maxbSo why is this not affecting non-CSCVS imports? :-/21:54
thumpermaxb: all the others use a common base class21:54
thumpermaxb: which I think has been fixed for the no-whoami thingy21:54
mwhudsonit's because they use a different api to create commits21:54
mwhudsonat least, i think21:55
mwhudsoncreate revisions rather21:55
thumpermwhudson: there was a bzrlib bug where the method was ignoring the param21:55
mwhudsonthumper: ah21:55
thumpermwhudson: so you still had to set BZR_EMAIL env21:55
maxbThis problem arose between 2010-09-08 and 2010-09-1221:56
thumpermwhudson: also, about the incorrectly stacked distro branches21:57
thumpermwhudson: we can fix it by running bzr reconfigure --unstacked on them21:57
thumpermwhudson: I'm trying to avoid writing a "special" fixit script21:57
mwhudsonthumper: seems fair enough21:57
thumperwallyworld___: morning21:58
sinzuithumper, I agree with you22:05
thumpersinzui: which bit?22:05
thumperabentley, rockstar, wallyworld___: standup?22:06
sinzuithumper, I reported bugs on blueprint and malone when I saw project was changed when the feature was about specifcationtarget and bugtarget. branch collections/target must be the on on launchpad and behave like all launchpad22:06
abentleythumper: okie22:06
thumpersinzui: I don't understand what you just said22:07
sinzuithumper, no 2.0 designs22:07
sinzuithumper, do not make IProduct an exception22:07
mbarnettmwhudson: sorry, overly distracted over here with a db issue.22:07
mbarnettmwhudson: will have to look at that in a few22:08
bacthumper: i think sinzui is saying we'll update the other pages as soon as possible.  is that right curtis?22:08
sinzuiyes. If someone else does not report the bug I will22:09
sinzuibesides jml, will not end bridging-the-gap until his vision of consistent UI and clarity of message is met22:09
=== matsubara is now known as matsubara-afk
rockstarwallyworld___, ping me when you're ready. :)22:47
wallyworld___rockstar: do you want to talk now or a bit later? if later, i'll go grab a coffee and breakfast22:48
rockstarwallyworld___, have breakfast first.  I can wait.22:51
wallyworld___rockstar: i saw your message after i sent mine. i can talk now if you like. might be good to get it done22:51
rockstarwallyworld___, if you're good to go, I am.22:51
wallyworld___rockstar: skype?22:52
=== salgado is now known as salgado-afk
rockstarwallyworld___, http://developer.yahoo.com/yui/3/io/23:10
james_wrockstar: merged your bzr-builder branch, apologies for the delay23:21
james_wwhile there I found a case where a bug was corrupting the text of recipes23:21
james_wbut there was in fact not enough checking in __str__ to catch the problem23:22
rockstarjames_w, ah!  Thank you for doing that.23:23
pooliehi rockstar, james_w23:24
rockstarMorning poolie.23:24
james_whi poolie23:24
james_whttps://code.edge.launchpad.net/~james-w/bzr-builder/fix-text-of-nest-parts/+merge/37076 if someone wants to review23:24
* rockstar walks dog23:26
* thumper afk for brief shopping23:29
james_werm, helps if I actually commit23:37
james_wok, third time lucks23:52
pooliehi mars? happy to help with your feature flag test if i can23:52
james_whttps://code.edge.launchpad.net/~james-w/bzr-builder/fix-text-of-nest-parts/+merge/37080 if someone wants to review23:52
marspoolie, sure: I would be glad for the help: https://code.edge.launchpad.net/~mars/launchpad/add-py25-lint23:54
marspoolie, this test fails: bin/test canonical.launchpad -t profiling.txt23:54
marspoolie, I have to run, but I can check back later23:54

Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!