[00:01] <StevenK> wgrant: Good enough, or you'd prefer other changes?
[00:02] <wgrant> Looks quite reasonable to me.
[00:02] <wgrant> Does it work?
[00:02] <StevenK> The tests all pass
[00:02] <StevenK> And I think I wrote enough of them. :-)
[00:09] <StevenK> wgrant: https://code.launchpad.net/~stevenk/launchpad/bugs-information_type-mail/+merge/104483 has been updated.
[00:14] <wgrant> StevenK: r=me
[00:59] <mwhudson> jcsackett, benji: ayt?
[01:02] <mwhudson> oh never mind
[01:45] <wallyworld> wgrant: when do you expect use_flat to be ripped out?
[01:46] <wgrant> wallyworld: Of get_bug_privacy_filter? Undefined.
[01:46] <wgrant> Hopefully within a week, but it's a bit hard to tell.
[01:46] <wallyworld> wgrant: ok. i want to use get_bug_privacy_filter() but it has no use_flat paparm; i'll add one
[01:48] <wgrant> wallyworld: I have one added in one of my local branches, but it's like two lines and your branch will probably land before mine, so go ahead.
[01:48] <wallyworld> rightio
[01:48] <wallyworld> or if yours lands first, i'll deal with it. i have a few more branches to go yet
[01:49] <wgrant> Yeah
[01:49] <wgrant> It's going to be a pretty trivial conflict.
[01:52] <wallyworld> wgrant: it seems the gbpf doesn't have a term which queries on bugtaskflat.access_policies, only access_grants
[01:53] <wgrant> wallyworld: Right, at present it doesn't use access_policies, because there are no policy grants.
[01:53] <wgrant> There's a card in Next about making it do that
[01:53] <wgrant> It's 3 lines of code, and if that function's used everywhere it'll apply everywhere
[01:54] <wallyworld> are there really no policy grants?
[01:54] <wgrant> There are no policy grants yet.
[01:54] <wgrant> That's an All permission
[01:54] <wallyworld> on qas there are :-)
[01:54] <wgrant> Which don't exist on prod, because the UI isn't write-enabled.
[01:55] <wgrant> True
[01:55] <wallyworld> wgrant:  so if i want to use gpbf i will need that extra functionality. so i will have to pick up that card from the board and do that
[01:55] <wgrant> wallyworld: Why?
[01:55] <wallyworld> i need it for the remove subscriptions job
[01:56] <wgrant> No.
[01:56] <wgrant> The current definition of access to bugs doesn't currently include policy grants.
[01:56] <wallyworld> yes, it was even in the query you pasted
[01:56] <wgrant> It was, yes.
[01:56] <wgrant> That'll be the final thing.
[01:56] <wallyworld> sure, but it's still correct to use it
[01:56] <wgrant> But if you're going to use the existing function, you don't have to care about that stuff.
[01:56] <wallyworld> i will be writing tests which require it or else trhey will fail
[01:57] <wgrant> Ah, right, it could make tests slightly awkward, indeed.
[01:57] <wgrant> You can steal the clause pretty much verbatim from the query I pasted yesterday.
[01:57] <wgrant> And stick it into the function.
[01:57] <wallyworld> yep
[01:57] <wgrant> And it will Just Work™
[01:57] <wallyworld> should do
[01:57] <wallyworld> hopefully
[01:59] <wallyworld> wgrant: is there a bug number?
[02:00] <wgrant> Don't think so
[02:01] <wallyworld> ok, will raise one
[02:01] <wgrant> You can probably find a bug search test to extend to cover APGs.
[02:02] <wallyworld> hope so :-)
[02:17] <wallyworld> wgrant: i can't find any bug search tests which create an artifact grant to give visibility; all the current tests seem to use subscription and reply on the triggers to update the artifact grant table. is that correct?
[02:37] <wgrant> wallyworld: That's correct. Everything still assumes that subscriptions are the only things that confer access to bugs.
[02:37] <wgrant> Now that legacy search is gone that can start to change.
[02:37] <wallyworld> wgrant: ok. i've found a doc test in bugtask.txt which i can add to i think
[02:38] <wgrant> (there was no performant way to respect grants/policies in legacy search, so flat search was a prereq for all this)
[04:46] <wallyworld_> wgrant: can you see why this test isn't working? the new one is the one at the bottom https://pastebin.canonical.com/65516/
[04:48] <wallyworld_> it fails because it still returns just the 3 bugs. perhaps it's not using the flat search?
[04:54] <wgrant> wallyworld_: Non-flat search doesn't exist any more.
[04:54] <wgrant> So it's probably using flat search.
[04:54] <wgrant> wallyworld_: The bug isn't PROPRIETARY
[04:54] <wgrant> wallyworld_: So the grant doesn't grant access to anything.
[04:54] <wallyworld_> ah, USERDATA
[04:55] <wallyworld_> doh
[04:55] <wallyworld_> thanks
[04:55] <wgrant> You'll want to ensure/find that, rather than create.
[04:55] <wgrant> But yes.
[04:55] <wgrant> USERDATA
[04:55] <wallyworld_> yep
[04:58] <StevenK> wallyworld_: Are you reviewing today?
[04:58] <wallyworld_> StevenK: yes
[04:58] <StevenK> wallyworld_: I have an MP up
[04:58] <StevenK> It's only 221 lines, disappointing.
[04:58] <wallyworld_> StevenK: sorry, wasn't running thunderbird
[04:59] <StevenK> wallyworld_: Given your recent RAM issues ... :-P
[04:59] <wallyworld_> yep :-(
[04:59] <StevenK> I don't see why.
[04:59] <wallyworld_> StevenK: i have to run out for 30mins to get kid from school
[05:00] <wallyworld_> and buy ram on the way home
[05:00] <StevenK> I have 4GiB of RAM with 64-bit, and I'm only 300MiB into swap with firefox, thunderbird, quod libet, gvim
[05:01] <StevenK> wallyworld_: Given wgrant has wedged the deployment pipeline, it's not urgent.
[05:01] <wallyworld_> ok. back soon.
[05:01] <StevenK> I don't think I'll finish the garbo job this afternoon, sadly.
[05:02] <StevenK> I wonder if the branch garbo job is doable using set
[05:11] <wgrant> It should be, es.
[05:11] <wgrant> yes
[05:11] <wgrant> Since everything will just go to USERDATA
[05:11] <StevenK> Everything?
[05:12] <StevenK> It's either PUBLIC or USERDATA
[05:12] <wgrant> Although I don't know if set() respects limits these days
[05:12] <wgrant> Everything private.
[05:12] <wgrant> Hm.
[05:12] <wgrant> There may be few enough branches that a manual UPDATE is quick enough, too.
[05:12] <StevenK> 500k on DF when I checked
[05:12] <StevenK> And lifeless will smack us
[05:14] <StevenK> Not sure how to use .set() against what I get back :-(
[05:21] <StevenK> wgrant: Is "CASE WHEN Branch.explicity_private IS NOT NULL OR Branch.transitelvy_private IS NOT NULL THEN USERDATA ELSE PUBLIC" expressible in Storm?
[05:22] <wgrant> StevenK: No, use an SQL().
[05:22] <wgrant> Also, that's redundant.
[05:22] <StevenK> Oh?
[05:22] <wgrant> explicitly_private implies transitively_private
[05:22] <wgrant> And they're not likely to be NULL
[05:22] <wgrant> They're boolean.
[05:22] <StevenK> Oh, so just check transitively_private IS TRUE ?
[05:23] <wgrant> s/ IS TRUE//
[05:24] <StevenK> So if I need to use SQL, I need the entire statement. Bleh.
[05:46] <wallyworld_> StevenK: you will export IBranch.transitionToInformationType() later?
[05:46] <StevenK> wallyworld_: After it does more than just blindly setting the information_type, yes.
[05:46] <wallyworld_> ok
[05:55] <wallyworld_> StevenK: i'm wondering if there needs to be a test like test_information_type_private_team to check that info type = public when required
[05:56] <StevenK> wallyworld_: There is, that's why there is a few self.assertEqual(InformationType.PUBLIC, ...) scattered around the place.
[05:56] <wallyworld_> ok
[05:57] <wallyworld_> r=me, thanks
[05:57]  * wallyworld_ sighs. sometimes lp is so slow at updating diffs :-(
[05:58] <StevenK> 528? When did that happen?
[05:59] <wgrant> Mine, last night.
[05:59] <wgrant> 526 527 were already taken
[05:59] <StevenK> Ah
[05:59] <wgrant> I suspect by stub test images, but that's merely a guess
[05:59] <stub> didn't one of them go live?
[06:00] <StevenK> wgrant: Do you think it's worthwhile adding self.log.debug() to the garbo job about progress?
[06:00] <wgrant> StevenK: The looptuner should be enough output
[06:00] <wgrant> At DEBUG2
[06:00] <wgrant> I think we run it at DEBUG2, too
[06:00] <StevenK> Right, then I'll bug wallyworld_ again. :-)
[06:00] <wgrant> stub: Your 525 was live.
[06:03] <wgrant> wallyworld_: r=me
[06:03] <wallyworld_> thanks!
[06:09] <StevenK> Haha, 3 underscore Ian
[06:09] <StevenK> wallyworld_{,__}: I have another one for you.
[06:09] <wgrant> wallyworld_: A pretty quick dev-only one for you, if you still have time: https://code.launchpad.net/~wgrant/launchpad/easier-remote-access/+merge/104684
[06:09] <wgrant> Heh
[06:09] <wallyworld___> me finds a screwdriver to add extra ramyou're so kind
[06:09] <wallyworld___> sure
[06:10] <StevenK> wgrant: You can review the garbo job branch, if you wish, it's +38
[06:11] <wgrant> I don't want to steal wallyworld________'s new non-swapping thunder.
[06:11] <wgrant> But I might anyway.
[06:11] <StevenK> Hahaha
[06:11] <wgrant> StevenK: Your linebreaks are criminal, sqlvalues is evil, and you don't test public.
[06:11] <wgrant> Also self.transaction wut?
[06:12] <wallyworld___> wgrant: the branch looks ok but i'm not really sure i know enough to give it a proper +1
[06:12] <wgrant> The worst it can do is break new dev installations :)
[06:12] <wgrant> And it doesn't break new dev installations.
[06:12] <wallyworld___> WCPGW
[06:13] <wallyworld___> r=me
[06:13] <wgrant> Thanks.
[06:13] <wallyworld___> np
[06:13] <StevenK> wgrant: Without sqlvalues, it puts in 'User Data' which doesn't help
[06:13] <wgrant> StevenK: Use the ?
[06:13] <wgrant> SQL("foo ? bar ?", params=('whatever', 'oh look something else unsafe'))
[06:14] <wgrant> Hm
[06:14] <wgrant> Although it's possible that won't work for a dbenum
[06:14] <wgrant> In which case I guess sqlvalues will have to do.
[06:14] <StevenK> I can try ? if you wish
[06:14] <wgrant> Or you could just use params= with .value directly. That's probably best.
[06:16] <wgrant> Oh good
[06:16] <wgrant> There's a third copy of those hostnames in lp-dev-utils
[06:17] <StevenK> Hahaha
[06:17] <StevenK> wgrant: http://pastebin.ubuntu.com/966263/
[06:17] <StevenK> wgrant: If it isn't duplicated it might be forgotten!
[06:19] <wgrant> StevenK: Your linebreaking in the first hunk is still likely to draw criminal charges.
[06:19] <StevenK> wgrant: http://pastebin.ubuntu.com/966266/
[06:20] <wgrant> Deindent line 13 and you're good
[06:31] <StevenK> wgrant: Diff updated, please have a look.
[06:34] <wgrant> StevenK: Branch.information_type is unindexed.
[06:34] <StevenK> Hmmm
[06:34] <wgrant> I sense a 2-statement DB patch in your fu=ture
[06:34] <StevenK> Haha
[06:35] <StevenK> wgrant: Two?
[06:36] <lifeless> index + metadata
[06:36] <wgrant> CREATE INDEX and INSERT INTO LDR
[06:36] <StevenK> Yah
[06:41] <jtv> I think devel's versions.cfg still relies on some zope versions that are too old for Precise or something.  I can't build on Precise, without bumping a bunch of versions in versions.cfg to the ones I actually have.
[06:44] <wgrant> jtv: What's the error?
[06:44] <wgrant> jtv: Perhaps it breaks if you have some old versions installed through apt (for maas)?
[06:44] <wgrant> Otherwise you should use LXC.
[06:44] <jtv> Error: Couldn't find a distribution for 'zope.publisher==3.12.0'
[06:44] <jtv> etc.
[06:45] <wgrant> Sure your download-cache is up to date?
[06:45] <jtv> No
[06:45] <jtv> I only have newer versions than what's in devel's versions.cfg.
[06:46] <wgrant> wgrant@lamuella:~/launchpad/lp-branches/devel$ bzr ls download-cache/dist | grep zope.publisher
[06:46] <wgrant> download-cache/dist/zope.publisher-3.12.0.zip
[06:46] <wgrant> download-cache/dist/zope.publisher-3.12.1.tar.gz
[06:46] <wgrant> wgrant@lamuella:~/launchpad/lp-branches/devel$ bzr ls download-cache/dist | grep zope.publisher
[06:46] <wgrant> download-cache/dist/zope.publisher-3.12.0.zip
[06:46] <wgrant> download-cache/dist/zope.publisher-3.12.1.tar.gz
[06:46] <wgrant> Your download-cache must be bad
[06:46] <jtv> Bad in what way?
[06:46] <jtv> Bear in mind that I've been running Precise since January, and so haven't been able to run any Launchpad setup since then.
[06:47] <jtv> But I seem to get past this if I edit versions.cfg and bump z3c.ptcompat from 0.5.3 to 0.5.5; zope.app.component from 3.8.3 to 3.8.4; and so on.
[06:47] <StevenK> stub: Can you have a look at https://code.launchpad.net/~stevenk/launchpad/db-branch-information_type-index/+merge/104689 ?
[06:48] <StevenK> I can't seem to request a review of lifeless, but it's one index.
[06:48] <wgrant> jtv: Which zope.publisher is in download-cache/dist in the branch you're trying to build?
[06:48] <wgrant> jtv: Perhaps try make clean
[06:48] <wgrant> jtv: It may still think it's using 2.6 or something
[06:49] <jtv> Crap — even “make clean” fails
[06:49] <jtv> rm: cannot remove `sourcecode/pygpgme/gpgme/*.so': Too many levels of symbolic links
[06:49] <stub> StevenK: r=stub. That should go live?
[06:49] <wgrant> jtv: I think your download-cache, eggs, sourcecode etc. symlinks might be a little screwed.
[06:50] <StevenK> stub: As long as backups aren't in our way, yup.
[06:50] <jtv> They may well be.  I haven't been able to run the rocketfuel scripts for so long.
[06:50] <wgrant> They haven't changed since 2009, but maybe.
[06:51] <jtv> The scripts maybe, but I'm sure the data has!
[06:51] <jtv> load_cache fails: No JSON object could be decoded
[06:51]  * StevenK stabs Level3
[06:52] <jtv> Trying more make clean & version bumps.
[06:53] <StevenK> jtv: make clean and then cd into your download-cache and bzr up
[06:53] <jtv> The download-cache in the branch?
[06:53] <StevenK> Which is a symlink
[06:55] <jtv> $ bzr up
[06:55] <jtv> Tree is up to date at revision 465 of branch bzr+ssh://bazaar.launchpad.net/%2Bbranch/lp-source-dependencies
[06:56] <jtv> Let's see what “make” does now.
[06:56] <StevenK> stub: DB patch is on db-devel r11569.
[06:56] <stub> ta. I've added it to my todo for when the backups complete
[06:56] <StevenK> stub: Thanks.
[06:56] <StevenK> wgrant: ^ Can haz approve for garbo job *now*? :-)
[06:58] <jtv> Wow, I managed to have a conflict in sourcedeps.cache.
[06:58] <StevenK> Unsurprising
[06:58] <jtv> Really?  And I was so proud.
[06:58] <wgrant> StevenK: Why db-devel?
[06:58] <wgrant> StevenK: Live patches go to devel
[06:59] <StevenK> Bleh
[06:59] <StevenK> wgrant: Merge it after Ian's FDT, then?
[06:59] <wgrant> I guess.
[07:00] <jtv> \o/ I can “make.”  Thanks guys.
[07:00] <wgrant> It's the first step toward recovery :)
[07:01] <jtv> Our Makefile really needs work again.  I had it doing parallel tasks, and not rebuilding mailman and css/js every time you rebuild the db schema at one point.  :(
[07:01] <StevenK> jtv: wallyworld___ has added things to prevent mailman building
[07:02] <jtv> I so hope I wasn't wrong in bug 994410.
[07:26] <wallyworld___> StevenK: what did i do?
[07:26] <wgrant> Hmm
[07:26] <wgrant> Do I want to teach ec2 to preserve the buildout egg cache?
[07:27] <noodles785> Hi people, is a tag added to an LP bug as soon as it is deployed to staging or qastaging? (we'd like to do integration testing as soon as bug 992691 lands on staging).
[07:27] <_mup_> Bug #992691: Special permissions for 'Archive.commercial' are not needed <tech-debt> <Launchpad itself:In Progress by jml> < https://launchpad.net/bugs/992691 >
[07:28] <wgrant> noodles785: Yes, it will have qa-needstesting added a few minutes after it's on either qastaging or staging. In this case you'll need to wait up to 12 more hours for it to hit staging, since it'll be on qastaging first.
[07:28] <noodles785> wgrant: the egg cache is left pristine isn't it? I don't see why not.
[07:28] <noodles785> wgrant: perfect, thanks!
[07:28] <wgrant> However, the branch failed ec2 a couple of hours back. It's trivial to fix.
[07:28] <wgrant> Also, you've been incremented by 10?
[07:29] <noodles785> 10? 10 credits?
[07:29] <wgrant> Ypi
[07:29] <wgrant> Bah
[07:29] <wgrant> You've traditionally been noodles775
[07:29] <noodles785> Hah, oh.
[07:30] <wgrant> noodles775: So, hopefully jml will be around to fix the test failure tonight. If not, I'll do it. Either way I'll reland it this evening, and it will be on staging when you start on Monday.
[07:30] <noodles775> wgrant: great, thanks!
[07:30] <wgrant> It's about 7-12 hours from landing to qastaging, and then another 13-20 after that until staging.
[07:31] <wgrant> This'll hopefully be down to 90 minutes and 120 minutes once parallel testing is complete.
[07:45] <adeuring> good morning
[07:51] <czajkowski> aloha
[07:51] <cjwatson> jtv: was bug 994410 supposed to be on LP rather than MAAS?
[07:56] <jtv> cjwatson: absolutely
[07:56] <jtv> And I work so hard to catch chromium at that.
[07:56] <jtv> Thanks.
[10:12] <jtv> Reviewer needed!  https://code.launchpad.net/~jtv/launchpad/bug-994410-stop-using-latest_message/+merge/104703
[10:13] <czajkowski> jtv: is there anything we need to be aware of when adding a new language in Ubuntu/lp
[10:13] <jtv> Yes: there are several very different things that people call “adding a language.”
[10:14] <czajkowski> https://answers.launchpad.net/launchpad/+question/195970
[10:14] <czajkowski> wanted to add BRX
[10:16] <jtv> czajkowski: these people have really done their research.  It answers all questions I might have; you're ready to go.
[10:17] <czajkowski> so the next bit is I've not added a langauge before and don't know where in lp I do this
[10:17] <czajkowski> so not sure I can  also :/
[10:17] <jtv> I don't think there's an ISO 639-2 code, so the 639-3 is fine.  It's an Individual language code, not a Collective which we can't accept.
[10:18] <jtv> This is properly "adding a language" in Launchpad.  Do you see the option just under “Available languages in Launchpad”?  https://translations.launchpad.net/+languages
[10:19] <czajkowski> jtv: yes
[10:19] <jtv> Well there you go then.  You've got the plural expression, language code, English name…
[10:20] <czajkowski> thank you
[10:21] <jtv> Please convey to them my appreciation for the care and effort they put into this request.
[10:21] <jml> hello
[10:21] <jml> I'm around to fix the test failure.
[10:21] <wgrant> jml: Did you get the email?
[10:21] <jml> wgrant: I think so.
[10:21] <wgrant> I didn't realise my suggestion was so offensive to the test suite :(
[10:24] <czajkowski> jtv: thank for the help.
[10:25] <jtv> no worries
[10:25] <jtv> czajkowski: don't forget to enter the plural-form data!
[10:25] <jml> wgrant: yeah.
[10:26] <jml> wgrant: I like that we have auto-generated tests for IPersonRole, but that we don't auto-generate it ourselves.
[10:26] <jtv> czajkowski: there's 2 plural forms, and the formula is “n != 1” (I think entering it in that form will work)
[10:26] <wgrant> jml: That is pretty cool, yeah.
[10:28] <jtv> stub: I want to drop a column from POFileTranslator, and stop the TranslationMessage trigger from writing it.  Would it be best to do that in one patch?  Or a sequence of smaller steps?
[10:29] <czajkowski> jtv: am trying but I keep getting an oops
[10:29] <jtv> Then I suppose the format isn't quite right.  Have a look at other languages; see if you can crib the format.
[10:29] <wgrant> jtv: One patch is fine, as long as the appservers have already stopped going near it.
[10:30] <jtv> OK thanks
[10:30] <jtv> Speaking of which, here's my MP for achieving that.  :)  https://code.launchpad.net/~jtv/launchpad/bug-994410-stop-using-latest_message/+merge/104703
[10:32] <stub> jtv: There is no problem doing both the column drop and the trigger change in one patch. Both are fast operations.
[10:33] <jtv> Thanks.  Just wondering if there were multi-server rollout implications.
[10:33]  * jml fixes now
[10:34] <stub> jtv: We *might* be able to twiddle the trigger live. The column drop has to go through fast downtime. Easier for everyone if it can go in one patch.
[10:34] <czajkowski> jtv: done
[10:34] <jtv> stub: very well
[10:35] <jtv> czajkowski: looking good
[10:35] <czajkowski> jtv: sorry about that peksy oops :)
[10:35] <jtv> Oh, the code change?  Didn't take long to figure out.  :)
[10:36] <jtv> stub: I suspect https://dev.launchpad.net/Database/LivePatching is far out of date
[10:37] <jml> hey
[10:37] <jml> are there any webservice tests written as unit tests?
[10:38] <jml> is there any non-historical reason for writing them as doctests?
[10:38] <jml> they clearly don't function as documentation for the actual users of the webservice.
[10:38] <stub> jtv: Was this an example of a CRITICAL (genuine) bug drowned out by CRITICAL (policy) bugs?
[10:39] <cjwatson> jml: lib/lp/soyuz/browser/tests/test_archive_webservice.py comes to mind
[10:39] <cjwatson> (Quite a few others, 'find -name \*webservice\*')
[10:39] <jml> cjwatson: thanks.
[10:40] <jtv> stub: arguably — in any case, we never actually seem to have made a decision to drive critical bugs to zero; we made a decision to allocate them some time, we announced that that would drive them to zero, and then we instated a policy for creating new ones at a faster rate.  :)
[10:40] <jml> :(
[10:40] <wgrant> cjwatson, jml: Note that those tests use launchpadlib, and launchpadlib tests are *very* slow.
[10:41] <wgrant> Most of the doctests avoid launchpadlib. I'm not sure if there are non-doctests that do.
[10:41] <jml> Hmm.
[10:41] <stub> Just thinking we knew this was a timebomb and we let it explode creating more work (cleanup)
[10:41] <jtv> stub: absolutely.
[10:42] <jtv> And the flood of policy-critical bugs gave us easier things to fix instead.
[10:44] <jtv> If you want to fill a bucket of dirty water with clean water, you can pour it out first and then refill it; or you can put it under the tap.  But the latter is something you only do if you can't stand to be without a full bucket for a while.
[10:44] <jml> wgrant: ok, that's fixed. r15203
[10:44] <stub> jtv: Are these tests you removing going to have to be rewritten when we maintain the latest translator asynchronously?
[10:44] <jtv> There will be other tests, but much simpler.
[10:45] <jtv> There won't be any complications from how data got where it is; it's either there or not there.
[10:45] <jtv> Dragging the data through the same changes as other data is what makes it difficult.
[10:50] <wgrant> jml: It's in devel.
[10:50] <jml> wgrant: oh, you just landed it directly?
[10:50] <wgrant> jml: Yeah
[10:50] <wgrant> No point re-ec2ing it
[10:51] <jml> wgrant: I guess not.
[10:55] <jtv> Thanks for the review stub
[10:56] <StevenK> Bah. Forgot the --incremental for r15204. :-(
[10:56] <jtv> Er… what's the current landing procedure?  I've been away for a while.
[10:57] <wgrant> jtv: bin/ec2 is now in lp:lp-dev-utils
[10:57] <wgrant> So grab that, put it in your PATH, 'ec2 land'
[10:57] <jtv> Thanks.
[11:41] <jtv> Grar.  The bugs that come out of the woodwork when you start simplifying POFileTranslator.  Should've done it years ago.
[11:54] <jml> jtv: I know what you mean.
[11:54] <jtv> You've looked at it?  Or seen the same thing elsewhere?
[11:54] <jml> jtv: I don't think I've ever thought, "man, I'm glad I postponed cleaning that up, that made everything go so much better and faster"
[11:54] <jml> jtv: same thing elsewhere.
[11:54] <jtv> Heh.
[11:55] <jtv> To be honest, I think by nature _not_ cleaning up pays off beforehand, unlike cleaning up which pays off afterwards.
[11:56] <jtv> But I don't think we've been sorry about having cleaned things up very often.  :)
[11:58] <jml> jtv: well, practically every time I've got some code and my understanding doesn't quite match it (or there's a clean up I can see), whenever I do finally fix it up, I *always* think I should have done it earlier.
[11:58] <jml> mostly because it would have made everything between the realization and the cleanup easier.
[11:59] <jtv> Then I wonder: if you adjust to that and clean up earlier, what would be the signal that you've overshot?  Or is there no such thing?
[12:03] <jtv> (I'm not entirely un-selfish in asking)
[12:04] <cjwatson> Sometimes cleanup is blocked on something else, and waiting means the cleanup is "delete big pile of stuff" rather than "spend two weeks fixing other stuff and then delete stuff".
[12:05] <jtv>  My current situation in a nutshell.  :)
[12:05] <jtv> But with a twist: I found a kind of circular dependency.
[12:05] <jtv> Which reminds me that I should keep dpm and Deryck updated.
[12:06] <jml> jtv: hmm.
[12:07] <dpm> heya jtv, what's up?
[12:07] <jtv> Speak of the  :)
[12:07] <jml> jtv: perhaps decreased certainty about what to clean up?
[12:07] <jtv> dpm: I'm just dealing with some breakage during the Quantal opening.
[12:07] <jtv> jml: something to keep an eye out for next time.  :)
[12:08] <jtv> dpm: no worries, we'll get it done.  But I've gone off on a bit of a tangent cleaning things up that made it hard to fix these problems we've been having.  And I found some things.
[12:09] <jml> jtv: sometimes I get a certain circular mental motion / paralysis when I'm trying to clean things up when I really should just solve the damn problem.
[12:09]  * dpm worries that "things" are not "good things"
[12:09] <jtv> dpm: Well the bad things have already happened and now we can work on fixing them.  Some deeply obscure PLPSQL code looks to me like this would happen:
[12:10] <jtv> jml: also a good thing to keep an eye out for then.  I'll try to remember.
[12:10] <jtv> dpm: say you have two POFiles that share.
[12:10] <jml> sorry, figuring this out myself now
[12:10] <jtv> And there's a record of your latest translation in one of them, but not in the other.
[12:11] <jml> I think the tighter the loop, the less cost in overshooting either way
[12:11] <jtv> Probably.
[12:11] <jtv> dpm: now you enter a new translation, in either of them.
[12:11] <jtv> dpm: the obscure code is the code that keeps track of who's contributed to which POFiles.
[12:12] <jtv> And it says “oh good, I've just updated an existing contribution record (POFileTranslator is the actual table); that means I don't need to create a new one.  I'm done.”
[12:12] <jtv> And so it never creates a similar record for that other POFile — even if that was the one you were translating in in the first plcae!
[12:12] <jtv> *place
[12:12] <jtv> dpm: sound familiar?
[12:13] <jtv> This is what users have been complaining about that we just couldn't figure out.
[12:27] <dpm> jtv, oh, so if I understand it correctly that's the bug we recently talked about?
[12:27] <jtv> dpm: I think it just might be.
[12:59] <rick_h_> anyone know what's up with Error: Couldn't find a distribution for 'pytz==2012c'.
[12:59] <rick_h_> ?
[12:59] <rick_h_> pulled source-deps, not there, reran buildout but can't find it
[12:59] <rick_h_> anyone know where I'm supposed to have gotten it from?
[12:59] <wgrant> rick_h_: download-cache
[12:59] <wgrant> bzr up it
[12:59] <rick_h_> hmm, did but said it was up to date
[13:00] <rick_h_> Tree is up to date at revision 465 of branch bzr+ssh://bazaar.launchpad.net/+branch/lp-source-dependencies
[13:00] <rick_h_> ?
[13:01] <wgrant> 2012c is in there for 2.6 and 2.7
[13:01] <rick_h_> ok, yea see it there, just buildout/make hating on me I guess.
[13:01] <rick_h_> ok, thanks for verifying it's where it should be
[13:04] <rick_h_> wgrant: doh, sorry was updating download-cache in ~/launchpad/ not realizing it wasn't symlinked into devel but was it's own copy
[13:04] <wgrant> rick_h_: Hm, that's meant to be symlinked.
[13:05] <rick_h_> yea, that's what I thought. I know I did some hackery due to precise install fun so probably my fault
[13:05] <wgrant> The default setup is that devel/download-cache is symlinked to ~/launchpad/lp-sourcedeps/download-cache, and all the other branches have theirs symlinked to devel
[13:05] <rick_h_> right
[13:18] <jml> tabs :(
[13:18] <wgrant> jml: Ew, where?
[13:18] <jml> wgrant: soyuz/configure.zcml
[13:19] <wgrant> Just the one.
[13:19] <wgrant> Still, destroy.
[13:21] <jml> wgrant: with enthusiasm
[13:22] <deryck> allenap, hi. You around?
[13:22] <deryck> ah, see the lunch afk message now.  sorry, catch you later.
[13:23] <allenap> deryck: Hi.
[13:23] <allenap> deryck: I forgot to update my status.
[13:32] <deryck> adeuring, rick_h_ -- sorry, got busy chatting and missed it was standup time.
[13:32] <deryck> adeuring, rick_h_ -- be right there.
[13:39] <jml> https://code.launchpad.net/~jml/launchpad/commercial-means-shut-up/+merge/104739
[13:39] <jml> patch to rename IArchive.commercial
[13:44] <rick_h_> deryck: https://code.launchpad.net/~rharding/launchpad/editstatus_timeout_874250/+merge/103912
[13:47] <bac> jml: looking
[13:47] <jml> bac: thanks.
[13:53] <deryck> rick_h_, just ping me when that MP is good to go.  I'm not watching it, so will need the reminder.
[13:54] <wgrant> rick_h_: It's midnight on Friday so I can't really give a proper review, but the _init_new_task thing strikes me as more than a little awkward. Have you considered merging that into createManyTasks, and having createTask just delegate to createManyTasks? That's the pattern we use elsewhere.
[13:58] <rick_h_> deryck: all good
[13:59] <rick_h_> wgrant: ah, good point. I can take a peek at that once I find why my MP isn't working right
[14:00] <wgrant> It should remove a lot of duplicated default stuff, and eliminate the reasonably hideous getattr stuff that this structure requires
[14:04] <rick_h_> yea, gotcha. started out smaller but grew over time, curse of taking too long
[14:05] <wgrant> Yeah, that often happens, particularly when setifying ancient code.
[14:05] <rick_h_> ok, need bzr help, I removed lib/lp/bugs/doc/bugtask.txt
[14:05] <rick_h_> so the file is gone, but hte MP says there's conflicts in it
[14:05] <rick_h_> and doesn't show the file deleted
[14:06] <rick_h_> my bzr branch shows no conflicts, file is gone, push and push --overwrite no helpy, any other ideas?
[14:07] <rick_h_> ah, sec...see this in the MP diff  renamed file 'lib/lp/bugs/doc/bugtask.txt' => 'lib/lp/bugs/doc/bugtask.txt.THIS'
[14:28] <jtv> Should downtime database branches still branch off db-devel?
[14:31] <rick_h_> jtv: I've yet to do one, but I do believe so
[14:33] <wgrant> jtv: Yes, but it doesn't really matter since there's rarely more than in progress at a time.
[14:33] <jtv> Naturally.  Thanks.
[14:33] <wgrant> fastdowntime patches -> db-devel, live -> devel
[14:33] <bac> hi jml -- your branch looks mostly good but i have a policy question about changing the signature of 'createPPA' and removing 'commercial' from being exported.  since those items were introduced in beta, we need to continue supporting them in beta and 1.0.
[14:33]  * wgrant sleeps.
[14:33] <jtv> nn wgrant
[14:33] <rick_h_> see wgrant
[14:33] <jml> bac: we (consumer-apps) are the only users permitted to do that
[14:33] <jml> bac: I myself added the 'commercial' parameter a week or two ago.
[14:34] <wgrant> Yeah, API stability is not a concern here.
[14:34] <wgrant> Possibly for reading Archive.commercial, but meh.
[14:34] <wgrant> That flag wasn't even around when 1.0 was released, I'm pretty sure
[14:34] <wgrant> Nobody but CA has any cause to use it.
[14:34] <jml> bac: and we're OK with the compat break, so I don't think it's a practical concern
[14:34] <jml> wgrant: thanks :)
[14:34] <jml> wgrant: sleep well & have a good weekend.
[14:35] <wgrant> jml, rick_h_: Thanks, you too.
[14:35] <czajkowski> rick_h_: please read all tweets today laced with thick irish accent - will make it funny for you , and hair pulling experience for me :)
[14:35] <rick_h_> czajkowski: :)
[14:35] <bac> jml: ok, so the createPPA signature change since beta was to add an optional parameter...so removing it is a-ok.  sounds reasonable
[14:37] <jml> bac: thanks.
[14:40] <wgrant> gary_poster: Argh, sorry. That cookie-authentication.txt failure is my fault.
[14:40] <wgrant> gary_poster: I missed a space when I updated the domain list in setuplxc
[14:41] <bac> jml: approved but your test is slightly broken
[14:41] <jml> bac: oh?
[14:44] <gary_poster> wgrant, oh cool.  good to know thx
[14:44] <gary_poster> will look after babysitting/lunch time, starting now :-)
[14:44] <wgrant> Heh
[14:44] <wgrant> pqm-submitting a fix
[14:55] <jml> bac: have pushed a fix for the test, as well as doc fixes that james_w recommended. Can you please land?
[14:55] <jml> note, I expect test failures.
[14:56] <bac> jml: would you set the commit message on the MP please.
[14:57] <jml> bac: done.
[14:57] <bac> jml: so you want me to 'ec2 land' but you don't think it'll make it?
[14:58] <jml> bac: pretty much. renames in Launchpad never work the first time.
[14:58] <bac> jml: alrighty
[14:58] <jml> bac: I've done what I can, but grep only takes one so farr.
[15:11] <bac> jml: is there a bug for this work ?
[15:11] <jml> Hmm.
[15:11] <jml> No. I can file one if you'd like.
[15:11]  * jml wishes lp had rev-based QA
[15:12] <bac> jml: where i guess there should be some qa, to ensure the UI doesn't break.  so a bug would be needed.
[15:12] <bac> s/where/well
[15:13] <jml> https://bugs.launchpad.net/launchpad/+bug/994644
[15:13] <_mup_> Bug #994644: IArchive.commercial is misleading name  <Launchpad itself:New> < https://launchpad.net/bugs/994644 >
[15:13] <jml> linked up
[15:18] <rick_h_> ok, going to take bzr out back and shoot it...any masters around know how to trick bzr into removing the THIS file it's creating for it's own evil puposes?
[15:18] <rick_h_> https://code.launchpad.net/~rharding/launchpad/editstatus_timeout_874250/+merge/103912
[15:18] <rick_h_> tried creating a new branch and merging the old in, creating and removing the file itself,
[15:19] <rick_h_> nothing in the conflict docs in bzr land is pointing me at any hints I can tell
[15:21] <jml> rick_h_: there's only a .THIS? no .OTHER?
[15:21] <rick_h_> jml: yea, I did at one point have a conflict I resolved there
[15:21] <rick_h_> but that .THIS isn't in tree anywhere, it's only in LP's MP head for some reason
[15:21] <rick_h_> so I can't do any sort of force delete, etc
[15:22] <jml> rick_h_: so the current head on LP has got most recent devel merged in & conflicts resolved?
[15:22] <rick_h_> jml: rgr
[15:22] <rick_h_> if I try to merge with devel is says nothing to do, push, nothing to push, pull my branch, nothing to pull
[15:22] <jml> rick_h_: what happens when you change into devel and merge your branch in to that?
[15:22] <jml> i.e. manually on your own system
[15:22] <rick_h_> but the MP says there's a conflict and refuses to delete the file
[15:23] <rick_h_> hmm, haven't tried that, what I did do was create a fresh branch off of devel, and merge these changes into that new branch
[15:23] <rick_h_> as soon as I pushed it up to create a new MP...it had this same problem
[15:23] <rick_h_> so I'd wager that merging into devel itself would have this issue
[15:24] <jml> rick_h_: because merging into devel is exactly what LP does, AIUI
[15:25] <jml> rick_h_: that rename is *really weird*
[15:25] <rick_h_> right, so figuring by some bzr voodoo I need to get rid of that .THIS which should never have been there after I did the bzr resolve on my conflicted file
[15:25] <rick_h_> but that voodoo is escaping me
[15:26] <cjwatson> Perhaps the file-ids for that file differ in the two branches
[15:26] <cjwatson> bzr ls --show-ids would tell you
[15:26] <rick_h_> looking
[15:26] <jml> that would account for the original conflict but wouldn't account for committing the rename
[15:27] <cjwatson> Well, it would suggest the simplest fix which would be to 'bzr rm --keep' the file with the wrong file-id and 'bzr add --file-ids-from=../devel' it with the same contents
[15:27] <cjwatson> (If true)
[15:28] <rick_h_> right, but the fiel isn't there at all, so I can't rm anything with it
[15:28] <rick_h_> file that is
[15:28] <rick_h_> there is no bugtask.txt.THIS
[15:28] <cjwatson> Oh, hmm
[15:29] <jml> underscores. sheesh.
[15:29] <cjwatson> I'm stumped then.  If it were me, at some point I'd probably give up, make a fresh branch, and apply the patch ...
[15:30] <rick_h_> so just manual diff/manually patch the trees vs trying to merge them with bzr itself
[15:30] <rick_h_> yea, guess that makes sense, the commit history isn't important with this atm so works for me
[15:30]  * jml tries.
[15:30] <stub> gary_poster: That nl_  helper occasional failure is special.
[15:30]  * rick_h_ cheers on jml
[15:31] <gary_poster> stub, heh, yeah. :-)  benji is looking at it
[15:31] <jml> watch this space: https://code.launchpad.net/~jml/launchpad/editstatus_timeout_874250/+merge/104763
[15:32] <rick_h_> will do
[15:32] <benji> stub: when I repeatedly run the test it fails about 1 in 30 times, when I edit the doctest that it's in to just include the failing test I can't get it to fail at all (in more than 100 runs)
[15:33] <stub> benji: Is this with PG 9.1 or 8.4?
[15:33] <benji> stub: 8.4
[15:34] <jml> ok same prob.
[15:34] <rick_h_> jml: ok thanks for trying. I'll feel better knowing it's not me just having a case of fridays not 'getting' something and work around it
[15:34] <benji> stub: I have some reason to believe that the bad behavior is generated in these two lines:
[15:34] <benji>         p = plpy.prepare("SELECT to_tsquery('default', $1) AS x", ["text"])
[15:34] <benji>         query = plpy.execute(p, [query], 1)[0]["x"]
[15:35] <stub> benji: I'd consider trying PG 9.1. If the problem goes away, we can blame plpythonu. It received significant updates in 9.0 and 9.1
[15:35] <stub> benji: If there is some sort of memory allocation area, bind variables like you point out would be where they are at.
[15:35] <stub> c/area/error/
[15:36] <benji> stub: I quake in fear of doing a postgres upgrade
[15:36] <stub> Its trivial really, and I just emailed saying devs will need to upgrade soon anyway :)
[15:37] <benji> once more into the breach
[15:37] <stub> benji: Have you managed to get debug information about what 'query' is being passed into plpy.execute from a failed run?
[15:40] <benji> stub: yep, I couldn't figure out how to enable/locate the output of plby.debug so I just transmuted those calls to write to a file; everything looks good (even when the tests fail) through the "15" debug output (just before the lines I quoted earlier) so I added a "16" line just after and when I got a failure the logged query was messed up as you would expect
[15:42] <stub> If the query string going in is correct, but the results from plpy.execute wrong, then I think we must be triggering a bug in plpythonu or tsearch2.
[15:42] <abentley> rick_h_: So the conflict makes sense to me.  This is a case where one side wants to modify a file, and the other side wants to delete it.  It's a "contents conflict" rather than a "text conflict", because the file was deleted.
[15:43] <stub> jtv: The 'IF EXISTS' clauses are likely 9.0 specific syntax
[15:43] <stub> (and are unnecessary)
[15:43] <abentley> rick_h_: Bazaar keeps the file around in case you choose to resurrect it.  But it renames it to foo.THIS because it's part of a conflict.
[15:44] <jtv> stub: they are probably unnecessary, yes.  They're more to avoid development hassle than anything else.
[15:44] <benji> stub: I'll try upgrading PG and see what happens
[15:44] <stub> jtv: yer, I like them too (but I'm currently writing 9.1 specific code ;) )
[15:45] <jtv> Grr.  Can't land: pqm says it can't verify my gpg sig.
[15:46] <stub> ec2 land or bzr lp-land ?
[15:46] <abentley> rick_h_: You can do diff -u lib/lp/bugs/doc/bugtask.txt.BASE lib/lp/bugs/doc/bugtask.txt.THIS to see what changes were made.
[15:47] <jtv> stub: I did pqm-submit.
[15:47] <jtv> (After ec2 land had the same problem)
[15:48] <jtv> Meanwhile, looks like “IF EXISTS” is 9.x only for the ALTER TABLE DROP COLUMN part of my patch.
[15:50] <jtv> benji: if you're still on PG 8.4, maybe you could try out a “make schema” on my branch?
[15:51] <jtv> I'd sleep easier knowing that that worked.  :)
[15:51] <jtv> It's lp:~jtv/launchpad/db-994410
[15:54] <rick_h_> abentley: right, but I marked it as resolved so I have no files in tree named bugtest.txt*
[15:54] <rick_h_> and I can't 'get' the file so I can see it by pulling/merging
[15:55] <jtv> Definitely time to go home.
[15:55] <abentley> rick_h_: Right, when you mark a conflict resolved, it deletes any files you had hanging around to help you resolve your conflict.
[15:56] <benji> jtv: I'm trying your branch now.
[15:56] <jtv> Thanks!
[15:56] <rick_h_> abentley: right, so I follow the 'usual' process and thought I did it right from that standpoint, but no idea how to make the MP 'happy' at this point from this state
[15:56] <abentley> rick_h_: You can retrieve the THIS version with "bzr cat lib/lp/bugs/doc/bugtask.txt"
[15:57] <rick_h_> bzr: ERROR: u'lib/lp/bugs/doc/bugtask.txt' is not present in revision rick.harding@canonical.com-20120504145817-vw1o8a0212a8olkv
[15:57] <jtv> benji: if it's not too much to ask, maybe run some arbitrary lp.translations tests to see if the trigger blows up on 8.4?
[15:57] <benji> jtv: will do
[15:58] <abentley> rick_h_: That would work in a devel branch, not in the editstatus_timeout branch.
[15:58] <rick_h_> ah ok
[15:58] <abentley> rick_h_: I don't think you'd have THIS in the editstatus_timeout.  It out to be OTHER in that case.
[15:59] <abentley> rick_h_: And to retrieve BASE,  bzr cat -r ancestor:lp:~jml/launchpad/editstatus_timeout_874250 lib/lp/bugs/doc/bugtask.txt
[16:00] <abentley> rick_h_: To make the mp "happy", you need to merge devel into editstatus_timeout_874250, fix the conflict, push.
[16:01] <abentley> rick_h_: Since you can't push into jml's copy of the branch, you need to push your own copy, and resubmit the merge proposal to use your copy.
[16:02] <rick_h_> abentley: right, but his branch is a copy of my branch which is already merged with devel, conflict resolved, and pushed
[16:02] <abentley> rick_h_: When I merge his branch into devel, I get a conflict, so I don't think the conflict was resolved.
[16:03] <rick_h_> ok, I merged devel with my branch and resolved it there
[16:03] <rick_h_> but gotcha, it's still unhappy. I'll rework it and carry on. Thanks for looking into it abentley
[16:10] <benji> jtv: no test failures in about 100 tests; unless you want me to run more I'll kill it now
[16:10] <jtv> benji: fantastic.  Thanks.  Feel free to stop it.
[16:10] <jtv> And with that, I'm off.  Have a good weekend everyone!
[16:10] <benji> cool
[16:10] <abentley> rick_h_: So, you merged 15195, but the file was changed in 15207.  That's why your merge didn't fix the conflict.
[17:58] <dratone> hi all.
[19:16] <benji> rick_h_: so, how did you fix http://paste.mitechie.com/raw/606/?  I'm upgrading to postgres 9.1 and have the same problem
[19:18] <lifeless> benji: you need to enable the fti procedures, which is done as part of rocketfuel setup IIRC
[19:18] <lifeless> benji: basically running the db setup part again
[19:19] <benji> lifeless: thanks
[19:19] <lifeless> (Thats a WAG, but I'm fairly sure about it)