[00:00] <lifeless> bah
[00:00] <lifeless> I selected a component of main
[00:00] <lifeless> status new
[00:00] <lifeless> 5.53 seconds
[00:00] <wgrant> main is tiny.
[00:00] <wgrant> Doesn't count.
[00:00] <LPCIBot> Project devel build #518: FAILURE in 4 hr 48 min: https://hudson.wedontsleep.org/job/devel/518/
[00:00] <lifeless> ok
[00:00] <lifeless> universe works
[00:03] <lifeless> and its the - guess what - spph join thats the problem
[00:03] <wallyworld> sinzui: if you are still online, can you tell me how to construct a PrettyOverlay but with a different BOUNDING_TEMPLATE. I'm not sure how to override the default one
[00:04] <sinzui> oh.
[00:04] <sinzui> That just went over my head. I think I got a nose bleed thinking about it
[00:04] <wallyworld> sorry :-)
[00:05] <wallyworld> i want to replace the close (x) with a pair of (tick) (cross) buttons
[00:05] <wallyworld> since the user needs to be able to poke some widgets on the overlay and then either accept or cancel
[00:06] <wallyworld> i could get the close div after construction and insert a new node
[00:06] <wallyworld> that would do it
[00:07] <lifeless> wgrant: whats silly is that we get back no packages
[00:07] <wgrant> getBuildRecords is a mess :(
[00:08] <wgrant> lifeless: ... oh?
[00:09] <lifeless> wgrant: see actual time=0.011..0.011 rows=0 loops=1276160)
[00:09] <lifeless> perhaps thats just misleading
[00:09] <lifeless> we do get 76 bugs
[00:10] <wgrant> Ursinha: Hi.
[00:11] <Ursinha> hi
[00:11] <Ursinha> it's carnaval, so this might be an automated response :P
[00:11] <wgrant> haha, OK.
[00:12] <Ursinha> wgrant, what's going on?
[00:12] <thumper> wallyworld: I'll take a look
[00:12] <wallyworld> thumper: ta
[00:12] <wgrant> Ursinha: I was just wondering what sends the daily error reports these days. launchpad-errors-yesterday in lp:oops-tools seems to be out of date.
[00:13]  * Ursinha checks lpqateam's crontab
[00:13] <lifeless> Ursinha: is matsubara_ around ?
[00:14] <Ursinha> lifeless, unlikely, 9pm of a carnaval's tuesday night
[00:14] <lifeless> heh, ok :)
[00:15] <Ursinha> wgrant, so, it's a script called report (heh)
[00:16] <Ursinha> generated automatically by oops-tools
[00:16] <Ursinha> wgrant, why?
[00:16] <Ursinha> lifeless, do you need something specific?
[00:17] <lifeless> Ursinha: I have a oops tool merge proposal, hadn't heard anything in a few days
[00:17] <wgrant> Ursinha: I want to change a prefix.
[00:17] <wgrant> Was hoping to change it there too.
[00:17] <Ursinha> wgrant, hm, prefixes now are changed using an admin django page
[00:17] <wgrant> Ah, handy.
[00:17] <Ursinha> prefixes are loaded dynamically from oops-tools database
[00:18] <Ursinha> I'm sure there's a wiki page explaining that
[00:18]  * Ursinha looks
[00:18] <lifeless> there is
[00:18] <lifeless> matsubara mailed it to me :)
[00:18] <Ursinha> wgrant, https://dev.launchpad.net/Foundations/QA/OopsToolsSetup
[00:20] <wgrant> Ursinha: Thanks.
[00:20] <wgrant> Who has access to add a new prefix to a report?
[00:24] <lifeless> ok, got that down to 800ms.
[00:24] <lifeless> nice
[00:24] <wgrant> Nice.
[00:24] <wgrant> What did you change?
[00:25] <wgrant> Hm, build breakage.
[00:25] <wgrant> Same one us hudson.
[00:25] <wgrant> Intriguing.
[00:25] <wgrant> Because that's a spurious failure.
[00:25] <wgrant> Must be time-based.
[00:25] <lifeless> https://bugs.launchpad.net/launchpad/+bug/731679/comments/3
[00:25] <_mup_> Bug #731679: distribution:+bugs timeout searching for bugs by component <timeout> <Launchpad itself:Triaged> < https://launchpad.net/bugs/731679 >
[00:25] <wgrant> I've seen this happen before, but dismissed it as coincidence :/
[00:26] <lifeless> time to dig into storm, I just wish it was more pleasant
[00:27] <wgrant> lifeless: Can't you easily word that as a subselect?
[00:28] <lifeless> wgrant: you'd think so
[00:28] <wgrant> Does it break the plan?
[00:31] <thumper> sinzui: still around?
[00:31] <sinzui> I am
[00:31] <thumper> I'm just encoding that video :)
[00:33] <sinzui> thumper: I was just about to have dinner. I will return in about an hour.
[00:33] <thumper> sinzui: ack
[00:55] <wgrant> wtf
[00:56] <lifeless> ???
[00:56] <wgrant> This test failure.
[00:56] <wgrant> It is real.
[00:56] <wgrant> But completely unrelated.
[00:56] <wgrant> 12555 must have changed test order somehow.
[00:57] <wgrant> 'bin/test -m lp.services.job.tests.test_runner -1cvvt TestTwistedJobRunner.test_timeout' reproduces the failure even before then. Dropping the -m works fine. So it needs some test module loaded before it will work.
[00:58] <wgrant> Hmm, ampoule can't import _pythonpath.
[01:07] <lifeless> mtaylor: does mysql 5.1 support 'with foo as (...) select ... where my.id in (select thing from foo)' ?
[01:31] <wgrant> Gnrgh.
[01:32] <lifeless> ?
[01:33] <wgrant> This test. A subprocess fails if lp.testing imports anything from canonical.testing.layers.
[01:33] <wgrant> Must be a circular import, I guess, but Twisted is good at swallowing details.
[01:34] <lifeless> eugh
[02:08] <lifeless> \o/ 176 /    0  LanguageSet:CollectionResource:#languages
[02:08] <lifeless> 58 /  155  BugTask:+index
[02:11] <mtaylor> lifeless: I have never seen 'with foo as (...)' - but I could of course have just missed it
[02:12] <lifeless> mtaylor: thanks, I will write this as a postgresql only tested feature then :)
[02:13] <mtaylor> lifeless: my pleasure!
[02:17] <wgrant> Excellent, the top 50 exceptions now encompass all with multiple occurrences.
[02:27] <lifeless> https://bugs.launchpad.net/storm/+bug/731739
[02:27] <_mup_> Bug #731739: with foo as (select...) SELECT ... not supported <Storm:New> < https://launchpad.net/bugs/731739 >
[02:29] <lifeless> now to figure out how to do a snapshot of storm
[02:30] <wgrant> You've done it already?
[02:30] <lifeless> yes
[02:30] <lifeless> sufficient for our needs anyhow
[02:32] <lifeless> https://code.launchpad.net/~lifeless/storm/with/+merge/52630
[02:47] <StevenK> lifeless: You linked to the Postgres 9.0 docs, it's supported earlier?
[02:50] <wgrant> StevenK: It's new in 8.4.
[02:51] <StevenK> That's probably okay, then.
[02:53] <lifeless> StevenK: its supported wherever lp is supported
[02:53] <thumper> I seem to be getting  lp.services.job.tests.test_runner.TestTwistedJobRunner.test_timeout failing every time on EC2
[02:53] <lifeless> wgrant is working on that
[02:53] <wgrant> thumper: Yes, I'm looking at that.
[02:53] <thumper> others getting this?
[02:53] <thumper> wgrant: awesome, thanks
[02:53] <wgrant> About to give up and just work around it, though.
[02:53] <StevenK> Jenkins is to
[02:53] <wgrant> Ampoule is almost more evil than I am willing to deal with.
[02:53] <thumper> wgrant: want to talk through it?
[02:54] <wgrant> thumper: That test breaks with an unknown subprocess failure if lib/lp/testing/__init__.py imports anything from canonical.testing.layers (r12555 introduces one such import).
[02:55] <wgrant> thumper: The test also fails when run on its own with -m lp.services.job.tests.test_runner, even before r12555.
[02:55] <thumper> hmm...
[02:55] <wgrant> But with a slightly different failure.
[02:55] <thumper> it was failing intermittantly
[02:55] <wgrant> It always has, yes.
[02:55] <thumper> and there is a bug for that
[02:55] <wgrant> There is.
[02:55] <wgrant> Bug #505913
[02:55] <_mup_> Bug #505913: TestTwistedJobRunner.test_timeout fails intermittently <lp-foundations> <spurious-test-failure> <Launchpad itself:In Progress by wgrant> < https://launchpad.net/bugs/505913 >
[02:56] <thumper> how are you working around it?
[02:56] <wgrant> Removing the import.
[02:57] <wgrant> Running with -m causes ampoule to fail to import _pythonpath. If I stop it from attempting to use _pythonpath, we get the same error that r12555 introduces.
[02:57] <wgrant> Any useful diagnostic information is swalled by Twisted, though.
[02:57] <wgrant> +ow
[02:57] <thumper> hmm...
[02:58] <wgrant> It also still happens if I stop the tests from running in a subprocess.
[03:00] <wallyworld> thumper: it seems one is not allowed to modify a CollectionField attribute via the web service? i get a "You tried to modify a collection attribute" error
[03:00] <wgrant> That's right.
[03:00] <wgrant> You might want a List instead.
[03:01] <wgrant> eg. IBug['tags']
[03:01] <wallyworld> wgrant: thanks. will have a look
[03:02] <wgrant> wallyworld: Modifying CollectionFields is hard... what does it mean to remove an object from a resultset?
[03:02] <thumper> wallyworld: hmm
[03:02] <thumper> wallyworld: I'm sure leonardr suggested to me that we'd need add and remove methods to post to.
[03:02] <thumper> wallyworld: or perhaps an update?
[03:02] <thumper> wallyworld: you're one of the first attempting this
[03:02] <wallyworld> wgrant: i thought a collectionfield attribute were like lists
[03:03] <thumper> wallyworld: so you get to blaze the way
[03:03] <wallyworld> thumper: i'm replacing the entire field's value so i may as well change it to a List
[03:04] <wgrant> wallyworld: Note that it changes the representation significantly; it becomes inline instead of a link.
[03:04] <wgrant> This has performance implications.
[03:05] <wallyworld> wgrant: i'm not familiar with the terminology in this context - inline vs link
[03:05] <thumper> wallyworld: I think you want an update method for the API
[03:06] <wallyworld> thumper: so leave it as CollectionField and use an update method? if List fields are mutable, given the size of the list is small (<20), does it make sense just to go with that for this case?
[03:07] <wgrant> Is this for SourcePackageRecipe.distroseries?
[03:08] <thumper> wgrant: it is for recipe build series
[03:08] <wgrant> Right.
[03:08] <wgrant> List is probably not appropriate there.
[03:08] <wgrant> I would have (add|remove)Series named ops, I think.
[03:09] <wallyworld> hmm. seems like overkill given the size of the collection and the fact that we really just replace one version with another
[03:09] <wallyworld> by version i should say value
[03:10] <wallyworld> add/remove is typically more suited to larger collection sizes
[03:10] <thumper> I think updateSeries should be sufficient
[03:10] <StevenK>     Store.find(LibraryFileAlias).set(content=None)
[03:10] <StevenK> TypeError: unbound method find() must be called with Store instance as first argument (got SQLObjectMeta instance instead)
[03:10]  * StevenK grumbles more
[03:10] <thumper> StevenK: Store.of(something).find
[03:10] <wgrant> StevenK: IStore(LibraryFileAlias)
[03:10] <wgrant> Or that.
[03:11] <StevenK> wgrant: I've reverted to getCandidates returns ids
[03:12] <wgrant> wallyworld: Hmm, a collection can have custom named operations.
[03:12] <wgrant> Possibly not collection fields, though... hmmm.
[03:13] <wallyworld> wgrant: i'm grepping the code as we speak to figure out how to do it :-)
[03:13] <wallyworld> i'll do  a named_post
[03:13] <wallyworld> SPR.updateseries() or something
[03:14] <wgrant> wallyworld: Right, that's easy enough, but from looking at lazr.restful it seems like it might be possible to do something on the collection itself.
[03:14] <wgrant> But updateSeries() is probably simpler.
[03:15] <wallyworld> i'll have a poke around. i'm generally against fine grained collection updates
[03:15] <wallyworld> especially when the semantics of this is a complete replacement
[03:15] <wallyworld> of the collection's values
[03:16] <wgrant> Right.
[03:16] <wallyworld> where it would make sense for example is adding/removing a single person from a team
[03:18] <lifeless> wgrant: not only done, done and working.
[03:18] <wgrant> lifeless: Excellent!
[03:23] <lifeless> I can has reviewer? https://code.launchpad.net/~lifeless/launchpad/bug-731679/+merge/52631
[03:23]  * StevenK looks
[03:24] <StevenK> No reviewer for you. Come back when you have a diff
[03:24] <StevenK> :-P
[03:24] <lifeless> don't blame me
[03:25] <StevenK> lifeless: Does it require the storm changes, or are those seperate?
[03:25] <lifeless> requires em
[03:25] <lifeless> I've a snapshot ready to commit to the downloadcache
[03:26] <lifeless> StevenK: it has a diff
[03:27] <StevenK> Yes, I'm reading it
[03:27] <lifeless> thanks!
[03:27] <StevenK> lifeless: orig_store, and then you call orig_store.with_(with_clause) when you'd already set store = store.with_(with_clause)?
[03:27] <wgrant> That version number looks like a lie.
[03:27] <lifeless> StevenK: different with_clause
[03:28] <StevenK> And yes, I'd prefer that the storm changes get approved and landed first
[03:28] <lifeless> StevenK: note the for loop
[03:28] <lifeless> StevenK: not going to happen
[03:28] <lifeless> StevenK: storm is basically stalled.
[03:28] <lifeless> StevenK: I've 0 intention of letting us stall just because its stalled.
[03:28] <StevenK> Orsum
[03:30] <lifeless> feel free to review my storm changes though
[03:30] <StevenK> thumper: https://code.launchpad.net/~lifeless/launchpad/bug-731679/+merge/52631 when you have a tick
[03:31] <lifeless> but its an odd coding style there
[03:31] <StevenK> lifeless: I glanced at them, but I'm not comfortable reviewing
[03:31]  * thumper has two ticks
[03:31] <lifeless> thumper: thanks
[03:32] <thumper> lifeless: what was the concern addressed on irc?
[03:32] <lifeless> thumper: which concern?
[03:32] <wgrant> I still think that version number is wrong.
[03:32] <thumper> lifeless: the one that StevenK says in the code review
[03:32] <StevenK> thumper: I'd prefer the changes to Storm landed
[03:32] <StevenK> [14:28] < lifeless> StevenK: storm is basically stalled.
[03:33] <StevenK> Ergo ...
[03:33] <wgrant> I don't care if they land, but that 0.18.0.99 is deceptive.
[03:33] <lifeless> wgrant: pick a version, I'll roll with it
[03:33] <lifeless> wgrant: note that it does string algebra
[03:34] <lifeless> wgrant: so 0.18.0.0r14 will be stupidly painful to make
[03:34] <wgrant> lifeless: We normally use a suffix like -lp-r1234
[03:34] <lifeless> wgrant: -how-
[03:34] <wgrant> lifeless: How are you building it?
[03:34] <lifeless> wgrant: like I say, storms setup.py / __init__ do algebra on a constant
[03:34] <lifeless> wgrant: setup.py sdist
[03:36] <StevenK> Store.of(LibraryFileAlias).find().set(content=None)
[03:36] <wgrant> All our previous custom Storms have had a good suffix.
[03:36] <StevenK> That doesn't work either, am I doing it wrong?
[03:36] <wgrant> StevenK: find(LibraryFileALias)
[03:37] <wgrant> lifeless: What about 'setup.py egg_info -b-lp-r1234 sdist'?
[03:37] <lifeless> wgrant: that worked, I'll use it
[03:40] <StevenK> wgrant: Still ends up with AttributeError: 'NoneType' object has no attribute 'find'
[03:40] <lifeless> StevenK: IStore(LibraryFileAlias)...
[03:41] <wgrant> Ah, didn't bother to look further than the find()
[03:41] <lifeless> StevenK: Store.of() only applies to instances that have either come from a store or been added to a store.
[03:41] <wgrant> Store.of(someobject)
[03:41] <wgrant> IStore(someclass)
[03:42] <StevenK> Ahh
[03:42] <lifeless> StevenK: this was the only thing wrong with your example I think
[03:42] <lifeless> StevenK: also, when something doesn't work, show how :)
[03:42] <lifeless> thumper: thanks
[03:43] <StevenK> IStore(LibraryFileAlias).find(LibraryFileAlias).set(content=None) results in:
[03:43] <StevenK> CompileError: Don't know how to compile type Reference of <storm.references.Reference object at 0x6a30450>
[03:44] <lifeless> StevenK: don't use reference columns
[03:44] <lifeless> contentID
[03:44] <wgrant> Huh, that should work.
[03:45]  * wgrant blames the SQLObject layer.
[03:45] <StevenK> Trying contentID
[03:45] <lifeless> reference columns are basically broken in storm and upstream are not receptive
[03:45] <lifeless> wgrant: no, its not sqlobject layer
[03:48]  * wgrant gives up.
[03:48] <StevenK> wgrant: Oh?
[03:49] <wgrant> The ampoule thing.
[03:49]  * wgrant is testfixing.
[03:50] <StevenK> Rargh, how can setting content == NULL for every LFA *still* have SPRs being processed!
[03:51] <lifeless> clearly something doesn't do what you expect
[03:51] <StevenK> I'm tempted to blindly ignore 'has no DSC' and continue
[03:55] <lifeless> wgrant: is 681770 a dupe?
[03:55] <wgrant> lifeless: Already done.
[03:56] <lifeless> cool
[04:00] <StevenK> wgrant: Any thoughts?
[04:00] <wgrant> StevenK: Can you push up your latest changes?
[04:03] <StevenK> wgrant: Done.
[04:04] <wgrant> StevenK: https://code.launchpad.net/~wgrant/launchpad/testfix/+merge/52632
[04:04] <wgrant> Looking at yours.
[04:05] <StevenK> Same
[04:06] <StevenK> thumper: https://code.launchpad.net/~wgrant/launchpad/testfix/+merge/52632
[04:14]  * StevenK nails jtv to the Internet
[04:14] <wgrant> lifeless: Thanks.
[04:14] <StevenK> It probably won't help, though.
[04:15] <jtv> StevenK: no, but at least you got some fun out of it
[04:15] <lifeless> wth is this exposed  TemporaryStorageManager:CollectionResource:#temporary_blobs
[04:17] <wgrant> Possibly only so apport can upload to it.
[04:17] <wgrant> I can't think why an API client would want to read the collection.
[04:18] <wgrant> StevenK: Is it only test_hourly_script failing for you?
[04:18] <StevenK> wgrant: Yes.
[04:18] <wgrant> StevenK: Are you killing the LFAs there?
[04:18] <wgrant> Ah, yes.
[04:20] <lifeless> wgrant: and yet, they are
[04:21] <wgrant> lifeless: Grep logs and strangle them?
[04:24] <wgrant> StevenK: I wonder if LaunchpadScriptLayer might not use the right DB.
[04:31] <lifeless> wgrant: Revision 12550 can not be deployed: needstesting
[04:31] <lifeless> Preload BinaryPackageNames before listing successful copies' displaynames.
[04:32] <wgrant> lifeless: Fixed.
[04:35] <StevenK> wgrant: I wonder if using IMasterStore would make any difference. I suspect not.
[04:36] <lifeless> sigh
[04:36] <lifeless> overengineering
[04:36] <lifeless> https://code.launchpad.net/~gmb/launchpad/processapportblobjob-api-bug-513191/+merge/20209
[04:36] <wgrant> StevenK: I doubt it.
[04:37] <wgrant> lifeless: Ah, of course. That doesn't need a listable collection, though.
[04:37] <wgrant> StevenK: Check the name of the DB that each uses.
[04:37] <lifeless> wgrant: precisely.
[04:38] <StevenK> wgrant: How can I do that?
[04:38] <lifeless> StevenK: wgrant: tests that use IMasterStore need to use it consistently or they will break
[04:38] <lifeless> you can't use IStore in any helper :)
[04:38] <lifeless> ditto ISlaveStore
[04:39] <wgrant> lifeless: This one runs a script.
[04:39] <lifeless> wgrant: I also particularly love the 'last 50' statement which is a total lie.
[04:39] <wgrant> lifeless: Ah, it's not actually a security hole, because the blobs don't have launchpad.View.
[04:39] <wgrant> So the list is empty.
[04:40] <lifeless> wgrant: but are filtered in the appserver
[04:40] <wgrant> lifeless: Sure, so it's slow. But my main concern is that we aren't leaking them.
[04:40] <lifeless> wgrant: sure, thats a concern too
[04:40] <lifeless> just noting
[04:40] <lifeless> this is basically utter crack
[04:41] <wgrant> What happens if you remove the collection_default_content?
[04:41] <lifeless> wgrant: look at the end of lib/canonical/launchpad/database/temporaryblobstorage.py
[04:41] <wgrant> Or does export_webservice_collection require it?
[04:41] <lifeless> wgrant: needs it AFAIK
[04:41] <lifeless> I've a faster query
[04:41] <wgrant> lifeless: Don't use it.
[04:41] <wgrant> Return an empty list instead.
[04:41] <mwhudson> wgrant: wow, r12557 is >:(
[04:42] <wgrant> mwhudson: YES.
[04:42] <mwhudson> did you annotate for great justice?
[04:42] <wgrant> Which bit?
[04:42] <mwhudson> who added the flamingly incorrect comment
[04:42] <wgrant> The circular import one?
[04:43] <wgrant> That was added in the problematic revision.
[04:43] <mwhudson> ah ok
[04:43] <mwhudson> anyway, i have a bbq to get to, better than being frustrated by software
[04:43] <lifeless> wgrant: you've checked you don't see the blobs
[04:43] <wgrant> So, I spent a couple of hours trying to work out what was up, but I only ended up finding out that buildout, Twisted and Ampoule all need to DIE.
[04:43] <lifeless> wgrant: right ?
[04:43] <wgrant> lifeless: Yes.
[04:43] <lifeless> wgrant: but - how does the 'is blob X processed' work then ?
[04:44] <wgrant> lifeless: lazr.restful always checks for launchpad.View.
[04:44] <wgrant> Regardless of what the security adapters say.
[04:44] <wgrant> Anyone can view a blob, if they know its UUID.
[04:44] <lifeless> wgrant: but does't htat also check for View ?
[04:45] <wgrant>     <allow
[04:45] <wgrant>       interface="canonical.launchpad.interfaces.temporaryblobstorage.ITemporaryBlobStorage" />
[04:45] <wgrant> So, no.
[04:46] <lifeless> glue the dots together gfor me
[04:46] <wgrant> lazr.restful always checks for launchpad.View. But the security declarations can make anything publicly available.
[04:46] <wgrant> lazr.restful checks for launchpad.View before it shows an object in a collection representation, that is.
[04:47] <wgrant> For returning a single entry resoruce it will respect the security proxies.
[04:47] <lifeless> ah
[04:47] <lifeless> so collections are spethial
[04:47] <wgrant> Yeth.
[04:49]  * lifeless considers lp-landing this
[04:49]  * wgrant defers getBuildRecords until tomorrow, and instead fixes a couple of quick checkwatches OOPSes, since even checkwatches is likely to incite less violence than getBuildRecords and test_timeout.
[04:49] <wgrant> lifeless: What have you changed?
[04:50] <lifeless> wgrant: lp:~lifeless/launchpad/bug-731736
[04:50] <LPCIBot> Project devel build #519: STILL FAILING in 4 hr 49 min: https://hudson.wedontsleep.org/job/devel/519/
[04:51] <lifeless> https://code.launchpad.net/~lifeless/launchpad/bug-731736/+merge/52634
[04:52] <wgrant> lifeless: As long as nothing else uses that property, and all the blob webservice tests pass. doit.
[04:52] <lifeless> wgrant: I've changed one test which psases and the default collection method which there are no other references to outside of the webservice definition and implementation
[04:52] <wgrant> Right.
[04:52] <wgrant> Just checked the diff in LH
[04:52] <lifeless> and there are no other hits for the webservice collection name either
[04:53] <wgrant> You have my codestar.
[04:53] <lifeless> heh
[04:53] <lifeless> I was just self reviewing :) - thanks though
[04:54] <wgrant> That works too.
[04:54] <lifeless> sounds like bsg (codestar)
[04:54] <wgrant> That was the feel I was going for.
[04:55] <lifeless> so top timoeuts 1, 2, 3, all have patches pending deploy
[04:55] <wgrant> And 1 and 3 are actually fixed.
[04:55] <wgrant> 2 maybe not.
[04:56] <lifeless> time to look at 4 again
[04:56] <lifeless> bah
[04:56] <lifeless> no oops in the report
[05:02] <StevenK> wgrant: Grabbing the DB name in garbo-hourly is launchpad-main-master
[05:02] <lifeless> hah
[05:02] <lifeless> they missed a status
[05:02] <lifeless> field.status: [u'NEW', u'INCOMPLETE_WITH_RESPONSE', u'INCOMPLETE_WITHOUT_RESPONSE', u'INVALID', u'WONTFIX', u'EXPIRED', u'CONFIRMED', u'TRIAGED', u'INPROGRESS', u'FIXCOMMITTED', u'FIXRELEASED']
[05:02] <wgrant> StevenK: The postgres DB name is the interesting one. That's the store name.
[05:03] <wgrant> lifeless: Just opinion missing?
[05:03] <StevenK> wgrant: Did you miss my question of how to actually get it, then?
[05:03] <lifeless> wgrant: yeah
[05:03] <lifeless> wgrant: this is a /bugs/ search.
[05:03] <lifeless> https://bugs.launchpad.net/launchpad/+bug/716774
[05:03] <_mup_> Bug #716774: Timeout on MaloneApplication:+bugs <timeout> <Launchpad itself:Triaged> < https://launchpad.net/bugs/716774 >
[05:03] <wgrant> StevenK: Well, you could check the postgres log.
[05:04] <wgrant> Rather than adding debug prints everywhere.
[05:04] <StevenK> /var/log/postgresql/postgresql-8.4-main.log is fairly useless
[05:04] <wgrant> Ah, you don't have statement logging on?
[05:05] <StevenK> Doubtful
[05:05] <wgrant> log_statement='all' in postgresql.conf
[05:05] <wgrant> It's very handy.
[05:10] <StevenK> wgrant: If that logs into the main log, I can't see it
[05:11] <wgrant> /var/log/postgresql/postgresql-8.4-main.log
[05:16] <StevenK> wgrant: Doesn't seem to work for me. :-(
[05:17] <wgrant> StevenK: You put it at the bottom of /etc/postgresql/8.4/main/postgresql.conf, and it doesn't have any other log_statement lines?
[05:17] <StevenK> Rargh, there's two!
[05:18] <wgrant> Heh.
[05:21] <StevenK> Now to distill the log
[05:22] <StevenK> Looks to use the correct DB name
[05:22] <wgrant> Both of them?
[05:23] <StevenK> wgrant: They all use launchpad_ftest_23325
[05:23] <thumper> StevenK: sorry was at guitar lesson, thanks lifeless for approving
[05:23] <wgrant> :(
[05:23] <wgrant> Can you see the LFA UPDATES?
[05:24] <StevenK> Yes
[05:24] <lifeless> and you commit ?
[05:24] <StevenK> Yes
[05:24] <wgrant> There is a transaction.commit()
[05:24] <wgrant> But can you see it in the log?
[05:24] <wgrant> Some layers might not have the transaction manager configured properly.
[05:24] <wgrant> But it still fails on LaunchpadZopelessLayer :/
[05:24] <StevenK> [2011-03-09 16:19:28 EST] launchpad_main@launchpad_ftest_23325 LOG:  statement: UPDATE LibraryFileAlias SET content=NULL
[05:24] <StevenK> [2011-03-09 16:19:28 EST] launchpad_main@launchpad_ftest_23325 LOG:  statement: COMMIT
[05:24] <wgrant> That seems pretty definitive.
[05:25] <StevenK> The next bits are all garbo_hourly, so that's the script running
[05:26] <wgrant> Oh.
[05:26] <wgrant> Ha ha ha.
[05:26] <wgrant> Ha ha ha ha ha.
[05:26] <wgrant> I bet those SPRs have *no files at all*.
[05:26] <wgrant> Fuck sampledata, seriously.
[05:27] <StevenK> wgrant: 0 rows for id 17, for instance, so agreed
[05:27] <wgrant> Looks like I need a sharper desk still.
[05:27] <StevenK> I wonder if we can fix the query to ignore SPRs with no rows in SPRF
[05:28] <wgrant> Difficult. Just set changelog=1 for all of the SPRs.
[05:28] <wgrant> Much easier.
[05:30]  * StevenK tries that.
[05:30]  * StevenK attempts to not put "# <wgrant> Fuck sampledata, seriously." as the comment
[05:35] <StevenK> wgrant: Thanks, changed pushed.
[05:36] <wgrant> Great.
[05:37] <StevenK> wgrant: Now approve it, damn it. :-(
[05:46] <wgrant> lifeless: Could you mentor https://code.launchpad.net/~stevenk/launchpad/populate-spr-changelogs/+merge/52363?
[05:50] <StevenK> wgrant: Sure, let me fix the comment.
[05:50] <wgrant> StevenK: Thanks.
[05:51] <StevenK> wgrant: http://pastebin.ubuntu.com/577696/
[05:52] <wgrant> StevenK: Say something about why we can't expire the LFAs.
[05:52] <wgrant> That's the really confusing bit.
[05:53] <StevenK> wgrant: http://pastebin.ubuntu.com/577697/
[05:54] <wgrant> StevenK: +1
[05:55] <StevenK> wgrant: Changes pushed, thanks.
[06:47] <huwshimi> Night people.
[06:51] <wgrant> Night huwshimi.
[08:41] <jam> I thought there was a way to do an ssh tunnel to get direct psql access to the staging db. I was hoping to find the instructions to see if I could do something similar for loganberry's tuolumne db. Anyone have a link?
[08:41] <jam> my google fu is weak (and it may be private docs)
[08:43] <wgrant> jam: TLs can access the staging DB from devpad.
[08:44] <wgrant> But I doubt !IS has access to loganberry.
[08:44] <wgrant> Given that it's a prod machine.
[09:00] <adeuring> good morning
[09:03] <mrevell> Hallo
[09:07] <henninge> mrevell: Hallo
[09:07] <henninge> Moin adeuring ;)
[09:07] <stub> jam: It is just standard port forwarding a local port to port 5432 on the target machine. Connect to it locally with 'psql --port=55432 prod_db'
[09:10] <jam> stub: right. I thought so, but I wanted to check that you have to have a valid ssh login on the target machine.
[09:10] <jam> And I was curious why local psql is better than just running it on the target machine
[09:11] <jam> since you have a valid login already
[09:11] <jam> more-interactive typing ?
[09:17] <stub> jam: Yes, you need an account on the target machine or machine that can get past ipfilter to port 5432 on the target machine
[09:17] <lifeless> + ident or similar auth permissions
[09:17] <stub> jam: Using local psql makes it interactive. Typing into remote shells from the other side of the planet sucks big time.
[09:18] <wgrant> jam is right next to the DC :(
[09:18] <stub> So no benefit whatsoever :)
[09:18] <jam> stub: well, it confirms what I thought, thanks
[09:19] <jam> i agree that when you have 200+ms latency, typing is rather cumbersome
[09:41] <LPCIBot> Yippie, build fixed!
[09:41] <LPCIBot> Project devel build #520: FIXED in 4 hr 50 min: https://hudson.wedontsleep.org/job/devel/520/
[09:45] <bigjools> ye
[09:45] <bigjools> gads
[09:45] <bigjools> 4h 50m
[09:48] <StevenK> 20 minutes of that is check-out
[09:48] <StevenK> Which is just *orsum*
[09:58] <jml> good morning Launchpadders.
[09:58] <lifeless> jml: I forgot two things to chat about the other day
[09:58] <lifeless> jml: do you have 15?
[09:59] <jml> lifeless: not immediately. I'll need a few minutes.
[09:59] <lifeless> jml: is that few ~=5 or few ~=30 ?
[10:00] <jml> ~10
[10:00] <lifeless> ok, cool, I'll hang for that
[10:09] <rvba> lifeless: Hi ... I'm trying to ec2 test on a very small branch of mine and I'm getting an error in a test bzr annotate told me you wrote.Julian thinks it might be spurious (and it certainly looks like it when I look at the test) but advised me to ask you anyway. Would you take a look:
[10:09] <rvba> The whole log file http://dl.free.fr/pMlVoHAN6
[10:09] <rvba> The only place where it seems there is a failure https://pastebin.canonical.com/44450/
[10:17] <wgrant> That one is know to be spurious.
[10:17] <wgrant> If it was the only failure, the branch is fine.
[10:17] <wgrant> rvba: ^^
[10:17] <wgrant> It shows up rarely enough that it's difficult to debug :/
[10:18] <rvba> wgrant: ok
[10:18] <rvba> I'll launch ec2 land (again) then
[10:18] <rvba> thx
[10:18] <wgrant> No point running it through ec2 again.
[10:18] <wgrant> Just 'bzr lp-land' it.
[10:18] <wgrant> That submits it directly to PQM.
[10:18] <rvba> ok
[10:29] <lifeless> gnight all
[10:57] <bigjools> lifeless: review? :)
[11:43] <gmb> Well, that's a new one: "TypeError: Result of expression 'protonode' [null] is not an object."
[11:43]  * gmb goes grepping
[11:46] <bigjools> rvba: did you lp-land your branch ok?
[11:47] <rvba> bigjools: I relaunched the test suite to make sure I'll receive and email all right
[11:47] <rvba> s/and/an/g
[11:47] <bigjools> rvba: fair enough
[11:50] <rvba> bigjools: I'm working on 727632. Reading though the schema change docs.
[11:50] <bigjools> rvba: great
[11:54] <maxb> Hmm, odd. Launchpad appears to no longer be notifying me of recipe build failures
[11:55] <maxb> Or rather, it's not notifying me about recipe build failures due to depwait
[11:59] <henninge> adeuring: http://people.canonical.com/~henninge/mockups/
[11:59] <henninge> adeuring: see also my mail
[11:59]  * henninge lunces
[11:59] <henninge> lunches, even
[11:59] <heninge-lunch> Hi deryck!
[12:00] <deryck> Morning, heninge-lunch
[12:00] <heninge-lunch> deryck: http://people.canonical.com/~henninge/mockups/
[12:00] <heninge-lunch> deryck: see also my mail
[12:00] <deryck> heninge-lunch, ok, looking at it all now.  Thanks!
[12:43] <jml> deryck: is there an ACL for Fix Released like there is for wontfix etc?
[12:44] <deryck> jml, yes.  Only bug supervisor, or the original reporter, IIRC.
[12:44] <maxb> I have a pending application to ~loggerhead-team, who should I talk to to get a yay or nay?
[12:45] <jml> deryck: what does it take to set it up
[12:45] <jml> deryck: or is it enabled by default?
[12:48] <deryck> jml, I think defining a bug supervisor is enough.  I'd have to look at code to remember for sure. ;)
[12:49] <jml> deryck: that's fine.
[12:49] <jml> deryck: thanks very much.
[12:51] <deryck> jml, so I just looked to be sure, and it's just a check in canTransitionToStatus.  So nothing required.
[12:52] <deryck> wanted to remember clearly anyway :-)
[12:52] <jml> deryck: not even defining a bug supervisor?
[12:53] <deryck> jml, no, not required.
[13:16] <jam> I'm trying to qa bug #726985 but I'm unable to get any requests to succeed at "http://bazaar.qastaging.launchpad.net/"
[13:16] <_mup_> Bug #726985: codebrowse OOPSes with GeneratorExit when connection closed early <qa-needstesting> <Launchpad itself:Fix Committed by jameinel> < https://launchpad.net/bugs/726985 >
[13:17] <jam> When I go to: http://bazaar.qastaging.launchpad.net/~jameinel/+junk/foo/changes
[13:17] <jam> It just issues a permanent redirect, *to the same URL*
[13:21] <deryck> henninge, adeuring -- let's spend the standup talking about these mockups and come to some conclusion about them.
[13:22] <adeuring> sure
[13:23] <jam> jelmer: I tried to approve your change, but you deleted the merge proposal
[13:24] <jelmer> jam: It was an uncontroversial one-liner so I just submitted it.
[13:24] <jam> jelmer: well, maybe I'll just take back my approval then :)
[13:24] <jelmer> jam: heh
[13:24] <jelmer> jam: Sorry for the noise :)
[13:27] <henninge> deryck: great idea
[13:28] <wgrant> jam: qastaging codebrowse was never set up properly.
[13:28] <jam> wgrant: good enough for me, I guess
[13:28] <jam> I'm not really sure how to qa the "doesn't oops anymore" anyway
[13:28] <jam> can we just set the bug to qa-ok?
[13:29] <wgrant> jam: I QA'd the last one by poking around from some internal hosts, but for this I don't think it's worth it.
[13:29] <wgrant> qa-untestable
[13:29] <jam> fine by me
[13:34] <wgrant> jtv: Any luck with QAing the DSD script?
[13:34] <jtv> wgrant: I filed an RT.  I gather the losas are rather busy.
[13:34] <wgrant> jtv: Can we qa-untestable it, given that it presumably won't be run unless you ask for it?
[13:35] <jtv> wgrant: yes, come to think of it that does make sense now
[13:37] <jtv> Done.
[13:40] <wgrant> Thanks.
[13:57] <deryck> adeuring, henninge -- let's delay the standup by 15 minutes, to give abentley time to see the mockups when he comes on.
[13:57] <adeuring> ok
[13:59] <allenap> Running `make JSFLAGS="--filetype=debug"` seems to invoke the correct bin/jsbuild command line, but it still uses "min" files. Does anyone know how I get it to DTRT?
[14:00] <deryck> allenap, it should work for lp js code.  but utilities/yui-deps.py hard codes the min version.
[14:00] <deryck> so yui code is still minified, but lp code is not.
[14:03] <deryck> allenap, also, I've been doing filetype=raw, too.  not sure if that matters.
[14:03] <deryck> and you can save time doing "jsbuild", i.e. make jsbuild JSFLAGS='--filetype=raw'
[14:05] <allenap> deryck: Does this end up in lib/canonical/launchpad/icing/build/launchpad.js?
[14:05] <deryck> allenap, yes
[14:07] <allenap> deryck: It's still using the min files. I must be doing something stupid.
[14:07] <deryck> allenap, make clean_js ?  maybe it's being dumb and doing nothing?
[14:08] <bigjools> rvba: congratulations, your branch landed
[14:09] <deryck> henninge, we can standup when you're ready.
[14:09] <rvba> bigjools: well, one of them
[14:10] <rvba> bigjools: the other one triggered an error sending the email
[14:10] <rvba> https://pastebin.canonical.com/44466/
[14:11] <rvba> the strange thing is that the email conf should be the same for this branch and the one which landed
[14:11] <bigjools> no idea then :/
[14:11] <bigjools> use bzr lp-land for now
[14:11] <rvba> bigjools: yes
[14:15] <allenap> deryck: It doesn't seem to want to play.
[14:15] <allenap> Any other ideas?
[14:30] <bigjools> benji: yo
[14:30] <benji> bigjools: yo
[14:31] <bigjools> benji: the MP of mine that you approved the other day was targeted to devel instead of db-devel, can you do me a favour and tick off the newer MP please? https://code.launchpad.net/~julian-edwards/launchpad/publisher-config-db-schema/+merge/52530
[14:31] <benji> sure
[14:31] <bigjools> benji: I added some code that you might want to look at, there were no security declarations in the old branch :)
[14:31] <benji> k
[14:37] <benji> bigjools: done
[14:37] <bigjools> benji: cheers
[14:38] <bigjools> benji: good catch with the zcml
[14:48] <allenap> deryck: Ah, I've found the culprit. utilities/lp-deps.py force minifies the lp code too. Btw, thank you for your help earlier.
[14:54] <deryck> allenap, cool.  sorry on call now.  I really thought it used to work for me.
[15:17] <deryck> abentley, hey.  How about bottom of the hour for a chat?  roughly 12-13 minutes from now?
[15:18] <abentley> deryck: sure.
[15:18] <deryck> abentley, ok, cool.  meet you in mumble then.
[15:24] <deryck> jml, I'd like to book 15 minutes with you tomorrow, if possible.
[15:24] <deryck> jml, for some voice chat time about how/when to turn of this translations story for your review.
[15:25] <bigjools> jml: thanks for triaging my bug, I didn't mean to be lazy but I am a forgetful bastard
[15:26] <jml> bigjools: that's OK. I actually thought there was a dupe (there wasn't), and have been using this as an opportunity to make sure all the ec2test bugs are tagged up
[15:26] <jml> found a couple that have already been fixed
[15:26] <jml> inspired to go on a bug fixing rampage later on
[15:26] <jml> in my copious free time
[15:26] <jml> speaking of which...
[15:26] <jml> deryck: that'd be great
[15:27] <jml> deryck: just put something on my calendar
[15:27] <deryck> jml, ok, great.  thanks!
[15:30] <bigjools> jml: what is this free time of which you speak?
[15:31] <bigjools> is it somehow associated with the weird bright glowing ball I see in the sky sometimes?
[15:31] <jml> bigjools: I think the medical term is insomnia
[15:32] <jml> (or, more honestly, being naughty and doing something I want to do instead of something I should be doing)
[15:33] <abentley> deryck: http://pastebin.ubuntu.com/577884/
[15:35] <abentley> deryck: http://pastebin.ubuntu.com/577886/
[15:55] <rvba> sinzui: I've started to work on bug 727632. I've worked on the first part of the bug we talked about yesterday: fixing distro's owner usage.
[15:55] <_mup_> Bug #727632: distro and distro series pages do not specify their owner <derivation> <distributions> <series> <Launchpad itself:In Progress by rvb> < https://launchpad.net/bugs/727632 >
[15:55] <rvba> sinzui: This is still in the works (no tests yet etc.) but I'd be happy if you could quickly review my branch and give me feedback (lp:~rvb/launchpad/distro-add-registrant).
[15:55] <sinzui> rvba: okay. I will now
[15:56] <deryck> abentley, so here's a bit on why we do namespacing the way we do:  https://dev.launchpad.net/JavaScriptReviewNotes#Naming%20Javascript%20Modules%20and%20Namespaces
[15:56] <deryck> abentley, I think the "do this so you only have to change it in one place" comment is the best reason.  some of the rest seems suspect to me.
[15:58] <deryck> abentley, YUI's way is to create the namespace and then address it directly:  http://developer.yahoo.com/yui/3/yui/#yuiadd
[15:59] <abentley> deryck: are modules and namespaces orthagonal?
[16:00] <deryck> abentley, by that you mean that what I name it in the add constructor has nothing to do with initialziing the namespace in said module?
[16:01] <abentley> deryck: yes.  (except that we have a convention connecting them)
[16:02] <deryck> abentley, yes.  you could have a single module that creates two different namespaces, for example.
[16:02] <abentley> deryck: the UI docs you point at, the module is named 'mymodules-mod1' and the namespace is "Y.mynamespace".
[16:02] <abentley> s/UI/YUI
[16:02] <deryck> abentley, right.  we only it do it by convention, as you noted.
[16:02] <leonardr> abentley, i have an exciting branch for your review: https://code.launchpad.net/~leonardr/lazr.restful/must-specify-version/+merge/52705
[16:03] <abentley> deryck: okay, that explains why we specify the same string twice, then.
[16:03] <abentley> leonardr: ack
[16:19] <rvba> sinzui: thx for the review. I'll go on with my branch then and fix the -almost- same problem for distro series.
[16:19] <sinzui> okay
[16:25] <rvba> sinzui: one question though. When I 'make lint' I've got a warning about the fact that database/sampledata/lintdata.sql differs from database/sampledata/current.sql ... and indeed I had to modify current.sql ...
[16:26] <rvba> the warning print also a few shell commands ... that will take me to erase my modifications if I follow them :-)
[16:26] <sinzui> rvba: we do not edit those files for schema changes, though I realsied you did
[16:27] <rvba> sinzui: oh I get it, they are supposed to be migrated I guess ...
[16:27] <sinzui> rvba: after you` make schema`, run `make newsampledata`, then rename the new files to the old files
[16:27] <rvba> sinzui: ok I get it. thx
[16:28] <sinzui> rvba: I am certain you will see more than your changes. Lp engineers rarely reconcile their schema changes to sampledata
[16:28] <rvba> sinzui: indeed, I saw a small difference.
[16:33] <abentley> leonardr: does zope.Cleanup do something that testtools.TestCase.addCleanup can't do?
[16:38] <abentley> leonardr: as_of is a nice improvement over current syntax, but did you consider allowing the default version to be 'devel'?
[16:48] <leonardr> abentley: we'd still need as_of (since the existing syntax is ugly), it just wouldn't be required
[16:48] <abentley> leonardr: agreed.
[16:49] <leonardr> i'm pretty sure we explored that possibility and rejected it, but i don't remember the reasoning
[16:49] <leonardr> i will say that with this implementation it's a lot easier to patch our existing code
[16:50] <leonardr> than with an implementation in which our existing code will change meaning (rather than fail) if we forget to patch something
[16:50] <abentley> leonardr: True, but we could generate before-and-after wadls to ensure the interface stayed the same.
[16:52] <leonardr> abentley: i think there's not a whole lot of difference between these implementations, and this is the one lifeless prefers, so i'm going with it
[16:52] <abentley> leonardr: okay.  Just seems like extra typing for no benefit.
[16:53] <abentley> leonardr: Now, about zope.Cleanup.  Does that do something similar to testtools.addCleanup?
[16:53] <leonardr> abentley: there's no net decrease in typing. if the default is devel, you're going to have to do the typing when you decide to cut a new release
[16:53] <leonardr> i don't know what zope.Cleanup does. i'll check
[16:54] <abentley> leonardr: Oh, so that was from a contributor?
[16:54] <leonardr> abentley: i didn't write that code, if that's what you're saying. i just refactored it
[16:54] <leonardr> and i remember what the problem was. zope.Cleanup takes care of the current interaction
[16:54] <abentley> leonardr: There is a net increase in typing, because first you set export_as='devel', and then afterward you change export_as to '1.1'.
[16:55] <lifeless> gmb: hi - qa plox - 12551
[16:55] <leonardr> fair enough. i'd say that's worth it because you can grep for 'devel', see everything that hasn't been changed, and make a decision based on that
[16:56] <leonardr> if there is something exported with no version specified, there's no way to know whether it's in devel or whether someone just made a mistake
[16:57] <abentley> leonardr: okay.  That takes care of the real issues.
[16:57] <gmb> lifeless: done
[16:59] <jcsackett> i'm trying to use bzr pipeline, and i think i might have screwed something up. when i try to test a particular pipe via ec2 i get a NotABranch error.
[16:59] <jcsackett> anyone have any thoughts?
[16:59] <abentley> leonardr: you have some extra blank lines at 679 and 669.  It would also be nice if you documented every test.
[16:59] <abentley> leonardr: But r=me.
[16:59] <leonardr> ok
[17:09] <leonardr> sinzui: i am getting totally random test failures when i try to land the branch we worked on yesterday
[17:09] <leonardr> InternalError: current transaction is aborted, commands ignored until end of transaction block
[17:09] <leonardr> this has happened twice, so i doubt it's spurious
[17:10] <leonardr> is it possible that adding the validation for duplicate bugtasks could cause that error?
[17:11] <leonardr> oh, and i'm getting the error locally, so it's not an ec2 problem
[17:13] <leonardr> sinzui: http://pastebin.ubuntu.com/577933/
[17:14] <leonardr> that's the source of the problem, but i don't know why it wouldn't have permission
[17:14] <sinzui> leonardr: this looks like a db issue something related to security.cfg
[17:15] <leonardr> is process_email running as a different database user?
[17:15] <sinzui> leonardr: yes it is
[17:16] <sinzui> leonardr: we just need to find the user and add the tables to the section.
[17:16] <sinzui> I think we only need insert
[17:16] <leonardr> i think we only need select, right?
[17:17] <sinzui> oh, yes, select. you are right.
[17:18] <leonardr> processmail user?
[17:18] <sinzui> I think so
[17:21] <leonardr> sinzui: the obvious thing doesn't help: http://pastebin.ubuntu.com/577935/
[17:21] <leonardr> any ideas?
[17:22] <leonardr> perhaps that's the wrong user??
[17:22] <lifeless> abentley: trying to get as complete a deploy tonight as possible - can you qa Revision 12555 ?
[17:22] <sinzui> leonardr: You had the identical error after making schema?
[17:23] <abentley> lifeless: in the middle of doing so.
[17:23] <lifeless> abentley: cool!
[17:23] <lifeless> henninge: ping
[17:23] <henninge> lifeless: pong
[17:23] <henninge> ;)
[17:24] <lifeless> henninge: if you're still around, and can qa https://bugs.launchpad.net/launchpad/+bug/705176 we should be able to include it in the deploy tonight
[17:24] <_mup_> Bug #705176: Improve sharing information on translation pages <qa-needstesting> <upstream-translations-sharing> <Launchpad itself:Fix Committed by henninge> < https://launchpad.net/bugs/705176 >
[17:24] <henninge> lifeless: I was just looking at it. something might be wrong but it is behind a feature flag anyway. so qa-ok
[17:25] <lifeless> henninge: nice! thanks
[17:25] <leonardr> ah, make schema
[17:25] <leonardr> the missing ingredient?
[17:25] <sinzui> leonardr: yes, after after the schema is setup, security.cfg is parsed and the users are setup
[17:28]  * jml heading back home & thus offline
[17:28] <leonardr> sinzui: unfortunately, yes, make schema doesn't stop the error
[17:33] <lifeless> matsubara: hi
[17:33] <lifeless> matsubara: you've probably got a tonne of mail
[17:33] <lifeless> matsubara: but I've got a oops-tools branch you may not have seen
[17:33] <lifeless> matsubara: which I'd love a hand with - test suite doesn't want to play for me
[17:36] <leonardr> oh, wait, it's slightly different! binarypackage*name*
[17:37] <leonardr> great, i just have to make schema over and over until i run out of errors. sinzui, is there a better way?
[17:37] <lifeless> what is going wrong?
[17:39] <leonardr> lifeless: i have a branch that adds some error checking to bugtask creation. the problem is that this requires bugtask creation to access database tables it formerly did not access
[17:39] <leonardr> so when it happens, eg., over email, the database user doesn't have permission to run the checks
[17:39] <sinzui> leonardr: I have not learned of a better way
[17:39] <lifeless> ah checkwatches
[17:39] <lifeless> security.py can update that
[17:39] <lifeless> you need to point it at the ftest *template* db
[17:39] <matsubara> lifeless, yep, saw that one. I'll review it today and investigate the test suite failure
[17:40] <lifeless> matsubara: thank you!
[17:40] <lifeless> matsubara: if you can take it over, that would be awesome.
[17:41] <leonardr> lifeless: i'm intrigued. this is a substitute for 'make schema'? what's the name of the ftest template db?
[17:42] <lifeless> launchpad_ftest_template
[17:42] <lifeless> make schema builds that db
[17:42] <lifeless> and as part of building it invokes security.py pointed at that template
[17:43] <leonardr> that should speed up the process considerably, thanks
[17:43] <lifeless> I don't know the magic runes to make that happen later, but it should be amenable to a little bit of spelunking in database/
[17:45] <leonardr> yes, this is making it go faster
[17:47] <lifeless> it might be nice to have a make target to do this to the ftest template, as a convenience for other developers
[17:49] <lifeless> gary_poster: my first patch to lp using 'with' has test failures, I'll be working through those today.
[17:49] <jam> anyone else running LP in a VM rather than raw on their system?
[17:49] <jam> I managed to get it all set up
[17:49] <jam> but realized I'd really like to access it from the container
[17:49] <jam> so have it serve in the VM's public address
[17:50] <jam> not just on the loopback
[17:50] <lifeless> gary_poster: one of them is in the subscription code - http://pastebin.com/bHJgS6DG - wondering why that is an exact check rather than a LessThan
[17:50] <lifeless> jam: I am
[17:50] <abentley> lifeless: I see a broken link on qastaging that I don't see on production.
[17:50] <lifeless> jam: https://dev.launchpad.net/Running/VirtualMachine
[17:50] <lifeless> abentley: on what page?
[17:51] <abentley> lifeless: the "Set up sharing" on https://translations.qastaging.launchpad.net/bzrtools/trunk
[17:51] <abentley> lifeless: this was not introduced by my code.  So far my code is qa-ok.
[17:52] <jam> lifeless: right, I got that part working
[17:52] <lifeless> abentley: I can't see +sharing-details on production either
[17:52] <jam> I just want to be able to access it from the host
[17:52] <jam> rather than eg having to run Firefox in the VM
[17:52] <abentley> lifeless: Yes, but the link isn't present on production.
[17:52] <lifeless> abentley: ok
[17:52] <lifeless> abentley: is it a feature flag hidden thing, or will everyone see it ?
[17:53] <abentley> lifeless: I believe everyone will see it.  It looks like it was introduced in r12559
[17:53] <lifeless> abentley: translations.sharing_information.enabled is set on qastaging
[17:54] <lifeless> abentley: in the template, I see 'tal:condition="features/translations.sharing_information.enabled'
[17:54] <abentley> lifeless: Okay.  I probably misunderstood henninge.
[17:54] <lifeless> abentley: so it won't show up on prod at all until that flag is enabled
[17:56] <lifeless> s/it/the link that is broken will be hidden/
[17:57] <lifeless> jam: right, I thought I put notes on that on the wiki page
[17:57] <jam> lifeless: there is a small link to an email thread, perhaps in there
[17:59] <lifeless> apparently not
[17:59] <lifeless> jam: so, copy the new stuff from /etc/hosts to your host os
[18:03] <maxb> I have a pending application for ~loggerhead-team, who should I talk to to get a yay or nay?
[18:03] <abentley> lifeless: qa-ok.
[18:05] <leonardr> sinzui: good news is i've got the tests running again. bad news is we have more tests that try to create bad bugtasks
[18:05] <sinzui> we suck
[18:06] <lifeless> abentley: cool, thank you
[18:06] <sinzui> leonardr: I favour deleting any test that possibly duplicates what is done in a lower level test. This is an opportunity to make the test suite faster
[18:07] <leonardr> sinzui: i like this plan but i don't trust my ability to quickly determine that one test duplicates another. let me get the big list of failures up and we can go through it
[18:07] <lifeless> maxb: what do you need that for ?
[18:07] <sinzui> leonardr: that is a good plan
[18:07] <lifeless> maxb: or better phrased: what are you trying to do that you can't today, that ~loggerhead-team membership will let you do
[18:08] <maxb> I would like to help out with getting some of the changes moved to the experimental branch re-landed. One of the convenient ways to do that would be to re-open merge proposals for stuff that has been un-landed.
[18:08] <lifeless> maxb: you don't need any special privilege to do that
[18:08] <maxb> I can't edit the status of merge proposals for other peoples landings on lp:loggerhead - being in ~loggerhead-team would enable this
[18:09] <lifeless> maxb: oh, I see what you're thinking. That won't do the right thing *anyway*
[18:09] <maxb> it wouldn't?
[18:09] <leonardr> sinzui: http://pastebin.ubuntu.com/577955/ is the full list, i'll trim it down to just the initial errors
[18:09] <lifeless> I don't think so
[18:10] <maxb> Doesn't being in the team owning the target branch of a MP entitle you to edit it?
[18:10] <lifeless> maxb: I think it would be better to make a branch containing a specific thing merged out of experimental - probably oldest-first is easy (can avoid cherrypicks)
[18:11] <lifeless> maxb: the problem is the source branch: unless the mp is /perfect already/ it will need further changes
[18:11] <maxb> Right.... I'd hope to convince the original submitter to do them, then :-)
[18:11] <lifeless> maxb: and you can't push to e.g. mkanat's branches
[18:11] <leonardr> sinzui: http://pastebin.ubuntu.com/577957/
[18:11] <lifeless> maxb: so a number of the changes were done on contract to Canonical
[18:12] <leonardr> not too bad, only 4 failures, and one of them is clearly my fault (somehow)
[18:12] <sinzui> I have got the files open
[18:12] <lifeless> maxb: and the contract is over. Its up to 'us' - folk interested in those changes - to pick them up and polish/revisit/land
[18:13] <leonardr> sinzui: i'm looking at the first one. would you take a look at the other 3?
[18:13] <sinzui> leonardr: 30-nominate-bug-for-distrorelease needs history like I added to another test. maybe...
[18:13] <leonardr> i'll push my changes
[18:13] <maxb> alright - not all the MPs I was thinking of were mkanat's, but I could opt for emailing people asking them to reopen their own MPs
[18:14] <lifeless> maxb: Personally, I would just make a new one
[18:14] <lifeless> maxb: we took on the burden of re-reviewing and assessing and fixing those changes when we pivoted the trunks arounds
[18:14] <leonardr> sinzui: lp:~leonardr/launchpad/bug-106338 is up to date
[18:14] <lifeless> maxb: I don't think its all that fair to push back to folk whom we have told their change is done with.
[18:15] <maxb> ok
[18:17] <lifeless> maxb: if you need bug triage permissions, or are going to get involved in code review & qa, I'd be delighted to talk loggerhead-team membership
[18:17] <lifeless> it is however the project committers team, so I think the usual 'get involved, be involved, ok heres the membership' process should apply
[18:18] <maxb> alright, I'll re-raise at an appropriate time
[18:27] <sinzui> leonardr: I think I have 4 fixed. 3 fails trying to show it will fail. The test is bad, but we might have a case where the use can oops the UI
[18:27] <leonardr> no luck on 1 so far
[18:28] <lifeless> sinzui: is there any way on e.g. https://launchpad.net/~loggerhead-team/+editproposedmembers to discuss with the applier why they are applying ?
[18:29] <sinzui> No
[18:29] <sinzui> That is a long requested feature
[18:29] <lifeless> sinzui: is there a bug complaining about same?
[18:34] <jam> lifeless: https://dev.launchpad.net/Running/RemoteAccess looks like it has what I was looking for
[18:35] <sinzui> lifeless: bug 4976
[18:35] <_mup_> Bug #4976: When trying to join a team, user should be able to give comment <feature> <lp-registry> <registry> <Launchpad itself:Triaged> < https://launchpad.net/bugs/4976 >
[18:36] <jam> I added it to the page
[18:36] <lifeless> sinzui: sigh, thats like 3 difference features rolled into one :(
[18:37] <lifeless> sinzui: so many things from users are like that
[18:40] <sinzui> leonardr: I been to build a pristine tree to test it will be a moment: this is what I have done: http://pastebin.ubuntu.com/577973/
[18:41] <leonardr> ok
[18:50] <leonardr> sinzui: the problem with #1 is that the productseries bugtask created by the factory is not conjoined to the bugtask for that productseries' product
[18:51] <leonardr> does that make sense? i'm not sure why it does happen
[18:52] <sinzui> leonardr: it does if the productseries is not the product's development_focus
[18:52] <leonardr> but it is
[18:52] <leonardr> conjoined_bugtask = self.factory.makeBugTask(bug=bug, target=product.development_focus)
[18:55] <leonardr> getConjoinedMaster has the right bugtask in its shortlist but for some reason decides that's not it...
[18:56] <leonardr> sinzui: the bugtask (whose target is a ProductSeries) does not have either IDistroBugTask or IUpstreamBugTask
[18:56] <leonardr> ie. doesn't provide
[18:56] <leonardr> should it be providing IDistroBugTask? that doesn't seem right
[18:57] <leonardr> but that's the only way it can have a conjoined master
[18:57] <sinzui> leonardr: we are testing the IUpstreamBugTask.providedBy condition
[18:58] <leonardr> ok, i get that now. any idea why this task doesn't provide that?
[18:59] <sinzui> leonardr: it doesn't
[18:59] <leonardr> i guess because its .product is none? it's got a productseries but no .product?
[19:00] <sinzui> product==upstream there is even a bug report to rename IUpstreamBugTask to IProductBugTask
[19:00] <sinzui> what!
[19:00] <sinzui> that is F'd up
[19:00] <leonardr> ok, probably a problem with createBugTask
[19:01] <leonardr> or, maybe in the factory
[19:01] <sinzui> is it I would have thought it was makeBugTask() which I know was making invalid series last year
[19:01] <lifeless> thats normal
[19:01] <lifeless>  count
[19:01] <lifeless> -------
[19:01] <lifeless>   7703
[19:01] <lifeless> (1 row
[19:01] <gary_poster> lifeless, pastebin you gave me doesn't work anymore.  can look if you like
[19:02] <lifeless> sinzui: launchpad_qastaging=> select count(*) from bugtask where product is null and productseries is not null;
[19:02] <lifeless> -> 7703
[19:02] <lifeless> sinzui: product is accessed via productseries.product
[19:02] <sinzui> leonardr: last year the factory was creating products without series, which is invalid. I fixed that. I think we just found the reverse is true
[19:02] <leonardr> sinzui: the product and series both _exist_
[19:03] <leonardr> but, the bug task is created with a series and no product, and nobody says 'wait, that's wrong'
[19:03] <lifeless> leonardr: thats how it should be
[19:03] <lifeless> gary_poster: http://pastebin.com/21LHXw6D
[19:03] <sinzui> lifeless: I think I need to put my hands over my ears and hum very loudly. I do not want to hear about how messed up production data is. It took me 4 months to clean up milestones without a series
[19:04] <lifeless> sinzui: this is not messed up
[19:04] <lifeless> sinzui: its /correct/
[19:04] <lifeless> WHEN product IS NOT NULL THEN productseries IS NULL AND distribution IS NULL AND distroseries IS NULL AND sourcepackagename IS NULL
[19:04] <lifeless> sinzui: I think you have an invalid assumption about how it should be, and you're wrong.
[19:05] <sinzui> How do I access a product series that has no product?
[19:05] <leonardr> sinzui: the product series has a product
[19:05] <lifeless> sinzui: thats not what leonardr is saying.
[19:05] <leonardr> the *bug task* has a .productseries but does not have a .product
[19:05] <lifeless> leonardr: thats normal.
[19:05] <leonardr> and the check for conjoined master only looks at .product
[19:05] <lifeless> leonardr: and thats also normal
[19:06] <lifeless> leonardr: the conjoined master is the *nonseries* task
[19:06] <sinzui> lifeless: we are looking at this
[19:06] <sinzui>         product = self.factory.makeProduct()
[19:06] <sinzui>         bug = self.factory.makeBug(product=product)
[19:06] <sinzui>         conjoined_bugtask = self.factory.makeBugTask(
[19:06] <sinzui>             bug=bug, target=product.development_focus)
[19:07] <sinzui> ^ I think there should be a bugtask with .product and a bugtask with .productseries
[19:07] <gary_poster> lifeless no idea on the face of it.  AFAIK I haven't/yellow hasn't touched that particular bit.  I encountered something somewhat like that before and investigated to discover that the difference in query count was in fact indicative of a problem, but that's certainly not the ideal way to show it. ...I'm sorry I can't offer more.
[19:07] <lifeless> sinzui: no, thats not possible.
[19:07] <lifeless> gary_poster: no worries, thanks for peeking
[19:07] <lifeless> sinzui: bugtask links to only one of (product, productseries, ...)
[19:08] <leonardr> lifeless: my only goal in this is to create something that launchpad will treat as a conjoined bugtask, so that i can test its behavior. what do you recommend?
[19:08] <lifeless> upstream or distro
[19:08] <sinzui> lifeless that are 2 bug tasks. how would you setup a conjoined condition in a test. the are not examples in factory as far as I can see
[19:09] <fjlacoste> lifeless: catch-up call?
[19:09] <lifeless> flacoste: yes, but give me 2 minutes to unblock sinzui and leonardr
[19:09] <lifeless> leonardr: upstream or distro ?
[19:10] <sinzui> lifeless: leonardr, I am mistaken, there is a test and it does what we are doing in the broken test
[19:10] <lifeless> task = factory.makeBugTask()
[19:11] <lifeless> conjoined = factory.makeBugTask(bug=task.bug, target=factory.makeProductSeries(product=task.product)
[19:11] <lifeless> sinzui: ^ those two lines will make a conjoined master for upstream
[19:12] <lifeless> erm, not quite
[19:12] <lifeless> conjoined = factory.makeBugTask(bug=task.bug, target=task.product.devel_focus)
[19:12] <lifeless> sinzui: ^ that will do it
[19:12] <sinzui> I am looking at a test that does this:
[19:12] <sinzui>     self.factory.makeBugTask(
[19:12] <sinzui>                 bug=bug, target=self.product.development_focus)
[19:12] <lifeless> following the definition of 'a conjoined master is a series task on the development focus of a project'
[19:13] <lifeless> what we should do is drop product bugs altogether.
[19:13] <sinzui> which identical to: self.factory.makeBugTask(
[19:13] <sinzui>             bug=bug, target=product.development_focus)
[19:13] <lifeless> sinzui: if your product is the same product
[19:14] <sinzui> yes, we make the bug with the same product, I pasted the lines
[19:14] <sinzui>         product = self.factory.makeProduct()
[19:14] <sinzui>         bug = self.factory.makeBug(product=product)
[19:14] <sinzui>         conjoined_bugtask = self.factory.makeBugTask(
[19:14] <sinzui>             bug=bug, target=product.development_focus)
[19:14] <lifeless> cool
[19:14] <sinzui> we could
[19:14] <sinzui>         product = self.factory.makeProduct()
[19:14] <sinzui>         bug = self.factory.makeBug()
[19:14] <sinzui>         conjoined_bugtask = self.factory.makeBugTask(
[19:14] <sinzui>             bug=bug, target=product.development_focus)
[19:14] <sinzui> ^ disregard that
[19:14] <lifeless> so, the quest is
[19:15] <sinzui> lifeless: but the issue is that the test fails
[19:15] <lifeless> what is the default_bugtask
[19:15] <lifeless> I think you should not call makeBug
[19:15] <lifeless> but call makeBugTask twice, because you *do care* about the target.
[19:16] <lifeless> that shouldn't affect the validity though
[19:16] <lifeless> sinzui: have you looked at what ends up in the db ?
[19:16] <lifeless> something I do in this sort of situation is put a import pdb;pdb.set_trace() call at the point of failure, and then go sniffing
[19:17] <lifeless> sinzui: I'll chat to flacoste and come back to help you in a bit
[19:17] <sinzui> that was what I was thinking. leonardr. we might need a IStore(conjoined_bugtas).flush()
[19:17] <leonardr> sinzui: in createBugTask?
[19:18] <sinzui> after the test created conjoined_bugtask, we can explicitly flush the state to the db for the query that is test a few lines later
[19:18] <leonardr> i see
[19:19] <leonardr> sinzui: does it change your suggestion to know that conjoined_bugtask.conjoined_master is None immediately after it's created?
[19:20] <sinzui> no, that is a sql query
[19:20] <leonardr> ok
[19:20] <leonardr> i disagree--i don't think it gets to the sql, but i'll try it
[19:23] <leonardr> sinzui: after the flush, conjoined_bugtask.conjoined_master is still None and I can still set that bugtask's status to Invalid without raising an exception
[19:24] <sinzui> leonardr: your line of code does not make sense. I think you are confused again. the series task is the master, the product/distro is the slave so  conjoined_bugtask.conjoined_master should always be None
[19:24] <sinzui> conjoined_bugtask.conjoined_slave == product
[19:25] <leonardr> so i am editing the wrong task?
[19:25] <sinzui> maybe product==slave, series==master
[19:27] <sinzui> conjoined bugs are like X servers. The relationship is reversed. It is like the odd fact that seahorse males give birth, not femals
[19:29] <leonardr> yes, that was the problem
[19:29] <leonardr> man
[19:29] <sinzui> leonardr: I just confirmed that 30-nominate-bug-for-distrorelease.txt is fixed by my change. I need to rethink 10-bug-requestdistrofix.txt.
[19:29] <leonardr> ok, we're getting there
[19:30] <sinzui> leonardr: I had to restore this line in sourcepackagerecipe.py to get the tests to run
[19:30] <sinzui> from lp.registry.interfaces.pocket import PackagePublishingPocket
[19:35] <lifeless> flacoste: grab devpad:~mthaddon/banana.html
[19:39] <sinzui> leonardr: #3 is a duplicate of bugtask editing tests. I will remove the redundant parts and hope for a pass
[19:44] <lifeless> flacoste:     WHEN productseries IS NOT NULL THEN distribution IS NULL AND distroseries IS NULL AND sourcepackagename IS NULL
[19:45] <leonardr> sinzui: ok, my side is committed and pushed
[19:45] <lifeless> flacoste: https://lpstats.canonical.com/graphs/AppServerRequestLpnet/
[19:46] <sinzui> leonardr: did you apply my changes to 30-... from http://pastebin.ubuntu.com/577973/
[19:46] <leonardr> sinzui: no, doing that now
[19:52] <lifeless> flacoste: http://www.mongodb.org/display/DOCS/Replication
[19:57] <leonardr> sinzui: for some reason my bin/test is not finding 30-nominate-bug
[19:57] <leonardr> but, i have applied your change
[19:57] <sinzui> leonardr: the test runner does not let you run tests our of sequence
[19:58] <sinzui> leonardr: you need to test the whole directory
[19:58] <sinzui> we were smoking a lot of crack in 2005 and 2006
[19:58] <leonardr> ah, i see. i was wondering how that test was run earlier, but earlier i'd run the whole directory
[20:00] <leonardr> ok, your test fixes it
[20:00] <mwhudson> i wonder how hard it would be to remove that sequential test feature
[20:00] <mwhudson> probably not hard, just very annoying
[20:02] <sinzui> leonardr: my effort to fix 10-... broke 20-... so I am fixing that too :(
[20:02] <leonardr> ok, don't be discouraged
[20:05] <jcsackett> sinzui: skipping our 1to1, based on how busy you seem to be?
[20:05] <sinzui> jcsackett: yes, we can try in an hour
[20:05] <jcsackett> sinzui: sure.
[20:06] <jcsackett> lifeless: any chance you would be available to talk a bit about bug 716786?
[20:06] <_mup_> Bug #716786: Product:+filebug-show-similar timeouts <timeout> <Launchpad itself:Triaged> < https://launchpad.net/bugs/716786 >
[20:06] <lifeless> sure
[20:06] <lifeless> in a little bit
[20:06] <jcsackett> cool.
[20:14] <leonardr> sinzui: argh, looking at the failure message more closely i see that i neglected failures in unit tests
[20:14] <sinzui> :(
[20:34] <sinzui> leonardr: my branch that increments lazr.restful to 0.17.4 passed, but was bounced because we are in rc mode. I think my branch will land before yours. maybe you should say your branch requests the same version before you submit it again
[20:35] <leonardr> sinzui: makes sense
[20:39] <sinzui> leonardr: I got rid of the duplicate parts in the 10-* test. There is still an error. I believe that bugalsoaffect.BugTaskCreationStep is not validating before creating the bugtask! We need to fix this in this branch
[20:40] <leonardr> sinzui: ok, give me a pastebin and i'll try to figure out what you mean
[20:41] <sinzui> leonardr: I mutated by test, but you can see it in your own paste. #2 and #3 are the same error. The view assumes the selected target is sane: http://pastebin.ubuntu.com/577957/
[20:49] <leonardr> sinzui: what would it mean to validate before creating the bugtask? we would call validate_new_distrotask/valid_upstreamtask and catch the exception?
[20:51] <leonardr> what would happen if createBugtask just threw a WidgetError? (that's what should happe nnow)
[20:52] <thumper> anyone know what happened to the builders?
[20:52] <thumper> they both seem to have failed mid test
[20:52] <thumper> with what looks like a missing database
[20:55] <sinzui> leonardr: I think we are too far into the view to raise an error. The field value (a distro in the test I am reading) is invalid. I do not think we should have entered the method. I expect something to set fieldError()
[20:57] <leonardr> sinzui: in that case... we have a lot of error checking code in createTask (determining the order of precedence). should we create a method that takes similar arguments to createTask and makes the check? then the view can catch the error and set fieldError
[20:57] <sinzui> oh, leonardr: I think DistroBugTaskCreationStep.validateStep() is at fault
[20:58] <leonardr> yeah, something like that. except that code doesn't know about the precedence order we decided on? is that hte problem?
[21:00] <lifeless> jcsackett: hi
[21:00] <jcsackett> lifeless: hi.
[21:00] <jcsackett> i think there may be a conflict in times to talk, unless sinzui is still too busy for 1-to-1, or is good with doing it tomorrow.
[21:01] <sinzui> jcsackett: we can talk tomorrow
[21:01]  * sinzui is free all dat
[21:01] <sinzui> day
[21:01] <jcsackett> okay, lifeless, no conflict after all. :-)
[21:01] <thumper> leonardr: mumble ping
[21:01] <lifeless> jcsackett: I can wait for you to talk with sinzui
[21:01] <leonardr> k
[21:02] <jcsackett> lifeless: sinzui and i can chat tomorrow--it's not a pressing thing, just  a regular checkin.
[21:02] <lifeless> ok
[21:02] <lifeless> so how can I help you
[21:02] <sinzui> lifeless: leonardr and I want to land this branch and never speak of it again
[21:02]  * jcsackett wants them to land the branch as well.
[21:03] <jcsackett> so, lifeless, i'm looking at bug 716786
[21:03] <_mup_> Bug #716786: Product:+filebug-show-similar timeouts <timeout> <Launchpad itself:Triaged> < https://launchpad.net/bugs/716786 >
[21:03] <jcsackett> the one about timeouts with insane fti strings.
[21:03] <lifeless> right
[21:03] <leonardr> jcsackett: actually you're in good shape, you just need the new version of lazr.restful, which will be installed anyway when _sinzui's_ branch lands
[21:03] <jcsackett> leonardr: oh, cool.
[21:03] <leonardr> my problem is that i pulled on a thread that turned out to be the tail of a dragon
[21:04] <jcsackett> lifeless: my naive thought, based on a comment by stub on a related bug, is that maybe we just need to kick out any super long bug titles?
[21:04] <jcsackett> but a) i'm not sure that's best and b) i'm not sure what the cap should be if it is.
[21:04] <lifeless> jcsackett: why does it do this search at all ?
[21:05] <jcsackett> show similar is using an fti search to find any bugs similar to the one you might be reporting.
[21:05] <jcsackett> so you can say "oh, yeah, that's the issue" rather than filing a new bug.
[21:05] <sinzui> leonardr: I am uncertain what happened in the two examples. I see DistroBugTaskCreationStep.validateStep() and it does validate_new_distrotask. either we skipped that line, or multistep() never called validateStep().
[21:05] <leonardr> sinzui: on call, 1 min
[21:06] <lifeless> jcsackett: oh, its not actually on the bug filing
[21:06] <lifeless> its in the prepatory workflow ?
[21:06] <jcsackett> no, though bug 722787, which you filed against +filebug, is the from the same thing.
[21:06] <_mup_> Bug #722787: Product:+filebug timeouts <timeout> <Launchpad itself:Triaged> < https://launchpad.net/bugs/722787 >
[21:06] <lifeless> so title=%5Barmel%5D%20mawk%20testsuite%20fails%20(-fif-conversion2%20problem) is the sesarch string
[21:06] <lifeless> so title=%5Barmel%5D%20mawk%20testsuite%20fails%20(-fif-conversion2%20problem) is the sesarch string
[21:07] <lifeless> jcsackett: that sample query runs in 800ms on qastaging
[21:07] <lifeless> jcsackett: this may not need code changes
[21:07] <jcsackett> lifeless: i lost a bit there; accidentally closed the channel.
[21:07] <lifeless> jcsackett: I repasted
[21:07] <jcsackett> ah.
[21:08] <jcsackett> so, the problem in fti is that whatever you search on, gets redone N ways, where N is number of tokens in your search string left over.
[21:08] <lifeless> yes, I put that together as a compromise
[21:08] <lifeless> we used to do some very expensive preprocessing to filter out common terms
[21:08]  * jcsackett nods.
[21:09] <jcsackett> so, you think this may not need code changes?
[21:09] <jcsackett> is our full text index just not up to snuff?
[21:09]  * jcsackett confesses fair amount of ignorance on tsearch2 and the like.
[21:10] <lifeless> jcsackett: I think its db layer, stub bounced it as crazy, but I want to talk through it with him
[21:10] <jcsackett> fair. i'll mosey on to something else then. :-)
[21:11] <lifeless> we may need to do a context-term document frequency index
[21:11] <lifeless> to permit getting back what we had
[21:11] <lifeless> -or-
[21:11] <lifeless> we need to move ahead on the lucene thing
[21:11] <jcsackett> lucene is pretty hot. we used it at my last job.
[21:11] <jcsackett> we were using mysql full text prior, so *huge* improvement.
[21:15] <sinzui> leonardr: I am still stepping through. We are validating, and I can see that we did a sane validation, so I now looking at the base class to see it if is creating the bugtask without the sourcepackage (which is what was validated)
[21:16] <leonardr> k
[21:16] <lifeless> jcsackett: may I humbly suggest bug 727023
[21:16] <_mup_> Bug #727023: Cve:+index timeouts <timeout> <Launchpad itself:Triaged> < https://launchpad.net/bugs/727023 >
[21:17] <lifeless> jcsackett: 20 timeouts a day
[21:17] <lifeless> jcsackett: needs eager loading
[21:17] <jcsackett> lifeless: sure, i'll take a look at that.
[21:19] <leonardr> sinzui: off the phone
[21:19] <thumper> sinzui: I'm getting google service problems
[21:19] <thumper> RuntimeError: Socket poll time exceeded.
[21:19] <thumper> starting the google service layer
[21:19] <thumper> WTF?
[21:20]  * thumper tries to rebuild
[21:23] <thumper> yeah, rebuild fixed it
[21:25] <sinzui> thumper: is it because you branch still says it changed something in lp.services.googlesearch
[21:25] <thumper> sinzui: no, this is the other branch :)
[21:27] <sinzui> thumper: you may need to do bin/buildout if you merged the change from from canonical or the rename from .search to .googlesearch
[21:27] <thumper> sinzui: yeah, the make fixed that
[21:29] <sinzui> leonardr: the data passed to validateStep and main_action is different. We validated spn=alsa-utils, but the action is working with spn=mozilla-firefox
[21:30]  * sinzui now wonders if there is a very old and deep bug in the code
[21:30] <leonardr> sinzui: remind me which test file you are in?
[21:30] <sinzui> xx-duplicate-bugwatches.txt, the first test
[21:31] <sinzui> leonardr: we are on mozilla-firefox, but we are adding alsa-utils
[21:31] <sinzui> We validate alsa-utils, but the main_action thinks we working with mozilla-firefox
[21:33] <leonardr> sinzui: ok, and which class has the main_action?
[21:35] <sinzui> leonardr: It was DistroBugTaskCreationStep.main_action. I am stepping though it again and I see everything right so far
[21:40] <lifeless> sinzui: leonardr: would you like me to look as well? if so whats the branch
[21:40] <leonardr> lifeless: knock yourself out. lp:~leonardr/launchpad/bug-106338
[21:40] <leonardr> i'll forward you the failure message
[21:40] <leonardr> we've fixed all but one error in the pagetests, so it's just the unit test failures after shis
[21:41] <lifeless> leonardr: if its under control I'll leave it with you guys; if you need a hand I'll pull the branch and have a look
[21:42] <leonardr> sinzui: how much longer can you spend on this?
[21:44] <sinzui> leonardr: we get a LaunchpadValidationError creating a task for debian alsa-utils. This causes the view to attempt to recover (maybe by backing up a step) and it rebuilds the data from the existing context.
[21:45] <leonardr> sinzui: so instead we should set a field error and let it go?
[21:45] <sinzui> leonardr: so I need to see know why the call to validate passed, but the call to create failed
[21:45] <leonardr> sinzui: probably because the call to validate was much simpler than the equivalent validation code in create
[21:45] <leonardr> remember the complicated order-of-operations code i wrote yesterday?
[21:46] <leonardr> that calls validate_new_distrotask in two places and calls valid_upstreamtask if neither of those was triggered?
[21:46] <leonardr> i bet the validate() only calls valid_upstreamtask
[21:47] <leonardr> well, we only call validate_new_distrotask
[21:48] <leonardr> however, a casual glance shows that in this case, createTask should be running the same code as validateStep()
[21:50] <StevenK> lifeless: I was having trouble working out the bug you said was in the temp directory handling, but first I wanted to get a pointer or two on fixtures, since I've not used them before.
[21:51] <lifeless> StevenK: from fixtures import TempDir
[21:51] <lifeless> with TempDir() as dir:
[21:51] <lifeless>     cwd = dir.path
[21:51] <lifeless>    ...
[21:52] <lifeless> StevenK: fixtures implement the context manager protocol
[21:52] <StevenK> lifeless: So, just replace the outside try: with that, and it will create the directory and remove it when it falls out of scope?
[21:52] <lifeless> StevenK: the bug is that you make a directory and may not clean it up because you don't have a try: finally:
[21:52] <lifeless> StevenK: yes.
[21:53] <StevenK> lifeless: Er, I do have a try: finally:. I might not in the test, but I do in the code
[21:54] <lifeless> StevenK: whats the mp again ?
[21:54] <leonardr> sinzui: let me throw this out there. the problem is that validate_new_distrotask is called twice, once for the source package and once for the distribution
[21:54] <leonardr> it should only be called for the source package
[21:54] <leonardr> that's what i'm seeing, anyway
[21:55] <StevenK> lifeless: https://code.launchpad.net/~stevenk/launchpad/populate-spr-changelogs/+merge/52363
[21:55] <wgrant> Do WIP MPs send mail still?
[21:55] <lifeless> StevenK: ok, so your test is definitely buggy
[21:56] <lifeless> StevenK: and your try:finally: is ok  but would be improved by using the fixture I think
[21:56] <StevenK> lifeless: Agreed. I'll look at them, thanks for the pointer and the help.
[21:57] <lifeless> StevenK: anytime
[21:58] <sinzui> leonardr: sorry, unity went belly up. validate_new_distrotas()k and bugtask.createTask() disagree about debian/alsa-utils
[21:59] <thumper> wallyworld: can you update your cards on the kanban board?
[21:59] <leonardr> sinzui: i got it
[22:00] <leonardr> the _reason_ it was trying again with firefox was that createTask was checking alsa-utils and then debia
[22:00] <leonardr> it shouldn't have been checking debian
[22:01] <leonardr> sinzui: see if this makes sense to you: http://pastebin.ubuntu.com/578052/
[22:03]  * leonardr hopefully checks to see if that fixed other bugs, even though it's unlikely
[22:04] <sinzui> leonardr: I think that may do the trick. I am in the condition that your paste is trying to stop
[22:04] <leonardr> yeah, not really fixing other bugs, oh well
[22:04] <sinzui> leonardr: does that fix xx-duplicate-bugwatches?
[22:05] <leonardr> sinzui: yes
[22:05] <leonardr> but not, apparently, anything else
[22:06] <sinzui> leonardr: yes. I think the first chunk I remove from the the 10.* test was needed. I will try your changes with mine
[22:06] <leonardr> ok, then we can spend a little time on the unit test failures
[22:08] <wallyworld> thumper: will do
[22:09] <leonardr> i think these test failures need a call to factory.makeDistroSeries
[22:17] <leonardr> sinzui: ok, i know what's going on for one of these tests, but i'm not sure what to do
[22:18]  * sinzui listens
[22:18] <leonardr> the test is calling self.factory.makeBugTask(sourcepackage)
[22:18] <leonardr> this code runs in makeBugTask
[22:18] <leonardr>         if ISourcePackage.providedBy(target):
[22:18] <leonardr>             # We can't have a series task without a distribution task.
[22:18] <leonardr>             prerequisite_target = target.distribution.getSourcePackage(
[22:18] <leonardr>                 target.sourcepackagename)
[22:18] <leonardr> that gives us a DistributionSourcePackage
[22:19] <leonardr> then we create a task for that DistributionSourcePackage, and that raises an error
[22:19] <leonardr> LaunchpadValidationError: u'Package generic-string927887 not published in Distribution927888'
[22:19] <sinzui> leonardr: I reverted my changes to 10-*. bug-also-affects passed with your change. So I think the doctests are now addressed.
[22:20] <leonardr> how can we get that error when prerequisite_target *is* the package generic-string... as published in Distribution...?
[22:20] <leonardr> that's why i said maybe we need to call makeSourcePackagePublishingHistory
[22:20] <leonardr> but i'm not sure how, or if that will really help
[22:20] <sinzui> leonardr: makeSourcePackagePublishingHistory() will fix that
[22:21] <leonardr> ok, 1) how do i call it, 2) what if there already is a publishing history?
[22:21] <sinzui> makeSourcePackagePublishingHistory(series=series, sourcepackagename=dsp.sourcepackagename)
[22:22] <leonardr> what is 'series' here?
[22:22] <leonardr> target?
[22:22] <sinzui> leonardr: a distroseries possibly the development_focus of the distro
[22:23] <sinzui> leonardr: since sample data has bugger all for SPPH, there is little chance we can collide. since SPPH is a record of changes, adding publishing records jut means a new version was released
[22:24] <wgrant> thumper: Do we send email for WIP MPs still?
[22:24] <thumper> wgrant: not on creation
[22:24] <lifeless> wgrant: not when creaeted WIP IIRC. possibly not on changes either.
[22:24] <wgrant> Great, thanks.
[22:24] <thumper> wgrant: there are a few nitty bits around commenting on WIP
[22:24] <thumper> if you change the description or commit message, it doesn't send email
[22:25] <lifeless> jcsackett: why is bug 731841 critical?
[22:25] <_mup_> Bug #731841: rosetta-admins team cannot change project translation settings <Launchpad itself:Triaged> < https://launchpad.net/bugs/731841 >
[22:25] <thumper> if you request reviews, it "shouldn't", I'm not sure if it odes
[22:25] <jcsackett> lifeless: i misread it earlier, thought they were getting an OOPS, not just a "you're not allowed here"
[22:26] <lifeless> fixing
[22:26] <leonardr> ok, that got rid of a lot of the errors
[22:26] <jcsackett> dig.
[22:27] <leonardr> sinzui: http://pastebin.ubuntu.com/578065/
[22:28] <sinzui> leonardr: that looks good
[22:28] <leonardr> ok, on to the next one
[22:28] <leonardr> we have this code:
[22:28] <leonardr>                bug = self.factory.makeBug(
[22:28] <leonardr>                     distribution=self.searchtarget.distribution)
[22:28] <leonardr>                 bugtask = self.factory.makeBugTask(
[22:28] <leonardr>                     bug=bug, target=self.searchtarget)
[22:29] <leonardr> that gives this error, i think correctly:
[22:29] <leonardr> This bug is already open on Distribution360034 with no package specified. You should fill in a package name for the existing bug.
[22:30] <sinzui> leonardr: I think the test is bad.
[22:30] <sinzui> we should provide a package or makeup a new distro
[22:30] <leonardr> self.searchtarget is a DistributionSourcePackage
[22:30] <leonardr> it has a .sourcepackagename
[22:31] <leonardr> so maybe it's not a bad test?
[22:34] <leonardr> sinzui: that error is given by validate_new_distrotask(bug, distribution, sourcepackagename) when sourcepackagename is not None
[22:34] <leonardr> i would only expect it when sourcepackagename is None
[22:35] <leonardr> ARGH
[22:35] <leonardr> now i understand the error message
[22:35] <leonardr> it's saying "you shouldn't be filing this bug"
[22:35] <thumper> leonardr: hi, how to I stop exporting an attribute in devel?
[22:35] <leonardr> thumper: pass in ('devel', dict(exported=False)) right after the field object
[22:36] <thumper> leonardr: no need
[22:36] <leonardr> k
[22:36] <thumper> leonardr: it is only exported in devel
[22:36] <thumper> not in the others
[22:36] <thumper> so I can just kill it
[22:36] <leonardr> yup
[22:36] <leonardr> sinzui: so, yes, i think the test is wrong. what would be a better test?
[22:36] <sinzui> leonardr: yes, the test is wrong, not the code
[22:36] <leonardr> i don't think "provide a package or make up a new distro" makes sense here, but i could be wrong
[22:36] <sinzui> we can create a new dsp
[22:37] <leonardr> sinzui: for context, this is test_bugtask_search.py line 327
[22:37] <wgrant> sinzui: I hear you... but I'm muted, and Unity is not showing Mumble's notification area icon.
[22:38] <wgrant> sinzui: Any hints?
[22:46] <leonardr> sinzui, i am eod and then some. i've pushed my changes. did i give you the list of unit tests that failed?
[22:46] <sinzui> leonardr: no you did not. I have few hours left in my day
[22:47] <leonardr> ok, let me give you the master list and you can chip away at it. i think there are only 2 or 3 distinct bugs, despite the huge number of failure
[22:47] <leonardr> s
[22:47] <leonardr> sinzui: http://pastebin.ubuntu.com/578071/
[23:09] <lifeless> ugh
[23:09] <lifeless> did something break date formatting recently?
[23:09] <lifeless> ----------------------------------------------------------------------
[23:09] <lifeless> File "lib/lp/registry/tests/../stories/webservice/xx-project-registry.txt", line 932, in xx-project-registry.txt
[23:09] <lifeless> Failed example:
[23:09] <lifeless>     print milestone['date_targeted']
[23:09] <lifeless> Differences (ndiff with -expected +actual):
[23:09] <lifeless>     - 2005-06-06T00:00:00
[23:09] <lifeless>     + 2005-06-06
[23:10] <lifeless> StevenK: wgrant: you guys watch CI like hawks - seen anything like that recently ?
[23:11] <StevenK> lifeless: I can't recall anything
[23:11] <StevenK> r12561 on Jenkins had no failures.
[23:12] <wgrant> lifeless: I've never seen that before.
[23:12] <wgrant> That field was changed a few months ago.
[23:12] <lifeless>     for milestone in firefox.milestones:
[23:12] <lifeless>         print '%-7s %s' % (milestone.name, milestone.dateexpected)
[23:12] <lifeless> Differences (ndiff with -expected +actual):
[23:12] <lifeless>     - 1.0     2056-10-16 18:31:44.293448
[23:12] <lifeless>     + 1.0     2056-10-16
[23:12] <lifeless>       1.0-rc1 2018-10-01
[23:12] <wgrant> Maybe lazr.restful has changed?
[23:13] <leonardr> the changes to it haven't landed recently, and there weren't any changes like that...
[23:13] <wgrant> lifeless: Where are you seeing this failure?
[23:14] <bac> hi sinzui, quick JS test question?
[23:14] <sinzui> I hope I know the answer
[23:15] <lifeless> wgrant: my storm branch
[23:15] <lifeless> wgrant: my with branch, I mean
[23:15] <wgrant> Maybe Storm has started returning proper dates...
[23:16] <lifeless> oh that would be just typical
[23:16] <wgrant> 386. Merged datetime-in-datevariable
[23:16] <wgrant> Last rev on lp:storm
[23:16] <lifeless> yeah
[23:17] <lifeless> I'll generate a storm tarball without it
[23:17] <wgrant> Please also push up a branch with it backed out...
[23:17] <lifeless> yes
[23:17] <lifeless> thats what I'm doing
[23:17] <wgrant> Great.
[23:18] <lifeless> after I confirm it, of course
[23:20] <lifeless> I've also got this weird thing with subscription tests
[23:20] <lifeless> one less query than expected
[23:20] <wgrant> Hah
[23:21] <wgrant> Reduce the number of queries issued when retrieving an object with
[23:21] <wgrant> Store.get() for which we have an existing invalidated Python object
[23:21] <wgrant> available.
[23:21] <StevenK> And that's a problem?
[23:21] <wgrant> Oh, that is awesome.
[23:21] <wgrant> Merge it merge it merge it.
[23:21] <lifeless> StevenK: it is when the tests are written with an exact match
[23:21] <StevenK> It certainly got wgrant excited
[23:21] <wgrant> It's going to make query tests not suck.
[23:22] <lifeless> yeah
[23:25] <StevenK> lifeless: http://pastebin.ubuntu.com/578091/
[23:27] <lifeless> with TempDir as tmp_dir:
[23:27] <lifeless> should be
[23:27] <lifeless> with TempDir() as tmp_dir:
[23:27] <lifeless> and
[23:28] <lifeless> +                    tmp_dir, 'extracted')],
[23:28] <lifeless> should be
[23:28] <lifeless> +                    tmp_dir.path, 'extracted')],
[23:28] <lifeless> ditto on its other uses
[23:29] <lifeless> StevenK: the fixture isn't a path itself, its an object with the path on .path
[23:29] <StevenK> Right
[23:29] <StevenK> lifeless: Then http://pastebin.ubuntu.com/578092/ should be good
[23:30] <lifeless> looks good
[23:30] <lifeless> tests will tell you
[23:30] <StevenK> Sure, I just wanted to hear your thoughts
[23:31] <lifeless> sure :) just saying my built in interpreter is an imperfect oracle :)
[23:31] <lifeless> as for my branch
[23:31] <lifeless> down to just a checkwatches fail
[23:32] <lifeless> wgrant:
[23:32] <lifeless>     - ...SELECT 1 FROM BugTracker WHERE BugTracker.id = ...
[23:32] <lifeless>     - ...Transaction completed, status: Active...
[23:32]  * StevenK waits for the car alarm to SHUT UP
[23:33] <lifeless> wgrant: i don't see that subselect
[23:33] <lifeless> and the last line of the timeline is
[23:33] <lifeless>     + 00016-00016@SQL-nostore Transaction completed, status: Committed
[23:36] <StevenK> RARGH, need to run make schema before tests, HULK SMASH
[23:39] <lifeless> wgrant: for future reference https://code.launchpad.net/~lifeless/storm/with-without-datetime
[23:41] <lifeless> wgrant: lp:~lifeless/launchpad/bug-731679 is all go except this checkwatches change now.
[23:43] <lifeless> wgrant: I'm nagging you cause the test looks like one you did :)
[23:47] <wgrant> lifeless: Back, sorry.
[23:47] <wgrant> lifeless: Do you have a full paste of that error?
[23:47] <lifeless> can do
[23:48] <wgrant> I think just dropping the SELECT 1 statement is probably best, but I'll check.
[23:48] <lifeless> http://pastebin.com/MV3yQfrt
[23:48] <wgrant> StevenK: Does loganberry have a memcache set up?
[23:49] <wgrant> It might not.
[23:49] <wgrant> Check the configs.
[23:50] <wgrant> lifeless: Replace the "...SELECT 1 [...]" line with "...SELECT BugTracker...", I guess. That test will hopefully die next week, but we might as well keep it mildly useful until then.
[23:50] <lifeless> what happened to the SELECT 1
[23:50] <wgrant> The Storm revision eliminates that.
[23:51] <lifeless> righto
[23:51] <wgrant> On an invalidated object it used to do a SELECT 1 to check its existence, then select its data when it was used.
[23:51] <wgrant> Now it just grabs the data in the first query.
[23:52] <wgrant> On a related matter, what is your opinion on LessThan vs Equals for query test?
[23:52] <wgrant> +s
[23:53] <wgrant> I think Equals is probably better, as it ensures we are never regressing.
[23:53] <lifeless> depends on the test
[23:53] <lifeless> so
[23:54] <lifeless> it would be sad to have to change 5K tests when we fix traversal to be better
[23:55] <wgrant> True.
[23:57] <lifeless> ok, pqmed
[23:57] <lifeless> wgrant: I think things that are really isolated and specific should be specific
[23:57] <wgrant> Right.
[23:57] <lifeless> wgrant: things that are not really isolated and depend on the behaviour of many subcomponets should be less sensitive
[23:57] <lifeless> wgrant: while still providing a solid backstop
[23:58] <StevenK> wgrant: Can you recall if there a bug for older SPRs not having .changelog?
[23:58] <wgrant> StevenK: I don't think so, but I don't know for sure.
[23:58] <StevenK> Meh, filing one