| sinzui | wgrant: mumble? | 00:01 |
|---|---|---|
| wgrant | sinzui: You can't hear me? | 00:02 |
| sinzui | Once again I think I need to restart to hear | 00:02 |
| lifeless | I'm amazed our test suite works at all | 00:09 |
| lifeless | os.environ['HOME'] = self.root | 00:10 |
| lifeless | in the buildd tactestsetup affects all subsequent tests. | 00:11 |
| lifeless | but | 00:11 |
| lifeless | it helpfully deletes that directory | 00:11 |
| wgrant | Heh | 00:11 |
| lifeless | so when rabbit tries to start up | 00:12 |
| lifeless | it uses that as home | 00:12 |
| lifeless | which doesn't exist | 00:12 |
| wgrant | Ahh | 00:12 |
| lifeless | two bugs: rabbit should isolate itself. | 00:12 |
| lifeless | and we should restore global state we futz with | 00:12 |
| lifeless | \o/ | 00:13 |
| lifeless | bug 780794 if you're interested | 00:14 |
| _mup_ | Bug #780794: tests of/using BuilddSlaveTestSetup mangle global state for other tests <build-infrastructure> <Launchpad itself:Triaged> < https://launchpad.net/bugs/780794 > | 00:14 |
| sinzui | wgrant: I am perplexed. My test script only passes the person changes. None of the question_target methods work | 00:30 |
| wgrant | sinzui: Did you change the inheritance hierarchy? | 00:30 |
| wgrant | sinzui: Each object has a single interface exported. | 00:31 |
| wgrant | So eg. IProduct needs to inherit IQuestionTarget. | 00:31 |
| sinzui | No, But I am wondering I I need to | 00:31 |
| wgrant | Product cannot implement IProduct and IQuestionTarget directly. | 00:31 |
| sinzui | okay, that is my mistake | 00:32 |
| sinzui | wgrant The branch is okay to release, but I need to followup with another branch to pass my test | 00:33 |
| wgrant | sinzui: Great, thanks. | 00:33 |
| LPCIBot | Yippie, build fixed! | 00:36 |
| LPCIBot | Project db-devel build #535: FIXED in 5 hr 31 min: https://lpci.wedontsleep.org/job/db-devel/535/ | 00:36 |
| lifeless | does Failure in test testEpochNonInteger (lp.archivepublisher.tests.test_debversion.VersionTests) | 00:49 |
| lifeless | happen for anyone else? | 00:49 |
| wgrant | Have you run update-sourcecode lately? | 00:51 |
| wgrant | That test is new. | 00:51 |
| wgrant | 30 rev deploy time :( | 00:53 |
| lifeless | no, I haven't | 00:55 |
| wgrant | lifeless: It was a bug in python-debian. | 00:59 |
| wgrant | So update-sourcecode and try again. | 00:59 |
| wgrant | Greeeeeeeen | 01:01 |
| wgrant | zomg | 01:02 |
| wgrant | Assignee picker on +filebug works on qastaging. | 01:02 |
| LPCIBot | Project windmill-devel build #63: STILL FAILING in 44 min: https://lpci.wedontsleep.org/job/windmill-devel/63/ | 01:20 |
| wgrant | Hmm. | 01:40 |
| wgrant | I wonder how many users the API export of archive.checkUpload has. | 01:41 |
| * wgrant checks the PPR. | 01:42 | |
| lifeless | \o/\o/\o/\o/\o/\o/ | 01:44 |
| wgrant | ? | 01:44 |
| wgrant | Rabbit working? | 01:44 |
| lifeless | yup | 01:44 |
| lifeless | lp-land in a second | 01:44 |
| wgrant | !!! | 01:44 |
| wgrant | Did you decrapify the U1 harness a bit? | 01:45 |
| lifeless | a tad more | 01:45 |
| wgrant | Great. | 01:45 |
| wgrant | Next stop: wildcherry. | 01:45 |
| lifeless | but I want the darn thing in tree | 01:45 |
| wgrant | Yeah. | 01:45 |
| lifeless | so that the next 'if we had queues' comment I see I can drop the bomb on | 01:45 |
| wgrant | Yes yes yse. | 01:45 |
| lifeless | of course, we have 1 person (jtv) with srs queue experience | 01:45 |
| lifeless | so its not at cargo cult point yet | 01:46 |
| wgrant | :( 127 checkUpload calls last month. | 01:46 |
| wgrant | Maybe I can grep logs and destroy whoever is doing that. | 01:46 |
| jtv | Stereotactic RadioSurgery? | 01:46 |
| wgrant | Because it's a shit API. | 01:46 |
| lifeless | jtv: serious | 01:47 |
| jtv | lifeless: ah... so, exciting rabbit things happening? Gimme! | 01:47 |
| lifeless | :) | 01:50 |
| jtv | oh damn crappy shitty damn bad connection I smegging hate this | 01:52 |
| lifeless | :> | 01:52 |
| lifeless | so I answered you with | 01:52 |
| lifeless | :) | 01:52 |
| jtv | lifeless: got the smileys, and would be joining in the smiles if it weren't for the broken connections! | 01:55 |
| spm | o/ jtv | 01:55 |
| jtv | Hi spm! | 01:55 |
| jtv | And the good thing about bad connections is you get to keep saying hello to your friends. :) | 01:56 |
| lifeless | jtv: the next step is a Layer for rabbit, which should be trivial with the fixture | 01:56 |
| jtv | Aigh! Not a layer! We want resources and fixtures! | 01:56 |
| lifeless | jtv: and after that we probably need some test support - things to say 'a message was sent' and so on | 01:56 |
| lifeless | jtv: we have a fixture | 01:56 |
| lifeless | jtv: but it needs to fit in the current test layout | 01:56 |
| jtv | Which we all hate :) | 01:57 |
| lifeless | sure | 01:57 |
| lifeless | orthogonal problem | 01:57 |
| jtv | True. | 01:58 |
| lifeless | anyhow | 01:58 |
| lifeless | pull devel | 01:58 |
| lifeless | its there | 01:58 |
| jtv | Great | 01:58 |
| jtv | Meanwhile, I thought I was being clever but am running into problems that you'd be able to help with. | 01:58 |
| lifeless | shoot | 01:58 |
| jtv | My test takes 6s or so. I thought I could speed it up by re-using test SPPH objects across tests, because creating those seems to be taking most of the time. | 01:59 |
| jtv | I had a really nice idea for that: a generator that caches a list of SPPHs. | 01:59 |
| jtv | You request one, it gives you the next one from its cached list. | 01:59 |
| lifeless | the db layer rolls back the db | 01:59 |
| jtv | Yes. :( | 01:59 |
| lifeless | either have one test, or accept the cost, or test narrower layers that don't need the db (oops the last one is [for now] science fiction) | 02:00 |
| jtv | I suppose I might produce fakes, but it gets a bit hackey. | 02:00 |
| jtv | (Thank you, alarm clock, but I've been at work for hours) | 02:00 |
| jtv | This particular one is pretty painful. It'd have been nice to avoid it. | 02:01 |
| jtv | I do have a plan B: | 02:01 |
| jtv | see if somewhere in the factory we do commits just for the Librarian's sake. | 02:01 |
| jtv | If so, use FakeLibrarian and replace the commit with the fake. | 02:02 |
| jtv | It's really ridiculous: at one point I'm spending 0.4s on creating 2 test objects! | 02:02 |
| lifeless | yeah | 02:02 |
| lifeless | thats about 10 tests in bzr :> | 02:02 |
| jtv | Hmm not a lot of committing going on in the factory these days. That's good, but it means the remaining fruit hangs higher. | 02:03 |
| wgrant | jtv: Have you profiled? | 02:03 |
| wgrant | That's hideous. | 02:03 |
| wgrant | Alternatively use makeSourcePackagePublishingHistory, which doesn't create any files AFAIK. | 02:04 |
| jtv | Only print-profiling so far. | 02:04 |
| wgrant | At least I was able to use it in DatabaseFunctionalLayer without a problem. | 02:04 |
| jtv | It _is_ makeSourcePackagePublishingHistory that's taking up that time. | 02:04 |
| jtv | It seems to be a bit irregular somehow. | 02:04 |
| wgrant | Some apparently trivial utility operations can be really slow. | 02:06 |
| wgrant | NFI why. | 02:06 |
| jtv | I'm going to try it in "make harness" with SQL logging on. | 02:08 |
| jtv | See if it's doing any outrageous DB work. | 02:08 |
| wgrant | Yeah. | 02:09 |
| wgrant | There's lots of SQL. | 02:09 |
| wgrant | Huh. | 02:10 |
| wgrant | makeSPPH repeatably takes 0.3s | 02:10 |
| wgrant | That's awful :/ | 02:10 |
| jtv | Quite. | 02:11 |
| jtv | Imagine what we could save if we had an easy way around that. | 02:11 |
| wgrant | I guess it creates lots of people. | 02:11 |
| wgrant | Which is slow. | 02:11 |
| wgrant | Because of Account. | 02:11 |
| jtv | I was going to ask about that: how does that currently work? | 02:12 |
| wgrant | Which? | 02:12 |
| wgrant | Account? | 02:12 |
| jtv | Account. I half expected to see commits to synchronize the account/LP dbs. | 02:12 |
| wgrant | Right, that was removed a while ago. | 02:12 |
| wgrant | Since we no longer have a HA auth store. | 02:12 |
| wgrant | Because SSO is a separate DB. | 02:12 |
| wgrant | So you no longer need to commit after creating people. | 02:13 |
| lifeless | and session is also not replicated | 02:13 |
| lifeless | if we fail over its trashed | 02:13 |
| wgrant | Looks like creating a person takes 15 queries :( | 02:13 |
| wgrant | Still should be quick, though. | 02:13 |
| * wgrant fires up a profiler. | 02:13 | |
| wgrant | :( no context manager profiler | 02:14 |
| jtv | Can't start one of the regular python profilers in "make harness"? | 02:16 |
| jtv | Looks like makeArchive() takes up a significant portion of the time of a makeSPPH(). | 02:17 |
| wgrant | Yes. But a context manager would probably be easier. | 02:17 |
| lifeless | bzrlib.lsprof.profile(...) is your friend | 02:18 |
| lifeless | and yes, it works in harness | 02:18 |
| wgrant | Can we just merge bzrlib into the standard library or something? | 02:18 |
| lifeless | *blink* | 02:18 |
| lifeless | lp.testing.storm | 02:19 |
| * lifeless sobs | 02:19 | |
| wgrant | Does not exist. | 02:19 |
| lifeless | lib/lp/testing/storm.py | 02:19 |
| lifeless | pull devel | 02:19 |
| lifeless | its quite buggy for new generic code | 02:20 |
| wgrant | I pull in trepidation. | 02:20 |
| lifeless | sinzui: when you said hack, I didn't imagine this | 02:21 |
| lifeless | sinzui: (sorry for being negative; it is a bit of a tricky problem) | 02:22 |
| wgrant | 28% in the DB, 20% clearing lazr.restful memcache... | 02:26 |
| lifeless | the representation cache? | 02:26 |
| wgrant | Yes. | 02:26 |
| lifeless | turn the f*er off | 02:26 |
| wgrant | 10% on top of the 28% in the storm tracer. | 02:26 |
| lifeless | I thought the representation cache was off in prod anyhow | 02:27 |
| wgrant | It is. | 02:27 |
| wgrant | Killing the representation cache only shaved off a bit under 10%. | 02:29 |
| wgrant | Now 50% in storm, 40% in the DB. | 02:30 |
| lifeless | OOPS-1957DY36 | 02:31 |
| lifeless | actually filing a bug should not lookup similar bugs >< | 02:32 |
| lifeless | right, bug 780835 | 02:32 |
| _mup_ | Bug #780835: representation cache a pessimism <lazr.restful:New> < https://launchpad.net/bugs/780835 > | 02:32 |
| jtv | Pessimism or pessimization? | 02:33 |
| lifeless | bah | 02:33 |
| lifeless | yes | 02:33 |
| lifeless | ESLEEPFFFFAIL | 02:33 |
| jtv | Hmm I got some savings of a few tens of seconds by eliding Person constructions. | 02:34 |
| lifeless | fixed | 02:34 |
| wgrant | jtv: createPersonAndEmail seems to take 33% of the time. | 02:35 |
| wgrant | And a third of that is EmailAddress.new... | 02:36 |
| wgrant | How. | 02:36 |
| lifeless | 02:37 | |
| lifeless | if self.getByEmail(email) is not None: | 02:37 |
| lifeless | raise EmailAddressAlreadyTaken( | 02:37 |
| lifeless | "The email address '%s' is already registered." % email) | 02:37 |
| lifeless | LBYL :( | 02:37 |
| wgrant | Yeah. | 02:37 |
| lifeless | assert status in EmailAddressStatus.items | 02:37 |
| lifeless | is inefficient (should be a type check) | 02:37 |
| wgrant | Still trivial. | 02:38 |
| lifeless | valid_email looks sane enough | 02:39 |
| lifeless | how many persons are you creating? | 02:39 |
| wgrant | 5, looks like. | 02:39 |
| lifeless | oh | 02:39 |
| lifeless | that LBYL isn't indexed. | 02:39 |
| wgrant | Aha. | 02:39 |
| lifeless | possibly | 02:39 |
| wgrant | It must be, surely. | 02:39 |
| lifeless | need to check one more thing | 02:39 |
| lifeless | "emailaddress__lower_email__key" UNIQUE, btree (lower(email)) | 02:39 |
| lifeless | ah, it is | 02:40 |
| wgrant | 6% in PersonSet.getByName :( | 02:40 |
| jtv | At <100 persons in the sample data, would it matter anyway? | 02:40 |
| lifeless | jtv: psosibly ;> | 02:40 |
| jtv | I nearly halved my test time just by eliminating makePerson calls. | 02:40 |
| lifeless | but | 02:40 |
| lifeless | its stupid to have an index on a function | 02:40 |
| wgrant | lifeless: | 02:40 |
| lifeless | when we can apply the function on data insertion | 02:40 |
| wgrant | Sort (cost=6.65..6.66 rows=1 width=39) (actual time=0.146..0.146 rows=0 loops=1) | 02:40 |
| wgrant | Sort Key: email | 02:41 |
| wgrant | Sort Method: quicksort Memory: 25kB | 02:41 |
| wgrant | But should still be fast, given how small it is :/ | 02:41 |
| wgrant | -> Seq Scan on emailaddress (cost=0.00..6.64 rows=1 width=39) (actual time=0.096..0.096 rows=0 loops=1) | 02:41 |
| wgrant | Filter: (lower(email) = 'email755122@example.com'::text) | 02:41 |
| wgrant | That's the plan. | 02:41 |
| wgrant | So WTF is it so slow. | 02:41 |
| wgrant | It's sub-ms. | 02:41 |
| jtv | lifeless: absolutely... especially because last I heard, function indexes weren't quite as effective as I expected. | 02:41 |
| lifeless | specifically, all queries on email come from our appservers | 02:41 |
| lifeless | joins are on id | 02:41 |
| lifeless | so we should be able to lower() in the appserver before querying rather than in the db | 02:42 |
| lifeless | anyhoo | 02:42 |
| lifeless | wgrant: how many ms does your profiling think it is? | 02:42 |
| lifeless | wgrant: and what profiler did you use? | 02:42 |
| wgrant | 5% of roughly 300ms. | 02:42 |
| wgrant | So 15ms. | 02:42 |
| wgrant | I used bzrlib.lsprof. | 02:43 |
| lifeless | ok | 02:43 |
| wgrant | The query is 200µs hot. | 02:43 |
| lifeless | so @ 15ms that suggtests 14ms in python | 02:43 |
| jtv | wgrant: what exactly was slow? | 02:43 |
| lifeless | wgrant: how long does the profiler think the cursor get took ? | 02:43 |
| wgrant | jtv: EmailAddressSet.getByEmail | 02:44 |
| jtv | Weirdness. | 02:44 |
| jtv | Unexpected ASCII/Unicode conversion issue? | 02:44 |
| jtv | (Since this is one of the few cases where ASCII would probably be fine) | 02:45 |
| wgrant | kcachegrind confuses me. | 02:45 |
| wgrant | Or maybe it's just lying to me. It's not showing any callees beyond selectOne. | 02:47 |
| lifeless | you've turned min-node down ? | 02:47 |
| wgrant | As low as it goes, yes. But it's still showing some nodes without children. Hrm. | 02:51 |
| lifeless | C modules add some fun | 02:51 |
| jtv | Stripped? | 02:51 |
| jtv | Or otherwise without debug info? | 02:51 |
| lifeless | they show as a monolithic block in the profiler | 02:52 |
| lifeless | because the profiler works by running code between every opcode | 02:52 |
| lifeless | (thats a lie, its every line, shrug) | 02:52 |
| lifeless | and when C modules call back into python the callgraph can be fun | 02:53 |
| spiv | 'perf' can sometimes show you where C modules are consuming time. | 02:53 |
| spiv | (Not tied into the Python callstack at all, of course) | 02:53 |
| wgrant | I think kcachegrind might just be broken. | 02:53 |
| wgrant | gprof2dot sees calls into EmailAddress.new as expected. kcachegrind sees 0. | 02:53 |
| lifeless | win | 02:54 |
| wgrant | Hmm. kcachegrind shows two createPersonAndEmail functions. | 02:55 |
| wgrant | I wonder if some C in the middle is breaking everything. | 02:55 |
| wgrant | One of them has no calls. | 02:55 |
| wgrant | And the one with no calls calls the uncalled functions! | 02:55 |
| wgrant | That's not very nice. | 02:55 |
| lifeless | \o/ | 02:55 |
| lifeless | that might be fixed by jams recent lsprof tweak in bzrlib actually | 02:55 |
| wgrant | Ahh, and the harness will be using ancient bzr... | 02:56 |
| wgrant | 2.2.2 :/ | 02:56 |
| wgrant | We should really fix that. | 02:57 |
| lifeless | yes! | 02:57 |
| lifeless | bzr team have been doing that | 02:57 |
| wgrant | Have they? | 02:57 |
| lifeless | thats my understanding | 02:57 |
| lifeless | IMBW | 02:57 |
| wgrant | I don't think it's been updated since Code was disbanded and then everyone ran away,. | 02:58 |
| wgrant | It was last changed 2011-02-09, when I applied a patch for a stacking bug. | 02:58 |
| wgrant | So it hasn't been upgraded since Code evaporated. | 02:58 |
| wgrant | => bzr probably doesn't do it. | 02:59 |
| wgrant | lifeless: Ahh, that's much better. | 03:12 |
| wgrant | The tree is no longer lots of trees. | 03:12 |
| wgrant | (after a quick port to 2.4b2) | 03:13 |
| wgrant | Perhaps I will toss it at ec2 and see what breaks. | 03:18 |
| wgrant | After I work out how to use weave_fmt properly as a plugin. | 03:18 |
| wgrant | Ahh | 03:24 |
| wgrant | 2/3 of the getByEmail time is flushing writes. | 03:24 |
| wgrant | Before it does the find. | 03:24 |
| wgrant | It would be more illuminating if we could turn off the write cache... | 03:25 |
| LPCIBot | Project windmill-devel build #64: STILL FAILING in 1 hr 8 min: https://lpci.wedontsleep.org/job/windmill-devel/64/ | 03:37 |
| LPCIBot | Project windmill-db-devel build #262: STILL FAILING in 1 hr 7 min: https://lpci.wedontsleep.org/job/windmill-db-devel/262/ | 03:42 |
| lifeless | wgrant: which write cache | 03:54 |
| wgrant | lifeless: Storm's. | 03:54 |
| lifeless | the same one that broke commits on the archive thingy the other week ? | 03:54 |
| wgrant | Hm? | 03:55 |
| wgrant | Anyway, I emulated disabling it by flushing after adding each new object. | 03:56 |
| lifeless | when I eager loaded source packages | 03:56 |
| lifeless | commits crawled to a halt because of O(cache size) behaviour in commit | 03:57 |
| wgrant | Ah. | 03:57 |
| wgrant | No, this is the dirty object cache. | 03:57 |
| lifeless | I'm trying to decide between | 04:09 |
| lifeless | fix a timeout | 04:09 |
| lifeless | write a rabbit layer | 04:09 |
| lifeless | $other | 04:09 |
| wgrant | Rabbit. | 04:09 |
| wgrant | Rabbit will fix lots of timeouts. | 04:10 |
| wgrant | lifeless: staging would like to have http://paste.ubuntu.com/605983/ run on it with an EXPLAIN ANALYZE. | 04:17 |
| lifeless | [qa]? | 04:18 |
| wgrant | Either. | 04:18 |
| lifeless | http://paste.ubuntu.com/605985/ | 04:18 |
| wgrant | lifeless: And hot? | 04:20 |
| lifeless | 340.216 | 04:20 |
| wgrant | Slower than DF :( | 04:20 |
| wgrant | 205ms hot. | 04:20 |
| lifeless | -> Hash (cost=13594.89..13594.89 rows=5168 width=8) (actual time=112.326..112.326 rows=5189 loops=1) | 04:20 |
| wgrant | But more like what I was expecting. | 04:20 |
| lifeless | -> Index Scan using securesourcepackagepublishinghistory__distroseries__idx on sourcepackagepublishinghistory (cost=0.00..19355.35 rows=16746 width=24) (actual time=0.259..214.622 rows=17420 loops=1) | 04:21 |
| StevenK | Hold on, my world is crashing down. | 04:21 |
| jtv | We'll have to revise the term "dog [food] slow" | 04:22 |
| lifeless | wgrant: you just restored; less bloat | 04:22 |
| StevenK | staging should have just done that, too? | 04:23 |
| lifeless | hahahahaahhahahahahaha | 04:24 |
| lifeless | no, restore failed | 04:24 |
| StevenK | :-( | 04:24 |
| lifeless | also this is qastaging | 04:24 |
| StevenK | Ah | 04:24 |
| lifeless | which is better to test on for things that aren't coupled to schema changes | 04:25 |
| StevenK | But still only restores on demand :-( | 04:25 |
| lifeless | (same hardware, longer lifetime - it gets its own bloat going, and its where we qa upcoming deploys) | 04:25 |
| jtv | wgrant: did you eliminate that silliness in packagecopier where a permissions error was suppressed if the user had Append privileges? | 04:26 |
| jtv | "if reason is not None: raise CannotCopy(reason)" instead of a nested "if" to give it another chance? | 04:27 |
| === jtv changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: jtv | https://code.launchpad.net/launchpad-project/+activereviews | ||
| jtv | Or maybe it was Gavin. | 04:28 |
| jtv | And by the way wgrant, would you care to review my "copy asynchronously if there's more than 100 packages to copy" branch? https://code.launchpad.net/~jtv/launchpad/bug-780319/+merge/60571 | 04:29 |
| lifeless | jtv: 100?! | 04:29 |
| lifeless | jtv: we struggle to copy 2 in the webapp | 04:29 |
| jtv | That's what's been specced. | 04:29 |
| lifeless | well, reality is going to intervene | 04:30 |
| jtv | Then feel free to lower the threshold to 1. :) | 04:30 |
| StevenK | Er, doesn't it make use of PackageCopyJob? | 04:31 |
| jtv | Yes. | 04:31 |
| StevenK | Then that's completly different. | 04:31 |
| StevenK | lifeless: ^ | 04:31 |
| lifeless | so if its using a job | 04:31 |
| lifeless | its already asynchronous | 04:32 |
| lifeless | isn't it? | 04:32 |
| lifeless | jtv: the time budget for backend tasks that hold transactions is effectively 7-8 seconds (allowing time for webapp changes after they start actually doing writes | 04:33 |
| jtv | This implements a choice between synchronous and asynchronous. | 04:33 |
| lifeless | does synchronous mean in-webapp | 04:34 |
| jtv | Yes. | 04:34 |
| lifeless | if that is set to 1, it might work sometimes. | 04:34 |
| jtv | Is someone planning to speed that up massively, perhaps? | 04:35 |
| lifeless | yes, by making it async | 04:35 |
| lifeless | 964 OOPS-1955CE140 Archive:+copy-packages | 04:35 |
| jtv | I mean the actual copying. | 04:35 |
| lifeless | 11 / 8 Archive:+copy-packages | 04:35 |
| wgrant | Copying is very fast. | 04:36 |
| wgrant | Copy checking is still slow sometimes. | 04:36 |
| lifeless | and when its slow its glacial | 04:36 |
| lifeless | 11 failures on 416 copies yesterday | 04:36 |
| lifeless | 2.5% failure rate | 04:36 |
| wgrant | Lots of those were probably delayed copies. | 04:36 |
| lifeless | 99th percentile is 8.3 seonds | 04:37 |
| lifeless | mean is 3 seconds | 04:37 |
| wgrant | Non-sql time: 4333 ms | 04:37 |
| lifeless | when you say 'fast' | 04:37 |
| wgrant | lifeless: +copy-packages or syncSource? | 04:37 |
| lifeless | Archive:+copy-packages | 04:37 |
| wgrant | That's got UI crap. | 04:37 |
| wgrant | Which is unoptimised and constant. | 04:37 |
| lifeless | Archive:EntryResource:sync has its 99th percentile 11 seconds | 04:37 |
| lifeless | mean 2.15 seconds | 04:38 |
| wgrant | Right, because it's used a lot for delayed copies, or copies from -proposed to -updates. | 04:38 |
| wgrant | Which close bugs. | 04:38 |
| wgrant | None of which is optimised. | 04:38 |
| lifeless | 99th percentil of db time is 6.5 seconds | 04:38 |
| wgrant | Actual copying is fast. Delayed copies are not. Bug closing is not. | 04:38 |
| wgrant | Checking when the source already exists in the target archive is not. | 04:38 |
| lifeless | sure, but does 'do a copy' trigger these other htings? | 04:38 |
| wgrant | Yes. They can and should be optimised. | 04:38 |
| lifeless | if it does, then a discussion about copying has to assume that copying is slow [and optimisable perhaps] | 04:39 |
| jtv | In any case, my branch isolates a policy choice for when to do which. | 04:39 |
| lifeless | which is great | 04:39 |
| lifeless | I guess I'm saying that right now, I would put the policy at 0 | 04:39 |
| jtv | I figure improving and tuning that choice is a separate step. | 04:39 |
| lifeless | jtv: in fact, there is a good request: please make sure its possible to disable synchronous copying entirely. | 04:40 |
| wgrant | As long as these both use pretty much the same code. | 04:40 |
| wgrant | And it isn't like delayed copies. | 04:40 |
| wgrant | If it is anything like delayed copies I will rs it out now. | 04:40 |
| jtv | rs? | 04:40 |
| wgrant | Delete without review, because delayed copies are just that hideous. | 04:41 |
| jtv | Where do delayed copies come into the picture? | 04:42 |
| wgrant | They are the old asynchronous copy mechanism. | 04:42 |
| jtv | I think I saw bits of that, deeper down in the call tree. | 04:43 |
| wgrant | Which is not even slightly like the synchronous copy mechanism. | 04:43 |
| wgrant | If both of these use the direct copy function, I am happy. | 04:43 |
| jtv | When do we use which? | 04:43 |
| jtv | They use do_copy. | 04:43 |
| wgrant | Delayed copies are used for copies from private to public archives. | 04:43 |
| jtv | So that's a policy decision, not some kind of time management? | 04:43 |
| wgrant | Half and half and half of some more hacks. | 04:44 |
| StevenK | Delayed copies are *hideous*, and I'm going to remove them | 04:44 |
| wgrant | It was initially because those copies needed to reupload files to the public librarian. | 04:44 |
| wgrant | Which was slow. | 04:44 |
| wgrant | Then more hacks were layered on top that were unrelated. | 04:44 |
| wgrant | And then it became clear that the initial rationale was wrong: you can just toggle the restricted flag in the DB. | 04:44 |
| wgrant | But it all remains. | 04:44 |
| jtv | So... it might make sense for the sync/async choice to check whether any of the source archives are private and the dest archive is public? | 04:45 |
| wgrant | This UI cannot go anywhere near production until delayed copies are dead and buried. | 04:45 |
| wgrant | Or we'll have doubly async copies. | 04:45 |
| wgrant | And badness. | 04:45 |
| jtv | If you're worried about doubly async copies, then I guess that would be a blocker for using PackageCopyJobs. | 04:47 |
| wgrant | Doubly async is stupid, and the behaviour is inconsistent. a PackageCopyJob should never use a delayed copy. | 04:47 |
| jtv | Well, both it and the view code call do_copy. | 04:47 |
| wgrant | And do_copy will create a delayed copy if the stars are aligned and it deems the witchcraft justfieid. | 04:48 |
| jtv | If do_copy may then defer yet again to another async kind of job, then that needs fixing but it's independent of my branch. | 04:48 |
| jtv | Anyway, I need to go out and find food and coffee. In that order: I'm that desperate. | 04:48 |
| wgrant | As long as this is all heavily flagged. | 04:49 |
| jtv | With wgrant telling me the safe choice is to go synchronous and lifeless telling me I should probably always go asynchronous. Whee! | 04:50 |
| wgrant | I didn't say to go synchronous. | 04:50 |
| wgrant | I said that synchronous can be fairly fast, and that you are to under no circumstances go *doubly* asynchronous by bringing delayed copies into the mix. | 04:50 |
| jtv | In other words, that "the safe choice is to go synchronous" | 04:51 |
| wgrant | Synchronous will do delayed copies in the same cases as asynchronous will. | 04:51 |
| jtv | And so delayed copies can't be handled synchronously as per robert and can't be handled asynchronously as per william? | 04:52 |
| wgrant | Delayed copies must not be handled at all. | 04:52 |
| wgrant | If you are creating them in this new workflow you are doing it wrong. | 04:52 |
| wgrant | But you may be doing it accidentally. | 04:52 |
| jtv | How would I know whether I'm creating them? | 04:52 |
| jtv | This still seems orthogonal to me. | 04:52 |
| wgrant | If you're not aware of them then it's not orthogonal. | 04:53 |
| wgrant | You need to be aware of the horrors that lurk within packagecopier. | 04:53 |
| wgrant | So, the reason I like synchronous copies for now is that our async UI story is completely hopeless. | 04:54 |
| wgrant | You don't get a response in a reasonable amount of time. | 04:54 |
| wgrant | Because a) the job probably won't run for more than a minute, and b) the client has to poll. | 04:54 |
| wgrant | So you end up with crap like the apport refreshing-every-10-seconds-don't-mind-me thing. | 04:54 |
| jtv | Which I want no part of. | 04:54 |
| wgrant | I think until we can fix both we should try to keep things synchronous. | 04:55 |
| spiv | wgrant: skimming this conversation, it strikes me you are warning loudly about the lurking horrors, without explaining how to actually find out about them. | 04:56 |
| wgrant | spiv: Because there's no way to solve this yet :) | 04:56 |
| spiv | (But I'm only skimming, so apologies if I've missed an important bit) | 04:56 |
| jtv | Welcome to R'lyeh. | 04:56 |
| lifeless | jtv: hi, I didn't mean to cause confusion | 04:57 |
| lifeless | jtv: have you ate? | 04:57 |
| jtv | No. | 04:57 |
| jtv | It's getting urgent. | 04:57 |
| lifeless | jtv: so go do that | 04:57 |
| lifeless | jtv: This is a years old mess | 04:57 |
| jtv | But wgrant keeps talking about the horrors that haunt his nightmares. | 04:57 |
| spiv | Um, I'm not talking about a solution. I'm just saying you're saying "must not handle delayed copies" without giving jtv any pointers on how to ensure that's the case. | 04:57 |
| lifeless | jtv: one meal won't make any difference. | 04:57 |
| wgrant | You hadn't skipped any solution :( CopyChecker takes an allow_delayed_copies argument. If it's True it might create delayed copies, which is the wrong thing. If it's False it won't create delayed copies, so they will be copied directly which then misses the extra delayed copy functionality so is also the wrong thing. | 04:57 |
| === jtv is now known as jtv-eat | ||
| wgrant | spiv: There is no solution to ensure that's the cae. | 04:57 |
| wgrant | +s | 04:57 |
| spiv | (Other than by not implementing things at all) | 04:57 |
| wgrant | Right. | 04:58 |
| wgrant | My advice would be to not try to extend the copy mechanism until the copy mechanism is not a nightmare. | 04:58 |
| wgrant | In more general terms, plan! | 04:58 |
| lifeless | mmm | 05:02 |
| lifeless | I agree that nuking tech debt makes things easier to do. | 05:03 |
| lifeless | This new functionality will be doing maaasive copies we don't do at all | 05:03 |
| lifeless | It may need to fix things up to move forward. | 05:03 |
| lifeless | now, all that said doing double-async is gross but would work. | 05:04 |
| wgrant | Sure. But it's currently *not possible* to implement this correctly. | 05:04 |
| wgrant | Unless delayed copies are nuked. | 05:04 |
| lifeless | wgrant: other than grossness, where am I wrong? | 05:04 |
| wgrant | You would have to use delayed copies for all copies. | 05:04 |
| wgrant | Which means making everything doubly-async and inconsistent. | 05:04 |
| lifeless | why? | 05:04 |
| wgrant | Because only delayed copies respect overrides and announce. | 05:05 |
| lifeless | ok, and this patch needs that ? | 05:05 |
| wgrant | That's why delayed copy elimination is part of this work. | 05:05 |
| wgrant | Because DDs cannot operate without overrides and announcements. | 05:05 |
| lifeless | hang on | 05:05 |
| lifeless | we have a narrow patch up | 05:05 |
| lifeless | to do async if > some N | 05:06 |
| lifeless | how will that *break* | 05:06 |
| wgrant | It won't break, it's just the first time I've looked at this additional mess in depth. | 05:06 |
| lifeless | ok | 05:06 |
| lifeless | so, if it won't break, we don't need to block jtv on this, as long as the overall arc will fix shit up. | 05:07 |
| wgrant | This is Launchpad. | 05:07 |
| wgrant | The overall arc will not. | 05:07 |
| lifeless | well, we can talk to julian about that | 05:07 |
| wgrant | Julian is under the impression that the feature squad is over in just a few weeks. | 05:07 |
| wgrant | I don't know how accurate that is. | 05:07 |
| wgrant | But it does not bode well. | 05:07 |
| lifeless | julian and flacoste :) | 05:07 |
| lifeless | we may need to tweak our size estimation process | 05:08 |
| lifeless | we definitely need to get faster | 05:08 |
| wgrant | We do. | 05:08 |
| lifeless | so, *personally* I would: | 05:08 |
| lifeless | - make it always async | 05:08 |
| wgrant | That would be no problem if the UI didn't suck. | 05:08 |
| wgrant | But the UI will suck. | 05:08 |
| wgrant | So we should avoid it. | 05:08 |
| lifeless | I think you need to consider more variables in assessing this | 05:09 |
| lifeless | do you agree that not all copies can be done in <1 second, and that in fact most will need > 1 second ? | 05:09 |
| wgrant | Sure. | 05:09 |
| lifeless | well, s/most/more than 1%/ | 05:09 |
| lifeless | ok | 05:10 |
| lifeless | so we need the async case to exist; we need it to have a reasonable UI. | 05:10 |
| wgrant | Launchpad has no facilities for reasonable async UI. | 05:10 |
| wgrant | We are not ready. | 05:10 |
| lifeless | we have sufficient facilities to get a tolerable UI that will be robust and amenable to optimisation. | 05:11 |
| wgrant | lol | 05:11 |
| lifeless | and we will save /significant/ complexity having one code path. | 05:11 |
| wgrant | But perhaps you consider "This page will refresh every 10 seconds." to be tolerable UI. | 05:11 |
| wgrant | It may be,. | 05:11 |
| wgrant | But it seems pretty crap to put that on a new feature. | 05:11 |
| lifeless | wgrant: vs a timeout, I think its a marvelous improvement. | 05:12 |
| wgrant | No, it's vs immediate feedback for small copies and frequent refreshes for large copies. And people probably don't care much about immediate feedback for large copies,. | 05:12 |
| lifeless | wgrant: we should refresh more often than that anyway, for a polled approach. | 05:12 |
| wgrant | lifeless: I was quoting from our existing async page refresher. | 05:12 |
| lifeless | wgrant: s/refresh/ajax-check/ | 05:12 |
| lifeless | wgrant: so thats an argument for having two UI's. | 05:13 |
| lifeless | wgrant: on a purely technical basis i think that that is suboptimal and harder for us to do (I don't mean tougher, I mean 'more total effort') | 05:14 |
| lifeless | wgrant: you have expressed a desire for us to finish things more completely; part of that is making things cheaper to do. That means less effort output to achieve stated goals. | 05:14 |
| wgrant | lifeless: Once we can quickly switch to a page probably with rows for each copy and a progress indicator, sure. | 05:14 |
| lifeless | wgrant: all the bits are in place to do that. | 05:14 |
| wgrant | But if it's a tradeoff between a little additional complexity and a completely terrible UI, we should go for that minimal extra complexity. | 05:15 |
| lifeless | thats not the tradeoff though | 05:15 |
| wgrant | We may well be able to quickly whip up an AJAX UI for this. If so, great! We can make everything async and it will be excellent. | 05:16 |
| lifeless | its between having 4 code paths and 2 (sync, delayed, job-sync, job-delayed) vs (job-sync, job-delayed) | 05:16 |
| wgrant | But I don't think there's time, given the huge amount of unscheduled trainwreck that's still to come. | 05:16 |
| wgrant | lifeless: Well, delayed copies probably don't have much life left (hi StevenK) | 05:17 |
| StevenK | lifeless: delayed copies are still a pox and need to die | 05:17 |
| lifeless | right, I agree, which is why we shold be making this as *simple* as possible to let work proceed on the bits that matter! | 05:17 |
| wgrant | Oh, it turns out that you get more meaningful results from the PPR if you're looking at the recent one in ~lpqateam, not the ancient one in ~stub. | 05:22 |
| wgrant | Oops. | 05:22 |
| wgrant | StevenK: With a new index I have the source query down to 80ms. | 05:28 |
| wgrant | 70ms... | 05:29 |
| StevenK | Whoa | 05:31 |
| StevenK | What index? | 05:31 |
| wgrant | sourcepackagepublishinghistory(archive, distroseries, pocket, status); | 05:31 |
| wgrant | That's 70ms. | 05:31 |
| wgrant | Without pocket is 80ms. | 05:31 |
| wgrant | Need to check how that works with a pocketless query, though. | 05:31 |
| wgrant | Because i need that fast for permission checks. | 05:31 |
| wgrant | (it's currently >300ms per source) | 05:32 |
| StevenK | We don't need a DB patch for the index? | 05:38 |
| wgrant | We'll do a live one. Just working out what's best. | 05:39 |
| wgrant | (archive, distroseries, status, pocket) is quick for both pocketful and pocketless queries. | 05:39 |
| wgrant | But I hope I can get even better. | 05:39 |
| lifeless | jtv: back? | 05:40 |
| lifeless | jtv: ah, just your irc client autoconnecting after network -fail- | 05:41 |
| wgrant | StevenK: Do you have a binary version of that query>? | 05:42 |
| wgrant | StevenK: Preferably with some common binaries. | 05:42 |
| StevenK | I do, but only for 'dpkg' | 05:46 |
| StevenK | Pasted in PM | 05:46 |
| wgrant | That is a bit slow cold :( | 05:50 |
| StevenK | Tis, yes :-( | 05:52 |
| StevenK | It also isn't as simple as the source query | 05:52 |
| wgrant | Hot it's 100ms for a few common binaries. | 05:55 |
| wgrant | In a single query. | 05:55 |
| wgrant | The plan is the opposite of the source one, though. | 05:56 |
| StevenK | Strange | 05:57 |
| lifeless | many more binaries than sources | 05:57 |
| StevenK | Sure, but sometimes the query planner gets strange ideas | 05:58 |
| wgrant | It always finds the BPRs first. | 05:58 |
| wgrant | Whereas it tries the SPPHs first. | 05:58 |
| StevenK | wgrant: What does the query look like for multiple binaries? | 05:59 |
| wgrant | StevenK: Well, I just did a binarypackagename IN (blah), so I'm not doing multiple DASes yet. | 06:00 |
| StevenK | Ah | 06:00 |
| StevenK | So, you're cheating | 06:00 |
| wgrant | That doesn't change the plan, though. | 06:00 |
| wgrant | Because it finds all BPRs with the right name, then indexes into BPPH on those IDs, then filters for DAS. | 06:01 |
| wgrant | Only thing it will change is the volume of input to the sort. | 06:01 |
| StevenK | wgrant: As you say, it's interesting it picks BPR first, whereas the source query starts with SPPH. | 06:02 |
| wgrant | Well, the source query actually grabs both and does a hash join. | 06:03 |
| wgrant | It's hard to say which is more efficient, because of the differing row counts. | 06:04 |
| wgrant | StevenK: Heh, I just did the ultimate source test. | 06:21 |
| wgrant | Finding overrides in maverick for every sourcepackagename in Ubuntu. | 06:21 |
| wgrant | 480ms. | 06:21 |
| wgrant | Not too bad. | 06:21 |
| wgrant | Better than the publisher's attempt at that, at least. | 06:21 |
| StevenK | Haha | 06:22 |
| StevenK | \o/ | 06:22 |
| StevenK | wgrant: And how evil was that query? | 06:22 |
| wgrant | Oh, I just selected all the SPNs into a temp table and used that. | 06:22 |
| StevenK | If we could get the publisher to use this, it would be awesome | 06:22 |
| StevenK | I'm not sure it fits | 06:23 |
| wgrant | It fits. | 06:23 |
| wgrant | Mmm, but it's already only a couple of seconds. | 06:24 |
| wgrant | Except for binaries. | 06:24 |
| wgrant | Which are like 20s. | 06:24 |
| wgrant | Need to work out why the planner does that. | 06:24 |
| StevenK | Doesn't it drop a bit of code duplication? | 06:25 |
| StevenK | (And so is a win from that perspective) | 06:25 |
| wgrant | Yes. | 06:25 |
| wgrant | But once we have the generic overrides stuff a lot gets simpler. | 06:25 |
| wgrant | Not just that :) | 06:25 |
| wgrant | Including two things I was going to work on this week, but have deferred until this work is complete. | 06:26 |
| StevenK | \o/ | 06:26 |
| StevenK | I think I need to format the query better and write docstrings and I'm down with Existing | 06:27 |
| StevenK | How to split the DISTINCT over mutiple lines, for instance | 06:27 |
| jtv | lifeless: back now... thanks for stepping in earlier btw | 06:37 |
| wgrant | lifeless: What do you think of http://bazaar.launchpad.net/~wgrant/storm/distinct-on/revision/391? | 06:40 |
| lifeless | hash joins are generally poor :( | 06:42 |
| lifeless | at least in our scale | 06:42 |
| wgrant | It's fast enough here. | 06:42 |
| wgrant | But yes, I would normally expect them to be bad. | 06:42 |
| wgrant | Not sure why it picks it, but it works. | 06:42 |
| lifeless | wgrant: thats lovely and minimal | 06:43 |
| wgrant | But... | 06:43 |
| lifeless | I don't know what raw=True implies | 06:43 |
| lifeless | no buts | 06:44 |
| wgrant | Oh. | 06:44 |
| wgrant | It lets you pass in strings directly. Not sure if it's desirable, but it's what's used for group_by and similar things. | 06:44 |
| wgrant | That was what I was going to check after you didn't reject this idea. | 06:44 |
| lifeless | thank you for having a low activation energy threshold | 06:45 |
| wgrant | Given it's so simple it seems better than perpetuating the string hack everywhere. | 06:45 |
| lifeless | god yes | 06:46 |
| lifeless | if I was a theist. | 06:46 |
| wgrant | Haha | 06:46 |
| StevenK | Ooh, when can I use it, then? | 06:46 |
| wgrant | With a bit of luck once I merge it into the LP branch in a few minutes. | 06:47 |
| wgrant | Then I'll write an LP branch to use it everywhere, and you can possibly merge that before it lands. | 06:47 |
| StevenK | \o/ | 06:47 |
| wgrant | I guess I should file a storm bug. | 06:47 |
| wgrant | Oh look, there's a storm bug already. | 06:48 |
| lifeless | there is one | 06:48 |
| wgrant | Bug #374777 | 06:48 |
| _mup_ | Bug #374777: DISTINCT ON queries <Storm:New> < https://launchpad.net/bugs/374777 > | 06:48 |
| wgrant | That's almost half a Launchpad ago. | 06:48 |
| wgrant | More than half a Launchpad ago. | 06:48 |
| wgrant | lifeless: If raw=True, .config(distinct=['something']) will work like .config(distinct=[SQL('something')]) | 06:52 |
| wgrant | Like order_by/group_by do: a string is automatically treated as an SQLRaw. | 06:53 |
| jtv | lifeless, wgrant: was there a conclusion to the question of synchronous/asynchronous immediate/delayed package copies? | 06:53 |
| wgrant | jtv: Do it all async and get at least a minimally unawful UI on it later. | 06:53 |
| wgrant | I think that was the result. | 06:53 |
| jtv | What happened to "on no account should you..."? | 06:54 |
| lifeless | jtv: there were a few specific things | 06:54 |
| wgrant | As long as you are aware of delayed copies and do not run this code in production before they are removed, it's OK. | 06:54 |
| lifeless | - what you have doesn't imply blowing up awfully; its fugly if you do delayed copied in an alyread async job, but fugly != broken. | 06:55 |
| lifeless | - you /will/ need async for probably a majority of your copies, even with the logic fixed, copying as a whole has a lot of work to do. | 06:55 |
| wgrant | lifeless: It will give inconsistent results. And inconsistency is brokenness when dealing with a primary archive. | 06:55 |
| lifeless | wgrant: what form of inconsistency | 06:56 |
| lifeless | wgrant: vs triggering the same delayed copy in the webapp | 06:56 |
| wgrant | lifeless: Delayed copies and direct copies perform difference overrides, copy different set of binaries, announce differently.' | 06:56 |
| wgrant | s/difference/different/ | 06:56 |
| lifeless | wgrant: I want to separate the issues in *this patch* vs those in the existing code | 06:56 |
| wgrant | OK. | 06:57 |
| wgrant | Sure. | 06:57 |
| lifeless | wgrant: the issues in the existing code are a topic for julian and work scheduling; the issues in or introduced by this patch are a topic for jtv & code review | 07:00 |
| jtv | Exactly. | 07:00 |
| lifeless | wgrant: so, are there any actual correctness, (vs fugliness) issues in this patch ? | 07:00 |
| lifeless | wgrant: my understanding is that there are not | 07:00 |
| jtv | lifeless: thank you | 07:00 |
| wgrant | Apart from building further on a shaky foundation without stabilisation scheduled, no, it's fine. | 07:01 |
| lifeless | wgrant: (pending a detailed code review for thinkos etc) | 07:01 |
| lifeless | ok | 07:01 |
| lifeless | so back to the things I think we concluded | 07:01 |
| lifeless | - because async will be needed for most copy operations, the async UI will need a bunch of work | 07:01 |
| lifeless | Now, *I* have a recommendation that you (jtv) just do async and put time into making the experience nice | 07:02 |
| lifeless | because that benefits all cases. | 07:02 |
| lifeless | Its only a recommendation | 07:02 |
| jtv | By the way, what exactly does "the async UI" mean? AIUI there don't need to be separate UIs for sync and async _requests_, but it'd be nice to have some _visibility_ into pending async requests. | 07:02 |
| lifeless | but I think it would save a bunch of duplicate code. | 07:02 |
| lifeless | jtv: minimally a spinner while the copy happens. | 07:03 |
| lifeless | Perhaps a progress bar showing x/Y completed. | 07:03 |
| lifeless | the details are a topic for you/julian/jml/sinzui/huwshimi :) | 07:03 |
| jtv | Some interesting design questions in those details; generally you'll probably have just one or two jobs for the whole thing. | 07:04 |
| jtv | Also makes it hard to see which requests are underway. | 07:04 |
| wgrant | Hmm, so there's a single job for all 10000 copies? | 07:04 |
| lifeless | Finally, the underlying mechanics of copies badly badly need rectification to be more robust and sane, given the level of current pitfalls | 07:04 |
| jtv | Could be, yes. That's the way those jobs work. | 07:04 |
| wgrant | OK. | 07:04 |
| jtv | I fully buy into the notion that soyuz needs a firm broom. | 07:05 |
| lifeless | now, where to go from here. I think you should discuss with julian - and I'm delighted to participate - about whether its worth having a sync+async api. | 07:05 |
| lifeless | s/api/ui/ | 07:05 |
| jtv | In fact just a very simple change with basically no UI implications almost hit the 800-line point because of the simple cleanups needed. | 07:05 |
| lifeless | but I'm totally serious when I say that the threshold to avoid timeouts is about 0. | 07:05 |
| jtv | That brings me to something that came up earlier. | 07:06 |
| jtv | Somebody mentioned that _checking_ the copies took up most of the time..? | 07:06 |
| jtv | Because the permissions check at least (part of the checking work) happens synchronously either way. | 07:06 |
| wgrant | Apart from one case (closing bugs when copying to -updates or -security), the actual copy is very fast. | 07:06 |
| lifeless | the soyuz datamodel is not enforced (nor can it be -today-) by DRI | 07:06 |
| lifeless | the mechanics of adding the new rows is fast | 07:07 |
| wgrant | jtv: Permission checking we can do in a few hundred milliseconds for the whole archive. | 07:07 |
| wgrant | jtv: Does it do the full check? | 07:07 |
| jtv | Just the permissions check. | 07:07 |
| lifeless | jtv: also async ui - error reporting is a component too | 07:07 |
| wgrant | jtv: (at the moment it will be ~400ms per source, but with StevenK's work and some refactoring I have planned for after that, it becomes 600ms for the whole archive) | 07:07 |
| jtv | wgrant: that's for the full copy including checks? | 07:08 |
| wgrant | jtv: Just permission checking, sorry. | 07:08 |
| wgrant | At the moment doing permission checks for more than a few packages at a time is infeasible. | 07:08 |
| jtv | Ah, so that's relatively costly and I can't even avoid it in the async case. | 07:08 |
| lifeless | per package you're doing ? | 07:08 |
| jtv | Yes. | 07:09 |
| lifeless | sorry, that was to wgrant | 07:09 |
| lifeless | is that 600/400ms 'for all the packages in a copy' or 'per' | 07:09 |
| wgrant | jtv: I thought we were just checking for any permission on the relevant archive, and leaving the details to syncpackagejob. | 07:09 |
| jtv | lifeless: AIUI 400ms is per package. | 07:09 |
| wgrant | lifeless: 400ms is per source. 600ms is for every source in one query. | 07:09 |
| wgrant | For a reasonable selection of sources is more like 100ms. | 07:10 |
| lifeless | wgrant: so 600ms is 'copy all sources from archive a to b and make sure you have perms to do that' ? | 07:10 |
| wgrant | lifeless: Just permission checking. | 07:10 |
| jtv | wgrant: from what I can see, the PackageCopyJob (or CopyPackageJob, I forget) does do_copy without a permissions check. Which makes sense semantically: why let people create jobs to sync packages they're not allowed to? | 07:10 |
| lifeless | wgrant: yes, I phrased it badly | 07:10 |
| wgrant | jtv: Well, do_copy only gained permission checking yesterday. | 07:10 |
| wgrant | jtv: So it's unsurprising that CopyPackageCopyJobCopyPackage doesn't use it. But it could. | 07:10 |
| jtv | wgrant: and when you say the 600ms is for "the" archive, you mean the destination archive, right? I'm asking since AIUI there can be multiple source archives in the same request and they may also be involved. | 07:11 |
| wgrant | jtv: But I saw an "any permission on the archive at all" check in the view | 07:11 |
| jtv | wgrant: did you get my question about that weird check earlier? | 07:11 |
| wgrant | Which weird check? | 07:11 |
| lifeless | grah, my graphs, my pretty pretty graphs are bust in daily chrom | 07:11 |
| lifeless | but I need daily chromium for speedtool thingy | 07:12 |
| wgrant | jtv: The expensive bit of permission checking is determining the destination component of each package. | 07:12 |
| lifeless | *fuckers* | 07:12 |
| wgrant | lifeless: :( | 07:12 |
| jtv | wgrant: the check for "if there is a reason to reject this, check again for Append permission and if the user has it, don't raise an error" | 07:12 |
| wgrant | jtv: You mean the thing I reverted last night? | 07:12 |
| jtv | wgrant: I guess--but that is exactly what I've been asking for the past hours. :) | 07:12 |
| jtv | I guess my damn connection trouble has been stopping that from getting through. :( :( :( | 07:13 |
| wgrant | I don't see how it's related. | 07:13 |
| wgrant | The launchpad.Append special case is now restricted to the Archive.syncSource(s) API methods. | 07:13 |
| wgrant | Nowhere near do_copy any more. | 07:13 |
| jtv | I was going to explain how it was related, but I needed to start somewhere. | 07:13 |
| jtv | (I wonder if my power problems are affecting my connections--UPS is beeping many times a minute) | 07:14 |
| wgrant | Hmm, that's not ideal. | 07:14 |
| jtv | Anyway: | 07:14 |
| jtv | I extracted the permissions check, because I needed to but also because it made the code a bit easier to work with. | 07:14 |
| wgrant | You extracted it from CopyChecker? | 07:15 |
| jtv | Yes. | 07:15 |
| wgrant | To where? | 07:15 |
| jtv | To a function right above it. | 07:15 |
| wgrant | ie. not a method? | 07:16 |
| jtv | No, but it can be if you want it to. | 07:16 |
| jtv | Ah no it can't | 07:16 |
| wgrant | No, no, just getting a better picture. | 07:16 |
| wgrant | Don't worry, my background is not Java :) | 07:16 |
| jtv | The problem was that the async case shouldn't be doing the full checks (right?) because it sort of defeats the purpose. | 07:16 |
| wgrant | Right. | 07:16 |
| jtv | Phew. :) | 07:16 |
| jtv | But I needed the permissions check because of how the jobs worked (and how the jobs worked seemed reasonable to me, apart from potential performance concerns) | 07:17 |
| jtv | So I needed it out of the CopyChecker, into a reusable function. | 07:17 |
| jtv | (Which by the way would also be unit-testable, hint hint :) | 07:17 |
| wgrant | CopyChecker isn't tooooo badly tested. | 07:18 |
| jtv | It's not that | 07:18 |
| wgrant | I think we may want to keep it on CopyChecker, in a partial check mode, but I guess either works. | 07:18 |
| wgrant | Depends on if we might want other pre-creation checks. | 07:18 |
| jtv | It's that it's better to have a "check permissions" method with its own unit tests, than to have unit tests on the surrounding method that exercises the crooks and nannies of the permissions-checking part. | 07:19 |
| jtv | Nooks and crannies, I mean. | 07:19 |
| StevenK | Criminals and grandmothers? | 07:19 |
| StevenK | :-) | 07:19 |
| jtv | Thank you. | 07:19 |
| lifeless | bankers and politicians? | 07:19 |
| wgrant | jtv: Sure. | 07:19 |
| jtv | Go wash your mouth. | 07:19 |
| wgrant | jtv: So that takes a list of sources? | 07:19 |
| jtv | wgrant: I believe it's currently one per SPN, but not sure. | 07:20 |
| * jtv checks | 07:20 | |
| jtv | Yes, currently one call per. | 07:21 |
| wgrant | It makes no difference at the moment, but we'll need it to take a set of SPNs/SPPHs if we want it to not take forever eventually. | 07:21 |
| jtv | But that's easily fixed. | 07:21 |
| wgrant | OK. | 07:21 |
| wgrant | What are the arguments it takes? | 07:21 |
| jtv | Yes, sounds like a solid plan. It's a small loop anyway, shouldn't be hard. | 07:21 |
| lifeless | jtv: oh btw | 07:21 |
| lifeless | have you seen load_related? | 07:21 |
| lifeless | jtv: it would make one of your helper functions smaller | 07:22 |
| jtv | wgrant: person, destination archive/pocket/..., and spn. But again, it's not set in stone. | 07:22 |
| jtv | lifeless: yes, I've seen it thanks. | 07:22 |
| jtv | I'm looking forward to applying it. | 07:22 |
| wgrant | jtv: The emerging pattern is that all underlying functions/methods like this will take something like (archive, distroseries, pocket, (spns)) | 07:23 |
| wgrant | jtv: So it deals with SPNs directly, not SPRs or SPPHs? | 07:23 |
| jtv | wgrant: then that's a very nice fit indeed. | 07:23 |
| lifeless | http://www.zdnet.com.au/telstra-scores-patent-win-over-amazon-339314845.htm | 07:23 |
| jtv | wgrant: right, it seemed cleaner to do it that way. | 07:23 |
| jtv | Eliminated some unneeded stuff. | 07:23 |
| jtv | (spph, spr) | 07:24 |
| wgrant | jtv: Great, because that's also the emerging pattern, although it goes against everything we've done for years. | 07:24 |
| jtv | It does? | 07:24 |
| wgrant | Sets of names works well for most stuff. | 07:24 |
| wgrant | But traditional we've passed around single SPRs or SPPHs instead. | 07:24 |
| wgrant | +ly | 07:24 |
| wgrant | jtv: The idea is that the copier and uploader will both share most of these functions, which will be set-of-name based and very fast. | 07:25 |
| wgrant | So we can delete tonnes of code and it will all be much faster too. | 07:25 |
| jtv | Great, great. | 07:25 |
| jtv | That shouldn't differ between the sync & async cases anyway. | 07:25 |
| wgrant | Yeah. | 07:25 |
| wgrant | jtv: For now I guess your function will just loop over the SPNs. | 07:26 |
| jtv | (I don't usually get upset about this distinction: easier to see where it's a problem and fix it without social upheaval :) | 07:26 |
| wgrant | Until we have a verifyUpload that takes a set of names. | 07:26 |
| wgrant | Right? | 07:26 |
| jtv | Sure, I could make it take a list-of-SPN right away. | 07:26 |
| jtv | (set-of-names would be particularly nice for easy testing... I had to write some fakes to get acceptable speed on my tests) | 07:27 |
| wgrant | Yeah. | 07:27 |
| wgrant | It also works well with lifeless grand plan. | 07:27 |
| wgrant | ' | 07:27 |
| jtv | Given all this, will there still ever be a case where we want to work synchronously in the web request? | 07:28 |
| wgrant | If we can get a basic UI scheduled soonish, I don't see why it would ever be required. | 07:29 |
| wgrant | That will fix +copy-packages, too. | 07:29 |
| jtv | Then maybe the sanest thing for me to do right now would be to get to work on that UI, and later we can rip the synchronous code out of my current branch. | 07:29 |
| jtv | (And by "get to work" I mean "start worrying about what the UI should actually do") | 07:30 |
| wgrant | I think get your branch fairly sensible now, and discuss the UI and delayed copy murder implementation and scheduling with Julian tonight? | 07:30 |
| wgrant | Not much consideration has been given to any of this, but it's not *that* much work. | 07:30 |
| StevenK | wgrant: The delayed copy murder implementation is already known. | 07:31 |
| wgrant | StevenK: I know you know it. | 07:31 |
| wgrant | I don't know that Julian knows all about it. | 07:31 |
| wgrant | Does he? | 07:31 |
| StevenK | Indeed he does | 07:31 |
| wgrant | Excellent. | 07:31 |
| jtv | And for me, who is struggling to understand the directly relevant parts, it's an undesirable side track. | 07:31 |
| StevenK | We spoke about it at length and he agrees | 07:31 |
| StevenK | wgrant: And you know the plan, so I don't need to re-iterate. | 07:31 |
| wgrant | Yup. | 07:32 |
| jtv | So... what's needed to get my branch fairly sensible now? | 07:32 |
| wgrant | It taking a bit of an unfortunate side-track with the overrides generalisation, but that should be better in the long run. | 07:32 |
| wgrant | jtv: Well, given that we can't use this in production yet anyway, just make it always async if that's quick? | 07:32 |
| jtv | Quickest & most flexible way to do that is to tweak the policy choice. | 07:33 |
| NCommander | wgrant: on writing soyuz tests, should I have it use breezy-autotest as the series? (if so, how do I add a new architecture?) | 07:33 |
| StevenK | NCommander: Do not use sampledata if you can at all help it | 07:33 |
| wgrant | NCommander: You would ideally create a new series using LaunchpadObjectFactory... but if you're using SoyuzTestPublisher then breezy-autotest is a good option. | 07:33 |
| StevenK | STP can be taught to respect a new series | 07:33 |
| StevenK | It just involves 20 lines of vomit | 07:34 |
| NCommander | wgrant: I was planning on recycling STP instead of writing an entire new test_.py file | 07:34 |
| wgrant | NCommander: Ideally try to use LaunchpadObjectFactory for everything, though. Can you outline the specifics of the test? | 07:34 |
| StevenK | NCommander: Don't recycle STP, either | 07:34 |
| NCommander | wgrant: I need a series with two architectures. Upload source package that has an arch all and an arch any bit which has a ${binary:Version} relationship on the Depends. Upload binaries for both architectures. Upload a 1.0-2, only upload arch all, and i386 bits. Test should attempt to see if the arch all packages from 1.0-1 are still published | 07:35 |
| StevenK | NCommander: distroseries = self.factory.makeDistroSeries() | 07:35 |
| StevenK | das1 = self.factory.makeDistroArchSeries(distroseries=distroseries) | 07:36 |
| StevenK | das2 = self.factory.makeDistroArchSeries(distroseries=distroseries) | 07:36 |
| StevenK | distroseries.nominatedarchindep = das1 | 07:36 |
| wgrant | Indeed, I suspect the factory will be better for this. | 07:36 |
| jtv | wgrant: of course one misgiving about async is we have no reporting mechanism (definitely not in the UI; don't know what support there is in the data model) | 07:36 |
| wgrant | NCommander: You can then use makeBinaryPackageBuild() to create a build, and makeBinaryPackageRelease(build=yourbuild) to create the binaries. Then publish them with makeBinaryPackagePublishingHistory. | 07:37 |
| NCommander | wgrant: the existing test_publishing.py uses SoyuzTestFactory, do I need to write a new test py? | 07:37 |
| wgrant | jtv: Right, that's my sole misgiving, and the reason I like synchronous for now. | 07:37 |
| StevenK | NCommander: Have a look at lib/lp/soyuz/tests/test_initialisedistroseriesjob.py, sub _create_child | 07:38 |
| wgrant | NCommander: It probably doesn't belong in test_publishing. But the location doesn't really matter for now. | 07:38 |
| StevenK | NCommander: That creates a new series and sets STP up to use it | 07:38 |
| wgrant | NCommander: Just write a new test class with a single test case. | 07:38 |
| wgrant | Then we can move that to wherever. | 07:38 |
| NCommander | StevenK: holy crap, my eyes, they bleed! | 07:38 |
| jtv | wgrant: so I was thinking the numeric threshold ain't so bad: we can tweak it to be ridiculously high (always synchronous) or ridiculously low (always asynchronous) and maintain our freedom of choice while we may possibly still want it. | 07:39 |
| NCommander | (thanks) | 07:39 |
| StevenK | NCommander: Frankly, that test is one of the better ones | 07:39 |
| wgrant | jtv: I guess. But once we have at least a basic async status reporting UI (hopefully very soon!) we can destroy sync. | 07:39 |
| NCommander | StevenK: no, just Soyuz's API in general | 07:39 |
| StevenK | NCommander: Most of that test is using LaunchpadObjectFactory | 07:40 |
| StevenK | So, uh, no. | 07:40 |
| * NCommander eats shoe | 07:40 | |
| NCommander | k | 07:40 |
| StevenK | s/test/function/ | 07:40 |
| jtv | wgrant: absolutely. Given though that it's the -- what comes below known-good? | 07:40 |
| jtv | wgrant: Given though that it's the devil we know, and that we have code that accommodates it, keeping that as an option seems an easy choice. | 07:41 |
| jtv | For now. | 07:41 |
| StevenK | NCommander: Hold on, I found a better example. | 07:42 |
| jtv | wgrant: I guess the ideal migration path would be: go async only for cases that currently don't work at all, then implement async UI, then drop sync. | 07:42 |
| StevenK | NCommander: It sets up two DASes and then the STP. lib/lp/soyuz/tests/test_build_set.py , function setUp() | 07:42 |
| wgrant | jtv: That's the path that I originally intended. But lifeless says to drop sync then implement async UI. | 07:43 |
| wgrant | Which is the best solution, if the UI is coming before the feature finishes. | 07:43 |
| jtv | wgrant: Also sensible, but at the moment, "drop sync" is work and would merely delay the UI work. So I'd say: let's get this branch reviewed, and start defining the UI. | 07:44 |
| wgrant | +1 | 07:44 |
| NCommander | StevenK: looks straightforward enough | 07:44 |
| StevenK | NCommander: Yes, it's an awesome amount of boilerplate | 07:44 |
| jtv | Thanks wgrant! Maybe lifeless can help me figure out the UI reqs. | 07:45 |
| jtv | Uh-oh. | 07:47 |
| jtv | wgrant: I don't suppose a do_copy that fails for whatever reason _beyond_ CannotCopy leaves any kind of paper trail? | 07:48 |
| wgrant | jtv: In the DB? | 07:48 |
| wgrant | No. | 07:48 |
| jtv | That is sad news, because neither does the job type. | 07:48 |
| wgrant | Hmm, that sounds suboptimal. | 07:49 |
| wgrant | We may need to fix that. | 07:49 |
| lifeless | 'may' :> | 07:49 |
| jtv | So the only way I see to report failure or indeed progress is to look for jobs that would apply to a given package (there may be several, and finding them is nontrivial) and inspect their status. | 07:49 |
| wgrant | jtv: So, I'd imagined that there would be a job for each package, so they'd be somewhat queryable. | 07:50 |
| wgrant | For the UI case, it can remember the job ID, perhaps? | 07:50 |
| jtv | Eh eh eh such dreams | 07:50 |
| jtv | I suppose Ajax-y UI could remember job IDs, yes. | 07:50 |
| jtv | (AIUI there may be multiple jobs, for copies from different source archives) | 07:50 |
| wgrant | Ah, those are separate jobs? | 07:51 |
| jtv | Yes. | 07:52 |
| wgrant | I thought the UI would just get a job ID back for each source, and then request the status for all those IDs regularly. | 07:52 |
| wgrant | Hard to do progress or failure counts if it's all one or two jobs :( | 07:52 |
| wgrant | Hard to do failure info, too. | 07:52 |
| jtv | Progress is in some ways the easier one. | 07:53 |
| jtv | The jobs should benefit, speed-wise, from doing multiple packages in one go I guess. Progress we can measure by how many jobs there are ahead of us in the queue. | 07:53 |
| wgrant | They will benefit significantly, yes. | 07:56 |
| wgrant | But it seems like that's more of something for the job runner to do. | 07:57 |
| wgrant | Grab jobs that can be done together, and do them together. | 07:57 |
| NCommander | setUp(self) is called by the test harnass automatically, correct? | 07:58 |
| jtv | wgrant: tempting discussion, but rabbit hole. :) | 07:59 |
| StevenK | NCommander: Yes | 07:59 |
| jtv | wgrant: Fairness, starvation, pathological cases, db cost vs. actual job work... what we have now ain't so bad I guess. :) | 08:00 |
| jml | lifeless: hi | 08:01 |
| jml | lifeless: want to /join #ubuntu -uds-mikszath? | 08:01 |
| wgrant | jtv: Yeah, but UI will be interesting, and just returning failure info at all :/ | 08:02 |
| lifeless | jml: coming | 08:04 |
| jml | lifeless: thanks. | 08:04 |
| jelmer | wallyworld: hi :) | 08:05 |
| jelmer | wallyworld: I think you still have my badge :) | 08:06 |
| wallyworld | jelmer: hello | 08:06 |
| wallyworld | jelmer: oh yeah. where are you now? | 08:06 |
| wgrant | Down to exactly three pages of criticals! | 08:08 |
| jtv | StevenK: maybe you would like to review, or at least comment on, my sync/async branch? Yes, it needs more. But I believe it's a step forward. | 08:08 |
| jtv | wgrant: ctrl-+ | 08:08 |
| StevenK | jtv: Do I have to? I'd like to finish this policy before EOD | 08:08 |
| jtv | StevenK: no, just asking if you'd like to. | 08:08 |
| StevenK | jtv: Then I'll respectly decline | 08:09 |
| jtv | No worries. | 08:09 |
| jelmer_ | wallyworld, I'm in the bug triaging session, mikszath | 08:09 |
| jelmer_ | wallyworld, there's no hurry, I can find you afterwards. I was just worried you would fly back to Brisbane with it. :) | 08:09 |
| wallyworld | jelmer: np. i'll find you | 08:09 |
| === jelmer_ is now known as jelmer | ||
| wgrant | lifeless: http://webnumbr.com/launchpad-critical-bugs and http://webnumbr.com/launchpad-oops-bugs need restarting. Looks like they failed like a month ago. | 08:14 |
| wgrant | lifeless: https://lp-oops.canonical.com/oops.py/?oopsid=OOPS-1956CH112 | 08:22 |
| wgrant | Copy checked and source copied in 300ms, then it goes and spends the next 9 seconds dealing with strucsubs :( | 08:23 |
| StevenK | Oh, bleh | 08:23 |
| wgrant | StevenK: ? | 08:26 |
| StevenK | wgrant: Your last comment | 08:27 |
| wgrant | Oh, right. | 08:27 |
| wgrant | Only eight bugs. Hm. | 08:30 |
| wgrant | Six, in fact. One is referenced three times... | 08:30 |
| adeuring | good morning | 08:37 |
| jtv | hi adeuring | 08:40 |
| adeuring | hi jtv! | 08:40 |
| wgrant | lifeless: BugMessageSet.createMessage currently calculates the index by counting the number of messages... any reason we can't take the max index and add one? | 08:43 |
| lifeless | +1 | 08:43 |
| wgrant | It's much faster. | 08:43 |
| wgrant | The existing one can take tens of ms :/ | 08:43 |
| lifeless | sorry, in 'redo lp bugs entirely for ubuntu' | 08:43 |
| wgrant | Oh *awesome*. | 08:44 |
| wgrant | Interestingly it's actually slightly faster still to ORDER BY index DESC and take the index of the first row. | 08:48 |
| wgrant | Chromium, why are you using 4GiB of RAM? | 08:55 |
| StevenK | Hah | 08:56 |
| lifeless | wgrant: also bug 424671 | 08:58 |
| _mup_ | Bug #424671: attachments: accessing attachment's message is very slow <api> <lp-foundations> <performance> <Launchpad itself:Triaged by leonardr> < https://launchpad.net/bugs/424671 > | 08:58 |
| lifeless | wgrant: the numbrs are broken | 09:00 |
| lifeless | spiv: do you have working webnumbr voodoo for lp bug searches? | 09:00 |
| spiv | lifeless: http://webnumbr.com/profile?name=https%3A%2F%2Flaunchpad.net%2F~spiv is what I've got | 09:00 |
| lifeless | wgrant: its xpat - substring-after(//div[@id='maincontent']/div/div[2]/div/div/div/div[1]/table//tr/td[1], 'of') | 09:00 |
| lifeless | substring-after(//div[@id='maincontent']/div/div[3]/div/div[1]/div/div[1]/table//tr/td[1], 'of') | 09:01 |
| lifeless | so, 2->3 | 09:01 |
| lifeless | no, deeper | 09:01 |
| lifeless | copying | 09:01 |
| bigjools | morning | 09:04 |
| jtv | moaning bigjools! | 09:04 |
| lifeless | wgrant: http://webnumbr.com/.join(launchpad-oops-bugs.all,launchpad-timeout-bugs.all,launchpad-critical-bugs.all) | 09:05 |
| lifeless | ok, I fail at search | 09:06 |
| lifeless | I'm trying to find the (long) analysis I wrote about our bug task UI/model | 09:06 |
| lifeless | cake credits to anyone that finds it | 09:06 |
| mrevell | Hello! | 09:08 |
| bigjools | wassup jtv | 09:09 |
| jtv | bigjools: where do I start :) | 09:09 |
| jtv | Hi mrevell | 09:09 |
| jtv | bigjools: currently trying to figure out what UI is needed (and reasonably implementable) for async package copying. | 09:10 |
| LPCIBot | Project devel build #709: FAILURE in 5 hr 33 min: https://lpci.wedontsleep.org/job/devel/709/ | 09:10 |
| StevenK | Hmph | 09:11 |
| bigjools | jtv: ah yes we need to talk about that | 09:11 |
| lifeless | wgrant: it used the count because the indices hadn't been populated | 09:11 |
| StevenK | lifeless: ^ Rabbit fail | 09:11 |
| bigjools | jtv: I previously chatted to allenap about the relevant page polling for copy jobs | 09:11 |
| jtv | Ah, that ties right in | 09:11 |
| bigjools | jtv: then the notifications can work mostly as they do now, just asynchronously | 09:12 |
| bigjools | but we need the addition of a "job in progress" one | 09:12 |
| lifeless | StevenK: win ip-172-56-124-252: nxdomain | 09:13 |
| jtv | bigjools: my mental image of this atm involves JS on the page knowing what job ids it has in progress, and seeing where the last of those is in the queue. | 09:13 |
| bigjools | jtv: well it can poll for all copy jobs for that series, there may be more than one | 09:14 |
| bigjools | but yes | 09:14 |
| jtv | Yes, but probably not many. | 09:14 |
| jtv | One per source archive, basically. | 09:14 |
| StevenK | lifeless: If rabbit is attempting to look up it's local hostname, then that's just stupid | 09:14 |
| bigjools | jtv: well it depends how many archive admins are using it at once | 09:14 |
| lifeless | StevenK: rabbit attempts to look up its local hostname | 09:14 |
| jtv | StevenK: I believe that's what broke shutdown on maverick. | 09:14 |
| lifeless | StevenK: thats what made maverick fail to shutdown, because not-manager went away too fast | 09:14 |
| jtv | lifeless, StevenK: fwiw my filesystem hasn't corrupted itself once since I upgraded to natty! | 09:15 |
| lifeless | jtv: congrats | 09:15 |
| lifeless | ! | 09:15 |
| === almaisan-away is now known as al-maisan | ||
| jtv | bigjools: there's another thing... it's hard to find other jobs that may be pending for the same copies. I'm hoping repeated copies will be no-ops. | 09:15 |
| bigjools | jtv: yes, the copy checker will fail the second one | 09:16 |
| StevenK | lifeless: So that's why it fails to start? Er, can we not have it do that. | 09:16 |
| bigjools | so not a no-op | 09:16 |
| jtv | bigjools: fail it? That seems a bit harsh, esp. if it's perfectly reasonable for multiple admins to request the same copy concurrently. | 09:16 |
| wgrant | lifeless: That's remarkably constant progress we've made over the last month! | 09:16 |
| lifeless | wgrant: it is! | 09:17 |
| lifeless | StevenK: I don't know. patches appreciated, or perhaps put localhost first in the hosts file? | 09:17 |
| bigjools | jtv: well possibly in the future, but right now it'll fail | 09:17 |
| jtv | Which won't do much except generate an oops, I guess, after which a maintenance squad will pick it up very quickly. | 09:17 |
| StevenK | ubuntu@ip-172-56-124-250:~$ cat /etc/hosts | 09:18 |
| StevenK | 127.0.0.1 localhost | 09:18 |
| StevenK | lifeless: ^ | 09:18 |
| jtv | bigjools: right now my main concern is that it be fast, regardless of whether it ends in front of the restaurant or partway through its outside wall. :) | 09:18 |
| jtv | Car analogy ^ | 09:18 |
| bigjools | jtv: it'll be fast! | 09:18 |
| StevenK | lifeless: (There's more, but ... :-) | 09:18 |
| jtv | bigjools: then sod the brakes :) | 09:19 |
| bigjools | jtv: no oops, just a failed job, which we need to show on the page | 09:19 |
| lifeless | StevenK: >< | 09:19 |
| StevenK | lifeless: Hm? | 09:21 |
| lifeless | StevenK: I'm unhappy that this -long- overdue patch has broken jenkins | 09:21 |
| StevenK | lifeless: I can run hostname localhost on builder bring-up, but ew | 09:22 |
| === danilos changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: jtv, danilos | https://code.launchpad.net/launchpad-project/+activereviews | ||
| jtv | hi danilos | 09:25 |
| danilos | jtv, hi :) | 09:25 |
| danilos | jtv, and the answer is "no" | 09:25 |
| jtv | damn, beat me to it! | 09:25 |
| jtv | The other OCR sure ain't gonna do it | 09:25 |
| danilos | jtv, sure thing, send it my way | 09:25 |
| jtv | :) | 09:26 |
| danilos | 780319? | 09:26 |
| jtv | danilos: thanks... it's largish: https://code.launchpad.net/~jtv/launchpad/bug-780319/+merge/60571 | 09:26 |
| jtv | Yup | 09:26 |
| wgrant | bigjools: Is DF's appserver meant to be not running? | 09:26 |
| jtv | Again!? | 09:26 |
| bigjools | wgrant: last I checked it was ok | 09:27 |
| * bigjools looks at log | 09:27 | |
| * bigjools fixes | 09:27 | |
| bigjools | nohup: cannot run command `bin/run': No such file or directory | 09:28 |
| bigjools | awsum :) | 09:28 |
| wgrant | Handy. | 09:28 |
| bigjools | grar, how do I stop this PoS mumble from starting the config wizard every time I start it | 09:28 |
| stub | jtv: Is the only usage of multitablecopy copying translations into a newly opened distro? | 09:29 |
| jtv | stub: afaics yes | 09:29 |
| jtv | (did a quick grep) | 09:29 |
| stub | jtv: I guess we can always recover from that job if things explode - we have done it a few times already. | 09:30 |
| jtv | And actually it's no longer even copying the translations themselves. | 09:30 |
| jtv | At the time it was a huge enough problem that the overhead of making this big a generic module wasn't a big deal. But I guess it hasn't been a smash success. | 09:31 |
| jtv | otp | 09:31 |
| StevenK | lifeless: So starting rabbit gives: Crash dump was written to: erl_crash.dump | 09:32 |
| NCommander | Grumble, I'm getting a ComponentLookupError when I try to do self.admin = getUtility(IPersonSet).getByEmail(ADMIN_EMAIL) | 09:36 |
| wgrant | NCommander: Does your test class have a layer set? | 09:40 |
| wgrant | NCommander: Try LaunchpadFunctionalLayer. | 09:40 |
| danilos | jtv, fwiw, you do already have conflicts :) | 09:47 |
| jtv | again!? | 09:47 |
| jtv | danilos: otp now, will fix soonest | 09:47 |
| danilos | jtv, oh, perhaps they are just in the email | 09:47 |
| danilos | can't see them on the MP, all good then | 09:47 |
| jtv | Phew! | 09:47 |
| StevenK | lifeless: So help to fix rabbit would be awesome, but I personally am *utterly* unimpressed with it. | 09:54 |
| NCommander | wgrant: ah, didn't know that was used by anything | 09:55 |
| NCommander | wgrant: now I've gotten librarian traceback errors. Obviously something is unhappy with my testing setup | 09:58 |
| adeuring | stub, lifeless: could you please have a look at my mp: https://code.launchpad.net/~adeuring/launchpad/bug-739075/+merge/60516 ? | 09:58 |
| wgrant | NCommander: Which layer did you use? | 09:59 |
| wgrant | And what's the traceback? | 09:59 |
| lifeless | jml: https://bugs.launchpad.net/launchpad/+bug/655385/comments/14 | 10:07 |
| _mup_ | Bug #655385: Allow bug status change from Triaged only for bug supervisor <accesscontrol> <acl> <bugs> <lp-bugs> <Launchpad itself:Opinion> < https://launchpad.net/bugs/655385 > | 10:07 |
| lifeless | jml: is the analysis I was looking for | 10:08 |
| lifeless | jml: I segue from the bug itself quite sharply | 10:08 |
| jml | lifeless: right, thanks. | 10:09 |
| lifeless | jml: I've mailed you and skaet | 10:12 |
| lifeless | should I have cc'd ali/ | 10:12 |
| lifeless | ? | 10:12 |
| jml | lifeless: probably ok for just skaet & I atm. Have linked it from IssueTracker as well. | 10:13 |
| lifeless | cool | 10:13 |
| lifeless | jml: do you need me for anything? I might go remind my self what my wife looks like... | 10:13 |
| jml | lifeless: no, I think we're good. | 10:13 |
| jml | lifeless: thanks for staying around | 10:13 |
| lifeless | kk, my pleasure | 10:14 |
| danilos | jtv, r=me with some minor comments; general impression is that it's mostly refactorings, and that's even easier to review as two separate branches :) | 10:16 |
| danilos | oh well | 10:17 |
| LPCIBot | Project windmill-db-devel build #263: STILL FAILING in 1 hr 7 min: https://lpci.wedontsleep.org/job/windmill-db-devel/263/ | 10:18 |
| danilos | <-- jtv has quit (Read error: Connection reset by peer) | 10:20 |
| danilos | <danilos> jtv, r=me with some minor comments; general impression is that it's mostly refactorings, and that's even easier to review as two separate branches :) | 10:20 |
| stub | adeuring: reviewed | 10:43 |
| adeuring | stub: thanks! | 10:43 |
| stub | was on phone | 10:43 |
| stub | jtv: https://code.launchpad.net/~stub/launchpad/bug-179821-lowercase-tablenames/+merge/60511 should be easy. https://code.launchpad.net/~stub/launchpad/bug-778338-transient-schema/+merge/60512 is a further uglification of your code. | 10:52 |
| jtv | stub: are you asking for reviews? | 10:52 |
| stub | jtv: If you want to. Otherwise I can grab an ocr. | 10:52 |
| jtv | stub: check again. | 10:53 |
| stub | oic. Yes, you can review it then :) | 10:53 |
| stub | I was just giving you first right of refusal since this was originally your work. | 10:53 |
| stub | The second one went ugly, as "foo.bar" is a table in the public schema, unlike "foo"."bar" | 10:54 |
| stub | Either that, or refactor most of canonical.database.postgresql | 10:56 |
| jtv | stub: as far as the first one is concerned, there's a quandary: we're probably supposed to quote variable table names, which is exactly what leads to the case sensitivity. If only there were a way to escape, but not quote them! | 10:56 |
| stub | Indeed. I think at one stage I had an assert to ensure the name didn't need to be quoted. Think I lost that somewhere. | 10:57 |
| jtv | stub: I'm not sure it's worth migrating existing tables... with single-master and just a few production-like environments, wouldn't it be easier to check for any existing copying tables? | 10:57 |
| stub | Oh... we are still quoting table names. I'm just forcing them to lowercase. | 10:57 |
| jtv | Yes. | 10:58 |
| jtv | Which should work fine, as long as you don't run anything in a Turkish locale. | 10:58 |
| stub | jtv: I don't want the rollout aborted because there was a job running at the time. | 10:58 |
| jtv | It's a manual job, and the one for Oneiric has already been run. | 10:58 |
| stub | ok. so if we promise not to run another one in the near future, I can land this on launchpad/devel then. | 10:59 |
| stub | Although there is little point landing this code until before the next time one of these jobs is run... so... whatever :) | 10:59 |
| wgrant | stub: The less we have in db-devel the better. | 11:01 |
| stub | jtv: I think lower() is fine and we can assume nobody is going to stick Turkish or worse in here. I think it is either lower(), or generate the tablename using punycode or sha1 hash. | 11:04 |
| jtv | stub: sure, I was just being flippant. | 11:05 |
| stub | Its a valid concern. At the moment, no callsites of this code will get non-ascii user input. | 11:05 |
| jtv | (And since I'm still feeling a _little bit_ flippant, let me point out that "I".lower() != "i" in Turkish. :) | 11:05 |
| stub | I think myself and all the losas are all native english speakers so that is fine :) | 11:06 |
| jtv | I think it's fine to have strong requirements for this. I've forgotten most of the details but maybe you could simply require lower-case strings and fail if that requirement is not met? | 11:06 |
| stub | sure. I'll have callsites to modify then though? Many? | 11:07 |
| NCommander | wg /win 24 | 11:07 |
| NCommander | bah | 11:07 |
| NCommander | wgrant: http://paste.ubuntu.com/606078/ | 11:07 |
| stub | The multitable tests are all lowercase | 11:08 |
| wgrant | NCommander: You're not using LP_PERSISTENT_TEST_SERVICES? | 11:08 |
| wgrant | (that's an environment variable... it should be unset these days) | 11:09 |
| NCommander | wgrant: its not set | 11:10 |
| wgrant | :( | 11:11 |
| wgrant | No pids floating around in /var/tmp? | 11:11 |
| NCommander | wgrant: no love | 11:12 |
| jtv | stub: sorry, food arrived... no, I don't think there'll be many callsites. | 11:13 |
| wgrant | NCommander: Can you start a dev instance OK? | 11:14 |
| NCommander | wgrant: works fine. | 11:14 |
| NCommander | correction | 11:15 |
| NCommander | I'm OOPSIng | 11:15 |
| stub | I suspect the Librarian won't be starting. | 11:15 |
| NCommander | now the question is why isn't it | 11:15 |
| === jtv changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: danilos | https://code.launchpad.net/launchpad-project/+activereviews | ||
| stub | NCommander: Got a librarian log file in logs directory? | 11:16 |
| NCommander | yeah, looks like librarian doesn't want to start, but I can't figure out why | 11:16 |
| stub | Might have a juicy traceback for you | 11:16 |
| NCommander | where do I find that? | 11:16 |
| stub | I think logs directory, top of the lp tree. | 11:17 |
| wgrant | Otherwise there could be something interesting in /var/tmp | 11:17 |
| NCommander | don't have any librarian logs | 11:17 |
| stub | Try bin/start_librarian | 11:18 |
| NCommander | bah, I'm going to reboot | 11:21 |
| wgrant | bigjools: You (or you squad) have a lucid_db_lp failure | 11:25 |
| wgrant | (<DistroSeries u'distroseries344950'>, 'getDifferencesTo', 'launchpad.Edit') | 11:25 |
| NCommander | Ok, I rebooted | 11:26 |
| NCommander | launchpad.dev works, my test still breaks (Connection refused is the error now) | 11:26 |
| wgrant | That's more encouraging. | 11:27 |
| NCommander | wgrant: http://paste.ubuntu.com/606086/ | 11:27 |
| wgrant | Can you try attaching adding a bug attachment on launchpad.dev, just to test that it's really working? | 11:27 |
| NCommander | can't login | 11:28 |
| NCommander | grumble | 11:28 |
| wgrant | Your /etc/hosts might be old. | 11:29 |
| wgrant | Does testopenid.dev resolve? | 11:29 |
| NCommander | 2011-05-11 03:25:07-0700 [-] twisted.web.server.Site starting on 58085 | 11:29 |
| NCommander | 2011-05-11 03:25:07-0700 [-] Starting factory <twisted.web.server.Site instance at 0x383c3b0> | 11:29 |
| NCommander | 2011-05-11 03:25:07-0700 [-] Not using upstream librarian | 11:30 |
| NCommander | 2011-05-11 03:25:07-0700 [-] daemon ready! | 11:30 |
| NCommander | wgrant: yes, it sresolves | 11:30 |
| NCommander | I can' tlogin cause it oops'ed | 11:30 |
| NCommander | 3745 pts/1 Sl+ 0:09 /usr/bin/python2.6 -S /home/mcasadevall/launchpad/lp-branches/arch-indep-skew-fix/bin/twistd --no_save --nodaemon --python /home/mcasadevall/launchpad/lp-branches/arch-indep-skew-fix/daemons/librarian.tac --pidfile /var/tmp/development-librarian.pid --prefix Librarian --logfile librarian.log | 11:30 |
| NCommander | I've got librarian running, nothing interesting in the log | 11:30 |
| wgrant | What was the OOPS? | 11:30 |
| NCommander | OOPS-1957X7 | 11:31 |
| wgrant | That refers to /var/tmp/lperr/2011-05-11/*X7, so it's not very helpful for me :( | 11:31 |
| NCommander | http://paste.ubuntu.com/606088/ | 11:32 |
| NCommander | wgrant: ^ | 11:34 |
| wgrant | What. | 11:34 |
| NCommander | wgrant: that's what that file has | 11:34 |
| wgrant | Do you have a broken proxy set or something? | 11:34 |
| NCommander | don't think so, this was workign fine ... | 11:34 |
| wgrant | Or a broken apache or something. | 11:34 |
| NCommander | grumble | 11:34 |
| === al-maisan is now known as almaisan-away | ||
| wgrant | NCommander: AFAICT that should be grabbing a page from blog.launchpad.net. | 11:35 |
| * NCommander make stop/make clean/make run's | 11:35 | |
| jtv | lifeless: it seems great minds think alike. | 11:41 |
| === henninge is now known as henninge-lunch | ||
| NCommander | this is really irritating me :-/ | 11:43 |
| * NCommander probably should throw LP into a VM and save myself some frustation | 11:43 | |
| === mrevell is now known as mrevell-lunch | ||
| LPCIBot | Project windmill-db-devel build #264: STILL FAILING in 1 hr 8 min: https://lpci.wedontsleep.org/job/windmill-db-devel/264/ | 12:49 |
| === henninge-lunch is now known as henninge | ||
| gary_poster | hi jml. When you get a chance (no rush), I'd like your review of my non-high triaging of bug 780568. All the other new high/critical bugs I agree need to be addressed before we move on. | 13:19 |
| _mup_ | Bug #780568: bug structural subscription auto tag complete <story-better-bug-notification> <Launchpad itself:Triaged> < https://launchpad.net/bugs/780568 > | 13:19 |
| jml | gary_poster: yeah. that's fair enough. | 13:20 |
| gary_poster | coolcool thanks | 13:20 |
| === mrevell-lunch is now known as mrevell | ||
| jml | how do I demo the unsubscribe-in-anger feature? | 13:40 |
| jml | all I need is a screenshot, actually | 13:40 |
| deryck | Morning, all. | 13:50 |
| danilos | jml, click on "Edit bug mail" link on any bug page | 13:54 |
| bigjools | wgrant: can you remember if someone fixed a conflict on db-devel just before buildbot whinged? | 13:54 |
| danilos | jml, (that page is going to be linked from all emails, but it was a mess to make the link conditional in the email template so we left it for when we go fully public) | 13:54 |
| wgrant | bigjools: There was no conflict. | 13:54 |
| bigjools | hmm | 13:55 |
| bigjools | wgrant: it's rvb's API change, not quite sure why it put it in EditRestricted.... and how it passed ec2 | 13:57 |
| wgrant | bigjools: It's not related to the deriveDistroSeries permission change? | 13:58 |
| wgrant | In devel? | 13:58 |
| wgrant | The API changes landed days ago. | 13:59 |
| bigjools | db-devel | 13:59 |
| wgrant | In db-devel, yes. | 13:59 |
| wgrant | Then deriveDistroSeries permission changes landed yesterday in devel. | 13:59 |
| bigjools | why would my change affect that? I only moved deriveDistroSeries | 13:59 |
| wgrant | And would have been merged into db-devel today... | 13:59 |
| wgrant | Hmm. | 13:59 |
| wgrant | Have you bisected? | 13:59 |
| * bigjools looks at self | 13:59 | |
| * bigjools is still in once piece | 13:59 | |
| wgrant | Sounds painful. | 13:59 |
| wgrant | It is 11pm, so it's your problem :) | 14:00 |
| bigjools | gee thanks | 14:00 |
| bigjools | :) | 14:00 |
| danilos | bigjools, I can help bisect you ;) | 14:00 |
| bigjools | danilos: no I am not rooming with you again | 14:00 |
| danilos | bigjools, a shame, a shame :) | 14:00 |
| bigjools | :) | 14:00 |
| deryck | henninge, ping for standup | 14:01 |
| === almaisan-away is now known as al-maisan | ||
| wgrant | bigjools: Any luck with the conflict? | 14:22 |
| bigjools | what conflict? | 14:22 |
| wgrant | Test failure, whatever it is these days. | 14:23 |
| bigjools | just sent a fix in | 14:23 |
| wgrant | Great. | 14:23 |
| bigjools | I have *no* idea how that *ever* passed *any* buildbot/ec2 run | 14:23 |
| === danilos changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: - | https://code.launchpad.net/launchpad-project/+activereviews | ||
| jml | gmb: I'm doing a lightning talk tomorrow on the bug subscription stuff. I'm trying to get a screenshot of the unsubscribe-in-anger stuff | 14:34 |
| danilos | jml, did you see my suggestions above? | 14:35 |
| jml | danilos: no, I didn't. | 14:35 |
| jml | sorry | 14:35 |
| jml | my IRC backlog spew is not working: ( | 14:35 |
| gmb | jml: $bug/+subscriptions will give you the UIA page for any bug. | 14:35 |
| danilos | <danilos> jml, click on "Edit bug mail" link on any bug page | 14:35 |
| danilos | <danilos> jml, (that page is going to be linked from all emails, but it was a mess to make the link conditional in the email template so we left it for when we go fully public) | 14:35 |
| gmb | (Or that) | 14:35 |
| jml | Thanks :) | 14:36 |
| jml | also, how do I remove the product team from the yellow team | 14:36 |
| danilos | jml, you may want to have a suitable set of subscriptions through teams, to duplicates and structural subscriptions | 14:36 |
| jml | this Launchpad thing can be very confusing at times | 14:36 |
| danilos | jml, yeah, you should talk to our strategist about that :) | 14:36 |
| jml | danilos: he says they know about it and would love to work on it but don't have the capacity right now. | 14:37 |
| danilos | jml, anyway, deactivated the product team in ~yellow | 14:37 |
| jml | danilos: thanks. | 14:37 |
| danilos | :) | 14:37 |
| jml | https://launchpad.net/~yellow/+mugshots still shows my mug :( | 14:38 |
| danilos | jml, sounds like a different bug :( | 14:39 |
| wgrant | jml: That page is memcached. We can turn that off with an FF. | 14:39 |
| gmb | Jeez, I still have that potato photo up. | 14:39 |
| gmb | Mind you, could be worse | 14:39 |
| mwhudson | it also says, results 1-5 of 5 and then shows 11 photos | 14:40 |
| gmb | Heh. Nice. | 14:40 |
| danilos | gmb, heh, I had photo set by kiko originally to something weird... forcing me to change it something like 18 months later | 14:40 |
| gmb | "There are probably at least five people in this team" | 14:40 |
| danilos | mwhudson, so obviously, a portlet is memcached only | 14:40 |
| danilos | or a page snippet, making it all the more fun | 14:41 |
| mwhudson | danilos: yeah | 14:41 |
| henninge | abentley: two gestions about API use: | 14:41 |
| wgrant | https://launchpad.net/~yellow/+mugshots?nocachethanks is a bit more sensible :) | 14:41 |
| henninge | abentley: how do I switch launchpadlib to use 'devel' instead of 1.0? | 14:42 |
| jml | wgrant: thanks. I didn't know about that trick. | 14:42 |
| henninge | abentley: how do I get to the HTML representation of an object, as mentioned in the bug description? | 14:42 |
| abentley | henninge: You specify the root URL to your initial client call. | 14:42 |
| wgrant | Of course now *that*'s cached. | 14:42 |
| jml | also, anyone got spare time for gravatar integration? | 14:42 |
| gmb | Heh. The three big problems, eh. | 14:42 |
| wgrant | henninge, abentley: Give Launchpad.login_with the kwarg version='devel'. | 14:42 |
| gmb | jml: +many | 14:42 |
| abentley | henninge: my bad. | 14:42 |
| henninge | abentley: no worries | 14:43 |
| henninge | wgrant: thanks | 14:43 |
| wgrant | Hacking the URL was right until Lucid. | 14:43 |
| abentley | henninge: The HTML representation is just the web service representation, as a list of javascript variables. | 14:44 |
| wgrant | Objects can define custom HTML representations. | 14:44 |
| henninge | abentley: what I get when I call the URL in a browser, right? | 14:44 |
| abentley | wgrant: you mean, the JSON representation provided by the LP.cache can be overridden? | 14:45 |
| abentley | henninge: No, I don't know what you mean. | 14:46 |
| wgrant | abentley: The JSON representation is distinct from the HTML representation. | 14:46 |
| abentley | wgrant: So this is an additional representation provided by the web service on top of JSON and XML representations? | 14:47 |
| wgrant | abentley: There are two types of representation that lazr.restful can give you: JSON and XHTML. You can also ask it to give you the XHTML inside the rest of the JSON. | 14:48 |
| wgrant | abentley: The default XHTML representation is just a list of attributes and their values, but interfaces can declare adapters to give custom representations. | 14:48 |
| wgrant | https://api.launchpad.net/devel/launchpad is the default. | 14:49 |
| wgrant | https://api.launchpad.net/devel/~wgrant is a custom one. | 14:49 |
| abentley | wgrant: Okay. So I guess henninge needs to know how you request that representation using launchpadlib. | 14:49 |
| wgrant | NFI | 14:49 |
| wgrant | You can override it with ws.accept | 14:49 |
| wgrant | But not sure how to tell launchpadlib about that. | 14:49 |
| henninge | abentley, wgrant: that's fine for now. Thanks | 14:50 |
| wgrant | eg. https://api.launchpad.net/devel/~wgrant?ws.accept=application/json | 14:50 |
| abentley | henninge: The JSON representation ends up in the LP.cache through lib/lp/app/templates/base-layout-macros.pt:174 | 14:51 |
| henninge | abentley: thanks | 14:52 |
| wgrant | You add stuff to that using IJSONRequestCache. | 14:53 |
| wgrant | Do not attempt to serialise JSON manually. | 14:53 |
| wgrant | Or I will come and get you :) | 14:53 |
| abentley | wgrant: he's fixing bug 740208 | 14:53 |
| wgrant | Ah. | 14:54 |
| wgrant | That's a fairly difficult one. | 14:54 |
| henninge | I always pick those it seems ... | 14:54 |
| wgrant | I have very little idea of how to fix it sanely. | 14:54 |
| wgrant | Apart from slapping people who think it's necessary. | 14:54 |
| wgrant | But that's less than productive :( | 14:54 |
| bigjools | wgrant, the voice of reason | 15:01 |
| mwhudson | i like wgrant's proposed solution | 15:05 |
| aalex-sat | hello | 15:09 |
| aalex-sat | eh I could work for Canonical, since I know Twisted. :) I didn't know that you guys were using it a lot. | 15:10 |
| aalex-sat | I'm currently working on something much similar to Ubuntu One. Maybe I shouldn't reinvent the wheel. | 15:11 |
| === abentley changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: abentley | https://code.launchpad.net/launchpad-project/+activereviews | ||
| abentley | deryck: I'm OCR, so could you please review https://code.launchpad.net/~abentley/launchpad/safe-branch-upgrade/+merge/60634 ? | 15:34 |
| deryck | abentley, definitely. | 15:34 |
| abentley | deryck: thanks.' | 15:34 |
| sinzui | jcsackett: any commercial admin can increase an archive size. I am one | 15:45 |
| * sinzui looks at question | 15:45 | |
| bigjools | sinzui: we need to fix that. commercial admins have far too many powers over PPAs | 15:48 |
| wgrant | bigjools: I guess we probably want to grant the OEM namespace an entitlement for unlimited private PPAs and quotas. | 15:51 |
| wgrant | Then we can remove commercial admins. | 15:51 |
| wgrant | I suppose. | 15:51 |
| bigjools | not quite | 15:52 |
| sinzui | bigjools: I think ~registry needs to change the archive size. That is all I want to support for the whole team | 15:52 |
| bigjools | we need a PPA admin celeb | 15:52 |
| wgrant | Why? | 15:52 |
| bigjools | the celeb will be able to do what commercial can currently do to PPAs and we restrict it better. We then need to come up with a way for certain people to self-service within some boundary | 15:53 |
| deryck | abentley, looks great. r=me. | 16:05 |
| abentley | deryck: thanks. | 16:06 |
| deryck | abentley, np! | 16:06 |
| deryck | benji, I'm trying to qa my fix from yesterday that you reviewed... the click unsubscribe link doesn't remove name issue.... | 16:43 |
| deryck | benji, but I wrote this, not being mindful of the feature flag, and can't qa against the unsubscribe link myself... | 16:43 |
| deryck | benji, but I notice the "Edit subscription" goes to loading when you're subscribed via a dupe and never completes. | 16:44 |
| deryck | benji, wondering if this is known or something I introduced? | 16:45 |
| === al-maisan is now known as almaisan-away | ||
| benji | deryck: that doesn't ring a bell, let me look at the full bug list real quick | 16:45 |
| === mpt_ is now known as mpt | ||
| benji | deryck: I don't see anything on https://bugs.edge.launchpad.net/launchpad-project/+bugs?field.tag=story-better-bug-notification that sounds like that, so either we haven't noticed or you did it ;) | 16:47 |
| deryck | benji, ok, thanks | 16:48 |
| benji | sinzui: since you've been on this click-to-close message box journey with me from the begining, would you like to look at the final (I hope) revision? https://code.launchpad.net/~benji/launchpad/bug-779538/+merge/60650 | 16:49 |
| deryck | benji, but is the expected behavior that I should be subscribed via a duplicate and click "edit subscription" and be able to drop my subscription via a dupe that way? | 16:49 |
| benji | deryck: that I don't know | 16:50 |
| deryck | benji, ok, thanks again | 16:50 |
| === matsubara is now known as matsubara-lunch | ||
| deryck | benji, I filed bug 781245 about what we talked about. I'll poke a bit more to see if its from me or not. | 17:27 |
| _mup_ | Bug #781245: Clicking "Edit subscription" when subscribed via a duplicate gets stuck at "Loading..." <story-better-bug-notification> <Launchpad itself:Triaged> < https://launchpad.net/bugs/781245 > | 17:27 |
| === deryck is now known as deryck[lunch] | ||
| LPCIBot | Project db-devel build #538: FAILURE in 5 hr 11 min: https://lpci.wedontsleep.org/job/db-devel/538/ | 18:00 |
| benji | abentley: do you have a minute to review https://code.launchpad.net/~benji/launchpad/bug-779538/+merge/60650 for me? | 18:09 |
| benji | I think the MP has all the background you need, but feel free to ask if you have any questions. | 18:09 |
| abentley | benji: okay. | 18:09 |
| abentley | benji: Why not just tweak the delegate callback to only affect the correct link? | 18:16 |
| === matsubara-lunch is now known as matsubara | ||
| benji | abentley: in the former approach there wasn't a link, the "Hide X" "link" was just CSS style, the on-click actually happened on the message box itself, that meant that clicking on a link, or selecting text, or control-/shift-clicking on a link would trigger the on-click | 18:18 |
| benji | this way there's a real DOM element to hook an on-click up to so we don't react to all those stray events | 18:18 |
| abentley | benji: So normally, we'd just hook this up when the DOM is ready. I assume the notifications are created after DOMReady? | 18:21 |
| benji | abentley: they can be; for example the new structural subscription bits create message boxes after doing AJAX requests | 18:22 |
| abentley | benji: So could the javascript that creates the notification also attach the handler? | 18:23 |
| benji | abentley: it could, but that would mean we have to remember every time; sounds easy to forget | 18:25 |
| abentley | benji: Really? There's no common code to create a notification? | 18:25 |
| benji | not that I'm aware of; all the ones I know of are done server-side; if there is, then the fact that we created ours directly (Y.Node.create) suggests to me that others will do the same | 18:26 |
| abentley | benji: Having everyone create their notifications using Y.Node.create sounds like something we should discourage. | 18:30 |
| === deryck[lunch] is now known as deryck | ||
| abentley | benji: instead of polling, what about a "contentready" event handler? | 18:30 |
| benji | abentley: I wasn't aware that contentready would fire when new elements were added to the DOM | 18:34 |
| abentley | benji: I don't know for sure, but it seems likely there's a way to detect new elements being added to the DOM. | 18:34 |
| benji | re. Y.Node.create: <shrug> It seems like embracing our medium to me. | 18:35 |
| abentley | benji: It seems like it would lead to discrepancies and confusion to me. | 18:36 |
| abentley | benji: Notifications have a higher-level meaning to us. They're not just HTML. | 18:36 |
| benji | threre's the DOMNodeInserted event, but it's still in draft and slightly older Firefoxes don't support it, and I'd be afraid of an event storm on our larger pages | 18:38 |
| benji | well, at this point I've exhausted my timebox, so I'll instead remove the current, broken click-to-close behavior | 18:39 |
| sinzui | benji: I agree they are not HTML, but we have several bug open that indicate that developers think they are HTML. Many pages show object state as a notification: bug-converted-to-question, product-deactivated, package-not-published. | 18:39 |
| sinzui | benji: I think reverting is a fair decision. The only case I want to close a notification is the broken cases above. I know the notification will be gone if I reload | 18:41 |
| LPCIBot | Project windmill-db-devel build #265: STILL FAILING in 44 min: https://lpci.wedontsleep.org/job/windmill-db-devel/265/ | 18:45 |
| benji | abentley: here's the remove-click-to-close MP: https://code.launchpad.net/~benji/launchpad/remove-click-to-close/+merge/60673 | 19:16 |
| abentley | benji: It's not true that it was deemed unsuitable in review. You terminated review before I reached a conclusion. | 19:17 |
| benji | abentley: I was thinking more of my conversation with sinzui. | 19:19 |
| benji | I didn't intend to cut the review short, but I don't see how we would have ended up landing the branch (more or less) as-is. | 19:20 |
| abentley | benji: r=me. | 19:21 |
| benji | thanks | 19:21 |
| abentley | benji: It's important to avoid polling unless there is no reasonable alternative. I was trying to find out whether the alternatives had been exhausted. | 19:22 |
| benji | yep, absolutely understandable | 19:24 |
| abentley | If I believed that there was no reasonable alternative, I would have been willing to approve a polling-based solution. | 19:25 |
| === Ursinha is now known as Ursinha-afk | ||
| deryck | abentley, hi. I have an easy one for review, if you're available. | 20:41 |
| abentley | deryck: sure. | 20:42 |
| deryck | abentley, thanks! https://code.launchpad.net/~deryck/launchpad/edit-subscription-always-loading-781245/+merge/60683 | 20:42 |
| abentley | deryck: r=me | 20:44 |
| deryck | abentley, awesome, thanks! | 20:44 |
| === lifeless_ is now known as lifeless | ||
| lifeless | gary_poster: hi | 20:50 |
| gary_poster | lifeless, hey | 21:06 |
| abentley | deryck: chat? | 21:11 |
| deryck | abentley: sure. | 21:12 |
| lifeless | gary_poster: wondering if you wanted to chat about the services draft | 21:12 |
| deryck | abentley: mumble or here? | 21:12 |
| abentley | deryck: mumble | 21:13 |
| deryck | ok | 21:13 |
| gary_poster | lifeless, it looks like it has grown further since my last skim. Lemme finish what I'm up to, then I'll read again and more carefully. I'm out in 45 min (and trying to be strict about it because of the baby). I'll touch base with you in 30 min and let you know if I have any immediate thoughts, and if I think I have anything worth talking about over a longer time. | 21:15 |
| lifeless | gary_poster: thats great | 21:17 |
| lifeless | sinzui: hi | 21:37 |
| lifeless | sinzui: I'm trying to do what bug 781326 asks for but | 21:37 |
| _mup_ | Bug #781326: Please remove erroneous LibreOffice Launchpad project website https://launchpad.net/libreoffice <Launchpad itself:New> < https://launchpad.net/bugs/781326 > | 21:37 |
| lifeless | it had a series linked to sid (the df-libreoffice has its series linked to natty and oneiric | 21:37 |
| lifeless | so I unlinked and relinked | 21:37 |
| lifeless | the https://launchpad.net/libreoffice/+admin form though is whinging that there is a series | 21:38 |
| lifeless | sorry, a package link | 21:38 |
| lifeless | which I just deleted :( | 21:38 |
| lifeless | sinzui: any thoughts on how to address ? | 21:38 |
| === abentley changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: - | https://code.launchpad.net/launchpad-project/+activereviews | ||
| lifeless | gary_poster: hi | 21:56 |
| lifeless | gary_poster: you're about to run; thought i would ping a touch-base before then | 21:56 |
| gary_poster | lifeless: hi, I didn't forget/ :-) I'm reading and making notes as I go. I think I do have some thoughts, though I don't know how helpful they are, since they are so casual | 21:57 |
| lifeless | gary_poster: cool! I had no doubts :) | 21:59 |
| lifeless | gary_poster: if you can email me what you have when you're done; I'll incorporate them, and then start wider distribution of this for discussion | 21:59 |
| gary_poster | lifeless, will do. they are of this sort--questions, casual thoughts, and so on. http://pastebin.ubuntu.com/606283/ Maybe they will be more substantial when done. :-) | 22:02 |
| lifeless | thats good stuff | 22:03 |
| gary_poster | ok, I'll keep going tomorrow and send it your way then lifeless. | 22:03 |
| gary_poster | have a great day | 22:03 |
| NCommander | wgrant: so I've been starring at Soyuz, and I'm kinda lost on using the factory, I get that SoyuzTestPublisher does things like setup BreezyAutoTest, and it seems its got enough methods to at least get my initial uploads in, publish them, etc. | 23:12 |
| NCommander | I'm very confused on where the Factory comes in, or why test_build_set has a lot of things that seem ignored by SoyuzTestPublisher ... | 23:12 |
| === cinerama_ is now known as cinerama | ||
Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!