| mbarnett | bah, gdb not installed | 00:02 |
|---|---|---|
| wgrant | rofl | 00:03 |
| mwhudson | wgrant: probably, but i suspect that it will be very boring, just twisted sitting in an select loop with no active sockets | 00:03 |
| lifeless | a clean shutdown *should* tell us about un-gc'd deferreds | 00:04 |
| lifeless | also | 00:04 |
| lifeless | triggering a gc from gdb should do that | 00:04 |
| wgrant | mwhudson: There should be lots of them, though... | 00:05 |
| mwhudson | yeah | 00:05 |
| lifeless | lsof output too then | 00:05 |
| wgrant | Aha. | 00:06 |
| wgrant | Got it. | 00:06 |
| wgrant | mbarnett: Can you check if there's a running PPA builder reset trigger? | 00:06 |
| wgrant | I'm not sure what it's called. | 00:06 |
| wgrant | vm_resume_command is the config key. | 00:07 |
| poolie | lifeless: flacoste suggested i should also land the flags ui into lp devel | 00:07 |
| poolie | since at the moment it seems to now be only in db-devel | 00:07 |
| poolie | but its precondition db changes should now be in devel | 00:07 |
| lifeless | poolie: yes, definitely | 00:07 |
| mbarnett | wgrant: i don't see one off hand | 00:08 |
| wgrant | Hm, now it won't respond to SIGTERM. | 00:08 |
| * wgrant is trying a few things locally. | 00:08 | |
| wgrant | mbarnett: There really should be one running. | 00:10 |
| lifeless | wgrant: a hung one ? | 00:10 |
| wgrant | lifeless: Ideally. | 00:10 |
| wgrant | It looks like it's an ssh process. | 00:10 |
| wgrant | ./ftpmaster/launchpad-lazr.conf:vm_resume_command: ssh -i /home/lp_buildd/.ssh/ppa-reset-builder ppa@%(vm_host)s | 00:11 |
| wgrant | It will hang if that does, but recovers if it dies, AFAICT. | 00:11 |
| mbarnett | here is everything running as the lp_buildd user: http://pastebin.ubuntu.com/479127/ | 00:12 |
| wgrant | Mmm. Zombies. | 00:13 |
| wgrant | So, it will hang until the resume trigger exits after a communication failure, bypassing even the normal timeout. But I don't see why it would continue to hang if there's no process running. | 00:14 |
| mbarnett | hah, i think lamont came along and kicked the hung process. | 00:15 |
| wgrant | Ah, yes, everything's running again. | 00:16 |
| mbarnett | yeah, now we are going | 00:16 |
| wgrant | So that was probably it. | 00:16 |
| mbarnett | lamont> or a child ssh process killed. /me did that, seems happier now | 00:17 |
| wgrant | Phew. | 00:17 |
| thumper | lifeless: ping | 00:18 |
| mbarnett | okay, i see the ssh process he killed | 00:18 |
| mbarnett | good to know that will get things moving agian | 00:18 |
| thumper | lifeless: I've got the bzr failure down to two test files | 00:19 |
| lifeless | COOL | 00:19 |
| lifeless | bah | 00:19 |
| thumper | lifeless: is it worthwhile trying to eliminate individual tests? | 00:19 |
| lifeless | Cool | 00:19 |
| lifeless | thumper: it may well be | 00:19 |
| thumper | lifeless: ok | 00:19 |
| mbarnett | well, all builders are now actually doing stuff, so that is good | 00:23 |
| wgrant | mbarnett: Bug #618955 | 00:27 |
| wgrant | Bug #618955? | 00:27 |
| wgrant | :( | 00:27 |
| mbarnett | thanks for filing that. I have subscribed us losas | 00:29 |
| thumper | lifeless: gc.collect likely to screw things up? | 00:31 |
| lifeless | thumper: I wouldn't expect it too | 00:31 |
| thumper | lp.translations.scripts.tests.test_message_sharing_migration.TestSharingMigrationPerformance | 00:32 |
| thumper | lifeless: that test makes the other one fail | 00:32 |
| thumper | lifeless: I'm just seeing if it is either test in that test case, or a specific one | 00:32 |
| lifeless | progress! | 00:32 |
| thumper | lifeless: either of the two tests in that testcase cause the failure | 00:34 |
| thumper | lifeless: calling TestSharingMigrationPerformance._listLoadedObjects triggers the error | 00:51 |
| thumper | which uses gc.get_objects | 00:51 |
| thumper | I'm guessing that is it | 00:52 |
| lifeless | zomg | 00:52 |
| lifeless | yes, a gc held reference to a lazy object is equivant to a bound alias | 00:52 |
| lifeless | don't-do-that | 00:52 |
| lifeless | :) | 00:52 |
| thumper | advice needed on how to fix this test | 00:53 |
| lifeless | hmm | 00:54 |
| lifeless | easiest not to call that damn thing at all | 00:54 |
| lifeless | its a high cost function | 00:55 |
| lifeless | perhaps installing a Storm Tracer | 00:55 |
| lifeless | its also a poor surrogate for what I think they want to test | 00:56 |
| thumper | I don't really understand their test | 00:56 |
| lifeless | so | 00:56 |
| lifeless | I think what they want to say is | 00:56 |
| lifeless | 'if there are 15 pomsgids in memory before you call this function, there are only those 15 in memory afterwards' | 00:57 |
| lifeless | and the implication is that no pomsgids were loaded in the interim | 00:57 |
| lifeless | this only works if there is a non-weak cache installed (we have one) in the storm layer, and nothing resets it or otherwise invalidates it. | 00:57 |
| lifeless | try this | 00:58 |
| lifeless | do a gc.collect() at the top of _listLoadedObjects | 00:58 |
| lifeless | this doesn't invalidate the test because: | 00:58 |
| lifeless | - gc collect can happen anytime | 00:58 |
| lifeless | this should fix the problem because: | 00:58 |
| lifeless | - we don't have any references to the scope replacer | 00:59 |
| thumper | trying | 00:59 |
| lifeless | but also please file a tech debt bug - this is a very roundabout way to say that something didn't happen - its a surrogate for something we can measure directly with only a little more effort | 00:59 |
| thumper | nope | 00:59 |
| thumper | still fails | 00:59 |
| lifeless | darn | 00:59 |
| lifeless | oh | 01:00 |
| lifeless | by bad | 01:00 |
| thumper | we have code for storm tracers | 01:00 |
| lifeless | so that isn't working because | 01:00 |
| thumper | if it will only take a little more to fix, I could do that now | 01:00 |
| lifeless | there is a module in bzrlib that hasn't triggered its lazy load. | 01:00 |
| lifeless | and we go past it twice, or something | 01:01 |
| lifeless | though I am a little surprised it didn't work. Does it blow up in that function, or in your test code? | 01:01 |
| thumper | we have StormStatementRecorder | 01:02 |
| thumper | lifeless: it blows up in the following test | 01:02 |
| lifeless | I would cjec that there are no Selects that return a <type> | 01:02 |
| lifeless | *check* | 01:02 |
| lifeless | that seems much more targeted, to me. | 01:02 |
| thumper | hmm... | 01:02 |
| thumper | the statement recorder can't check that | 01:02 |
| thumper | it records the raw sql | 01:03 |
| lifeless | SELECT.*POSMSGSETID\..*FROM -> boom | 01:03 |
| thumper | I'm recording the statements now, with a pdb set | 01:05 |
| thumper | to look | 01:05 |
| lifeless | well | 01:05 |
| lifeless | we don't expect to see that, because the test 'passes' :) | 01:05 |
| lifeless | Ursinha: timeout bugs get high + triaged :) | 01:07 |
| lifeless | zero-oops-policy | 01:07 |
| lifeless | just fyi | 01:07 |
| thumper | you don't see POSMSGSETID.* in the SQL | 01:07 |
| lifeless | good | 01:07 |
| thumper | it gets expanded to all columns | 01:07 |
| thumper | storm expands the * | 01:07 |
| lifeless | right | 01:07 |
| lifeless | and it qualifies them | 01:07 |
| lifeless | posmsgsetid.foo | 01:07 |
| thumper | ah, I see what you mean now | 01:07 |
| lifeless | is what you'd see when pomsgsetid's are looked up | 01:08 |
| Ursinha | lifeless, cool, should I triage them as such when filing them? | 01:08 |
| lifeless | Ursinha: where there is as clear a policy as the zero-oops-one, I absolutely think so | 01:08 |
| Ursinha | cool | 01:08 |
| lifeless | Ursinha: where its going to be team-determined, then no - or at least, I don't in those cases except | 01:08 |
| Ursinha | lifeless, what's the difference | 01:09 |
| lifeless | where I am putting my TA hat on and asking as a favour-to-the-TA to do something | 01:09 |
| lifeless | Ursinha: well a non-oops non-timeout bug doesn't have a predetermined importance | 01:09 |
| thumper | is any in python 2.5? | 01:09 |
| Ursinha | lifeless, ah, I was only talking about timeout bugs :) | 01:09 |
| lifeless | Ursinha: :) | 01:10 |
| lifeless | Ursinha: I was giving a general answer :) | 01:10 |
| lifeless | sorry for de confusion | 01:10 |
| Ursinha | lifeless, no problem, thanks for the information :) | 01:10 |
| thumper | lifeless: can you check out this? http://pastebin.ubuntu.com/479160/ | 01:29 |
| thumper | lifeless: do you think it is testing enough of the same thing? | 01:29 |
| lifeless | +1 | 01:29 |
| lifeless | I certainly do | 01:30 |
| lifeless | should be faster too | 01:30 |
| thumper | lifeless: I'll break it into a separate branch | 01:30 |
| thumper | lifeless: and no bad interaction | 01:32 |
| mwhudson | lifeless: about performance tuesday, is the looking up of branch subscriptions because of the checking for launchpad.View ? | 01:33 |
| mwhudson | lifeless: if so, have you seen how the branchlistings avoid this particular death by sql? | 01:33 |
| * thumper files a bug on rosetta | 01:35 | |
| lifeless | mwhudson: I don't know | 01:36 |
| lifeless | mwhudson: I'm trying to get a test that is precisely the same | 01:36 |
| mwhudson | ok | 01:36 |
| lifeless | mwhudson: for now, I have one that shows 'one more query as you add a private bug' | 01:37 |
| lifeless | let me paste | 01:37 |
| lifeless | so, I can fix one cause of too many queries | 01:38 |
| lifeless | but I know I'll be missing the particular one :( | 01:38 |
| lifeless | http://pastebin.com/wPNqcuzn is my test | 01:38 |
| lifeless | mwhudson: AIUI you retrieve too much and then do a custom security check | 01:39 |
| mwhudson | lifeless: 'you'? | 01:39 |
| lifeless | 'branhc listings' | 01:39 |
| mwhudson | that's the launchpad default usually, it's not what branch listings do | 01:39 |
| lifeless | https://lp-oops.canonical.com/oops.py/?oopsid=OOPS-1689EB3170 | 01:40 |
| lifeless | is the oops I'm trying to duplicate | 01:40 |
| mwhudson | branch listings do a query to find the branches the user can see and then pre-populate the permission cache | 01:40 |
| lifeless | 3286439196420launchpad-main-slaveSELECT BugSubscription.bug, BugSubscription.date_created ... | 01:40 |
| lifeless | blah | 01:40 |
| mwhudson | so the security adapter doesn't do any queries | 01:40 |
| lifeless | 328 6439 19 6420 launchpad-main-slaveSELECT BugSubscription.bug, BugSubscription.date_created, BugSubscription.id, | 01:40 |
| lifeless | is what I'm trying to make my test trigger | 01:40 |
| lifeless | but I'm having trouble making it do it | 01:40 |
| lifeless | possibly due to caching :) | 01:41 |
| mwhudson | lifeless: also, another trick | 01:41 |
| mwhudson | lifeless: do you know about LP_DEBUG_SQL_EXTRA ? | 01:41 |
| lifeless | do | 01:41 |
| lifeless | what doth that do? | 01:41 |
| mwhudson | make run LP_DEBUG_SQL=1 | 01:41 |
| thumper | gives a metric shit-load of output | 01:41 |
| mwhudson | prints all queries as the output | 01:41 |
| thumper | extra gives a stacktrace too | 01:41 |
| mwhudson | LP_DEBUG_SQL_EXTRA gives tracebacks as well as the sql | 01:41 |
| lifeless | ok | 01:42 |
| thumper | very very handy | 01:42 |
| mwhudson | as thumper says,. it's mounds of output | 01:42 |
| thumper | in optimisation work | 01:42 |
| mwhudson | but can be handy to figure out where that query is coming from | 01:42 |
| thumper | I had 70k lines of output for the branch index page | 01:42 |
| lifeless | I should capture that for these performance tests | 01:42 |
| lifeless | toss it into kcachegrind | 01:42 |
| thumper | lifeless: https://code.edge.launchpad.net/~thumper/launchpad/fix-TestSharingMigrationPerformance/+merge/32828 | 01:43 |
| thumper | lifeless: if you make the mark on that, I'll land the fix | 01:43 |
| thumper | lifeless: and that was pretty much the only thing blocking bzr 2.2 landing | 01:43 |
| === Ursinha is now known as Ursinha-afk | ||
| === Ursinha-afk is now known as Ursinha | ||
| lifeless | mwhudson: so the problem with reproducing the query is that I don't know how to do it *at all*, not just in-a-test, other than 'visit this page that oopses' :) | 01:49 |
| mwhudson | lifeless: challenging | 01:50 |
| lifeless | even sprinking store.invalidate()'s doesn't trigger it | 01:53 |
| lifeless | ah well, so I'll fix the problem I *have* demonstrated. | 01:53 |
| lifeless | it would be nice to be able to say | 01:56 |
| lifeless | 'now make run should give me this test environment' | 01:56 |
| lifeless | or something | 01:56 |
| mwhudson | barry had some terrible hacks for that iirc | 01:57 |
| lifeless | sync_to_demo() | 01:58 |
| lifeless | ? | 01:58 |
| lifeless | do windmill tests hate everyone, or just me ? | 02:23 |
| StevenK | lifeless: It's everyone, and life itself | 02:26 |
| lifeless | does this mean anything to anyone ? | 02:31 |
| lifeless | ====================================================================== | 02:31 |
| lifeless | ERROR: lp.registry.windmill.tests.test_person_picker.TesPersonPickerWidget.test_person_picker_widget | 02:31 |
| lifeless | ---------------------------------------------------------------------- | 02:31 |
| lifeless | _StringException: Text attachment: garbage | 02:31 |
| lifeless | ------------ | 02:31 |
| lifeless | [<Thread(Thread-657, started daemon 47266133427984)>] | 02:31 |
| lifeless | ------------ | 02:31 |
| wgrant | I see that most times I run that. | 02:33 |
| wgrant | So I just sort of ignore it. | 02:33 |
| wgrant | While giving it a look of extreme disapproval. | 02:33 |
| lifeless | blocking landing is asad | 02:33 |
| wgrant | Oh, it did that in EC2? | 02:34 |
| lifeless | yes | 02:34 |
| wgrant | :/ | 02:34 |
| lifeless | yes | 02:34 |
| lifeless | >< | 02:37 |
| lifeless | my registry /participants fix works here | 02:37 |
| lifeless | but there is an extra query on prod | 02:37 |
| lifeless | sorry | 02:37 |
| lifeless | ec2 | 02:37 |
| mwhudson | lifeless: for reasons i have (shamefully) never really dug into, i find that when i have a genuine failure in ec2, i get a windmill failure too | 02:38 |
| mwhudson | when i fix the genuine failure, the branch lands ok | 02:38 |
| mwhudson | maybe this is just luck or maybe something else | 02:39 |
| lifeless | sigh | 02:39 |
| lifeless | a new SELECT Person.account, Person.creation_comment, Person.creation_rationale, Person.datecreated, Person.defaultmembershipperiod, Person.defaultrenewalperiod, Person.displayname, Person.hide_email_addresses, Person.homepage_content, Person.icon, Person.logo, Person.mailing_list_auto_subscribe_policy, Person.merged, Person.mugshot, Person.name, Person.personal_standing, Person.personal_standing_reason, Person.registrant, Person | 02:39 |
| lifeless | mwhudson: interesting | 02:39 |
| lifeless | I've sent cachedproperty off on its own with the registry branch fix | 02:40 |
| lifeless | while I try to figure out wtf I see an extra query even with devel merged in | 02:40 |
| lifeless | maybe something landed in the last few hours | 02:40 |
| lifeless | oh wow | 02:57 |
| lifeless | we trigger a DB lookup on every person lookup via this code path | 02:57 |
| lifeless | shudder | 02:57 |
| lifeless | yay storm bugs | 03:08 |
| lifeless | object cache coherency, can you imagine it? | 03:08 |
| mwhudson | it sounds fairly unlikely :-) | 03:12 |
| lifeless | har har har | 03:15 |
| lifeless | thats twice in 1 month | 03:15 |
| lifeless | https://bugs.edge.launchpad.net/storm/+bug/619017 | 03:15 |
| lifeless | I'm not trusting the layer much now, as a result | 03:15 |
| lifeless | I'd rather storm didn't have an object cache. | 03:16 |
| lifeless | it would make it much simpler | 03:16 |
| lifeless | and more focused | 03:16 |
| lifeless | mwhudson: or perhaps you were being sardonic ? | 03:19 |
| mwhudson | lifeless: a touch, yes | 03:20 |
| lifeless | ha! | 03:21 |
| thumper | poolie: ping | 03:34 |
| thumper | mwhudson: know the simplest solution to this? http://pastebin.ubuntu.com/479196/ | 03:35 |
| mwhudson | thumper: :( | 03:36 |
| thumper | mwhudson: could trim the input on _show_lline | 03:36 |
| thumper | mwhudson: but we'd still have the | | 03:36 |
| mwhudson | thumper: let me try to remember what the test is actually testing | 03:37 |
| thumper | mwhudson: most of the TestLoggingUIFactory tests fail | 03:37 |
| thumper | with similar errors | 03:37 |
| thumper | I'm guessing the base class was updated in 2.2 | 03:37 |
| mwhudson | thumper: yeah, looks like progress reporting changes in bzr | 03:38 |
| mwhudson | thumper: possibly needs a code fix, as that's going to make the code import logs look rather lame i suspect | 03:38 |
| thumper | :-| | 03:38 |
| thumper | I wonder if the _render_bar was renamed | 03:40 |
| mwhudson | i'm just updating my bzr branch | 03:40 |
| mwhudson | (i think it's fairly ancient) | 03:40 |
| * thumper runs away again | 03:41 | |
| mwhudson | thumper: got a branch i can run tests in? | 03:44 |
| mwhudson | I wonder if overriding _avail_width to always return None in LoggingTextProgressView will help | 03:45 |
| lifeless | poolie: | 03:45 |
| lifeless | ^ | 03:45 |
| mwhudson | alternatively def _render_line(self): return self._render_bar() will probably do it | 03:49 |
| lifeless | mwhudson: so | 03:55 |
| lifeless | File "/home/robertc/launchpad/lp-branches/working/lib/canonical/launchpad/webapp/authorization.py", line 207, in checkPermission | 03:55 |
| lifeless | principal.account) | 03:55 |
| lifeless | is that the code path you're talking about w.r.t. checking permissions doing db access ? | 03:56 |
| mwhudson | probably | 03:56 |
| mwhudson | oh | 03:56 |
| mwhudson | is the next call down in canonical.launchpad.security? | 03:56 |
| mwhudson | lifeless: ^ | 03:57 |
| lifeless | checkAccountAuthenticated | 03:58 |
| lifeless | yes | 03:58 |
| mwhudson | then yes | 03:58 |
| lifeless | so the *menu* is generating this | 03:58 |
| lifeless | :/ | 03:58 |
| mwhudson | enabled_with_permission ? | 03:58 |
| mwhudson | what's the context object being checked? | 03:58 |
| lifeless | dunno | 03:59 |
| lifeless | but probably milestone/product | 03:59 |
| mwhudson | mm | 03:59 |
| mwhudson | lifeless: can you pastebin the whole traceback? | 04:00 |
| lifeless | sure | 04:00 |
| lifeless | http://pastebin.com/EkXrtUK6 | 04:01 |
| thumper | mwhudson: yeah, I do | 04:03 |
| mwhudson | lifeless: wee, it's checking if the user can review the page | 04:04 |
| mwhudson | er | 04:04 |
| mwhudson | lifeless: wee, it's checking if the user can review the product | 04:04 |
| thumper | mwhudson: lp:~thumper/launchpad/new-bzr | 04:04 |
| mwhudson | ReviewByRegistryExpertsOrAdmins | 04:04 |
| lifeless | mwhudson: how did you figure that out ? | 04:04 |
| thumper | although I'm about to run out yet again | 04:04 |
| mwhudson | lifeless: # | 04:05 |
| mwhudson | File "/home/robertc/launchpad/lp-branches/working/lib/canonical/launchpad/security.py", line 232, in checkAuthenticated | 04:05 |
| mwhudson | # | 04:05 |
| mwhudson | return user.in_admin or user.in_registry_experts | 04:05 |
| mwhudson | -> look at line 232 of security.py | 04:05 |
| mwhudson | hi tech haxorring! | 04:05 |
| lifeless | oh | 04:05 |
| lifeless | I figured that would be generic. serves me right | 04:05 |
| mwhudson | it's crummy, but it's a once per page check | 04:05 |
| thumper | mwhudson: found it | 04:06 |
| mwhudson | thumper: oh? | 04:06 |
| thumper | mwhudson: we set _render_bar to return '' | 04:06 |
| thumper | mwhudson: the base _render_line method does bar_string = self._render_bar() first | 04:06 |
| thumper | then | 04:06 |
| thumper | if (task_part or trans) and not bar_string: | 04:07 |
| thumper | bar_string = '| ' | 04:07 |
| thumper | further down | 04:07 |
| lifeless | mwhudson: then why do I see two of them ? :) | 04:07 |
| mwhudson | aah | 04:07 |
| thumper | :( | 04:07 |
| mwhudson | lifeless: both sides of the or execute a query i guess | 04:07 |
| * lifeless facepalms | 04:07 | |
| mwhudson | lifeless: i guess i should have said "fixed number of queries per page" | 04:07 |
| mwhudson | lifeless: ORMs r grate | 04:07 |
| mwhudson | thumper: i guess overriding _render_line() is one fix | 04:08 |
| thumper | mwhudson: yeah | 04:09 |
| thumper | I'll look at it again when I get back | 04:10 |
| lifeless | mwhudson: want to have a rotation in 6 months or so, to work on a separated mapper approach ? :) | 04:11 |
| mwhudson | lifeless: heh | 04:12 |
| mwhudson | lifeless: we should rewrite launchpad in haskell and use meta-programming to compute the set of needed queries before we actually start executing code for the page!! | 04:13 |
| lifeless | mwhudson: funny you should say that | 04:13 |
| lifeless | we do need something along those lines (the needed queries, not haskell) | 04:14 |
| mwhudson | lifeless: given the pypy background, perhaps not so much | 04:14 |
| lifeless | mwhudson: :) | 04:14 |
| mwhudson | it some sense, it would be fairly easy to do the page rendering twice, once to collect the queries and then again for real | 04:16 |
| mwhudson | you could even do the first bit ahead of time | 04:16 |
| mwhudson | the fun part is conditionals, of course | 04:16 |
| ajmitch | fsvo fun? | 04:17 |
| mwhudson | you _could_ abstractly interpret the code repeatedly, taking a different path and giving a different answer for each branch | 04:18 |
| mwhudson | pypy does exactly this | 04:18 |
| lifeless | see | 04:19 |
| lifeless | while this sounds extremely promising | 04:19 |
| lifeless | I'm actually ok with just putting a cap on the queries | 04:19 |
| mwhudson | :) | 04:19 |
| lifeless | and failing tests if the cap is exceeded | 04:19 |
| poolie | hi thumper, mwhudson | 04:30 |
| poolie | mwhudson: all ok now? | 04:32 |
| lifeless | how can I tell if an Interface is exported via APIS ? | 04:33 |
| james_w | if it has export* decorators on it | 04:34 |
| lifeless | ok | 04:35 |
| lifeless | so BugTaskSearchParams isn't exported | 04:35 |
| lifeless | \o/ I can mess with it | 04:35 |
| mwhudson | poolie: i think so, although it's really thumper who's worrying about this | 04:43 |
| lifeless | can anyone explain what a 'non conjoined task' is ? | 05:00 |
| wgrant | lifeless: A task that isn't conjoined. Do you know what a conjoined task is? | 05:05 |
| lifeless | no | 05:09 |
| StevenK | lifeless: When you're done with wgrant, do you have time to mumble? | 05:09 |
| lifeless | sure | 05:09 |
| lifeless | I think I've *finally* found where this private check is kicking in | 05:09 |
| lifeless | StevenK: but skype please | 05:10 |
| lifeless | it works | 05:10 |
| wgrant | lifeless: If you have a task for a project and its development series, the project task is hidden, and becomes a conjoined slave to the development series task. | 05:10 |
| wgrant | (that's then you see "Status tracked in SomeSeries") | 05:10 |
| lifeless | ok | 05:10 |
| lifeless | StevenK: rbtcollins | 05:10 |
| wgrant | Setting the status on the series task sets it on the project task too. | 05:10 |
| * StevenK grumbles how lifeless hates freedom, and hides | 05:10 | |
| lifeless | StevenK: do you have skype? | 05:11 |
| StevenK | lifeless: Yes | 05:12 |
| lifeless | and do you want to talk | 05:12 |
| StevenK | lifeless: Yes -- and I cope with mumble's lag every stand-up, so ... :-) | 05:12 |
| StevenK | lifeless: So I'm only teasing | 05:12 |
| lifeless | arch_tag in [%s] | 05:22 |
| lifeless | where the %s is | 05:22 |
| lifeless | sqlvalues(tags) | 05:22 |
| lifeless | or something | 05:22 |
| lifeless | you need , | 05:22 |
| lifeless | 's | 05:23 |
| lifeless | so I guess ",".join(sqlvalues(*arch_tags)) | 05:23 |
| StevenK | include = "architecturetag IN (%s)" % sqlvalues( | 05:23 |
| StevenK | ','.join(self.arches)) | 05:23 |
| lifeless | only | 05:23 |
| lifeless | include = "AND whatyou haveabove | 05:23 |
| wgrant | sqlvalues(list(self.arches))? | 05:24 |
| lifeless | wgrant: I guess | 05:25 |
| lifeless | wgrant: but why the list ? | 05:25 |
| StevenK | wgrant: But I can't just blat that into a string with %s? | 05:25 |
| poolie | lifeless: so i am thinking of just using a textarea for editing the rules | 05:25 |
| lifeless | its a tuple already | 05:25 |
| poolie | at least to start with | 05:25 |
| lifeless | poolie: +1 | 05:25 |
| poolie | since it's a very nerd-oriented thing | 05:25 |
| wgrant | lifeless: Ah, not already a resultset? | 05:26 |
| wgrant | s/already/just/ | 05:26 |
| StevenK | wgrant: Damn it, it's my code. :-) It's a tuple | 05:26 |
| thumper | poolie: hi | 05:26 |
| wgrant | Just sqlvalues(self.arches) should work, then. | 05:26 |
| lifeless | wgrant: does that return a iterable of strings ? | 05:26 |
| poolie | hi thumper | 05:26 |
| thumper | poolie: I was discussing the issue we have with our code import bzr logger | 05:26 |
| thumper | poolie: and that things changed | 05:26 |
| thumper | poolie: http://pastebin.ubuntu.com/479196/ | 05:26 |
| thumper | poolie: is an example of the failure | 05:27 |
| thumper | poolie: just working out what to change in our logger | 05:27 |
| poolie | huh | 05:27 |
| thumper | was thinking overriding _render_line | 05:27 |
| poolie | i would guess this is actually a uifactory problem | 05:27 |
| poolie | and that these tests want to test TestTextUI | 05:27 |
| poolie | but they don't explicitly set it, they just assume it's the default | 05:27 |
| poolie | which it may not be in your context | 05:27 |
| wgrant | >>> 'foo IN %s' % sqlvalues([1, 2, 3]) | 05:28 |
| wgrant | 'foo IN (1, 2, 3)' | 05:28 |
| wgrant | lifeless, StevenK: ^^ | 05:28 |
| thumper | poolie: I'm actually guessing part of the issue is line 392 of bzrlib.ui.text.py | 05:28 |
| thumper | poolie: as our logger has _render_bar defined to return '' | 05:29 |
| thumper | poolie: FYI, this is the lp.codehosting.codeimport.uifactory.LoggingTextProgressView | 05:29 |
| poolie | oh this is your test, not mine | 05:30 |
| thumper | poolie: yes | 05:30 |
| StevenK | LINE 5: FROM DistroArchSeries WHERE distroseries = 3 '' | 05:30 |
| poolie | thumper: i guess what's broken it is that _render_line now does the whitespace filling, not repaint | 05:32 |
| poolie | or something like that | 05:32 |
| thumper | actually, I'm thinking line 391 should be: if (task_part and trans) and not bar_string: | 05:32 |
| thumper | rather than if (task_part or trans) and not bar_string: | 05:32 |
| thumper | you only need the bar if there are both task_part and trans to separate them | 05:33 |
| thumper | perhaps? | 05:33 |
| thumper | oh, and the whitespace filling | 05:33 |
| poolie | thumper: you seem to have two problems, one being the whitespace padding and the other is the bar | 05:33 |
| thumper | yep | 05:33 |
| thumper | poolie: I'm thinking it is easier just to override _render_line | 05:33 |
| poolie | i agree about the 'and' in that line | 05:34 |
| thumper | ok | 05:34 |
| thumper | I could just change the _avail_width to return None | 05:34 |
| thumper | but we'd still have the bar issue | 05:35 |
| poolie | but if you want complete control you should probably just override _render_line | 05:35 |
| poolie | to me that is most consistent with the intention | 05:35 |
| * thumper nods | 05:35 | |
| thumper | ack | 05:35 |
| lifeless | storm/database.py +236 | 05:38 |
| thumper | lifeless: whatcha doin? | 05:38 |
| lifeless | talking about doing INSERT INTO in storm | 05:40 |
| lifeless | compile(expr, State() | 05:44 |
| lifeless | ) | 05:45 |
| lifeless | -> sql | 05:45 |
| lifeless | expr - Select(table, And(x, y)) | 05:45 |
| lifeless | storm.expr.compile_insert | 05:46 |
| lifeless | StevenK: so I'm debugging https://lp-oops.canonical.com/oops.py/?oopsid=OOPS-1689EB3170 | 06:00 |
| lifeless | StevenK: do you have lpadmin account access in the dogfood environment ? | 06:01 |
| lifeless | (I need to be put in a group there, to trigger this) | 06:01 |
| lifeless | if you don't, don't worry, I'll get mthaddon later on staging. | 06:01 |
| StevenK | lifeless: Oh, you need a duck on dogfood? | 06:02 |
| lifeless | I need to be in the landscape team | 06:03 |
| lifeless | I am on prod, now | 06:03 |
| lifeless | how that is arranged may need a duck | 06:03 |
| wgrant | Um, dogfood is really not a good place to test timeouts. | 06:03 |
| lifeless | wgrant: I don't want to test the timeout | 06:03 |
| StevenK | lifeless: And yes, I can ... force you into landscape | 06:04 |
| lifeless | wgrant: I want to get a definitive backtrace of the cause of the query | 06:04 |
| lifeless | StevenK: hold off for a bit | 06:04 |
| lifeless | like I said just before our call, I think I'm onto the cause. | 06:04 |
| lifeless | bug.py 1561 looks like a clear culprit | 06:07 |
| lifeless | though I still need to figure out WTF I can't trigger this in a test | 06:09 |
| lifeless | possibly a non-proxied bug object ? surely not. | 06:09 |
| lifeless | I'm using userBrowser | 06:09 |
| poolie | lifeless: you should try 'ssh launchpad-vm ./bin/test --subunit|tribunal-subunit -' | 06:12 |
| lifeless | poolie: :) | 06:15 |
| lifeless | ok, so userBrowser goes through publication | 06:16 |
| lifeless | wtf | 06:16 |
| lifeless | wtf | 06:16 |
| lifeless | wtf | 06:16 |
| wgrant | Why wouldn't it? | 06:16 |
| lifeless | its good that it does | 06:17 |
| lifeless | wtf am I not seeing this security code popping up I mean | 06:17 |
| lifeless | \o/ cachedproperty branch passed ec2 | 06:22 |
| poolie | can someone point me to a good example of non-doctest pagetesting? | 06:26 |
| lifeless | I pasted an ok example in last weeks perf tuesday thread | 06:28 |
| poolie | ah test_archive_packages | 06:30 |
| poolie | ok | 06:30 |
| lifeless | they seem to generally live in browser/tests/* | 06:30 |
| poolie | yep i found some more | 06:31 |
| poolie | obvious really | 06:31 |
| poolie | the ones that do exist seems pretty clean | 06:31 |
| lifeless | man, I hate debugging by printf | 06:43 |
| poolie | has anyone ever tried running pgsql on a tmpfs, or with libeatmydata? | 06:45 |
| lifeless | yes | 06:46 |
| lifeless | not a huge difference | 06:46 |
| mwhudson | it helps if you have lots of ram and a slow hd | 06:46 |
| poolie | where was the build pydoctor stuff? | 06:52 |
| mwhudson | poolie: http://people.canonical.com/~mwh/canonicalapi/ | 06:52 |
| mwhudson | but there's a good chance apidoc.launchpad.dev is more useful these days | 06:53 |
| wgrant | StevenK: Shouldn't your packageset thing be copying memberships too? | 07:07 |
| wgrant | At the moment it's just copying empty sets. | 07:07 |
| StevenK | Empty sets? | 07:08 |
| StevenK | wgrant: I'm about to head out for an hour or so, can you dump what you think I'm missing and how the tests could be extended in a PM? | 07:09 |
| wgrant | StevenK: I suspect that the new package sets will have nothing in them. | 07:09 |
| StevenK | That can be tested trivially | 07:09 |
| wgrant | It can be. | 07:10 |
| wgrant | It needs to copy packageset<->packageset and packageset<->package relationships. | 07:10 |
| lifeless | \o/ reproduced the query in the test | 07:11 |
| lifeless | that makes me so much happier | 07:17 |
| lifeless | I thought I was going nuts | 07:17 |
| poolie | yay, a pagetest passes | 07:17 |
| lifeless | \o/ | 07:19 |
| poolie | ok, might take a berak | 07:21 |
| poolie | tests for merge of the existing flags code into devel are still running | 07:21 |
| lifeless | time for a break for me too, that was a marathon head beating session to find out that userBrowser 'works' with the wrong password | 07:22 |
| poolie | oh, nice | 07:25 |
| poolie | btw if i want to have /+features | 07:26 |
| poolie | is it reasonable to say that's a view of ILaunchpadRoot? | 07:26 |
| poolie | that doesn't seem quite right but it does seem stadnard | 07:26 |
| lifeless | yes | 08:01 |
| === almaisan-away is now known as al-maisan | ||
| StevenK | wgrant: Still here? | 08:17 |
| wgrant | StevenK: Indeed. | 08:20 |
| StevenK | wgrant: Just looking at the branch now | 08:24 |
| lifeless | wgrant: btw, registry branch has landed | 08:29 |
| lifeless | wgrant: your worst gears may come true next time edge updates | 08:29 |
| StevenK | Worst gears? | 08:31 |
| StevenK | First, second, third, fourth or fifth? | 08:31 |
| StevenK | Damn it, wgrant is right | 08:32 |
| lifeless | aboot? | 08:32 |
| StevenK | Alpha? | 08:32 |
| wgrant | Yay. | 08:33 |
| lifeless | StevenK: about | 08:33 |
| StevenK | a = [] | 08:33 |
| StevenK | b = [u'pmount'] | 08:33 |
| StevenK | wgrant: ^ | 08:33 |
| lifeless | what causes a security proxy to be wrapped around something | 08:44 |
| lifeless | securedutility... ah layers | 08:44 |
| lifeless | ...nope | 08:45 |
| lifeless | databasefunctionallayer should have sec proxies around utilities, no? | 08:46 |
| wgrant | It should, yes. | 08:47 |
| lifeless | is there an idiom for checking that? | 08:47 |
| lifeless | and it does | 08:48 |
| lifeless | hmm | 08:48 |
| adeuring | good morning | 08:51 |
| StevenK | wgrant: So, which table includes the package names for packagesets? | 08:58 |
| lifeless | guess how many queries this takes: | 09:07 |
| lifeless | (in the assert call) | 09:07 |
| lifeless | target = self.makeBugTarget() | 09:07 |
| lifeless | person = self.login() | 09:07 |
| lifeless | self.factory.makeBug(product=target, private=True, owner=person) | 09:07 |
| lifeless | self.factory.makeBug(product=target, private=True, owner=person) | 09:07 |
| lifeless | login_person(person) | 09:07 |
| lifeless | tasks =target.searchTasks(BugTaskSearchParams(person, omit_dupes=True, | 09:08 |
| lifeless | orderby=['status', '-importance', 'id'])) | 09:08 |
| lifeless | self.assertStatementCount(0, lambda:[task.getConjoinedMaster for task | 09:08 |
| lifeless | in tasks]) | 09:08 |
| lifeless | no takers? | 09:09 |
| StevenK | lifeless: 100? | 09:09 |
| lifeless | 5 | 09:09 |
| lifeless | each security check is 2 queries | 09:11 |
| lifeless | sanity check: if we'v edone a search limited by a users view, saving that user in the results as someone we know can see them is sane? | 09:12 |
| mrevell | Yo | 09:16 |
| bigjools | hey wgrant | 09:23 |
| lifeless | bigjools: btw, b-m had a hiccup midday, wgrant has the details | 09:24 |
| bigjools | lifeless: I read the bug | 09:24 |
| bigjools | lifeless: thankfully it's nothing to do with the recent changes | 09:25 |
| lifeless | phew | 09:31 |
| lifeless | ok, down to 2 queries | 09:43 |
| lifeless | lets see if I can go for 1 | 09:43 |
| lifeless | \o/ | 09:57 |
| lifeless | milestone pages should be happier soon :) | 09:57 |
| bigjools | woot | 10:01 |
| lifeless | it may also help a bunch of other private bug related perf issues | 10:02 |
| lifeless | due to cachedproperty. | 10:02 |
| lifeless | whats the preferred way to spell 'adapt this' | 10:07 |
| lifeless | e.g. | 10:07 |
| lifeless | if I have a person_or_personroles | 10:07 |
| lifeless | is it | 10:07 |
| lifeless | person = IPerson(person_or_personroles) ? | 10:07 |
| thumper | yes | 10:08 |
| lifeless | hmm, I think I'll add 'id' to IPersonRoles | 10:11 |
| lifeless | jml: like you, I can't write code without tests | 10:13 |
| lifeless | jml: but its ok, its a good place to be | 10:13 |
| lifeless | now I have a weird situation | 10:13 |
| lifeless | the second page load with more bugs takes less queries :) | 10:14 |
| wgrant | StevenK: Sorry, was travelling. | 10:14 |
| wgrant | StevenK: PackageSetInclusion and PackageSetSource, IIRC. | 10:14 |
| wgrant | There's also PackageSetGroup, but I don't know if that's used for anything. | 10:14 |
| wgrant | bigjools: It isn't? | 10:14 |
| wgrant | bigjools: It looks like it's meant to be asynchronous... | 10:14 |
| wgrant | And given that the whole Deferred management thing changed last week... | 10:15 |
| bigjools | wgrant: nothing changed in what is Deferred | 10:16 |
| bigjools | it just interleaves them more | 10:16 |
| thumper | \o/ | 10:30 |
| thumper | bzr 2.2 upgrade in the pipe | 10:30 |
| lifeless | \o/ | 10:31 |
| lifeless | lp.registry.browser.tests.test_milestone.TestMilestoneIndex.test_more_private_bugs_query_count_is_constant | 10:31 |
| lifeless | Ran 1 tests with 0 failures and 0 errors in 2.864 seconds. | 10:31 |
| lifeless | ignoring the pathetic sped | 10:32 |
| lifeless | -it works- muahahhaaha | 10:32 |
| jml | lifeless, \o/ | 10:32 |
| lifeless | win 68 | 10:33 |
| lifeless | so | 10:33 |
| lifeless | I'm going to push | 10:33 |
| lifeless | propose for merge | 10:33 |
| lifeless | and foff to relax | 10:33 |
| lifeless | @930pm | 10:33 |
| lifeless | oh yay, apt-get update ruined ca-cert | 10:39 |
| * bigjools switches conversation to -dev | 10:46 | |
| bigjools | wgrant: I plan on converting the rest of the sync stuff to async at some point anyway | 10:46 |
| wgrant | Yeah, it needs it. | 10:46 |
| jml | good plan :) | 10:46 |
| wgrant | Although the next significant step is to remove the p-u invocations. | 10:47 |
| * bigjools chuckles at jml | 10:47 | |
| wgrant | That should just about get it to an acceptable speed. | 10:47 |
| bigjools | wgrant: jelmer has a branch that does that, it's on dogfood :) | 10:47 |
| wgrant | Does it dump it into a directory with a daemon watching it? | 10:47 |
| bigjools | cronjob for now, but yeah | 10:47 |
| wgrant | Ah. | 10:47 |
| wgrant | How does that interact with the build state? It sits FULLYBUILT without binaries for up to a minute? | 10:48 |
| bigjools | something like that yeah, but it's still quicker than what we have right now | 10:48 |
| wgrant | Probably. | 10:49 |
| bigjools | definitely :) | 10:56 |
| bigjools | wgrant: any comments on https://bugs.edge.launchpad.net/soyuz/+bug/619088 | 10:57 |
| wgrant | bigjools: NMAF does it. I don't know why a-f wouldn't. | 11:01 |
| bigjools | it's a-f .... all bets are off | 11:01 |
| wgrant | True. | 11:01 |
| wgrant | Even Debian's trying to kill it now. | 11:02 |
| lifeless | ok, have a good performance tuesday | 11:11 |
| lifeless | bigjools: I'll pull the performance tuesday topic-element off tomorrow morning - I'd like it to be up there for a full 24 if thats ok:) | 11:11 |
| bigjools | lifeless: yeah, did I remove it last week? | 11:12 |
| lifeless | bigjools: yeah :) | 11:13 |
| bigjools | lifeless: oops. sorry :) | 11:13 |
| lifeless | bigjools: de nada | 11:13 |
| poolie | is there an official way to make the right pqm command to land a branch i've already tested locally | 11:15 |
| poolie | i guess lp-land? | 11:15 |
| bigjools | poolie: I do it by hand | 11:15 |
| bigjools | old skool :) | 11:15 |
| poolie | echo blah|gpg|mail? | 11:15 |
| bigjools | you need the pqm plugin for bzr | 11:15 |
| bigjools | then I do "bzr pqm-submit -m ...." | 11:16 |
| noodles775 | lp-land should work. | 11:16 |
| bigjools | you can use your approach if you can remember the pqm commands I guess | 11:16 |
| noodles775 | bigjools: you don't need to, lp-land gets them from the MP. | 11:17 |
| bigjools | even betterer, I've not used it | 11:17 |
| poolie | OAuth needs a "yeah, yeah" button | 11:17 |
| noodles775 | heh | 11:17 |
| jml | I have come up with the perfect solution for this ravenous hunger: Food! | 11:18 |
| poolie | you can have a lamb, leek and mint stew | 11:18 |
| poolie | if you can get over here in time | 11:18 |
| poolie | urk | 11:19 |
| poolie | File "/usr/lib/python2.6/dist-packages/lazr/restfulclient/resource.py", line 308, in __getattr__ | 11:19 |
| poolie | % (self.__class__.__name__, attr)) | 11:19 |
| poolie | AttributeError: 'Entry' object has no attribute 'source_branch' | 11:19 |
| poolie | i know this indicates some internal error but i can't remember what | 11:19 |
| lifeless | it may mean Entry isn't an IMergeProposal | 11:21 |
| poolie | i've seen that happen when you call into lplib again after previously getting an exception | 11:21 |
| lifeless | poolie: red dead redemption is kinda fun | 11:21 |
| poolie | yes it is | 11:21 |
| poolie | i guess a cache is corrupt | 11:21 |
| jml | umm | 11:25 |
| jml | poolie, actually, it's because you are giving it a branch URL not a mp URL | 11:26 |
| jml | I think | 11:26 |
| lifeless | thumper: btw | 11:26 |
| lifeless | thumper: CodeImportSchedulerApplication:CodeImportSchedulerAPI - https://devpad.canonical.com/~stub/ppr/lpnet/latest-daily-timeout-candidates.html - click on sort-by-total-queries | 11:26 |
| lifeless | thumper: thats doing a _lot_ of db work | 11:26 |
| lifeless | only three per hit on avg, but boy - a huge number of hits. Or something like that | 11:27 |
| lifeless | wgrant: Distribution:+archivemirrors | 11:27 |
| lifeless | wgrant: if you're going to look at perf - that looks to be a classic death-by-sql | 11:28 |
| lifeless | 94 hits | 11:28 |
| lifeless | 69715 queries | 11:28 |
| poolie | hm i got a "failed to verify gpg signature" | 11:29 |
| poolie | i might not actually be allowed to submit? | 11:29 |
| lifeless | hah, thats possible | 11:29 |
| lifeless | losa should be able to fix that fr you | 11:29 |
| poolie | ok | 11:29 |
| wgrant | lifeless: Is that a typo? | 11:29 |
| lifeless | wgrant: is what a typo? | 11:30 |
| wgrant | lifeless: 69715 | 11:30 |
| lifeless | no | 11:30 |
| wgrant | queries. | 11:30 |
| wgrant | That is insane. | 11:30 |
| lifeless | yes | 11:30 |
| lifeless | 700 per hit | 11:30 |
| lifeless | and you wonder why its timing out :) | 11:30 |
| poolie | lifeless: i'll ask tomorrow; could you send https://code.edge.launchpad.net/~mbp/launchpad/flags-webapp/+merge/32833 for me now to reduce latency? | 11:30 |
| wgrant | Oh. | 11:30 |
| lifeless | wgrant: its still about 100 times too many queries | 11:30 |
| poolie | i may put up the edit ui tomorrow, depending | 11:30 |
| wgrant | lifeless: I wonder how much we can cut the base query count. | 11:31 |
| lifeless | poolie: please specify a commit message | 11:31 |
| lifeless | poolie: sounds great on the edit ui | 11:32 |
| poolie | done | 11:32 |
| lifeless | wgrant: I think you could get it to 8 | 11:32 |
| lifeless | wgrant: 4 for oauth & other auth cruft, 3 for cookies and session | 11:32 |
| lifeless | 1 for the content. | 11:32 |
| jpds | lifeless/wgrant: Not timing out here; you two should move to LDN. | 11:32 |
| lifeless | jpds: whttps://devpad.canonical.com/~stub/ppr/lpnet/latest-daily-timeout-candidates.html | 11:32 |
| lifeless | 1% of hits are timing out | 11:33 |
| lifeless | 40% are over 10 seconds long | 11:33 |
| jpds | Yeah; the freshness stuff needs fixing. | 11:33 |
| lifeless | jpds: its doing *an average* of 700 queries per page load :) | 11:35 |
| lifeless | jpds: thats very much 'needs fixing' :) | 11:35 |
| lifeless | poolie: ok I think i've told it the magic nuts n bolts to make it do stuff | 11:43 |
| lifeless | gnight | 11:50 |
| lifeless | tag, somebody is it; make things better | 11:50 |
| deryck | Morning, all. | 12:01 |
| jml | deryck, good morning | 12:01 |
| stub | Do people look at the old patch-???.sql files for examples, or can I just trash them when they are no longer needed? Its about 85 files. | 12:05 |
| wgrant | I use them as examples sometimes, but only because I'm lazy. | 12:07 |
| wgrant | For everything else, there's VCS history. | 12:07 |
| stub | Just wondering if people want them left around for laziness, or trashed for performance. | 12:14 |
| StevenK | wgrant: I can't see anything in packagesetinclusion, but I'm about to did through .addSource() code | 12:14 |
| wgrant | StevenK: I'm pretty sure it's used for packageset inheritance. | 12:16 |
| StevenK | I was afraid of that | 12:16 |
| StevenK | I'm already having to write one horrible SQL transform, what's two more :-( | 12:16 |
| wgrant | StevenK: What about packagesetgroup? | 12:17 |
| StevenK | wgrant: Although, the code for .addSource() only touches packagesetsources, so perhaps packagesetinclusion is done by trigger? | 12:17 |
| StevenK | wgrant: I don't touch it at all, currently | 12:18 |
| wgrant | StevenK: packagesetinclusion is for packageset inheritance. It doesn't touch sources. | 12:18 |
| StevenK | Sigh. | 12:19 |
| jml | stub, I vote trashing for performance | 12:19 |
| wgrant | And packagesetgroup seems to be to group them across series? Not entirely sure. | 12:19 |
| StevenK | wgrant: I'm not either | 12:19 |
| jtv | danilos, henninge: about setCurrentTranslation, submitSuggestion & approve… ISTM we'll be calling setCurrentTranslation only from tests and from the new approval method. (Doing it this way even for imports would probably simplify the conflict checks as well). If that is right, that would mean we could handle things like karma in submitSuggestion and approveSuggestion, completely outside of setCurrentTranslation. | 12:33 |
| jtv | Oh, and from other scripts perhaps where we most likely don't _want_ karma involved. | 12:34 |
| === mrevell is now known as mrevell-lunch | ||
| === matsubara-afk is now known as matsubara | ||
| jml | hmm. | 13:50 |
| jml | why doesn't Mismatch just take some descriptions and details? | 13:50 |
| mars | jml, to reduce coupling between Matcher and Mismatch? | 13:57 |
| jml | they are pretty tightly coupled anyway | 13:58 |
| mars | well, off the top of my head, lets say you had two matchers | 13:58 |
| mars | they each create an instance of the same mismatch class | 13:59 |
| mars | if you pass in the error message or change the detail handling, then both matchers are affected | 13:59 |
| mars | under the current design only changes to the mismatch constructor API will cause callers to change | 14:00 |
| jml | oh, sorry, I wasn't being clear. | 14:02 |
| jml | I'm not proposing removing describe() or get_details(), just changing the base Mismatch to take optional parameters for description and details so every single Matcher doesn't need to make its own Mismatch subclass. | 14:02 |
| jml | (alternatively, make a SimpleMismatch that does that) | 14:03 |
| mars | ah, I was thinking of a similar idea, but using a factory function that returns some Mismatch template class instance | 14:03 |
| mars | same diff | 14:03 |
| mars | so yes, something like that would be nice for convenience | 14:04 |
| mars | jml, looking through testtools.matchers - it might be convenient, but there would be a real temptation to eliminate all the mismatch classes, or for developers to not create new ones. It would be very easy (too easy?) to just pull the mismatch-specific code (often one line) up into the matcher. | 14:09 |
| === fjlacoste is now known as flacoste | ||
| === mrevell-lunch is now known as mrevell | ||
| jml | mars, you mean less code that does the same thing and preserves the current API? | 14:10 |
| mars | well, it does the same thing, but in a different form (no more well-named Mismatch subclasses) | 14:11 |
| mars | not saying that's a bad thing, just pointing out one consequence of the convenience | 14:12 |
| jml | mars, is that such a loss? | 14:12 |
| jml | right | 14:12 |
| jml | well, the patch is up | 14:14 |
| gmb | sinzui, I've just filed https://bugs.edge.launchpad.net/malone/+bug/619218 against malone, but I'm wondering if it should be a registry task instead. | 14:17 |
| gmb | What do you think? | 14:17 |
| sinzui | I think it is bugs since not all projects use bugs | 14:17 |
| sinzui | gmb: is this a feature we are considering? | 14:18 |
| gmb | sinzui, No, just something that I've seen mentioned twice in as many days. | 14:19 |
| gmb | I'll leave it on bugs. | 14:19 |
| gmb | Thanks. | 14:19 |
| sinzui | EdwinGrubbs, gmb: I landed a fix to count only open bugs on a dsp for https://edge.launchpad.net/ubuntu/maverick/+needs-packaging Do I need to wait for a daily proc to run that updated bug heat on dsps (max_bug_heat bug_heat_count)? | 15:27 |
| EdwinGrubbs | sinzui: I believe that it runs whenever a bugtask is added or removed on a sourcepackage, so they won't all get updated at once. | 15:31 |
| sinzui | Thanks EdwinGrubbs | 15:32 |
| EdwinGrubbs | matsubara: ping | 16:00 |
| matsubara | hi EdwinGrubbs | 16:01 |
| EdwinGrubbs | matsubara: to make it easier to debug errors where mailman sends an email to launchpad for moderation via xmlrpc, I want to add the full text of the email to the oops. I've looked at how oops-tools parses it, and I want to discuss the various possible solutions. | 16:03 |
| barry | /join #ubuntu+1 | 16:05 |
| barry | 16:05 | |
| matsubara | EdwinGrubbs, sure, here or mumble? (IRC is easier for me as I'm in the middle of something) | 16:06 |
| EdwinGrubbs | matsubara: we can do it on irc. Before I start listing off the possibilities, do you have any thoughts on how you would like this to be solved? | 16:07 |
| gary_poster | sinzui: hey. does registry do gpg stuff usually (https://bugs.launchpad.net/bugs/618347) or is this a foundations thing? foundations hasn't done much of any gpg stuff on my watch. | 16:09 |
| sinzui | gary_poster, That is registry. I believe it is the last piece of code that uses logintoken | 16:11 |
| jcsackett | lifeless, ping? | 16:11 |
| gary_poster | ok thank you sinzui | 16:11 |
| EdwinGrubbs | matsubara: ok, I'll start then. Option 1) Just add a delimiter at the end of the traceback to add a single section. | 16:12 |
| EdwinGrubbs | matsubara: Option 2) Use the python multifile module to add mulltiple delimited sections after the traceback. | 16:13 |
| matsubara | EdwinGrubbs, sorry, I'm looking for an oops as an example. I think one idea is to make like the codehosting oopses and add the email as part of the request variables | 16:13 |
| matsubara | kinda hackish though | 16:13 |
| matsubara | EdwinGrubbs, option 1 is interesting | 16:13 |
| EdwinGrubbs | matsubara: but aren't the request variables limited to one line normally. Even if the oops-tools can parse a single req var that is multiple lines, it would probably look really bad when browsing oops files on devpad. | 16:15 |
| matsubara | I just talked to the ISD guys and they want to have a way to parse oopses like this: https://admin.isd.canonical.com/oops/?oopsid=1607carambola1 and render the extra data info as html. perhaps extra_data should be a section after the traceback | 16:15 |
| jml | james_w, are your html matchers available for use in Launchpad (sorry, I've asked you before) | 16:15 |
| james_w | jml: I don't think so | 16:16 |
| jml | james_w, where could I find them? | 16:16 |
| james_w | lp:soupmatchers | 16:17 |
| EdwinGrubbs | matsubara: Option 3) expand on the email formatting idea. Add a header to indicate a new oops format, and turn it into a multipart/mixed mime email. This has the added benefit of providing name/value pairs for each part and unlimited number of sections. | 16:17 |
| jml | james_w, thanks. | 16:17 |
| matsubara | EdwinGrubbs, option 3 sounds interesting | 16:18 |
| EdwinGrubbs | matsubara: my one concern about rendering html in an oops is that the html might contain javascript attacks. | 16:19 |
| matsubara | EdwinGrubbs, yep, I'm aware of that. one of the things I'd like to discuss with the ISD guys before they implement that option | 16:20 |
| EdwinGrubbs | matsubara: so, should I wait until we can get more info from ISD, or can one of the solutions be started on. Technically, the oops-tool can just format extra data as text now and be updated to not escape the html later when the security concerns are addressed. The mime solution would be the handiest for that, since we would be able to add a content type to each section. | 16:23 |
| matsubara | EdwinGrubbs, I think we can start and then adapt the work to address ISD's needs in the future. | 16:25 |
| matsubara | EdwinGrubbs, once you get the changes needed on the LP side could you ping me so I can update oops-tools to understand this new format? | 16:25 |
| EdwinGrubbs | matsubara: well, I was planning on updating oops-tools at the same time for testing purposes. Do you want me to go with Option 3? | 16:29 |
| === salgado is now known as salgado-lunch | ||
| EdwinGrubbs | matsubara: the main drawback to option 3 is that oops-tools has to be updated before launchpad starts producing oopses in that format. | 16:31 |
| EdwinGrubbs | just appending to the end of the traceback is sorta backwards compatible. | 16:31 |
| matsubara | EdwinGrubbs, right. I'd rather go with option 3 given that we will need this work in the future anyway | 16:33 |
| EdwinGrubbs | ok | 16:37 |
| jcsackett | what time does lifeless usually get active on IRC? | 17:07 |
| leonardr | i have a question for anyone who might know | 17:07 |
| leonardr | i have a test that's suddenly started failing on ec2 | 17:07 |
| leonardr | and the failures would be explained if the AppServerLayer used to make Launchpad available at "launchpad.dev", but recently it was changed to be available at "launchpad.dev:8085" | 17:08 |
| leonardr | did anything like that happen, or has the test launchpad always been available at launchpad.dev:8085? | 17:08 |
| leonardr | i know for some test layers it's been at :8085, but maybe it was formerly inconsistent and the inconsistency was fixed recently? | 17:08 |
| === Ursinha is now known as Ursinha-lunch | ||
| leonardr | another possibility is that the tests used to be run with a sitename of 'testrunner' and now they're run as 'development' | 17:19 |
| === jcsackett is now known as jcsackett|afk | ||
| jml | jcsackett|afk, early NZ time. | 17:23 |
| jml | leonardr, sorry, I don't know of any recent changes along those lines. | 17:24 |
| jml | leonardr, I'd be very surprised if the latter possibility were true. | 17:24 |
| === matsubara is now known as matsubara-lunch | ||
| leonardr | well, damn | 17:28 |
| jml | leonardr, does the test fail locally? | 17:30 |
| leonardr | jml: no | 17:31 |
| leonardr | jml, more specifics about the test | 17:31 |
| jml | leonardr, even if you merge in the latest devel? it's possible that something broken landed recently. | 17:31 |
| leonardr | jml, i think i know why it doesn't fail locally, let me explain | 17:32 |
| jml | leonardr, please do. | 17:32 |
| leonardr | when you run 'make' in launchpad, it generates some wadl for your instance | 17:32 |
| === salgado-lunch is now known as salgado | ||
| leonardr | the wadl defines the root url of the wadl as being http://api.launchpad.dev/ | 17:32 |
| leonardr | s/of the wadl/of the web service/ | 17:33 |
| leonardr | in local usage (and i'll test this to make sure), when you run a test, launchpad starts up as launchpad.dev, and the launchpadlib test makes requests to http://api.launchpad.dev/, and gets wadl that tells it to make more requests to http://api.launchpad.dev/... | 17:33 |
| leonardr | on EC2, when you run a test, launchpad starts up as launchpad.dev:8085 | 17:34 |
| leonardr | the launchpadlib test makes a request to https://api.launchpad.dev:8085/ | 17:34 |
| leonardr | and gets wadl that tells it to make requests to https://api.launchpad.dev/ | 17:34 |
| leonardr | but there's no such host, and the test fails | 17:34 |
| jml | I see. | 17:35 |
| jml | I'd be disappointed if ec2 tests were running with a different configuration to the local test runner. | 17:35 |
| === benji is now known as benji-lunch | ||
| leonardr | gary, jml: i have 100% confirmed that ec2 environment differs _somehow_ from up-to-date local environment | 18:01 |
| leonardr | such that the WADL served by ec2 sends you to api.launchpad.dev, but the WADL served by a local instance keeps you on api.launchpad.dev:8085 | 18:02 |
| jml | leonardr, I guess moving from fear to unpleasant certainty is an incremental improvement. | 18:02 |
| leonardr | i'm also fairly certain that the cached WADL is not contributing to the problem, because the problem persists even when i move the cached wadl out of the way on ec2 | 18:04 |
| jml | leonardr, do you know if this difference exists on up-to-date stable? | 18:04 |
| leonardr | jml: no, i don't | 18:04 |
| leonardr | ideally i would be able to watch the wadl being generated on ec2, but i can't, because the problem only shows up during the launchpadlib tests. and a breakpoint in launchpad, triggered by the launchpadlib tests, will just hang, since launchpad is running in the background | 18:06 |
| jml | yeah. interprocess bugs suck. | 18:08 |
| leonardr | testing another hypothesis... | 18:08 |
| leonardr | ok, even disabling the wadl cache altogether on ec2 doesn't get rid of the problem. so there is one specific request that goes to api.launchpad.dev:8085, and launchpad thinks "ok, what is my hostname? aha! it's api.launchpad.dev!" and generates the wadl accordingly | 18:15 |
| === matsubara-lunch is now known as matsubara | ||
| jml | mars, you might be interested in reviewing: https://code.edge.launchpad.net/~jml/launchpad/different-ec2-mail/+merge/32905 | 18:30 |
| jml | mars, it's the first deliverable chunk of work from the branch we started at the epic | 18:30 |
| === Ursinha-lunch is now known as Ursinha | ||
| Green00000 | hi | 18:38 |
| === benji-lunch is now known as benji | ||
| Green00000 | i have a problem with deleting my launchpad-account ...... can anybody answer?? | 18:41 |
| benji | Green00000: what's the problem? | 18:43 |
| Green00000 | hi. | 18:43 |
| Green00000 | i loged in first. | 18:45 |
| Green00000 | then i deactivated the account. | 18:45 |
| Green00000 | wait. | 18:47 |
| Green00000 | okay. | 18:49 |
| Green00000 | once again. | 18:49 |
| Green00000 | . | 18:49 |
| Green00000 | . | 18:49 |
| Green00000 | . | 18:49 |
| Green00000 | i log in launchpad. | 18:49 |
| Green00000 | then the point "change details" | 18:50 |
| Green00000 | down on the site | 18:51 |
| Green00000 | "Deactivate your account" | 18:51 |
| jml | Green00000, https://answers.edge.launchpad.net/launchpad/+faq/968 might be helpful. | 18:52 |
| Green00000 | the message "Your account has been deactivated." | 18:52 |
| Green00000 | but it's just possible to login. | 18:53 |
| Green00000 | okay. | 18:54 |
| Green00000 | read. | 18:54 |
| Green00000 | but | 18:54 |
| Green00000 | .... | 18:54 |
| Green00000 | why can i log in any more?????? | 18:55 |
| jml | Green00000, you mean, you can still log in even after deactivating? | 18:56 |
| Green00000 | (btw -- thx to jml.) | 18:56 |
| Green00000 | yes. | 18:56 |
| jml | Green00000, I don't know why that's the case. It probably has something to do with Launchpad using the Ubuntu One account service. | 18:56 |
| Green00000 | i loged out and delete all cookies. | 18:58 |
| Green00000 | but the log in with my emailadress and the password is still working. | 18:59 |
| leonardr | gary, jml: it's official. i get exactly the same error when i run launchpadlib on an unaltered 'stable' on ec2 | 19:23 |
| gary_poster | huh | 19:23 |
| gary_poster | well, my ec2 instance is *almost* ready to go... | 19:23 |
| jcsackett|afk | mars, ping | 19:24 |
| === jcsackett|afk is now known as jcsackett | ||
| leonardr | so the other question is, why is it possible for anyone to land anything? | 19:24 |
| mars | hi jcsackett | 19:24 |
| jcsackett | mars: you're oncall reviewer today, correct? | 19:25 |
| gary_poster | yeah, that question occurred to me | 19:25 |
| mars | jcsackett, why yes, I am :) | 19:25 |
| * mars does not like that question | 19:26 | |
| jcsackett | mars: awesome! i've got two mps that sinzui may be getting to, but indicated he didn't mind getting beaten to the punch, and i just realized the mp notices went out long enough go they may have been lost. https://code.edge.launchpad.net/~jcsackett/+activereviews | 19:26 |
| mars | (about the ec2 landings) | 19:26 |
| jcsackett | mars: don't hate me. :-) | 19:26 |
| mars | jcsackett, my bad, crossed messages | 19:26 |
| jcsackett | mars: cool. | 19:26 |
| sinzui | jcsackett, I started https://code.edge.launchpad.net/~jcsackett/launchpad/unsubscription/+merge/32680 a few minutes ago | 19:27 |
| sinzui | you will hate my nitpicks | 19:27 |
| jcsackett | sinzui: cool. | 19:27 |
| jcsackett | so, mars: only one mp. :-) | 19:27 |
| mars | jcsackett, sure, I can review those. Head on over to #launchpad-reviews and add what you have to the queue | 19:27 |
| jcsackett | sinzui: i never hate your nitpicks. :-) | 19:27 |
| mars | jcsackett, depends if I or sinzui gets to them first | 19:27 |
| leonardr | gary: i have a hypothesis why other people have been landing things. if you run the full test suite it hangs, and after about 6 hours the ec2 instance silently shuts down. if someone is using the --merge option (or whatever it is, i don't use it), maybe the ec2 script interprets this silence as success? | 19:28 |
| gary_poster | I hope not leonardr. mars, any evidence to the contrary? ^^^ | 19:29 |
| leonardr | and we don't have the problem in production because production servers don't have the :8085 trickery | 19:30 |
| mars | ergh - lost with that. leonardr, your branch fails on stable ec2? Or just plain old stable ec2? | 19:30 |
| leonardr | mars: plain old stable ec2 | 19:30 |
| leonardr | there are test failures and a hang when running the launchpadlib tests | 19:31 |
| mars | that is weird | 19:31 |
| mars | leonardr, how about when you run just the one test? | 19:31 |
| leonardr | mars: same thing | 19:31 |
| mars | ec2 test -o '-t introduction.txt' | 19:31 |
| leonardr | oh, start up a new ec2 instance just to run the one test? | 19:31 |
| leonardr | i can try that | 19:31 |
| mars | leonardr, it is introduction.txt that it hangs on so far? | 19:32 |
| leonardr | mars: yes, there are test failures in other launchpadlib tests, but introduction.txt causes the hang | 19:32 |
| mars | leonardr, two tests then: first, just introduction.txt. Next, the suite that introduction.txt is contained in. | 19:33 |
| leonardr | ok | 19:33 |
| mars | leonardr, don't bother with the second if introduction.txt alone hangs | 19:34 |
| leonardr | mars: i'll test with stable? | 19:34 |
| mars | leonardr, you don't have to | 19:34 |
| mars | stable and devel don't drift that far | 19:34 |
| leonardr | ok, but i'll test with the basic code instead of the code with my changes in it | 19:35 |
| mars | and there is only moderate risk of you stepping on someone else's accident while trying this | 19:35 |
| mars | right | 19:35 |
| mars | leonardr, run with --attached if you want to get a better idea when the test finishes | 19:35 |
| mars | instead of busy-waiting on your INBOX for the failure mail | 19:36 |
| leonardr | ok | 19:36 |
| leonardr | i predict there won't be a failure mail--i never got one before | 19:36 |
| mars | well, if you run just that one test, it should be pretty obvious when it is hung :) | 19:36 |
| mars | ... kind of. (I hope output buffering doesn't bite us here) | 19:37 |
| === al-maisan is now known as almaisan-away | ||
| gary_poster | leonardr: ok, I think I have duped. I ran ./bin/test -vvt launchpadlib and launchpadlib/docs/introduction.txt has been sitting for about a minute. | 19:41 |
| leonardr | gary, did you get any other failures? | 19:41 |
| gary_poster | no leonardr | 19:41 |
| leonardr | gary: control-c and paste me the traceback | 19:41 |
| leonardr | i kind of suspect there are 2 problems | 19:42 |
| gary_poster | leonardr: http://pastebin.ubuntu.com/479533/ | 19:42 |
| gary_poster | that doesn't look very wadllib-y | 19:43 |
| leonardr | gary: run bin/test -vvt hosted-files.txt and see if that works | 19:43 |
| gary_poster | k | 19:43 |
| salgado | rockstar, I've used lp-submit and the m-p was created but I was redirected to the wrong url (https://code.edge.launchpad.net/1.0/~salgado/launchpad/upload-policy-utility/+merge/32911). it has a leading '1.0' | 19:44 |
| salgado | is that a known bug? | 19:44 |
| rockstar | salgado, probably not a known bug, but I see why it would do that. | 19:46 |
| gary_poster | leonardr: it passed | 19:46 |
| gary_poster | this is with your branches, btw, leonardr | 19:46 |
| leonardr | gary: yes! that's the problem i originally had on thursday (and, like you, i didn't have the wadl problem) | 19:46 |
| salgado | rockstar, should I report it against bzr or is this a plugin? | 19:46 |
| leonardr | and i think that is because of the new launchpadlib--the test launchpad thinks it's launching a web browser | 19:47 |
| gary_poster | so I should try to dig into that bug, leonardr? | 19:47 |
| leonardr | but there could be any number of causes, and it could even be caused by an inaccurate wadl file | 19:47 |
| leonardr | gary: yes, please. put a breakpoint in the TestableLaunchpad constructor. coming up with more specific advice now | 19:48 |
| gary_poster | ok | 19:48 |
| jam | hey all, what is the correct way to run a subset of the lp test suite? Is it just 'bin/test -m lp.module' ? | 19:49 |
| gary_poster | jam, use -t | 19:49 |
| gary_poster | (my suggestion) | 19:49 |
| jam | gary_poster: in what way? | 19:49 |
| jam | (can you give an actual invocation) | 19:49 |
| dobey | how does one set the status on a bug_task via launchpadlib? task.setStatus() doesn't seem to exist, and task.status = status gives me a HTTP 401 :( | 19:50 |
| gary_poster | ./bin/test -t launcpadlib | 19:50 |
| gary_poster | jam, I suggest searching for -t TEST in the ./bin/test --help output | 19:50 |
| leonardr | dobey: how did you set up your Launchpad object? | 19:51 |
| rockstar | salgado, bzr-pipes has an implementation of it. | 19:51 |
| jam | gary_poster: you guys really need to play with the bzr selftest suite... this is rather, uh, clunky. :) | 19:51 |
| rockstar | salgado, it might be a bzr bug, but let abentley sort that out. | 19:51 |
| gary_poster | jam, k :-) | 19:51 |
| dobey | leonardr: launchpad.Launchpad(creds, EDGE_SERVICE_ROOT, cachedir) | 19:51 |
| jam | so far, I'm sitting at a couple of minutes, and I don't think I've actually seen tests being run yet | 19:52 |
| leonardr | dobey: how did you get the creds? if they don't have write permission, then attempting to change the dataset will give you 401 | 19:52 |
| jam | time bzr selftest -s bt.test__chk ... ran 50 tests in 1.4s | 19:52 |
| leonardr | gary: ok, the section 'authorizing the request token' in launchpadlib.txt deserves to fail. at the very least, it prints out stuff like 'come back here and press enter', which isn't there anymore | 19:53 |
| leonardr | gary: but i bet it's hanging there instead of printing stuff out | 19:53 |
| jam | also not sure why there are 3 200MB instances of python running | 19:53 |
| mars | jam, three instances when running bin/test? | 19:53 |
| jam | mars: yes | 19:53 |
| mars | might be the servers from different layers | 19:53 |
| dobey | leonardr: is task.status = status the correct way to do it? | 19:54 |
| leonardr | dobey: yes | 19:54 |
| leonardr | task.status = status; task.lp_save() | 19:54 |
| gary_poster | that's what I was thinking, mars, but not sure this is particularly productive. old ground. | 19:54 |
| jam | mars: one is the librarian it seems | 19:54 |
| salgado | rockstar, are you suggesting that I file it against bzr-pipes? that seems weird -- the help for that command says "From: plugin "launchpad"" | 19:54 |
| mars | gary_poster, yep | 19:54 |
| jam | mars: two of them are 'python bin/test' | 19:54 |
| dobey | leonardr: ok, thanks | 19:55 |
| mars | jam, ah, that is because the zope testrunner fires up subprocesses of itself while testing | 19:55 |
| rockstar | salgado, that means you should file the bug on bzr then. | 19:55 |
| jam | mars: it also seems to be doing stuff like 'branch-distro.py' | 19:56 |
| rockstar | salgado, there are two implementations, one in bzr and one in bzr-pipes | 19:56 |
| jam | is this all setup stuff? | 19:56 |
| jam | Does it run on every test suite run? | 19:56 |
| mars | jam, what test command did you run? | 19:56 |
| jam | mars: bin/test -t lp.codehosting | 19:56 |
| gary_poster | leonardr: going to launchpad-foundations | 19:56 |
| leonardr | ok | 19:56 |
| jam | I'd *like* to just run the 1 test I just wrote | 19:57 |
| jam | but I'm still sorting out how to ever run the test suite | 19:57 |
| jam | I'm also seeing it say: | 19:57 |
| jam | https://lists.ubuntu.com/archives/bazaar/2010q3/069549.html | 19:57 |
| mars | jam, what was the file name of the test you just wrote? | 19:57 |
| jam | sorry bad paste | 19:57 |
| jam | lib/lp/codehosting/tests/test_lpserve.py | 19:57 |
| jam | I also see my invocation saying it is running the 'canonical.testing.layers.BaseLayer' tests | 19:58 |
| jam | which seems like they should be skipped by '-t' | 19:58 |
| lifeless | moin | 19:58 |
| salgado | rockstar, I think in my case it's the one in bzr because I don't seem to have bzr-pipes installed. thanks for the help | 19:58 |
| jam | did I type something wrong? | 19:58 |
| jam | hi lifeless | 19:58 |
| jam | lifeless: thank you very much for making 'bzr selftest' what it is today :) | 19:58 |
| lifeless | jam: my pleasure | 19:58 |
| rockstar | salgado, no prob. | 19:59 |
| mars | jam, 'bin/test -t test_lpserve' or just 'bin/test -t lpserve' | 19:59 |
| jam | mars: any idea how I can stop it gracefully? | 19:59 |
| mars | ^C | 19:59 |
| mars | and wait | 19:59 |
| jam | seems to need waiting for a *long* time | 19:59 |
| mars | yep | 20:00 |
| jam | it did eventually stop, though | 20:01 |
| salgado | I don't wait; I press ^C multiple times | 20:01 |
| salgado | but it's either that or wait | 20:01 |
| jam | any ideas why 'lp.codehosting' is running canonical.* tests? | 20:01 |
| jam | I guess 19s to run the one test file isn't too bad | 20:02 |
| jam | though 19s real time to 'Ran 4 tests ... in 7.3s' is a bit high | 20:02 |
| jam | (the problem with -t is I'm guessing it has to load all possible tests, and then filter them, which is why bzr switched to '-s' for prefix matching, and only loading tests that could possibly match the pattern) | 20:03 |
| jam | anyway, thanks mars and gary_poster I at least have it running | 20:03 |
| mars | jam, np. You are right about the -t thing, discovery takes a while. IIRC discovery parses every doctest it finds, too | 20:04 |
| mars | not optimal | 20:04 |
| mars | I looked into fixing that, found the code, but no solution jumped out at me. | 20:06 |
| jam | mars: so bzr went the route of using a custom TestLoader and modules that expose 'load_tests' (which is also added to python 2.7 and unittest2) | 20:06 |
| jam | so it loads the root, which knows about loading children, and all check against the pattern to see if any of these children should be evaluated | 20:07 |
| lifeless | sinzui: jc sackett pinged me, do you know what about ? | 20:07 |
| sinzui | I think it was about the review of his branch | 20:08 |
| mars | jam, makes sense, some adaptation of that may work for our pagetests too. | 20:09 |
| mars | However, I don't know if our testrunner makes a distinction between load versus find | 20:09 |
| mars | I know that for our pagetest suites our code does not make that distinction | 20:10 |
| jcsackett | lifeless: i just wanted to make sure i had answered your questions re: plus-participation. | 20:10 |
| lifeless | jcsackett: ok, let me look up the response now | 20:11 |
| lifeless | jcsackett: I tried to tab complete your name here, before but failed - sorry! (Thats why I asked sinzui :P) | 20:11 |
| jcsackett | lifeless: it's all good. | 20:11 |
| lifeless | ok | 20:19 |
| lifeless | so, I've put forward a possible algorithm to do this in two queries | 20:19 |
| lifeless | if you liked the previous UI | 20:20 |
| === lifeless changed the topic of #launchpad-dev to: Launchpad Development Channel | Week 1 of 10.09 | PQM is OPEN | firefighting: - | https://dev.launchpad.net/ | Get the code: https://dev.launchpad.net/Getting | On-call review in irc://irc.freenode.net/#launchpad-reviews | ||
| jcsackett | lifeless, was that to me? | 20:22 |
| lifeless | jcsackett: yes | 20:22 |
| lifeless | jcsackett: just now, in the MP | 20:22 |
| lifeless | poolie: webapp flags is in devel now | 20:22 |
| lifeless | gmb: ^ | 20:22 |
| jcsackett | lifeless: i think i like that, but the graph part goes by a little quickly (or may in fact go over my head). | 20:24 |
| lifeless | oh, I missed a step, darn | 20:25 |
| jcsackett | sinzui: i've pushed up the changes on my subscription mp. | 20:26 |
| sinzui | thanks | 20:26 |
| jcsackett | lifeless: that makes me feel a bit better. | 20:26 |
| lifeless | updating in a sec | 20:32 |
| lifeless | updated | 20:36 |
| lifeless | flacoste: thank you | 20:37 |
| flacoste | bac, sinzui: so official_codehosting where EXISTS (SELECT 1 FROM Branch JOIN ProductSeries ON (Branch.id = ProductSeries.branch) JOIN Product ON (ProductSeries.id = Product.development_focus) WHERE BranchType = HOSTED AND ProductId = <product_id>)? | 20:48 |
| flacoste | i mean it becomes | 20:48 |
| flacoste | or can we consider a MIRRORED or REMOTE branch has officially codehosting? | 20:48 |
| sinzui | flacoste, HOSTED only | 20:49 |
| flacoste | sinzui: ack | 20:49 |
| lifeless | jcsackett: does it make more sense now ? | 20:57 |
| jcsackett | lifeless: i think so; graph theory isn't my forte, so i'm sort of mulling it over in a terminal to get a handle on it. | 20:58 |
| lifeless | ok | 20:58 |
| lifeless | please do ping me | 20:58 |
| lifeless | if you want to chat about it | 20:58 |
| jcsackett | lifeless: will do. probably shortly. :-P | 20:58 |
| flacoste | sinzui: that actually gives us about ~2000 more official_codehosting projects :-) | 20:58 |
| flacoste | sinzui: make that 4000! | 20:59 |
| flacoste | sinzui: about the same number than we have projects officially using Launchpad for Bugs | 20:59 |
| sinzui | yes. as back suspected when he learned we were counting it. He was pretty sure that users could not set the value over the last year | 20:59 |
| sinzui | s/as back/as bac/ | 21:00 |
| flacoste | sinzui: looking at https://lpstats.canonical.com/graphs/ProductsFlaggedOfficial/, I'd say they can't flag it since May | 21:01 |
| sinzui | I would have though March, but May is close enough | 21:02 |
| flacoste | thumper will be happy | 21:02 |
| sinzui | flacoste, lets add a constraint product.active = TRUE | 21:04 |
| flacoste | sinzui: ok | 21:04 |
| flacoste | sinzui: 2000 less projects | 21:04 |
| sinzui | We had a lot of project try launchpad. | 21:05 |
| jcsackett | lifeless: you want me to PM you about this or just carry on the conversation for all to see? | 21:05 |
| lifeless | jcsackett: more eyes, more thoughts | 21:06 |
| sinzui | I saw a few hundred projects with empty branches in my review of all projects. There never was code for a lot of projects | 21:06 |
| jcsackett | lifeless, works for me. | 21:06 |
| jcsackett | okay, lifeless, if i understand you right, you suggest we pull all teams at once, then pull all the relationships for those teams, and out of the resulting query sets map out the relationships. | 21:07 |
| jcsackett | so if we pull user in a, b, and c, and then pull that a is in b and c, and then that b is in c, we can build out the link from that. | 21:08 |
| lifeless | right | 21:09 |
| lifeless | user in a,b,c is one query | 21:09 |
| lifeless | a in b,c and b in c is one query | 21:09 |
| lifeless | but - the second query would be direct memberships only so it would look like | 21:10 |
| lifeless | user in a,b,c | 21:10 |
| lifeless | user in c c in b b in a | 21:10 |
| sinzui | flacoste, I think the ProductsFlaggedOfficial graph is suspicious. Blueprints is not used as much as it is claimed to be used | 21:10 |
| lifeless | as the two resultsets | 21:10 |
| jcsackett | wouldn't a in b,c and b in c be two queries? or are you thinking (in pseudo sql) something like "select team_participant, team, where team_participant is in [list_of_teams]" | 21:11 |
| lifeless | well it won't be the team participation exploded table | 21:14 |
| lifeless | it will be the direct table | 21:14 |
| lifeless | uhm, looking | 21:14 |
| lifeless | and yes, 'in' is operator | 21:14 |
| lifeless | we want all the rows that directly list any of (user, a, b, c) as the member of a team | 21:15 |
| lifeless | memberships = store.find(TeamMembership, TeamMembership.person in (user, a, b, c)) | 21:15 |
| lifeless | print list(memberships) | 21:16 |
| lifeless | [(user, c), (c, b), (b, a)] | 21:16 |
| lifeless | # thats fiction, but pretty darn close to reality | 21:16 |
| lifeless | we can actually do all of this in one query | 21:17 |
| lifeless | I've been describing two to show the algorith, | 21:17 |
| lifeless | mwhudson: you were right about the windmill error being bogus | 21:23 |
| jcsackett | so, lifeless, basically once we have that final dict you suggest walking through it to find the paths from user to indirect teams? seems like that could get expensive computationally in these pathalogical cases. | 21:27 |
| lifeless | walk in the other direction | 21:28 |
| lifeless | you have all the terminal points | 21:28 |
| lifeless | they all walk directly to the user | 21:28 |
| lifeless | bam | 21:28 |
| lifeless | the api you have - 'find path to ' is a wonky api | 21:28 |
| lifeless | we can just iterate this graph and spit out all in a tabular form all the paths we want to render | 21:29 |
| lifeless | then in the template just loop on the paths rendering them | 21:29 |
| mwhudson | lifeless: it's very strange that | 21:29 |
| lifeless | mwhudson: eparse | 21:30 |
| mwhudson | lifeless: the windmill error thing is very strange | 21:30 |
| lifeless | yes, yes indeed | 21:30 |
| lifeless | jcsackett: if we start at the user, and for every team they are in (direct connections in that graph), we start a path - [user, teamx], and then for each path we repeat that expansion - [[user, teamx], teamy] and so on | 21:37 |
| lifeless | jcsackett: that will be fast - we don't permit cycles AIUI in the db | 21:37 |
| lifeless | there may be multiple paths, but thats ok. when we're done we just tabulate and discard any duplicate terminal nodes | 21:38 |
| lifeless | so we'll have | 21:38 |
| jcsackett | lifeless: AIUI == ? | 21:38 |
| lifeless | [user, teamx] | 21:38 |
| lifeless | As I Understand It | 21:38 |
| jcsackett | ah. | 21:38 |
| lifeless | [user, teamx] | 21:38 |
| lifeless | [user, teamx, teamy] | 21:38 |
| lifeless | etc | 21:38 |
| lifeless | len(path) !=2 => indirect path | 21:39 |
| jcsackett | lifeless: okay, i think i've got it. | 21:39 |
| lifeless | oh final tweak I think - prefer the shortest paths when trimming - | 21:39 |
| lifeless | [user, teamy] should win over [user, teamx, teamy] in the discard stage | 21:39 |
| jcsackett | lifeless: yeah, i sort of assumed that when you mentioned discards. | 21:39 |
| jcsackett | lifeless: i do think i like that better; it's worth a shot at least before basically killing the UI. | 21:40 |
| lifeless | cool | 21:40 |
| lifeless | I think for nearly all of LP perf problems we can do something like this | 21:40 |
| lifeless | its been the experience [for most] issues in bzr anyhow | 21:41 |
| lifeless | and bzr has a harder problem in many ways : no persistent database with hot disk caches :) | 21:41 |
| jcsackett | that does seem like a much harder issue. :-) | 21:43 |
| === salgado is now known as salgado-afk | ||
| lifeless | losa ping | 22:11 |
| lifeless | I want to know what our current ssl session lifetime is set to | 22:11 |
| lifeless | and also the SSLSessionCache setting | 22:12 |
| mbarnett | lifeless: SSLSessionCache shmcb:/var/run/apache2/ssl_scache(512000) | 22:15 |
| mbarnett | SSLSessionCacheTimeout 300 | 22:15 |
| lifeless | thanks | 22:18 |
| lifeless | rt'd | 22:18 |
| mbarnett | welcome | 22:18 |
| lifeless | flacoste: rt 40918 | 22:18 |
| flacoste | lifeless: bumped up | 22:29 |
| lifeless | so what makes in_admin etc work | 22:30 |
| lifeless | my new branch made a change that looked fine and works some of the time but not always | 22:30 |
| lifeless | should one adapt to IPersonRoles? | 22:31 |
| thumper | lifeless: do you realise that it takes 30 minutes for a trivial branch to land on devel using PQM? | 22:36 |
| thumper | AFAICS much of that time is cleaning the working directory | 22:36 |
| lifeless | yes | 22:36 |
| thumper | why does it take so long? | 22:36 |
| lifeless | would like to fix | 22:36 |
| lifeless | thumper: its doing a heavyweight branch | 22:37 |
| lifeless | because we have calls that look at the bzr history but need to run in the chroot | 22:37 |
| lifeless | if we do a lightweight checkout of the http readonly branch url, they would work | 22:37 |
| lifeless | and we could bzr switch to do the commit | 22:37 |
| jelmer | 'evening lifeless, thumper | 22:38 |
| lifeless | hi jelmer | 22:39 |
| mwhudson | gary_poster: hi, can i ask you a couple of questions about buildout? | 22:42 |
| === matsubara is now known as matsubara-afk | ||
| lifeless | Segmentation fault in cron mail worries me | 22:44 |
| lifeless | on pear | 22:44 |
| jelmer | pear.. that's one of the importds isn't it? | 22:45 |
| mwhudson | yes | 22:45 |
| * rstat1 pokes salgado | 22:46 | |
| mwhudson | gary_poster: nm, afk | 22:48 |
| jam | anyone know where you get "zope.preferences.interfaces.IPreferenceGroup" from ? | 22:56 |
| jam | I just did 'apt-get update' and now I can't run the test suite anymore without tracking down stuff like this | 22:56 |
| wgrant | jam: Run 'make' | 22:57 |
| wgrant | It's an egg that was added a few weeks ago. | 22:57 |
| wgrant | You might also need to update-sourcecode, if you're really out of date. | 22:57 |
| jam | wgrant: to give a fuller description, I ran apt-get update, which broke bzr-builder because it imported something bzr-builddeb didn't like, when I updated bzr-builder it broke my source code. when I merged devel, it broke because it was missing interfaces | 22:58 |
| jam | wgrant: is this par for the course? | 22:58 |
| flacoste | thumper: https://lpstats.canonical.com/graphs/BuilddLagProductionSupportedArch/ | 22:58 |
| jam | (of doing development on launchpad) | 22:58 |
| lifeless | jam: sometimes | 22:59 |
| lifeless | jam: bzr 2.2 just landed in devel | 22:59 |
| lifeless | jam: so you need to do ./utilities/update-sourcecode | 22:59 |
| lifeless | jam: and I'd like to fix this up, move to totally deb dependencies, that sort of thing; however this isn't [yet] a widely held view. | 22:59 |
| lifeless | jam: I think its stockholm syndrome. | 22:59 |
| flacoste | thumper: https://lpstats.canonical.com/graphs/BuilddLagPPASupportedArch/ | 23:00 |
| jam | lifeless: I think you need 1 dependency structure | 23:00 |
| jam | I'm not sure that it has to be debs, since I don't really want apt-get update to break my dev environment each time | 23:00 |
| lifeless | jam: I'm not sure why it would break your dev environment | 23:00 |
| lifeless | but yes | 23:00 |
| jam | (I need the latest firefox patch, and oh, my test suite won't run anymore) | 23:00 |
| jam | lifeless: see above. It seems that we use the system bzr-builddeb? but the source tree bzr-builder? | 23:00 |
| lifeless | jam: well, given we use windmill, we do have dependencies on firefox | 23:01 |
| lifeless | jam: its a little magic, we'll use one if its available or an egg if its not. | 23:01 |
| rstat1 | s | 23:02 |
| rstat1 | whoops | 23:02 |
| jam | lifeless: If the answer is "don't update your system until your current patch has been landed" I guess that would be a different answer | 23:02 |
| jam | I realize it is a significantly more complex project. | 23:02 |
| lifeless | jam: I get the impression thats what folk do. IMBW | 23:02 |
| jam | but stuff like this is really pretty bad | 23:02 |
| lifeless | jam: its got many more dependencies, code base isn't that different in size | 23:02 |
| wgrant | I rarely find that an update breaks it. | 23:03 |
| wgrant | Except on a dev series. | 23:03 |
| wgrant | And even then it's normally the autosync that kills things. | 23:03 |
| flacoste | thumper: https://lpstats.canonical.com/graphs/CodeRecipeBaseTypeCounts/20090818/20100818/ | 23:04 |
| jam | lifeless: as an outsider, it isn't like I can change things. But I do find it strange that it is the current status quo | 23:04 |
| jam | as you say, stockholm syndrome :) | 23:04 |
| lifeless | jam: right | 23:04 |
| lifeless | jam: we're improving the dev process at the moment via the release-features-when-done project | 23:05 |
| jam | *sigh* and after all the update-sourcecode, etc. it is still broken | 23:05 |
| lifeless | jam: once that is bedded in, other bits of friction will be bubbling to the top | 23:05 |
| lifeless | jam: ok, current issue ? | 23:05 |
| jam | "DeprecationWarning: please use 'debian' instead of 'debian_bundle' | 23:05 |
| lifeless | check in aptitude that you have updated the lp deps | 23:06 |
| jam | from bzrlib.plugins.builder import RecipeParser imports something from bzr-builddeb and builddeb complains | 23:06 |
| lifeless | apt held-back that update for me for some reason | 23:06 |
| lifeless | poolie: are you successfully running concurrent vm tests ? | 23:21 |
| poolie | in multiple vms? yes | 23:22 |
| lifeless | \o/ | 23:22 |
| lifeless | you might like to write that up | 23:22 |
| poolie | iwbn to automatically CoW-fork them | 23:22 |
| poolie | so i can have as many as i like | 23:22 |
| lifeless | as well as your please-use-flags mail :) | 23:22 |
| poolie | but i haven't got there yet | 23:22 |
| lifeless | flags is in devel how | 23:22 |
| lifeless | *now* | 23:22 |
| flacoste | thumper: https://devpad.canonical.com/~stub/ppr/lpnet/latest-daily-timeout-candidates.html | 23:22 |
| poolie | mm i have a draft of that but wanted it to actually be landed first | 23:22 |
| poolie | thanks for that | 23:22 |
| lifeless | anytime | 23:23 |
| jml | I have a big hairy patch to ec2test/remote.py that I would love to have reviewed. | 23:25 |
| rstat1 | and I have an issue with openid if anyone is interested :) | 23:26 |
| wgrant | rstat1: What's the problem? | 23:28 |
| lifeless | rstat1: you probably need to describe the issue | 23:28 |
| rstat1 | https://answers.edge.launchpad.net/launchpad-foundations/+question/120150 << This explains it quite well. | 23:28 |
| jml | https://code.edge.launchpad.net/~jml/launchpad/different-ec2-mail/+merge/32905 | 23:29 |
| flacoste | bac, sinzui: https://code.edge.launchpad.net/~flacoste/tuolumne-lp-configs/fixup-officical_codehosting | 23:30 |
| rstat1 | the jist of the matter is: authenication on a lp.dev instance i have on a Lucid VM is impossible. I get an oops. "DiscoveryFailure: No usable openid services found for https://testopenid.dev | 23:30 |
| wgrant | It sounds like an Apache config issue. | 23:30 |
| rstat1 | Thats's what I'm thinking...https://testopenid.dev redirects to lp.dev and https://testopenid.dev/+openid coughs up a 503 | 23:31 |
| rstat1 | BUT my config for apache comes straight from the lp repos...save for a modifcation to only use one ip address. | 23:32 |
| lifeless | jml: I saw it | 23:32 |
| lifeless | jml: I think its great but a pain to review :( | 23:32 |
| wgrant | rstat1: What't the 503? Just a normal Apache one? | 23:34 |
| bac | flacoste: that branch looks good. i am not allowed to review it, though. | 23:34 |
| wgrant | What does the Apache error log say? | 23:34 |
| bac | flacoste: and thanks for doing it! | 23:34 |
| rstat1 | Yea its a normal 503...I'll go dig up the log. | 23:34 |
| flacoste | bac, a losa need to review and deploy it | 23:34 |
| bac | flacoste: right. i didn't know if you pasted just a FYI or wanted a review. | 23:35 |
| rstat1 | My apache log doesn't appear to have anything useful in it...just a bunch of debug crap from OpenSSL | 23:39 |
| === rstat1 is now known as rstat1-brb | ||
| jml | lifeless, yeah, I'm sorry about that. | 23:43 |
| jml | lifeless, I'm not sure I could stand the pain of breaking it up into multiple patches though. | 23:44 |
| lifeless | man, some of these failures are nuts | 23:52 |
| rockstar | Hm, sourcepackagerecipebuild views don't really have unit tests, just leaky sourcepackagerecipe view tests. | 23:56 |
| lifeless | oh | 23:59 |
| * lifeless hates on prejoins | 23:59 | |
Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!