/srv/irclogs.ubuntu.com/2011/10/18/#launchpad-dev.txt

lifelesswhoever is trying to and00:19
lifeless*land*00:19
lifelessPQMException: 'Failed to verify signature: gpgv exited with error code 2'00:19
lifelessplease be signing, or update your key with losa00:19
StevenKOr it could be OOPS tools branches have PQM subscribed again00:20
* lifeless checks00:20
lifelessnope00:21
lifelesshttps://code.launchpad.net/~launchpad-pqm/python-oops-tools/trunk00:21
lifelessbesides, this isn't the super-fast-busy-loop that normally creates :)00:21
StevenKYes, wgrant had that one fixed last night00:21
lifelesswhat was subscribed /00:22
wgrantlaunchpad-pqm00:22
wgrantThe branch owner.00:22
lifelessbahhh00:22
lifelessugh wikipedia00:24
lifelesswhy do I click on links.00:24
=== wgrant changed the topic of #launchpad-dev to: Performance Tuesday! | https://dev.launchpad.net/ | On call reviewer: wgrant | Critical bugtasks: 263
wgrantStevenK: https://code.launchpad.net/~wgrant/launchpad/bug-876171/+merge/7963900:54
=== leguin.freenode.net changed the topic of #launchpad-dev to: Performance Tuesday! | https://dev.launchpad.net/ | On call reviewer: - | Critical bugtasks: 273
StevenKwgrant: IDistroSeries._getAll{Sources,Binaries} do not take a status argument00:56
StevenKOh, sorry, misreading00:57
wgrantStevenK: I would hope not.00:57
wgrantRight.00:57
StevenKThere's a .find() hiding in those calls00:57
wgrantI am working around our lack of a PublicationCollection,.00:57
StevenKwgrant: Ignoring my misunderstanding -- r=me00:59
wgrantwallyworld_: Around?00:59
wallyworld_wgrant: yep00:59
wgrantwallyworld_: Bug #87653301:00
_mup_Bug #876533: changing the values in Comboboxes on my code page does not update the branch-list shown <regression> <ui> <Launchpad itself:Triaged> < https://launchpad.net/bugs/876533 >01:00
* wallyworld_ still waiting for lp01:01
wallyworld_wgrant: ah, ok. it works with the ff on01:02
wallyworld_but not off01:02
wallyworld_i'll fix it01:02
wgrantThanks.01:04
wallyworld_wgrant: wanna review it, and i'll get it in the pipeline https://code.launchpad.net/~wallyworld/launchpad/broken-branch-listing-876533/+merge/7964001:22
wgrantOh god, Julian's domination change has landed :(01:23
wgrantIt will be months before we can deploy!01:23
wallyworld_what's the domination change?01:23
StevenKUtter madness01:24
wgrantwallyworld_: Shouldn't that be unconditonal?01:24
wallyworld_wgrant: nope01:24
StevenKDon't dominate arch-indep binaries if any sources build them01:24
wgrantOr does the post_refresh_hook fire on load too?01:24
wallyworld_if the ff is there, the setup is done in the js01:24
wallyworld_that's why i didn't notice it01:24
wallyworld_because it works fine for us01:24
wgrant                    post_refresh_hook: hookUpFilterSubmission01:24
wallyworld_yep01:25
wgrantThat's the only call I can see.01:25
wgrantSo either post_refresh_hook is a lie, or it's buggy.01:25
wallyworld_wgrant: nope, that bit is behind the feature flag01:25
wallyworld_so we need to provide a call when the ff is off01:25
wgrantI realise.01:25
wallyworld_that's the basis of the bug01:25
wgrantBut does post_refresh_hook fire on page load?01:25
wallyworld_no, only if the ff is on01:26
wgrantIt looks like it should be broken when the ff is on as well.01:26
wgrantThat's what I mean.01:26
wallyworld_otherwise that bit of javascript is not executed01:26
wgrantDoes post_refresh_hook fire on page load?01:26
wgrantAssuming it's configured.01:26
wallyworld_yes01:26
wgrantAhhhh01:26
wgrantHow confusing.01:26
wallyworld_and it works fine then01:26
wgrantOK.01:26
wallyworld_it will be better when the ff is taken away and it's the same for everyone01:27
wgrantRight, it's just very confusing that post_refresh_hook is executed on load too.01:27
wgrantApproved.01:27
wallyworld_thanks.  post_refresh_hook is not executed though unless ff is on01:28
lifelessanyone up for an oops_amqp revu ?01:28
wgrantwallyworld_: Clearly.01:28
wgrantBut it's confusing that it doesn't just apply to refreshes.01:28
wallyworld_perhaps. the js module calls the hookup when it is done loading and also on refresh01:30
wallyworld_ie whenever the html form changes01:30
lifelesshttps://code.launchpad.net/~lifeless/python-oops-amqp/0.0.2/+merge/7963701:31
lifeless^ hint, hint01:31
wallyworld_lifeless: i'll have a look but am  working get a web service test to run properly. i'll need a bit of time to ramp up on the ampq stuff. can i do it a bit later?01:34
lifelesswallyworld_: well, I want low turnaround so that I can use it in the next project along01:35
wallyworld_ah, right01:35
wallyworld_i'll look now01:36
lifelesswallyworld_: tuesday wgrant is on OCR01:36
lifelesswgrant: ^ ;)01:36
wgrantSorry, just developing a test case to revert Julian's revision.01:36
lifelesswallyworld_: you're working on a critical regression, I think you should focus on that :)01:37
wallyworld_lifeless: already fixed01:37
wallyworld_and in ec201:37
lifelessah cool01:37
lifelessin which case, thank you!01:37
wallyworld_just a few lines of tales01:37
StevenKwgrant: You want to revert it?01:37
wgrantStevenK: It's far easier to QA if it's reverted. Plus I think it's buggy.01:38
lifelesswgrant: julian didn't dogfoodinate it before landing ?01:38
wgrantHe did.01:38
lifelessunity needs a cat-safe mode01:41
lifeless37 screenshots later..01:41
mwhudson:)01:41
lifelessall with the same second timestamp01:41
lifelessargh, this use-oops-twisted branch is becoming a branch of doom01:54
lifelessfix one test failure, 30 more pop out01:54
wgrantStevenK: http://paste.ubuntu.com/711550/01:59
wgrantStevenK: That test now fails.01:59
wgrantPasses at devel-301:59
wgrantThink that's revert-worthy?01:59
StevenKwgrant: The source is not superseded?02:02
StevenKIE, 1.0's source is not02:02
wgrantAh, sorry. The binary isn't.02:02
wgrantArch-indep can no longer supersede arch-specific, and vice-versa.02:03
StevenKHmmmm02:03
wgrantI think that's pretty clearly undesirable.02:03
StevenKI think you should add that test and then revert Julian's branch02:03
StevenKOr vice-versa02:03
wgrantOther way around, but yes.02:03
wgrantThanks.02:03
nigelbStevenK: \o/02:13
nigelbPurple Assasins!02:13
nigelbI lol'd02:13
lifelesswallyworld_: so for clarity, you're doing that review, or wgrant is ?02:14
wallyworld_lifeless: sorry, i assumed wgrant was02:15
wgrantI'll be done with the revert+test in a few minutes.02:15
wgrantCan do it then.;02:15
lifelesswallyworld_: I think we tied ourselves up in knots :)02:15
lifelesswallyworld_: wgrant: ok, wgrant it is02:15
wallyworld_ouch02:15
lifelessbwah I missed one more pending change. ><02:18
lifelessright, pushed. *now* we're cooking. Vit gahs02:26
* wgrant glares disapprovingly at lifeless' implicit relative imports.02:30
wgrantlifeless: Is r10 relevant to anything at all?02:31
lifelesswgrant: very02:33
wgrantOh?02:34
lifelesswgrant: fallback when amqp is down is a little cronscript that reads from the datedir repo and sends to amqp02:34
wgrantIt seems entirely unrelated to the rest of the branch.02:34
lifelessthe branches theme is 'stuff I learnt while hacking elsewhere'02:35
wgrantThat's slightly unpleasant.02:35
wgrantBut OK.02:35
wgrant__init__ no longer sets self.stopping. Deliberate?02:35
lifelessyeah02:35
lifelesscalling run_forever twice should work02:36
wgrantAlso, are you sure I can't send a message with a body of None?02:36
wgrantYou probably can't, but you never know.02:36
lifelessmoderately02:36
lifelesscan make it a random nonce / guard it if you like02:36
lifelessI'm not sure if its needed or not02:36
lifelessI've seen no reference to optional message bodies in amqp02:37
wgrantlifeless: Won't the retry logic not work if it goes away more than two minutes after the connection was established?02:40
lifelessgrah, you are right02:42
lifelessI will fix02:42
wgrantThanks.02:42
wgrantlifeless: Reviewed, with a few more comments.02:55
* wgrant unsubscribes canonical-launchpad-branches from review mail on python-oops-amqp02:58
wgrantSince PQM is unhappy.02:58
lifelesswgrant: you know that refactoring that (mild) duplicate keeps the line count in each test as long due to PEP8 line wrapping rules...03:07
wgrantlifeless: But it's going to be much clearer.03:07
StevenKwgrant: The revert and test have landed?03:09
wgrantStevenK: yes.03:09
StevenKExcellent.03:09
lifelesswgrant: actually, its a rabbit hole into that testtools 'function' object has no attribute getDetails03:12
lifelesswgrant: so I want to defer it; I've added a fixture but using it -> death spiral03:12
wgrantlifeless: at least turn it into a helper method.03:12
wgrantOn the test class.03:12
lifelesswgrant: so I think that that hurts clarity03:13
lifelessmost of those tests have two distinct actors03:13
lifelessthe channel and the queue03:13
wgrantAll the present boilerplate certainly doesn't aid clarity.03:13
lifelessright03:13
lifelessso I wrote a fixture to get a channel03:13
lifelessand something is wrong and its going to be a yak shaving session to figure it out03:13
lifelesswhich needs doing(but not now)03:14
lifelesshmm, I have a possible handle, poking03:15
lifelesswgrant: +70 lines -48 lines03:18
lifelesswgrant: (Which is why this was below my tolerance threshold)03:18
wgrantHow!?03:18
lifelesspastebin.com/bTKHC5wN03:19
wgrantYou could collapse the channel = channel_fixture.channel into the previous statement.03:21
lifeless+22 from the helper, docs, _all_ entry03:21
wgrantAnd I was imagining the QueueFixture would go into the same helper method.03:21
wgrantSince it's also shared between every test.03:21
wgrantPossibly even the publisher as well.03:21
lifelessway too much hidden in that03:21
lifelesshowever collapsing the assignment is good03:22
wgrantHm? Why can't the test just start with a queue name and publisher as instance variables?03:22
lifelessI've done that plenty before03:22
lifelessso I'm going to play the ETIRED card for sympathy :P03:22
nigelbSympathy card, well played.03:23
wgrantSo, all the tests need a channel and a queue, and there's nothing test-specific about that setup.03:24
wgrantPull it out into setUp(). Problem solved.03:24
wallyworld_why do i get a 'AttributeError: 'thread._local' object has no attribute 'interaction' error when i try and use 'with person_logged_in(xxx)'? i've tried a few things but can't seem to fix it03:25
lifelesswgrant: Expensive things being in setUp is a mispattern03:25
wallyworld_surely it matter not that there's no interaction before i try and log in03:26
lifelesswgrant: they are out-of-site, out-of-mind03:26
mwhudsonwallyworld_: full traceback?03:26
lifelesswgrant: I much prefer seeing whats going on03:26
wallyworld_Traceback (most recent call last):03:26
wallyworld_  File "/home/ian/projects/lp-branches/devel-sandbox/lib/lp/bugs/model/tests/test_bugtask.py", line 2390, in test_delete_bugtask03:26
wallyworld_    with person_logged_in(db_bug.owner):03:26
wallyworld_AttributeError: 'thread._local' object has no attribute 'interaction'03:26
wgrantlifeless: Put them in a fixtures class attribute, or something, then?03:26
wgrantThis is currently really cluttering boilerplate.03:26
lifeless+48-36 now, which is much better03:26
wgrantI want it killed.03:26
wgrantSomehow.03:26
lifelesswgrant: its not03:27
mwhudsonwallyworld_: bet it's the security check on bug.owner :/03:27
lifelesswgrant: yes its the same, but they are the essential actors in these tests03:27
lifelesswgrant: which is different to boilerplate that isn't an essential actor, and which I have great joy encapsulating and hiding03:27
wgrantIt's like 12 consecutive lines that's shared between every test.03:27
wgrantThey are essential actors for this *class of test*.03:28
lifeless3, 5 if you could the publisher (which has different params sometimes)03:28
wallyworld_mwhudson: you mean i have to remove the proxy?03:28
wallyworld_mwhudson: i've used with person_logged_in(bugtask.owner) ok03:28
mwhudsonwallyworld_: i don]'03:29
mwhudsonwallyworld_: i don't know, it's just a guess :-)03:29
* wallyworld_ tries it03:29
lifelesswgrant: I've pushed up rev 1203:29
wallyworld_mwhudson: that got me one step closer thank you. it would have been nice if the error said what was wrong though03:32
wallyworld_as in authorised attribute access or whatever03:33
mwhudsonwallyworld_: note that if you inherit from TestCaseWithFactory, you are logged in anonymously in setUp03:33
StevenKLike Zope is going to be nice.03:33
mwhudsonhandling security proxied objects without an interaction is like some metaphor involving explosives03:34
wallyworld_mwhudson: yes, but before the bit that was failing i was doing a web service call and other stuff so the initial login was gone i think03:35
mwhudsonah03:35
mwhudsonthat's a bit naughty03:35
mwhudsonyou could try restoreInteraction()03:35
mwhudsonthat always seemed a bit magical03:35
StevenKAh, webservice tests are special03:35
wallyworld_ok. the test case extends WebServiceTestCase. i have followed what others have done i think03:35
lifelesss/c/th/03:35
StevenKYou can't mix factory calls with webservice calls03:35
mwhudsonwallyworld_: without wanting to be rude03:36
wallyworld_you need to make actory calls to set up the objects03:36
mwhudsonwallyworld_: the "others" might not have understood what was going on :-p03:36
lifelessso, webservice calls wipe the interaction03:36
StevenKCreate or grab *everything* from the factory/zope, logout() and then deal with only the webservice03:36
lifelessI think someone has a patch up to fix that03:36
wallyworld_mwhudson: clearly i don't quite either :-)03:36
lifelessStevenK: or login again03:36
wallyworld_and then if i want to look at the state of the db after making the ws calls, i need to logout and login again?03:37
StevenKI prefer my way. Less chance of blowing off both legs03:37
mwhudsonwallyworld_: yeah, i have an email i need to write about that, come to think of it03:37
StevenKwallyworld_: Yes03:37
lifelesswgrant: what do you think?03:37
wgrantlifeless: Oh, the diff is updated?03:38
wallyworld_mwhudson: StevenK: thanks for the help.03:38
lifelesswgrant: yes, or loggerhead :)03:38
wgrant401+ queue = self.useFixture(03:38
wgrant402+ QueueFixture(channel, self.getUniqueString))03:39
wgrantDoes that really not fit on one line?03:39
lifelesswgrant: bah, I missed one - all the others were shuffled by channel_fixture03:39
lifelesss/1/3/03:39
lifelessno, really just the one03:40
mwhudsonwallyworld_: where is the code that makes a webservice request?03:41
wallyworld_mwhudson: it's local. i'll push up something in a sec03:41
wgrantlifeless: You could also s/connection_factory/conn_factory/ in the test and get the ChannelFixture stuff down to a single line.03:42
wallyworld_as it turns out, calloing logout before webservice_for_person doesn't work03:42
StevenKWhat traceback do you get?03:42
lifelesswgrant: when we feel the urge to do that, its a hint that PEP8 is wrong :)03:42
wallyworld_'no local interaction', no real traceback03:42
wgrantPEP 8 is not relevant here. Every coding standard ever is.03:43
wgrantIt's not an indentation problem, it's a line length problem.03:43
lifelessprecisely03:43
lifelessanyhow, yes I could do that03:43
lifelesshowever, the focus right now is on this patch03:43
wgrantIt looks more reasonable now.03:44
StevenKwallyworld_: If you're doing webservice_for_person(bugtask.owner) that wants you logged in03:44
StevenKYou need to grab out the person beforehand03:44
StevenKowner = bugtask.owner ; logout(); webservice_for_person(owner)03:45
wgrantlifeless: Approved.03:45
StevenKThat's what I meant by *everything*03:45
lifelesswgrant: thanks03:45
* StevenK heads out for about 003:45
StevenKSigh03:45
StevenKs/\(0\)/3\1/03:46
* wgrant is tempted to make LP mailing lists strip text/html parts.03:47
lifelesss/.*/bye/ 'Fixed'03:47
wallyworld_mwhudson: so this gets passed the ws stuff but the delete doesn't 'stick' so the last assert fails https://pastebin.canonical.com/54477/03:52
mwhudsonwallyworld_: i hope those commit() calls aren't really required03:55
wallyworld_mwhudson: the first one is03:55
mwhudsonwallyworld_: i dunno what's going on though, maybe turn on statement logging in postgres?03:56
wallyworld_or else the ws cannt see the newly created bug03:56
mwhudsonuh really?03:56
mwhudsondelightful03:56
wallyworld_mwhudson: yeah, doing the sql logging now. nb the normal unit tests for this all work03:56
wallyworld_i've tried removing the first commit and it blows up with a 'not found' error or similar03:57
mwhudsonoh right, webservicecaller makes an actual http request03:57
mwhudsonwell, not quite03:58
mwhudsonwallyworld_: i'm in well over my head now04:00
wallyworld_mwhudson: np. thanks for your input04:00
=== almaisan-away is now known as al-maisan
* wallyworld_ goes to get flat tyre fixed05:07
StevenKThe one on your car, or your belly?05:08
wallyworld_ha05:09
wallyworld_smart-ass05:09
wallyworld_the new one on my car that went flat overnight :-(05:09
nigelbwallyworld_: clearly you need to get it exchanged ;)05:20
=== al-maisan is now known as almaisan-away
stublifeless: So a quick scan of your oops/ampq work worries me a bit due to the amount of crap you need to use Rabbit sanely. I'll have a better scan later, but maybe ZeroMQ has less of an implementation burden than I thought (service registry etc.)05:54
lifelesscan you expand on what you mean ?05:57
stublifeless: Reconnection logic, EPIPE handling, local queuing - that sort of stuff. That seems flaky with the Rabbit libs making us do the work. In ZeroMQ we get that for free but need to implement some sort of exchange (such as a service registry to allow peer-peer, or a cookbook broker).05:59
lifelessstub: could you do an 'approve' vote on https://code.launchpad.net/~lifeless/python-oops-tools/amqp/+merge/79644 - I think tarmac is not setup for self-review yet06:00
stubdone06:01
lifelessfingers crossed06:01
lifelessstub: so, we're very close to done with a working rabbit; I think you have a pretty solid point though06:01
wgrantHow is 0mq better on those fronts?06:02
wgrantI don't see how it can do local queueing.06:02
lifelesswgrant: you give it a disk path and it spools to disk if the endpoints are unavailable06:02
lifelesswgrant: up to a threshold you set, at which point it either errors or discards (again, you set the policy)06:02
stubSo messages are retransmitted when it gets back up. But the local storage is not durable, so you lose it if the process terminates.06:03
lifelesswgrant: only applies to pubsub obviously06:03
stubBut it is free06:03
wgrantRight, we need durability across process restarts.06:03
stubThere are a lot of 'we needs'. What I'm wondering is if the stuff we need to implement for rabbit is going to be more work or less reliable than the stuff we need to implement for an alternative like ZeroMQ.06:04
stubI've got rose coloured glasses on for ZeroMQ though, as I've not used it in anger and haven't gone over the ampq-oops work seriously to see how much of it we would have needed to do with ZeroMQ too.06:05
lifelessstub: disk swaps don't reset on proces death do they ?06:05
lifelessah, it does06:06
lifeless'The SWAP file is not recoverable, so if a publisher dies and restarts, it will lose data that was in its transmit buffers, and that was in the network I/O buffers.'06:06
lifelessstub: so, catching EPIPE is pretty shallow; it would be nice not to. We'd still need (for our use case) offline persistent queuing06:07
stubI think it has to do with the runtime state (next time you restart, your connections might be completely different and incompatible), and maybe security (if your spool is durable, you need to worry about who can read it rather than keep it in a private temporary file)06:07
lifelessmakes sense to me06:14
lifelessstub: OTOH I think its well encapsulated in oops-amqp (I'm about to cleanup the stuff in my useoops lp branch to use the 0.0.2 improvements)06:16
lifelessstub: also once we are on -a- mq, I think its quite easy to transition, message-pair at a time06:16
stubI suspect we will want to move some of this encapsulation to lazr.amqp or something for reuse. lazr.ipc? lazr.messaging?06:17
stubYer. I doubt I could argue a switch to ZeroMQ at this point. The Rabbit APIs bug me though. Weird terminology and promote ugly code.06:18
lifelessyeah. There is some stuff in lp.servicces.messsaging, but its a little to keen to be zope integrated to be reusable (plus some of it doesn't carry its own weight)06:18
lifelessI so want the compose-key-combo for theta06:19
lifelesserm, not theta06:19
lifelessthe greek letter whose name escapes me that they use06:19
wgrantø?06:20
lifelessyes06:20
wgrantØ06:20
wgrantAlt+/ O06:20
lifelessno06:20
wgrants/Alt/Compose/06:20
lifelessø woo06:20
lifelessyeah, its my left windows key06:20
wgrantI use RAlt, since LSuper is for Unity.06:21
stubI refuse to perpetuate that annoyance and promote ZeroMQ :-)06:22
lifelessøøø06:22
wgrantøMQ does look reasonably nice, but I'm a bit concerned that they promote it with a video from a PHP conference.06:23
wgrantOh.06:24
wgrant#is has a better one.06:24
wgrant06:24
wgrantAlthough it's not monospaced :/06:25
stubIt is a tool though for writing sane PHP since your PHP can easily become a thin layer talking to a middle layer written in something else :-)06:26
lifelesswgrant: compose for that ?06:26
wgrantlifeless: No clue.06:26
stubØMQ is fine06:26
wgrantI wonder if xkeyboard-config knows.06:27
wgrant∅MQ fails in a monospace font.06:27
wgrantNeed a smallish space..06:27
stubØMQ looks better than ∅MQ in my non-monospace font too.06:28
wgrantWe really need to get rid of embedded JS.06:30
huwshimiwgrant: You mean all the inline JS? We certainly do need to do that06:35
wgrantYes.06:35
huwshimiwgrant: We really need a global dom ready and a place for all the js to exists06:35
wgrantIt's hideous and buggy and blah.06:35
wgrantAnd TAL makes it even worse.06:35
huwshimisure is06:36
huwshimiwgrant: Fixing that up should have a small speed improvement too06:37
wgrantIndeed.06:38
poolielifeless previously said that .suspend on a bmp diff job would cause it to retry later07:50
pooliebut is that true07:51
poolieit looks a bit more like 'suspend' means paused indefinitely07:52
adeuringgood morning08:02
jelmerhi Abel08:02
wgrantpoolie: The solution should be clear.08:06
wgrantJust schedule a BranchMergeProposalUpdatePreviewDiffUnsuspensionJob!08:06
wgrantSorry, BranchMergeProposalUpdatePreviewDiffJobUnsuspensionJob08:07
poolie:)08:07
pooliei think i can add it to the list of retriable exceptions08:08
mrevellHallo08:08
pooliehi there08:09
=== almaisan-away is now known as al-maisan
=== jtv1 is now known as jtv
pooliemrevell: https://dev.launchpad.net/LEP/SSH_OAuth08:52
wallyworld_so why does the web service interface hate me so? i have a LaunchpadWebServiceCaller instance, i have logged in, but when i try invoking a delete method exposed via  @export_destructor_operation,  i get a 405  and only GET PUT PATCH POST allowed08:57
wallyworld_and it swallows the error so i have to hack in debugging to see the response08:57
lifelessits not personal08:58
lifelessit hates everyone equally08:58
wallyworld_:-) so how so i get it to allow me to invoke a http delete?08:58
wallyworld_i see for example there's a story test for deleting branches using a similar bit of code but there's no exposed branch delete method that i can see, so i'm not sure how that works09:01
rvbaadeuring: Hi, I'd like to ask you another question about lazr.batchnavigator if you have 5 minutes.09:17
adeuringrvba: sure09:17
rvbaThis time it's a *real* question.09:17
rvbaI'm working on bug 872086.09:17
_mup_Bug #872086: Distribution:+ppas and Distribution:+questions issue unlimited queries when memo=0&direction=backwards <regression> <timeout> <Launchpad itself:Triaged> <Storm:In Progress by rvb> < https://launchpad.net/bugs/872086 >09:17
adeuringrvba: well, your last one wass quite real  too ;)09:17
rvbaI've done the Storm (trivial) fix but I wonder if I should try to also add a small fix to the batchnavigator like I first envisioned.09:18
* adeuring is reading the bug report...09:18
rvbaMy idea was to try to improve how the batchnavigator deals with empty slices (like res[x:x]) to avoid issuing a query.09:18
rvbaBut, reading the code in src/lazr/batchnavigator/z3batching/batch.py I see that the way it works, slicing with [start:end+1] to get cleverly get the information about whether or not there are other elements, will make this improvement rather "clumsy" ...09:20
rvbaadeuring: The storm fix will fix the problem so my question to you is: do you think I should try to improve the navigator or not? (after looking at the code I confess I'm not sure it's worth it).09:21
adeuringrvba: in general, it is of course a good idea to "short-cut" queries where you know that the result will be empty, like sequence[n:n]. The problem with StormRangeFactory is that it does not have any knowledge about the possible size of the result set: The SQL OFFSET clause is gone, so you don't have anything that would map to [N:N]. Or am I missing something?09:22
rvbaadeuring: right, that's why I was thinking that the fix could be in src/lazr/batchnavigator/z3batching/batch.py.09:23
rvbaadeuring: But I'm just reading this code for the first time so I might be wrong.09:23
adeuringrvba: this code is a bit messy: It mixes the old batching parameters, like start, with the new ones -- no realy clear separation09:24
adeuringrvba: The class is supposed to rely _only_ on memo values and on a requested batch size09:25
lifelessadeuring: rvba is talking about the List range factory09:25
lifelessadeuring: I think.09:25
adeuringlifeless: ah, right!09:25
adeuringrvba: ok, in that case, tweaking ListRangeFactory is of course reasonable09:26
* adeuring needs more coffee09:26
rvbaRight, ListRangeFactory, I confess this code messes with my brain a little ;)09:27
lifelessthats entirely my fault09:27
lifelessI took some confused code and made it more capable09:27
lifeless-> and more confused09:27
rvbahehe09:27
rvbaOkay, I'll see what I can do with ListRangeFactory, thanks lifeless, adeuring.09:28
=== jam1 is now known as jam
=== gmb changed the topic of #launchpad-dev to: Performance Tuesday! | https://dev.launchpad.net/ | On call reviewer: gmb | Critical bugtasks: 273
jtvgmb: hi there.  Could I grovel you for a review?  https://code.launchpad.net/~jtv/launchpad/bug-873421/+merge/7966209:58
gmbjtv: If you missed a preposition in that sentence, sure. If not, I don't know what being grovelled by you actually entails...10:06
jtvJust verbing my way around the language.10:07
jtvIt involves me grovelling.10:07
nigelbI think the grovelling comes if you refuse to review :P10:07
gmbjtv: That's fine, then. I didn't know if it was the Launchpad version of Thailand's annual scheduled coup.10:08
nigelblol10:09
gmbjtv: Approved with one comment, discardable as appropriate.10:35
gmbs/discard/disregard/g10:36
jtvgmb: thanks.  Perhaps you were thinking of “shovelling someone” rather than “grovelling someone” earlier.10:36
gmbHeh.10:36
* gmb -> adding tea to brain; bbiab10:41
jtvMaking an interface attribute doNotSnapshot does not affect our definition of API compatibility, right?  ISTR some programs could theoretically break because of it, but we considered such programs buggy.10:48
gmbjtv: Right, it shouldn't make a difference to API compatibility.10:50
jtvThanks.10:50
=== deryck_ is now known as deryck
deryckMorning, all.13:07
abentleyderyck: morning.13:07
lifelessbigjools: allenap: what would cause txlongpoll to timeout on 'make rune'13:29
lifelesss/rune/run/13:29
allenaplifeless: Blimey, let me see...13:30
lifeless'Exception: Timeout waiting for txlongpoll to start.'13:30
lifelessplus a trace back - > total output13:30
lifelesspresumably the fixture has a log in getDetails but we're not printing that13:31
bigjoolsmmm it tries to connect on the frontend port13:31
bigjoolsand won't continue until it connects13:31
bigjoolsor times out13:31
lifelessok; is that something I would need to fiddle, add to hosts, ... ?13:31
bigjoolsso I expect txlongpoll is failing to start up13:31
bigjoolscan you run it standalone?13:31
lifelessI don't know. how do I check?13:32
bigjoolsrun the binary!13:33
bigjoolswell, "executable"13:33
lifelesshelp me out here, its 0230 :)13:33
bigjoolsbin/twistd-for-txlongpoll13:33
lifeless[only awake due to unhappy baby]13:33
allenaplifeless: I suggest adding a try...except: print $fixture_details in TxLongPollServer.setUp() (in the LP code base).13:34
lifelessthat returns a twistd options page13:34
bigjoolsbin/twistd-for-txlongpoll  amqp-longpoll -f 9999 -u guest -a guest13:34
lifelessah! thanks13:34
bigjoolsunless the port that the fixture is using is conflicting with something...13:35
lifelessassert fallback is None or callable(fallback)13:35
lifelessI'm guessing I dont' have your fixed txlongpoll13:35
lifelessin this branch13:35
lifelessyah, trunk has 0.2.8->0.2.913:39
lifelessthanks!13:39
lifelessallenap: bigjools: does 'make run' run its own rabbit up ?13:52
=== flacoste changed the topic of #launchpad-dev to: Launchpad should be engineered like an AK-47 | Performance Tuesday! | https://dev.launchpad.net/ | On call reviewer: gmb | Critical bugtasks: 273
allenaplifeless: Yes, should do.13:54
lifelessallenap: yes, I see13:56
jcsackettwallyworld: you still actually around?14:10
wallyworldjcsackett: sadly yes14:10
wallyworldbut off to bed soon14:10
* rvba takes a look at the world clock.14:10
jcsackettrvba: he's up very late. :-P14:10
rvbaVery very late indeed.14:11
lifelesspish14:11
jcsackettlifeless: your habits can't be used to judge anyone else, for various reasons. :-P14:11
jcsackettalthough you're up late by most standards too. :-)14:12
nigelbActually, with lifeless its hard to judge if he's up late or awake eearly.14:12
jcsacketthm. that's *very* true.14:13
lifeless\o/ \o/ \o/ \o/ \o/ \o/14:15
lifelessinstant-oops-delivery end to end manual test is go.14:15
lifelesscode.launchpad.dev/++oops++ -> 'received OOPS-...' on the amqp2disk console (run with -v); paste that into oops-tools web UI and tada immediately visible.14:16
jcsackettnice!14:22
lifelessgnight14:45
rvbann14:46
flacostervba, allenap: i'm trying to find the cause of bug 830676, can you help me navigate the involved templates?14:54
_mup_Bug #830676: Regression in PPA AJAX build status indicators <critical-analysis> <regression> <Launchpad itself:Triaged> < https://launchpad.net/bugs/830676 >14:54
allenapflacoste: Do you mind if I leave you in rvba's hands? I'm meant to be on leave :)14:55
rvbaflacoste: sure.14:55
flacosteallenap: please be on leave!14:55
rvbaLet me have a look.14:55
flacostervba: i'm utterly unfamiliar with how this is all hooked up together14:55
flacostemy guess is that the change is done by doUpdate in archive-packages.pt14:56
flacostebut i don't see where doUpdate is called from14:56
flacosteor even if this is the right place to look14:57
rvbalooking14:57
flacostehmm, although i think this is for the expander part, not the portlet that the bug report talks about14:59
* rvba uploads a test package to a ppa to see the ajax request.15:02
flacostervba: ok, i think this is related to lp.soyuz.dynamic_dom_updater15:02
rvbaflacoste: sounds like the soyuz way indeed.15:06
flacostervba: do you know where the domupdater is hooked to a specific row in the build status column?15:06
flacosteah, update_build_status.js15:07
rvbaflacoste: The key is to provide the domUpdateFunction.15:08
flacostervba: right, i'm looking at how this is all hooked up, like everytime you disturb someone to ask questions, you start making progress yourself!15:08
rvbaflacoste: so that's good, let's continue ;)15:09
flacostervba: thanks for helping, i'll continue to investigate and bug you again if i get stuck :-)15:09
flacostervba: i need to hop on a call15:09
rvbak15:09
abentleygmb: flacoste thinks my work on reordering bugs via ajax may overlap with your work on loading comments incrementally via ajax.  Can we chat?15:19
gmbabentley: Certainly. I'll be free to talk in ~30mins if that's okay.15:21
abentleygmb: works for me.15:21
gmbabentley: Cool. I'll ping when I'm free.15:21
gmbabentley: I'm free whenever you are. What works for you, mumble or skype?15:55
abentleygmb: let's mumble in Orange 10115:55
gmbabentley: Okidoke. 1 sec...15:55
elmoa lot of the launchpad buildds are going down  for power work15:55
jtvgmb: still ocr'ing?  If so, https://code.launchpad.net/~jtv/launchpad/bug-870130/+merge/7969115:58
gmbjtv: Certainly; bear with me a little while and I'll take a look.15:58
jtvI often feel bad about hitting the same reviewer twice in one day, though nowhere near as much as for reviewing the same developer twice in one day.15:58
jtvIt's a good thing I enjoy watching them suffer, or I'd be depressed.15:58
jtvThanks gmb15:58
=== al-maisan is now known as almaisan-away
=== mtaylor_ is now known as mtaylor
abentleyderyck: We have form fields with ids such as "field.searchtext".  I'm having trouble retrieving them with YUI.one.  I think they may be invalid identifiers: http://www.w3.org/TR/CSS2/syndata.html16:23
deryckabentley, they are invalid.16:23
deryckabentley, there's some selector work around, but it escapes me now.  let me search.16:23
abentleyderyck: okay.16:23
abentleyderyck: [id=field.searchtext] seems to work.16:25
deryckabentley, ah right.16:26
deryckabentley, I was thinking of this construction:  Y.one(Y.DOM.byId('field.searchtext'))16:27
abentleyderyck: Y.one accepts a DOM node?  Weird.16:28
abentleyderyck: also, we seem to have a bunch of ids that are not unique, e.g. field.status.  Should I just give up and jam this stuff into the IJSONCache instead?16:29
deryckabentley, yeah.  turns it into the Y.Node form.16:29
deryckhmmm, thinking about that....16:30
deryckabentley, I think I would stick it in the cache, yes.  Just to be careful and specific.16:30
abentleyderyck: Okay, I'll do that.16:31
=== deryck is now known as deryck[lunch]
=== gmb changed the topic of #launchpad-dev to: Launchpad should be engineered like an AK-47 | Performance Tuesday! | https://dev.launchpad.net/ | On call reviewer: - | Critical bugtasks: 273
=== beuno is now known as beuno-lunch
gmbjtv: Approved.16:55
=== matsubara is now known as matsubara-lunch
jtvthanks gmb17:02
gmbnp17:02
gmbAnd with that...17:02
* gmb -> afk for the evening17:03
jtvThen next continent over is giving up?  Classic cue to stop working.17:07
Ursinhalol17:08
jtvhey Ursinha!17:09
Ursinhahey jtv!17:11
Ursinha:)17:11
=== deryck[lunch] is now known as deryck
nigelbjtv: heh.17:15
flacosteabentley, deryck: we should fix the the non-unique ids at some point17:32
abentleyflacoste: and make them valid ids, too.17:42
=== beuno-lunch is now known as beuno
jtvArrrgh buildbot failure.  Something about bzr and twisted, I think.17:58
dobeyhaha. launchpad /is/ engineered like an AK-47 no? jams all the time, and lubricated with vodka?17:59
jtvdobey: no, that's just barry.17:59
jtvAt least, I hear he jams a lot.  Not 100% sure about the vodka.18:00
jtvjelmer: do you have any idea what might have earned us the ongoing buildbot failure?  https://lpbuildbot.canonical.com/builders/lucid_lp/builds/1476/steps/shell_6/logs/summary18:03
jtvMost of the failing tests are related to something I touched the other day, but not the first one.  So I'm hoping that first one is closer to the root cause. :)18:04
jelmerjtv: hmm18:05
jelmerjtv: that's an odd one. I'm not aware of any other changes in this area. How long has this been failing?18:06
abentleyderyck: Having trouble finding the spot where the search params are defined.  Any suggestions?18:06
jtvjelmer: only just now, I think18:06
jtvIt looks like it might be transient, maybe timing-sensitive.18:06
deryckabentley, are you looking for lp.bugs.interfaces.bugtask.BugTaskSearchParams ?18:07
jtvallenap: nothing to do with your fix to un-hose our librarian tests?  https://lpbuildbot.canonical.com/builders/lucid_lp/builds/1476/steps/shell_6/logs/summary18:08
abentleyderyck: No, I18:08
abentleyderyck: 'm trying to figure out how it gets populated, so I can do it in the right place.18:09
=== jtv is now known as jtv-zzz
deryckabentley, lp.bugs.browser.bugtask.BugTaskSearchListingView.buildSearchParams ?18:11
abentleyderyck: Yes, but how does that get called?18:11
abentleyderyck: it gets supplied with extra_params, and I don't know how to get them.'18:12
deryckabentley, same file, look for search --> searchUnbatched --> buildSearchParams.  Or are you asking how extra_params gets supplied from that top function?18:14
=== matsubara-lunch is now known as matsubara
abentleyderyck: Yeah, I still need to know how extra_params get supplied to search().18:15
deryckabentley, IIRC, I don't think that's used much.18:16
deryckabentley, I'm looking around.18:16
abentleyderyck: I backtraced to eggs/zope.tales-3.4.0-py2.6.egg/zope/tales/expressions.py(211)_eval()18:17
deryckabentley, I don't see it actually used anywhere.  You have some error, I take it, where you see extra_params being passed in?18:23
abentleyI don't know if extra_params is used anywhere, but I thought it was how the extra search parameters were getting into the page.18:23
deryckabentley, so I think buildSearchParams is just doing the work. and data gets populated with the self._validate call.  after that it's Zope magic to me.18:35
flacostegary_poster: WebserviceTestCase uses the AppServerLayer, wouldn't it be possible to use wsgi.intercept instead and run in the LaunchpadLayer?19:00
flacosteor even DatabaseLayer19:00
flacosteDatbaseFunctionalLayer19:00
flacosteand let users who need the librarian to upgrade their layer19:00
flacostegary_poster: i remember you adding support to use launchpadlib with wsgi.intercept for tests19:00
flacostebut it seems the default testcase base class for that (and that is getting significant traction by devs) isn't taking advantage of that19:01
gary_posterflacoste on call, off soon19:05
gary_posterflacoste, I did add wsgi.intercept for launchpadlib, yeah19:06
gary_posterlemme see where that was19:06
gary_posterflacoste, it is in the FunctionalLayer19:08
gary_posterSo in DatabaseFunctionalLayer19:09
gary_posterLaunchpadFunctionalLayer19:09
flacostegary_poster: so in theory, we should be able to change WebserviceTestCase to remove the layer attribute19:09
flacosteor change it to layer = DatabaseFunctionalLayer19:09
flacosteand everything should be honky dory?19:10
flacostegary_poster: ready on skype whenever you are19:13
gary_posterflacoste, or any of the non-appserver layers, yeah.  it expects to answer on "localhost:80" and "api.launchpad.dev:80"19:13
gary_posterok19:13
benjiDoes anyone want to review an exciting and well-written MP?19:19
* benji pretends as if he's not exaggerating.19:19
=== almaisan-away is now known as al-maisan
=== al-maisan is now known as almaisan-away
flacostebenji: if this is the stakeholders' bug i'm thinking it is, i can have a shot at it20:03
benjiflacoste: it probably is: https://code.launchpad.net/~benji/launchpad/bug-828572/+merge/7973320:03
flacostebenji: ok, i'm on it20:05
benjicool20:05
flacostebenji: looks very good, but i have two questions20:21
flacoste1) is there anything we should remove since the garbo migration is complete?20:21
flacoste(code)20:22
flacoste2) since any() in this case is all about status items, couldn't we use search_value_to_where_condition() instead of using recursion and joining with ' OR '?20:22
benjiflacoste: 1) probably; I was thinking the same thing but I was temted to wait until I can ask Brad when he gets back, but honestly it's very, very likely that we should just remove the code from garbo20:23
flacosteyes, if the transition is complete, we should remove it20:24
benjiflacoste:  2) I don't see an easy way because we have the odd-man-out of having a special case for INCOMPLETE; we could do it but the code would get quite a bit more complex20:25
benjiflacoste: that was odd, my internet connection dropped on me, did you see my response to 2?20:33
flacostebenji: i did, you basically mean that we'd need to support any() in search_value_to_where_condition20:37
flacostebenji: which it seems to support :-)20:38
flacostehmm20:38
flacosteactually20:39
flacosteno20:39
flacosteit supports a single any20:39
flacosteok, so that should be fine then20:39
benjiflacoste: no (although that may be true), I mean that since the user can provide something like any(NEW, INCOMPLETE) we would have to generate something like "status IN (1, 2, 3)", where 2 and 3 are the enum values for INCOMPLETE_WITH_RESPONSE and INCOMPLETE_WITHOUT_RESPONSE20:39
flacosteright20:39
flacostethat's what i understood20:40
flacostebenji: how about:20:42
flacosteif INCOMPLETE in status.query_values:20:42
flacoste   status = any([value for value in status.query_values if value != INCOMPLETE] + [INCOMPLETE_WITH, INCOMPLETE_WIHTOUT)20:43
benjiflacoste: I assume you mean for that to be inside the "if zope_isinstance(status, any)" clause.20:48
flacosteyep20:50
flacostebenji: feel free to use it or something like that if you feel it simplifies20:50
flacosteyour mp is approved anyway :-)20:50
benjiflacoste: thanks; I'm experimenting with a variation that might result in equally straight-forward code while also generating the slightly nice SQL you're looking for20:53
flacostethat'd be awesome20:53
abentleywallyworld_: flacoste has pointed out that your work on batchnavigator and mine on dynamic bug listings may overlap.  Can we chat?21:03
wallyworld_abentley: sure21:04
abentleywallyworld_: skype or mumple?21:04
wallyworld_either. let's try mumble21:04
benjiflacoste: it turned out quite nicely, thanks for the prod in that direction: http://paste.ubuntu.com/712484/21:14
flacostebenji: indeed!21:16
flacostethat's nice21:17
wallyworld_wgrant: https://code.launchpad.net/~wallyworld/launchpad/delete-bugtasks-1324/+merge/7954122:24
lifelessflacoste: wow you're churning out critical-analysis tags now :)22:52
lifelesssinzui: ping23:02
lifelesssinzui: I want to talk failfan with you23:02
lifelesssinzui: at a mutually convenient time23:02
lifelesssinzui: also, StevenK wgrant and I were talking about private teams & socialisation, without you :( [you were on public holiday I think :))] - I'd like to confirm the conclusions with you23:04
wgrantlifeless: He's out at the moment. Was planning to talk about private teams in 1.5 or 2 hours or so.23:11
wgrantfailfan?23:11
lifelessmailman23:11
lifelessbut I wanted to say fail23:11
lifelesswgrant: ok, well ping me (perhaps on skype) - I'm not here today, but meh... will swap things around on thurs/fri23:12
wgrantAh!23:12
lifelessI was working at 4am...23:15
wgrant... bad lifeless23:16
lifelessbut working amqp oopses23:16
lifeless\o/23:16
cjwatsonwgrant: so I've just been talking with slangasek about getting cron.germinate optimisation onto my official to-do list for 12.04, which will hopefully happen23:46
cjwatsonwgrant: it looks to me that its runtime is currently 9-10 minutes, 8 flavours * 5 architectures, and I would be surprised if I couldn't trim at least three-quarters of that23:47
wgrantYeah, it should be pretty cuttable.23:47
wgrantThat's excellent news.23:47
cjwatsonwgrant: that would mean that quite a few, but not all, publisher runs would fit within 30 minutes (looking at a sample of today)23:48
cjwatsonwhat do you think the threshold might be for switching to */30 runs?23:48
wgrantAnd we can cut it down further if we split partner out...23:48
cjwatsonalso parallelising ls-lR against other stuff, perhaps23:49
* wgrant finds a recent log.23:49
wgrantI really should get it bumped up to DEBUG.23:49
wgrantIt's a bit quiet at the moment.23:49
cjwatsonls-lR runtime isn't logged at the moment, no23:49
cjwatsonI had to watch it in action.  I didn't time that step specifically but it took a couple of minutes.23:50
cjwatsonactually from 23:27:00 to I think something around 23:30:4023:50
wgrantI had a detailed timeline in a Tomboy note a year or so ago. I wonder if that's still around, or if that was in the batch that U1 deleted...23:50
wgrantAh, there it is.23:51
wgrantI have ls-lR down as taking 2.5-3 minutes.23:51
wgrantGerminate 7, but we have more flavours now.23:51
cjwatsonand one extra arch23:51
wgrantI think this may have been before the powerpc/sparc elimination, but I don't quite recall.23:52
cjwatsonah, could be23:52
cjwatson(powerpc hasn't been eliminated though, maybe you mean lpia)23:52
cjwatson(or hppa)23:52
wgrantI meant ia6423:52
wgrantia64/sparc23:52
cjwatsonall these doorstops23:52
StevenKHaha23:52
wgrantI count sparc and powerpc as real archs, so I forgot about ia64.23:53
cjwatsonI haven't measured (I know ...) but it strikes me that ls-lR is I/O-bound while at least cron.germinate (and possibly other finalize.d type stuff) really aren't23:54
wgrantThat would indeed make sense.23:54
cjwatsonand cocoplum has four CPUs ...23:54
cjwatsonIs splitting out partner a near-future kind of thing?23:56
cjwatsonor merely priority High?23:56
wgrantNear future doesn't even come from Critical importance.23:56
wgrantit comes from someone doing it in violation of policy :)23:56
cjwatsonwell, LP queue ordering policy doesn't apply to external contributors so no violation involved :-)  but I don't think I'd know where to start23:57
wgrantI'm not sure that partner actually takes more than a few seconds, though, so it might not be worth it now that it's all in Python.23:58
mwhudsondata data data23:58
cjwatsonmm, I don't see much in the log I can clearly attribute to it23:58
wgrantPreviously partner had ~a minute of LP process startup time. Now it's all in one process, so it's almost free.23:58
wgrantI'm getting data :)23:58

Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!