| poolie | hi | 00:20 |
|---|---|---|
| wgrant | Morning poolie. | 00:20 |
| wgrant | They're multiplying. | 00:22 |
| poolie_ | poolies? | 00:23 |
| wgrant | That's the one. | 00:23 |
| wgrant | Or two. | 00:23 |
| huwshimi | wgrant: And now there are none | 00:31 |
| wgrant | :( | 00:39 |
| wgrant | jtv revived Zopeless stuff in his megalint again. | 00:40 |
| StevenK | Oh, bleh | 00:40 |
| StevenK | nigelb: 1) Your branch doesn't seem to include any changes; 2) It should be targetted against db-devel | 00:42 |
| wgrant | 46 conflicts encountered. | 00:42 |
| StevenK | wgrant: Whee | 00:42 |
| StevenK | Sigh, it does include changes. | 00:43 |
| StevenK | MPJ has lost its mind again | 00:43 |
| wgrant | StevenK: Link? | 00:43 |
| StevenK | wgrant: https://code.launchpad.net/~nigelbabu/launchpad/kill-statusexplanation/+merge/77211 | 00:43 |
| wgrant | StevenK: It doesn't have any changes. | 00:44 |
| wgrant | It was merged then reverted. | 00:44 |
| wgrant | nigelb will need to revert the revert. | 00:45 |
| StevenK | 41- var warning = overlay.one('.private-bug-warning'); | 00:45 |
| StevenK | 42+ warning = overlay.one('.private-bug-warning'); | 00:45 |
| StevenK | lol? | 00:45 |
| StevenK | Oh, let me guess. In JavaScript 'warning' is a reserved word? | 00:46 |
| nigelb | wgrant: wait, revert the revert? | 02:05 |
| nigelb | thta sounds convoluted :) | 02:05 |
| StevenK | nigelb: It's true | 02:08 |
| wgrant | nigelb: Reverting in a DAG-based DVCS involves doing a reverse-cherrypick of the revison that you want to revert. | 02:21 |
| wgrant | nigelb: So to revert the revert, it's probably easiest to do a reverse-cherrypick of the reversion revision. | 02:21 |
| poolie | mwhudson, lifeless, huwshimi, i drafted https://dev.launchpad.net/LEP/SSH_OAuth based on my discussion with robert yesterday | 03:02 |
| poolie | not planning to run with it at the moment but thought i'd get it down | 03:02 |
| StevenK | So I have brutally hacked twisted to fix the text/html Content-Type pollution, do I tar it up and stuff it into the download-cache, or do I need to do more? | 03:07 |
| nigelb | wgrant: Let me try this. (lost power for an hour) | 03:07 |
| wgrant | StevenK: Is it the fix that they've applied to trunk? | 03:09 |
| StevenK | wgrant: It is | 03:09 |
| StevenK | wgrant: I had to apply it manually since patch couldn't work it out | 03:09 |
| wgrant | StevenK: you'll need to create the tarball using whatever mechanism we use for Twisted, add it to download-cache, update the version in versions.cfg, and hope that you got everything right. | 03:10 |
| StevenK | Yes, it's the "create the tarball using whatever mechanism we use for Twisted" that I'm concerned about | 03:10 |
| wgrant | I normally try a few different things, and then take the one that looks the most like the old tarball. | 03:11 |
| StevenK | It looks like a source tarball | 03:11 |
| mwhudson | poolie: nice | 03:12 |
| mwhudson | poolie: fwiw, ssh is extensible enough that we could define an auth_oauthtoken method or something | 03:13 |
| mwhudson | poolie: you'd have to use paramiko rather than openssh for that sort of game though | 03:13 |
| mwhudson | (tbh just using the token as a passphrase would seem to be ok) | 03:13 |
| StevenK | wgrant: Just tarring up what I had looks very similiar | 03:16 |
| * StevenK runs bzr add, bzr ci, because what could possibly go wrong | 03:16 | |
| wgrant | StevenK: Don't commit until you've tested it.. | 03:17 |
| StevenK | Test it how? | 03:18 |
| wgrant | Add it to versions.cfg, make, and see if LP starts | 03:20 |
| poolie | mwhudson, yeah adding a new mechanism was my first thought too | 03:21 |
| poolie | seems cleaner; robert suggested just pretending it's a password may be easier | 03:21 |
| poolie | perhaps also we would log in as mbp+oauth@bazaar.launchpad.net and do it there | 03:21 |
| poolie | bit kludgy perhaps | 03:21 |
| poolie | it's basicalyl bug 297398 | 03:22 |
| _mup_ | Bug #297398: using ssh keys to connect to bazaar.launchpad.net confuses users a great deal <confusing-ui> <lp-code> <Launchpad itself:Triaged> < https://launchpad.net/bugs/297398 > | 03:22 |
| poolie | mwhudson, ok that's all i'm planning to actively do on it | 03:24 |
| poolie | if you want to work on it i'll help though | 03:24 |
| mwhudson | poolie: i guess i should try to land my feature flags xmlrpc branch again | 03:25 |
| poolie | oh i thought it did land | 03:25 |
| poolie | that would be great in its own right | 03:25 |
| mwhudson | it did, and was reverted because it broke feature flags in the webapp | 03:25 |
| mwhudson | changing launchpad is hard :/ | 03:25 |
| poolie | :/ | 03:26 |
| poolie | at least this patch will make it incrementally easier | 03:28 |
| nigelb | mwhudson++ | 03:31 |
| nigelb | :) | 03:31 |
| poolie | probably somebody with more zope blood on their hands could have made flags provide global state in a better way than i did | 03:31 |
| * mwhudson runs ec2 land | 03:34 | |
| nigelb | Hrm, what's with me and db-devel. Second patch also seems to be having issues. | 03:34 |
| StevenK | It does? | 03:35 |
| poolie | does anyone disagree with me calling bug 858605 untestable? | 03:35 |
| _mup_ | Bug #858605: launchpad sends mail for people with an inactive account <mail> <qa-untestable> <Launchpad itself:Fix Committed by mbp> < https://launchpad.net/bugs/858605 > | 03:35 |
| StevenK | Bleh, my source tarball still proclaims itself as Twisted 11.0.0 | 03:37 |
| nigelb | Argh. How do I revert? | 03:37 |
| poolie | revert what? | 03:38 |
| nigelb | I need to revert a revert. | 03:39 |
| poolie | tell me more? | 03:39 |
| mwhudson | nigelb: find the revision that reverted your change | 03:39 |
| mwhudson | run bzr merge -r $rev..before:$rev . | 03:39 |
| nigelb | Well, I overeagerly landed a db-devel change, which wgrant reverted. I need to revert that revert. | 03:40 |
| nigelb | ah, I can find the revision number in the bug. | 03:40 |
| nigelb | Let me hunt it down. | 03:40 |
| nigelb | mwhudson: that didn't work. maybe my revno was wrong. | 03:44 |
| nigelb | What am I doing wrong - http://paste.ubuntu.com/698277/ | 03:46 |
| mwhudson | well, conflicts don't necessarily mean you are doing it wrong | 03:48 |
| mwhudson | nigelb: looks like you forgot the '.' at the end though | 03:48 |
| nigelb | ah | 03:49 |
| nigelb | mwhudson: Worked. Thanks! | 03:52 |
| mwhudson | nigelb: cool | 03:52 |
| nigelb | StevenK: do you need stub's ack to land that? | 03:58 |
| nigelb | Or can you ack it on the basis of the earlier "okay" | 03:58 |
| StevenK | nigelb: It is easier if you have DB sign-off again | 04:04 |
| nigelb | okay | 04:05 |
| nigelb | I wonder what timezone stub is living in today. | 04:05 |
| * StevenK renews his hatred of Twisted | 04:25 | |
| wgrant | StevenK: https://code.launchpad.net/~wgrant/launchpad/bug-860920/+merge/77271 desires your perusal. | 04:27 |
| nigelb | wgrant: Very dry branch name ;) | 04:28 |
| wgrant | nigelb: Yeah, I'm mostly adding code, unfortunately. | 04:28 |
| wgrant | But I will hopefully delete half of our test layers this weekend :) | 04:28 |
| nigelb | ... | 04:28 |
| nigelb | What's going away? Zopeless? | 04:28 |
| wgrant | Zopeless went away on Monday. | 04:29 |
| wgrant | But there are still *ZopelessLayer. | 04:29 |
| wgrant | I'm hopefully going to merge them into *FunctionalLayer. | 04:29 |
| StevenK | wgrant: Can you explain the changes to configure.zcml? | 04:29 |
| StevenK | The addition is fine, the removsls | 04:30 |
| StevenK | Sigh | 04:30 |
| wgrant | StevenK: DistributionSourcePackage was unique among IBugTarget implementers in that it named the attributes on IBugTarget explicitly, rather than just allowing the whole interface. | 04:30 |
| wgrant | Since I was adding a new attribute to the interface, rather than add another name I opted to follow the rest of the implementers and allow the whole interface. | 04:30 |
| StevenK | Which resulting in removing a whole slate of stuff from the ZCML. Right. | 04:31 |
| wgrant | Right, since you can't declare multiple permissions for the one name, even if those permissions are identical. | 04:31 |
| StevenK | wgrant: r=me | 04:32 |
| wgrant | Thankyou sir. | 04:32 |
| * StevenK goes back to bashing his head against Twisted or Zope | 04:33 | |
| wgrant | Oh? | 04:33 |
| StevenK | You know the latter already, the former is trying to get my tarball of Twisted to identify itself with a different version | 04:34 |
| StevenK | Can't find the arguments to setup.py, or even if that will help one iota | 04:36 |
| StevenK | And Twisted's build system is worse than ours. | 04:37 |
| === jtv-afk is now known as jtv | ||
| wgrant | wallyworld: Is the deval failure yours? | 04:58 |
| wgrant | devel | 04:58 |
| wgrant | https://lpbuildbot.canonical.com/builders/lucid_lp/builds/1414/steps/shell_6/logs/summary | 04:58 |
| wallyworld | wgrant: don't know. haven't seen it yet. let me look | 04:58 |
| wallyworld | wgrant: could be. but how did it pass ec2 i wonder | 04:59 |
| wgrant | wallyworld: It's possible that you conflicted with megalint-II | 05:00 |
| wgrant | eg. it revived a test that I deleted on Monday. | 05:00 |
| wallyworld | ah. oh well. i'll update from devel and fix | 05:00 |
| wgrant | Thanks. | 05:00 |
| wgrant | It may not be yours, but it certainly looks likely... | 05:00 |
| wallyworld | wgrant: curtis landed the branch for me late last night after i went to bed. i assume it was put through ec2 | 05:06 |
| wgrant | I saw he made a last-minute change. | 05:06 |
| wgrant | He appeared in the blamelist of the merge. | 05:06 |
| wallyworld | wgrant: passes locally with latest devel | 05:07 |
| wgrant | wallyworld: That is extremely unfortunate. | 05:08 |
| wallyworld | wgrant: yes. let me look at the source code vs the test failure output | 05:08 |
| wallyworld | wgrant: i get the same error locally if i comment out the settin gof the feature flag in the doc test | 05:12 |
| wallyworld | so an initial guess is that the feature flag was ignored by the doc test | 05:13 |
| wallyworld | even though it was set | 05:13 |
| wgrant | wallyworld: The doctest isn't run more than once? | 05:13 |
| wallyworld | not that i know of | 05:14 |
| wallyworld | but the part of the doc test that needs the ff is inside a set/clear block | 05:14 |
| wallyworld | wonder if it's a genuine issue or a spurious failure | 05:16 |
| wallyworld | wgrant: does it fail for you locally? | 05:17 |
| wallyworld | bin/test -vvct initial-bug-contacts.txt | 05:17 |
| wgrant | No :( | 05:19 |
| wallyworld | wgrant: so resubmit and hope for the best? | 05:22 |
| wgrant | wallyworld: Force the build when it fails? I guess so. | 05:23 |
| wallyworld | yeah, that what i meant sorry | 05:23 |
| wallyworld | maybe jenkins would build it ok :-) | 05:24 |
| wgrant | Heh | 05:25 |
| wallyworld | so ec2 is happy and it runs locally. stupid buildbot | 05:26 |
| wgrant | Wow. | 05:48 |
| wgrant | The rabbitmq people have made their build system even worse. | 05:48 |
| wgrant | 2.3.1 was already one of the most distro-hostile I've seen, and 2.5.0 is worse... | 05:48 |
| StevenK | wgrant: How so? | 05:50 |
| wgrant | StevenK: For starters, see http://hg.rabbitmq.com/rabbitmq-mochiweb/file/b41f8ab516d6/Makefile | 05:51 |
| wgrant | StevenK: Yes, the Makefile in the root directory of the package does one thing: include umbrella.mk from outside the package. | 05:52 |
| wgrant | The rabbitmq-mochiweb test suite is also convinced that it wants to build its own rabbitmq-server. | 05:53 |
| wgrant | Because dependencies are hard, I guess. | 05:53 |
| wgrant | They seem to have basically rewritten the build system twice between 2.3.1 and 2.5.0, so my old fixes don't work :( | 05:53 |
| wgrant | Ahh. | 05:54 |
| wgrant | The plugin in Ubuntu (rabbitmq-stomp) works around this by not running the test suite. | 05:54 |
| wgrant | But I got it working with 2.3.1... so I will do it again. | 05:54 |
| wgrant | Perhaps all Erlang developers are crazy. | 05:54 |
| StevenK | I'd agree with that. | 05:55 |
| StevenK | For starters, they use Erlang. | 05:55 |
| wgrant | Meh, Erlang has some pretty nice bits. | 05:55 |
| wgrant | Why will people not stop writing their own shitty application-specific packaging systems!? | 06:01 |
| wgrant | Oh cool, they're now including webmachine stuff in rabbitmq-mochiweb. | 06:05 |
| * wgrant sobs. | 06:05 | |
| * jtv consoles wgrant | 06:12 | |
| jtv | "There, there. This is all part of growing up. Eventually you'll learn that it all goes away for a while after large quantities of beer." | 06:12 |
| jtv | Yeah, I know. I'm a natural at this. | 06:14 |
| wgrant | Heh | 06:14 |
| * StevenK kicks LaunchpadSearchView for being horrid. | 06:16 | |
| nigelb | jtv++ | 06:34 |
| jtv | nigelb: that returns the old value of jtv. | 06:35 |
| jtv | Although it does increase me, for which I am grateful. | 06:35 |
| nigelb | heh | 06:35 |
| jtv | Now, what are we talking about? | 06:35 |
| nigelb | Beer? | 06:35 |
| jtv | Ah! | 06:35 |
| === jtv1 is now known as jtv | ||
| === almaisan-away is now known as al-maisan | ||
| adeuring | good morning | 07:37 |
| === danilos changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: StevenK, danilos | Critical bugs: 261 - 0:[########*** stack smashing detected ***: ./lp terminated | ||
| lifeless | wgrant: how do you build txlongpoll from tarball? Perhaps julian means build from branch ? | 07:55 |
| wgrant | lifeless: What might Julian mean where? | 07:56 |
| lifeless | https://bugs.launchpad.net/python-oops-twisted/+bug/859545 | 07:56 |
| _mup_ | Bug #859545: Fallback code corrupts the output <python-oops-twisted:Triaged> < https://launchpad.net/bugs/859545 > | 07:56 |
| wgrant | lifeless: I presume he means from the branch, rather than an egg. | 07:57 |
| rvba | lifeless: I also think he meant from the branch. Also note that we renamed the generated executable from twistd to txlonpoll. | 07:58 |
| wgrant | rvba: Hm, why? | 07:58 |
| wgrant | That seems very confusing. | 07:58 |
| rvba | huwshimi: Hi! Thanks for the longpoll icons. I'll see what the others have to say but I like them :). | 07:59 |
| lifeless | there is a separate txlongpoll executable that just runs it isn't there? | 07:59 |
| rvba | lifeless: that's is still in the works AFAIK. | 08:00 |
| rvba | wgrant: why confusing? The goal was to avoid clashing with other twistd services. | 08:00 |
| wgrant | rvba: 'bin/txlongpoll amqp-longpoll' is more confusing. | 08:01 |
| wgrant | twistd is slightly more obvious. | 08:01 |
| lifeless | the problem will be file collisions I suspect | 08:01 |
| wgrant | When included in LP, sure. | 08:01 |
| rvba | True | 08:01 |
| wgrant | But that's a matter for LP's buildout.cfg. | 08:01 |
| wgrant | Not txlongpoll' | 08:01 |
| wgrant | s. | 08:01 |
| wgrant | wallyworld: Hm. | 08:04 |
| wgrant | wallyworld: My ec2 run has just failed with that error... | 08:04 |
| wallyworld | wtf? | 08:05 |
| * wgrant runs the whole layer locally. | 08:05 | |
| huwshimi | rvba: No problems! | 08:08 |
| rvba | huwshimi: one remark: | 08:08 |
| rvba | huwshimi: allenap noted that the failure icon looks a little bit like the Ubuntu Circle of Friends. | 08:09 |
| huwshimi | rvba: Ah, good point. I might make it a little less like that :) | 08:09 |
| huwshimi | rvba: I'm just heading off, if you have any other comments can you send me an email and I'll get to it tomorrow. | 08:10 |
| allenap | huwshimi: Fwiw, I like the spinning circle :) | 08:10 |
| huwshimi | allenap: OK cool :) | 08:10 |
| huwshimi | night all | 08:10 |
| StevenK | jml: O hai | 08:13 |
| jtv | cjwatson: are debian imports working normally for you? We got gina's run time back down to half an hour or so, and everything goes through proper domination now. Packages deleted from Debian also have their publications deleted. | 08:14 |
| cjwatson | jtv: the last thing I uploaded to Debian (yesterday evening) has been imported, which is about all I can say really | 08:32 |
| lifeless | bigjools: looking at txlongpoll | 08:32 |
| cjwatson | hmm | 08:32 |
| lifeless | bigjools: why do you have a custom log *and* twistd.log ? | 08:32 |
| cjwatson | jtv: https://launchpad.net/debian/+source/debian-installer-utils - is the 1.84 "not published" there normal? | 08:32 |
| cjwatson | on the front page? | 08:32 |
| jtv | cjwatson: I'll have to check up on that -- otp now, gimme a few | 08:33 |
| wgrant | cjwatson, jtv: gina wasn't setting datepublished for a while. We should set it to datecreated where it's not set. | 08:37 |
| wgrant | bigjools: ppa:wgrant/rabbitmq has rabbitmq-management 2.5.0 for oneiric now. I had to disable the full functional tests, since the new plugin build system is even more distro-hostile than the old one, but the unit tests all pass and it works fine. | 08:39 |
| bigjools | lifeless: are you running it with -n ? | 08:41 |
| bigjools | wgrant: \o/ | 08:42 |
| lifeless | bigjools: yes | 08:42 |
| bigjools | wgrant: awesome, thanks | 08:42 |
| bigjools | lifeless: that buggers up logging for some reason | 08:42 |
| lifeless | bigjools: not at all | 08:43 |
| jtv | cjwatson, wgrant: it does indeed look like datepublished was simply not set at that point (should've been done in May for this one I guess). More interesting is that 1.84's publication in sid is Published; that should only be the case if 1.84 is still occurring in sid's Sources.gz. | 08:43 |
| lifeless | bigjools: -n makes it log to stdout, and disables the default twistd.log | 08:43 |
| lifeless | bigjools: if you run without -n, you get *two* logs | 08:43 |
| bigjools | ah ok | 08:43 |
| bigjools | lifeless: in that case, the FileLogObserver is not being added correctly | 08:43 |
| lifeless | bigjools: its because the default setup includes another observer | 08:44 |
| lifeless | bigjools: see https://bugs.launchpad.net/txlongpoll/+bug/861234 | 08:44 |
| bigjools | I think I need to add my code back that adds it separately | 08:44 |
| _mup_ | Bug #861234: logs to both custom log and twistd.log <txlongpoll:Triaged> < https://launchpad.net/bugs/861234 > | 08:44 |
| lifeless | bigjools: the mylog observer is being added just fine | 08:44 |
| bigjools | lifeless: yeah, the bug describes what I meant :) | 08:44 |
| lifeless | I have fixed the extra spaces thing | 08:45 |
| lifeless | I can't reproduce the other failure yet | 08:45 |
| lifeless | bug 860490 | 08:45 |
| _mup_ | Bug #860490: oops reports with no type/value crash the fallback reporting code <python-oops-twisted:Triaged> < https://launchpad.net/bugs/860490 > | 08:45 |
| bigjools | that line used to be in the code but I took it out by mistake when I converted to using oops_twisted | 08:45 |
| bigjools | that's weird | 08:45 |
| lifeless | bigjools: https://bugs.launchpad.net/python-oops-twisted/+bug/860490/comments/6 | 08:47 |
| _mup_ | Bug #860490: oops reports with no type/value crash the fallback reporting code <python-oops-twisted:Triaged> < https://launchpad.net/bugs/860490 > | 08:47 |
| bigjools | lifeless: I can't figure out wtf is wrong then, it's 100% reproducible here | 08:48 |
| bigjools | lifeless: when I break on that method, the first call succeeds, the 2nd fails | 08:48 |
| bigjools | with the deferred failure | 08:48 |
| lifeless | it gets called once per connection attempt for me | 08:49 |
| bigjools | there's the difference then | 08:49 |
| bigjools | I exepct you're getting a different failure | 08:49 |
| bigjools | have you got the same versions of everything I wonder | 08:49 |
| * bigjools otp in a bit | 08:50 | |
| lifeless | anyhow, I'm going to implement the defensive programming solution | 08:50 |
| bigjools | ok | 08:50 |
| bigjools | without the bare except I hope | 08:51 |
| lifeless | but we'll still need to figure out why you are not getting an exception data there | 08:51 |
| lifeless | because that will devalue your oops reports | 08:51 |
| wgrant | jtv: 1.85 is new today, so 1.84 still being around is not surprising. | 08:56 |
| bigjools | jtv, wgrant: I am about to initialise p-series on DF, are you guys done with all the long-running stuff you were doing on there? | 08:56 |
| jtv | I am. | 08:56 |
| wgrant | I'm not using it. | 08:56 |
| bigjools | ta | 08:56 |
| wgrant | So feel free to break it :) | 08:56 |
| jtv | Concur. | 08:56 |
| jtv | wgrant: whether it's surprising is one thing, but I'm interested in another: is it actually the case? | 08:57 |
| wgrant | jtv: Yes, both versions are in http://ftp.debian.org/debian/dists/sid/main/source/Sources.gz | 09:00 |
| jtv | Good! | 09:00 |
| jtv | cjwatson: you should also see older package releases becoming Superseded, but the domination we implemented for gina does support multiple live versions in one release / pocket / archive. | 09:01 |
| cjwatson | Right. (When are we going to get similar behaviour of keeping multiple sources around until everything builds, I wonder ... or, more importantly, multiple arch: all) | 09:01 |
| jtv | The underlying mechanics are there now, for both Gina's domination and regular domination. | 09:02 |
| wgrant | bigjools and I have some disagreements on how that should be done. | 09:02 |
| jtv | The hard part for regular domination is getting the right information about what versions should be considered live, and feeding it into those mechanics. | 09:02 |
| lifeless | bigjools: what try:except ? | 09:10 |
| lifeless | bigjools: 0.0.3 pushed | 09:13 |
| lifeless | bigjools: (and on pypi) | 09:15 |
| bigjools | ta | 09:15 |
| lifeless | bigjools: we need to sort out whats up still, and as I can't reproduce yet, that means stepping through some with you :) | 09:16 |
| bigjools | ok will need to be tomorrow | 09:16 |
| lifeless | bigjools: sure hting | 09:17 |
| bigjools | (still otp) | 09:23 |
| bigjools | jml: do you know if there's a way to stop a Deferred firing until a write transaction is finished? | 09:25 |
| lifeless | bigjools: wrap it in a deferred that will tie into the transaction | 09:26 |
| bigjools | we want to block any deferred firing until a txn is done, so if a callback calls code that returns deferred we want the txn done before that deferred fires | 09:27 |
| lifeless | I don't see any possible way to do that | 09:28 |
| bigjools | we want to make sure txn boundaries don't cross callbacks, basically | 09:28 |
| jtv | We'd be fine if we had a way to stop twisted from giving any time to another thread of execution while we're in a given stretch of code. | 09:28 |
| lifeless | a deferred is equivalent to a function call - it would be like making *all* function calls in any thread block based on some other state | 09:28 |
| wgrant | bigjools: Aren't transactions restricted to inside the function? | 09:28 |
| bigjools | this is a problem in the buildd-manager | 09:28 |
| wgrant | bigjools: So a transaction won't be in progress when it returns to the reactor... | 09:28 |
| lifeless | bigjools: you could push the db access into the API and use that instead; thats the long term plan *anyway* (per SOA & data encapsulation) | 09:29 |
| jtv | The core problem, I think, is that we want to be sure in a particular stretch of code that we don't accidentally call something that will do anything deferred. | 09:29 |
| bigjools | lifeless: yes, but we need a very fast api first :) | 09:29 |
| lifeless | bigjools: we have one | 09:30 |
| wgrant | Also, pushing this to the API would just make things even worse. | 09:30 |
| lifeless | bigjools: the overhead for a single call is very low | 09:30 |
| wgrant | The problem is not that transactions are being held around. It's that we're doing data modification in places that we shouldn't be. | 09:30 |
| bigjools | lifeless: xmlrpc yes, webservice no | 09:30 |
| wgrant | Eliminating transactions from that just makes it harder to track down. | 09:30 |
| lifeless | bigjools: !cite - I've looked, and the webservice overhead is low | 09:30 |
| nigelb | lifeless: can do you have time to ack a db patch? | 09:30 |
| lifeless | bigjools: also, internally I'd be suggesting using xmlrpc *anyway* as we have that infrastructure setup. | 09:30 |
| bigjools | lifeless: not low enough for this | 09:31 |
| nigelb | s/can you/do you/ | 09:31 |
| lifeless | nigelb: no, I'm on annual leave today :P also, see policy - stub approves except when he's not here. | 09:31 |
| lifeless | bigjools: is 2ms low enough ? | 09:31 |
| bigjools | no | 09:32 |
| bigjools | that's slow | 09:32 |
| lifeless | bigjools: because thats the size it was looking like last time I compared the delta from xmlrpc to json api | 09:32 |
| bigjools | if it were 2us I'd be happy | 09:32 |
| lifeless | bigjools: that doesn't make any sense to me - we can't even query the db twice in 2ms. | 09:33 |
| nigelb | lifeless: oh. right. wednesday. Well, stub hasn't been here yet. Which is why I asked. I'll wait for him :) | 09:33 |
| bigjools | so you want to add more latency? | 09:33 |
| lifeless | bigjools: *if* it had to, I could tolerate 8ms per slave build, which would allow for 4 API calls in the critical path for dispatching and completing a build | 09:34 |
| lifeless | bigjools: but we're not going to get clarity by tossing sketches back and forth on IRC | 09:35 |
| bigjools | lifeless: I used to work in sub-millisencond latency over networks. If you take the Db out of the equation, replacing an internal python call with a 2ms latency remote call doesn't sound clever | 09:38 |
| jml | jtv: you can 'stop Twisted from giving any time to another thread of execution' by not using threads and doing a blocking call | 09:39 |
| jtv | jml: yes I had figured that bit out, thank you | 09:39 |
| bigjools | jtv: ^ that is why I said not to call them threads :) | 09:39 |
| jtv | (notice irony there) | 09:39 |
| lifeless | bigjools: of course, but you don't do a literal transcription just replacing db calls with api calls. | 09:39 |
| jml | jtv: ok. | 09:39 |
| lifeless | bigjools: that wouldn't work well for many reasons, transactions being one of them | 09:39 |
| jtv | I'm not talking about threads, of course, merely about threads of execution in twisted. | 09:39 |
| lifeless | bigjools: so measuring the performance by estimating overhead from a direct transcription isn't a good estimator | 09:40 |
| bigjools | lifeless: ok, this is a conversation better left for in-person I think | 09:40 |
| jml | jtv: I'm not interested in having an ironic conversation right now. | 09:40 |
| jtv | (Seeing how much taller than lifeless bigjools is) | 09:40 |
| jtv | jml: neither am I. What I'm trying to do is ensure that twisted does not give any time to another job deeper down in my call tree. | 09:41 |
| bigjools | lifeless: I am sure we can come to a good agreement somewhere, but it's hard to explain without waving arms around | 09:41 |
| lifeless | bigjools: yup, I get that | 09:41 |
| wgrant | wallyworld: bin/test -vvct initial-bug-contacts.txt -t bugtracker-tokens.txt | 09:41 |
| lifeless | jtv: don't return from your function then. | 09:41 |
| wallyworld | wgrant: you want me to run those? | 09:41 |
| lifeless | jtv: (and don't yield either) | 09:42 |
| wgrant | wallyworld: That reproduces the failure. | 09:42 |
| wgrant | wallyworld: We have a test isolation issue. | 09:42 |
| wallyworld | wgrant: bugtracker-tokens wasn't even changed in that branch | 09:42 |
| wgrant | wallyworld: Correct. | 09:42 |
| wallyworld | so somehow the ff is not being homoured | 09:42 |
| jtv | lifeless: I don't want to "just not do" the thing I want to prevent; I want to establish some certainty that it's not inadvertently happening somewhere deeper down in the call tree. | 09:43 |
| wallyworld | in the failing doctest | 09:43 |
| wgrant | wallyworld: Right, presumably bugtracker-tokens is corrupting the state. | 09:43 |
| wallyworld | that sucks :-( | 09:43 |
| wallyworld | has this happened before, the same way? | 09:43 |
| wallyworld | is there something we can recall that will help solve it? | 09:44 |
| wgrant | Not identical, but we do occasionally have stuff like this. | 09:44 |
| wgrant | I'm tracking down the problematic call. | 09:44 |
| wallyworld | ok | 09:44 |
| wallyworld | i'm hacking javascript. for the disclosure mockups | 09:44 |
| wallyworld | let me knoe if i can do anything | 09:45 |
| wgrant | Ah, it looks like it's XMLRPCTestTransport. | 09:45 |
| wgrant | That is an evil piece of code. I recall it from my Python 2.7 experiments. | 09:46 |
| jtv | lifeless: don't forget to "make lint" your branches! From a quick glance at your self-approved branch, I get the impression that there will be some warnings in there. | 10:21 |
| jtv | Or is it not a Launchpad branch? That'd explain why it ended up in my main mailbox. | 10:21 |
| jtv | stub: as promised... https://code.launchpad.net/~jtv/launchpad/pre-717969/+merge/77299 | 10:22 |
| stub | jtv: I'm not sure I like DatabaseTransactionPolicy having an implicit commit. It goes against your reasoning in the MP synopsis, and means you can't mix read-only and read-write code paths in a transaction. | 10:28 |
| jtv | stub: I feel the same, but didn't see support in the underlying APIs for a non-committing version. | 10:28 |
| jtv | For example, how do you reset policy when you're in a failed transaction? | 10:29 |
| stub | Why do you need it? | 10:29 |
| jtv | When an exception is taking you out of the policy, it needs to reset previous policy. But at that point the transaction may be broken. | 10:31 |
| jtv | I could abort at that point, start a new transaction, set policy, and commit -- but that brings us back to it being a transaction boundary. | 10:32 |
| stub | Sounds like we need to issue a transaction.abort() if we are exiting the context handler because of a psycopg2.Error | 10:32 |
| stub | Although a higher level could be handling it... | 10:33 |
| stub | blah | 10:33 |
| stub | Hmm... I wonder if we can setlocal instead? | 10:33 |
| stub | doesn't help in this case, but set local might be nicer anyway | 10:35 |
| * stub looks into savepoints | 10:36 | |
| stub | jtv: savepoints make this nicer, and you get better nesting behavior too | 10:37 |
| jtv | Are we talking about a savepoint in the postgres sense? | 10:38 |
| jtv | Nested transactions, basically? | 10:38 |
| stub | jtv: savepoint mytoken; set local default_transaction_readonly true; [...]; release savepoint mytoken; | 10:39 |
| stub | yet | 10:39 |
| stub | yer | 10:39 |
| stub | or rollback to savepoint mytoken | 10:39 |
| jtv | But that won't work if the transaction breaks, will it? | 10:40 |
| stub | It will rollback the 'set' too (per documentation on the SET statement) | 10:40 |
| stub | jtv: transaction breaks gets an implicit release of the savepoint | 10:41 |
| jtv | (Also, I couldn't swear that the big performance problems with savepoints were solved in 8.4 as opposed to 9.0 :) | 10:41 |
| stub | its all aborted and it is all rolling back until the code that wants to handle it starts a new transaction | 10:41 |
| bigjools | cjwatson: I just did a test run of initializing p-series on dogfood, do you want to check it? It seems to be ok to me. https://dogfood.launchpad.net/ubuntu/p-series | 10:41 |
| wgrant | bigjools: It used the cloner, not the copier, right? | 10:42 |
| stub | If there is a performance problem, we could consider a switch to turn it off on production. The goal of this code is to prove what sections of code are doing, so mainly applicable to test suite and maybe staging | 10:42 |
| bigjools | cloner | 10:42 |
| jtv | stub: I don't see nested transactions playing well with other transaction boundaries though. | 10:43 |
| stub | jtv: Yes, it won't play well if the code wrapped by the context manager does commit or rollback | 10:43 |
| jtv | There are commits and rollbacks in there already. We could perhaps get rid of those, but it makes me slightly more nervous. | 10:44 |
| stub | Implicit commits where previously there were no commits seems to be making the original problem worse rather than better (updates that should be atomic being made in separate transactions) | 10:46 |
| stub | So what happens if the context manager, if it catches a psycopg2.Error, does a transaction.abort(), reset policy, reraise exception? | 10:48 |
| stub | We could even break the transaction again after resetting the policy if we want to be totally transparent | 10:48 |
| jtv | That doesn't play very well with other transaction handlers. | 10:48 |
| jtv | Sorry: | 10:49 |
| jtv | with other *exception* handlers. | 10:49 |
| jtv | We could try it with a slight twist: | 10:49 |
| stub | I don't see why. The db transaction is broken. There is nothing to do with the db transaction except issue an abort | 10:49 |
| jtv | reset policy | 10:49 |
| jtv | if *that* raises a psycopg2 error, abort & reset policy. | 10:50 |
| jtv | And then commit. | 10:50 |
| stub | oic. yes, because the exception might have been caught inside but left the db connection broken | 10:50 |
| stub | Your idea seems sane to me | 10:50 |
| jtv | That worries me. | 10:50 |
| stub | You had to get a good idea eventually. | 10:51 |
| jtv | There is another problem, however. | 10:51 |
| jtv | Assuming the transaction was not broken, then what if the context outside of the policy breaks or aborts the transaction that we reset the policy in? | 10:51 |
| stub | Or does me thinking an idea sane cause you problems? ;) | 10:51 |
| nigelb | stub! Morning! | 10:51 |
| stub | Morning! | 10:52 |
| nigelb | stub: Could you review https://code.launchpad.net/~nigelbabu/launchpad/kill-statusexplanation-field-forever/+merge/77273 | 10:52 |
| jtv | (See why I never liked the primary use case for this python feature?) | 10:52 |
| jtv | nigelb: I saw him first! | 10:52 |
| stub | jtv: What feature do you need to solve this problem correctly? | 10:53 |
| nigelb | jtv: My review is quicker. Already approved once. Need a fresh approval for safety :D | 10:53 |
| jtv | stub: for that one, clairvoyance. Thanks. :) | 10:53 |
| jtv | nigelb: I should never have looked at your MP. :) | 10:53 |
| jtv | stub: it was more things like "figure out the state of the ongoing transaction" that could be solved with API extensions. But this one is really nasty. | 10:54 |
| jtv | There is completely different approach: set hooks client-side to manipulate every transaction as it's created. | 10:54 |
| nigelb | jtv: hahaha :D | 10:55 |
| stub | Just looking at psycopg2 docs.... connection.set_session(readonly=True)... API has become fatter in recent years with active development | 10:55 |
| stub | We could hook into the transaction easily enough... | 10:56 |
| stub | connection.get_transaction_status() seems useful | 10:56 |
| stub | or connection.status even | 10:57 |
| stub | http://initd.org/psycopg/docs/connection.html | 10:57 |
| jtv | Ah, thanks. I browsed around the python objects but didn't find that stuff. | 10:57 |
| stub | We might need to update our psycopg2 | 10:58 |
| stub | (which would be a good thing probably - it seems to have moved out of maintenance only to active development mode, but we only update it when we trip over crashers). | 10:59 |
| stub | nigelb: So what changed with that db patch? | 11:00 |
| jtv | stub: gah. set_session sounded so nice, but: "The function must be invoked with no transaction in progress." | 11:00 |
| nigelb | stub: Nothing. It was reverted because the devel changes weren't deployed everywhere. I un-reverted it. | 11:00 |
| nigelb | The code changes are now deployed everywhere. Or so StevenK tells me. | 11:01 |
| bigjools | wgrant: do you have any idea what port the rabbit management plugin uses for additional server instances? | 11:01 |
| stub | So it is the same patch and my mind isn't playing tricks on me | 11:01 |
| nigelb | No, it isn't :) | 11:01 |
| wgrant | stub, nigelb I ensured that 14041 was deployed everywhere yesterday. | 11:01 |
| wgrant | https://devpad.canonical.com/~wgrant/production-revnos | 11:01 |
| nigelb | wgrant: I owe you a beer. Or a few actually. | 11:02 |
| wgrant | bigjools: rabbitfixture disables it. | 11:02 |
| stub | wgrant: Saw that on the mailing list. Thanks! | 11:02 |
| bigjools | wgrant: what if it doesn't? | 11:02 |
| wgrant | bigjools: It'll fail to start, because it will try to use 55672 again. | 11:03 |
| nigelb | StevenK: OHAI, around? | 11:03 |
| === al-maisan is now known as almaisan-away | ||
| bigjools | wgrant: ok ta | 11:03 |
| wgrant | bigjools: You can configure it, however. | 11:03 |
| stub | wgrant: If you are looking for suggestions, map server names to meaningful Launchpad terminology (cronscripts, appservers, buildd-manager etc.) | 11:03 |
| wgrant | bigjools: And rabbitfixture can be given a list of plugins to whitelist. | 11:03 |
| bigjools | wgrant: it's a blocker for that test fix you had | 11:03 |
| rvba | wgrant: how can I enable it again (if I shutdown the machine's instance)? | 11:04 |
| nigelb | jtv: ah, can you land that MP? (want a link?) | 11:04 |
| wgrant | bigjools: http://www.rabbitmq.com/management.html describes how to configure the port. | 11:04 |
| stub | jtv: So does being able to get the connection status help us? | 11:04 |
| bigjools | wgrant: thanks | 11:04 |
| rvba | wgrant: tweaking rabbitfixture I guess. | 11:05 |
| jtv | stub: don't know, really -- set_session certainly doesn't seem to. | 11:05 |
| wgrant | rvba: Actually, I may never have landed the whitelisting support. For now you can just remove the RABBITMQ_PLUGINS_DIR fixture from rabbitfixture. | 11:06 |
| stub | jtv: I guess it is pretty much the same just trying the reset and if it fails with a db error abort the transaction and reset | 11:06 |
| rvba | wgrant: okay, thanks. | 11:06 |
| stub | jtv: connection.status is cleaner, but need to poke into Storm internals to get the psycopg2 connection | 11:06 |
| jtv | stub: I think one of the bigger problems is nested commits. | 11:06 |
| stub | Why? | 11:07 |
| jtv | Things get a bit complex for my liking if we try to do this without transaction boundaries: | 11:07 |
| jtv | set policy | 11:07 |
| jtv | register hook to repeat setting policy for next transaction | 11:07 |
| jtv | on exit: | 11:08 |
| jtv | reset policy | 11:08 |
| jtv | handle failure if any | 11:08 |
| jtv | unregister hook | 11:08 |
| stub | context manager does set session read_only, which persists over transaction boundaries. On exit, resets with set session read_only again. | 11:08 |
| jtv | Doesn't work with nested commits. | 11:09 |
| stub | Why not? | 11:09 |
| stub | oh... set session might not work inside a transaction... | 11:10 |
| jtv | Well according to the psycopg2 docs it doesn't, but even assuming that it does, there is some transaction weirdness there. | 11:10 |
| stub | I'm looking at the pg docs, and 'set session' inside a transaction block rolls back with the transaction | 11:11 |
| jtv | Right. | 11:11 |
| jtv | I think that was the weirdness that was bothering me: persists across transactions but rolls back with the transaction it's in. | 11:12 |
| jtv | Made it hard for libpqxx. :) | 11:12 |
| cjwatson | bigjools: not really sure exactly what to look at, but it looks ok from a preliminary glance | 11:12 |
| cjwatson | bigjools: shouldn't https://dogfood.launchpad.net/ubuntu/+source/man-db have p-series somewhere? | 11:12 |
| cjwatson | bigjools: or is that pending a publisher run? | 11:12 |
| bigjools | yeah pending publishing, and the series needs to be frozen | 11:12 |
| bigjools | cjwatson: what checks do you normally make? | 11:13 |
| wgrant | It's not pending publishing. | 11:13 |
| stub | jtv: "A special case is SET followed by SET LOCAL within a single transaction: the SET LOCAL value will be seen until the end of the transaction, but afterwards (if the transaction is committed) the SET value will take effect. " | 11:13 |
| wgrant | It's just because the series is Future. | 11:13 |
| cjwatson | I guess it would be interesting to try an initial publisher run on df and run compare-archives between oneiric and p-series | 11:13 |
| wgrant | So it doesn't show up on DSP:+index. | 11:13 |
| wgrant | https://dogfood.launchpad.net/ubuntu/+source/man-db/+publishinghistory | 11:13 |
| cjwatson | compare-archives being http://paste.ubuntu.com/698441/ | 11:13 |
| wgrant | cjwatson: Can't really do that on DF without spending 1.5 days publishing oneiric first :/ | 11:13 |
| stub | jtv: bah... but not rollback :-P | 11:13 |
| cjwatson | wgrant: oh dear | 11:13 |
| bigjools | cjwatson: that'd help if we already had all of oneiric published on DF :) | 11:13 |
| jtv | stub: not relevant, I hope -- "set transaction" is separate to "set" and inherently transaction-local. | 11:14 |
| wgrant | cjwatson, bigjools: Or we could just debmirror in a few minutes... | 11:14 |
| wgrant | Let's do that. | 11:14 |
| cjwatson | the interesting changes to initialisation were in things like copying custom uploads around, I thought | 11:14 |
| bigjools | disk space might be an issue | 11:14 |
| cjwatson | so that seems more interesting to check if we want to dry-run init | 11:14 |
| wgrant | bigjools: Not for one series and four archs of oneiric. | 11:14 |
| bigjools | shurg | 11:14 |
| bigjools | and shrug | 11:14 |
| jtv | stub: I hope this illustrates why I (1) went with commits here and (2) didn't buy the primary use case for python context managers. :) | 11:14 |
| cjwatson | shurg sounds like a star wars monster | 11:15 |
| bigjools | :) | 11:15 |
| jtv | wgrant: oneiric _is_ a series | 11:15 |
| wgrant | Shh | 11:15 |
| * jtv shhes | 11:15 | |
| bigjools | cjwatson: yeah copying custom AND auto-initialisation of all pockets on the first run | 11:15 |
| bigjools | I shall do a publication | 11:16 |
| bigjools | wgrant: you probably know more about debmirror than me, you can either do it or I can take monkey instructions | 11:16 |
| wgrant | bigjools: It's pretty trivial to use, with the minor issue that it's probably not installed on mawson. | 11:17 |
| stub | jtv: file:///usr/share/doc/postgresql-doc-8.4/html/sql-set-transaction.html , so 'set transaction' is the current transaction and 'set session characteristics as transaction' is future transactions. I don't know if the latter rollsback. | 11:17 |
| jtv | stub: interestingly, "set transaction" can switch back and forth during a transaction. I remembered it as working only at the beginning of one. | 11:17 |
| stub | jtv: The docs make you think that, but they are describing the isolation level. | 11:18 |
| jtv | You can happily commit a transaction with changes that is currently "set transaction"ed as read-only. | 11:18 |
| * jtv looks up the docs | 11:18 | |
| stub | set transaction to set the isolation level has to happen at the start. read only/read write is apparently allowed anywhere. | 11:19 |
| jtv | ah | 11:20 |
| stub | bah.... it rollsback too | 11:20 |
| stub | Can't seem to set state inside a transaction that survives a rollback | 11:20 |
| stub | Which would normally be a good thing. | 11:20 |
| jtv | Well IIRC there is one exception to that: | 11:21 |
| jtv | prepared statements. | 11:21 |
| jtv | Or maybe what upset me at the time was that they did roll back... forget now. :) | 11:21 |
| stub | ohh... nasty idea. | 11:21 |
| stub | nm... pointless nasty idea involving temporary tables. | 11:22 |
| jtv | Oh give it up. :) | 11:22 |
| stub | I don't see much use to this as it stands though, because issuing implicit commits is making our problem worse I think. | 11:22 |
| stub | And the work around is ugly and painful involving hooking into the transaction machinery | 11:23 |
| cjwatson | wgrant: but OTOH it is a fairly self-contained script | 11:23 |
| stub | jtv: Its pretty much like @write_transaction | 11:24 |
| jtv | stub: I don't see how. Same arguments as for read_transaction. | 11:25 |
| wgrant | cjwatson: It is. | 11:25 |
| wgrant | Well, it was. | 11:25 |
| wgrant | It may have done an sbuild and turned into a horrid Perl module mess since I last looked :) | 11:25 |
| stub | jtv: Yes, and we are not using read_transaction because of this behaviour, and the proposed solution also has this behaviour. | 11:25 |
| jtv | stub: read_transaction/write_transaction are half-hearted about it. At least what I've got here enforces the read-only property and supports nesting. | 11:26 |
| jtv | (Also, read_transaction/write_transaction could easily make a horrible mess if they have to retry ongoing transactions, AFAICS) | 11:27 |
| stub | jtv: Not sure if we want to support nesting though unless we also break commit/rollback | 11:27 |
| jtv | I hate the implicit commits, but... | 11:27 |
| jtv | For what I'm doing, yes, nesting would be very useful. | 11:28 |
| jtv | It's basically a complex, multi-function version of the second example I gave in the docstring. | 11:28 |
| stub | jtv: Implicit commits can lead to corruption and data loss. Can we work with an implicit rollback? | 11:29 |
| cjwatson | wgrant: no, I believe it's still pretty self-contained | 11:29 |
| jtv | stub: could you elaborate? | 11:30 |
| stub | Scattering new commits randomly around the codebase is dangerous as you end up committing things you were hoping to rollback. Even if you are careful, the next person might not be. Scattering new rollbacks randomly around the codebase is safer (but still not 'safe' as we might be dealing with non-db resources too). | 11:31 |
| stub | The problems with 'try: reset(): except psyocpg2.Error: abort(), reset()' also affect the current implementation don't they? | 11:33 |
| jtv | Just that one, AFAICS, and that's about the easiest problem we have here. | 11:34 |
| stub | If code inside the policy issues commits or rollbacks, the policy is no longer in effect. | 11:34 |
| jtv | Not in my code. | 11:34 |
| jtv | See the documentation & tests. | 11:34 |
| wgrant | Perhaps you want a decorator that monkeypatches out transaction.commit and transaction.abort. | 11:34 |
| jtv | go play outside! | 11:35 |
| stub | jtv: You only test it survives commit, not rollback | 11:35 |
| jtv | stub: You're preaching to the choir as far as implicit commits are concerned, but I'd much rather have well-defined transaction boundaries on this (with another curse in the direction of G. v. Rossum on this topic) than something that's complicated to follow. | 11:35 |
| jtv | stub: I can add a test case for that; won't affect the code. | 11:36 |
| stub | jtv: How will it work? Rollback rolls back the SET | 11:37 |
| jtv | stub: actually, I do test that case. | 11:37 |
| stub | If it works, the other solution works too | 11:37 |
| jtv | It's the very last test in the test case. | 11:37 |
| jtv | No, because my present solution can commit the SET. :) | 11:37 |
| jtv | "test_policy_survives_abort": "Even after aborting the initial transaction, a transaction policy still applies." | 11:38 |
| jtv | The point is to get well-defined behaviour. | 11:38 |
| stub | So we have two implicit commits | 11:38 |
| stub | I really don't see how this can be used safely. | 11:39 |
| stub | Hmm | 11:39 |
| jtv | There is one small tweak that could make it more palatable: abort at the end, rather than committing. | 11:40 |
| stub | The implicit commit at the start is the same. | 11:40 |
| stub | You might as well just put the connection into autocommit mode. | 11:41 |
| jtv | Not the same. You have an explicit statement there doing stuff with the database. | 11:41 |
| jtv | At least there you can read it and think "this will do something to the database." | 11:41 |
| jtv | Exit is the really nasty part. | 11:41 |
| stub | __enter__ would need to 'transaction.abort(); set' and __exit__ would need to 'transaction.abort(); set' | 11:41 |
| jtv | (See my line of reasoning against python context handlers as transaction managers at the time) | 11:41 |
| stub | Sorry... __enter__ transaction.abort(); set; commit() | 11:42 |
| jtv | What I tried to do, and couldn't make work unfortunately, was ensure that if a transaction is ongoing, it's "blank" | 11:42 |
| jtv | (in the sense of what SET TRANSACTION <isolation level> would accept) | 11:42 |
| jtv | If we can query a connection's transaction state, we could require an explicit commit/abort just prior to the context. | 11:43 |
| stub | That might be connection.status == TRANSACTION_STATUS_IDLE | 11:43 |
| jtv | Yeah. | 11:43 |
| jtv | Or get_transaction_status. | 11:44 |
| jtv | I believe connection.status returns a different enum. | 11:45 |
| stub | I think they are the same thing, just different spelling | 11:45 |
| stub | oh... different enums | 11:45 |
| jtv | An initial implicit commit becomes a lot more palatable if we can check that no transaction is ongoing. | 11:46 |
| jtv | Basically, who's going to notice an extra commit in that case? | 11:46 |
| stub | jtv: yes, that is fine. | 11:47 |
| jtv | We can turn the implicit final commit into an abort, raising a resolute middle finger to the whole context-manager design process. | 11:47 |
| jtv | Or better yet! | 11:47 |
| stub | jtv: And I think that helps solve the original problems. | 11:47 |
| jtv | "if exiting normally, assert that no transaction is ongoing." | 11:47 |
| stub | Yup. 'This chunk of code is called without an ongoing transaction and will commit or abort' | 11:48 |
| jtv | That's what I really wanted, but couldn't find the supporting API to implement. | 11:48 |
| jtv | And when you exit, it had better be either with an exception or after you have cleared out any open transactions. | 11:48 |
| stub | Nesting means a commit or abort would have to happen before the nesting happens, which is also good I suspect. | 11:48 |
| jtv | Yes, that's expected. | 11:49 |
| stub | I think this design is no longer dangerous and helpful. | 11:50 |
| jtv | Could you disambiguate that with some parentheses? | 11:50 |
| stub | (no longer dangerous) and helpful | 11:51 |
| jtv | Phew. Thanks. | 11:51 |
| jtv | stub: I can create a new exception type for "transaction still open," but do you happen to know of one I could reuse? | 11:55 |
| stub | RuntimeError ? | 11:55 |
| stub | I can't think of anything suitable that already exists | 11:55 |
| jtv | Isn't really a runtime error; more like a logic error. | 11:55 |
| jtv | I'll just create a new exception class then. | 11:56 |
| stub | Yer... just subclass Exception I guess | 11:56 |
| jtv | But now I need to leave! | 11:56 |
| stub | Night | 11:57 |
| jtv | stub: oh, and: read-only doesn't need an explicit commit really, does it? | 11:57 |
| stub | jtv: it does if you want it to persist over multiple transactions, especially if you hit code protected by the @read_transaction decorator. | 11:58 |
| jtv | I mean it would be safe to commit a read-only transaction implicitly at the end of the policy. | 11:59 |
| stub | flush first | 11:59 |
| jtv | Of course. | 11:59 |
| jtv | Or wait, not on commit. | 11:59 |
| jtv | On aborting a read-only one, flush. | 11:59 |
| jtv | (I had that in an earlier iteration) | 11:59 |
| jtv | I think abort is probably needlessly expensive for a read-only transaction. | 12:00 |
| stub | If you are in read only mode, abort or commit doesn't matter. You want to flush anyway, so ensure any exceptions that should be raised are raised. | 12:00 |
| jtv | Exactly. | 12:01 |
| stub | For read only mode, you can go with flush, abort, set, commit in the __exit__ | 12:02 |
| stub | Sorry. flush, confirm status, abort, set commit | 12:02 |
| jtv | Wasn't abort horribly expensive? | 12:02 |
| stub | Same for read write, since the inside is supposed to have committed or aborted already | 12:02 |
| stub | I don't know if abort is more expensive than commit. | 12:03 |
| stub | I suspect not if there are no changes, which there shouldn't be. | 12:03 |
| jtv | What I mean is, we don't need to require an explicit commit/abort for a read-only transaction. | 12:03 |
| jtv | For a read-write transaction, you want an explicit commit at the end of a successful run. | 12:04 |
| jtv | For a read-only transaction, why bother? | 12:04 |
| stub | point | 12:04 |
| jtv | Then again, it does keep the bracketing explicit. | 12:04 |
| jtv | I mean, having a transaction.commit() at the end of the policy's code region keeps bracketing explicit. | 12:05 |
| stub | So with read_only: stuff followed by with read_write: stuff, the second block would fail because we are already in a transaction | 12:05 |
| jtv | No, because the un-setting of the policy does need a commit. | 12:06 |
| jtv | (Which is implicit). | 12:06 |
| stub | oic | 12:06 |
| stub | Anyway, basic behavior of '__enter__ confirms no transaction in progress and sets persistent state' and '__exit__ confirms no transaction in progress and resets persistent state' is fine. And 'persistent state' means set followed by commit. | 12:08 |
| stub | jtv: But you were leaving | 12:08 |
| jtv | Trying to anyway. :) | 12:08 |
| jtv | The main tests are written & documented; can implement tomorrow. | 12:09 |
| stub | yup. | 12:09 |
| jtv | Thanks! | 12:09 |
| stub | I just hope connection.status or connection.get_transaction_status() does what we need :-) | 12:10 |
| === jtv is now known as jtv-afk | ||
| allenap | StevenK, danilos: Could either of you do a short review for me? https://code.launchpad.net/~allenap/launchpad/silence-amqplib-logger/+merge/77310 | 12:27 |
| danilos | allenap, I have a call in a few minutes, I'll pick it up after the call if it's still there :) | 12:28 |
| allenap | danilos: Ta. | 12:28 |
| === almaisan-away is now known as al-maisan | ||
| danilos | allenap, r=me, thanks for fixing this :) | 12:46 |
| nigelb | danilos: Hey, could you land something for me? Its already approved for db-devel. | 12:47 |
| allenap | danilos: Thanks! | 12:50 |
| danilos | nigelb, sure, though if it's the same thing benji was on a few days ago, I think he might get it sorted for you today :) | 12:51 |
| nigelb | danilos: Its another one :) | 12:54 |
| nigelb | https://code.launchpad.net/~nigelbabu/launchpad/kill-statusexplanation-field-forever/+merge/77273 | 12:55 |
| deryck | Morning, all. | 12:58 |
| danilos | nigelb, sure thing then | 12:59 |
| nigelb | danilos: Thanks! | 13:05 |
| nigelb | So, instead of one db patch, I'm landing 2 db patches in one day :) | 13:06 |
| danilos | nigelb, it probably means we'll have to serialize and roll one-by-one to production on consecutive days, but we'll see emails from people who do the rollout on the list :) | 13:08 |
| nigelb | danilos: heh :) | 13:10 |
| deryck | adeuring, abentley -- let's hold off on standup for 10 minutes. compiling some stuff to send you both that we need to chat about. | 13:28 |
| adeuring | deryck: ok | 13:28 |
| abentley | deryck: ack. | 13:28 |
| deryck | abentley, ready when you are. | 13:41 |
| === al-maisan is now known as almaisan-away | ||
| deryck | abentley, don't hear you. | 13:41 |
| abentley | deryck: same here. | 13:42 |
| deryck | abentley, we heard you then | 13:42 |
| stub | gary_poster: Here yet? http://paste.ubuntu.com/698496/ | 13:47 |
| gary_poster | stub, hi, yes. I was going to try to fiddle with what your email said. reading pastebin... | 13:48 |
| stub | gary_poster: So the first test passes, although I'm not sure if that is a good thing | 13:48 |
| gary_poster | heh | 13:48 |
| stub | gary_poster: The second test demonstrates a case we are not handling | 13:48 |
| stub | gary_poster: an OperationalError is raised if the Store is attempting to connect for the first time and the server isn't there. | 13:49 |
| stub | Which I guess is technically correct, as you can't disconnect unless you have previously managed to connect... | 13:49 |
| gary_poster | stub: for first test, what did you change to get things fixed (I see change in 23-29 but that is more tests, not a change to get things to pass) | 13:50 |
| stub | This would be the 'appserver bounced while the db is down' situation. We don't have to handle it this branch if you don't want to. | 13:50 |
| gary_poster | stub, ack for second test, thinking. | 13:51 |
| gary_poster | stub, won't we want to register a similar/identical view for OperationalError then? | 13:52 |
| gary_poster | That seems reasonable on the face of it | 13:52 |
| stub | gary_poster: Yes. We might see operational error in other more serious cases (like disk full), but 503 would be acceptable behaviour I think. | 13:52 |
| gary_poster | ok stub. I can do that easily enough. | 13:53 |
| gary_poster | stub, what am I missing from the first test though (test_error_view_integration)--did you initially dupe the problem I reported, and then fix it somehow? | 13:53 |
| stub | gary_poster: Yes. If you move the 'use_fixture' to before you register the wsgi stuff it explodes like you found. | 13:54 |
| gary_poster | Ah! | 13:54 |
| stub | I don't see why it should do that. It worries me. | 13:54 |
| gary_poster | how odd | 13:55 |
| gary_poster | yeah | 13:55 |
| gary_poster | ok, stub, thank you very much! This helps a lot. I'll proceed and ignore the worry myself unless you suggest otherwise. | 13:56 |
| deryck | adeuring, abentley -- ready on Skype when you guys are. | 13:58 |
| abentley | deryck: on. | 13:59 |
| deryck | abentley, you're showing offline for me and can't dial you. | 14:00 |
| abentley | deryck: 2 different versions of Skype hate me. | 14:01 |
| deryck | heh. | 14:02 |
| deryck | abentley, shall I try the regular number? | 14:02 |
| === aalex-sat is now known as aalex | ||
| flacoste | man, wadllib API sucks | 14:40 |
| flacoste | i've never seen such a convulated API | 14:40 |
| flacoste | and i probably reviewed some of it... *sigh* | 14:40 |
| nigelb | Historic moment of self realization? :) | 14:42 |
| cr3 | hi folks, has there been any interest in running launchpad with pypy to potentially improve performance? | 14:46 |
| benji | APIs (like code) don't age well | 14:46 |
| cr3 | benji: I don't think many things age well, even in nature things just die and are born again :( | 14:48 |
| benji | cr3: there's certainly interest but there are (at least) two negatives: we have lots of dependencies that don't work with pypy and LP isn't especially CPU bound (as far as I know) | 14:48 |
| benji | cr3: true, but we often think of digital (mathematical) constructs being immune from aging (like, say, Dijkstra's algorithm); IMO, code and APIs don't age well because we know more now than we did when we created them | 14:50 |
| cr3 | benji: same might apply to mathematics as well, consider hilbert threw a wrench in euclidian space. same reason as you mention though, "we know more now..." | 14:52 |
| cr3 | axioms change, just like in code, which breaks everything built on top of them | 14:52 |
| cr3 | apache had a great axiom of pre-forking but then we found that asynchronous sockets might be preferable in lots of cases | 14:54 |
| benji | this is true; I guess we need to embrace refactoring like Turritopsis nutricula (http://en.wikipedia.org/wiki/Turritopsis_nutricula) | 14:56 |
| cr3 | benji: to keep the analogy, I wonder what would be the equivalent for code to return in a polyp state... stable release? :) | 15:03 |
| benji | cr3: I was thinking more like a rewrite of a chunk of code such that it still performs its essential functions while shedding accumulated, non-essential functionality | 15:05 |
| cr3 | benji: I've always had difficulty with rewrites which others have most likely felt as well: people usually expect the same functionalities as before, sometimes even the quirky ones! | 15:07 |
| cr3 | benji: the problem is more about managing expectations than a technological one | 15:07 |
| benji | cr3: indeed | 15:08 |
| benji | I aspire to be minimalist, especially with code, and one of my mantras is "do less". | 15:09 |
| cr3 | benji: ironically, I've seen the amount of code I write increases with experience because I happen to know more about doing things right which is usually more verbose :( | 15:14 |
| jcsackett | sinzui: are you up for some mumbling? | 15:18 |
| sinzui | barely. I am ill and struggling to fix the broken test suite. | 15:18 |
| sinzui | I will start mumble in a few minutes when I get control of my computer | 15:19 |
| jcsackett | sinzui: sure. no worries. | 15:21 |
| gary_poster | stub, fwiw, I couldn't dupe your solution for the first test by rearranging the lines--I had made some name changes but it was otherwise identical. So, I'm thinking that solution may be to fragile to pursue after all. I've tried the three request solution (primary, secondary, clear) but that caused errors for subsequent tests (the ProgrammingError showed up for them). I've tried reconnect_stores() and helps me dupe | 15:27 |
| gary_poster | 't let me do a DisconnectError. | 15:27 |
| gary_poster | So, I'll dig in some more and send you an email at my EOD if I still don't have a solution | 15:27 |
| stub | gary_poster: garh | 15:31 |
| stub | gary_poster: Have you got anything more verbose out of that ProgrammingError? | 15:31 |
| gary_poster | no, stub. no. any hints on how to get more? | 15:32 |
| stub | Just wondering if our reporting is not outputting the informative string in the exception, or we are somehow getting a ProgrammingError('') | 15:32 |
| gary_poster | stub, I can instrument closer to the error | 15:33 |
| gary_poster | I'll check it out | 15:33 |
| stub | I can have a further poke tomorrow anyway at the underlying issues, but can't see how it is possible to get a ProgrammingError if the database is inaccessible, nor how to get one that only triggers when the PGBouncerFixture is loaded but not being connected through. | 15:34 |
| gary_poster | stub, it actually happens in one of my tracebacks in a rollback | 15:35 |
| gary_poster | thanks stub | 15:35 |
| stub | gary_poster: might be an untested codepath then? ProgrammingError means SQL syntax error, or attempting to access a table that doesn't exist or you have no permissions on, or something like that. | 15:36 |
| gary_poster | hm...I saw the pgbouncer sets up security | 15:36 |
| gary_poster | but I didn't know enough about the details to have an opinion or thought about it | 15:37 |
| gary_poster | the pgbouncer fixture, I should say | 15:37 |
| stub | possibly, if it is attempting to connect as some user not defined in security.cfg | 15:38 |
| stub | or the default user (your unix username) | 15:38 |
| gary_poster | stub, where the "it" attempting to connect is LP here, I guess? the bouncer itself is dead by the time we get the programming error, according to the bouncerlogs I looked at yesterday... | 15:40 |
| gary_poster | OK, I got more SQL than I shake a stick at...wading through... | 15:41 |
| gary_poster | hm, going to try that again with more info about whether there was an error... | 15:43 |
| gary_poster | yeah, SELECT getlocalnodeid() is the only interesting one | 15:47 |
| gary_poster | um | 15:47 |
| * gary_poster goes to take a break :-P | 15:47 | |
| === abentley changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: StevenK, danilos, abentley | Critical bugs: 261 - 0:[########*** stack smashing detected ***: ./lp terminated | ||
| === beuno is now known as beuno-lunch | ||
| === beuno-lunch is now known as beuno | ||
| === deryck is now known as deryck[lunch] | ||
| === deryck[lunch] is now known as deryck | ||
| lifeless | morning | 18:53 |
| nigelb | Morning lifeless | 18:56 |
| nigelb | benji: Hey, just checking if you were able to sort out your ec2 land troubles | 18:57 |
| nigelb | Also, \o/ my first db-devel patch landed | 18:57 |
| lifeless | nigelb: second surely? | 18:58 |
| benji | nigelb: I just now got evidence that it's working for me again. I'll resubmit your branch within the hour. (If you can easily remind me which one it is, that would be appreciated.) | 18:58 |
| nigelb | lifeless: second one is one the way :D | 18:58 |
| nigelb | benji: https://code.launchpad.net/~nigelbabu/launchpad/create-description-5283/+merge/76818 | 18:58 |
| lifeless | nigelb: wasn't the first 2 weeks back ? | 18:58 |
| nigelb | lifeless: well, that got reverted. | 18:59 |
| lifeless | right | 18:59 |
| nigelb | I just un-reverted it. | 18:59 |
| lifeless | still count s:P | 18:59 |
| nigelb | ha! | 18:59 |
| benji | I just got the same failure on two ec2 runs of different branches: http://paste.ubuntu.com/698675/. I can't reproduce the failure locally. Anyone seen this? | 19:26 |
| benji | nigelb: bad news, conflicts: http://paste.ubuntu.com/698689/ | 19:38 |
| tedg | Hello, I'm looking for where in the LP codebase the bugzilla sync code is. | 19:43 |
| tedg | Could someone give me a pointer? | 19:43 |
| lifeless | at a guess, lib/lp/bugs/ somewhere. I'd do a 'find . -name "*bugz*" | 19:45 |
| tedg | lifeless, Are bugz, the new hip bugs? ;-) | 19:46 |
| tedg | Heh, I just got an timeout oops browsing the LP source code ;-) | 19:47 |
| lifeless | yeah, loggerhead still needs a performance love-in | 19:50 |
| tedg | lifeless, Okay, found what I was looking for. Thanks! | 20:07 |
| === abentley changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: - | Critical bugs: 261 - 0:[########*** stack smashing detected ***: ./lp terminated | ||
| * mwhudson wonders, could ~launchpad-reviewers be set as the reviewer team for lp:txlongpoll? | 21:03 | |
| mwhudson | oh it is | 21:03 |
| mwhudson | then could ~lazr-developers _not_ be subscribed to all code review mail? | 21:03 |
| mwhudson | er, failures in lib/lp/bugs/doc/initial-bug-contacts.txt -- not my fault? | 21:04 |
| mwhudson | maybe not | 21:05 |
| mwhudson | bug 861510 | 21:06 |
| _mup_ | Bug #861510: XMLRPCTestTransport is not torn down properly <test-system> <Launchpad itself:Triaged> < https://launchpad.net/bugs/861510 > | 21:06 |
| _thumper_ | wallyworld_: ping | 21:18 |
| === _thumper_ is now known as thumper | ||
| mwhudson | ok, bug 861510 is very very screwed | 21:36 |
| _mup_ | Bug #861510: XMLRPCTestTransport is not torn down properly <test-system> <Launchpad itself:Triaged> < https://launchpad.net/bugs/861510 > | 21:36 |
| mwhudson | arughjkfsdafdaslkjhfalsdkjfh | 21:58 |
| mwhudson | omg omg | 22:04 |
| lifeless | mwhudson: cats? | 22:08 |
| mwhudson | lifeless: no | 22:08 |
| mwhudson | lifeless: thread locals | 22:08 |
| mwhudson | much more terrifying | 22:08 |
| lifeless | rather like regexes | 22:08 |
| mwhudson | lifeless: looking at | 22:09 |
| mwhudson | <mwhudson> lifeless: no | 22:09 |
| mwhudson | argh | 22:09 |
| mwhudson | https://bugs.launchpad.net/launchpad/+bug/861510 | 22:09 |
| _mup_ | Bug #861510: XMLRPCTestTransport is not torn down properly <test-system> <Launchpad itself:Triaged> < https://launchpad.net/bugs/861510 > | 22:09 |
| lifeless | yeah | 22:09 |
| lifeless | I saw it go past, decided to err on the side of sanity and close my eyes ;( | 22:09 |
| mwhudson | i don't get all the details | 22:09 |
| mwhudson | but the problem is that doing the XMLRPCTestTransport thing ends up calling set_permit_timeout_from_features(True) | 22:10 |
| mwhudson | and then everything else goes wrong | 22:10 |
| mwhudson | lifeless: in general i am becoming more of the opinion that all 'global-ish' things in launchpad should be explicitly tied to the request/participation | 22:33 |
| mwhudson | and using thread locals is just a recipe for confusiong | 22:34 |
| mwhudson | (mostly confusion in test suites rather than real code) | 22:34 |
| mwhudson | of course the interaction is a thread-local, but there's only one of it and we're in general _reasonably_ good about keeping its state in check | 22:35 |
| lifeless | mwhudson: IIRC we were both of that opinion, and firmly so, when I filed that bug about scripts and participations :) | 22:36 |
| mwhudson | yeah | 22:36 |
| mwhudson | i guess i'm even more of it now :) | 22:36 |
| mwhudson | lifeless: i think i hadn't realised how bad having lots of thread locals is | 22:36 |
| lifeless | and its viral | 22:37 |
| lifeless | once you have one its hard to not have another | 22:37 |
| mwhudson | i'd say it's more like when you have _two_ it's hard not to have another | 22:42 |
| mwhudson | i think it would be possibly to have one thread local to rule them all | 22:42 |
| lifeless | mmm | 22:45 |
| mwhudson | https://bugs.launchpad.net/launchpad/+bug/861510/comments/3 | 22:45 |
| _mup_ | Bug #861510: XMLRPCTestTransport is not torn down properly <test-system> <Launchpad itself:Triaged> < https://launchpad.net/bugs/861510 > | 22:45 |
| lifeless | I don't think 2 drives three more than 1 drives 2, TBH | 22:45 |
| wallyworld_ | jcsackett: hi | 22:49 |
| wallyworld_ | thumper: hi | 22:52 |
| thumper | wallyworld_: hi | 22:52 |
| thumper | wallyworld_: how's tricks? | 22:52 |
| wallyworld_ | how goes it? | 22:52 |
| wallyworld_ | busy | 22:52 |
| wallyworld_ | doing some mockups for disclosure, lots of javascript | 22:52 |
| wallyworld_ | finished improving pickers | 22:52 |
| thumper | cool | 22:53 |
| thumper | I have a minor feature request | 22:53 |
| wallyworld_ | how's dx? | 22:53 |
| thumper | :) | 22:53 |
| wallyworld_ | sure | 22:53 |
| thumper | when we attach diffs to outgoing code review and branch emails | 22:53 |
| thumper | can we utf-8 encode plz? | 22:53 |
| wallyworld_ | sure, can't see why not | 22:53 |
| thumper | we have people with weird names committing or .po file stuff et al | 22:54 |
| wallyworld_ | not surprised :-) | 22:54 |
| thumper | dx is interesting | 22:55 |
| thumper | so so much going on | 22:55 |
| thumper | almost scarey | 22:55 |
| wallyworld_ | good stuff? | 22:55 |
| wallyworld_ | i guess you are focused on squashing the last few oneiric bugs | 22:56 |
| wgrant | There's two weeks to go! Not the final stretch yet. | 22:58 |
| wgrant | Still got another 10 FF violations to land, probably :P | 22:58 |
| thumper | wallyworld_: final cut for upload is today | 23:00 |
| wallyworld_ | thumper: does it fix the stacking issues? | 23:00 |
| thumper | wallyworld_: the rest of the bug fixes are going in as distro patches :) | 23:00 |
| thumper | wallyworld_: almost all of them are fixed | 23:01 |
| wallyworld_ | \o/ | 23:01 |
| thumper | wallyworld_: apparently there are still and edge case or two out there | 23:01 |
| wallyworld_ | been driving me crazy | 23:01 |
| thumper | the big problem is it needs a fundamental refactoring to compiz internals | 23:01 |
| wallyworld_ | thumper: my main other gripe is with the global menu behaviour, but you know that :-) | 23:01 |
| thumper | X sucks | 23:01 |
| wallyworld_ | yep | 23:01 |
| thumper | yeah... I know that | 23:01 |
| wallyworld_ | i'm fining it's taking more mouse clicks to do what i want :-( | 23:02 |
| wallyworld_ | oh well | 23:02 |
| wgrant | wallyworld_: I'm guessing there's no standup today? | 23:03 |
| wallyworld_ | wgrant: don't know? | 23:03 |
| wgrant | Given sinzui's email. | 23:03 |
| wallyworld_ | i was hoping to catch up with jcsackett briefly | 23:04 |
| wallyworld_ | wgrant: if there's nothing in particular to discuss, we can skip it | 23:05 |
| wallyworld_ | thumper: do you know if "Show Desktop" on the launcher is broken? well it is for me | 23:43 |
| wgrant | Does that still exist? | 23:43 |
| wallyworld_ | on mine it does | 23:43 |
| wgrant | Maybe I just removed it and forgot... | 23:43 |
| wallyworld_ | i updated just now and its icon has changed but it still doesn't work | 23:44 |
| * wallyworld_ has to go run an errand | 23:47 | |
| poolie | hi all | 23:57 |
Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!