[00:20] hi [00:20] Morning poolie. [00:22] They're multiplying. [00:23] poolies? [00:23] That's the one. [00:23] Or two. [00:31] wgrant: And now there are none [00:39] :( [00:40] jtv revived Zopeless stuff in his megalint again. [00:40] Oh, bleh [00:42] nigelb: 1) Your branch doesn't seem to include any changes; 2) It should be targetted against db-devel [00:42] 46 conflicts encountered. [00:42] wgrant: Whee [00:43] Sigh, it does include changes. [00:43] MPJ has lost its mind again [00:43] StevenK: Link? [00:43] wgrant: https://code.launchpad.net/~nigelbabu/launchpad/kill-statusexplanation/+merge/77211 [00:44] StevenK: It doesn't have any changes. [00:44] It was merged then reverted. [00:45] nigelb will need to revert the revert. [00:45] 41 - var warning = overlay.one('.private-bug-warning'); [00:45] 42 + warning = overlay.one('.private-bug-warning'); [00:45] lol? [00:46] Oh, let me guess. In JavaScript 'warning' is a reserved word? [02:05] wgrant: wait, revert the revert? [02:05] thta sounds convoluted :) [02:08] nigelb: It's true [02:21] nigelb: Reverting in a DAG-based DVCS involves doing a reverse-cherrypick of the revison that you want to revert. [02:21] nigelb: So to revert the revert, it's probably easiest to do a reverse-cherrypick of the reversion revision. [03:02] mwhudson, lifeless, huwshimi, i drafted https://dev.launchpad.net/LEP/SSH_OAuth based on my discussion with robert yesterday [03:02] not planning to run with it at the moment but thought i'd get it down [03:07] So I have brutally hacked twisted to fix the text/html Content-Type pollution, do I tar it up and stuff it into the download-cache, or do I need to do more? [03:07] wgrant: Let me try this. (lost power for an hour) [03:09] StevenK: Is it the fix that they've applied to trunk? [03:09] wgrant: It is [03:09] wgrant: I had to apply it manually since patch couldn't work it out [03:10] StevenK: you'll need to create the tarball using whatever mechanism we use for Twisted, add it to download-cache, update the version in versions.cfg, and hope that you got everything right. [03:10] Yes, it's the "create the tarball using whatever mechanism we use for Twisted" that I'm concerned about [03:11] I normally try a few different things, and then take the one that looks the most like the old tarball. [03:11] It looks like a source tarball [03:12] poolie: nice [03:13] poolie: fwiw, ssh is extensible enough that we could define an auth_oauthtoken method or something [03:13] poolie: you'd have to use paramiko rather than openssh for that sort of game though [03:13] (tbh just using the token as a passphrase would seem to be ok) [03:16] wgrant: Just tarring up what I had looks very similiar [03:16] * StevenK runs bzr add, bzr ci, because what could possibly go wrong [03:17] StevenK: Don't commit until you've tested it.. [03:18] Test it how? [03:20] Add it to versions.cfg, make, and see if LP starts [03:21] mwhudson, yeah adding a new mechanism was my first thought too [03:21] seems cleaner; robert suggested just pretending it's a password may be easier [03:21] perhaps also we would log in as mbp+oauth@bazaar.launchpad.net and do it there [03:21] bit kludgy perhaps [03:22] it's basicalyl bug 297398 [03:22] <_mup_> Bug #297398: using ssh keys to connect to bazaar.launchpad.net confuses users a great deal

< https://launchpad.net/bugs/297398 > [03:24] mwhudson, ok that's all i'm planning to actively do on it [03:24] if you want to work on it i'll help though [03:25] poolie: i guess i should try to land my feature flags xmlrpc branch again [03:25] oh i thought it did land [03:25] that would be great in its own right [03:25] it did, and was reverted because it broke feature flags in the webapp [03:25] changing launchpad is hard :/ [03:26] :/ [03:28] at least this patch will make it incrementally easier [03:31] mwhudson++ [03:31] :) [03:31] probably somebody with more zope blood on their hands could have made flags provide global state in a better way than i did [03:34] * mwhudson runs ec2 land [03:34] Hrm, what's with me and db-devel. Second patch also seems to be having issues. [03:35] It does? [03:35] does anyone disagree with me calling bug 858605 untestable? [03:35] <_mup_> Bug #858605: launchpad sends mail for people with an inactive account

< https://launchpad.net/bugs/858605 > [03:37] Bleh, my source tarball still proclaims itself as Twisted 11.0.0 [03:37] Argh. How do I revert? [03:38] revert what? [03:39] I need to revert a revert. [03:39] tell me more? [03:39] nigelb: find the revision that reverted your change [03:39] run bzr merge -r $rev..before:$rev . [03:40] Well, I overeagerly landed a db-devel change, which wgrant reverted. I need to revert that revert. [03:40] ah, I can find the revision number in the bug. [03:40] Let me hunt it down. [03:44] mwhudson: that didn't work. maybe my revno was wrong. [03:46] What am I doing wrong - http://paste.ubuntu.com/698277/ [03:48] well, conflicts don't necessarily mean you are doing it wrong [03:48] nigelb: looks like you forgot the '.' at the end though [03:49] ah [03:52] mwhudson: Worked. Thanks! [03:52] nigelb: cool [03:58] StevenK: do you need stub's ack to land that? [03:58] Or can you ack it on the basis of the earlier "okay" [04:04] nigelb: It is easier if you have DB sign-off again [04:05] okay [04:05] I wonder what timezone stub is living in today. [04:25] * StevenK renews his hatred of Twisted [04:27] StevenK: https://code.launchpad.net/~wgrant/launchpad/bug-860920/+merge/77271 desires your perusal. [04:28] wgrant: Very dry branch name ;) [04:28] nigelb: Yeah, I'm mostly adding code, unfortunately. [04:28] But I will hopefully delete half of our test layers this weekend :) [04:28] ... [04:28] What's going away? Zopeless? [04:29] Zopeless went away on Monday. [04:29] But there are still *ZopelessLayer. [04:29] I'm hopefully going to merge them into *FunctionalLayer. [04:29] wgrant: Can you explain the changes to configure.zcml? [04:30] The addition is fine, the removsls [04:30] Sigh [04:30] StevenK: DistributionSourcePackage was unique among IBugTarget implementers in that it named the attributes on IBugTarget explicitly, rather than just allowing the whole interface. [04:30] Since I was adding a new attribute to the interface, rather than add another name I opted to follow the rest of the implementers and allow the whole interface. [04:31] Which resulting in removing a whole slate of stuff from the ZCML. Right. [04:31] Right, since you can't declare multiple permissions for the one name, even if those permissions are identical. [04:32] wgrant: r=me [04:32] Thankyou sir. [04:33] * StevenK goes back to bashing his head against Twisted or Zope [04:33] Oh? [04:34] You know the latter already, the former is trying to get my tarball of Twisted to identify itself with a different version [04:36] Can't find the arguments to setup.py, or even if that will help one iota [04:37] And Twisted's build system is worse than ours. === jtv-afk is now known as jtv [04:58] wallyworld: Is the deval failure yours? [04:58] devel [04:58] https://lpbuildbot.canonical.com/builders/lucid_lp/builds/1414/steps/shell_6/logs/summary [04:58] wgrant: don't know. haven't seen it yet. let me look [04:59] wgrant: could be. but how did it pass ec2 i wonder [05:00] wallyworld: It's possible that you conflicted with megalint-II [05:00] eg. it revived a test that I deleted on Monday. [05:00] ah. oh well. i'll update from devel and fix [05:00] Thanks. [05:00] It may not be yours, but it certainly looks likely... [05:06] wgrant: curtis landed the branch for me late last night after i went to bed. i assume it was put through ec2 [05:06] I saw he made a last-minute change. [05:06] He appeared in the blamelist of the merge. [05:07] wgrant: passes locally with latest devel [05:08] wallyworld: That is extremely unfortunate. [05:08] wgrant: yes. let me look at the source code vs the test failure output [05:12] wgrant: i get the same error locally if i comment out the settin gof the feature flag in the doc test [05:13] so an initial guess is that the feature flag was ignored by the doc test [05:13] even though it was set [05:13] wallyworld: The doctest isn't run more than once? [05:14] not that i know of [05:14] but the part of the doc test that needs the ff is inside a set/clear block [05:16] wonder if it's a genuine issue or a spurious failure [05:17] wgrant: does it fail for you locally? [05:17] bin/test -vvct initial-bug-contacts.txt [05:19] No :( [05:22] wgrant: so resubmit and hope for the best? [05:23] wallyworld: Force the build when it fails? I guess so. [05:23] yeah, that what i meant sorry [05:24] maybe jenkins would build it ok :-) [05:25] Heh [05:26] so ec2 is happy and it runs locally. stupid buildbot [05:48] Wow. [05:48] The rabbitmq people have made their build system even worse. [05:48] 2.3.1 was already one of the most distro-hostile I've seen, and 2.5.0 is worse... [05:50] wgrant: How so? [05:51] StevenK: For starters, see http://hg.rabbitmq.com/rabbitmq-mochiweb/file/b41f8ab516d6/Makefile [05:52] StevenK: Yes, the Makefile in the root directory of the package does one thing: include umbrella.mk from outside the package. [05:53] The rabbitmq-mochiweb test suite is also convinced that it wants to build its own rabbitmq-server. [05:53] Because dependencies are hard, I guess. [05:53] They seem to have basically rewritten the build system twice between 2.3.1 and 2.5.0, so my old fixes don't work :( [05:54] Ahh. [05:54] The plugin in Ubuntu (rabbitmq-stomp) works around this by not running the test suite. [05:54] But I got it working with 2.3.1... so I will do it again. [05:54] Perhaps all Erlang developers are crazy. [05:55] I'd agree with that. [05:55] For starters, they use Erlang. [05:55] Meh, Erlang has some pretty nice bits. [06:01] Why will people not stop writing their own shitty application-specific packaging systems!? [06:05] Oh cool, they're now including webmachine stuff in rabbitmq-mochiweb. [06:05] * wgrant sobs. [06:12] * jtv consoles wgrant [06:12] "There, there. This is all part of growing up. Eventually you'll learn that it all goes away for a while after large quantities of beer." [06:14] Yeah, I know. I'm a natural at this. [06:14] Heh [06:16] * StevenK kicks LaunchpadSearchView for being horrid. [06:34] jtv++ [06:35] nigelb: that returns the old value of jtv. [06:35] Although it does increase me, for which I am grateful. [06:35] heh [06:35] Now, what are we talking about? [06:35] Beer? [06:35] Ah! === jtv1 is now known as jtv === almaisan-away is now known as al-maisan [07:37] good morning === danilos changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: StevenK, danilos | Critical bugs: 261 - 0:[########*** stack smashing detected ***: ./lp terminated [07:55] wgrant: how do you build txlongpoll from tarball? Perhaps julian means build from branch ? [07:56] lifeless: What might Julian mean where? [07:56] https://bugs.launchpad.net/python-oops-twisted/+bug/859545 [07:56] <_mup_> Bug #859545: Fallback code corrupts the output < https://launchpad.net/bugs/859545 > [07:57] lifeless: I presume he means from the branch, rather than an egg. [07:58] lifeless: I also think he meant from the branch. Also note that we renamed the generated executable from twistd to txlonpoll. [07:58] rvba: Hm, why? [07:58] That seems very confusing. [07:59] huwshimi: Hi! Thanks for the longpoll icons. I'll see what the others have to say but I like them :). [07:59] there is a separate txlongpoll executable that just runs it isn't there? [08:00] lifeless: that's is still in the works AFAIK. [08:00] wgrant: why confusing? The goal was to avoid clashing with other twistd services. [08:01] rvba: 'bin/txlongpoll amqp-longpoll' is more confusing. [08:01] twistd is slightly more obvious. [08:01] the problem will be file collisions I suspect [08:01] When included in LP, sure. [08:01] True [08:01] But that's a matter for LP's buildout.cfg. [08:01] Not txlongpoll' [08:01] s. [08:04] wallyworld: Hm. [08:04] wallyworld: My ec2 run has just failed with that error... [08:05] wtf? [08:05] * wgrant runs the whole layer locally. [08:08] rvba: No problems! [08:08] huwshimi: one remark: [08:09] huwshimi: allenap noted that the failure icon looks a little bit like the Ubuntu Circle of Friends. [08:09] rvba: Ah, good point. I might make it a little less like that :) [08:10] rvba: I'm just heading off, if you have any other comments can you send me an email and I'll get to it tomorrow. [08:10] huwshimi: Fwiw, I like the spinning circle :) [08:10] allenap: OK cool :) [08:10] night all [08:13] jml: O hai [08:14] cjwatson: are debian imports working normally for you? We got gina's run time back down to half an hour or so, and everything goes through proper domination now. Packages deleted from Debian also have their publications deleted. [08:32] jtv: the last thing I uploaded to Debian (yesterday evening) has been imported, which is about all I can say really [08:32] bigjools: looking at txlongpoll [08:32] hmm [08:32] bigjools: why do you have a custom log *and* twistd.log ? [08:32] jtv: https://launchpad.net/debian/+source/debian-installer-utils - is the 1.84 "not published" there normal? [08:32] on the front page? [08:33] cjwatson: I'll have to check up on that -- otp now, gimme a few [08:37] cjwatson, jtv: gina wasn't setting datepublished for a while. We should set it to datecreated where it's not set. [08:39] bigjools: ppa:wgrant/rabbitmq has rabbitmq-management 2.5.0 for oneiric now. I had to disable the full functional tests, since the new plugin build system is even more distro-hostile than the old one, but the unit tests all pass and it works fine. [08:41] lifeless: are you running it with -n ? [08:42] wgrant: \o/ [08:42] bigjools: yes [08:42] wgrant: awesome, thanks [08:42] lifeless: that buggers up logging for some reason [08:43] bigjools: not at all [08:43] cjwatson, wgrant: it does indeed look like datepublished was simply not set at that point (should've been done in May for this one I guess). More interesting is that 1.84's publication in sid is Published; that should only be the case if 1.84 is still occurring in sid's Sources.gz. [08:43] bigjools: -n makes it log to stdout, and disables the default twistd.log [08:43] bigjools: if you run without -n, you get *two* logs [08:43] ah ok [08:43] lifeless: in that case, the FileLogObserver is not being added correctly [08:44] bigjools: its because the default setup includes another observer [08:44] bigjools: see https://bugs.launchpad.net/txlongpoll/+bug/861234 [08:44] I think I need to add my code back that adds it separately [08:44] <_mup_> Bug #861234: logs to both custom log and twistd.log < https://launchpad.net/bugs/861234 > [08:44] bigjools: the mylog observer is being added just fine [08:44] lifeless: yeah, the bug describes what I meant :) [08:45] I have fixed the extra spaces thing [08:45] I can't reproduce the other failure yet [08:45] bug 860490 [08:45] <_mup_> Bug #860490: oops reports with no type/value crash the fallback reporting code < https://launchpad.net/bugs/860490 > [08:45] that line used to be in the code but I took it out by mistake when I converted to using oops_twisted [08:45] that's weird [08:47] bigjools: https://bugs.launchpad.net/python-oops-twisted/+bug/860490/comments/6 [08:47] <_mup_> Bug #860490: oops reports with no type/value crash the fallback reporting code < https://launchpad.net/bugs/860490 > [08:48] lifeless: I can't figure out wtf is wrong then, it's 100% reproducible here [08:48] lifeless: when I break on that method, the first call succeeds, the 2nd fails [08:48] with the deferred failure [08:49] it gets called once per connection attempt for me [08:49] there's the difference then [08:49] I exepct you're getting a different failure [08:49] have you got the same versions of everything I wonder [08:50] * bigjools otp in a bit [08:50] anyhow, I'm going to implement the defensive programming solution [08:50] ok [08:51] without the bare except I hope [08:51] but we'll still need to figure out why you are not getting an exception data there [08:51] because that will devalue your oops reports [08:56] jtv: 1.85 is new today, so 1.84 still being around is not surprising. [08:56] jtv, wgrant: I am about to initialise p-series on DF, are you guys done with all the long-running stuff you were doing on there? [08:56] I am. [08:56] I'm not using it. [08:56] ta [08:56] So feel free to break it :) [08:56] Concur. [08:57] wgrant: whether it's surprising is one thing, but I'm interested in another: is it actually the case? [09:00] jtv: Yes, both versions are in http://ftp.debian.org/debian/dists/sid/main/source/Sources.gz [09:00] Good! [09:01] cjwatson: you should also see older package releases becoming Superseded, but the domination we implemented for gina does support multiple live versions in one release / pocket / archive. [09:01] Right. (When are we going to get similar behaviour of keeping multiple sources around until everything builds, I wonder ... or, more importantly, multiple arch: all) [09:02] The underlying mechanics are there now, for both Gina's domination and regular domination. [09:02] bigjools and I have some disagreements on how that should be done. [09:02] The hard part for regular domination is getting the right information about what versions should be considered live, and feeding it into those mechanics. [09:10] bigjools: what try:except ? [09:13] bigjools: 0.0.3 pushed [09:15] bigjools: (and on pypi) [09:15] ta [09:16] bigjools: we need to sort out whats up still, and as I can't reproduce yet, that means stepping through some with you :) [09:16] ok will need to be tomorrow [09:17] bigjools: sure hting [09:23] (still otp) [09:25] jml: do you know if there's a way to stop a Deferred firing until a write transaction is finished? [09:26] bigjools: wrap it in a deferred that will tie into the transaction [09:27] we want to block any deferred firing until a txn is done, so if a callback calls code that returns deferred we want the txn done before that deferred fires [09:28] I don't see any possible way to do that [09:28] we want to make sure txn boundaries don't cross callbacks, basically [09:28] We'd be fine if we had a way to stop twisted from giving any time to another thread of execution while we're in a given stretch of code. [09:28] a deferred is equivalent to a function call - it would be like making *all* function calls in any thread block based on some other state [09:28] bigjools: Aren't transactions restricted to inside the function? [09:28] this is a problem in the buildd-manager [09:28] bigjools: So a transaction won't be in progress when it returns to the reactor... [09:29] bigjools: you could push the db access into the API and use that instead; thats the long term plan *anyway* (per SOA & data encapsulation) [09:29] The core problem, I think, is that we want to be sure in a particular stretch of code that we don't accidentally call something that will do anything deferred. [09:29] lifeless: yes, but we need a very fast api first :) [09:30] bigjools: we have one [09:30] Also, pushing this to the API would just make things even worse. [09:30] bigjools: the overhead for a single call is very low [09:30] The problem is not that transactions are being held around. It's that we're doing data modification in places that we shouldn't be. [09:30] lifeless: xmlrpc yes, webservice no [09:30] Eliminating transactions from that just makes it harder to track down. [09:30] bigjools: !cite - I've looked, and the webservice overhead is low [09:30] lifeless: can do you have time to ack a db patch? [09:30] bigjools: also, internally I'd be suggesting using xmlrpc *anyway* as we have that infrastructure setup. [09:31] lifeless: not low enough for this [09:31] s/can you/do you/ [09:31] nigelb: no, I'm on annual leave today :P also, see policy - stub approves except when he's not here. [09:31] bigjools: is 2ms low enough ? [09:32] no [09:32] that's slow [09:32] bigjools: because thats the size it was looking like last time I compared the delta from xmlrpc to json api [09:32] if it were 2us I'd be happy [09:33] bigjools: that doesn't make any sense to me - we can't even query the db twice in 2ms. [09:33] lifeless: oh. right. wednesday. Well, stub hasn't been here yet. Which is why I asked. I'll wait for him :) [09:33] so you want to add more latency? [09:34] bigjools: *if* it had to, I could tolerate 8ms per slave build, which would allow for 4 API calls in the critical path for dispatching and completing a build [09:35] bigjools: but we're not going to get clarity by tossing sketches back and forth on IRC [09:38] lifeless: I used to work in sub-millisencond latency over networks. If you take the Db out of the equation, replacing an internal python call with a 2ms latency remote call doesn't sound clever [09:39] jtv: you can 'stop Twisted from giving any time to another thread of execution' by not using threads and doing a blocking call [09:39] jml: yes I had figured that bit out, thank you [09:39] jtv: ^ that is why I said not to call them threads :) [09:39] (notice irony there) [09:39] bigjools: of course, but you don't do a literal transcription just replacing db calls with api calls. [09:39] jtv: ok. [09:39] bigjools: that wouldn't work well for many reasons, transactions being one of them [09:39] I'm not talking about threads, of course, merely about threads of execution in twisted. [09:40] bigjools: so measuring the performance by estimating overhead from a direct transcription isn't a good estimator [09:40] lifeless: ok, this is a conversation better left for in-person I think [09:40] jtv: I'm not interested in having an ironic conversation right now. [09:40] (Seeing how much taller than lifeless bigjools is) [09:41] jml: neither am I. What I'm trying to do is ensure that twisted does not give any time to another job deeper down in my call tree. [09:41] lifeless: I am sure we can come to a good agreement somewhere, but it's hard to explain without waving arms around [09:41] bigjools: yup, I get that [09:41] wallyworld: bin/test -vvct initial-bug-contacts.txt -t bugtracker-tokens.txt [09:41] jtv: don't return from your function then. [09:41] wgrant: you want me to run those? [09:42] jtv: (and don't yield either) [09:42] wallyworld: That reproduces the failure. [09:42] wallyworld: We have a test isolation issue. [09:42] wgrant: bugtracker-tokens wasn't even changed in that branch [09:42] wallyworld: Correct. [09:42] so somehow the ff is not being homoured [09:43] lifeless: I don't want to "just not do" the thing I want to prevent; I want to establish some certainty that it's not inadvertently happening somewhere deeper down in the call tree. [09:43] in the failing doctest [09:43] wallyworld: Right, presumably bugtracker-tokens is corrupting the state. [09:43] that sucks :-( [09:43] has this happened before, the same way? [09:44] is there something we can recall that will help solve it? [09:44] Not identical, but we do occasionally have stuff like this. [09:44] I'm tracking down the problematic call. [09:44] ok [09:44] i'm hacking javascript. for the disclosure mockups [09:45] let me knoe if i can do anything [09:45] Ah, it looks like it's XMLRPCTestTransport. [09:46] That is an evil piece of code. I recall it from my Python 2.7 experiments. [10:21] lifeless: don't forget to "make lint" your branches! From a quick glance at your self-approved branch, I get the impression that there will be some warnings in there. [10:21] Or is it not a Launchpad branch? That'd explain why it ended up in my main mailbox. [10:22] stub: as promised... https://code.launchpad.net/~jtv/launchpad/pre-717969/+merge/77299 [10:28] jtv: I'm not sure I like DatabaseTransactionPolicy having an implicit commit. It goes against your reasoning in the MP synopsis, and means you can't mix read-only and read-write code paths in a transaction. [10:28] stub: I feel the same, but didn't see support in the underlying APIs for a non-committing version. [10:29] For example, how do you reset policy when you're in a failed transaction? [10:29] Why do you need it? [10:31] When an exception is taking you out of the policy, it needs to reset previous policy. But at that point the transaction may be broken. [10:32] I could abort at that point, start a new transaction, set policy, and commit -- but that brings us back to it being a transaction boundary. [10:32] Sounds like we need to issue a transaction.abort() if we are exiting the context handler because of a psycopg2.Error [10:33] Although a higher level could be handling it... [10:33] blah [10:33] Hmm... I wonder if we can setlocal instead? [10:35] doesn't help in this case, but set local might be nicer anyway [10:36] * stub looks into savepoints [10:37] jtv: savepoints make this nicer, and you get better nesting behavior too [10:38] Are we talking about a savepoint in the postgres sense? [10:38] Nested transactions, basically? [10:39] jtv: savepoint mytoken; set local default_transaction_readonly true; [...]; release savepoint mytoken; [10:39] yet [10:39] yer [10:39] or rollback to savepoint mytoken [10:40] But that won't work if the transaction breaks, will it? [10:40] It will rollback the 'set' too (per documentation on the SET statement) [10:41] jtv: transaction breaks gets an implicit release of the savepoint [10:41] (Also, I couldn't swear that the big performance problems with savepoints were solved in 8.4 as opposed to 9.0 :) [10:41] its all aborted and it is all rolling back until the code that wants to handle it starts a new transaction [10:41] cjwatson: I just did a test run of initializing p-series on dogfood, do you want to check it? It seems to be ok to me. https://dogfood.launchpad.net/ubuntu/p-series [10:42] bigjools: It used the cloner, not the copier, right? [10:42] If there is a performance problem, we could consider a switch to turn it off on production. The goal of this code is to prove what sections of code are doing, so mainly applicable to test suite and maybe staging [10:42] cloner [10:43] stub: I don't see nested transactions playing well with other transaction boundaries though. [10:43] jtv: Yes, it won't play well if the code wrapped by the context manager does commit or rollback [10:44] There are commits and rollbacks in there already. We could perhaps get rid of those, but it makes me slightly more nervous. [10:46] Implicit commits where previously there were no commits seems to be making the original problem worse rather than better (updates that should be atomic being made in separate transactions) [10:48] So what happens if the context manager, if it catches a psycopg2.Error, does a transaction.abort(), reset policy, reraise exception? [10:48] We could even break the transaction again after resetting the policy if we want to be totally transparent [10:48] That doesn't play very well with other transaction handlers. [10:49] Sorry: [10:49] with other *exception* handlers. [10:49] We could try it with a slight twist: [10:49] I don't see why. The db transaction is broken. There is nothing to do with the db transaction except issue an abort [10:49] reset policy [10:50] if *that* raises a psycopg2 error, abort & reset policy. [10:50] And then commit. [10:50] oic. yes, because the exception might have been caught inside but left the db connection broken [10:50] Your idea seems sane to me [10:50]