[02:11] wgrant: i've pushed the latest changes, seems to work ok [02:58] wallyworld__: /me looks [02:58] Sorry, was at lunch [02:58] ok [02:58] np [04:37] wgrant: you had a look at the garbo job? [05:06] wallyworld__: Sorry, fighting with a combination of Unity, fglrx, and Storm [05:07] wallyworld__: It looks good, although lp:~wgrant/launchpad/another-reporting-cache-garbojob-1071581 has a significantly shorter version. [05:24] wgrant: Your security.py changes landed? - 2012-11-13 01:19:45,892 INFO Outage complete. 0:00:00.755889 [05:25] stub: Yeah [05:25] stub: It sped up prod by not as much as expected :( [05:25] Only a bit more than it did staging [05:26] But little more can be done with the scripts, which is good. Any other improvements need to happen in pgbouncer or our PG setup if we can identify the issues [05:26] Yeah [05:27] unlikely to be worth it I think [05:27] We have a -v run from prod with my changes, somewhere [05:27] still haven't got the basic pgbouncer patch accepted upstream :-/ No responses when I brought it up on the mailing list. [05:30] Ah, you brought it up on the ML? [05:33] hmm... might have posted from the wrong address and got silently spamfiled [05:33] wgrant: just got back from school pick up, looking now [05:34] oh, no. it is there. http://permalink.gmane.org/gmane.comp.db.postgresql.pgbouncer.general/945 [05:35] wallyworld__: Just pushing some further cleanups (and a fix to my changes) [05:35] ok [05:36] I think I've beaten Storm sufficiently near death that it's not completely hideous. [05:36] The only thing that needs improving before it's genericised is typecasting in the VALUES expression, but it's easy enough to work around here [05:37] Now I can actually read it, I will recheck correctness :) [05:37] Is this in Storm or Launchpad? [05:37] LP for now [05:37] Once it's stable it'll go back to Storm [05:38] stub: This is a bulk UPDATE helper, which in postgres can get a liiiittle messy [05:38] We should railroad through our other patch too. Given the speed of new Storm feature work, the argument that our mods could cause problems is a bit flaccid. [05:38] Heh [05:39] wallyworld__: Won't this think that a key already exists in the cache table if there's any row for that (archive, distroseries, spn), without regard for the creator or maintainer? [05:40] wgrant: it loads those and then constructs map keys based on creator and maintainer for in memory processing [05:41] wallyworld__: Oh right, I'm stupid, don't mind me :) [05:41] maybe i could tighten the filter a bit [05:41] That's going to be slightly excessive in terms of loaded objects, but it shouldn't matter too much [05:41] Though it's also not difficult to fix [05:41] i couldn't be bothered doing a Values() class first up, but it's there now [05:42] It means we can reuse _dbify_value, which simplifies the compilation of the values a lot [05:42] Otherwise I wouldn't have bothered either [05:42] But I basically just lifted the code from my bulk INSERT work [05:43] ah, ok. i lifted the code from Update() to get the core BulkUpdate SET compiling [05:43] so you changed that to allow for the dbify stuff [05:46] I changed bulkupdate not because of the dbify stuff, but because I needed it to include the table name in the RHS of each SET expression [05:47] looks good to me. i like the ClassAlias stuff [05:47] Previously it always used the COLUMN_NAME compilation mode, which meant it said "SET sourcepackagerelease=sourcepackagerelease", which has an ambiguous RHS [05:47] That means Storm's compile_update is buggy, but people don't tend to do complex UPDATEs in Storm [05:47] yeah [05:48] i exported Values in stormexpr, that is my only change i think [05:48] The ClassAlias stuff is a fairly gross but very effective and relatively attractive hack [05:48] Ah yeah, that too [05:48] Also, the stormexpr import in garbo is in the wrong place [05:48] except the test fails [05:48] Forgot to fix that [05:48] i fixed that import [05:48] I assumed it was just failing due to my test DB being out of date [05:48] What's the failure? [05:48] test_garbo hides the failure sadly [05:49] i'll debug it [05:50] Ah [05:50] So yeah, I learnt some interface-violating Storm tricks during my bulk-insert, bugtaskflat, bugsummary, and targetnamecache arcs which hopefully make this a bit more pleasant [05:51] we really should commit out stormexpr stuff to storm and get to using trunk [05:51] s/out/our [05:52] The main delta between Storm trunk and our branch is the With stuff [05:53] The stormexpr stuff can all be easily upstreamed independently of that [05:53] there's a datettime related diff too? [05:53] Oh yeah [05:54] Storm trunk Date columns now return a datetime.date, rather than a datetime.datetime [05:54] There's a bit of test fallout that we opted to avoid sorting out immediately [05:54] right [05:57] It seems like our BulkUpdate could actually be merged straight into upstream's Update [05:57] By just making Update take a tables= [05:58] But later :) [05:58] Update does take a tables already I think? [05:58] unless i misremember [05:58] It takes a table and a default_table [05:58] I do not know what the difference is [05:59] me either, haven't dug that far down [05:59] * wallyworld__ afk, bbiab [05:59] It doesn't have an awful lot of callsites [06:00] In Storm itself there are 0 [06:24] wallyworld__: Ah, the job works better when I haven't completely broken inserts [06:24] As it turns out [06:26] wallyworld__: You'll need a new index if you continue with the (archive, distroseries, spn) filter, but other than that it works fine on DF [06:28] 2012-11-14 06:28:20 DEBUG2 [PopulateLatestPersonSourcePackageReleaseCache] Done. 60852 items in 140 iterations, 283.422512 seconds, average size 434.659565 (214.705383185/s) [06:28] Which is not bad for DF [06:46] Ah, damn, we need more casts [07:00] wgrant: just merged your latest stuff, got a conflict, resolving [07:01] wallyworld__: Yeah, and even your version of the BulkUpdate breaks if it ends up UPDATEing a single row, as it infers the NULL creator or maintainer to be text, not integer [07:01] Testing a fix now [07:01] ah ok, thanks [07:04] I guess it can't really infer all the way up in an UPDATE, unlike an INSERT === almaisan-away is now known as al-maisan === al-maisan is now known as almaisan-away [07:22] wallyworld__: OK, fix with explicit types pushed [07:22] ok, i was still getting test failures with the latest prior to that [07:22] Now restarting unity again... [07:23] :-( [07:24] <3 fglrx [07:24] gnome-terminal no longer feels like I am SSHed to the moon [07:30] wgrant: the update is broken [07:30] update sql [07:31] i don't think it likes the ::integer syntax [07:31] if i run it in an sql editor it asks for a value "for parameter :integer" [07:32] What's the generated SQL? [07:32] The test passes here [07:33] https://pastebin.canonical.com/78351/ [07:33] wallyworld__: ::type syntax is PG specific, it might be confusing your editor [07:33] bin/test fails though [07:34] It is missing quotes around that date string [07:34] and with sql logging it stops just after trying to execute the update [07:34] wallyworld__: That's not valid SQL [07:34] Did you get that from LP_DEBUG_SQL=1? [07:35] yes [07:35] I have a patch for that [07:35] sec [07:35] The existing one just uses python string formatting to do it [07:35] Rather than psycopg2's method [07:35] so you end up with None instead of NULL === almaisan-away is now known as al-maisan [07:35] yes [07:35] u'FOO' instead of E'FOO', etc [07:35] http://pastebin.ubuntu.com/1357406/ [07:35] I might land that soon [07:36] i didn't notice the None first up [07:36] Hmm, BulkUpdate. That might be awesome for IBug's method to change information type [07:36] but the garbo test still fails for me sadly [07:37] so there's something else not quite right [07:37] wallyworld__: Odd, it passes here. Possibly a mismerge? [07:38] perhaps. there were several conflicts, so i copied your entire garbo.py [07:38] i'll try again [07:38] (the problems with the SQL you pasted are just the None and the unquoted timestamps, AFAICT) [07:38] Hm [07:39] Your stormexpr.py is odd [07:39] UPDATE LatestPersonSourcePackageReleaseCache SET sourcepackagerelease=sourcepackagerelease, [07:39] publication=publication, [07:39] My version doesn't do that [07:39] date_uploaded=date_uploaded [07:39] The RHS of those SET expressions is meant to be qualified with a table name [07:39] yeah [07:39] the None vs NULL is a logging issue as you say [07:40] Oh, also, while you're looking at compile_bulkupdate, the 'state.push("context", EXPR)' should just be 'state.context = EXPR' [07:40] Won't make a difference since it's the root expression, but it's slightly wrong [07:41] my stormexpr was slightly out of date i think [07:41] test passes [07:43] wgrant: so i've just pushed changes to the mp, incl that state.context change. test passes too [07:44] Great, will approve [07:46] thanks for all the help making it even better [07:46] we can add the index tomorrow i guess if required [07:47] We'll certainly need it [07:47] As otherwise it has to seqscan [07:47] but should be ok to land now so that we get data into the table [07:48] Not really. I can get the index added tonight if you want [07:49] But we can't sensibly deploy this without the index [07:49] But r=me, and thanks for obliging my experimentation [07:49] np. [07:49] let's do the index tonight [07:50] i'm keen to see this land [07:50] and data to be there [07:50] Yep [07:50] Do you have time to do the branch, or should I? [07:50] i am going out for dinner soon but can do it when i get back. we can do it in devel right? [07:51] Yep [07:51] Then get someone to apply it live [07:51] s/someone/some poor webops [07:52] so just an index on (archive, distroseries, spn) [07:52] Right [07:54] ffs, db-devel broken === al-maisan is now known as almaisan-away [08:51] good morning [10:13] wgrant: https://code.launchpad.net/~wallyworld/launchpad/report-cache-index-1071581/+merge/134274 [10:15] wallyworld__: Index names are in the public schema scope, not a particular table, so we usually prefix them with the table name [10:15] sometable__some_col__some_other_col__idx [10:15] ok [10:16] wgrant: name os too long - says it will be truncated, is that ok? [10:17] wallyworld__: It's better to have the end cut off than the table name, but sometimes we'll use abbreviations in the column namesa [10:17] ok, will abbreviate the col names [10:18] wgrant: It's a minor point, but do you have any feeling on the logging question I asked at the end of the comments in https://code.launchpad.net/~cjwatson/launchpad/always-copy-packages-asynchronously-2/+merge/131928 ? [10:18] cjwatson: We historically ran scripts on production at INFO for not very good reasons [10:19] I tend to tweak them up to DEBUG when I have cause to touch their crontab lines [10:19] I'd suggest that might be a good idea here [10:19] Righto [10:24] wgrant: diff updated, index still about 6 chars too long, but i guess it will do [10:26] wallyworld__: Right, r=me [10:26] thanks [10:28] wallyworld__: This should be the last branch apart from the one to remove the old inline calculation, right? [10:28] bloody hope so [10:28] Heh [10:28] but there will be one to remove feature flag [10:29] stub: could you possibly apply this index to qastaging? https://pastebin.canonical.com/78357/ [10:29] That will presumably be the one that rips the old stuff out [10:29] yeah [10:29] wallyworld__: Got a patch number to go with that? Or are we going to remove it again in a tick? [10:30] 2009-38-2 [10:30] 2009? [10:30] 2209 sorry [10:31] all done on qastaging [10:31] thanks [10:31] wallyworld__: You might want to increase the garbo-frequently log verbosity on qas (and potentially prod; I forget what it's set to there) [10:32] stub: rev 16267 is landing now (has the patch). when do we apply to prod? [10:32] wgrant: ok, can a web ops do that [10:33] wallyworld__: whenever you like [10:33] stub: now would be great [10:33] wallyworld__: Yeah [10:34] wallyworld__: It's in the crontabs, the --log-file option [10:34] You probably want DEBUG2 [10:34] to get the looptuner logging [10:34] With row counts etc. [10:34] ok [10:34] wallyworld__: it is running [10:34] (will take a bit longer on prod depending on other transactions) === matsubara is now known as matsubara-lunch [14:54] rick_h, I moved our call by 1/2 hour later, but the TL call wraps then, too. So I may be a couple minutes late. [14:54] rick_h, if that's cool, I'll ping when I get free. [14:55] deryck: np, whenever works for me. Thanks for the heads up [14:55] deryck: just ping when you're free [14:55] deryck: I just created an embargoed project, and it has the correct policies: https://qastaging.launchpad.net/asdff/+sharing [14:56] abentley, gah! I wonder what happened differently for me. [14:57] I don't know. I doubt it's a difference between doing it locally and doing it on qastaging. [14:58] deryck: You created it with Embargoed, you didn't change it to Embargoed afterwards, right? [14:58] abentley, right. [14:58] abentley, trying again now locally to see what happens. [14:59] abentley, hmmm, yeah, no blueprints. I wonder if it's because I don't have the blueprints flag enabled. [14:59] deryck: That seems possible. [15:00] abentley, yeah, that's it. sorry man. [15:00] abentley, I'll update that bug to be just about bugs sharing policy then. [15:00] deryck: +1 [15:04] sinzui: Could we talk about sharing policies? Doesn't have to be right away. [15:05] abentley, sure. [15:05] I can talk now if you like [15:06] sinzui: Cool. I've invited you to a hangout. [15:06] sinzui: https://plus.google.com/hangouts/_/31454706a4303baaad1308627a542146c5b83ded?authuser=0&hl=en-GB# [15:07] abentley, one moment as I fight an authorisation failed at google [15:22] deryck: Outcome is: we will support inconsistency between old artifacts and current sharing policies. This was requested by stakeholders. [15:23] abentley, ok, sounds fine to me. we need to do some work to support this, or existing behavior is okay? [15:24] deryck: We need to do some work. We need to consider artifact information types when doing AccessPolicy pruning. [15:25] deryck: It's possible we should also not complain that a project has public artifacts when switching information type to proprietary. [15:27] abentley, ok. we need to add a bug/card for the first item. personally, I think it's good to complain about the mismatch, though. We don't have to block it, if we need to support this. [15:27] abentley, but a warning is useful. [15:29] deryck: I think a warning is most consistent with what we promised stakeholders they would be able to do. [15:29] abentley, sounds good. [15:29] I will add a bug/card for the first item. [15:33] abentley, thanks for doing that! [15:35] rick_h: could you please review this MP: https://code.launchpad.net/~adeuring/launchpad/product-lp-limitedview-branch-view/+merge/134315 ? [15:36] adeuring: sure thing [15:36] ricthanks! [15:36] rick_h: tanks === rick_h changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On Call Reviewer: - | Firefighting: - | Critical bugs: ~200 === abentley changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On Call Reviewer: abentley | Firefighting: - | Critical bugs: ~200 [15:40] adeuring: r=me with one typo [15:40] rick_h: thanks! [15:40] rick_h: he has to add a tyop for you to approve? [15:41] mgz: requirement, leaves something for me to fix later as a nice simple bug when I feel like I'm not getting much done :P [15:41] mgz: move those cards across the board! [15:41] :) [16:35] rick_h, how about meet in 5 minutes? [16:35] deryck: rgr === Ursinha is now known as Ursinha-afk === deryck is now known as deryck[lunch] === almaisan-away is now known as al-maisan === al-maisan is now known as almaisan-away === deryck[lunch] is now known as deryck === Ursinha-afk is now known as Ursinha [19:34] jcsackett, I marked your branch qa-ok. This is my test: https://bazaar.qastaging.launchpad.net/+branch/launchpad/revision/14654 [19:34] expand all works === yofel_ is now known as yofel === matsubara_ is now known as matsubara === Ursinha is now known as Ursinha-afk [22:29] wgrant, do you have any insight into https://bugs.launchpad.net/launchpad/+bug/1078796 [22:45] sinzui: I thought that was a new bug I created hw did you find the other duplicate? [22:46] czajkowski, wgrant agreed with my speculation that the root cause is a naughty process locking the archive...nothing to change it not the web, not the api, not even another process running on our servers [22:47] ok [22:47] we have a number of bugs that have this root cause. We think we need to change our scripting architecture to honor timeouts like the web and API does so everyone can share the data [22:47] strange how it only happens some ppas and not all but just noticed it a lot this week given I'm increasing a lot of ppas [22:48] czajkowski if the person notices that they are out of space, Lp is accepting and building those packages...so those are the most likely have a lock :( [22:52] ahh [22:55] I thought we accepted binary uploads from buildds if they're over quota, but not source uploads [23:00] Yes === Ursinha-afk is now known as Ursinha