| mwhudson | wgrant: is this the form of replication that lp uses? http://www.postgresql.org/docs/9.1/static/warm-standby.html#STREAMING-REPLICATION | 00:07 |
|---|---|---|
| wgrant | mwhudson: Yes | 00:07 |
| wgrant | It works excellently, though it's somewhat arcane at times | 00:08 |
| mwhudson | it's postgres, i think that last bit is implied | 00:08 |
| wgrant | And it's not meant to be possible to demote the master to a slave without rebuilding, but we do it anyway with a bit of evil. | 00:09 |
| wgrant | Hopefully it'll be a bit better in 9.3, but it works very well once you work it out | 00:09 |
| wgrant | A tonne less annoying than slony | 00:09 |
| mwhudson | wgrant: can you restore a slave from a backup that's say about 24 hours old and have it catch up from the master? | 00:09 |
| wgrant | mwhudson: Um, sort of. | 00:10 |
| wgrant | It can't be a pg_dump backup | 00:10 |
| wgrant | It has to be a backup of the raw data files | 00:10 |
| mwhudson | ah | 00:10 |
| wgrant | And you have to have WAL retention set high enough | 00:10 |
| mwhudson | yeah, that last bit was the thing that was starting to scare me a little | 00:11 |
| mwhudson | well not scare me | 00:11 |
| mwhudson | intimidate me with the prospect of reading a lot more documentation | 00:11 |
| wgrant | Once we have enough disk space around we're planning on moving away from pg_dump backups | 00:11 |
| wgrant | Just rsync the data dir to a backup volume, and keep WAL for weeks | 00:11 |
| wgrant | Lets you easily bring on a new replica, plus gives you arbitrary PITR for DR. | 00:11 |
| mwhudson | hmm | 00:12 |
| mwhudson | hmm | 00:12 |
| wgrant | For a less insane DB it's probably worth doing both from the start | 00:12 |
| mwhudson | the reason I'm wondering about this is that i want to move pg off a server temporarily while we upgrade | 00:12 |
| mwhudson | so maybe i don't need replication at all (a little downtime is ok) | 00:13 |
| wgrant | Yeah, probably better to take the few minutes of downtime in that case | 00:13 |
| mwhudson | we could just rsync the data dir from one server to the backup | 00:13 |
| wgrant | If you have an existing replication setup then sure | 00:13 |
| mwhudson | stop the db, rsync again, bring up the backup | 00:13 |
| mwhudson | upgrade the original server | 00:13 |
| mwhudson | stop the back up pg, rsync back to the original server and start things up again | 00:14 |
| wgrant | Or just have the service down for the 5 minutes that it takes to upgrade the original server? | 00:15 |
| mwhudson | wgrant: i mean os upgrade | 00:15 |
| wgrant | So do I :) | 00:16 |
| mwhudson | not expecting that to take 5 minutes | 00:16 |
| mwhudson | hm | 00:16 |
| wgrant | This isn't Windows or a pandaboard or a desktop, so it should usually be pretty fast, but I guess it depends | 00:16 |
| mwhudson | I guess i'm a bit paranoid | 00:18 |
| wgrant | That's always good, but sometimes not justified depending on the particular service | 00:18 |
| StevenK | wgrant: Does https://code.launchpad.net/~stevenk/launchpad/destroy-pylint-comments/+merge/142053 frighten you? | 02:58 |
| wgrant | StevenK: Naturally | 03:10 |
| StevenK | wgrant: Is that code for 'self-review that, damn it' ? | 03:14 |
| wgrant | No | 03:14 |
| wgrant | Nearly done | 03:14 |
| wgrant | StevenK: Done | 03:17 |
| StevenK | wgrant: Can you glance at the changes in r16406 for that branch to see if I missed something? | 03:40 |
| wgrant | StevenK: Can you just read through the new revision diffs and remove the many comments that are now obsolete? | 03:41 |
| StevenK | I already did that before commiting, I'm wondering if I missed something | 03:42 |
| StevenK | r16406 only changed 36 files | 03:42 |
| wgrant | # A zope interface doesn't have self as a parameter for its methods. | 03:43 |
| wgrant | - # pylint: disable-msg=E0211 | 03:43 |
| wgrant | # This class should know about the private _window attribute. | 03:43 |
| wgrant | - # pylint: disable-msg=W0212 | 03:43 |
| wgrant | # We are deliberately not calling PullerWorkerProtocol.__init__: | 03:43 |
| wgrant | - # pylint: disable-msg=W0231 | 03:43 |
| wgrant | # We know we are not using dirnames. | 03:43 |
| wgrant | - # pylint: disable-msg=W0612 | 03:43 |
| wgrant | # The super constructor is a no-op. | 03:43 |
| wgrant | - # pylint: disable-msg=W0231 | 03:43 |
| wgrant | - # pylint thinks this raises None, which is clearly not | 03:44 |
| wgrant | - # possible. That's why this test disables pylint message | 03:44 |
| wgrant | # E0702. | 03:44 |
| StevenK | wgrant: So it seems getUtility(IPackageDiffSet).getDiffsToReleases(sprs, preload_for_display=True) preloads the LFAs and LFCs, but the page goes ahead and queries again anyway. Surely it should be cached since it's just a key lookup? | 03:57 |
| wgrant | StevenK: Right, you'll need to work out why they're not | 03:58 |
| StevenK | Hmm, looks like the LFA ids are not the ids from the packagediff query | 04:03 |
| * StevenK peers closer | 04:03 | |
| StevenK | (Pdb) lfa.filename | 04:06 |
| StevenK | u'filename-100051' | 04:06 |
| * StevenK facepalms | 04:06 | |
| wgrant | Hmmmm? | 04:09 |
| StevenK | I'm poking around in the DB for that id, so I can work out what it is | 04:10 |
| StevenK | It isn't a SPRF, BPF or a PackageDiff | 04:10 |
| wgrant | Ah | 04:10 |
| wgrant | Check the traceback on the query? | 04:10 |
| StevenK | The test isn't giving a traceback | 04:11 |
| wgrant | If it's a query count test then the statement collector will have tracebacks | 04:12 |
| StevenK | AH HA | 04:12 |
| StevenK | It's the files from the PUCs | 04:13 |
| wgrant | StevenK: https://code.launchpad.net/~wgrant/launchpad/bfj-mixin/+merge/142055 | 04:19 |
| StevenK | 170+class TestBuildFarmJobBase: | 04:21 |
| StevenK | And then you call super() ? | 04:21 |
| wgrant | I didn't write that, but yes, it's fine to call super() in a class with no superclasses | 04:22 |
| wgrant | Because of multiple inheritance, there still may be classes further up the MRO | 04:22 |
| wgrant | Otherwise if I inherit from (TestBuildFarmJobBase, TestCaseWithFactory) it might not go all the way up | 04:23 |
| StevenK | 417# The url of the upload log file is determined by the PackageBuild. | 04:23 |
| StevenK | That comment is sort of wrong now, no? | 04:23 |
| wgrant | The tests are all very strange now | 04:24 |
| wgrant | I just needed them to pass | 04:24 |
| wgrant | They'll obviously need to be significantly reworked over the next couple of days when PackageBuild and BuildFarmJob cease to exist :) | 04:24 |
| StevenK | Yeah | 04:24 |
| StevenK | Niggle dropped, since you'll need to revisit the tests, r=me | 04:25 |
| wgrant | Thanks | 04:25 |
| wgrant | StevenK: Why's the invalidate in getDiffTo? | 04:50 |
| StevenK | Because that creates a PackageDiff? | 04:50 |
| wgrant | No | 04:50 |
| wgrant | That's a selectOneBy | 04:50 |
| wgrant | You might mean requestDiffTo | 04:51 |
| StevenK | Yeah, I've just shifted it | 04:51 |
| wgrant | Great | 04:51 |
| StevenK | Any other issues? | 04:51 |
| wgrant | Doesn't look like it | 04:52 |
| wgrant | Does it work? | 04:52 |
| wgrant | Have you tested the query count on DF? | 04:52 |
| StevenK | I've seen appreciable drops in the query count from the tests | 04:53 |
| StevenK | I can cowboy it onto DF | 04:53 |
| wgrant | Right, but it's worth a cowboy to see that you've actually caught everything | 04:53 |
| StevenK | wgrant: +queue == At least 56 queries/external actions issued in 0.99 seconds | 05:22 |
| wgrant | StevenK: Checked various Done pages? | 05:22 |
| StevenK | Just loading Done | 05:22 |
| StevenK | Which timed out | 05:22 |
| StevenK | At least 66 queries/external actions issued in 12.24 seconds | 05:23 |
| wgrant | What are the extra 10? | 05:24 |
| StevenK | Reload gets us to 3.26 | 05:24 |
| StevenK | wgrant: Checking the query log without JS? | 05:25 |
| StevenK | You evil man | 05:25 |
| wgrant | Because Firefox and Chromium have become stupid, you might need to select the entire page content and View Selection Source | 05:25 |
| wgrant | Because View Source seems to reload the page without cookies | 05:25 |
| StevenK | Ugh | 05:27 |
| wgrant | StevenK: https://code.launchpad.net/~wgrant/launchpad/minimise-bfj-interfaces/+merge/142058 | 05:32 |
| bigjools | wtf does view source even reload the page ... | 05:33 |
| wgrant | It's possible that IPackageBuildDB will need build_farm_job_id, but I don't think so | 05:33 |
| wgrant | bigjools: Probably because LP sets no-cache, and browsers are retarded | 05:33 |
| wgrant | Although do we actually set no-cache nowadays... | 05:33 |
| StevenK | Items in the NEW queue have no SPRs and so a portition of the preloading is skipped? | 05:34 |
| wgrant | They have SPRs | 05:36 |
| wgrant | But no SPPHs | 05:36 |
| wgrant | But +queue shouldn't care about SPPHs except for detecting newness | 05:36 |
| StevenK | Ah ha | 05:36 |
| StevenK | No binaries for source in NEW | 05:37 |
| StevenK | wgrant: http://pastebin.ubuntu.com/1505886/ is a diff between the queries | 05:39 |
| StevenK | Between the query logs, even | 05:39 |
| wgrant | Well | 05:41 |
| wgrant | Obvious one is sections | 05:41 |
| wgrant | Are you failing to preload binary sections, perhaps? | 05:41 |
| StevenK | I don't grab bprs at all, so likely | 05:41 |
| * StevenK shovels more coal into mawson | 05:45 | |
| wgrant | While you're waiting for it to run 'bzr st' you can review my branch :) | 05:46 |
| StevenK | I have | 05:46 |
| wgrant | Oh, so you have | 05:46 |
| wgrant | StevenK: Hideous!? | 05:47 |
| StevenK | Right, NEW is now 58, and DONE is 62. | 05:47 |
| wgrant | We only have IBuildFarmJob, IBuildFarmJobOld and IBuildFarmJobDB | 05:47 |
| wgrant | I don't know WHAT you are talking about | 05:47 |
| wgrant | What did you change? | 05:47 |
| StevenK | load_referencing(BPR) and load Section for sprs + bprs | 05:49 |
| StevenK | wgrant: I can diff the query logs again if you wish | 05:51 |
| wgrant | Aha | 05:52 |
| wgrant | Are you sure the BPRs weren't already loaded somewhere? | 05:52 |
| wgrant | It already preloads the BPFs | 05:52 |
| wgrant | Possibly directly | 05:52 |
| wgrant | It may be worth replacing that prejoin | 05:53 |
| StevenK | That's inside the browser code, I can rip that out if you wish | 05:53 |
| StevenK | But it's a bit disgusting and it links into CPU | 05:53 |
| StevenK | So I'm hoping I can leave it alone | 05:53 |
| StevenK | Or I could just preload the BPR sections in the browser code | 05:54 |
| wgrant | Well, SQLObject prejoins are thoroughly deprecated anyway, so it'd be nice to eliminate it if it's easy | 05:55 |
| wgrant | And I suspect it would be cleaner given all the other preloading you're doing now | 05:56 |
| StevenK | It's calling into IBinaryPackageFileSet.getByPackageUploadIDs | 05:56 |
| wgrant | Oh god why | 05:56 |
| wgrant | It may be worth eliminating that method | 05:56 |
| wgrant | Rather than double-querying the BPrs | 05:56 |
| wgrant | (Since that obviously joins through them, though it might not actually grab them) | 05:57 |
| StevenK | browser/queue is the only callsite | 05:57 |
| StevenK | And some hideous doctest | 05:57 |
| wgrant | Kill | 05:57 |
| wgrant | Crush | 05:57 |
| wgrant | Destroy | 05:57 |
| StevenK | WITH FIRE | 05:57 |
| StevenK | wgrant: Haha, IBinaryPackageFileSet and ISourcePackageReleaseFileSet only have stuff used by that page | 06:05 |
| StevenK | So I can delete the whole interface and model classes | 06:05 |
| * wgrant loses | 06:06 | |
| wgrant | Excellent | 06:06 |
| StevenK | NEW = 65 and DONE = 72 | 06:33 |
| wgrant | StevenK: Nice easy one if you're still around: https://code.launchpad.net/~wgrant/launchpad/no-+buildjob-redirect/+merge/142066 | 08:20 |
| === almaisan-away is now known as al-maisan | ||
| StevenK | wgrant: Was distracted by Orks in SM, but r=me | 08:46 |
| wgrant | Thanks | 08:52 |
| === teknico_away is now known as teknico | ||
| === al-maisan is now known as almaisan-away | ||
| === benji changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On-call reviewer: benji | Firefighting: - | Critical bugs: <150 | ||
| czajkowski | mgz: best place to send someone intersted in bzr community disucssions? | 14:34 |
| === slank_away is now known as slank | ||
| === almaisan-away is now known as al-maisan | ||
| === salgado is now known as salgado-lunch | ||
| mgz | czajkowski: the mailing list? | 15:24 |
| czajkowski | mgz: ah yes what is the sign up page for that | 15:26 |
| czajkowski | it's not n the signature of the list | 15:26 |
| mgz | it's on lists.ubuntu.com | 15:26 |
| mgz | linked from the main page | 15:26 |
| czajkowski | ah | 15:29 |
| czajkowski | cheers | 15:29 |
| === slank is now known as slank_away | ||
| === slank_away is now known as slank | ||
| === salgado-lunch is now known as salgado | ||
| === gary_poster is now known as gary_poster|away | ||
| === gary_poster|away is now known as gary_poster | ||
| === al-maisan is now known as almaisan-away | ||
| === gary_poster|away is now known as gary_poster | ||
| === teknico_ is now known as teknico | ||
| === deryck is now known as deryck[lunch] | ||
| === teknico_ is now known as teknico | ||
| === deryck[lunch] is now known as deryck | ||
| === Ursinha_ is now known as Ursinha | ||
| dobey | anyone have any idea what might cause the dailydeb issues happening in https://launchpadlibrarian.net/127602321/buildlog.txt.gz ? running dailydeb locally works fine (save for my needed to use debupstream instead of debversion, otherwise it fails immediately on that) | 19:58 |
| dobey | #launchpad seems dead, so asking here | 19:58 |
| czajkowski | abentley: allenap gary_poster ^^^ | 20:57 |
| abentley | czajkowski: Sorry, don't know. | 21:02 |
| === benji changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On-call reviewer: - | Firefighting: - | Critical bugs: <150 | ||
| gary_poster | czajkowski, I have no idea I'm afraid. | 22:33 |
| hloeung | morning | 23:11 |
| czajkowski | hloeung: ello | 23:12 |
Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!