[00:01] wgrant: are you looking at the same url ? if so, sure [00:02] spm: is OOPS-1698EA2488 mirrored yet ? [00:02] or something wrong with lp-oops ? [00:03] wgrant: 4.18 seconds, hitting it again. [00:03] wgrant: so it -can- be fast; we may have caused disk io [01:17] working on blueprints presents interesting challenges [01:17] like not trying to fix absolutely everything [01:18] Delete delete delete. [01:40] i mean really [01:42] Hm? [02:00] oh sorry just venting [02:00] and some silly code [02:13] mwhudson: I feel your pain [02:13] been there [02:23] mwhudson: have you looked at the query counts for blueprints [02:52] lifeless: no, but given the naivety of the code i've just been looking at, i imagine they might be crazy [02:59] thumper: you around ? [02:59] kinda [03:03] thumper: can i call you at some point? [03:03] no massive hurry [03:03] sure, I'll ping you [03:03] cool [03:04] i think i'm going to end up with another preparatory branch [03:04] first was: move code i want to change [03:04] second looks like being: de-stupid code i want to change [03:06] OMG [03:06] ls lib/lp/blueprints/stories/blueprints [03:06] sequential page tests! [03:08] mwhudson: almost certainly [03:09] eek: I haven't used make lint in a while, and I just ran it on a branch. it seemed to have linted just about every file in the tree. That wasn't particularly useful. ./bin/lint.sh doesn't seem to point to any glaring mistake I made either :-/ [03:10] gary_poster: if there are no changed files (bzr st) it should do a diff against the parent (bzr info) i think? [03:10] gary_poster: did you have any uncommitted files ? [03:10] if not - what mwhudson said [03:10] no uncommitted files [03:10] gary_poster: while you're here [03:11] gary_poster: did you see the thread on lp dev about scripts and participations ? [03:11] gary_poster: bzr st -r parent: ? [03:11] bzr: ERROR: Requested revision: u'parent:' does not exist in branch: BzrBranch7('file:///home/gary/launchpad/lp-branches/devel/') [03:11] heh, that just blew up for me [03:11] if you didn't, mwhudson and I would appreciate your thoughts [03:11] parent branch: bzr+ssh://bazaar.launchpad.net/~launchpad-pqm/launchpad/devel/ [03:11] mwhudson: :parent I think [03:12] lifeless: oh right, that's what blew up [03:12] crashed for me [03:12] i just said the wrong thing to gary [03:12] lifeless: I saw it fly by but did not look closely; will look [03:12] "ReadOnlyError: A write attempt was made in a read only transaction on LockableFiles" [03:12] yeah, same here [03:12] mwhudson: personally, I'd do 'bzr st -r ancestor:..' [03:13] where ... is (for me) ../devel [03:13] lifeless: ah yeah, probably better [03:14] bzr st -r ancestor:../devel/ gives me the kind of list I expected make lint to use === Ursinha-afk is now known as Ursinha [03:14] * gary_poster will try make lint again for amusement sake [03:15] eek, no [03:16] * gary_poster googles "bzr set parent branch" [03:18] gary_poster: maybe you should just fix all the lint in the tree! [03:18] thanks a lot mwhudson :-) [03:19] OK, so I think this is the problem: [03:19] parent branch: bzr+ssh://bazaar.launchpad.net/~mars/launchpad/lsprof-on-demand/ [03:19] I want it to be my local devel (or a remote devel, whatever) [03:19] ah yes [03:20] i think bzr pull --remember will set parent [03:20] I tried bzr merge --remember ../devel/ [03:20] although of course bzr pull won't actually work [03:20] gary_poster: vim .bzr/branch/branch.conf is probably as easy as anything :/ [03:21] ok :-/ if it works that would be good though :-) [03:23] i don't know if the --remember config setting happens before or after the 'pull' part [03:23] all fixed, thank you [03:25] aararargh [03:25] if (ISpecification.providedBy(self.context) and [03:25] specurl == getattr(self.context, 'specurl')): [03:25] # The specurl wasn't changed [03:25] return [03:26] why getattr there? [03:37] lifeless: I have to go, but you were talking about the email thread from this week titled "performance tuesday - transparency," right? [03:58] gary_poster: yes [03:58] ok, will read and reply tomorry [04:20] mwhudson: it is just there to make you tear your hair out [04:20] mwhudson: shall I get a coffee before our call? [04:20] thumper: a cunning plan [04:21] thumper: yeah, i've become distracted by the craptitude of this code, i need to remember what i wanted to talk to you about :-) [04:21] :) [04:24] i want to cry [04:25] * spm hands mwhudson a box of kleenex [04:26] synmpathy from a losa only goes so far..... [04:28] How long is it since Blueprint was maintained? 3ish years? [04:38] possibly because it drives people mad? [04:40] it's a bit of a chicken and egg problem there [04:40] i'm making a small corner of it better at least [04:41] wgrant—hi! Say, how is BuildFarmJob coupled to a Job or BuildQueue? Is there any chance that the state of a BuildFarmJob fell out of sync with that of the Job? [04:42] We've got a bunch of TranslationTemplateBuildJobs that were starved out of the build farm as far as we can tell, but now the build farm is mostly idle again and those jobs aren't being picked up. [04:42] jtv: Yes, there are a few ways they can become inconsistent. [04:42] Although less for translations jobs. [04:42] let's see... [04:43] jtv: The i386 virt queue is still 500 items and 11 hours deep. [04:43] That's not what I'd call idle. [04:43] ah [04:43] so many i386 builders idle… all nonvirtual? [04:43] Yes. [04:43] oic [04:44] There used to be just three, but as of yesterday there are five. [04:44] I'm not sure where all the virt builders have been for the last week. [04:44] so that ought to help a little [04:44] How? [04:44] Translations jobs build on virt builders. [04:44] So the extra non-virt ones just slow down buildd-manager for you :) [04:45] Oh, the extra ones are non-virtual. I misunderstood that. [04:46] There are normally 15 or so i386 virt builders. [04:46] Now there are 6. [04:46] that'd explain a few things… [04:47] wgrant: I don't see anything that would associate a BuildFarmJob with a Job or BuildQueue. How does it connect to things? [04:48] jtv: Ha ha ha. [04:48] jtv: Well. [04:48] jtv: http://people.ubuntu.com/~wgrant/launchpad/buildfarm/current-build-model.png [04:49] That's how it roughly is at the moment. [04:49] Except that it's been partially converted to http://people.ubuntu.com/~wgrant/launchpad/buildfarm/new-build-model-again.png [04:49] But the link tables (BuildPackageJob, SourcePackageRecipeBuildJob) are still in place. [04:49] (That's a relief, because I didn't see any BuildFarmJob at all in the former) [04:50] (I still say the arrows look the wrong way around to me) [04:51] The parts of the new model that have so far been adopted are the merging and splitting of BinaryPackageBuild and SourcePackageRecipeBuild into BuildFarmJob, PackageBuild, BinaryPackageBuild, and SourcePackageRecipeBuild. [04:51] Yes, shutup :P [04:51] Everything else hangs on the translations jobs being ported to sit on top of BFJ. [04:51] BuildPackageJob and other link tables need to remain in place until we have you off the old infrastructure. [04:53] * jtv is disappointed to see that shaking his head doesn't clear the cobwebs [04:54] Hmm? [04:54] It's just dizzying. [04:55] Being somewhere inbetween these two schemata. [04:55] Yes :( [04:55] With the arrows—sorry, but it's really not helping!—all pointing the other way. [04:59] wgrant: also, I _still_ don't see how the translation template build jobs are linked to buildfarmjobs. [04:59] jtv: Even in the second diagram? [04:59] A TranslationsTemplateBuild has a BuildFarmJob and a Branch. [04:59] wgrant: I mean in the current schema—there being no TranslationTemplatesBuild table. [04:59] Ah. [05:00] In the current model, they're not linked to a BuildFarmJob. [05:00] The link to BuildFarmJob has to remain memory-only, which ISTM is getting more and more tenuous. [05:01] There's just a TranslationTemplatesBuildJob (BranchJob) linked to a Job linked to a BuildQueue. [05:01] You're different from the others, since you don't have a permanent object. [05:04] So does the absence of a buildfarmjob currently matter? [05:05] Only in that it prevents us from radically simplifying the mess. [05:07] So it's not likely that we've got jobs stuck forever just because of this, but all the more urgent that we update our part of the schema. [05:07] Right. [05:07] I don't think you have stuck jobs. [05:07] I think they're just at the end of the queue. [05:09] Almost certainly, if it's true that the farm is currently swamped by PPA jobs coming in mostly at a score of 2505. Until the cherrypick that spm deployed for us just now, our jobs went in at 1000. [05:10] jtv: You can see at https://launchpad.net/builders that the queue is large. [05:11] wgrant: quite [05:11] It's just hard to get an impression of more subtle aspects such as "what's the queue like for the group of builders that can handle our jobs?" [05:26] Whoops, did my local launchpad repository just break? [05:26] Can't pull any of the LP branches now. [05:26] No such file or directory: u'/home/jtv/canonical/lp-branches/.bzr/repository/indices/f16f01b514931e8ff2433001ed359d47.rix' [05:27] lifeless: what do I do about a broken repo? [05:27] bzr check? [05:28] * jtv tries [05:28] That's busily checking away [05:28] argl [05:28] mwhudson: try argv—it's called a vector, not a list for some reason [05:28] ECONTEXT [05:30] oh, ./bin/ec2 is slightly broken [05:32] argh [05:33] really? [05:33] hmm [05:33] I still use utilities/ec2 :) [05:34] well [05:34] that's just a symlink [05:34] it only matters if you have "debug_flags = hpss" in your config, i think [05:34] https://bugs.edge.launchpad.net/launchpad-foundations/+bug/624434 [05:34] <_mup_> Bug #624434: ec2 needs to initialize bzrlib [05:35] about 50% of the time needed to fix this bug is waiting for "make build" :( [05:37] mwhudson: make compile isn't enough? [05:37] :) [05:37] It skips the WADL and related crap. [05:37] So it doesn't take five minutes. [05:37] yeah, that probably would have been ok here [05:37] the WADL generation is being moved out of make build [05:38] and WADL will be checked into the tree [05:38] see gary for details :) [05:39] that'd be nice [05:41] Um, what happens if it changes? [05:45] jtv: depends on the breakage [05:45] jtv: #bzr is a good place to ask [05:46] lifeless: thanks. "bzr check" has been recommended to me, and I'm running that now. [05:47] jtv: do you have other files called /home/jtv/canonical/lp-branches/.bzr/repository/indices/f16f01b514931e8ff2433001ed359d47.* ? [05:47] or well /home/jtv/canonical/lp-branches/.bzr/repository/*/f16f01b514931e8ff2433001ed359d47.* [05:48] obsolete_packs/f16f01b514931e8ff2433001ed359d47.cix [05:48] obsolete_packs/f16f01b514931e8ff2433001ed359d47.iix [05:48] obsolete_packs/f16f01b514931e8ff2433001ed359d47.pack [05:48] obsolete_packs/f16f01b514931e8ff2433001ed359d47.rix [05:48] obsolete_packs/f16f01b514931e8ff2433001ed359d47.six [05:48] obsolete_packs/f16f01b514931e8ff2433001ed359d47.tix [05:48] that's it [05:48] mmm [05:48] looks like pack-names didn't get updated properly? [05:49] does it? [05:49] maybe [05:49] * jtv is devoid of clue [05:49] lifeless would know more than me [05:49] jtv: bzr stores all data in packs, named after some checksum of the contents [05:50] jtv: the pack-names file lists which pack files are curreent [05:50] jtv: repacking coalesces pack files, so removes some [05:51] so one way you could get in this situation is if packs were combined, the old ones moved to obsolete_packs and then the update of pack-names file didn't succeed for some reason [05:51] but i don't know the order things happen in off the top of my head [05:51] in fact i would guess the pack-names update would be done _before_ obsoleting the packs [05:51] I think this happened while I was pulling several branches at the same time. Maybe a race condition? [05:52] let me also try if my fs is still writeable; it corrupts itself sometimes [05:52] yup, still writable [05:54] a missing index [05:54] means your fs failed to complete a write [05:56] so a previous instance of fs corruption is the most likely cause… odd though that it started failing in the middle of a bunch of pulls though. [05:59] we only do IO as required [05:59] so we don't check that every index is there every time [06:00] so, a pack-name update that wasn't flushed looks plausible [06:00] you could see if there is a tmp file with a newer date [06:00] and if so, (in a copy of the repo, of course) [06:00] try putting it in place [06:04] lifeless: where would I find that? And shouldn't I wait for "bzr check" to complete first? [06:10] check doesn't handle this sort of problem [06:10] .bzr/repository [06:10] and - seriously - hop into #bzr [06:11] grab - probably spiv right now [06:27] * mwhudson eods === almaisan-away is now known as al-maisan [08:39] jtv: Looks like your build score changes worked. [08:39] wgrant: are we building one? [08:39] jtv: Yeah. [08:39] https://edge.launchpad.net/builders/rhenium [08:39] yup, see it! [08:40] Thanks for the heads up [08:41] However, the i386 queue is looking pretty sad. [08:41] I wonder if lamont rolled out the new lp-buildd this morning. [08:42] Then we can easily reassign some amd64 builders to i386... [08:43] good morning [08:45] hi adeuring [08:45] hi jtv! [09:00] allenap: will you be landing your alternative property cache branch soon? It looks great (tests will be much nicer to read :) ). [09:06] actually, just saw that there is canonical.cachedproperty.is_cached too. [09:15] Hello [09:33] bigjools: Yes, db-devel merges into devel have really bad revnos :( [10:48] mrevell: just dumping a branch on you may have been a little rough yesterday… would you prefer a series of HTML snapshots? Won't show the full interaction, but will show the different forms the message can take. [10:50] jtv, No a branch is fine, I'm just in the middle of transcribing a particularly difficult to understand interview. I'll have that finished this morning and will get to the translations help this afternoon, thanks. [10:50] mrevell, particularly hard-to-understand? I don't remember talking to you! [10:50] danilos, heh [10:50] mrevell: thanks—looking forward to it! [10:50] (ignore the danilo) [10:50] mrevell, who deals with commercial emails on feedback@lp? [10:53] danilos, me [10:54] mrevell, there's one from 17th that I haven't seen a response to [10:54] mrevell, and another one 5 days ago [10:56] Anything that old will definitely have been dealt with. [10:59] wgrant: Thankyou for removing BuildBase :) [11:00] Q: do we have anything that actually runs the tests in lib/canonical/launchpad/windmill/jstests/launchpad_ajax.js ? [11:00] noodles775: Still need to move the last couple of things from the interface file. [11:00] noodles775: I'll move BuildStatus to BuildFarmJob, but I'm not sure where to put BUILDD_MANAGER_LOG_NAME. [11:01] wgrant: I'm assuming BuildStatus should go to lp.buildmaster.enums? [11:01] noodles775: I guess it could. [11:01] if anybody other than Code is doing that now, which it seems is the case now. [11:01] Yep. [11:02] bigjools: Where should lp.buildmaster.interfaces.buildbase.BUILDD_MANAGER_LOG_NAME go? [11:02] I think it should probably not be BUILDD_MANAGER_*, since that's an implementation detail. [11:13] wgrant: ummm, I wonder if it's worth making lp.buildmaster.constants [11:14] or it could go in enums [11:16] Why does it even exist? It's used in two places and represents a logging config name... [11:17] slave-scanner is still hardcoded in several places. [11:17] It should be used more widely, or we should pass loggers around more aggressively. [11:17] wgrant, my branch makes us do that [11:17] Or we could just resolve to hardcode it more heavily. [11:17] jelmer: Ah! [11:17] (pass loggers around more aggressively) [11:17] Excellent. [11:18] jelmer: Is BUILDD_MANAGER_LOG_NAME still used anywhere? [11:19] jelmer: Is this your Popen removal branch? [11:19] wgrant: yeah [11:19] * noodles775 would prefer to see the module names used with getLogger (ie. getLogger('lp.buildmaster.manager') and the logging configs updated to do the right thing - where possible) [11:19] wgrant: IIRC BUILDD_MANAGER_LOG_NAME is still used once or twice. [11:19] noodles775: But I don't think that's entirely correct, since it's not necessarily buildd-manager that's calling things. [11:20] noodles775: Although it will be the only caller, once we remove SlaveScanner. [11:20] jelmer: :( [11:20] wgrant, I'm sure we can fix that afterwards.. [11:21] Yeah. [11:22] wgrant: I thought the reason for using the module name as a standard for getLogger was to indicate the context of the log message, not the call-site? [11:23] noodles775: Ah, I just took your lp.buildmaster.manager example in the context of the calls I was thinking of, which are not in fact in buildd-manager itself. [11:23] Never mind me. [11:32] +1 for passing loggers around [11:34] jml: is that from experience, or is there an good article you'd recommend? From reading about pythons logging configuration options, I'd assumed part of their intention was to remove the need to pass loggers around? [11:34] noodles775, experience [11:35] noodles775, every time I think mutable global state is a good idea and actually try it, it turns out to be a bad idea [11:36] last I heard, flacoste was not so keen on passing loggers around [11:38] danilos: I question the fix suggestion in bug #522800. From what was it derived? [11:38] <_mup_> Bug #522800: Broken link in PPA package details page (404) [11:38] passing them around is easy to read, easy to write, easy to change and doesn't reduce functionality [11:38] The problem is a bit less serious than that fix suggests. [11:39] wgrant: I forgot what you said about it [11:39] it also makes it _really_ obvious what needs a logger and what doesn't [11:39] bigjools: The query returns the oldest LFA with the given name. [11:39] bigjools: It should filter out expired ones. [11:39] ah right [11:40] would you mind updating the bug? I think you said you were going to do that :) [11:40] There are unexpired ones there. I can see them through the librarian's search interface. [11:40] Heh, possibly. Uni has been keeping me busy. [11:40] * wgrant updates the bug. [11:40] Uni didn't keep me busy [11:40] well - not academically :) [11:41] Heh. [11:41] The team project is actually taking up time. The first subject to actually do so :( [11:42] wgrant, the fix suggestion came from bigjools, but I see you already resolved that :) [11:42] I'll tell you a story about one of mine next time we meet up [11:42] Heh. [11:42] danilos: yeah I was wrong in my assumption :() [11:42] See, this project has classically been done over a year. [11:42] This is the first time they've crushed it into a semester. [11:42] wgrant, can you update the description or should I (while I have the bug open)? [11:42] I'm not sure they've really thought it through. [11:43] danilos: I'm doing it. [11:43] wgrant, cool, thanks [11:43] geez, LP's test suite even brings my four-core to its knees. [11:43] is it using more than one core? [11:43] jml: but to add a logging statement to a method that previously didn't log anything, you've got to update the method signature and (possibly) all call-sites, and potentially the signatures of the callsites which may not yet have the logger themselves? I'm not clear why you think it's good that the decision to log something in a method changes the method. [11:43] bigjools: Need more SSD! [11:44] jml: yes, but not in the way you think [11:44] disk access is the killer [11:44] jml: but that scenario might just indicate badly-factored code anyway. [11:44] and memory hogging [11:44] noodles775, for the case where "log" is little more than a print statement, that's a fair point [11:44] wgrant: I think I can soon start heating my office from the machines in here [11:44] noodles775, but so often with Launchpad, logging is srs bsns [11:45] bigjools: Have you acquired a new one to replace your previous quad-core? [11:45] I dunno. I'm also going through a phase of making interfaces with methods that correspond to events I care about, and using that for logging. [11:46] wgrant: nope I'm still happy with this one [11:46] unless you count the pseudo 4-core in the i5 :) [11:46] That's not a real quad-core [11:47] jml: personally I agree with noodles775 here, changing method signatures to accommodate logging objects sounds quite evil [11:47] indeed, "pseudo" could imply little else. [11:47] yeah I was about to say ... [11:47] Oh, bleh [11:47] * StevenK goes back to terrorizing dogfood [11:47] * bigjools likes to call it a fork-whore in honour of the Two Ronnies [11:48] bigjools: The only thing I know from Two Ronnies is "and that's good night from 'im" :-( [11:48] bigjools, I've more frequently encountered evil where I've had to mix code that each does it's own logging and neither bits of code have given me permission to touch their logging [11:49] because it's acquired from a global rather than passed in. [11:49] or refactoring a chunk of code that has log statements all over the place [11:49] in any case, more often I pass logging objects into constructors [11:50] ah, that would help :) [11:50] StevenK: http://www.youtube.com/watch?v=qu9MptWyCB8 [11:52] jml: but I wonder if the evil you've encountered is related to logging not being configured correctly in the first place? (/s/correctly/optimally) Not sure. [11:53] noodles775, maybe. I can't recall enough detail. [11:53] noodles775, things not being right in the first place is certainly a strong theme in the soundtrack of my programming career :) [11:54] jml: but being right in the end I'm sure too :) [11:54] noodles775, software is never finished, it just runs out of funding [12:01] Ursinha, hi (when you are around), does this bug activity look like a problem with qa-tagging scripts? https://bugs.edge.launchpad.net/rosetta/+bug/618987 [12:01] <_mup_> Bug #618987: gc.get_objects() interacts badly with bzrlib lazy imports

[12:05] Morning, all. [12:05] morning === al-maisan is now known as almaisan-away === matsubara-afk is now known as matsubara === henninge_ is now known as henninge [14:27] noodles775: I tried to land it yesterday but there were some test failures. I'm away until Tuesday and I probably won't have enough time to fix it before then, sorry. I'll migrate any code that lands before then though. [14:27] allenap: np. Enjoy your break! [14:29] noodles775: Cheers :) [14:33] salgado, i'm looking into the oauth token scope thing we talked about yesterday, but i'm remembering other problems with it. for instance, a web service client with a scoped oauth token can't get the service root [14:33] because the service root is not in scope [14:33] an oauth token is also scoped to a person, but the person will be out of scope [14:34] leonardr, when things are out of scope the client should have read access still, no? [14:34] salgado: we'll find out. i don't know if i actually tested this [14:34] so it could still work [14:34] maybe that's not how it works today, but I think that's how it should work [14:36] that makes sense [14:43] wgrant, we have a solution to the timeout problem i last talked with you about on july 20th [14:44] i'm delegating to benji the responsibility for updating the script you're running on optusnet.com.au [14:44] basically, you need to upgrade to the latest versions of launchpadlib and lazr.restfulclient [14:45] you need to make your script use edge (until the changes are deployed in the next launchpad release) [14:45] and if you were using the ._wadl_resource.total_size hack to get the length of a collection, you need to change that code to use len() [14:47] gary -^ delegating to benji [14:48] ack thanks [14:53] salgado, i see how the oauth context could be a specific project or distribution or whatever, but not how it could be a particular _class_ of object. do you have any ideas? [14:56] leonardr, indeed, the scope is a db object and not a python class. you could scope it to the person, but I guess that wouldn't help you in this case? [14:56] scoping it to the person would be a good start [14:59] or maybe not... since the person's oauth tokens are probably not considered to be in scope === almaisan-away is now known as al-maisan [14:59] i'll give it a try [14:59] wow, comment fonts are huge on my system now. [15:00] Feel like I'm reading a large print old-folks book. [15:00] deryck: I just noticed the same thing; I like big text but that is taking it a bit far :) [15:01] benji, indeed :-) [15:01] It's falling back to monospace for me and at 116% [15:01] perhaps the percentage worked for whatever font we used to request. [15:03] I'm not aware of a way to have different sizes for different type faces... but I'm no CSS expert either. [15:06] I don't seem to have a mono version of the ubuntu font either. [15:08] jml, hi. See my good-god-that's-a-huge font comments above. ^^ [15:08] jml, should I file a bug on launchpad-web? [15:08] deryck, huh what, yes. [15:08] heh [15:08] deryck, there's no Ubuntu monospace yet. [15:08] soon, I'm told. [15:09] ok, I didn't think so. [15:09] jml, should we just scale back the percentage on the monospace then, since we know we're only using that currently? [15:11] deryck, sorry, otp, that sounds reasonable [15:11] jml, np. Sorry to interrupt. I'll work up a branch now and see how it looks. [15:11] deryck, np [15:11] deryck, a bug with screenshots would help the reviewer. [15:12] jml, ack [15:26] deryck, beuno, fwiw, they don't look huge to me. [15:27] oh, 'comment fonts' [15:27] jml, right [15:27] and merge proposal diffs [15:27] no, same as ever. [15:27] beuno, they do look huge to you? [15:27] deryck, yes, huge [15:27] I'll take a screenshot [15:28] beuno, yeah, they look huge to me, and benji confirmed. But just making sure. [15:28]