[01:36] * mwhudson shoves branching-ubuntu into ec2 land (at last!) and goes to lunch [01:37] \o/ [01:37] thanks mwhudson [01:38] james_w: you can see the diff at https://code.edge.launchpad.net/~mwhudson/launchpad/branching-ubuntu/+merge/13269 if you want [01:38] james_w: shouldn't you be asleep? [01:38] I should [01:38] so I will look tomorrow [01:40] cool [02:22] thumper: didn't i worry at you about which zcml was executing in scripts vs the webapp? [02:23] thumper: looking at the errorreportingutility lookup failures [02:23] mwhudson: yes you did [02:23] oh well, use globalErrorReportingUtility and don't worry over much i guess :/ [02:24] mwhudson: my branch adds the registration of the utility for scripts [02:24] mwhudson: after talking with flacoste_afk [02:24] thumper: ah cool, that's a better solution [02:24] mwhudson: the code should be moved from c.l.webapp to lp.services.error [02:24] but hey [02:25] thumper: world hunger also needs fixing [02:26] mwhudson: :) [02:26] my desire to file bugs about ubuntu drops significantly as soon as things work again, even if intermittantly [02:37] hmm [02:37] the commit message area in https://code.edge.launchpad.net/~mwhudson/launchpad/branching-ubuntu/+merge/13269 looks seriously ugly [02:39] mwhudson: yes it does [02:39] * thumper has a cunning plan [03:54] mwhudson: https://pastebin.canonical.com/23283/ [03:54] https://code.edge.launchpad.net/~andrea-bs/launchpadlib/fix-set_data_store/+merge/619 [03:54] has a null date review requested [03:54] causing the oops [03:54] now how the hell did that happen? [03:54] thumper: that's really olkd [03:54] old [03:55] 619? yeah [03:55] I wonder if it slipped through early before we knew what we were doing [03:55] ... [03:58] rejected the proposal and now we have a working active reviews again [03:58] should look into that at some stage [03:59] maybe add a DB constraint [04:05] mwhudson: http://pastebin.ubuntu.com/292816/ ideas? [04:05] thumper: apt-get install python-boto ?? [04:05] what is boto? [04:06] like launchpadlib is to launchpad, boto is to amazon web services [04:06] thumper: have you run ec2 on this laptop yet? [04:06] just about to [04:07] mwhudson: does land have --dry-run? [04:07] thumper: yes [04:07] heh, it tells me the mp is not approved :) [04:07] if I approve it does that make it r=me or you? [04:08] thumper: it looks at the votes [04:08] thumper: finish autoapprove already :-) [04:08] mwhudson: help me come up with the db schema and we can land it this cycle [04:20] * thumper needs to copy across aws credentials [05:50] sigh [05:51] so it turns out that writing a simulator for a confusingly concurrent situation is, in and of itself, confusingly concurrent [05:52] Can mocker help? [05:53] stub: several processes involved so maybe not [06:10] hmmm straage, two successive calls to acquireBranchToPull are returning the same branch [06:32] well, that's a bug to fix i guess [06:54] argh [06:55] there's no way to tell the difference between a branch which failed last time that the puller is currently working on and one that failed last time :( [06:56] i can think of various stupid ways that you could encode this, but maybe we should just use jobs [06:59] right *now* however, i'm going to the pub [08:21] good morning [09:15] morning [09:27] bigjools: I think I am attempting the impossible (establishing the matching publication for a deb's ddeb). [09:28] wgrant: it should be possible [09:28] you need to join via the BPR [09:29] bigjools: I know, but there is no way to identify which of the potentially several BPPHs matches (think overrides). [09:29] wgrant: the ddeb should not be overridden differently [09:29] wgrant: let me know if you've got any improvements for bug 450124 [09:29] Bug #450124: Improvements to IBuilder.findBuildCandidate() [09:29] Malone bug 450124 in soyuz "Improvements to IBuilder.findBuildCandidate()" [High,In progress] https://launchpad.net/bugs/450124 [09:29] Bug #450124: Improvements to IBuilder.findBuildCandidate() [09:29] Bug #450124: Improvements to IBuilder.findBuildCandidate() [09:29] GAH [09:30] bigjools: Matching based on overrides sounds horribly fragile. [09:30] who keeps putting bots in here [09:30] And it's not been donebefore. [09:30] * wgrant hunts out jussi01. [09:30] it's not an override, it's a matching publication in the same series, component and pocket [09:31] But there's nothing unique about that. [09:31] apart from the archive [09:32] I mean, I can easily have duplicate (bpr, archive, distroseries, pocket, component, section) [09:33] wgrant: just get one of your ops to quiet it for now is probably the best course of action [09:33] wgrant: I dont have access to ubot3, thats nalioth, but if you just quiet it, it sits here and isnt a problem [09:33] jussi01: ubot3 says you own it :( [09:34] wgrant: unfortunately thats because it syncs the factoids from ubottu [09:34] jussi01: Ahh. [09:34] wgrant: well yes any model code can screw up the DB, but if it's encapsulated properly and the rules are self-contained then I don't see a problem [09:35] bigjools: I have a deb and ddeb, in universe. I promote them to main, then demote them again. [09:35] Now there are three publications for each, but two are duplicates. [09:35] I now remove the deb, and it tries to drag along the ddeb. [09:35] But how does it know whihc publication is the right one? [09:36] wgrant: the one that's PUBLISHED [09:36] bigjools: But I did all three override changes semi-accidentally in a single publication cycle. [09:36] or the one with the most recent ID [09:36] (it has happened) [09:37] then you need to supersede old publications if you find inconsistencies [09:37] Inconsistencies? [09:37] gimme 10 I need to have a call [09:37] Sure. [09:49] hello [09:50] Hi, has anyone else seen failures on lib/canonical/launchpad/pagetests/webservice/xx-wadl.txt ? [09:52] hmm [09:52] That name is familiar. What sort of failure? [09:52] wadl_schema.validate(wadl) returning False [09:52] maxb: passes here locally (r9691 on karmic) [09:53] noodles775: How "interesting" :-) [09:56] maxb: the wadl file is generated automatically, so my guess is that something went wrong generating it [09:56] maxb, I get that failure when running the test on the stable branch. [09:57] updating download-cache & running 'make build' [09:59] still get the error [10:00] wgrant: ok back [10:00] so, you just need to take the publication with the most recent ID [10:00] the publisher should supersede the others [10:01] bigjools: Even if the latest one is no longer active? That sounds convenient. [10:01] Although still a bit ew. [10:01] yeah [10:02] which makes me think, you might want to check that the publisher will work with debug archives :) [10:02] Actually, this isn't safe. [10:02] deb_bpph.supersede() should also supersede the corresponding ddeb. [10:03] yes [10:03] This is to handle the case that a package ceases to produce a ddeb. [10:03] But what if it doesn't cease to produce it? [10:03] Hmm. [10:03] I guess it should be safe as long as we only look within the build. [10:04] Yeah, I guess that will work. [10:04] I don't quite follow this case [10:05] I wasn't thinking properly. There is no problem. [10:05] ok :) [10:05] It is guessing, but very safely. [10:05] if we assume the publisher will deal with old publications, taking the most recent one is always safe [10:06] Right. I had forgotten about other restrictions on the search. [10:07] Maybe it's not safe. How I hate guessing. [10:08] I have a deb and ddeb, both with Published publications. I promote them, giving them a new Pending publication each. [10:08] The publisher runs. [10:08] It sees that the deb is superseded, and calls deb_bpph.supersede() [10:08] The Pending ddeb publication screams and dies. [10:08] This is not correct behaviour. [10:09] s/deb_bpph/deb_old_bpph/ [10:10] bigjools, are there docs anywhere describing how the buildmaster works? [10:10] Ha ha ha. [10:10] jml, this is Soyuz. [10:10] ROFL [10:11] fair point. [10:11] buildd-dispatching.txt ? [10:11] wgrant, do _you_ have docs describing how the buildmaster works? [10:11] thge buildmaster was oringally hacked up by the early LP devs (daniel etc) and more recently re-written by Celso in a hurry [10:11] yes, the doctest is the closest we have [10:11] jml: Although I know it internally fairly well now, I only have docs on how to run it. [10:11] yeah, I remember him being in a hurry. :) [10:12] when wasn't he? :) [10:13] jml we have a plan on how to generalise the buildmaster side but it was written before the re-write of the scanner [10:13] bigjools: Any ideas on the above example? [10:13] * bigjools missed that, just a sec [10:13] bigjools, is the plan written down? [10:13] https://dev.launchpad.net/Soyuz/Specs/BuilddGeneralisation [10:13] I might have a poke around the code once I do today's CHR duties & finish off this damn imports branch and review al-maisan's branch. [10:13] I guess that means tomorrow. [10:14] wgrant, thanks. [10:14] wgrant: why would it scream and die? [10:14] [10:14] [10:14] ^ Well, oops :-) [10:15] bigjools: Because it has the latest ID, so gets superseded. [10:15] jml: the code is pretty bad, most of it was written before we had real coding standards. Feel free to hassle me to explain stuff. [10:15] * al-maisan takes a peek at lp:~jml/launchpad/upload-permission-joy [10:16] wgrant: ok I see [10:18] wgrant: if the last publication is pending then it should be clever enough to go to the previous one [10:19] bigjools: This is getting more and more into inaccurate guess territory, and sounds like it will go horribly wrong if there are multiple pending publications at once. [10:19] bigjools, I remember someone asking me for a generic sftp server based on the one we have in codehosting... [10:19] I'd like one of those too :-) [10:19] bigjools, I guess that's for a different thing [10:20] jml: yeah we could do with replacing poppy [10:20] maxb, remind me to actually put license text in txsshserver one of these days. [10:21] wgrant: there is already code like this in the publisher [10:21] jml: Your verification of the xx-wadl.txt failure was on jaunty/karmic? py2.4? [10:21] wgrant: I suggest we try and map out all of the operations that can happen like this, and work through them carefully [10:22] maxb, karmic. [10:22] maxb, ./bin/test -1cvvt xx-wadl.txt in stable branch [10:23] * maxb wonders if this is a mysterious jaunty vs. karmic oddity [10:23] maxb, I'm deleting the unversioned files in lib/canonical/launchpad/apidoc and running the test again. [10:23] jml: the wadl tested in the test is generated on the fly [10:24] maxb, ... as my experiment just demonstrated :) [10:25] I'll have to dig out a jaunty system and retest there [10:27] * bigjools laughs a lot at https://answers.launchpad.net/soyuz/+question/85773 [10:29] jml --^ [10:33] How does the wadl generation actually work? There is stuff in the comments in the wadl which I don't find by grepping anywhere in the source?! [10:36] bigjools, heh heh. [10:36] bigjools, what's the blueprint URL? [10:36] maxb: Possibly stuff in lazr.restful(client)? [10:36] jml: I haven't done one yet, I will sort it out soon [10:38] bigjools, no worries. [10:38] jml: I cleaned up the buildd manager one yesterday as that's more important to me right now :) [10:39] Hmm... looks like the bugged wadl is copied verbatim from wadl-root.py in lazr.restful [10:39] bigjools, ahh sure. [10:39] bigjools, I can just register the blueprint and link it to the already-written spec [10:40] jml: I can do it, it's on my list of things to do anyway [10:41] bigjools, ok. [10:41] * jml would really love to scrub the launchpad-code blueprints at some point [10:41] Hi - is there a good way to debug traversal errors? I'm getting a backtrace from base-layout-macros.pt for "json" and I didn't really find a way to find the original error. [10:43] hmm [10:43] jmux, nothing leaps to mind. [10:43] https://dev.launchpad.net/Debugging might help. [10:44] eww. lazr.restful has no tags :-( [10:44] does logger.exception in LP log sys.exc_info() or similar? [10:45] james_w, it really depends. [10:46] 258 + except: [10:46] 259 + ok = False [10:46] 260 + self.logger.exception( [10:46] 261 + "Unexpected error checking %s!", db_branch) [10:46] james_w, that behaviour comes standard in Python :) [10:46] I'd like to know that this logs the exception, otherwise we are going to be scratching our heads when something goes wrong [10:46] james_w, http://www.python.org/doc/2.4.4/lib/node341.html [10:47] ah, so it does [10:47] eg. try to catch an exception and call logging.exception("blah") [10:47] james_w, I'll need more context [10:47] thanks, will be useful in future [10:47] james_w, because it's the log _handlers_ that really make the difference [10:47] I'm looking at https://code.edge.launchpad.net/~mwhudson/launchpad/branching-ubuntu/+merge/13269 [10:47] def checkNewBranches(self): [10:48] so in this case logger comes from LaunchpadScript [10:49] james_w, which in turn uses canonical.launchpad.scripts.logger.logger [10:49] james_w, which uses a default Python StreamHandler [10:50] which is fine. [10:51] cool, thanks for checking [10:52] np [10:52] thanks for asking :) === dpm_ is now known as dpm [11:12] bigjools: http://williamgrant.id.au/f/1/2009/ddeb-supersedure-cases.html describes most supersedure cases, I think. [11:15] wgrant: ok, thanks, I'll look through it in a bit [12:37] bigjools, the buildd-generalizing spec is for running arbitrary jobs, right [12:37] jml: yes [12:38] bigjools, is there another spec for generalizing across machine type? (physical machine, local vm, ec2 instance etc) [12:38] there's a bug [12:38] well [12:39] there's a bug for building any type of arch/virtual on the existing builders [12:39] there's no spec for going as far as what you say there [12:39] ok, thanks. [12:40] which I think is slightly crazy BTW :) [12:40] perhaps it is. [12:41] what I'm actually worried about is build capacity [12:41] yes, me too [12:41] s/worried/thinking/ [12:41] we'll need a lot more builders [12:41] yeah. [12:41] we'll also need some more optimisation done to the buildd-manager [12:41] because finished builds are uploaded synchronously [12:42] oh right, I remember talking to cprov about that. [12:42] it's a very hard problem to solve [12:42] err, why? [12:42] error conditions mainly [12:43] I forgot the exact details [12:43] mmk [12:43] I may be overlooking something, but Debian has done uploads asynchronously for over a decade [12:43] bigjools, ISTR it as more being a case of getting round to it. [12:44] well don't let me stop you fixing the buildd-manager then ;) [12:44] it just means the extra section of the state machine where a package can be either 'uploaded' or 'installed' (i.e. the upload completed successfully) and you just have to make sure nothing stays in 'uploaded' for overly long [12:44] also, why are we worried about capacity? [12:44] ah I remember more about this now [12:44] dammit. etsu lunch again. [12:44] the upload processor is single-threaded [12:44] But it's run in a subprocess. [12:45] with a lock [12:45] Oh. Right. [12:45] and no, it's not a subprocess when uploading binaries [12:45] Then why does the builddmaster have a config option for the command to run to upload the binaries? [12:45] NFI [12:46] It spawns process-upload.py. [12:46] This is why it takes /forever/. [12:46] hmmm last time I looked it didn't do that [12:47] but anyway, it's still has a bunch of issues that prevent it running more than one copy at once [12:47] I am full of unhappiness and empty of food. [12:47] brb. [12:47] and there are other issues with the buildd-manager just dumping the binary in an upload queue - for example how to deal with upload failures [12:47] bigjools: lp.buildmaster.buildergroup.BuilderGroup.buildStatus_OK certainly seems to spawn a subprocess. [12:54] bigjools, how do you deal with them now? [12:54] it marks the build as failed [12:56] bigjools, I see. [12:56] which the upload processor can't do, since it doesn't know about the build [12:57] It does, actually. [12:57] bigjools, you could write the builder asynchronously [12:57] The upload processor sets the build to Successfully Built, and attaches the binaries. [12:57] If it completes successfully, the buildmaster attaches the build log, sets the finish time, unassigns the builder, deletes the buildqueue. [12:58] If it fails, it attaches the upload log and marks the build as Failed to Upload. [12:58] But the upload processor does (and has to) know about the build. === matsubara-afk is now known as matsubara [13:11] wgrant: of course, you're right. I think I need more coffee. === salgado-afk is now known as salgado === mrevell is now known as mrevell-lunch [14:33] EdwinGrubbs: barry: bac: standup in two minutes [14:39] sinzui: standup? [14:55] reviewers, interested developers, and lurkers: ameu reviewers meeting -> #launchpad-meeting in 5m [15:00] ameu reviewers meeting -> #launchpad-meeting [15:04] abentley: -> #launchpad-meeting. i'm about to talk about pipelines :) === mrevell-lunch is now known as mrevell [15:48] hi all [15:49] people do you know the place where I can get documentation about putting Launchpad on Debian [15:49] ? [16:00] Fox_1_: you are in unexplored territory. I am not aware is has been done, let alone documented [16:01] bac: ready for our call? [16:01] yes === flacoste_afk is now known as flacoste [16:46] leonardr, were you making a trusted client for the API? [16:46] jml: yes, it is on my list of things to do, but pretty far down the list [16:46] i can point you to the bug if you want to take over [16:47] leonardr, what if I don't want to take over but just want to leer speculatively, one day dreaming of having enough time to want to take over? [16:48] i think pointing you to the bug would still be appropriate [16:48] it might not be as difficult as you think, i did write the client, i just didn't write the tests for it [16:48] heh. [16:49] leonardr, in that case, please point me to the bug report. [16:49] I'm actually planning on working with launchpadlib on Canonical time during the Bazaar sprint in November. [16:49] so I might do trusted client work on the plane or something. maybe. [16:52] jml: https://bugs.launchpad.net/ubuntu/+source/ubuntu-dev-tools/+bug/387297 [16:52] Bug #387297: manage-credentials should not ask for Launchpad password directly [16:52] Malone bug 387297 in ubuntu-dev-tools "manage-credentials should not ask for Launchpad password directly" [Medium,Triaged] [16:52] https://code.launchpad.net/~leonardr/launchpadlib/trusted-client [16:52] Bug #387297: manage-credentials should not ask for Launchpad password directly [16:53] I thought we asked ubot3 to leave. :( [16:53] leonardr, thanks. [17:06] bigjools: halp. i don't know anything about upload permissions. can you give me some pointers? do you know how the driver permission for a distroseries is calculated? [17:06] intellectronica: I don't know that, I only know about upload permissions [17:08] intellectronica, for a distroseries? [17:08] hm [17:08] wait. I'm focusing. [17:08] jml: nm, i think i just found out [17:08] intellectronica, glad to hear it. === matsubara is now known as matsubara-lunch === beuno is now known as beuno-lunch === salgado is now known as salgado-lunch === beuno-lunch is now known as beuno [18:04] night all === matsubara-lunch is now known as matsubara === salgado-lunch is now known as salgado [19:21] danilos: ping [19:27] barry: danilos is ill today, and it's pretty late for him, I don't think he'll answer in a reasonable time :) [19:28] Ursinha: okay, thanks. danilos hope you feel better soon! === ubot3` is now known as ubot3 === EdwinGrubbs is now known as Edwin-lunch [21:05] * jgay attempts to install launchpad. [21:10] Does anybody know if I will likely get a successful build and install on intrepid? [21:11] jgay: unlikely [21:11] jgay: nothing fundamental, but some of the packages needed haven't been built for intrepid i think [21:12] mwhudson: ok, thanks. I'll see how far I get before things start to breakdown. [21:16] mwhudson, abentley, who's hosting today? [21:16] rockstar: good question! [21:16] mwhudson, would you like me to? [21:16] rockstar: Go for it. [21:16] rockstar: sure [21:30] The windmill tests don't run concurrently as threads, do they? [21:41] rockstar, mwhudson thumper's off line today. do you still want to meet today? [21:41] barry, yeah, abentley said today was pretty exciting. [21:41] rockstar: okay cool. i'll update the minutes and we can meet in 19m [21:42] barry, ack [21:42] barry: ok === Edwin-lunch is now known as EdwinGrubbs [22:01] rockstar, mwhudson, wgrant -> #launchpad-meeting === salgado is now known as salgado-afk [22:12] * rockstar goes on a short walk [22:50] intellectronica: jml refactored the upload permission stuff recently, so it should be pretty easy to remove the duplicated logic from the bug nomination permissions. === ursula is now known as Ursinha === matsubara is now known as matsubara-afk