[00:11] flacoste: we're waiting for my patch disabling the edge redirect code to propogate through QA [00:15] lifeless: What about recipes? [00:23] wgrant: we're waiting for N things [00:23] wgrant: one at a time [00:32] lifeless: But if the redirect removal lands, then recipes vanish from Launchpad. [00:32] lifeless: Which may be slightly undesirable. [00:33] s/lands/deloys/ [00:33] +p [00:37] huh, no [00:38] food, whilst I leave you to look closer. [02:11] Project devel build (156): FAILURE in 3 hr 33 min: https://hudson.wedontsleep.org/job/devel/156/ [02:11] * Launchpad Patch Queue Manager: [r=bac][ui=none][bug=666580] Make getMessageDispositions much more [02:11] efficient in the number of DB queries. [02:11] * Launchpad Patch Queue Manager: [r=jml][ui=none][no-qa] Remove with_statement. [02:11] * Launchpad Patch Queue Manager: [r=jml][ui=none][bug=54946, 496574, 497282, 611258, [02:11] 618955] New almost-fully-asynchronous buildd-manager with lots of hot [02:11] sexy bug fixes. [02:31] lifeless: I see nothing in that MP which enables BFB on lpnet. [02:31] Unless you've also changed the prod configs, I guess. [02:34] wgrant: disabling the edge redirect doesn't remove the edge appservers [02:34] and yes, the prod configs have (naturally) changed [02:36] lifeless: It doesn't remove them, but it stops people from being redirected, so it effectively removes them. [02:36] lifeless: did you change the prod config to enable recipes? [02:37] lifeless: it isn't is_edge [02:37] no [02:37] we're adding a feature scope to support recipes [02:37] we're stalled on the edge removal on that [02:38] ... that is what I was asking. [02:38] Aha. [02:38] wgrant: see lp-foundations bugs [02:43] lifeless: why stalled on edge removal? [02:44] jml doesn't want recipes on prod for all users [02:44] only beta [02:44] (and preferrably a dedicated smaller beta team) [02:44] so we need to add a scope that selects teams. [02:45] ok [02:46] thumper: there's no panic on removing edge, we have time to do it methodically [02:46] ok [02:47] all due haste, no panic. [02:50] lifeless: So you just CP'd up to r11738? [02:50] deployed, yes. [02:50] Hopefully the publisher won't explode. [02:50] CP's no longer exist. [02:50] Shh. [02:50] wgrant: why would it? We didn't deploy to soyuz machines [02:50] Ah. [02:50] So the Fix Released was a lie. [02:51] wgrant: which bug [02:51] Bug #655690 [02:52] wgrant: well, its a nonfunctional change, right? [02:52] wgrant: or is it broken? [02:53] lifeless: No, it's fine. Shouldn't be a problem, except it's happened in the past that a CP has been partially deployed, marked as Fix Released, something has broken, and then confusion abounds when you're looking at the wrong code. [02:53] well the rev is on the bug [02:54] True. [02:54] I'm not sure what we should do here. [02:54] if it was a functional change, I wouldn't have toggled it. [02:54] Merge Fix Committed and Fix Released into Fixed, like mpt has wanted to do forever? [02:55] :) [02:57] thumper, did you see hudson failed on your branch? [02:58] thumper, well not just your branch... [03:01] no [03:10] * thumper afk to collect car [03:12] bac: looks screwed up [03:12] bac: as in hudson looks screwed up, not the branch [03:52] * StevenK prods at hudson before bed [03:53] Project db-devel build (103): FAILURE in 3 hr 57 min: https://hudson.wedontsleep.org/job/db-devel/103/ [03:54] That devel change is odd [03:54] Er, s/change/test failure/ [03:55] The test suite is run under sudo, so I am tempted to blame a recent landing [03:55] yes, yes it is [04:17] wallyworld_: skype? [04:17] ok [04:22] thumper: now? [04:22] wallyworld_: I can't hear you :) [04:22] i can't hear you either [04:23] !#@#@%@ pulse or whatever [04:23] i may have to reboot :-( [04:23] ack [05:00] contains(@class, "js-action") [05:36] thumper: you might like https://code.edge.launchpad.net/~lifeless/launchpad/zope.testing/+merge/39423 [05:36] jml: or you may like it. [05:36] gnight [06:03] Project devel build (157): STILL FAILING in 3 hr 32 min: https://hudson.wedontsleep.org/job/devel/157/ [06:03] Launchpad Patch Queue Manager: [r=lifeless][ui=none][no-qa] Prevent the atexit warning that occurs [06:03] when tests are run within the LibrarianLayer. [08:54] good morning [09:03] Hello [09:12] hello mrevell, adeuring [09:17] Someone is going to need to manually apply the permission changes from db-devel r9888, or we are going to have an awful lot of branch scan failures. [09:18] (that was merged into devel later on, and has security.cfg changes, which can't be deployed directly without downtime) [09:18] (but the revs that need the new perms were deployed earlier today) === thekorn is now known as th3k0rn [09:31] wgrant: interesting - this seems to be something of a failure in process... :/ [09:32] mthaddon: One that people have been warned about. [09:32] I don't recall exactly where. [09:32] But it was brought up recently. [09:34] yeah [09:35] so allegedly qastaging should fix this, but it seems this wasn't exactly qa-ed there... [09:36] qastaging may have been set up too late. [09:36] So it may have had the right perms from the start. [09:36] well, and we're not running scripts on it yet... [09:36] Ah, that would also do it. [09:36] but my point is the revno was blessed for rollout without having really been QA-ed [09:37] It's almost impossible to completely QA something like that. [09:37] But more could have been done. [09:37] anyway, thx for bringing it up [09:37] It's also possible that it was QA'd properly on db-devel, and then merged without properly being QA'd on devel. [09:37] But anyway. === th3k0rn is now known as thekorn [09:42] jtv: yup, the main facet feigns ignorance about a series branch to non-privileged users. [09:42] if that branch is private. [09:43] henninge: and ours does the same for the development branch but not for the export branch? Or does it fail to feign and so feint when either is present? [09:44] * jtv gets confused about spelling [09:44] jtv: ;-) the latter. [09:45] jtv: so, if translations would say somthing like "translations are imported from/exported to a private branch" it would already give away more information than the main page which claims "No revision control details recorded for ..." [09:46] Right. [09:46] Well, mystery solved I guess. Good job. :) [09:47] So our page will have to look to non-priv users as if no branch sync had been set up. === almaisan-away is now known as al-maisan [10:05] Yippie, build fixed! [10:05] Project devel build (158): FIXED in 3 hr 33 min: https://hudson.wedontsleep.org/job/devel/158/ [10:05] Launchpad Patch Queue Manager: [testfix][rs=bac][ui=none][bug=54946, 496574, 497282, 611258, [10:05] 618955][rollback=11801] === al-maisan is now known as almaisan-away === almaisan-away is now known as al-maisan === matsubara-afk is now known as matsubara [13:11] jml: oh hai [13:11] lifeless: hello [13:12] jml: I've fixed layers [13:12] lifeless: glad to hear it. [13:13] jml: it needs a review :) [13:13] lifeless: cool. I'll look at it once I'm done w/ email [13:17] -> food [13:39] Project devel build (159): SUCCESS in 3 hr 34 min: https://hudson.wedontsleep.org/job/devel/159/ [13:39] Launchpad Patch Queue Manager: [r=allenap][ui=none][bug=665407] Add invalid-link style to invalid lp [13:39] branch short links and prevent click through [13:47] jml: have you picked up a key? [13:47] lifeless: I don't think so. [13:51] I'll go get one [14:02] lifeless, jml: around? [14:04] jml: bony 3 [14:04] flacoste: yes [14:04] lifeless: I don't understand... [14:04] It sounds like a room title to me [14:05] lifeless, jml: call me when you are ready [14:06] jml: all the meeting rooms are booked; marianna has put us in bonaire 3 [14:07] flacoste: skype ? [14:08] lifeless: skype or POTS, whatever works best for you guys [14:08] flacoste: rephrasing; please join skype [14:08] lifeless: i should be there already [14:09] lifeless: are you hearing me?% [14:10] jml: I have some grants for production that accompany a security.cfg change that has already landed in devel. Are you the person I should ask to review these? === jtv is now known as jtv-afk [14:11] It's probably already in stable too, but I think all the grants are for script users so it hasn't caused problems so far. [14:14] allenap: he's in a session [14:15] bigjools: Thanks. I'll email him (unless you happen to know who else I could/should contact?). [14:16] allenap: I can review/grant for you if you want [14:17] bigjools: Awesome, thanks. http://paste.ubuntu.com/520757/ [14:18] allenap: how do you know you have all the necessary changes? [14:19] bigjools: Because you're reviewing it for me to check ;) [14:19] ! [14:19] allenap: it's my job to ask probing questions :) [14:20] bigjools: Do you know if it's possible to get Postgres to puke up it's permissions so I can compare before and after? [14:21] allenap: yeah I think I've seen something, can't remember what it is [14:21] bigjools: Okay, I'll investigate. Thank you for your probe. [14:22] allenap: so assuming that you've already landed the changes with these then it looks ok - my only concern is that you've not missed anything and it would be nice to somehow verify that. [14:22] allenap: FWIW, I also bow to your grep skillz [14:22] If you could get a permissions dump from a running development instance, that would probably help for comparion [14:23] *comparsion [14:23] StevenK: Yeah, that's what I have in mind. [14:24] jml: when you have a moment I could do with chatting about the new buildd-manager - the twistd process is hitting all of one core on mawson and strace is not helpful. [14:39] deryck: did you see bug 667215? :) [14:42] gary_poster: hi [14:42] hey on call but off soon bigjools [14:42] gary_poster: np, I am in a session soon though. Just wanted to talk about bug 667183 [14:44] I'll comment on the bug [14:45] ack [14:48] bigjools, yeah :-) I haven't replied or un-security'ed it yet. It's crying out for a clever response, and I don't want to disappoint. :-) [14:48] * deryck kids obviously [14:48] I just haven't replied yet. [14:48] deryck: seems ripe for "opinion" to me :) [14:49] heh === adeuring1 is now known as adeuring [14:49] abuse of the security tag is annoying though [14:51] yeah [14:53] launchpad session! [14:55] bigjools: will do. [14:55] bigjools: You were very right to ask that question: http://paste.ubuntu.com/520782/ [14:58] allenap: :) === al-maisan is now known as almaisan-away === almaisan-away is now known as al-maisan [15:33] mars ping [15:33] Hi bac [15:33] bac, regarding your earlier question, Ursinha would know [15:33] oh, ok [15:33] bac, and your EOD was 7:00am for me, I think? :) [15:33] Ursinha: ping... [15:34] sinzui, give me an example of your perfect pagetest story. [15:34] mars: yeah [15:34] mars: so you don't know which tools are supposed to set the bad-commit tags? [15:34] Ursinha: do you ? ^^ [15:35] if that could be documented on the the QAForContinuousRelease pages i think people would understand the process better [15:35] bac, not off the top of my head, no. Ursinha and lifeless worked that out, and I remember something on the list about it having the tags be manual(?) === Ursinha is now known as Ursinha-afk [15:35] mars: really? not what the wiki says at all. [15:36] well, i won't be getting my answer today, then. :) [15:36] hey, bac, as long as your online did you see my comment on the bug(s) we were talking about yesterday? [15:36] bac, that's why I did not reply. I thoguht lp-land or ec2 land should have done everything for you [15:36] mars: me too. this branch was full of mysteries. [15:37] bugs didn't get linked, diff didn't update, tags didn't update, MP didn't get set to 'merged'. all very odd. [15:37] some of it may be to branch scanner weirdness today [15:37] jcsackett: i did. thanks for following through. [15:37] bac: no problem. were you able to qa your stuff? [15:37] jcsackett: in the future, could you send me email? sometimes my IRC client tells me i got pinged but i cannot find it in the scrollback [15:38] jcsackett: i did [15:38] bac: oh, sure. [15:38] excellent. [15:46] bac: hello, would you mind testing a patch for me to fix that failing test of mine? === Ursinha-afk is now known as Ursinha === Ursinha is now known as Ursinha-afk === Ursinha-afk is now known as Ursinha [16:10] flacoste: can you invite me to the ISO call tomorrow/today ? [16:13] lifeless: sure [16:13] lifeless: not sure, it it's going to happen though given that elmo is at UDS, but we can probably have the call with tom === flacoste changed the topic of #launchpad-dev to: Launchpad Development Channel | Week 1 of 10.11 | PQM is open | firefighting: - | https://dev.launchpad.net/ | Get the code: https://dev.launchpad.net/Getting [16:16] flacoste: I'd like to talk with you, charlie & tom about some ops stuff from chatting with statik [16:17] flacoste: that call seems ideally structured [16:17] lifeless: sure [16:17] lifeless: you should have the invite [16:18] lifeless: btw, you aware that converting daily builds to feature flag is a blocker for edge removal? [16:18] flacoste: yes, I said so on the prereq bug [16:19] what bug is it? [16:19] flacoste: bug 666538 [16:19] (was just looking) [16:19] flacoste: we don't need to panic about removing edge - all due caution, and all due haste IMO [16:19] flacoste: we're already better off [16:20] matsubara: ping [16:20] lifeless: agreed, but the daily build conversion, did you intend to take it, or do we need to hand this off to the code team? [16:20] hi lifeless [16:20] matsubara: I'd like to interrupt you for a small but important thing. [16:20] shoot [16:20] matsubara: could you fold the 'edge' OOPS report summaries into lpnet. [16:20] matsubara: they are now running the exact same code. [16:20] matsubara: so we don't benefit from a separate report. [16:21] lifeless: ok, i see that the bug is assigned to poolie [16:21] lifeless, yes, are those instance not using the E* prefixes anymore? [16:21] matsubara: prefixes haven't changed [16:21] lifeless: so they are running the same code, but with a different config right? [16:22] flacoste: -very- slightly different config; only practical change I know of is the recipe stuff being enabled. [16:22] ok [16:22] flacoste: oh, and the timeout is 13000 not 15000 - but again, the oops /types/ and /causes/ should be identical. [16:23] right [16:23] should we harmonize the timeouts? [16:23] flacoste: on my todo :) [16:23] flacoste: waiting for the 600 a day to reduce [16:23] right, because of the 8.4 regressions [16:23] flacoste: which will happen as more things are qad and deployed - we have several timeout fixes blocked on qa [16:23] flacoste: right [16:23] awesome [16:24] this is really better [16:24] StevenK: https://devpad.canonical.com/~lpqateam/qa_reports/deployment-stable.html - --please-- I know you're at UDS, but unqa'd code in trunk is a teamwide stall. [16:24] lifeless: I'm trying! [16:24] StevenK: what can I do to help? [16:25] lifeless: Tell me why the function appears in the WADL but not in the actual API code [16:26] flacoste: so the call is in 23.5 hours? [16:26] yep [16:26] StevenK: how do you mean? leonardr: we may need your help. [16:26] flacoste: thanks, wicked. [16:27] StevenK, give me a link? [16:27] matsubara: so, for clarity - you're folding the E prefix into the lpnet reports ? [16:27] lifeless: I'm in a session that I'm talking in, so I don't want to walk out [16:27] what do you mean by "the api code"? you can't access it from launchpadlib? [16:27] lifeless, yep [16:27] leonardr: Right. I'm looking at say dir(maverick) [16:27] matsubara: awesome, thank you very much. We can drop the edge-oops.html on lpqateam etc too, with this change. [16:29] lifeless, I seem to remember you saying that the weekly summary is not useful, so I'll get rid of that as well [16:30] Ursinha, what do you think ^? [16:30] matsubara: yes please [16:31] reading [16:31] matsubara, that's cool, want me to do that along with qastaging changes? [16:32] Ursinha, sure, getting you a diff, just a sec [16:32] StevenK: i assume you're busy in the presentation, but when you get a minute, i need a link to the problematic branch (or maybe lifeless can give it) [16:34] leonardr: rev 9303 or so in db-devel, I think [16:34] http://bazaar.launchpad.net/~launchpad-pqm/launchpad/db-devel/revision/9903 [16:35] Ursinha, merge from here: lp:~matsubara/oops-tools/edge-no-more [16:36] thanks matsubara, will do that right now [16:36] Ursinha, will go for lunch and post office errands. please add a MP and I'll take a look after lunch and will deploy this so we'll get the new reports tomorrow. thanks a lot! === matsubara is now known as matsubara-lunch [16:37] sure, thanks matsubara-lunch! [16:40] mwhudson: https://bugs.edge.launchpad.net/launchpad-code/+bug/667322 [16:46] StevenK: ok, so you have some tests that use launchpadlib, and those tests pass, so it should be working. i'd like to eliminate the possibility that your client-side launchpadlib is using a cached wadl file [16:47] can you run your test script but set "httplib2.debuglevel = 1" before using Launchpad.login_with or equivalent? [16:47] leonardr: Okay, doing [16:48] lifeless: i think the port already comes from launchpad-lazr.conf [16:48] lifeless: config.codebrowse.port [16:49] leonardr: What do you need from that? [16:49] StevenK: paste me the whole thing, but i especially need the first part [16:49] * mwhudson comments on the bug [16:50] leonardr: http://paste.ubuntu.com/520833/ [16:50] mwhudson: thanks, I'll make new configs [16:52] StevenK: you are using a cached wadl file. go into ~/.launchpadlib/api.dogfoot.launchpad.net/cache/ and look at the file that starts api.dogfood.launchpad.net,beta,-application,vnd.sun.wadl+xml [16:53] see if derivedistroseries is in there [16:53] leonardr: I don't have that directory, is: launchpad = Launchpad.login_with('qa-mawson', 'dogfood') [16:54] leonardr: Is that right to to talk to dogfood? [16:54] yeah. i misspelled dogfood in my earlier msg as "dogfoot" [16:54] do you have a ~/.launchpadlib/api.dogfood.launchpad.net? [16:54] Nope [16:55] Which is why I'm questioning if my login call is right === jcsackett_ is now known as jcsackett|aft [16:55] you're making calls to the web service [16:56] do you have .launchpadlib at all? [16:56] mwhudson: whats 'secret_path' for ? [16:56] lifeless: signing cookies [16:56] is it sharable across processes ? [16:56] leonardr: Sorry, I have .launchpadlib/api.dogfood.launchpad.net/ but not .launchpadlib/cache/api.dogfood... [16:56] yeah, it's a path to a file that contains the secret [16:56] like, if we're running two codebrowsers [16:57] lifeless: different processes _need_ to share the same file content [16:57] do you have cache *inside* api.dogfood? [16:58] ie. when i do `ls /home/leonardr/.launchpadlib/api.dogfood.launchpad.net/cache/`, i see a file [16:58] lifeless: afk [16:58] leonardr: Right [16:58] leonardr: I do have that directory [16:59] ok, is there a wadl file inside it, and if so, does the wadl file mention derivedDistroSeries? [16:59] leonardr: Let me delete everything in that directory [16:59] StevenK: ok, that's another effective tactic [17:00] leonardr: Before I deleted it, no, it doesn't [17:00] Let me do the effective tactic [17:02] leonardr: Ah ha! Now it shows up [17:02] StevenK: ok, that is your problem. wadl files are cached for one week [17:02] this is fine for casual users, but when you do web service development, you need to make sure you clear them out [17:03] leonardr: Why the two directories? [17:03] lifeless: back again [17:03] StevenK: which two directories? [17:05] leonardr: .launchpadlib//cache versus .launchpadlib/cache/ [17:05] StevenL: you should not have .launchpadlib/cache/ [17:06] it may have been created by a very old version [17:06] you can delete it [17:06] Okay [17:06] mwhudson: is cachepath safe to share ? [17:06] lifeless: yes [17:06] it's sqlite & there's no real difference between sharing between threads and sharing between processes [17:08] Can I get a vic^Wvolunteer to run a test on revno 11801 of devel for me please? [17:08] mwhudson: so log_folder, error_dir and oops_prefix are the only unique things ? [17:09] leonardr: http://paste.ubuntu.com/520844/ [17:09] matsubara-lunch: when you return, can you add CB1 and CB2 oops prefixes? [17:09] lifeless: well, and port [17:09] lifeless: otherwise, yes [17:10] StevenK: would it be easy for you to try to reproduce that error locally? [17:10] just start up a local launchpad and run launchpadlib against 'dev' [17:11] leonardr: I can try, but I also have code access to dogfood directly [17:12] mwhudson: lp:~lifeless/lp-production-configs/codebrowse [17:13] StevenK: ok, the most likely problem is that your named operation takes an argument that is of a schema field type for which there is no unmarshaller [17:13] let me check your code [17:13] i don't know why the automated tests wouldn't have caught this [17:14] leonardr: if thats the case, will deploying it break other APIs ? [17:14] if not, then we can qa-ok this patch and move on with deployments; we can iterate on the feature [17:14] lifeless: no, it only means that this particular named operation is broken [17:15] leonardr: great, thanks. [17:15] bigjools: ok, so bad-commit explanation. [17:15] StevenK: yeah, i don't think you can have a named operation take a List [17:15] leonardr: Oh, drat [17:15] bigjools: bad-commit-XXXX will block deployments until either a) the tag is removed or b) a branch with [rollback=XXXX] is landed (even if its not a rollback, thats just the tag to use. [17:15] leonardr: Is there anything else I can do? [17:16] let me look at the code/think [17:16] bigjools or someone else who's been doing actual web service work might know [17:16] the qa-* tag doesn't override the bad-commit [17:16] once the rollback is present, then as long as the rollback rev will be deployed, it *unblocks* the bad-commit [17:16] and at *that point* the qa-tag gets evaluated. [17:17] StevenK, leonardr: nothing springs to mind but looking at IArchive might give inspiration. [17:18] lifeless: where is the link between the rollback rev and the bad rev so it knows to do that? [17:18] StevenK: the automated tests passed because you never passed in a value for any of the List arguments [17:18] and I'm not sure I am really following this [17:18] bigjools: StevenK: are you guys planning on going to https://blueprints.edge.launchpad.net/ubuntu/+spec/ubuntutheproject-foundations-n-releaseprocess ? [17:18] bigjools: in the commit message for the rollback rev [17:18] jml: yes [17:18] bigjools: we need to iterate on this process support stuff too, to make it more dynamic. [17:18] bigjools: cool. I really want to go but something just came up. glad you guys are. [17:19] bigjools: IArchive.getBuildSummariesForSourceIds takes a List [17:19] jml: when is it? [17:19] 5:10 [17:20] StevenK: there's your inspiration then [17:21] bigjools, StevenK, do we know that getBuildSummaries.. will actually work when passed a list? [17:21] lifeless: I am still confused, let's go through it after this session [17:22] lifeless: i don't think you need the launchpad.conf & mail-configure-normal.zcml crap [17:22] leonardr: yes. it's use in a UI page [17:22] lifeless: otherwise +1 [17:22] used [17:22] StevenK: in that case, i think your problem is that you need to specify a value_type for the List [17:23] otherwise, lazr.restful says "a list of what?" and can't validate the incoming json [17:23] sounds sane [17:24] hey jelmer, have you got time later today to talk about that task list? [17:25] bigjools: do you want me to run your patch through ec2 land? [17:26] jml: it would be nice to re-create it on someone's box first, but yes, that would help, thanks. [17:26] jml: how's this look? http://pastebin.ubuntu.com/520807/ [17:26] leonardr: And then regenerate the wadl and remove the cached wadl? [17:27] StevenK: i don't think you even need to do that, but it wouldn't hurt. you should be able to write an automated test that fails pretty easily, and then make it stop failing [17:27] bigjools: you need to add a cleanup to revert the patch when the test is done [17:27] bigjools: otherwise, perfect. [17:27] mwhudson: everything has it :( don't want to experiment offhand [17:27] leonardr: Right [17:27] jml: why? it doesn't hurt to leave it [17:27] bigjools: ok [17:28] given that is going to be the "fix" [17:28] bigjools: because you should only break test isolation when there's a very, very, very good reason [17:28] hmm [17:28] lifeless: well, i'm pretty sure you don't need it, but fair enough i guess [17:29] bigjools, jml: the other option is to install the monkey patch at import time i guess [17:29] deryck: I just trod on your toes wrt bug 667347. [17:29] doing it in test setup and then not doing it in teardown is ick [17:29] deryck: Can you look at my comment in there and let me know if I should revert my change? [17:31] allenap, ah, no worries. Thank thank thank you for triaging. ;) :) It's never stepping on my toes. :-) === matsubara-lunch is now known as matsubara [17:31] allenap, the comment is fine and good. I just wanted to have the bugs together about trac, since his bugs came out of a UDS session. [17:31] lifeless, the CB prefix is already known by oops-tools. [17:31] deryck: Cool. [17:31] matsubara: so it will handle CB1 and CB2 automagically ? [17:31] matsubara: there are two new error dirs as well [17:32] well, CB1 and CB2 can't be prefixes [17:32] you can't use digits in the prefix [17:32] oh grah [17:32]