[00:00] Right. [00:09] StevenK: its not part of lp-setup ? yes, I'd like to see it [00:35] lifeless: It is, it's shipped as an example. [00:36] lifeless: http://pastebin.ubuntu.com/1255107/ [00:37] ok [00:37] can you pastebin the lxc-start-ephemeral script you have ? [00:38] StevenK: and what version of testtools and subunit and testrepository do you have (outside the lxc containers) === matsubara-afk is now known as matsubara [00:38] lifeless: Versions http://pastebin.ubuntu.com/1255115/ [00:39] wgrant: heat cam eback, the very next year [00:40] lifeless: lxc-start-ephemeral is from precise, lxc 0.7.5-3ubuntu63 [00:44] ugh [00:44] its got an exit 0 in there. That will cause bad signal, if ssh exits non-zero, we want that to propogate. [00:44] Someone has been 'fixing' it I bet. [00:44] wgrant: ^ [00:45] that said [00:45] StevenK: Stopping lxc goes to [00:45] &2 [00:45] I wonder if the lxc from precise is not right and I need a different version? [00:45] StevenK: we need to figure out why its ending up in stdout [00:46] StevenK: I wonder, perhaps jenkins is insane [00:47] wallyworld: How did you build the 0.19.7 tarball? [00:48] python setup sdist [00:48] setup.py even [00:48] wallyworld: So did I, but I have 3 more files than you [00:48] in the dist directory? [00:48] No, in the tarball [00:48] which ones? [00:49] +/setup.cfg [00:49] +/src/lazr.restful.egg-info/not-zip-safe [00:49] +/src/lazr.restful.egg-info/PKG-INFO [00:49] lifeless: Heat came back? [00:49] i should have had the pkg-info in mine [00:49] not sure about the others [00:50] i don't know why there's a difference [00:50] wgrant: see the bug bryceh filed [00:50] lifeless: Ah yeah, that [00:50] lifeless: That's why I'm interested in locks :) [00:50] indeed [00:51] lifeless: However [00:51] lifeless: Note the time there. [00:52] lifeless: I don't think this particular case is mostly DB locks, but I don't know for sure which is why I wanted to get reports rolling :) [00:52] I suspect this is just Python [00:52] As the statement timeout should mean the query times out in well under 23 seconds... [00:53] wgrant: query times are not checked during lock waits [00:53] or weren't, its possible that 9.1 fixed that. [00:53] lifeless: Are so. [00:53] In 9.1 they definitely are [00:53] ok, so thats nice. [00:53] I'm pretty sure they were in 8.4 too [00:54] 24 seconds is slow for python, and there is no data to be returning there. [00:54] But I don't remember for sure [00:54] By "Python" I mean "someone else probably had a lock" [00:54] Because as you say, there is no data. [00:54] It may be interesting to check postgres logs for the killed transaction. [00:54] See when it actually happened. [00:54] We log errors, right? [00:55] pretty sure [00:55] StevenK: jml: jelmer: so the testrepository you have is old, the testing cabal builds are failing. [00:56] lifeless: Ah [00:56] thats not the cause of the issue AFAICT, but its a nuisance. [00:56] wallyworld: How did you add the 0.19.7 milestone when https://launchpad.net/lazr.restful/trunk times out? [00:56] StevenK: url hack [00:56] Huh, that times out? [00:56] * wgrant guesses workitems [00:56] No [00:57] Maintenance is not allowed to URL hack as the first step [00:57] Fix timeout instead :) [00:57] wgrant: OOPS-539a88c269cd05876823126d6dbb9e50 [00:57] wgrant: i'm not going to fix a timeout to delay adding a release [00:57] add release first [00:57] Yeah, it's the workitems regression :( [00:57] 23 150ms queries [00:58] wallyworld: What's the URL then? [00:58] +addmilestone [00:58] IIRC [00:58] hmm. +addmilestone [00:58] i think [00:58] so, working through this [00:59] lxc-start-ephemeral, assuming my copy == stevenk's copy, as he did not pastebinit :), outputs 'Stopping lxc' to &2, stderr [00:59] testrepository, which spawns the lp-setup-lxc-test processes uses stdout=subprocess.PIPE, stdin=subprocess.PIPE) [00:59] lifeless, StevenK: It's odd that we don't see this problem in the DC [00:59] Unless lp-setup-lxc-test does it [01:00] We use a puppeted version of that in the DC [01:00] so we'd expect stderr to be the jenkins provided stderr fd inherited from testrepository [01:00] echo "Stopping lxc" >&2 [01:01] however [01:01] we see all the ephemeral chatter in this file [01:01] Setting up ephemeral container... [01:01] Starting up the container... [01:01] ah, thats on stdout [01:02] That's from lp-setup-lxc-build I think [01:03] http://lpbuildbot.canonical.com/builders/lucid_lp_lxc/builds/70/steps/shell_9/logs/stdio [01:03] red == stderr [01:03] black == stdout [01:04] StevenK: its from lxc-start-ephemeral [01:05] StevenK: see 'setup_container' [01:05] Ah, indeed [01:06] wallyworld, lifeless: Can haz pypi rights for lazr.restful? [01:06] sure [01:06] so in http://lpbuildbot.canonical.com/builders/lucid_lp_lxc/builds/70/steps/shell_9/logs/stdio, Stopping lxc is on stderr [01:06] as I pasted above, testrepository doesn't capture stderr at all, it does the default for subprocess [01:07] StevenK: user name is SteveK? [01:07] StevenK even [01:07] ? [01:07] wallyworld: Yeah, stevenk [01:08] StevenK: should be done [01:09] Hmmm [01:09] Postgres really really seems to hate BitmapOr [01:09] Even when the alternative is two seqscans. [01:10] a quick re-check of subprocess doesn't given any obvious bugs [01:12] StevenK: can I login on the executor, will need root access [01:13] lifeless: ubuntu@184.73.75.214 [01:13] I've fed it your lifeless-64 ssh key [01:14] waaaay but its busay [01:14] or [01:14] StevenK: is port 22 open in the sec group ? [01:15] I'm not sure if it's open to 0/0 [01:15] ssh: connect to host 184.73.75.214 port 22: Connection timed out [01:15] Right [01:16] lifeless: Should I add just your IP or a netblock? [01:16] shrug [01:17] lifeless: You should be good [01:21] * StevenK drops lazr.restful-0.19.[2-5].tar.gz from the download-cache [01:22] lifeless: Ah, nice. You're right for the mo? [01:23] jenkins, you mad bro [01:23] testr 12536 jenkins 1w FIFO 0,8 0t0 65754 pipe [01:23] testr 12536 jenkins 2w FIFO 0,8 0t0 65754 pipe [01:24] Oh [01:25] testr 12536 jenkins 0r FIFO 0,8 0t0 65753 pipe [01:26] testr 12536 jenkins 1w FIFO 0,8 0t0 65754 pipe [01:26] testr 12536 jenkins 2w FIFO 0,8 0t0 65754 pipe [01:26] testr 12536 jenkins 3u REG 202,1 6975862 32201 /home/jenkins/launchpad/lp-branches/workspace/devel/.testrepository/tmp2CwhL_ [01:26] testr 12536 jenkins 5r FIFO 0,8 0t0 69881 pipe [01:26] thats the full set [01:26] testr's in, out, err, the temp file its stream its inputs too, 4 would have been the stdin for the child process, which is closes, and 5 is where its reading hte child output from [01:26] Hah, so it's Jenkins bug/misfeature [01:27] well [01:27] we have correlation [01:27] not causation [01:28] StevenK: so, we run sudo /usr/local/bin/lp-setup-lxc-test lptests /home/jenkins/.ssh/id_rsa $IDOPTION $LISTOPT ? [01:28] what does the DC run ? [01:29] sudo /usr/local/sbin/lp-setup-lxc-test I think [01:29] I think the only difference is the DC script has stuff hardcoded [01:29] fd's for lp setup look ok [01:29] lp-setup- 12540 root 0r FIFO 0,8 0t0 69880 pipe [01:29] lp-setup- 12540 root 1w FIFO 0,8 0t0 69881 pipe [01:29] lp-setup- 12540 root 2w FIFO 0,8 0t0 65754 pipe [01:30] err is as we expect, out is the pipe [01:30] lxc-start 12541 root 0r FIFO 0,8 0t0 69880 pipe [01:30] lxc-start 12541 root 1w FIFO 0,8 0t0 69881 pipe [01:30] lxc-start 12541 root 2w FIFO 0,8 0t0 65754 pipe [01:30] that looks fine too [01:31] and finally ssh [01:31] ssh 13831 root 0r FIFO 0,8 0t0 69880 pipe [01:31] ssh 13831 root 1w FIFO 0,8 0t0 69881 pipe [01:31] ssh 13831 root 2w FIFO 0,8 0t0 65754 pipe [01:31] so, theres nothing down through to the ssh layer that can explain stderr output ending up on stdout for testr [01:32] other stderr output *does* end up in the jenkins console [01:32] the added address to .ssh/known-hosts specifically [01:32] https://lpci.wedontsleep.org/job/devel/1102/console [01:32] Warning: Permanently added '10.0.3.39' (RSA) to the list of known hosts. [01:33] wallyworld, wgrant: https://code.launchpad.net/~stevenk/launchpad/preview-diff-none-type/+merge/126601 [01:35] StevenK: perhaps the assertVoteReference could be _assertVoteReference? [01:35] otherwise looks good [01:36] StevenK: kill the build? run again as concurrency 4 ? the stream written so far is intact. [01:36] Project devel build #1102: ABORTED in 2 hr 23 min: https://lpci.wedontsleep.org/job/devel/1102/ [01:37] lifeless: #1103 away [01:38] wallyworld: http://pastebin.ubuntu.com/1255162/ [01:39] +1 [01:44] Bleh, my oops got pruned again [01:45] Clearly, I need to log into neem and make copies [01:58] lifeless: See also OOPSes like https://oops.canonical.com/oops/?oopsid=OOPS-c182f7f94210ede212ae5b835ef993c7 [01:58] lifeless: 12s query that couldn't be blocked by any normal lock [01:58] We have a general problem, and I suspect that Bryce's bug is mostly that general problem. [02:15] wgrant: sounds plausible [02:24] this is mystifying [02:27] What is? [02:27] I was having trouble finding any evidence of a problem [02:27] xvfb-run 8494 jenkins 0r FIFO 0,8 0t0 11112539 pipe [02:27] xvfb-run 8494 jenkins 1w FIFO 0,8 0t0 11112540 pipe [02:27] xvfb-run 8494 jenkins 2w FIFO 0,8 0t0 11112540 pipe [02:27] Ah [02:27] but - this is such evidence [02:28] xvfb-run has stderr and stdout linked [02:28] so we can corrupt a stream within the subprocess [02:28] across the ssh barrier [02:28] lifeless: You think this is jenkins level or lower? [02:29] I'm not sure if Jenkins does something like exec 2>&1 that it will hit processes it spawns [02:30] well, I'm still only at correlation stage [02:30] jenkins definitely is doing 2>&1 [02:31] but that shouldn't have any way of affecting children of children if they make their own pipes [02:31] which testr does for stdout [02:33] ssh normally preserves stderr as a separate fd [02:33] e.g. [02:33] ssh localhost 'echo true >&2' 2>/dev/null [02:33] ssh localhost 'echo true >&2' [02:34] compare the output [02:34] it knows whats stdout and stderr on the slave, and faithfully lines that up on the invoking side [02:34] so WTF would xvfb-run be seeing stderr=stdout [02:35] ssh -n -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -i /home/jenkins/.ssh/id_rsa jenkins@10.0.3.132 -- env -u LANG xvfb-run --error-file=/var/tmp/xvfb-errors.log --server-args='-screen 0 1024x768x24' -a /home/jenkins/launchpad/lp-branches/workspace/devel/bin/test -vvv --shuffle --subunit --load-list /home/jenkins/launchpad/lp-branches/workspace/devel/temp/tmp0bW4iD [02:35] is the command being run [02:36] ssh -n localhost 'env echo true >&2' 2>/dev/null [02:36] still good [02:39] I'll file a bug on the silly exit 0 / exit 1 stuff [02:43] https://bugs.launchpad.net/ubuntu/+source/lxc/+bug/1059943 [02:43] <_mup_> Bug #1059943: lxc-start-ephemeral masks process exit code < https://launchpad.net/bugs/1059943 > [02:44] lifeless: I already filed and fixed that in quantal [02:45] wgrant: oh, can you dupe it appropriately please then ? [02:45] And I have a precise backport branch. [02:45] DISPLAY=:$SERVERNUM XAUTHORITY=$AUTHFILE "$@" 2>&1 [02:45] ^ xvfb-run [02:49] StevenK: so, smoking gun. [02:50] StevenK: this may not explain all the corruption we're encountering, but I think it covers a pretty good chunk thereof [02:50] StevenK: are you running lucid or precise in the lxc containers? [02:50] StevenK: https://bugs.launchpad.net/ubuntu/+source/xorg-server/+bug/1059947 [02:50] <_mup_> Bug #1059947: xvfb-run munges stdout and stderr together < https://launchpad.net/bugs/1059947 > [02:52] StevenK: it doesn't explain HTF 'Stopping lxc' got from lxc-start-ephemeral &2 into &1 [03:03] blink [03:04] Ah [03:05] wgrant: ? [03:05] MilestoneTag.specifications had me confused for a while [03:05] Why it was intersecting all the resulting specs [03:06] But it makes some sense now [03:09] Project devel build #1103: STILL FAILING in 1 hr 32 min: https://lpci.wedontsleep.org/job/devel/1103/ [03:13] lifeless: Whatever lp-setup does, which I guess is lucid? [03:14] Right, inside the containers is lucid [03:14] xvfb-run bug? :-( [03:15] StevenK: contributing factor at most [03:15] I've just edited the bug out of the root container and started another run [03:16] So buildbot doesn't xvfb-run, then? [03:17] timing bugs are terrible [03:17] buildbot does [03:17] But isn't the problematic stderr coming from *outside* xvfb-run? [03:18] wgrant: one case [03:18] which as I said [03:18] 15:52 < lifeless> StevenK: it doesn't explain HTF 'Stopping lxc' got from lxc-start-ephemeral &2 into &1 [03:18] Ah, right [03:19] but [03:19] it may be confused signals [03:19] see [03:19] https://lpci.wedontsleep.org/job/devel/1103/console [03:19] where we have test data in the jenkins console [03:19] that happens when tests are mangled at a lower layer [03:19] and subunit's parser decides its seeing junk rather than tests [03:20] e.g. the subprocess race/rhing [03:20] which we need to get a reproducable test case for [03:20] _StringException: lost connection during test 'lp/registry/javascript/tests/test_milestone_creation' [03:20] the end of https://lpci.wedontsleep.org/job/devel/1103/console [03:20] shows [03:20] test: lp.codehosting.pukill: 186: No such process [03:20] Stopping lxc [03:20] ller.tests.test_scheduler.TestPullerWireProtocol.test_methodDispatchWithArguments [03:20] which is, I think, testr's stdout intermingled with the shared stderr, and - *this is fine*. [03:21] Its expected given jenkins mangling of the two onto one stream [03:22] StevenK: you probably want two forked packages; lxc (for the lxc-start-ephemeral fix from wgrant) and a xvfb-run with fixed stderr handling [03:23] https://code.launchpad.net/~wgrant/ubuntu/precise/lxc/bug-1050351 [03:36] StevenK: so we'll see if thats any better, and if it isn't we'll see about making a reproducable test case. [03:37] for now, I'm going to context switch a bit [03:48] StevenK: Thoughts on bug #810716 and bug #690834? [03:48] <_mup_> Bug #810716: Translations section builds have highest priority in copy builds < https://launchpad.net/bugs/810716 > [03:48] <_mup_> Bug #690834: Language packages in P3As are also scored 0 < https://launchpad.net/bugs/690834 > [03:48] For context, if section == 'translations' then the score short-circuits to 0 [03:48] Ignoring any archive offset [03:48] Ywah [03:48] I suspect we just want to apply the archive offset to the 0 [03:48] I'll be hitting that code to [03:48] So they'll sit at the bottom of the archive [03:49] Oh? [03:49] Oh [03:49] I see you took the card [03:49] I was just collecting those three bugs to fix [03:49] Oh, you can do it if you want [03:49] If you've been busy with jenkins and haven't actually started... [03:50] I've created a branch and found the code, so 'not really' [03:50] Ah, well I just wrote a test, so I'm probably slightly ahead. [03:50] Nothing of consequence, so feel free to steal the card and I'll find another bug [03:50] Or re-package lxc [03:50] Thanks [03:50] Heh [03:53] Haha [03:53] I can see the edit page in bug 1059324, but the reporter can't [03:54] <_mup_> Bug #1059324: Can't edit branch import <403> < https://launchpad.net/bugs/1059324 > [03:54] StevenK: ~vcs-imports can see +edit-import, mortals can't. [03:54] It's likely that the link should just be hidden. [03:55] It may just be a matter of adding one line of decorator. [03:55] Right [03:55] @with_permission or so? [03:55] @enabled_with_permission, IIRC [03:59] Bleh, the qa-tagger hates multi-task bugs? :-( [04:00] Should work, normally. [04:00] What's the issue? [04:01] It stated orphaned for my rev [04:01] Maybe it insists on default_bugtask, but that would be silly [04:01] Hm [04:01] That's odd [04:03] wgrant: Hmm, lib/lp/code/browser/branch.py edit_import checks perms already. Is it checking the wrong thing? [04:04] StevenK: That's a very good point [04:05] Although I wouldn't trust that function to not be insane [04:05] Given that it sets enabled = True a line earlier [04:06] Haha, I wonder if that's the fix [04:06] It's just a normal local [04:06] So I doubt it [04:09] 6 Unauthorized: (, 'updateFromData', 'launchpad.Edit') [04:10] That seems more like he can view the page and the submit is blowing up [04:10] Right, that means the user doesn't hold launchpad.Edit on the CodeImport [04:10] What's the view permission? [04:12] launchpad.Edit on IBranch, looks like [04:13] Ah [04:13] That doesn't explain why there's a link, though [04:13] Perhaps there is no link [04:13] And the user URL-hacked. [04:13] Try to reproduce locally or on qastaging? [04:13] How did he POST, then? [04:13] The view is Branch:+edit-import, so the permission is wrong [04:14] It'll load for the branch owner [04:14] We have a few OOPSes, so we should be able to work it out [04:14] But the edit_import menu item correctly checks against branch.codeimport [04:14] OOPS-1d2dccea10b7b7c5f4bc11cf0eaa9ebf [04:14] The OOPS won't say [04:14] It'll show the referer as +edit-import either way [04:15] Right, so the ZCML is wrong [04:15] And can't really be right. [04:15] Not if we use for="...", right [04:19] wgrant: I'm not clear how to create a branch import [04:21] +new-import [04:21] http://code.qastaging.launchpad.net/launchpad/+new-import [04:25] wgrant: Right, so no link. URL hacking as the branch owner shows the form and submitting gives Not allowed here [04:26] Maybe we want to redirect away in initialize() [04:26] "You are not permitted to edit the import for this branch" etc [04:27] Or raise a Forbidden or so [04:27] No point being pretty if you have to URL-hack to get there. [04:27] Indeed [04:28] It already does for anonymous, indeed [04:28] If you waste more than two lines of code on this, you're probably doing it wrong. [04:30] StevenK: I think I'll take this opportunity to finally eliminate time-based scoring. [04:30] Any objections? [04:31] * wgrant murders. [04:32] wgrant: We have time based scoring? [04:33] If it will complete quickly, score it higher? [04:33] No [04:33] If it's been in the queue for 5 minutes, give it a score bump of 5 [04:33] an hour is a bump of 20 [04:33] Oh, right [04:33] This ignores the fact that the tiebreaker is id, which is age anyway [04:33] And we don't run queue-builder any more [04:33] wgrant: Yes, murder it. [04:33] So the score is never calculated except when the build is 0 seconds old. [04:33] Thanks [04:34] Can we destroy queue-builder too? [04:34] I already got rid of most of it a while ago [04:34] wgrant: I'm at -4 with this branch with some cleanups and no test [04:34] :) [04:34] We have a lot of crap in our code. [04:35] Not sure if raise Forbidden is okay [04:35] Unauthorized may be more common, I think [04:35] But I forget [04:38] StevenK: ok, so the xvfb-run isn't the root cause, and whether it contributed is an open q [04:39] StevenK: tomorrow I shall poke further, the artifacts gathered should be sufficient [04:39] Ah [04:39] https://lpci.wedontsleep.org/job/devel/1104/console [04:39] shows subunit [04:39] so its escaped the parser again [04:40] lifeless: So I can kill the executor ec2 when the current one fails? [04:41] wgrant: Do you think a test is worth it? [04:42] StevenK: If you've deleted enough to still leave you net-negative :) [04:44] Project devel build #1104: STILL FAILING in 1 hr 28 min: https://lpci.wedontsleep.org/job/devel/1104/ [04:53] StevenK, wallyworld_: https://code.launchpad.net/~wgrant/launchpad/bug-1058310/+merge/127414 https://code.launchpad.net/~wgrant/launchpad/bug-1056617/+merge/127412 [04:56] StevenK: wgrant kicking ass on bugs I see :) very nice [04:59] wgrant: r=me on 2nd one [05:00] Today should take us below 300 criticals :) [05:00] wallyworld_: Thanks [05:00] wgrant: And r=me on the first [05:00] Did we implement ParallelReviewing while I wasn't watching? [05:00] wallyworld_: ah and you [05:01] tab complete and me are not working at this hour [05:01] czajkowski: well, today i'm removing an old feature flag for dynamic bug listings which was released a while back. not sure if that's critical, but it needs to be done to keep the code clean and maintainable [05:02] tedious work [05:02] which is needed [05:02] That's probably our biggest single bit of tech debt :) === matsubara is now known as matsubara-afk [05:02] i'm almost finished. a few more doc tests to go [05:03] maybe a bit optimistic to say "almost". "substantially" is better [05:03] * wgrant plays buildbot roulette [05:03] ah hat reminds me I need to find the bug on signing the CoC to make it easier to see if we can get some sort of check box implemented :D [05:03] Heh [05:10] StevenK: You have a couple of ancient branches on https://code.launchpad.net/launchpad/+activereviews [05:11] No I don't. :-) [05:11] Thanks. [05:11] 5 files changed, 24 insertions(+), 32 deletions(-) [05:11] I think that's far enough [05:25] StevenK: wgrant: have you noticed that the branch badges on the dynamic bug listing are no longer anchors linking to the associated branch? [05:25] wallyworld_, wgrant: https://code.launchpad.net/~stevenk/launchpad/no-edit-import-for-you/+merge/127417 [05:34] StevenK: r=me [06:19] wgrant: I wonder if bug 1022334 can be closed due to YUI 3.5.1 [06:20] StevenK: No [06:20] StevenK: The bug *listing* back behaviour may be fixed, but the bug reporting one probably isn't [06:21] I thought zope handled at this stuff, TBH [06:21] Our JavaScript is not Zope [06:21] Oh, it's a JS bug? [06:22] Well, +filebug is very JSey, so it's probably not unrelated. [06:22] Yeah [06:22] Or it could be that the view is too custom and doesn't let initial_values through properly. [06:29] wgrant: In https://oops.canonical.com/oops/?oopsid=OOPS-91e85c5a76936c13c1993df1985491e5 I guess the branch is None? [06:30] StevenK: Potentially because the requesting user can't see it [06:30] But it looks like it. [06:31] Yeah [06:32] StevenK: Yeah, it happens whenever the development focus is an import that isn't visible. [06:32] Heh, right [06:32] wgrant: So we should skip that bit? [06:32] That may be appropriate. [06:33] wgrant: It seems I broke buildbot. But it doesn't look to be my fault. [06:33] It just has a personal vendetta against you [06:33] Ah, yeah [06:33] We see that one sometimes :( [06:35] wgrant: QA! :-P [06:35] Did that like 90 seconds ago [06:39] lib/lp/code/templates/product-branch-summary.pt makes me sad [06:57] wgrant: I should force away? [06:57] StevenK: Yep [07:02] good morning [07:02] StevenK: tsk tsk [07:03] wgrant: Hm? [07:04] StevenK: You're not very good at deleting views :) [07:04] I was supposed to delete one? [07:04] You deleted +daily-builds [07:05] wgrant: Sure, but? [07:05] You missed 300 lines of code that was used only by it :) [07:05] Have you binned them, or shall I have at it? [07:05] They're miiiiiiine [07:06] * StevenK makes a note to talk to wgrant's mother about sharing [07:06] We already finished sharing last month :) [07:06] s/sharing/closing legs/ [07:06] No need for any more. [07:06] Ah, make that 400 lines. [07:07] wgrant: Meh, I'm -3500 and you're -1235 :-P [07:09] I won't be dictated by fact-checkers. [07:09] Also, http://webnumbr.com/launchpad-critical-bugs [07:10] We're 3 bugs away from where we were at the start of December. [07:10] Getting there. [07:35] StevenK: Your new test seems to be bad [07:36] If you're in AppServerLayer, you're probably trying to catch the exception over HTTP [07:36] Which isn't likely to be a gigantic success. === mmrazik is now known as mmrazik|otp [07:40] Hmmm, I thought it was passing locally [07:41] Let me roll it back [07:48] stub: Huh [07:48] "specification_product_name_uniq" UNIQUE CONSTRAINT, btree (name, product) [07:48] Why is it that way around? :/ [07:48] lookup by name? [07:49] Possibly, but it seems pretty odd [07:49] I prefer the theory that someone tried to make it as useless as possible. [07:49] it is the only suitable lookup by name, which would be for traversal [07:49] Right, but the URL is /product/+spec/foo [07:49] I don't know much that looks up by unqualified name. [07:50] yeah, so it can use that index. [07:50] Right, but the distribution index is around the right way :) [07:50] The product one is just inverted for probably no reason [07:50] it would have been inverted to look up by unqualified name. I see that is probably pointless, yes. [07:51] I'm not sure productseries and distroseries indices are useful, but I guess we might as well [07:51] The table isn't exactly high-write... [07:51] * wgrant adds three [07:52] I didn't even consider that product might have been unindexed [07:52] Since that's basically the primary thing we've queried on for 7 years... [07:52] w [08:03] stub: I've added new indices (including UNIQUE (product, name) so we can kill off the other one), though the diff is taking its time to update [08:03] It's possible we want one on just name, but I don't think so [08:05] yup [08:05] unqualified doesn't make sense [08:05] It certainly doesn't make sense. [08:05] But it still might happen [08:06] Because a lot of blueprints is a little behind its sense quota. [08:06] But let's go without === frankban changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: frankban | Firefighting: - | Critical bugs: ~300 [08:10] stub: If that looks good I'll land it [08:11] frankban: Hi. I'm trying to clean up the review queue, and you have three branches on https://code.launchpad.net/launchpad/+activereviews. I think they're all for stuff that's now in lp:lpsetup, so they can be rejected from LP, right? [08:12] wgrant: yeah, that looks good. It might make more sense for these to be partial indexes, but I don't think we know enough yet to be sure. [08:12] wgrant: yes [08:13] stub: Given the data volumes and rates we're looking at, it won't matter for another few years :) [08:17] frankban: Thanks [08:18] frankban: Can you set https://code.launchpad.net/~frankban/launchpad/setuplxc-remove-sleep/+merge/96422 from Approved to Rejected to get it off the list? It's proposed to a branch that I don't have review privs on. [08:21] wgrant: thank you! and done [08:22] Great, thanks. === mmrazik|otp is now known as mmrazik [08:45] stub: should francis be the 2nd db reviewer going forward, or someone else ? [08:54] morning folks [08:57] lifeless: Depends if Francis is comfortable giving the go ahead for patches. One feature of the reviews is to avoid hard to recover failures and dataloss. [08:59] stub: Indeed. [09:09] gnight [09:09] Night lifeless. === mmrazik is now known as mmrazik|lunch === mmrazik|lunch is now known as mmrazik === matsubara-afk is now known as matsubara [12:36] frankban: could you please review this MP: https://code.launchpad.net/~adeuring/launchpad/product-sharing-sec-adapter/+merge/127473 ? [12:38] adeuring: sure [12:38] thanks! [12:38] adeuring: A couple of notes: [12:38] Product.information_type exists on production now [12:39] Though not on qastaging; you might be able to convince ops or stub to fix that [12:39] And also, LimitedView is probably not what you want, unless you've discussed that with Curtis/Robert already [12:39] wgrant: makes sense [12:39] View is what makes sense here [12:39] wgrant: i discussed this with sinzui for Ispecification [12:40] there were some pitfalls with lp.View [12:40] adeuring: LimitedView is intended to give people enough to know that something exists, but basically nothing else. [12:40] correct [12:40] Oh, sinzui's here [12:40] sinzui: would you be free later on for a call re maintenance, have stand up now. [12:41] LimitedView reveals the spec name, displayname, and project, .public, and .information_type. They do not have icons that need to be revealed [12:41] yes [12:41] yes czajkowski [12:42] It's not clear that LimitedView is necessary for products at this point [12:43] But even if it is, it would probably expose name, displayname, owner, and that's about it [12:43] sinzui: thanks gew Q's need to be handed over and want to go through them first [12:43] wgrant, subscribing someone to a bug, blueprint, or branch will require limited view to the product and possibly series [12:44] sinzui: I think those are likely to give full View to the product. [12:44] But that may be yet to be established. [12:44] wgrant, adeuring, without limited view, you cannot traverse to get to the artefact. [12:44] Possession of View implies possession of LimitedView by default [12:45] wgrant, We are not here to establish that, Deryck's squad are. We know that traversal requires access to .name to get the a subscribed private arttfact [12:45] Sure [12:46] But that doesn't mean that 90% of the attributes that were zope.Public should be launchpad.LimitedView now [12:46] 89% should be launchpad.View, 1% should be launchpad.LimitedView [12:46] Otherwise LimitedView is meaningless. [12:46] wgrant, I agree. I think 7 attributes are about the right number [12:46] Right :) [12:48] limitedview is intended to convey enough information to allow someone to identify the thing they are looking at. It deters spying and deceit when a private something is in public relationship [12:51] adeuring, OEM projects are only willing to reveal their name, displayname, icon, logo, .public, .information_type, and maybe .active to contractors that they subscribe to bugs and branches [12:52] sinzui: ok; just working replacining lp.LimitedView with lp.View for the current branch [12:52] okay. [12:57] adeuring: should I wait for you to push these changes? [12:57] frankban: make sense, I think [12:57] adeuring: ok [13:01] frankban: i renamed the permission to lp.view now. I'll add lp.limitedview in a later branch for the properties that need it. Testing both permissions would let the diff size explode ;) [13:01] adeuring: ok [13:32] abentley, adeuring -- always have troubles getting into the hangout the first time, still trying.... [14:30] deryck: I'm confused about where we stand wrt NULLs in Product.information_type. [14:31] abentley, we need a garbo job to fill in values for existing products. nulls are currently allowed. [14:42] deryck: I think we don't have a card for the garbo job. And we also need to fix the sampledata. [14:43] abentley, I lumped the garbo job into your current card back when I created it. And I can add a card now for sampledata. [14:43] abentley, and I don't mind if you split off that garbo stuff into a new card, too. [14:44] deryck: I read my card as stuff to do "after information_type + garbo job for current values" [14:45] abentley, yeah, I probably phrased it badly. :) [14:45] deryck: I see what you meant now. [14:58] deryck: Do you know whether sampledata updates can land in stable? ISTM there would be no value to landing them in db-stable. [14:58] abentley, I believe they can indeed land in stable. [15:23] Nothing except db patches lands on db-devel/db-stable now [15:23] Occasionally some tests to go with db patch, but that is really rare. [15:25] stub: Thanks. [15:26] stub: But if a db patch requires sampledata changes, we should still land both as one branch, right? [15:27] yeah [15:28] The bare minimum required to keep the tests passing and the build working, or if your db patch contains stored procedures tests to prove the code works. [15:30] abentley, what was the feature flag you used for the new project UI? [15:31] deryck: 'disclosure.private_projects.enabled' [15:31] abentley, thanks [15:53] what is initial LOC credit for somebody who never committed to lp? [15:54] -100, 0, 100 ? [15:54] 0 [15:55] frankban: if you are still around, could you do another review? https://code.launchpad.net/~adeuring/launchpad/correct-permission-check-for-iproduct/+merge/127518 [15:56] adeuring: on a call now, but I will take a look in 30 or so... [15:57] frankban: thanks! (this MP is a bit smaller, BTW) [16:01] abentley, adeuring, just to confirm, we do still need PROJECTGROUP/+newproject updated to match projects/+new [16:01] so I'll leave the card on the board, obviously. [16:01] deryck: roger. [16:01] ack === frankban changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: - | Firefighting: - | Critical bugs: ~300 [16:44] abentley, adeuring -- qastaging is updated for Product.information_type now. [16:47] deryck: cool [16:50] sinzui: ok, thanks. [16:50] xnox, do you need help finding cruft to remove? [16:51] sinzui: well there is a link to bug reports that are trianged with the cruft-removal-bug tag ;-) so I'm good. [16:51] okay. Ping me if you need help. [16:51] trying to pick something easy, as I haven't done lp development yet, but there are a couple bugs that hinder my productivity ;-) =))))) [16:52] but are "low" priority for now. [18:07] deryck: There's no OCR. Could you please review https://code.launchpad.net/~abentley/launchpad/model-product-info-type/+merge/127558 ? [18:32] abentley, I will shortly. Just grabbing me a bite to eat. [18:33] deryck: Actually, I missed a spot. Fixing now. [19:15] deryck: ready for review again: https://code.launchpad.net/~abentley/launchpad/model-product-info-type/+merge/127558 [19:21] abentley, ack [19:31] deryck: Garbo job was really easy with your suggestion: https://code.launchpad.net/~abentley/launchpad/product-info-type-garbo/+merge/127579 [19:33] oh nice [19:36] abentley, r=me for both. [19:37] deryck: thanks. [19:38] np [19:43] deryck: How do we decide whether EMBARGOED/PROPRIETARY are reasonable information_types for a given Product? Do we look for a commercial subscription or something? [19:43] abentley, yes, you have to have a commercial subscription to use them. [19:47] deryck: [19:47] deryck: I'm going to restrict it in the model for now, and we can add db constraints afterwards. [19:47] abentley, sounds good. [19:52] * deryck goes AFK for a little bit [22:30] lifeless: Moar Jenkins? [22:46] wgrant: StevenK: sinzui: you seen bug 1055766? [22:46] <_mup_> Bug #1055766: grep -R doesn't automatically search amazon < https://launchpad.net/bugs/1055766 > [23:29] wallyworld_, wgrant: https://code.launchpad.net/~stevenk/launchpad/no-edit-import-for-you-redux/+merge/127613 === Ursinha is now known as Jorjao [23:31] StevenK: the test comment should be something like "Unauthorised users are forbidden....." since it's not just the branch owner who is not allowed === Jorjao is now known as Ursinha [23:32] wallyworld_: http://pastebin.ubuntu.com/1257063/ [23:32] StevenK: Given lifeless is probably terribly busy, might be better to throw broken Jenkinses at me sometimes [23:32] hi [23:32] I'm going to be poking it [23:32] r=me [23:32] StevenK: can you setup an executor ? [23:33] lifeless: I thought you only needed the build artifacts, but sure. [23:34] StevenK: reproducability is key, until I have a core recipe its an open question what will be needed [23:39] running just http://paste.ubuntu.com/1257072/ should give us a fail single-threaded. [23:39] StevenK: so what I want is an executor, and jenkins told not to run anything [23:40] StevenK: run one build, to seed it and recor da failure [23:40] and then I'll poke around as jenkins and reproduce etc [23:40] StevenK: so I want ssh, root access etc [23:40] lifeless: Said executor is up and in the middle of lp-setup [23:41] lifeless: Right, so I should add it to Jenkins and kick off a devel build too? [23:41] StevenK, I pulled the latests critical data. The chart is updated [23:42] StevenK: please, and we can configure jenkins to only do manual builds [23:42] lifeless: I already have [23:42] coolio [23:43] * StevenK waits for Firefox to choke down sinzui's JS [23:44] StevenK: will you prep a fixed xvfb for the lp PPA [23:44] Not the js, it is 5 megs of bug data [23:44] I can, but as you say, it's no smoking gun [23:44] StevenK: it a necessary fix for reliability [23:45] For lucid, I guess [23:45] StevenK: any stderr IO from anything in the test suite or under it can corrupt a stream w/out that fixed. [23:45] StevenK: for precise too [23:45] I'm not sure if it's inside the container or outside that is the important one [23:45] inside [23:45] xvfb isn't present outside the container [23:49] * StevenK stares at the xorg source package [23:49] xorg-server ? [23:50] Bleh [23:55] lifeless: http://pastebin.ubuntu.com/1257092/ [23:56] * StevenK stabs Contact this team's admins [23:57] StevenK: looks good to me [23:57] lifeless: If you have no complaints about the version, I'll upload it [23:58] StevenK: seems fine to me