[00:01] wgrant: https://code.launchpad.net/~lifeless/python-oops-datedir-repo/bug-891251/+merge/82469 [00:03] lifeless: Why is the umask 077 in the first place? [00:04] I don't know the backstory [00:04] we have multiple services like this though, have for ages [00:17] StevenK: FYI, in my hide bug comments mp, i've added 2 methods to Bug which i believe may be similar to what you need - userCanSetCommentVisibility and userInProjectRole [00:18] wallyworld_: I fear we may step on each others toes, then. [00:18] wallyworld_: Let me finish what I'm in the middle of, and I'll show you what I have. [00:18] ok [00:19] we should try and consolidate [00:19] Exactly [00:34] wallyworld_: http://pastebin.ubuntu.com/740741/ [00:37] StevenK: so it looks like code under # Otherwise, if you're a member of the pillar owner, drivers..... is similar to my userInProjectRole(), but my method iterates over all affected pillars and also checks for security contact [00:37] * StevenK grumbles at Unity [00:38] If I run indicator-weather, I want you to stay in the notification area! [00:38] StevenK: i've added new methods to IPersonRole: isBugSupervisor(obj) and isSecurityContact(obj). you could use these, plus the existing isDriber() API instead of your code [00:38] isDriber()? Sounds interesting. [00:38] * StevenK ducks. [00:38] you know i can't type :-P [00:39] Why all affected pillars? [00:39] because it's to control who can hide a bug comment [00:40] Ah [00:40] win [00:40] Right [00:40] win win win win win [00:40] ie the user is validated against the bug, not an individual bugtask [00:40] 2011-11-17 05:45:37+0530 [-] self._oops_datedir_repo = DateDirRepo(config[section_name].error_dir) [00:40] 2011-11-17 05:45:37+0530 [-] File "/var/launchpad/tmp/eggs/oops_datedir_repo-0.0.12-py2.6.egg/oops_datedir_repo/repository.py", line 93, in __init__ [00:40] wallyworld_: Right. [00:40] 2011-11-17 05:45:37+0530 [-] self.metadatadir = os.path.join(self.root, 'metadata') [00:40] 2011-11-17 05:45:37+0530 [-] File "/usr/lib/python2.6/posixpath.py", line 67, in join [00:40] * StevenK kicks lifeless [00:40] 2011-11-17 05:45:37+0530 [-] elif path == '' or path.endswith('/'): [00:40] 2011-11-17 05:45:37+0530 [-] AttributeError: 'NoneType' object has no attribute 'endswith' [00:40] we have an error_dir setting of None somewhere. [00:40] StevenK: its 6 lines, deal [00:41] lifeless: Except with your internets, it takes 20 seconds per line [00:41] TacException: Unable to start /var/launchpad/test/daemons/poppy-sftp.tac. Content of /tmp/tmp_Nitmg/poppy-sftp.log: [00:41] this might explain a lot, actually. [00:41] StevenK: not sure if we want to introduce a isUserInProjectRole(pillar). your stuff doesn't require isSecurityContact(). maybe it's sufficient for you to just use the IPersonRoles stuff [00:44] StevenK: https://pastebin.canonical.com/55928/ [00:44] plus there's the other existing IPersonRoles helpers [00:44] lifeless: We have lots of missing errordirs. [00:44] lifeless: You've not seen that before? === gmb` is now known as gmb [00:45] wgrant: it wasn't fatal before [00:46] wgrant: (for some bizarre reason) [00:46] That traceback was always there. [00:47] wgrant: poppy is failing to start in ec2 [00:47] wgrant: this is the error [00:47] Hmmm [00:47] I see [00:47] wgrant: I dispute 'always there' [00:47] wgrant: what may well look similar is raising an oops barfing if errordir is None [00:47] wgrant: this evaluates it for sanity a little earlier, because it was simpler. [00:47] Right.; [00:48] as such I think its a net win, but it puts back my landing my oops polish branch [00:50] poolie: which reminds me, any word on that email oops raising critical? [01:14] StevenK, do you have time to mumble about goals [01:15] sinzui: Certainly. [01:15] * StevenK fights with Unity on his laptop [01:15] how does this even work on prod [01:16] we've no error dir or oops prefix set for poppy. [01:18] poppy isn't meant to OOPS, is it? [01:18] hah [01:18] I didn't know it had anything installed to raise OOPSes. [01:21] SSHService [01:21] takes a parameter for the oops config section to use, and installs twisted top level oops glue [01:21] so yeah, it does [01:21] but it has never ever worked [01:21] Hah [01:22] Well, poppy-sftp is not yet 18 months old. === lifeless changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: Me! | Critical bugtasks: 277 [01:29] wgrant: can you please do an incremental review on https://code.launchpad.net/~lifeless/launchpad/oops-polish/+merge/82359 for me ? [01:31] wgrant: if you do, I can lp-land this bad boy (all the tests that failed are fixed by these commits) [01:34] * StevenK kicks subunit for being stupid === lifeless is now known as subunit [01:35] * subunit kicks StevenK for using me badly === subunit is now known as lifeless [01:35] StevenK: seriously, whats the issue? [01:37] lifeless: I have a stream, I want a list of failing tests only. [02:00] huh [02:00] MP diff in less than 30 seconds [02:04] Something must be wrong, then. :-P [02:06] lifeless: OCR: Me! is about as useful as [rs=me] in a commit log. [02:09] StevenK: if you can't figure it out you don't need a review [02:09] lifeless: I have a review for you :P [02:09] ok, ok, enough with the hassling [02:10] StevenK: | subunit-filter --no-skips | subunit-ls [02:10] StevenK: I seem to be giving you this recipe weekly, or something [02:13] * wallyworld_ goes to have lunch with bigjools who is melting in the heat [02:13] wallyworld_: he's in.au? say hi [02:14] wgrant: so, incremental on https://code.launchpad.net/~lifeless/launchpad/oops-polish/+merge/82359 ? [02:14] lifeless: here's in brisbane for 2 weeks to check out schools and houses [02:15] wallyworld_: Tell him to move somewhere a little further south [02:15] somewhere nice :P [02:15] lifeless, yeah i might try changing the counters [02:15] ec2 on tmpfses is doing approximately zero io [02:15] that's nice [02:15] lifeless: Sure [02:16] i haven't measured the 'before' though [02:16] huwshimi: it still gets hot down there too. adelaide for example is often hotter than brisbane [02:16] Yes, but it's *Adelaide*. [02:17] could be worse - could be sydney or melbourne [02:17] * wallyworld_ runs away [02:18] lifeless: Why are you UTF-8ing the URL in errorlog.py? [02:20] wgrant: to eliminate the double escaping [02:20] Mmm, OK [02:20] Anyway, approved. [02:21] wgrant: the u1/isd wsgi stack gets de-%coded, de-utf8 coded urls [02:21] this whole visiting countries before moving to them is very cautious! [02:21] wgrant: so oops tools treats a unicode url as probably garbage and immediately utf8 and % encodes it [02:21] wgrant: thats why our oopses look bong url wise atm [02:22] wgrant: I figure the only semi sane way out is to treat bytestring urls as 'from the wire' and unicode ones as 'from noone knows where' [02:23] lifeless: That might be hint that subunit needs better docs [02:23] deleting 'import threading' and 'import random' from test code can only be good [02:24] wallyworld_: How is he melting in the heat? It's like 29 there? [02:24] Eep [02:25] regression [02:25] 1716 InconsistentBuildFarmJobError: Could not find all the related specific jobs. [02:25] Crap [02:38] review for build failure please? https://code.launchpad.net/~mbp/launchpad/891028-notunique [02:41] poolie: The critical bit is probably fixed by the rollback. We'll see in half an hour or so. [02:41] oh you rolled back everything? [02:41] all 5 revisions or whatever [02:41] [Branch ~launchpad-pqm/launchpad/devel] Rev 14304: [testfix][r=bac][rollback=14301] [02:41] it's still a bloody stupid bug, if it is what it seems to be [02:43] heh, that's what I was planning on doing last night [02:43] poolie: so when I brought that up in our stand up, it got brought up that there's some part of the tests that use the number for a "number of iterations" or something? [02:43] hmm, well you're still count() though nvm [02:44] StevenK: well, TBH you're the only person that has this trouble [02:44] if they are counting on getting particular versions they need to be shot [02:44] StevenK: perhaps the other users are unix diehards [02:48] StevenK: have you considered using testr ? [02:53] okies, time to vote, and due to early start, EOD too. [02:53] enjoy y'all [03:09] lifeless: Clearly, I am not smart enough to use subunit. [03:15] can anyone give me a one liner to search for critical and high bugs using launchpadlib? [03:16] and I don't suppose I can search in one go over both packages and projects can I? [03:16] wgrant, poolie, StevenK: ^^? [03:16] I don't think searchTasks() will bend to that particular case. [03:18] somecontext.searchTasks(importance=['Critical', 'High']), I expect [03:18] It will only search over one context, or all contextys. [03:19] So you probably want to fetch your DSPs and then sprinkle in your products and call searchTasks over each of them [03:19] jtv: Can we have our builders back yet? [03:20] wgrant: thanks [03:20] wgrant: actually I was hoping to get you to try it for ubuntu uploads first. [03:21] Only my mouse pointer went off somewhere and using my system's a bit hard. [03:25] lifeless: Are the tracebacks not going to ENOSPC us reasonably quickly? [03:30] wgrant: how do I get a source package from a distro (ubuntu) in launchpadlib to use as a bug target? [03:30] wgrant: any chance you could try out a bunch of concurrent ubuntu builds on dogfood? [03:33] thumper: lp.distributions['ubuntu'].getSourcePackage(name='some-broken-compiz-plugin') [03:33] wgrant: the new pruner is ready and usable [03:33] lifeless: Ah! [03:33] lifeless: That is good news. [03:34] wgrant: its in oops-datedir-repo 0.0.12 [03:34] lifeless, i'm having a bit of trouble understanding why you would want bugs in the help text to just not be tracked at all [03:34] ta [03:34] wgrant: and I've ping in a few rt's about it :P [03:34] lifeless: Right, but there are like 100 RTs :) [03:34] And not much IS this week. [03:35] poolie: uhm, I may well have knee jerked. [03:35] jtv: Maybe. [03:35] poolie: the combination of not-code and critical got to me [03:35] jtv: Ubuntu in particular, or just non-virtual? [03:36] poolie: perhaps a better reaction would have been to retarget it to launchpad-moin-help-theme and priority low [03:36] imbw, i thought broken links were critical [03:36] it's not a bug in the theme either [03:36] i will as you suggest justnot mention the ppa [03:36] people can use the one on oneiric [03:36] poolie: oopses are critical; broken links generated from within LP oops (by design) and those are thus critical [03:39] \o/ mirror finally got onto multiverse [03:39] now I can install locally, in principle. [03:41] mirror? [03:42] I've been bootstrapping an ubuntu archive mirror for a while now [03:42] gah [03:42] slow nets [03:42] long pipe [03:42] * thumper foiled by unicode again [03:42] Oh. [03:42] 150kB peak rate/s [03:42] I was about to say, the mirror package has been around for as long as I can remember. [03:45] jesus [03:46] the use of randomness in tests just gets worse [03:48] allenap: https://bugs.launchpad.net/txlongpoll/+bug/891251 may interest you [03:48] <_mup_> Bug #891251: txlongpoll oopses are recorded with the wrong permission, causing oops loader script to fail with a permission denied error < https://launchpad.net/bugs/891251 > [03:48] wgrant: in principle, we want to exercise all kinds of job we have. I just don't feel we can realistically do it. I think I'll try TTBJs though. [03:58] lifeless, thanks for the review of the randomness patch [03:58] it turns out the testtools one will not quite fit [04:05] ah well [04:05] it's good to know it's there though [04:15] wgrant: I think it would make sense to do some ubuntu uploads on dogfood, in case it exercises other code paths that write to the database without having been permitted to do so. Once my branch lands, there's a good chance that we'll find more of these. We should minimize that risk though. [04:18] jtv: You can't throw your 10 zillion hellos at Ubuntu? [04:18] Can I? [04:19] Why not? [04:19] Indeed, why not? [04:23] wgrant: what location should I feed to dput? My “incoming” setting is “%(dogfood)s”. [04:23] jtv: dogfood:ubuntu [04:23] Thanks. [04:23] Or dogfood:ubuntu/oneiric to override the series [04:26] it's kind of interesting how the lp test suite doesn't use a full cpu even when it's not doing io [04:26] i don't know what it is doing [04:26] poolie: sledgehammer trick — ctrl-C it and look at the backtrace. [04:27] mm i know [04:27] My apologies. [04:28] I guess we're not doing enough context switches (yet) to make that the cause. [04:28] it's ok [04:28] I mean, to make context switches the cause of such symptoms. [04:28] we might be [04:28] there is a lot of interprocess io [04:29] mwhudson, still here? could you read https://code.launchpad.net/~mbp/launchpad/ec2-region/+merge/82487 [04:29] poolie: looking [04:31] * mwhudson goes cross-eyed [04:31] time to try meld [04:31] :/ [04:31] poolie: I'd be quite interested to learn how you see the interprocess I/O going on, incidentally! [04:32] strace [04:32] lots of reading and writing on sockets [04:32] Ah :) [04:32] i guess talking to the db [04:32] Does strace show the difference between a socket and a file? [04:32] there is also iotop i guess? [04:32] * jtv runs that [04:33] Fun, thanks! [04:33] not directly [04:33] though if you see it previously polling on the fd, or things like that, it's a good clue [04:33] True. [04:34] There's also the "wa" column in "top," but that's only a global number. [04:34] poolie: Multiverse is required [04:35] StevenK, it's added by the sed line a bit higher up [04:35] in a way that does not hardcode the region [04:35] Right [04:36] wgrant: where do I observe my uploads to ubuntu? /ubuntu/+builds? Or is there more? [04:37] /ubuntu/+source/ [04:37] Thanks. [04:37] StevenK, is that what you were asking about? [04:38] poolie: Yes, but your branch still makes me ... uneasy [04:38] cause it's changing something dangerous? [04:39] Because it's changing something that isn't well tested. [04:39] yeah i know [04:39] And ec2 issues are hard to debug [04:39] much like testing lplib clients, it's hard to test it without a whole server cluster on the other end [04:39] fwiw this makes several failure modes easier to understand [04:39] Which isn't helped by the ec2 test runner going "Hrm, can't work out what is going on. sudo halt" [04:40] yeah [04:40] we could easily have an option to not do that [04:42] poolie: surely you can delete _convert_instance_type now? [04:42] fwiw i've run it several times here with no unfixed errors [04:43] done [04:45] * mwhudson is reminded that, now we don't do the private dependency thing, ec2 test could detach plenty quicker [04:45] that would be nice [04:45] especially when travelling [04:46] mwhudson: Fix eet? [04:46] StevenK, i'm kind of glad i broke my personal ice of finding this too scary to change [04:46] StevenK: not today [04:46] poolie: \o/ [04:46] Well, there are several tests or [04:46] functions reached from tests that reset the random seed, for instance [04:46] test_token_creation just sets it flat out to zero [04:46] What [04:46] it's not that bad code [04:46] seriously? [04:46] "wgrant moment" [04:47] Heh [04:47] !!!!!! [04:47] someone from ubuntu will tut me if i write down here what i actually said [04:47] "i want something kinda random but not actually random" [04:48] the reason for the randomness iirc was to stop people putting 'person-name15' in doctests [04:48] :( [04:48] haha [04:48] Yeah :( [04:48] it now starts at 100,000 and counts up across all tests [04:48] Which didn't actually work [04:49] people will realize pretty soon it's not predictable [04:49] :/ [04:49] Since people started doing factory.makePerson(name='fred') [04:49] i hope they don't count on it being 6 digits but we can only do so much [04:49] TBH, I write my tests to not care. I just want an IPerson. [04:50] StevenK: that's ok [04:50] wgrant: process-upload.py doesn't like “dogfood:ubuntu” as my dput upload path. It says: Could not find distribution 'dogfood:ubuntu'. [04:50] if you care about the name, you can care about the name [04:50] you just have to be explicit about it [04:50] jtv: dogfood:ubuntu, not dogfood:dogfood:ubuntu [04:50] so can i get an upvote? [04:50] https://code.launchpad.net/~mbp/launchpad/891028-notunique/+merge/82480 [04:50] DId you guys fix the random thing not being actually random? :) [04:50] wgrant: whoops, you're right. Sorry. [04:50] i think doing random.seed(0) at the start of the tests is arguably a good idea [04:50] poolie: approved your branch btw [04:51] let's not pretend [04:51] thanks [04:51] poolie: yeah, seed dependent failures are bad [04:51] nigelb, see above [04:51] \o/ [04:51] Nice. [04:51] * nigelb reads the mail thread as well [04:53] How did these tests work until this week. [04:54] there used to be say 2500 getUnique calls before you reached this test [04:54] now there are 2501, and that causes the counters to collide [04:54] or 2512312 and 2512313 [04:54] lol, damn +1 [04:54] not necessarily exactly one more [04:54] oh. wow. [04:54] but the patterns aligned [04:54] i'm not totally sure but it looks a lot like that [04:55] poolie: approved your other branch too [04:55] i have to confess i have not run its own tests locally yet [04:56] for ec2 [04:57] lifeless: oops raising question for you. attempting to delete the last bug task raises a CannotDeleteBugtask exception and oops in browser. this doesn't normally happen except if two users are involved and step on each others' toes. i am going to fix the web ui to display a nice message. if i catch the exception in the view code, will that still generate an oops on the server side? do we care? [04:57] wallyworld_: If you catch and don't reraise, it won't log an oops. [04:57] the view code is server side [04:57] Only unhandled exceptiongs log oopses. [04:58] if you catch it, its handled, fin. [04:58] lifeless: cool. i was hoping that would be the case, just wanted to double check. thanks [04:59] wgrant: thanks to you too [05:01] wgrant: found out why my builds aren't getting through — I have no upload rights to Ubuntu Main. [05:01] Anything simple I can poke in the database to change that? [05:02] Add yourself to ubuntu-core-dev [05:02] * jtv fiddles [05:03] wallyworld_: Following up on yesterday, I'm just now taking a look, and not quite sure how I override the io request for the test (in https://code.launchpad.net/~huwshimi/launchpad/tag-cloud-removal-709009/+merge/81689) [05:03] * wallyworld_ looks [05:04] wallyworld_: It's trying to get a URL for the request from the HTML [05:04] wallyworld_: I can pass in the config, but what I really need to do is pass it the response [05:04] wallyworld_: I might be missing something here [05:06] huwshimi: so you want to force io_provider to be an instance of MockIO so that your test can check the correct url is being used in the get and also to allow you to poke in a return value [05:06] so in the call to setup_taglist(config), you make config = {io_provider: your_mockio_instance} [05:07] your test will call setup_taglist [05:08] once setup_taglist is called by your test, then you poke in the return value [05:08] by calling a method on the mockio instance [05:09] are there any objections to me landing the deletion of lib/canonical/buildd? [05:09] it will then only be an external dependency [05:09] If the tests pass, go ahead/ [05:09] we have already deployed from the external tree to all the buildds [05:09] thanks [05:10] huwshimi: see for an example get_intensity_listing in test_buglisting.js (there are many others also) [05:11] huwshimi: ListingNavigator is constructed with a mockio instance [05:11] huwshimi: thwn later, mock_io.last_request.successJSON(xxxx) is called to simulate a value sent back from the xhr call [05:12] clear as mud? [05:13] wallyworld_: hmmm [05:14] is that a good hmmm or a bad hmmm [05:15] wallyworld_: Sorry I'm dealing with too many things at the moment, it's a slightly unsure hmmm, I'll have a proper look in a sec [05:15] np. take your time [05:24] wgrant: I've got some uploads, which process-upload.py said it set to UNAPPROVED. What comes next? [05:25] wallyworld_: Oh, so I need to call the function that would be called on io:success manually? [05:25] jtv: Approve them :) [05:25] How? [05:25] Add yourself to ubuntu-archive [05:25] * jtv fiddles [05:26] And go to https://launchpad.net/ubuntu/someseries/+queue [05:26] +dogfood [05:26] huwshimi: no, that is done when you poke a successful response back into the mock io provider using for example successJSON(xxx) [05:27] huwshimi: you can also pass in data simulating an error response [05:27] wallyworld_: Ah I see [05:28] huwshimi: so, if you factor out the success handler to it's own method, you can do stuff like test that method separately without and io involved at all [05:29] just call it from your test with data and see that the DOM is manipulated as expected [05:29] there's a few ways to skin the proverbial cat [05:29] huwshimi: more examples in test_bugtask_delete.js [05:30] wallyworld_: Yeah, was looking at that [05:30] wallyworld_: I think I'm understanding now [05:30] :-) [05:30] wallyworld_: I didn't realise that setting the mockio.success() would call "io:success" [05:31] ah, right. it's handy like that. [05:31] it allows you to simulate a server request -> response without getting into all the io glue [05:39] poolie: Did you break lp-buildd? [05:39] make: dh_lintian: Command not found [05:39] Trying to build 100 in hardy [05:48] wallyworld_: Is there a way to make the log console more verbose? It's telling me about an error, but doesn't tell me where [05:49] huwshimi: you looking at the log in the page or the browser console (F12 in firefox) [05:49] ? [05:49] wgrant, i hope i didn't break it [05:49] where did that happen? [05:49] https://launchpad.net/~launchpad/+archive/ppa/+build/2932600 [05:49] huwshimi: i thing firebug's browser log may be more verbose [05:49] Maybe it was jelmer(?)'s cleanups. [05:50] huwshimi: other than that, it's debug js time or add Y.log()s [05:50] wallyworld_: Hmm.. that just shows me an error in yui.js [05:50] huwshimi: ah, you haven't included all dependencies in the html file [05:51] you need to include each lp js file required, one by one [05:51] that' [05:51] s most likely what's wrong [05:51] wgrant, i don't think i've landed any changes to it since the last deployment [05:51] if you look at an existing html file, you'll see a block for including all the yui stuff, then each lp file included under that [05:51] wallyworld_: Ah I see [05:51] i have one outstanding change up for review [05:53] wallyworld_: Thanks, I was thinking, but I've added it to use(), but we don't have a combo loader :) [05:53] yep :-) [05:53] wallyworld_: :( [05:53] wgrant, yes, jelmer changed launchpad-buildd to run dh_lintian during its own build [05:54] not yet anyway. it keeps getting talked about from time to time [05:54] i guess he assumed it was in build essential or something [05:54] It's in debhelper, just the version in hardy is too old [05:54] Check if dh_lintian is in $PATH in debian/rules, or backport debhelper [05:55] k [05:55] i have to go out for a bit [05:55] please file a bug or fix it or both [05:57] * StevenK is down to 7 test failures, from 132. [05:57] running on tmpfs is 10 minutes, 5% faster [05:58] * wallyworld_ goes to get the kid from school [05:58] At 5pm? [05:58] Oh, bloody daylight savings. [05:59] lol qld [06:10] I thought australia was sane about DST? [06:11] No DST is sane. [06:11] No [06:11] But we are a bit special. [06:12] it's getting better [06:12] The eastern states are all in the same timezone, except that Queensland doesn't do DST. [06:12] So they stay an hour behind. [06:12] WA no longer does DST [06:12] And occasionally one of the other states will decide to change their DST rules one year because why not and what could go wrong. [06:12] I'm not sure about SA and NT [06:12] And then the WA government will announce with two weeks' notice that they're doing DST. [06:13] And then abolish it again the next year. [06:13] nigelb: So no, not really :) [06:14] Currently, New South Wales, Victoria, Tasmania, Australian Capital Territory and South Australia apply DST each year, from the first Sunday in October to the first Sunday in April. The Northern Territory, Queensland and Western Australia do not observe DST. [06:14] With notable exceptions. [06:15] In December 2008, the Daylight Saving for South East Queensland (DS4SEQ) political party was officially registered, advocating the implementation of a dual-time zone arrangement for Daylight Saving in South East Queensland while the rest of the state maintains standard time. [06:15] Oh, sure. That won't be confusing at *all* [06:16] "On 12 April 2007, New South Wales, Victoria, Tasmania and the Australian Capital Territory agreed to common starting and finishing dates for DST." [06:16] IIRC there were changes due to the Commonwealth Games in '06 [06:16] And probably the '00 Olympics as well. [06:16]