[02:05] yay @ 3KB/s from LP [02:05] (not LP's fault - the weather is messing up my internets) [02:08] The cold should be improving conductivity! [02:36] lifeless: Morning [02:36] lifeless: It was just one page [02:36] wgrant: Everyone must be home and using the internet in the cold ;) [02:38] http://www.stuff.co.nz/national/5434460/NZ-set-for-perfect-snowstorm [02:39] Also, it was a db heavy page, so now I think it was probably okay. [02:39] Sounds right. You should listen to lifeless talk about his Internets at 5pm [02:39] *hates* [02:40] haha [02:40] oh. Today's a Monday and its a holiday! <3 [02:43] grah [02:43] so.umask [02:43] ... [02:43] and then restores it [02:43] no finally. [02:43] no cleanup. [02:43] -> how to mess up an entire test run if one test fails. [03:13] poolie: could you spare a few minutes to teddy bear a design with me ? [03:31] nigelb: better pics - http://www.stuff.co.nz/national/5443420/Major-weather-disruption-around-NZ [03:32] lifeless: it looks beautiful [03:33] of course, I'm saying this while bathed in sunlight ;) [03:38] nigelb: better pics - http://www.stuff.co.nz/national/5443420/Major-weather-disruption-around-NZ [04:38] nice [04:45] time to move the python-oops code to python-oops_datedir_repo [04:49] StevenK: any chance I can get a couple of reviews from you? [04:49] https://code.launchpad.net/~lifeless/python-oops/extraction/+merge/71507 and https://code.launchpad.net/~lifeless/launchpad/useoops/+merge/71506 [04:49] StevenK: mainly deletes again [04:53] lifeless: Having desktop issues [04:54] steven@liquified:~% ps axw | grep -c "" [04:54] 153 [04:54] StevenK: ugh [04:54] StevenK: \o/ [04:54] ^ That isn't helping [04:54] StevenK: sure its not a lp test run parent ? [04:54] No, they're all gnome stuff [04:56] heh [04:56] https://launchpad.net/python-launchpad-filesystem [05:08] # XXX AaronBentley 2009-11-26 bug=488950: There should be separate storage for oops messages. [05:08] lifeless: Haven't you dealt with that? [05:10] StevenK: no [05:10] StevenK: by the end of the arc I will have [05:10] but changing that means fixing the oops-tools bug about actually using the common extracted code. [05:16] lifeless: Both approved [05:16] thank you! [05:17] * StevenK tries to swap back the state he had before the reviews, fails [05:17] :( [05:17] sorry! [05:21] File "/home/steven/launchpad/lp-branches/link-branch-to-private-bug/lib/lp/code/model/branch.py", line 181, in setPrivate [05:21] raise BranchCannotBePrivate() [05:21] :-( [05:21] probably nee da privacy policy [05:21] or non-junk or non-distro branch [05:21] I've been trying to work that out [05:22] The code isn't being very helpful [05:26] lifeless: So I have a product branch, and it still raises the exception. But I don't want the branch to be private by default -- I'm trying to go for what we do with LP itself. [05:32] are you making it private as an admin ? [05:32] No, I'm not. [05:32] I'm wondering if I should [05:32] I suspect you need to., [05:32] AFAIK we have no 'user can opt in' story that is polished :) [05:33] This is in model code, so I probably need to removeSecurityProxy() [05:33] Which makes me sad. [05:49] * StevenK grumbles [05:50] IBugBranchSet.getBranchesWithVisibleBugs iterates over a collection of branches it is passed, and doesn't deal with unauthorized errors [06:14] lifeless: Can haz review? [06:22] if you give me a url [06:23] StevenK: I see it [06:23] uhm [06:23] lifeless: Um? [06:25] uhm [06:26] reviewed; twice :) [06:26] I have to go cook v. soon but I can discuss later, or tomorrow [06:27] lifeless: Public branches can become private -- we use this in LP occasionally [06:27] admin only functionality [06:27] Right [06:28] with this change as it stands, I can make anyone's branch private [06:28] I will discuss it with Curtis tomorrow [06:28] and without a privacy policy, they can't even undo that IIRC [06:31] I've commented in the bug [06:43] you're going to love the next review :P [06:44] Find someone else? Like Henning? :-) [06:44] sure :) [06:44] I've deleted the guts of python-oops, moved it to python-oops-datedir-repo, and added a new new core :) [06:45] datedir-repo? [06:45] Why? [06:45] because thats what the code really is all about [06:45] the structure we need is getting clearer the more I pull things apart [06:46] the lp oops system currently uses an oops repository based around directory named by the date [06:49] anyhow - for whomever feels like it - https://code.launchpad.net/~lifeless/python-oops/extraction/+merge/71510 +248 - 847 [06:51] lifeless: not sure if it isn't core. Can you imagine an installation that doesn't use the directory storage? I thought it would be required as a backup for when more advanced data stores fail. [06:53] lifeless: I'm confused in the first lines of the readme - factory = Config(). Is it a misnamed method or a misnamed variable? [06:55] I'm used to factories having __call__ methods rather than create, but I suspect that is just personal style. I guess create is clearer as the factory needs to be configured before use. [06:57] Not really a factory is it with .publish - more an oops_handler [06:58] lifeless: The report is created by the oops_handler. Any reason to prefer oops_handler.publish(report) over report.publish() ? [07:03] stub: reports are just dicts [07:03] stub: see the README in the lp:python-oops [07:03] stub: thats deliberate for interop [07:03] stub: yes, I can imagine an install that doesn't use the directory storage [07:04] stub: e.g. local mq on the box, let that take care of downtime handling [07:04] stub: or even queue in process: by the time you can restart the process, you've gotten network back :) [07:05] I was wondering if the core library would use it as a fallback for exceptions happening in the pluggable storage. [07:05] stub: I think its best not to, so that the core really is lean - e.g. for inclusion on the Ubuntu CD [07:06] stub: where they'd want to use the apport *style* of disk store (one crash per program per user stored at max) [07:06] So exception handler would be logging.log.exception('Unable to store OOPS') [07:07] stub: perhaps; I think there is room for discussion around that. I can well imagine getting bug reports either way. [07:07] stub: for now, my focus is on a reasonable structure for the project(s), and acheiving enough extraction that gpgverify can import and use the library to do oops reporting [07:07] unblocking the SOA project [07:08] stub: if that makes sense [07:09] stub: another way to do fallbacks would be a publisher clients can use that: looks for an id, if not present then log via logging.error that something is wrong [07:09] lifeless, hi, what's the status of our collided changes (re 728220): I could confirm OOPSes produced tonight have the data properly sanitized, so I just marked the bug qa-ok [07:09] stub: I don't think there is a single right policy for all users [07:09] Sure. I do think the OOPS system should never hard fail, so will need some sort of global exception handler and ability to emit meeps when your publication messes up or your dict isn't serializable. [07:09] danilos: I resolved it all [07:10] lifeless, excellent, thanks [07:10] stub: agree on never hard failing; want to think more about what/how to do for that - at the moment we do hard fail if things blow up, so its not a regression in any sense [07:10] danilos: I moved it into the *make* rather than the *write* [07:11] lifeless: We currently put that burden on the call sites in Launchpad. [07:11] stub: overnight scan-branches blew up, its oops writer failed (directory permissions), so it propogated an exception to the root :) [07:11] lifeless, I assume make happens before the write, which is even better [07:11] (appservers get it for free with publication, and scripts don't bother and would just explode on disk full) [07:11] stub: yeah [07:12] stub: so there is room for us to think about an interface which will let scripts to better and not burden appservers. [07:12] danilos: yes, exactly. [07:13] danilos: you can poke at my merge - rev 13684 - if you're interested. [07:14] Yup. try: [code calling non-core] except Exception: log.exception('OOPception') would be fine, assuming all the logging system callbacks are also bulletproof :) [07:17] lifeless, thanks [07:24] stub: perhaps, though for an appserver that would double-log [07:25] stub: so I think this needs a bit more consideraiton [07:30] When did we start naming_modules_with_underscore? [07:34] stub: well, its not lp core, so its more pep8 than anything else [07:34] which says 'when it aids clarity' [07:35] good morning [07:56] stub: so was that by way of review or just discussion ? [08:10] lifeless: r=stub with comments [08:11] stub: thanks! [08:25] lifeless: O hai -- you remember the denorm branch you looked at on Friday? That should be fine to just lp-land, right? [08:29] StevenK: hahahahahahahahahaahahah [08:36] stub: O hai. So you think we should add indexes for BPPH.BPN and SPPH.SPN before we do the population? [08:39] lifeless: that was brilliant reply. [08:39] Need to have a quotes db. [08:39] one of these days I'll host it myself. [08:39] nigelb: SILENCE [08:54] henninge, hi, will you be reviewing today? if so, I've got a branch up at https://code.launchpad.net/~danilo/launchpad/bug-690568/+merge/71394 ;) === henninge changed the topic of #launchpad-dev to: Das Thema für #launchpad-dev ist: https://dev.launchpad.net/ | On call reviewer: henninge | Critical bugs: 238 - 0:[#######=]:256 [08:55] danilos: I will. ;-) [08:55] danilos: can you fix the lint, please? [08:56] henninge, sure, I didn't want it to pollute the diff [08:56] danilos: that's something we'll have to live with if we ever want to get any transitions done. [08:56] henninge, I usually do it post-review though [08:57] oh, that is a good idea, too. [08:57] danilos: cool [09:03] henninge: O hai, are you free? [09:04] StevenK: not right now but feel free to queue ;-) [09:04] henninge: https://code.launchpad.net/~stevenk/launchpad/use-shutdown-ec2/+merge/71522 when you have a tick. Nice small one. [09:07] StevenK: If you don't have an index in place, the population will go really slow (it will take ages to calculate WHERE spn IS NULL) [09:08] stub: Right. I'm doing development of this in a pipe, I will shift a pipe into the middle for indinces [09:12] danilos: r=me [09:12] StevenK: Looking now [09:12] henninge, thanks [09:12] henninge: Thanks! [09:15] StevenK: did you change from shutdown -h to shutdown -P on purpose? [09:15] sounds better to me , just wondering if there may have been a reason not to use -P before. [09:15] henninge: Since the other shutdown call used -P [09:16] StevenK: Ah! (that's not in the diff obviously) [09:17] StevenK: you mention a bug report but there is no bug linked [09:18] henninge: No, but the bug is mentioned in the code I removed [09:18] duh [09:22] StevenK: I assume you have already tried this out? [09:22] I have not [09:23] henninge: It was going to be tested with the landing [09:23] ah, of course ;-) [09:23] StevenK: r=me [09:23] henninge: Thanks! [09:48] gmb: aaaaa [09:48] gmb: Not sure that's a good one to pick. [09:49] wgrant: Any reason other than "That bug will eat your face and you will die?" [09:49] given it's a) difficult, b) DB changes, c) we [09:49] .. might delete nominations later this year [09:49] d) difficult [09:49] e) face eating [09:49] f) kitten will die? [09:50] nom nom nom [09:50] There are pretty unfortunate complexities around SourcePackage tasks that this would eliminate. [09:50] But... [09:50] Ubuntu and lifeless want a rework of how series tasks work. [09:50] Ah. Hmm. [09:51] So I'm not sure it's worth putting a huge amount of time into fixing my second oldest Launchpad bug. [09:51] wgrant: I have up trying to read Zope 3. Much eeasier to read LP code sadly. [09:51] wgrant: Is there any kind of concrete schedule for said reworking? I'm more than happy to punt this if so, but I want it off the Critical list because I keep seeing it and fooling myself into thinking I can fix it. [09:51] At least not until we know what's happening. [09:51] *gave [09:51] gmb: You could just fix the OOPS. [09:52] wgrant: Ah, good idea. [09:52] It only affects a few bugs from before 2007. [09:56] Do changes to security.cfg need to be submitted to db-devel? [09:58] henninge: no [09:58] lifeless: thanks [09:58] dev.launchpad.net/PolicyAndProcess/DatabaseSchemaChangeProcess 9r something like that [09:59] henninge: New permissions are applied before every nodowntime rollout, but revocations are only performed during downtime. [09:59] Regardless, should be landed on devel. [09:59] ok [10:00] gmb: which bug? [10:04] lifeless: https://bugs.launchpad.net/launchpad/+bug/110195 [10:04] <_mup_> Bug #110195: Nominating a bug for a distro release affects all package tasks for that distro < https://launchpad.net/bugs/110195 > [10:07] gmb: I would fix it by making the thing that fails fail cleanly, vs oopsing [10:07] gmb: that will cover the oops bit, because its not something we need to investigate each time it happens [10:08] grr. posting to +check-links with extra information creates 500 error :| [10:10] gmb: you could fix it fully if you want, but as wgrant says, we probably stand to gain a lot by massively simplifying things - no non-series tasks, and nomination a task status [10:13] lifeless: Yes, I agree; I'll make the OOPS not an OOPS and leave it at that for now. [10:16] guys, if I have a bug number, how do I lookup the validity of the bug? [10:16] err, inside LP that is. [10:18] Is there something similar to lp.code.interfaces.branchlookup.IBranchLookup? [10:18] I wonder if I bit more than I can chew. [10:19] IBugSet.getByNumbers() [10:19] or just .get() [10:20] bigjools: where does it live? [10:20] henninge: time to review? https://code.launchpad.net/~jtv/launchpad/pre-824499/+merge/71536 [10:20] lp/bugs/ ? :) [10:20] you need to install bzr grep [10:20] * nigelb installs [10:21] hm, don't see anything like that. [10:21] bzr-grep I think is the package [10:22] ah, maverick+ [10:22] I'm on lucid. [10:22] tsk :) [10:22] lp's deployed on lucid! So its a technically valid excuse :P [10:23] look in lib/lp/bugs/interfaces/bug.py [10:23] there is a pattern to our files y'know :) [10:23] heh, yeah. I should have guessed :) [10:23] there's a PPA with bzr-grep in it somewhere [10:24] it's a cracking plugin [10:24] bzr branch lp:bzr-grep ~/.bazaar/plugins/grep [10:25] I wonder if we can give nigelb the PyCharm licence key [10:25] maybe not, he doesn't have commit access [10:25] To LP? [10:25] yeah [10:25] I thought that's only for Canonical employees? [10:25] if you commit to other OSS projects you could get your own licence [10:26] indeed [10:26] I've given up on PyCharm for now. [10:26] why? [10:26] wgrant: Oh? [10:26] I like vim tbh. [10:26] I have the vim plugin for PyCharm [10:27] it's almost like using vim except you get something from the 21st century [10:27] heh [10:27] but seriously, it's kinda nice having a decent IDE [10:29] how does the Interface thing work? [10:29] standard OOP? [10:30] e_context [10:30] Like, where do I see the function defitions for fuctions in Ibug [10:31] s/interfaces/model/ [10:31] Oh. [10:31] well he can't call stuff that's not in the interface [10:31] the interface is the best place to look [10:31] it also has docstrings [10:32] nigelb: look in the file I pasted earlier and search down for IBugSet [10:32] class IBugSet [10:32] Yeah, I was looking at that. [10:33] jtv: looking [10:34] jtv: why does "rsyncBackupDists" need a distribution parameter? I see no change in the method body. [10:34] henninge: it becomes necessary for accessing configs later, in the follow-up branch. [10:34] ok [10:35] (self.configs becomes indexed by distro) [10:36] jtv: ah yes, I see there are more lilke that! [10:36] bigjools: (maybe a stupid question), does IbugSet.get() correctly take care of hiding private bugs from the person requesting? [10:37] (the docstrings only tell baout bug not found. Not about bug found, but you can't see it situation) [10:37] nigelb: no idea, I've never done any bugs work [10:37] * bigjools would hope that the security adapter does that [10:37] *whee* grep time :) [10:37] check in the security.py [10:38] Nothing about IBugSet in there [10:39] So we don't want to land code changes on db-devel, yet there are some db changes that require code updates. So the best way of handling this is wranging two branches? One launchpad/stable branch with the code changes, and merge this to a launchpad/db-devel branch with the schema change, and run tests for both branches and land separately? [10:39] (yes, pipes should make this workable) [10:39] stub: DB changes aren't allowed to require code updates. [10:39] stub: I think we _have_ to make the code and db changes work independently. [10:39] stub: Or if they do, it must already be landed on devel and deployed. [10:39] I'm adding a NOT NULL constraint. That requires code to make sure it is never NULL. [10:39] Not sure how renaming columns works. [10:40] It doesn't. [10:40] wgrant: Yes. I'm thinking about how to best wrangle this. [10:40] Well, you could do it with triggers. [10:40] Same as renaming tables. [10:40] stub: What needs wranglement? [10:40] I need to land code changes so I can land db changes and keep the test suite passing. [10:41] I'm adding a NOT NULL constraint. Some tests fail. I need to fix these tests on launchpad/devel. [10:41] The code needs to cope with both old and new schemas. [10:41] The code lands on devel. Once it's merged to db-devel, you can land the schema change. [10:41] Yes. [10:41] The code change is then deployed. [10:41] fastdowntime happens later. [10:41] Then it's all done. [10:41] I'm just wondering if the method of wrangling the branches is most efficient. [10:42] I need to do coordinated landing of changes to launchpad/devel and launchpad/db-devel. [10:42] jtv: that's an almost mechanical change ... ;-) r=me [10:42] stub: Why can't you make the devel changes first? [10:42] I am! [10:42] I mean, at an arbitrary earlier point. [10:42] Yes, I sketched out a method of coordinating this. [10:43] henninge: yes, I wanted to keep that separate from the less-mechanical changes. Thanks! [10:43] My code changes are defined as 'what code changes are needed to allow this database patch to land' [10:44] stub: so you need to sequence it [10:44] stub: a) land code that makes sure its never NULL [10:44] let that get deployed. [10:44] then land the schema constraint [10:44] Yes. As I said before, I'm wondering on the best way of wrangling the branches. [10:44] then deploy that, and merge the db side of it to devel after the deploy [10:45] I'm thinking a pipe branch, first pipe from launchpad/stable containing code changes. [10:45] Second pipe from launchpad/db-devel containing the db-changes [10:45] yes, that should work fine [10:45] Run tests for both [10:45] I'd probably just do separate branhces [10:45] but thats me :P [10:46] lifeless: Speaking of you, when are we likely to lose you? [10:46] Due to the impending lifeless-ng. [10:46] wgrant: 38w1d [10:47] Eep. [10:47] The important things seem to be code branch from stable (as this will be merged into the db changes branch, so we don't want devel leaking in there), and to run the tests for code changes (don't break trunk) and code changes + db patch (the changes actually let me land the patch) [10:47] wgrant: normal births occur from 37w0d to 42w0d [10:47] I know that much :) [10:47] stub: landing separately will naturally run the changes separately :) [10:47] yes [10:48] wgrant: I don't know what they teach at aussie schools ;P [10:48] Hah. [10:48] lifeless-ng? [10:48] in this case, knowing what code changes are necessary requires running the test suite with the dbpatch [10:48] bugid is er what type of variable? [10:49] stub: I would be lazy; make the db branch, submit to ec2, if it lands \o/ if it doesn't, the error list will tell you what you need to change :) [10:50] lifeless: So skew in LaunchpadDatabaseRevision might actually be a good thing, as it gives us a little transition state during the period when patches are deployed on some databases and not others. Or it might just be a footgun. [10:51] stub: depends on what you mean by skew [10:51] stub: with transactions we have skew between slaves [10:51] stub: but never between the version on a slave, and the schema on the slave. [10:52] stub: what I meant by skew was a schema that doesn't match LDR [and not just an index existing] [10:53] ! http://mp3fs.sourceforge.net/ [10:53] Need to think about it anyway. [10:54] I have an oops from my local instance, how I see the full OOPS? [10:54] (its an XHR call, so I can't actually see the page) [10:54] That FUSE idea is an interesting solution. [10:56] nigelb: /var/tmp/lperr/*/* [10:56] lifeless: thanks! [10:57] I may have run into a security problem. Gotta try and confirm. [10:57] nigelb: what are you hacking up ? [10:58] lifeless: making the linkage of bugs similar to branches, greyed out if the bug doesn't exist [10:59] nigelb: uhm [10:59] nigelb: do you mean linkification ? [10:59] yeah [10:59] bug 4595 actually I think [10:59] so, a few tips [10:59] <_mup_> Bug #4595: Don't auto-linkify non-existent bug reports

< https://launchpad.net/bugs/4595 > [10:59] hah, whoever claimed easy is probably lying [11:00] I agree [11:00] I touched javascript and some python, not easy, but fun :) [11:00] I've got it down to harvesting, posting the bug numbers via xhr call, returning invalid bug numbers. [11:01] are you doing it via a webservice call post lost ? [11:01] I'm doing it via an xhr call to +check-links [11:03] I almost got it I think :) [11:03] (except for a possible security concern) [11:04] \o/ It works [11:06] Bah. Need separate branches or db-devel ends up in the code changes. Duh. [11:08] lifeless: It works \o/ Sadly, the I run into the security issue I feared. [11:08] henninge: here's the follow-up to my earlier branch, if you still have time: https://code.launchpad.net/~jtv/launchpad/bug-824499/+merge/71544 [11:18] jtv: time? who owns time? [11:19] henninge: Not saying you have to own it. You might be holding someone else's, for instance. [11:34] jtv: I won't be able to start for another hour or so, I am sorry. [11:34] * henninge is off to lunch === henninge is now known as henninge-lunch [11:38] henninge-lunch: OK — I may be gone when you get back. [11:46] who has worked on bugs that I can take help of? [11:53] nigelb: most folk [11:55] lifeless: oh. [11:55] I just need some help trying to make sure I show only bugs that are visible to the current user, not just valid. [11:56] IBugSet.get() only checks if I have e permission. [11:56] so, I would use search [11:56] not BugSet.get [11:57] because you don't want to check every bug one sql line at a time [11:57] thats pretty much a guaranteed timeout right there [11:57] you can see how to do a narrow search by id if you poke around the bugbranch link code [11:57] wow, that +localpackagediff page is awesome [11:58] BugSet.getByNumbers() is what I recommended earlier [11:58] jelmer: :D [11:58] lifeless: ouch, branches is done that way. [11:58] or so I think. [11:59] Also, bug branch link code? [12:00] I don't know what that is to look for it [12:00] bigjools: that will perform horribly [12:01] nigelb: lib/lp/code/model/branchcollection.py line 408 has some related code [12:01] that might be enough to get you rolling [12:02] *looks* [12:05] bigjools: (the lookup would be fast, but every private bug would trigger separate just-in-time subscription lookups) [12:05] I was only asked about lookup [12:05] bigjools: I know [12:05] * bigjools heads to lunch for real [12:07] adeuring: grahhh regex?! [12:07] adeuring: just use iso8601 [12:07] lifeless: can you explain a bit? [12:08] bin/py -m pydoc 'iso8601.parse_date' [12:08] adeuring: rather than using a regex, use pre-existing code. [12:08] adeuring: if someone messes with a memo, they can get a UDF and we don't have to care [12:09]