[00:09] jelmer: the diff on https://code.launchpad.net/~jelmer/launchpad/bzr-code-imports/+merge/70684 *still* seems to have extraneous changes [00:09] jelmer: which must be quite frustrating by now :) [00:09] jelmer: ah, seems you merged db-devel? [00:17] mwhudson, yeah, as it has the bzr-2.4b4 branch merged, which is targetted to db-devel [00:17] mwhudson, let me resubmit [00:19] pop quiz [00:19] who cares about informational oopses? [00:19] lifeless: people presumably care about the ones made with ++oops++ [00:20] mwhudson: I don't think they would suffer if that was not 'informational' [00:20] jelmer: does the code change depend on schema changes that are not yet deployed ? [00:24] lifeless: No, but it has merged some revisions from an older revision of db-devel which seems to confuse the MP code [00:25] so we don't deploy from db-devel [00:25] -ever- [00:25] -at all- [00:25] landing it on db-devel will just cause conflicts and pain [00:25] ok [00:25] (conflicts when folk changing devel conflict with your code) [00:25] (pain when conflicts happen :P) [00:26] is it perhaps triggering a criss-cross merge case? [00:26] what happens when you merge (but don't commit) devel into it ? [00:27] conflicts :P [00:27] jelmer: so, I would fix that :P [00:27] and propose to devel [00:27] mwhudson: I'm thinking to nuke the 'informational' flag [00:27] mwhudson: and just use specific exceptions like 'ProfilingOops' instead to filter them on the reporting side [00:28] lifeless: ah i see [00:28] mwhudson: things like the hwdb code that logs an oops when it gets a POST missing a field [00:28] suggest that the informational side has been misunderstood and misused [00:29] lifeless: I think it is useful to have an OOPS-like mechanism for non-critical logging. [00:29] wgrant: well, I'm going through our uses [00:30] wgrant: so far I've found one case that makes sense - profiling / ++oops++ - both of which use a custom exception [00:35] the disconnected handling in publisher, for instance - that deserves to be a real oops [00:35] its a problem if its happening [00:35] (it should never be happening) [00:52] yep, all uses were either a) noddy or b) trivially classified on the exception anyway. [00:52] *nuke* [00:55] wallyworld_, did you need me to look at something? [00:55] sinzui: yes, just to approve the ff request: https://wiki.canonical.com/InformationInfrastructure/OSA/LaunchpadProductionStatus#Feature_Flag_Changes [00:55] thanks [01:44] Project devel build #976: STILL FAILING in 6 hr 7 min: https://lpci.wedontsleep.org/job/devel/976/ [01:47] jelmer: lol; meep - your branch is really confused isn't it :) [02:07] StevenK: want a trivial review? https://code.launchpad.net/~wallyworld/launchpad/remove-answer-contact-827051/+merge/71632 [02:32] wallyworld_: heya [02:32] nigelb: hi, got your email, still catching up with things, sorry haven't replied yet [02:33] speaking of reviews [02:33] wallyworld_: still pending on loggerhead;sorry [02:33] https://code.launchpad.net/~lifeless/python-oops-datedir-repo/extraction/+merge/71633 and https://code.launchpad.net/~lifeless/launchpad/useoops/+merge/71634 [02:33] lifeless: np. i was going to remind you later today :-P [02:33] are the next stage of my ripping oops out of LP [02:33] wallyworld_: great thanks :) [02:34] nigelb: will look very shortly [02:34] \o/ [02:34] I think the only thing left is an http module and either a canonical module or some sensible-defaults copied into the core oops system. [02:52] Any reviewers in the house? Got a job for you! https://code.launchpad.net/~jtv/launchpad/bug-826659/+merge/71636 [02:53] Short diff, long cover letter because I vent some steam and formulate some rules for testing. [02:59] jtv: i'll look if someone can mentor [02:59] hi wallyworld_! Thanks. I see a slight conflict of interest with me doing that, obviously. :) [03:00] jtv: yep :-) i and looking at something for nigelb just now but will do the review asap after that [03:01] Thanks. Let me know if you need any mentoring for branches other than mine. And maybe it's time to look at graduation — I have a feeling we may have lost our procedure for that. [03:09] nigelb: your @call_with(user=REQUEST_USER) is having no effect since it's in the wrong place. [03:10] nigelb: you need to add this to an interface method and register the interface with the web service parsing infrastructure so it gets slurped up [03:10] wallyworld_: oh. *thats* why. [03:11] nigelb: there's various webservice.py files in the code base which is where the webservice stuff looks to see what it needs to register [03:11] with your code at the moment, user is set to None [03:11] ah, that explains that. [03:12] you could pick an example interface what is exported to the web service and model what you need to do on that [03:13] so, write a wrapper over the search function using a webservice interface? [03:13] you also should write a test for the web service side of it [03:13] I should write a javascript test too. I've been lazy ;) [03:14] nigelb: you don't need to write a wrapper - you just need to figure out how to get the current logged in user inside your check_bug_links method. [03:15] ah! [03:15] your code looks ok so long as you know the user [03:16] we uased to use ILaunchbag to arbitarily get the user for a given interaction but that's frowned upon now [03:16] That's because ILaunchBag needs to die horribly. [03:16] And painfully. [03:16] the @call_with is one way of doing it if the method is an existing web service [03:17] there may be a better way without having to export teh method as a web service but i'm not sure off hand what that would be [03:17] this has been done before? and I just need to find some example code and work my code the same way? [03:17] StevenK: yes, agree about ILaunchbag. do you know any other way to get the logged in user other than @call_with ? [03:18] nigelb: @call_with is used in many places [03:18] wallyworld_: heh, except I didn't understand the correct way of using it :) [03:19] if you are in a view, you can type self.user i think. but we are not in a view here [03:19] i don't know enough about the zope machinery to understand how the user is extracted from the request [03:21] the only place call_with works is in interfaces? [03:21] nigelb: LinkChecker is initialised with the request object [03:21] nigelb: i think you can get the user from in there [03:21] nigelb: afaik, all web service decorators need to be added to interface methods [03:22] ah, cool. [03:23] nigelb: try user = request.principal.person [03:23] not sure if that's kosher but it works [03:23] heh, okay :) [03:24] nigelb: for anonymous requests, i suspect principal will be None so check for that [03:24] sure, will do [03:25] so forget the web service stuff [03:28] i think IPerson(request.principal) might be more kosher [03:29] wallyworld_: @call_with for web service methods/properties is best practice. [03:29] mwhudson: ah, thanks [03:29] lifeless: it's not exposed as a web service method atm [03:30] otherwise @call_with would be the best solution i agree [03:31] nigelb: see mwhudson's comment above - use IPerson(request.principal) [03:32] wallyworld_: already changed code :) [03:32] excellent [03:32] wallyworld_: ah, I didn't think it could be anything -other- than a web service clal [03:32] wallyworld_: :) [03:34] lifeless: it's defined in zcml as a browser:page implementation [03:34] and the __call__ method is implemented [03:34] yeah, I can imagine [03:36] web service is sort of overkill for what it does perhaps, it's only an "internal" api [03:36] in principle the web service would take care of encoding, auth checking, marshalling etc [03:37] doing it by hand is, well, by hand. [03:37] err, what's the result of a searchBugIds call? I don't seem to be able to understand this. [03:38] Here's the call I'm making and its associated traceback http://paste.ubuntu.com/666969/ [03:38] a list of ids :) [03:39] nigelb: your bug_ids set is full of strings. [03:39] nigelb: also, that python loop will be -very- slow [03:40] nigelb: you need to map it into a set - [03:40] found_ids = set(bug_ids) [03:40] missing_bugs = found_ids - bugs [03:40] oh. [03:40] I didn't know I could do that :| [03:41] nigelb: I wouldn't use an english description either, you want minimal data flowing - this is an implementation detail, not a UI [03:41] so the thing you serialise out should just be the ids of the inaccessible bugs, no explanation why [03:41] lifeless: the why explanation shows up as tooltips. [03:41] nigelb: there is only one explanation [03:41] that's how the javascript end of things seem to be configured. [03:42] nigelb: sure, but you can do that on the client. [03:42] jtv: you and your fancy latin words [03:42] nigelb: disclosing whether the bug was private or just a gap in the bug sequence would leak information about the bug. [03:42] heh, that means creating a new javascript function on the client ;) [03:42] wallyworld_: I'm sorry if I urinated you off, amice [03:43] nigelb: well, up to you where you do it; just saying what I see ;) [03:43] jtv: no, you didn't, just saying :-P [03:43] :) [03:43] lifeless: heh, my first non-trivial branch, so I think I might aim for perfection or as much close to it as I can get. [03:45] nigelb: as long as its fit for purpose [03:45] nigelb: beyond that, shrug :) [03:45] heh [04:14] lifeless: for some reason the missing_bugs = found_ids - bugs isn't working [04:17] nigelb: break in with pdb and have a look around [04:18] in the middle of doing that [04:26] wait, what. [04:26] search returns bug ids as int. [04:26] * nigelb cries [04:26] is there a way to convert that to string or make it return strings? [04:27] nigelb: like I said, your thing was full of strings [04:28] nigelb: no, you should be converting your strings to ints before you query in fact ;) [04:28] lifeless: bug numbers can't exceed the length of ints, can't they? [04:28] [because the data type is int] [04:28] nigelb: do you mean as in MAXINT ? [04:28] yeah [04:29] we can upgrade the column to 64-bit before that [04:29] but we'll be a few decades yet [04:29] oh. ok. [04:35] IT WORKS \O/ [04:36] lifeless: So I should write a loop to create 200,000 bugs to help it along? [04:38] StevenK: ... [04:39] poolie: around ? [04:41] lifeless: :-P [04:44] hi lifeless, otp atm [04:44] to statik [04:44] kk [04:45] wgrant: you are terrible at leave. [04:45] lifeless: like you can talk [04:45] Haaha [04:45] wallyworld_ is right, you so can't. [04:45] mr pot, meet mr kettle [04:45] wallyworld_: I'm talking from a position of knowledge. [04:47] so I'm thinking about oopses here [04:47] lifeless: And I've just fallen into the trap of correcting incorrect question answers. [04:48] wgrant: i hope you take time in lieu :) [04:48] Hah. [04:48] It will only take a few minutes. [04:48] cognitive overhead :) [04:48] I *know* it takes me a fortnight +- to spin down from work, so I plan leave accordingly [04:49] that's a loooong time [04:49] wallyworld_: yes, and if you think back to last december - after about two weeks I was around much less [04:49] Indeed. [04:50] wallyworld_: if I take < 2 weeks leave with the intent of relaxing, its an utter fail [04:50] lifeless: i was in NZ for most of Dec, so didn't notice :-) [04:50] wallyworld_: nice, did you drive around ? [04:50] wgrant: btw, talking about generic names [04:50] http://pypi.python.org/pypi/Products.Collage/1.3.6 [04:51] wgrant: 'Products' as a top level package in python. [04:51] yes, started in Dunedin, all through south island, then onto north island [04:51] lifeless: Sounds like Zope. [04:51] Is it? [04:51] wgrant: plone, so more or less yes [04:51] I think Zope 2 is allowed to have the global namespace. [04:51] RIght. [04:52] Since Zope invented just about everything, and they use sensible namespaces now :) [04:52] so anyhow [04:52] I'm thinking about how to pass in things like the request object, urls, exc_info [04:53] the generic way to wire up for DI would be to supply a container callback [04:53] but this is going to be fuugly for the simple case [04:54] so I'm thinking of having a 'context' parameter to config.create()', and documenting that on_create hooks will receive this, and it can be used to psas things to the hooks, and some well known hooks will look for things there. [04:54] like - attaching exception info would look for context['exc_infp'] [04:54] lifeless: is the preforking server unblocked now? or almost? [04:54] poolie: not until we have the disconnect stuff done [04:55] wgrant: what do you think? The alternative seems to be tighter coupling between report population/creation and the point the report is created. [04:55] StevenK: wallyworld_: ^ [04:56] lifeless: i'm not sure i have enough context - you want to do IOC with the new oops module? [04:57] to allow the oops module to poke stuff back into the container? [04:57] wallyworld_: I want it to play with IOC environments (like lp's) and with non IOC stuff (like LaunchpadScript, loggerhead, gpgverify) [04:57] wallyworld_: so here is an example case [04:58] in LP we use zope.exceptions.exception_formatter to format the traceback in an oops report [04:58] this is pretty zope specific [04:58] even though, thankfully, its a the edge of the dependency graph I think. [04:59] yes, i can imagine it would be zope specific [04:59] so oops *could* use that [05:00] and have config.attachException(report, exc_info) [05:00] this would drag in zope.exceptions and zope.interfaces [05:00] which isn't too bad [05:00] Did you know that imposing Zope dependencies aparent from zope.interface and zope.component is considered a crime against humanity? [05:00] s/aparent/apart/ [05:00] wgrant: well this is why the thought going into this [05:01] an alternative would be to say 'if you want to attach an exception, do it yourself', but thats cruel. [05:01] a third way would be to have an explicit config knob 'call this function to configure the traceback formatter' [05:01] lifeless: i'd rather pass in some sort of context with redirects to the preferred implementation as you say [05:01] the way I am considering is to say 'put your exception in content['exc_info'], and your exception on-create hook will format [05:02] if you want a different formatter, don't put the supplyed on-create hook in there, instead put your own one. [05:02] supplied. [05:02] this kindof shifts the complexity around [05:02] but [05:02] I think it starts to pay off when you consider handling of 'request' objects [05:02] and the like. [05:03] where the core really shouldn't have any concept of such a beast. [05:03] i don't think it introduces too much extra complexity any simple case [05:04] so create(content=dict(request=current_browser_request(), exc_info=sys.exc_info(), program=sys.argv[0]) [05:04] ) [05:04] and let the hooks suck out what they want [05:05] I'm down to *one function* to pick apart [05:05] and then launchpadloggerhead can lose a canonical import, and gpgverify can work. [05:05] maybe a namedtuple rather than a dict? [05:05] wallyworld_: blah :) [05:06] wallyworld_: theres no particular reason to make it immutable [05:06] not so much to make it immutable but for type safety and extensibility [05:06] How's that more typesafe or extensible? [05:06] you'll need to expand on how it gives those in a way a dict would not ? [05:07] i guess you could subclass dict if you needed to [05:07] To do what? [05:07] add extra functionality besides name-values pairs to the context [05:07] wallyworld_: uhhh, you've *totally* lost me. [05:08] wallyworld_: so the point of the context is to pass data from outside 'create' to each on_create hook, without needing create-and-every-callbacj to take **kwargs [05:08] context could be more than just a container for name-value pairs- it couild actually provide some other encapsulated functionality [05:08] wallyworld_: it could; what do you have in mind ? [05:08] just a sec, otp [05:13] lifeless: i dont' have any specific functionality in mind for this case, but in practice such context objects can be used for more than data passing - they can provide useful behaviour such as access to container specific services [05:13] so a dict may be too simple [05:13] wallyworld_: so, I would say that if someone in a container environment needs access to the container, they do: [05:13] context=dict(container=realcontainer, exc_info=sys.exc_info(), ... etc) [05:13] wallyworld_: and have their custom on_create hook do content['container'].containerservicecall() [05:14] wallyworld_: or even implement __getitem__ on their container and pass it in directly, if the protocols line up well enough [05:15] wallyworld_: remember we're duck typed, so there is no need for it to *be* a dict, not even a subclass :) [05:15] that would work. i guess it comes down a little to personal preference. i tend to dislike using simple data structures as integration "glue" since often i find more is needed [05:15] I've come to really appreciate minimalism [05:16] but maybe duck typing removes the need for that as you say [05:18] anyhow, that can be iterated on [05:18] given we have 1(1!) user so far :P [05:18] sure, so +1 for option 3 :-) [05:20] however it's done, we just need to be clear on the contract between the modules and ensure that is satified from both sides [05:20] Now now, clear and sensible contracts are not the Launchpad way. [05:22] lifeless: i'm off the phone but i might have some lunch now [05:22] will ping you after [05:30] lifeless: quick question - what's the storm syntax for a query term checking for "is null" that I can pass to "where=And(*terms)"? instead of [MyClass.property == 'name'] I want [MyClass.property is null] or whatever [05:34] poolie: ok [05:34] wallyworld_: ==None [05:34] wallyworld_: == None [05:35] lifeless: StevenK: thanks! [05:36] wallyworld_: I think you've broken devel on buildbot [05:36] Failure in test lp.answers.tests.test_questiontarget.QuestionTargetAnswerContactTestCase.test_canUserAlterAnswerContact_owner [05:37] StevenK: i'll look [05:43] StevenK: ffs. i didn't remove a test when i did the revert earlier [05:43] FAIL [05:43] yes :-( [05:43] There's a JS test and a python test both failing [05:44] i'll check the js one [05:45] RARGH [05:45] Stupid librarian logs attached to test failures [05:45] USELESS [05:46] StevenK: which js test? bb only shows 1 failure, the py one [05:47] StevenK: You should fix the fixture to only attach the log for the current test. [05:47] wgrant: I should fix the fixture to not attach the log in the first place! [05:47] wallyworld_: Read the summary, there are two. [05:48] StevenK: attaching the log is great [05:49] StevenK: what needs to change is to only attach the *new entries* for a test. [05:49] Sure, great. 3,800 lines between the test failure and the traceback [05:56] Any review mentors available? https://code.launchpad.net/~jtv/launchpad/bug-826659/+merge/71636 [05:57] Well. Bugger. [05:58] Calling the column sourcepackagename hurts when you join against SPR [05:58] :-( [06:02] StevenK: can you possibly +1 this? do I land with 'testfix' now or wait till bb barfs? https://code.launchpad.net/~wallyworld/launchpad/remove-tests-for-badrev-13696/+merge/71642 [06:02] wgrant: are you reviewing today? [06:02] jtv: wgrant is on holidays [06:02] Then he shouldn't be here! [06:03] jtv: *supposed* to be [06:03] Ah yes [06:03] wallyworld_: r=me [06:03] Well, I'll just go have lunch then or something. [06:03] wallyworld_: If you land it now, you may have to force buildbot when it finishes. [06:04] wallyworld_: I added a follow-up note on my MP, even though I'm not formally mentoring that one. :) [06:04] jtv: thanks, will look [06:05] StevenK: i'll be at soccer when it barfs but will force as soon as i return, about an hour later. so i'll land now so it's all ready [06:06] StevenK: since you're familiar with the code... https://code.launchpad.net/~lifeless/python-oops/extraction/+merge/71510 please? [06:07] lifeless: 1169 lines!? [06:07] StevenK: bah, thats the old one. [06:07] one sec [06:07] StevenK: https://code.launchpad.net/~lifeless/python-oops-datedir-repo/extraction/+merge/71633 is the one I meant. [06:08] lifeless: So, calling the new column on SPPH sourcepackagename leads to problems [06:08] StevenK: existing queries barf ? [06:08] Right [06:08] StevenK: lets use a different name; easier and safer than fixing all queries in advance. [06:08] spn_cache or something [06:09] lifeless: I've switched to 'name' [06:09] StevenK: I'd prefer something other than that, because of 'name' being the url component in so many other classes. [06:09] StevenK: however, up to you. [06:09] lifeless: 'spn' / 'bpn' ? [06:09] StevenK: sure [06:11] Subject: RETURNED MAIL: DATA FORMAT ERROR [06:11] heh [06:12] lifeless: r=me, with a slight wrist-slap [06:14] sexy times [06:15] ah yes, the version number. [06:15] that was oversight in the move before [06:48] StevenK: if you are around in about 1hr30 and force a build, i'll buy you a beer :-) [06:50] wallyworld_: You should have landed it with [testfix] [06:51] StevenK: bollocks. can i change it now? [06:52] No, it's landed. [06:52] bollocks x 2 [06:53] * wallyworld_ gotta run to soccer [07:05] lifeless, hello? [07:09] can has reviews? tiny: https://code.launchpad.net/~lifeless/python-oops/extraction/+merge/71644 and getting big but its really simple stuff... just rearranged code: https://code.launchpad.net/~lifeless/launchpad/useoops/+merge/71634 [07:09] poolie: hi [07:09] poolie: sure, if you want to ring the landline ? [07:25] poolie: ? [07:35] Project db-devel build #812: STILL FAILING in 5 hr 50 min: https://lpci.wedontsleep.org/job/db-devel/812/ [07:42] oh sorry, sure [07:48] good morning [08:09] http://bazaar.launchpad.net/~launchpad-pqm/launchpad/stable/view/head:/lib/canonical/launchpad/webapp/dbpolicy.py?file_id=x_Andrew_Bennetts_%3Candrew.bennetts%40canonical.com%3E_Fri_Oct__1_11%3A57%3A48_2004_12702.0 is failing for me, but otherwise code browse was reasonably snappy [08:10] "redirecting the request for this address in a way that will never complete" [08:12] stub: bug 819444 [08:14] stub: O hai. I've had to rename {binary,source}packagename to {b,s}pn, but you think an index on just that column is fine to start with? [08:15] Why the rename, apart from long names suck? [08:15] An index on just that column is fine. If you know the queries it is needed for, we might be able to make a compound index that satisfies those queries as well as the population job. [08:16] stub: Breaks existing queries that join against SPR and SPN [08:16] I'm not sure I like that rationale. We know we need to fix code when changing the schema. [08:17] stub: we can't be sure we've caught all the existing queries [08:17] I would go for 'because the long names suck, so we should just use spn and bpn everywhere' but that doesn't solve your problem. [08:18] stub: the reason for the breakage was a previously unique column name becoming non-unique + hand crafted queries not putting the table name in place [08:18] lifeless: You could say that about every new column addition we make, and it has never been a problem in the past. [08:18] stub: previously we did code + schema changes in lockstep, with several days (or more) qa on staging [08:18] stub: we're explicitly not doing that now... [08:19] Yes, and now we need to run the test suite with both /stable and /db-devel [08:19] Or we are going to have to start naming new columns with UUIDs [08:19] stub: well, its a risk assessment. [08:20] Yes, and next time we denormalize further we are going to call the column spn2? [08:20] stub: I guess I'm saying I'm fine with a name that reduces the risk; if you want a more risky but more consistent name, I can live with that [08:20] stub: I'd hope that we don't use spn unqualified, having learnt that lesson [08:21] stub, lifeless: So, I will kill this ec2 run, and I will investigate fixing the queries that use the full name of SPN and BPN unqualified. [08:21] I want consistent names. If we want to switch to spn/bpn I'd like it to be consistent. This also gets tied up with migrating to text strings over integer ids [08:22] stub: such a migration might be a pessimisation for some queries (absent trigrams...) [08:22] Yes. We don't know yet. [08:22] stub: I think we need more investigation on that particular thing (though I know I was one of the proponents of it before :P ) [08:23] For now, I'd rather stick with the full names and do two ec2 test runs (-b launchpad=stable, -b launchpad=db-devel) [08:23] stub: given stable is merged to db-devel all the time, just db-devel should be robust [08:24] stub: I know the test failures for db-devel anyway, so I will investigate those tomorrow. [08:24] True. So this isn't any different from before then? [08:24] stub: remember that someone could land a break in devel at any time; we probably need a deploy rule 'can only deploy a schema change if the current *live* revno has been tested by buildbot on db-devel' [08:24] stub: we'll figure out all the corner cases soon :P [08:25] matsubara-afk: are you around ? === matsubara-afk is now known as matsubara [08:25] Yes. We knew there would be extra landing complexity and more branches for allowing live patches. People thought the trade off was worth it. [08:25] lifeless, hey, yes. I'm in the QA sprint [08:25] stub: yup, and I still do :P [08:26] Which reminds me, I have branches to land since I couldn't get my branch to push last night due to major sucky 'Net. [08:26] matsubara: I want to patch oops-tools. It wants its own db cluster - is that still correct ? [08:27] lifeless, what do you mean by it wants its own db cluster? [08:27] matsubara: are there any docs on hacking it ? [the README.txt only talks about running locally...] === almaisan-away is now known as al-maisan [08:28] lifeless, hmm README.txt is the closest I have for docs. [08:28] StevenK: So you need to do what I was doing yesterday - separate branch from stable containing the required test fixes to land on devel, which gets merged into your db-devel so you can run the tests to ensure the required fixes really are the required fixes. And don't use devel for the source of your fixes or you risk leaking untested changes from devel -> db-devel. [08:31] lifeless, are you not able to run it locally to test your changes? [08:33] matsubara: I don't know how to run its test suite ;) [08:33] lifeless, make check [08:36] let me have another go at it :) [08:39] matsubara: I want to get rid of the custom oops parser [08:42] lifeless, right. basically you'll have to add your package to the download-cache, update versions.cfg and setup.py with the new package, remove the code from src/oopstools/oops/models.py and update it to use the code in the package. Do you need help with this? [08:42] matsubara: hopefully not :P [08:44] wallyworld_: my MP now works. I think I'll spend tonight / tomorrow morning writing tests. Is there any more clean up that you'd suggest? === gmb changed the topic of #launchpad-dev to: Performance Tuesday | https://dev.launchpad.net/ | On call reviewer: gmb | Critical bugs: 238 - 0:[#######=]:256 [09:13] jml: Hi! [09:13] Dear Thunderbird, why are you taking up 1.2 cores? [09:13] StevenK: you will have to email that question, not put it on irc ... [09:14] Bwaha [09:14] gmb: oh hai [09:14] lifeless: Wotcher [09:14] can haz review? [09:15] https://code.launchpad.net/~lifeless/python-oops/extraction/+merge/71644 [09:15] and [09:15] https://code.launchpad.net/~lifeless/launchpad/useoops/+merge/71634 [09:15] lifeless: Sure. I'll attend to those presently. [09:15] * gmb grabs a cup of tea before getting down to work. [09:16] gmb: thanks! [09:16] np [09:28] hey guys :) [09:29] I have a team subscribed to several packages, is it possible to mute bugmail in only one of them? [09:32] Ursinha: yes [09:35] Daviey: using the ui? [09:36] Ursinha: It might be team admins only.. but for example, https://bugs.launchpad.net/ubuntu/+source/euca2ools/+subscriptions [09:36] I can edit the, "Ubuntu Server Team subscription" listed there [09:42] Daviey: pedro was trying to edit the subscription of desktop-bugs of a package but there was no mute option [09:43] matsubara: so updated all the bits, ran make check: [09:43] bin/test [09:43] Creating test database for alias 'default'... [09:43] Traceback (most recent call last): [09:43] File "/home/robertc/source/launchpad/oops-tools/parts/django/django/db/backends/postgresql_psycopg2/base.py", line 140, in _cursor [09:44] self.connection = Database.connect(**conn_params) [09:44] psycopg2.OperationalError: could not connect to server: No such file or directory [09:44] Is the server running locally and accepting [09:44] connections on Unix domain socket "/var/run/postgresql/.s.PGSQL.5433"? [09:45] lifeless, does `psql -p 5433 lpoops` works for you? it looks like your postgres db for oops tools is not there [09:54] matsubara: thats what I was asking about needing a special cluster ! :) [09:54] matsubara: so the answer is 'yes, it needs a special cluster' [09:55] Ursinha, I believe muting is person-only, teams can't have mutes (i.e. since you can't stop bug mail without leaving the team, we introduced muting; if teams don't want bug mail for a certain package, they should unsubscribe basically) [09:57] right, doesn't apply for ubuntu use case :/ [09:58] jelmer, re your mps for lptools [09:58] i think it's better to bring them in than to quibble about exactly how they're written [09:58] so, 3x +1 [09:59] poolie: gracias [09:59] jelmer: btw that loggerhead open-branch thing is failing 1000x a day [09:59] lifeless, I'm working on fixing it as we speak [09:59] jelmer: so if you're going to do other stuff first, let me know so I can roll it back [09:59] jelmer: ah cool [10:00] lifeless, :-) [10:02] matsubara: the README.txt tells how t make a pg 8.3 cluster [10:02] matsubara: does that need to be updated ? [10:04] lifeless, it should work the same for 8.4. if you can update: s/8.3/8.4/, I appreciate [10:06] matsubara: that seems happier [10:11] jelmer: It looks like the prereq in https://code.launchpad.net/~jelmer/launchpad/bzr-code-imports/+merge/71625 might be out of date; the diff is >3000 lines suddenly. [10:12] allenap: Whoops, thanks - I'll set it back to wip for the moment while I get these other things sorted. [10:16] jelmer: I just approved https://code.launchpad.net/~jelmer/launchpad/bzr-2.4b4/+merge/71624. Want me to run it through EC2 for you? [10:20] * gmb does so anyway [10:20] gmb: Thanks, though my current branch will actually conflict with it (and has higher priority) [10:20] jelmer: Oh, okay. [10:21] gmb: Thanks for the review though :) [10:21] np :) [10:21] gmb: thanks; sorry about the size [10:22] lifeless: No worries. After years of reviewing Celso's branches, 1500 lines is nothing. [10:22] gmb: and all fairly focused :) [10:22] Indeed [10:40] query: archive.getPermissionsForPerson returns an empty list when connected anonymously. Is there any good reason for this? (I can't find a related bug) [10:44] probably means yiou have no permission to read that persons details anonymously [10:45] are such details not provided to non-ubuntu-developers? [10:45] for per-component permissions, it's pretty obvious from team membership [10:46] ubuntu has nothing to do with it [10:46] if you're getting an empty list the api probably got a list with inacessible entries in it [10:46] which it will filter [10:46] so try logging in :) [10:46] what I'm getting at is why does one need to be authenticated to read these [10:48] thats a good question [10:48] why shouldn't you be authenticated though ? [10:48] because this is for ubuntu-sponsoring. There is no good reason to be authenticated [10:48] (except this) [10:50] what does "expired token" mean? [10:50] that the actual login has expired, or something else? [10:50] oauth tokens can be time limited [10:50] e.g. to a month o fuse [10:50] you need to request a new one when that happens [10:51] gar [10:52] # TODO: handle the case of having credentials that have expired etc [10:52] [10:57] lifeless: would you like a bug report? its easily worked-around, but very non-obvious to the API client [10:58] lifeless: I sent an email about a batching issue (too long for IRC) to the lp-dev mailing list. could you have a look at itß [10:58] ...it? [10:58] ß sure [10:59] tumbleweed: up to you, but I'm not convinced its a bug [10:59] lifeless: what is it then? It seems wrong to have to authenticate to read public information [10:59] lifeless: and having to authenticate cronjobs is an administrative pain [11:00] (IIRC this isn't the first time we've tried to do this, but we didn't remember the issue) [11:00] tumbleweed: we have restrictions on visibility on a lot of data [11:01] I'm not saying there is a good reason for this particular case [11:01] I have notice that I can use isSourceUploadAllowed (packageset authorisation), unauthenticated. But I can't determine PPU or component-based upload rights [11:02] anyhow, its much better for us if cronjobs are authenticated [11:02] because when they go rogue and use a tonne of resources, its a pain to track anonymous sessions down [11:02] fair enough [11:03] this one is quite a beast (~10 min running time) [11:06] Ursinha: Why can't the team unsubscribe? [11:07] oh ffs [11:07] any http error maps into keyerror [11:08] wgrant: because we used the list of packages a team is subscribed to to say which packages should the team care about [11:08] *we use [11:09] henninge: hi [11:10] which is bug 626960 [11:10] <_mup_> Bug #626960: Collection dictionary access should not return KeyError when the server is down < https://launchpad.net/bugs/626960 > [11:10] jml: Hi! I resolved it already. [11:10] jml: testtools' setup.py does not know egg_info [11:10] and does not need it [11:10] indeed it shouldn't :) [11:11] I am quite new to setuptools, so I am not sure why [11:11] but sdist produced the right package [11:11] o/ jml [11:13] poolie: hi [11:14] Ursinha: Ah :( [11:15] wgrant: so we're workarounding it by creating teams only to be collections of packages, let's say [11:15] You could also just keep a list of names around :) [11:15] But yeah, using the subscription list is handy. [11:15] wgrant: haha no way... [11:15] I have a few teams that I do it for. [11:15] Much easier to manage. [11:16] yet another list of things... better keep in launchpad [11:16] But unfortunate for notifications, indeed. [11:18] hello gmb [11:18] could you review https://code.launchpad.net/~mbp/lazr.restfulclient/626960-keyerror/+merge/71667 please [11:18] and perhaps land https://code.launchpad.net/~james-w/lazr.restfulclient/reject-non-json-responses/+merge/16960 === al-maisan is now known as almaisan-away [11:35] poolie: Sure, I'll take a look shortly. [11:53] thanks matey [11:59] danilos: did you just make qastaging send me email? I thought that was supposed to be captured? === henninge is now known as henninge-lunch [12:07] poolie: can you land my lptools branches or should I apply for team membership? === almaisan-away is now known as al-maisan [12:13] matsubara: so, have tests running, I will fiddle with this tomorrow more [12:13] lifeless, cool. [12:20] is there a publically accessible list of things that are in a specific package set? [12:20] jelmer: You can query that sort of thing over the API [12:25] StevenK: close enough - thanks [12:26] jtv, hum, interesting, I did some QA this morning for subscribing large teams to PPAs, but qastaging shouldn't send any emails [12:26] That's what I thought. [12:26] And then I got that email about your privates. [12:27] About your empty privates, in fact. Which, on reflection, may not have been your first choice for a message to be sent out to the world. [12:27] jtv, empty privates, though [12:27] :) [12:27] jtv, actually, it sucks that the emails have gone through (all of ~canonical would have gotten it) [12:27] We can only hope that qastaging email is at least whitelisted. [12:27] As they were at the time. [12:27] I didn't see any such mail [12:27] jtv, I didn't get anything, so I hope we are all good [12:28] I got two. [12:28] But then I think I'm whitelisted for staging email. [12:28] jtv, right, that explains it then [12:29] I hope it does! [12:31] Wow. Still hearing a thunderclap that started about 20 seconds ago. [12:32] The lightning was pretty far off (5km at least) but shockingly bright. [12:32] lifeless, hi, do you perhaps know what's our limit for storm caches? I am trying some load_referencing and similar magic, and it seems I am hitting the limit because depending on the order, different statements show up [12:33] I don't, no. I am rather of the opinion that we need to set it as high as it can be without thrashing, because otherwise it just forces pathological behaviour - and we wipe the cache between requests anyway. [12:33] danilos: there is a policy knob in the lazr config that controls it, IIRC [12:34] all that said, every time I've looked at similar symptoms, we haven't had a cache limit problem, rather the cache working on the 2nd page in a test [12:34] or similar [12:35] (because test view instantiation doesn't reset the cache - the storm resetting only happens in thread 1+, not 0+ [12:35] 56 [12:35] lifeless, right, thanks, I'll try to dive deeper (I am using ++profile++show to look at the queries being issued) [12:36] danilos: qastaging, or dev? [12:36] lifeless, dev [12:36] don't forget celebrities do get cached [12:37] hmmm, way past bedtime. I have to crash() [12:37] gnight all [12:41] night-night, lifeless [13:07] Morning, everyone. === henninge-lunch is now known as henninge [13:22] Yippie, build fixed! [13:22] Project devel build #977: FIXED in 5 hr 46 min: https://lpci.wedontsleep.org/job/devel/977/ [13:22] 5 hr 46 min! === al-maisan is now known as almaisan-away === almaisan-away is now known as al-maisan [13:39] gmb: got reviews for you if you've got time. You'll enjoy this one: https://code.launchpad.net/~jtv/launchpad/more-820456/+merge/71689 [13:41] jtv: Just nomming at the moment. Queue 'em up, though and I'll come to them when I return. [13:42] gmb: there's one that may not jump out at you from the queue—brad started on it once but never had time to finish it. [13:42] So it's claimed, but actually looking for a new reviewer. [13:42] Bon appétit. [13:42] jtv: okay. Can you add me as a reviewer so I don't lose track of it? [13:42] Aye-aye. Thanks. [13:43] Cool, and np [13:55] abentley, ping. I can chat whenever you like now. [13:55] deryck: I'll just get coffee... [13:55] abentley, hmmm, me too :) [14:08] What's the old qa report page we used to cut-n-paste bug numbers from when asking for a nodowntime deploy? [14:08] matsubara, can you point me at it? ^^ [14:10] deryck, You mean the report that used to be on lpqateam.devpad.com? If yes, see: https://bugs.launchpad.net/lp-devops-dashboard/+bug/823886 [14:10] <_mup_> Bug #823886: New dashboard design removed available deployable revision and fixed bugs list < https://launchpad.net/bugs/823886 > [14:11] matsubara, yeah, I was looking for that list. Didn't know it didn't exist anymore. [14:12] sorry about that. I'll get to it once I get back from the QA sprint. Meanwhile I guess the only way to get that info is to see the deployment report itself [14:14] matsubara, gotchas. no worries, thanks! [14:32] deryck: actually, wallyworld has landed a rollback as 13696. [14:32] abentley, ah, nice! So we should be deploy ready then, once the qa pages catch up? [14:33] deryck: Looks like. [14:33] abentley, awesome. === al-maisan is now known as almaisan-away [14:38] matsubara: buildbot approved revision 13697 an hour ago, but it's not included on https://devpad.canonical.com/~lpqateam/qa_reports/deployment-stable.html Do you know why? [14:41] abentley, will look into it in a moment [14:41] matsubara: thanks. [14:44] gmb: can I add another MP to your queue? [14:51] bigjools: I haz success \o/ [14:51] Well, not entirely, but mostly :) [14:52] nigelb: congrats [14:52] The MP looks small now, but I need ta add tests :) [14:56] jelmer: Sure. [14:56] gmb: Great :) The MP is https://code.launchpad.net/~jelmer/launchpad/safe-opener-threading/+merge/71670 [14:57] k [14:59] Who is the owner of imported bug comments? [15:01] stub: The Person who made the comment (we create one if they don't exist in LP) [15:01] k. not a robot. good. [15:10] abentley, r13697 is in the deployment report now [15:10] matsubara: thanks. [15:11] np [15:12] deryck: The board is green for a deployment. Can you talk me through it? [15:13] thanks for the review gmb [15:13] abentley, yay! Indeed I can. [15:13] jtv: np. [15:13] abentley, looking for links.... [15:14] abentley, so it's really only a matter of filling out the table under "Requested stable Deployments" on the LaunchpadProductionStatus wiki page.... [15:15] abentley, you have to paste all the bug numbers fixed with links in that table. but the revno is the highest revno to deploy. [15:16] abentley, approved by is "qa-tagger" and deploy to is "nodowntime" [15:16] abentley, and you can see example just above that have been deployed. then you ping a losa once the wiki page is filled out. [15:16] Okay. [15:17] abentley, it takes around 2 hours to do the deploy and you want to be on hand for that time. [15:17] I think that's it. [15:22] abentley: great to see that fix for bug 386596, that also helps towards build-from-branch-into-primary. [15:22] <_mup_> Bug #386596: pushing to a packaging branch can't create a new package < https://launchpad.net/bugs/386596 > [15:27] losa: please deploy 13697. I have updated LPS. [15:27] jelmer: cool. [15:31] gmb: Fancy a short one? https://code.launchpad.net/~allenap/launchpad/wrong-needs-linking-count-bug-795359/+merge/71715 [15:31] allenap: I'll add it to the queue. [15:31] gmb: Ta :) [15:31] np [15:51] thanks again gmb! And I just put up another one. [15:51] jtv: Hah. Might just be able to get that one done before my head melts. [15:51] :) [15:51] gmb: I'll stop here then. :-) [15:52] Performance Tuesday | https://dev.launchpad.net/ | On call reviewer^W^W^Wjtv's review monkey: gmb | Critical bugs: 238 - 0:[#######=]:256 [15:52] * gmb is glad he forgot to put /topic in there, actually :) [15:56] cjwatson: did you get my email about custom-uploads copying? Unless you see any problems with the design, it's all set to go in at your word. [15:58] Thanks gmb :) [15:58] allenap: np :) === gmb changed the topic of #launchpad-dev to: Performance Tuesday | https://dev.launchpad.net/ | On call reviewer: - | Critical bugs: 238 - 0:[#######=]:256 === deryck is now known as deryck[lunch] [16:05] jtv: approved. [16:05] gmb: thanks, and feel free to take the rest of the day off. :) [16:05] Ta :) === beuno is now known as beuno-lunch === deryck[lunch] is now known as deryck === beuno-lunch is now known as beuno [18:25] lifeless: if I'm consuming launchpad openid auth, are timeouts your end, my end, or just the way life is? [19:10] Yippie, build fixed! [19:10] Project db-devel build #813: FIXED in 5 hr 48 min: https://lpci.wedontsleep.org/job/db-devel/813/ [19:18] deryck: there's no OCR, so could you please review https://code.launchpad.net/~abentley/launchpad/fix-unicode-path/+merge/71762 ? [19:18] abentley, yeah, definitely. [19:19] deryck: thanks. [19:19] np [19:25] mtaylor: +login ? [19:26] lifeless: yeah? [19:26] lifeless: (basically, I keep having to re-auth to launchpad all the time in my jenkins, and I was wondering if that was a config thing on my end, or just the way life goes) [19:27] oh, you don't mean 'the login process times out', you mean 'the tokens seem to run out of life' ? [19:28] lifeless: are you around when the LOSAs BOD? [19:29] BOD ? [19:29] oh, start. [19:29] I can be. [19:30] lifeless: We've done a no-downtime deploy of the push-creates-source-package, but they didn't have time for the codehosting deploy. [19:30] lifeless: I can handle it tomorrow if it's not convenient for you. [19:31] abentley: the losas want stuff RT'd while they are sprinting; can you: put it on LPS as a request, RT it (and cc: me on the mail to RT so I get copied), and if I'm around I will touch base with the first losa to show up and see about doing it [19:32] lifeless: Okay. [19:32] thanks [19:36] lifeless: can I use you for "Approved by"? [19:38] yes [19:41] abentley, r=me on the review. Looks good. [19:42] deryck: thanks. [19:42] abentley, np! [19:44] jcsackett, do you have time to mumble? [19:45] sinzui: 15 min? [19:45] sure [19:50] deryck, bug 173019 may not be as easy as you think and I doubt it is high given the age of the bug and the lack of duplicates [19:50] <_mup_> Bug #173019: "Renewal period" is marked as "(Optional)" when it isn't < https://launchpad.net/bugs/173019 > [19:52] sinzui, hey. I thought I marked that low actually. [19:53] :) [19:53] maybe my fingers didn't do what my brain did :-) [19:53] deryck, Past discussion was to use a multistep form, or an overlay to only ask for information that is needed [19:54] sinzui, and I just thought "easy" because I assumed we could drop the "optional" label. If that's not possible without a larger fix, forgive my lack of knowledge. [19:55] deryck, understood. We do not know how to express invariants as a state between optional and required [19:56] sinzui, ah, gotchas. [20:08] sinzui: i'm on mumble whenever you're free. [20:10] can we delete shipit.css now ? [20:34] lifeless: could i get your thoughts on https://code.launchpad.net/~bac/launchpad/bug-813322-2/+merge/71733? not asking for a code review, just feedback on the approach. [20:35] sure [20:35] looking [20:36] sweet [20:37] bac: looks fine; I think that we will find we need to another pass for backend efficiency (e.g. perhaps a separate xaction for each dup we move across, but thats a different ratchet - we're not at that point in our optimisation arc yet. [20:38] bac: so a big +1 from me. [20:38] lifeless: great, thanks [20:45] I need brainstorming help [20:45] we have page ids [20:45] which are kindof a unique part-of-the-codebase-to-look-at [20:46] cronscripts are also unique-parts-of-the-codebase-to-look-at [20:46] but putting 'sendbranchmail' in as a pageid rubs me the wrong way. [20:47] I want to find a generic term that I can use in the public contract for oopses, and migrate towards for the serialized form as well [20:48] "task-id", "action-id", "activity-id", "functionality-something" [20:48] good, good, keep em coming! [20:48] another one I thought of was 'context' [20:49] I like "facet" but we already use that term in LP [20:50] thats ok [20:50] this is for python-opos [20:50] python-oops [20:50] which LP will use [20:50] but needs to be suitable for u1, landscape, possibly isd, possibly ubuntu, package-importer [20:50] its kind of like user-agent [20:51] 'agent' ? [20:51] like, agent: zope3/Person:+index [20:52] "concern" [20:52] thats good [20:52] "agent" is very close to "client" in my head, so that seems a little confusing [20:52] I see [20:53] I don't really like it, but "relm" comes to mind. [20:54] realm ? [20:54] thats another possibility [20:54] heh, yeah I typoed the "a" [20:55] if you saw : "Lucid/apport;program=evolution" , would that make sense to you ? [20:56] or would : "evolution" with other fields noting it was from a lucid box be better? [20:56] I'm thinking about how to write up a recommended use [20:57] to me, "Lucid/apport;program=evolution" communicates well (I think, here's what I think it means: "this error was reported by apport on Lucid but the app generating the error was evolution") [20:58] bac: yeah [20:58] you might break those into two fields, the reporter (which might well be an "agent") and the topic being reported ("program: evolution") [20:58] bah [20:58] benji: yeah [20:58] topic and reporter, I like. [20:58] * lifeless things [20:58] * lifeless thinks. [20:59] so the difference between pageid and url [20:59] is that one pageid applies to many urls. [20:59] theres one other concept we might want, which is for multitenancy [21:00] which I think reporter really fulfils [21:00] claiming a reporter of launchpad gets you into the launchpad oops bucket that only lp devs can see [21:00] folk can inject crap in there if we have an anonymous submission thing, but we can filter that out programattically. === almaisan-away is now known as al-maisan === al-maisan is now known as almaisan-away [21:01] I wonder if some loose hierarchy would work. Something like "ubuntu/lucid/appport" and "canonical/launchpad". [21:02] Although, you can get in several forms of one-size-fits-all trouble with that approach. [21:03] so reporter: 'Describes the program reporting the oops. For instance you might put your program name in here, or your website domain.' [21:04] topic: 'The subject or context for the oops. For instance with a command line program you might put the subcommand in here, in a website you might put the template or function name (something that is common to many different urls)' [21:05] apport would do 'reporter: lucid/apport' topic:'evolution' [21:06] gpverify would do 'reporter: launchpad/gpgverify' or just 'reporter:gpgverify' and for topic the api being called. [21:06] why would apport include "lucid"? I would think that'd be included in some other field if needed (along with all kinds of other environmental info). [21:06] our main appservers would do 'reporter: launchpad/main' or just 'launchpad' and for topic what we currently put in pageid [21:07] benji: oh, so one of the big things ubuntu have is that cross-release crashes are generally considered irrelevant [21:07] benji: that would just be an easy way to trigger partitioning of the analysis [21:08] seems reasonable, "lucid" really is intrinsic to the identity of the reporter then [21:09] I like this - thanks for the brainstorm [21:09] * lifeless goes to make it realityish [21:10] yep, any time [22:11] lifeless: yes - I mean "the tokens seem to run out of life" [22:11] mtaylor: I don't believe we have a mandatory expiry. [22:11] mtaylor: what expiry are you choosing when you create the tokens? [22:11] lifeless: ok- so that's more likely something on my side then [22:11] lifeless: not sure - let me poke around (it's in java, so it takes a moment) :) [22:13] back in a ninute [22:35] back [22:40] mtaylor: ^ [22:42] sinzui: I had a wtf moment when I saw your branch name 'private-bug-1' [23:03] lifeless, I think I could make that bug private without a brach [23:05] sinzui: I think yo ucould too [23:05] sinzui: still, I had a moment :) [23:06] We could retitle the bug to be Microsoft has a dominate share of the increasing irrelevant desktop market [23:07] :) [23:07] well [23:07] we could try. [23:07] Who knows, it might not timeout. [23:08] yeah. We probable cannot close the bug without making Lp twice as fast [23:34] sinzui: http://irclogs.ubuntu.com/2011/08/10/%23launchpad-dev.html