[00:00] lk [00:02] lifeless, hi there, [00:02] we're just cleaning out some bugs like bug 722043 where bzr users report ssl timeouts getting to launchpad [00:02] hi poolie [00:02] <_mup_> Bug #722043: bzr crashed on launchpad-login

< https://launchpad.net/bugs/722043 > [00:03] obviously there is one easy bug here which is that bzr should not give a traceback [00:03] assuming we are not doing something dumb that gives a short timeout or causes a spurious failure [00:03] and also assuming it's not a local network problem [00:03] i guess it may be an lp issue [00:03] but, probably just operational or intermittent, and perhaps not worth fliing as a bug [00:03] wgrant: mumble? [00:04] poolie: this is the thing that that thomas guy from openstack whas having issues with, Ithink [00:04] there are a few dupes [00:04] mmm [00:04] so launchpad-login talks to xmlrpc [00:04] the get one is talking to a different vhost [00:05] and not using the xmlrpc proxy [00:05] perhaps its the same, perhaps not [00:05] anyhow [00:05] so, soon if not already, we should cease using xmlrpc on trunk [00:05] we want to know of these issues so we can fix, where the fix is at all doable by us [00:05] (maybe not for anonymous access?) [00:05] sinzui: mumble mumble sure. [00:06] but the one talking to the webapp will still exist [00:06] sinzui: we can hear you [00:07] yup === sidnei_ is now known as sidnie === sidnie is now known as sidnei [00:54] lifeless: I eliminated packagebuild last night. [00:54] buildfarmjob will hopefully die today. [00:55] Apart from a couple of Storm bugs it was fairly simple. [00:55] (inherited References == breakage) [00:56] \o/ [00:57] wgrant: and bpb ? [00:57] lifeless: PB has been flattened into SPRB and BPB. [00:57] oh right [00:57] I mismentallyhandled your change [00:58] They both now reference BuildFarmJob, which I will hopefully flatten into them today,. [00:58] But it's slightly harder. [00:58] so bfj is all that remains to flatten [00:58] Since we query based on it at times. [00:58] it has three users [00:58] Yeah. [00:58] There are about two or three methods that use BuildFarmJob in a generic way, so I just need to fix them. [01:26] * wallyworld off to doctor for blood test [02:52] hmm [02:52] we needs a deploy [02:53] We do. [02:53] * wgrant QAs process-accepted. [02:56] * wallyworld shakes fist at windmill... again [02:56] wallyworld: Oh? [02:56] why dies waits.forElement(xpath=xxx) work but asserts.assertNode(xpath=xxx) fail [02:56] same xpath [02:56] Hah [02:56] haven't looked at the code, but you would think it would work [02:57] anyhoo, another test passes [02:57] again [02:57] wallyworld: Which? The broken one? [02:57] till next time something breaks [02:57] StevenK: Is canonicloud broken? [02:57] test_filebug_extra_options [02:57] Likely suffering from the release DDoS [02:57] Yeah. [02:57] wallyworld: Hmm. [02:58] wgrant: plus the xpath was actually wrong [02:58] wgrant: i fixed it and then was wondering why it still failed [02:58] wgrant: Yes, tis broken. [02:59] StevenK: Great. [02:59] As long as it's not our problem. [03:08] Ha. Those nodes are going to take a WHILE to come up. [03:09] Oh? [03:09] a.u.c overloaded? [03:09] Yup [04:00] hmm [04:00] now I have unity on my primary machine, more quirks emerge/ [04:01] Oh? [04:01] the machine scrollbars don't show up reliably [04:02] I'm having to do a binary chop along the length of the window to find the horizontal scroll handle [04:02] Hah [04:03] wgrant: you don't have that problem? [04:04] lifeless: It seems to only appear next to the thumb. [04:04] wgrant: thumb? [04:04] lifeless: The orange/filled bit of the scrollbar. [04:04] I just have a thin grey line along the bottom of the screen [04:05] Oh, not sure about horizontal scrollbars. [04:05] Ok, squinting I can just make out that there is a difference ther. [04:06] I think this sill last until I get a headache [04:06] :( [04:06] I quite like the new scrollbars. [04:07] Except the whole "1px should be enough for everybody" thing. [04:08] I can't make out the thumb on the vertical on my CRT at all [04:08] Hmm, it's fairly obvious here. [04:08] But I don't use a CRT. [04:09] I have to move the mouse downthe window carefully to stay in the grab region width - until the grab appears [04:09] to figure out how far through the document I am [05:19] * StevenK tries to work out how to get from a ResultSet to a hash [05:20] a hash? [05:20] Er, sorry, Perl-ism [05:20] dict [05:20] your perl is showing ... [05:20] :-) [05:20] map/dict/whatever you want [05:20] the default is to give you objects [05:20] Hm, which aren't really keyable [05:21] But the code looks it up anyway [05:21] perhaps some context would help [05:21] lifeless: I'm fixing populate_distroseriesdiff to use DistroSeriesParent [05:22] lifeless: DistroSeriesParent is a table that shows the relationship between a parent and a derived series -- please note that this is a Many-to-1 relationship WRT to the child. [05:23] lifeless: So I'm fixing the code to deal with multiple parents, but there is code that looks up all derived series -- this used to be simple, since we assumed that if IDistroSeries.parent_series was set, and wasn't the same Distribution as the child, they are derived. [05:24] StevenK: isn't it just a join now ? [05:25] Since I deal with multiple parents, I need to loop, which is fine, but I'm trying to work out how to store the mapping of child to parent(s) -- a dict of {'child': ['parent1', 'parent2']} sounds like a good solution [05:25] child_distroes = list(store.find(distroseries, distroseriesparent.childID==distroseries.id, distroseriesparent.parentID==self.id)) [05:26] do you need transitive children or direct children ? [05:26] Actually, this is seperate code, and is wanting to find all derived distroseries, regardless of parent [05:26] if you need transitive children you want a with clause and a recusrsive query. [05:26] StevenK: so you want all distroseries that are derivatives? [05:26] (IE, self.id makes no sense) [05:27] derivatives = list(store.find(distroseries, distroseriesparent.childID==distroseries.id)) [05:27] lifeless: I can find that easily enough -- I'm trying to figure out how to store the information that doesn't result in a massive loop over a ResultSet [05:27] derivates = set(store.find(distroseries, distroseriesparent.childID==distroseries.id)) [05:27] ? [05:28] But how does that help the multiple parents case? [05:28] I'd also like to get the parents at the same time -- pre-loading them [05:29] StevenK: you didn't say you wanted the parents :) [05:29] Oh, sorry. I do. :-) [05:30] derivatives = set(DecoratedResultSet(store.find((DistroSeries, DistroSeriesParent), DistroSeriesParent.childID==DistroSeries.id), operator.itemgetter(0))) [05:31] mmm [05:31] .parents will be a join [05:31] so you'll need a more complex one - you need a cached property on DistroSeries that can remember the parents for you [05:31] then [05:31] def eager_load(rows): [05:31] for distroseries, parent in rows: [05:32] cachedpropery_manager(distroseries)._parents.append(parent) [05:32] [with the obvious tweak to handle the first parent] [05:32] and the query becomes [05:32] derivatives = set(DecoratedResultSet(store.find((DistroSeries, DistroSeriesParent), DistroSeriesParent.childID==DistroSeries.id), pre_iter_hook=eager_load) [05:33] ) [05:43] I finally realized why people wear trousers at work. [05:44] Just now? [05:44] Yes. Sometimes coffee mugs just break at the handle, and I just realized how much worse things could have been. [05:44] I knew there had to be a reason, since so many people were doing it. [05:44] But does that imply a good reason? Many people use Windows. [05:45] So quite an epiphany. [05:47] *However there is the corollary of multi-touch trackpads, Unity gestures, and electricity-conducing liquids not mixing well. [05:47] No amount of trousing is going to help with that. [05:49] Better shut down and let this dry up... no way I'm getting anything done like this. [05:52] lifeless: Where is cachedproperty_manager defined? My grep didn't find it. [05:53] from lp.services.propertycache [05:53] Ta, I shall dig there to make sure of _ placement and so on [05:54] StevenK: I've given you a brief sketch; if you have trouble shout and I'll explain missing bits [05:54] StevenK: this idiom is in reasonably widespread use [05:54] FWIW [05:55] Do you mean cachedproperty(distroseries)._parent, by the way? [06:12] lifeless: ^ I don't think you do, since cachedproperty is a decorator [06:12] right [06:13] get_property_cache [06:13] lets you get at the cache [06:14] .parents should be the attribute, if the cachedproperty will be 'def parents' [06:14] and you want the regular join attribute to be on _parents [06:14] so you have [06:14] @cachedproperty [06:14] def parents(self):; [06:14] return list(self._parents) [06:14] which will support folk doing late evaluation of the parents [06:16] I see where you're going -- adding a decorated method onto IDistroSeries [06:22] when eager loading you inject into the property [07:44] evening stub [07:44] I haven't made any trigger progress [07:44] had a day of copying vms around basically ;< [07:44] yo [07:44] ok [08:00] hi all [08:00] that '-' bastard is a slack reviewer [08:03] Haha [08:04] poolie: There is no OCR for Friday APAC [08:04] StevenK, i don't recall often seeing one in apac timezones [08:04] in fact i had kind of thought the concept was dead entirely [08:04] but apparently not in eu/us [08:04] poolie: https://dev.launchpad.net/ReviewerSchedule [08:05] what should i do to get https://code.launchpad.net/~mbp/launchpad/mail-cleanup/+merge/58867 going again? [08:05] APAC reviewers tend to defect to other parts of Canonical. [08:05] Haha [08:05] oh huh [08:06] poolie: Given the time, I'd suggest waiting for adeuring and asking him. [08:06] realistically i may not be able to land it and catch fallout before i leave for uds etc [08:06] k thanks [08:06] I think wallyworld should become a reviewer* [08:07] * StevenK drops thumper and leonardr from the table [08:07] good idea [08:08] And rockstar :-( [08:10] poolie: That MP looks approved to me [08:11] because of the 'approve code'? [08:11] Yes, benji approves [08:11] lp is a bit unclear about votes vs mp status [08:11] StevenK: i sort of already am, but maybe not "formally" [08:11] so i can set the state to approved? [08:12] poolie: If you feel it has been reviewed enough, sure. [08:12] i would actually like a response to my comments [08:12] or a sanity check of them [08:12] StevenK: but i see i'm not listed. i'll get that fixed [08:13] wallyworld: I'd suggest you talk to bac [08:13] StevenK: ok. i originally talked to tim [08:13] poolie: Right, if you're not sure, then I suggest you do what I first suggested and discuss your concerns with the OCR today, which is Abel. [08:13] StevenK: i'm rooming with bac at the summit and uds, so i'll bring it up in person :-) [08:13] poolie: I'm looking over it now. First thing, is bare execpts are always evil and should be except Exception:, or else you cause grief with Ctrl-C handling etc. [08:14] Or you could wait for stub to finish. :-) [08:14] wallyworld: Sounds like a plan. Then he can't escape. :-) [08:14] thanks stub [08:14] yup :-) [08:14] the bare except was already there, but i don't mind changing it [08:15] since it's generating an oops, arguably it should be catching internal errors [08:15] It would be nice. I consider it a bug unless there are big fat comments explaining why BaseException exceptions need to be caught too. [08:15] but as i said, recovering from that gets messy [08:16] From my POV, anything that it doesn't catch should either not log an oops, or will mean the environment is so unstable you probably can't log an OOPS. [08:17] So +1 for the big fat comment, but I don't think it justifies the bare except in this case but haven't thought it through as much as you might have :) [08:18] * stub wonders how this interacts with the OOPS handling built into the logging system now [08:18] I guess it isn't invoked since you are not calling log.exception() [08:19] And you need to do it yourself to email the OOPS id. [08:19] y'know, friday night. [08:19] I might have a *beer* [08:19] You rebel. [08:20] poolie: Do we want an expiry time for the email we stuff in the Librarian? Or is the default 72 hour stay of execution enough? [08:20] lifeless: Fixed your machines yet? [08:20] lifeless, you promised me a beer last week [08:21] stub, i'm just moving the exception code around not changing it [08:21] similarly for the oops logging [08:21] if people are finding 72h enough so far perhaps that is enough [08:21] Yes, but we have so much bad code it's always good to be pedantic about stuff as we see it :) [08:22] ok [08:22] poolie: I did [08:22] poolie: if you're ok with the odd 'one sec while I turn the potatoes' we could chat ~now [08:24] poolie: You can fix a long standing bug here if you like by converting the domain to lowercase around line 415 [08:28] second review in anyway, not that it is needed :) === almaisan-away is now known as al-maisan [08:32] poolie: so, did you want skypebeer ? [08:37] wallyworld: sorry mate, bad news for you [08:41] thanks stub i appreciate it [08:41] good morning [08:41] adeuring: Morning! [08:41] hi StevenK! [08:43] hi adeuring [08:43] hi poolie! [08:43] i put up a tiny mp at https://code.launchpad.net/~mbp/launchpad/feature-scope-cleanup/+merge/59460 [08:43] edging up on more scope work [08:44] i'd appreciate a review [08:44] also a meta review on whether it is too small of a chunk to land [08:44] (otoh it is probably all i will do on that today, so it's worth pushing anyhow) [08:47] lifeless: ?? bad news about? [08:47] wallyworld: gpghandler branch [08:47] lifeless: yeah, but i redid it and it's back in ec2 [08:48] wallyworld: your redo tries to run twisted code in the main appserver stack [08:48] wallyworld: I assure you that thats not a good idea [08:48] oh. why? [08:48] because its zope [08:48] not twisted [08:49] zope is an app server framework, no? so surely it can handle services eunning within it? [08:49] its threaded [08:49] zope is callbacks in a single thread [08:49] entirely different models [08:49] :-( [08:49] not like the EJB environments you may be used to [08:50] yeah, looks like i found ou tthe hard way [08:50] imagine running tomcat specific code in a jboss container [08:50] so i'll cancel the ec2 run [08:50] it might work [08:50] or it might appear to work and crash and burn after several days uptime [08:50] I also noticed another bug [08:51] well, the tests which instantiate the zope layer "work" in isolation. so perhaps the latter [08:52] yah [08:52] (and tomcat and jboss are near identical when you consider the zope / twisted differences) [08:52] I've commented on the other bug [08:52] lifeless: perhaps the best solution is to create a cron script which just touches anything under /tmp/gpg-* ?? [08:52] we have several hours analysis to do I suspect [08:53] wallyworld: *Ew* [08:53] ok, i'll look at the comment, after i play soccer [08:53] wallyworld: Enjoy the weekend [08:53] wallyworld: I think the right thing to do is to disable this 'feature' for now. [08:53] StevenK: i know, but atm i'm out of ideas [08:53] this is actually non-trivial engineering to fix. [08:54] the gpg check on upload is nice but not -required- [08:54] lifeless: so it seems [08:54] it was added as part of migrating away from the zope ftp stack [08:54] conflating two things [08:54] I would like to see this work properly [08:54] lifeless: so, i'll back out my changes and just disable the check on upload? [08:54] wallyworld: yeah [08:55] ok [08:55] I don't like the idea of a maintenance squad disappearing down this rats nest at the moment [08:55] yeah [08:55] we can have a high bug open to reinstate it [08:56] and note that its a couple days work because having a static dir leads to conflicts, having dynamic dirs needs this if there is a tmpreaper, but 'this' needs to be conditional on the specific appserver thats running [08:56] and AFAIK we don't have 'if' statements in our zcml [08:57] I suspect that we process zcml so early in the piece you can't tell if its a twisted thing or not reliably. IMBW === adeuring changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: adeuring | https://code.launchpad.net/launchpad-project/+activereviews [09:10] wgrant: Hi! As promised, I'm back with a question if you have a few minutes :) [09:11] I finally decided to use Archive.checkUpload and not Archive.canUploadSuiteSourcePackage because if the user does not have the right perm, I need to be able to display the reason why not. [09:11] But my question is related to what you mentionned yesterday: the fact that in my first version I was using the source component instead of the destination component. Even looking at how this is handled inside canUploadSuiteSourcePackage I'm not really sure what to do: what component to pass to checkUpload that it. [09:12] More precisely (please correct me if I'm wrong): the sync can be on a package that might not already exist in the destination series (so the sourcepackage.latest_published_component trick and the usage of a suitesourcepackage is not possible) [09:13] rvba: Yeah, that's difficult. At the moment any component (but not packageset) uploader can upload a new package. [09:13] It's not a good solution. [09:13] But it is at least consistent. [09:14] this strict_component parameter seemed kinda hacky indeed [09:14] Hacks? Surely not. [09:14] ho? [09:14] I would replicate the hacky check for now. [09:15] So I guess my next question would be: why is this strict_component needed? [09:15] (I suppose it's a use case I don't know about) [09:17] rvba: So, originally only component-based upload privileges existed. [09:17] Packageset- and sourcepackage-specific privileges are a new addition. [09:17] I see [09:17] So initially it just took a component and person, basically. [09:17] strict_component is used when the package is new. [09:17] So there's no component to check. [09:18] # strict_component is True because the source package already [09:18] # exists (otherwise we couldn't have a suitesourcepackage [09:18] # object) and nascentupload passes True as a matter of policy [09:18] Since there's no component to verify privileges against, and the package will end up in the NEW queue anyway (where it will be reviewed by an archive admin), it was decided to let any uploader upload it. [09:18] # when the package exists. [09:18] Right. [09:19] Not sure I understand what you're saying when I look at the comments I just pasted ... [09:19] (comments com from canUploadSuiteSourcePackage) [09:20] That's a new method, from after the new permission scopes were added. [09:20] Have a look at Archive.verifyUpload. [09:20] It is a little confusingly structured. [09:21] I thought that strict_component=True would be used when a package already exists (and thus is in a specific component) [09:21] But it first checks if you can upload to the package or its packageset: if so, success. Then it checks if you can upload to any components: if not, failure. [09:21] Right. [09:21] * rvba looks at verifyUpload [09:21] That's right. [09:22] Then, if strict_component is set and the uploader doesn't have privileges on that component, failure. [09:22] Otherwise success. [09:22] 18:17:52 < wgrant> strict_component is used when the package is new. [09:22] By 'used' I mean 'changed from the default', ie. set to False. [09:22] ok, makes sense ... if thought "used" = True. [09:22] Sorry, yeah, I was really unclear. [09:24] so ... what component could I use for the sync to make sense ... I guess I have to do 2 different things. One if the package exists in the dest series and one other if it does not [09:25] Right. If it does exist, use the destination component. If it doesn't, strict_component=False. [09:25] makes sense ... but how can I figure out what the dest component is? [09:26] rejection_reason = self.policy.archive.checkUpload( [09:26] uploader, self.policy.distroseries, source_name, [09:26] self.changes.dsc.component, self.policy.pocket, not self.is_new) [09:26] This is what the uploader does. [09:26] dsc.component is the old component, if it's not new. [09:26] You can probably just pass None if it's new. [09:27] You need to get the SourcePackage object (distroseries.getSourcePackage(sourcepackagename)), then use latest_published_component. [09:27] ok, makes sense. Thanks for the clarification, I really needed it :). [09:30] self.changes.dsc.component likely won't exist for what you want to check. [09:30] Ah [09:31] Maybe I should keep reading [09:31] I think so :) [10:56] adeuring, could i have an OCR of https://code.launchpad.net/~mbp/launchpad/feature-scope-cleanup/+merge/59460 [10:56] poolie: sure [10:56] thanks [10:56] is not very big === al-maisan is now known as almaisan-away [11:18] poolie: r=me [11:52] adeuring, thanks! [12:00] Morning, all. [12:02] Hi deryck! [12:03] deryck: great to hear you are ok [12:03] henninge, thanks! it's been pretty rough here in Alabama the last day or so. lots of loss. [12:04] yes, it's been on the news here, too. [12:06] My town was pretty hard hit. I have friends who lost homes and loved ones. [12:14] sorry to hear :( [12:37] jtv: Hi! [12:38] jtv: can you help me with triggering gettext validation errors? [12:41] danilos: Hi! ;) [12:41] danilos: maybe you can help me? [12:42] henninge, hi, how about 'msgid "Blah %d"\nmsgstr "Foo %s"'? :) [12:43] danilos: yes, I got that. I am actually looking for that than format specifiers mismatch. [12:43] danilos: do you know any other way? [12:44] danilos: I am trying to see if bug 385086 can or needs to be fixed. [12:44] <_mup_> Bug #385086: Non-ascii in error messages causes OOPS in +translate < https://launchpad.net/bugs/385086 > [12:44] danilos: the problem is that the OOPS is gone and I am missing clues. [12:51] henninge, imho, if you can't reproduce it, close it, and if it re-appears, we should re-open it with a more recent OOPS if we can find it [13:19] danilos: yes, that's what I was thinking. [13:19] henninge, fwiw, looking at gettext source code, it seems to try very hard to not include any non-ASCII in error messages anymore, so closing it seems a good idea [13:20] danilos: thanks [13:21] henninge, gettext-tools/src/format-invalid.h for reference :) [13:23] danilos: close as "Invalid"? [13:23] henninge, yeah [13:25] staging is dead. doesn't appear to be my fault. :-P [13:26] no stub around. :-/ [13:26] flacoste, fyi ^^ [13:26] lifeless, I assume/hope you are weekending, but just in case, also ^^ [13:27] will ask losas on ops if they have any ideas [13:46] hi all, could someone tell me where I could check when the staging database was last synced with production? I know there is a url where I can see that, but I cannot find it right now [13:46] jcsackett: will you be able to QA bug 769173 today? There are a few things behind it that I'd like to get deployed. [13:46] <_mup_> Bug #769173: "Hidden" comments should be visible to registry experts < https://launchpad.net/bugs/769173 > [14:00] dpm: https://staging.launchpad.net/successful-updates.txt -- last was restored on Sunday from a Friday or Saturday backup. [14:02] awesome, thanks a lot wgrant [14:02] dpm: (of course it's down at the moment, but with luck will be back in a couple of hours) [14:03] wgrant, that's fine, the info you gave me was exactly what I needed [14:15] benji: i see that i got a ping from you about qa, but my alert only showed part of the message. if it was about bug 769173, it's done. [14:15] <_mup_> Bug #769173: "Hidden" comments should be visible to registry experts < https://launchpad.net/bugs/769173 > [14:15] jcsackett: great, thanks! [14:31] dpm, wgrant: i usually look at https://lpstats.canonical.com/graphs/StagingRestoreDurations/ [14:31] a green bar shows a successful production restore/upgrade [14:31] and red bars are successful partial upgrade [14:32] adeuring, hi, would you have time to look at https://code.launchpad.net/~danilo/launchpad/subscribe-unsubscribe/+merge/59511? [14:32] danilos: sure [14:32] flacoste, ah, great, thanks! [14:33] adeuring, it's a very short JS branch, and unfortunately, lacks tests because I am also working some more on the refactoring of that code [14:33] ok [14:34]