[00:23] StevenK: Right, that's the most likely cause4 [00:24] And I've been trying to figure out how to defeat it without success [00:29] * wallyworld_ has a few errands to do, bbiab [00:35] StevenK: It may be some subtle difference between lazr.uri and urlparse, but it's not exactly obvious. Have you tried grepping for other occurrences? [00:36] wgrant: On neem? find . | xargs grep -H "valid_vcs_details" gives no results [00:37] StevenK: :( [00:37] And I don't think our postgres logs go back far enough [00:37] Although they might [00:39] I can recall the original OOPS mentioning https://lomse.svn.sourceforge.net/svnroot/lomse/trunk. (yes, with the extra .), but qas doesn't succumb to handing over an OOPS. [00:43] OH [00:43] That's easy, then [00:44] Yeah, there we are [00:44] (Error ID: OOPS-45b1bf391bcdcaec312388d7a2907957) [00:46] Now, can you work out how I did that? :P [00:59] wgrant: No, actually. [00:59] It's not actually visible in the page [00:59] Because I did the same thing and it didn't OOPS [00:59] But it's in the source [01:00] wgrant: The generated HTML or ProductSeries:+setbranch ? [01:00] StevenK: The HTML of the OOPS page [01:01] OH [01:01] It's a SPACE [01:01] A leading space, yes. [01:02] Hmm, does that make it a lazr.uri bug? [01:02] No [01:02] I thought zope would have stripped fields [01:02] URIField strips it before giving to lazr.uri [01:02] lib/lp/services/fields/__init__.py [01:02] I see that [01:03] _validate() calls lazr.uri.URI on a normalized version [01:03] But all it returns is the superclass' validated version of the input [01:03] And it's just a TextLine, not a StrippedTextLine [01:03] Hah [01:03] It should return uri, not value [01:04] No [01:04] We probably don't want to store the normalised value. [01:05] wgrant: Then I'm confused [01:05] I would suggest that it be turned into a StrippedTextLine, normalize be taught to stop stripping, and _validate to do the super() call first, before passing the validated value into lazr.uri === jamesh_ is now known as jamesh [01:10] wgrant: http://pastebin.ubuntu.com/1270322/ [01:11] StevenK: Something like that, yeah [01:12] wgrant: Did you see the copyright fix? :-) [01:14] wgrant: And how can I test that fix? [01:15] StevenK: There should be existing tests for URIField [01:15] I'd look at them for inspiration [01:16] wgrant: There's already a test that checks that URIField.normalise() returns a stripped version [01:17] Maybe it actually does intend to return the normalised version [01:17] But fails [01:18] wgrant: But it does, the _validate() actually uses it [01:18] StevenK: Sure, it uses it for validation. [01:19] StevenK: But then it returns the superclass' validated version of the *original* value, not the normalise one. [01:21] wgrant: Hah, uri-field.txt fails when calling normalize on ' http://www.ubuntu.com/ ' [01:59] Aw, my fix worketh not. [02:00] :( [02:03] wgrant: You said you don't think we want to store the normalized value? We do already? [02:03] Or is .set() not used for storing? [02:04] set()'s probably used for setting the value in the *field*, usually from the one in the model. [02:05] Given my changes have had no effect, and there is already a test for URIField stripping input, I'm inclined to think the bug is at another level [02:07] Perhaps the field infrastructure is slightly different from the rest of the form stuff [02:08] I'd trace through and see where it might be going wrong [02:08] Since we can reproduce it now. [02:10] ah [02:10] so guys you should learn something bad about code imports [02:12] i am sure there is a bug for this [02:12] but code import editing does not use the usual methods of updating the object [02:12] i.e. LaunchpadFormView.updateFromData or whatever it's called [02:12] so the field validators/cleaners do not get run [02:13] Yeah, luckily we're not quite near that piece of evil atm [02:13] ok [02:13] This is ProductSeries:+setbranch, which uses normalish form stuff and only calls newCodeImport [02:13] But thanks for the warning [02:23] jelmer: Is https://code.launchpad.net/~jelmer/launchpad/retry-update-node-tags meant to be against MAAS? === ccxCZ_ is now known as ccxCZ [02:43] wgrant: So does URIField even factor into it? [02:43] How does. [02:44] wgrant: ProductSeries:+setbranch just calls url = data.get('repo_url') and then tosses it straight into getUtility(ICodeImportSet).new() [02:45] StevenK: But data.get('repo_url') should be the post-validation value, shouldn't it? [02:46] request.form will be pre-validation, but data should be post-validation [02:46] AIUI [02:47] wgrant: Ah. So tracing in update_action is after the horse has bolted [02:48] StevenK: Yes [02:48] StevenK: The form infrastructure will usually validate request.form to produce data, then call the action based on that [02:48] I believe [02:49] But don't quote me on that, unless I'm right [02:49] * StevenK shifts the pdb call to URIField._validate [02:53] So it validates correctly, [02:53] And the superclass validate is just checking if the string not empty [02:54] So then zope.app.form.browser.widget calls getInputValue and gets the non-stripped value [02:54] So maybe it's a bug in URIWidget [02:54] So perhaps field validation and setting stuff are separate, unlike forms. [02:59] OH [02:59] Dear URIWidget, 2004 called, and wants you to start using super. [02:59] Hah [03:00] Hah, I think I can see another bug [03:00] Yeah [03:01] If you pass it a list, it will raise UnexpectedFormData. [03:01] If you pass it a tuple, it won't. [03:02] wallyworld_: Hmmm [03:03] wallyworld_: Why do you not just call getPrecachedPersonsFromIds? [03:03] i do [03:03] wallyworld_: You don't usually need to do messy caching stuff like that if you're using the PK [03:03] you do if the view sets up a decorstor [03:03] Since calling bugactivity.person will look up the PK in the Storm cache [03:04] i found i had to add that or else it didn't work but i can retry [03:04] Make sure you're actually evaluating the resultset you get from getPrecachedPersonsFromIds [03:05] ok, i'll try again and see what happens [03:07] Yeah, I suspect you just forgot the unobvious fact that you have to eg. call list() on it, or it doesn't do anything [03:07] Since I just tried that without the other changes, and the query count test still passes. [03:09] i knew i had to evaluate the resultset and thought i had. clearly not :-( [03:09] It's really easy to miss, sadly. [03:11] * wallyworld_ makes an optometrist appointment [03:11] Heh [03:12] changes pushed, thanks for noticing that [03:15] Great [03:17] wgrant: the "ALL" in the doc string is the sharing permission "ALL", not all pillars [03:17] I know [03:18] But if you skim the docstring, its emphasised that all the pillars are checked [03:18] so you still think it's confusing? [03:18] Indeed. Not sure if it's important, but it's certainly confusing. [03:18] hmm, ok. can't say i agree but will reword [03:26] wallyworld_, wgrant: https://code.launchpad.net/~stevenk/launchpad/urifield-with-leading-space/+merge/128855 [03:31] StevenK: Do you get to remove the manual stripping? [03:35] yay - [03:35] 91.85s OOPS-0ad9cf4a4b00e8030eb9033a83b705cf Person:branches.atom [03:36] :) [03:38] wgrant: Oh, from URIField? [03:38] Hmm, maybe [03:39] wgrant: Yeah, if I kill the normalize() strips test, the URIWidget test still passes [03:40] Yay [03:42] wgrant: http://pastebin.ubuntu.com/1270464/ [03:47] StevenK: Right, something like that [03:48] ... Why is RootObject:+openid-callback doing 875 SQL statements? [03:49] Probably name collisions [03:49] Do you have an OOPS ID? [03:49] OOPS-3bd6bd0eddf3594799308c1c32b6b4e6 [03:49] From the error report [03:49] Yes, but that's all the way over there [03:51] Yeah, name collisions [03:52] We have the branch scanner and celery competing again? OOPS-0d92c20fe1d6766c6637b1ee558aca28 [03:53] Right, that's an ongoing issue [03:53] there's a bug for it [03:53] Assuming that's the illegal job state transition thing [03:53] scan_branches.py sometimes picks up a job before celery can get to it [03:53] Then celery tries to start it, which dies [03:53] Why are we running both? :-( [03:54] NFI [03:54] celery was turned on by nobody at nobody's request, from what I can tell [03:54] As nobody was aware it was running except me [03:54] And I was only aware because I saw it in logs [03:55] wgrant: MP updated [03:56] wgrant: Yeah, there's two. One is caused by that and there is another that implicates the PCJ machinery [03:56] wallyworld beat me to it [04:18] wgrant: are you ok to +1 my mps? [04:20] wallyworld_: r=me, thanks for the fixes [04:20] np. thnk for finding them [06:00] StevenK: bzr lp-land doesn't grab any dependent branches like ec2 land does it? [06:01] if not i think bb will break [06:01] ec2 land doesn't either? [06:01] it does [06:01] wallyworld_: Your dependent branch was merged into the one you landed [06:01] ah right, good [06:01] ec2 land and bzr lp-land ignore prereqs [06:02] so what happens if you ec2 land a branch with a pre req? [06:02] i could have sworn it worked, even if not merged in? [06:02] maybe not [06:03] It doesn't know it exists [06:03] So if it's not merged then it doesn't exist [07:27] good morning [08:24] wgrant: Is bug 1062291 another dupe of bug 839141? [08:24] <_mup_> Bug #1062291: timeout on Bugtask:+Index in subscription lookups < https://launchpad.net/bugs/1062291 > [08:24] <_mup_> Bug #839141: Bug page times out getting subscriptions for logged in users on bugs with many duplicates < https://launchpad.net/bugs/839141 > [08:27] Hmmm, maybe not [08:28] StevenK: The fix for one may fix the other [08:28] And they are caused by the same thing [08:28] But they're different bugs [08:29] Yeah, I came to that conclusion too [08:29] Maybe we need to denorm subscriber ids into BTF [08:29] That idea might make you murder me [08:30] No real need for that [08:30] Even assuming 1000 dupes, the lookups shouldn't be that bad [08:31] If you do it all in one query [08:31] Right [10:51] allenap, jelmer, adeuring: You guys each have a fairly old branch on . Could you dispose of them one way or another? [10:51] Just trying to get the queue cleaned up [10:51] wgrant: marked mine as rejected [10:51] jelmer: Thanks [10:52] Mine probably wants a feature flag added, which I haven't had time to do [10:52] I left yours out because its situation was clear, yeah [10:53] wgrant: Done. [10:53] allenap: Thanks [12:26] adeuring: ping, do you know much about the garbo job ProductInformationTypeDefault ? [12:27] rick_h_: no that much. but what is the problem? [12:27] so my branch failed to pass tests because it's doing queries against information_type and I changed it to _information_type [12:27] so if I update the field the tests/query pass [12:28] but I'm nervous that this thing is ok to do and we don't need to update it in some way to make sure that the ensurePolicies bit gets taken care of [12:28] if this thing is going on in the background changing the informatino type, then again I guess it's just setting None to PUBLIC [12:28] so shold be safe [12:29] rick_h_: i had to add the leading '_' for other reasons. I am aware that of this prblem; once the gabro job is finished, we can omit the '_' again. [12:30] adeuring: ok, I think I should be fine with the change for now. Thanks [12:30] just got nervous there for a second [12:50] rick_h_: have ye broken bp search ? [12:50] rick_h_: https://bugs.launchpad.net/launchpad/+bug/1064996 [12:50] <_mup_> Bug #1064996: Blueprints search is totally broken < https://launchpad.net/bugs/1064996 > [12:50] rick_h, adeuring: the garbo job shold have finished yesterday. i suspect if you ask ops to check the database, all the information types will be set. [12:51] czajkowski: but but but... [12:51] wgrant: ah, ok good to know thanks. [12:51] the job can probably be deleted. [12:51] * czajkowski peers at rick_h_ [12:51] wgrant: cool, I just merged but will add a card to follow up on it thanks [12:51] great [12:51] czajkowski: so *I* didn't, looking to see if we have a bug on that already [12:52] stub: aha! can you please apply my index before librarian-gc spoils my plan? [12:52] ok [12:52] and we can kill the librarian-gc if it is ever a problem, it will catch up the next day [12:53] sure, but i prefer not to encourage evil workarounds like that [12:55] czajkowski: hmmm, so if you search for arm it works. arm-m doesn't, but it looks like we probably tokenize on the - so not sure it would match. [12:55] czajkowski: I'm not sure if this worked before so I can't say for sure, but it sure seems he just hit a bad search term [12:55] if you just search for arm and sort the blueprints the arm-m ones all jump to the top peachy [12:55] czajkowski: that shows a limit in postgresql's search capabilities. [12:56] adeuring: nods [12:56] czajkowski: so I'm going to note that and mark it as an invalid bug [12:56] rick_h_: aye I did try but getting conflicting results [12:56] czajkowski: searchinf for "arm m" should work [12:56] i'll add a comment [12:56] or let adeuring do it [12:56] yea, arm m works fine [12:56] thanks adeuring [12:56] wgrant: done [12:57] stub: thanks [13:00] sinuzi beat us all to the bug and marked as a duplicate [13:02] ah, even better [13:03] https://bugs.launchpad.net/launchpad/+bug/1025357 [13:03] <_mup_> Bug #1025357: BluePrint searchtext= not returning correct results < https://launchpad.net/bugs/1025357 > [13:04] it seems like a trivial fix would be to just do prefix or trigram matching on the name in addition to fti [13:04] you'd need to build the extra indexes though right? /me tries to remember when he setup pgsql fulltext [13:05] well, it would just be a normal string index next to the fti one [13:05] nvm, you mean just or'ing/distinct on matching the name itself [13:05] yea, gotcha [13:06] we use this pattern for eg. person searches, where we do fti on displayname, but also prefix on name and email [13:07] because fti works for text, and those names are barely text === _rvba is now known as rvba [14:43] adeuring: I made buildbot happy so my changes are landed [14:44] adeuring: might want to pull updated trunk before you go do ec2 land in case we have conflicts in my adding back in the _information_type and getter/setter that you also had in [14:44] rick_h_: thanks for the heads-up! [14:57] sinzui: wgrant said to talk to you rehttps://answers.launchpad.net/launchpad/+question/210838 possible subkey issue again? [15:05] I will look [15:09] czajkowski, he has only one key: http://keyserver.ubuntu.com:11371/pks/lookup?op=vindex&search=0xD543BD02B52EF7A5 [15:10] but he may want to be certain he is signing with the one key that Lp knows about [15:10] * sinzui looks for command [15:12] czajkowski, If he has more than one key in his keyring, he needs to sign using this command that selects the same key that Lp knows about: [15:12] gpg -u B52EF7A5 --clearsign [15:12] ahh ok [15:12] didnt know that [15:12] yes, we have no faq on that [15:13] considering how common this issue is, maybe we want an faq explain this [15:15] nods === abentley changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: abentley | Firefighting: - | Critical bugs: ~280 === matsubara is now known as matsubara-lunch [17:45] abentley: review for you if you get a sec please https://code.launchpad.net/~rharding/launchpad/bp_default_1062207/+merge/129001 === matsubara-lunch is now known as matsubara [18:01] abentley, do you have time to review https://code.launchpad.net/~sinzui/launchpad/moderated-messages-0/+merge/129013 [18:25] abentley: another review request: https://code.launchpad.net/~adeuring/launchpad/authentication-for-private-products/+merge/129014 [18:28] rick_h_: could you review my MP: https://code.launchpad.net/~adeuring/launchpad/authentication-for-private-products/+merge/129014 (abentley seems to be away/busy, and I'd like to land the branch soon...) [18:29] adeuring: looking [18:29] rick_h_: thanks! [18:34] adeuring: ok, so in looking at that. Can sinzui look that one over since you guys had a chat. I just feel that I don't know that well enough to help protect against another round of issues. [18:34] sinzui: ^^^? [18:34] and I'll try to pick up for sinzui's mp in return? [18:34] * sinzui looks [18:40] sinzui, rick_h_: Sorry, was lunching. [18:41] abentley: np on my end. looked over sinzui's [18:42] I am reading adeuring's branch. I will be done a few minutes. The branch looks good, I want to browse an instance as a few users to verify nothing odd is shown to non-registry users [18:44] rick_h_: So your review still needs doing. [18:44] abentley: yes please [18:45] rick_h_: Does your branch handle the case where PUBLIC is not a valid value? [18:46] abentley: yes, the second test covers that case. [18:46] because right after the default being set to PUBLIC, it gets updated based on allowed values [18:48] rick_h_: I see. [18:49] rick_h_: I think the right way to do this is to use lp.blueprints.model.specification.SPECIFICATION_POLICY_DEFAULT_TYPES [18:49] looking [18:50] rick_h_: You should do that via Product.getDefaultSpecificationInformationType [18:52] abentley: ah ok that makes sense. I didn't see that. [18:52] rick_h_: Yeah, I'm currently mucking around in that area, too. [18:53] rick_h_: getDefaultSpecificationInformationType appears to be provided by all the other pillars, too, so you should be good to go. [18:54] abentley: ah ok that's what I was going to say that it already hits that for product, but we were getting it None so it must have been a diff context [18:54] rick_h_: This is exactly at the seam between my work and deryck's. Clearly we left a gap. [18:54] right, makes sense once it comes up [18:54] abentley: so if you're in there did you want to apply the fix to your branch or am I clear enough not to break up your work? [18:55] rick_h_: You're clear. I'm dealing with all those dicts in a different context. [18:55] ok cool [18:55] rick_h_: Speaking of that, want to review my new branch? [18:55] https://code.launchpad.net/~abentley/launchpad/specification-policy/+merge/128998 [18:57] abentley: sure thing [18:57] rick_h_: thanks. [19:02] rick_h_: Hmm, so I think I retract my suggestion on your branch. Since the value will be set to something reasonable if it's a product, defaulting to PUBLIC if it's not is fine. [19:02] rick_h_: Because the context doesn't necessarily provide a pillar. [19:04] rick_h_: NewSpecificationFromNonTargetView and its descendants don't know what the pillar will be, so PUBLIC is the best default we have. [19:05] abentley: ah ok cool. Still good to know about the getDefault... [19:11] abentley: r=me with a small nitpick on updating the comment [19:13] rick_h_: Thanks. [19:35] adeuring, I found an issue in the branch, but I think we can tweak the security checker to keep the current behaviours. We can adjust the interfaces and checker in future branches maybe [19:36] sinzui: ok, what is the issue? [19:36] nm, found your comment === mordred is now known as mtaylor === mtaylor is now known as mordred [21:03] gary_poster, can I have your plus-one for these instructions to fix a production mailing list. https://pastebin.canonical.com/76259/ [21:03] * gary_poster looks [21:10] sinzui, you are the expert we have here, so I'll happily approve if you say thsi is right. That said, I have suggestions/thoughts. after first test query, I suggest you clarify what youe expect the output to be (I assume one row). You did not include instructions to tar or remove the mhonarc bits--that seems like it might be intentional? You aren't asking for any of those bits to be archived, actually; if t [21:10] hat's what you want cool, but that's not what the page says. Finally, is 2500 really our timeout these days for writes? I thought it would be shorter [21:10] 2.5secods is the timeout [21:10] for changes [21:10] cool [21:11] gary_poster, I revised to comment about the deletes: https://pastebin.canonical.com/76265/ [21:12] Lp and mailman are in disagreement. I think the first 'rm' will do nothing because I believe mailman did delete the list directory. [21:13] cool, sinzui, looks good. If you don't think we need the tars, great. +1. :-) I have to run [21:14] gary_poster. apparently we do not because there never were any messages. [21:14] heh [21:14] ok [22:09] wgrant: hey [22:09] wgrant: do you know if the new FDT stuff is all live ? [22:11] lifeless: Yes [22:12] wgrant: is there timing data to be foubd? [22:12] lifeless: There are still changes planned to speed up security changes, and to make appservers fall back more [22:12] wgrant: how did the appservers handle it ? [22:12] lifeless: Slave->master failover worked fine [22:12] in phase2? [22:12] Yes [22:12] Though it was very quick [22:12] kk [22:13] Let me find times [22:13] It was something odd like 4.2s [22:13] that sees slow [22:13] Yes [22:13] We need to start logging milliseconds [22:13] yes === Ursinha_ is now known as Ursinha