/srv/irclogs.ubuntu.com/2012/10/10/#launchpad-dev.txt

wgrantStevenK: Right, that's the most likely cause400:23
StevenKAnd I've been trying to figure out how to defeat it without success00:24
* wallyworld_ has a few errands to do, bbiab00:29
wgrantStevenK: It may be some subtle difference between lazr.uri and urlparse, but it's not exactly obvious. Have you tried grepping for other occurrences?00:35
StevenKwgrant: On neem? find . | xargs grep -H "valid_vcs_details" gives no results00:36
wgrantStevenK: :(00:37
wgrantAnd I don't think our postgres logs go back far enough00:37
wgrantAlthough they might00:37
StevenKI can recall the original OOPS mentioning https://lomse.svn.sourceforge.net/svnroot/lomse/trunk. (yes, with the extra .), but qas doesn't succumb to handing over an OOPS.00:39
wgrantOH00:43
wgrantThat's easy, then00:43
wgrantYeah, there we are00:44
wgrant (Error ID: OOPS-45b1bf391bcdcaec312388d7a2907957)00:44
wgrantNow, can you work out how I did that? :P00:46
StevenKwgrant: No, actually.00:59
wgrantIt's not actually visible in the page00:59
StevenKBecause I did the same thing and it didn't OOPS00:59
wgrantBut it's in the source00:59
StevenKwgrant: The generated HTML or ProductSeries:+setbranch ?01:00
wgrantStevenK: The HTML of the OOPS page01:00
StevenKOH01:01
StevenKIt's a SPACE01:01
wgrantA leading space, yes.01:01
StevenKHmm, does that make it a lazr.uri bug?01:02
wgrantNo01:02
StevenKI thought zope would have stripped fields01:02
wgrantURIField strips it before giving to lazr.uri01:02
wgrantlib/lp/services/fields/__init__.py01:02
StevenKI see that01:02
wgrant_validate() calls lazr.uri.URI on a normalized version01:03
wgrantBut all it returns is the superclass' validated version of the input01:03
wgrantAnd it's just a TextLine, not a StrippedTextLine01:03
StevenKHah01:03
StevenKIt should return uri, not value01:03
wgrantNo01:04
wgrantWe probably don't want to store the normalised value.01:04
StevenKwgrant: Then I'm confused01:05
wgrantI would suggest that it be turned into a StrippedTextLine, normalize be taught to stop stripping, and _validate to do the super() call first, before passing the validated value into lazr.uri01:05
=== jamesh_ is now known as jamesh
StevenKwgrant: http://pastebin.ubuntu.com/1270322/01:10
wgrantStevenK: Something like that, yeah01:11
StevenKwgrant: Did you see the copyright fix? :-)01:12
StevenKwgrant: And how can I test that fix?01:14
wgrantStevenK: There should be existing tests for URIField01:15
wgrantI'd look at them for inspiration01:15
StevenKwgrant: There's already a test that checks that URIField.normalise() returns a stripped version01:16
wgrantMaybe it actually does intend to return the normalised version01:17
wgrantBut fails01:17
StevenKwgrant: But it does, the _validate() actually uses it01:18
wgrantStevenK: Sure, it uses it for validation.01:18
wgrantStevenK: But then it returns the superclass' validated version of the *original* value, not the normalise one.01:19
StevenKwgrant: Hah, uri-field.txt fails when calling normalize on '  http://www.ubuntu.com/   '01:21
StevenKAw, my fix worketh not.01:59
wgrant:(02:00
StevenKwgrant: You said you don't think we want to store the normalized value? We do already?02:03
StevenKOr is .set() not used for storing?02:03
wgrantset()'s probably used for setting the value in the *field*, usually from the one in the model.02:04
StevenKGiven my changes have had no effect, and there is already a test for URIField stripping input, I'm inclined to think the bug is at another level02:05
wgrantPerhaps the field infrastructure is slightly different from the rest of the form stuff02:07
wgrantI'd trace through and see where it might be going wrong02:08
wgrantSince we can reproduce it now.02:08
mwhudsonah02:10
mwhudsonso guys you should learn something bad about code imports02:10
mwhudsoni am sure there is a bug for this02:12
mwhudsonbut code import editing does not use the usual methods of updating the object02:12
mwhudsoni.e. LaunchpadFormView.updateFromData or whatever it's called02:12
mwhudsonso the field validators/cleaners do not get run02:12
wgrantYeah, luckily we're not quite near that piece of evil atm02:13
mwhudsonok02:13
wgrantThis is ProductSeries:+setbranch, which uses normalish form stuff and only calls newCodeImport02:13
wgrantBut thanks for the warning02:13
wgrantjelmer: Is https://code.launchpad.net/~jelmer/launchpad/retry-update-node-tags meant to be against MAAS?02:23
=== ccxCZ_ is now known as ccxCZ
StevenKwgrant: So does URIField even factor into it?02:43
StevenKHow does.02:43
StevenKwgrant: ProductSeries:+setbranch just calls url = data.get('repo_url') and then tosses it straight into getUtility(ICodeImportSet).new()02:44
wgrantStevenK: But data.get('repo_url') should be the post-validation value, shouldn't it?02:45
wgrantrequest.form will be pre-validation, but data should be post-validation02:46
wgrantAIUI02:46
StevenKwgrant: Ah. So tracing in update_action is after the horse has bolted02:47
wgrantStevenK: Yes02:48
wgrantStevenK: The form infrastructure will usually validate request.form to produce data, then call the action based on that02:48
wgrantI believe02:48
wgrantBut don't quote me on that, unless I'm right02:49
* StevenK shifts the pdb call to URIField._validate02:49
StevenKSo it validates correctly,02:53
StevenKAnd the superclass validate is just checking if the string not empty02:53
StevenKSo then zope.app.form.browser.widget calls getInputValue and gets the non-stripped value02:54
StevenKSo maybe it's a bug in URIWidget02:54
wgrantSo perhaps field validation and setting stuff are separate, unlike forms.02:54
StevenKOH02:59
StevenKDear URIWidget, 2004 called, and wants you to start using super.02:59
wgrantHah02:59
StevenKHah, I think I can see another bug03:00
StevenKYeah03:00
StevenKIf you pass it a list, it will raise UnexpectedFormData.03:01
StevenKIf you pass it a tuple, it won't.03:01
wgrantwallyworld_: Hmmm03:02
wgrantwallyworld_: Why do you not just call getPrecachedPersonsFromIds?03:03
wallyworld_i do03:03
wgrantwallyworld_: You don't usually need to do messy caching stuff like that if you're using the PK03:03
wallyworld_you do if the view sets up a decorstor03:03
wgrantSince calling bugactivity.person will look up the PK in the Storm cache03:03
wallyworld_i found i had to add that or else it didn't work but i can retry03:04
wgrantMake sure you're actually evaluating the resultset you get from getPrecachedPersonsFromIds03:04
wallyworld_ok, i'll try again and see what happens03:05
wgrantYeah, I suspect you just forgot the unobvious fact that you have to eg. call list() on it, or it doesn't do anything03:07
wgrantSince I just tried that without the other changes, and the query count test still passes.03:07
wallyworld_i knew i had to evaluate the resultset and thought i had. clearly not :-(03:09
wgrantIt's really easy to miss, sadly.03:09
* wallyworld_ makes an optometrist appointment03:11
wgrantHeh03:11
wallyworld_changes pushed, thanks for noticing that03:12
wgrantGreat03:15
wallyworld_wgrant: the "ALL" in the doc string is the sharing permission "ALL", not all pillars03:17
wgrantI know03:17
wgrantBut if you skim the docstring, its emphasised that all the pillars are checked03:18
wallyworld_so you still think it's confusing?03:18
wgrantIndeed. Not sure if it's important, but it's certainly confusing.03:18
wallyworld_hmm, ok. can't say i agree but will reword03:18
StevenKwallyworld_, wgrant: https://code.launchpad.net/~stevenk/launchpad/urifield-with-leading-space/+merge/12885503:26
wgrantStevenK: Do you get to remove the manual stripping?03:31
lifelessyay -03:35
lifeless    91.85s  OOPS-0ad9cf4a4b00e8030eb9033a83b705cf  Person:branches.atom03:35
wgrant:)03:36
StevenKwgrant: Oh, from URIField?03:38
StevenKHmm, maybe03:38
StevenKwgrant: Yeah, if I kill the normalize() strips test, the URIWidget test still passes03:39
wgrantYay03:40
StevenKwgrant: http://pastebin.ubuntu.com/1270464/03:42
wgrantStevenK: Right, something like that03:47
StevenK... Why is RootObject:+openid-callback doing 875 SQL statements?03:48
wgrantProbably name collisions03:49
wgrantDo you have an OOPS ID?03:49
StevenKOOPS-3bd6bd0eddf3594799308c1c32b6b4e603:49
StevenKFrom the error report03:49
wgrantYes, but that's all the way over there03:49
wgrantYeah, name collisions03:51
StevenKWe have the branch scanner and celery competing again? OOPS-0d92c20fe1d6766c6637b1ee558aca2803:52
wgrantRight, that's an ongoing issue03:53
wgrantthere's a bug for it03:53
wgrantAssuming that's the illegal job state transition thing03:53
wgrantscan_branches.py sometimes picks up a job before celery can get to it03:53
wgrantThen celery tries to start it, which dies03:53
StevenKWhy are we running both? :-(03:53
wgrantNFI03:54
wgrantcelery was turned on by nobody at nobody's request, from what I can tell03:54
wgrantAs nobody was aware it was running except me03:54
wgrantAnd I was only aware because I saw it in logs03:54
StevenKwgrant: MP updated03:55
StevenKwgrant: Yeah, there's two. One is caused by that and there is another that implicates the PCJ machinery03:56
wgrantwallyworld beat me to it03:56
wallyworld_wgrant: are you ok to +1 my mps?04:18
wgrantwallyworld_: r=me, thanks for the fixes04:20
wallyworld_np. thnk for finding them04:20
wallyworld_StevenK: bzr lp-land doesn't grab any dependent branches like ec2 land does it?06:00
wallyworld_if not i think bb will break06:01
StevenKec2 land doesn't either?06:01
wallyworld_it does06:01
wgrantwallyworld_: Your dependent branch was merged into the one you landed06:01
wallyworld_ah right, good06:01
wgrantec2 land and bzr lp-land ignore prereqs06:01
wallyworld_so what happens if you ec2 land a branch with a pre req?06:02
wallyworld_i could have sworn it worked, even if not merged in?06:02
wallyworld_maybe not06:02
wgrantIt doesn't know it exists06:03
wgrantSo if it's not merged then it doesn't exist06:03
adeuringgood morning07:27
StevenKwgrant: Is bug 1062291 another dupe of bug 839141?08:24
_mup_Bug #1062291: timeout on Bugtask:+Index in subscription lookups <bugs> <subscriptions> <timeout> <Launchpad itself:Triaged> < https://launchpad.net/bugs/1062291 >08:24
_mup_Bug #839141: Bug page times out getting subscriptions for logged in users on bugs with many duplicates <critical-analysis> <timeout> <Launchpad itself:Triaged by stevenk> < https://launchpad.net/bugs/839141 >08:24
StevenKHmmm, maybe not08:27
wgrantStevenK: The fix for one may fix the other08:28
wgrantAnd they are caused by the same thing08:28
wgrantBut they're different bugs08:28
StevenKYeah, I came to that conclusion too08:29
StevenKMaybe we need to denorm subscriber ids into BTF08:29
StevenKThat idea might make you murder me08:29
wgrantNo real need for that08:30
wgrantEven assuming 1000 dupes, the lookups shouldn't be that bad08:30
wgrantIf you do it all in one query08:31
StevenKRight08:31
wgrantallenap, jelmer, adeuring: You guys each have a fairly old branch on <https://code.launchpad.net/launchpad/+activereviews>. Could you dispose of them one way or another?10:51
wgrantJust trying to get the queue cleaned up10:51
jelmerwgrant: marked mine as rejected10:51
wgrantjelmer: Thanks10:51
cjwatsonMine probably wants a feature flag added, which I haven't had time to do10:52
wgrantI left yours out because its situation was clear, yeah10:52
allenapwgrant: Done.10:53
wgrantallenap: Thanks10:53
rick_h_adeuring: ping, do you know much about the garbo job ProductInformationTypeDefault ?12:26
adeuringrick_h_: no that much. but what is the problem?12:27
rick_h_so my branch failed to pass tests because it's doing queries against information_type and I changed it to _information_type12:27
rick_h_so if I update the field the tests/query pass12:27
rick_h_but I'm nervous that this thing is ok to do and we don't need to update it in some way to make sure that the ensurePolicies bit gets taken care of12:28
rick_h_if this thing is going on in the background changing the informatino type, then again I guess it's just setting None to PUBLIC12:28
rick_h_so shold be safe12:28
adeuringrick_h_: i had to add the leading '_' for other reasons. I am aware that of this prblem; once the gabro job is finished, we can omit the '_' again.12:29
rick_h_adeuring: ok, I think I should be fine with the change for now. Thanks12:30
rick_h_just got nervous there for a second12:30
czajkowskirick_h_: have ye broken bp search ?12:50
czajkowskirick_h_: https://bugs.launchpad.net/launchpad/+bug/106499612:50
_mup_Bug #1064996: Blueprints search is totally broken <Launchpad itself:New> < https://launchpad.net/bugs/1064996 >12:50
wgrantrick_h, adeuring: the garbo job shold have finished yesterday. i suspect if you ask ops to check the database, all the information types will be set.12:50
rick_h_czajkowski: but but but...12:51
rick_h_wgrant: ah, ok good to know thanks.12:51
wgrantthe job can probably be deleted.12:51
* czajkowski peers at rick_h_ 12:51
rick_h_wgrant: cool, I just merged but will add a card to follow up on it thanks12:51
wgrantgreat12:51
rick_h_czajkowski: so *I* didn't, looking to see if we have a bug on that already12:51
wgrantstub: aha! can you please apply my index before librarian-gc spoils my plan?12:52
stubok12:52
stuband we can kill the librarian-gc if it is ever a problem, it will catch up the next day12:52
wgrantsure, but i prefer not to encourage evil workarounds like that12:53
rick_h_czajkowski: hmmm, so if you search for arm it works. arm-m doesn't, but it looks like we probably tokenize on the - so not sure it would match.12:55
rick_h_czajkowski: I'm not sure if this worked before so I can't say for sure, but it sure seems he just hit a bad search term12:55
rick_h_if you just search for arm and sort the blueprints the arm-m ones all jump to the top peachy12:55
adeuringczajkowski: that shows a limit in postgresql's search capabilities.12:55
czajkowskiadeuring: nods12:56
rick_h_czajkowski: so I'm going to note that and mark it as an invalid bug12:56
czajkowskirick_h_: aye I did try but getting conflicting results12:56
adeuringczajkowski: searchinf for "arm m" should work12:56
adeuringi'll add a comment12:56
rick_h_or let adeuring do it12:56
rick_h_yea, arm m works fine12:56
rick_h_thanks adeuring12:56
stubwgrant: done12:56
wgrantstub: thanks12:57
czajkowskisinuzi beat us all to the bug and marked as a duplicate13:00
rick_h_ah, even better13:02
czajkowskihttps://bugs.launchpad.net/launchpad/+bug/102535713:03
_mup_Bug #1025357: BluePrint searchtext= not returning correct results <lp-blueprints> <search> <specifications> <Launchpad itself:Triaged> < https://launchpad.net/bugs/1025357 >13:03
wgrantit seems like a trivial fix would be to just do prefix or trigram matching on the name in addition to fti13:04
rick_h_you'd need to build the extra indexes though right? /me tries to remember when he setup pgsql fulltext13:04
wgrantwell, it would just be a normal string index next to the fti one13:05
rick_h_nvm, you mean just or'ing/distinct on matching the name itself13:05
rick_h_yea, gotcha13:05
wgrantwe use this pattern for eg. person searches, where we do fti on displayname, but also prefix on name and email13:06
wgrantbecause fti works for text, and those names are barely text13:07
=== _rvba is now known as rvba
rick_h_adeuring: I made buildbot happy so my changes are landed14:43
rick_h_adeuring: might want to pull updated trunk before you go do ec2 land in case we have conflicts in my adding back in the _information_type and getter/setter that you also had in14:44
adeuringrick_h_: thanks for the heads-up!14:44
czajkowskisinzui: wgrant said to talk to you rehttps://answers.launchpad.net/launchpad/+question/210838  possible subkey issue again?14:57
sinzuiI will look15:05
sinzuiczajkowski, he has only one key: http://keyserver.ubuntu.com:11371/pks/lookup?op=vindex&search=0xD543BD02B52EF7A515:09
sinzuibut he may want to be certain he is signing with the one key that Lp knows about15:10
* sinzui looks for command15:10
sinzuiczajkowski, If he has more than one key in his keyring, he needs to sign using this command that selects the same key that Lp knows about:15:12
sinzuigpg -u B52EF7A5 --clearsign15:12
czajkowskiahh ok15:12
czajkowskididnt know that15:12
sinzuiyes, we have no faq on that15:12
sinzuiconsidering how common this issue is, maybe we want an faq explain this15:13
czajkowskinods15:15
=== abentley changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: abentley | Firefighting: - | Critical bugs: ~280
=== matsubara is now known as matsubara-lunch
rick_h_abentley: review for you if you get a sec please https://code.launchpad.net/~rharding/launchpad/bp_default_1062207/+merge/12900117:45
=== matsubara-lunch is now known as matsubara
sinzuiabentley, do you have time to review https://code.launchpad.net/~sinzui/launchpad/moderated-messages-0/+merge/12901318:01
adeuringabentley: another review request: https://code.launchpad.net/~adeuring/launchpad/authentication-for-private-products/+merge/12901418:25
adeuringrick_h_: could you review my MP: https://code.launchpad.net/~adeuring/launchpad/authentication-for-private-products/+merge/129014 (abentley seems to be away/busy, and I'd like to land the branch soon...)18:28
rick_h_adeuring: looking18:29
adeuringrick_h_: thanks!18:29
rick_h_adeuring: ok, so in looking at that. Can sinzui look that one over since you guys had a chat. I just feel that I don't know that well enough to help protect against another round of issues.18:34
adeuringsinzui: ^^^?18:34
rick_h_and I'll try to pick up for sinzui's mp in return?18:34
* sinzui looks18:34
abentleysinzui, rick_h_: Sorry, was lunching.18:40
rick_h_abentley: np on my end. looked over sinzui's18:41
sinzuiI am reading adeuring's branch. I will be done a few minutes. The branch looks good, I want to browse an instance as a few users to verify nothing odd is shown to non-registry users18:42
abentleyrick_h_: So your review still needs doing.18:44
rick_h_abentley: yes please18:44
abentleyrick_h_: Does your branch handle the case where PUBLIC is not a valid value?18:45
rick_h_abentley: yes, the second test covers that case.18:46
rick_h_because right after the default being set to PUBLIC, it gets updated based on allowed values18:46
abentleyrick_h_: I see.18:48
abentleyrick_h_: I think the right way to do this is to use lp.blueprints.model.specification.SPECIFICATION_POLICY_DEFAULT_TYPES18:49
rick_h_looking18:49
abentleyrick_h_: You should do that via Product.getDefaultSpecificationInformationType18:50
rick_h_abentley: ah ok that makes sense. I didn't see that.18:52
abentleyrick_h_: Yeah, I'm currently mucking around in that area, too.18:52
abentleyrick_h_: getDefaultSpecificationInformationType appears to be provided by all the other pillars, too, so you should be good to go.18:53
rick_h_abentley: ah ok that's what I was going to say that it already hits that for product, but we were getting it None so it must have been a diff context18:54
abentleyrick_h_: This is exactly at the seam between my work and deryck's.  Clearly we left a gap.18:54
rick_h_right, makes sense once it comes up18:54
rick_h_abentley: so if you're in there did you want to apply the fix to your branch or am I clear enough not to break up your work?18:54
abentleyrick_h_: You're clear.  I'm dealing with all those dicts in a different context.18:55
rick_h_ok cool18:55
abentleyrick_h_: Speaking of that, want to review my new branch?18:55
abentleyhttps://code.launchpad.net/~abentley/launchpad/specification-policy/+merge/12899818:55
rick_h_abentley: sure thing18:57
abentleyrick_h_: thanks.18:57
abentleyrick_h_: Hmm, so I think I retract my suggestion on your branch.  Since the value will be set to something reasonable if it's a product, defaulting to PUBLIC if it's not is fine.19:02
abentleyrick_h_: Because the context doesn't necessarily provide a pillar.19:02
abentleyrick_h_: NewSpecificationFromNonTargetView and its descendants don't know what the pillar will be, so PUBLIC is the best default we have.19:04
rick_h_abentley: ah ok cool. Still good to know about the getDefault...19:05
rick_h_abentley: r=me with a small nitpick on updating the comment19:11
abentleyrick_h_: Thanks.19:13
sinzuiadeuring, I found an issue in the branch, but I think we can tweak the security checker to keep the current behaviours. We can adjust the interfaces and checker in future branches maybe19:35
adeuringsinzui: ok, what is the issue?19:36
adeuringnm, found your comment19:36
=== mordred is now known as mtaylor
=== mtaylor is now known as mordred
sinzuigary_poster, can I have your plus-one for these instructions to fix a production mailing list. https://pastebin.canonical.com/76259/21:03
* gary_poster looks21:03
gary_postersinzui, you are the expert we have here, so I'll happily approve if you say thsi is right.  That said, I have suggestions/thoughts.  after first test query, I suggest you clarify what youe expect the output to be (I assume one row).  You did not include instructions to tar or remove the mhonarc bits--that seems like it might be intentional?  You aren't asking for any of those bits to be archived, actually; if t21:10
gary_posterhat's what you want cool, but that's not what the page says.  Finally, is 2500 really our timeout these days for writes?  I thought it would be shorter21:10
sinzui2.5secods is the timeout21:10
sinzuifor changes21:10
gary_postercool21:10
sinzuigary_poster, I revised to comment about the deletes: https://pastebin.canonical.com/76265/21:11
sinzuiLp and mailman are in disagreement. I think the first 'rm' will do nothing because I believe mailman did delete the list directory.21:12
gary_postercool, sinzui, looks good. If you don't think we need the tars, great. +1.   :-) I have to run21:13
sinzuigary_poster. apparently we do not because there never were any messages.21:14
gary_posterheh21:14
gary_posterok21:14
lifelesswgrant: hey22:09
lifelesswgrant: do you know if the new FDT stuff is all live ?22:09
wgrantlifeless: Yes22:11
lifelesswgrant: is there timing data to be foubd?22:12
wgrantlifeless: There are still changes planned to speed up security changes, and to make appservers fall back more22:12
lifelesswgrant: how did the appservers handle it ?22:12
wgrantlifeless: Slave->master failover worked fine22:12
lifelessin phase2?22:12
wgrantYes22:12
wgrantThough it was very quick22:12
lifelesskk22:12
wgrantLet me find times22:13
wgrantIt was something odd like 4.2s22:13
lifelessthat sees slow22:13
wgrantYes22:13
wgrantWe need to start logging milliseconds22:13
lifelessyes22:13
=== Ursinha_ is now known as Ursinha

Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!