/srv/irclogs.ubuntu.com/2012/10/17/#launchpad-dev.txt

=== matsubara-dinner is now known as matsubara
=== Ursinha-afk is now known as Ursinha
wallyworldstub: hi, i'm getting this error from a query containing an IN expr with strings: "ProgrammingError: operator does not exist: text = bytea". i can copy and paste the sql from the traceback into a sql editor and it works fine. any clues?07:10
wallyworldthe sql statement contains a call to trim if it matters07:12
wallyworldSELECT trim(leading 'pending-' from CommercialSubscription.sales_system_id) FROM CommercialSubscription WHERE (trim(leading 'pending-' from CommercialSubscription.sales_system_id)) IN ('foo', 'bar')07:12
stubwallyworld: I think you are sending in a raw string instead of unicode, and it is being cast to a binary blob (BYTEA)07:12
wallyworldi'm using storm07:12
stubwallyworld: And your logging doesn't notice, because to it a string is a string07:12
wallyworldStore.of(self).using(CommercialSubscription).find(SQL(voucher_expr), SQL(voucher_expr).is_in(voucher_ids))07:13
wallyworldvoucher_ids is a list of strings07:13
wgrantRight, that's a unicode vs str issue07:13
stubAre you sure about that?07:13
wallyworldi tried casting to unicode07:13
wallyworldlet me try again07:13
stubok.07:13
wgrantsales_system_id needs a unicode07:13
stubsurprised storm didn't pick it up - it is normally very picky07:14
wallyworldi could have sworn i did that, i'll recheck07:14
wgrantstub: Not when you use SQL()07:14
stubOh right, foot gun07:14
wgrantYeah07:14
wgrantI'm not sure that using pending- as a prefix here is sensible07:14
wgrantRather than adding a new column, for example07:14
wallyworldwgrant: so using SQL() breaks it?07:15
wgrantwallyworld: No07:15
wgrantIt depends how you're building the string that you give to it07:15
wallyworldah, hang on, it worked that time07:15
stubSQL() is a way of bypassing in built Storm checks07:15
wallyworldi used SQL() because i wanted an expr with trim()07:15
wgrantHm07:16
stubSure, it has uses. Just have to be more careful since you are saying 'this is valid SQL, trust me'.07:16
wgrantIt's more likely that voucher_ids is a sequence of strs, actually07:16
wgrantAs the rest of the types should be inferred.07:16
wallyworldi must have messed up the unicode cast first time07:16
wgrantvoucher_ids needs to be a sequence of unicodes, since the DB column is Unicode.07:17
wallyworldi cast the voucher_ids to unicode07:17
wallyworldi must have had fat fingers the first time07:17
adeuringgood morning07:40
=== jtv1 is now known as jtv
=== almaisan-away is now known as al-maisan
=== al-maisan is now known as almaisan-away
rick_h_adeuring: ping, got a sec?12:22
adeuringrick_h_: sure12:22
rick_h_adeuring: I'm looking at https://bugs.launchpad.net/launchpad/+bug/106327212:22
_mup_Bug #1063272: Cannot view +related-projects because of private-project <403> <private-projects> <Launchpad itself:In Progress by rharding> < https://launchpad.net/bugs/1063272 >12:22
rick_h_and from the bug report, it appears the fix could just be a simple check for a PUBLIC information type12:23
rick_h_is there any reason to deal with things like checking your access via access policy and such to this you think?12:23
adeuringrick_h_: yes: i think the user himself should be able to see  a products he can access12:24
rick_h_I've not really used these pages before so my thought is that it's really only useful to other people and only showing PUBLIC types shold work12:24
rick_h_ok12:24
adeuringafter all, that's an easy way to navigate to the a product12:24
adeuringrick_h_: but for a quick fix, returning only public prodcts is foine, I think12:25
rick_h_ok, :P don't let me cheat12:25
rick_h_adeuring: well it's just done in raw sql atm with a series of unioned joins so will be a bit messier12:25
adeuringyeah, I can imagine ;)12:26
abentleywgrant: The new Person.specifications has landed.12:29
=== Ursinha is now known as Ursinha-afk
=== abentley changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: abentley | Firefighting: - | Critical bugs: ~260
abentleyabel: Do you have time for some reviews?  I'm the OCR, so I can't ask the OCR.14:42
sinzuijcsackett, I lowered the priority of the gmail attachment bug and remove you from it because gmail is forcing CRLF every 76 characters when it does it's base64 encoding. I don't think this is a critical issue.14:48
jcsackettsinzui: dig.14:49
adeuringabentley, deryck, rick_h_: my branch with the security adapter for milestones causes again test failures for a recently added  test:  test_listings_consider_spec_visibility . Could you give me a few hours to try to get my branch landed (I'd like to avoid "race conditions" like those we had for the adapter for products...)15:19
rick_h_adeuring: yep, thanks for the heads up15:19
deryckadeuring, I can wait for my stuff.  still doing calls, so not able to fix my own tests yet.15:20
abentleyadeuring: Okay.15:20
adeuringrick_h_, abentley, deryck: Thanks! The alternative is of course to merge this brnach: lp:~adeuring/launchpad/milestone-sec-adapter (though now I'm getting "connection closed" errors when i try to push the most recnet version...)15:21
cjwatsonabentley: would appreciate thoughts on lp:~cjwatson/launchpad/pcj-queue-ordering, particularly on whether you agree with my performance handwaving15:23
abentleycjwatson: Okay, give me a couple minutes.15:23
abentleycjwatson: Can you add "Not(Job.id.is_in(seen))" to the find, and so skip the loops?15:31
adeuringabentley, deryck, rick_h_: I started another EC2 test  a few minutes ago15:32
deryckadeuring, cool, thanks for the heads-up.15:32
cjwatsonabentley: Oh, is_in doesn't have to take a Select or similar?15:33
abentleycjwatson: No, it takes values.15:33
cjwatsonNeat.15:33
cjwatsonI'll give that a try.15:34
abentleycjwatson: I don't fundamentally see away around changing the number of DB queries to scaling O(n) with next().15:37
abentleys/away/a way15:37
cjwatsonMe neither.15:37
cjwatsonUnfortunately I don't have subsecond profiling data on how long those queries usually take at the moment.15:38
cjwatsonAt least it's in a script, so worst case it goes slow not times out ...15:38
cjwatsonabentley: Not(Job.id.is_in(seen)) causes one of the tests to enter an infinite loop in the kind of way I'd expect if that test was entirely missing.15:39
cjwatson(i.e. a test which doesn't actually process the jobs, just iterates over them all.)15:40
abentleycjwatson: Can I see the diff?15:41
cjwatsonSure.  http://paste.ubuntu.com/1285251/ - the failing test is test_iterReady_orders_by_copy_policy.15:41
cjwatsonOh, wait, PCJ.id != Job.id perhaps15:42
abentleycjwatson: Yes, but seen and the is_in are both in terms of Job.id.15:44
cjwatsonNo, seen is in terms of PCJ.id15:45
cjwatsonseen.add(job.id) -> seen.add(job.job_id) fixes it15:45
abentleycjwatson: great.15:46
cjwatsonPushed.15:46
=== Ursinha-afk is now known as Ursinha
abentleycjwatson: grr, diff still not updated.  I think this is reasonable.  You're aware you're making it possible for interactive jobs to starve mass jobs, right?15:55
cjwatsonYes.15:57
abentleycjwatson: The And() in the find statement is superfluous-- find accepts multiple clauses and ANDs them automatically.15:57
cjwatsonThe volume of interactive jobs is a couple of orders of magnitude below capacity, and hardly ever goes over maybe a couple of dozen at a time.15:58
cjwatsonThanks, removing the And.15:58
abentleycjwatson: I would probably leave out the else after "if job is None", since everything following will be the remainder of the loop.15:58
cjwatsonFair enough15:59
abentleycjwatson: As for LOC, are you saying this work is part of an arc that will ultimately remove code?15:59
cjwatsonYes.15:59
cjwatsonLet me double-check numbers quickly.15:59
abentleyCool.16:00
abentleycjwatson: r=me.16:04
cjwatsonThe arc is, er, not exactly sure at this point but I think a few hundred so far; this should be near the end of it.  I have 1465 lines queued for deletion.16:04
abentleycjwatson: if you did care about starving mass jobs, I'd suggest splitting interactive and mass into separate lists and yielding from each in turn in a ratio.  (Say 10 interactive for every 1 mass).16:07
cjwatsonRight, thanks.  My preliminary analysis a week or two back suggested it wouldn't be necessary; the queue is empty or very close most of the time, except when we're auto-syncing from Debian.16:08
cjwatson(Which will be in a week or two, so I want to make sure PS doesn't come and shout at me around then ...)16:08
abentley:-)16:08
cjwatsonIn fact we'll hopefully (!) be doubling our copy job load by staging everything in -proposed first, if we get the infrastructure for that working in time.16:09
cjwatsonIncidentally, recent LoC progress has been pretty fantastic.  http://people.canonical.com/~cjwatson/tmp/loc-cum.png16:18
czajkowskicjwatson: you're on the TB right?16:35
czajkowskicjwatson: a mail was sent to it earlier today, if it hasn't been moderated could you please moderate it . thanks16:36
cjwatsonczajkowski: Yes.  Done.16:38
czajkowskicjwatson: thank you16:38
sinzuijcsackett, I think I found the strip() that lost the last line when making the attachment16:51
sinzuijcsackett, I would like you thoughts about the comment I added to https://bugs.launchpad.net/launchpad/+bug/89822717:01
_mup_Bug #898227: Newlines are removed from the end of bug attachments submitted by email <Launchpad itself:Triaged> < https://launchpad.net/bugs/898227 >17:01
=== matsubara is now known as matsubara-lunch
abentleyderyck: I think that disabling privacy checks should be a separate feature flag, to avoid changing the meaning of existing tests, and to reduce risks on production (where we'd never turn it on).17:21
deryckabentley, I agree.17:27
=== deryck is now known as deryck[lunch]
abentleyrick_h_: Could you please review https://code.launchpad.net/~abentley/launchpad/flag-enables-privacy-checks/+merge/130185 ?18:05
rick_h_abentley: loading18:06
=== deryck[lunch] is now known as deryck
abentleyderyck: I'm the OCR.  Could you please review https://code.launchpad.net/~abentley/launchpad/product-specifications-tests/+merge/130000 https://code.launchpad.net/~abentley/launchpad/product-specifications-storm/+merge/130002 and https://code.launchpad.net/~abentley/launchpad/product-specifications-privacy/+merge/130003 ?18:17
deryckabentley, sure.  That is 3 MPs, correct?18:18
abentleyderyck: Yes, 1. add tests 2. stormify 3. add privacy.18:19
deryckabentley, ah, gotchas.  ok, jumping on them now.18:19
abentleyderyck: Thanks.18:19
derycknp!18:19
=== matsubara-lunch is now known as matsubara
rick_h_abentley: r=me18:20
abentleyrick_h_: Thanks!18:20
deryckabentley, so for the first branch…. should there be some tests removed too?  If this is copied, should the other place die?  Or these are slightly different?18:25
abentleyderyck: The behaviour is different.  You can list the specifications of an inactive product.  You just can't see them elsewhere (like off a Person).18:27
abentleyderyck: After Stormifying, you can't list the specifications of an inactive product.  I wasn't sure whether that would cause test failures, but apparently not.  I don't know what the behaviour should be.18:31
deryckabentley, I don't follow, sorry.  The tests are no good then, i.e. you don't get the failure you expect?  or you mean something else?18:34
abentleyderyck: The existing tests of Products don't care whether it's possible to list the specifications of a disabled product.  I don't know whether we care.18:36
deryckabentley, ah.18:37
abentleyderyck: If we do care, then we should add tests to ensure we get whatever behaviour we want.18:37
deryckabentley, right, I follow now.  Was just thinking myself if we care.18:37
deryckabentley, I don't think we care.  no need for new tests.18:40
abentleyderyck: Okay.18:40
jcsackettsinzui: what file is the block you posted from? i think you're right, but i don't recall seeing that chunk in the handler.18:47
jcsackett(it also doesn't -- to me, anyway -- explain why there was no reproducing it in a test).18:47
sinzuijcsackett, I am not convinced. It is in messages/model/messaging.py18:48
sinzuiI have a test with the real message from the librarian, but it does not fail18:48
deryckabentley, all these look fine.  good work.  (r=me)*318:49
abentleyderyck: Thanks.18:49
jcsackettsinzui: yeah--that's the problem. i couldn't seem to get any sort of similar failure.18:49
sinzuijcsackett, I found the real message that is stored in the librarian. I am feeing it into a test that created the inbox copy, then sends it through the bugs mail handler. the base64 message always has the proper new lines18:50
jcsackettoh, that's just maddening.18:50
jcsackettand yet; in a dire hope the bug was gone i emailed an attach to the bug from my gmail account, and proved it's still an issue.18:50
sinzuiI can see that block will do odd things, but I just don't see the real example step into the block18:50
sinzuiI saw18:51
jcsackettsinzui: do we know all the callsites? could it be mutated at some other point?18:52
sinzuijcsackett, I guess the real new news is that we can be sure the message was not corrupted when we took it from the inbox and put it into the librarian. I think the bug/message/bugcomment rules mess with it18:52
jcsackettsinzui: must be.18:53
sinzuijcsackett, message does mutate. It does forced decoding18:53
abentleyderyck: These three branches take us over the WIP limit.18:53
deryckabentley, ok, thanks for the heads up.  Most of this is my fault.18:54
abentleynp18:54
deryckabentley, if I don't get my stuff out of the way here shortly, I'll up the WIP limit.18:55
sinzuiThe real message https://pastebin.canonical.com/76759/18:56
sinzui^ jcsackett note that there are two encoding which are fine, but I think Message can do something wrong18:58
jcsackettsinzui: good to know that the raw message at least is stored properly.19:00
jcsackettsinzui: how did you check that, btw?19:00
sinzuiI looked at the message ids for the attachments to the bug in staging's db...19:00
sinzui...then guess that message.id - 1 would show me what incoming.py got19:01
sinzuiincoming creates a random file name and and sequenced id. I typed the url in to my browser to pull it out of the librarian19:02
jcsackettaaah. clever.19:06
sinzuijcsackett, also, before the suspected block, there is `content = part.get_payload(decode=True)`19:08
sinzuiThis is correct when I step though, but I am using that latest python 2.719:09
jcsackettthe suspected block is the bit you posted, yes?19:09
sinzuiyes19:09
jcsackettsinzui: are we on 2.6 on production?19:09
sinzuicorrect19:09
jcsacketthuh.19:09
deryckabentley, I have my fix for that DecoratedBranch stuff now.  So I can submit to ec2 now.19:12
sinzuijcsackett, \o/ http://bugs.python.org/issue714319:15
jcsackettsinzui: w00t!19:15
jcsackettsinzui: so, we can't reproduce it in tests because we're using a fixed version. so we need to upgrade to fix it in prod.19:16
sinzuijcsackett, which we plan to do19:16
jcsackettsinzui: so hurrah, this will be fixed then.19:17
sinzuiThe bug is not critical, but I will tag it as python-upgrade so that we know to close it when we complete the task19:18
jcsackettsinzui: cool.19:19
adeuringabentley, deryck, rick_h_: my branch landed19:56
deryckadeuring, awesome!19:57
abentleysinzui: I'm having trouble reproducing the milestones side of bug #106329120:29
_mup_Bug #1063291: Project groups are broken by private projects <milestones> <private-projects> <projectgroups> <Launchpad itself:In Progress by abentley> < https://launchpad.net/bugs/1063291 >20:29
sinzuihmm20:31
abentleysinzui: This passes: http://pastebin.ubuntu.com/1285813/20:31
sinzuiabentley, did I have urls in the original bug?20:31
abentleysinzui: No.20:32
* sinzui looks on qastaging again20:32
sinzuiabentley, still looking. anon users cannot see https://qastaging.launchpad.net/launchpad-project20:35
abentleysinzui: Yes, I've got a fix for the private product part of that.20:36
sinzuiI thought so20:36
abentleysinzui: There's proprietary blueprints on that-- that's probably the reason.20:37
sinzuiI am getting timeouts creating a milestone on a private project that matches a milestone on a public project, so I am being patient20:39
sinzuiabentley, non-priv user gets a 403 on qastaging for https://qastaging.launchpad.net/launchpad-project/+milestone/3.120:41
sinzuishe can see https://qastaging.launchpad.net/launchpad-project/+milestones, but not the milestone itself20:42
abentleysinzui: So I can rephrase as "If the project has a milestone with a bug or blueprint targeted, then I cannot see the  the groups's own milestone page"?20:43
sinzuiyes.20:44
abentleysinzui: Thanks.20:44
sinzuiabentley, I think your blueprint speculation is correct. The page loads for the no-priv user and it has just a private bug targeted https://qastaging.launchpad.net/launchpad-project/+milestone/3.220:48
sinzuiThe 3.1 test has a proprietary blueprint from a previous test20:49
abentleysinzui: Cool.20:51
=== Ursinha is now known as Ursinha-afk
sinzuiwgrant, https://bugs.launchpad.net/launchpad/+bug/105985322:30
_mup_Bug #1059853: BugTask:+editstatus timeout blocked on bug update setting heat <timeout> <Launchpad itself:Triaged> < https://launchpad.net/bugs/1059853 >22:30

Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!