/srv/irclogs.ubuntu.com/2011/08/08/#launchpad-dev.txt

wallyworld_wgrant: some already filed a bug http://youtrack.jetbrains.net/issue/PY-274100:14
wallyworld_someone00:14
wallyworld_actually, it seems like it pertains to code completion but it will be related00:20
lifelesswallyworld_: this is the whole 'our packaging can only support one version at a time' issue.00:31
wallyworld_lifeless: thanks. btw, a gentle reminder about https://code.launchpad.net/~xaav/loggerhead/export-tarball/+merge/66408 :-)00:33
lifelessthanks ;P00:33
wallyworld_an adaptor question - i think we should be able to specify a class as being an adaptor for more than one interface, wither via @adaptor(foo, bar) or in zcml for="foo bar"? it appears not since it doesn't work. and so i have to have one zcml adaptor element per interface, which the same adapter class/factory specfified. is there something i'm missing?00:37
wgrantwallyworld_: I'm not sure... but it's probably not an adapter for more than one interface.00:39
wgrantThose interfaces probably share a subinterface.00:40
wallyworld_wgrant: in this case it is the same adapter implementation for more than one interface that i would like to specify. they do share a subinterface (in fact > 1) and the adaptor makes use of those >1 subinterfaces. you'd think it would be possible00:41
wallyworld_in this case, i want to say "this specified class adapts both IDistribution and IProduct"00:42
wallyworld_i can do it but need to have multiple adapter declarations with just a different for="xxx" which seems dumb00:43
wallyworld_and i can't user @adapter() so am forced to use zcml00:43
wgrantStevenK: QA!00:46
StevenKwgrant: Yeah yeah :-P00:47
StevenKlifeless: Can haz bazaar-experts team deleted?01:01
lifelessStevenK: we've deployed everywhere since?01:01
lifelessStevenK: including crannies like soyuz ?01:01
StevenKI know cocobanana has been deployed to since, and one of germanium's trees is in NDT. Not sure about the other tree01:02
wgrantpoppy doesn't use bazaar-experts, so meh.01:10
StevenKI'd be a little concerned if it did ...01:11
nigelbGood morning01:48
wgrantlifeless: Do you think the timeouts are regressions, or bloat?02:05
wgrantWe went from 300 to 1100 in a week.02:05
StevenKWheee02:06
wgrantI suspect the subscription changes.02:06
wgrantBut that doesn't explain Person:EntryResource:searchTasks02:06
wgrantI don't think.02:06
lifelesswgrant: the soft counts last week would have been large if it was creeping bloatism02:08
StevenKlifeless: So, can haz bazaar-experts deleted, or do you want to wait until every single tree doesn't mention the celebrity?02:10
lifelessI'd like to wait02:10
lifelesseasy to do, reduces risk, doesn't slow anything down.02:10
StevenKlifeless: Until when? :-)02:10
StevenKIf you say the heat death of the universe, I'm booking a flight to NZ :-P02:11
lifelessno live trees with the celeb in them02:11
StevenKThe only services I can think of are germanium's poppy and probably forster02:12
StevenKOh, and the librarian?02:12
lifelesslikely, yes.02:17
lifelesswallyworld_: hi02:35
wallyworld_hello02:35
lifelesswallyworld_: your affiliation work; it looks like its going to do lots of eager loading.02:35
wallyworld_i put in tests for the query count (for bugtask) - it's only 4 quesies soince stuff is already loaded02:36
lifelessper person02:36
wallyworld_yes, that is true02:37
lifelesss/eager loading/lazy loading/02:37
wallyworld_so i probably need to fix that02:37
lifelesswallyworld_: anytime you end up writing adapters, and object orientated code that does data access, its probably going to involve late evaluation/lazy loading problems.02:38
lifelesswallyworld_: the -only- case I know where such code is fine is when it happens /once/ on a page/http request as part of establishing the context (rather than potentially in inner loops etc)02:38
wallyworld_i'm starting to see that. i still haven't got it hard wired in my brain that zope object navigation = queries02:39
lifelessalso inTeam() for instance.02:39
lifelessuserA.inTeam(teamT)02:39
wallyworld_i'm used to popo/pojo type domain models02:39
lifelessuserB.inTeam(teamT)02:39
lifelesswill do 2 queries, even if userA, userB and teamT are all already loaded.02:40
wallyworld_to access the join table?02:40
lifelessyup02:40
wallyworld_ah, well it's behind a feature flag at least. i'll do a branch to optimise. that's for pointing it out02:40
wallyworld_except i think we turn on the feature flag recently, must check02:42
wgrantpicker_enhancements is on.02:42
wallyworld_lifeless: the issue with the implementation is that the call to the affiliation adapter is done inside getPickerEntry(), which itself is an adapter method which is done per picker row02:47
wallyworld_so there's potentially a bit of work to do to change all that02:48
wallyworld_still, must be done02:49
wallyworld_not quite so straightforward, since each item in a batch might not use the same picker entry adaptor02:56
lifelessright02:56
lifelessIt's common to run into friction like this02:57
wallyworld_yeah, so now picker entry adapters and HugeVocabularyJSONView are getting a bit of rework03:05
wallyworld_lifeless: the current affiliation adaptor implmentation may not be so bad. the inTeam checks will be cached and the 4 queries i mentioned before are not per person i don't believe - eg once the pillar.bug_supervisor  is fetched, it is not required to hit the db again for each person to check03:49
wallyworld_actually, the inteam caching assertion is bogus03:51
lifelessthe current caching implementation is (team, member) specific03:52
lifelesswhat you need is an interface for populating many such cache tuples at once03:52
wallyworld_there's lots of other places where we do inTeam(xxx) or inTeam(yyy) eg canBeUnsubscribedBy()03:53
lifelessand to call that with all the teams, for all the rows, and all the row person-or-team objects at once.03:53
wallyworld_that is a solution but it severely breaks encapulation of the adapter implementation03:54
lifelessyes03:55
lifelesssuch encapasulation is harmful in our environment TBH03:55
wallyworld_it may be easier to load the team members inside the adapter implementatiin and cache those03:55
wallyworld_encapsulation is preserved and cachiing can still occur03:56
lifelessis the adapter per row ?03:56
wallyworld_the adapter is per row but the teams are static for a given adaper instance03:56
lifelessto solve the problem you need to have set based interfaces.03:57
lifelessanything that is object based or row based will have late evaluation symptoms03:57
wallyworld_or load the team members the adapter knows it needs up front03:57
lifelessno03:57
lifelessthat won't work because the adapter is per row03:58
wallyworld_it will work - each row is checking for emebrship in  fixed set of teams, known in advance03:58
lifelessuhm03:58
wallyworld_so if those team members are loaded and cached, no extra db access is needed03:58
wgrantBug #82250003:59
_mup_Bug #822500: select <Launchpad itself:New> < https://launchpad.net/bugs/822500 >03:59
lifelesswallyworld_: This doesn't fit with my understanding of the code.03:59
StevenKNice title03:59
lifelesswallyworld_: calling personA.inTeam(teamT) will not seed the cache for calling personB.inTeam(teamT)04:00
StevenKIn fact, I found something odd last night -- https://code.launchpad.net/~aaacavazos/+recipe/launchpad-daily-104:00
wallyworld_lifeless: yes, that's why i am saying we need to load and cache these relationships04:00
wallyworld_do it once up front for the known, fixed teams04:01
lifelesswallyworld_: which requires knowing all the personX's and all the teamY's, doesn't it ?04:01
StevenKAnd there are seven recipes for stable ...04:01
wallyworld_lifeless: a person (row) is affiliated with the context (eg bug task) if they are a member of (eg) the security contacts team or the bug supervisors team04:01
wallyworld_so we can load and cache the team membership of those 2 teams04:02
wallyworld_then as each row is checked, no extra db acces sis needed04:02
wallyworld_i guess that doesn't account for indirect membership04:02
wallyworld_but it could04:03
lifelesswallyworld_: this seems both more work, and more code, than accumulating the set of (person, team) pairs we need an doing one call to check them all.04:03
wallyworld_but it preserves encapulation and the inheritance model used but the adapter implenmentation04:04
lifelessby more work I mean more runtime work.04:04
lifelessYou have to be sure what you're encapsulating; if you're encapsulting the wrong things its a pessimisation.04:05
wallyworld_hmm. not sure. maybe.04:05
wallyworld_what's being encapulated is the method to find out a person's affiliation with a given context04:06
lifelessyes, (I've read the patches)04:06
lifelessthis is *exactly* the sort of thing I've had to rip out to fix performance issues up and down the LP stack. bugs. soyuz. questions. blueprints.04:06
wallyworld_and as dicussed, it hooks into the existing workflow of one call per picker row04:06
lifelessproductseries has its own driver04:07
lifelessa product with 15 series will have 15 series teams to check.04:07
wallyworld_i don't think that sort of check is done currently04:07
lifeless(if you honour them all)04:07
wallyworld_if the context is a product, then series doesn't come into it04:08
lifelessanyhow, I can see this isn't making sense to you04:08
lifelessso we can drop it and be evidence based.04:08
wallyworld_it is, i'm just not quite agreeing with the suggested approach (yet)04:08
lifelessAll I can say, is that everything I've seen over the last year predisposes me to consider *any* per-single-object interface flawed by design.04:09
wallyworld_sure, but there's a balance between that and adhering to good oo design principals04:09
lifelessnot for code involved in batches/iteration/queries04:10
wallyworld_yes, i can see the merit in that statement04:11
lifeless(oh and for loops, which is implicit in iteration)04:11
lifelessOO design hasn't scaled to non-main-memory datasets IMO04:11
wallyworld_there's two ways of solving it - use set based interfaces or provide a caching implementation that prevents single calls from hitting the db04:12
lifelessI agree there are two approaches, but the latter isn't one of the two - it doesn't work.04:12
wallyworld_ie for a given use case, load the domain model known to be required to satisfy the request04:13
lifeless(the second approach I'd layout is the object query language approach)04:13
wallyworld_i've used that in combination with my second approach above04:14
lifelessthere are two disjoint  problems with caching implementations04:14
lifelessif you avoid one you get the other04:14
lifelessthe first problem is bog standard late evaluation04:15
lifelesseven if you cache each single call, when you iterate you make a new call for each person, and thats a cache miss04:15
wallyworld_but if the domain model that is being reference for each call is the same...04:16
lifelessthe answer to that is (broadly) readahead: when you get two similar lookups your cache can decide to load everything related04:16
lifelesswallyworld_: its not in your case, but we'll come back to that :)04:16
lifelessthe problem with readahead is that it blows out in size04:16
lifelesse.g. readahead in the inTeam case would be to select person from teamparticipation where team=self.id, in inTeam04:17
wallyworld_readahead is suboptimal - i normally don't like tyo use it except ib very narrow, specific circumstances04:17
lifelessrather than looking up single members.04:17
lifelessfor LP, we have some very big teams:04:17
lifelessselect count(*) from teamparticipation group by team order by count(*) desc limit 20;04:17
lifeless count04:17
lifeless-------04:17
lifeless 1849004:18
lifeless(dropping to 855 at the 20th)04:18
lifelessbut still, those big teams, in a naive implementation, will be a bit of a headache.04:18
lifeless(for instance, loading 18K rows from disk, if we had a cold cache situation, is ~ 40 seconds on its own)04:19
wallyworld_what's that large team called?04:19
lifelessthat one is locoteams04:20
lifeless 18490 | 2968414 | locoteams04:20
lifeless 11026 | 2979678 | locoteams-approved04:20
lifeless  3842 | 2388044 | launchpad-users04:20
lifeless  2936 | 1121852 | ubuntu-us04:20
lifelesswallyworld_: but I only brought that up to illustrate the solution-is-worse-than-the-problem nature of using on demand caching strategies in this sort of problem04:22
wallyworld_given the size of the largest teams, i can see the issue.04:23
wallyworld_although i do wonder whether such team would ever be a product owner or bug supervisor etc04:24
wallyworld_i had the feeling such attributes would be individuals or small teams04:24
lifelessfor loco team issue tracking, sure ;)04:24
wallyworld_fair enough. one of my underlying assumptions was that the team sizes involved in affiliation would be smallish04:25
wallyworld_thus allowing other interfaces than a set based one04:26
lifelessubuntu-bugs has 416 people in it04:26
lifelessopenstack has 130004:26
lifelessubuntu-marketing 140004:27
wallyworld_that's not too bad - caching 1300 id tuples04:27
wallyworld_but anyway, that seems like suboptimal approach now04:28
wallyworld_it also means that the adaptor based approach needs to be rethought i suspect04:29
wallyworld_actually, maybe not04:29
lifelessif you expose the things that need looking up04:29
wallyworld_just the getAffiliation interface04:30
lifelessa single call can seed the caches of the various teams with the results for all the queries that may be made04:30
lifelessthis is a set based interface04:30
lifelessset based doesn't mean 'no abstraction' :)04:30
wgrantI think we probably need to do away with adapters at some point.04:30
wgrantIt is always tempting to use them because they are nice and we use them already, but they make it awkward to setify.04:31
wallyworld_i think i just need to change getAffiliationBadge(self, person) to getAffiliationBadges(self, people)04:31
wgrantOh, if the API is like that, that's not too bad.04:31
wallyworld_or something like that, plus all the upstream changes for hooking it into the vocab04:31
wgrantAs long as there aren't many affiliation contexts.04:31
wallyworld_well, atm, each batch of picker results is for the same context, so there's only one04:32
wallyworld_lifeless: i think the TeamParticipation table is good for indirect memberships too, right?04:33
wgrantYes.04:33
wgrantTeamParticipation includes indirect memberships.04:34
wallyworld_right. makes it a bit easier.04:34
lifelesswallyworld_: yes, I would encourage any implementation to start with refactoring inTeam() to have a single multi-lookup method it goes through.04:34
wallyworld_lifeless: agreed. inTeam will likely not be used now04:34
lifelesswallyworld_: cool; I suggest a single method so that we don't have duplication04:38
wallyworld_lifeless: depending on how many usages of inTeam there are, I may optimise the affiliation stuff separately04:39
wallyworld_i want to get that done asap without the baggage associated with an inTeam refactoring04:39
wallyworld_across the entire product04:40
lifelesswallyworld_: you shouldn't need to touch other callers of inTeam04:40
lifelessthey are either fine, or separate problems.04:40
lifelessI'm just talking 'extract Method'04:40
wgrantAnd leave an inTeam wrapper around the new method, for compatibility.04:41
wallyworld_lifeless: oh, i see what you are saying i think. as if i would cut'n'paste code everywhere :-/04:41
lifelesswgrant: compat/convenience04:41
lifelesswgrant: all the searchTasks timeouts are on one person04:45
wgrantAha.04:45
wgrantI didn't look that far.04:45
lifelesscould well be patching04:45
lifeless    276 https://api.launchpad.net/1.0/%7Eubuntu-security-sponsors (Person:EntryResource:searchTasks)04:45
lifeless       OOPS-2045A18, OOPS-2045A33, OOPS-2045A34, OOPS-2045A39, OOPS-2045A5204:45
lifeless276 SELECT BugTask.assignee, BugTask.bug, BugTask.bugwatch, BugTask.date_assigned, BugTask.date_close ...  = FALSE)) AS BugTask JOIN Bug ON BugTask.bug = Bug.id ORDER BY BugTask.id LIMIT $INT OFFSET $INT:04:45
lifelessI suspect its the high offset issue04:46
wgrantSo adeuring will save the world.04:46
=== danilo_ is now known as danilos
=== henninge changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: henninge | Critical bugs: 247 - 0:[#######=]:256
bigjoolsmorning08:00
adeuringgood morning08:22
mrevellHi08:24
bigjoolswgrant or StevenK, care to talk about bug 820941 for a moment?09:47
_mup_Bug #820941: Distro uploads always email XXXX_derivatives@packages.qa.debian.org <derivation> <Launchpad itself:Triaged> < https://launchpad.net/bugs/820941 >09:47
wgrantbigjools: New field on distribution to provide a template?09:47
bigjoolsyeah that's my first reaction too09:48
bigjoolswell, since it's not going to change it can be fixed, we just need to decide whether a distro is going to email or not09:48
wgrantNobody is ever going to want to email anybody except Debian?09:49
bigjoolsthere's currently zero requirement to do that09:50
bigjoolsand I don't want to fix nonexistent bugs09:50
wgrantWe can almost freely remove a hard-coded Debian-specific string.09:50
bigjoolsit should move to config09:50
wgrantWhy?09:51
wgrantIt's distribution-specific.09:52
bigjoolsno, it's Debian specific09:52
wgrantAnd not everybody wants to send stuff to Debian.09:52
bigjoolsthere is no requirement to do this for any other LP-hosted distro09:52
wgrantThey may want to send it elsewhere.09:52
bigjools"may"09:52
wgrantWe need either add a bool or a string field.09:52
bigjoolsbut they don't, right now.  I am not interested in fixing nonexistent bugs09:53
wgrantIf we add a bool, we get to keep the hardcodedness for another 5 years.09:53
wgrantIf we add a string field instead, we get an extra line of code and remove the hardcoded string.09:53
wgrantThe overhead of making it customisable is probably somewhere around two minutes.09:54
bigjoolsloving your optimism09:54
bigjoolsyou're usually pessimistic ;)(09:54
bigjoolswhere's lifeless's wiki page on the DB change process?09:56
wgrantAt the moment, don't.09:57
lifelessthe usual place09:57
wgrantThere's https://dev.launchpad.net/PolicyAndProcess/DatabaseSchemaChangesProcess09:57
wgrantBut it will probably be at least two weeks before we can do it.09:57
bigjoolslifeless: that's not helpful :)09:57
lifelessbigjools: :P I was looking fo rit09:57
bigjoolsPolicyAndProcess isn't linked from the front page09:58
bigjoolslifeless: are we blocked on DB changes until it's implemeted?09:59
lifelessbigjools: fastdowntime? not blocked no. *If* we reach the end of the monthly cycle without being ready to use it, we can manually schedule a 90 minute legacy downtime.10:02
bigjoolslifeless: ok thanks10:02
jmllifeless: so that announcement about something being the last 90 minute downtime was optimistic?10:12
lifelessjml: slightly10:14
lifelessjml: its certainly the last as-regular-schedule10:14
lifelessjml: I would consider us having to do another as a big deal.10:15
bigjoolsstub: where do you keep your list of db patch numbers?  I need one and I can't remember where it lives. :(10:15
lifelessbigjools: its linked on that wiki page10:15
jmllifeless: ok. fair enough. I sort of thought from skimming it that fast db rollouts were a done thing.10:15
stublp:~launchpad/+junk/dbpatches10:15
jmlbut that was a reckless assumption.10:16
lifelessjml: we're in pretty good shape10:16
bigjoolsstub: ta10:16
lifelessjml: code is written, staging is updating using it10:16
jmllifeless: that's pretty good.10:21
wgrantWe really have no idea how long it will take on production, though :(10:23
lifelesswell10:26
lifelessthe first one will be nerve racking10:26
lifelessand will change nothing.10:26
wgrantYes.10:26
lifelessStuart is investigating slony 1.2 as a possible major improvement10:26
wgrantWell.10:26
wgrantWe *hope* it will change nothing.10:26
lifelesswgrant: -har-, yes.10:27
wgrantIt's slony1 v2, not slony 1.2 :)10:27
lifelessblah :P10:27
wgrant(the product is named slony-i or slony1)10:27
* lifeless dons the not-a-details-job hat10:27
wgrantHeh10:28
StevenKbigjools: I agree with William, make it a string10:36
stubMy attention span:  |---|10:36
stubPackaging learning curve: |-------------------------------------------------------------|10:36
wgrantstub: What are you packaging?10:36
wgrantNot Slony-I 2?10:37
stubWill need a 2.0.7 package, but basic packaging has been on my todo list for 6 years now...10:37
wgrantThat's easy.10:37
stub:-p10:37
wgrantAssuming none of the patches need changing, apt-get source slony1-2; cd slony1-2-2.0.6; uupdate /path/to/new/tarball, dch -e and edit the changelog as fit, debuild -S -sa, dput ppa:what/ever ../slony1-2*_source.changes10:39
stubIts arcane. Hunger is winning out over this introductory documentation atm.10:39
bigjoolsStevenK: I am doing that.  I just dislike only having one solution discussed.10:42
StevenKbigjools: So, you can either kill it, or make it generic.10:43
jtvhenninge: could I beg for a review?  https://code.launchpad.net/~jtv/launchpad/bug-820796/+merge/7070110:43
StevenKKilling it requires a discussion with distro and Debian10:43
StevenKMaking it generic with a string is a my vote.10:43
henningejtv: sure, you may beg ;)10:43
jtvGood, that's step 1.10:43
jtv"henninge, please, _please_ review my branch for me!"10:44
jtvThat's step 2.10:44
henningejtv: I will do that.10:44
jtvThanks!  That's step 3.10:44
henningejtv: welcome10:44
henningebaby steps? ;-)10:44
StevenKbigjools: There. Two solutions discussed. :-P10:45
henningewhy is this diff taking so long ... Is it long?10:45
jtvhenninge: nope—looks like the differ broke.  :(10:45
jtvIt's a very simple branch.10:46
henningecool, I'll pull10:46
henningebranch, I mean10:46
henningewhatever10:46
bigjoolsStevenK: you've been taking lessons in diplomacy from your wife10:46
jtvcjwatson, are you back?10:48
StevenKHe is back today10:49
adeuringlifeless: fancy a review of a StormRangeFactory MP? https://code.launchpad.net/~adeuring/launchpad/bug-739052-4/+merge/7071710:56
danilosmrevell, hi11:04
wgrantstub: Any luck?11:09
lifelessadeuring: conflicts11:09
stubeh?11:09
lifelessadeuring:11:09
lifelessDiff against target:1338 lines (+879/-25) 21 files modified (has conflicts)11:09
lifelessText conflict in lib/canonical/launchpad/webapp/batching.py11:09
wgrantstub: With the evils of packaging.11:09
lifelessetc11:09
bigjoolsstub: db MP winging its way to you11:10
adeuringlifeless: gahhh11:10
stubwgrant: your cheat sheet + common sense + google got me as far as a debsign failure. Docs bored me. Still hungry.11:10
wgrantstub: Heh. Probably just your name + email in debian/changelog not matching your OpenPGP identity exactly.11:11
wgrantstub: You can use debsign's -k option to specify a different key ID.11:11
stubYer  - need a (Work) in there11:12
adeuringlifeless: devel merged; new revision pushed11:12
stubwgrant: But that would involve me knowing what I'm doing and where to edit that :)11:12
wgrantstub: Should be reasonably obvious in debian/changelog.11:12
wgrantAlthough this is Debian-obvious, not classical obvious.11:12
stubwgrant: I meant getting different options passed to debsign11:13
wgrantstub: Should just be able to give it to debuild.11:13
stubwgrant: debbuild done now. Now to learn about ppas and uploading11:13
wgranthttp://help.launchpad.net/Packaging/PPA/Uploading11:14
stubwgrant: At what point to I get to specify 'lucid and above'?11:17
wgrantstub: You have to upload to one, and copy to the rest later.11:17
wgrantNormally one would upload to the earliest, as that is what it will build for.11:17
wgrantThen the binaries and source will be copied to maverick/natty/oneiric.11:17
wgrantAFAICT this should build fine on Lucid, but I guess we're about to find out.11:18
wgrantstub: The upload series is specified on the top line in debian/changelog, in case you hadn't seen that.11:18
wgrantDefaults to 'unstable'.11:19
wgrantWhich obviously won't work too well for Ubuntu.11:19
stubta11:19
stubwgrant: version number 2.0.7-0~ppa1~lucid1 or 2.0.7-0~ppa1 ?11:20
wgrantstub: That's one of the great unanswered questions of the universe.11:20
wgrantstub: I'd probably use the latter.11:20
wgrantI actually tend to use -0ppa111:21
stubwhat I have atm. docs are trying to confuse me :)11:21
stubAhh... what I had earlier before something told me to put in a twiddly11:21
wgrantstub: It depends.11:23
wgrantWe often backport stuff, in which case we'll normally use $(original version)~ppa111:23
wgrantBut -0ppa1 < -0ubuntu1 < -1, so it's cleaner and still works fine.11:23
stubits in there now... lets see what lp does with it while I get fud11:24
wgrantAnd building already.11:26
lifelessjml: can we unsubscribe testtools-dev from the bug tracker ?11:27
lifelesshttps://bugs.launchpad.net/testtools/+subscriptions11:28
jmllifeless: you just want to be on the mailing list?11:28
lifelessjml: I think having the mailing list get a copy of every bug change makes it useless for discussion11:29
jmllifeless: oh huh. I'm not getting those mails.11:29
lifelessjml: see the page I linked; you may need to look under 'other subscriptions' or something11:30
jmllifeless: I'm not getting those mails.11:30
lifelessjml: thats because they have the same message id11:30
lifelessjml: and you are also subscribed directly11:31
jmlah ok.11:31
lifelessjml: and you are using a MUA that dedupes on message id.11:31
lifelesshttps://lists.launchpad.net/testtools-dev/11:31
lifelessjml: its also getting merge proposals11:32
lifelessjml: have you, by chance, combined a discussion-group and a permission-granting-team ?11:32
jmlsigh11:32
jmlno.11:32
jmlI uncombined it.11:32
jtvhenninge: any questions about the review so far?  I'm hoping to scoot off soon.11:33
henningejtv: no, sorry, did not get that far yet.11:35
jtvhenninge: in the meantime, I replaced the core of the Ubuntu distribution infrastructure and it performed its first-ever production run.  I think I need to peddle out the tension on my bicycle soon.  :-)11:36
henningejtv: you mean, you pushed a new revisionß11:36
henningejtv: but go ahead and peddle11:36
henninges/ß/?/11:37
jtvhenninge: sort of — Tom rolled out the necessary cron change for me.  And then we all sat around worrying until the script ran.11:37
henningeah that11:38
lifelessadeuring: reviewed; couple of tiny things11:38
adeuringlifeless: thanks!11:38
adeuringlifeless: >>> def f(a,b):11:40
adeuring...  return a+b11:40
adeuring...11:40
adeuring>>> reduce (f, 1, 2, 3)11:40
adeuringTraceback (most recent call last):11:40
adeuring  File "<stdin>", line 1, in <module>11:40
adeuringTypeError: reduce expected at most 3 arguments, got 411:40
adeuringso, reduce(And, clauses) is right, I think11:41
lifelessadeuring: huh? that seems unrelated ;)11:41
adeuringlifeless: argh, sorry, got your comment wrong11:41
lifelessAnd is a CompoundExpr11:42
lifeless    def __init__(self, *exprs):11:42
lifeless        self.exprs = exprs11:42
adeuringright11:42
henningejtv: no questions, r=me11:44
jtvthanks henninge!  That's step… 4?11:44
henningejtv: wasn't that 5 already?11:45
jtvCould be, could be.11:45
wgrantstub: So, looks like slony1-2 all went well.12:13
=== benji changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: henninge, benji | Critical bugs: 247 - 0:[#######=]:256
stubNeed to copy it into natty so I can test with it12:13
wgrantYou should see a handy https://launchpad.net/~stub/+archive/launchpad/+copy-packages12:14
wgrantMake sure to select to copy with binaries, and to the right archive.12:14
stubNot rebuild?12:14
stubGuess better if I test the actual lucid binaries12:15
stubWhat are p-series and q-series?12:16
wgrantstub: Ubuntu 11.10 and 12.04 LTS, which should not be shown on +copy-packages.12:17
wgrantBut we don't really have a way to model that properly yet.12:17
wgrantEr, 12.04 LTS and 12.10.12:18
wgrantI am 6 months behind, apparently.12:18
=== danilo_ is now known as danilos
bigjoolswgrant: should be very easy to filter in the vocab - check for len(supported archs ) == 012:21
wgrantbigjools: Yeah, but what if it's partially initialised and oh god everything comes back to this doesn't it.12:22
bigjoolsargh12:26
bigjoolswgrant: we're going to have to suck that bullet and just fix this12:27
wgrantProbably.12:27
StevenKBut we have a field in DSP for this purpose?12:27
wgrantDSP is not relevant here; not everything has a parent.12:27
bigjoolsneeds to be a bool on the series I expect12:28
StevenKWhich we did have ...12:28
StevenKWhich hooked into DSP, and was a horrid hack, IIRC12:30
StevenKbenji: You haz QA to do!12:31
StevenKgary_poster: QA for you as well :-P12:31
benjithanks, StevenK; I'll look now12:31
gary_posterStevenK, looking thanks12:31
jtvWho will review some security.cfg extensions?  benji, I've already bothered your co-OCR today so I'm angling for you: https://code.launchpad.net/~jtv/launchpad/bug-822640/+merge/7073012:48
benjijtv: sure, I'll take a look now12:48
jtvthanks12:49
benjijtv: done, looks fine12:50
jtvGreat, thanks!12:50
deryckMorning, all.13:13
abentleyderyck: welcome back.13:14
deryckabentley: thanks!13:14
danilosabentley, hi, I was wondering about bug 820511, where it would be most useful to have the job ID logged? I was thinking of doing it in the existing debug statements in the runner, but I wonder if that'd be enough for the circumstances we were in?13:33
_mup_Bug #820511: Job scripts don't report job ids <jobs> <Launchpad itself:In Progress by danilo> < https://launchpad.net/bugs/820511 >13:33
abentleydanilos: on the phone13:33
abentleydanilos: The normal logging should be fine.  I think it should be "info", so it still shows when the script is run non-verbosely.13:44
danilosabentley, right, that's what I was thinking as well13:45
danilosabentley, thanks13:45
abentleydanilos: cool.13:45
bigjoolsjml: is it wrong that when I saw the sperm whale stranded on the Australian coast earlier, I thought "Beached as!"13:57
jmlbigjools: not at all :)13:57
jmlalthough I do believe that was a kiwi whale in the yt video13:57
bigjoolsit may have crossed the Tasman13:58
jmlheh13:58
henningebenji: Hi15:11
henningebenji: are you currently reviewing something?15:12
benjihi, henninge15:12
benjihenninge: not at the moment15:12
henningebenji: ok, I'll take jelmer's big one, then.15:12
jcsackettsinzui: can you mumble?15:13
sinzuiyes15:13
jcsackettcan you hear me, sinzui?15:16
henningejelmer: Hi!15:32
bigjoolshi henninge or benji, can I bother one of you for a review please?15:35
benjibigjools: I can after I finish this one.15:35
bigjoolsbenji: thank you15:35
bigjoolshttps://code.launchpad.net/~julian-edwards/launchpad/code-bug-820941/+merge/7075315:35
henningebenji: which one are you doing?15:38
benjihenninge: gary asked me to do https://code.launchpad.net/~gary/launchpad/bug821531/+merge/7073815:38
jelmerhenninge: hi!15:38
henningebenji: ah, ok15:38
henningejelmer: Hi!15:39
henningejelmer: I just wanted to ask if your safe-branch-open branch is strictly refactoring.15:39
henningejelmer: I assumed so, just wanted to double-check.15:39
jelmerhenninge: yes, it should not change any of the existing behaviour15:40
henningejelmer: cool, I already approved it.15:40
jelmerhenninge: Great, thanks for the review :)15:40
=== henninge changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: benji | Critical bugs: 247 - 0:[#######=]:256
henningejelmer: I'll do your other one before I finish.15:41
jelmerhenninge: Cool15:44
henningejelmer: Line 120 of the diff: I think you wanted to use the "port" parameter here.15:51
benjibigjools: I'm done with the review.  I had one small question.15:51
bigjoolsok15:52
bigjoolsbenji: In the pre-requisite schema branch the SQL updates Ubunbu address.  Nothing else needs one right now.15:53
bigjoolsUbuntu, even!15:53
benjibigjools: cool, I figred it was something like that15:53
bigjoolsbenji: thanks for reviewing15:53
benjimy pleasure15:54
jelmerhenninge, that's a good point. or potentially, we could get rid of the port parameter altogether15:55
henningeor that15:55
gary_posterbac, someone wants is to "_REALLY_ DELETE ACCOUNT" (https://answers.launchpad.net/launchpad/+question/167045).  "please delete my account permanently. i still can sign in even when i use the deactivate procedure recommended."  Are they out of luck?16:04
gary_posterI'm going to say "no" and point them to https://answers.launchpad.net/launchpad/+faq/96816:10
sinzuigary_poster, SSO reenables the user profile. Deactivate works, just do not sign in again to make it active16:10
=== Ursinha` is now known as Ursinha-afk
=== Ursinha-afk is now known as Guest6606
=== Guest6606 is now known as Ursula
gary_posterthanks sinzui, yes, that's where I was going.  While I have your attention, someone wants to convert a person to a team (~lintian-maint fwiw).  Am I right that they cannot do this, and that they should create a new team and then fiddle with the names if absolutely necessary?16:13
=== Ursula is now known as Ursinha
* gary_poster going to lunch; will be back later16:20
=== deryck is now known as deryck[lunch]
sinzuigary_poster, a person can be converted (claimed?) to a team in one case, but I cannot remember how that is done.16:34
sinzuigary_poster, see lp/registry/stories/team/xx-team-claim.txt16:36
gary_posterthank you sinzui16:47
=== beuno is now known as beuno-lunch
benjiallenap: I think there's a prblem with the Python-dict-in-JS branch, my thoughts are in the MP: https://code.launchpad.net/~allenap/launchpad/javascript-utils-2/+merge/7076016:53
gary_posterabentley, would you mind if I assigned you a code import question?  https://answers.launchpad.net/launchpad/+question/167251  Alternatively, I'm happy to formulate a reply if you give me the gist, or do other necessary grunt work.17:18
abentleygary_poster: looking17:18
gary_poster(that is, I'm happy to do other necessary grunt work if directed)17:18
gary_posterthank you17:18
abentleygary_poster: I'll take it.  I looks like the answer is "this is a bug in bzr-hg".17:22
gary_posterok thanks abentley17:22
=== deryck[lunch] is now known as deryck
=== beuno-lunch is now known as beuno
=== jacob_ is now known as j-johan-edwards
j-johan-edwardsHello, is there any way to get a dump for soyuz packaging records in the Ubuntu archives?17:48
j-johan-edwardsThe sample data is pretty sparse...17:48
j-johan-edwardsHey, sorry to repeat (won't again) but is there any way to get a dump for soyuz packaging records in the Ubuntu archive?18:20
abentleybenji: could you please review https://code.launchpad.net/~abentley/launchpad/ubuntu_merges/+merge/70772 ?18:35
benjiabentley: sure18:35
benjiI appreciate the courtesy space after the URL. ;)18:36
benjiabentley: done18:42
lifelessmorning y'all18:57
lifelesssinzui: hi; re bug 29815219:10
_mup_Bug #298152: privacy portlet is not visible enough for indicating private objects <disclosure> <lp-web> <oem-services> <privacy> <qa-ok> <ui> <Launchpad itself:Fix Released by jcsackett> < https://launchpad.net/bugs/298152 >19:10
sinzuiyes19:10
* jcsackett perks up.19:10
lifelesssinzui: 2 questions;. A) there was some user feedback that the new ribbon is less noticable than the old border thing, particularly when someone has been switching tabs and is no longer at the top of page.19:11
lifelessand B) have we enabled the feature flag to show it to everyone ?19:11
sinzuilifeless, the privacy ribbon cannot be scrolled off.19:11
lifelesssinzui: it used to have a close button19:12
jcsackettlifeless: sinzui and i were going to add it to beta-testers. i forgot to ping a losa on that point.19:12
sinzuiI asked jcsackett to enable the ribbon for all beta testers. We will blog about it to encourage feedback19:12
jcsacketti'll correct that now.19:12
lifelesssinzui: so I guess the scenario is 'close, scroll, switch windows, come back and forget'.19:12
sinzuilifeless, I question the hide button because it is not obvious how to unhide it19:13
jcsackettsinzui, lifeless: i'm plus one on removing the hide part of it.19:13
lifelesssinzui: anyhow, my question (A) is, has that user reaction been taken into account? Perhaps the bug should stay open, or a new follow on be opened, if not.19:13
thedacjcsackett: did you need something done now?19:13
sinzuilifeless, https://bugs.launchpad.net/launchpad/+bug/789566 < scroll and confirm it is fixed at the top of your screen19:14
jcsackettthedac: yeah, i'm looking up a feature flag rule i need. :-)19:14
thedacok19:14
sinzuilifeless, the ribbon is fixed using css2. if your browser does not support it, please update to a browser released in the last 10 years19:14
lifelesssinzui: I don't actually have an opinion about it myself, just noticed the feedback in https://bugs.launchpad.net/launchpad/+bug/298152/comments/35 and no apparent response19:14
_mup_Bug #298152: privacy portlet is not visible enough for indicating private objects <disclosure> <lp-web> <oem-services> <privacy> <qa-ok> <ui> <Launchpad itself:Fix Released by jcsackett> < https://launchpad.net/bugs/298152 >19:14
lifelesssinzui: yes, its fixed when not hidden.19:15
lifelesssinzui: I have no concerns about the implementation :)19:15
sinzuisorry 'fixed' is a css position property. I did not mean this bug is fixed when the user can see the ribbon19:16
lifelesssinzui: I meant 'pinned to the top of the visible area' when I said fixed.19:16
sinzuilifeless, We had positive reaction, but NO reaction from non Canonical staff so I asked that we broaden the test19:17
lifelesssinzui: that was because only canonical staff could see it :P19:17
sinzuiindeed that is why I asked19:17
lifelesssinzui: sounds like its all under control... cool19:17
lifelessthanks19:17
sinzuilifeless, I wanted to remove the hide action, but that would be me second guessing that users do not want to shoot themselves in the foot19:18
lifelesssinzui: could we tell how many people hide the ribbon somehow?19:19
sinzuino I cannot, nor did I agree with the reason for that addition19:20
jcsackettsinzui, lifeless: maybe we should just create a mockup with hide and without hide on some page and throw it to user testing?19:20
sinzuiI think there was confusion between the desire to show state and the need for a universal notification presentation that did not suck19:20
sinzuijcsackett, I am hoping for someone to be reasonable a realise that the cost of a leak could be millions of dollars and individuals should not be given an opportunity to be the source of the leak19:22
jcsackettsinzui: fair point.19:22
sinzuijcsackett, as we discussed last week. The privacy portlet is not on subordinate pages. So the hide code that changes that portlet will not work19:25
jcsackettsinzui: right, the hide code doesn't do the highlight portlet thing if there is no portlet.19:25
jcsackettthe original code already had a toggle for hiding without highlighting; based on our conversation i thought we elected to not remove "hide" and simply go with "only try to highlight the portlet if it exists".19:27
sinzuijcsackett, mpt reported a bug where the ribbon separates the state from when you change the state. The privacy portlet is largely an artefact of the 2/3x designs. Moving the control to change privacy to the ribbon would 1 Remove the need for the hide button's effect.19:27
abentleylifeless: AIUI, Launchpad URLs should never contain whitespace or non-ascii characters in the path portion of the URL.  Bug 819841 and bug 819859 would be solved if such URLs automatically 404ed.  Do you think it makes sense to apply that at the URL parsing level?19:27
_mup_Bug #819841: OOPs rendering page with non-ascii URL <oops> <Launchpad itself:Triaged> < https://launchpad.net/bugs/819841 >19:27
lifelesssinzui: what stops you removing the hide button ?19:27
_mup_Bug #819859: launchpad urls permit spurious whitespace <oops> <Launchpad itself:Triaged> < https://launchpad.net/bugs/819859 >19:27
lifelessabentley: do these urls currently work sometimes, or not at all ?19:28
jcsackettsinzui: interesting. so we get rid of the privacy portlet (or at least the edit part of it) and move that into the ribbon, simultaneously killing the "hide" button?19:28
sinzuiunderstanding of why it was added. As I wrote above. I think it was a misunderstanding. Privacy is not a notification, it should not be dismissable19:28
lifelessabentley: and by parsing level, do you mean in the incoming url routing area, or in our 'regex to figure out what is a url in a page' ?19:28
abentleylifeless: urls with ascii whitespace work.  urls with non-ascii whitespate oops.19:28
abentleys/whitespate/whitespace/19:28
abentleylifeless: I mean in the incoming url routing area.19:29
sinzuilifeless, yet one requirement may have been that the notice cannot take up vertical space that is needed on small screens19:29
lifelessabentley: ok. You probably don't have this at hand, but do we emit such urls-with-ascii-whitespace from our regex linkifier code? E.g. would we break some in-site links from user content doing this?19:29
lifelesssinzui: you could have a button that moves it to the left hand side ?19:29
sinzuiThat was my original plan that was scraped by Huw's testing/design/work19:30
lifelesssinzui: FWIW I accept your assertion that being able to hide it seems liable to increase the risk of accidental disclosure19:30
lifelessabentley: I think 404ing on urls that cannot possibly resolve to urls we would generate makes a great deal of sense.19:31
abentleylifeless: I can test whether we linkify them.19:31
lifelessabentley: and even if we do break some things we linkify today, thats ok - we can fix the linkifier19:31
lifelessabentley: I would ask, if we expect nontrivial fallout, that we do both at once; if the fallout is going to be minor, 2 landings, or even 1 with the second maybe-or-maybe-not happening would be fine with me19:32
sinzuilifeless, the ribbon is officially in customer acceptance as of Wednesday. I expect to get feedback about the design and bugs about what is or is not displayed private19:32
abentleylifeless: urls with ascii whitespace can and do resolve today.  So you would forbid only the non-ascii ones?19:32
lifelessabentley: I would be ok with either all or only non-ascii as you choose; my thinking is that if we don't emit it, there is no reason to accept it, and room for interesting bugs if we do.19:33
lifelessabentley: s/don't emit it/don't emit it or explicitly expect it [e.g. search form submission]/19:34
abentleylifeless: Right.  And with with search form submission I believe we use the query, not the path.19:34
lifelessabentley: yes19:35
lifelessabentley: (you said 'urls' rather than 'url paths' before, and I was trying to avoid being pedantic while still giving useful accurate answers)19:35
abentleylifeless: I started with "path portion of the url".  Sorry if I was unclear.19:36
lifelessabentley: ah, I failed to parse!19:36
lifelessabentley: so, we're synced :)19:36
abentleylifeless: We do linkify URLs with whitespace: https://bugs.qastaging.launchpad.net/bzrtools/+bug/76043519:37
_mup_Bug #760435: failUnless & co cause PendingDeprecationWarning <easy> <selftest> <verification-done> <Bazaar:Fix Released by mbp> <Bazaar 2.3:Fix Released by vila> <bzr-pipeline:Fix Committed> <BzrTools:Fix Committed> <bzr (Ubuntu):Fix Released> <bzr (Ubuntu Natty):Fix Released by jelmer> < https://launchpad.net/bugs/760435 >19:37
lifelessabentley: that particular case I think is fine, cause its already hex escaped19:38
lifelessI meant something like <https://foo bar/>19:38
abentleylifeless: I meant something like %20 or %A019:38
lifelessabentley: in the web request yes; I was worried about someone finding a way to cause that from what looks like whitespace19:39
lifelessabentley: e.g. the example I've put into that same qastaging bug19:39
sinzuiI am having network difficulties. Maybe no one will see this. I am restarting.19:41
abentleylifeless: So to give examples, I think https://bugs.qastaging.launchpad.net/bzrtools/+bug/760435 should work, https://bugs.qastaging.launchpad.net/bzrtools/+bug/760435%20 and https://bugs.qastaging.launchpad.net/bzrtools/+bug/760435%C2%A0 should 404.19:42
_mup_Bug #760435: failUnless & co cause PendingDeprecationWarning <easy> <selftest> <verification-done> <Bazaar:Fix Released by mbp> <Bazaar 2.3:Fix Released by vila> <bzr-pipeline:Fix Committed> <BzrTools:Fix Committed> <bzr (Ubuntu):Fix Released> <bzr (Ubuntu Natty):Fix Released by jelmer> < https://launchpad.net/bugs/760435 >19:42
_mup_Bug #760435: failUnless & co cause PendingDeprecationWarning <easy> <selftest> <verification-done> <Bazaar:Fix Released by mbp> <Bazaar 2.3:Fix Released by vila> <bzr-pipeline:Fix Committed> <BzrTools:Fix Committed> <bzr (Ubuntu):Fix Released> <bzr (Ubuntu Natty):Fix Released by jelmer> < https://launchpad.net/bugs/760435 >19:42
_mup_Bug #760435: failUnless & co cause PendingDeprecationWarning <easy> <selftest> <verification-done> <Bazaar:Fix Released by mbp> <Bazaar 2.3:Fix Released by vila> <bzr-pipeline:Fix Committed> <BzrTools:Fix Committed> <bzr (Ubuntu):Fix Released> <bzr (Ubuntu Natty):Fix Released by jelmer> < https://launchpad.net/bugs/760435 >19:42
abentleylifeless: Right now, the second is equivalent to the first, and the third oopses.19:43
jcsackett?19:44
* jcsackett fails at keyboards again.19:45
lifelessabentley: I agree19:45
abentleylifeless: excellent.19:45
lifelessabentley: I don't think the linkifier needs touching19:46
abentleylifeless: cool.19:46
lifelessabentley: based on this discussion + experiment19:46
lifelessif someone hex encodes whitespace into a url in a page, we should linkify that19:46
abentleylifeless: do you know where we parse/validate incoming urls?19:46
lifelessabentley: A reasonable place to start would be lib/canonical/launchpad/webapp/publisher.py19:46
abentleylifeless: Yes, I'll have a look there.19:47
lifelessabentley: I don't know precisely19:47
thedacjcsackett: to confirm feature flag change is on production, correct?19:55
abentleygary_poster: do you happen to know where would be a good place to add extra validation rules for incoming URLs (i.e. when a browser does a GET or POST on us)?19:55
jcsackettthedac: yes.19:55
thedacjcsackett: and do you have a one liner log message I can put in?19:56
gary_posterabentley, so, at the very beginning of processing a URL?19:56
abentleygary_poster: Yes.  I want to verify that the "path" portion does not contain any whitespace or non-ascii characters.19:57
abentleygary_poster: And by "whitespace", I mean %20 as well as " ".19:57
gary_posterabentley, and I assume the goal is to not produce an OOPS.  It would relatively easy to do this if we didn't mind an OOPS with a IStartRequestEvent handler... I wonder if we can somehow short-circuit anything there.  alternatively...maybe we have overridden enough of the publisher that you can do it in there.  Lemme dig for a sec abentley19:59
abentleygary_poster: okay, thanks.20:00
gary_posterabentley, so, I suggest that you work after the request has been made, probably in beforeTraverse.  Take a look at eggs/zope.publisher-3.12.0-py2.6.egg/zope/publisher/publish.py and in lib/canonical/launchpad/webapp/publication.py .20:13
gary_posterIn publish.py, you'll see a publish function.  That's the basics of how Zope publishes a request.  It uses a publication object, which we have customized in the second file I listed.  As you can see in publish, if you raise an error in beforeTraverse, then the publication's handleException will be called.20:13
gary_posterYou'll want to raise an exception that has a view registered for it that gives the user the information they will want about the error.20:13
gary_posterThere are examples of times we do that in the LP code now; see zope.app.publication.browser.BrowserPublication.handleException (we override it but the meat you want is elsewhere) for how that works if you want the details.20:13
gary_posterYou might want to advertise this to devs after you are done; I wonder if there are any other "quit now" shortcut rules we might want to implement this way.20:13
abentleygary_poster: Okay, thanks for the pointers.20:14
gary_posterwelcome20:16
gary_posterlifeless, thanks for review!  I agree and normally self-review those, but I'm asking benji to make sure everything works for him in Gnome.  It's kind of a double-check for Oneiric20:47
lifelessah cool20:49
lifelessI didn't realise20:49
deryckabentley: hey, was talking and didn't realize I was on mute. ;)20:49
abentleyderyck: no worries.  I didn't mean to be in Mumble anyhow :-)20:50
deryckabentley: heh.  Was coming back to my normal home from another call :)20:50
=== benji changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: - | Critical bugs: 247 - 0:[#######=]:256
=== lifeless changed the topic of #launchpad-dev to: Performance Tuesday | https://dev.launchpad.net/ | On call reviewer: - | Critical bugs: 247 - 0:[#######=]:256
lifelessbah, missed gary22:01
lifelessbenji: still here, perchance ?22:01
lifelessabentley: btw - def publishTraverse(self, request, name):22:08
lifeless in publisher.py22:08
lifelessseems to do the ascii only enforcement, so is close to what you were looking at22:09

Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!