/srv/irclogs.ubuntu.com/2011/02/15/#launchpad-dev.txt

* thumper throws up his hands in frustration00:41
thumperdamn windmill testing00:41
wgrantlifeless: Erm, your PQM submission had a very odd failure.00:42
wgrantlifeless: I guess it could have been the codehosting connection refusal.00:42
thumperanyone else getting failures in  lp.services.features.browser.tests.test_feature_info.TestFeatureControlPage ?00:42
wgrantthumper: Yes.00:43
wgrantlifeless has a testfix for that.00:43
wgrantBut PQM is being crap.00:43
wgrantThere are four failures relating to this change. See the latest buildbot run for details.00:43
* thumper sighs00:44
* thumper gets frustrated at jQuery selectors00:48
thumperpossible pebkac00:49
mwhudsonjquery?00:49
thumpermwhudson: used for windmill tests to select elements00:50
mwhudsonoh right00:50
thumperyep PEBKAC00:50
lifelesswgrant: I have submitted, again, again.01:03
lifeless4th attempt to ec2 land this db patch01:12
wgrantlifeless: Thanks.01:17
* thumper sprinkles some '[0]' values over the jquery selectors01:17
lifelessgrrr01:26
lifelesshttp://checkip.amazonaws.com -> efail01:26
lifelessjust for me01:26
wgrantWhy? Proxy?01:27
lifelessdunno01:27
lifelessthere is a nz wide proxy :(01:27
wgrantAwesome.01:27
wgrantYay, landed.01:28
* wgrant lands stuff.01:28
wgrantlifeless: '[ui=none]' :(01:28
* thumper lands daily-ajax01:33
=== Ursinha is now known as _starbuck-gone
wgrantSo, we need to do something about all these codehosting OOPSes. I think UnknownRemoteImportanceError probably wants to generate informational OOPSes?02:00
wgrantExternalBugtracker for BugTrackerType 'GOOGLE_CODE' is not known.02:00
wgrantThat seems thoroughly useless.02:00
* wgrant removes.02:00
mwhudsonwgrant: i don't think you mean codehosting02:00
wgrantmwhudson: Er, checkwatches.02:01
wgrantNo idea where codehosting came from.02:01
spmwgrant: about the same place as "ssh codebounce.c.c" comes from02:01
wgrantAlthough there are lots of codehosting OOPSes too.02:01
wgrantThey are next after checkwatches.02:02
lifelessspm: oh hai02:02
wgrantSo HA02:02
lifelessspm: I can has those haproxy?02:02
spmDENIED02:02
spmnot really, I just like saying that.02:02
lifelessspm: you've been saving that up for weeks02:02
spmmonths....02:02
lifelessspm: ok, so not denied... urlof screen cap ?02:06
wallyworldanyone know how to get the text to show up beside the image for a lazr-btn styled submit button? by default, using class=lazr-btn hides the text and just shows the image02:08
thumperanyone? https://code.launchpad.net/~thumper/launchpad/recipe-inline-edit-recipe-text/+merge/4958502:09
thumperwallyworld: call?02:09
thumperwallyworld: mumble? and I'll try to answer your question02:10
wallyworldthumper: ok02:10
pooliehi lifeless02:27
pooliecan i borrow your ear briefly?02:28
wgrantlifeless: That's a nice Message query improvement you have there.02:32
lifelesswhats wrong with these two lines02:38
lifeless        comments[0].text_for_display = ''02:38
lifeless        assert len(comments) > 0, "A bug should have at least one comment."02:38
lifelesswgrant: yeah, should be pretty swet02:38
lifelesspoolie: sure02:38
lifelesspoolie: voice or irc?02:38
lifelessspm: thank yo02:38
pooliepots? i can call you?02:39
beunolen of the list is > 0!  guaranteed success, can't beat that02:39
wallyworldanyone want to claim a code review? https://code.launchpad.net/~wallyworld/launchpad/request-build-popup/+merge/4886403:06
wallyworldthumper: do you have a bug number handy for the recipe "build now" work03:12
wgrantspm: qas is down... is this the DB restore?03:20
spmwgrant: aye03:29
spmhrm. actually I could bring it back up for now. the db restore is into an _new db. need it so?03:34
wgrantNot really.03:35
wgrantHow long will the DB restore take?03:35
wgrantI am hoping for another deployment tonight to unbreak nightly.sh.03:35
* StevenK appears.03:40
wgrantHi StevenK.03:41
wgrantWhen are you back?03:41
wgrantthumper: What's changed in the new lazr.restful?03:41
StevenKAt the airport, waiting for boarding.03:41
StevenKPlane delayed 40 minutes already.03:41
wgrantWhere've you been? Adelaide?03:41
StevenKRight.03:41
pooliewgrant, i commented on https://code.launchpad.net/~wgrant/launchpad/bug-708999-ff-docs/+merge/4975803:46
pooliehappy to discuss that03:46
spmwgrant: how long? no idea. based on staging restores somewhere around 12+ hours would be my wag.03:46
wgrantspm: :( OK03:46
wgrantpoolie: Yeah, we do need a boolean standard :(03:47
wgrantI reused the existing [empty|nonempty] notation, but am not very happy with it.03:47
pooliei think at least changing the docs would be good03:47
pooliewe can file a bug for booleans03:47
pooliei regret not adding such a mechanism in the first place but at least it's fairly easy to change03:48
wgrantDo you have a preferred replacement for '[empty|nonempty]'?03:48
poolie[true|]03:52
pooliemaybe?03:52
wgrantI guess, yeah.03:52
wgrantI can probably leave out the "if true" then, too.03:53
pooliei can really imagine people either putting 'non-empty' or more likely thinking it's something to do with how to handle empty queues03:53
pooliei did, on the first reading03:53
wgrantpoolie: I initially had the "if non-empty" at the start, but felt that made it less readable by putting the useful information later.03:56
wgrantBut this looks OK now.03:57
wgrantpoolie: http://people.canonical.com/~wgrant/launchpad/feature-info.png03:59
wgrantBut we really need some standardisation of names and values :/03:59
lifelessuhm04:01
lifeless[true|] seems less clear to me04:01
lifelessthis is a dsl, its not actually python.04:01
lifelessjust saying.04:01
wgrantFor some of the keys true makes sense.04:02
wgranteg. visible_render_time and code.incremental_diffs.enabled04:03
lifelessand disable_targetnamesearch04:03
wgrantRight.04:03
lifelessbut04:04
wgrantSo only really code.branchmergequeue needs to be 'enabled'04:04
lifelessFalse04:04
lifelessis not the opposite of true04:04
wgrantRight.04:04
lifelesshmm04:06
lifelessthis might be even easy04:06
wgrantIt is tempting to define a new boolean standard.04:07
wgrantDescribe it on +feature-info as empty or non-empty.04:07
wgrantAnd then make all of the existing rules fit it.04:07
poolieyes04:07
pooliei think for now, if it's just empty or not04:07
pooliei care more about making the comment clear than what's in the value domain04:08
wgrantlifeless: You had a suggestion?04:08
poolielifeless, i see launchpadlib's tip still has uris in uris.py, surprisingly04:09
poolies/surprisingly04:10
pooliei think leonard was only saying hypothetically he could change them04:10
lifelesswgrant: BugTask:+index may be easy04:12
wgrantMy current thoughts are that I should standardise all boolean flags to empty/non-empty, call this value domain 'boolean', document the empty/non-emptiness of this new domain on +feature-info, and beat the existing boolean feature flags until they have names and implementations compatible with this new standard.04:13
wgrantlifeless: Oh?04:13
lifelesswgrant: sure, sounds reasonable04:13
poolieor not04:13
pooliewgrant, that sounds fairly reasonable04:14
pooliei wonder if 'defined to be ""' vs 'not defined' will be a bit confusing, for flags that essentially default to be on04:14
wgrantFlags won't default to be on.04:14
wgrantThe only one that does that is disable_targetnamesearch.04:14
wgrantAnd that is one I will rename.04:14
lifelesswgrant: mmm04:15
lifelesswgrant: why rename it?04:15
wgrantBecause it is inverted.04:15
lifelessits inverted so that when we deploy it doesn't suddenly disable the search04:15
lifelessit has the vector it needs04:16
wgrantRight, but that can also be achieved by adding the FF beforehand.04:16
lifelessand the name matches the vector, doesn't it ?04:16
wgrantOr we could add a default to getFeatureFlag.04:16
lifelesswgrant: or just accept that sometimes we need to disable rather than enable04:16
wgrant:(04:16
lifelesswgrant: I think changing the name for the reason you give is pointless consistency04:17
lifelesswgrant: harmful even04:17
wgrantOK.04:17
wgrantlifeless: Do you also object to my renaming of malone.advanced-subscriptions.enabled to malone.advanced_subscriptions.enabled, code.branchmergequeue to code.branchmergequeues.enabled, code.recipes_enabled to code.recipes.enabled?04:19
wgrantI fear that if I do not then yet more variants will spring up.04:20
wgrantWhereas if we standardise now, people will see there is a standard and follow it.04:20
wgrantBecause currently it is a little crazy.04:20
poolie+1 to that04:21
poolieadding a default is a bit harder in TAL04:21
pooliewe could set a default in the code that registers the flag definition04:21
wgrantAhh, true.04:21
pooliei have also been wondering about features that grow from being experimental and off-by-default to being stable and on-by-default04:22
pooliewe want to be able to garden the dynamic rules04:22
pooliebut, ideally, while still being able to flip things off in response to late-breaking emergencies04:22
poolie(but perhaps not; the code may have rusted in place)04:22
wgrantRight. The 'default' column on +feature-info seems a little pointless if there is no default.04:23
poolieif there's a default we probably shouldn't have things expressed in the negative sense like 'hidden' or 'disabled'04:24
pooliei guess the question is whether the default is expressed in just one place, or diffused throughout the code04:24
poolieat the moment i suspect the latter04:24
wgrantRight, that was my point with lifeless' new disabled_targetnamesearch.04:25
wgrantI don't like it. But without defaults it's hard to sensibly do the opposite.04:25
wgrantFor now I'll do the boolean standardisation. We can think about defaults later.04:27
lifelessuhm04:29
wgrantHm?04:29
lifelessso, I don't care about the renames; all the ones you list are flags we're going to delete in the next 4-5 months.04:29
wgrantWe hope.04:29
lifelessI think the _ is better than the -04:29
wgrantYep.04:30
lifelessI could care less about the namespacing; I think the cost of communication will outweight several years development :)04:30
wgrantWaitwaitwait.04:30
wgrantCould, or couldn't?04:30
lifelessI have a small aesthetic preference for namespacing04:30
lifelessbut I really think it doesn't matter at all04:31
lifelessif you rename04:31
lifelessyou need to communicate very clearly04:31
lifelesscheck losa docs04:31
lifelesscheck outstanding rts04:31
lifelessand check with teams setting these.04:31
wgrantOf course.04:31
lifelessthats a lot of work; if you want to do it, its your choice. But you asked for my opinion.04:32
lifelesswhich is 'why bother'04:32
wgrantIf I don't, then someone will need to police all reviews until we are standardised.04:32
lifelesswhy?04:32
wgrantBecause conventions are good, but people don't follow unimplemented conventions.04:32
lifelesswhat does this standard buy us; what is the benefit? how does it make us more efficient or less fragile ?04:33
lifelessconventions are conventions, they are most definitely not intrinsically good04:33
wgrantSure.04:33
wgrantBut code.branchmergequeue is not a clear name.04:33
wgrantIf its values were [enabled|disabled] it might be.04:34
lifelessLook, I don't /object/ to a mass rename if you do all the work, I just think I weigh the costs and benefits very differently here. I think the cost in losa time is a huge shame.04:35
lifelessand I certainly wouldn't want reviews being bounce back if a clear but 'nonstandard' name were chosen; feature flags are meant to be low key and cheap constructs.04:35
lifelessI agree that code.branchmergequeue is unclear.04:36
thumperwgrant: you mean with respect to my change?04:38
wgrantthumper: You upgraded us to the proper 0.16.0, with no-qa.04:38
wgrantI am wondering if anything significant has changed that needs testing.04:38
wgrantSince things went bad with the last upgrade.04:39
thumperwgrant: the only change is that non-5xx error codes no longer have tracebacks04:40
wgrantOK, great. Thanks.04:40
pooliewgrant, i think the main change we want is to say that in future it should be 'code.branchemergequeue.visible' or whatever04:40
poolieto establish the pattern for future things04:40
wgrantpoolie: Right.04:40
* lifeless is saying that structure really doesn't matter04:41
lifelessthe silos are gone04:41
lifelesswhat matters now is *doing*04:41
lifelessand with that, I'm back to working on timeouts, a pasttime I hope you all share my passion for04:41
wgrantlifeless: Our favourite BugTask:+index?04:41
lifelessyes04:42
wgrantExcellent.04:42
wgrantWhat.04:46
wgrantWhy does lib/lp/services/features/templates/feature-info.pt have several <p>s without </p>s?04:46
wgrantThis is not the 1990s :(04:47
wgrantWhat :/04:48
poolie:)04:48
wgrantWhat.04:49
wgrantIf I attempt to close one of the <p>s after another element, TAL complains the <p> is not open.04:49
pooliei wonder if tal has special inferences for it?04:49
wgrantHmmm.04:52
wgrantPARA_LEVEL_HTML_TAGS = frozenset([ # List of HTML elements that close open paragraph-level elements # and are themselves paragraph-level. "h1", "h2", "h3", "h4", "h5", "h6", "p", ])04:54
wgrantXHTML does not work that way, TAL...04:54
mwhudsonchameleon ftw04:56
wgrantSo it uses XML namespaces heavily, but apparently does not use an XML parser.04:56
wgrantHow odd.04:56
huwshimiwgrant: Those <p>s look like they shouldn't be there anyway04:57
wgranthuwshimi: I suspect they are there to space it nicely. But I am hoping to remove them.04:57
wgrantFor now they are <p/>s instead.04:57
wgrantI am also tempted to delete zope.tal in retaliation for it being braindead, but that may be slightly excessive.04:58
huwshimiwgrant: Well they should be at least after the <h2>s and certainly not wrapping the <table>04:58
wgranthuwshimi: Right.04:58
wgrantI initially presumed they were wrapping the h2 and text.04:58
wgrantBut the h2 closes them immediately.04:58
wgrantBecause pre-XHTML HTML is cool.04:59
huwshimiwgrant: It's nice that it closes those tags for us, but not nice in that it doesn't help us write good html05:01
wgrantNo, it's not nice that it closes them at all!05:02
huwshimiwgrant: What do you mean?05:03
wgrantIt makes us write something that looks like an XML document and makes us specify namespaces and all that. And then treats the document like something that is not XML.05:04
wgrantThat is not useful.05:04
wgrantIt is stupid :/05:04
wgrantI guess it may have made sense when it was written.05:06
huwshimiwgrant: Right, so it's trying to be smart in a way that is unhelpful. It is good however that we don't deploy broken html pages.05:06
wgranthuwshimi: Ah, but a real XML parser would crash, not render broken XHTML.05:08
lifelesshmm05:09
lifelesswhats a good bug to look at that has synchronising bugwatches ?05:09
wgrantOnes that work?05:10
wgrantBug 184131 has two.05:10
_mup_Bug #184131: Apache 2.2 SNI support <Apache2 Web Server:Fix Released> <apache2 (Ubuntu):Fix Released> <apache2 (Debian):New> < https://launchpad.net/bugs/184131 >05:10
wgrantExcept the debbugs one is broken at the moment.05:10
huwshimiwgrant: It would be good if it broken depending on the spec (i.e. it breaks when we don't add alt tags to images etc.)05:10
lifelesstrying to see why we process bug watched messages05:11
lifeless(at all)05:11
wgrantlifeless: Ah, you want a bug with synced comments?05:11
lifelessyes05:11
lifelessbuild_comments_from_chunks is -bong-05:11
lifelessAFAICT our templates don't use half the work it does05:11
stubI suspect it is not using a real XML parser because they are slow, and Chameleon's very reason for existence is because someone got sick of ZPT being slow.05:13
wgrantlifeless: Bug #476397 has several imported comments.05:16
_mup_Bug #476397: does not display correctly an inkscape pattern fill <apport-bug> <i386> <Poppler:Confirmed> <poppler (Ubuntu):Triaged by desktop-bugs> < https://launchpad.net/bugs/476397 >05:16
lifelessah05:17
lifelessright05:17
lifelesswgrant: thanks, very helpful05:20
wgrantlifeless: You'd not seen that feature before?05:20
lifelesswgrant: I was trying to figure out if the loop on imported bug messages was global state for the template or local to the comment05:22
lifelessit was local to the comment, so we can skip the size(bug messages) lookup entirely.05:22
lifelessalso05:22
lifelessbugtask:+index is doing lazy evaluation *of the bug watches*05:22
lifelessand the irony is05:22
lifelessits doing that in a region of code with a comment saying 'we want to avoid one query per item in the for loop'.05:23
* lifeless facewalls05:23
wgrantHaha.05:23
lifelesscheck out build_comments_from_chunks if you want to see the evidence05:24
lifelessstub: hi; there was something I wanted your advice on, some less-normalisation I think.05:24
lifelessstub: reckon yoiu could spare a few minutes to look at this?05:25
stubSure05:25
lifelessits uhm, forget the bug number - binarypackagerelease.packagename though05:25
lifelessforces materialisation to sort archive pages05:25
wgrantbinarypackagename and sourcepackagename are two normalisations that I don't really agree with.05:28
lifelessi'm just trying to find the bug, without luck05:28
wgrantgetPublishedBinaries?05:28
wgrantThat one?05:28
lifelessbug 70620005:28
_mup_Bug #706200: Archive:EntryResource:getPublishedBinaries timeouts <qa-ok> <timeout> <Launchpad itself:Triaged by julian-edwards> < https://launchpad.net/bugs/706200 >05:28
wgrantYeah.05:29
stubI'd happily drop the *PackageName tables - never liked them.05:30
wgrantHah.05:30
lifeless+105:32
wgrantJulian wasn't precisely happy with them last time I talked to him about this.05:33
wgrantSo it sounds like everyone agrees.05:33
stubI wouldn't even call it denormalization since the *PackageName tables contain a single value and are write once - not 3NF as implemented by SQL databases.05:35
stubOh - I recall the rationale I think...05:36
wgranthuwshimi: I can see why the <p>s are there.05:37
wgranthuwshimi: Removing them leaves too little space between a table and the following h2.05:37
huwshimiwgrant: Oh yeah?05:37
wgranth2 needs more like margin-top: 1em05:38
wgrantIt now has 0.3em.05:38
stubThere was no SourcePackage or BinaryPackage tables, so *PackageName was there to ensure via referential integrity that bugs could only be targetted to valid packages. It is pretty weak, but I wasn't able to argue against it well enough back then.05:38
huwshimiwgrant: Is that a general thing? I'm not sure where we use h2s currently05:38
wgranthuwshimi: I'm not sure where else we use them either.05:38
wgrantstub: That's completely invalid.05:39
wgrantstub: So we can throw them out, it seems.05:39
stubCould be, or I could be recalling wrong. It was a subject of much debate anyway, and I lost.05:39
wgrantstub: It's a very plausible reason.05:40
wgrantIt's just also completely invalid.05:40
lifelesswgrant: bah, indices exist for a reason05:40
stubActually, a lot of our indexes don't exist for a reason (any more). I'll be detecting and cleaning out the useless ones soon ;)05:40
lifeless\o/05:41
wgrantlifeless: Hm?05:41
wgranthuwshimi: Most of the existing h2s seem to be in portlets (that have their own padding) or surround div.actions (that have their own margin)05:41
wgrantlifeless: Where did I not use indices?05:43
wgrantHah.05:46
wgrantcode.recipes_enabled lies.05:46
wgrantIt is already empty|non-empty, not on|off as it says.05:46
wgrantIn fact I think they all are.05:48
* wgrant headdesks.05:48
wgrantHmm.05:55
wgrantmemcache defaults to enabled.05:55
wgrantThat's a bit awkward with the standardised boolean type.05:56
lifelesshmm05:58
lifelesswe need a 'slices to range query' helper05:59
lifeless'nother day05:59
wgrantlifeless: Huh?05:59
wgrantOh.05:59
wgrantYeah, that would be handy sometimes.05:59
wgrantlifeless: Do you have an opinion on the memcache feature? Should I detect the absence of a flag and default to on?06:00
wgrantI think that's probably best unless we have a pressing need for more global defaults.06:00
wgrantunless/until06:01
wgrantpoolie: What do you think?06:03
wgrantlifeless: What was the point of your indices comment? Have I missed an index somewhere?06:04
lifelesswgrant: hmm? unset is on at the moment because its a killswitch06:05
lifelesswgrant: it should stay that way.06:05
lifelesswgrant: I think you're making flags more complex and harder to use, if you're running into trouble.06:05
wgrantIt is not trouble. There are easy ways to do it. I just want to set a good standard.06:06
lifelesswgrant: so, its a killswitch, it should stay as one06:07
lifeless'normally on'06:07
wgrantRight.06:07
wgrantThe value on prod will need to change, but that is easy enough.06:08
* poolie looks06:08
pooliei think that making memcache behave as "if not set, on" just inline in the code is reasonable for now06:09
pooliei think "memcache" is a pretty broad name for a flag too06:09
poolieif it actually means tal caching or something06:09
wgrantThat's what I've done. I check if there is no flag, and default to on.06:09
pooliecool06:09
wgrantIt's actually not just TAL caching.06:10
wgrantIt's the whole client.06:10
poolieok06:10
wgranthttp://people.canonical.com/~wgrant/launchpad/feature-info-again.png06:10
wgrantIt turns out that most of the current https://launchpad.net/+feature-info is a lie :(06:11
pooliewgrant, i like your screenshot06:11
poolieyes, but a Bright Shining Lie06:12
wgranthuwshimi: Some <th>s seem to be black, others grey. Is this intentional?06:13
huwshimiwgrant: I don't think so06:13
pooliewgrant,  so again i think saying "this turns off memcache altogether" would be good too06:14
pooliebut we can get there06:14
wgranthuwshimi: Compare https://launchpad.net/+feature-info and https://code.launchpad.net/launchpad/+activereviews06:14
lifelessoh my06:14
lifelessthe ss queries on bugtask:+index are -huge-06:14
wgrantlifeless: ss?06:15
wgrantStrucsubs?06:15
poolie"process'" is odd grammar06:15
wgrantpoolie: Is it?06:15
wgrantIt's not mine, but it seems right to me.06:15
poolieprocess's? process?06:15
huwshimiwgrant: yeah that's wrong06:15
wgranthuwshimi: Missing thead?06:16
wgrantYeah.06:16
huwshimiwgrant: That's one way to solve it :)06:16
lifelesswgrant: yes06:17
lifelesswgrant: visit bug 2 in the same data with LP_DEBUG_SQL=106:17
pooliei guess it's a known bug that there seems to be a different acl for triaging while filing vs triaging after filing?06:17
wgrantpoolie: Not a bug.06:18
lifelessalso wasDescriptionModified is fugly06:18
wgrantpoolie: It was quite intentional at the time.06:18
lifelesspoolie: I wouldn't assume so, I think its surely a bug06:18
lifelesspossibly not filed.06:18
wgrantpoolie: Bug supervisors were meant to be able pre-triage their bugs, but normal users were to be discouraged from doing so.06:18
pooliewgrant, something about people being more likely to triage badly if it's on the initial form?06:19
pooliehm06:19
wgrantThis annoys me to no end.06:19
pooliethere's a weird mispattern here about it being good to make actions hard if some people will abuse them06:19
wgrantPeople are more likely to set to Confirmed if the initial form tells them that they can, I guess.06:19
poolietrue06:19
wgrantBut it is very inconvenient.06:19
pooliei suffer this on lp:kanban for instance06:19
lifelesssetting confirmed is irrelevant06:20
lifelessbecause confirmed should die in a fire06:20
wgrantlifeless: Yes.06:20
wgrantThere were proposals at UDS-Jaunty to remove it.06:20
pooliethat kind of gets into the new/confirmed/triaged thing: does confirmed mean "multilpe people reproduced this" or "someone knowledgeable reproduced it" or "a developer read it"?06:20
wgrantBut that never eventuated.06:20
poolie:)06:20
pooliei have a fire06:20
wgrantIt's too warm today to destroy things in fires.06:20
wgrantExcept for zope.tal.06:20
lifelesszomg06:28
lifelesswhats wrong with this06:28
lifelesstal:replace="view/visible_oldest_comments_for_display/count:len">23</span>06:28
wgrantpoolie: Any comments on http://people.canonical.com/~wgrant/launchpad/feature-info-again-again.png? I've hopefully improved the subscriptions.06:29
wgrantlifeless: What is count:len?06:29
lifelesswgrant: count is a sql query06:29
lifelessas in .count()06:29
lifelessIIRC my tal, :len is a formatter to say 'this is a length'. IMBW06:30
wgrantAh.06:30
lifelessbut - we're doing a sql query to figure out how many rows we told the sql engine to get for us.06:30
lifelessor something approximately like that06:30
wgrantAlthough it may be less than what we requested.06:30
lifelessbizarre06:30
* lifeless is willing to live with that discrepancy06:31
lifelessat least for a bit06:31
wgrantOh.06:31
wgrantIn that context.06:32
wgrantThe requested value should always be right, right?06:32
wgrantSince if there are fewer comments then we display all of them.06:32
lifelessright06:32
wgrantSo that is pointless.06:32
wgrantKill it.06:32
lifelessright06:33
lifelessanother related change06:37
lifelesscurrently the threshold is 'visible comments > limit'06:38
lifelessI am changing that to 'comments > limit' because 'visible comment' requires interpretation06:38
lifeless /sigh06:38
wgrantlifeless: You mean you have to ask the DB to look at every comment?06:39
pooliewgrant, i see you stripped off the warning about scopes not being used?06:40
poolieor is that just screenshot cropping?06:40
poolieit looks very wide but i assume it scales06:40
wgrantpoolie: Just screenshot cropping.06:40
lifelesswgrant: check out get_visible_comments in browser/bugtask.py06:40
poolieso memcache default=enabled is just pure documentation at this point?06:41
wgrantlifeless: I did not want to see that.06:41
wgrantpoolie: No.06:41
poolieie it's describing a policy embedded in the code, rather than actually setting the default?06:41
wgrantRight.06:41
pooliek06:41
wgrantI think once more things want defaults it may become more than just documentation.06:41
poolieright06:41
poolieit looks really nice06:41
pooliei wonder if we should make them sentence case to be consistent with the usual thing for ui and docstrings06:42
poolieanything in particular you want comments on?06:42
poolieoh while you're there you could link to the dev wiki about this, perhasp06:42
poolieso we can add more docs06:42
wgrantA good idea.06:42
wgrantOn both counts.06:42
wgrantWas mostly wondering if the descriptions are OK now.06:42
wgrantBut those comments are good too.06:42
wgrantThe memcache one would be easier to word as a negative :(06:43
wgrantpoolie: Hm, there's no really relevant page on the dev wiki, I don't think.06:44
pooliehm06:46
pooliei'll create https://dev.launchpad.net/Foundations/FeatureFlags as at least a stub06:46
pooliethere's a bug open asking for something beyond the lep06:46
wgrantFoundations is dead.06:47
pooliewhat would be a good URL?06:47
poolieit seems like there should be a /Guide/ or /Components/ or something06:48
wgrantI'd like a namespace, but I don't think we have a relevant one.06:48
poolieso although Foundations doesn't exist as a team, it still exists on the dev wiki home page06:50
pooliewe can always rename with a redirect later06:50
wgrantOK.06:50
poolie... unless someone has a better name now?06:50
wgrantI don't.06:50
pooliebug 69248006:52
_mup_Bug #692480: need better docs on how to use/test feature flags <feature-flags> <Launchpad itself:Triaged> < https://launchpad.net/bugs/692480 >06:52
pooliei may well be wrong but it looks like edge's api is 30% faster than lpnet06:52
wgrantpoolie: Less load.06:52
wgrantSo smaller queues.06:52
huwshimiBye people.06:52
pooliethat's measuring across many requests06:52
wgrantlifeless is trying to fix this.06:52
wgranthuwshimi: Night.06:52
pooliei'm surprised it's so high it shows up above the rtt06:52
lifeless2.5 seconds06:54
lifelesslast time I measured it06:54
lifelessits sneaking up06:55
lifelesswe've got 2 new 8 core appservers, a complete reconfig of the per appserver process thread count and an haproxy rearrangement to address this06:55
poolie\o/06:55
lifelesseta is probably 3-4 weeks06:55
lifelessI /think/ we'll last that long06:56
poolie2.5s... queue time before service?06:56
wgrantOverhead on edge is usually around 600ms.06:56
wgrantlpnet is rarely below 1.5s.06:56
wgrantIn my quick tests here.06:56
wgrantpoolie: Yes.06:56
lifelesspoolie: yes; just subtract the time to first byte and the reported render time.06:56
poolieouchy06:56
lifelesspoolie: (and rtt)06:56
pooliejust wondered why my attempted optimizations (which also included moving things to lpnet) seemed to be making things so much worse06:56
pooliesilly me06:56
lifelesswell, lpnet is where we want you06:57
wgrantHmmm, yeah, for the API that would reallllly suck.06:57
lifelessbecause those edge resources will make lpnet faster as we move them across06:57
poolieit makes kanban generation 30% slower overall06:57
poolieusure06:57
lifelessthe thing making edge faster isn't queuing so much06:57
cody-somervillehmmm... are you guys saying edge is faster currently then lpnet? :P06:57
lifelessas that we don't have as many bloated pages served on edge06:57
lifeless*that* drives the queue times06:57
lifelesscody-somerville: yes, and if you use it I will personally block your uid there06:58
lifeless(just saying)06:58
poolielifeless, what do you mean?06:58
wgrantSpeaking of which... in the debacle last night did we tell the complainers to use lpnet instead?06:58
lifelesswgrant: which debacle?06:59
* cody-somerville hardly ever uses his personal uid... just saying :P06:59
wgrantlifeless: The edge API incident.06:59
lifelesscody-somerville: :P06:59
pooliewhat happened?06:59
lifelesswgrant: I haven't read about that yet; we should.06:59
wgrantlifeless: edge was serving api.launchpad.net URLs for an hour.06:59
poolieoh, very nice of it to pitch in like that :)06:59
lifelesswgrant: (I mean I haven't read the detail .. I knew the outline)06:59
wgrantAh.06:59
wgrantWell, my edge config minimisation broke things because our vhost config is terrible :(07:00
lifelessyeah07:00
lifelesswgrant: as a data point, this is the sort of reason I was ignoring the edge config07:00
wgrantAlso because people don't remove cruft from configs, though.07:00
lifelesswgrant: nonzero chance of trouble, short term win, long term we're killing it.07:00
cody-somervilleYou guys should have like a duplicated setup for like staging this stuff :P07:00
wgrantIt would not have broken if people had removed cruft instead of trimming it.07:00
poolielifeless, so the difference is basically just that there's less load on edge relative to it's hardware07:01
poolie?07:01
lifelesscody-somerville: we do, but the config is the thing that is different07:01
lifelesspoolie: shorter requests are more common on edge07:01
pooliehuh07:01
lifeless99th percentile on lpnet is 3s, on edge its 2.something07:01
wgrantlifeless: Oh, really?"07:01
wgrantDidn't know that.07:01
lifelessapi's 99th percentile is 2s across both07:01
pooliebecaues it gets relatively more api load?07:02
lifelessedge has slower servers07:02
wgrantSince the reports were merged it's less easy to check.07:02
lifelessthats my current theory.07:02
lifelessnot that I really care; its going bye bye07:02
cody-somervilleIf launchpad has no load, how fast would it be?07:02
cody-somerville*had07:02
* lifeless sobs /home/robertc/launchpad/lp-branches/working/lib/lp/bugs/browser/bugtask.py(986)wasDescriptionModified()07:02
wgrantlifeless: Heh heh heh.07:02
lifelesscody-somerville: about 1.5 to 2 seconds faster on every page07:03
lifelesscody-somerville: you're seeing the render times on pages now, right ?07:03
cody-somervillewell... 'query/external actions issued' time07:04
lifelessthats the render time07:04
lifelesson *top* of that there is07:04
lifelessRTT07:04
lifelesstcp window opening07:04
lifelessSSL07:04
lifelessin datacentre queuing07:04
pooliewell, for this i think i have ssl keepalives07:04
lifelesspython GIL contributes maybe 25% to the render time07:04
poolieimbw07:04
lifelessin datacentre queuing is doing a variable 1.5-2 seconds07:05
lifelessno load would remove in datacentre queuing07:05
lifelessand the GIL contention issues07:05
lifelessslow sql would still be slow07:05
pooliepfft07:06
cody-somervilleI guess my question is... these seem like 'common' problems. How do other websites manage to have better performance? Is it a code issue or are we just need investing the cash to have a powerful enough 'cloud' to serve lp as fast?07:08
cody-somervilleerr... s/need/not/07:08
wgrantThe DC overhead fix is scheduled.07:08
wgrantThe GIL contention will also soon be minimised by these same changes.07:08
cody-somervilleLP faster would make LP like 50% better. Folks would have so much more tolerance for other issues. At least thats my theory.07:09
wgrantIt's mostly a matter of people not really caring about performance for 5 years, and it's taking a while to fix it.07:09
lifelesscody-somerville: I agree; and its on the basis that we're buying the new front end servers etc.07:09
wgrantcody-somerville: Definitely.07:09
lifelesswgrant: well, folk gave a crap, but I think the tools in use, and the particular sorts of analysis needed to solve performance were challenging for the team.07:10
lifelesswgrant: there are places in LP that have been heavily optimised along one or two axis07:11
wgrantlifeless: Sure.07:11
cody-somervilleI don't think folks really recognized how big of an impact performance had on the overall perception of the service - in the same fashion that in the early days folks didn't recognize the impact the UI had on the overall impression.07:12
lifelessI think thats totally untrue07:12
cody-somervillemaybe thats just a personal reflection07:12
lifeless*every* current and emeritus LP dev I know cares passionately about performance and UI07:12
lifelesson the UI side there was a bunch of conflicting directives07:13
lifelessthe distro wanted bugzilla07:13
lifelessin terms of UI, not actual bugzilla07:13
lifelessMark wanted what he'd now call crisp and clean07:13
cody-somervilleYea... but thats relatively 'recent'07:13
lifelesslots of churn resulted07:13
* cody-somerville remembers launchpad from one or two 'ui refreshes' ago.07:14
lifelesscody-somerville: on day one he wanted the same basic outcome; the language for describing it has evolved.07:14
cody-somervilleah07:14
lifelessanyhow07:14
lifelessthere is a /tonne/ of discipline needed to make the site fast07:14
lifelessand zope is not intrinsically kind - zope is optimised for same-machine ~0 latency DB calls.07:15
lifelesswhich we don't have07:15
cody-somervilleUgh... omg... I cant believe the snow plows are already starting! Its only 3am!!! :(07:15
cody-somervilleerrr...07:15
cody-somerville2am07:15
LPCIBotProject devel build (442): STILL FAILING in 5 hr 40 min: https://hudson.wedontsleep.org/job/devel/442/07:23
pooliewgrant, bug 71918007:36
_mup_Bug #719180: need central defaults for flags <feature-flags> <Launchpad itself:Triaged> < https://launchpad.net/bugs/719180 >07:36
poolienot sure if that should be higher07:36
wgrantIt's not hard, and there is a squad assigned to take the remaining high-priority FF bugs. So perhaps.07:37
wgrantThat may be a Strategist question, I'm not sure.07:38
lifelesswgrant: there isn't one, is there? it got close off07:41
wgrantlifeless: The remaining bugs were thrown at Green this morning.07:41
lifelessoh, nice.07:42
wgrantThat's why I was working on it.07:43
lifelessisn't green on maintenance?07:43
wgrantYes.07:43
* lifeless twitches07:43
lifelesswho threw them at you?07:43
wgrantlifeless: The Strategist hiimself.07:44
wgrantThere were only two or so.07:44
pooliealso https://bugs.launchpad.net/launchpad/+bug/71918207:44
_mup_Bug #719182: boolean interpretation for feature flags <feature-flags> <Launchpad itself:Triaged> < https://launchpad.net/bugs/719182 >07:44
wgrantAnd it within a maintenance squad's mandate to finish off bugs resulting from a feature -- normally it would be the squad that did the feature initially, but that's obviously not possible here.07:45
pooliewhat else should go into https://dev.launchpad.net/Foundations/FeatureFlags#preview07:51
wgrantpoolie: That looks excellent.07:53
wgrantParticularly the conventions bit :)07:53
pooliethanks :)07:54
pooliejust going to fill out the two blank sections of course07:54
lifelesswgrant: kk08:00
pooliewgrant, also hard_timeout, contrary to its docs, seems to be in milliseconds08:02
poolienot seconds08:02
wgrantpoolie: Hah, so it is.08:03
* wgrant fixes docs.08:03
poolielifeless,  did you add server.?08:04
poolieas a scope08:04
poolieany particular reason it uses a dot rather than a colon?08:04
* wgrant vanishes for a while.08:06
wgrantThanks poolie, lifeless.08:06
pooliethank you; have a good night08:06
lifelesspoolie: you added it08:07
pooliehah08:07
poolieok08:07
lifelesspoolie: if I may, -1 on colons08:07
poolieso team.admins ?08:07
lifelesserm08:08
lifelesson flags, not scopes08:08
* lifeless confuses the issue08:08
poolieoh, yeah, i agree with that08:08
lifelesslet me start over08:08
pooliewas writing about scopes08:08
lifelessso, server. was what you had08:08
lifelessI left it as is when doing team: because of diminishing returns etc08:08
poolieistm colon makes sense for scopes because it's followed by user parameters, but dots are better for flags which are more like tokens08:11
pooliebut i mostly want to document stuff that will avoid unnecessary inconsistency08:11
lifelesscool08:12
poolieto make things faster maybe you can read <https://dev.launchpad.net/Foundations/FeatureFlags#preview> and tell me if you either disagree with anything, or think anything ought to be there08:12
lifelessI may have seemed grumpy earlier; I'm glad folk care about this08:13
lifelessI think there is a balance though08:13
lifelesswhat happened to the LEP ?08:13
poolieit still exists08:14
* lifeless thought we'd agreed to clean it up and keep it current08:14
pooliebut people complained that it had too much speculative stuff08:14
poolieoh, i thought we agreed we would add separate documentation08:14
lifelessright, which I have a note here to go delete.08:14
pooliei think "here's what you need to know to use it" is separate from "here's what we thought about in writing it"08:15
pooliethe LEP still has some ideas we could add later08:15
pooliethey link to each other08:15
lifelessthere is a risk in wikis of ending up with a link farm08:15
lifelessI think this runs very close to that08:15
poolietrue08:15
lifelessanyhow the prose is fine08:15
pooliewe could certainly shove some of the LEP into LEP/ff/Future08:15
poolieor /Archive08:15
lifelessit shouldn't really be in 'Foundations' as those team silos are gone08:16
poolieas i said i would be happy to see the dev wiki get a /Guide namespace or something08:16
pooliewe can rename and redirect08:16
pooliei just wanted to get _some_ developer-oriented docs08:17
lifelesssure08:17
lifelessany as I say, the prose there seems fine, both content and style08:17
poolieif you want to take a view of what the dev wiki should look like that would be great08:17
pooliecool, thanks08:18
lifelessPersonally, I would expect to find in a LEP enough useful stuff to use something we built08:18
lifelessmuch like one does in a PEP08:18
pooliewell08:18
poolieeventually the PEPs are redundantly documented in What's New and in the actual user manual08:18
poolieso it's kind of analogous08:19
lifelesswgrant: /count:len means 'call len() on the thing and show it as a count' as far as I can tell..08:19
poolieoh, https://bugs.launchpad.net/launchpad/+bug/692480/comments/108:19
_mup_Bug #692480: need better docs on how to use/test feature flags <feature-flags> <Launchpad itself:Triaged> < https://launchpad.net/bugs/692480 >08:19
pooliemaybe i should just move it to /FeatureFlags now?08:20
lifelesspoolie: when I need to know about context managers, I read the context manager PEP; I think if we're not going to delete the LEPs the same should be true.08:20
lifelessyes, /FeatureFlags would be a reasonable home08:21
poolieso i would probably start with the user manual and drill down to the pep08:21
poolieperhaps we should make it part of the lep process that by the time it's implemented, the lep is up to date08:21
poolieyour impatience before was good08:23
pooliei just heard some frustration with the lack of concise docs at the epic, so i felt i ought to add some08:23
lifelessthank you08:23
lifelessuhm08:24
jtv-afkMorning folks.  Still massive test fail?08:24
lifelessI'd roll the testing one into the main page for it I think08:24
=== jtv-afk is now known as jtv
poolieyeah, that would be good08:24
lifelessjtv: nope08:24
poolie"This page is about SOMETHING. More details can be found in CategoryCategory."08:24
pooliegood to know it's about something08:24
poolielifeless, ok testing stuff rolled in08:28
poolieyou're probably right they should merge actually08:29
pooliewith future stuff or background shunted off into appendices08:29
wgrantpoolie: Shall I drop the /Foundations from the link?08:29
lifelesspoolie: its sounding nicer and nicer08:30
lifeless\o/08:30
lifelesspartial loading bugtask:+index in place08:30
wgrantlifeless: Just messages for now?08:31
lifelesswgrant: its still loading all activities, yes.08:31
wgrant:(08:32
wgrantStill, not so bad.08:32
lifelessalso found some lovely bugs08:33
lifelessdid you know a bug import change the 'original description' for a bug ?08:34
wgrantYup.08:34
wgrantThere's a bug for that.08:34
lifelesswell, fixed.08:34
jtvlifeless: the same tests that were failing in ec2 are also failing for me locally, even in devel.  So why not still massive test fail?08:34
lifelessjtv: buildbot is green08:34
wgrantlifeless: Since the index is explicit now?08:34
lifelesswgrant: yes08:35
wgrantYay.08:35
pooliewgrant, yes08:35
poolieyes drop /Foundations/ from the link08:36
wgrantAlready done.08:36
wgrantThanks.08:36
jtvlifeless: rf-setup seems to have fixed most failures, but not all08:39
lifelessjtv: what failures are you seeing?08:43
jtvlifeless: in ec2test.tests.test_remote, a bunch of cases where a human-readable string was expected and something crypt-y comes out instead ("VnA5Gz…") and one where "utf-8" was expected on an email header but "utf8" was received.08:44
jtvAnd perhaps more, but there's rather a lot of output.08:45
jtv7 failures in that test (on devel)08:45
wgrantjtv: Are you on natty?08:45
jtvMaverick still.08:45
wgrantHmm.08:45
jtvI've been seeing these over the past 7 hours or so.08:46
jtvOn EC2, I even saw the number of failures in that particular test increase over that time.08:46
wgrantHmm.08:46
poolienight08:54
lifelessjtv: sorry, no insights here; buildbot is happy though, which is (a) reference platform08:55
jtvIs nobody else seeing failures from ec2test.tests.test_remote?08:57
wgrantNot I.08:57
wgrantI have had two ec2 runs today.08:57
wgrantOne failed because lifeless' testfix bounced, one succeeded after that landed.08:57
jtvAnd mine failed in the same way that devel failed locally.  :(08:58
jtvIt's as if email is encrypted but the tests expect it not to be.08:59
adeuringgood morning08:59
jtvhi adeuring08:59
wgrantjtv: passes fine for me locally.08:59
wgrantjtv: Checked your bzr config?08:59
bigjoolsmorning all08:59
wgrantjtv: That is copied to the ec2 slave.08:59
jtvwgrant: I wouldn't know what to look for.  I didn't mess with it recently.08:59
jtvhi bigjools08:59
jtvwgrant: there's a gpg_signing_command there09:00
jtvand create_signatures = always09:00
wgrantI don't sign my revisions, although I probably should.09:01
wgrantI suppose it's possible that's relevant.09:01
jtvI thought that was required setup.09:01
wgrantClearly not :)09:01
stubjtv: Sounds like Python mail library issues.09:02
wgrantI have seen the utf8 vs utf-8 issue before.09:02
wgrantBut I do not remember in what circumstances.09:02
stubjtv: In particular, I'd guess the 'encrypted' message is base64. We have a monkey patch lurking somewhere that is supposed to change the default from base64 to quoted printable.09:03
wgrantAha.09:03
stubAnd the preferred character set to UTF-809:03
jtvThat sounds like it would help these tests… but why did this start failing for me so recently?09:03
stubYou switched to Python 2.7 or something?09:04
jtvNot that I know.09:04
jtvDiscovery: the failures I've been getting until 8 hours ago were actually completely different ones than the ones I've been seeing since then.09:04
stubjtv: You get this on a fresh branch?09:05
jtvdevel09:05
jtvalso, on EC2 with a branch that was fresh yesterday.09:05
jmlgood morning09:06
lifelesshi jml09:06
jtvmorning jml09:07
wgrantEvening jml.09:07
bigjoolshelleau jml09:08
stubjtv: So my guess is that it is something to do with test ordering or which subprocess layer is being run.09:08
jtvstub: I'll try running just one failing test in isolation then.09:09
jtvThat fails too.09:09
stubjtv: With tests being run that haven't imported canonical.launchpad.mail or lp.services.mail.sendmail failing because the monkey patch hasn't installed.09:09
stubStick an 'import lp.services.mail.sendmail' at the top of the test's .py file?09:09
* jtv tries09:09
lifelessit might be useful to have something run over all our tests and run them one by one in a new process; file bugs on any that fail.09:11
jtvstub: yup, that fixes it!09:11
stubdel Charset.CHARSETS['utf-8']09:11
stubCharset.add_charset('utf-8', Charset.SHORTEST, Charset.QP, 'utf-8')09:11
stubCharset.add_alias('utf8', 'utf-8')09:11
jmllifeless: wouldn't be too hard to do. I suspect some of our tests would behave better with such an runner09:12
jtvstub: that's in that module?09:12
stubSo those lines and associated comments and imports needs to move to lib/lp_sitecustomize.py. I think that is the 'correct' solution ('correct' is an odd term to use when talking about monkey patches)09:12
jtvstub: strange that this should affect me so selectively… have I got the wrong locale configured or something?09:16
stubjtv: Are you running a subset of tests or the complete suite? If you run a subset of tests, you will be running things in a different order.09:17
jtvstub: it happens either way.09:17
stubDunno then :)09:17
stubTests should be run in ascii order IIRC, and the layer forking should be deterministic.09:17
stubI guess the tests might run in locale specific order... which could give weird results.09:20
stubI feel old whenever I see my old comments, dated 2005...09:21
=== elmo_away is now known as elmo
stuband 2004 in the module docstring...09:23
* StevenK finally gets home.09:26
StevenKFail, Virgin Blue09:26
lifelessStevenK: grats09:29
wgrantbigjools: That's going to be *quite* a branch.09:50
jmlremove SourcePackageName and BinaryPackageName?09:52
lifelesssourcepackagename may be tricky to remove09:52
adeuringhenninge: could you have a looka t this mp: https://code.launchpad.net/~adeuring/launchpad/bug-702468/+merge/49205 ?09:52
jmllifeless: you think?09:52
jml(yes, it will be tricky to remove)09:52
wgrantlifeless: It is referenced in a few more places. But it's not fundamentally significantly harder.09:53
wgrantJust a few more tables.09:53
lifelesswgrant: package branches09:53
wgrantlifeless: Hm?09:53
wgrantNo harder than bugs.09:53
lifelesspersonally, I'd love for that constraint to be fixed, its a showstopper for turning off dput09:53
lifelesswgrant: we constrain the namespace of package branches to existing sourcepackagenames.09:54
wgrantlifeless: That's only because it's easier.09:54
jmlwgrant's right.09:54
lifelesseasier than what?09:55
jmllifeless: easier than implementing it not constrained09:55
jmllifeless: at the time09:55
=== gmb changed the topic of #launchpad-dev to: :
gmbAaaaah09:58
wgrantSuccinct!09:58
gmbBotheration.09:58
gmbIf someone's got a recent topic line, now's the time to save the day.09:58
=== wgrant changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | firefighting: - | On call reviewer: - | https://code.launchpad.net/launchpad-project/+activereviews
henningeadeuring: of course! I was already looking at it yesterday but things got in the way. Sorry.09:59
adeuringhenninge: no problem10:00
gmbwgrant: Thankyou kindly.10:00
=== gmb changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | firefighting: - | On call reviewer: gmb | https://code.launchpad.net/launchpad-project/+activereviews
wgranthenninge: Thanks for reviewing that branch.10:00
henningewgrant: pleasure ;)10:01
bigjoolswgrant: yeah I was being slightly tongue-in-cheek in the bug comment.10:02
bigjoolssarcasm doesn't translate easyily to the written word10:03
bigjoolsand my typing doesn't help10:03
wgrantCome on qastaging, restore quicker.10:03
allenapI would like to increase the hard time-out for Person:+delete so I can delete a team. Should I do that, or should I fix the bug first (which is probably very hard)?10:10
wgrant~ubuntu-main-sponsors?10:11
stubSo the packaging table allows distroseries to be NULL, although there are now rows in the database with a NULL distroseries.10:11
allenapwgrant: Yes :)10:12
wgrantallenap: Good luck with that.10:12
stubShould this column be NOT NULL?10:12
wgrantallenap: It could have many thousands of bugsubscriptions.10:12
allenapwgrant: Yeah, that's what I was afraid of.10:12
stub(I have two indexes on (distroseries,sourcepackagename), and only one is correct)10:13
stubbigjools: ^^10:13
wgrantstub: That's a Registry thing.10:13
stubTa10:13
wgrantstub: I'm pretty sure our code won't handle it if distroseries IS NULL.10:13
* bigjools reads10:13
wgrantCertainly nothing uses it.10:13
wgrantOr creates them like that.10:13
wgrantBecause it means nothing.10:13
* bigjools always seems to get pinged when "packaging" is involved :)10:14
lifelessallenap: you should fix the bug10:15
lifelessallenap: there is no guarantee that even a 20 sec timeout would be sufficient10:16
henningeadeuring: r=me with just a little comment. Thanks!10:17
allenaplifeless: Gah. Bugger. Okay, I try and figure out what it'll take.10:17
lifelessallenap: bug 16251010:18
_mup_Bug #162510: Person:+delete timeouts : Person merging needs to be done asynchronously <canonical-losa-lp> <chr> <feature> <lp-registry> <merge-deactivate> <tech-debt> <timeout> <Launchpad itself:Triaged> < https://launchpad.net/bugs/162510 >10:18
lifelessallenap: its half done already.10:18
allenaplifeless: Awesome, thank you.10:18
lifelessjml: got 5 ?10:22
StevenKlifeless: Actually package branches don't use SourcePackageName directly, but SourcePackage10:22
jmllifeless: yeah. just let me capture this thought.10:22
wgrantStevenK: SourcePackage doesn't exist.10:23
jtvdanilos, henninge: bug 71924710:23
_mup_Bug #719247: Translations branch auto-approver not working <Launchpad itself:Triaged> < https://launchpad.net/bugs/719247 >10:23
wgrantStevenK: It is a Python myth.10:23
danilosjtv, yay?10:23
jtvdanilos: not yay10:23
StevenKwgrant: ISourcePackage, lib/lp/registry/interfaces/sourcepackage.py10:23
jtvwgrant: you lie—I saw a SourcePackage once and if my camera hadn't broken down just then…10:24
adeuringhenninge: thanks!10:24
henningeadeuring: np10:24
henningejtv: We need to replace the approver.10:24
wgrantStevenK: Yes, a Python myth. Goes nowhere near the DB, and most of this work is going to be in the DB interface code.10:24
danilosjtv, any indication as to what might be the problem?10:25
jtvdanilos: none whatsoever10:25
jtvhenninge: this sounds like it may be an operational issue though10:25
henningejtv: without looking further into it, I remember that the auto-approver for the branch imports was too careful.10:26
jtvhenninge: this was a simple single-template setup though10:26
jmllifeless: skype me when you are ready10:26
henningehm10:26
jmlwgrant: that's backwards, imo.10:26
* henninge looks closer10:26
wgrantjml: Yes, but so are lots of things.10:26
wgrant:(10:27
wgrantDistributionSourcePackage is both!10:27
wgrantDistributionSourcePackageInDatabase, yay.10:27
gmbbigjools: Is cron.germinate Soyuz territory?10:28
bigjoolsgmb: yes10:28
bigjoolsit's maintained by the ubuntu platform guys though10:29
wgrantRiddell's change is fine.10:29
henningejtv: that is odd indeed10:29
gmbwgrant: Thanks :)10:29
gmbThat's what I was asking about.10:29
wgrantgmb: Platform does just about all work on it.10:30
wgrantBut it lives in our tree so we must wave them through.10:30
gmbRighto; duly noted. Thanks again.10:30
jtvhenninge: sbackup looks like it may be having the same problem since september.10:30
gmbdanilos, jtv, henninge: I'm looking at abentley's translations branch and, whilst it's small, I have no context on it to know whether it makes sense or not. Can one of you guys take a look?10:35
gmbhttps://code.launchpad.net/~abentley/launchpad/fix-undiverging/+merge/4971810:35
jtvgmb: I can10:35
gmbjtv: Bedankt.10:35
jtvGraag gedaan10:35
jtv(prononounce each "g" as if it were your last)10:35
gmbHah.10:36
jtvNo, more like khh10:36
jtvCheck your keyboard.  If the spittle is visible there, you're doing it right.10:36
gmb:)10:36
henningeadeuring: If you still have a chance to fix it: The "Comment instead of docstring" exception is only for actual tests (methods that start with test_). SelectUpstreamTranslation should have a """docstring""" not  a # Comment. ;-)10:41
adeuringhenninge: ah, right... but I'va already started "ec2 land"...10:42
henningeadeuring: if it fails, you'll have your chance ... otherwise just include that in one of your next branches.10:42
henninge;-)10:42
jtvgmb: done10:44
gmbjtv: Thanks10:45
henningejtv: You wrote: The script requires a distroseries to be specified among the options, but assumes a distroseries to be specified.  There may be a way to specify to optparse that a particular option is required10:52
jtvhenninge: I meant it assumes a distro to be specified.10:53
henningejtv: ah10:53
jtvAnd source package, IIRC10:53
henningejtv: it defaults to "ubuntu"10:53
henningevia optparse10:53
jtvI see10:53
henningeno, source package is checked10:53
StevenKOSError: [Errno 2] No such file or directory: './lib/canonical/launchpad/icing/yui/event/event-base-ie-min.js'10:54
StevenKDuring 'make'10:54
wgrantStevenK: rm -r lazr-js10:54
wgrantmake clean may do that.10:55
wgrantBut I just remove it manually.10:55
StevenKwgrant: Ta.10:55
wgrantIt happens occasionally on YUI upgrades.10:55
=== _starbuck-gone is now known as _starbuck
wgrantallenap: Do you have any ideas for getting rid of all the UnknownRemoteImportanceErrors and "ExternalBugtracker for BugTrackerType 'FOO' is not known." errors?10:59
lifelessgmb: hi11:01
lifelessgmb: I think bug 719249 is a risky way to go11:01
_mup_Bug #719249: The new direct subscription overlay takes too long to load <story-better-bug-notification> <Launchpad itself:Triaged> < https://launchpad.net/bugs/719249 >11:01
lifelessgmb: most of the time most users won't be changing subscriptions, by preloading you are going to pay for that on *every* page load.11:02
allenapwgrant: Get rid of them entirely, or from the OOPS reports? For the latter I think they should be added to (a|the) blacklist.11:04
lifelessallenap: if its in a blacklist, its pointless overhead to write an oops11:05
lifelessallenap: log it to the log file and move on - you don't need the http request count, backtrace, etc etc for these.11:05
allenaplifeless: Okay. wgrant, I'll file a bug for lifeless's suggestion.11:06
wgrantThanks!11:06
lifelessallenap: rule of thumb: if a *human* should do something *now*, we record an OOPS.11:08
lifelessallenap: otherwise, we shouldn't.11:08
allenaplifeless: Noted :)11:09
=== matsubara-afk is now known as matsubara
gmblifeless: Hmm. I see your point, but since we load all the portlets and initialise all the JS for the subscription links asynchronously already I'm not sure I entirely agree that it's risky.11:18
wgrantjtv: Hi.11:22
jtvhi wgrant11:22
bacgmb: i've just put up a MP but am not really here today.  could you review it at your leisure?11:22
gmbbac: Sure thing.11:23
bacthx11:23
wgrantjtv: update-stats.py is now the second-slowest piece of nightly.sh. Looking at DistroSeriesLanguage.updateStatistics, it seems to grab every POFile then sum the results of methods that return a value directly from a DB column. Am I missing something here, or can that be replaced with one query to select the sums directly?11:24
wgrantjtv: Same with the POTemplate iteration in DistroSeries.updateStatistics.11:26
jtvwgrant: we haven't looked at the DistroSeriesLanguage version of that for a while—as it happens I was just looking at POFile.updateStatistics performance and that seems to be quite fast now.11:27
jtvBut yes, I believe that's something we simply never had time to optimize.11:27
henningewgrant is the guy that screws booster rockets onto scripts ... ;-)11:28
wgranthenninge: I don't think I can do 1000x again, though :(11:28
henningewgrant: 100x is fine, too :-P11:28
wgrantjtv: I gather that those attributes are caches that are relatively new?11:28
jtvwgrant: not at all, but their mechanics and intricacies have changed11:29
wgrantjtv: Hmm.11:29
jtvThe attributes are positively ancient.11:29
wgrantIt just seemed odd to have methods that just wrap apparently cached values.11:29
wgrantI presumed they were historical relics.11:29
jtvThose methods are from ancient interfaces, though adi simplified them last year.11:30
wgrantBut there is no evil magic that prevents me from doing the sum in the DB?11:31
jtvThey're still nastily spread out through the code, unfortunately.11:31
jtvNo really evil magic, no, though you may run into some attributes that are computed from others.11:31
wgrantI don't believe there are any of those. But I will check.11:31
wgrantIt all just seemed too simple!11:31
jtvYes.11:32
jtvdanilos, henninge, this may be related to the translations branch approver problem: bug 71926711:33
_mup_Bug #719267: Translation template approval times out <timeout> <Launchpad itself:Triaged> < https://launchpad.net/bugs/719267 >11:33
=== jtv is now known as jtv-eat
lifelessgmb: ah, if its async its better, but still - unneeded load is waste11:38
lifelessgmb: do have a look see - and remember that tests against launchpad.net are currently pessimised due to being overcapacity11:39
gmbRight.11:39
jmlwhen are we getting more capacity?11:39
lifelessjml: shortly after we get losa cycles11:40
lifelesswe have 8 cores we can expand onto with11:40
lifeless'simple'11:40
lifelessreconfiguration11:40
lifelessand 12 more cores coming11:40
lifelesswe should knock ~ 1 second of in-data centre latency off immediately11:40
lifelessto test; hit a page up and note the render time shown11:41
lifelesscompare the time-to-first-byte in speed tracer of the same request11:41
lifelessthe difference is ssl + rtt + in-dc queuing11:41
lifelessnight all11:41
jmlg'night.11:41
bigjoolsa pox on python files with a - in their name11:50
jmlbigjools: indeed11:52
jmland also ones that can't be imported11:52
bigjoolsthat is my problem :/11:52
bigjoolstrying my best to write a test for something I changed that is totally untested right now11:53
jmlyeah.11:59
jmlbigjools: with the file formerly known as ec2test-remote.py, I renamed it and added some tests for the current behaviour in a branch before landing my behaviour change11:59
* bigjools is delving into buildd module territory where all bets are off12:00
deryckMorning, all.12:02
allenapMorning deryck :)12:02
=== Guest28688 is now known as NCommander
LPCIBotProject db-devel build (366): STILL FAILING in 5 hr 40 min: https://hudson.wedontsleep.org/job/db-devel/366/12:23
leonardrgmb, take a look at https://code.launchpad.net/~leonardr/launchpadlib/silently-replace-edge-with-production/+merge/4979412:25
=== leonardr changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | firefighting: - | On call reviewer: gmb, leonardr | https://code.launchpad.net/launchpad-project/+activereviews
LPCIBotProject devel build (443): STILL FAILING in 5 hr 16 min: https://hudson.wedontsleep.org/job/devel/443/12:39
LPCIBot* Launchpad Patch Queue Manager: [r=henninge][bug=198778,12:39
LPCIBot718004] Rewrite update-bugtask-targetnamecaches to use a set-based12:39
LPCIBotmethod, approximately 1000x faster.12:39
LPCIBot* Launchpad Patch Queue Manager: [r=sinzui][no-qa] Remove unused icing.12:39
LPCIBot* Launchpad Patch Queue Manager: [r=deryck, lifeless,12:39
LPCIBotwgrant][bug=673530][incr] Add a popup choice widget for the build12:39
LPCIBotfrequency.12:39
bigjoolsgmb: review ping please: https://code.launchpad.net/~julian-edwards/launchpad/buildd-regex-bug-719162/+merge/4979612:47
abentleyhenninge, deryck, adeuring: My internet is busted.  May not be able to Mumble.  Is POTS an option?13:00
deryckabentley, we could skype and dial a land line.13:01
abentleyderyck: logging off to preserve cell data...13:04
gmbleonardr, bigjools: Just got back from lunch; will peer owlishly at your MPs shortly.13:04
leonardrgreat13:05
leonardrbigjools: i can take yours13:05
gmbleonardr: edge_server_equivalent_string_becomes_production() and edge_web_server_equivalent_string_becomes_production() look like tests but they don't have the test_ prefix so (I think) the testrunner won't run them. Is that deliberate (or am I wrong)?13:10
leonardrgmb: it's not deliberate, let me check13:11
gmbleonardr: line 121 and 127 of the diff, for reference13:11
leonardrgmb: yeah, just adding test_ fixes the problem13:11
gmbCool13:12
gmbleonardr: r=me then.13:12
leonardrok13:12
=== jtv-eat is now known as jtv
leonardrbigjools, r=me13:30
salgadoleonardr, does launchpadlib mock Launchpad for any of its tests or do we still need a local Launchpad to run launchpadlib's test suite?13:47
leonardrsalgado: you still need a local launchpad13:47
leonardrlaunchpad has unit tests but they don't test that launchpad returns specific data13:48
leonardrthey test that launchpadlib behaves in a certain way13:48
leonardrer, launchpadlib has unit tests...13:48
salgadoright13:48
salgadothanks leonardr.  I was afraid that'd be the case13:49
leonardrnp. there is a project to make a mock launchpad but it's stalled afaik13:49
salgadoleonardr, do you know where I can find more info about it?13:49
leonardrsalgado: no, i don't remember who was doing it.13:50
james_wleonardr, do you mean the one on https://code.launchpad.net/launchpadlib/+activereviews ?13:52
leonardrlet's see...13:53
leonardryeah, that's the one.13:53
leonardri guess it's not stalled--there's recent activity13:54
=== henninge_ is now known as henninge
salgadowell, about 8 months ago was the last non-merge commit13:56
salgadoalthough there's review activity. :)13:57
bigjoolsleonardr: thanks13:57
=== almaisan-away is now known as al-maisan
danilosgmb, leonardr: hi guys, anyone want to take a look at my "split bugtask_index.js into two" branch?14:48
gmbdanilos: I'll take it14:48
gmbSince I set you on that path....14:49
danilosgmb, heh, excellent, thanks14:49
danilosgmb, https://code.launchpad.net/~danilo/launchpad/bugtask-index-portlet-setup/+merge/4981814:49
danilosgmb, diff is probably not there yet :)14:49
gmbRight :)14:49
gmbTime for me to get coffee14:50
henningejtv: can you help me with some SQL foo?14:50
danilosgmb, heh, yeah, the diff itself is >2k lines, but you know what that is14:50
jtvhenninge: sure14:51
gmbRight14:51
henningejtv: it might be a good idea to consolidate the input queue before re-enabling imports.14:51
jtvhenninge: consolidate how?14:52
henningejtv: dpm reckons there are been multiple uploads for a source package in the queue and we really only need the latest.14:53
henningejtv: so from all entries with the same sourcepackage and path, we could delete all but the latest.14:54
henningeI was just wondering if that could be done in SQL or if I need a script.14:54
jtvhenninge: don't they all have the same uploaders?14:55
henningejtv: dpm says not.14:55
jtvSometimes not?  Never?  Not a lot?14:55
henningegood questions.14:55
jtvBecause AFAICS that's the only duplication we're going to see: from multiple people uploading the same file.14:56
henningejtv: True14:56
jtvMeanwhile, unfortunately, it's become impossible to approve some templates.14:56
danilosjtv, :(14:56
henningejtv: I guess we could get some statistics on that14:56
jtvTimeout while running statistics for new POFiles.14:56
jtvHow ironic.14:57
henningewhy is that?14:57
bigjoolsallenap: thanks for fixing the buildd-manager14:57
jtvhenninge: we're both talking about statistics.14:57
allenapbigjools: No worries, though it's not in stable yet.14:57
henningehah!14:57
jtvhenninge: redundant uploads should normally go into the pre-existing queue entries—and that means all of them for package uploads.  Is there really a problem here?14:58
henningejtv: not sure15:01
henningejtv: I will see if I can put together a query to get some numbers on that.15:01
henningeafter lunch (well, tea really)15:02
=== henninge is now known as henninge-tea
jtvhenninge: one complication… if two uploads are by different people, who's to say they don't both contain valid, original data?15:02
henninge-teajtv: dpm says just the latest matters.15:03
henninge-teabut maybe he is wrong?15:03
=== matsubara is now known as matsubara-lunch
jtvWell I wonder how it's possible to tell, since it's different people.15:03
jtvYou'd have coordinate that with all the translation teams.15:04
jtv(the impersonal "you," not you as such)15:04
danilosjtv, for packaged uploads, even if they are from different people, we can safely ignore non-latest ones (in theory, that might still cause different behaviour, but doesn't matter in practice)15:09
jtvdanilos: ISWYM15:09
jtvEven so, I'd be interested to see how much it'd save us.15:09
danilosjtv, sure15:10
dpmhenninge-tea, jtv, my question was not whether the two uploaders upload valid data (which I assume they'd do). The question is whether, assuming one uploads valid translation data on Monday and the other one on Tuesday, why do we need to import the data from Monday, if it's the same (or contains less translations)?15:18
jamspm: ping, if you haven't killed the extra processes on crowberry yet, it would be good if we could do some debugging.15:18
jtvdpm: in the meanwhile we've been convinced of that, but our current counterquestion is: how many imports would it save us?15:19
dpmjtv, henninge-tea, anyway, you guys know the technical details better, that was just my brainstorming to make the disruption in translations shorter15:19
dpmjtv, I don't know :) But I'm guessing it would be a big saving in big uploads such as KDE15:19
dpmit's a matter of finding out how many "duplicate" uploads there have been since we stopped imports15:20
jtvdpm: the real disruption is the ongoing delays though—there's no disruption to non-Ubuntu imports, and for Ubuntu ones it may not (←guess, to be verified) make much of a difference now relative to the existing delay.15:20
jtvAlso bear in mind that duplicate uploads are processed much faster.15:20
dpmjtv, I'm talking about Ubuntu only, I know the other queues are not stopped15:21
dpmjtv, I didn't know duplicate uploads were processed faster, I understood from your comments that was only the case if they came from the same uploader15:22
jtvIf they come from the same uploader, then the new upload simply replaces the old one so only 1 is processed at all.15:22
jtvBut processing a msgstr that's identical to the current translation of that msgid is very fast.15:23
dpmjtv, anyway, I just thought I'd ask, I'll let you guys figure it out. The reason why I thought about this is because we did this once in the past for OO.o uploads (i.e. discard all but the latest upload), and it was my understanding that it did speed things up15:24
jtvdpm: it did, yes… we'll have a look to see if there's a similar effect.15:25
jtvOO.o would slow down anything.  :)15:25
dpm:)15:25
dpmthe same with KDE, it's a big import15:26
gmbdanilos: r=me with a couple of tweaks (one of which is a longstanding bug that I put there and never noticed ;)).15:35
danilosgmb, heh, sorry, I don't want to fix *your* stuff :P15:38
danilosgmb, anyway, thanks :)15:38
gmbHah.15:38
=== salgado is now known as salgado-lunch
danilosgmb, btw, since I am not too comfortable around JS, should it really be "var namespace.blah_blah = ..." or just "namespace.blah_blah = ..."?15:43
danilosgmb, (i.e. without the "var")15:43
gmbdanilos: Argh. Without the var.15:43
gmbI always make that mistake15:43
danilosgmb, heh, ok, thanks :)15:43
gmbAnd it takes me ages to debug it. Good catch.15:43
=== deryck is now known as deryck[lunch]
danilosgmb, for testing the advanced_subscription stuff, do I have to worry about malone-alpha memberships and such?15:49
gmbdanilos: Not on lp.dev, no.15:49
danilosgmb, cool, thanks15:51
=== matsubara-lunch is now known as matsubara
=== beuno is now known as beuno-lunch
sinzuiallenap: what do you intend to do about the merge/delete person timeout?16:10
allenapsinzui: I'm going to look at the code then, actually, talk to you about it :)16:11
allenapsinzui: I was going to see if there's a registry job already set up that I could piggy back on.16:11
sinzuiyep. That was Edwin's intentions16:11
allenapCool. I'll head in that direction.16:12
sinzuiallenap: I updated the question because the timeout is related to a second bug. The proc is transferring the bug subscriptions to ~registry. That is just wrong16:12
sinzuiallenap: They are separate issues. But I know how to remove the 196 bug subscriptions. Either dholbach can use the script I posted, or make ~registry the team owner so that I can clean up the bugs first16:14
=== salgado-lunch is now known as salgado
=== henninge-tea is now known as henninge
allenapsinzui: Is there are bug for "transfer to ~registry on delete"? I assume it should just unsubscribe.16:17
sinzuiallenap: bug 61360416:18
sinzuiwrong16:18
sinzuiallenap: bug 61360316:18
_mup_Bug #613603: team merge subscribes/assignes bugs to ~registry <lp-registry> <merge-deactivate> <Launchpad itself:Triaged> < https://launchpad.net/bugs/613603 >16:18
allenapsinzui: I was going to do a work-around like the script you suggest, but I'll try and get this bug fixed first... it would be nice to see it work.16:18
allenapsinzui: Thanks.16:20
sinzuiallenap: Edwin intended to use PersonTransferJob. The job could be membership change, person/team merging or person to person emails16:22
allenapCool.16:23
jcsackettcould someone help me figure out an lp.dev codehosting issue? i've run sync_branches but still can't get a branch scanned. :-/16:29
jcsackettah, got it. competing process hurting codehosting.16:40
allenapsinzui: It looks like a lot of the logic in AdminTeamMergeView should be folded into the new job, if not into Person.merge() itself.16:43
sinzuiallenap: oh, thanks for bringing that up16:43
sinzuiI think I reported a bug that the view logic can be pushed lower I was thinking the model, but the job may also be the right place. We may want to initiate a merge/delete over the api.16:44
* sinzui looks for bug16:44
sinzuiallenap: there is no bug. I think the conversation took place in an MP. Once we got the view logic cleaned up, it looked like we could finally get the rules into something beneath the view16:47
=== deryck[lunch] is now known as deryck
allenapsinzui: Yeah, I am kind of pointing out the obvious. I think I might move only AdminMergeBaseView.doMerge() into the job at first.16:49
sinzuiallenap: yes. Just do what is obviously needed. We have fumbled pushing the logic lower before. It has only been in a state that looked doable since December of last year16:50
allenapsinzui: Okay, thanks.16:50
jkakarFWIW, and maybe it's just from years of being used to the old, I'm finding the grey headings make things harder to see... again, could just be that I'm just very used to the old maroon, but this is my initial experience.16:52
sinzuijkakar: it is harder. huwshimi and I discussed it.16:53
sinzuijkakar: The colour was bad, but at least is hid the font-size and spacing issues that are *really* bad in Lp16:53
sinzuijkakar: I am doing a spike this evening to fix the font-size issue. If it works, huwshimi will do a followup to address heading16:54
jkakarsinzui: Awesome, thanks. :)16:56
jmljkakar: what about the old blue, orange, green etc?16:56
jmljkakar: is it just maroon that's easier to read?16:56
jkakarjml: I miss the red 'Report a bug' link.16:56
sinzuijkakar: is landscape using Canonical web guidelines16:56
jkakarSomethings are a bit weird... for example, the THEADs on the +activereview page are much stronger than the headings that describe what each section is, which feels backwards.16:57
sinzuijkakar: oh, yes, we discussed the participation links too. We are uncertain. We decided to try consistency first. We lost something though.16:57
sinzuijkakar: wow, it's like you are reading my reviews and emails to huwshimi16:58
jkakarsinzui: :)16:58
sinzuiyes the headers are odd, many are inconsistent because the engineer hacked the css/style rules16:58
sinzuiWe need to tear out a lot of exceptions in templates16:58
jkakarOne thing I thought, particularly from looking at a merge proposal, was, "Oh, they're trying to make user content jump out and be more prominent than all the chrome"... if that's the case, great, but I think it needs a bit more iteration. :)16:59
jkakarsinzui: Yeah, that kind of work is really tedious, but important.16:59
sinzuijkakar: it certainly does need iteration. Unlike other updates, we are not stopping releases for a few months. I think the font-size issue is becoming a blocker. Canonical Web guildlines have some good advice, but we do not know why px is used instead of ems.17:01
jkakarsinzui: I don't know if would help, but we've been through a lot of careful iteration and selection of colours in Landscape with the redesign we did last year.  It might be a place to look for ideas/inspiration.17:01
jkakarsinzui: I don't why they're used, either.17:02
jkakarsinzui: I think it's because designers use Photoshop and think naturally in terms of px's.  em's are a bit of a random measurement.17:02
jkakarsinzui: alejandra and yaili are good people to talk to about the guidelines, especially to get insight about questions like that.17:02
sinzuijkakar: thanks! I think there is something else at play. browser engines are now running on a many tiny and huge screens. I suspect the engines are using px and a baseline rather than absolute units for fonts.17:04
jkakarsinzui: They're also good people to report problems regarding the guidelines, since they basically maintain them.  We found a number of issues during our redesign that resulted in tweaks to the guidelines.17:05
sinzuifab, I do not think the bug I reported were seen17:06
=== gmb changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | firefighting: - | On call reviewer: leonardr | https://code.launchpad.net/launchpad-project/+activereviews
abentleyderyck, we need to have the inverse of translation merging ("splitting") for when a Packaging is deleted, so we should probably put that on Kanban.17:24
deryckabentley, ok.  Do you mind adding a bug and card for that?17:25
abentleyderyck, okay.17:25
deryckthanks!17:25
LPCIBotProject db-devel build (367): STILL FAILING in 5 hr 16 min: https://hudson.wedontsleep.org/job/db-devel/367/17:39
LPCIBotLaunchpad Patch Queue Manager: [rs=buildbot-poller] automatic merge from stable. Revisions: 12377,17:39
LPCIBot12378, 12379 included.17:39
=== leonardr is now known as leonardr-afk
jmljust wrapping up for the day, around for another 20 or so minutes.18:10
lifelessjml: hi18:11
jmllifeless: hi18:11
=== deryck_ is now known as deryck
LPCIBotProject devel build (444): STILL FAILING in 5 hr 36 min: https://hudson.wedontsleep.org/job/devel/444/18:16
LPCIBot* Launchpad Patch Queue Manager: [r=henninge][bug=718761] Ensure that NewBuildersScanner only detects18:16
LPCIBota new builder once. Previously,18:16
LPCIBotany builder registered after start-up would be detected as "new" every18:16
LPCIBot5 minutes.18:16
LPCIBot* Launchpad Patch Queue Manager: [r=adeuring][ui=none] [r=adeuring][bug=715802] Add a vestigial page18:16
LPCIBotfor the new Bug:+subscriptions view.18:16
LPCIBot* Launchpad Patch Queue Manager: [r=lifeless][no-qa] SQL statements logged in OOPs and ++profile++18:16
LPCIBotoutput include the value of their bind variables.18:16
LPCIBot* Launchpad Patch Queue Manager: [r=leonardr][bug=715854] Don't set filereferences when creating18:16
LPCIBotPOTMsgSets during translation import.18:16
jmlYou know, that bot would be better if it just showed the failing tests18:17
jmlrather than the merges18:17
jmlStevenK: LPCIBot should show the failing tests, rather than the merges.18:17
lifelessthats a great idea18:20
LPCIBotProject db-devel build (368): STILL FAILING in 48 min: https://hudson.wedontsleep.org/job/db-devel/368/18:28
LPCIBotLaunchpad Patch Queue Manager: [rs=buildbot-poller] automatic merge from stable. Revisions: 12380,18:28
LPCIBot12381 included.18:28
=== al-maisan is now known as almaisan-away
=== leonardr-afk is now known as leonardr
=== beuno-lunch is now known as beuno
beunoso, what's up with the color change in bug titles?19:38
lifelesscolour change?19:43
beunothey're not red anymore19:43
beunothey're gray19:43
lifelessthey were red?19:44
lifelessanyhow, the h1 colours are now consistent on all the domains19:44
beunoheh19:44
beunoyes, they where always red on bugs, bugs was already red19:44
beunoso all h1's are light-ish gray?19:45
lifelessthats my understanding; sinzui and huwshimi are going to tweak it some more though19:45
beunoI'd propose a more title-y color, something that instead of blending into the background stands out19:46
sinzuibeuno: the shades will change. If Lp had a colour (aubergine, orange, etc...) we could use that19:47
sinzuibeuno: I am going to attempt a final font-size fix tonight19:47
=== _starbuck is now known as _starbuck-bbl
thumpermorning20:07
lifelessjcsackett: is bug 242461 a dup20:12
_mup_Bug #242461: Bug search returning results for inactive projects. <404> <lp-bugs> <oem-services> <search> <Launchpad itself:Triaged> < https://launchpad.net/bugs/242461 >20:12
* jcsackett looks20:14
jcsackettlifeless: looks like it. bug 632847 is specifically about an oops when following links b/c of that, but it's the same issue.20:15
_mup_Bug #632847: Bug page OOPS when viewed in deactivated project context <404> <bad-commit-12363> <lp-bugs> <oops> <qa-bad> <Launchpad itself:In Progress by jcsackett> < https://launchpad.net/bugs/632847 >20:15
lifelessjcsackett: I was looking for some other stuff and it seemed awfully familiar :)20:16
jcsackettyes indeed. :-P20:21
jcsackettlifeless: i'll go ahead and mark it.20:21
=== matsubara is now known as matsubara-afk
benjiAnyone know what's wrong with my lint? http://pastebin.ubuntu.com/567459/20:51
benjileonardr: hi, have time for a quick review? https://code.edge.launchpad.net/~benji/launchpad/bug-649252/+merge/4987620:58
leonardrbenji, sure20:58
benjithanks20:59
wallyworldthumper: i cn hear you21:03
wallyworldthumper: i'll try restarting :-(21:04
wallyworldthumper: not sure what'21:10
wallyworldfaaaark21:10
wallyworldstupid mic21:10
* wallyworld tries something else21:11
wallyworldthumper: let me try and get it sorted and i'll get back to you21:11
thumperleonardr: mumble?21:12
thumperStevenK: around?21:12
wallyworldthumper: i tries skype and that worked21:14
thumperhaha21:15
thumpermumble hates you21:15
wallyworlddon't know what's up with mumble though21:15
leonardrthumper, yeah21:15
leonardrthumper, for future reference: http://pastebin.ubuntu.com/567472/21:19
=== cr3_ is now known as cr3
=== lifeless_ is now known as lifeless
leonardrbenji, r=me with very minor changes, sorry for the wait21:44
benjileonardr: no problem; gave me time to figure out why lint hates me21:45
=== leonardr changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | firefighting: - | On call reviewer: - | https://code.launchpad.net/launchpad-project/+activereviews
wgrantmwhudson: We didn't know at the time that it was running out of file handles :(21:52
mwhudsonwgrant: yeah21:53
mwhudsonproduction debugging is such a joy!21:53
=== salgado_ is now known as salgado-afk
benjileonardr: I don't know if you have time or not, but speaking of lint breakage, I have a branch ready for review that fixes the problem: https://code.edge.launchpad.net/~benji/launchpad/make-lint-ignore-grep-env/+merge/4988421:56
leonardrsure, why not21:56
leonardrbenji, makes sense to me22:05
benjicool22:06
pooliebenji, thanks for the doc feedback22:25
poolieleonardr, hi22:25
leonardrpoolie, hi22:25
benjimy pleasure22:26
poolieleonardr, i'm just wondering if there's a news-feed-like-thing that client authors ought to be reading to tell them about this22:28
leonardrpoolie, what is this? the changes to launchpadlib?22:28
poolieyes, launchpadlib22:29
poolieor a what's new webpage or something22:29
leonardrthere's no news feed. we can turn the CHANGELOG into a news fee22:30
leonardrd22:30
pooliei don't care so much about it being rss (i don't use that)22:30
poolieas just being told "oh you shouldn't use launchpadlib.launchpad anymore" somehow22:30
poolieperhaps deprecation warnings or api breaks are the most direct means to do so :)22:30
poolieoh i finished reading "Restful Web Services" on the weekend btw; it was good22:31
leonardrthanks22:31
leonardrit's difficult to deprecate a constant, or you would have been getting deprecation warnings for a while22:31
pooliei know22:32
wgrantCan't you detect the deprecated service root when it is used?22:32
pooliei know it's hard22:32
pooliewell, i guess there are two things being deprecated22:33
pooliethe edge server; also the symbol that points to the url22:33
thumperleonardr: I seem to be having trouble with the launchpadlib test for the setRecipeText22:33
thumperleonardr: it is saying that AttributeError: 'Entry' object has no attribute 'setRecipeText'22:33
thumperwhen using version='1.0' to launchpadlib_for22:34
wgrantYour WADL is probably out of date.22:34
pooliefor warning people not to use 'edge' we can easily warn when you use that URL22:34
pooliefor the name, i don't know22:34
pooliejust generally speaking i think python deprecation warnings are pretty poor22:35
pooliebecause they tend to appear to users who can't necessarily do anything about them22:35
pooliea warning at compile time (if there was such a thing) would be better22:36
lifelessupstream python has turned off display of deprecation warnings by default22:36
lifeless!22:36
leonardrthere's also no way to force developers to read anything or to comply with any of our standards--witness the oauth problems22:37
leonardrthumper: i would also suspect out-of-date wadl, but that shouldn't be a problem in a launchpad test22:38
leonardralso, if you had out of date wadl it would have setRecipeText22:38
lifelessrun 'make'22:38
leonardrrun make just to be sure22:38
thumperok22:39
* wallyworld takes son to doctor, back soon22:40
poolieleonardr, so i'm kind of inclined to focus on making things fail in an obvious way22:44
poolieand warning people ahead of time by documentation, eg blog posts22:45
poolierather than changes in the program22:45
poolienot sure though22:45
leonardrpoolie: well, there was a blog post back in october that nobody read, since when there was a fire-drill failure yesterday one common response was "it's too much trouble to fix this"22:46
leonardri'm not saying that a deprecation warning will do any better22:46
leonardrbut the other thing i did yesterday was make the old code give the new behavior22:46
leonardrwhich will help22:46
pooliei think your thing of just making edge send people to lpnet is good22:46
poolieconsidering the constraints of not being able to rewrite on the server side22:47
poolieand that clients were for a long time required or encouraged to ask for edge22:47
poolie "it's too much trouble to fix this" --> meaning what?22:47
poolietoo hard to update clients to not use edge?22:47
leonardrpoolie: right. "i have 20 machines" kind of thing22:47
leonardrthat was how i found out it was an accident, actually--one person wouldn't take "make this trivial change" for an answer, and escalated22:48
poolieanyhow22:49
pooliehttps://code.launchpad.net/~leonardr/launchpadlib/silently-replace-edge-with-production/+merge/49794 seems reasonable to me22:49
thumperleonardr: lp:~thumper/launchpad/recipe-inline-edit-recipe-text is still getting the attribute error22:54
leonardrchecking22:57
leonardrthumper: here are a couple things to try22:57
leonardr1. don't bother removing the method and re-exporting it. just publish it as a mutator write operation22:58
leonardrthat might give you the behavior you want22:58
StevenKthumper: My setRecipeText change just landed, too.22:58
thumperStevenK: which change is that?22:59
StevenKthumper: https://code.launchpad.net/~stevenk/launchpad/set-recipe-text-bad-data/+merge/4885322:59
thumperleonardr: you mean remove;     @export_write_operation(),   @operation_for_version("devel") from the middle?23:04
leonardrthumper: yes23:04
leonardri think we can treat this the way we treated transitionToStatus23:04
leonardrno, unfortunately not23:05
leonardrtransitionToStatus was changed to a mutator in 'beta', not '1.0'23:05
leonardrhow long has this setRecipeText operation been in place, anyway? isn't this whole feature new?23:05
thumperleonardr: it complains if I delete them23:06
leonardrlet me see the complaint23:06
thumperhttp://pastebin.ubuntu.com/567499/23:08
leonardrthumper, how new is this feature? unless it's older than version 1.0 of the web service there's no need to do this23:08
leonardrie. you can change the 'devel' version around however you want, without respect for backwards compatibility23:11
leonardrthumper: you should also remove the 'operation_removed_in' annotation if you didn't already23:14
thumperleonardr: how do I find out if they are in 1.0?23:14
leonardrthumper: bzr annotate is probably the best way23:14
thumperleonardr: when did we cut 1.0?23:15
leonardrquite a while ago, possibly a year23:15
leonardr"It was introduced in March 2010"23:15
thumperhmm... very possible recipes weren't in 1.023:19
thumperhow do we generate the wadl for 1.0?23:20
leonardrthumper: it's based on the annotations23:21
leonardrthe thing is that once someone published setRecipeText as a named operation, it showed up in all the versions, including 1.023:21
leonardrsince it wasn't explicitly excluded from other versions23:22
wgrantCould we invert that behaviour?23:22
leonardrbut, given that it wasn't in the original 1.0, and that i suspect it was added pretty recently, i think we can just get rid of it23:22
wgrantSo things only show up in devel unless they are explicitly annotated?23:22
thumperleonardr: I agree23:22
leonardrwgrant: it would be a big project since we'd have to manually grandfather in everything else23:22
wgrantleonardr: But otherwise versions are not versioned :/23:22
leonardrwgrant: our policy is to let older versions have new stuff, but not to *change* behavior. we decided that anything else was a lot of work to prevent things that generally range from "slightly harmful" to "actually good"23:24
wgrantleonardr: But then we need to explicitly restrict things to devel if we think we ever want them to change.23:25
wgrantWe should introduce features to devel and then push them back later.23:26
wgrantas an explicit step.23:26
leonardrwe already have an explicit step "don't publish named operations that are effectively mutator methods", and setRecipeText already slipped through that crack23:26
wgrantWell, that's this particular case.23:27
wgrantBut in general, an immature recipe API should not have slipped into 1.0.23:27
leonardrbut no policy i could have set would have prevented it, since the much simpler policy i set wasn't followed23:28
thumperleonardr: source package recipes were not exported at all at the 1.0 time23:28
wgrantleonardr: This is why we need safe defaults, I think.23:28
wgrantPolicies are easily missed.23:28
thumperleonardr: so... the first adapter (on the bottom) should be     @operation_for_version("devel")?23:29
leonardrthumper: i suggest simply removing all references to versions23:30
leonardr@operation_for_version("devel") will remove setRecipeText from 1.0 and replace it with nothing23:30
thumperleonardr: it seams reasonable that we should be able to annotate an Entry to say "added in devel"23:30
leonardrthumper: we decided not to add support for that because it was too much work for too little benefit23:31
thumperleonardr: so how do I say "this method is only in devel"?23:31
leonardrthumper: that's how you'd do it. @operation_for_version("devel")23:31
james_wok23:31
leonardrbut it's already in 1.0, so when you fix it in devel you might as well let the fix propagate to 1.023:31
thumperleonardr: so ... http://pastebin.ubuntu.com/567508/23:32
thumperhmm..23:32
thumperwhat if we take recipe stuff out of 1.0?23:33
lifelessthumper: its new right ?23:33
leonardrwgrant: if we were twitter or some other very popular web service i would be very anal about this. but as it is, we don't have the manpower to maintain velocity with strict separation between versions23:33
thumperlifeless: new since 1.0 yes23:33
lifelessthumper: I think new stuff should be devel only23:33
lifelessIMO we shouldn't change 1.0 at all23:33
thumperlifeless: we are discussing how to manage this23:33
leonardrok. when you publish recipes on the web service, you can publish all of its named operations in devel only23:34
leonardryou can publish all of its fields in devel only23:34
thumperit would be nice to just have checked in WADL for defined versions23:34
wgrantleonardr: We can take a big step towards that by just making the default devel-only.23:34
leonardryou can publish all links to recipe type objects in devel only23:34
wgrantleonardr: When a feature is done, we promote it to 1.0.23:35
wgrantIf people use a feature in devel, it's their problem if their script breaks.23:35
leonardrwgrant: if the default is devel only then we (i) must manually promote hundreds of existing features23:35
wgrantAnd they will realise it's unstable.23:35
leonardrthumper: but, you can't stop something called a 'recipe' from showing up in the wadl for earlier versions, and the syntax for restricting all this stuff will be ugly23:36
* lifeless would like the default to be devel only; and when we cut 1.1 we shift the things that are stable from devel to 1.123:37
leonardrbecause our original design was based around doing version restrictions *when there was a problem with letting stuff bleed through into older versions*23:37
thumperleonardr: why not?23:37
leonardrwe can re-evaluate this, we can always re-evaluate this, but it's going to be a big project during which imo we are not delivering value to users23:38
thumperI guess it depends if you are a user of the 1.0 webservice and using things you thought were stable, but weren't23:38
leonardrto clarify, our original design was based around doing version restrictions when something *changed* between versions, not when something was added23:39
thumperI can see that23:39
leonardrwe can do this:23:39
leonardr"we created the named operation setRecipeText. that was a mistake"23:40
leonardr"let's leave setRecipeText alone in 1.0 and change it in devel"23:40
thumperthat's what we were trying to do23:40
leonardrright23:40
thumperbut the tests weren't working23:40
thumperI got a launchpadlib for 1.023:40
leonardrthen i proposed checking to see how recent the feature is23:40
thumperand it says setRecipeText doesn't exist23:40
leonardrbecause maybe we can just change it everywhere23:41
leonardrand short-circuit this problem23:41
thumperit may well be that no-one uses this at all through the web service23:41
thumperI'd be happy to just change this everywhere for this method23:41
thumperbut my concerns are over the versioning as a whole for the web service23:42
leonardrwell, as a whole the rule is to do this by the book23:42
leonardrso let's do this by the book23:42
leonardrand then you'll see how it works23:42
thumperok...23:42
thumperso lets try to get the annotations working then23:42
leonardrso, why is the test failing? i don't know. i'll investigate23:42
leonardrit's possible there's a bug in lazr.restful. i don't know if we have a real case where a method is removed and then reinstated as a mutator23:43
thumperok23:43
thumpercan I leave this test with you to check?23:43
thumperlp.code.model.tests.test_sourcepackagerecipe.TestWebservice.test_recipe_text_setRecipeText_in_one_zero23:43
thumperthat is the test that fails23:43
thumperin the branch23:43
leonardrok23:45
leonardrdoing it by the book this time is annoying, but the advantage this has over 'devel only by default' is that in the majority of cases where we didn't make a mistake, we don't have to add cumbersome annotations, and people who are using 1.0 get the new stuff for free23:46
thumperIMO 1.0 should not get new stuff23:48
thumperbut that's just me23:49
* thumper needs food23:49
leonardrthumper: it's not just you, but the amount of work required to stop 1.0 from getting new stuff is nontrivial. and what are you doing? making 1.0 less useful23:49
StevenKThe other side of the coin is 'making 1.0 more stable, and doing what we said we would'23:50
lifelessleonardr: FWIW we make mistakes most of the time23:51
leonardrlifeless: maybe 'bleed through by default' was just another mistake then23:53
leonardrthumper: the operation works in beta but not 1.0. that makes me think there may be a bug in lazr.restful23:53
lifelessleonardr: the reason I say we make lots of mistakes is because we do incremental development23:54
lifelessleonardr: so its baked in that we'll iterate and tweak23:54
lifelessI think this is a positive thing, but the friction it has with support guarantees like 1.0 is pretty high.23:55
leonardri say this because lazr.restful has a feature that let us deprecate 'transitionToStatus' et al. in a reasonable way, and that might be interfering with what we're doing here23:55
lifelessinteresting23:55
=== Ursinha is now known as _starbuck
leonardrlifeless: so, it sounds the early-2010 web service versioning strategy conflicts with the early-2011 development strategy23:56
lifelessleonardr: tell me about this deprecation feature23:59
leonardrlifeless: look at lib/canonical/launchpad/rest/configuration.py23:59
leonardrlast_version_with_mutator_named_operations = "beta"23:59

Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!