[00:41]  * thumper throws up his hands in frustration
[00:41] <thumper> damn windmill testing
[00:42] <wgrant> lifeless: Erm, your PQM submission had a very odd failure.
[00:42] <wgrant> lifeless: I guess it could have been the codehosting connection refusal.
[00:42] <thumper> anyone else getting failures in  lp.services.features.browser.tests.test_feature_info.TestFeatureControlPage ?
[00:43] <wgrant> thumper: Yes.
[00:43] <wgrant> lifeless has a testfix for that.
[00:43] <wgrant> But PQM is being crap.
[00:43] <wgrant> There are four failures relating to this change. See the latest buildbot run for details.
[00:44]  * thumper sighs
[00:48]  * thumper gets frustrated at jQuery selectors
[00:49] <thumper> possible pebkac
[00:49] <mwhudson> jquery?
[00:50] <thumper> mwhudson: used for windmill tests to select elements
[00:50] <mwhudson> oh right
[00:50] <thumper> yep PEBKAC
[01:03] <lifeless> wgrant: I have submitted, again, again.
[01:12] <lifeless> 4th attempt to ec2 land this db patch
[01:17] <wgrant> lifeless: Thanks.
[01:17]  * thumper sprinkles some '[0]' values over the jquery selectors
[01:26] <lifeless> grrr
[01:26] <lifeless> http://checkip.amazonaws.com -> efail
[01:26] <lifeless> just for me
[01:27] <wgrant> Why? Proxy?
[01:27] <lifeless> dunno
[01:27] <lifeless> there is a nz wide proxy :(
[01:27] <wgrant> Awesome.
[01:28] <wgrant> Yay, landed.
[01:28]  * wgrant lands stuff.
[01:28] <wgrant> lifeless: '[ui=none]' :(
[01:33]  * thumper lands daily-ajax
=== Ursinha is now known as _starbuck-gone
[02:00] <wgrant> So, we need to do something about all these codehosting OOPSes. I think UnknownRemoteImportanceError probably wants to generate informational OOPSes?
[02:00] <wgrant> ExternalBugtracker for BugTrackerType 'GOOGLE_CODE' is not known.
[02:00] <wgrant> That seems thoroughly useless.
[02:00]  * wgrant removes.
[02:00] <mwhudson> wgrant: i don't think you mean codehosting
[02:01] <wgrant> mwhudson: Er, checkwatches.
[02:01] <wgrant> No idea where codehosting came from.
[02:01] <spm> wgrant: about the same place as "ssh codebounce.c.c" comes from
[02:01] <wgrant> Although there are lots of codehosting OOPSes too.
[02:02] <wgrant> They are next after checkwatches.
[02:02] <lifeless> spm: oh hai
[02:02] <wgrant> So HA
[02:02] <lifeless> spm: I can has those haproxy?
[02:02] <spm> DENIED
[02:02] <spm> not really, I just like saying that.
[02:02] <lifeless> spm: you've been saving that up for weeks
[02:02] <spm> months....
[02:06] <lifeless> spm: ok, so not denied... urlof screen cap ?
[02:08] <wallyworld> anyone know how to get the text to show up beside the image for a lazr-btn styled submit button? by default, using class=lazr-btn hides the text and just shows the image
[02:09] <thumper> anyone? https://code.launchpad.net/~thumper/launchpad/recipe-inline-edit-recipe-text/+merge/49585
[02:09] <thumper> wallyworld: call?
[02:10] <thumper> wallyworld: mumble? and I'll try to answer your question
[02:10] <wallyworld> thumper: ok
[02:27] <poolie> hi lifeless
[02:28] <poolie> can i borrow your ear briefly?
[02:32] <wgrant> lifeless: That's a nice Message query improvement you have there.
[02:38] <lifeless> whats wrong with these two lines
[02:38] <lifeless>         comments[0].text_for_display = ''
[02:38] <lifeless>         assert len(comments) > 0, "A bug should have at least one comment."
[02:38] <lifeless> wgrant: yeah, should be pretty swet
[02:38] <lifeless> poolie: sure
[02:38] <lifeless> poolie: voice or irc?
[02:38] <lifeless> spm: thank yo
[02:39] <poolie> pots? i can call you?
[02:39] <beuno> len of the list is > 0!  guaranteed success, can't beat that
[03:06] <wallyworld> anyone want to claim a code review? https://code.launchpad.net/~wallyworld/launchpad/request-build-popup/+merge/48864
[03:12] <wallyworld> thumper: do you have a bug number handy for the recipe "build now" work
[03:20] <wgrant> spm: qas is down... is this the DB restore?
[03:29] <spm> wgrant: aye
[03:34] <spm> hrm. actually I could bring it back up for now. the db restore is into an _new db. need it so?
[03:35] <wgrant> Not really.
[03:35] <wgrant> How long will the DB restore take?
[03:35] <wgrant> I am hoping for another deployment tonight to unbreak nightly.sh.
[03:40]  * StevenK appears.
[03:41] <wgrant> Hi StevenK.
[03:41] <wgrant> When are you back?
[03:41] <wgrant> thumper: What's changed in the new lazr.restful?
[03:41] <StevenK> At the airport, waiting for boarding.
[03:41] <StevenK> Plane delayed 40 minutes already.
[03:41] <wgrant> Where've you been? Adelaide?
[03:41] <StevenK> Right.
[03:46] <poolie> wgrant, i commented on https://code.launchpad.net/~wgrant/launchpad/bug-708999-ff-docs/+merge/49758
[03:46] <poolie> happy to discuss that
[03:46] <spm> wgrant: how long? no idea. based on staging restores somewhere around 12+ hours would be my wag.
[03:46] <wgrant> spm: :( OK
[03:47] <wgrant> poolie: Yeah, we do need a boolean standard :(
[03:47] <wgrant> I reused the existing [empty|nonempty] notation, but am not very happy with it.
[03:47] <poolie> i think at least changing the docs would be good
[03:47] <poolie> we can file a bug for booleans
[03:48] <poolie> i regret not adding such a mechanism in the first place but at least it's fairly easy to change
[03:48] <wgrant> Do you have a preferred replacement for '[empty|nonempty]'?
[03:52] <poolie> [true|]
[03:52] <poolie> maybe?
[03:52] <wgrant> I guess, yeah.
[03:53] <wgrant> I can probably leave out the "if true" then, too.
[03:53] <poolie> i can really imagine people either putting 'non-empty' or more likely thinking it's something to do with how to handle empty queues
[03:53] <poolie> i did, on the first reading
[03:56] <wgrant> poolie: I initially had the "if non-empty" at the start, but felt that made it less readable by putting the useful information later.
[03:57] <wgrant> But this looks OK now.
[03:59] <wgrant> poolie: http://people.canonical.com/~wgrant/launchpad/feature-info.png
[03:59] <wgrant> But we really need some standardisation of names and values :/
[04:01] <lifeless> uhm
[04:01] <lifeless> [true|] seems less clear to me
[04:01] <lifeless> this is a dsl, its not actually python.
[04:01] <lifeless> just saying.
[04:02] <wgrant> For some of the keys true makes sense.
[04:03] <wgrant> eg. visible_render_time and code.incremental_diffs.enabled
[04:03] <lifeless> and disable_targetnamesearch
[04:03] <wgrant> Right.
[04:04] <lifeless> but
[04:04] <wgrant> So only really code.branchmergequeue needs to be 'enabled'
[04:04] <lifeless> False
[04:04] <lifeless> is not the opposite of true
[04:04] <wgrant> Right.
[04:06] <lifeless> hmm
[04:06] <lifeless> this might be even easy
[04:07] <wgrant> It is tempting to define a new boolean standard.
[04:07] <wgrant> Describe it on +feature-info as empty or non-empty.
[04:07] <wgrant> And then make all of the existing rules fit it.
[04:07] <poolie> yes
[04:07] <poolie> i think for now, if it's just empty or not
[04:08] <poolie> i care more about making the comment clear than what's in the value domain
[04:08] <wgrant> lifeless: You had a suggestion?
[04:09] <poolie> lifeless, i see launchpadlib's tip still has uris in uris.py, surprisingly
[04:10] <poolie> s/surprisingly
[04:10] <poolie> i think leonard was only saying hypothetically he could change them
[04:12] <lifeless> wgrant: BugTask:+index may be easy
[04:13] <wgrant> My current thoughts are that I should standardise all boolean flags to empty/non-empty, call this value domain 'boolean', document the empty/non-emptiness of this new domain on +feature-info, and beat the existing boolean feature flags until they have names and implementations compatible with this new standard.
[04:13] <wgrant> lifeless: Oh?
[04:13] <lifeless> wgrant: sure, sounds reasonable
[04:13] <poolie> or not
[04:14] <poolie> wgrant, that sounds fairly reasonable
[04:14] <poolie> i wonder if 'defined to be ""' vs 'not defined' will be a bit confusing, for flags that essentially default to be on
[04:14] <wgrant> Flags won't default to be on.
[04:14] <wgrant> The only one that does that is disable_targetnamesearch.
[04:14] <wgrant> And that is one I will rename.
[04:15] <lifeless> wgrant: mmm
[04:15] <lifeless> wgrant: why rename it?
[04:15] <wgrant> Because it is inverted.
[04:15] <lifeless> its inverted so that when we deploy it doesn't suddenly disable the search
[04:16] <lifeless> it has the vector it needs
[04:16] <wgrant> Right, but that can also be achieved by adding the FF beforehand.
[04:16] <lifeless> and the name matches the vector, doesn't it ?
[04:16] <wgrant> Or we could add a default to getFeatureFlag.
[04:16] <lifeless> wgrant: or just accept that sometimes we need to disable rather than enable
[04:16] <wgrant> :(
[04:17] <lifeless> wgrant: I think changing the name for the reason you give is pointless consistency
[04:17] <lifeless> wgrant: harmful even
[04:17] <wgrant> OK.
[04:19] <wgrant> lifeless: Do you also object to my renaming of malone.advanced-subscriptions.enabled to malone.advanced_subscriptions.enabled, code.branchmergequeue to code.branchmergequeues.enabled, code.recipes_enabled to code.recipes.enabled?
[04:20] <wgrant> I fear that if I do not then yet more variants will spring up.
[04:20] <wgrant> Whereas if we standardise now, people will see there is a standard and follow it.
[04:20] <wgrant> Because currently it is a little crazy.
[04:21] <poolie> +1 to that
[04:21] <poolie> adding a default is a bit harder in TAL
[04:21] <poolie> we could set a default in the code that registers the flag definition
[04:21] <wgrant> Ahh, true.
[04:22] <poolie> i have also been wondering about features that grow from being experimental and off-by-default to being stable and on-by-default
[04:22] <poolie> we want to be able to garden the dynamic rules
[04:22] <poolie> but, ideally, while still being able to flip things off in response to late-breaking emergencies
[04:22] <poolie> (but perhaps not; the code may have rusted in place)
[04:23] <wgrant> Right. The 'default' column on +feature-info seems a little pointless if there is no default.
[04:24] <poolie> if there's a default we probably shouldn't have things expressed in the negative sense like 'hidden' or 'disabled'
[04:24] <poolie> i guess the question is whether the default is expressed in just one place, or diffused throughout the code
[04:24] <poolie> at the moment i suspect the latter
[04:25] <wgrant> Right, that was my point with lifeless' new disabled_targetnamesearch.
[04:25] <wgrant> I don't like it. But without defaults it's hard to sensibly do the opposite.
[04:27] <wgrant> For now I'll do the boolean standardisation. We can think about defaults later.
[04:29] <lifeless> uhm
[04:29] <wgrant> Hm?
[04:29] <lifeless> so, I don't care about the renames; all the ones you list are flags we're going to delete in the next 4-5 months.
[04:29] <wgrant> We hope.
[04:29] <lifeless> I think the _ is better than the -
[04:30] <wgrant> Yep.
[04:30] <lifeless> I could care less about the namespacing; I think the cost of communication will outweight several years development :)
[04:30] <wgrant> Waitwaitwait.
[04:30] <wgrant> Could, or couldn't?
[04:30] <lifeless> I have a small aesthetic preference for namespacing
[04:31] <lifeless> but I really think it doesn't matter at all
[04:31] <lifeless> if you rename
[04:31] <lifeless> you need to communicate very clearly
[04:31] <lifeless> check losa docs
[04:31] <lifeless> check outstanding rts
[04:31] <lifeless> and check with teams setting these.
[04:31] <wgrant> Of course.
[04:32] <lifeless> thats a lot of work; if you want to do it, its your choice. But you asked for my opinion.
[04:32] <lifeless> which is 'why bother'
[04:32] <wgrant> If I don't, then someone will need to police all reviews until we are standardised.
[04:32] <lifeless> why?
[04:32] <wgrant> Because conventions are good, but people don't follow unimplemented conventions.
[04:33] <lifeless> what does this standard buy us; what is the benefit? how does it make us more efficient or less fragile ?
[04:33] <lifeless> conventions are conventions, they are most definitely not intrinsically good
[04:33] <wgrant> Sure.
[04:33] <wgrant> But code.branchmergequeue is not a clear name.
[04:34] <wgrant> If its values were [enabled|disabled] it might be.
[04:35] <lifeless> Look, I don't /object/ to a mass rename if you do all the work, I just think I weigh the costs and benefits very differently here. I think the cost in losa time is a huge shame.
[04:35] <lifeless> and I certainly wouldn't want reviews being bounce back if a clear but 'nonstandard' name were chosen; feature flags are meant to be low key and cheap constructs.
[04:36] <lifeless> I agree that code.branchmergequeue is unclear.
[04:38] <thumper> wgrant: you mean with respect to my change?
[04:38] <wgrant> thumper: You upgraded us to the proper 0.16.0, with no-qa.
[04:38] <wgrant> I am wondering if anything significant has changed that needs testing.
[04:39] <wgrant> Since things went bad with the last upgrade.
[04:40] <thumper> wgrant: the only change is that non-5xx error codes no longer have tracebacks
[04:40] <wgrant> OK, great. Thanks.
[04:40] <poolie> wgrant, i think the main change we want is to say that in future it should be 'code.branchemergequeue.visible' or whatever
[04:40] <poolie> to establish the pattern for future things
[04:40] <wgrant> poolie: Right.
[04:41]  * lifeless is saying that structure really doesn't matter
[04:41] <lifeless> the silos are gone
[04:41] <lifeless> what matters now is *doing*
[04:41] <lifeless> and with that, I'm back to working on timeouts, a pasttime I hope you all share my passion for
[04:41] <wgrant> lifeless: Our favourite BugTask:+index?
[04:42] <lifeless> yes
[04:42] <wgrant> Excellent.
[04:46] <wgrant> What.
[04:46] <wgrant> Why does lib/lp/services/features/templates/feature-info.pt have several <p>s without </p>s?
[04:47] <wgrant> This is not the 1990s :(
[04:48] <wgrant> What :/
[04:48] <poolie> :)
[04:49] <wgrant> What.
[04:49] <wgrant> If I attempt to close one of the <p>s after another element, TAL complains the <p> is not open.
[04:49] <poolie> i wonder if tal has special inferences for it?
[04:52] <wgrant> Hmmm.
[04:54] <wgrant> PARA_LEVEL_HTML_TAGS = frozenset([ # List of HTML elements that close open paragraph-level elements # and are themselves paragraph-level. "h1", "h2", "h3", "h4", "h5", "h6", "p", ])
[04:54] <wgrant> XHTML does not work that way, TAL...
[04:56] <mwhudson> chameleon ftw
[04:56] <wgrant> So it uses XML namespaces heavily, but apparently does not use an XML parser.
[04:56] <wgrant> How odd.
[04:57] <huwshimi> wgrant: Those <p>s look like they shouldn't be there anyway
[04:57] <wgrant> huwshimi: I suspect they are there to space it nicely. But I am hoping to remove them.
[04:57] <wgrant> For now they are <p/>s instead.
[04:58] <wgrant> I am also tempted to delete zope.tal in retaliation for it being braindead, but that may be slightly excessive.
[04:58] <huwshimi> wgrant: Well they should be at least after the <h2>s and certainly not wrapping the <table>
[04:58] <wgrant> huwshimi: Right.
[04:58] <wgrant> I initially presumed they were wrapping the h2 and text.
[04:58] <wgrant> But the h2 closes them immediately.
[04:59] <wgrant> Because pre-XHTML HTML is cool.
[05:01] <huwshimi> wgrant: It's nice that it closes those tags for us, but not nice in that it doesn't help us write good html
[05:02] <wgrant> No, it's not nice that it closes them at all!
[05:03] <huwshimi> wgrant: What do you mean?
[05:04] <wgrant> It makes us write something that looks like an XML document and makes us specify namespaces and all that. And then treats the document like something that is not XML.
[05:04] <wgrant> That is not useful.
[05:04] <wgrant> It is stupid :/
[05:06] <wgrant> I guess it may have made sense when it was written.
[05:06] <huwshimi> wgrant: Right, so it's trying to be smart in a way that is unhelpful. It is good however that we don't deploy broken html pages.
[05:08] <wgrant> huwshimi: Ah, but a real XML parser would crash, not render broken XHTML.
[05:09] <lifeless> hmm
[05:09] <lifeless> whats a good bug to look at that has synchronising bugwatches ?
[05:10] <wgrant> Ones that work?
[05:10] <wgrant> Bug 184131 has two.
[05:10] <_mup_> Bug #184131: Apache 2.2 SNI support <Apache2 Web Server:Fix Released> <apache2 (Ubuntu):Fix Released> <apache2 (Debian):New> < https://launchpad.net/bugs/184131 >
[05:10] <wgrant> Except the debbugs one is broken at the moment.
[05:10] <huwshimi> wgrant: It would be good if it broken depending on the spec (i.e. it breaks when we don't add alt tags to images etc.)
[05:11] <lifeless> trying to see why we process bug watched messages
[05:11] <lifeless> (at all)
[05:11] <wgrant> lifeless: Ah, you want a bug with synced comments?
[05:11] <lifeless> yes
[05:11] <lifeless> build_comments_from_chunks is -bong-
[05:11] <lifeless> AFAICT our templates don't use half the work it does
[05:13] <stub> I suspect it is not using a real XML parser because they are slow, and Chameleon's very reason for existence is because someone got sick of ZPT being slow.
[05:16] <wgrant> lifeless: Bug #476397 has several imported comments.
[05:16] <_mup_> Bug #476397: does not display correctly an inkscape pattern fill <apport-bug> <i386> <Poppler:Confirmed> <poppler (Ubuntu):Triaged by desktop-bugs> < https://launchpad.net/bugs/476397 >
[05:17] <lifeless> ah
[05:17] <lifeless> right
[05:20] <lifeless> wgrant: thanks, very helpful
[05:20] <wgrant> lifeless: You'd not seen that feature before?
[05:22] <lifeless> wgrant: I was trying to figure out if the loop on imported bug messages was global state for the template or local to the comment
[05:22] <lifeless> it was local to the comment, so we can skip the size(bug messages) lookup entirely.
[05:22] <lifeless> also
[05:22] <lifeless> bugtask:+index is doing lazy evaluation *of the bug watches*
[05:22] <lifeless> and the irony is
[05:23] <lifeless> its doing that in a region of code with a comment saying 'we want to avoid one query per item in the for loop'.
[05:23]  * lifeless facewalls
[05:23] <wgrant> Haha.
[05:24] <lifeless> check out build_comments_from_chunks if you want to see the evidence
[05:24] <lifeless> stub: hi; there was something I wanted your advice on, some less-normalisation I think.
[05:25] <lifeless> stub: reckon yoiu could spare a few minutes to look at this?
[05:25] <stub> Sure
[05:25] <lifeless> its uhm, forget the bug number - binarypackagerelease.packagename though
[05:25] <lifeless> forces materialisation to sort archive pages
[05:28] <wgrant> binarypackagename and sourcepackagename are two normalisations that I don't really agree with.
[05:28] <lifeless> i'm just trying to find the bug, without luck
[05:28] <wgrant> getPublishedBinaries?
[05:28] <wgrant> That one?
[05:28] <lifeless> bug 706200
[05:28] <_mup_> Bug #706200: Archive:EntryResource:getPublishedBinaries timeouts <qa-ok> <timeout> <Launchpad itself:Triaged by julian-edwards> < https://launchpad.net/bugs/706200 >
[05:29] <wgrant> Yeah.
[05:30] <stub> I'd happily drop the *PackageName tables - never liked them.
[05:30] <wgrant> Hah.
[05:32] <lifeless> +1
[05:33] <wgrant> Julian wasn't precisely happy with them last time I talked to him about this.
[05:33] <wgrant> So it sounds like everyone agrees.
[05:35] <stub> I wouldn't even call it denormalization since the *PackageName tables contain a single value and are write once - not 3NF as implemented by SQL databases.
[05:36] <stub> Oh - I recall the rationale I think...
[05:37] <wgrant> huwshimi: I can see why the <p>s are there.
[05:37] <wgrant> huwshimi: Removing them leaves too little space between a table and the following h2.
[05:37] <huwshimi> wgrant: Oh yeah?
[05:38] <wgrant> h2 needs more like margin-top: 1em
[05:38] <wgrant> It now has 0.3em.
[05:38] <stub> There was no SourcePackage or BinaryPackage tables, so *PackageName was there to ensure via referential integrity that bugs could only be targetted to valid packages. It is pretty weak, but I wasn't able to argue against it well enough back then.
[05:38] <huwshimi> wgrant: Is that a general thing? I'm not sure where we use h2s currently
[05:38] <wgrant> huwshimi: I'm not sure where else we use them either.
[05:39] <wgrant> stub: That's completely invalid.
[05:39] <wgrant> stub: So we can throw them out, it seems.
[05:39] <stub> Could be, or I could be recalling wrong. It was a subject of much debate anyway, and I lost.
[05:40] <wgrant> stub: It's a very plausible reason.
[05:40] <wgrant> It's just also completely invalid.
[05:40] <lifeless> wgrant: bah, indices exist for a reason
[05:40] <stub> Actually, a lot of our indexes don't exist for a reason (any more). I'll be detecting and cleaning out the useless ones soon ;)
[05:41] <lifeless> \o/
[05:41] <wgrant> lifeless: Hm?
[05:41] <wgrant> huwshimi: Most of the existing h2s seem to be in portlets (that have their own padding) or surround div.actions (that have their own margin)
[05:43] <wgrant> lifeless: Where did I not use indices?
[05:46] <wgrant> Hah.
[05:46] <wgrant> code.recipes_enabled lies.
[05:46] <wgrant> It is already empty|non-empty, not on|off as it says.
[05:48] <wgrant> In fact I think they all are.
[05:48]  * wgrant headdesks.
[05:55] <wgrant> Hmm.
[05:55] <wgrant> memcache defaults to enabled.
[05:56] <wgrant> That's a bit awkward with the standardised boolean type.
[05:58] <lifeless> hmm
[05:59] <lifeless> we need a 'slices to range query' helper
[05:59] <lifeless> 'nother day
[05:59] <wgrant> lifeless: Huh?
[05:59] <wgrant> Oh.
[05:59] <wgrant> Yeah, that would be handy sometimes.
[06:00] <wgrant> lifeless: Do you have an opinion on the memcache feature? Should I detect the absence of a flag and default to on?
[06:00] <wgrant> I think that's probably best unless we have a pressing need for more global defaults.
[06:01] <wgrant> unless/until
[06:03] <wgrant> poolie: What do you think?
[06:04] <wgrant> lifeless: What was the point of your indices comment? Have I missed an index somewhere?
[06:05] <lifeless> wgrant: hmm? unset is on at the moment because its a killswitch
[06:05] <lifeless> wgrant: it should stay that way.
[06:05] <lifeless> wgrant: I think you're making flags more complex and harder to use, if you're running into trouble.
[06:06] <wgrant> It is not trouble. There are easy ways to do it. I just want to set a good standard.
[06:07] <lifeless> wgrant: so, its a killswitch, it should stay as one
[06:07] <lifeless> 'normally on'
[06:07] <wgrant> Right.
[06:08] <wgrant> The value on prod will need to change, but that is easy enough.
[06:08]  * poolie looks
[06:09] <poolie> i think that making memcache behave as "if not set, on" just inline in the code is reasonable for now
[06:09] <poolie> i think "memcache" is a pretty broad name for a flag too
[06:09] <poolie> if it actually means tal caching or something
[06:09] <wgrant> That's what I've done. I check if there is no flag, and default to on.
[06:09] <poolie> cool
[06:10] <wgrant> It's actually not just TAL caching.
[06:10] <wgrant> It's the whole client.
[06:10] <poolie> ok
[06:10] <wgrant> http://people.canonical.com/~wgrant/launchpad/feature-info-again.png
[06:11] <wgrant> It turns out that most of the current https://launchpad.net/+feature-info is a lie :(
[06:11] <poolie> wgrant, i like your screenshot
[06:12] <poolie> yes, but a Bright Shining Lie
[06:13] <wgrant> huwshimi: Some <th>s seem to be black, others grey. Is this intentional?
[06:13] <huwshimi> wgrant: I don't think so
[06:14] <poolie> wgrant,  so again i think saying "this turns off memcache altogether" would be good too
[06:14] <poolie> but we can get there
[06:14] <wgrant> huwshimi: Compare https://launchpad.net/+feature-info and https://code.launchpad.net/launchpad/+activereviews
[06:14] <lifeless> oh my
[06:14] <lifeless> the ss queries on bugtask:+index are -huge-
[06:15] <wgrant> lifeless: ss?
[06:15] <wgrant> Strucsubs?
[06:15] <poolie> "process'" is odd grammar
[06:15] <wgrant> poolie: Is it?
[06:15] <wgrant> It's not mine, but it seems right to me.
[06:15] <poolie> process's? process?
[06:15] <huwshimi> wgrant: yeah that's wrong
[06:16] <wgrant> huwshimi: Missing thead?
[06:16] <wgrant> Yeah.
[06:16] <huwshimi> wgrant: That's one way to solve it :)
[06:17] <lifeless> wgrant: yes
[06:17] <lifeless> wgrant: visit bug 2 in the same data with LP_DEBUG_SQL=1
[06:17] <poolie> i guess it's a known bug that there seems to be a different acl for triaging while filing vs triaging after filing?
[06:18] <wgrant> poolie: Not a bug.
[06:18] <lifeless> also wasDescriptionModified is fugly
[06:18] <wgrant> poolie: It was quite intentional at the time.
[06:18] <lifeless> poolie: I wouldn't assume so, I think its surely a bug
[06:18] <lifeless> possibly not filed.
[06:18] <wgrant> poolie: Bug supervisors were meant to be able pre-triage their bugs, but normal users were to be discouraged from doing so.
[06:19] <poolie> wgrant, something about people being more likely to triage badly if it's on the initial form?
[06:19] <poolie> hm
[06:19] <wgrant> This annoys me to no end.
[06:19] <poolie> there's a weird mispattern here about it being good to make actions hard if some people will abuse them
[06:19] <wgrant> People are more likely to set to Confirmed if the initial form tells them that they can, I guess.
[06:19] <poolie> true
[06:19] <wgrant> But it is very inconvenient.
[06:19] <poolie> i suffer this on lp:kanban for instance
[06:20] <lifeless> setting confirmed is irrelevant
[06:20] <lifeless> because confirmed should die in a fire
[06:20] <wgrant> lifeless: Yes.
[06:20] <wgrant> There were proposals at UDS-Jaunty to remove it.
[06:20] <poolie> that kind of gets into the new/confirmed/triaged thing: does confirmed mean "multilpe people reproduced this" or "someone knowledgeable reproduced it" or "a developer read it"?
[06:20] <wgrant> But that never eventuated.
[06:20] <poolie> :)
[06:20] <poolie> i have a fire
[06:20] <wgrant> It's too warm today to destroy things in fires.
[06:20] <wgrant> Except for zope.tal.
[06:28] <lifeless> zomg
[06:28] <lifeless> whats wrong with this
[06:28] <lifeless> tal:replace="view/visible_oldest_comments_for_display/count:len">23</span>
[06:29] <wgrant> poolie: Any comments on http://people.canonical.com/~wgrant/launchpad/feature-info-again-again.png? I've hopefully improved the subscriptions.
[06:29] <wgrant> lifeless: What is count:len?
[06:29] <lifeless> wgrant: count is a sql query
[06:29] <lifeless> as in .count()
[06:30] <lifeless> IIRC my tal, :len is a formatter to say 'this is a length'. IMBW
[06:30] <wgrant> Ah.
[06:30] <lifeless> but - we're doing a sql query to figure out how many rows we told the sql engine to get for us.
[06:30] <lifeless> or something approximately like that
[06:30] <wgrant> Although it may be less than what we requested.
[06:30] <lifeless> bizarre
[06:31]  * lifeless is willing to live with that discrepancy
[06:31] <lifeless> at least for a bit
[06:31] <wgrant> Oh.
[06:32] <wgrant> In that context.
[06:32] <wgrant> The requested value should always be right, right?
[06:32] <wgrant> Since if there are fewer comments then we display all of them.
[06:32] <lifeless> right
[06:32] <wgrant> So that is pointless.
[06:32] <wgrant> Kill it.
[06:33] <lifeless> right
[06:37] <lifeless> another related change
[06:38] <lifeless> currently the threshold is 'visible comments > limit'
[06:38] <lifeless> I am changing that to 'comments > limit' because 'visible comment' requires interpretation
[06:38] <lifeless>  /sigh
[06:39] <wgrant> lifeless: You mean you have to ask the DB to look at every comment?
[06:40] <poolie> wgrant, i see you stripped off the warning about scopes not being used?
[06:40] <poolie> or is that just screenshot cropping?
[06:40] <poolie> it looks very wide but i assume it scales
[06:40] <wgrant> poolie: Just screenshot cropping.
[06:40] <lifeless> wgrant: check out get_visible_comments in browser/bugtask.py
[06:41] <poolie> so memcache default=enabled is just pure documentation at this point?
[06:41] <wgrant> lifeless: I did not want to see that.
[06:41] <wgrant> poolie: No.
[06:41] <poolie> ie it's describing a policy embedded in the code, rather than actually setting the default?
[06:41] <wgrant> Right.
[06:41] <poolie> k
[06:41] <wgrant> I think once more things want defaults it may become more than just documentation.
[06:41] <poolie> right
[06:41] <poolie> it looks really nice
[06:42] <poolie> i wonder if we should make them sentence case to be consistent with the usual thing for ui and docstrings
[06:42] <poolie> anything in particular you want comments on?
[06:42] <poolie> oh while you're there you could link to the dev wiki about this, perhasp
[06:42] <poolie> so we can add more docs
[06:42] <wgrant> A good idea.
[06:42] <wgrant> On both counts.
[06:42] <wgrant> Was mostly wondering if the descriptions are OK now.
[06:42] <wgrant> But those comments are good too.
[06:43] <wgrant> The memcache one would be easier to word as a negative :(
[06:44] <wgrant> poolie: Hm, there's no really relevant page on the dev wiki, I don't think.
[06:46] <poolie> hm
[06:46] <poolie> i'll create https://dev.launchpad.net/Foundations/FeatureFlags as at least a stub
[06:46] <poolie> there's a bug open asking for something beyond the lep
[06:47] <wgrant> Foundations is dead.
[06:47] <poolie> what would be a good URL?
[06:48] <poolie> it seems like there should be a /Guide/ or /Components/ or something
[06:48] <wgrant> I'd like a namespace, but I don't think we have a relevant one.
[06:50] <poolie> so although Foundations doesn't exist as a team, it still exists on the dev wiki home page
[06:50] <poolie> we can always rename with a redirect later
[06:50] <wgrant> OK.
[06:50] <poolie> ... unless someone has a better name now?
[06:50] <wgrant> I don't.
[06:52] <poolie> bug 692480
[06:52] <_mup_> Bug #692480: need better docs on how to use/test feature flags <feature-flags> <Launchpad itself:Triaged> < https://launchpad.net/bugs/692480 >
[06:52] <poolie> i may well be wrong but it looks like edge's api is 30% faster than lpnet
[06:52] <wgrant> poolie: Less load.
[06:52] <wgrant> So smaller queues.
[06:52] <huwshimi> Bye people.
[06:52] <poolie> that's measuring across many requests
[06:52] <wgrant> lifeless is trying to fix this.
[06:52] <wgrant> huwshimi: Night.
[06:52] <poolie> i'm surprised it's so high it shows up above the rtt
[06:54] <lifeless> 2.5 seconds
[06:54] <lifeless> last time I measured it
[06:55] <lifeless> its sneaking up
[06:55] <lifeless> we've got 2 new 8 core appservers, a complete reconfig of the per appserver process thread count and an haproxy rearrangement to address this
[06:55] <poolie> \o/
[06:55] <lifeless> eta is probably 3-4 weeks
[06:56] <lifeless> I /think/ we'll last that long
[06:56] <poolie> 2.5s... queue time before service?
[06:56] <wgrant> Overhead on edge is usually around 600ms.
[06:56] <wgrant> lpnet is rarely below 1.5s.
[06:56] <wgrant> In my quick tests here.
[06:56] <wgrant> poolie: Yes.
[06:56] <lifeless> poolie: yes; just subtract the time to first byte and the reported render time.
[06:56] <poolie> ouchy
[06:56] <lifeless> poolie: (and rtt)
[06:56] <poolie> just wondered why my attempted optimizations (which also included moving things to lpnet) seemed to be making things so much worse
[06:56] <poolie> silly me
[06:57] <lifeless> well, lpnet is where we want you
[06:57] <wgrant> Hmmm, yeah, for the API that would reallllly suck.
[06:57] <lifeless> because those edge resources will make lpnet faster as we move them across
[06:57] <poolie> it makes kanban generation 30% slower overall
[06:57] <poolie> usure
[06:57] <lifeless> the thing making edge faster isn't queuing so much
[06:57] <cody-somerville> hmmm... are you guys saying edge is faster currently then lpnet? :P
[06:57] <lifeless> as that we don't have as many bloated pages served on edge
[06:57] <lifeless> *that* drives the queue times
[06:58] <lifeless> cody-somerville: yes, and if you use it I will personally block your uid there
[06:58] <lifeless> (just saying)
[06:58] <poolie> lifeless, what do you mean?
[06:58] <wgrant> Speaking of which... in the debacle last night did we tell the complainers to use lpnet instead?
[06:59] <lifeless> wgrant: which debacle?
[06:59]  * cody-somerville hardly ever uses his personal uid... just saying :P
[06:59] <wgrant> lifeless: The edge API incident.
[06:59] <lifeless> cody-somerville: :P
[06:59] <poolie> what happened?
[06:59] <lifeless> wgrant: I haven't read about that yet; we should.
[06:59] <wgrant> lifeless: edge was serving api.launchpad.net URLs for an hour.
[06:59] <poolie> oh, very nice of it to pitch in like that :)
[06:59] <lifeless> wgrant: (I mean I haven't read the detail .. I knew the outline)
[06:59] <wgrant> Ah.
[07:00] <wgrant> Well, my edge config minimisation broke things because our vhost config is terrible :(
[07:00] <lifeless> yeah
[07:00] <lifeless> wgrant: as a data point, this is the sort of reason I was ignoring the edge config
[07:00] <wgrant> Also because people don't remove cruft from configs, though.
[07:00] <lifeless> wgrant: nonzero chance of trouble, short term win, long term we're killing it.
[07:00] <cody-somerville> You guys should have like a duplicated setup for like staging this stuff :P
[07:00] <wgrant> It would not have broken if people had removed cruft instead of trimming it.
[07:01] <poolie> lifeless, so the difference is basically just that there's less load on edge relative to it's hardware
[07:01] <poolie> ?
[07:01] <lifeless> cody-somerville: we do, but the config is the thing that is different
[07:01] <lifeless> poolie: shorter requests are more common on edge
[07:01] <poolie> huh
[07:01] <lifeless> 99th percentile on lpnet is 3s, on edge its 2.something
[07:01] <wgrant> lifeless: Oh, really?"
[07:01] <wgrant> Didn't know that.
[07:01] <lifeless> api's 99th percentile is 2s across both
[07:02] <poolie> becaues it gets relatively more api load?
[07:02] <lifeless> edge has slower servers
[07:02] <wgrant> Since the reports were merged it's less easy to check.
[07:02] <lifeless> thats my current theory.
[07:02] <lifeless> not that I really care; its going bye bye
[07:02] <cody-somerville> If launchpad has no load, how fast would it be?
[07:02] <cody-somerville> *had
[07:02]  * lifeless sobs   /home/robertc/launchpad/lp-branches/working/lib/lp/bugs/browser/bugtask.py(986)wasDescriptionModified()
[07:02] <wgrant> lifeless: Heh heh heh.
[07:03] <lifeless> cody-somerville: about 1.5 to 2 seconds faster on every page
[07:03] <lifeless> cody-somerville: you're seeing the render times on pages now, right ?
[07:04] <cody-somerville> well... 'query/external actions issued' time
[07:04] <lifeless> thats the render time
[07:04] <lifeless> on *top* of that there is
[07:04] <lifeless> RTT
[07:04] <lifeless> tcp window opening
[07:04] <lifeless> SSL
[07:04] <lifeless> in datacentre queuing
[07:04] <poolie> well, for this i think i have ssl keepalives
[07:04] <lifeless> python GIL contributes maybe 25% to the render time
[07:04] <poolie> imbw
[07:05] <lifeless> in datacentre queuing is doing a variable 1.5-2 seconds
[07:05] <lifeless> no load would remove in datacentre queuing
[07:05] <lifeless> and the GIL contention issues
[07:05] <lifeless> slow sql would still be slow
[07:06] <poolie> pfft
[07:08] <cody-somerville> I guess my question is... these seem like 'common' problems. How do other websites manage to have better performance? Is it a code issue or are we just need investing the cash to have a powerful enough 'cloud' to serve lp as fast?
[07:08] <cody-somerville> err... s/need/not/
[07:08] <wgrant> The DC overhead fix is scheduled.
[07:08] <wgrant> The GIL contention will also soon be minimised by these same changes.
[07:09] <cody-somerville> LP faster would make LP like 50% better. Folks would have so much more tolerance for other issues. At least thats my theory.
[07:09] <wgrant> It's mostly a matter of people not really caring about performance for 5 years, and it's taking a while to fix it.
[07:09] <lifeless> cody-somerville: I agree; and its on the basis that we're buying the new front end servers etc.
[07:09] <wgrant> cody-somerville: Definitely.
[07:10] <lifeless> wgrant: well, folk gave a crap, but I think the tools in use, and the particular sorts of analysis needed to solve performance were challenging for the team.
[07:11] <lifeless> wgrant: there are places in LP that have been heavily optimised along one or two axis
[07:11] <wgrant> lifeless: Sure.
[07:12] <cody-somerville> I don't think folks really recognized how big of an impact performance had on the overall perception of the service - in the same fashion that in the early days folks didn't recognize the impact the UI had on the overall impression.
[07:12] <lifeless> I think thats totally untrue
[07:12] <cody-somerville> maybe thats just a personal reflection
[07:12] <lifeless> *every* current and emeritus LP dev I know cares passionately about performance and UI
[07:13] <lifeless> on the UI side there was a bunch of conflicting directives
[07:13] <lifeless> the distro wanted bugzilla
[07:13] <lifeless> in terms of UI, not actual bugzilla
[07:13] <lifeless> Mark wanted what he'd now call crisp and clean
[07:13] <cody-somerville> Yea... but thats relatively 'recent'
[07:13] <lifeless> lots of churn resulted
[07:14]  * cody-somerville remembers launchpad from one or two 'ui refreshes' ago.
[07:14] <lifeless> cody-somerville: on day one he wanted the same basic outcome; the language for describing it has evolved.
[07:14] <cody-somerville> ah
[07:14] <lifeless> anyhow
[07:14] <lifeless> there is a /tonne/ of discipline needed to make the site fast
[07:15] <lifeless> and zope is not intrinsically kind - zope is optimised for same-machine ~0 latency DB calls.
[07:15] <lifeless> which we don't have
[07:15] <cody-somerville> Ugh... omg... I cant believe the snow plows are already starting! Its only 3am!!! :(
[07:15] <cody-somerville> errr...
[07:15] <cody-somerville> 2am
[07:23] <LPCIBot> Project devel build (442): STILL FAILING in 5 hr 40 min: https://hudson.wedontsleep.org/job/devel/442/
[07:36] <poolie> wgrant, bug 719180
[07:36] <_mup_> Bug #719180: need central defaults for flags <feature-flags> <Launchpad itself:Triaged> < https://launchpad.net/bugs/719180 >
[07:36] <poolie> not sure if that should be higher
[07:37] <wgrant> It's not hard, and there is a squad assigned to take the remaining high-priority FF bugs. So perhaps.
[07:38] <wgrant> That may be a Strategist question, I'm not sure.
[07:41] <lifeless> wgrant: there isn't one, is there? it got close off
[07:41] <wgrant> lifeless: The remaining bugs were thrown at Green this morning.
[07:42] <lifeless> oh, nice.
[07:43] <wgrant> That's why I was working on it.
[07:43] <lifeless> isn't green on maintenance?
[07:43] <wgrant> Yes.
[07:43]  * lifeless twitches
[07:43] <lifeless> who threw them at you?
[07:44] <wgrant> lifeless: The Strategist hiimself.
[07:44] <wgrant> There were only two or so.
[07:44] <poolie> also https://bugs.launchpad.net/launchpad/+bug/719182
[07:44] <_mup_> Bug #719182: boolean interpretation for feature flags <feature-flags> <Launchpad itself:Triaged> < https://launchpad.net/bugs/719182 >
[07:45] <wgrant> And it within a maintenance squad's mandate to finish off bugs resulting from a feature -- normally it would be the squad that did the feature initially, but that's obviously not possible here.
[07:51] <poolie> what else should go into https://dev.launchpad.net/Foundations/FeatureFlags#preview
[07:53] <wgrant> poolie: That looks excellent.
[07:53] <wgrant> Particularly the conventions bit :)
[07:54] <poolie> thanks :)
[07:54] <poolie> just going to fill out the two blank sections of course
[08:00] <lifeless> wgrant: kk
[08:02] <poolie> wgrant, also hard_timeout, contrary to its docs, seems to be in milliseconds
[08:02] <poolie> not seconds
[08:03] <wgrant> poolie: Hah, so it is.
[08:03]  * wgrant fixes docs.
[08:04] <poolie> lifeless,  did you add server.?
[08:04] <poolie> as a scope
[08:04] <poolie> any particular reason it uses a dot rather than a colon?
[08:06]  * wgrant vanishes for a while.
[08:06] <wgrant> Thanks poolie, lifeless.
[08:06] <poolie> thank you; have a good night
[08:07] <lifeless> poolie: you added it
[08:07] <poolie> hah
[08:07] <poolie> ok
[08:07] <lifeless> poolie: if I may, -1 on colons
[08:07] <poolie> so team.admins ?
[08:08] <lifeless> erm
[08:08] <lifeless> on flags, not scopes
[08:08]  * lifeless confuses the issue
[08:08] <poolie> oh, yeah, i agree with that
[08:08] <lifeless> let me start over
[08:08] <poolie> was writing about scopes
[08:08] <lifeless> so, server. was what you had
[08:08] <lifeless> I left it as is when doing team: because of diminishing returns etc
[08:11] <poolie> istm colon makes sense for scopes because it's followed by user parameters, but dots are better for flags which are more like tokens
[08:11] <poolie> but i mostly want to document stuff that will avoid unnecessary inconsistency
[08:12] <lifeless> cool
[08:12] <poolie> to make things faster maybe you can read <https://dev.launchpad.net/Foundations/FeatureFlags#preview> and tell me if you either disagree with anything, or think anything ought to be there
[08:13] <lifeless> I may have seemed grumpy earlier; I'm glad folk care about this
[08:13] <lifeless> I think there is a balance though
[08:13] <lifeless> what happened to the LEP ?
[08:14] <poolie> it still exists
[08:14]  * lifeless thought we'd agreed to clean it up and keep it current
[08:14] <poolie> but people complained that it had too much speculative stuff
[08:14] <poolie> oh, i thought we agreed we would add separate documentation
[08:14] <lifeless> right, which I have a note here to go delete.
[08:15] <poolie> i think "here's what you need to know to use it" is separate from "here's what we thought about in writing it"
[08:15] <poolie> the LEP still has some ideas we could add later
[08:15] <poolie> they link to each other
[08:15] <lifeless> there is a risk in wikis of ending up with a link farm
[08:15] <lifeless> I think this runs very close to that
[08:15] <poolie> true
[08:15] <lifeless> anyhow the prose is fine
[08:15] <poolie> we could certainly shove some of the LEP into LEP/ff/Future
[08:15] <poolie> or /Archive
[08:16] <lifeless> it shouldn't really be in 'Foundations' as those team silos are gone
[08:16] <poolie> as i said i would be happy to see the dev wiki get a /Guide namespace or something
[08:16] <poolie> we can rename and redirect
[08:17] <poolie> i just wanted to get _some_ developer-oriented docs
[08:17] <lifeless> sure
[08:17] <lifeless> any as I say, the prose there seems fine, both content and style
[08:17] <poolie> if you want to take a view of what the dev wiki should look like that would be great
[08:18] <poolie> cool, thanks
[08:18] <lifeless> Personally, I would expect to find in a LEP enough useful stuff to use something we built
[08:18] <lifeless> much like one does in a PEP
[08:18] <poolie> well
[08:18] <poolie> eventually the PEPs are redundantly documented in What's New and in the actual user manual
[08:19] <poolie> so it's kind of analogous
[08:19] <lifeless> wgrant: /count:len means 'call len() on the thing and show it as a count' as far as I can tell..
[08:19] <poolie> oh, https://bugs.launchpad.net/launchpad/+bug/692480/comments/1
[08:19] <_mup_> Bug #692480: need better docs on how to use/test feature flags <feature-flags> <Launchpad itself:Triaged> < https://launchpad.net/bugs/692480 >
[08:20] <poolie> maybe i should just move it to /FeatureFlags now?
[08:20] <lifeless> poolie: when I need to know about context managers, I read the context manager PEP; I think if we're not going to delete the LEPs the same should be true.
[08:21] <lifeless> yes, /FeatureFlags would be a reasonable home
[08:21] <poolie> so i would probably start with the user manual and drill down to the pep
[08:21] <poolie> perhaps we should make it part of the lep process that by the time it's implemented, the lep is up to date
[08:23] <poolie> your impatience before was good
[08:23] <poolie> i just heard some frustration with the lack of concise docs at the epic, so i felt i ought to add some
[08:23] <lifeless> thank you
[08:24] <lifeless> uhm
[08:24] <jtv-afk> Morning folks.  Still massive test fail?
[08:24] <lifeless> I'd roll the testing one into the main page for it I think
=== jtv-afk is now known as jtv
[08:24] <poolie> yeah, that would be good
[08:24] <lifeless> jtv: nope
[08:24] <poolie> "This page is about SOMETHING. More details can be found in CategoryCategory."
[08:24] <poolie> good to know it's about something
[08:28] <poolie> lifeless, ok testing stuff rolled in
[08:29] <poolie> you're probably right they should merge actually
[08:29] <poolie> with future stuff or background shunted off into appendices
[08:29] <wgrant> poolie: Shall I drop the /Foundations from the link?
[08:30] <lifeless> poolie: its sounding nicer and nicer
[08:30] <lifeless> \o/
[08:30] <lifeless> partial loading bugtask:+index in place
[08:31] <wgrant> lifeless: Just messages for now?
[08:31] <lifeless> wgrant: its still loading all activities, yes.
[08:32] <wgrant> :(
[08:32] <wgrant> Still, not so bad.
[08:33] <lifeless> also found some lovely bugs
[08:34] <lifeless> did you know a bug import change the 'original description' for a bug ?
[08:34] <wgrant> Yup.
[08:34] <wgrant> There's a bug for that.
[08:34] <lifeless> well, fixed.
[08:34] <jtv> lifeless: the same tests that were failing in ec2 are also failing for me locally, even in devel.  So why not still massive test fail?
[08:34] <lifeless> jtv: buildbot is green
[08:34] <wgrant> lifeless: Since the index is explicit now?
[08:35] <lifeless> wgrant: yes
[08:35] <wgrant> Yay.
[08:35] <poolie> wgrant, yes
[08:36] <poolie> yes drop /Foundations/ from the link
[08:36] <wgrant> Already done.
[08:36] <wgrant> Thanks.
[08:39] <jtv> lifeless: rf-setup seems to have fixed most failures, but not all
[08:43] <lifeless> jtv: what failures are you seeing?
[08:44] <jtv> lifeless: in ec2test.tests.test_remote, a bunch of cases where a human-readable string was expected and something crypt-y comes out instead ("VnA5Gz…") and one where "utf-8" was expected on an email header but "utf8" was received.
[08:45] <jtv> And perhaps more, but there's rather a lot of output.
[08:45] <jtv> 7 failures in that test (on devel)
[08:45] <wgrant> jtv: Are you on natty?
[08:45] <jtv> Maverick still.
[08:45] <wgrant> Hmm.
[08:46] <jtv> I've been seeing these over the past 7 hours or so.
[08:46] <jtv> On EC2, I even saw the number of failures in that particular test increase over that time.
[08:46] <wgrant> Hmm.
[08:54] <poolie> night
[08:55] <lifeless> jtv: sorry, no insights here; buildbot is happy though, which is (a) reference platform
[08:57] <jtv> Is nobody else seeing failures from ec2test.tests.test_remote?
[08:57] <wgrant> Not I.
[08:57] <wgrant> I have had two ec2 runs today.
[08:57] <wgrant> One failed because lifeless' testfix bounced, one succeeded after that landed.
[08:58] <jtv> And mine failed in the same way that devel failed locally.  :(
[08:59] <jtv> It's as if email is encrypted but the tests expect it not to be.
[08:59] <adeuring> good morning
[08:59] <jtv> hi adeuring
[08:59] <wgrant> jtv: passes fine for me locally.
[08:59] <wgrant> jtv: Checked your bzr config?
[08:59] <bigjools> morning all
[08:59] <wgrant> jtv: That is copied to the ec2 slave.
[08:59] <jtv> wgrant: I wouldn't know what to look for.  I didn't mess with it recently.
[08:59] <jtv> hi bigjools
[09:00] <jtv> wgrant: there's a gpg_signing_command there
[09:00] <jtv> and create_signatures = always
[09:01] <wgrant> I don't sign my revisions, although I probably should.
[09:01] <wgrant> I suppose it's possible that's relevant.
[09:01] <jtv> I thought that was required setup.
[09:01] <wgrant> Clearly not :)
[09:02] <stub> jtv: Sounds like Python mail library issues.
[09:02] <wgrant> I have seen the utf8 vs utf-8 issue before.
[09:02] <wgrant> But I do not remember in what circumstances.
[09:03] <stub> jtv: In particular, I'd guess the 'encrypted' message is base64. We have a monkey patch lurking somewhere that is supposed to change the default from base64 to quoted printable.
[09:03] <wgrant> Aha.
[09:03] <stub> And the preferred character set to UTF-8
[09:03] <jtv> That sounds like it would help these tests… but why did this start failing for me so recently?
[09:04] <stub> You switched to Python 2.7 or something?
[09:04] <jtv> Not that I know.
[09:04] <jtv> Discovery: the failures I've been getting until 8 hours ago were actually completely different ones than the ones I've been seeing since then.
[09:05] <stub> jtv: You get this on a fresh branch?
[09:05] <jtv> devel
[09:05] <jtv> also, on EC2 with a branch that was fresh yesterday.
[09:06] <jml> good morning
[09:06] <lifeless> hi jml
[09:07] <jtv> morning jml
[09:07] <wgrant> Evening jml.
[09:08] <bigjools> helleau jml
[09:08] <stub> jtv: So my guess is that it is something to do with test ordering or which subprocess layer is being run.
[09:09] <jtv> stub: I'll try running just one failing test in isolation then.
[09:09] <jtv> That fails too.
[09:09] <stub> jtv: With tests being run that haven't imported canonical.launchpad.mail or lp.services.mail.sendmail failing because the monkey patch hasn't installed.
[09:09] <stub> Stick an 'import lp.services.mail.sendmail' at the top of the test's .py file?
[09:09]  * jtv tries
[09:11] <lifeless> it might be useful to have something run over all our tests and run them one by one in a new process; file bugs on any that fail.
[09:11] <jtv> stub: yup, that fixes it!
[09:11] <stub> del Charset.CHARSETS['utf-8']
[09:11] <stub> Charset.add_charset('utf-8', Charset.SHORTEST, Charset.QP, 'utf-8')
[09:11] <stub> Charset.add_alias('utf8', 'utf-8')
[09:12] <jml> lifeless: wouldn't be too hard to do. I suspect some of our tests would behave better with such an runner
[09:12] <jtv> stub: that's in that module?
[09:12] <stub> So those lines and associated comments and imports needs to move to lib/lp_sitecustomize.py. I think that is the 'correct' solution ('correct' is an odd term to use when talking about monkey patches)
[09:16] <jtv> stub: strange that this should affect me so selectively… have I got the wrong locale configured or something?
[09:17] <stub> jtv: Are you running a subset of tests or the complete suite? If you run a subset of tests, you will be running things in a different order.
[09:17] <jtv> stub: it happens either way.
[09:17] <stub> Dunno then :)
[09:17] <stub> Tests should be run in ascii order IIRC, and the layer forking should be deterministic.
[09:20] <stub> I guess the tests might run in locale specific order... which could give weird results.
[09:21] <stub> I feel old whenever I see my old comments, dated 2005...
=== elmo_away is now known as elmo
[09:23] <stub> and 2004 in the module docstring...
[09:26]  * StevenK finally gets home.
[09:26] <StevenK> Fail, Virgin Blue
[09:29] <lifeless> StevenK: grats
[09:50] <wgrant> bigjools: That's going to be *quite* a branch.
[09:52] <jml> remove SourcePackageName and BinaryPackageName?
[09:52] <lifeless> sourcepackagename may be tricky to remove
[09:52] <adeuring> henninge: could you have a looka t this mp: https://code.launchpad.net/~adeuring/launchpad/bug-702468/+merge/49205 ?
[09:52] <jml> lifeless: you think?
[09:52] <jml> (yes, it will be tricky to remove)
[09:53] <wgrant> lifeless: It is referenced in a few more places. But it's not fundamentally significantly harder.
[09:53] <wgrant> Just a few more tables.
[09:53] <lifeless> wgrant: package branches
[09:53] <wgrant> lifeless: Hm?
[09:53] <wgrant> No harder than bugs.
[09:53] <lifeless> personally, I'd love for that constraint to be fixed, its a showstopper for turning off dput
[09:54] <lifeless> wgrant: we constrain the namespace of package branches to existing sourcepackagenames.
[09:54] <wgrant> lifeless: That's only because it's easier.
[09:54] <jml> wgrant's right.
[09:55] <lifeless> easier than what?
[09:55] <jml> lifeless: easier than implementing it not constrained
[09:55] <jml> lifeless: at the time
=== gmb changed the topic of #launchpad-dev to: :
[09:58] <gmb> Aaaaah
[09:58] <wgrant> Succinct!
[09:58] <gmb> Botheration.
[09:58] <gmb> If someone's got a recent topic line, now's the time to save the day.
=== wgrant changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | firefighting: - | On call reviewer: - | https://code.launchpad.net/launchpad-project/+activereviews
[09:59] <henninge> adeuring: of course! I was already looking at it yesterday but things got in the way. Sorry.
[10:00] <adeuring> henninge: no problem
[10:00] <gmb> wgrant: Thankyou kindly.
=== gmb changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | firefighting: - | On call reviewer: gmb | https://code.launchpad.net/launchpad-project/+activereviews
[10:00] <wgrant> henninge: Thanks for reviewing that branch.
[10:01] <henninge> wgrant: pleasure ;)
[10:02] <bigjools> wgrant: yeah I was being slightly tongue-in-cheek in the bug comment.
[10:03] <bigjools> sarcasm doesn't translate easyily to the written word
[10:03] <bigjools> and my typing doesn't help
[10:03] <wgrant> Come on qastaging, restore quicker.
[10:10] <allenap> I would like to increase the hard time-out for Person:+delete so I can delete a team. Should I do that, or should I fix the bug first (which is probably very hard)?
[10:11] <wgrant> ~ubuntu-main-sponsors?
[10:11] <stub> So the packaging table allows distroseries to be NULL, although there are now rows in the database with a NULL distroseries.
[10:12] <allenap> wgrant: Yes :)
[10:12] <wgrant> allenap: Good luck with that.
[10:12] <stub> Should this column be NOT NULL?
[10:12] <wgrant> allenap: It could have many thousands of bugsubscriptions.
[10:12] <allenap> wgrant: Yeah, that's what I was afraid of.
[10:13] <stub> (I have two indexes on (distroseries,sourcepackagename), and only one is correct)
[10:13] <stub> bigjools: ^^
[10:13] <wgrant> stub: That's a Registry thing.
[10:13] <stub> Ta
[10:13] <wgrant> stub: I'm pretty sure our code won't handle it if distroseries IS NULL.
[10:13]  * bigjools reads
[10:13] <wgrant> Certainly nothing uses it.
[10:13] <wgrant> Or creates them like that.
[10:13] <wgrant> Because it means nothing.
[10:14]  * bigjools always seems to get pinged when  "packaging" is involved :)
[10:15] <lifeless> allenap: you should fix the bug
[10:16] <lifeless> allenap: there is no guarantee that even a 20 sec timeout would be sufficient
[10:17] <henninge> adeuring: r=me with just a little comment. Thanks!
[10:17] <allenap> lifeless: Gah. Bugger. Okay, I try and figure out what it'll take.
[10:18] <lifeless> allenap: bug 162510
[10:18] <_mup_> Bug #162510: Person:+delete timeouts : Person merging needs to be done asynchronously <canonical-losa-lp> <chr> <feature> <lp-registry> <merge-deactivate> <tech-debt> <timeout> <Launchpad itself:Triaged> < https://launchpad.net/bugs/162510 >
[10:18] <lifeless> allenap: its half done already.
[10:18] <allenap> lifeless: Awesome, thank you.
[10:22] <lifeless> jml: got 5 ?
[10:22] <StevenK> lifeless: Actually package branches don't use SourcePackageName directly, but SourcePackage
[10:22] <jml> lifeless: yeah. just let me capture this thought.
[10:23] <wgrant> StevenK: SourcePackage doesn't exist.
[10:23] <jtv> danilos, henninge: bug 719247
[10:23] <_mup_> Bug #719247: Translations branch auto-approver not working <Launchpad itself:Triaged> < https://launchpad.net/bugs/719247 >
[10:23] <wgrant> StevenK: It is a Python myth.
[10:23] <danilos> jtv, yay?
[10:23] <jtv> danilos: not yay
[10:23] <StevenK> wgrant: ISourcePackage, lib/lp/registry/interfaces/sourcepackage.py
[10:24] <jtv> wgrant: you lie—I saw a SourcePackage once and if my camera hadn't broken down just then…
[10:24] <adeuring> henninge: thanks!
[10:24] <henninge> adeuring: np
[10:24] <henninge> jtv: We need to replace the approver.
[10:24] <wgrant> StevenK: Yes, a Python myth. Goes nowhere near the DB, and most of this work is going to be in the DB interface code.
[10:25] <danilos> jtv, any indication as to what might be the problem?
[10:25] <jtv> danilos: none whatsoever
[10:25] <jtv> henninge: this sounds like it may be an operational issue though
[10:26] <henninge> jtv: without looking further into it, I remember that the auto-approver for the branch imports was too careful.
[10:26] <jtv> henninge: this was a simple single-template setup though
[10:26] <jml> lifeless: skype me when you are ready
[10:26] <henninge> hm
[10:26] <jml> wgrant: that's backwards, imo.
[10:26]  * henninge looks closer
[10:26] <wgrant> jml: Yes, but so are lots of things.
[10:27] <wgrant> :(
[10:27] <wgrant> DistributionSourcePackage is both!
[10:27] <wgrant> DistributionSourcePackageInDatabase, yay.
[10:28] <gmb> bigjools: Is cron.germinate Soyuz territory?
[10:28] <bigjools> gmb: yes
[10:29] <bigjools> it's maintained by the ubuntu platform guys though
[10:29] <wgrant> Riddell's change is fine.
[10:29] <henninge> jtv: that is odd indeed
[10:29] <gmb> wgrant: Thanks :)
[10:29] <gmb> That's what I was asking about.
[10:30] <wgrant> gmb: Platform does just about all work on it.
[10:30] <wgrant> But it lives in our tree so we must wave them through.
[10:30] <gmb> Righto; duly noted. Thanks again.
[10:30] <jtv> henninge: sbackup looks like it may be having the same problem since september.
[10:35] <gmb> danilos, jtv, henninge: I'm looking at abentley's translations branch and, whilst it's small, I have no context on it to know whether it makes sense or not. Can one of you guys take a look?
[10:35] <gmb> https://code.launchpad.net/~abentley/launchpad/fix-undiverging/+merge/49718
[10:35] <jtv> gmb: I can
[10:35] <gmb> jtv: Bedankt.
[10:35] <jtv> Graag gedaan
[10:35] <jtv> (prononounce each "g" as if it were your last)
[10:36] <gmb> Hah.
[10:36] <jtv> No, more like khh
[10:36] <jtv> Check your keyboard.  If the spittle is visible there, you're doing it right.
[10:36] <gmb> :)
[10:41] <henninge> adeuring: If you still have a chance to fix it: The "Comment instead of docstring" exception is only for actual tests (methods that start with test_). SelectUpstreamTranslation should have a """docstring""" not  a # Comment. ;-)
[10:42] <adeuring> henninge: ah, right... but I'va already started "ec2 land"...
[10:42] <henninge> adeuring: if it fails, you'll have your chance ... otherwise just include that in one of your next branches.
[10:42] <henninge> ;-)
[10:44] <jtv> gmb: done
[10:45] <gmb> jtv: Thanks
[10:52] <henninge> jtv: You wrote: The script requires a distroseries to be specified among the options, but assumes a distroseries to be specified.  There may be a way to specify to optparse that a particular option is required
[10:53] <jtv> henninge: I meant it assumes a distro to be specified.
[10:53] <henninge> jtv: ah
[10:53] <jtv> And source package, IIRC
[10:53] <henninge> jtv: it defaults to "ubuntu"
[10:53] <henninge> via optparse
[10:53] <jtv> I see
[10:53] <henninge> no, source package is checked
[10:54] <StevenK> OSError: [Errno 2] No such file or directory: './lib/canonical/launchpad/icing/yui/event/event-base-ie-min.js'
[10:54] <StevenK> During 'make'
[10:54] <wgrant> StevenK: rm -r lazr-js
[10:55] <wgrant> make clean may do that.
[10:55] <wgrant> But I just remove it manually.
[10:55] <StevenK> wgrant: Ta.
[10:55] <wgrant> It happens occasionally on YUI upgrades.
=== _starbuck-gone is now known as _starbuck
[10:59] <wgrant> allenap: Do you have any ideas for getting rid of all the UnknownRemoteImportanceErrors and "ExternalBugtracker for BugTrackerType 'FOO' is not known." errors?
[11:01] <lifeless> gmb: hi
[11:01] <lifeless> gmb: I think bug 719249 is a risky way to go
[11:01] <_mup_> Bug #719249: The new direct subscription overlay takes too long to load <story-better-bug-notification> <Launchpad itself:Triaged> < https://launchpad.net/bugs/719249 >
[11:02] <lifeless> gmb: most of the time most users won't be changing subscriptions, by preloading you are going to pay for that on *every* page load.
[11:04] <allenap> wgrant: Get rid of them entirely, or from the OOPS reports? For the latter I think they should be added to (a|the) blacklist.
[11:05] <lifeless> allenap: if its in a blacklist, its pointless overhead to write an oops
[11:05] <lifeless> allenap: log it to the log file and move on - you don't need the http request count, backtrace, etc etc for these.
[11:06] <allenap> lifeless: Okay. wgrant, I'll file a bug for lifeless's suggestion.
[11:06] <wgrant> Thanks!
[11:08] <lifeless> allenap: rule of thumb: if a *human* should do something *now*, we record an OOPS.
[11:08] <lifeless> allenap: otherwise, we shouldn't.
[11:09] <allenap> lifeless: Noted :)
=== matsubara-afk is now known as matsubara
[11:18] <gmb> lifeless: Hmm. I see your point, but since we load all the portlets and initialise all the JS for the subscription links asynchronously already I'm not sure I entirely agree that it's risky.
[11:22] <wgrant> jtv: Hi.
[11:22] <jtv> hi wgrant
[11:22] <bac> gmb: i've just put up a MP but am not really here today.  could you review it at your leisure?
[11:23] <gmb> bac: Sure thing.
[11:23] <bac> thx
[11:24] <wgrant> jtv: update-stats.py is now the second-slowest piece of nightly.sh. Looking at DistroSeriesLanguage.updateStatistics, it seems to grab every POFile then sum the results of methods that return a value directly from a DB column. Am I missing something here, or can that be replaced with one query to select the sums directly?
[11:26] <wgrant> jtv: Same with the POTemplate iteration in DistroSeries.updateStatistics.
[11:27] <jtv> wgrant: we haven't looked at the DistroSeriesLanguage version of that for a while—as it happens I was just looking at POFile.updateStatistics performance and that seems to be quite fast now.
[11:27] <jtv> But yes, I believe that's something we simply never had time to optimize.
[11:28] <henninge> wgrant is the guy that screws booster rockets onto scripts ... ;-)
[11:28] <wgrant> henninge: I don't think I can do 1000x again, though :(
[11:28] <henninge> wgrant: 100x is fine, too :-P
[11:28] <wgrant> jtv: I gather that those attributes are caches that are relatively new?
[11:29] <jtv> wgrant: not at all, but their mechanics and intricacies have changed
[11:29] <wgrant> jtv: Hmm.
[11:29] <jtv> The attributes are positively ancient.
[11:29] <wgrant> It just seemed odd to have methods that just wrap apparently cached values.
[11:29] <wgrant> I presumed they were historical relics.
[11:30] <jtv> Those methods are from ancient interfaces, though adi simplified them last year.
[11:31] <wgrant> But there is no evil magic that prevents me from doing the sum in the DB?
[11:31] <jtv> They're still nastily spread out through the code, unfortunately.
[11:31] <jtv> No really evil magic, no, though you may run into some attributes that are computed from others.
[11:31] <wgrant> I don't believe there are any of those. But I will check.
[11:31] <wgrant> It all just seemed too simple!
[11:32] <jtv> Yes.
[11:33] <jtv> danilos, henninge, this may be related to the translations branch approver problem: bug 719267
[11:33] <_mup_> Bug #719267: Translation template approval times out <timeout> <Launchpad itself:Triaged> < https://launchpad.net/bugs/719267 >
=== jtv is now known as jtv-eat
[11:38] <lifeless> gmb: ah, if its async its better, but still - unneeded load is waste
[11:39] <lifeless> gmb: do have a look see - and remember that tests against launchpad.net are currently pessimised due to being overcapacity
[11:39] <gmb> Right.
[11:39] <jml> when are we getting more capacity?
[11:40] <lifeless> jml: shortly after we get losa cycles
[11:40] <lifeless> we have 8 cores we can expand onto with
[11:40] <lifeless> 'simple'
[11:40] <lifeless> reconfiguration
[11:40] <lifeless> and 12 more cores coming
[11:40] <lifeless> we should knock ~ 1 second of in-data centre latency off immediately
[11:41] <lifeless> to test; hit a page up and note the render time shown
[11:41] <lifeless> compare the time-to-first-byte in speed tracer of the same request
[11:41] <lifeless> the difference is ssl + rtt + in-dc queuing
[11:41] <lifeless> night all
[11:41] <jml> g'night.
[11:50] <bigjools> a pox on python files with a - in their name
[11:52] <jml> bigjools: indeed
[11:52] <jml> and also ones that can't be imported
[11:52] <bigjools> that is my problem :/
[11:53] <bigjools> trying my best to write a test for something I changed that is totally untested right now
[11:59] <jml> yeah.
[11:59] <jml> bigjools: with the file formerly known as ec2test-remote.py, I renamed it and added some tests for the current behaviour in a branch before landing my behaviour change
[12:00]  * bigjools is delving into buildd module territory where all bets are off
[12:02] <deryck> Morning, all.
[12:02] <allenap> Morning deryck :)
=== Guest28688 is now known as NCommander
[12:23] <LPCIBot> Project db-devel build (366): STILL FAILING in 5 hr 40 min: https://hudson.wedontsleep.org/job/db-devel/366/
[12:25] <leonardr> gmb, take a look at https://code.launchpad.net/~leonardr/launchpadlib/silently-replace-edge-with-production/+merge/49794
=== leonardr changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | firefighting: - | On call reviewer: gmb, leonardr | https://code.launchpad.net/launchpad-project/+activereviews
[12:39] <LPCIBot> Project devel build (443): STILL FAILING in 5 hr 16 min: https://hudson.wedontsleep.org/job/devel/443/
[12:39] <LPCIBot> * Launchpad Patch Queue Manager: [r=henninge][bug=198778,
[12:39] <LPCIBot> 718004] Rewrite update-bugtask-targetnamecaches to use a set-based
[12:39] <LPCIBot> method, approximately 1000x faster.
[12:39] <LPCIBot> * Launchpad Patch Queue Manager: [r=sinzui][no-qa] Remove unused icing.
[12:39] <LPCIBot> * Launchpad Patch Queue Manager: [r=deryck, lifeless,
[12:39] <LPCIBot> wgrant][bug=673530][incr] Add a popup choice widget for the build
[12:39] <LPCIBot> frequency.
[12:47] <bigjools> gmb: review ping please: https://code.launchpad.net/~julian-edwards/launchpad/buildd-regex-bug-719162/+merge/49796
[13:00] <abentley> henninge, deryck, adeuring: My internet is busted.  May not be able to Mumble.  Is POTS an option?
[13:01] <deryck> abentley, we could skype and dial a land line.
[13:04] <abentley> deryck: logging off to preserve cell data...
[13:04] <gmb> leonardr, bigjools: Just got back from lunch; will peer owlishly at your MPs shortly.
[13:05] <leonardr> great
[13:05] <leonardr> bigjools: i can take yours
[13:10] <gmb> leonardr: edge_server_equivalent_string_becomes_production() and edge_web_server_equivalent_string_becomes_production() look like tests but they don't have the test_ prefix so (I think) the testrunner won't run them. Is that deliberate (or am I wrong)?
[13:11] <leonardr> gmb: it's not deliberate, let me check
[13:11] <gmb> leonardr: line 121 and 127 of the diff, for reference
[13:11] <leonardr> gmb: yeah, just adding test_ fixes the problem
[13:12] <gmb> Cool
[13:12] <gmb> leonardr: r=me then.
[13:12] <leonardr> ok
=== jtv-eat is now known as jtv
[13:30] <leonardr> bigjools, r=me
[13:47] <salgado> leonardr, does launchpadlib mock Launchpad for any of its tests or do we still need a local Launchpad to run launchpadlib's test suite?
[13:47] <leonardr> salgado: you still need a local launchpad
[13:48] <leonardr> launchpad has unit tests but they don't test that launchpad returns specific data
[13:48] <leonardr> they test that launchpadlib behaves in a certain way
[13:48] <leonardr> er, launchpadlib has unit tests...
[13:48] <salgado> right
[13:49] <salgado> thanks leonardr.  I was afraid that'd be the case
[13:49] <leonardr> np. there is a project to make a mock launchpad but it's stalled afaik
[13:49] <salgado> leonardr, do you know where I can find more info about it?
[13:50] <leonardr> salgado: no, i don't remember who was doing it.
[13:52] <james_w> leonardr, do you mean the one on https://code.launchpad.net/launchpadlib/+activereviews ?
[13:53] <leonardr> let's see...
[13:53] <leonardr> yeah, that's the one.
[13:54] <leonardr> i guess it's not stalled--there's recent activity
=== henninge_ is now known as henninge
[13:56] <salgado> well, about 8 months ago was the last non-merge commit
[13:57] <salgado> although there's review activity. :)
[13:57] <bigjools> leonardr: thanks
=== almaisan-away is now known as al-maisan
[14:48] <danilos> gmb, leonardr: hi guys, anyone want to take a look at my "split bugtask_index.js into two" branch?
[14:48] <gmb> danilos: I'll take it
[14:49] <gmb> Since I set you on that path....
[14:49] <danilos> gmb, heh, excellent, thanks
[14:49] <danilos> gmb, https://code.launchpad.net/~danilo/launchpad/bugtask-index-portlet-setup/+merge/49818
[14:49] <danilos> gmb, diff is probably not there yet :)
[14:49] <gmb> Right :)
[14:50] <gmb> Time for me to get coffee
[14:50] <henninge> jtv: can you help me with some SQL foo?
[14:50] <danilos> gmb, heh, yeah, the diff itself is >2k lines, but you know what that is
[14:51] <jtv> henninge: sure
[14:51] <gmb> Right
[14:51] <henninge> jtv: it might be a good idea to consolidate the input queue before re-enabling imports.
[14:52] <jtv> henninge: consolidate how?
[14:53] <henninge> jtv: dpm reckons there are been multiple uploads for a source package in the queue and we really only need the latest.
[14:54] <henninge> jtv: so from all entries with the same sourcepackage and path, we could delete all but the latest.
[14:54] <henninge> I was just wondering if that could be done in SQL or if I need a script.
[14:55] <jtv> henninge: don't they all have the same uploaders?
[14:55] <henninge> jtv: dpm says not.
[14:55] <jtv> Sometimes not?  Never?  Not a lot?
[14:55] <henninge> good questions.
[14:56] <jtv> Because AFAICS that's the only duplication we're going to see: from multiple people uploading the same file.
[14:56] <henninge> jtv: True
[14:56] <jtv> Meanwhile, unfortunately, it's become impossible to approve some templates.
[14:56] <danilos> jtv, :(
[14:56] <henninge> jtv: I guess we could get some statistics on that
[14:56] <jtv> Timeout while running statistics for new POFiles.
[14:57] <jtv> How ironic.
[14:57] <henninge> why is that?
[14:57] <bigjools> allenap: thanks for fixing the buildd-manager
[14:57] <jtv> henninge: we're both talking about statistics.
[14:57] <allenap> bigjools: No worries, though it's not in stable yet.
[14:57] <henninge> hah!
[14:58] <jtv> henninge: redundant uploads should normally go into the pre-existing queue entries—and that means all of them for package uploads.  Is there really a problem here?
[15:01] <henninge> jtv: not sure
[15:01] <henninge> jtv: I will see if I can put together a query to get some numbers on that.
[15:02] <henninge> after lunch (well, tea really)
=== henninge is now known as henninge-tea
[15:02] <jtv> henninge: one complication… if two uploads are by different people, who's to say they don't both contain valid, original data?
[15:03] <henninge-tea> jtv: dpm says just the latest matters.
[15:03] <henninge-tea> but maybe he is wrong?
=== matsubara is now known as matsubara-lunch
[15:03] <jtv> Well I wonder how it's possible to tell, since it's different people.
[15:04] <jtv> You'd have coordinate that with all the translation teams.
[15:04] <jtv> (the impersonal "you," not you as such)
[15:09] <danilos> jtv, for packaged uploads, even if they are from different people, we can safely ignore non-latest ones (in theory, that might still cause different behaviour, but doesn't matter in practice)
[15:09] <jtv> danilos: ISWYM
[15:09] <jtv> Even so, I'd be interested to see how much it'd save us.
[15:10] <danilos> jtv, sure
[15:18] <dpm> henninge-tea, jtv, my question was not whether the two uploaders upload valid data (which I assume they'd do). The question is whether, assuming one uploads valid translation data on Monday and the other one on Tuesday, why do we need to import the data from Monday, if it's the same (or contains less translations)?
[15:18] <jam> spm: ping, if you haven't killed the extra processes on crowberry yet, it would be good if we could do some debugging.
[15:19] <jtv> dpm: in the meanwhile we've been convinced of that, but our current counterquestion is: how many imports would it save us?
[15:19] <dpm> jtv, henninge-tea, anyway, you guys know the technical details better, that was just my brainstorming to make the disruption in translations shorter
[15:19] <dpm> jtv, I don't know :) But I'm guessing it would be a big saving in big uploads such as KDE
[15:20] <dpm> it's a matter of finding out how many "duplicate" uploads there have been since we stopped imports
[15:20] <jtv> dpm: the real disruption is the ongoing delays though—there's no disruption to non-Ubuntu imports, and for Ubuntu ones it may not (←guess, to be verified) make much of a difference now relative to the existing delay.
[15:20] <jtv> Also bear in mind that duplicate uploads are processed much faster.
[15:21] <dpm> jtv, I'm talking about Ubuntu only, I know the other queues are not stopped
[15:22] <dpm> jtv, I didn't know duplicate uploads were processed faster, I understood from your comments that was only the case if they came from the same uploader
[15:22] <jtv> If they come from the same uploader, then the new upload simply replaces the old one so only 1 is processed at all.
[15:23] <jtv> But processing a msgstr that's identical to the current translation of that msgid is very fast.
[15:24] <dpm> jtv, anyway, I just thought I'd ask, I'll let you guys figure it out. The reason why I thought about this is because we did this once in the past for OO.o uploads (i.e. discard all but the latest upload), and it was my understanding that it did speed things up
[15:25] <jtv> dpm: it did, yes… we'll have a look to see if there's a similar effect.
[15:25] <jtv> OO.o would slow down anything.  :)
[15:25] <dpm> :)
[15:26] <dpm> the same with KDE, it's a big import
[15:35] <gmb> danilos: r=me with a couple of tweaks (one of which is a longstanding bug that I put there and never noticed ;)).
[15:38] <danilos> gmb, heh, sorry, I don't want to fix *your* stuff :P
[15:38] <danilos> gmb, anyway, thanks :)
[15:38] <gmb> Hah.
=== salgado is now known as salgado-lunch
[15:43] <danilos> gmb, btw, since I am not too comfortable around JS, should it really be "var namespace.blah_blah = ..." or just "namespace.blah_blah = ..."?
[15:43] <danilos> gmb, (i.e. without the "var")
[15:43] <gmb> danilos: Argh. Without the var.
[15:43] <gmb> I always make that mistake
[15:43] <danilos> gmb, heh, ok, thanks :)
[15:43] <gmb> And it takes me ages to debug it. Good catch.
=== deryck is now known as deryck[lunch]
[15:49] <danilos> gmb, for testing the advanced_subscription stuff, do I have to worry about malone-alpha memberships and such?
[15:49] <gmb> danilos: Not on lp.dev, no.
[15:51] <danilos> gmb, cool, thanks
=== matsubara-lunch is now known as matsubara
=== beuno is now known as beuno-lunch
[16:10] <sinzui> allenap: what do you intend to do about the merge/delete person timeout?
[16:11] <allenap> sinzui: I'm going to look at the code then, actually, talk to you about it :)
[16:11] <allenap> sinzui: I was going to see if there's a registry job already set up that I could piggy back on.
[16:11] <sinzui> yep. That was Edwin's intentions
[16:12] <allenap> Cool. I'll head in that direction.
[16:12] <sinzui> allenap: I updated the question because the timeout is related to a second bug. The proc is transferring the bug subscriptions to ~registry. That is just wrong
[16:14] <sinzui> allenap: They are separate issues. But I know how to remove the 196 bug subscriptions. Either dholbach can use the script I posted, or make ~registry the team owner so that I can clean up the bugs first
=== salgado-lunch is now known as salgado
=== henninge-tea is now known as henninge
[16:17] <allenap> sinzui: Is there are bug for "transfer to ~registry on delete"? I assume it should just unsubscribe.
[16:18] <sinzui> allenap: bug 613604
[16:18] <sinzui> wrong
[16:18] <sinzui> allenap: bug 613603
[16:18] <_mup_> Bug #613603: team merge subscribes/assignes bugs to ~registry <lp-registry> <merge-deactivate> <Launchpad itself:Triaged> < https://launchpad.net/bugs/613603 >
[16:18] <allenap> sinzui: I was going to do a work-around like the script you suggest, but I'll try and get this bug fixed first... it would be nice to see it work.
[16:20] <allenap> sinzui: Thanks.
[16:22] <sinzui> allenap: Edwin intended to use PersonTransferJob. The job could be membership change, person/team merging or person to person emails
[16:23] <allenap> Cool.
[16:29] <jcsackett> could someone help me figure out an lp.dev codehosting issue? i've run sync_branches but still can't get a branch scanned. :-/
[16:40] <jcsackett> ah, got it. competing process hurting codehosting.
[16:43] <allenap> sinzui: It looks like a lot of the logic in AdminTeamMergeView should be folded into the new job, if not into Person.merge() itself.
[16:43] <sinzui> allenap: oh, thanks for bringing that up
[16:44] <sinzui> I think I reported a bug that the view logic can be pushed lower I was thinking the model, but the job may also be the right place. We may want to initiate a merge/delete over the api.
[16:44]  * sinzui looks for bug
[16:47] <sinzui> allenap: there is no bug. I think the conversation took place in an MP. Once we got the view logic cleaned up, it looked like we could finally get the rules into something beneath the view
=== deryck[lunch] is now known as deryck
[16:49] <allenap> sinzui: Yeah, I am kind of pointing out the obvious. I think I might move only AdminMergeBaseView.doMerge() into the job at first.
[16:50] <sinzui> allenap: yes. Just do what is obviously needed. We have fumbled pushing the logic lower before. It has only been in a state that looked doable since December of last year
[16:50] <allenap> sinzui: Okay, thanks.
[16:52] <jkakar> FWIW, and maybe it's just from years of being used to the old, I'm finding the grey headings make things harder to see... again, could just be that I'm just very used to the old maroon, but this is my initial experience.
[16:53] <sinzui> jkakar: it is harder. huwshimi and I discussed it.
[16:53] <sinzui> jkakar: The colour was bad, but at least is hid the font-size and spacing issues that are *really* bad in Lp
[16:54] <sinzui> jkakar: I am doing a spike this evening to fix the font-size issue. If it works, huwshimi will do a followup to address heading
[16:56] <jkakar> sinzui: Awesome, thanks. :)
[16:56] <jml> jkakar: what about the old blue, orange, green etc?
[16:56] <jml> jkakar: is it just maroon that's easier to read?
[16:56] <jkakar> jml: I miss the red 'Report a bug' link.
[16:56] <sinzui> jkakar: is landscape using Canonical web guidelines
[16:57] <jkakar> Somethings are a bit weird... for example, the THEADs on the +activereview page are much stronger than the headings that describe what each section is, which feels backwards.
[16:57] <sinzui> jkakar: oh, yes, we discussed the participation links too. We are uncertain. We decided to try consistency first. We lost something though.
[16:58] <sinzui> jkakar: wow, it's like you are reading my reviews and emails to huwshimi
[16:58] <jkakar> sinzui: :)
[16:58] <sinzui> yes the headers are odd, many are inconsistent because the engineer hacked the css/style rules
[16:58] <sinzui> We need to tear out a lot of exceptions in templates
[16:59] <jkakar> One thing I thought, particularly from looking at a merge proposal, was, "Oh, they're trying to make user content jump out and be more prominent than all the chrome"... if that's the case, great, but I think it needs a bit more iteration. :)
[16:59] <jkakar> sinzui: Yeah, that kind of work is really tedious, but important.
[17:01] <sinzui> jkakar: it certainly does need iteration. Unlike other updates, we are not stopping releases for a few months. I think the font-size issue is becoming a blocker. Canonical Web guildlines have some good advice, but we do not know why px is used instead of ems.
[17:01] <jkakar> sinzui: I don't know if would help, but we've been through a lot of careful iteration and selection of colours in Landscape with the redesign we did last year.  It might be a place to look for ideas/inspiration.
[17:02] <jkakar> sinzui: I don't why they're used, either.
[17:02] <jkakar> sinzui: I think it's because designers use Photoshop and think naturally in terms of px's.  em's are a bit of a random measurement.
[17:02] <jkakar> sinzui: alejandra and yaili are good people to talk to about the guidelines, especially to get insight about questions like that.
[17:04] <sinzui> jkakar: thanks! I think there is something else at play. browser engines are now running on a many tiny and huge screens. I suspect the engines are using px and a baseline rather than absolute units for fonts.
[17:05] <jkakar> sinzui: They're also good people to report problems regarding the guidelines, since they basically maintain them.  We found a number of issues during our redesign that resulted in tweaks to the guidelines.
[17:06] <sinzui> fab, I do not think the bug I reported were seen
=== gmb changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | firefighting: - | On call reviewer: leonardr | https://code.launchpad.net/launchpad-project/+activereviews
[17:24] <abentley> deryck, we need to have the inverse of translation merging ("splitting") for when a Packaging is deleted, so we should probably put that on Kanban.
[17:25] <deryck> abentley, ok.  Do you mind adding a bug and card for that?
[17:25] <abentley> deryck, okay.
[17:25] <deryck> thanks!
[17:39] <LPCIBot> Project db-devel build (367): STILL FAILING in 5 hr 16 min: https://hudson.wedontsleep.org/job/db-devel/367/
[17:39] <LPCIBot> Launchpad Patch Queue Manager: [rs=buildbot-poller] automatic merge from stable. Revisions: 12377,
[17:39] <LPCIBot> 12378, 12379 included.
=== leonardr is now known as leonardr-afk
[18:10] <jml> just wrapping up for the day, around for another 20 or so minutes.
[18:11] <lifeless> jml: hi
[18:11] <jml> lifeless: hi
=== deryck_ is now known as deryck
[18:16] <LPCIBot> Project devel build (444): STILL FAILING in 5 hr 36 min: https://hudson.wedontsleep.org/job/devel/444/
[18:16] <LPCIBot> * Launchpad Patch Queue Manager: [r=henninge][bug=718761] Ensure that NewBuildersScanner only detects
[18:16] <LPCIBot> a new builder once. Previously,
[18:16] <LPCIBot> any builder registered after start-up would be detected as "new" every
[18:16] <LPCIBot> 5 minutes.
[18:16] <LPCIBot> * Launchpad Patch Queue Manager: [r=adeuring][ui=none] [r=adeuring][bug=715802] Add a vestigial page
[18:16] <LPCIBot> for the new Bug:+subscriptions view.
[18:16] <LPCIBot> * Launchpad Patch Queue Manager: [r=lifeless][no-qa] SQL statements logged in OOPs and ++profile++
[18:16] <LPCIBot> output include the value of their bind variables.
[18:16] <LPCIBot> * Launchpad Patch Queue Manager: [r=leonardr][bug=715854] Don't set filereferences when creating
[18:16] <LPCIBot> POTMsgSets during translation import.
[18:17] <jml> You know, that bot would be better if it just showed the failing tests
[18:17] <jml> rather than the merges
[18:17] <jml> StevenK: LPCIBot should show the failing tests, rather than the merges.
[18:20] <lifeless> thats a great idea
[18:28] <LPCIBot> Project db-devel build (368): STILL FAILING in 48 min: https://hudson.wedontsleep.org/job/db-devel/368/
[18:28] <LPCIBot> Launchpad Patch Queue Manager: [rs=buildbot-poller] automatic merge from stable. Revisions: 12380,
[18:28] <LPCIBot> 12381 included.
=== al-maisan is now known as almaisan-away
=== leonardr-afk is now known as leonardr
=== beuno-lunch is now known as beuno
[19:38] <beuno> so, what's up with the color change in bug titles?
[19:43] <lifeless> colour change?
[19:43] <beuno> they're not red anymore
[19:43] <beuno> they're gray
[19:44] <lifeless> they were red?
[19:44] <lifeless> anyhow, the h1 colours are now consistent on all the domains
[19:44] <beuno> heh
[19:44] <beuno> yes, they where always red on bugs, bugs was already red
[19:45] <beuno> so all h1's are light-ish gray?
[19:45] <lifeless> thats my understanding; sinzui and huwshimi are going to tweak it some more though
[19:46] <beuno> I'd propose a more title-y color, something that instead of blending into the background stands out
[19:47] <sinzui> beuno: the shades will change. If Lp had a colour (aubergine, orange, etc...) we could use that
[19:47] <sinzui> beuno: I am going to attempt a final font-size fix tonight
=== _starbuck is now known as _starbuck-bbl
[20:07] <thumper> morning
[20:12] <lifeless> jcsackett: is bug 242461 a dup
[20:12] <_mup_> Bug #242461: Bug search returning results for inactive projects. <404> <lp-bugs> <oem-services> <search> <Launchpad itself:Triaged> < https://launchpad.net/bugs/242461 >
[20:14]  * jcsackett looks
[20:15] <jcsackett> lifeless: looks like it. bug 632847 is specifically about an oops when following links b/c of that, but it's the same issue.
[20:15] <_mup_> Bug #632847: Bug page OOPS when viewed in deactivated project context <404> <bad-commit-12363> <lp-bugs> <oops> <qa-bad> <Launchpad itself:In Progress by jcsackett> < https://launchpad.net/bugs/632847 >
[20:16] <lifeless> jcsackett: I was looking for some other stuff and it seemed awfully familiar :)
[20:21] <jcsackett> yes indeed. :-P
[20:21] <jcsackett> lifeless: i'll go ahead and mark it.
=== matsubara is now known as matsubara-afk
[20:51] <benji> Anyone know what's wrong with my lint? http://pastebin.ubuntu.com/567459/
[20:58] <benji> leonardr: hi, have time for a quick review? https://code.edge.launchpad.net/~benji/launchpad/bug-649252/+merge/49876
[20:58] <leonardr> benji, sure
[20:59] <benji> thanks
[21:03] <wallyworld> thumper: i cn hear you
[21:04] <wallyworld> thumper: i'll try restarting :-(
[21:10] <wallyworld> thumper: not sure what'
[21:10] <wallyworld> faaaark
[21:10] <wallyworld> stupid mic
[21:11]  * wallyworld tries something else
[21:11] <wallyworld> thumper: let me try and get it sorted and i'll get back to you
[21:12] <thumper> leonardr: mumble?
[21:12] <thumper> StevenK: around?
[21:14] <wallyworld> thumper: i tries skype and that worked
[21:15] <thumper> haha
[21:15] <thumper> mumble hates you
[21:15] <wallyworld> don't know what's up with mumble though
[21:15] <leonardr> thumper, yeah
[21:19] <leonardr> thumper, for future reference: http://pastebin.ubuntu.com/567472/
=== cr3_ is now known as cr3
=== lifeless_ is now known as lifeless
[21:44] <leonardr> benji, r=me with very minor changes, sorry for the wait
[21:45] <benji> leonardr: no problem; gave me time to figure out why lint hates me
=== leonardr changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | firefighting: - | On call reviewer: - | https://code.launchpad.net/launchpad-project/+activereviews
[21:52] <wgrant> mwhudson: We didn't know at the time that it was running out of file handles :(
[21:53] <mwhudson> wgrant: yeah
[21:53] <mwhudson> production debugging is such a joy!
=== salgado_ is now known as salgado-afk
[21:56] <benji> leonardr: I don't know if you have time or not, but speaking of lint breakage, I have a branch ready for review that fixes the problem: https://code.edge.launchpad.net/~benji/launchpad/make-lint-ignore-grep-env/+merge/49884
[21:56] <leonardr> sure, why not
[22:05] <leonardr> benji, makes sense to me
[22:06] <benji> cool
[22:25] <poolie> benji, thanks for the doc feedback
[22:25] <poolie> leonardr, hi
[22:25] <leonardr> poolie, hi
[22:26] <benji> my pleasure
[22:28] <poolie> leonardr, i'm just wondering if there's a news-feed-like-thing that client authors ought to be reading to tell them about this
[22:28] <leonardr> poolie, what is this? the changes to launchpadlib?
[22:29] <poolie> yes, launchpadlib
[22:29] <poolie> or a what's new webpage or something
[22:30] <leonardr> there's no news feed. we can turn the CHANGELOG into a news fee
[22:30] <leonardr> d
[22:30] <poolie> i don't care so much about it being rss (i don't use that)
[22:30] <poolie> as just being told "oh you shouldn't use launchpadlib.launchpad anymore" somehow
[22:30] <poolie> perhaps deprecation warnings or api breaks are the most direct means to do so :)
[22:31] <poolie> oh i finished reading "Restful Web Services" on the weekend btw; it was good
[22:31] <leonardr> thanks
[22:31] <leonardr> it's difficult to deprecate a constant, or you would have been getting deprecation warnings for a while
[22:32] <poolie> i know
[22:32] <wgrant> Can't you detect the deprecated service root when it is used?
[22:32] <poolie> i know it's hard
[22:33] <poolie> well, i guess there are two things being deprecated
[22:33] <poolie> the edge server; also the symbol that points to the url
[22:33] <thumper> leonardr: I seem to be having trouble with the launchpadlib test for the setRecipeText
[22:33] <thumper> leonardr: it is saying that AttributeError: 'Entry' object has no attribute 'setRecipeText'
[22:34] <thumper> when using version='1.0' to launchpadlib_for
[22:34] <wgrant> Your WADL is probably out of date.
[22:34] <poolie> for warning people not to use 'edge' we can easily warn when you use that URL
[22:34] <poolie> for the name, i don't know
[22:35] <poolie> just generally speaking i think python deprecation warnings are pretty poor
[22:35] <poolie> because they tend to appear to users who can't necessarily do anything about them
[22:36] <poolie> a warning at compile time (if there was such a thing) would be better
[22:36] <lifeless> upstream python has turned off display of deprecation warnings by default
[22:36] <lifeless> !
[22:37] <leonardr> there's also no way to force developers to read anything or to comply with any of our standards--witness the oauth problems
[22:38] <leonardr> thumper: i would also suspect out-of-date wadl, but that shouldn't be a problem in a launchpad test
[22:38] <leonardr> also, if you had out of date wadl it would have setRecipeText
[22:38] <lifeless> run 'make'
[22:38] <leonardr> run make just to be sure
[22:39] <thumper> ok
[22:40]  * wallyworld takes son to doctor, back soon
[22:44] <poolie> leonardr, so i'm kind of inclined to focus on making things fail in an obvious way
[22:45] <poolie> and warning people ahead of time by documentation, eg blog posts
[22:45] <poolie> rather than changes in the program
[22:45] <poolie> not sure though
[22:46] <leonardr> poolie: well, there was a blog post back in october that nobody read, since when there was a fire-drill failure yesterday one common response was "it's too much trouble to fix this"
[22:46] <leonardr> i'm not saying that a deprecation warning will do any better
[22:46] <leonardr> but the other thing i did yesterday was make the old code give the new behavior
[22:46] <leonardr> which will help
[22:46] <poolie> i think your thing of just making edge send people to lpnet is good
[22:47] <poolie> considering the constraints of not being able to rewrite on the server side
[22:47] <poolie> and that clients were for a long time required or encouraged to ask for edge
[22:47] <poolie>  "it's too much trouble to fix this" --> meaning what?
[22:47] <poolie> too hard to update clients to not use edge?
[22:47] <leonardr> poolie: right. "i have 20 machines" kind of thing
[22:48] <leonardr> that was how i found out it was an accident, actually--one person wouldn't take "make this trivial change" for an answer, and escalated
[22:49] <poolie> anyhow
[22:49] <poolie> https://code.launchpad.net/~leonardr/launchpadlib/silently-replace-edge-with-production/+merge/49794 seems reasonable to me
[22:54] <thumper> leonardr: lp:~thumper/launchpad/recipe-inline-edit-recipe-text is still getting the attribute error
[22:57] <leonardr> checking
[22:57] <leonardr> thumper: here are a couple things to try
[22:58] <leonardr> 1. don't bother removing the method and re-exporting it. just publish it as a mutator write operation
[22:58] <leonardr> that might give you the behavior you want
[22:58] <StevenK> thumper: My setRecipeText change just landed, too.
[22:59] <thumper> StevenK: which change is that?
[22:59] <StevenK> thumper: https://code.launchpad.net/~stevenk/launchpad/set-recipe-text-bad-data/+merge/48853
[23:04] <thumper> leonardr: you mean remove;     @export_write_operation(),   @operation_for_version("devel") from the middle?
[23:04] <leonardr> thumper: yes
[23:04] <leonardr> i think we can treat this the way we treated transitionToStatus
[23:05] <leonardr> no, unfortunately not
[23:05] <leonardr> transitionToStatus was changed to a mutator in 'beta', not '1.0'
[23:05] <leonardr> how long has this setRecipeText operation been in place, anyway? isn't this whole feature new?
[23:06] <thumper> leonardr: it complains if I delete them
[23:06] <leonardr> let me see the complaint
[23:08] <thumper> http://pastebin.ubuntu.com/567499/
[23:08] <leonardr> thumper, how new is this feature? unless it's older than version 1.0 of the web service there's no need to do this
[23:11] <leonardr> ie. you can change the 'devel' version around however you want, without respect for backwards compatibility
[23:14] <leonardr> thumper: you should also remove the 'operation_removed_in' annotation if you didn't already
[23:14] <thumper> leonardr: how do I find out if they are in 1.0?
[23:14] <leonardr> thumper: bzr annotate is probably the best way
[23:15] <thumper> leonardr: when did we cut 1.0?
[23:15] <leonardr> quite a while ago, possibly a year
[23:15] <leonardr> "It was introduced in March 2010"
[23:19] <thumper> hmm... very possible recipes weren't in 1.0
[23:20] <thumper> how do we generate the wadl for 1.0?
[23:21] <leonardr> thumper: it's based on the annotations
[23:21] <leonardr> the thing is that once someone published setRecipeText as a named operation, it showed up in all the versions, including 1.0
[23:22] <leonardr> since it wasn't explicitly excluded from other versions
[23:22] <wgrant> Could we invert that behaviour?
[23:22] <leonardr> but, given that it wasn't in the original 1.0, and that i suspect it was added pretty recently, i think we can just get rid of it
[23:22] <wgrant> So things only show up in devel unless they are explicitly annotated?
[23:22] <thumper> leonardr: I agree
[23:22] <leonardr> wgrant: it would be a big project since we'd have to manually grandfather in everything else
[23:22] <wgrant> leonardr: But otherwise versions are not versioned :/
[23:24] <leonardr> wgrant: our policy is to let older versions have new stuff, but not to *change* behavior. we decided that anything else was a lot of work to prevent things that generally range from "slightly harmful" to "actually good"
[23:25] <wgrant> leonardr: But then we need to explicitly restrict things to devel if we think we ever want them to change.
[23:26] <wgrant> We should introduce features to devel and then push them back later.
[23:26] <wgrant> as an explicit step.
[23:26] <leonardr> we already have an explicit step "don't publish named operations that are effectively mutator methods", and setRecipeText already slipped through that crack
[23:27] <wgrant> Well, that's this particular case.
[23:27] <wgrant> But in general, an immature recipe API should not have slipped into 1.0.
[23:28] <leonardr> but no policy i could have set would have prevented it, since the much simpler policy i set wasn't followed
[23:28] <thumper> leonardr: source package recipes were not exported at all at the 1.0 time
[23:28] <wgrant> leonardr: This is why we need safe defaults, I think.
[23:28] <wgrant> Policies are easily missed.
[23:29] <thumper> leonardr: so... the first adapter (on the bottom) should be     @operation_for_version("devel")?
[23:30] <leonardr> thumper: i suggest simply removing all references to versions
[23:30] <leonardr> @operation_for_version("devel") will remove setRecipeText from 1.0 and replace it with nothing
[23:30] <thumper> leonardr: it seams reasonable that we should be able to annotate an Entry to say "added in devel"
[23:31] <leonardr> thumper: we decided not to add support for that because it was too much work for too little benefit
[23:31] <thumper> leonardr: so how do I say "this method is only in devel"?
[23:31] <leonardr> thumper: that's how you'd do it. @operation_for_version("devel")
[23:31] <james_w> ok
[23:31] <leonardr> but it's already in 1.0, so when you fix it in devel you might as well let the fix propagate to 1.0
[23:32] <thumper> leonardr: so ... http://pastebin.ubuntu.com/567508/
[23:32] <thumper> hmm..
[23:33] <thumper> what if we take recipe stuff out of 1.0?
[23:33] <lifeless> thumper: its new right ?
[23:33] <leonardr> wgrant: if we were twitter or some other very popular web service i would be very anal about this. but as it is, we don't have the manpower to maintain velocity with strict separation between versions
[23:33] <thumper> lifeless: new since 1.0 yes
[23:33] <lifeless> thumper: I think new stuff should be devel only
[23:33] <lifeless> IMO we shouldn't change 1.0 at all
[23:33] <thumper> lifeless: we are discussing how to manage this
[23:34] <leonardr> ok. when you publish recipes on the web service, you can publish all of its named operations in devel only
[23:34] <leonardr> you can publish all of its fields in devel only
[23:34] <thumper> it would be nice to just have checked in WADL for defined versions
[23:34] <wgrant> leonardr: We can take a big step towards that by just making the default devel-only.
[23:34] <leonardr> you can publish all links to recipe type objects in devel only
[23:35] <wgrant> leonardr: When a feature is done, we promote it to 1.0.
[23:35] <wgrant> If people use a feature in devel, it's their problem if their script breaks.
[23:35] <leonardr> wgrant: if the default is devel only then we (i) must manually promote hundreds of existing features
[23:35] <wgrant> And they will realise it's unstable.
[23:36] <leonardr> thumper: but, you can't stop something called a 'recipe' from showing up in the wadl for earlier versions, and the syntax for restricting all this stuff will be ugly
[23:37]  * lifeless would like the default to be devel only; and when we cut 1.1 we shift the things that are stable from devel to 1.1
[23:37] <leonardr> because our original design was based around doing version restrictions *when there was a problem with letting stuff bleed through into older versions*
[23:37] <thumper> leonardr: why not?
[23:38] <leonardr> we can re-evaluate this, we can always re-evaluate this, but it's going to be a big project during which imo we are not delivering value to users
[23:38] <thumper> I guess it depends if you are a user of the 1.0 webservice and using things you thought were stable, but weren't
[23:39] <leonardr> to clarify, our original design was based around doing version restrictions when something *changed* between versions, not when something was added
[23:39] <thumper> I can see that
[23:39] <leonardr> we can do this:
[23:40] <leonardr> "we created the named operation setRecipeText. that was a mistake"
[23:40] <leonardr> "let's leave setRecipeText alone in 1.0 and change it in devel"
[23:40] <thumper> that's what we were trying to do
[23:40] <leonardr> right
[23:40] <thumper> but the tests weren't working
[23:40] <thumper> I got a launchpadlib for 1.0
[23:40] <leonardr> then i proposed checking to see how recent the feature is
[23:40] <thumper> and it says setRecipeText doesn't exist
[23:41] <leonardr> because maybe we can just change it everywhere
[23:41] <leonardr> and short-circuit this problem
[23:41] <thumper> it may well be that no-one uses this at all through the web service
[23:41] <thumper> I'd be happy to just change this everywhere for this method
[23:42] <thumper> but my concerns are over the versioning as a whole for the web service
[23:42] <leonardr> well, as a whole the rule is to do this by the book
[23:42] <leonardr> so let's do this by the book
[23:42] <leonardr> and then you'll see how it works
[23:42] <thumper> ok...
[23:42] <thumper> so lets try to get the annotations working then
[23:42] <leonardr> so, why is the test failing? i don't know. i'll investigate
[23:43] <leonardr> it's possible there's a bug in lazr.restful. i don't know if we have a real case where a method is removed and then reinstated as a mutator
[23:43] <thumper> ok
[23:43] <thumper> can I leave this test with you to check?
[23:43] <thumper> lp.code.model.tests.test_sourcepackagerecipe.TestWebservice.test_recipe_text_setRecipeText_in_one_zero
[23:43] <thumper> that is the test that fails
[23:43] <thumper> in the branch
[23:45] <leonardr> ok
[23:46] <leonardr> doing it by the book this time is annoying, but the advantage this has over 'devel only by default' is that in the majority of cases where we didn't make a mistake, we don't have to add cumbersome annotations, and people who are using 1.0 get the new stuff for free
[23:48] <thumper> IMO 1.0 should not get new stuff
[23:49] <thumper> but that's just me
[23:49]  * thumper needs food
[23:49] <leonardr> thumper: it's not just you, but the amount of work required to stop 1.0 from getting new stuff is nontrivial. and what are you doing? making 1.0 less useful
[23:50] <StevenK> The other side of the coin is 'making 1.0 more stable, and doing what we said we would'
[23:51] <lifeless> leonardr: FWIW we make mistakes most of the time
[23:53] <leonardr> lifeless: maybe 'bleed through by default' was just another mistake then
[23:53] <leonardr> thumper: the operation works in beta but not 1.0. that makes me think there may be a bug in lazr.restful
[23:54] <lifeless> leonardr: the reason I say we make lots of mistakes is because we do incremental development
[23:54] <lifeless> leonardr: so its baked in that we'll iterate and tweak
[23:55] <lifeless> I think this is a positive thing, but the friction it has with support guarantees like 1.0 is pretty high.
[23:55] <leonardr> i say this because lazr.restful has a feature that let us deprecate 'transitionToStatus' et al. in a reasonable way, and that might be interfering with what we're doing here
[23:55] <lifeless> interesting
=== Ursinha is now known as _starbuck
[23:56] <leonardr> lifeless: so, it sounds the early-2010 web service versioning strategy conflicts with the early-2011 development strategy
[23:59] <lifeless> leonardr: tell me about this deprecation feature
[23:59] <leonardr> lifeless: look at lib/canonical/launchpad/rest/configuration.py
[23:59] <leonardr> last_version_with_mutator_named_operations = "beta"