[00:05] <wgrant> lifeless: re. bug #137448: I fixed the model to permit full retargeting, and added support for that to the legacy, non-AJAX UI.
[00:05] <_mup_> Bug #137448: New UI is confusing and counter inuitive for changing affected package <lp-bugs> <Launchpad itself:Triaged> < https://launchpad.net/bugs/137448 >
[00:06] <lifeless> wgrant: bug 163244 needs you to pick *A* flaw in it ;)
[00:06] <_mup_> Bug #163244: CVE list needs some triaging features <lp-bugs> <Launchpad itself:Triaged> < https://launchpad.net/bugs/163244 >
[00:06] <lifeless> wgrant: so 137448 is fix released ?
[00:06] <lifeless> wgrant: or you mean 'not fixed because the ajax ui isn't fixed'
[00:08] <wgrant> lifeless: I thought that it was referring to the AJAX UI, but I think that bug might predate AJAX. I suspect the specific change the bug complains about is in fact the introduction of a link to the target in the task row, rather than just in the expander where it was initially.
[00:08] <wgrant> lifeless: However, until packages can be reassigned by AJAX and the expander is removed, I think the bug is probably still valid.
[00:09] <lifeless> wgrant: yup, I get that
[00:09] <lifeless> thanks
[00:16] <huwshimi> wallyworld_: Ready whenever you are
[00:16] <wallyworld_> huwshimi: ok. mumble?
[00:16] <huwshimi> wallyworld_: sure
[00:22] <lifeless> wgrant: get spec by mail ..
[00:23] <lifeless> (the email fail)
[00:23] <wgrant> Yeah, but that's not the main issue here.
[00:23] <wgrant> I've only seen that error once.
[00:23] <lifeless> oh, I see dkim ?
[00:23] <lifeless> is it casting str to unicode?
[00:24] <wgrant> I can see a particular DKIM log message missing from process-mail.log each time we get that cronspam.
[00:24] <lifeless> done for today, down to 679 medium bugs
[00:24] <wgrant> 2011-09-22 18:51:13 INFO    Attempting DKIM authentication of message id=<db6701cc7969$046cade0$718d875f@BOOTHRpy> from=<BOOTHRpy@ffni.com> sender=None
[00:24] <wgrant> That one.
[00:24] <wgrant> I suspect the crash is in from/sender.
[00:25] <wgrant> Can we upgrade to Python 3 yet?
[00:25] <lifeless> anytime
[00:25] <lifeless> if it works
[00:26] <wgrant> Hmm, do I want to %r or try to unicodeify them...
[00:26] <wgrant>     log.info(
[00:26] <wgrant>         'Attempting DKIM authentication of message id=%s from=%s sender=%s'
[00:26] <wgrant>         % (signed_message['Message-ID'],
[00:26] <wgrant>             signed_message['From'],
[00:26] <wgrant>             signed_message['Sender']))
[00:26] <wgrant> That's what we're doing now.
[00:53]  * wallyworld_ goes out for lunch
[00:54] <lifeless> wgrant: looks like one of those is unicode then
[00:56] <wgrant> Traceback (most recent call last):
[00:56] <wgrant>   File "/usr/lib/python2.6/logging/__init__.py", line 791, in emit
[00:56] <wgrant>     stream.write(fs % msg.encode("UTF-8"))
[00:56] <wgrant> UnicodeDecodeError: 'ascii' codec can't decode byte 0x98 in position 120: ordinal not in range(128)
[00:56] <wgrant> lifeless: It's a decode error on encode.
[00:57] <wgrant> Doesn't that mean it's trying to encode a str?
[00:57] <lifeless> implicit conversions
[00:58] <lifeless> 'foo %s' % u'bar' -> u'foo bar'
[00:58] <wgrant> Fair point, missed the % bit.
[00:58] <wgrant> Focused on the encode.
[00:58] <wgrant> Anyway, -> maintenance peeps
[00:59] <lifeless> the log message ends up being unicode because its % (...)
[01:00] <lifeless> or, ah
[01:00] <lifeless> it may be a str that is utf8, but 'fs' isn't ascii decodable
[01:00] <lifeless> so fs % msg-as-unicode blows up
[01:00] <lifeless> need to see the values to know
[01:03] <lifeless> blah, my turn to miss stuff :)
[01:03] <lifeless> anyhow, yes we need to fix
[01:04] <wgrant> wallyworld_: Could you review <https://code.launchpad.net/~wgrant/launchpad/moar-fragile/+merge/76666>? About as trivial as they get.
[01:12] <lifeless> wgrant: why don't you self review it ?
[01:13] <wgrant> I didn't really want to self-review a change like that, but might as well.
[01:14] <lifeless> why not?
[01:14] <wgrant> It has the potential to do bad things to fastdowntime unless I know what I'm doing.
[01:14] <wgrant> This stuff is already fragile enough :)
[01:15] <lifeless> you're adding one name
[01:16] <lifeless> I can't imagine a reviewer at the code style level having anything to contribute
[01:16] <lifeless> you're either right, or wrong ;)
[01:16] <wgrant> I didn't want a style review :)
[01:22] <lifeless> wgrant: less use of imperatives in bug reports!
[01:30] <wgrant> lifeless: Fair point.
[01:42] <wgrant> wallyworld_: How goes QA for your trigger?
[02:13] <wallyworld_> wgrant: i'll ping stub when he is available. easier that way
[02:13] <wallyworld_> wgrant: sorry i missed your review. i was having an early lunch with a friend
[02:13] <lifeless> what do you need done?
[02:14] <wallyworld_> lifeless: i need to run some sql to update branch.private = true for say launchpad, and then check on of the branches stacked on lp to see that transitively_private is true
[02:15] <wallyworld_> plus say, once lp is private, unstack a branch and see that transitively_private is false
[02:19] <lifeless> so, future ref, use a losa
[02:20] <lifeless> 24h coverage, which stub cannot supply.
[02:20] <wallyworld_> sure. i thought in this case it would be easier sonce he was familiar with the tirgger etc
[02:20] <wallyworld_> less hand holding required
[02:21] <lifeless> a little yes, but it adds latency, and we're kindof latency sensitive once things are in trunk
[02:21] <wallyworld_> this is in db-devel
[02:21] <lifeless> yees
[02:21] <lifeless> thats a trunk
[02:21] <lifeless> as is devel
[02:21] <wallyworld_> doh
[02:21] <lifeless> s/trunk/thing/ we deploy from
[02:22] <wallyworld_> for some reason i only think of devel as the "real" trunk
[02:24] <lifeless> when you're not dealing with the schema thats a pragmatic view
[02:25] <lifeless> but with fastdowntime in the mix, db-devel has the ability to deploy as often as nodowntime has been deploying (but not as often as nodowntime /can/ deploy
[02:25] <wallyworld_> yes, 95% of the time it's just devel but you are right about db-devel
[02:52] <wallyworld_> lifeless: i need to catch when a model object property is changed. is the correct pattern in our architecture to make it a r/o property and introduce a setter, marked as @mutator_for(xxx) in the web services interface. the setter would publish an ObjectModifiedEvent which there would be a listener for
[02:53] <wgrant> There are several different ways to do it.
[02:54] <wgrant> What exactly are you trying to achieve?
[02:54] <StevenK> Can anyone reproduce a overlay appearing at the top of a bug page on up-to-date devel? make run and then https://bugs.launchpad.dev/redfish/+bug/15
[02:54] <_mup_> Bug #15: PO file import errors should be more verbose <feature> <lp-translations> <Launchpad itself:Fix Released by carlos> < https://launchpad.net/bugs/15 >
[02:54] <wallyworld_> i need to know when a bug supervisor or security contact has changed on a project
[02:54] <wallyworld_> i need the old and the new values
[02:55] <wallyworld_> StevenK: give me a sec and i'll do it
[02:56] <wgrant> wallyworld_: You're going to attempt to do a mass subscription/unsubscription?
[02:57] <wallyworld_> wgrant: no. if a new bug supervisor is added to a project, they need to be auto subscribed if the project is related to any private bug tasks etc
[02:57] <wgrant> That sounds like a "yes" to me.
[02:57] <wgrant> And a very bad idea.
[02:57] <wgrant> Really slow, and attempting to perform sensible disclosure on top of legacy disclosure.
[02:58] <wgrant> s/perform/implement/
[02:58] <wallyworld_> i've been assigned a bug to do this work
[02:59] <wallyworld_> well, there's a 5 digit but and one i assigned myself which is related which curtis agreed to this morning
[02:59] <wallyworld_> s/but/bug
[02:59] <wgrant> And I will happily revert any attempt to do that, because it will time out.
[03:00] <wallyworld_> that's very prsumptuous
[03:00] <wgrant> It's really not, unfortunately :/
[03:00] <wgrant> We can do this once we have project observers.
[03:00] <wgrant> But updating a couple of hundred thousand bug subscriptions in-request is not cheap.
[03:01] <wallyworld_> this is only for private bugs
[03:02] <wallyworld_> surely there's not 100000's of private bugs
[03:02] <wallyworld_> StevenK: what am i looking for?
[03:04] <StevenK> wallyworld_: http://people.canonical.com/~stevenk/overlay.png
[03:04] <wallyworld_> StevenK: don't see that
[03:05] <StevenK> That is upsetting.
[03:05] <StevenK> Then WTF is causing it for me.
[03:05] <wallyworld_> try make jsbuild? is your js up to date
[03:05] <StevenK> That is from a clean devel
[03:06] <wallyworld_> hmmm. i just pulled the latest devel tip myself and did make clean run
[03:06] <wallyworld_> is it just on the bug index page?
[03:07] <StevenK> Hm, no.
[03:07] <wgrant>  product | distribution | count
[03:07] <wgrant> ---------+--------------+-------
[03:07] <wgrant>          |            1 | 29441
[03:07] <wgrant>     1694 |              |  3598
[03:07] <wgrant>     9514 |              |  3196
[03:07] <wgrant>     2982 |              |  1868
[03:07] <wgrant>     8892 |              |  1607
[03:07] <wgrant>    10294 |              |  1241
[03:07] <StevenK> Many private bugs
[03:09] <StevenK> Shows up on any bug page and https://launchpad.dev/~landscape-developers
[03:09] <wgrant> StevenK: I don't see it either.
[03:09] <StevenK> :-(
[03:09] <wgrant> Whatever Firefox Oneiric has this week.
[03:09] <StevenK> Then why do I? :-(
[03:10] <wallyworld_> i'm runnign latest ff beta and don't see it
[03:11] <StevenK> Perhaps Firebug is confounding me
[03:12] <StevenK> Where's the option for halting on errors?
[03:12] <wgrant> StevenK: Tried Chromium?
[03:12] <wallyworld_> StevenK: can't remember
[03:12] <wallyworld_> wgrant: i assume distro number 1 is ubuntu?
[03:13] <StevenK> Yes, Distribution 1 is Ubuntu
[03:13] <StevenK> wgrant: No. Don't really plan on, either. :-P
[03:15] <wallyworld_> so i could do it as a cron task. don't think we have a proper jobs infrastructure with persistent state etc yet?
[03:15] <wgrant> wallyworld_: Why don't we do this in a few months when we have non-legacy disclosure, with observer roles?
[03:15] <wgrant> wallyworld_: Adding a job is not lightweight enough that we should just throw it away in a few months.
[03:17] <wallyworld_> observers will probably work for this, yes
[03:17] <wallyworld_> or we can make sure it works
[03:17] <wgrant> It's meant to do *exactly* this sort of thing.
[03:17] <wgrant> Quickly, too.
[03:18] <wgrant> Without having to write-lock thousands of rows.
[03:18] <wallyworld_> so we would have to have some sort of denormalisation built into the model
[03:18] <wgrant> Oh?
[03:19] <wgrant> Bug supervisor will no longer imply visibility, just like it no longer implies notification.
[03:19] <wgrant> The default will probably still be supervisor == visibility == notification, but like with structural subscriptions that will just be a default.
[03:20] <wallyworld_> oh ok. i would expect that as a bug supervisor i would have visibility, however it's done
[03:20] <wgrant> That doesn't work for Ubuntu.
[03:20] <wallyworld_> because?
[03:20] <wgrant> We have many non-Canonical people in the bug supervisor role, but we have coredumps and stuff that should not be visible to them.
[03:21] <wgrant> (I won't mention security bugs, because sinzui seems to be convinced that they should remain a special case)
[03:21] <wallyworld_> ok. seems like we are saying to someone to do a role, but not necessarily ggiving them all the info they might need ti make decisions?
[03:22] <wallyworld_> or have i misunderstood?
[03:22] <wgrant> The bug supervisor role's primary purpose is to confer access to the restricted statuses, and the ability to set bug importance.
[03:22] <wgrant> It was hijacked to be the default subscriber back when default-private-bugs were introduced, because it was convenient.
[03:23] <wgrant> And that has functioned reasonably well for a few years.
[03:23] <wallyworld_> what does "confer access to the restricted statuses" mean?
[03:24] <wgrant> Allow setting and unsetting of Won't Fix and Triaged, and unsetting of Fix Released.
[03:24] <wallyworld_> ok. but the info need to fix the bug, once triaged and assigned to someone, might remain invisible to the supervisor?
[03:25] <wgrant> They don't know that that bug exists.
[03:26] <wgrant> Unless they are also defined as an observer.
[03:26] <wallyworld_> how can they set bug status if they don't know it exists?
[03:26] <wgrant> that will be a sensible default for most projects, but it cannot be a universal rule.
[03:26] <wgrant> They can't.
[03:26] <wallyworld_> but didn;t you just say that's what the supervisor does?
[03:26] <wgrant> They can set status and importance on the bugs they can see, yes.
[03:26] <wgrant> They obviously can't do it to the bugs they can't see.
[03:27] <wallyworld_> so there's no point making someone a bug supervisor of a bug they can't see
[03:27] <wallyworld_> and yet the rules now are that bug supervisors are subscribed to private bugs
[03:28] <wallyworld_> automatically
[03:28] <wgrant> They are subscribed to private bugs when they are created in a default-private-bugs project.
[03:28] <wgrant> Which is a tiny subset of projects.
[03:29] <wallyworld_> sure
[03:29] <wgrant> And an inflexibility on which we already have bugs filed.
[03:29] <wallyworld_> so with upcoming disclosure work, this will all change?
[03:32] <wgrant> Right. The bug supervisor will probably return to its original purpose, no longer implicitly providing visibility by default in some projects.
[03:32] <wgrant> There's a separate observer list for that purpose.
[03:32] <wallyworld_> makes sense. thanks for the explanation
[03:34] <wgrant> So, while it would be nice to do the subscriber transition on bug supervisor change in the current model, it's probably impractical to do now, and is trivially done in a much simpler way once we have the new privacy model up and running.
[03:43] <wallyworld_> sure
[03:48] <lifeless> guys
[03:49] <wgrant> Uhoh.
[03:50] <lifeless> if we're doing any sort of mass-change, it should be backend driven, and use short (<=2 secs) transactions
[03:50] <wgrant> Yes.
[03:50] <lifeless> for bug supervisors
[03:50] <lifeless> question - is the intent that bug supervisors will be observers?
[03:51] <wgrant> I hope not.
[03:51] <wgrant> If there is, the intent is incorrect.
[03:51] <lifeless> because I expect most projects will want the bug supervisor to see all private non-sec bugs, as they will want the project security team to see all private sec bugs
[03:51] <lifeless> !cite, I know.
[03:52] <wgrant> That is the most common situation that I envisage.
[03:52] <lifeless> How do we intend to help folk achieve that ?
[03:52] <wgrant> I believe the current proposal, which I vehemently disagree with, is to continue special-casing security.
[03:53] <wgrant> Observers won't be able to see bugs with the security flag set.
[03:53] <wgrant> (I haven't heard this confirmed for a couple of months, but it's the last thing I heard)
[03:53] <lifeless> observers not seeing security is correct
[03:53] <lifeless> but
[03:53] <wgrant> sure.
[03:53] <lifeless> perhaps we need types of observers, or something.
[03:53] <wgrant> But I don't think it should be a special case.
[03:53] <wgrant> Yes!
[03:54] <wgrant> But it was argued that that is too complicated or something.
[03:54] <lifeless> it veers close to generic acls
[03:54] <lifeless> but not too close, I think.
[03:54] <wgrant> To me the most obvious solution is to specify a privacy policy on each bug, sort of like what branches have now except explicit and not crippled.
[03:54] <wgrant> Most projects would have two.
[03:55] <wgrant> Private and Security, I guess.
[03:55] <wgrant> Two I can think of for Ubuntu are Apport and Security.
[03:55] <lifeless> right, this harks back to the prague, or was it texas, discussion
[03:55] <wgrant> They probably wouldn't have Private.
[03:55] <wgrant> sinzui has basically vetoed this, I believe.
[03:55] <wgrant> But I don't think we can not have it.
[03:55] <lifeless> where shared-namespace projects would have multiple policies.
[03:56] <lifeless> the big ui issues I see are clearly communicating the impact of choosing such a policy for a bug.
[03:56] <lifeless> implementation wise I think it would be easy and execute fast
[03:57] <wgrant> Exactly.
[03:57] <wgrant> We don't even have to expose it by default.
[03:57] <wgrant> Security could be a UI special case, if we want to keep it just how it is now.
[03:57] <wgrant> Until we smooth things out and open up the general UI.
[03:57] <wgrant> But the key is that the backend is general, just as fast as a security flag.
[03:57] <wgrant> And it lets us handle Ubuntu very well.
[03:57] <wallyworld_> what are some examples of shared namespace projects?
[03:57] <wgrant> Where the security special case does not, at all.
[03:58] <lifeless> wallyworld_: storm
[03:58] <lifeless> wallyworld_: but more seriously, they are in an awkard halfway house at the moment
[03:59] <lifeless> I think we should -either- ditch them and say that projects are single-owner (e.g. if a private entity wants to work on some other public project, they can't do so in private: they need a separate private project which forks the other)
[03:59] <lifeless> or we need to do a lot of remedial work
[03:59] <wallyworld_> dumb question - what about storm makes it shared namespace?
[03:59] <lifeless> it has multiple different branch privacy policies
[03:59] <lifeless> when I push a branch there, you cannot see it.
[04:00] <lifeless> if you push a branch there, I can see it.
[04:00] <lifeless> this is an example of how it works, not how it should be :)
[04:00] <wallyworld_> i didn't realise that
[04:00] <lifeless> theres some enterprise integration branches hardware-qa folk have written
[04:00] <lifeless> not yet opened
[04:01] <wallyworld_> lifeless: so what sets your branches private - an attribute on your IPerson instance?
[04:01] <lifeless> I'm in the hardware-certification team
[04:01] <wallyworld_> ok. and that's a private team?
[04:01] <lifeless> that team matches one of the policies, and wins for whatever reason.
[04:01] <lifeless> the team doesn't have to be private for this to happen
[04:02] <lifeless> (it may be in this case, I genuinely don't know)
[04:02] <wallyworld_> but it's policy based per project
[04:02] <lifeless> yes
[04:02] <wallyworld_> right
[04:03] <wallyworld_> lifeless: so how do people get to do code reviews on your storm mp's if your branches are private?
[04:03] <wallyworld_> i guess only certain people can
[04:03] <lifeless> they need to be in the hardware-certification team
[04:03] <lifeless> its like a whole other project, sharing the namespace
[04:03] <lifeless> own rules, own policies
[04:03] <wallyworld_> sounds blah
[04:03] <lifeless> but its not a complete separation
[04:03] <lifeless> like series are shared
[04:03] <lifeless> so stacking is shared
[04:04] <lifeless> wallyworld_: well, the intent is for things where multiple organisations want to do independent private work, for them to do that.
[04:05] <lifeless> wallyworld_: it also is intended to support things like unity where we have private branches for a while which get opened when hardware vendors ship the final product and we are allowed to open source the code
[04:05] <lifeless> I'm neither justifying nor critiqing this
[04:05] <lifeless> just trying to explain why, what it is, and its current status
[04:06] <lifeless> in the former folk want partitions
[04:06] <wallyworld_> lifeless: sure, and i appreciate the insights
[04:06] <lifeless> in the latter folk want public-private partitioning and then free-for-all behind the partition
[04:06] <lifeless> oem are unique in doing this multiple times with multiple partitions, and they don't use shared namespaces, instead they run N projects (and want us to link things cross-project)
[04:07] <lifeless> arguably these things are equivalent, but choosing one approach and really polishing it probably makes a lot of sense
[04:07] <wgrant> There are also non-commercial instances of this sort of thing.
[04:07] <wgrant> eg. Ubuntu has really private apport bugs, sort of private apport bugs, and security bugs.
[04:08] <wallyworld_> i'm all for making *one* nicely polished process that can be tuned to fit the required use cases
[04:08] <lifeless> thats closely related yes. I wouldn't call it a shared namespace as such
[04:09]  * wallyworld_ sighs. another compiz crash :-(
[04:18] <wgrant> lifeless: Exactly: it's not shared-namespace, but it is an example of the security special case not working.
[04:48]  * wallyworld_ reboots
[05:05] <lifeless> jamesh: hi; I don't suppose you've had a chance to try glueing oops and u1 together again ?
[05:06] <jamesh> lifeless: sadly not.
[05:06] <lifeless> no worries
[05:06] <lifeless> btw, I have a native twisted oops config, if that is useful
[05:07] <lifeless> should be publishing the code today
[05:07] <jamesh> the python-oops bug I filed came from my chasing down a bug in the wrong direction because of similarly named exceptions
[05:08] <lifeless> yeah
[05:08] <lifeless> it makes total sense
[05:08] <lifeless> we did similar to testtools output
[05:08] <jamesh> Python's DB-API encourages lots of similarly named exceptions, including one called "Error"
[05:10] <lifeless> \o/
[05:11] <lifeless> bbiab
[05:11] <jamesh> the spec could really have benefited from a stdlib module for all the adapters to build on top of
[05:22] <rsalveti> sorry asking here, wasn't able to find at launchpad, but is there a bug already on the code import about the ability to only import trunk when importing a git tree?
[05:23] <rsalveti> would like to also be able to import a git branch
[05:23] <rsalveti> I know it's probably because of a limitation of bzr-git or similar, but would like to also see a bug for that at launchpad
[05:25] <rsalveti> guess jelmer should know this :-)
[05:26] <rsalveti> https://answers.launchpad.net/bzr-git/+question/171522
[05:26] <rsalveti> cool, seems the support is already at bzr-git trunk
[05:37] <wgrant> rsalveti: You can actually import them in LP now.
[05:37] <wgrant> rsalveti: Since a week or two ago.
[05:37] <wgrant> It's not documented or obvious yet, however.
[05:37] <rsalveti> wgrant: oh, ok, any special syntax?
[05:38] <wgrant> eg. https://code.launchpad.net/~wgrant/mochiweb/R12B
[05:38] <wgrant> Add ,branch=whatever to the end of the URL.
[05:38] <rsalveti> wgrant: oh, awesome :-)
[05:38] <rsalveti> I love you guys, launchpad rocks
[05:39] <wgrant> There'll hopefully be some UI for it soon, but at least it's possible now :)
[05:39] <rsalveti> just when I needed \o/
[05:39] <rsalveti> we'll be doing CI and daily builds of the linaro components
[05:39] <rsalveti> and they are mostly tracked using git
[05:39] <wgrant> Excellent.
[05:40] <rsalveti> wgrant: thanks
[06:07] <nigelb> lifeless: Hey! You asked me to ping you today for a project skeleton and sketch, etc :-)
[06:10] <nigelb> wgrant: So now, everything that touches db need to have the pre-requisite branches deployed everywhere before landing?
[06:11] <wgrant> nigelb: Sadly not yet. mizuho, cocoplum and germanium still need fixing. mizuho is having other work done to it on probably Monday, which means we should be able to upgrade it then.
[06:11] <wgrant> (it's awkward because it's not HA yet, so we can't safely restart it while LP is up)
[06:11] <wgrant> The work on Monday should make it HA.
[06:11] <nigelb> \o/
[06:12] <wgrant> It was meant to be done last night, but, er, things did not go smoothly.
[06:13] <nigelb> heh
[06:13] <nigelb> Oh well, time to head to work. Laters.
[06:28] <lifeless> nigelb: I did
[06:28] <lifeless> nigelb: uhm, give me half an hour or so
[06:28] <StevenK> wallyworld: Wow, no underscore.
[06:28] <wgrant> Heh
[06:29] <wallyworld> yeah, it ran away
[06:29] <StevenK> wallyworld: No green animated awesomeness for that branch.
[06:29] <wallyworld> ?
[06:29] <StevenK> wallyworld: But I will hit you up on Monday about it.
[06:30] <wallyworld> you mean the latest bug confirmation work
[06:30] <StevenK> confirming unsubscribe on private bugs, yes.
[06:30] <wallyworld> cool
[07:03]  * wallyworld hates debugging @^@$@%$^ doc tests :-(
[07:04] <rvba> Morning.
[07:05] <wallyworld> g'day
[07:05] <rvba> Hey, how's it hanging wallyworld? ;)
[07:06] <wallyworld> rvba: shit. ec2 gave me the finger and now I've got to find an error in a doc test :-(
[07:07]  * rvba hates doc tests too.
[07:07]  * wgrant stabs Storm.
[07:07] <wallyworld> what did Storm do to you?
[07:08] <wgrant> It's generating UPDATE queries with = NULL instead of IS NULL.
[07:08] <wallyworld> wgrant: it also does that for =/is True
[07:09] <rvba> wgrant: I've got a question for you: I have four dependent branches: A -> B -> C -> D; a certain wgrant reverted the whole thing (and he was right) ...
[07:09] <wgrant> Hmmm, I guess I can't exactly use a nullable field in a Storm primary key anyway.
[07:09] <rvba> Now I've created C': A -> B -> C'
[07:10] <rvba> But when I try to land it, it gets merged with devel which has A and B reverted so ec2 gives me the finger (like wallyworld says ;))
[07:11] <wgrant> rvba: What we normally do in that case is create a branch to revert the revert (bzr merge -r$(revert)..$(revert-1) .), then develop the fix on top of that.
[07:11] <rvba> I could simply merge devel, recreate A' (similar to A) and B' (similar to B) but maybe there is a better way to do this?
[07:11] <wgrant> Or you could probably merge C' onto the branch with the reverted revert.
[07:11] <wgrant> That would probably work.
[07:11] <rvba> If I could avoid having to create new MP for thing that have been approved already it would be nice ...
[07:12] <rvba> things* even
[07:27] <rvba> wgrant: I think that since all the changes from A, B, C and D have been reverted all at once, I can't revert the revert properly...
[07:28] <wgrant> rvba: Ah. In that case, cherrypick the A and B merges from devel back on top of your branch.
[07:28] <rvba> wgrant: Okay, I'll do that, thanks for your help...
[07:30] <wgrant> That'll hopefully work. I haven't had cause to partially unrevert a multi-branch reversion before :/
[07:31] <wgrant> If it doesn't, I will poke further.
[07:31] <rvba> Seems all the changes where committed as one commit in devel.
[07:31] <wgrant> Ah, of course, so they were.
[07:32] <rvba> I tried to cherry pick from my branch (as opposed to devel) but something seems to be wrong with that ...
[07:32] <wgrant> That'll probably go wrong if you merged devel into your branch during its development.
[07:33] <rvba> I did merge devel during its development.
[07:33] <rvba> Let see if I can create a simple patch from the dev branch and apply it manually to the new branch then.
[07:38] <wgrant> Indeed, you could do a diff -rancestor:../devel to get the patch.
[07:38] <rvba> I've done a patch using cherry picking (i.e. specifying the revisions) but this seems to work.
[07:45] <lifeless> wgrant: what evil are you working on?
[07:47] <wgrant> lifeless: PillarObserver (currently just for optimising display and revocation of legacy restricted disclosure, but in the future it will probably be used to grant non-legacy observer privs, I guess)
[07:49] <lifeless> how does it optimise the legacy stuff?
[07:50] <wgrant> lifeless: My current theory is that access to private artifacts will depend on holding either full Observer privileges, or Restricted Observer and a subscription.
[07:50] <wgrant> We need to be able to efficiently show who has access to private artifacts on a project.
[07:50] <wgrant> And revoke that access.
[07:51] <lifeless> sure
[07:51] <lifeless> I humbly suggest that you want a fact table here
[07:51] <lifeless> rather than doing it in your online processing table
[07:51] <lifeless> we can talk on monday if that doesn't make sense
[07:51] <wgrant> My online processing table?
[07:52] <lifeless> the live table where you change things
[07:52] <lifeless> the one optimised for letting users search bugs etc
[07:52] <lifeless> -> the one that actually grants access
[07:52] <wgrant> I have a table PillarObserver(pillar, person, permission), which subscribing someone to a bug currently adds a RESTRICTED permission to, if it doesn't already exist.
[07:52] <wgrant> It's not flattened, so you do have to join against TP, but that seems to be fast enoguh.
[07:52] <lifeless> wgrant: you've shoved real world data sizes into it ?
[07:53] <lifeless> joining against TP should be fine
[07:53] <wgrant> Attempted to, but it's really hard to say what is a representative DB.
[07:53] <wgrant> Right, that's what I thought.
[07:53] <wgrant> It should be a small enough set of useful rows that the join should work fine.
[07:54] <wgrant> Unless someone is in 100000 teams.;
[07:54] <wgrant> In which case we might be in trouble.
[07:54] <lifeless> Now, IIRC there was a requirement to show the artificats flk have access to
[07:54] <wgrant> Yes.
[07:54] <lifeless> 2K is the largest in-team set at the moment.
[07:54] <wgrant> I plan to initially use bugsummary to display counts, possibly even on the listing.
[07:54] <wgrant> Then once we drill down list subscriptions.
[07:54] <lifeless> that might be shiny
[07:55] <wgrant> We should really check bugsummary's consistency at some point.
[07:55] <lifeless> I keep meaning to file a bug saying that its either confusing output, or wrong - oem let me know its off by 25% in some pages
[07:55] <lifeless> which is -way- more than my simulations predicted
[07:55] <wgrant> OEM is a bit pathological in a lot of cases, though.
[07:55] <lifeless> disclosure will help a lot
[07:55] <wgrant> Yeah.
[07:56] <lifeless> wgrant: I've seen odd numbers myself, I just can't remember where
[07:56] <wgrant> We can hopefully mostly summarise on privacy policy.
[07:56] <wgrant> WIth only exception subscriptions throwing everything off.
[07:56] <wgrant> And private projects should have very few of them.
[07:58] <wgrant> I've tried various different strategies here, and I think this approach is going to be the simplest, most efficient and most future-proof.
[07:59] <lifeless> will there be a blacklist facility ?
[07:59] <wgrant> Others probably won't outlast legacy disclosure without a complete redesign, and/or run into what wallyworld was trying to do earlier.
[07:59] <wgrant> ie. an explicit DENY?
[07:59] <lifeless> e.g. 'foo has restricted, but thats wrong, reject them -> leads to them not being addable again'
[07:59] <lifeless> at least not implicitly via subscriptions
[08:00] <wgrant> So, at present I'm emulating legacy disclosure. Subscribing someone to a bug adds a Restricted permission if they don't already have at least that. I imagine in future that will probably not implicitly happen, or projects will be able to reject subscriptions who don't have a permission explicitly added.
[08:01] <wgrant> We can't sensibly enable the revocation functionality until we've worked out how the new artificat visibility UI will work, but I don't want to design a legacy disclosure viewing model that won't be future-proof.
[08:01] <lifeless> sure
[08:02] <lifeless> will po be part of bug queries ?
[08:02] <mrevell> Hi!
[08:04] <wgrant> lifeless: Yes.
[08:04] <wgrant> lifeless: I initially tried to avoid that.
[08:04] <wgrant> lifeless: But then realised that it will have to be in future anyway.
[08:04] <wgrant> lifeless: For full observer roles.
[08:04] <wgrant> It should be quick enough to join through that and TP that it won't be too slow. I have tried on reasonably large datasets, but who know how it will perform in the wild.
[08:04] <wgrant> FF FTW.
[08:06] <adeuring> fgood morning
[08:17] <StevenK> Dear Freenode, stop sucking.
[08:19] <rvba> StevenK: Hi ... and congrats on the JS branch! ;)
[08:19] <lifeless> nigelb: doing your thing in 5
[08:23] <StevenK> rvba: Thanks, couldn't have done without you. :-)
[08:24] <rvba> StevenK: Np ;)
[08:26] <bigjools> morning all
[08:44] <nigelb> lifeless: \o/
[08:45] <lifeless> bigjools: http://pypi.python.org/pypi/oops_twisted
[08:45] <nigelb> I'm at work, so my response times are horrible :|
[08:45] <bigjools> lifeless: yay
[08:45] <lifeless> bigjools: I'm just adding its tarball to the download cache for you
[08:45] <nigelb> Morning bigjools!
[08:46] <bigjools> lifeless: no Launchpad download? *cough*
[08:46] <bigjools> lifeless: anyway that's awesome, thanks
[08:46] <bigjools> hello nigelb
[08:47] <lifeless> bigjools: it should be straight forward to use
[08:48] <bigjools> lifeless: yeah we talked about that already, sounds great
[09:00] <stub> bigjools: So does the db permissions approach I'm taking in https://code.launchpad.net/~stub/launchpad/distinct-db-users/+merge/76535 , or do you still think it will be painful and things will break?
[09:00]  * bigjools looks
[09:01] <stub> security.cfg bits, where I'm just declaring the distinct db users as aliases of a more powerful group
[09:02] <bigjools> stub: I think it's fine. ideally we'd have role groups, but I guess that would be painful to do now
[09:03] <bigjools> I already did the same trick for packagecopyjob
[09:03] <stub> bigjools: I expect if someone can think of a good name we just rename 'fiera', 'uploader', 'queued' etc. to start with and stick with them as the roles.
[09:03] <bigjools> well I was thinking more granular
[09:03] <bigjools> because queued, for example, closes bugs
[09:03] <stub> bigjools: Yes, but that takes time as you say :-)
[09:04] <bigjools> yes :)
[09:07] <nigelb> lifeless: were you able to do that thing for me? :)
[09:22] <lifeless> nigelb: download cache upload just finished
[09:24] <lifeless> bigjools: committed to the cache for you
[09:25] <bigjools> lifeless: ta
[09:25] <bigjools> lifeless: why did you call it oops_twisted on pypi?
[09:26] <lifeless> thats the package
[09:26] <lifeless> import oops_twisted
[09:27] <wgrant> stub: Is it deliberate that LP defaults to serializable isolation, while things that call initZopeless default to read committed?
[09:27] <bigjools> yes, but we have python-oops
[09:27] <lifeless> bigjools: http://pypi.python.org/pypi/oops
[09:27] <bigjools> ok
[09:28] <stub> wgrant: yes
[09:29] <wgrant> stub: Some scripts don't call initZopeless, so they end up serializable :(
[09:29] <stub> wgrant: need to fix that. hang - on phone
[09:35] <bigjools> lifeless: I am writing tests for my txlongpollfixture and my buildout test runner doesn't have the path to the executable but the interpreter does, any idea how I can fix it?
[09:35] <bigjools> so Popen("bin/txlongpoll") works inside bin/py, but not in a test
[09:35] <lifeless> whats cwd in the test ?
[09:37] <bigjools> it gets changed to the tempdir
[09:38] <bigjools> on the Popen call
[09:38] <bigjools> but removing it makes no difference
[09:38] <lifeless> so . isn't in your PATH, you'll need to use the abspath to the executable
[09:38] <bigjools> lifeless: how does that work when I am using an egg?
[09:39] <bigjools> I don't know where it is
[09:39] <bigjools> only buildout does
[09:39] <lifeless> erm, I'm not sure offhand
[09:40] <lifeless> sorry, ELOCAL
[09:40] <stub> wgrant: Can you point me at a script that doesn't call initZopeless?
[09:41] <stub> I thought this was all handled in the LaunchpadScript base class
[09:43] <stub> And everything was supposed to use that (or LaunchpadCronScript)
[09:47] <wgrant> stub: All LaunchpadScripts do it, yeah. But some stuff isn't a LaunchpadScript.
[09:47] <wgrant> eg. I ported a dozen scripts to LaunchpadScript on Sunday, because most of them called initZopeless directly.
[09:48] <wgrant> But I also ran into a couple that I can't remember right now, that call execute_zcml_for_scripts but not initZopeless.
[09:48] <wgrant> poppy-sftp seems to be the only thing left.
[09:49] <wgrant> There are some others, but they are short-lived and rarely run on prod.
[09:53] <wgrant> stub: I think it's somewhat unfortunate that ISOLATION_LEVEL_DEFAULT isn't actually the default any more, though.
[09:53] <wgrant> Perhaps now that just about everything uses LaunchpadScript it will be less bad.
[10:25] <stub> wgrant: The reason for using read committed over serializable is that if you use serializable you will randomly get serialization exceptions and are expected to deal with them.
[10:26] <bigjools> for my sanity, if I add a new dependency in launchpad-developer-dependencies it's not going to affect  buildbot is it?
[10:26] <stub> wgrant: So a foot gun, but scripts only hurt themselves.
[10:26] <wgrant> bigjools: It won't automatically, but it will when somebody next tries to update them.
[10:26] <wgrant> bigjools: Same with ec2.
[10:26] <wgrant> stub: Right.
[10:27] <wgrant> stub: I'm just strategising around saner script DB config APIs.
[10:29] <bigjools> wgrant: why is pdr showing script warnings this morning?
[10:29] <wgrant> bigjools: was killed for fastdowntime.
[10:30] <bigjools> how long is it taking to run now?
[10:30] <wgrant> 1.5 hours still.
[10:30] <bigjools> :/
[10:30] <bigjools> 8 mins to 1.5h is not goo
[10:30] <bigjools> d
[10:31] <wgrant> Indeed not, but we now have everything close enough to fixed that I think we know everything that is wrong.
[10:35] <wgrant> jelmer: What have you done to https://code.launchpad.net/~jelmer/wireshark/svn?
[10:35] <wgrant> jelmer: It seems rather impressively broken.
[10:39] <bigjools> wgrant: can you review: https://code.launchpad.net/~julian-edwards/meta-lp-deps/add-rabbit-management/+merge/76710
[10:40] <wgrant> bigjools: This needs some thought.
[10:40] <wgrant> bigjools: It will make LP uninstallable on oneiric.
[10:40] <wgrant> And launchpad-messagequeue-dependencies uninstallable in the DC.
[10:40] <bigjools> the latter easily fixed
[10:40] <bigjools> why the former?
[10:41] <bigjools> (this is a losa request BTW)
[10:41] <wgrant> Because oneiric has rabbitmq-server 2.5.0
[10:41] <wgrant> So rabbitmq-management 2.3.1 won't install.
[10:41] <bigjools> hmm
[10:42] <bigjools> this is why I asked you to review this :)
[10:43] <stub> So given we should all have switched to Oneiric per policy...
[10:47] <bigjools> wgrant: I am at a loss. I think the only way forward is to package the 2.5.0 plugin but ....
[10:49] <jtv> wgrant: are we also quite quite sure that nothing and nobody will mark an SPPH as Deleted after Gina is done importing the package but before its domination run?
[10:53] <wgrant> jtv: Nobody is meant to have privileges to do that. And even if they do, what's the worst that happens? It gets dominated 6 hours late.
[10:54] <jtv> wgrant: unless the number of versions increases to offset the decrease in Published publications.
[10:54] <wgrant> bigjools: I think so too. Which is why I was saying a week or so ago that the way forward is 2.5.0 or possibly even 2.6.0.
[10:55] <bigjools> wgrant: yes, and I agreed with you, but then I found out what a total bitch it is to package that stuff
[10:56] <bigjools> oneiric is getting 2.6.0
[10:56] <wgrant> It's actually really simple!
[10:56]  * wgrant calls StevenK in for backup.
[10:56] <bigjools> wgrant: well you said it wasn;t :)
[10:56] <wgrant> bigjools: For a packager it is simple :)
[10:56] <bigjools> plugins requiring the parent's Makefile
[10:56] <bigjools> odd version requirements
[10:56] <wgrant> Sure, there are a couple of messy bits, but those are already solved.
[10:57] <bigjools> FSVO solved :)
[10:57] <bigjools> but it'll do
[10:57] <bigjools> so can I invoke the Antipodean Packaging Brigade?
[10:58] <wgrant> Damn, you're not Soyuz any more, so I can't say you should finally learn packaging :(
[10:58] <bigjools> you can still say that
[10:58] <bigjools> it's not like I don;t want to
[10:58] <bigjools> but $TIME .... :/
[10:58] <bigjools> and I don't want to block this
[10:59] <wgrant> Bah.
[10:59] <wgrant> I'll see what I can do on Monday.
[10:59] <wgrant> If the upgrades are trivial enough, it should only take an hour or two.
[10:59] <wgrant> If they're not, I'll have to see.
[11:01] <bigjools> we could ask someone in distro, if they are already packaging 2.6.0
[11:01] <wgrant> That may also be a good option.
[11:01] <bigjools> they had a FFe for it
[11:02] <wgrant> Late!
[11:02]  * nigelb waves
[11:02] <wgrant> Evening nigelb.
[11:03] <nigelb> Lazy Friday evening :)
[11:03] <bigjools> packaged by Marc Cluet it seems
[11:05] <wgrant> Sigh, their rabbitmq-erlang-client packaging is still stupid.
[11:05] <wgrant> It installs both the extracted files and the archive containing all of them.
[11:06] <StevenK> wgrant: Grab me on Monday and I'll do part of it too
[11:06] <bigjools> thanks guys
[11:07] <nigelb> bigjools: Starting work on longpoll? :)
[11:07] <wgrant> Upstream provides reasonable 2.6.0 packages, so it's just a matter of updating the plugin infrastructure and getting the versions to match.
[11:07] <bigjools> nigelb: a while ago
[11:07] <bigjools> wgrant: oh, they've done packages?
[11:07] <nigelb> bigjools: Can't wait to see it in production :-)
[11:08] <bigjools> nigelb: we're also moving to a slightly different deployment model, so it's been a bit trickier than usual
[11:08] <wgrant> bigjools: They provide a rabbitmq-server package which appears to be based on Debian's, or vice-versa.
[11:08] <bigjools> but nothing for the management plugin?
[11:08] <nigelb> Oh, how different is the deployment going to be now?
[11:08] <wgrant> No.
[11:08] <bigjools> balls
[11:08] <bigjools> nigelb: SOA-based
[11:08] <wgrant> Their deployment story for plugins is to download the .ez from the website and drop it in the system dirs.
[11:09] <nigelb> Ah. Right.
[11:09] <bigjools> lots of small microservices
[11:09] <jelmer> wgrant: it's a bzr code import
[11:09] <bigjools> which will be *awesome* when we can start splitting up LP itself
[11:09] <nigelb> I'm writing one of those services :P
[11:10] <wgrant> jelmer: Ah.
[11:30] <jtv> wgrant: wanna review?  https://code.launchpad.net/~jtv/launchpad/bug-857155/+merge/76715
[11:35] <wgrant> jtv: Looks good. Should we perhaps take this opportunity do some logging during domination?
[11:35] <wgrant> jtv: We presently log two lines per published package; another one per package being dominated would be useful and only 50% larger :)
[11:35] <jtv> wgrant: if you like.  I'm also preparing a post-branch with additional improvements (bulk-loading and a renaming), but logging could go into the original.
[11:36] <wgrant> It should be a single line, so why not :)
[11:38] <jtv> Well it's not _that_ easy.  The place where I'd most like to log this doesn't know the package name.
[11:40] <wgrant> :(
[11:40] <wgrant> Depends where you want it, I guess.
[11:52] <jtv> wgrant: I've added some debug and debug2 logging: debug will give you the name and stats for a package being dominated; debug2 also says which packages it skips *and* what it does to each publication.  Pushing.
[11:54] <jtv> In the follow-up branch I'll change the name of that domination method (because they no longer have to be “removed” versions) and add bulk-loading of the sprs.
[11:54] <wgrant> jtv: That sounds like a good option.
[11:54] <jtv> If you _really_ want, you'll be follow each step of the algorithm.
[11:55] <bigjools> dogfood is going down for 2-3 days for DB refresh
[11:55] <wgrant> Hm, that is awkward.
[11:55] <jtv> Hmm maybe I should request a Q/A run of my branch on staging now.
[11:55] <wgrant> Right when we need to QA gina!
[11:55] <wgrant> But yeah.
[11:56] <bigjools> staging FTW
[11:56] <jtv> We knew dogfood would go down, and postponing it would just give us more grief next week.
[11:56] <jtv> Yup.
[11:56] <wgrant> We'll need to cowboy the config and get a partial archive onto there.
[11:56] <wgrant> But we should try it.
[11:56]  * jtv toys with idea of running old & new versions on staging & qastaging respectively, just so he can compare results
[11:56] <bigjools> we need to shift onto staging more
[11:57] <jtv> Och aye, we can't run gina's import there.
[11:57] <wgrant> That debug2 logging looks marvellous.
[11:57] <wgrant> Approvalised.
[11:57] <jtv> Thanks.
[11:57] <jtv> stub defined some extra debug levels, including “BLATHER”
[11:58] <bigjools> launchpad_dogfood=# drop database launchpad_dogfood;
[11:58] <bigjools> muahahha
[11:58] <wgrant> Will it let you do that with active connections?
[11:58] <bigjools> what active connections.... :D
[11:58] <wgrant> 21:58:26 < bigjools> launchpad_dogfood=# drop database launchpad_dogfood;
[11:59] <nigelb> bigjools: You need to work on that evil laugh :D
[11:59] <lifeless> bah, bad oops_wsgi 0.0.4 tarball.
[11:59] <bigjools> it's only semi-evil
[11:59] <lifeless> silly setuptools manifest handling
[12:02] <bigjools> you'd think that typing a ticket number into the twisted trac search, with only "tickets" selected, would find the exact ticket first.  But oh no.
[12:05] <lifeless> jamesh: oops-wsgi had a bad tarball - stale MANIFEST, I've done a 0.0.5 to fix.
[12:11] <lifeless> nigelb: ok
[12:11] <nigelb> lifeless: "free-er"? :)
[12:11] <lifeless> lp:js-oopsd is populated
[12:12] <nigelb> lifeless: Thanks! Looking now
[12:12] <lifeless> no tests, and nothing useful captured, but it should be within reach rather than being from a standing-stat for you
[12:12] <nigelb> wow
[12:14] <lifeless> there is some science fiction in the README etc, so read carefully - you'll need to fill those bits out :)
[12:14] <nigelb> lifeless: Thanks, that's awesome. You just did all the bits I had no clue (buildout, etc), and I there's boilerplate wsgiref code \o/
[12:14] <lifeless> to experiment
[12:14] <lifeless> run it with
[12:14] <lifeless> bin/py -m js_oopsd.main
[12:14] <lifeless> and use your browser to visit the url it prints
[12:14] <nigelb> okay@
[12:15] <lifeless> then look for a directory like 2011-09-24
[12:15] <nigelb> s/@/!
[12:15] <lifeless> in that will be OOPSes
[12:15] <lifeless> one for each request your browser made
[12:16] <nigelb> cool!
[12:17] <lifeless> so you can see its -very- shallow :)
[12:18] <bigjools> lifeless: so, oops logging added to longpoll already.  Nice and easy!
[12:18] <lifeless> bigjools: worth the wait ?
[12:18] <bigjools> lifeless: of course
[12:18] <lifeless> :P
[12:18] <bigjools> :)
[12:18] <lifeless> ok, time for bed
[12:18] <lifeless> gnight all
[12:18] <nigelb> g'nite lifeless
[12:18] <bigjools> lifeless: so one more question!
[12:18] <lifeless> heh, nick of time!
[12:19] <bigjools> lifeless: if I add more log observers will it affect any of this?
[12:19] <bigjools> or vice-versa
[12:19] <bigjools> since Twisted bug 638 got fixed I can add a proper rotatable log file now
[12:19] <lifeless> twisted dispatches all log items to all log observers
[12:19] <_mup_> Bug #638: Incorrect link in the bug's task listing <lp-bugs> <Launchpad itself:Fix Released by bradb> < https://launchpad.net/bugs/638 >
[12:19] <bigjools> poifect
[12:19] <bigjools> ta
[12:19] <lifeless> bigjools: however
[12:19] <bigjools> you may sleep now
[12:20] <lifeless> bigjools: I suggest using the OOPSObserver fallback parameter
[12:20] <lifeless> to point at the rotatable log file observer, rather than having it as a direct observer.
[12:20] <bigjools> right
[12:20] <lifeless> this will mean you get the oops ids logged to the log file, rather than having the tracebacks in that file and no oops id
[12:20] <lifeless> ta-ra
[12:21] <bigjools> right
[12:31] <bac> hi adeuring, looks mighty quiet here
[12:31] <adeuring> morning bac, yes, is very quiet today
[12:51] <StevenK> Are you two hunting MPs?
[13:00] <stub> jtv: I can't take credit for BLATHER - that is brought over from Zope2 to Zope3. I just downgrade its integer value :)
[13:10] <jtv> Reviewer wanted, while I go off and have weekend!  https://code.launchpad.net/~jtv/launchpad/post-857155/+merge/76733
[13:10] <nigelb> heh
[13:11] <nigelb> not very tempting offer :P
[13:11] <jtv> Why not?
[13:17] <adeuring> morning deryck:, could you please run this query on staging: https://pastebin.canonical.com/53295/ ?
[13:18] <deryck> adeuring, sure.
[13:18] <adeuring> thanks!
[13:21] <deryck> adeuring, https://pastebin.canonical.com/53296/ (run twice to see with/without cache)
[13:21] <adeuring> deryck: cool, thanks!
[13:21] <deryck> np!
[13:22] <mrevell> Hey deryck, do you have time for a ten minute call this afternoon?
[13:22] <deryck> mrevell, sure.  you want to calendar a time that works for you?
[13:23] <adeuring> deryck: just to be a bore sur, could also try this one: https://pastebin.canonical.com/53297/ (should be sloooow)
[13:25] <deryck> adeuring, sure
[13:27] <mrevell> deryck, thanks, will do
[13:28] <deryck> adeuring, https://pastebin.canonical.com/53298/
[13:28] <adeuring> deryck: thanks!
[13:28] <deryck> np!
[13:33] <deryck> adeuring, abentley -- coming. just running a couple minutes behind, sorry.
[13:38] <deryck> adeuring, abentley -- https://answers.launchpad.net/launchpad/+question/172089
[13:52] <deryck> adeuring, hey.  see recent mails from me about that question, where I cc'ed you.
[13:52] <bigjools> MP diffs are slow today
[13:53] <adeuring> deryck: ack
[13:54] <deryck> adeuring, if it ends up being a timeout bug, or something that needs coding to fix ;)  we should pull it back to the next lane….
[13:54] <deryck> adeuring, and let whoever gets free first deal with it.  since there's the other escalated bug already in next.
[13:54] <adeuring> ok
[13:57] <flacoste> gary_poster: you are not really fixing bug 724609, right?
[13:57] <_mup_> Bug #724609: Rewrite lp.client tests using yuitest <build-infrastructure> <qa-ok> <Launchpad itself:In Progress by gary> < https://launchpad.net/bugs/724609 >
[13:57] <gary_poster> flacoste, I was intending to soonish, yes, as an example of using the new YUI XHR tests.
[13:58] <bigjools> gary_poster: hello, I have another buildout question for you if you have 5 minutes?
[13:58] <mrevell> deryck, Skype or Mumble?
[13:58] <gary_poster> In fact, I was about to include that in my announcement flacoste, so if that's bad somehow lemme know. :-)
[13:58] <flacoste> gary_poster: cool, but it's not fixed yet (as qa-ok makes me believe)
[13:58] <flacoste> gary_poster: or did you just rewrite the tests already
[13:59] <deryck> mrevell, let's mumble.  give me 2 minutes.
[13:59] <mrevell> k
[14:00] <gary_poster> flacoste, I have to say qa-ok for incremental steps.  I landed with --incr because the branch introduced the infrastructure needed to write the test, but then when you do that you still have to assert that you have qa'd your incremental changes (I thought I used --no-qa actually, but that's a separate issue)
[14:00] <flacoste> gary_poster: ok, makes sense
[14:00] <flacoste> gary_poster: and i'm very happy that you are fixing this bug!
[14:00] <gary_poster> flacoste, cool :-)
[14:00] <gary_poster> bigjools, sure
[14:01] <bigjools> gary_poster: thanks you might be my saviour again.
[14:01] <gary_poster> :-)
[14:01] <bigjools> gary_poster: I have a txlongpoll egg, which build the txlongpoll server. I have another egg, txlongpollfixture which depends on the former
[14:02] <bigjools> gary_poster: I am struggling to work out how the fixture can find the executable that the first egg builds, so it can run it up
[14:02] <deryck> mrevell, orange 1 o 1 when ready
[14:05] <gary_poster> bigjools, ah fun.  Mm, if I understand you, you may have to use setuptools APIs to do that the "right" way.  The "wrong" way might be easy: it might be in the egg, in EGG-INFO/bin.  Another alternative is to have the fixture call the code that the executable "fronts," if that works for you.
[14:07] <bigjools> gary_poster: we discussed the wrong way earlier, it could get messy depending on what else builds it I think. I'm not sure I understand the 3rd alternative there though, can you expand?
[14:09] <bigjools> (bearing in mind we need to Popen the executable)
[14:10] <gary_poster> bigjools, typically you create executables by simply pointing setup.py at a function and saying "make a bin for this function and call it ${whatever}."  What you can do then in this case is set up your module to be run as __main__ (IIRC) and run "${executable} -m txlongpoll.module_with_your_executable ...options..."
[14:11] <gary_poster> if they are equivalent that you are good to go
[14:12] <bigjools> gary_poster: have you got an example of that?
[14:12] <gary_poster> bigjools, :-) I am looking for the pertinent docs, 1 sec
[14:13] <jcsackett> is anyone else regularly getting rabbitmq timeouts when trying to run launchpad.dev?
[14:15] <bigjools> gary_poster: is this the entry_points thing I can see in LP's setup.py?
[14:20] <gary_poster> bigjools, that's how you typically make an executable, yes.  So, in txlongpoll's setup.py, you would define a binary there.  If it were installed via something like setuptools in a virtualenv or real Python, the bin would be created.
[14:20] <gary_poster> Then for code that uses it as a library, you would define the same entry_points thing in our setup.py or in our buildout.cfg.  That entry point would also be run in the associated module's def __main__ (http://docs.python.org/using/cmdline.html#cmdoption-unittest-discover-m) so that code like your fixture would not have to find (or even have!) the binary, just run {executable} -m {module} {any other arguments that you
[14:20] <bigjools> gary_poster: it's not a library
[14:21] <bigjools> it's a standalone daemon
[14:21] <bigjools> I already have it building from from a buildout recipe
[14:21] <gary_poster> bigjools, ok, but the code can still be imported, right?
[14:21] <bigjools> not easily
[14:21] <bigjools> it's twisted
[14:22] <gary_poster> bigjools, ohhhh, so they don't play the normal Python games?
[14:22] <gary_poster> :-(
[14:22] <bigjools> not in the slightest
[14:22] <gary_poster> ah :-(
[14:22] <gary_poster> um
[14:23] <bigjools> you have to put a plugin in a special place
[14:23] <bigjools> then run twisted with the right args
[14:23] <bigjools> we'll have this same problem when we eventually make the librarian a SOA part
[14:23] <gary_poster> bigjools, well, then setuptools may not help you either.  what's an example of the command you want to run?
[14:25] <bigjools> gary_poster: https://bazaar.launchpad.net/~julian-edwards/txlongpollfixture/FIRST/view/head:/txlongpollfixture/server.py
[14:25] <bigjools> gary_poster: see the _start() method
[14:25] <bigjools> line 73 onwards
[14:26] <bigjools> if we could get the bin directory in the PATH then we're golden
[14:26] <bigjools> or even the egg directory
[14:28] <gary_poster> bigjools (I have a call in 20, fwiw.  can return to this afterwards.)
[14:28] <gary_poster> actually about 15 now
[14:29] <bigjools> gary_poster: np, thanks
[14:29] <gary_poster> I'm going to go look at how you are creating bin/txlongpoll...
[14:29] <bigjools> script recipe
[14:33] <gary_poster> OK so txlongpoll=twisted.scripts.twistd:run , which is identical to the twistd in setup.py except it has its own custom path that includes the txlongpoll egg.  So...it really is an arbitrary binary from the perspective of other code.  It could be running a different Python version even.
[14:37] <gary_poster> bigjools, I don't see an opportunity for magic snarfing here.  If you don't want to tie txlongpollfixture to the LP build (and I would understand not wanting to), you need to make this something that is configured--for instance, the fixture requires you to inform it where the binary is.
[14:37] <gary_poster> Once you do that, you can have Launchpad either hardcode the binary name/location in the test fixture initialization, or do buildout tricks.  I'd be inclined to do the former, relying on config's ability to easily get you LP's root, and then hardcoding 'bin' and 'txlongpoll'.
[14:37] <bigjools> gary_poster: yeah, I was considering just ensuring that the person using the fixture has the executable in $PATH
[14:38] <bigjools> yeah, hardcoding bin/txlongpoll was the original plan, I figured that buildout might set the path. I was wrong :)
[14:39] <StevenK> What's wrong with figuring out your current path with os.path.abspath(os.path.dirname(__file__)) ?
[14:39] <bigjools> because it's not necessarily always in the same relative place to the executable
[14:39] <bigjools> it depends how the egg was built
[14:39] <bigjools> the dependent egg I should say
[14:40] <bigjools> but I might be wrong, I had discarded this way earlier, maybe it was too hasty
[14:40] <gary_poster> bigjools, eh, there's a million ways for that kind of config. :-) but yeah, I'm afraid it does need to be config, AFAIK.  buildout can tell you what has been generated, particularly while it is generating it, but since this is really buildout generating a separate binary with a separate context (that could even be in another language as far as the main LP code is concerned)
[14:41] <bigjools> indeed
[14:42] <bigjools> StevenK: actually thinking about it that definitely won't work because the bin/ directory can move around
[14:42] <bigjools> gary_poster: ok I will set the PATH in my fixture's tests so at least they work, then whoever uses the fixture needs to set the path as well
[14:42] <bigjools> thanks for looking
[14:43] <gary_poster> bigjools, does the txlongpollfixture depend on txlongpoll, and does/will LP depend on txlongpollfixture?
[14:43] <bigjools> gary_poster: yes, exactly that (will)
[14:44] <gary_poster> bigjools, in that case, all the nice separation you've done in buildout is a bit for naught
[14:44] <bigjools> gary_poster: blame lifeless!
[14:44] <gary_poster> because LP will still have txlongpoll as a dependency
[14:44] <bigjools> well it was to prevent txlongpoll having a dep on the test fixtures
[14:44] <gary_poster> Does the fixture really need txlongpoll as a dependency?
[14:45] <bigjools> and this is only for the dev environment
[14:45] <bigjools> in prod we'll run it differently
[14:45] <gary_poster> ...
[14:45] <bigjools> prod doesn't need the fixture
[14:45] <gary_poster> yeah, I understand that part.
[14:46] <gary_poster> So...
[14:46] <adeuring> deryck: to be a bit more sure, could you try this query: http://paste.ubuntu.com/695674/ ?
[14:46] <gary_poster> This is maybe a discussion for another forum, but just to see if I understand
[14:47] <deryck> adeuring, sure.
[14:47] <gary_poster> bigjools, txlongpollfixture is needed to test txlongpoll within itself *and* needed for LP to test txlongpoll?
[14:47] <gary_poster> I mean, test the use of txlongpoll?
[14:47] <gary_poster> What I'm getting at is this:
[14:47] <bigjools> gary_poster: txlongpollfixture is not needed to test txlongpoll; it's just a way of starting it for other tests and for "make run"
[14:48] <gary_poster> bigjools, got it.  So, actually, it never imports txlongpoll, right?
[14:48] <bigjools> nope
[14:48] <bigjools> just starts the twistd daemon up
[14:48] <gary_poster> bigjools, but it has a Python egg dependency on txlongpoll even though it never imports it?
[14:49] <bigjools> yeah that does sound wrong :)
[14:49] <bigjools> I did it for convenience so that the egg gets built
[14:49] <gary_poster> bigjools, yeah, so I suggest breaking that dependency
[14:49] <bigjools> ok
[14:49] <bigjools> although
[14:50] <gary_poster> because that's what we will need for other similar microfixtures, particularly those not written in Python
[14:50] <gary_poster> microservices I mean
[14:50] <bigjools> that will make it harder to make the txlongpollfixture tests to work
[14:50] <gary_poster> bigjools, maybe make it a test dependency?  That should be easy with or without buildout
[14:50] <bigjools> we need a general solution for PATH with microservices
[14:51] <bigjools> ah yes, I can do a test dep
[14:51] <gary_poster> bigjools, you don't really mean PATH you mean bin, right?
[14:51] <gary_poster> I mean, PATH is one mechanism for communicating bin
[14:51] <gary_poster> the location of the bin
[14:51] <bigjools> I mean a way for these services to get started without knowing where they live
[14:51] <gary_poster> yeah
[14:51] <bigjools> otherwise the local .dev will be a nightmare
[14:52] <bigjools> we just want to be able to type make run and have it start stuff
[14:52] <gary_poster> that has to be configuration IMO, but we can certainly make that configuration easy some how
[14:52] <bigjools> we could put bin in the path
[14:53] <gary_poster> bigjools, yeah, sure, my point is simply that you are framing the problem in terms of your current solution, AFAICT.  no big deal.  I need to have my call :-)
[14:54] <bigjools> gary_poster: I guess I am, yeah
[14:54] <bigjools> enjoy, thanks for the help
[14:54] <gary_poster> welcome bigjools
[15:02] <deryck> adeuring, taking a bit, sorry.  but I'm getting the results now.
[15:02] <adeuring> deryck: np
[15:07] <sinzui> jcsackett, I might loose power in a few minutes. The rain is torrential.
[15:08] <jcsackett> sinzui: i am sorry to hear that, do you want to chat now in case you can't later, and we'll just accept the risk of sudden disconnect?
[15:08] <sinzui> sure
[16:08] <rsalveti> bigjools: hey, any news when we'll be able to create the ubuntu-leb derived distro?
[16:08] <rsalveti> at production
[16:09] <bigjools> rsalveti: you can do it now, I was holding off telling you because there's no Apache set up yet to make the archive available
[16:10] <rsalveti> bigjools: well, guess we can create it now and set up apache later
[16:10] <bigjools> there's an RT for it
[16:10] <rsalveti> not critical yet but I'd like to get it going
[16:11] <adeuring> bac: fancy a a review of an MP with a short diff? https://code.launchpad.net/~adeuring/launchpad/bug-739052-11/+merge/76772
[16:12] <bac> adeuring: surely
[16:30] <bac> adeuring: nice, r=bac
[16:51] <mrevell> Night all
[16:52] <nigelb> gary_poster: YUI XHR tests sound neat :-)
[16:52] <bigjools> hey bac, can I poke a review your way please?
[16:52] <bac> bigjools: shirley
[16:53] <bigjools> bac: https://code.launchpad.net/~julian-edwards/txlongpollfixture/FIRST/+merge/76777
[16:53] <bac> bigjools: i was about to grab a late lunch, though.  you in a hurry?
[16:53] <bigjools> it's a new standalong project
[16:53] <bigjools> bac: no it can wait, I will be eating dinner shortly
[16:53] <bigjools> bac: I will be back on later
[16:53] <bac> ok
[16:53] <bigjools> cheers
[16:55] <gary_poster> thanks nigelb :-)
[18:20] <sinzui> bac: do you have time to review https://code.launchpad.net/~sinzui/launchpad/valid-targets-1/+merge/76781
[18:20] <bac> sinzui: i do and will when i'm done with julian's
[18:40]  * deryck goes offline for lunch
[18:50] <bac> sinzui: i accidentally approved your MP without comment.  i'm updating it now.
[18:51] <sinzui> cool, I saw the approve without a comment and thought that was odd
[18:54] <bac> sinzui: hah, and i mispelled 'typo'
[18:55] <sinzui> I can tell you were thinking of me
[18:55] <bac> :)
[18:55] <sinzui> I will fix the comment once I am certain I know what I meant.
[19:02] <nigelb> sinzui: Hey, do you have a few minutes? I'd like to hear your take on bug 5283
[19:02] <_mup_> Bug #5283: "Home page" vs. "Description" is misleading <easy> <lp-registry> <tech-debt> <ui> <Launchpad itself:Triaged by nigelbabu> < https://launchpad.net/bugs/5283 >
[19:03] <sinzui> I do
[19:03] <sinzui> I think I did a report in the past to see what would happen is we merged fields
[19:08] <nigelb> sinzui: what was the eventual decision back then?
[19:08] <sinzui> nigelb, We really want to remove .homepage
[19:09] <nigelb> the whole field?
[19:10] <sinzui> nigelb, for 2 years we have presented .homepage_content as the leading text to teamdescription
[19:10]  * nigelb opens up psql
[19:11] <sinzui> We want to update the db to concatenate the two fields
[19:11] <sinzui> We want to rename teamdescription to description
[19:11] <nigelb> Db patch \m/
[19:12] <sinzui> so there is one field name description in Person for both teams and users
[19:12] <nigelb> that makes sense
[19:12] <nigelb> so, do you want me to talk to stub and lifeless to get started on a db patch for this.
[19:12] <sinzui> The fixing the edit forms and templates is easy, but I think timing is difficult
[19:13] <nigelb> I would first do the rename
[19:13] <nigelb> and then land it and deploy
[19:13] <sinzui> nigelb, I think there are three db tasks and one code task  to ensure no interuption to users
[19:13] <nigelb> yeah
[19:13] <sinzui> 1. create a description field in the schema
[19:14] <sinzui> 2. garbo job to to fill it from homepage_content + \n + team description
[19:14] <nigelb> what's a garbo job?
[19:15] <sinzui> 3. update the index pages and forms to use the new field (do not show the old fields if the description is not empty).
[19:15] <sinzui> 4. remove the old columns when the garbo job is complete
[19:15] <sinzui> 5. remove the old field from the models and templates
[19:16] <sinzui> nigelb, a garbo job is a db process used to clean up data while th production system is running
[19:16] <nigelb> ah, I've read that somewhere
[19:16] <sinzui> nigelb, while we call it garbo, we often co-opt it to populate new data
[19:17] <sinzui> nigelb, there are more than 1,000,000 users and team. I expect the job needs a few days to complete
[19:17] <nigelb> okay
[19:18] <nigelb> I'll paste the list you just mentioned into the bug and start working :)
[19:18] <sinzui> Please do.
[19:19] <nigelb> Could give me a patch ID please?
[19:19] <nigelb> s/ID/number
[19:20] <sinzui> nigelb, only stub can give an id. When I write patches I end mine in a very high number
[19:21] <sinzui> nigelb, use patch-2208-97-0.sql for development
[19:21] <nigelb> The process says any launchpad team member can
[19:22] <nigelb> I will, however, poke stub on Monday
[19:22] <sinzui> oh, that's right, let me check the wiki page to claim a number
[19:24] <sinzui> and I should not have recommend -0. this will be an incremental for the fast rollout
[19:24] <nigelb> Yeah :)
[19:25] <lifeless> norming
[19:25] <sinzui> nigelb, i am getting the branch now. I will also get you real numbers for affected teams and person from staging afterwards
[19:26] <nigelb> cool!
[19:26] <nigelb> lifeless: Good morning! I see you still need caffeine :-)
[19:28] <lifeless> I very rarely drink caffeine ;) - that was a deliberate switch
[19:28] <sinzui> nigelb, 2208-90-1 is your number to add the column
[19:28] <nigelb> lifeless: ha
[19:39] <muffinresearch> Hi, I seem to have trouble entering text into some of the edit boxes on merge proposals in chrome in the last couple of days. This is chrome 14.0.835.186. When it doesn't work I can't type anything only pasting text works.
[19:44] <matsubara> sinzui, https://bugs.launchpad.net/launchpad/+bug/857697 oops in the affiliation code for the person picker
[19:44] <_mup_> Bug #857697: AttributeError: 'NoneType' object has no attribute 'owner'  get affiliation in person picker <disclosure> <oops> <person-picker> <Launchpad itself:Triaged> < https://launchpad.net/bugs/857697 >
[19:45] <matsubara> I just triggered the oops in the bug report so that one is not available yet on oops-tools. the other one I added as an example can be seen there
[19:45] <sinzui> thanks matsubara
[19:45] <matsubara> np
[20:56] <cr3> benji: hi there, might you have a moment for some lazr.restful assistance?
[20:56] <benji> cr3: sure, what's up?
[20:57] <cr3> benji: so, I'm getting this error when getting a collection of system objects: There isn't enough context to get URL information. This is probably due to a bug in setting up location information.
[20:58] <cr3> benji: the problem is with the processor attribute of each system object which probably doesn't have a url information by defining the __parent__ and __url_path__ attributes
[20:59] <cr3> benji: however, I don't know how to define the __parent__ attribute because a processor object doesn't know its system container. in fact, it's the other way around, system has a foreign key to the processor
[20:59] <cr3> benji: ideally, I'd like the url for a processor to be /system/<id>/processor, not sure how to pull it off though
[21:00] <benji> cr3: is this a Django app?
[21:02] <cr3> benji: yep
[21:04] <benji> cr3: if __parent__ isn't known statically, then you'll have to make it a property that looks it up
[21:04] <cr3> benji: so, I wonder if System should inherit from TraverseWithGet, do something to the request object, so that Processor.__parent__ can then know the context
[21:04]  * benji looks up TraverseWithGet
[21:05] <benji> cr3: I don't see how that will help.
[21:06] <cr3> benji: I just tried it, it doesn't help indeed :)
[21:06] <benji> :)
[21:06] <cr3> benji: not sure how a processor can know it's system container at runtime though
[21:07] <benji> cr3: now, you could use something like TraverseWithGet that wraps the traversed object in a LocationProxy
[21:07] <benji> LocationProxy is for objects that don't know their __parent__ or __name__, but someone else does and can wrap the object in a proxy that passes through all requests except for those two attributes
[21:08] <benji> of course, you may need a DjangoLocationProxy because Django doesn't like __name__ as an attribute name
[21:09] <cr3> benji: I think I've rolled my own to look something like: class SystemProcessor: delegates(IProcessor, context="processor"); def __init__(self, system, processor)...
[21:11] <cr3> benji: however, whether I create something like a DjangoLocationProxy (good call about __name__, by the way!) or roll my own, I'm not sure how to apply that to the System class. For now, I have class System: processor = Reference(processor_id, Processor.id), ie I'm conveniently using Storm to reference the processor
[21:12] <cr3> benji: instead, I believe I'd have to change that to class System: def _getProcessor(self): processor = Store.of(self).get(Processor, self.processor_id); return SystemProcessor(self, processor); processor = property(_getProcessor)
[21:12] <benji> cr3: that's where you'd use something like TraverseWithGet, you have to hook some part of the traversal so that when you have both the system and processor "in hand" you can create your wrapper and tell it about it's __parent__ and __name__ (er, __url_path__)
[21:12] <cr3> ... which seems like a lot of noise to do something apparently simple :(
[21:14] <benji> cr3: that would work too, and it is a bit noisy
[21:14] <cr3> benji: I'm not sure where I'd have both the system and processor "in hand" though, I'll try hooking here and there to figure it out and then fallback to making noise :)
[21:15] <nigelb> I have a confusion with a db patch.
[21:15] <benji> cr3: good luck
[21:15] <cr3> benji: thanks man!
[21:15] <nigelb> I'm comparing the diff of current.sql and newsampledata.sql. I see a new column transitively private :|
[21:16] <nigelb> Is that supposed to happen?
[21:19] <bac> nigelb: you'll see new items unrelated to your change if previous branches added things to the db but did not generate new sampledata
[21:20] <bac> nigelb: you can verify that it isn't your change but generating sampledata in a fresh branch from devel
[21:20] <bac> you'll probably see the same additions
[21:20] <nigelb> ah
[21:20] <nigelb> so, I should probably mention this in my MP for stub I guess.
[21:21]  * nigelb looks for culprits
[21:21] <nigelb> bac: Thanks!
[21:21] <bac> nigelb: np.
[21:52] <wallyworld_> jcsackett: you still there?
[21:52] <jcsackett> for some definition of "there". :-P what's up?
[21:53] <wallyworld_> jcsackett: just to let you know, i had a chat to huw about the managing disclosure mockups
[21:53] <wallyworld_> were are looking to use some static html with js hacked in and hosted on people.canonical.com
[21:53] <jcsackett> wallyworld_: sounds good to me.
[21:54] <wallyworld_> cool. i may on monday muck around with it a bit
[21:54] <wallyworld_> we can chat about it a bit more next week, just wanted to keep you in the loop
[21:54] <jcsackett> wallyworld_: is huw providing the static and we hack in the js, or is that part not yet figured?
[21:55] <wallyworld_> jcsackett: he is still ramping up on what we need to do - i ran him through the functionality etc. i was going to do the html myself initially
[21:55] <jcsackett> wallyworld_: dig.
[21:55] <jcsackett> and now, i must attend to starting dinner. thanks for keeping me up to date. :-)
[21:56] <wallyworld_> np. have a good weekend