/srv/irclogs.ubuntu.com/2011/09/23/#launchpad-dev.txt

wgrantlifeless: re. bug #137448: I fixed the model to permit full retargeting, and added support for that to the legacy, non-AJAX UI.00:05
_mup_Bug #137448: New UI is confusing and counter inuitive for changing affected package <lp-bugs> <Launchpad itself:Triaged> < https://launchpad.net/bugs/137448 >00:05
lifelesswgrant: bug 163244 needs you to pick *A* flaw in it ;)00:06
_mup_Bug #163244: CVE list needs some triaging features <lp-bugs> <Launchpad itself:Triaged> < https://launchpad.net/bugs/163244 >00:06
lifelesswgrant: so 137448 is fix released ?00:06
lifelesswgrant: or you mean 'not fixed because the ajax ui isn't fixed'00:06
wgrantlifeless: I thought that it was referring to the AJAX UI, but I think that bug might predate AJAX. I suspect the specific change the bug complains about is in fact the introduction of a link to the target in the task row, rather than just in the expander where it was initially.00:08
wgrantlifeless: However, until packages can be reassigned by AJAX and the expander is removed, I think the bug is probably still valid.00:08
lifelesswgrant: yup, I get that00:09
lifelessthanks00:09
huwshimiwallyworld_: Ready whenever you are00:16
wallyworld_huwshimi: ok. mumble?00:16
huwshimiwallyworld_: sure00:16
lifelesswgrant: get spec by mail ..00:22
lifeless(the email fail)00:23
wgrantYeah, but that's not the main issue here.00:23
wgrantI've only seen that error once.00:23
lifelessoh, I see dkim ?00:23
lifelessis it casting str to unicode?00:23
wgrantI can see a particular DKIM log message missing from process-mail.log each time we get that cronspam.00:24
lifelessdone for today, down to 679 medium bugs00:24
wgrant2011-09-22 18:51:13 INFO    Attempting DKIM authentication of message id=<db6701cc7969$046cade0$718d875f@BOOTHRpy> from=<BOOTHRpy@ffni.com> sender=None00:24
wgrantThat one.00:24
wgrantI suspect the crash is in from/sender.00:24
wgrantCan we upgrade to Python 3 yet?00:25
lifelessanytime00:25
lifelessif it works00:25
wgrantHmm, do I want to %r or try to unicodeify them...00:26
wgrant    log.info(00:26
wgrant        'Attempting DKIM authentication of message id=%s from=%s sender=%s'00:26
wgrant        % (signed_message['Message-ID'],00:26
wgrant            signed_message['From'],00:26
wgrant            signed_message['Sender']))00:26
wgrantThat's what we're doing now.00:26
* wallyworld_ goes out for lunch00:53
lifelesswgrant: looks like one of those is unicode then00:54
wgrantTraceback (most recent call last):00:56
wgrant  File "/usr/lib/python2.6/logging/__init__.py", line 791, in emit00:56
wgrant    stream.write(fs % msg.encode("UTF-8"))00:56
wgrantUnicodeDecodeError: 'ascii' codec can't decode byte 0x98 in position 120: ordinal not in range(128)00:56
wgrantlifeless: It's a decode error on encode.00:56
wgrantDoesn't that mean it's trying to encode a str?00:57
lifelessimplicit conversions00:57
lifeless'foo %s' % u'bar' -> u'foo bar'00:58
wgrantFair point, missed the % bit.00:58
wgrantFocused on the encode.00:58
wgrantAnyway, -> maintenance peeps00:58
lifelessthe log message ends up being unicode because its % (...)00:59
lifelessor, ah01:00
lifelessit may be a str that is utf8, but 'fs' isn't ascii decodable01:00
lifelessso fs % msg-as-unicode blows up01:00
lifelessneed to see the values to know01:00
lifelessblah, my turn to miss stuff :)01:03
lifelessanyhow, yes we need to fix01:03
wgrantwallyworld_: Could you review <https://code.launchpad.net/~wgrant/launchpad/moar-fragile/+merge/76666>? About as trivial as they get.01:04
lifelesswgrant: why don't you self review it ?01:12
wgrantI didn't really want to self-review a change like that, but might as well.01:13
lifelesswhy not?01:14
wgrantIt has the potential to do bad things to fastdowntime unless I know what I'm doing.01:14
wgrantThis stuff is already fragile enough :)01:14
lifelessyou're adding one name01:15
lifelessI can't imagine a reviewer at the code style level having anything to contribute01:16
lifelessyou're either right, or wrong ;)01:16
wgrantI didn't want a style review :)01:16
lifelesswgrant: less use of imperatives in bug reports!01:22
wgrantlifeless: Fair point.01:30
wgrantwallyworld_: How goes QA for your trigger?01:42
wallyworld_wgrant: i'll ping stub when he is available. easier that way02:13
wallyworld_wgrant: sorry i missed your review. i was having an early lunch with a friend02:13
lifelesswhat do you need done?02:13
wallyworld_lifeless: i need to run some sql to update branch.private = true for say launchpad, and then check on of the branches stacked on lp to see that transitively_private is true02:14
wallyworld_plus say, once lp is private, unstack a branch and see that transitively_private is false02:15
lifelessso, future ref, use a losa02:19
lifeless24h coverage, which stub cannot supply.02:20
wallyworld_sure. i thought in this case it would be easier sonce he was familiar with the tirgger etc02:20
wallyworld_less hand holding required02:20
lifelessa little yes, but it adds latency, and we're kindof latency sensitive once things are in trunk02:21
wallyworld_this is in db-devel02:21
lifelessyees02:21
lifelessthats a trunk02:21
lifelessas is devel02:21
wallyworld_doh02:21
lifelesss/trunk/thing/ we deploy from02:21
wallyworld_for some reason i only think of devel as the "real" trunk02:22
lifelesswhen you're not dealing with the schema thats a pragmatic view02:24
lifelessbut with fastdowntime in the mix, db-devel has the ability to deploy as often as nodowntime has been deploying (but not as often as nodowntime /can/ deploy02:25
wallyworld_yes, 95% of the time it's just devel but you are right about db-devel02:25
wallyworld_lifeless: i need to catch when a model object property is changed. is the correct pattern in our architecture to make it a r/o property and introduce a setter, marked as @mutator_for(xxx) in the web services interface. the setter would publish an ObjectModifiedEvent which there would be a listener for02:52
wgrantThere are several different ways to do it.02:53
wgrantWhat exactly are you trying to achieve?02:54
StevenKCan anyone reproduce a overlay appearing at the top of a bug page on up-to-date devel? make run and then https://bugs.launchpad.dev/redfish/+bug/1502:54
_mup_Bug #15: PO file import errors should be more verbose <feature> <lp-translations> <Launchpad itself:Fix Released by carlos> < https://launchpad.net/bugs/15 >02:54
wallyworld_i need to know when a bug supervisor or security contact has changed on a project02:54
wallyworld_i need the old and the new values02:54
wallyworld_StevenK: give me a sec and i'll do it02:55
wgrantwallyworld_: You're going to attempt to do a mass subscription/unsubscription?02:56
wallyworld_wgrant: no. if a new bug supervisor is added to a project, they need to be auto subscribed if the project is related to any private bug tasks etc02:57
wgrantThat sounds like a "yes" to me.02:57
wgrantAnd a very bad idea.02:57
wgrantReally slow, and attempting to perform sensible disclosure on top of legacy disclosure.02:57
wgrants/perform/implement/02:58
wallyworld_i've been assigned a bug to do this work02:58
wallyworld_well, there's a 5 digit but and one i assigned myself which is related which curtis agreed to this morning02:59
wallyworld_s/but/bug02:59
wgrantAnd I will happily revert any attempt to do that, because it will time out.02:59
wallyworld_that's very prsumptuous03:00
wgrantIt's really not, unfortunately :/03:00
wgrantWe can do this once we have project observers.03:00
wgrantBut updating a couple of hundred thousand bug subscriptions in-request is not cheap.03:00
wallyworld_this is only for private bugs03:01
wallyworld_surely there's not 100000's of private bugs03:02
wallyworld_StevenK: what am i looking for?03:02
StevenKwallyworld_: http://people.canonical.com/~stevenk/overlay.png03:04
wallyworld_StevenK: don't see that03:04
StevenKThat is upsetting.03:05
StevenKThen WTF is causing it for me.03:05
wallyworld_try make jsbuild? is your js up to date03:05
StevenKThat is from a clean devel03:05
wallyworld_hmmm. i just pulled the latest devel tip myself and did make clean run03:06
wallyworld_is it just on the bug index page?03:06
StevenKHm, no.03:07
wgrant product | distribution | count03:07
wgrant---------+--------------+-------03:07
wgrant         |            1 | 2944103:07
wgrant    1694 |              |  359803:07
wgrant    9514 |              |  319603:07
wgrant    2982 |              |  186803:07
wgrant    8892 |              |  160703:07
wgrant   10294 |              |  124103:07
StevenKMany private bugs03:07
StevenKShows up on any bug page and https://launchpad.dev/~landscape-developers03:09
wgrantStevenK: I don't see it either.03:09
StevenK:-(03:09
wgrantWhatever Firefox Oneiric has this week.03:09
StevenKThen why do I? :-(03:09
wallyworld_i'm runnign latest ff beta and don't see it03:10
StevenKPerhaps Firebug is confounding me03:11
StevenKWhere's the option for halting on errors?03:12
wgrantStevenK: Tried Chromium?03:12
wallyworld_StevenK: can't remember03:12
wallyworld_wgrant: i assume distro number 1 is ubuntu?03:12
StevenKYes, Distribution 1 is Ubuntu03:13
StevenKwgrant: No. Don't really plan on, either. :-P03:13
wallyworld_so i could do it as a cron task. don't think we have a proper jobs infrastructure with persistent state etc yet?03:15
wgrantwallyworld_: Why don't we do this in a few months when we have non-legacy disclosure, with observer roles?03:15
wgrantwallyworld_: Adding a job is not lightweight enough that we should just throw it away in a few months.03:15
wallyworld_observers will probably work for this, yes03:17
wallyworld_or we can make sure it works03:17
wgrantIt's meant to do *exactly* this sort of thing.03:17
wgrantQuickly, too.03:17
wgrantWithout having to write-lock thousands of rows.03:18
wallyworld_so we would have to have some sort of denormalisation built into the model03:18
wgrantOh?03:18
wgrantBug supervisor will no longer imply visibility, just like it no longer implies notification.03:19
wgrantThe default will probably still be supervisor == visibility == notification, but like with structural subscriptions that will just be a default.03:19
wallyworld_oh ok. i would expect that as a bug supervisor i would have visibility, however it's done03:20
wgrantThat doesn't work for Ubuntu.03:20
wallyworld_because?03:20
wgrantWe have many non-Canonical people in the bug supervisor role, but we have coredumps and stuff that should not be visible to them.03:20
wgrant(I won't mention security bugs, because sinzui seems to be convinced that they should remain a special case)03:21
wallyworld_ok. seems like we are saying to someone to do a role, but not necessarily ggiving them all the info they might need ti make decisions?03:21
wallyworld_or have i misunderstood?03:22
wgrantThe bug supervisor role's primary purpose is to confer access to the restricted statuses, and the ability to set bug importance.03:22
wgrantIt was hijacked to be the default subscriber back when default-private-bugs were introduced, because it was convenient.03:22
wgrantAnd that has functioned reasonably well for a few years.03:23
wallyworld_what does "confer access to the restricted statuses" mean?03:23
wgrantAllow setting and unsetting of Won't Fix and Triaged, and unsetting of Fix Released.03:24
wallyworld_ok. but the info need to fix the bug, once triaged and assigned to someone, might remain invisible to the supervisor?03:24
wgrantThey don't know that that bug exists.03:25
wgrantUnless they are also defined as an observer.03:26
wallyworld_how can they set bug status if they don't know it exists?03:26
wgrantthat will be a sensible default for most projects, but it cannot be a universal rule.03:26
wgrantThey can't.03:26
wallyworld_but didn;t you just say that's what the supervisor does?03:26
wgrantThey can set status and importance on the bugs they can see, yes.03:26
wgrantThey obviously can't do it to the bugs they can't see.03:26
wallyworld_so there's no point making someone a bug supervisor of a bug they can't see03:27
wallyworld_and yet the rules now are that bug supervisors are subscribed to private bugs03:27
wallyworld_automatically03:28
wgrantThey are subscribed to private bugs when they are created in a default-private-bugs project.03:28
wgrantWhich is a tiny subset of projects.03:28
wallyworld_sure03:29
wgrantAnd an inflexibility on which we already have bugs filed.03:29
wallyworld_so with upcoming disclosure work, this will all change?03:29
wgrantRight. The bug supervisor will probably return to its original purpose, no longer implicitly providing visibility by default in some projects.03:32
wgrantThere's a separate observer list for that purpose.03:32
wallyworld_makes sense. thanks for the explanation03:32
wgrantSo, while it would be nice to do the subscriber transition on bug supervisor change in the current model, it's probably impractical to do now, and is trivially done in a much simpler way once we have the new privacy model up and running.03:34
wallyworld_sure03:43
lifelessguys03:48
wgrantUhoh.03:49
lifelessif we're doing any sort of mass-change, it should be backend driven, and use short (<=2 secs) transactions03:50
wgrantYes.03:50
lifelessfor bug supervisors03:50
lifelessquestion - is the intent that bug supervisors will be observers?03:50
wgrantI hope not.03:51
wgrantIf there is, the intent is incorrect.03:51
lifelessbecause I expect most projects will want the bug supervisor to see all private non-sec bugs, as they will want the project security team to see all private sec bugs03:51
lifeless!cite, I know.03:51
wgrantThat is the most common situation that I envisage.03:52
lifelessHow do we intend to help folk achieve that ?03:52
wgrantI believe the current proposal, which I vehemently disagree with, is to continue special-casing security.03:52
wgrantObservers won't be able to see bugs with the security flag set.03:53
wgrant(I haven't heard this confirmed for a couple of months, but it's the last thing I heard)03:53
lifelessobservers not seeing security is correct03:53
lifelessbut03:53
wgrantsure.03:53
lifelessperhaps we need types of observers, or something.03:53
wgrantBut I don't think it should be a special case.03:53
wgrantYes!03:53
wgrantBut it was argued that that is too complicated or something.03:54
lifelessit veers close to generic acls03:54
lifelessbut not too close, I think.03:54
wgrantTo me the most obvious solution is to specify a privacy policy on each bug, sort of like what branches have now except explicit and not crippled.03:54
wgrantMost projects would have two.03:54
wgrantPrivate and Security, I guess.03:55
wgrantTwo I can think of for Ubuntu are Apport and Security.03:55
lifelessright, this harks back to the prague, or was it texas, discussion03:55
wgrantThey probably wouldn't have Private.03:55
wgrantsinzui has basically vetoed this, I believe.03:55
wgrantBut I don't think we can not have it.03:55
lifelesswhere shared-namespace projects would have multiple policies.03:55
lifelessthe big ui issues I see are clearly communicating the impact of choosing such a policy for a bug.03:56
lifelessimplementation wise I think it would be easy and execute fast03:56
wgrantExactly.03:57
wgrantWe don't even have to expose it by default.03:57
wgrantSecurity could be a UI special case, if we want to keep it just how it is now.03:57
wgrantUntil we smooth things out and open up the general UI.03:57
wgrantBut the key is that the backend is general, just as fast as a security flag.03:57
wgrantAnd it lets us handle Ubuntu very well.03:57
wallyworld_what are some examples of shared namespace projects?03:57
wgrantWhere the security special case does not, at all.03:57
lifelesswallyworld_: storm03:58
lifelesswallyworld_: but more seriously, they are in an awkard halfway house at the moment03:58
lifelessI think we should -either- ditch them and say that projects are single-owner (e.g. if a private entity wants to work on some other public project, they can't do so in private: they need a separate private project which forks the other)03:59
lifelessor we need to do a lot of remedial work03:59
wallyworld_dumb question - what about storm makes it shared namespace?03:59
lifelessit has multiple different branch privacy policies03:59
lifelesswhen I push a branch there, you cannot see it.03:59
lifelessif you push a branch there, I can see it.04:00
lifelessthis is an example of how it works, not how it should be :)04:00
wallyworld_i didn't realise that04:00
lifelesstheres some enterprise integration branches hardware-qa folk have written04:00
lifelessnot yet opened04:00
wallyworld_lifeless: so what sets your branches private - an attribute on your IPerson instance?04:01
lifelessI'm in the hardware-certification team04:01
wallyworld_ok. and that's a private team?04:01
lifelessthat team matches one of the policies, and wins for whatever reason.04:01
lifelessthe team doesn't have to be private for this to happen04:01
lifeless(it may be in this case, I genuinely don't know)04:02
wallyworld_but it's policy based per project04:02
lifelessyes04:02
wallyworld_right04:02
wallyworld_lifeless: so how do people get to do code reviews on your storm mp's if your branches are private?04:03
wallyworld_i guess only certain people can04:03
lifelessthey need to be in the hardware-certification team04:03
lifelessits like a whole other project, sharing the namespace04:03
lifelessown rules, own policies04:03
wallyworld_sounds blah04:03
lifelessbut its not a complete separation04:03
lifelesslike series are shared04:03
lifelessso stacking is shared04:03
lifelesswallyworld_: well, the intent is for things where multiple organisations want to do independent private work, for them to do that.04:04
lifelesswallyworld_: it also is intended to support things like unity where we have private branches for a while which get opened when hardware vendors ship the final product and we are allowed to open source the code04:05
lifelessI'm neither justifying nor critiqing this04:05
lifelessjust trying to explain why, what it is, and its current status04:05
lifelessin the former folk want partitions04:06
wallyworld_lifeless: sure, and i appreciate the insights04:06
lifelessin the latter folk want public-private partitioning and then free-for-all behind the partition04:06
lifelessoem are unique in doing this multiple times with multiple partitions, and they don't use shared namespaces, instead they run N projects (and want us to link things cross-project)04:06
lifelessarguably these things are equivalent, but choosing one approach and really polishing it probably makes a lot of sense04:07
wgrantThere are also non-commercial instances of this sort of thing.04:07
wgranteg. Ubuntu has really private apport bugs, sort of private apport bugs, and security bugs.04:07
wallyworld_i'm all for making *one* nicely polished process that can be tuned to fit the required use cases04:08
lifelessthats closely related yes. I wouldn't call it a shared namespace as such04:08
* wallyworld_ sighs. another compiz crash :-(04:09
wgrantlifeless: Exactly: it's not shared-namespace, but it is an example of the security special case not working.04:18
* wallyworld_ reboots04:48
lifelessjamesh: hi; I don't suppose you've had a chance to try glueing oops and u1 together again ?05:05
jameshlifeless: sadly not.05:06
lifelessno worries05:06
lifelessbtw, I have a native twisted oops config, if that is useful05:06
lifelessshould be publishing the code today05:07
jameshthe python-oops bug I filed came from my chasing down a bug in the wrong direction because of similarly named exceptions05:07
lifelessyeah05:08
lifelessit makes total sense05:08
lifelesswe did similar to testtools output05:08
jameshPython's DB-API encourages lots of similarly named exceptions, including one called "Error"05:08
lifeless\o/05:10
lifelessbbiab05:11
jameshthe spec could really have benefited from a stdlib module for all the adapters to build on top of05:11
rsalvetisorry asking here, wasn't able to find at launchpad, but is there a bug already on the code import about the ability to only import trunk when importing a git tree?05:22
rsalvetiwould like to also be able to import a git branch05:23
rsalvetiI know it's probably because of a limitation of bzr-git or similar, but would like to also see a bug for that at launchpad05:23
rsalvetiguess jelmer should know this :-)05:25
rsalvetihttps://answers.launchpad.net/bzr-git/+question/17152205:26
rsalveticool, seems the support is already at bzr-git trunk05:26
wgrantrsalveti: You can actually import them in LP now.05:37
wgrantrsalveti: Since a week or two ago.05:37
wgrantIt's not documented or obvious yet, however.05:37
rsalvetiwgrant: oh, ok, any special syntax?05:37
wgranteg. https://code.launchpad.net/~wgrant/mochiweb/R12B05:38
wgrantAdd ,branch=whatever to the end of the URL.05:38
rsalvetiwgrant: oh, awesome :-)05:38
rsalvetiI love you guys, launchpad rocks05:38
wgrantThere'll hopefully be some UI for it soon, but at least it's possible now :)05:39
rsalvetijust when I needed \o/05:39
rsalvetiwe'll be doing CI and daily builds of the linaro components05:39
rsalvetiand they are mostly tracked using git05:39
wgrantExcellent.05:39
rsalvetiwgrant: thanks05:40
nigelblifeless: Hey! You asked me to ping you today for a project skeleton and sketch, etc :-)06:07
nigelbwgrant: So now, everything that touches db need to have the pre-requisite branches deployed everywhere before landing?06:10
wgrantnigelb: Sadly not yet. mizuho, cocoplum and germanium still need fixing. mizuho is having other work done to it on probably Monday, which means we should be able to upgrade it then.06:11
wgrant(it's awkward because it's not HA yet, so we can't safely restart it while LP is up)06:11
wgrantThe work on Monday should make it HA.06:11
nigelb\o/06:11
wgrantIt was meant to be done last night, but, er, things did not go smoothly.06:12
nigelbheh06:13
nigelbOh well, time to head to work. Laters.06:13
lifelessnigelb: I did06:28
lifelessnigelb: uhm, give me half an hour or so06:28
StevenKwallyworld: Wow, no underscore.06:28
wgrantHeh06:28
wallyworldyeah, it ran away06:29
StevenKwallyworld: No green animated awesomeness for that branch.06:29
wallyworld?06:29
StevenKwallyworld: But I will hit you up on Monday about it.06:29
wallyworldyou mean the latest bug confirmation work06:30
StevenKconfirming unsubscribe on private bugs, yes.06:30
wallyworldcool06:30
* wallyworld hates debugging @^@$@%$^ doc tests :-(07:03
rvbaMorning.07:04
wallyworldg'day07:05
rvbaHey, how's it hanging wallyworld? ;)07:05
wallyworldrvba: shit. ec2 gave me the finger and now I've got to find an error in a doc test :-(07:06
* rvba hates doc tests too.07:07
* wgrant stabs Storm.07:07
wallyworldwhat did Storm do to you?07:07
wgrantIt's generating UPDATE queries with = NULL instead of IS NULL.07:08
wallyworldwgrant: it also does that for =/is True07:08
rvbawgrant: I've got a question for you: I have four dependent branches: A -> B -> C -> D; a certain wgrant reverted the whole thing (and he was right) ...07:09
wgrantHmmm, I guess I can't exactly use a nullable field in a Storm primary key anyway.07:09
rvbaNow I've created C': A -> B -> C'07:09
rvbaBut when I try to land it, it gets merged with devel which has A and B reverted so ec2 gives me the finger (like wallyworld says ;))07:10
wgrantrvba: What we normally do in that case is create a branch to revert the revert (bzr merge -r$(revert)..$(revert-1) .), then develop the fix on top of that.07:11
rvbaI could simply merge devel, recreate A' (similar to A) and B' (similar to B) but maybe there is a better way to do this?07:11
wgrantOr you could probably merge C' onto the branch with the reverted revert.07:11
wgrantThat would probably work.07:11
rvbaIf I could avoid having to create new MP for thing that have been approved already it would be nice ...07:11
rvbathings* even07:12
rvbawgrant: I think that since all the changes from A, B, C and D have been reverted all at once, I can't revert the revert properly...07:27
wgrantrvba: Ah. In that case, cherrypick the A and B merges from devel back on top of your branch.07:28
rvbawgrant: Okay, I'll do that, thanks for your help...07:28
wgrantThat'll hopefully work. I haven't had cause to partially unrevert a multi-branch reversion before :/07:30
wgrantIf it doesn't, I will poke further.07:31
rvbaSeems all the changes where committed as one commit in devel.07:31
wgrantAh, of course, so they were.07:31
rvbaI tried to cherry pick from my branch (as opposed to devel) but something seems to be wrong with that ...07:32
wgrantThat'll probably go wrong if you merged devel into your branch during its development.07:32
=== wallyworld changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: - | Critical bugs: 256 - 0:[#########]:256
rvbaI did merge devel during its development.07:33
rvbaLet see if I can create a simple patch from the dev branch and apply it manually to the new branch then.07:33
wgrantIndeed, you could do a diff -rancestor:../devel to get the patch.07:38
rvbaI've done a patch using cherry picking (i.e. specifying the revisions) but this seems to work.07:38
lifelesswgrant: what evil are you working on?07:45
wgrantlifeless: PillarObserver (currently just for optimising display and revocation of legacy restricted disclosure, but in the future it will probably be used to grant non-legacy observer privs, I guess)07:47
lifelesshow does it optimise the legacy stuff?07:49
wgrantlifeless: My current theory is that access to private artifacts will depend on holding either full Observer privileges, or Restricted Observer and a subscription.07:50
wgrantWe need to be able to efficiently show who has access to private artifacts on a project.07:50
wgrantAnd revoke that access.07:50
lifelesssure07:51
lifelessI humbly suggest that you want a fact table here07:51
lifelessrather than doing it in your online processing table07:51
lifelesswe can talk on monday if that doesn't make sense07:51
wgrantMy online processing table?07:51
lifelessthe live table where you change things07:52
lifelessthe one optimised for letting users search bugs etc07:52
lifeless-> the one that actually grants access07:52
wgrantI have a table PillarObserver(pillar, person, permission), which subscribing someone to a bug currently adds a RESTRICTED permission to, if it doesn't already exist.07:52
wgrantIt's not flattened, so you do have to join against TP, but that seems to be fast enoguh.07:52
lifelesswgrant: you've shoved real world data sizes into it ?07:52
lifelessjoining against TP should be fine07:53
wgrantAttempted to, but it's really hard to say what is a representative DB.07:53
wgrantRight, that's what I thought.07:53
wgrantIt should be a small enough set of useful rows that the join should work fine.07:53
wgrantUnless someone is in 100000 teams.;07:54
wgrantIn which case we might be in trouble.07:54
lifelessNow, IIRC there was a requirement to show the artificats flk have access to07:54
wgrantYes.07:54
lifeless2K is the largest in-team set at the moment.07:54
wgrantI plan to initially use bugsummary to display counts, possibly even on the listing.07:54
wgrantThen once we drill down list subscriptions.07:54
lifelessthat might be shiny07:54
wgrantWe should really check bugsummary's consistency at some point.07:55
lifelessI keep meaning to file a bug saying that its either confusing output, or wrong - oem let me know its off by 25% in some pages07:55
lifelesswhich is -way- more than my simulations predicted07:55
wgrantOEM is a bit pathological in a lot of cases, though.07:55
lifelessdisclosure will help a lot07:55
wgrantYeah.07:55
lifelesswgrant: I've seen odd numbers myself, I just can't remember where07:56
wgrantWe can hopefully mostly summarise on privacy policy.07:56
wgrantWIth only exception subscriptions throwing everything off.07:56
wgrantAnd private projects should have very few of them.07:56
wgrantI've tried various different strategies here, and I think this approach is going to be the simplest, most efficient and most future-proof.07:58
lifelesswill there be a blacklist facility ?07:59
wgrantOthers probably won't outlast legacy disclosure without a complete redesign, and/or run into what wallyworld was trying to do earlier.07:59
wgrantie. an explicit DENY?07:59
lifelesse.g. 'foo has restricted, but thats wrong, reject them -> leads to them not being addable again'07:59
lifelessat least not implicitly via subscriptions07:59
wgrantSo, at present I'm emulating legacy disclosure. Subscribing someone to a bug adds a Restricted permission if they don't already have at least that. I imagine in future that will probably not implicitly happen, or projects will be able to reject subscriptions who don't have a permission explicitly added.08:00
wgrantWe can't sensibly enable the revocation functionality until we've worked out how the new artificat visibility UI will work, but I don't want to design a legacy disclosure viewing model that won't be future-proof.08:01
lifelesssure08:01
lifelesswill po be part of bug queries ?08:02
mrevellHi!08:02
wgrantlifeless: Yes.08:04
wgrantlifeless: I initially tried to avoid that.08:04
wgrantlifeless: But then realised that it will have to be in future anyway.08:04
wgrantlifeless: For full observer roles.08:04
wgrantIt should be quick enough to join through that and TP that it won't be too slow. I have tried on reasonably large datasets, but who know how it will perform in the wild.08:04
wgrantFF FTW.08:04
adeuringfgood morning08:06
=== almaisan-away is now known as al-maisan
=== al-maisan is now known as almaisan-away
=== leguin.freenode.net changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: wallyworld | Critical bugs: 256 - 0:[#########]:256
StevenKDear Freenode, stop sucking.08:17
rvbaStevenK: Hi ... and congrats on the JS branch! ;)08:19
lifelessnigelb: doing your thing in 508:19
StevenKrvba: Thanks, couldn't have done without you. :-)08:23
rvbaStevenK: Np ;)08:24
bigjoolsmorning all08:26
nigelblifeless: \o/08:44
lifelessbigjools: http://pypi.python.org/pypi/oops_twisted08:45
nigelbI'm at work, so my response times are horrible :|08:45
bigjoolslifeless: yay08:45
lifelessbigjools: I'm just adding its tarball to the download cache for you08:45
nigelbMorning bigjools!08:45
bigjoolslifeless: no Launchpad download? *cough*08:46
bigjoolslifeless: anyway that's awesome, thanks08:46
bigjoolshello nigelb08:46
lifelessbigjools: it should be straight forward to use08:47
bigjoolslifeless: yeah we talked about that already, sounds great08:48
stubbigjools: So does the db permissions approach I'm taking in https://code.launchpad.net/~stub/launchpad/distinct-db-users/+merge/76535 , or do you still think it will be painful and things will break?09:00
* bigjools looks09:00
stubsecurity.cfg bits, where I'm just declaring the distinct db users as aliases of a more powerful group09:01
bigjoolsstub: I think it's fine. ideally we'd have role groups, but I guess that would be painful to do now09:02
bigjoolsI already did the same trick for packagecopyjob09:03
stubbigjools: I expect if someone can think of a good name we just rename 'fiera', 'uploader', 'queued' etc. to start with and stick with them as the roles.09:03
bigjoolswell I was thinking more granular09:03
bigjoolsbecause queued, for example, closes bugs09:03
stubbigjools: Yes, but that takes time as you say :-)09:03
bigjoolsyes :)09:04
nigelblifeless: were you able to do that thing for me? :)09:07
=== adeuring changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: adeuring | Critical bugs: 256 - 0:[#########]:256
lifelessnigelb: download cache upload just finished09:22
lifelessbigjools: committed to the cache for you09:24
bigjoolslifeless: ta09:25
bigjoolslifeless: why did you call it oops_twisted on pypi?09:25
lifelessthats the package09:26
lifelessimport oops_twisted09:26
wgrantstub: Is it deliberate that LP defaults to serializable isolation, while things that call initZopeless default to read committed?09:27
bigjoolsyes, but we have python-oops09:27
lifelessbigjools: http://pypi.python.org/pypi/oops09:27
bigjoolsok09:27
stubwgrant: yes09:28
wgrantstub: Some scripts don't call initZopeless, so they end up serializable :(09:29
stubwgrant: need to fix that. hang - on phone09:29
bigjoolslifeless: I am writing tests for my txlongpollfixture and my buildout test runner doesn't have the path to the executable but the interpreter does, any idea how I can fix it?09:35
bigjoolsso Popen("bin/txlongpoll") works inside bin/py, but not in a test09:35
lifelesswhats cwd in the test ?09:35
bigjoolsit gets changed to the tempdir09:37
bigjoolson the Popen call09:38
bigjoolsbut removing it makes no difference09:38
lifelessso . isn't in your PATH, you'll need to use the abspath to the executable09:38
bigjoolslifeless: how does that work when I am using an egg?09:38
bigjoolsI don't know where it is09:39
bigjoolsonly buildout does09:39
lifelesserm, I'm not sure offhand09:39
lifelesssorry, ELOCAL09:40
stubwgrant: Can you point me at a script that doesn't call initZopeless?09:40
stubI thought this was all handled in the LaunchpadScript base class09:41
stubAnd everything was supposed to use that (or LaunchpadCronScript)09:43
wgrantstub: All LaunchpadScripts do it, yeah. But some stuff isn't a LaunchpadScript.09:47
wgranteg. I ported a dozen scripts to LaunchpadScript on Sunday, because most of them called initZopeless directly.09:47
wgrantBut I also ran into a couple that I can't remember right now, that call execute_zcml_for_scripts but not initZopeless.09:48
wgrantpoppy-sftp seems to be the only thing left.09:48
wgrantThere are some others, but they are short-lived and rarely run on prod.09:49
wgrantstub: I think it's somewhat unfortunate that ISOLATION_LEVEL_DEFAULT isn't actually the default any more, though.09:53
wgrantPerhaps now that just about everything uses LaunchpadScript it will be less bad.09:53
stubwgrant: The reason for using read committed over serializable is that if you use serializable you will randomly get serialization exceptions and are expected to deal with them.10:25
bigjoolsfor my sanity, if I add a new dependency in launchpad-developer-dependencies it's not going to affect  buildbot is it?10:26
stubwgrant: So a foot gun, but scripts only hurt themselves.10:26
wgrantbigjools: It won't automatically, but it will when somebody next tries to update them.10:26
wgrantbigjools: Same with ec2.10:26
wgrantstub: Right.10:26
wgrantstub: I'm just strategising around saner script DB config APIs.10:27
bigjoolswgrant: why is pdr showing script warnings this morning?10:29
wgrantbigjools: was killed for fastdowntime.10:29
bigjoolshow long is it taking to run now?10:30
wgrant1.5 hours still.10:30
bigjools:/10:30
bigjools8 mins to 1.5h is not goo10:30
bigjoolsd10:30
wgrantIndeed not, but we now have everything close enough to fixed that I think we know everything that is wrong.10:31
wgrantjelmer: What have you done to https://code.launchpad.net/~jelmer/wireshark/svn?10:35
wgrantjelmer: It seems rather impressively broken.10:35
bigjoolswgrant: can you review: https://code.launchpad.net/~julian-edwards/meta-lp-deps/add-rabbit-management/+merge/7671010:39
wgrantbigjools: This needs some thought.10:40
wgrantbigjools: It will make LP uninstallable on oneiric.10:40
wgrantAnd launchpad-messagequeue-dependencies uninstallable in the DC.10:40
bigjoolsthe latter easily fixed10:40
bigjoolswhy the former?10:40
bigjools(this is a losa request BTW)10:41
wgrantBecause oneiric has rabbitmq-server 2.5.010:41
wgrantSo rabbitmq-management 2.3.1 won't install.10:41
bigjoolshmm10:41
bigjoolsthis is why I asked you to review this :)10:42
stubSo given we should all have switched to Oneiric per policy...10:43
bigjoolswgrant: I am at a loss. I think the only way forward is to package the 2.5.0 plugin but ....10:47
jtvwgrant: are we also quite quite sure that nothing and nobody will mark an SPPH as Deleted after Gina is done importing the package but before its domination run?10:49
wgrantjtv: Nobody is meant to have privileges to do that. And even if they do, what's the worst that happens? It gets dominated 6 hours late.10:53
jtvwgrant: unless the number of versions increases to offset the decrease in Published publications.10:54
wgrantbigjools: I think so too. Which is why I was saying a week or so ago that the way forward is 2.5.0 or possibly even 2.6.0.10:54
bigjoolswgrant: yes, and I agreed with you, but then I found out what a total bitch it is to package that stuff10:55
bigjoolsoneiric is getting 2.6.010:56
wgrantIt's actually really simple!10:56
* wgrant calls StevenK in for backup.10:56
bigjoolswgrant: well you said it wasn;t :)10:56
wgrantbigjools: For a packager it is simple :)10:56
bigjoolsplugins requiring the parent's Makefile10:56
bigjoolsodd version requirements10:56
wgrantSure, there are a couple of messy bits, but those are already solved.10:56
bigjoolsFSVO solved :)10:57
bigjoolsbut it'll do10:57
bigjoolsso can I invoke the Antipodean Packaging Brigade?10:57
wgrantDamn, you're not Soyuz any more, so I can't say you should finally learn packaging :(10:58
bigjoolsyou can still say that10:58
bigjoolsit's not like I don;t want to10:58
bigjoolsbut $TIME .... :/10:58
bigjoolsand I don't want to block this10:58
wgrantBah.10:59
wgrantI'll see what I can do on Monday.10:59
wgrantIf the upgrades are trivial enough, it should only take an hour or two.10:59
wgrantIf they're not, I'll have to see.10:59
bigjoolswe could ask someone in distro, if they are already packaging 2.6.011:01
wgrantThat may also be a good option.11:01
bigjoolsthey had a FFe for it11:01
wgrantLate!11:02
* nigelb waves11:02
wgrantEvening nigelb.11:02
nigelbLazy Friday evening :)11:03
bigjoolspackaged by Marc Cluet it seems11:03
wgrantSigh, their rabbitmq-erlang-client packaging is still stupid.11:05
wgrantIt installs both the extracted files and the archive containing all of them.11:05
StevenKwgrant: Grab me on Monday and I'll do part of it too11:06
bigjoolsthanks guys11:06
nigelbbigjools: Starting work on longpoll? :)11:07
wgrantUpstream provides reasonable 2.6.0 packages, so it's just a matter of updating the plugin infrastructure and getting the versions to match.11:07
bigjoolsnigelb: a while ago11:07
bigjoolswgrant: oh, they've done packages?11:07
nigelbbigjools: Can't wait to see it in production :-)11:07
bigjoolsnigelb: we're also moving to a slightly different deployment model, so it's been a bit trickier than usual11:08
wgrantbigjools: They provide a rabbitmq-server package which appears to be based on Debian's, or vice-versa.11:08
bigjoolsbut nothing for the management plugin?11:08
nigelbOh, how different is the deployment going to be now?11:08
wgrantNo.11:08
bigjoolsballs11:08
bigjoolsnigelb: SOA-based11:08
wgrantTheir deployment story for plugins is to download the .ez from the website and drop it in the system dirs.11:08
nigelbAh. Right.11:09
bigjoolslots of small microservices11:09
jelmerwgrant: it's a bzr code import11:09
bigjoolswhich will be *awesome* when we can start splitting up LP itself11:09
nigelbI'm writing one of those services :P11:09
wgrantjelmer: Ah.11:10
jtvwgrant: wanna review?  https://code.launchpad.net/~jtv/launchpad/bug-857155/+merge/7671511:30
wgrantjtv: Looks good. Should we perhaps take this opportunity do some logging during domination?11:35
wgrantjtv: We presently log two lines per published package; another one per package being dominated would be useful and only 50% larger :)11:35
jtvwgrant: if you like.  I'm also preparing a post-branch with additional improvements (bulk-loading and a renaming), but logging could go into the original.11:35
wgrantIt should be a single line, so why not :)11:36
jtvWell it's not _that_ easy.  The place where I'd most like to log this doesn't know the package name.11:38
wgrant:(11:40
wgrantDepends where you want it, I guess.11:40
jtvwgrant: I've added some debug and debug2 logging: debug will give you the name and stats for a package being dominated; debug2 also says which packages it skips *and* what it does to each publication.  Pushing.11:52
jtvIn the follow-up branch I'll change the name of that domination method (because they no longer have to be “removed” versions) and add bulk-loading of the sprs.11:54
wgrantjtv: That sounds like a good option.11:54
jtvIf you _really_ want, you'll be follow each step of the algorithm.11:54
bigjoolsdogfood is going down for 2-3 days for DB refresh11:55
wgrantHm, that is awkward.11:55
jtvHmm maybe I should request a Q/A run of my branch on staging now.11:55
wgrantRight when we need to QA gina!11:55
wgrantBut yeah.11:55
bigjoolsstaging FTW11:56
jtvWe knew dogfood would go down, and postponing it would just give us more grief next week.11:56
jtvYup.11:56
wgrantWe'll need to cowboy the config and get a partial archive onto there.11:56
wgrantBut we should try it.11:56
* jtv toys with idea of running old & new versions on staging & qastaging respectively, just so he can compare results11:56
bigjoolswe need to shift onto staging more11:56
jtvOch aye, we can't run gina's import there.11:57
wgrantThat debug2 logging looks marvellous.11:57
wgrantApprovalised.11:57
jtvThanks.11:57
jtvstub defined some extra debug levels, including “BLATHER”11:57
bigjoolslaunchpad_dogfood=# drop database launchpad_dogfood;11:58
bigjoolsmuahahha11:58
wgrantWill it let you do that with active connections?11:58
bigjoolswhat active connections.... :D11:58
wgrant21:58:26 < bigjools> launchpad_dogfood=# drop database launchpad_dogfood;11:58
nigelbbigjools: You need to work on that evil laugh :D11:59
lifelessbah, bad oops_wsgi 0.0.4 tarball.11:59
bigjoolsit's only semi-evil11:59
lifelesssilly setuptools manifest handling11:59
bigjoolsyou'd think that typing a ticket number into the twisted trac search, with only "tickets" selected, would find the exact ticket first.  But oh no.12:02
lifelessjamesh: oops-wsgi had a bad tarball - stale MANIFEST, I've done a 0.0.5 to fix.12:05
=== matsubara-afk is now known as matsubara
lifelessnigelb: ok12:11
nigelblifeless: "free-er"? :)12:11
lifelesslp:js-oopsd is populated12:11
nigelblifeless: Thanks! Looking now12:12
lifelessno tests, and nothing useful captured, but it should be within reach rather than being from a standing-stat for you12:12
nigelbwow12:12
lifelessthere is some science fiction in the README etc, so read carefully - you'll need to fill those bits out :)12:14
nigelblifeless: Thanks, that's awesome. You just did all the bits I had no clue (buildout, etc), and I there's boilerplate wsgiref code \o/12:14
lifelessto experiment12:14
lifelessrun it with12:14
lifelessbin/py -m js_oopsd.main12:14
lifelessand use your browser to visit the url it prints12:14
nigelbokay@12:14
lifelessthen look for a directory like 2011-09-2412:15
nigelbs/@/!12:15
lifelessin that will be OOPSes12:15
lifelessone for each request your browser made12:15
nigelbcool!12:16
lifelessso you can see its -very- shallow :)12:17
bigjoolslifeless: so, oops logging added to longpoll already.  Nice and easy!12:18
lifelessbigjools: worth the wait ?12:18
bigjoolslifeless: of course12:18
lifeless:P12:18
bigjools:)12:18
lifelessok, time for bed12:18
lifelessgnight all12:18
nigelbg'nite lifeless12:18
bigjoolslifeless: so one more question!12:18
lifelessheh, nick of time!12:18
bigjoolslifeless: if I add more log observers will it affect any of this?12:19
bigjoolsor vice-versa12:19
bigjoolssince Twisted bug 638 got fixed I can add a proper rotatable log file now12:19
lifelesstwisted dispatches all log items to all log observers12:19
_mup_Bug #638: Incorrect link in the bug's task listing <lp-bugs> <Launchpad itself:Fix Released by bradb> < https://launchpad.net/bugs/638 >12:19
bigjoolspoifect12:19
bigjoolsta12:19
lifelessbigjools: however12:19
bigjoolsyou may sleep now12:19
lifelessbigjools: I suggest using the OOPSObserver fallback parameter12:20
lifelessto point at the rotatable log file observer, rather than having it as a direct observer.12:20
bigjoolsright12:20
lifelessthis will mean you get the oops ids logged to the log file, rather than having the tracebacks in that file and no oops id12:20
lifelessta-ra12:20
bigjoolsright12:21
=== bac changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: adeuring, bac | Critical bugs: 256 - 0:[#########]:256
bachi adeuring, looks mighty quiet here12:31
adeuringmorning bac, yes, is very quiet today12:31
StevenKAre you two hunting MPs?12:51
stubjtv: I can't take credit for BLATHER - that is brought over from Zope2 to Zope3. I just downgrade its integer value :)13:00
jtvReviewer wanted, while I go off and have weekend!  https://code.launchpad.net/~jtv/launchpad/post-857155/+merge/7673313:10
nigelbheh13:10
nigelbnot very tempting offer :P13:11
jtvWhy not?13:11
=== jtv is now known as jtv-afk
adeuringmorning deryck:, could you please run this query on staging: https://pastebin.canonical.com/53295/ ?13:17
deryckadeuring, sure.13:18
adeuringthanks!13:18
deryckadeuring, https://pastebin.canonical.com/53296/ (run twice to see with/without cache)13:21
adeuringderyck: cool, thanks!13:21
derycknp!13:21
mrevellHey deryck, do you have time for a ten minute call this afternoon?13:22
deryckmrevell, sure.  you want to calendar a time that works for you?13:22
adeuringderyck: just to be a bore sur, could also try this one: https://pastebin.canonical.com/53297/ (should be sloooow)13:23
deryckadeuring, sure13:25
mrevellderyck, thanks, will do13:27
deryckadeuring, https://pastebin.canonical.com/53298/13:28
adeuringderyck: thanks!13:28
derycknp!13:28
deryckadeuring, abentley -- coming. just running a couple minutes behind, sorry.13:33
deryckadeuring, abentley -- https://answers.launchpad.net/launchpad/+question/17208913:38
deryckadeuring, hey.  see recent mails from me about that question, where I cc'ed you.13:52
bigjoolsMP diffs are slow today13:52
adeuringderyck: ack13:53
deryckadeuring, if it ends up being a timeout bug, or something that needs coding to fix ;)  we should pull it back to the next lane….13:54
deryckadeuring, and let whoever gets free first deal with it.  since there's the other escalated bug already in next.13:54
adeuringok13:54
flacostegary_poster: you are not really fixing bug 724609, right?13:57
_mup_Bug #724609: Rewrite lp.client tests using yuitest <build-infrastructure> <qa-ok> <Launchpad itself:In Progress by gary> < https://launchpad.net/bugs/724609 >13:57
gary_posterflacoste, I was intending to soonish, yes, as an example of using the new YUI XHR tests.13:57
bigjoolsgary_poster: hello, I have another buildout question for you if you have 5 minutes?13:58
mrevellderyck, Skype or Mumble?13:58
gary_posterIn fact, I was about to include that in my announcement flacoste, so if that's bad somehow lemme know. :-)13:58
flacostegary_poster: cool, but it's not fixed yet (as qa-ok makes me believe)13:58
flacostegary_poster: or did you just rewrite the tests already13:58
deryckmrevell, let's mumble.  give me 2 minutes.13:59
mrevellk13:59
gary_posterflacoste, I have to say qa-ok for incremental steps.  I landed with --incr because the branch introduced the infrastructure needed to write the test, but then when you do that you still have to assert that you have qa'd your incremental changes (I thought I used --no-qa actually, but that's a separate issue)14:00
flacostegary_poster: ok, makes sense14:00
flacostegary_poster: and i'm very happy that you are fixing this bug!14:00
gary_posterflacoste, cool :-)14:00
gary_posterbigjools, sure14:00
bigjoolsgary_poster: thanks you might be my saviour again.14:01
gary_poster:-)14:01
bigjoolsgary_poster: I have a txlongpoll egg, which build the txlongpoll server. I have another egg, txlongpollfixture which depends on the former14:01
bigjoolsgary_poster: I am struggling to work out how the fixture can find the executable that the first egg builds, so it can run it up14:02
deryckmrevell, orange 1 o 1 when ready14:02
gary_posterbigjools, ah fun.  Mm, if I understand you, you may have to use setuptools APIs to do that the "right" way.  The "wrong" way might be easy: it might be in the egg, in EGG-INFO/bin.  Another alternative is to have the fixture call the code that the executable "fronts," if that works for you.14:05
bigjoolsgary_poster: we discussed the wrong way earlier, it could get messy depending on what else builds it I think. I'm not sure I understand the 3rd alternative there though, can you expand?14:07
bigjools(bearing in mind we need to Popen the executable)14:09
gary_posterbigjools, typically you create executables by simply pointing setup.py at a function and saying "make a bin for this function and call it ${whatever}."  What you can do then in this case is set up your module to be run as __main__ (IIRC) and run "${executable} -m txlongpoll.module_with_your_executable ...options..."14:10
gary_posterif they are equivalent that you are good to go14:11
bigjoolsgary_poster: have you got an example of that?14:12
gary_posterbigjools, :-) I am looking for the pertinent docs, 1 sec14:12
jcsackettis anyone else regularly getting rabbitmq timeouts when trying to run launchpad.dev?14:13
bigjoolsgary_poster: is this the entry_points thing I can see in LP's setup.py?14:15
gary_posterbigjools, that's how you typically make an executable, yes.  So, in txlongpoll's setup.py, you would define a binary there.  If it were installed via something like setuptools in a virtualenv or real Python, the bin would be created.14:20
gary_posterThen for code that uses it as a library, you would define the same entry_points thing in our setup.py or in our buildout.cfg.  That entry point would also be run in the associated module's def __main__ (http://docs.python.org/using/cmdline.html#cmdoption-unittest-discover-m) so that code like your fixture would not have to find (or even have!) the binary, just run {executable} -m {module} {any other arguments that you14:20
bigjoolsgary_poster: it's not a library14:20
bigjoolsit's a standalone daemon14:21
bigjoolsI already have it building from from a buildout recipe14:21
gary_posterbigjools, ok, but the code can still be imported, right?14:21
bigjoolsnot easily14:21
bigjoolsit's twisted14:21
gary_posterbigjools, ohhhh, so they don't play the normal Python games?14:22
gary_poster:-(14:22
bigjoolsnot in the slightest14:22
gary_posterah :-(14:22
gary_posterum14:22
bigjoolsyou have to put a plugin in a special place14:23
bigjoolsthen run twisted with the right args14:23
bigjoolswe'll have this same problem when we eventually make the librarian a SOA part14:23
gary_posterbigjools, well, then setuptools may not help you either.  what's an example of the command you want to run?14:23
bigjoolsgary_poster: https://bazaar.launchpad.net/~julian-edwards/txlongpollfixture/FIRST/view/head:/txlongpollfixture/server.py14:25
bigjoolsgary_poster: see the _start() method14:25
bigjoolsline 73 onwards14:25
bigjoolsif we could get the bin directory in the PATH then we're golden14:26
bigjoolsor even the egg directory14:26
gary_posterbigjools (I have a call in 20, fwiw.  can return to this afterwards.)14:28
gary_posteractually about 15 now14:28
bigjoolsgary_poster: np, thanks14:29
gary_posterI'm going to go look at how you are creating bin/txlongpoll...14:29
bigjoolsscript recipe14:29
gary_posterOK so txlongpoll=twisted.scripts.twistd:run , which is identical to the twistd in setup.py except it has its own custom path that includes the txlongpoll egg.  So...it really is an arbitrary binary from the perspective of other code.  It could be running a different Python version even.14:33
gary_posterbigjools, I don't see an opportunity for magic snarfing here.  If you don't want to tie txlongpollfixture to the LP build (and I would understand not wanting to), you need to make this something that is configured--for instance, the fixture requires you to inform it where the binary is.14:37
gary_posterOnce you do that, you can have Launchpad either hardcode the binary name/location in the test fixture initialization, or do buildout tricks.  I'd be inclined to do the former, relying on config's ability to easily get you LP's root, and then hardcoding 'bin' and 'txlongpoll'.14:37
bigjoolsgary_poster: yeah, I was considering just ensuring that the person using the fixture has the executable in $PATH14:37
bigjoolsyeah, hardcoding bin/txlongpoll was the original plan, I figured that buildout might set the path. I was wrong :)14:38
StevenKWhat's wrong with figuring out your current path with os.path.abspath(os.path.dirname(__file__)) ?14:39
bigjoolsbecause it's not necessarily always in the same relative place to the executable14:39
bigjoolsit depends how the egg was built14:39
bigjoolsthe dependent egg I should say14:39
bigjoolsbut I might be wrong, I had discarded this way earlier, maybe it was too hasty14:40
gary_posterbigjools, eh, there's a million ways for that kind of config. :-) but yeah, I'm afraid it does need to be config, AFAIK.  buildout can tell you what has been generated, particularly while it is generating it, but since this is really buildout generating a separate binary with a separate context (that could even be in another language as far as the main LP code is concerned)14:40
bigjoolsindeed14:41
bigjoolsStevenK: actually thinking about it that definitely won't work because the bin/ directory can move around14:42
bigjoolsgary_poster: ok I will set the PATH in my fixture's tests so at least they work, then whoever uses the fixture needs to set the path as well14:42
bigjoolsthanks for looking14:42
gary_posterbigjools, does the txlongpollfixture depend on txlongpoll, and does/will LP depend on txlongpollfixture?14:43
bigjoolsgary_poster: yes, exactly that (will)14:43
gary_posterbigjools, in that case, all the nice separation you've done in buildout is a bit for naught14:44
bigjoolsgary_poster: blame lifeless!14:44
gary_posterbecause LP will still have txlongpoll as a dependency14:44
bigjoolswell it was to prevent txlongpoll having a dep on the test fixtures14:44
gary_posterDoes the fixture really need txlongpoll as a dependency?14:44
bigjoolsand this is only for the dev environment14:45
bigjoolsin prod we'll run it differently14:45
gary_poster...14:45
bigjoolsprod doesn't need the fixture14:45
gary_posteryeah, I understand that part.14:45
gary_posterSo...14:46
adeuringderyck: to be a bit more sure, could you try this query: http://paste.ubuntu.com/695674/ ?14:46
gary_posterThis is maybe a discussion for another forum, but just to see if I understand14:46
deryckadeuring, sure.14:47
gary_posterbigjools, txlongpollfixture is needed to test txlongpoll within itself *and* needed for LP to test txlongpoll?14:47
gary_posterI mean, test the use of txlongpoll?14:47
gary_posterWhat I'm getting at is this:14:47
bigjoolsgary_poster: txlongpollfixture is not needed to test txlongpoll; it's just a way of starting it for other tests and for "make run"14:47
gary_posterbigjools, got it.  So, actually, it never imports txlongpoll, right?14:48
bigjoolsnope14:48
bigjoolsjust starts the twistd daemon up14:48
gary_posterbigjools, but it has a Python egg dependency on txlongpoll even though it never imports it?14:48
bigjoolsyeah that does sound wrong :)14:49
bigjoolsI did it for convenience so that the egg gets built14:49
gary_posterbigjools, yeah, so I suggest breaking that dependency14:49
bigjoolsok14:49
bigjoolsalthough14:49
gary_posterbecause that's what we will need for other similar microfixtures, particularly those not written in Python14:50
gary_postermicroservices I mean14:50
bigjoolsthat will make it harder to make the txlongpollfixture tests to work14:50
gary_posterbigjools, maybe make it a test dependency?  That should be easy with or without buildout14:50
bigjoolswe need a general solution for PATH with microservices14:50
bigjoolsah yes, I can do a test dep14:51
gary_posterbigjools, you don't really mean PATH you mean bin, right?14:51
gary_posterI mean, PATH is one mechanism for communicating bin14:51
gary_posterthe location of the bin14:51
bigjoolsI mean a way for these services to get started without knowing where they live14:51
gary_posteryeah14:51
bigjoolsotherwise the local .dev will be a nightmare14:51
bigjoolswe just want to be able to type make run and have it start stuff14:52
gary_posterthat has to be configuration IMO, but we can certainly make that configuration easy some how14:52
bigjoolswe could put bin in the path14:52
gary_posterbigjools, yeah, sure, my point is simply that you are framing the problem in terms of your current solution, AFAICT.  no big deal.  I need to have my call :-)14:53
bigjoolsgary_poster: I guess I am, yeah14:54
bigjoolsenjoy, thanks for the help14:54
gary_posterwelcome bigjools14:54
deryckadeuring, taking a bit, sorry.  but I'm getting the results now.15:02
adeuringderyck: np15:02
sinzuijcsackett, I might loose power in a few minutes. The rain is torrential.15:07
jcsackettsinzui: i am sorry to hear that, do you want to chat now in case you can't later, and we'll just accept the risk of sudden disconnect?15:08
sinzuisure15:08
=== matsubara is now known as matsubara-lunch
=== beuno is now known as beuno-lunch
rsalvetibigjools: hey, any news when we'll be able to create the ubuntu-leb derived distro?16:08
rsalvetiat production16:08
bigjoolsrsalveti: you can do it now, I was holding off telling you because there's no Apache set up yet to make the archive available16:09
rsalvetibigjools: well, guess we can create it now and set up apache later16:10
bigjoolsthere's an RT for it16:10
rsalvetinot critical yet but I'd like to get it going16:10
adeuringbac: fancy a a review of an MP with a short diff? https://code.launchpad.net/~adeuring/launchpad/bug-739052-11/+merge/7677216:11
bacadeuring: surely16:12
bacadeuring: nice, r=bac16:30
=== matsubara-lunch is now known as matsubara
mrevellNight all16:51
nigelbgary_poster: YUI XHR tests sound neat :-)16:52
bigjoolshey bac, can I poke a review your way please?16:52
bacbigjools: shirley16:52
bigjoolsbac: https://code.launchpad.net/~julian-edwards/txlongpollfixture/FIRST/+merge/7677716:53
bacbigjools: i was about to grab a late lunch, though.  you in a hurry?16:53
bigjoolsit's a new standalong project16:53
bigjoolsbac: no it can wait, I will be eating dinner shortly16:53
bigjoolsbac: I will be back on later16:53
bacok16:53
bigjoolscheers16:53
gary_posterthanks nigelb :-)16:55
=== adeuring changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: bac | Critical bugs: 256 - 0:[#########]:256
=== beuno-lunch is now known as beuno
sinzuibac: do you have time to review https://code.launchpad.net/~sinzui/launchpad/valid-targets-1/+merge/7678118:20
bacsinzui: i do and will when i'm done with julian's18:20
* deryck goes offline for lunch18:40
bacsinzui: i accidentally approved your MP without comment.  i'm updating it now.18:50
sinzuicool, I saw the approve without a comment and thought that was odd18:51
bacsinzui: hah, and i mispelled 'typo'18:54
sinzuiI can tell you were thinking of me18:55
bac:)18:55
sinzuiI will fix the comment once I am certain I know what I meant.18:55
nigelbsinzui: Hey, do you have a few minutes? I'd like to hear your take on bug 528319:02
_mup_Bug #5283: "Home page" vs. "Description" is misleading <easy> <lp-registry> <tech-debt> <ui> <Launchpad itself:Triaged by nigelbabu> < https://launchpad.net/bugs/5283 >19:02
sinzuiI do19:03
sinzuiI think I did a report in the past to see what would happen is we merged fields19:03
nigelbsinzui: what was the eventual decision back then?19:08
sinzuinigelb, We really want to remove .homepage19:08
nigelbthe whole field?19:09
sinzuinigelb, for 2 years we have presented .homepage_content as the leading text to teamdescription19:10
* nigelb opens up psql19:10
sinzuiWe want to update the db to concatenate the two fields19:11
sinzuiWe want to rename teamdescription to description19:11
nigelbDb patch \m/19:11
sinzuiso there is one field name description in Person for both teams and users19:12
nigelbthat makes sense19:12
nigelbso, do you want me to talk to stub and lifeless to get started on a db patch for this.19:12
sinzuiThe fixing the edit forms and templates is easy, but I think timing is difficult19:12
nigelbI would first do the rename19:13
nigelband then land it and deploy19:13
sinzuinigelb, I think there are three db tasks and one code task  to ensure no interuption to users19:13
nigelbyeah19:13
sinzui1. create a description field in the schema19:13
sinzui2. garbo job to to fill it from homepage_content + \n + team description19:14
nigelbwhat's a garbo job?19:14
sinzui3. update the index pages and forms to use the new field (do not show the old fields if the description is not empty).19:15
sinzui4. remove the old columns when the garbo job is complete19:15
sinzui5. remove the old field from the models and templates19:15
sinzuinigelb, a garbo job is a db process used to clean up data while th production system is running19:16
nigelbah, I've read that somewhere19:16
sinzuinigelb, while we call it garbo, we often co-opt it to populate new data19:16
sinzuinigelb, there are more than 1,000,000 users and team. I expect the job needs a few days to complete19:17
nigelbokay19:17
nigelbI'll paste the list you just mentioned into the bug and start working :)19:18
sinzuiPlease do.19:18
nigelbCould give me a patch ID please?19:19
nigelbs/ID/number19:19
sinzuinigelb, only stub can give an id. When I write patches I end mine in a very high number19:20
sinzuinigelb, use patch-2208-97-0.sql for development19:21
nigelbThe process says any launchpad team member can19:21
nigelbI will, however, poke stub on Monday19:22
sinzuioh, that's right, let me check the wiki page to claim a number19:22
sinzuiand I should not have recommend -0. this will be an incremental for the fast rollout19:24
nigelbYeah :)19:24
lifelessnorming19:25
sinzuinigelb, i am getting the branch now. I will also get you real numbers for affected teams and person from staging afterwards19:25
nigelbcool!19:26
nigelblifeless: Good morning! I see you still need caffeine :-)19:26
lifelessI very rarely drink caffeine ;) - that was a deliberate switch19:28
sinzuinigelb, 2208-90-1 is your number to add the column19:28
nigelblifeless: ha19:28
muffinresearchHi, I seem to have trouble entering text into some of the edit boxes on merge proposals in chrome in the last couple of days. This is chrome 14.0.835.186. When it doesn't work I can't type anything only pasting text works.19:39
matsubarasinzui, https://bugs.launchpad.net/launchpad/+bug/857697 oops in the affiliation code for the person picker19:44
_mup_Bug #857697: AttributeError: 'NoneType' object has no attribute 'owner'  get affiliation in person picker <disclosure> <oops> <person-picker> <Launchpad itself:Triaged> < https://launchpad.net/bugs/857697 >19:44
matsubaraI just triggered the oops in the bug report so that one is not available yet on oops-tools. the other one I added as an example can be seen there19:45
sinzuithanks matsubara19:45
matsubaranp19:45
=== m4n1sh_ is now known as m4n1sh
cr3benji: hi there, might you have a moment for some lazr.restful assistance?20:56
benjicr3: sure, what's up?20:56
cr3benji: so, I'm getting this error when getting a collection of system objects: There isn't enough context to get URL information. This is probably due to a bug in setting up location information.20:57
cr3benji: the problem is with the processor attribute of each system object which probably doesn't have a url information by defining the __parent__ and __url_path__ attributes20:58
cr3benji: however, I don't know how to define the __parent__ attribute because a processor object doesn't know its system container. in fact, it's the other way around, system has a foreign key to the processor20:59
cr3benji: ideally, I'd like the url for a processor to be /system/<id>/processor, not sure how to pull it off though20:59
benjicr3: is this a Django app?21:00
cr3benji: yep21:02
benjicr3: if __parent__ isn't known statically, then you'll have to make it a property that looks it up21:04
cr3benji: so, I wonder if System should inherit from TraverseWithGet, do something to the request object, so that Processor.__parent__ can then know the context21:04
* benji looks up TraverseWithGet21:04
benjicr3: I don't see how that will help.21:05
cr3benji: I just tried it, it doesn't help indeed :)21:06
benji:)21:06
cr3benji: not sure how a processor can know it's system container at runtime though21:06
benjicr3: now, you could use something like TraverseWithGet that wraps the traversed object in a LocationProxy21:07
benjiLocationProxy is for objects that don't know their __parent__ or __name__, but someone else does and can wrap the object in a proxy that passes through all requests except for those two attributes21:07
benjiof course, you may need a DjangoLocationProxy because Django doesn't like __name__ as an attribute name21:08
cr3benji: I think I've rolled my own to look something like: class SystemProcessor: delegates(IProcessor, context="processor"); def __init__(self, system, processor)...21:09
cr3benji: however, whether I create something like a DjangoLocationProxy (good call about __name__, by the way!) or roll my own, I'm not sure how to apply that to the System class. For now, I have class System: processor = Reference(processor_id, Processor.id), ie I'm conveniently using Storm to reference the processor21:11
cr3benji: instead, I believe I'd have to change that to class System: def _getProcessor(self): processor = Store.of(self).get(Processor, self.processor_id); return SystemProcessor(self, processor); processor = property(_getProcessor)21:12
benjicr3: that's where you'd use something like TraverseWithGet, you have to hook some part of the traversal so that when you have both the system and processor "in hand" you can create your wrapper and tell it about it's __parent__ and __name__ (er, __url_path__)21:12
cr3... which seems like a lot of noise to do something apparently simple :(21:12
benjicr3: that would work too, and it is a bit noisy21:14
cr3benji: I'm not sure where I'd have both the system and processor "in hand" though, I'll try hooking here and there to figure it out and then fallback to making noise :)21:14
nigelbI have a confusion with a db patch.21:15
benjicr3: good luck21:15
cr3benji: thanks man!21:15
nigelbI'm comparing the diff of current.sql and newsampledata.sql. I see a new column transitively private :|21:15
nigelbIs that supposed to happen?21:16
=== bac changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: - | Critical bugs: 256 - 0:[#########]:256
bacnigelb: you'll see new items unrelated to your change if previous branches added things to the db but did not generate new sampledata21:19
bacnigelb: you can verify that it isn't your change but generating sampledata in a fresh branch from devel21:20
bacyou'll probably see the same additions21:20
nigelbah21:20
nigelbso, I should probably mention this in my MP for stub I guess.21:20
* nigelb looks for culprits21:21
nigelbbac: Thanks!21:21
bacnigelb: np.21:21
=== matsubara is now known as matsubara-afk
wallyworld_jcsackett: you still there?21:52
jcsackettfor some definition of "there". :-P what's up?21:52
wallyworld_jcsackett: just to let you know, i had a chat to huw about the managing disclosure mockups21:53
wallyworld_were are looking to use some static html with js hacked in and hosted on people.canonical.com21:53
jcsackettwallyworld_: sounds good to me.21:53
wallyworld_cool. i may on monday muck around with it a bit21:54
wallyworld_we can chat about it a bit more next week, just wanted to keep you in the loop21:54
jcsackettwallyworld_: is huw providing the static and we hack in the js, or is that part not yet figured?21:54
wallyworld_jcsackett: he is still ramping up on what we need to do - i ran him through the functionality etc. i was going to do the html myself initially21:55
jcsackettwallyworld_: dig.21:55
jcsackettand now, i must attend to starting dinner. thanks for keeping me up to date. :-)21:55
wallyworld_np. have a good weekend21:56

Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!