/srv/irclogs.ubuntu.com/2012/06/14/#launchpad-dev.txt

* StevenK stabs the lack of completion for bzr rm01:42
wgrantStevenK: How's the ec2 run looking?02:30
StevenKbranch-privacy-properties => devel        [OK]       (up for 2:33:55) i-2ca4375502:33
wgrantGood good02:38
* StevenK smacks wgrant.02:38
StevenKsteven@undermined:~/launchpad/lp-branches/devel% grep -c '^# ' lib/lp/bugs/model/bugtaskflat.py02:38
StevenK002:38
wgrantOops, indeed.02:39
StevenKwgrant: Do I need to add access_policy and access_grants to {I,}Branch, or are they internal only?02:39
StevenKI note BTF is missing access_policies and access_grants on its model.02:40
wgrantStevenK: We'll likely add them later, but you can't add them in this branch.02:40
StevenKwgrant: Sure, they have to wait until the DB patch hits prod and devel02:40
wgrantAnd then you'll add them if you need them.02:41
wgrantAnd not add them if you don't need them.02:41
wgrantSame as any other piece of code :)02:41
StevenKI was plotting writing tests for the DB patch, which made me ponder model code.02:41
wgrantlib/lp/bugs/tests/test_bugtaskflat_triggers.py may be of some interest.02:42
wgrantwallyworld: Why'd you delete that branch? It attracted the Scanner's Curse?02:43
wallyworldyes02:44
wgrant:(02:44
wgrantI still haven't been able to reproduce that locally.02:44
wallyworldit won't delete properly either it seems02:44
wgrantYeah, you can't delete lp:launchpad branches any more.02:44
wgrantNot once they've started to scan.02:44
wallyworldwell it lets me try02:44
wgrantSure.02:44
wallyworldand then errors in a funny way02:44
wgrantThen it 502s.02:44
wallyworldwhy?02:45
wallyworldit sholdn't let me delete02:45
wallyworldif i'm not supposed to02:45
wgrantIt should let you delete.02:45
wgrantYou are supposed to.02:45
wgrantIt just doesn't work.02:45
StevenKBecause BranchRevision exists02:45
wallyworldhooray!02:45
wgrantI'm not sure exactly what's going on, but I *believe* that the BranchRevision delete is unflushed at the end of the request.02:45
wgrantSo the commit() in the response post-processing flushes the 90krow delete outside the timeout.02:46
wgrantAnd crashes.02:46
wallyworldnot good02:46
wgrants/outside the timeout/outside the timeout exception handler/02:46
wgrantIt probably turns into a normal timeout if we flush explicitly after the delete.02:46
wgrantwallyworld: I'll review that branch after lunch, btw.02:52
wallyworldwgrant: no problem. whenever :-)02:52
StevenKWow, a smile from wallyworld.02:53
wallyworld>:-(02:53
mwhudsonisn't branchrevision deleted by an ON DELETE CASCADE?02:54
mwhudsonwhich is another way of saying the same thing i guess02:54
mwhudsonbtw buildout is very slow if you strace it :)02:55
StevenKs/ if you strace it//02:56
wgrantmwhudson: Ah, indeed.03:03
wgrantThat explains why it's unflushed.03:03
mwhudsoncould this be a pg9.1 regression?03:04
wgrantNo03:04
mwhudsonor has it been a problem for a while?03:04
wgrantIt's been a problem forever, it's just more prevalent now with the lower timeout.03:04
wgrantYou used to have to retry a couple of times to delete big branches.03:04
StevenKCan we bump the timeout for Branch:+delete?03:05
StevenKThe 'proper' fix is the death of destruction of most of BranchRevision03:05
mwhudsonyes03:06
wgrantwallyworld: 70+ if safe_hasattr(user_or_reference, 'id'):03:11
wgrantwallyworld: Can that be replaced by IPerson.providedBy(user_or_reference)?03:11
wallyworldno, cause it's not always a person, could be PersonRoles03:12
wgrantwallyworld: Perhaps try adapting it to IPersonRoles?03:13
wgrantThen you can just say IPersonRoles(user_or_reference).in_admin03:13
wgrantIf the adaption fails, it's not a person or an IPersonRoles.03:14
wallyworldi guess it will fail by returning None?03:14
wgrantwallyworld: By default it'll raise a TypeError03:15
wgrantBut the second arg specifies a default03:15
wgrantSo IPersonRoles(1, None) will return None03:15
wallyworldthat's the one i'll use, i don't like flow control by exception03:15
wgrantIt is good to avoid where possible, indeed.03:15
wgrantwallyworld: Looks like the branch fixes bug #1009360, bug #1009358 (maybe?), bug #1009281?03:25
_mup_Bug #1009360: RemoveBugSubscriptionsJob duplicates get_bug_privacy_filter <disclosure> <tech-debt> <Launchpad itself:Triaged> < https://launchpad.net/bugs/1009360 >03:25
_mup_Bug #1009358: Unsharing information from a team doesn't remove the members' bug subscriptions <disclosure> <Launchpad itself:Triaged by wallyworld> < https://launchpad.net/bugs/1009358 >03:25
_mup_Bug #1009281: SharingService.deletePillarSharee revokes all access to any involved artifacts <disclosure> <Launchpad itself:Triaged> < https://launchpad.net/bugs/1009281 >03:25
wgrant1009358 is linked already, nevermind03:25
wgrantThe other two aren't, though.03:25
wallyworldah, didn't realise thise were filed03:25
wallyworldbut yes03:25
wallyworldi'll link those to the branch03:25
wgrantGreat, thanks03:25
wallyworldwgrant: that should be fixed now03:41
wgrantwallyworld: Great. I'm experimenting with refactoring the admin specialcase.03:42
wgrantSince everywhere that needs to do it ends up mildly hideous.03:43
wallyworldwgrant: should that be done in a separate branch? i'd like to get this one (and the next) landed03:44
wgrant318+ getUtility(IRemoveBugSubscriptionsJobSource).create(03:53
wgrant319+ user, bugs=None, information_types=information_types)03:53
wgrantwallyworld: Doesn't that need a pillar restriction?03:53
wgrantOtherwise it's going to do a full consistency check over every bug of that information type.03:53
wallyworldyes, that's the intent at the moment03:54
wallyworldbut i could add a pillar check in03:54
wgrantWe can't run it on production like that, but I guess it's a reasonable intermediate.03:55
wallyworldthat bit was ported across from the deleted remove grantee job03:55
wgrantIn the long-term we probably want to restrict to bugs with the relevant AP and a subscriber in the team.03:55
wgrantThe join through BugSubscription + TeamParticipation is not completely cheap, but it's far cheaper than a full consistency check.03:55
wallyworldwgrant: i have to go pick up the kid, be back later. i'll add the pillar check before landing04:03
wgrantwallyworld: Thanks, I'm almost done.04:03
wallyworldok, no rush04:03
wgrantWould be done quicker if the celery job tests weren't hanging...04:03
wallyworldyeah, that happens sometimes to me too04:03
StevenKwgrant: branch-privacy-properties => devel        [OK]       (up for 4:05:35) i-2ca4375504:05
wgrantStevenK: Excellent. It's landing rather than testing?04:05
StevenKIt will land04:05
StevenKwgrant: r1541404:13
wgrantMarvellous.04:14
StevenKIt's about seven hours away from stable.04:16
wgrantWe're fixing that.04:16
StevenKWe are?04:16
wgrantI also filed RTs this morning to make qastaging updates faster.04:16
wgrantWell, the test suite will take half an hour soon :)04:16
StevenKwgrant: More than one RT?04:17
wgrantWell, one thing was only tangentially related.04:17
wgrantOh04:27
wgrantBah04:27
wgrantTest was failing because the job was crashing because of a typo, and the test didn't notice04:27
micahgBug #590523 doesn't seem to have been triaged to a specific type of failure on that page, if I have a new OOPS, should I just add it to the page or file another bug?05:10
_mup_Bug #590523: Archive:+delete-packages times out <boobytrap> <lp-soyuz> <timeout> <Launchpad itself:Triaged> < https://launchpad.net/bugs/590523 >05:10
micahgs/page/bug/05:10
wgrantmicahg: Add it to that page.05:12
micahgthanks05:12
StevenKwallyworld, wgrant: https://code.launchpad.net/~stevenk/launchpad/fix-copyrights/+merge/11022905:23
wgrantStevenK: lib/lp/app/__init__.py wants reverting05:23
wgrantIt's a file of monkeypatches without an import section, really.05:24
StevenKwgrant: Done.05:26
wgrantStevenK: lib/lp/services/database/locking.py's __all__ is in the wrong place.05:27
StevenKwgrant: Also fixed05:28
StevenKThe branch is now at +105:28
wgrantOtherwise it's good05:30
wgrantThanks05:30
wgrantmicahg: Hah05:35
wgrantmicahg: Copying 4 sources with a total of more than 250 binaries in one operation.05:36
wgrantmicahg: I am not surprised it timed out.05:36
micahgI'm special :)05:36
wgrantSo, the solution to the bug is to not do that.05:39
wgrantBecause it's not sensible :)05:39
micahgwgrant: umm, to copy 4 source packages to 4 different series in the same PPA shouldn't require 4 times through the page05:39
wgrantFor a sensible source package, sure.05:40
micahgmaybe it should just be done asynchronously05:40
wgrantYou can't make any assumptions about sources averaging 62 binaries each05:40
wgrantProbably, yes.05:40
StevenKPCJ?05:40
wgrantdie05:40
bigjoolsI have a plan for async ppa copies05:41
* micahg loves copyPackage05:41
bigjoolsjust never got around to implementing it05:41
wgrantmicahg: So, how can there possibly be more than 250 binaries?05:41
wgrantl10n?05:41
micahgwgrant: yes, the l10n packages were made arch:any because of skew issues05:41
wgrantblink05:42
wgrantWhen was this?05:42
StevenKmicahg: !!!05:42
micahgor was there another reason for that...05:42
wgrantRecall that we now keep arch-all published until all the arch-any are gone.05:42
micahgStevenK: I didn't do it :)05:42
wgrantBut making them all arch-any is just completely ridiculous and not justifiable :)05:42
micahgwgrant: http://bazaar.launchpad.net/~mozillateam/thunderbird/thunderbird.lucid/view/head:/debian/changelog#L12305:45
micahgit's a waste of mirror space and bandwidth as well05:46
wgrantEw05:46
lifelesswgrant: ok, so, hot patches05:47
wgrantOh right.05:47
wgrantlifeless: Indeed05:47
lifelessthey have the following properties:05:48
lifeless - they may break running code05:48
wgrantmicahg: Amusingly, the copy actually succeeded a few milliseconds before the timeout, but generating the notification timed out.05:48
lifeless - they will apply as regular dbpatches05:48
wgrantI think it might be loading too many objects for the cache to survive.05:48
lifeless - but we can apply them by-hand superfast.05:48
wgrantlifeless: Right.05:48
lifelessso, I'm not sure why we would land them on devel.05:49
wgrantlifeless: Though generally they're just index additions, so cannot break running code (unless postgres misplans and performance suffers)05:49
lifelessI can't find any reference to landing them on devel in the wiki page history05:49
lifelessthough I'm not quite at rev 005:50
micahgwgrant: interesting05:50
wgrantI quote:05:50
wgrant10:51 < wgrant> lifeless: What's the process for live index creation these days?05:51
wgrant10:51 < wgrant> Apply before or after landing?05:51
wgrant10:51 < wgrant> It should take about 60ms to create :)05:51
wgrant10:52 < lifeless> wgrant: land on qastaging, add and QA there, then prod.05:51
wgrant10:53 < wgrant> land on devel, add and QA on qastaging, then prod?05:51
wgrant10:53 < lifeless> yes05:51
lifelesswgrant: so clearly I'm inconsistent05:51
lifelesshmm05:51
lifelessstub went back to sleep05:51
wgrantLanding on db-devel provides no benefit.05:51
wgrantAnd it's extremely awkward to deal with, because db-stable can be undeployed for weeks.05:52
wgrantBecause of the one-patch-per-downtime.05:52
wgrantSo we'd end up with db-stable being deployed out of sequence.05:52
wgrantFor no benefit whatsoever.05:52
lifelesswell, thats a very absolute statement.05:53
wgrantIt's also absolutely correct :)05:53
* lifeless wonders if disproving it is worthwhile or a distraction05:53
lifelessELOCAL, bbiaw05:53
wgrantk05:53
lifelesswgrant: so what we originally wrote was to have the hot patch qaed by application to qastaging06:26
wgrantlifeless: Right, that makes sense.06:30
wgrantlifeless: In practice we often go live directly to production with indices, because there's no meaningful QA until the code is there.06:31
wgrant(though they usually have been tested with real queries on one of the three stagings first)06:32
StevenKqastaging, staging, and the staging that is spelt with no s, t, a, i, n, one less g, and with two d's, three o's, and an f?06:33
wgrantThose, indeed.06:34
wgrantlifeless: So, I don't see that there's anything achieved by landing on db-devel.06:35
wgrantWe shouldn't do something just because the policy that's never been executed in full says we should do it.06:36
wgrantThere has to actually be a reason :)06:37
wgrantwallyworld: 204+ return [06:39
wgrant205+ enumerated_type_registry[InformationType.name].items[value]06:39
wgrant206+ for value in self.metadata['information_types']]06:39
wgrantwallyworld: Isn't that just this:06:39
wgrantreturn [InformationType.items[value] for value in self.metadata['information_types']]?06:39
wallyworldnot sure, i can try it06:40
wallyworldi wrote that code ages ago06:40
wgrantIIRC the registry was created for de-JSONing things.06:40
wgrantBut you have the actual enum here06:40
wgrantSo you don't need the registry.06:40
wallyworldno i don't :-)06:40
wallyworldthe job state is saved06:40
wallyworldand then read later from json06:40
wgrantHmmm?06:41
wgrantYou do!06:41
wgrantYou get the name of the enum from the enum06:41
wgrantAnd then use it to look up the enum06:41
wgrantin the enum registry06:41
wallyworldi'll try it06:42
wallyworldi may have copied the code from when i didn't have the enum name06:42
wgrantThat sounds most plausible.06:42
wgrant# The enumerated_type_registry is a mapping of all enumerated types to the06:42
wgrant# actual class.06:42
wgrantSo it indeed just returns the class06:43
wgrantSo enumerated_type_registry[InformationType.name] is just InformationType06:43
wallyworldsounds about right, i have to go and re-read the source06:43
wgrantIt's also tempting to ignore Guido and replace the overlong list comprehension with "map(attrgetter('value'), information_types or [])"06:45
wallyworlddoes he not like map?06:46
wgrantHe tried to remove it from Python 306:46
wallyworldwhy?06:46
wgranthttp://www.artima.com/weblogs/viewpost.jsp?thread=9819606:47
wgrantBasically, Guido doesn't like functional-style code :)06:49
wallyworldwgrant: seems like an opinion on style rather than anything of substance at first glance06:49
wgrantSure06:49
wgrantBut he actually tried to remove them from the language :(06:49
wallyworldthat's one of the good things about pythin, that it's multiparidigm06:50
wgrantPrecisely.06:50
wgrant(I think listcomps might not be so bad if we didn't have a policy against using single-letter variable names)06:51
lifelesswgrant: I agree, I'm trying to reconstruct our reasoning06:57
lifelesswgrant: we failed to write some of it down06:57
wgrantlifeless: Perhaps stub remembers?07:02
wgrantI don't remember much of it from IRC, so it was probably mostly between you two07:02
lifelessondeed07:02
lifelesshe is asleep just now07:02
wgrant:(07:03
lifelessI think there are at least minor downsides to going straight to devel, but also less duplicate work merging stuff up etc07:03
lifelessone of the undone longer term things is making hot patch application be -done- (in make schema etc) using the same sql we use to apply it live.07:04
lifelessto avoid manual handling07:04
wgrantwallyworld, StevenK: Looks like the in-progress branch denorm will make lp:launchpad branch search about 12x faster.07:04
wgrantYay07:04
wallyworldgreat07:04
wgrantlifeless: Right, it'd be nice to automate that.07:04
wgrantlifeless: It is infeasible at present due to slony.07:04
lifelessits totally doable07:05
lifelessjust not done07:05
wgrantWell.07:05
wgrantAwkward.07:05
lifelessand not worth doing now.07:05
wgrantAnd probably not worth it given slony's demise07:05
wgrantRight07:05
wgrantThe remaining big issue is CONCURRENTLY. Not sure how best to work around that.07:05
=== almaisan-away is now known as al-maisan
adeuringgood morning07:43
wallyworldwgrant: fancy a quick +1 on a +16/-471 mp (deletes RemoveGranteeJob). then i can chuck at ec2 https://code.launchpad.net/~wallyworld/launchpad/delete-RemoveGranteeSubscriptionsJob/+merge/11025608:04
wgrantwallyworld: Looking08:05
wallyworldthanks08:05
wallyworldthe additonal lines are just in a generic base class test since the test job has been replaced08:06
wgrantwallyworld: Well that was challenging.08:07
wgrantr=me08:07
wgrantAlso just looked over your amendments to the one I reviewed earlier. That looks good too.08:07
wgrantThanks for applying my suggestions.08:07
wallyworldnp, thanks for the reviews08:07
wallyworldwgrant: so, i think if i add the filter for grantee when leaving a team the job should be performant08:08
wgrantThe pillar filtering isn't perfect, but in another iteration or two it should be good.08:08
wallyworldwgrant: you think it should be on AP?08:08
wgrantwallyworld: That's probably best. eg. now it will miss series bugs.08:09
wgrantAP is probably more direct.08:09
wgrantAnd accurate.08:09
wgrantWe'll need grantee filtering for normal removals too08:09
wgranteg. if I revoke someone's access to Ubuntu userdata.08:09
wgrantThat's a couple of hundred thousand bugs.08:09
wallyworldyes, makes sense08:10
wgrantWe care about the bugs that have the relevant AP and have a subscriber that is a participant in the grantee.08:10
wallyworldthe grantee will be removed first though08:10
wallyworldbefore running the job08:10
wallyworldbut that shouldn't matter i don't think08:11
wgrantRight.08:11
wgrantThat certainly makes things look a bit nicer.08:11
wallyworldfor team membership removal, the filtering will be a bit different08:11
wgrantYeah08:11
wgrantThat has no AP filter.08:11
wallyworldsince the team itself still has access08:12
wgrantIt's just any private bug with a participating subscriber.08:12
wgrantRight, the team isn't relevant to the job.08:12
wgrantThe removed member is.08:12
wgrantAny subscriber participating in the former member is affected.08:12
wallyworldthe team could be used to narrow the search08:12
wgrantHmm.08:12
wgrantIndeed.08:12
wgrantI'm not sure if that's worth it, but it is indeed a good thought.08:13
wgrantIn general the set of subscriptions should be very small.08:13
wallyworldso we need to record in the metadata (former) grantee and optionally team08:13
wgrantWell08:13
wgrantThe job only really cares about a set of criteria to identify the bugs to reconcile.08:14
wgrantI don't think it needs to know about grantees directly.08:14
wgrantWe just need to say "please reconcile subscriptions for participants of ~launchpad in Ubuntu's user data bugs"08:14
wgrantThat could mean that ~launchpad had a grant revoked.08:14
wgrantOr it could mean that ~launchpad was removed from a team.08:15
wallyworldsure, but i'm more specifically talking about removing a team member08:15
wgrantEr, in the team removal case there wouldn't be the AP restriction, but yes.08:15
wgrantwallyworld: That's the same.08:15
wallyworldso we only care about bugs the former member has access to via the team08:15
wgrantRight.08:15
wallyworldhad08:15
wallyworldanyways, i'll code it up and see what falls out08:16
wgrantIn most cases it won't be worth filtering by "had access to via the team"08:16
wgrantIt will be more efficient just to consider all subscriptions.08:16
wgrantBut it is something to consider for bad cases, and it may not be too bad to do it all the time.08:16
wallyworldok, i wasn't going to assume that, but i'll defer to you for that08:16
wallyworldi think if the cost of doing it is small, we may as well08:16
wgrantwallyworld: Subscriptions are going to become pretty rare, I think.08:17
wgrantStructural subscriptions handle widespread notification.08:17
wgrantAPGs handle widespread access.08:17
wgrantThere's no more use case for widespread subscriptions :)08:17
wallyworldbut the subquery to get the subscriptions08:17
StevenKI like the idea that subscriptions are going to die.08:18
wallyworldwould benfit from the extra filtering08:18
wgrantwallyworld: So08:18
wgrantMaybe.08:18
wgrantBut, for example, I'm removing a user with 5 subscriptions from a team with access to Ubuntu's user data bugs.08:18
wgrantIt's clearly more efficient there to just look at the subscribed bugs.08:18
wgrantThe converse case can also be argued, of course.08:19
wgrantBut huge numbers of subscriptions shouldn't be common at all.08:19
wgrantHuge access grants will be.08:19
wallyworldthe structure of the job though is to select bugsubscription where bug in (...)08:20
wallyworldand it's the subsquery that needs to be fast08:20
StevenKWe still need to transition to APGs/APAs.08:20
StevenKThat is going to be ... fun.08:20
wallyworldyep :-)08:20
lifelessalso subscriptions will be a separate service in future too.08:31
StevenKSo you keep saying08:31
lifelessI do08:31
lifelesshow is auditor ?08:31
StevenKI've not touched it this week, due to Monday and I'm still stuck having the fixture being able to run manage.py or some other form of it.08:32
wgrantStevenK: Huh? What transition?08:35
wgrantStevenK: We're already using them internally for all bug access checks.08:35
wgrantIt's just the UI that's not rolled out yet.08:35
wgrantAnd it mostly works.08:35
wgrantThese are the final pieces coming together.08:35
StevenKPerhaps I am misrecalling the Marvellous, Foolproof and Excellent Disclosure Plan.08:36
StevenKwgrant: However, branch access is another matter. But, that is progressing, slowly.08:38
wgrantNot particularly slowly.08:39
wgrantWe just have to wait for the bug stuff to mature and then refactor it to do branches too.08:39
wgrantBranches are a faaaar simpler problem.08:39
wgrantOnce you discard stacking.08:39
StevenKI was about to say "*cough* stacking" :-)08:39
lifeless'discard'08:40
jmlheh heh08:40
StevenKwgrant: I say slowly, because I have a DB patch that drags it forward and I can't land it for a week.08:41
StevenKAnd when it deploys is another matter entirely.08:42
wgrantStevenK: You can probably land the DB patch on Monday night.08:42
wgrantIt can be deployed on Tuesday.08:42
StevenKwgrant: That plan requires that the model code is dropped in a deployment on Monday.08:44
StevenKBut okay.08:44
wgrantStevenK: That's the plan.08:44
wgrantAlthough if we push it we can drop the model code tomorrow.08:44
wgrantLet's push it :)08:44
StevenKI haven't written the branch yet.08:45
wgrantIt's about -1008:45
wgrantWe can deploy today's branch in about 3 hours08:45
wgrantLand the model removal08:45
wgrantDeploy model removal tomorrow08:45
wgrantLand DB patch08:45
wgrantDeploy DB patch Monday night :)08:45
StevenKThere are at least seven branches that need QA before we can deploy in three hours.08:47
wgrantThe ones with non-trivial QA are ∅08:48
wgrantWell, r15407 and r15410 require a small amount of thought.08:49
StevenKwgrant: So, model droppage now, or tomorrow morning?08:49
wgrantStevenK: I'd prepare the branch now, since it's trivial.08:50
wgrantIt shouldn't be landed until today's is deployed.08:50
wgrantI think you can just delete any statement mentioning _transitively_private or _explicitly_private.08:51
StevenKOf which there are two ...08:51
wgrant3 lines mention explicitly, 8 transitively08:52
wgrantBut close enough08:52
wgrantSo it's going to be like -1308:52
wgrantNot -10 as I said :(08:52
StevenKwgrant: I'm at -18/+0 already09:00
jmlhttps://bugs.launchpad.net/launchpad/+bug/101305609:00
_mup_Bug #1013056: Creating a PPA requires one to be a team admin <ppa> <Launchpad itself:New> < https://launchpad.net/bugs/1013056 >09:00
wgrantjml: Argh09:01
wgrantMake it go away09:01
jmlI'm trying.09:01
wgrantThat one comes up occasionally09:01
wgrantI don't have a good answer.09:01
czajkowskiI was thinking something similar09:01
jmlI filed it.09:01
wgrantI know you filed it.09:01
jmlHere are some answers I can think of. I won't claim they are good.09:01
wgrantThe issue comes up in discussion occasionally.09:01
jmlmake it launchpad.Append09:01
jmlmake a new object, say, ArchiveCollection, and hook permissions on to that.09:02
bigjoolsappend on what?09:02
wgrantSo, allowing it technically is easy.09:02
wgrantSocially it's less obvious whether it's sensible.09:02
jmlbigjools: IPerson09:02
bigjoolsah ok09:02
wgrantjml: I'm a member of ~ubuntu-bugcontrol09:02
wgrantShould I be able to create a PPA?09:02
wgrantNo.09:02
wgrantubuntu-bugcontrol isn't a PPA team.09:02
jmlwgrant: right. similar for mailing list teams.09:02
StevenKwgrant: http://pastebin.ubuntu.com/1040482/09:03
jmland the last thing Launchpad needs is another knob.09:03
StevenKjml: No, we have enough of those working on it already. :-P09:03
bigjoolsbadumtish09:03
wgrantjml: Precisely.09:03
wgrantjml: So I would just make your robot a team admin.09:03
wgrantAnd ignore the question.09:03
jmlwgrant: we have a contractor too.09:03
bigjoolscan we apply a Knob-count approach like LOC? :)09:03
wgrantjml: Can't SCA have an API to do it or something?09:04
wgrantThere's no sane way to do it in LP without adding a knob.09:05
jmlwgrant: we can probably get by, but I'd rather leave the bug open for the moment. I don't like bots with team admin privs either.09:05
wgrantSure.09:05
wgrantWe need something more flexible.09:05
wgrantBut we have no way to do that without making everything suck right now :/09:05
jmlwgrant: by 'have an API', you mean write a front-end and secure that?09:05
wgrantjml: Maybe.09:06
wgrantjml: I mean, SCA already accepts API requests from webapps to manage subscriptions.09:06
wgrantAnd those webapps have admin UIs09:06
wgrantThat could be used to ask SCA to perform admin operations like setting up a PPA for a new app.09:06
jmlwgrant: yeah, we could possibly do that. We weren't planning on having SCA create PPAs, but that might work.09:07
wgrantStevenK: Ah, comments, I see. That's cheating.09:07
StevenKHahaha09:07
StevenKwgrant: Do you think it's reasonable?09:07
wgrantStevenK: It does what we want, it works, it doesn't fail tests, it doesn't add ugly code.09:09
wgrantI don't see how it could not be reasonable.09:09
wgrantStevenK: Deployment report is green and the two remaining revs before yours are 30s QA.09:12
StevenKwgrant: I think this branch model may also have to kill the transitively_private triggers tests. I'm not certain.09:14
wgrantOh, indeed.09:14
wgrantbzr rm lib/lp/code/model/tests/test_branch_privacy_triggers.py09:15
cjwatsonwgrant: map/listcomp> the tradeoff has changed a bit in Python 3 though - map returns an iterator, so if you need a concrete list then the map approach is an extra six characters versus Python 209:15
wgrantcjwatson: True.09:15
wgrantStevenK: Actually, it shouldn't matter. The trigger tests don't use the model.09:15
wgrantStevenK: It's all string-based SQL09:15
wgrantStevenK: But might as well delete them anyway.09:16
cjwatson15412 is a bit more than 30s QA since it's gone wrong before, but hopefully not overly much.09:17
wgrantMeh, it's only worth two API calls.09:18
cjwatsonActually, I could do that on dogfood now couldn't I09:19
wgrantYou could indeed.09:20
wgrantIt won't be on qastaging for a couple more hours.09:20
StevenKwgrant: https://code.launchpad.net/~stevenk/launchpad/destroy-branch-privacy/+merge/11027409:24
wgrantStevenK: r=me, ec2test time.09:26
StevenKwgrant: Tossed at ec2 test.09:27
wgrantStevenK: Great.09:29
StevenKIt will be great if it doesn't fail any tests. Not before.09:31
wgrantMeh09:31
bigjoolsjml: I saw a car today with a sticker in the back window that said "Beached as!"09:32
bigjoolsand I thought of you09:32
jmlbigjools: :)09:33
al-maisanhmm .. can someone please take a look at this build : https://launchpad.net/~openquake/+archive/testing/+build/357513409:58
al-maisanthe upload of the resulting binaries seems stuck09:58
wgrantal-maisan: Cronjobs are disabled for a DB upgrade in a few seconds.10:00
wgrantShould be processed in a few minutes.10:00
al-maisanah, I see, thanks!10:00
wgrantal-maisan: The webapp downtime is down to 70s, but we're still disabling cronjobs about 15min before for safety.10:01
al-maisanno problem at all -- at least I know what's going on now :)10:02
wgrantal-maisan: That uploaded a few minutes ago.10:11
al-maisanthank you William!10:11
jmlooh, a db upgrade10:13
jmlwill this include my patch?10:13
wgrantjml: This was stub's index work. Yours is nearly 24 hours away.10:13
jmlwgrant: ta10:14
=== matsubara is now known as matsubara-lunch
=== al-maisan is now known as almaisan-away
=== mrevell_ is now known as mrevell
wgrantcjwatson: Did you get around to doing that QA on DF?11:19
rick_hStevenK: you get +10 internet points for your email signature I noticed today11:22
cjwatsonwgrant: Yes; I've been waiting for qastaging to update.11:25
wgrantIt'll be there in about 30s.11:25
cjwatson(So I can mark it qa-ok.)11:25
cjwatsonCool.11:25
wgrantAh right11:25
wgrantThat'll be another 5 minutes.11:25
wgrantThanks.11:25
wgrantThere we go, right on time.11:26
cjwatsonwgrant: Bug 648611: any particular reason you feel it necessary to include upload status granularity in that?11:29
wgrantcjwatson: I don't remember who that was discussed with. ScottK and others. But if it's no longer an issue for you, then there's no need for it.11:30
jmlwgrant: do you know if the codehosting puller is still in use?11:33
cjwatsonMaybe I should talk with ScottK then.11:34
wgrantjml: Mirrors and imports still use it. Mirrors can't be created any more, but there are still some old ones in use.11:34
cjwatsonIt'd be easier to just make it per-pocket, since that's almost exactly analogous to per-pocket upload.11:34
jmlwgrant: ah ok. I'm surprised imports use it.11:34
jmloh wait11:34
wgrantjml: Same as ever.11:35
jmlto pull it from internal location where branch is11:35
wgrantimportds push their branches up to escudero11:35
wgrantLP pulls from escudero11:35
wgrantBecause who cares about sanity :)11:35
jmlwgrant: I thought I recalled some discussion about using the codeimport system to do pulling11:35
jmlddaa had a reason for that, I'm sure.11:36
jmlhuh11:36
jmlI wonder. Could / should the ScriptActivity table have a place in the web ui, exposed to LP developers & admins.11:36
StevenKrick_h: Haha11:37
wgrantjml: s/table/service/11:37
jmlwgrant: even better.11:38
StevenKwgrant: Are you going to be awesome and do my QA, since dinner for Sarah and I is about 3 minutes away from serving?11:38
wgrantStevenK: Mostly done already11:38
jmlhmm. hmm. hmm.11:38
StevenKwgrant: ec2 has another ~ 110 minutes, but it should be good to lp-land when it finishes.11:39
wgrantjml: Hm hm?11:39
jmlwgrant: I was thinking about starting work to pull codehosting out of LP11:39
jmlwgrant: and then about how I wonder how much work it would be to make & deploy a scriptactivity service as we just described11:39
cjwatsonOh, hey, maybe I lied when I said to technical-board@ that my permissions branch would probably land tomorrow or failing that Monday.11:40
jmlwgrant: and then about how I don't know whether anything *has* been pulled out of LP into a separate service successfully11:40
wgrantcjwatson: Pocket permissions? With a bit of luck it'll be deployed 40 minutes after gnuoy returns from lunch.11:40
wgrantjml: It depends on how much of codehosting you want to pull out of LP.11:41
cjwatsonYeah, I missed your chatter about pushing it.11:41
wgrantHow much of codehosting do you want to pull out of LP?11:41
jmlwgrant: and how if I started I'd be afraid that the work would spiral out of control with lifeless, yourself and others telling me that I'm doing it wrong or adding new requirements or elucidating requirements that I'm not fully conscious of yet.11:41
jmlwgrant: I think the ssh server / vfs bit makes sense.11:41
jmlwgrant: maybe the apache rewrite support too. I'm not sure.11:42
wgrantNothing like this has been extracted before.11:42
jmlwgrant: an obvious pre-req would be extracting lp.services.ssh into a library.11:42
wgrantI was going to do poppy to start. But I ran into trouble with naming the libraries.11:42
wgrantRight.11:42
wgrantThat's pretty easy to do.11:43
jmland I don't know how I'd answer questions like what to use for config.11:43
wgrantAnd it's probably most of the integration points.11:43
jmlwgrant: poppy would also be a good place to start.11:44
wgrantI'd probably just use a separate minimal lazr.config-based thing. But configglue or something might be easier and better.11:44
wgrantjml: I'd suggest starting with poppy, as it uses the sshserver and is much simpler in every other respect.11:44
jmlwgrant: it seems to me that doing one small thing, and (crucially) writing down the steps taken would be a giant leap for Launchpad.11:44
wgrantMost definitely.11:44
wgrantpoppy is probably the smallest, easiest piece to rip out like that.11:45
wgrantIt's extremely isolated.11:45
jmlYeah. The list for me goes: poppy, codehosting ssh, maybe codeimport, builddmaster.11:45
jmlAlthough you mentioned scriptactivity, which probably goes somewhere toward the right.11:46
wgrantscriptactivity is sort of the opposite.11:47
wgrantIt's an internal service used by everything, but it's very isolated and an excellent example of something that should be a microservice.11:48
wgrantscriptactivity is probably a prereq for getting rid of other things.11:49
wgrantSince they'll want a way to record activity.11:49
jamanyone have much experience with lxc? I'm running low on space on '/' so I'm trying to put the rootfs on an alternate drive. But once I do 'lxc-start' just returns without starting anything, or giving me anything informative.11:51
jamI've tried playing with the config file, to point the root to another location11:51
jammaybe if you know of an error log or something like that?11:52
wgrantjam: Tried 'lxc-start -n foo -o /tmp/lxclog -l DEBUG'?11:53
jmlwgrant: yeah. it's realizing that it's a pre-req that got me 'hmming' in the first place11:53
wgrantYeah11:54
jmlSpecs fall apart, the scope it cannot hold; more work items are loosed upon the world11:54
jmlwgrant: anyway, I'll keep it in mind as a possible skunkworks project11:55
wgrantSounds good.11:58
wgrantSo, poppy and scriptactivity, in that order, would be my recommendation for anyone wanting to do this sort of thing. poppy because it's simple, and scriptactivity because it's simple and a core pre-req from the other end.11:59
jmlwgrant: doesn't poppy need to use scriptactivity11:59
* jml just remembered something.11:59
wgrantjml: It's a daemon.11:59
wgrantNot a cronjob.11:59
jmlah, ok.11:59
jamwgrant: it appears /sbin/init was just dying, but that appears to be because I had that device mounted with "nosuid,nodev"12:01
jamit seems to work now12:01
wgrantjam: That would be unhelpful, indeed.12:01
wgrantIt was GNOME-mounted removable media?12:01
jamwgrant: I added it to fstab, using the params done by GNOME disk-utility12:02
cjwatsonudisks mounts everything with nosuid,nodev.12:02
jamI couldn't find a Gnome utility that would let me say "always mount this disk"12:02
wgrantAh yes, udisks, I had forgotten what was underneath today.12:03
cjwatsonudisks2 in quantal, because it would be a shame to leave anything the same.12:05
jmlwgrant: you don't appear to be subscribed to DisklessArchives. I made an edit, and james_w added a comment the other day for which we'd welcome your feedback. (Can wait until tomorrow though)12:06
wgrantjml: I couldn't find a subscribe button, and /UserPreferences is like all the way over there.12:06
jmlwgrant: URL hack! ?action=subscribe12:06
wgrantAh, didn't think that would still work.12:07
wgrantSurely new moin has fixed mutating GETd12:07
wgrants12:07
wgrantEvidently not.12:07
wgrantThanks.12:07
jmlwgrant: np.12:08
wgrantjml: The API service will live in the LP tree.12:08
wgrantThe squid bits will not.12:08
wgrantIdeally the API service would be part of the main Zope stack, but the main Zope stack is so terrible that it's not feasible at present, so we have to hack.12:09
jmlwgrant: that makes sense. it's a shame from a test cycle pov.12:10
wgrantGiving the testing hostname to real clients is an interesting idea. I'm not sure if it's a good one.12:10
wgrantjml: Certainly.12:11
lifelesswgrant: we an use the testing hostname as the new, browser-safe permanent hostname12:11
lifelesswgrant: and initiate a very slow migration.12:11
wgrantTrue.12:11
jmlwgrant: however, once we have one non-zope api server in LP to facilitate fast requests, why not move the existing api services to that same technology12:11
lifelesssee #ca-internal for a chat between me and james_w about this12:11
wgrantjml: That's the plan.12:11
lifelesswgrant: btw, LP's stack *can* answer in 3ms.12:11
wgrantjml: Whether the plan will happen this decade is unclear :)12:11
wgrantlifeless: hahahah12:12
jmlI hope not using zope stack means that at least the unit tests will be fast12:12
wgrant+opstats maybe12:12
lifelesswgrant: definitely, see the haproxy stats page.12:12
lifelesswgrant: I noting that the publication stack through to render and return is likely fast enough for our needs; there is definitely potential issues below that.12:13
lifelesswgrant: I'm inclined to suggest trying in the main stack in the first instance; it means we know fairly likely places for the rot to be located.12:13
wgrantRight, for a no-op publication to a no-op view that has hardcoded hacks to avoid most of the stack, it can be fast.12:13
jmlalas for my TDD cycle time!12:14
lifelessjml: the TDD cycle time for the horizontally scaling nodes should be fine.12:14
lifelessjml: it will need a test harness, but that by definition has to be fast to start and use.12:15
wgrantThat's like 3% of the effort.12:15
jmlbtw, at some point before we actually start work it'd be good if you two, james_w and I could have a call to walk through the implementation plan. I realize that TZ wise it's a huge pita.12:16
wgrantlifeless: Are you sure you didn't hallucinate the conversation with james_w?12:16
jmlalso, it looks more like 6-8 weeks than 4-6.12:16
wgrantI don't see it.12:16
wgrantjml: It mostly depends on how far you want to rewrite the backend publication stack.12:17
wgrantlifeless wants you to rewrite them a lot.12:17
wgrantI don't.12:17
wgrantlifeless: Which day was it?12:18
lifelesswgrant: today12:18
lifelessmay have been here. Lets see12:18
lifelesswgrant: u1-internal12:20
wgrantOh, sneaky.12:20
wgrantI don't think I'm in there.12:20
lifelessvery very so12:20
* wgrant grabs logs.12:20
=== almaisan-away is now known as al-maisan
=== _mup__ is now known as _mup_
stubjml: Yes, I think scriptactivity should definitely be exposed. I always imagined a page of green/red/yellow blobs for each component we can track.12:24
stubjml: I think the trick is to start simple, as bikeshedding and feature creep turn it into a big problem very quickly12:25
jmlstub: agreed12:25
jmlstub: the main thing I'm interested in is knowing which cronjobs are still being run and with what options so I can delete more code.12:25
jmlalthough I don't think SA tracks options12:25
wgrantjml: bzr branch lp:lp-production-crontabs12:26
jmlmirable dictu!12:26
lifelessdeleting scriptactivity itself would be a start12:28
=== matsubara-lunch is now known as matsubara
wgrantcjwatson: Perhaps we should revert your revision quickly, just to prevent the rest of UE from getting the idea that LP deadlines don't always slip.12:31
cjwatsonwgrant: haha12:38
wgrantAlso, we're not really in a hurry for something else, I just wanted to stop StevenK complaining that things are too slow.12:39
StevenKwgrant: You know, scriptactivity could probably be replaced by auditor ...12:43
wgrantPotentially.12:45
stuband the bikeshedding and feature creep starts...12:47
lifelessjml: pretotyping12:47
StevenKstub: For auditor? I don't think the feature could creep much :-)12:48
wgrantStevenK: That just means it's already crept.12:48
StevenKwgrant: But it hasn't?12:48
StevenKLog events via POST, grab them via GET. Done.12:49
StevenKIt might need some squinting in terms of actor/object acted on, but shrug.12:49
StevenKwgrant: It was good natured complaining about things going slowly. :-)12:50
wgrantSure.12:50
wgrantBut we might as well make things faster where we can.12:50
stubStevenK: It started as exposing some information in the DB, now you are posting information into it :)12:51
StevenKstub: No, it was always the plan that we had to post information into it, so nyah.12:52
wgrantjml: How soon are you likely to want that call?12:58
jmlwgrant: before we start implementing, after you guys think it's ready for us to start implementing.13:00
jmlwgrant: modulo some scheduling stuff that I need to talk to james_w and mvo about.13:00
wgrantRight.13:00
ivoryjelmer: could you review this please? https://code.launchpad.net/~ivo-kracht/launchpad/bug-999662/+merge/11032213:08
jelmerivory: sure, I'll add it to my queue13:11
ivoryjelmer: thanks13:11
flacostegary_poster: i really liked the format of your latest parallel tests update13:11
flacostegary_poster: it read very well!13:11
gary_posterflacoste, thanks!  I'll go review it to see what I did differently. ;-)13:12
mgzivory: skim-by, is that change wrap the <a> in xx-distirbution-changes.txt valid? I'd think id'd break a doctest13:12
wgrantivory: Did you consider using fmt:approximatedate? It handles timestamps within the last 24 hours a little more useful, by saying eg. "2 hours ago"13:12
flacostegary_poster: i think the summary rocked, or maybe is that the news are particularly good this week ;-)13:13
wgrantFor more than a day it just uses the date.13:13
StevenK"a moment ago" \o/13:13
gary_posterflacoste, lol, cool, I'll take either one :-)13:13
wgrantmgz: Doctests default to normalising whitespace.13:14
wgrantmgz: So you can normally wrap however you want.13:14
ivorywgrant: i did but somehow it didn't work, maybe i did something wrong but then abel proposed to do it like that ...13:14
cjwatsonNot that you'd know it by looking at lots of our existing doctests.13:14
wgrantcjwatson: Shhh13:14
mgzwgrant: they don't generally, but does launchpad set that flag?13:15
wgrantmgz: I meant our doctests, yeah.13:15
mgzthanks, good to know.13:15
wgrantdefault_optionflags = (doctest.REPORT_NDIFF | doctest.NORMALIZE_WHITESPACE | doctest.ELLIPSIS)13:15
StevenKSome of our unit tests also make use of doctest.NORMALIZE_WHITESPACE, which I find hideous.13:15
cjwatsondoctest.ELLIPSIS is handy when converting doc to unit though.13:17
ivorymgz: i don't really understand what you mean.i didn't change xx-distribution-changes.txt13:17
mgzivory: *-packages.txt, but wgrant settled that question13:19
=== jcsackett changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | Welcome our new intern: ivory | On call reviewer: jcsackett | Firefighting: - | Critical bugs: 3.47*10^2
wgrantcjwatson: It is done.13:34
cjwatsonNeat.13:42
StevenKwgrant: ec2 is done.13:43
=== al-maisan is now known as almaisan-away
wgrantStevenK: Might as well land it. The deploy isn't completely done yet, but the appservers are and nothing's going to cause a revert.13:44
StevenKwgrant: r1541713:47
wgrantStevenK: Excellent, thanks.13:47
wgrantHopefully by the end of the year we'll be doing this in two hours, not two days.13:47
ivoryjelmer: thank you :)13:57
danilosflacoste, mrevell, jamestunnicliffe, matsubara: hi, shall we try hangout again: https://plus.google.com/hangouts/_/40c23fb864bd02615f8a2bc4d8919084f9053f90?authuser=0&hl=sr14:59
danilosmrevell: hi, shall we try hangout again: https://plus.google.com/hangouts/_/40c23fb864bd02615f8a2bc4d8919084f9053f90?authuser=0&hl=sr15:02
mrevellTh15:02
mrevelldanilos, Cheers!15:02
mrevellflacoste, Hangout: ^^^15:03
mrevelldanilos, Google Plus is now talking to me in Serbian.15:04
danilosmrevell, don't you feel happy about it? just get hl=sr off the URL, sorry :)15:05
matsubarahttps://dev.launchpad.net/Projects/WorkItems15:05
mrevellczajkowski, In case you're interested, hangout ^^^15:07
jamestunnicliffehttps://launchpad.net/~linaro-infrastructure/+upcomingwork15:12
danilosmatsubara, and https://launchpad.net/people/+me/+upcomingwork15:13
danilosmrevell, https://blueprints.launchpad.net/linaro-infrastructure-misc/+spec/engineering-view-fixes15:17
danilosmrevell, https://bugs.launchpad.net/launchpad/+bug/100441615:18
_mup_Bug #1004416: Work Items not allowing users to edit them properly <work-item-tracker> <Launchpad itself:Triaged by linaro-infrastructure> < https://launchpad.net/bugs/1004416 >15:18
=== salgado is now known as salgado-lunch
sinzuijcsackett, can you review https://code.launchpad.net/~sinzui/launchpad/commercial-jobsource-zcml/+merge/110365 today?16:29
jcsackettsinzui: sure.16:36
jcsackettsinzui: r=me, but i'm confused by moving the lp.services imports? is that a merge artifact, or was there a problem with standard sort order?16:44
sinzuijcsackett, someone sorted imports and I merged16:45
sinzuiI can resort16:45
jcsackettsinzui: cool, thanks. :-)16:48
jmldist/zope.testing-3.9.4-p13.tar.gz. p13? my sympathies!16:58
=== matsubara is now known as matsubara-afk
jmlhttplib2.SSLHandshakeError: [Errno 1] _ssl.c:504: error:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed17:03
jml^^ got that error while trying to do API call against a dev instance17:03
jmlhow can I work around?17:03
jml(alternatively, how can I test creating a PPA on Launchpad as a user who does *not* have a commercial subscription?)17:04
cjwatsonCan anyone advise on the problem I'm having in https://code.launchpad.net/~cjwatson/launchpad/export-change-override/+merge/109549 ?17:11
jmlcjwatson: looking.17:13
jmlcjwatson: perplexing. copy_field doesn't seem to have any positional args beyond the first, so I'm in favour of the "Not being called" idea. Maybe it's something to do with the circular import hackery?17:18
cjwatsonI thought I used a keyword arg.17:20
jmlcjwatson: you did. yes. I checked for an error that in retrospect couldn't be possible.17:22
cjwatsonI'm not really convinced this proposed change actually makes the code any easier to maintain, but I'm not the reviewer ...17:22
jmlwhy do lazr.restful, lazr.restfulclient and launchpadlib only include the first 6 points of the GPLv3 license?17:23
cjwatsonI suppose I could start print-debugging through copy_field.17:23
jmlhttp://www.gnu.org/licenses/gpl.txt vs http://bazaar.launchpad.net/~lazr-developers/launchpadlib/trunk/view/head:/COPYING.txt17:23
jmlsame date & version, I think.17:23
jmloh. lgpl.17:24
jmlnever mind.17:24
=== salgado-lunch is now known as salgado
jmlsinzui: fwiw, my build was not from a public source.18:46
jmlmischan18:47
sinzuijml: the same issue is happening. builds are not private, so they cannot issue a 403 or show a privacy banner. the privacy is very smart as it looks at the object, then the view to ensure the banner is shown18:49
jmlsinzui: then the bug has the wrong title.18:50
sinzuifix it18:50
jmlsinzui: ok.18:50
evlifeless: if I may briefly go back to the conversation of not being able to use launchpadlib directly from a wsgi app because it's not thread safe.19:30
evWhat are your thoughts on talking straight HTTP to the webservice, rather than going through lplib?19:30
evthis being an alternative to adding a rabbit/celery layer19:30
lifelessev: love it.20:13
evso that's a yes, avoid lplib for this then? :)20:14
evlifeless: and good morning20:14
lifelessev: I have no soft spot for it :P20:15
evheh20:15
evawesome20:15
evthat makes things much simpler20:15
=== salgado is now known as salgado-afk
=== jcsackett changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | Welcome our new intern: ivory | On call reviewer: - | Firefighting: - | Critical bugs: 3.47*10^2

Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!