[01:42] * StevenK stabs the lack of completion for bzr rm [02:30] StevenK: How's the ec2 run looking? [02:33] branch-privacy-properties => devel [OK] (up for 2:33:55) i-2ca43755 [02:38] Good good [02:38] * StevenK smacks wgrant. [02:38] steven@undermined:~/launchpad/lp-branches/devel% grep -c '^# ' lib/lp/bugs/model/bugtaskflat.py [02:38] 0 [02:39] Oops, indeed. [02:39] wgrant: Do I need to add access_policy and access_grants to {I,}Branch, or are they internal only? [02:40] I note BTF is missing access_policies and access_grants on its model. [02:40] StevenK: We'll likely add them later, but you can't add them in this branch. [02:40] wgrant: Sure, they have to wait until the DB patch hits prod and devel [02:41] And then you'll add them if you need them. [02:41] And not add them if you don't need them. [02:41] Same as any other piece of code :) [02:41] I was plotting writing tests for the DB patch, which made me ponder model code. [02:42] lib/lp/bugs/tests/test_bugtaskflat_triggers.py may be of some interest. [02:43] wallyworld: Why'd you delete that branch? It attracted the Scanner's Curse? [02:44] yes [02:44] :( [02:44] I still haven't been able to reproduce that locally. [02:44] it won't delete properly either it seems [02:44] Yeah, you can't delete lp:launchpad branches any more. [02:44] Not once they've started to scan. [02:44] well it lets me try [02:44] Sure. [02:44] and then errors in a funny way [02:44] Then it 502s. [02:45] why? [02:45] it sholdn't let me delete [02:45] if i'm not supposed to [02:45] It should let you delete. [02:45] You are supposed to. [02:45] It just doesn't work. [02:45] Because BranchRevision exists [02:45] hooray! [02:45] I'm not sure exactly what's going on, but I *believe* that the BranchRevision delete is unflushed at the end of the request. [02:46] So the commit() in the response post-processing flushes the 90krow delete outside the timeout. [02:46] And crashes. [02:46] not good [02:46] s/outside the timeout/outside the timeout exception handler/ [02:46] It probably turns into a normal timeout if we flush explicitly after the delete. [02:52] wallyworld: I'll review that branch after lunch, btw. [02:52] wgrant: no problem. whenever :-) [02:53] Wow, a smile from wallyworld. [02:53] >:-( [02:54] isn't branchrevision deleted by an ON DELETE CASCADE? [02:54] which is another way of saying the same thing i guess [02:55] btw buildout is very slow if you strace it :) [02:56] s/ if you strace it// [03:03] mwhudson: Ah, indeed. [03:03] That explains why it's unflushed. [03:04] could this be a pg9.1 regression? [03:04] No [03:04] or has it been a problem for a while? [03:04] It's been a problem forever, it's just more prevalent now with the lower timeout. [03:04] You used to have to retry a couple of times to delete big branches. [03:05] Can we bump the timeout for Branch:+delete? [03:05] The 'proper' fix is the death of destruction of most of BranchRevision [03:06] yes [03:11] wallyworld: 70+ if safe_hasattr(user_or_reference, 'id'): [03:11] wallyworld: Can that be replaced by IPerson.providedBy(user_or_reference)? [03:12] no, cause it's not always a person, could be PersonRoles [03:13] wallyworld: Perhaps try adapting it to IPersonRoles? [03:13] Then you can just say IPersonRoles(user_or_reference).in_admin [03:14] If the adaption fails, it's not a person or an IPersonRoles. [03:14] i guess it will fail by returning None? [03:15] wallyworld: By default it'll raise a TypeError [03:15] But the second arg specifies a default [03:15] So IPersonRoles(1, None) will return None [03:15] that's the one i'll use, i don't like flow control by exception [03:15] It is good to avoid where possible, indeed. [03:25] wallyworld: Looks like the branch fixes bug #1009360, bug #1009358 (maybe?), bug #1009281? [03:25] <_mup_> Bug #1009360: RemoveBugSubscriptionsJob duplicates get_bug_privacy_filter < https://launchpad.net/bugs/1009360 > [03:25] <_mup_> Bug #1009358: Unsharing information from a team doesn't remove the members' bug subscriptions < https://launchpad.net/bugs/1009358 > [03:25] <_mup_> Bug #1009281: SharingService.deletePillarSharee revokes all access to any involved artifacts < https://launchpad.net/bugs/1009281 > [03:25] 1009358 is linked already, nevermind [03:25] The other two aren't, though. [03:25] ah, didn't realise thise were filed [03:25] but yes [03:25] i'll link those to the branch [03:25] Great, thanks [03:41] wgrant: that should be fixed now [03:42] wallyworld: Great. I'm experimenting with refactoring the admin specialcase. [03:43] Since everywhere that needs to do it ends up mildly hideous. [03:44] wgrant: should that be done in a separate branch? i'd like to get this one (and the next) landed [03:53] 318 + getUtility(IRemoveBugSubscriptionsJobSource).create( [03:53] 319 + user, bugs=None, information_types=information_types) [03:53] wallyworld: Doesn't that need a pillar restriction? [03:53] Otherwise it's going to do a full consistency check over every bug of that information type. [03:54] yes, that's the intent at the moment [03:54] but i could add a pillar check in [03:55] We can't run it on production like that, but I guess it's a reasonable intermediate. [03:55] that bit was ported across from the deleted remove grantee job [03:55] In the long-term we probably want to restrict to bugs with the relevant AP and a subscriber in the team. [03:55] The join through BugSubscription + TeamParticipation is not completely cheap, but it's far cheaper than a full consistency check. [04:03] wgrant: i have to go pick up the kid, be back later. i'll add the pillar check before landing [04:03] wallyworld: Thanks, I'm almost done. [04:03] ok, no rush [04:03] Would be done quicker if the celery job tests weren't hanging... [04:03] yeah, that happens sometimes to me too [04:05] wgrant: branch-privacy-properties => devel [OK] (up for 4:05:35) i-2ca43755 [04:05] StevenK: Excellent. It's landing rather than testing? [04:05] It will land [04:13] wgrant: r15414 [04:14] Marvellous. [04:16] It's about seven hours away from stable. [04:16] We're fixing that. [04:16] We are? [04:16] I also filed RTs this morning to make qastaging updates faster. [04:16] Well, the test suite will take half an hour soon :) [04:17] wgrant: More than one RT? [04:17] Well, one thing was only tangentially related. [04:27] Oh [04:27] Bah [04:27] Test was failing because the job was crashing because of a typo, and the test didn't notice [05:10] Bug #590523 doesn't seem to have been triaged to a specific type of failure on that page, if I have a new OOPS, should I just add it to the page or file another bug? [05:10] <_mup_> Bug #590523: Archive:+delete-packages times out < https://launchpad.net/bugs/590523 > [05:10] s/page/bug/ [05:12] micahg: Add it to that page. [05:12] thanks [05:23] wallyworld, wgrant: https://code.launchpad.net/~stevenk/launchpad/fix-copyrights/+merge/110229 [05:23] StevenK: lib/lp/app/__init__.py wants reverting [05:24] It's a file of monkeypatches without an import section, really. [05:26] wgrant: Done. [05:27] StevenK: lib/lp/services/database/locking.py's __all__ is in the wrong place. [05:28] wgrant: Also fixed [05:28] The branch is now at +1 [05:30] Otherwise it's good [05:30] Thanks [05:35] micahg: Hah [05:36] micahg: Copying 4 sources with a total of more than 250 binaries in one operation. [05:36] micahg: I am not surprised it timed out. [05:36] I'm special :) [05:39] So, the solution to the bug is to not do that. [05:39] Because it's not sensible :) [05:39] wgrant: umm, to copy 4 source packages to 4 different series in the same PPA shouldn't require 4 times through the page [05:40] For a sensible source package, sure. [05:40] maybe it should just be done asynchronously [05:40] You can't make any assumptions about sources averaging 62 binaries each [05:40] Probably, yes. [05:40] PCJ? [05:40] die [05:41] I have a plan for async ppa copies [05:41] * micahg loves copyPackage [05:41] just never got around to implementing it [05:41] micahg: So, how can there possibly be more than 250 binaries? [05:41] l10n? [05:41] wgrant: yes, the l10n packages were made arch:any because of skew issues [05:42] blink [05:42] When was this? [05:42] micahg: !!! [05:42] or was there another reason for that... [05:42] Recall that we now keep arch-all published until all the arch-any are gone. [05:42] StevenK: I didn't do it :) [05:42] But making them all arch-any is just completely ridiculous and not justifiable :) [05:45] wgrant: http://bazaar.launchpad.net/~mozillateam/thunderbird/thunderbird.lucid/view/head:/debian/changelog#L123 [05:46] it's a waste of mirror space and bandwidth as well [05:46] Ew [05:47] wgrant: ok, so, hot patches [05:47] Oh right. [05:47] lifeless: Indeed [05:48] they have the following properties: [05:48] - they may break running code [05:48] micahg: Amusingly, the copy actually succeeded a few milliseconds before the timeout, but generating the notification timed out. [05:48] - they will apply as regular dbpatches [05:48] I think it might be loading too many objects for the cache to survive. [05:48] - but we can apply them by-hand superfast. [05:48] lifeless: Right. [05:49] so, I'm not sure why we would land them on devel. [05:49] lifeless: Though generally they're just index additions, so cannot break running code (unless postgres misplans and performance suffers) [05:49] I can't find any reference to landing them on devel in the wiki page history [05:50] though I'm not quite at rev 0 [05:50] wgrant: interesting [05:50] I quote: [05:51] 10:51 < wgrant> lifeless: What's the process for live index creation these days? [05:51] 10:51 < wgrant> Apply before or after landing? [05:51] 10:51 < wgrant> It should take about 60ms to create :) [05:51] 10:52 < lifeless> wgrant: land on qastaging, add and QA there, then prod. [05:51] 10:53 < wgrant> land on devel, add and QA on qastaging, then prod? [05:51] 10:53 < lifeless> yes [05:51] wgrant: so clearly I'm inconsistent [05:51] hmm [05:51] stub went back to sleep [05:51] Landing on db-devel provides no benefit. [05:52] And it's extremely awkward to deal with, because db-stable can be undeployed for weeks. [05:52] Because of the one-patch-per-downtime. [05:52] So we'd end up with db-stable being deployed out of sequence. [05:52] For no benefit whatsoever. [05:53] well, thats a very absolute statement. [05:53] It's also absolutely correct :) [05:53] * lifeless wonders if disproving it is worthwhile or a distraction [05:53] ELOCAL, bbiaw [05:53] k [06:26] wgrant: so what we originally wrote was to have the hot patch qaed by application to qastaging [06:30] lifeless: Right, that makes sense. [06:31] lifeless: In practice we often go live directly to production with indices, because there's no meaningful QA until the code is there. [06:32] (though they usually have been tested with real queries on one of the three stagings first) [06:33] qastaging, staging, and the staging that is spelt with no s, t, a, i, n, one less g, and with two d's, three o's, and an f? [06:34] Those, indeed. [06:35] lifeless: So, I don't see that there's anything achieved by landing on db-devel. [06:36] We shouldn't do something just because the policy that's never been executed in full says we should do it. [06:37] There has to actually be a reason :) [06:39] wallyworld: 204 + return [ [06:39] 205 + enumerated_type_registry[InformationType.name].items[value] [06:39] 206 + for value in self.metadata['information_types']] [06:39] wallyworld: Isn't that just this: [06:39] return [InformationType.items[value] for value in self.metadata['information_types']]? [06:40] not sure, i can try it [06:40] i wrote that code ages ago [06:40] IIRC the registry was created for de-JSONing things. [06:40] But you have the actual enum here [06:40] So you don't need the registry. [06:40] no i don't :-) [06:40] the job state is saved [06:40] and then read later from json [06:41] Hmmm? [06:41] You do! [06:41] You get the name of the enum from the enum [06:41] And then use it to look up the enum [06:41] in the enum registry [06:42] i'll try it [06:42] i may have copied the code from when i didn't have the enum name [06:42] That sounds most plausible. [06:42] # The enumerated_type_registry is a mapping of all enumerated types to the [06:42] # actual class. [06:43] So it indeed just returns the class [06:43] So enumerated_type_registry[InformationType.name] is just InformationType [06:43] sounds about right, i have to go and re-read the source [06:45] It's also tempting to ignore Guido and replace the overlong list comprehension with "map(attrgetter('value'), information_types or [])" [06:46] does he not like map? [06:46] He tried to remove it from Python 3 [06:46] why? [06:47] http://www.artima.com/weblogs/viewpost.jsp?thread=98196 [06:49] Basically, Guido doesn't like functional-style code :) [06:49] wgrant: seems like an opinion on style rather than anything of substance at first glance [06:49] Sure [06:49] But he actually tried to remove them from the language :( [06:50] that's one of the good things about pythin, that it's multiparidigm [06:50] Precisely. [06:51] (I think listcomps might not be so bad if we didn't have a policy against using single-letter variable names) [06:57] wgrant: I agree, I'm trying to reconstruct our reasoning [06:57] wgrant: we failed to write some of it down [07:02] lifeless: Perhaps stub remembers? [07:02] I don't remember much of it from IRC, so it was probably mostly between you two [07:02] ondeed [07:02] he is asleep just now [07:03] :( [07:03] I think there are at least minor downsides to going straight to devel, but also less duplicate work merging stuff up etc [07:04] one of the undone longer term things is making hot patch application be -done- (in make schema etc) using the same sql we use to apply it live. [07:04] to avoid manual handling [07:04] wallyworld, StevenK: Looks like the in-progress branch denorm will make lp:launchpad branch search about 12x faster. [07:04] Yay [07:04] great [07:04] lifeless: Right, it'd be nice to automate that. [07:04] lifeless: It is infeasible at present due to slony. [07:05] its totally doable [07:05] just not done [07:05] Well. [07:05] Awkward. [07:05] and not worth doing now. [07:05] And probably not worth it given slony's demise [07:05] Right [07:05] The remaining big issue is CONCURRENTLY. Not sure how best to work around that. === almaisan-away is now known as al-maisan [07:43] good morning [08:04] wgrant: fancy a quick +1 on a +16/-471 mp (deletes RemoveGranteeJob). then i can chuck at ec2 https://code.launchpad.net/~wallyworld/launchpad/delete-RemoveGranteeSubscriptionsJob/+merge/110256 [08:05] wallyworld: Looking [08:05] thanks [08:06] the additonal lines are just in a generic base class test since the test job has been replaced [08:07] wallyworld: Well that was challenging. [08:07] r=me [08:07] Also just looked over your amendments to the one I reviewed earlier. That looks good too. [08:07] Thanks for applying my suggestions. [08:07] np, thanks for the reviews [08:08] wgrant: so, i think if i add the filter for grantee when leaving a team the job should be performant [08:08] The pillar filtering isn't perfect, but in another iteration or two it should be good. [08:08] wgrant: you think it should be on AP? [08:09] wallyworld: That's probably best. eg. now it will miss series bugs. [08:09] AP is probably more direct. [08:09] And accurate. [08:09] We'll need grantee filtering for normal removals too [08:09] eg. if I revoke someone's access to Ubuntu userdata. [08:09] That's a couple of hundred thousand bugs. [08:10] yes, makes sense [08:10] We care about the bugs that have the relevant AP and have a subscriber that is a participant in the grantee. [08:10] the grantee will be removed first though [08:10] before running the job [08:11] but that shouldn't matter i don't think [08:11] Right. [08:11] That certainly makes things look a bit nicer. [08:11] for team membership removal, the filtering will be a bit different [08:11] Yeah [08:11] That has no AP filter. [08:12] since the team itself still has access [08:12] It's just any private bug with a participating subscriber. [08:12] Right, the team isn't relevant to the job. [08:12] The removed member is. [08:12] Any subscriber participating in the former member is affected. [08:12] the team could be used to narrow the search [08:12] Hmm. [08:12] Indeed. [08:13] I'm not sure if that's worth it, but it is indeed a good thought. [08:13] In general the set of subscriptions should be very small. [08:13] so we need to record in the metadata (former) grantee and optionally team [08:13] Well [08:14] The job only really cares about a set of criteria to identify the bugs to reconcile. [08:14] I don't think it needs to know about grantees directly. [08:14] We just need to say "please reconcile subscriptions for participants of ~launchpad in Ubuntu's user data bugs" [08:14] That could mean that ~launchpad had a grant revoked. [08:15] Or it could mean that ~launchpad was removed from a team. [08:15] sure, but i'm more specifically talking about removing a team member [08:15] Er, in the team removal case there wouldn't be the AP restriction, but yes. [08:15] wallyworld: That's the same. [08:15] so we only care about bugs the former member has access to via the team [08:15] Right. [08:15] had [08:16] anyways, i'll code it up and see what falls out [08:16] In most cases it won't be worth filtering by "had access to via the team" [08:16] It will be more efficient just to consider all subscriptions. [08:16] But it is something to consider for bad cases, and it may not be too bad to do it all the time. [08:16] ok, i wasn't going to assume that, but i'll defer to you for that [08:16] i think if the cost of doing it is small, we may as well [08:17] wallyworld: Subscriptions are going to become pretty rare, I think. [08:17] Structural subscriptions handle widespread notification. [08:17] APGs handle widespread access. [08:17] There's no more use case for widespread subscriptions :) [08:17] but the subquery to get the subscriptions [08:18] I like the idea that subscriptions are going to die. [08:18] would benfit from the extra filtering [08:18] wallyworld: So [08:18] Maybe. [08:18] But, for example, I'm removing a user with 5 subscriptions from a team with access to Ubuntu's user data bugs. [08:18] It's clearly more efficient there to just look at the subscribed bugs. [08:19] The converse case can also be argued, of course. [08:19] But huge numbers of subscriptions shouldn't be common at all. [08:19] Huge access grants will be. [08:20] the structure of the job though is to select bugsubscription where bug in (...) [08:20] and it's the subsquery that needs to be fast [08:20] We still need to transition to APGs/APAs. [08:20] That is going to be ... fun. [08:20] yep :-) [08:31] also subscriptions will be a separate service in future too. [08:31] So you keep saying [08:31] I do [08:31] how is auditor ? [08:32] I've not touched it this week, due to Monday and I'm still stuck having the fixture being able to run manage.py or some other form of it. [08:35] StevenK: Huh? What transition? [08:35] StevenK: We're already using them internally for all bug access checks. [08:35] It's just the UI that's not rolled out yet. [08:35] And it mostly works. [08:35] These are the final pieces coming together. [08:36] Perhaps I am misrecalling the Marvellous, Foolproof and Excellent Disclosure Plan. [08:38] wgrant: However, branch access is another matter. But, that is progressing, slowly. [08:39] Not particularly slowly. [08:39] We just have to wait for the bug stuff to mature and then refactor it to do branches too. [08:39] Branches are a faaaar simpler problem. [08:39] Once you discard stacking. [08:39] I was about to say "*cough* stacking" :-) [08:40] 'discard' [08:40] heh heh [08:41] wgrant: I say slowly, because I have a DB patch that drags it forward and I can't land it for a week. [08:42] And when it deploys is another matter entirely. [08:42] StevenK: You can probably land the DB patch on Monday night. [08:42] It can be deployed on Tuesday. [08:44] wgrant: That plan requires that the model code is dropped in a deployment on Monday. [08:44] But okay. [08:44] StevenK: That's the plan. [08:44] Although if we push it we can drop the model code tomorrow. [08:44] Let's push it :) [08:45] I haven't written the branch yet. [08:45] It's about -10 [08:45] We can deploy today's branch in about 3 hours [08:45] Land the model removal [08:45] Deploy model removal tomorrow [08:45] Land DB patch [08:45] Deploy DB patch Monday night :) [08:47] There are at least seven branches that need QA before we can deploy in three hours. [08:48] The ones with non-trivial QA are ∅ [08:49] Well, r15407 and r15410 require a small amount of thought. [08:49] wgrant: So, model droppage now, or tomorrow morning? [08:50] StevenK: I'd prepare the branch now, since it's trivial. [08:50] It shouldn't be landed until today's is deployed. [08:51] I think you can just delete any statement mentioning _transitively_private or _explicitly_private. [08:51] Of which there are two ... [08:52] 3 lines mention explicitly, 8 transitively [08:52] But close enough [08:52] So it's going to be like -13 [08:52] Not -10 as I said :( [09:00] wgrant: I'm at -18/+0 already [09:00] https://bugs.launchpad.net/launchpad/+bug/1013056 [09:00] <_mup_> Bug #1013056: Creating a PPA requires one to be a team admin < https://launchpad.net/bugs/1013056 > [09:01] jml: Argh [09:01] Make it go away [09:01] I'm trying. [09:01] That one comes up occasionally [09:01] I don't have a good answer. [09:01] I was thinking something similar [09:01] I filed it. [09:01] I know you filed it. [09:01] Here are some answers I can think of. I won't claim they are good. [09:01] The issue comes up in discussion occasionally. [09:01] make it launchpad.Append [09:02] make a new object, say, ArchiveCollection, and hook permissions on to that. [09:02] append on what? [09:02] So, allowing it technically is easy. [09:02] Socially it's less obvious whether it's sensible. [09:02] bigjools: IPerson [09:02] ah ok [09:02] jml: I'm a member of ~ubuntu-bugcontrol [09:02] Should I be able to create a PPA? [09:02] No. [09:02] ubuntu-bugcontrol isn't a PPA team. [09:02] wgrant: right. similar for mailing list teams. [09:03] wgrant: http://pastebin.ubuntu.com/1040482/ [09:03] and the last thing Launchpad needs is another knob. [09:03] jml: No, we have enough of those working on it already. :-P [09:03] badumtish [09:03] jml: Precisely. [09:03] jml: So I would just make your robot a team admin. [09:03] And ignore the question. [09:03] wgrant: we have a contractor too. [09:03] can we apply a Knob-count approach like LOC? :) [09:04] jml: Can't SCA have an API to do it or something? [09:05] There's no sane way to do it in LP without adding a knob. [09:05] wgrant: we can probably get by, but I'd rather leave the bug open for the moment. I don't like bots with team admin privs either. [09:05] Sure. [09:05] We need something more flexible. [09:05] But we have no way to do that without making everything suck right now :/ [09:05] wgrant: by 'have an API', you mean write a front-end and secure that? [09:06] jml: Maybe. [09:06] jml: I mean, SCA already accepts API requests from webapps to manage subscriptions. [09:06] And those webapps have admin UIs [09:06] That could be used to ask SCA to perform admin operations like setting up a PPA for a new app. [09:07] wgrant: yeah, we could possibly do that. We weren't planning on having SCA create PPAs, but that might work. [09:07] StevenK: Ah, comments, I see. That's cheating. [09:07] Hahaha [09:07] wgrant: Do you think it's reasonable? [09:09] StevenK: It does what we want, it works, it doesn't fail tests, it doesn't add ugly code. [09:09] I don't see how it could not be reasonable. [09:12] StevenK: Deployment report is green and the two remaining revs before yours are 30s QA. [09:14] wgrant: I think this branch model may also have to kill the transitively_private triggers tests. I'm not certain. [09:14] Oh, indeed. [09:15] bzr rm lib/lp/code/model/tests/test_branch_privacy_triggers.py [09:15] wgrant: map/listcomp> the tradeoff has changed a bit in Python 3 though - map returns an iterator, so if you need a concrete list then the map approach is an extra six characters versus Python 2 [09:15] cjwatson: True. [09:15] StevenK: Actually, it shouldn't matter. The trigger tests don't use the model. [09:15] StevenK: It's all string-based SQL [09:16] StevenK: But might as well delete them anyway. [09:17] 15412 is a bit more than 30s QA since it's gone wrong before, but hopefully not overly much. [09:18] Meh, it's only worth two API calls. [09:19] Actually, I could do that on dogfood now couldn't I [09:20] You could indeed. [09:20] It won't be on qastaging for a couple more hours. [09:24] wgrant: https://code.launchpad.net/~stevenk/launchpad/destroy-branch-privacy/+merge/110274 [09:26] StevenK: r=me, ec2test time. [09:27] wgrant: Tossed at ec2 test. [09:29] StevenK: Great. [09:31] It will be great if it doesn't fail any tests. Not before. [09:31] Meh [09:32] jml: I saw a car today with a sticker in the back window that said "Beached as!" [09:32] and I thought of you [09:33] bigjools: :) [09:58] hmm .. can someone please take a look at this build : https://launchpad.net/~openquake/+archive/testing/+build/3575134 [09:58] the upload of the resulting binaries seems stuck [10:00] al-maisan: Cronjobs are disabled for a DB upgrade in a few seconds. [10:00] Should be processed in a few minutes. [10:00] ah, I see, thanks! [10:01] al-maisan: The webapp downtime is down to 70s, but we're still disabling cronjobs about 15min before for safety. [10:02] no problem at all -- at least I know what's going on now :) [10:11] al-maisan: That uploaded a few minutes ago. [10:11] thank you William! [10:13] ooh, a db upgrade [10:13] will this include my patch? [10:13] jml: This was stub's index work. Yours is nearly 24 hours away. [10:14] wgrant: ta === matsubara is now known as matsubara-lunch === al-maisan is now known as almaisan-away === mrevell_ is now known as mrevell [11:19] cjwatson: Did you get around to doing that QA on DF? [11:22] StevenK: you get +10 internet points for your email signature I noticed today [11:25] wgrant: Yes; I've been waiting for qastaging to update. [11:25] It'll be there in about 30s. [11:25] (So I can mark it qa-ok.) [11:25] Cool. [11:25] Ah right [11:25] That'll be another 5 minutes. [11:25] Thanks. [11:26] There we go, right on time. [11:29] wgrant: Bug 648611: any particular reason you feel it necessary to include upload status granularity in that? [11:30] cjwatson: I don't remember who that was discussed with. ScottK and others. But if it's no longer an issue for you, then there's no need for it. [11:33] wgrant: do you know if the codehosting puller is still in use? [11:34] Maybe I should talk with ScottK then. [11:34] jml: Mirrors and imports still use it. Mirrors can't be created any more, but there are still some old ones in use. [11:34] It'd be easier to just make it per-pocket, since that's almost exactly analogous to per-pocket upload. [11:34] wgrant: ah ok. I'm surprised imports use it. [11:34] oh wait [11:35] jml: Same as ever. [11:35] to pull it from internal location where branch is [11:35] importds push their branches up to escudero [11:35] LP pulls from escudero [11:35] Because who cares about sanity :) [11:35] wgrant: I thought I recalled some discussion about using the codeimport system to do pulling [11:36] ddaa had a reason for that, I'm sure. [11:36] huh [11:36] I wonder. Could / should the ScriptActivity table have a place in the web ui, exposed to LP developers & admins. [11:37] rick_h: Haha [11:37] jml: s/table/service/ [11:38] wgrant: even better. [11:38] wgrant: Are you going to be awesome and do my QA, since dinner for Sarah and I is about 3 minutes away from serving? [11:38] StevenK: Mostly done already [11:38] hmm. hmm. hmm. [11:39] wgrant: ec2 has another ~ 110 minutes, but it should be good to lp-land when it finishes. [11:39] jml: Hm hm? [11:39] wgrant: I was thinking about starting work to pull codehosting out of LP [11:39] wgrant: and then about how I wonder how much work it would be to make & deploy a scriptactivity service as we just described [11:40] Oh, hey, maybe I lied when I said to technical-board@ that my permissions branch would probably land tomorrow or failing that Monday. [11:40] wgrant: and then about how I don't know whether anything *has* been pulled out of LP into a separate service successfully [11:40] cjwatson: Pocket permissions? With a bit of luck it'll be deployed 40 minutes after gnuoy returns from lunch. [11:41] jml: It depends on how much of codehosting you want to pull out of LP. [11:41] Yeah, I missed your chatter about pushing it. [11:41] How much of codehosting do you want to pull out of LP? [11:41] wgrant: and how if I started I'd be afraid that the work would spiral out of control with lifeless, yourself and others telling me that I'm doing it wrong or adding new requirements or elucidating requirements that I'm not fully conscious of yet. [11:41] wgrant: I think the ssh server / vfs bit makes sense. [11:42] wgrant: maybe the apache rewrite support too. I'm not sure. [11:42] Nothing like this has been extracted before. [11:42] wgrant: an obvious pre-req would be extracting lp.services.ssh into a library. [11:42] I was going to do poppy to start. But I ran into trouble with naming the libraries. [11:42] Right. [11:43] That's pretty easy to do. [11:43] and I don't know how I'd answer questions like what to use for config. [11:43] And it's probably most of the integration points. [11:44] wgrant: poppy would also be a good place to start. [11:44] I'd probably just use a separate minimal lazr.config-based thing. But configglue or something might be easier and better. [11:44] jml: I'd suggest starting with poppy, as it uses the sshserver and is much simpler in every other respect. [11:44] wgrant: it seems to me that doing one small thing, and (crucially) writing down the steps taken would be a giant leap for Launchpad. [11:44] Most definitely. [11:45] poppy is probably the smallest, easiest piece to rip out like that. [11:45] It's extremely isolated. [11:45] Yeah. The list for me goes: poppy, codehosting ssh, maybe codeimport, builddmaster. [11:46] Although you mentioned scriptactivity, which probably goes somewhere toward the right. [11:47] scriptactivity is sort of the opposite. [11:48] It's an internal service used by everything, but it's very isolated and an excellent example of something that should be a microservice. [11:49] scriptactivity is probably a prereq for getting rid of other things. [11:49] Since they'll want a way to record activity. [11:51] anyone have much experience with lxc? I'm running low on space on '/' so I'm trying to put the rootfs on an alternate drive. But once I do 'lxc-start' just returns without starting anything, or giving me anything informative. [11:51] I've tried playing with the config file, to point the root to another location [11:52] maybe if you know of an error log or something like that? [11:53] jam: Tried 'lxc-start -n foo -o /tmp/lxclog -l DEBUG'? [11:53] wgrant: yeah. it's realizing that it's a pre-req that got me 'hmming' in the first place [11:54] Yeah [11:54] Specs fall apart, the scope it cannot hold; more work items are loosed upon the world [11:55] wgrant: anyway, I'll keep it in mind as a possible skunkworks project [11:58] Sounds good. [11:59] So, poppy and scriptactivity, in that order, would be my recommendation for anyone wanting to do this sort of thing. poppy because it's simple, and scriptactivity because it's simple and a core pre-req from the other end. [11:59] wgrant: doesn't poppy need to use scriptactivity [11:59] * jml just remembered something. [11:59] jml: It's a daemon. [11:59] Not a cronjob. [11:59] ah, ok. [12:01] wgrant: it appears /sbin/init was just dying, but that appears to be because I had that device mounted with "nosuid,nodev" [12:01] it seems to work now [12:01] jam: That would be unhelpful, indeed. [12:01] It was GNOME-mounted removable media? [12:02] wgrant: I added it to fstab, using the params done by GNOME disk-utility [12:02] udisks mounts everything with nosuid,nodev. [12:02] I couldn't find a Gnome utility that would let me say "always mount this disk" [12:03] Ah yes, udisks, I had forgotten what was underneath today. [12:05] udisks2 in quantal, because it would be a shame to leave anything the same. [12:06] wgrant: you don't appear to be subscribed to DisklessArchives. I made an edit, and james_w added a comment the other day for which we'd welcome your feedback. (Can wait until tomorrow though) [12:06] jml: I couldn't find a subscribe button, and /UserPreferences is like all the way over there. [12:06] wgrant: URL hack! ?action=subscribe [12:07] Ah, didn't think that would still work. [12:07] Surely new moin has fixed mutating GETd [12:07] s [12:07] Evidently not. [12:07] Thanks. [12:08] wgrant: np. [12:08] jml: The API service will live in the LP tree. [12:08] The squid bits will not. [12:09] Ideally the API service would be part of the main Zope stack, but the main Zope stack is so terrible that it's not feasible at present, so we have to hack. [12:10] wgrant: that makes sense. it's a shame from a test cycle pov. [12:10] Giving the testing hostname to real clients is an interesting idea. I'm not sure if it's a good one. [12:11] jml: Certainly. [12:11] wgrant: we an use the testing hostname as the new, browser-safe permanent hostname [12:11] wgrant: and initiate a very slow migration. [12:11] True. [12:11] wgrant: however, once we have one non-zope api server in LP to facilitate fast requests, why not move the existing api services to that same technology [12:11] see #ca-internal for a chat between me and james_w about this [12:11] jml: That's the plan. [12:11] wgrant: btw, LP's stack *can* answer in 3ms. [12:11] jml: Whether the plan will happen this decade is unclear :) [12:12] lifeless: hahahah [12:12] I hope not using zope stack means that at least the unit tests will be fast [12:12] +opstats maybe [12:12] wgrant: definitely, see the haproxy stats page. [12:13] wgrant: I noting that the publication stack through to render and return is likely fast enough for our needs; there is definitely potential issues below that. [12:13] wgrant: I'm inclined to suggest trying in the main stack in the first instance; it means we know fairly likely places for the rot to be located. [12:13] Right, for a no-op publication to a no-op view that has hardcoded hacks to avoid most of the stack, it can be fast. [12:14] alas for my TDD cycle time! [12:14] jml: the TDD cycle time for the horizontally scaling nodes should be fine. [12:15] jml: it will need a test harness, but that by definition has to be fast to start and use. [12:15] That's like 3% of the effort. [12:16] btw, at some point before we actually start work it'd be good if you two, james_w and I could have a call to walk through the implementation plan. I realize that TZ wise it's a huge pita. [12:16] lifeless: Are you sure you didn't hallucinate the conversation with james_w? [12:16] also, it looks more like 6-8 weeks than 4-6. [12:16] I don't see it. [12:17] jml: It mostly depends on how far you want to rewrite the backend publication stack. [12:17] lifeless wants you to rewrite them a lot. [12:17] I don't. [12:18] lifeless: Which day was it? [12:18] wgrant: today [12:18] may have been here. Lets see [12:20] wgrant: u1-internal [12:20] Oh, sneaky. [12:20] I don't think I'm in there. [12:20] very very so [12:20] * wgrant grabs logs. === almaisan-away is now known as al-maisan === _mup__ is now known as _mup_ [12:24] jml: Yes, I think scriptactivity should definitely be exposed. I always imagined a page of green/red/yellow blobs for each component we can track. [12:25] jml: I think the trick is to start simple, as bikeshedding and feature creep turn it into a big problem very quickly [12:25] stub: agreed [12:25] stub: the main thing I'm interested in is knowing which cronjobs are still being run and with what options so I can delete more code. [12:25] although I don't think SA tracks options [12:26] jml: bzr branch lp:lp-production-crontabs [12:26] mirable dictu! [12:28] deleting scriptactivity itself would be a start === matsubara-lunch is now known as matsubara [12:31] cjwatson: Perhaps we should revert your revision quickly, just to prevent the rest of UE from getting the idea that LP deadlines don't always slip. [12:38] wgrant: haha [12:39] Also, we're not really in a hurry for something else, I just wanted to stop StevenK complaining that things are too slow. [12:43] wgrant: You know, scriptactivity could probably be replaced by auditor ... [12:45] Potentially. [12:47] and the bikeshedding and feature creep starts... [12:47] jml: pretotyping [12:48] stub: For auditor? I don't think the feature could creep much :-) [12:48] StevenK: That just means it's already crept. [12:48] wgrant: But it hasn't? [12:49] Log events via POST, grab them via GET. Done. [12:49] It might need some squinting in terms of actor/object acted on, but shrug. [12:50] wgrant: It was good natured complaining about things going slowly. :-) [12:50] Sure. [12:50] But we might as well make things faster where we can. [12:51] StevenK: It started as exposing some information in the DB, now you are posting information into it :) [12:52] stub: No, it was always the plan that we had to post information into it, so nyah. [12:58] jml: How soon are you likely to want that call? [13:00] wgrant: before we start implementing, after you guys think it's ready for us to start implementing. [13:00] wgrant: modulo some scheduling stuff that I need to talk to james_w and mvo about. [13:00] Right. [13:08] jelmer: could you review this please? https://code.launchpad.net/~ivo-kracht/launchpad/bug-999662/+merge/110322 [13:11] ivory: sure, I'll add it to my queue [13:11] jelmer: thanks [13:11] gary_poster: i really liked the format of your latest parallel tests update [13:11] gary_poster: it read very well! [13:12] flacoste, thanks! I'll go review it to see what I did differently. ;-) [13:12] ivory: skim-by, is that change wrap the in xx-distirbution-changes.txt valid? I'd think id'd break a doctest [13:12] ivory: Did you consider using fmt:approximatedate? It handles timestamps within the last 24 hours a little more useful, by saying eg. "2 hours ago" [13:13] gary_poster: i think the summary rocked, or maybe is that the news are particularly good this week ;-) [13:13] For more than a day it just uses the date. [13:13] "a moment ago" \o/ [13:13] flacoste, lol, cool, I'll take either one :-) [13:14] mgz: Doctests default to normalising whitespace. [13:14] mgz: So you can normally wrap however you want. [13:14] wgrant: i did but somehow it didn't work, maybe i did something wrong but then abel proposed to do it like that ... [13:14] Not that you'd know it by looking at lots of our existing doctests. [13:14] cjwatson: Shhh [13:15] wgrant: they don't generally, but does launchpad set that flag? [13:15] mgz: I meant our doctests, yeah. [13:15] thanks, good to know. [13:15] default_optionflags = (doctest.REPORT_NDIFF | doctest.NORMALIZE_WHITESPACE | doctest.ELLIPSIS) [13:15] Some of our unit tests also make use of doctest.NORMALIZE_WHITESPACE, which I find hideous. [13:17] doctest.ELLIPSIS is handy when converting doc to unit though. [13:17] mgz: i don't really understand what you mean.i didn't change xx-distribution-changes.txt [13:19] ivory: *-packages.txt, but wgrant settled that question === jcsackett changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | Welcome our new intern: ivory | On call reviewer: jcsackett | Firefighting: - | Critical bugs: 3.47*10^2 [13:34] cjwatson: It is done. [13:42] Neat. [13:43] wgrant: ec2 is done. === al-maisan is now known as almaisan-away [13:44] StevenK: Might as well land it. The deploy isn't completely done yet, but the appservers are and nothing's going to cause a revert. [13:47] wgrant: r15417 [13:47] StevenK: Excellent, thanks. [13:47] Hopefully by the end of the year we'll be doing this in two hours, not two days. [13:57] jelmer: thank you :) [14:59] flacoste, mrevell, jamestunnicliffe, matsubara: hi, shall we try hangout again: https://plus.google.com/hangouts/_/40c23fb864bd02615f8a2bc4d8919084f9053f90?authuser=0&hl=sr [15:02] mrevell: hi, shall we try hangout again: https://plus.google.com/hangouts/_/40c23fb864bd02615f8a2bc4d8919084f9053f90?authuser=0&hl=sr [15:02] Th [15:02] danilos, Cheers! [15:03] flacoste, Hangout: ^^^ [15:04] danilos, Google Plus is now talking to me in Serbian. [15:05] mrevell, don't you feel happy about it? just get hl=sr off the URL, sorry :) [15:05] https://dev.launchpad.net/Projects/WorkItems [15:07] czajkowski, In case you're interested, hangout ^^^ [15:12] https://launchpad.net/~linaro-infrastructure/+upcomingwork [15:13] matsubara, and https://launchpad.net/people/+me/+upcomingwork [15:17] mrevell, https://blueprints.launchpad.net/linaro-infrastructure-misc/+spec/engineering-view-fixes [15:18] mrevell, https://bugs.launchpad.net/launchpad/+bug/1004416 [15:18] <_mup_> Bug #1004416: Work Items not allowing users to edit them properly < https://launchpad.net/bugs/1004416 > === salgado is now known as salgado-lunch [16:29] jcsackett, can you review https://code.launchpad.net/~sinzui/launchpad/commercial-jobsource-zcml/+merge/110365 today? [16:36] sinzui: sure. [16:44] sinzui: r=me, but i'm confused by moving the lp.services imports? is that a merge artifact, or was there a problem with standard sort order? [16:45] jcsackett, someone sorted imports and I merged [16:45] I can resort [16:48] sinzui: cool, thanks. :-) [16:58] dist/zope.testing-3.9.4-p13.tar.gz. p13? my sympathies! === matsubara is now known as matsubara-afk [17:03] httplib2.SSLHandshakeError: [Errno 1] _ssl.c:504: error:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed [17:03] ^^ got that error while trying to do API call against a dev instance [17:03] how can I work around? [17:04] (alternatively, how can I test creating a PPA on Launchpad as a user who does *not* have a commercial subscription?) [17:11] Can anyone advise on the problem I'm having in https://code.launchpad.net/~cjwatson/launchpad/export-change-override/+merge/109549 ? [17:13] cjwatson: looking. [17:18] cjwatson: perplexing. copy_field doesn't seem to have any positional args beyond the first, so I'm in favour of the "Not being called" idea. Maybe it's something to do with the circular import hackery? [17:20] I thought I used a keyword arg. [17:22] cjwatson: you did. yes. I checked for an error that in retrospect couldn't be possible. [17:22] I'm not really convinced this proposed change actually makes the code any easier to maintain, but I'm not the reviewer ... [17:23] why do lazr.restful, lazr.restfulclient and launchpadlib only include the first 6 points of the GPLv3 license? [17:23] I suppose I could start print-debugging through copy_field. [17:23] http://www.gnu.org/licenses/gpl.txt vs http://bazaar.launchpad.net/~lazr-developers/launchpadlib/trunk/view/head:/COPYING.txt [17:23] same date & version, I think. [17:24] oh. lgpl. [17:24] never mind. === salgado-lunch is now known as salgado [18:46] sinzui: fwiw, my build was not from a public source. [18:47] mischan [18:49] jml: the same issue is happening. builds are not private, so they cannot issue a 403 or show a privacy banner. the privacy is very smart as it looks at the object, then the view to ensure the banner is shown [18:50] sinzui: then the bug has the wrong title. [18:50] fix it [18:50] sinzui: ok. [19:30] lifeless: if I may briefly go back to the conversation of not being able to use launchpadlib directly from a wsgi app because it's not thread safe. [19:30] What are your thoughts on talking straight HTTP to the webservice, rather than going through lplib? [19:30] this being an alternative to adding a rabbit/celery layer [20:13] ev: love it. [20:14] so that's a yes, avoid lplib for this then? :) [20:14] lifeless: and good morning [20:15] ev: I have no soft spot for it :P [20:15] heh [20:15] awesome [20:15] that makes things much simpler === salgado is now known as salgado-afk === jcsackett changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | Welcome our new intern: ivory | On call reviewer: - | Firefighting: - | Critical bugs: 3.47*10^2