[00:08] garh [00:08] this search stuff is so frustratsing [00:15] Hm. Is loggerhead running? [00:35] \o/ my conflict resolution branch just landed [00:52] there's a lot of confusion between tests in lp.archiveuploader and some of those in soyuz [00:53] I moved a whole lot a couple of days ag. [00:53] But yes. [00:53] It's pretty bad. [00:53] which way did you move them? [00:53] I moved doctests from soyuz to archiveuploader. [00:53] wgrant: seems to be working for me? no alerts, and a test looked good. ?? [00:53] I left soyuz-upload.txt alone, though, since it's not clear where it should live. [00:54] spm: Odd, was 502ing for me almost immediately. [00:54] wgrant: can you see http://ec2-75-101-196-90.compute-1.amazonaws.com/summary.log? [00:54] james_w: No. [00:55] james_w: you need to open the firewall more, manually. [00:55] wgrant: http://paste.ubuntu.com/472006/ [00:55] that confused me for a while, the second isn't actually at the path that it says it is [00:55] it's in soyuz, and it appears to test exactly the same thing [00:56] That's one I just moved. [00:56] Sure you're up to date? [00:56] plus I just wrote a bunch of tests for checkUpload, and it now turns out that it is tested in archiveuploader [00:56] ah, that's probably why, I'm probably based of an older devel [00:56] Right. It used to live in archiveuploader. [00:56] Then it was factored out. [00:57] ah [00:57] the tests were changed but not moved I guess [00:57] how frustrating [00:58] Have you seen some of those doctests? [00:59] yes [00:59] these were unit tests though [01:00] Ah. [01:05] james_w: Sorry, that must be my fault. [01:07] The way that code worked was a bit messy, where an Archive method called out to a plain function and then function then called out to the Archive's methods again. I cleaned that up (by merging the logic into Archive) but never got around to moving the tests along. [01:07] jelmer: I think I can move all of lp.archiveuploader.tests.test_permission in to lp.soyuz.tests.test_archive? [01:08] * jelmer looks [01:09] james_w: yep [01:09] great [01:43] I have renewed my hate relationship with doctests [01:43] also with layering, fragile tests and DB's. [04:17] wgrant: so, on this private librarian issue [04:17] wgrant: I think we are, today, completely insecure. [04:17] wgrant: by which I mean many things that are on private objects are on the public librarian [04:18] lifeless: Right. Only P3As really use the restricted librarian. [04:18] Oh, and MP diffs. [04:19] I'm wondering how risky it is, to do it without C-D, and add C-D in later [04:19] oh, other angle [04:19] all restricted librarian content can attack all othe restricted librarian content at the moment too [04:19] in that it can get project names, urls, do same-site scripting. [04:20] thumper: ping (private librarian usage) [04:21] thumper: can we make the stuff we put in the librarian for private MP's be totally preformatted ? then the client could access it directly. [04:22] lifeless: Restricted librarian is purely proxied. [04:22] And the proxied stuff is always C-D'd if dangerous. [04:23] In fact, it may always be. [04:23] I forget. [04:23] wgrant: I'm pretty sure that that is a porky [04:23] The restricted librarian is not accessible from the outside. [04:23] Only via the webapp. [04:23] yes [04:23] thats rather my point [04:23] its in the LP domain [04:24] lib/canonical/launchpad/browser/librarian.py [04:24] like 141 [04:24] we set C-E [04:24] and C-T [04:24] and thats it. [04:24] Yes, I just found that. [04:24] That's broken. [04:25] so [04:25] I'm saying: [04:25] - do we add C-D:attachment now [04:25] - or get direct access now and add it later [04:25] I'd rather not do both direct access *and* add C-D, because of the potential for interactions and confusion. [04:26] though clearly, we want to end up doing both (or doing something crazy like uuid DNS names to isolate content more. [04:26] I guess that it's no worse than now as long as it's served separate from the public librarian. [04:26] wgrant: yes, we'll want a new hostname for sure. [04:26] is private.launchpadlibrarian.net going to be good enough ? [04:27] or will it need to be yet-another-sibling ? [04:27] No. [04:27] launchpadlibrarian.net is full of untrusted content. [04:27] Wait. [04:27] That might be safe, then. [04:33] lifeless: re private librarian, we use the same diff for both email and web usage [04:33] lifeless: so, no, not easily [04:34] thumper: so to fix your timeout you have two choices. [04:34] fix the librarian client bug that it doesn't set/use a timeout [04:34] or make the content you store immediately deliverable [04:45] ✁☹ [04:45] WTF! [04:45] bzr is lying [04:46] spm: hey [04:46] spm: nevermind [04:46] or should I say ec2 test is lying [04:46] * thumper wonders how... [04:48] I have a success mail from EC2 from two hours ago. [04:48] No sign of it in PQM yet, though... [04:48] wgrant: I have a branch that it "merged" into devel [04:48] wgrant: but failed the build step [04:48] wgrant: on a line that isn't in the resulting branch [04:48] Nice. [04:49] * thumper tries again [04:56] poolie: ping [04:58] spm: how is rt 39643 ? [04:58] hi lifeless [04:58] what's up? [04:58] poolie: how is the featureflags stuff ? Can I encourage folk to use it, or is there more needed before it can be adopted ? [04:58] :( [04:58] buildbot all read [04:58] red [04:58] poolie: I realise its not-all-finished [04:59] they can use it now [04:59] i don't expect to pull anything out [04:59] it will be easier when i land the thing currently rated tweak [04:59] oh, i think it's only in db-devel at the moment [04:59] so that may limit things a bit [04:59] (this would incidentally have been a pretty safe change to apply onto the live db [04:59] since it's just creating a new empty table [05:00] I think the table exists [05:00] if it doesn't, you can ask stub to do it, and then merge your code to devel [05:01] so next up is: [05:01] add some scope detection for beta users etc [05:01] web api to tweak it [05:02] logging [05:02] better way to test things tat depend on features [05:02] lifeless: how to I merge in the reverse of the last revision? [05:02] but none of these should block use now [05:02] lifeless: to back out a change [05:02] thumper: merge -r X..X-1 . [05:04] lifeless: ta [05:05] lifeless: a place of suffering and pain is where it's at, unf. am hoping to grab Mr K shortly for some thoughts and guidance on reducing the pain [05:06] spm: cause, I'm told that getting that done is blocking my two fav lp rt's [05:06] 40477 and 40480 [05:06] which in turn are blocking making *everyone* a lot happier. [05:06] that happens [05:07] spm: Moi? [05:07] indeedily; but shoo - lunch. grab you after. [05:07] Hehe [05:16] spm: when you get back, I would like 5-10 minutes to talk roadmap of deployments etc [05:16] not to change anything, just to get me on the same page [05:17] lifeless: sure. I'm back, it was SK heading lunchwardly. [05:17] oh [05:17] * lifeless checks the clock :) [05:17] spm: so, whats in the losa pipeline for lp [05:17] pain, misery suffereing. the usual. [05:18] spm: have you seen - https://dev.launchpad.net/LEP/ReleaseFeaturesWhenTheyAreDone [05:19] not in a final form like that; but was aware of the broad outline [05:19] ok [05:20] this is going to be a pretty big shift, I want to tie it in nicely with your workload [05:20] AIUI the main thing on your plates is the lucid migration, which involves pg8.4 too [05:21] for LP, yeah pretty much [05:22] how deep is that pipeline, realistically ? [05:22] in terms of? [05:22] just LP or everything? [05:23] are we there yet :) [05:23] hmm [05:23] I'm trying to assess the following: [05:23] heh, I wish. we haven't even done staging as a trial run [05:24] - are we blocked on manpower on the operations side to effect the changes desired [05:24] tho we have done LS successfully; some gotchas in that; and at least one issue that may (I stress may in the unknown sense) be a blocker for LP/U1 et al [05:24] - is there *anything* the developers need to do that isn't yet done, for the pg8.4, lucid, and RFWTAD changes [05:24] RFWTAD ? [05:24] https://dev.launchpad.net/LEP/ReleaseFeaturesWhenTheyAreDone [05:25] oh right, of course. [05:25] - if there is nothing developers can do, and we're not blocked on manpower, then I can start asking folk to use the new process; otherwise I need to rally something on one side or the other. [05:26] or if we're blocked on decisions, technical input, etc [05:26] for pg8.4/lucid - no idea. I have no visibility into what has been done/tested there. so NFI if it'll work or not. ?? [05:26] that we don't have a working test setup for lucid doesn't bode well in the first instance [05:26] I think there is a certain swings-and-roundabouts aspect there [05:26] it's been made critical though, AIUI [05:27] ok, so thats 10 minutes - thanks, we're cooked for now :) [05:27] yeah, so's everythingelse (critical) tho. :-( [05:27] critical on the dev side [05:28] * poolie off to lunch [05:28] lifeless: happy to talk about flags later if you like [05:28] coolio [05:28] i wasn't planning to work on it for the next couple of days other than to tweak & land what's already there [05:29] we're about to enter freeze, I think [05:29] i wanted to work out how to make the docs visible [05:29] go have tood [05:29] perhaps putting them in a module docstring would be best [05:29] *food* [05:29] we can talk when you return [05:29] late food [05:29] :) [05:29] ok [05:37] spm: also, is there an RT for daily-staging ? I think that that may have slipped through the cracks. [05:38] lifeless: ? we already update staging every day; several times a day infact [05:38] yeah, names are terrible. [05:38] this is staging-with-production-schema [05:38] there's no difference. ?? [05:38] yes there is [05:39] rt 40482 [05:39] as in; if we just rollout code to staging without coresponding DB schema changes; staging simply won't start. [05:39] Isn't staging with production schema just edge? [05:40] wgrant: can't do destructive testing on edg.e [05:40] Oh, right. [05:40] wgrant: there is /no/ place that you can QA edge properly. [05:40] wgrant: and thus, edge does not get QA'd. [05:40] oh! right. sorry - my bad. Prod schema. not prod+1 schema. [05:41] lifeless: given how little used staging is atm; is there any value in having another QA env? edge has the advantage of at least being used by lots? [05:51] spm: QA != users using it. [05:51] spm: see the RT - same hardware. [05:52] and we do, until we delete db-devel, require two environments because we have two revisions that can be deployed: db-stable:tip and stable:tip [05:53] lifeless: one minor ~ish nitpick. the staging db only restores once a week. it takes something like 20+ hours to happen. [05:53] hah [05:53] well, $same-schedule-please [05:53] yup :-) [05:53] But you can surely upgrade the schema without restoring it fully. [05:54] wgrant: we don't want to upgrade the schema [05:54] Um, yes, ignore me. [06:13] thumper: File "/var/launchpad/test/lib/canonical/shipit/browser/shipit.py", line 73, [06:13] ^^^ [06:13] bah [06:13] ^^^ [06:13] better [06:14] thumper: shipit is out of tree now, so you probably need to import the errors from the old location, fix shipit, and then do a three-branch landing dance. [06:14] * lifeless takes a break [06:14] Can't we just stop rolling out LP to shipit? [06:16] I don't see why we still do it. [06:19] does it have any data in common now ? [06:19] It doesn't use Person any more. [06:19] So I don't think so. [06:19] Except for Account, which is what we want to destroy. [06:22] Oh, no. [06:23] Looks like it does still use Person: for checking shipit-admins membership, and confirming karma. [06:35] lifeless: oh i meant to ask, is there going to be an architectural overview, and if so where? [06:35] are yougonig to build on the one bjorn started? [06:36] poolie: no [06:36] (i'm not assuming it needs to be all your work) [06:36] uh [06:36] I think we want to improve the docs [06:36] so things people should/might want to know before hacking on launchpad will go where? [06:37] for now, where they go at the moment - the launchpad.dev wiki [06:37] having many different places to look would add confusion, not reduce it. [06:37] I rather like what bzr has, and a low-pri task would be to start working towards that [06:38] right now its sufficiently far down the problem-scale that its unprioritised, at least from my perspective. [06:40] wiki is fine with me [06:41] i think it's just good to have a centre of gravity [06:41] for your flags stuff, I would like the following written up somewhere. [06:41] what should LOSA's know [06:41] so you can say "that should go in X" and gradually get the habit [06:41] what should developers writing code know/think of [06:41] what should maintainers-of-the-module know/think of [06:42] and possibly, what should users know (we might want users to see their active flags?) [06:42] mm [06:42] advanced users perhaps [06:42] I think having a good module docstring is really nice. [06:42] right, especially if that's built onto the web somewhere [06:42] then the wiki can point to that [06:42] I think it is on the apidocs [06:42] right [06:43] but not https://launchpad.net/+apidoc/devel.html - thats different again. [06:43] there is a zope thing [06:43] anyhow, if someone can do 'pydoc lp.services.flags', I think thats pretty cool, but it doesn't really help with discoverability. [06:44] https://dev.launchpad.net/Hacking is the centre of gravity for 'getting started doing something in LP' [06:46] i'll put something on the wiki and link to the api docs [06:46] i'm glad someone built them to html [06:46] (gary?) [06:46] we just need to make that public to non-canonical people [06:47] poolie: You mean pydoctor output? [06:47] mm [06:47] i thought it was on chinstrap or similar [06:48] http://people.canonical.com/~mwh/canonicalapi/ [06:49] that's great [06:50] http://people.canonical.com/~mwh/canonicalapi/lp.services.features.html ! :-) [06:50] wgrant: has that always been there? [06:50] poolie: since mwh put it together 3-4 years back :) [06:51] ok so salgado said about gary's [06:51] > Unlike the other docs we have, these ones know about interfaces and the classes [06:51] which implement them, adapters, views, global utilities, zcml directives [06:51] and a few other things, so go check it out. [06:52] Oh, Zope apidoc? [06:52] Yeah. [06:52] it would still be nice to move that [06:52] That's on devpad. [06:52] But accessible easily from a local instance. [06:52] which is private? [06:52] devpad's private, yes. [06:57] lifeless: ah... I've been fixing something in a symlink :( [07:19] * wgrant wonders what lp.soyuz.xmlrpc is about. [07:19] It's empty, and Soyuz has never done XML-RPC. [07:19] thumper: is your git-test fixing fix landed ? [07:19] lifeless: yes [07:19] \o/. [07:20] * lifeless tries to land his malone patch AGAIN [07:22] * wgrant ponders a massive c.l.i extermination branch. [07:27] I wonder if apidoc.launchpad.net is on the cards [07:27] Unlikely. It provides source access. [07:28] So does launchpad.net/~launchpad-pqm/devel ? [07:28] They like to pretend that we don't have access to shipit. [07:29] Apparently giving away free CDs is a risk? [07:30] * thumper EODs [07:33] wgrant: huh ? [07:35] lifeless: Well, it's somewhat irritating that I have to revert to devel r8967 to see if I can safely delete something. That's when shipit was removed -- which I really don't see the benefit of. [07:35] "Oh no, we can't have them giving away free CDs." [07:36] wgrant: was it perhaps back in the 'not all will be open sourced' period ? [07:38] lifeless: It was at the same time SSO was (somewhat less inexplicably) removed. [07:38] And around the time they were considering omitting most of Soyuz. [07:38] so [07:38] you could propose a merge back in :P [07:38] Well, I'd prefer that the split was finished. [07:39] The only benefit at the moment is that it can see karma. [07:39] sadly the split is along fairly nuts lines [07:39] if you wanted to make it use the API to do what it does, that would be great. [07:39] Apart from that, it could live in an entirely separate DB with a static copy of LP. [07:41] lifeless: How is the split 'along fairly nuts lines'? [07:41] vertically rather than front/back end [07:41] Well, ShipIt could sanely be a separate application. [07:41] SSO not so much. [07:42] ok, -> awol for a while [08:29] good morning [08:40] lifeless: Hi! [08:41] lifeless: If I remember correctly __nonzero__ was explicitly not added to ResultSet because it can cause performance problems (though, right now I can't remember what those exact reasons are). [08:42] it is equivalent to try: memo = self.any() except IsEmpty: return False, isn't it ? [08:42] (mumble details) [08:45] lifeless, stub: Thanks for the reviews. [09:15] Hello [09:32] bigjools: Hi. Should I get the last ddeb DB change landed this cycle (the checkbox on Archive:+admin to enable ddeb building)? Although it works OK for PPAs, it won't really be safe to turn it on anywhere until 10.09, I suspect, since the primary archive handling isn't done yet. [09:33] wgrant: yes that's fine as long as the feature is turned off [09:33] wgrant: we should consider moving that checkbox to +edit though? [09:34] OTOH I could make the copier reject DDEB copies to primary, then it would be fine for PPAs. [09:34] there's only one team that can copy from a PPA to primary [09:34] bigjools: Probably, yes. But we might need to work out expiration policies and the like before we make it widely available. [09:34] Moving the checkbox can be done at any time; adding the DB column can't. [09:37] Although PPA binaries are expired really quickly anyway. [09:37] yes, it should not be enabled until everything works :) [09:37] * bigjools -> OTP [09:37] k [09:39] lifeless: I've seen .count() == 0 around. But that's probably far less efficient than .any(). [09:40] ResultSet needs __nonzero__ then we can use bool(blah) [09:43] But it's possible that it's omitted for the same reason __len__ is. [09:43] Callers assume it's cheap. [09:52] __nonzero__ lives on SQLObjectResult though [09:53] I've learned not to assume anything when dealing with Storm code now :) === bigjools changed the topic of #launchpad-dev to: Launchpad Development Channel | Week 2 of 10.08 | PQM will be closing 22:00 UTC Friday | firefighting: - | https://dev.launchpad.net/ | Get the code: https://dev.launchpad.net/Getting | On-call review in irc://irc.freenode.net/#launchpad-reviews | Use http://paste.ubuntu.com/ for pastes === bigjools changed the topic of #launchpad-dev to: Launchpad Development Channel | Week 3 of 10.08 | PQM will be closing 22:00 UTC Friday | firefighting: - | https://dev.launchpad.net/ | Get the code: https://dev.launchpad.net/Getting | On-call review in irc://irc.freenode.net/#launchpad-reviews | Use http://paste.ubuntu.com/ for pastes [10:12] Hm. [10:12] http://blog.ganneff.de/blog/2010/08/02/removalstxt---removals822.html might make it easy enough to implement relibale deletion support in gina. === stub1 is now known as stub [10:21] StevenK: hi [10:21] StevenK: why do you propose things then move them to WIP immediately ? [10:23] bigjools: https://code.edge.launchpad.net/~wgrant/launchpad/multi-arch-builders will let us switch amd64 builders to i386 through the LP UI, letting admins easily alleviate situations like the current crisis... it removes the master's arch check, sends the build architecture to the slave, and the slave respects that. [10:24] Any objections? [10:24] (also handy for when LP learns about processor compatibility...) [10:26] wgrant: yes I object - until lamont looks at it, because I know it needs work on the slave. [10:26] bigjools: Well, of course. [10:27] but yes I'm sure he'll enjoy any branch that makes his life easier [10:28] Although the i386 builders have been gone for nearly four days now. [10:28] Is something up with them? [10:28] Unless alpha 3 is, uh, reallllly slow... [10:37] thumper: I still see from ec2 [10:37] lifeless: which revision of devel did your testrun use? [10:38] jelmer: Hi. Do you have a PQM email for that branch you ec2'd for me earlier? I got an ec2 success email, but it never made it through PQM... [10:40] bzr+ssh://bazaar.launchpad.net/~launchpad-pqm/launchpad/devel/, revision 11272 [10:40] thumper: ^ [10:40] lifeless: hmm... that should have the reverted sourcecode change [10:40] lifeless: not sure what is going on there [10:41] me neither [10:41] oh [10:42] I see, my bad, I had some overlapping stuff going on [10:42] thumper: sorry for the noise. [10:42] ack [10:48] lifeless: Because bigjools is lazy and wants to look at my branch [10:48] oxymoron [10:48] lifeless: So rather than pulling down my branch, he likes me to use a WIP MP [10:49] StevenK: you can tell lp-propose to start it in WIP [10:49] can you create them as WIP, I think he means [10:49] Oh, right [10:49] you *will* get reviews from me otherwise [10:49] I use the web UI to create them [10:49] lol [10:49] and from anyone else tracking the feed [10:49] StevenK: try clicking on advanced [10:50] lifeless: Yes, so noted [10:50] It'd be nice if everything had a WIP MP. [10:51] wgrant: its call 'a branch' [10:52] lifeless: Yes, but I can't get a diff unless I grab it or sacrifice some goats to loggerhead. [10:52] guffaw [10:52] It never works the first time. [10:53] I don't know why. [10:53] It sometimes works on the second refresh. And normally on the third. [10:58] lifeless: Actually, you should look at that MP [10:59] lifeless: *Why* is dsilvers effectively running SQL by hand, and surely there is a better way? [10:59] storm :) [10:59] and collections [11:00] lifeless: Mind looking at the code before you say so? ;-) [11:00] sure, I'll look tomorrow am [11:01] lifeless: this was before Storm days, I'm not sure if the same methods are needed any more [11:01] I don't know a way of doing such mass inserts in Storm, though. [11:02] there isn't - store.execute() FTW :) [11:02] Well, except for just doing a direct SQL -> Storm conversion. [11:18] * lifeless hates on the testfix glue [11:19] lifeless, yeah, we should do something about that :) [11:19] jml: hi [11:19] thumper, hello [11:19] jml: big branch today moving errors [11:20] running through ec2 now [11:20] jml: check my kanban [11:20] thumper, yay [11:20] why is PQM rejecting non-testfix when its also building ? [11:20] jml: most of the bigness however was changing imports :-| [11:20] losa ^ ? [11:21] lifeless, https://dev.launchpad.net/Trunk/Glue might have a hint [11:21] lifeless: I think there needs to be a successful build again to get it out of testfix [11:21] thumper, why's that bad? [11:21] jml: not bad, just a lot of boring changes [11:21] mthaddon: I thought it was 'branch started' was all that was needed. [11:21] thumper: https://lpbuildbot.canonical.com/builders/db_lp/builds/977/steps/shell_7/logs/summary [11:21] thumper, *nod* [11:22] lifeless: I don't think so - I'm not sure how it would be aware of that [11:22] mthaddon: its what the wiki page said [11:22] thumper, I've been meaning to challenge someone to write an emacs equivalent to that neat vim import statement thing EdwinGrubbs showed at the epic [11:22] thumper, that'd make it less boring. [11:22] (or, you know, write it myself). [11:23]