[00:14] bigjools: ... thunderbird [00:14] bigjools: it did, then it stopped ;) [00:14] yeah I just remembered that :) [00:15] ;) === mwhudson_ is now known as mwhudson [01:14] wgrant, hi, I wonder what's going on with db-stable r11611 rollout: LPS indicates it's out, but it's not merged yet in stable/devel [01:15] danilos: It will not be until it's deployed [01:15] danilos: Let me check [01:15] wgrant, https://wiki.canonical.com/InformationInfrastructure/OSA/LaunchpadProductionStatus lists it in 'past deployments' [01:17] danilos: Your DB patch is r11610, and the next deployment for tonight is r11615, which includes your patch. [01:18] danilos: Sorry, I forgot to merge it [01:18] danilos: Doing now [01:18] danilos: It was deployed 12 hours ago [01:18] wgrant, ah, ok, that explains it, thanks [01:18] Ah, so up next is my patch. [01:18] wgrant, no worries, sorry for being pushy :) [01:18] Thanks for the reminder. [01:25] danilos: That's landed. [01:25] danilos: You can now land the code removal [01:25] wgrant, cool, thanks [01:26] wgrant: I might up put up my builder.description destruction branch then [01:26] wgrant, except that I can't (at least not that I know, if I can, woohoo and "how"), but I can ask someone to do that for me :) [01:26] danilos: Bug lifeless about emeritus [01:26] StevenK: Hm? [01:26] StevenK: Which branch? [01:27] StevenK: danilos: I think danilos is already in emeritus, isn't he? [01:27] and yes, emeritus can lnad [01:27] *land* [01:27] wgrant: The one that drops builder.description from the code since the DB patch to DROP NOT NULL is up next. [01:27] while they are still @ canonical [01:27] Can by policy, but you have to be in the keyring... [01:27] StevenK: Right, can't land for another 9 hours, though [01:27] wgrant: Sure, what I meant was "Actually push and put up for review" [01:28] Right [01:28] Makes sense [01:28] Thought that was already done, in fact. [01:28] steven@undermined:~/launchpad/lp-branches/drop-builder-description% utilities/rocketfuel-mp-status . [01:28] * . [01:29] - Remote branch is up to date [01:29] Hm, it seems you're right. [01:29] lifeless, ah, cool, so I have to worry myself about running the full test suite, and then can pqm-submit? (how is ec2 test used these days?) [01:30] danilos: ec2 land is still used, its unchanged [01:30] Well [01:30] danilos: it lives in lp-dev-tools now [01:30] It's in lp:lp-dev-utils [01:30] bah utils [01:30] Rather than utilities/ [01:30] But otherwise the same. [01:30] In the branch, 'ec2 land' [01:30] ah, ok [01:33] wgrant: https://code.launchpad.net/~stevenk/launchpad/drop-builder-description/+merge/107698 would enjoy your review [01:35] StevenK: Can you use the factory to replace most of create_builder? [01:36] Most or all [01:36] It would end up looking pretty much the same, and the doctest already wants IBuilderSet [01:36] factory.makeBuilder(name='hamburger', processor=i386, virtualized=True) [01:36] No helper required [01:38] wgrant: Although that points I missed out fixing factory.makeBuilder() :-) [01:51] wgrant: Diff updated, can you have another look? [02:44] wgrant: hey, you know what? We don't need that private API for sso anymore [02:45] wgrant: not with your new regular API method [03:07] lifeless: For performance I think we do. [03:08] wgrant: two in-dc calls vs 1 ? [03:09] lifeless: How do you plan to find all of the relevant team memberships? [03:09] participation [03:10] It would require a new call. [03:10] But could be done. [03:11] wgrant: super_teams ? [03:12] lifeless: Is a collection that returns a limited batch of participations. [03:13] pass a batch in greater than the current high-end, and iterate if needed [03:14] wgrant: most folk will be <= 1 batch [03:15] Yeah, multiple 300ms batch loads per authentication attempt isn't high on my list of things that are good ideas :) [03:17] where are you getting that time from ? [03:18] wgrant: but looking at it, it includes the whole metacrazydata [03:18] rather than just providing references. [03:18] \o/ [03:19] whee [03:19] person:+contactuser [03:19] 99% under 32 seconds. [03:19] Yup [03:19] +cve is still horrid too [03:19] 23 [03:20] lifeless: OK, so it looks like without auth you can make an API request in about 100-120ms. [03:20] auth will increase that a fair bit. [03:21] wgrant: because? [03:22] lifeless: Because our auth code is awful [03:22] fixable or unfixable ? [03:23] Fixable. [03:23] there we go then [03:25] DistroSeries:+builds is heinous [03:25] ever seen Branch:+try-again ??? [03:26] Yeah [03:26] It is always either very fast or extremely slow [03:26] Probably due to lock contention, I guess. [03:26] But maybe not. [03:26] Distribution:+topcontributors is good [03:27] I'm not sure why that's so bad [03:27] I haven't looked at a query log. [03:27] But it doesn't make sense for it to be slow. [03:27] Unless it's all unindexed, I guess. [03:31] mhm [03:31] Unknown [03:31] 9.5M hits [03:31] (over a month) [03:31] 404? [03:31] not sure [03:32] need to heckle someone to dig and identify causes [03:33] ahhaha [03:33] on https://devpad.canonical.com/~lpqateam/ppr/lpnet/latest-monthly-pageids.html [03:33] 'SimpleViewClass from /srv/launchpad.net/production/launchpad-rev-15099/lib/lp/bugs/browser/../templates/buglisting-embedded-advanced-search.pt:JsonModelNamespaceView' [03:33] page it [03:33] page id [03:33] it really is [03:43] I need a guesstimate [03:43] how long to instantiate 122 feature rules [03:44] They don't have enumcols, so it should be pretty quick. [03:44] yah [03:44] and a linear lookup of simple values is fast, given we cache results [03:44] so it should be safe [03:44] Also, where'd you get 122? [03:44] Oh [03:45] you should have enough data to infer [03:45] You want to add 122 exceptions? [03:45] And lower the timeout? [03:45] yes [03:47] I'm not sure that all of them deserve exceptions. [03:47] But perhaps. [03:47] it makes reasoning about it easy [03:47] I prefer reasoning recklessly. [03:47] It generally gives much better results. [03:47] More rapidly. [04:34] wgrant: Can haz review? [04:39] StevenK: Oh right, it's still trying to submit. [04:39] I had too many MPs open [04:39] Done [04:41] So tempted to put a Portal 2 reference in the commit message. [04:41] "Delete Builder.description from the code for standing around, smelling and being useless." [04:44] wgrant: http://pastebin.ubuntu.com/1012579/ is that what you envisioned, or am I on the wrong track? [04:47] StevenK: Looks like a reasonable start, indeed. [04:48] Right, excellent. [04:48] Needs tests for Branch:+edit and then it's done, I think. [04:49] wgrant: I'm a little unhappy about the IBranch.implementedBy() bit, but that should hopefully be short lived. [04:49] StevenK: You probably want providedBy, too, but indeed [04:55] Blink. [04:55] On Bug:+index the text turns up on page load and then disappears. [04:55] * StevenK blames wallyworld_. [05:53] StevenK: what have i done? [05:56] wallyworld_: I've refactored out the Information Type portlet into a seperate class that is subclassed by BugView. Loading up Bug:+index shows the right text in the portlet for a few seconds which then disappears. I think the JS is to blame. [05:59] StevenK: this isn't on prod is it? seems to work there [05:59] wallyworld_: It is not on prod, no. [05:59] ok, so only in your local branch [06:00] when you say text, you mean the description that sits below the choice source popup widget? [06:00] wallyworld_: Nope, everything [06:01] hmmm [06:03] wallyworld_: Do you want me to commit and push this mess, or you want to mull it over? [06:03] i was going to look at the mp for the branch i recently landed but since it's merged, the diff is no longer available :-( [06:03] so go ahead and commit it [06:04] wallyworld_: I *HATE* that bug [06:04] i didn't realise it was a bug [06:04] i though it was intended [06:05] It didn't used to happen, I think. [06:05] it's when you push new revisions after merge IIRC [06:05] i can't remember [06:05] or something causes a scan on it [06:05] bigjools: I thought it was because the destination branch changes [06:05] so why does it discard the diff? [06:06] I'm not aware that matters [06:06] it regenerates a diff - which is empty [06:06] hmm. ok. [06:06] bigjools: Because the target branch now incorporates the changes that in the source branch [06:07] so perhaps it should retain the last diff in that case [06:07] StevenK: wouldn't that mean all of our diffs, ever, get regenerated? [06:07] for ever [06:07] bigjools: That seems to be happening anyway [06:11] wallyworld_: https://code.launchpad.net/~stevenk/launchpad/branch-information_type-ui [06:11] * wallyworld_ looks [06:23] StevenK: the initialise of the InfoTypePort;et is not being called [06:23] stupid python class inheritance stuff [06:24] Oh, it won't call *all* subclasses that have an initialize() method? [06:25] nope, not unless they call super themselves [06:25] afaiui [06:25] wallyworld_: But InformationTypePortlet doesn't have a superclass [06:26] wallyworld_: Do you want to try that? lib/lp/app/browser/information_type.py [06:26] StevenK: i think it's stopping the chain at LaunchpadView [06:26] LaunchpadView is the first superclass [06:26] since it's initialise() just passes [06:26] and so the other superclasses are nw called [06:27] Perhaps it will stop after the first one [06:27] StevenK: put LaunchadView last in the inheritance list [06:28] it works then [06:28] StevenK: afaiui, it calls super in order from left->right [06:28] and if one stops, then boom [06:28] they each have to call super to keep it going [06:29] And BugViewMixin does not? [06:29] It doesn't have an initialize(), so I guess not [06:29] BVMixin does not implement that method [06:33] bit of a trap that one [06:33] and so of course it was the bugtask_index javascript that didn;t see the expected stuff in the request cache [06:33] so it duely hid the info type stuff [07:23] ok, how can I convince LP that actually, yes, my GPG signature valid on MP emails [07:39] bigjools: use MIME? [07:40] * bigjools does a Marcel Marceau impression [07:41] What is mime for fucked? [07:49] * stub mimes 'fucked' [07:50] good morning [08:03] aloha [08:04] bigjools: I think you want the hip thrust for that one [08:37] hello [08:38] ohai === danhg_ is now known as danhg [09:18] so, to upgrade to pg 9.1 in dev env, I just remove pg 8.4, install 9.1 and then run database-setup? [09:20] jml: Pretty much [09:20] jml: Also make sure 9.1 is listening on 5432 [09:25] i once tried to figure out how the debian packages created clusters listening on different ports [09:30] so the metapackages are all broken atm then? [09:30] ok, I'm confused. [09:30] launchpad-database-setup configures pg to run on 5433 [09:31] so how can I make sure it's running on 5432 [09:39] ffs [09:39] now I'm getting "invalid data directory" [09:39] graet. === gmb changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: gmb | Firefighting: - | Critical bugs: 3.47*10^2 [09:55] http://paste.ubuntu.com/1012845/ what am I doing wrong? [09:58] nothing obvious, tries 5432... what's in /var/run/postgresql actually? [10:00] jml@lpdev:/var/run/postgresql$ sudo cat 9.1-main.pid [10:00] 7623 [10:00] just that. [10:03] also, [10:03] $ ls -a [10:03] . .. 9.1-main.pid .s.PGSQL.5433 .s.PGSQL.5433.lock [10:03] okay, so it is listening on the wrong port [10:05] change that in the postgre conf and restart? [10:07] and then manually run the createuser commands, I guess? [10:09] there's probably a better way, but naturally it just worked for me previously. [10:10] psql:launchpad-2209-00-0.sql:17: ERROR: language "plpgsql" already exists [10:10] psql:launchpad-2209-00-0.sql:2646: ERROR: could not access file "$libdir/plpython": No such file or directory [10:10] are they fatal errors? [10:14] jml: The second is - postgresql-plpython-9.1 is not installed. [10:14] stub: thanks. [10:15] stub: Ubuntu says it's already installed though. [10:15] jml: If you run 'psql', does it say the server version is 9.1 in the startup banner message? [10:16] psql (9.1.3) [10:16] Type "help" for help. [10:16] yes. [10:17] jml: oic. No, that isn't a fatal error. It will go away soon now we are 9.1 everywhere and can clean that up. [10:17] stub: cool. thanks. [10:17] while you're around... [10:18] I'm aiming to rename a DB column. The branch I submit against db-devel needs to update the code for the rename as well as adding the patch that does the rename, right? [10:19] jml: You can not do that. [10:20] StevenK: Do what? Rename a column? [10:20] jml: You can't change code at the same time [10:21] (Because database and appserver rollouts are decoupled.) [10:22] hmm. [10:22] cjwatson: Are you rotating with LP this cycle? [10:22] so either I can't do a rename [10:22] nigelb: No, I just have a number of LP-related projects [10:22] or I have to update the code to somehow work with both naes. [10:22] names, rather. [10:22] cjwatson: Ah, nice. :) [10:22] jml: If you figure out a neat approach, I have a low-priority bug that could use it. IIRC wgrant told me that we haven't renamed any columns since the introduction of fastdowntime. [10:23] *grin* [10:24] cjwatson: Ah, I see. [10:24] jml: In my case (bug 1000570) I renamed the API but left the DB column for "later". [10:24] <_mup_> Bug #1000570: "Packageset.score" is badly named

< https://launchpad.net/bugs/1000570 > [10:24] cjwatson: right, that's what I've done. I'm just making later, now. [10:25] William suggested a trigger approach that I'm afraid I didn't understand. [10:25] stub: any thoughts on how to do a column rename? [10:25] jml: Yes. Ideally, minimize the code using a property or something [10:25] jml: Actually, hang on [10:26] * jml does so [10:26] jml: So code changes and db changes are not synchronized. [10:26] jml: Which means first a code change that makes things work with the old column name and the new column name. [10:27] jml: Then the db patch doing the rename. Then a cleanup. [10:27] jml: Which all sucks. [10:27] Welcome to FDT. [10:27] ok. hmm. [10:27] Who asked for this again? [10:27] And oh, your patch will wait for a week because lifeless lives in 1990. [10:27] me. [10:29] I'm renaming Archive.commercial to Archive.suppress_subscription_notifications. Have done so in code, using a property. [10:29] StevenK: Aren't we almost up to date on DB deployments now [10:29] jml: There could be some magic we do with PG, but I think it will end up more complex. Might be necessary if making a Storm class not care which of two column names is in use is a pita. [10:29] ? [10:29] cjwatson: I'm just bitter. [10:30] cjwatson: It is usually pretty good, stuff has been held up due to 9.1 so we had a two week backlog [10:31] I know [10:31] (magic being an updatable view, making code use the updatable view, do the rename, make the code use the table again) [10:31] And that is counting the five or six DB patches Purple wants to land. [10:39] stub: Archive inherits from SQLBase and we don't ever query on this column, so I think I can hack something up in Python. [10:39] testing will be a pain though. [10:42] jml: Or if that code isn't used, extract it, do the rename, put it back + fix. [10:42] stub: it is used, I'm afraid. [10:46] jml: Just saw your email. Did you fix the PG port issue? [10:47] stub: manually. I changed the port again in postgresql.conf; restarted; ran createuser by hand. [10:47] jml: I think you might see that if /etc/postgresql/8.4/main/postgresql.conf still exists and specifies port=5432 [10:47] Can you check? Means there is another step after uninstalling the packages. [10:47] stub: make schema seemed to work OK, apart from the errors I mentioned earlier re plpython === Pendulum_ is now known as Pendulum === garyposter is now known as gary_poster [13:33] if a test is failing because it is escaping isolation due to a /+branch-id/ path being interpreted by bzr as being a filesystem path, does that mean that the test has to run with more of codehosting working? [13:34] or perhaps it needs more plugins loaded beforehand? [13:38] jml, do you know? [13:39] james_w: hi [13:39] james_w: plugins, I think. [13:41] thanks [13:51] hmm [13:53] thinking out loud about column renaming [13:54] if we want one code base to work with both column names, then we either need some way of interrogating the schema at class creation time or some way of trapping errors at query time and updating the class then [13:55] I'm probably missing other options, but those seem to be it for me [13:56] the problem with doing something on class creation is that it will add a query for everything that imports archive.py [13:56] which is probably everything [13:57] a problem with trapping at query time is that there's no obvious single choke point [13:58] Turn the attribute into a @property? [13:58] (This may not be the best approach, more an existence proof) [13:58] cjwatson: well, that's already been done [13:59] suppress_subscription_notifications = BoolCol( [13:59] dbName='commercial', notNull=True, [13:59] default=False) [13:59] in my branch [13:59] note that dbName is the old name. [14:00] but how do I make something that works both before and after the 'commercial' column gets renamed to 'suppress_...' [14:00] storm uses the dbName (somehow) to generate SQL [14:05] jml: There's probably no feasible way to handle that in the app. [14:05] jml: We've not done a column rename with fastdowntime before. [14:05] I would personally duplicate the column on the table and use triggers to keep it up to date while the application is flipped to use the new name. [14:06] Rather than trying to get Storm to cope, which it won't. [14:07] (unless you make suppress_subscription_notifications a property that executes manual SQL, rather than using Storm) [14:07] It's take 1 hour to unsubscribe ~registry for linux/unity bugs reported between 2005 and 2006. We may have another 6 hours of bug mail going to ~registry [14:07] (which might work in this case, since it's probably not used for filtering or anything) [14:07] wgrant: huh, you mean something like this: [14:07] @property [14:08] def suppress...(self): [14:08] try: [14:08] return do_query('SELECT suppress_... FROM Archive WHERE id = %s' % self.id) [14:08] except ProgrammingError: [14:08] return do_query('SELECT commercial FROM Arch...' % ...) [14:08] jml: Right [14:08] wgrant: interesting. [14:09] jml: I changed SourcePackageRelease.copyright to use a similar manual SQL approach for performance reasons. [14:09] But it could also work well here. [14:10] wgrant: I have looked pretty thoroughly, and it's not used for WHERE clauses to the best of my knowledge. [14:10] It obviously doesn't work if the column is involved deeply in queries, but since you're probably only getting/setting it it should be OK [14:10] Right, it wouldn't make much sense for it to be. [14:10] wgrant: I guess that will slow down code that uses the attribute, but maybe we can bear that for a little while. [14:10] jml: By 1 query for about 48 hours. [14:10] I think we can live. [14:12] wgrant: ok. I'll give that a shot. Thanks. [14:13] jml: Great. [14:13] * wgrant disappears [14:13] wgrant: nn [14:42] 15:39 < jcastro> czajkowski: do you remember the URL to add bug trackers to lp? [14:42] anyone help me out here? [14:42] gmb: ^^ [14:44] czajkowski, https://bugs.launchpad.net/bugs/bugtrackers/+newbugtracker [14:44] gmb: thanks [15:26] wgrant: SourcePackageRelease.copyright defaults to Python's copyright if it's not provided as a keyword argument, I think. [15:45] how do I get a cursor from a store object? [15:46] hmm. just cursor() [15:46] ok. [16:01] wgrant, stub: https://code.launchpad.net/~jml/launchpad/archive-commercial-rename-support/+merge/107819 [16:01] gmb also, ^^ (but I talked about it with them) [16:01] * jml has to dash. [16:01] jml, k, thx. [16:02] bye [16:02] ... can't believe I just did that. [16:02] tee hee === gmb changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: - | Firefighting: - | Critical bugs: 3.47*10^2 [18:13] morning [22:02] Anyone fancy giving me some light test education? http://paste.ubuntu.com/1013802/ [22:03] (AFAICT sourcepackage is a page such as https://launchpad.net/ubuntu/+source/mono) [22:10] Laney: sourcepackage is a DistributionSourcePackage [22:11] StevenK: yeah, I think that's what I linked to. [22:11] so, assuming that it's right, can I cure the ForbiddenAttribute? [22:11] then I'll get on to figuring out how to make it match the href :-) [22:12] (porting a doctest here) [22:12] StevenK: Um [22:12] StevenK: SourcePackage should be named Distro*Series*SourcePackage [22:13] Ah, but Distribution.getSourcePackage returns a DistributionSourcePackage, indeed. [22:15] In any case the forbidden attribute is probably because you're using LaunchpadFunctionalLayer (so including the security model) but your test doesn't log in. [22:16] The security model confuses me very much [22:16] is there any documentation on it? [22:16] Don't look at me, I zen it from the surroundings. [22:17] And occasionally whine in a confused manner. [22:17] Well [22:17] LaunchpadZopelessLayer avoids the security model for cases where it doesn't matter, but (a) I'm never quite sure how evil it is and (b) I don't know whether it'll work in this case [22:17] ForbiddenAttribute isn't caused by an incorrect layer [22:17] Unauthorized can be. [22:17] (Real LP devs will now tell me I'm on crack) [22:17] ForbiddenAttribute means you've used the wrong attribute name. [22:17] Laney: Distribution.name does not exist. [22:17] Ah, doh [22:18] Laney's test does not actually mention .name anywhere directly; presumably it's in some tested code [22:18] Sigh, I should learn to read. [22:18] Wait, Distribution.name *should* exist. It's on IDistributionPublic. [22:18] Probably trying to set it. [22:19] WHich isn't allowed. [22:19] If the attribute is entirely unsettable, setting it will ForbiddenAttribute. [22:19] it comes from the last line there [22:19] view() [22:19] That is, if there's no permission that can possibly be held to grant write rights. [22:19] Laney: What's the traceback? [22:19] You can figure out the security on any given attribute by looking at lib/lp/*/configure.zcml, sometimes in combination with lib/lp/security.py. [22:20] sec [22:20] wgrant: http://paste.ubuntu.com/1013837/ [22:20] 53 is the assertThat line [22:20] jml: That column workaround is disappointingly unevil. [22:26] My queue API branch is getting to the point where it might arguably almost work. [22:27] I might split it up, though, since some of the information about binary uploads is done by exporting bits of BPR and I suppose that might be contentious or something. [22:28] cjwatson: Well [22:28] cjwatson: Queue overrides are terrible. [22:29] The bit about exporting BPR isn't even so much for overrides, just for displaying information on builds. [22:30]