[00:16] lifeless: I'm pretty sure that 566339 is not fixed by the branch in question. [00:17] And 564491 needs somebody to drive DF. [00:17] wgrant: is it made worse? [00:17] No. [00:17] Just not fixed. [00:17] so remove qa-needstesting from it [00:17] for the other can you arrange that with StevenK ? [00:17] True. [00:17] Well, can somebody else remove qa-needstesting? It's not my bug, and I'm not really Soyuz, so I don't really want to screw with their QA :) [00:18] revno: 11556 [merge] [00:18] [r=stevenk, thumper][ui=none][bug=564491, [00:18] 566339] Fix an OOPS when an archive admin uses the +queue page to [00:18] accept uploads that close private bugs. [00:18] its qa-needs testing because of that landing [00:18] It is, yes. [00:18] oh, bigj [00:18] bigjools thinks it fixes it, but I'm pretty sure it's irrelevant. [00:18] sorry [00:19] you only filed it ;) [00:19] Yep. [00:27] sinzui: What's the UI for pages like +configure-translations going to look like? [00:27] a lot like bugs and answer [00:27] oh [00:28] Because the current one that almost made it out to production is very confusing :( [00:28] wgrant: what rev? [00:28] lifeless: The one I mentioned yesterday. r11552 [00:29] So not in your initial set. [00:29] wgrant: it is in the second set [00:29] Revision 11547 can be deployed: qa-ok [00:29] Revision 11548 can be deployed: qa-ok [00:29] Revision 11549 can be deployed: qa-ok [00:29] Revision 11550 can be deployed: qa-ok [00:29] Revision 11551 can be deployed: qa-ok [00:29] Revision 11552 can be deployed: qa-ok [00:29] Revision 11553 can be deployed: qa-ok [00:29] Revision 11554 can be deployed: qa-ok [00:29] Revision 11555 can be deployed: qa-ok [00:29] Revision 11556 can not be deployed: not-ok [00:29] This is what I was worried about. [00:30] * wgrant wonders why it isn't FF'd out. [00:30] wgrant: I explicitly asked about it [00:30] wgrant: and sinzui believes its an improvement even though not perfect [00:31] wgrant, the registry team is only worked on the unknown, not-applicable, launchpad enum. The translations team can consider doing a unified page that does everything in one view like bugs [00:33] sinzui: IMO the current translations/answers UI is a regression. [00:33] What does it even mean? [00:34] I don't know -- how will users? [00:34] wgrant: wgrant huh [00:34] wgrant: sample url? I don't recall it being confusing [00:34] lifeless: https://edge.launchpad.net/launchpad/+configure-translations [00:34] wgrant, I do not think so. consistent UI is better, being clear about what is being used or where something can be found is better. [00:35] sinzui: Why not FF it out for a couple more weeks, until the new pages are built? [00:35] And not roll out a half-finished awesome feature? [00:35] Awesome features are much better when people don't see broken implementations first. [00:35] wgrant, I do not understand. My entire team is working on this and we need edge feed back there are only a few things left to be done [00:36] I think there is plenty of time to complete this [00:36] sinzui: edge is fine. [00:36] I don't think lpnet is the best idea yet. [00:36] god knows spending a year on a so-called 6 month vision has burned me out [00:36] Heh. [00:38] My only regret is that I still think the DSP page is near useless and I want to be able to state where external questions and FAQs can be found [00:38] We need to work out a strategy for linking them better to projects. [00:39] Particularly since there are a lot of Canonical projects now that only want to use one bug tracker. [00:43] s/Canonical/Ubuntu ? [00:43] lifeless: Sometimes. [00:44] But if you categorise Ubuntu One as Ubuntu, I will not be amused :) [00:46] u1's ubuntu-only components are open source [00:46] lifeless, mind if I bug you with a question about bug 598289? [00:46] I think *if* a healthy set of packages in say fedora nd suse were to spring up, that they would see the need for 2 trackers for those components. [00:46] mars: shoot [00:46] https://bugs.edge.launchpad.net/launchpad-foundations/+bug/598289 [00:47] lifeless: They're not Ubuntu-only, but OK. [00:47] wgrant: the developers are only targeting ubuntu today; [00:47] unless something radical has changed [00:48] Yes, but there's no reason they can't run elsewhere. Like Software Center, and other stuff like that. [00:48] lifeless, So the question was: is it safe to use feature flags to protect access to profile generation. The answer was... yes, I think? zope-funcertainty [00:48] lack of zope-fu leads to zope-funcertainty [00:49] wgrant: thats true; the question is whether upstream get bug reports that aren't relevant to Ubuntu [00:49] wgrant: and until they are geting patches and so forth routinely, the broad answer is going to be 'no'. [00:49] True. [00:49] wgrant: which is why I talked about healthy packages in other distros [00:49] mars: certainly [00:50] mars: you'll need two patches. [00:50] mars: one patch to add a team membership scope [00:50] see rev 11536 for a very similar scope definition that does team membership scoping [00:51] bah [00:51] that does pageid scoping [00:51] and please add it to https://dev.launchpad.net/LEP/FeatureFlags at the bottom [00:51] mars: or fix the self-documenting bit [00:52] because team lookup is a db thing, we probably want a note that using that scope will trigger team membership checks, so we don't go overboard on it. [00:53] mars: secondly, you need a patch to force 'permit_profiling' or whatver it is on via a feature flag [00:53] mars: see e.g. the memcache feature flag patch for an example of what that might look like [00:53] these two patches are entirely unrelated to each other [00:53] but together much power can be had [00:54] If you don't want to hit the DB, you could use the pre-populated developer flag in ILaunchBag. [00:55] That's populated for every request. [00:55] thats true [00:55] we could have team: and special case well known names. [00:55] You could. [00:55] but [00:55] that shouldn't be needed if its populated sanely we'll get cache hits.. [00:56] you could also have a flag called user_is_developer [00:56] which is less general but fits the actual intent here a little more closely. [00:56] I'm easy. [00:56] bah [00:56] not flag [00:56] *scope* called user_is_developer [00:57] mars: I'd add the flag ('permit_profiling') or whatever name you choose, first. [00:57] mars: because onces thats added sysadmins can turn profiling on and off on staging without restarting it [00:57] cool [00:58] this is why I like pre-implementation calls :) [00:58] I think flag are still misunderstood [00:58] how so? [00:58] and what is the 'fix the self-documenting' bit? [00:59] (third discussion I've had today showing the difference between scopes/flags and how it hangs together. [00:59] Does the infrastructure support scope conjunction? [00:59] mars: no [00:59] for a given flag, the rules are evaluated in priority order [01:00] mars: self documenting: we should be able to query the code to find out what scopes(including pageid: magic ones) and flags (like memcache, gmaps2 etc) exist [01:00] this would let us: [01:00] - display that as help in the fictional editing UI [01:00] - not have a redundant list in the wiki page [01:01] find -iname '*py' -print0 | xargs -n0 grep getFeatureFlag | sort -u [01:03] mars: you'll miss all the page template ones [01:03] TAL flag isn't mentioned in the LEP [01:03] only in the listmail [01:04] I'm not sure there are ones deployed in tal yet, but there probably are and we should have some way of documenting them [01:04] mars: find might be the answer [01:04] find -iname '*py' -o -iname '*pt' -print0 | xargs -n0 grep -e 'getFeatureFlag' -e 'features/' | sort -u [01:04] yuck [01:04] do two find calls [01:05] add a 'make doc' target [01:05] and do the work in there [01:05] mars: what might be nice is a makefile rule to do it [01:05] along with the API docs [01:05] I'd like something out that the admin UI can include and show [01:06] mars: in templates getFeatureFlag isn't it [01:06] its view/flags/flagname [01:06] I think [01:06]

hello world!"

[01:06] ah yes [01:06] thats it [01:07] lifeless, so, the difference between flags and scopes: a flag only applies in-scope I assume? Sounds like a scope is a flag to turn on and of a set of flags [01:07] or override a flag [01:08] lifeless: Can I see OOPS-1721K2125? [01:08] so you could have feature:X scope:default :off [01:09] and feature:X scope:server.edge :on [01:09] Which saves you from having to write: [01:10] if feature:X and feature:server.lpnet then :on [01:10] err [01:10] if feature:X and feature:server.lpnet then ... [01:11] which is messy. You have to change the code if either of those two assumptions needs to change. [01:13] mars: code is controlled by flags [01:14] the value of a flag is determined by the highest priority rule with a matching scope [01:14] rules are a tuple (scope, flag, value) sorted by priority [01:20] mars: so yes, you have the point [01:20] we define a name for condition we want to control [01:20] and use rules to dynamically implement the particular flow control we want [01:25] \o/ I get yet another opportunity to use the awesome bzr pipeline plugin [01:26] mars: for this? [01:26] lifeless, three patches, so yes [01:26] mars: personally I wouldn't, they are totally separate patches [01:26] flag, scope [01:27] mars: unrelated [01:27] mars: thats the point really ;) [01:27] lifeless, well, you could add the flag in one patch, and not use it [01:28] then add the scope, and use it as well [01:28] mars: huh? once the flags added we can use it. [01:28] lifeless, I must be missing something: when I write getFeatureFlag(), I have to pass in a scope, no? [01:28] NO [01:28] you pass in a FLAG [01:28] so where did I get the wrong idea from... [01:29] reading over what I read again [01:33] Ok, so I thought I had read getFeatureFlag(flag, scope=foo) somewhere as the API. That is wrong, and I can not find the reference. Maybe I was confusing the "getFlag(scopes => None)" line from the LEP with the actual implementation. [01:35] heh [01:40] ok, this stuff needs a cheat sheet [01:41] wgrant: that oops [01:44] lifeless: That oops? [01:45] ywa [01:45] looks the same as the one in https://bugs.edge.launchpad.net/soyuz/+bug/276950 [01:46] Ah, yes. [01:46] In fact it's the same OOPS. [01:46] I was interested in the content. [01:46] Because I don't see what is so slow. [01:46] Unless it's the bad copy checker stuff. [01:46] its a different oops # [01:47] wgrant: the huge numbers of repeats [01:47] lifeless: It's the one in comment 3. [01:47] are at least 2.5 seconds [01:47] Ew. [01:47] Oh, there are details in the description. I see. [01:48] 600 bug lookups [01:48] 200 das lookups [01:48] 200 archive lookups [01:48] Ew. [01:48] its just horribly inefficient [01:49] I might be able to reproduce it locally, then. [01:50] if you can't, shout [01:50] the params aren't visible so I can't see what packages are being manipulated [01:50] I'd guess at something stupid like all-bugs-ever-on-the-package being manipulated [01:50] one at a time [01:52] with an added helping of 'cache miss due to complex query results in reading what we had already' or some such [01:53] komodo only found 5 occurrances of the string getFeatureFlag() in the devel source tree... odd. [01:54] looks a little few [01:54] but not far off [01:55] Martin's mail said to "use lp.services.features.per_thread.features to look things up" [01:55] so maybe people were doing that [01:55] no [01:55] because it has getFeatureFlag as the method [01:56] hmm [01:56] there are < 10 flags in use - I know of three in python code, and a couple in templates [01:56] lifeless, which three in Python? I can look up the exact string/symbol [01:56] getFeatureFlag [01:57] I documented them on https://dev.launchpad.net/LEP/FeatureFlags [01:57] 11519 claims to be behind a flag... [01:57] ah, like I said, my editor only found 5 occurrances of getFeatureFlag in the code base. Over to grep! [01:58] oh fugly [01:58] folk are using FeatureController and in_scope manually. [01:58] nooooo [01:58] see? Quickstart guides needed :) [01:58] Looks like rf-get did the wrong thing [01:59] and didn't update my source tree [02:00] eh? No - 'bzr switch -b' did something odd instead [02:01] mail sent to the list [02:02] ok, we need to fix up [02:02] distroseries.py [02:02] lifeless, did you work around that ec2 database issue you had earlier? [02:02] mars: no, I'm assuming ec2 AMI is fucked and won'y try yo land stuff till tuesday [02:03] ugh, sorry :( [02:03] and registry/browser/__init__.py [02:04] mars: possibly a postgresql dep change or something [02:05] lifeless, should feature flags return a boolean? Because the memcached one does "features.getFeatureFlag('memcache') != 'disabled'" [02:06] which feels a bit strange - what other values for the flag could there be? 'maybe'? [02:07] mars: they return the value [02:07] mars: look up the bug for timeouts via flags [02:07] I pasted a sample interpreting flag there [02:07] mars: other values are useful if we wanted to (for instance) keep memcache on a couple of pages but disable it globally [02:08] we'd set a priorty (say) 50 rule for (default, memcache, disabled) [02:08] and add priority 0, 1, 2 etc for [02:08] (pageid:pagetousememcacheon, memcache, '') [02:09] :/ [02:11] mars: __nonzero__ is perfectly cromulent python [02:11] as long as it doesn't have unusual consequences (like being terribly slow, or some such) [02:12] lifeless, comparison to the string 'disabled' for a flag is a poor API imho. featureIsEnabled('foo') => returns Boolean True or False, sounds like it would cover 99% of the cases [02:14] with this code, bool(getFeatureFlag('memcache')) == True, even when the memcache flag's value is 'disabled' [02:15] that's just not right [02:15] anyway, I need to run, have to put the kids to bed. ttyl [02:17] mars: ttyl [02:17] mars: I think bool would be overly restrictive for no benefit [02:17] mars: this is schemaless. Its a *benefit*. [02:23] Hard / Soft Page ID [02:23] 164 / 335 BugTask:+index [02:23] 107 / 277 CodeImportSchedulerApplication:CodeImportSchedulerAPI [02:23] 70 / 16 Distribution:+search [02:23] 39 / 18 DistributionSourcePackage:+filebug [02:23] 16 / 0 Product:+bugs [02:23] 14 / 69 Distribution:+bugtarget-portlet-bugfilters-stats [02:23] 14 / 18 DistroSeries:+queue [02:23] 13 / 43 Archive:+packages [02:23] 13 / 4 Archive:EntryResource:syncSource [02:23] 8 / 4 Person:+rdf [02:30] mars: I've tweaked the LEP [02:30] lifeless, by schemaless, do you mean key/value pair storage? Don't we get that already from lazr.config and from memcache? [02:36] mars: lazr.config is horrific [02:36] mars: and memcache is transient [02:36] mars: by schemaless I mean no defined schema [02:37] this is more than just keyvalue (but not a great deal more) [02:37] lifeless, ok, put another way: is there a problem with: [02:38] if getFeatureFlag('foo'): [02:38] bar = config.feature.value [02:38] yes [02:38] it will take hours to change. [02:38] it will be static [02:38] right [02:39] we'd need a new config.feature section for every rule in the DB [02:39] and rules are dynamic, so I don't see how we'd do that [02:39] ok [02:40] lazr.config is a very powerful, very complex, schemad system which pushed all integrity checking out to the setting-of-it [02:40] this has the following consequence: [02:40] - its hard to use [02:40] - and complex to use. [02:40] we're now, judging from what people are doing, abandoning it rapidly. [02:41] I wouldn't be surprised if folk route around it completely in the next 12 months and we have a bug called 'remove the last dependencies on it' [02:42] lifeless, so lazr.config stores data - what are they using instead? [02:42] stub has added ini files [02:42] poolie added feature flags [02:43] stub is looking with interest at zookeepr [02:43] who knows what else will turn up. [02:44] lazr.config is also very intolerant, making it hard to evolve. [02:44] From what I've seen of it, I'd be entirely happy to just switch to configparser now [02:45] but perhaps I've missed something. [02:45] (but even doing that owuldn't do what feature flags do). [02:45] feature flags are: very powerful, very simple, very dynamic. [02:46] lifeless, ok, then I would suggest a convention: 'on' or None [02:46] we can do A/B testing; dynamic configuration and more [02:46] mars: Why? [02:46] mars: what do we gain by limiting it [02:46] lifeless, because then bool(getFeatureFlag()) still works everywhere [02:46] mars: what value does that *have* [02:47] lifeless, bool('on') == True, bool(None) == False. That makes sense. The memcached flag (and it is a Bool flag right now) doesn't follow that convention. [02:48] because that convention doesn't exist. [02:48] And I don't see what value we'd get from it. [02:48] We'd destroy the usefulness of flags. [02:48] how? [02:48] for a consistency we don't need. [02:48] I am not saying "only use 'on' or None" [02:48] mars: did you look at the bug I suggested you do, about pageid based timeout control? [02:49] I am saying "if it is a flag, try to stick to 'on' or None [02:49] if it is not a flag, but a keyvalue, then you're on your own :) [02:49] its not a flag, its not a keyvalue [02:51] lifeless, I saw the example you pasted in the channel, but not a bug # [02:51] mars: look at launchpad-foundations bug [02:56] found it, https://bugs.edge.launchpad.net/launchpad-foundations/+bug/627701 [03:18] I think my weekend plans just went tits up. Has anyone noticed that ec2 images use PG 8.3, but that our own scripts to make the environment are 8,4? [03:19] lifeless, ^ that might explain the error you saw [03:19] * sinzui thinks [03:19] did someone land a fix to the scripts to get Maverick running? [03:20] I do not know I think there is just one real error in setup: /etc/init.d/postgresql: command not found [03:20] I can update my branch to try 8.4 then fallback to 8.3 [03:22] ah, we have two script and the image does not know which one to use. This may be a one line fix [03:23] sinzui, yep: http://bazaar.launchpad.net/~launchpad-pqm/launchpad/devel/revision/11570 [03:24] sinzui, consider switching to the /usr/sbin/service utility [03:24] then it will always do the right thing [03:24] okay thanks for the advice. [03:47] How will that work with version selection, though? [04:03] wgrant: no idea [04:03] wgrant, I see I can pass the version to service, but I do not know if service supports version in Hardy. I assume there are still some machines on Hardy [04:04] we stil have dc machines on hardy [04:04] but I don't know that we run any developer pg controlling scripts on them [04:04] take the safe path by adding a switch between the old command and the new one [04:07] james_w: http://pypi.python.org/pypi/fixtures updated [04:22] wgrant, lifeless. My pg hack played on my maverick instance and in ec2. If my branch passes, we will have a fix in the tree [04:30] sinzui: thanks [04:30] we'll need that CP'd to production-devel [04:30] by hand [04:31] or is it the local content on disk when you submit that controls this? [04:32] Why does production need it? [04:33] Isn't production still 8.3, so EC2 should probably use 8.3 too? [04:34] wgrant: we need to keep running ec2 land for prod-devel branches [04:35] do you have reason to think that that isn't affected in the same way? [04:35] True. [04:38] one way to make sure its not affected is to run pg 8.3 and only 8.3 on it and make the dev scripts do stuff for 8.3; I don't know if thats the case. [04:38] its also of minimal use because production doesn't take db patches [04:38] it hatses them [04:44] Although there was a bit of specialness WRT that this cycle... [04:46] this is not the patch you are looking for [05:17] Hm. [05:17] The Django Debug Toolbar is handy. [05:19] wgrant: how are you going with the +queue perf thing [05:19] lifeless: Doing some uni stuff first. +queue is being attacked this evening. [05:20] wgrant: K, I'll probably be around if you want to bounce ideas [05:21] lifeless: I was just adding an "assuming that maverick continues to be well-behaved"... when I got my first X hang in a few months. [05:22] Ahem. [05:23] :) [05:37] lifeless: Do you know if there's a known issue with the revision karma allocator? [05:40] I've checked a few users, and nobody seems to have any in the last couple of days. [05:40] Hm, there was some two days ago. [05:40] So maybe it just didn't run yesterday. [05:42] not aware of any incident report about it [05:50] * StevenK kicks Maven until pieces fall off [08:16] anyone know the list that zope testing patches and plans are designed on? [09:21] wgrant: https://bugs.edge.launchpad.net/soyuz/+bug/629921 -please- [09:21] I'm begging you. [09:23] Hmm. That was filed two weeks ago, and we are around 12000 bugs further along now... does that mean we are having 1000 filed a day!? [09:24] But anyway, let's see how difficult that is... [09:24] Ahh, I see. [09:25] lifeless: Am I allowed to violate the no-implicit-truth guideline? That would reduce the fix to deletion of a few characters... [09:26] wgrant: no implicit truth? [09:27] lifeless: It's suggested not to say 'if whatever' -- instead we are told to say 'if whatever is not None' [09:28] This particular case is currently 'if requested_name_filter is not None', but actually wants to be 'if requested_name_filter is not None and len(requested_name) > 0'. But 'if requested_name_filter' would work just as well. [09:28] if requested_name_filter is clearer IMO [09:28] doit. [09:28] Excellent. [09:29] given I just wrote email to the list suggesting *just that* for flags earlier today :) [09:48] * wgrant fixes the Maverick psycopg2 compatibility breakage. [09:49] its compatible; honest. [09:49] It is, but our celebrity code sucks. [10:06] This does not inspire confidence: [10:06] def source_fies_dict(self, package_upload_source_dict, source_files): [10:06] """Return a dictionary of source files keyed on PackageUpload ID.""" [10:08] WTF [10:08] WTF [10:08] WTF [10:09] lifeless: So... DistroSeries:+queue redirects after post. [10:09] lifeless: I was looking around to see how the action handler was called. [10:09] It turns out it's called by the template. [10:09] After it calculates all the data to display in the event of a GET. [10:09] Gar. [10:10] wgrant: hah [10:10] I think I speculated on that somewhere. [10:11] like in the bug or something [10:11] Heh, so you did. [10:11] I knew it redirected, so I didn't suspect this... but I went digging, and couldn't find anything calling the handler. [10:12] Sigh. [10:12] I'm glad you've confirmed it [10:12] it suggests an immediate obvious improvement ;) [10:12] Heh, yes. [10:12] Now to work out how to make it not do that. [10:12] is it a systemic thing [10:12] that is, do all our template render-the-GET< process-the-POST ? [10:12] No. [10:13] This one isn't a real form. [10:13] whew [10:13] So it's speical. [10:13]