[00:05] wgrant: http://pastebin.ubuntu.com/673437/ === StevenK changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: StevenK | Critical bugs: 238 - 0:[#######=]:256 [00:28] https://answers.launchpad.net/launchpad/+question/158269 [00:28] Intriguing. [00:29] People really seem to care about revision numbers matching. [00:30] yes [00:30] we could make bzr handle sparse mainlines ;) [00:30] You are a bad person. [00:31] Still. [00:31] At least not as bad as whoever designed Mercurial revnos. [00:31] I am an awesome person [00:31] solving the world, one line at a time. [00:31] Compared to whoever designed Mercurial revnos, yes. [00:31] Do you know how they work? [00:31] They are really, really useless. [00:31] yes [00:31] they are perfectly useful for the purpose designed [00:32] just extremely local in scsope [00:32] (and sparse on any given branch) [00:39] Grarrr. [00:41] >>> u.newPackagesetUploader(packageset=lp.packagesets.getByName(name='zope', distroseries='/ubuntu/oneiric'), person=lp.people['wgrant']) [00:41] [00:41] newPackagesetUploader is in IArchiveView, so we are qa-bad. [00:41] Yay. [00:41] We will be doing a 60-70 rev deploy tomorrow :D [00:41] oh, assign-me-permissions to anhyone ? [00:41] Yeah. [00:41] win [00:44] This deploy will also have some really risky stuff in it. [00:44] * wgrant dies. [00:44] :( [00:44] anything I can do to help? [00:44] No. [00:45] Apart from declaring devel frozen. [00:45] production is now 57 revisions behind tip :/ [00:45] I guess we should deploy the first 10 that we can do today. [00:47] if its qa ok yeth [00:47] Hm? [00:47] I agree conditional on that being qa-ok :) [00:47] Heh. [00:48] We have 13 deployable. [00:48] Let's do that. [00:54] wallyworld: (Wow, no underscores) WRT https://code.launchpad.net/~wallyworld/launchpad/rename-private-team-795771/+merge/72516 , does that mean that private teams can be renamed or this is just UI gubbins? [01:07] StevenK: just ui [01:08] wallyworld: So you can't use the UI to perform a rename? [01:08] StevenK: you can once the branch lands [01:08] for private teams [01:09] whereas before private teams you couldn't [01:09] wallyworld: Then I'd like a test that actually *tests* that. At the moment your test only checks the widgets on the +edit page [01:09] And the error conditions [01:09] StevenK: that's all the doc tests that were replaced by unit tests ecer did [01:10] the testing checks that the name field is or is not readonly in the right places [01:11] ie it is read only for those situations where the team cannot be renamed [01:11] from there on, when the user hits Save, it's just a normal domain model update operation [01:12] wallyworld: Even so, you (correctly) removed a test that private teams can not be renamed -- I'd still like to see an end-to-end test where you initialize the +edit view, submit it and assert the team name has changed. [01:13] but if the name field is not display only, then existing tests will cover that [01:13] scenario [01:13] all we need to test here is whether the name field is display only or not [01:14] since that's what the previous functionality did for private teams [01:14] seems like we would be duplicating tests [01:15] wallyworld: No, existing tests will not. Since the code was previously written to forbid renaming private teams, I find it highly suspect that there is another test for what I'm asking. [01:16] so you think there is an existng end-end test for private team renaming? there's nothing at the db layer to prevent private teams being renamed (there is for the mailing list restriction though) [01:17] the code that was previously written to forbid private team renaming was just inthe view and alit did was disable the name field [01:18] wallyworld: No, I think there is *not* an existing end-to-end test for private team renaming. [01:19] If you're removing a special case like this, it's probably not immensely beneficial to test the special case that you're removing. [01:19] I'm concerned that something else will pop up during QA [01:20] i can add an end-end test if you insist but it will be a brand new test - afaik we don't normally test simple crud stuff in the view tests? [01:23] so it wouldn't be a view test but a test in the model layer? [01:24] view tests normally just check that the view has rendered correctly and the actions have been wired up? [01:25] anyone up for a simple review ? https://code.launchpad.net/~lifeless/python-oops-wsgi/foru1/+merge/72640 [01:26] * wallyworld has to go pick up kid from school for orthodontist appointment $$$$ :-( [01:29] StevenK: ^ [01:29] Yes, yes, I'm already looking [01:29] thanks [01:29] I couldn't tell [01:30] I'd be worried if you could [01:32] lifeless: So why is there a lone '\' on line 284 of the diff? [01:39] StevenK: that was a test. You pass. [01:41] StevenK: (deleting it ;)) [01:46] hah [01:46] StevenK: I forgot one commit [01:46] to wire it all up ><. Doing that now [01:57] StevenK: rev 18 pushed - should be able to look at it incrementally anytime now [01:59] lifeless: r=me [02:00] I already voted as approve, so meh [02:00] StevenK: thanks! [02:02] jamesh: python-oops-wsgi 0.0.3 (in trunk now) should have everything [02:10] lifeless: looks good. Have you guys looked at things like SQL statement logging yet? (or any other data collection mechanism) [02:10] jamesh: yes, thats handled via the on_create hooks [02:11] adding something to the WSGI environment that could be used to log additional info during the request processing stage could be useful [02:11] jamesh: yes. Another way is thread locals, which is how I'd expect storm glue to be done, given storms global variable tracer implementation [02:12] jamesh: FWIW lp is now running with on_create hooks for its sql gathering [02:12] cool. [02:12] there was code for that in wsgi-oops, but it probably doesn't make sense to put storm-specific stuff in python-oops [02:13] so tracer -> request_timeline and in the on_create there is an attach_timeline which grabs the request_timeline (which ends up being a TLS thing) and snarfs it out into th db_statements variable [02:13] jamesh: right, that was one of the headaches I had when considering generalising wsgi-oops [02:16] jamesh: tarballs of 0.0.3 in lp/pypi and the lp download cache (in a minute) [02:17] and soon in the Ubuntu archive? :) [02:17] in freeze at the moment [02:18] hopefully one this stabilises a little (e.g. some of the key names are noddy) I can talk to pitti about adsorbing some of apport [02:22] jamesh: so for instance, to do the 404 thing: [02:22] set oops_on_status=['404'] in make_app [02:22] and separate [02:22] I read the diff. What you've added looks pretty good. [02:23] add to config.filters a callback to decide which 404's you trapped you actually want. [02:23] that callback might be generic enough with currying that we can put it in -wsgi [02:23] or even (for twisted & zope environments) a common oops-http or even oops itself. [02:24] lifeless: for the WSGI case, one issue is that I might not know what oops config the middleware is using. That is why I was suggesting putting something in the wsgi environment [02:24] jamesh: I don't quite follow how you wouldn't know [02:25] do you suggest storing the config in a global variable somewhere? [02:26] jamesh: perhaps. What bit of code wants to know the config ? [02:26] * jamesh should probably read through the rest of the python-oops code [02:28] http://bazaar.launchpad.net/~canonical-launchpad-branches/python-oops/trunk/view/head:/oops/createhooks.py is the default hooks [02:28] and [02:28] http://bazaar.launchpad.net/~canonical-launchpad-branches/python-oops/trunk/view/head:/oops/config.py [02:28] is the core event notifier [02:30] lifeless: so for example, lets say I want to add the Django user name to my OOPS report. I create a special Django middleware class to run after the auth middleware and read request.user [02:31] where should I stash that so that it makes it into the OOPS report created by python-oops-wsgi? [02:32] ok, so we'd want a on_create hook to grab the data from the context [02:32] stashing the data in the environment would be the easiest thing there [02:32] as an example, if python-oops-wsgi added a "oops.data" value to the wsgi environment as a dictionary that would be merged into the new OOPS, it could be as simple as: request.environ['oops.data']['username'] = request.user.username [02:33] def attach_username(report, context): report['username'] = context.get(wsgi_environ, {}).get('django_user', None) [02:33] I suppose I could do my own hook for that, but it seems like it could be a useful addition as a general purpose way of collecting data. [02:34] sure, i can see that [02:34] with the current rfc822 serializer we're quite limited [02:34] I really want to nuke that, hopefully oops-tools will be using the interface soon and then we're very close to migrating away [02:34] def attach_data(report, context): report.update(context.get(wsgi_environ, {}).get('oops.data', {})) :-) [02:35] I understand that. I'm just saying that the wsgi environment is pretty easily available in Django (and is definitely available if you're doing a plain wsgi app) [02:36] rather than doing N different collection methods, we might be able to provide one that can be used for many simple cases. [02:38] jamesh: yes, I'm in favour of that. [02:38] I would be able to rewrite the hacky traceback capture code U1 is using at the moment to work with that kind of API [02:38] to work without patched Django [02:39] request.environ['oops.data']['exc_info'] = exc_info [02:39] we need to decide if it goes into the oops context, or the report directly. === jtv-afk is now known as jtv [02:39] that exc_info example would make sense for going into the context. === jtv changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: StevenK, jtv | Critical bugs: 238 - 0:[#######=]:256 [02:40] pehraps two variables [02:40] oops.report oops.context [02:40] include a default hook in install_hooks to copy oops.report verbatim [02:40] and do an update of the context from oops.context. [02:41] note that the rfc822 serializer drops anything unknown on the floor :( so need to work within that for the mean time - but you probably already are. [02:41] again, I haven't read all of your new oops code yet. I'd probably be a bit more coherent if I had [02:41] sure [02:42] you're making plenty of sense. [02:43] While arbitrary data collection would be nice (and hopefully we can support it with a new report format), I'd be happy with a simple way to collect the standard data from deep in the request [02:43] yah [02:43] I need to context swtich for a bit [02:43] care to file a bug? [02:43] sure. [03:08] lifeless: filed: https://bugs.launchpad.net/python-oops-wsgi/+bug/832477 [03:08] <_mup_> Bug #832477: Provide a simple way to capture oops info from deep in the request < https://launchpad.net/bugs/832477 > [03:11] thanks [03:49] StevenK: can I bug you for a follow-on ? https://code.launchpad.net/~lifeless/python-oops-wsgi/foru1/+merge/72649 [03:55] Project db-devel build #819: STILL FAILING in 5 hr 55 min: https://lpci.wedontsleep.org/job/db-devel/819/ [04:07] hm, when is next push to production? [04:08] nigelb: wgrant will laugh at you shortly [04:08] nigelb: in seriousness, i think there are quite a stack of revisions due to be deployed [04:08] s/laugh/sob/ [04:09] There's around 60 revisions waiting to be deployed. [04:09] mwhudson: hehe, oh QAblocking? [04:09] Currently held up by the buildbot slaves being contaminated by some nasty substances. [04:09] wow [04:10] wgrant: thats worth noting to the list I think [04:10] lifeless: I was hoping it would be resolved before EOD. [04:10] But if it is not, I might notify the list. [04:11] even if it is [04:11] Ok, timeto find another bug to work on. [04:11] helps folk be aware of what caused grief/headaches [04:11] doesn't need to be a -long- message ;) [04:12] Subject: Deployment blocked by something nasty on buildbot slaves [04:12] :D [04:22] Project devel build #988: STILL FAILING in 5 hr 33 min: https://lpci.wedontsleep.org/job/devel/988/ [04:25] so [04:26] txtacfixture [04:26] mwhudson: ping [04:26] mwhudson: how do you spell 'pick an ephemeral port' for .tac files [04:27] lifeless: specify 0 as the port number maybe? not sure though [04:27] so [04:27] and this will make folk weep [04:27] in the config we specify 0 as the port [04:27] start the service [04:27] then rewrite the config with the real port. Win! [04:28] + parser to get the ports out of the readyservice handler I guess. [04:39] StevenK: hi [04:39] lifeless: Hm? [04:39] Slightly off-topic, but every time I try to subscribe someone to a bug [04:39] report, the first time fails with "() Couldn't get subscriber details [04:39] bah [04:39] 15:49 < lifeless> StevenK: can I bug you for a follow-on ? https://code.launchpad.net/~lifeless/python-oops-wsgi/foru1/+merge/72649 [04:39] lifeless: No. You've reached your branch limit. [04:40] 1 is the limit ? [04:40] I didn't specify the duration [04:41] heh. So are you serious or teasing? [04:41] Seriously teasing? :-P [04:42] lifeless: No real comments, r=me [04:42] thanks! [04:43] * StevenK grumbles at Jenkins faulures [04:43] jamesh: ok, so thats in place. I'm going to hold off releasing for a day or two, let you see what else is needed. [04:44] lifeless: Oh, did you bump the version, or no point? [04:45] StevenK: I didn't [04:45] StevenK: waiting for more feedback from jamesh :) [04:45] Ah [04:46] StevenK: That's interesting. The hg equivalent of those failed with a timeout or something on buildbot this morning. [04:51] Interesting [04:51] I suspect the only failures we're left with are races or other buggy stuff [04:55] Yeah. [05:39] wallyworld: you're approved. [05:39] jtv: excellent, thanks [05:40] StevenK: very simple review? https://code.launchpad.net/~jtv/launchpad/fix-more-utilities/+merge/72666 [05:42] jtv: I'm concerned about the clonePackages() change [05:42] Good. [05:43] StevenK: but taunts aside (sorry), what's the problem with it? [05:44] jtv: It seems I had misremembered -- r=me [05:44] jtv: Please don't land it yet [05:45] Thanks. You were right to be concerned about it, but I think the change I made here isn't the problem. [05:46] Why not land it? [05:47] jtv: Because we have a large amount of revisions to deploy, and wgrant and I would prefer people consider devel "soft-frozen" so we can get into a deployable state. [05:47] Fair enough. [05:48] Let's not try to break the record of 4 interlocked reversions. [05:48] so I shouldn't r=me the upgrade to wsgi-oops 0.0.3 ? [05:48] Or was it 5. [05:48] bah oops-wsgi [05:48] cause you know you want me to [05:48] yes, after all that testing you did on loggerhead :P [05:49] something like that [05:50] wgrant: I think 4 === StevenK changed the topic of #launchpad-dev to: devel "soft-frozen" until we're deployable | https://dev.launchpad.net/ | On call reviewer: StevenK, jtv | Critical bugs: 238 - 0:[#######=]:256 [06:23] wgrant: It looks like DSP.latest_overall_component can die horribly [06:27] Hmmm, that is a surprise. [06:27] That sort of thing is still used a bit, but apparently not that one. [06:28] +0/-8 makes me a little happy [06:34] jtv: Tiny MP for you: https://code.launchpad.net/~stevenk/launchpad/lose-dsp-loc/+merge/72676 [06:36] wgrant: That also removes components from DSP [06:39] StevenK: Ah, most stuff uses latest_overall_publication.component. [06:39] Amusing. [06:39] Haha [06:47] so it turns out [06:47] our keyserver implementation is much more than we use [06:47] it has a human play-with-me interface [06:47] which I doubt anyone has used in 3 years [06:48] jamesh: I dunno if you've had a chance to look at python-oops-wsgi tip, but if you have feedback is solicted ;) [06:49] lifeless: I looked at the merge proposal you attached to the bug report, and it looked pretty good. [06:51] cool === almaisan-away is now known as al-maisan [06:51] jamesh: that djano specific bug, I'm not sure its actionable in python-oops-wsgi, is it ? [06:52] like, if upstream reject it and the log hacking route is still needed, that isn't a generic wsgi glue, its django specific [06:52] lifeless: probably not, after having investigated it. [06:52] how would you feel about marking it invalid in oops-wsgi? [06:53] lifeless: and I think we could do equivalent hacks to what we currently do using the oops.report/oops.context extensions. [06:53] sure. [06:53] unless you want a tracking bug for "python-oops-wsgi should work with django apps" [06:53] I'll leave that up to you though. [06:53] i think it can do that with the upstream task [06:54] uhm [06:55] yeah [06:55] actioned [06:56] jamesh: thanks very much for kicking the tires on this beast === stub1 is now known as stub === al-maisan is now known as almaisan-away [07:31] StevenK: I'm looking at your review—sorry, everybody bothering me at once. [07:32] StevenK: done === danilo_ is now known as danilos === stub1 is now known as stub [07:50] good morning [08:03] hi adeuring [08:03] hi jtv! [08:20] Morning [08:20] Good Morning! [08:25] adeuring, you may be the best person to deal with this: my experimental interfaces checker found one or two small mismatches in HWDB utilities. [08:25] g22 [08:25] gah [08:30] ok, calling it. see you all tomorrow [08:31] wgranted (n): to roll back a bad commit [08:31] adeuring: canonical.launchpad.systemhomes.HWDBApplication does not fully implement lp.hardwaredb.interfaces.hwdb.IHWDBApplication: the title attribute was not provided. [08:31] jtv: argh [08:32] bigjools: (vt) surely [08:32] It's not serious, but it is the sort of thing I want to automate checking against. [08:32] right, i can look into this [08:32] (ahem, I'm talking to adeuring here) [08:32] lifeless: it's too early to think about verbs and nouns and I've only had one sip of coffee [08:32] early hah! [08:33] we can talk early at the TL meeting :) [08:33] adeuring: thanks. The other one was HWSubmissionDeviceSet.create having too many parameters, but if that was a "self" then I've got a fix in EC2. [08:33] 2 hours kip all night, feels like I've not actually been to bed :) [08:33] stub: any word on pgbouncer? [08:33] bigjools: ugh [08:33] bigjools: kids ? [08:33] I wish it were but no [08:34] medical issues ... [08:34] :( [08:34] lifeless: same same. Waiting on the config file rollout, which is really frustrating when I keep seeing these NDT rollouts happening. [08:34] lifeless: Think we need to bump the priority to 90? [08:34] I'm not quite there yet. [08:34] I will ping mthaddon now - he's said its getting worked on this week [08:35] There's still a lot of puppet breakage right now. [08:35] Several hosts failed to roll out. [08:36] :( [08:36] not unexpected tho [08:36] No. [08:36] yeah, there's a puppet branch in progress that should fix it [08:36] Great, thanks. [08:37] Then i guess I should RT the rest of the cronspam. === mpt_ is now known as mpt === almaisan-away is now known as al-maisan [09:30] wgrant: buildbot done, waiting for asuka [09:48] Project devel build #989: STILL FAILING in 5 hr 26 min: https://lpci.wedontsleep.org/job/devel/989/ [10:13] mrevell: i'm glad the bugs page is going to get some love [10:14] i wonder if some of the ones you're retagging may be better off with a task-oriented view rather than getting the user to customize them though [10:23] poolie, Hey there, at the moment I'm tagging anything that could fall in the scope of our "customisable bug search columns" project, which is coming up next. I'm not sure what you mean by a "task-oriented view". Could you help me understand,please? [10:24] sure [10:25] for instance, if i ask about inprogress bugs, the assignee is likely to be relevant [10:25] if i ask about new bugs, the heat is more likely to be relevant [10:25] if i ask for bugs sorted by #affected, the #affected is probably interesting [10:25] Oh, yes, I see. [10:25] i don't know if this actually works but istm it is at least indicated by some of the bugs and comments [10:26] it's a chance to do better than the typical nerdy "you can configure every possible query, why aren't you happy" :) [10:26] it may be too hard to predict what people want to see [10:27] Heh. I like the idea. I think that some user research could come up with some pointers towards what people want to see. [10:27] I was the QA CoP sprint for a day last week and it seems they have some scenarios where they need particular info; they've partially solved that with a Greasemonkey script so far [10:28] were it me, I'd still try to provide the 'customizable columns on advanced search' feature first [10:28] oh [10:28] 'by milestone' is especially poignant because it doesn't show you the milestones [10:30] perhaps adding the generic thing first is better, it's just an idea i was reminded of seeing them go past [10:30] jml, Yeah, agreed, poolie's idea seems to me to build on that. A set of canned customisations, almost ... for want of a better way of describing it. [10:30] configuration plus sensibel defaults [10:31] right [10:32] fwiw, the reason it's been scheduled next is because it was requested by a bunch of stakeholders. it's also the first feature we've had in the feature queue that isn't massive. [10:34] for my part, more than privacy or derived distros or whatever, I think that each week spent on it is another week without, say, a combined issue tracker. [10:34] but maybe it's better to improve the default bug searches [10:34] Mmmm, the last two features didn't need to be massive. [10:34] not my place to say, and I don't really have an thought out opinion. [10:34] They could have been split into several each. [10:35] But I guess they'd been hanging around for three years and needed to be dealt with :/ [10:35] At least they're nearly done now. [10:35] wgrant: basically, I agree with what you just said. :) [10:35] privacy seems not nearly done, but maybe that's just because project pages aren't being updated. [10:35] It's barely started. [10:36] But DDs is nearly done sort of maybe. [10:48] 80% done, 80% to go? [10:48] poolie: before I forget, are you happy for me to land the import of the other three tools from ubuntu-dev-tools in lptools as well? [10:48] yes, thanks! [10:49] i wonder if it ought to be mentioned on u-devel or something [10:49] or perhaps it can be just handled through dpkg dependencies [10:49] to make sure people don't see those tools just disappear [10:50] poolie: We were going to handle it through dependencies (add a Recommends: lptools) [10:50] wfm, thanks [10:50] we'll need a new upload of lptools to Debian as well though (ubuntu-dev-tools is in Debian) [10:50] * jelmer looks at lifeless [10:50] don't rely on mail to humans to do what machines can d [10:51] can i persuade someone to make a release of lp:restfulclient to get the fix for bug 626960 out? [10:51] <_mup_> Bug #626960: Collection dictionary access incorrectly folds all HTTP errors to KeyError < https://launchpad.net/bugs/626960 > === matsubara-afk is now known as matsubara [11:05] what am I doing wrong to get http://paste.ubuntu.com/673718/ from buildmailman.py? [11:05] lib/lp/services/mailman/monkeypatches/defaults.py exists; lib/canonical/launchpad/mailman/monkeypatches/defaults.py doesn't [11:06] cjwatson: you might need to make clean;make [11:06] trying that, thanks [11:06] that's guess, don't shoot me if it doesn't work :) [11:06] * cjwatson nods [11:07] shoot barry instead! [11:09] bigjools, hi :-) [11:10] matsubara: ok so ""Parent series has sources waiting in its upload queues that match your selection, see help text for more information." means that there's stuff in +queue [11:10] the help text doesn't exist - yet :) [11:10] bigjools, right. it's in my notes to file a bug about that help text [11:11] matsubara: we recently added extra checks to make sure that you cannot init from a series that has builds or binaries in +queue that match the sources you're copying [11:11] bigjools, is there any way for me to cancel whatever is in +queue so I can play with the initialization? [11:11] bigjools: excellent, thanks, that seems to have worked [11:11] matsubara: check with rvba, he's doing some QA. Otherwise, pick different packagesets. [11:11] ah, so if I choose different sources that those in the +queue, I should be able to initialize? [11:11] cjwatson: the Windows approach..... [11:11] good. thanks! [11:14] bigjools, I don't see anything here: https://qastaging.launchpad.net/testbuntu/foo/+queue. is it another +queue page? [11:15] I tried all the filters and they return nothing [11:17] jelmer: great stuff btw. [11:27] matsubara: it's the parent's queue [11:30] bigjools, ah right. I tried with natty as the parent and the series initialized successfully. thanks! [11:30] matsubara: do you understand what is happening? [11:31] bigjools, I understood the check, don't know why it's in place though. [11:32] matsubara: that's what we'll explain in the help text :) [11:32] if the parent has the same packagesets queued, it won't allow derived ones to be initialized with the same packages. is that right? [11:32] ah, cool! [11:33] matsubara: it's packages that matter, not packagesets [11:33] but basically if there's a pending build upload or a build in progress, you can't inherit it [11:34] unfortunately I can't see https://qastaging.launchpad.net/ubuntu/oneiric/+queue as it's timing out. [11:34] yes that page is a PITA [11:35] right. is that because would be a waste of resources to initialize from a old, previously built for that parent series, given that we know there's a new build in progress? [11:36] matsubara: not exactly - the problem is that you get complications later because you built your own binaries that will differ from the ones the parent compiled [11:37] bigjools, ok. and could we not schedule the initialization to fire automatically after the in progress ones are finished? or this adds too much complexity to the use case? [11:40] matsubara: way too much :) [11:40] jml: thanks, which stuff? :) [11:40] bigjools, :-) [11:41] jelmer: getting rid of lptools stuff from ubuntu-dev-tools [11:41] jml: ah === henninge is now known as henninge-lunch [11:49] rvba: Will you be able to QA bug #826870 today? [11:49] <_mup_> Bug #826870: native syncs from Debian non-free default to Ubuntu universe < https://launchpad.net/bugs/826870 > [11:50] We reverted bzr to 2.3 ? Looking at the merge conflict stable -> db-devel [11:50] stub: I believe 2.4 is still only in db-devel. [11:50] For testing on staging. [11:50] stub: no, but db-devel has 2.4 for testing [11:50] jelmer: ^^ [11:50] Heh. [11:50] Ta. I'll keep db-devel on 2.4 then. [11:51] wgrant: for that I would need my changes to make it into db-devel (I need to test that on DF) [11:51] rvba: stub should be arranging that as we speak. [11:51] rvba: But you can always merge devel on DF. [11:52] is there any news on pgbouncer and when staging will be updated? [11:52] jelmer: The next staging update is meant to work. [11:52] If it hasn't disabled itself due to failures... [11:52] Somewhat concerningly, it stopped logging 9 days ago. [11:52] stub: Do you know what's going on there? [11:53] stub: Doesn't look like it's going to retry, so the sudo access isn't much good. [11:53] (looking at https://staging.launchpad.net/successful-updates.txt) [11:53] losa ping: Lock needing to be killed for staging updates to work again? [11:53] wgrant: *I* cannot since I don't have shell access to DF ;). But I'll QA this as soon as r13769 is included into db-devel then. [11:54] rvba: Ah, forgot that detail. [11:56] wgrant: I'll monitor db-devel and ask you to update DF and restart it when my changes will be db-devel. [11:56] rvba: Sure. [11:56] Thanks. [12:02] Could somebody with access update the EC2 test images (https://dev.launchpad.net/EC2Test/Image)? There was some discussion about it on #soyuz but it was inconclusive [12:11] cjwatson: I've done so [12:11] stub: You're not in the access list... [12:11] Grar. [12:11] StevenK: ^^ [12:11] cjwatson: I'm waiting for a deployment, and then I'll land my branch that adds me to VALID_AMI_OWNERS [12:12] Ah. [12:12] (As well as two others) [12:14] rvba: Or you could just ask for your branch to merge anyway [12:14] DF is currently restarting... [12:15] Well, duh. If it's on the deployment report, it's certainly in db-devel [12:15] (The duh is at me for being a numpty.) [12:15] StevenK: Not necessarily. [12:15] StevenK: There was a conflict. [12:16] Sitting around for a few hours until stub resolved it a few minutes ago. [12:16] Again? That's like the fourth in 2 days. [12:16] We haven't merged db-devel for several weeks. [12:16] 6.5 weeks, in fact. [12:17] The proposal to merge lp:~jtv/launchpad/fix-more-utilities into lp:launchpad has been updated. [12:17] Status: Approved => Merged [12:18] JTV! [12:18] And he's not here. How convient. [12:18] Who can I yell at now? :-( [12:19] Meh, it was harmless. [12:19] If need be we can kill buildbot. [12:19] Again. [12:19] rvba: dogfood should have your rev now. [12:21] And then gary has 3 revisions of QA. === al-maisan is now known as almaisan-away [12:21] And bac [12:22] Buildbot b0rk is fixed? [12:23] StevenK: mine is easy...doing it now [12:23] nigelb: Yes [12:23] \o/ [12:35] gary_poster: O HAI! You haz two lots of QA to do so we can deploy many many lots revisions. [12:36] just when I thought I was getting the hang of strine [12:37] "many many lots" is from one of the Discworld books [12:37] ouch, now I am embarrassed === almaisan-away is now known as al-maisan [12:38] Where many many lots in this instance is 45 revisions [12:39] wgrant: Thanks for the heads up. [12:39] StevenK: that is indeed quite a few [12:40] StevenK, ack, on it === danilo_ is now known as danilos === henninge-lunch is now known as henninge [12:58] wgrant: I'm trying to QA #826870 and think I need your intervention: I've called """archive.copyPackage(source_name="emacs23-non-dfsg", version="23.3+1-1", from_archive=from_archive, to_pocket='release', to_series=oneiric.name)""" [12:58] <_mup_> Bug #826870: native syncs from Debian non-free default to Ubuntu universe < https://launchpad.net/bugs/826870 > [12:58] But now I suppose the job needs to run on DF. [12:58] wgrant: could you please do that for me? [13:01] rvba: It's hopefully running now. [13:01] wgrant: Thank you, I'll see if my package shows up in oneiric's queue. [13:03] rvba: Ah, it actually runs automatically every 2 minutes. [13:03] Rejected: [13:03] emacs23-non-dfsg 23.3+1-1 in wheezy (same version has unpublished binaries in the destination archive for Oneiric, please wait for them to be published before copying) [13:03] * StevenK prepares a bug list [13:03] 45 revisions, 2 rollbacks, 31 bugs [13:04] wgrant: Ok, I'll find another package from non-free. Thanks again. [13:05] StevenK: *wheee* :) [13:07] nigelb: One of them is your four-digit bug closure ... [13:08] StevenK: I know! I'm waiting ernestly and patiently to see it marked Fix Released. [13:12] Morning, all. [13:19] wgrant: done: qa-ok [13:20] StevenK: the archiveuploader tests all pass for me with dpkg-xz-support and updated launchpad-dependencies, FYI [13:20] cjwatson: Excellent. [13:20] I thought that would be the case but had to check ... [13:22] Gah! [13:22] abentley: QA! You're lucky last. [13:23] StevenK: take a pill [13:24] abentley: I'm perfectly calm. But I'd like to deploy before I go to bed, and that time is fast approaching [13:24] StevenK: my work day just started, and I am in fact investigating wgrant's QA of my work. [13:29] any code hosting/reviews masters around? i am seeing a very odd issue that i'm trying to debug. there is a branch proposed for merge into another, but the target thinks it has no proposals. [13:31] adeuring, ping for standup. [13:32] deryck: oops thanks [13:32] np [13:32] making it very hard for tarmac to land this branch :( [13:42] StevenK: qa done. [13:43] abentley, that's bug 829460 [13:43] <_mup_> Bug #829460: TypeError: object of type 'NoneType' has no len() loading qastaging oops report < https://launchpad.net/bugs/829460 > [13:43] abentley: Thanks! [13:44] matsubara: okay. [13:44] abentley, I'll try to get that fixed this week. [13:47] bigjools, does the series initialization process needs a script to run to successfully complete? [13:47] bigjools, on qastaging I mean [13:50] benji: are you around? (I'm not sure what your regular work schedule is) [13:50] jam: yep, I'm here [13:50] benji: just wondering if you worked out some of the loggerhead stuff or not. [13:51] it sounds like it got put on the back burner [13:51] but I figured I could try to help out [13:51] I'm working on it now. I may have some questions in a couple of minutes. [13:54] jam: when I try to push with a specified port number I get a connection refused: http://paste.ubuntu.com/673831/ [13:54] but when I remove the port number I get a password prompt but the user's LP password doesn't work [13:54] benji: are you running raw on your machine, not in a vm, etc? [13:54] benji: well you probably have openssh running on your machine [13:55] so it makes sense that you can get a password prompt [13:55] jam: it's in a VM [13:55] ah, I forgot that bzr+ssh runs over the normal ssh port [13:55] benji: then 'localhost' doesn't make sense, or are you running that in the VM as well [13:55] jam: I am [13:55] but certainly the VM is running openssh, or you wouldn't be connected [13:56] to clarify: I'm running these commands in a shell on the VM [13:56] right [13:57] matsubara: yes [13:57] matsubara: we're not running the DD scripts on staging yet until the new hardware arrives [13:58] benji: so you have to use "bzr+ssh://user@127.0.0.88:5022" [13:58] it doesn't bind to all addresses [13:58] http://bazaar.launchpad.net/+branch/launchpad/view/head:/configs/development/launchpad-lazr.conf [13:58] 'localhost' usually resolves to 127.0.0.1 I think [13:58] bigjools, can you run them for me? Or it can't be run on the current hardware? [13:58] I set up an ssh config alias, and forgot about it [13:59] jam: progress! It's complaining about not having an ssh key now, I'll add my public key to the user I'm using [13:59] right [14:01] jam: hmm: http://paste.ubuntu.com/673840/ [14:01] progress [14:01] sort of [14:01] what command are you running to run the host? [14:02] (to run launchpad + codehosting) [14:02] In the past, I've seen that failure when the Twisted ssh service is running, but the LPForkingService is not [14:02] I think "make run_codehosting" should be enough, though. [14:03] benji: I found that the forking socket is sometimes left around. rm /var/tmp/launchpad_forking_service.sock and make run_codehosting again [14:03] benji: You can also use 'utilities/make-lp-user $yourusername' to create a user with the name and your SSH key. [14:04] wgrant: killing the socket and restarting worked! [14:04] * benji needs to write this down somewhere. [14:04] benji: The service will fail to create it if it already exists. [14:05] I guess it should try to remove it. [14:06] matsubara: I can't run them personally, you need to poke a losa [14:06] matsubara: this one will init the new series: cronscripts/run_jobs.py -vv initializedistroseries [14:06] jam (or wgrant): now when I visit http://bazaar.launchpad.dev/~name16/+junk/1/files I get redirected to https://launchpad.dev/ [14:06] bigjools, will do. thanks! [14:07] oh, maybe I need to kick off a branch scan [14:07] benji: try bazaar.launchpad.dev:8080 (still in the VM)? [14:07] bigjools, how long should it take approximately? [14:07] benji: Restart apache. [14:07] benji: shouldn't need a scan for loggerhead [14:07] matsubara: how many packages did you copy? [14:07] benji: branch-rewrite probably isn't running (it's a LP script run by apache, yeah, a bit evil) [14:08] jam: adding the port worked (no I generally run my browser on my host, not the VM) [14:08] bigjools, I initialized 2 series, both derived from Natty, one with the core package set and the second only with the kernel. the first series is set to copies the packages while the second to rebuild the packages [14:08] wgrant: thanks, I'll try that [14:08] benji: If you run :8080 it goes directly to the loggerhead process. Apache proxies it [14:08] matsubara: should be quick-ish, probably 15-30m [14:09] jam: gotcha [14:09] bigjools, great! thank you [14:09] bigjools: That's almost /Quotes worthy. [14:09] the apache restart got the proxying working [14:09] jam: thanks much; now I just need to reproduce the original race and then verify that my branch fixes it [14:09] wgrant: /me waving hand around furiously [14:10] benji: right. So if you have the original code, "make run_codehosting" and load something that is 'slow' in 2 browser windows should work [14:10] The key, is that after the *first* request succeeds, generally you have no more lazy-import objects around [14:11] "slow"? [14:11] oh, it doesn't have to be a loggerhead request per se? [14:13] benji: it needs to be a request on loggerhead, such that you get 2 of them at the same tim. [14:13] time [14:13] k [14:13] so the first time a loggerhead instance comes up, it gets > 1 request immediately [14:13] well, I should say [14:14] whenever it gets 1 request, there is already a second request. [14:14] (the first request) [14:21] jam: hmm, I'm having trouble getting an exception; I have http://bazaar.launchpad.dev:8080/~name16/+junk/1/files loaded in five tabs, I stop and restart codehosting, and tell firefox to reload all tabs; all of them load fine and there are no errors reported on stdout [14:22] I'm trying now with ab doing 10 concurrent requests. [14:24] deryck, abentley: bad news, the problem already existed in my last branch, so it will break js on some pages but I don't know how badly. [14:24] henninge: Should the rollout be aborted? [14:24] wgrant: oh, are we having a roll-out? [14:24] henninge: oh dear. [14:25] henninge, "the problem" ? What is the problem? And yeah, should we abort and do a rollback? [14:25] I thought that was held off [14:25] henninge: It is in progress. [14:25] We've been holding off for a week now :) [14:25] wgrant: let me check how bad the breakage is. [14:25] henninge, what is the problem exactly? [14:26] henninge, not including the mockio from testing? [14:26] importing from app/testing [14:26] it is, yes [14:26] henninge: If it is rollout-blocking-critical, you need to revert it now. [14:26] We have already got two reverts and 50 revisions in the pipeline; we cannot afford to risk a fix. [14:26] And also poke someone to abort the rollout :) [14:26] wgrant: np, nothing depends on it. [14:27] henninge, yeah, what wgrant said :) abort the rollout, revert, and land it correctly. [14:27] Don't land it correctly until we are safely rolled out and confirmed good. [14:28] wgrant: it hadn't started yet. [14:28] Ah, good. [14:28] Anyway, you have 90 minutes to land the revert. [14:28] Before the next buildbot run. === wgrant changed the topic of #launchpad-dev to: please avoid landing things before we've rolled out -- qa is a mess right now | https://dev.launchpad.net/ | On call reviewer: StevenK, jtv | Critical bugs: 238 - 0:[#######=]:256 [14:30] wgrant, I don't follow why we should avoid landing things? not arguing, just not sure I see what not landing gets us. [14:30] deryck: Landing gets us into this situation. [14:31] We now have three chained rollbacks, making around 55 revisions undeployable. [14:31] wgrant, how does a new rev get us into this? marking qa-ok when it's not gets us into this, no? [14:32] deryck: If there is another bad rev in there, and people land stuff before we can detect and roll it back, we can then not deploy until all those new revisions are QAed, and we have to hope they are not bad. [14:32] The last three times this has happened this week, one of those revisions has been bad. [14:32] So we end up in a chain of non-deployability, and have to deploy a week of work in a very risky and difficult to roll back operation. [14:33] That is, we now have: ... bad revision a ... bad revision b ... rollback a ... bad revision c ... rollback b ... rollback c [14:34] Nothing between bad revision a and rollback c can be deployed. [14:36] * deryck is thinking [14:36] Basically, we need a more thorough, quicker test suite, so QA is less important and able to be done more quickly. [14:37] wgrant, I don't think saying "stop landing things" is the right way to fix this. if we feel it's serious enough to stop landings, then we should block the builders merging to stable while we sort it out. [14:37] wgrant, and completely agree we need a quicker test suite :) [14:38] It's half-past midnight and I'm not sure I can be bothered walking a LOSA through reconfiguring PQM. [14:38] But I also don't want to break the rollback record. [14:40] We currently have more than 6 days of work blocked. [14:40] By the time this is fixed, it'll be 7. [14:40] Any more breakage that lands in the meantime will take it over 8. [14:40] And large deployments have a very bad track record of reliability. [14:42] wgrant, I take the seriousness of it. I don't think changing the topic here is the way to fix it. I'll start a thread on the launchpad-dev list taking up the issue there if that's cool with you. [14:42] Sure. [14:42] wgrant, cool. thanks for the chat about it. [14:42] Reconfiguring PQM is heavyweight and awkward, so I am reluctant to do it. [14:42] completely understand. === deryck changed the topic of #launchpad-dev to: deployment is a mess right now, be cautious or avoid landings until fixed | https://dev.launchpad.net/ | On call reviewer: StevenK, jtv | Critical bugs: 238 - 0:[#######=]:256 [14:46] jam: ok, I think I'm now able to reproduce the race, but I'd like to make sure this 500 is the one I'm looking for, where does loggerhead log errors? [14:48] deryck: congratulations. i choose you! there seems to be an issue with merge proposals, and private branches? [14:51] dobey, ok, rockstar is also telling me something similar. [14:52] dobey, I'm already chatting with deryck about it. [14:52] :) [14:52] Possibly r13716 [14:52] I wonder if our aborted rollout is somehow to blame, or a bad landing. [14:52] If it was a regression just today. [14:52] We deploy 13715-13727 today. [14:52] deployed [14:52] Well, I don't know if it's a regression just for today. [14:52] At some point, we did have a test that covered this situation. [14:53] i am pretty sure it's a regression just for today [14:53] at least, today is when i get people asking me about it :) [14:53] Yeah, today is also when we noticed (it breaks tarmac's ability to find things to land) [14:54] danilo_: ^^ [14:54] wgrant, and is the rollback of r13716 in the stuff we aborted? [14:54] wgrant, or no one knows r13716 is bad? [14:55] I'm guessing this is new, just looking at qa pages and bzr logs. [14:56] deryck: 13716 is not known to be bad, but is very relevant. [14:56] I understand now. Thanks, wgrant. [14:56] It was deployed today, and alters the MP privacy code. [14:56] gotcha, [14:56] Woo, #4/ [14:56] Going for the record. [14:58] abentley, it seems merge proposals and private branches have an issue now. see scrollback. Can you take this issue, open an incident report, and start tracking down what's going on? [14:59] It would be excellent if it was diagnosed and rolled back in the next 60 minutes :) [14:59] But it is 1am, so I should really leave again. [14:59] deryck: looking... [14:59] wgrant, go sleep. :) we got it. === jam1 is now known as jam [15:00] abentley, thanks! [15:00] Thanks abentley, deryck. [15:00] * wgrant sleeps. === matsubara is now known as matsubara-lunch [15:05] mrevell, I fell off IRC and you may have thought I was off this week [15:06] sinzui, Company calendar says you are :) [15:06] yuck [15:06] I am pretty sure I am off next week [15:09] abentley: what was it that I had to do to keep the rollback-revision from removing my stuff next time I merge devel? [15:10] is it bzr revert --forget-merges ? [15:10] henninge: merge the revision immediately before the rollback; commit; merge the rollback revision, do "bzr revert ."; commit. [15:10] oh, the ".", right. [15:10] abentley: thanks [15:11] henninge: np [15:11] deryck: this preloading stuff is not my forté. [15:12] abentley, so we need a db/preloading expert to jump in here? [15:12] deryck: that would help, yes. Also, we could use a test that used to pass, but fails now. [15:13] abentley, I would have recommended allenap help, but he is afk. [15:13] * deryck looks around more.... [15:13] sinzui, otp just now, ping you in a mo [15:14] okay [15:14] rockstar: can you give me an idea what test I need to write to detect this issue? [15:15] abentley, yeah, let's pursue that angle, i.e. finding a test to confirm, and I'll look at the suspect rev more closely with you here in a second. [15:16] abentley: we have merge proposals that are generally fine, except the target doesn't think it has any merges proposed for it. and this seems to only affect private branches. [15:16] abentley, it appears that a private branch doesn't have any landing candidates even though there are merge proposals. [15:17] rockstar: do we know whether all users are affected, or just Tarmac? [15:17] abentley, it appears that even the website is affected. [15:18] So you can look at the Branch Merges section of a branch page and see that they are missing. [15:19] rockstar: okay, so it sounds like the view check is broken, then [15:20] sinzui, Are you available to talk now? [15:21] yes [15:23] hi bigjools, would you have a moment for a preimp call about a build recipe bug? [15:23] bac: I am not an expert on recipes but I can try [15:24] bigjools: i'm looking at bug 828914 [15:24] <_mup_> Bug #828914: +request-daily-build oops with an AttributeError: 'NoneType' object has no attribute 'published_archives' < https://launchpad.net/bugs/828914 > [15:24] bigjools: skype? [15:24] yup [15:24] or I am already on mumble [15:25] i haven't used mumble in a while and may take some setup [15:25] ok === jkakar_ is now known as jkakar === al-maisan is now known as almaisan-away [15:39] abentley, did you start and incident report yet? [15:40] s/and/an/ [15:40] deryck: no. [15:40] abentley, I'll start one and update topics. === almaisan-away is now known as al-maisan === al-maisan is now known as almaisan-away [15:45] any of you guys run wiki.ubuntu.org? === deryck changed the topic of #launchpad-dev to: deployment is a mess right now, be cautious or avoid landings until fixed; abentley looking into issues with merge proposals and private branches | https://dev.launchpad.net/ | On call reviewer: StevenK, jtv | Critical bugs: 238 - 0:[#######=]:256 [15:46] and/or have any idea how they integrated it with login.ubnutu.com? [15:46] mtaylor, no, not us. maybe ask in #ubuntu-devel? [15:46] mtaylor, it's a moin open id plugin [15:46] https://launchpad.net/moin-openid [15:47] abentley, I started https://wiki.canonical.com/IncidentReports/2011-08-24-LP-merge-proposals-and-private-branches [15:47] abentley, it's a skeleton, but I'll fill in what we know now. [15:50] beuno: thanks! [15:51] beuno: wow. that's so not open source :) [15:56] mtaylor, it is AFAIK [15:56] * beuno looks [15:58] deryck, rockstar: does this test look like it would pass if the error is fixed? http://pastebin.ubuntu.com/673926/ [15:59] hm [15:59] flacoste, ping. The moin-openid page says you have the task of looking to open sourcing [15:59] any ideas on where that's stalled? [15:59] beuno: really!!?! [16:00] beuno: i don't know, that's now owned by ISD, I'd ask maris or stuartm [16:00] flacoste, that's what this says in the comment: https://launchpad.net/moin-openid [16:00] abentley, as I understand the problem, yes, I think that's good. I'll wait on rockstar to say for sure, though. [16:00] thanks flacoste, I'll chase them [16:01] deryck: I've gone back 400 revisions and it's not passing. [16:01] mtaylor, it seems we're not using that anymore, the plugin is now in upstream as of moin 1.9 [16:02] abentley, hmmm, that doesn't sound good then. can't be correct. [16:02] beuno: oh neat! [16:02] rockstar: are you using getMergeProposals or landing_candidates? [16:03] mtaylor, if you run into problems integrating into login.u.c, you can head over to #canonical-isd [16:07] abentley, landing_candidates === matsubara-lunch is now known as matsubara [16:07] rockstar: okay, I think the problem is that landing_candidates should be the same as getMergeProposals, but is not. [16:07] abentley, yeah, I was thinking they were. [16:08] rockstar: or rather, it was not, but now is, and getMergeProposals has always been broken. [16:08] abentley, does that also explain why the merge proposals aren't listed on the page? [16:08] rockstar: dunno what that page uses. [16:08] Oh, I guess that would explain it. [16:09] rockstar, abentley -- are we certain the web version is broken and not just an api/tarmac thing? [16:10] deryck, I am, yeah. That's what made me escalate. dobey pointed it out when we were debugging. [16:10] ok [16:11] rockstar, deryck: I rolled-back 13716 and the landing_candidates version of my test passed. [16:12] abentley, ok, cool. so let's land a rollback of that and update the corresponding bug. [16:12] deryck: we could, but I think the fix is a one-liner. [16:12] abentley, hmmmm [16:14] abentley, here's why I'm hesitant..... [16:15] abentley, the deploy has already been delayed a lot. I worry this fix either blocks the deploy another day or doesn't make it in the deploy, and then is 2 days out from being deployed. [16:15] so I wonder if the straight rollback isn't easier to get this fixed quicker. we can easily hold the deploy for one more rollback, I think. [16:17] deryck: Here's the fix: http://pastebin.ubuntu.com/673945/ === salgado is now known as salgado-lunch [16:17] deryck: A rollback would affect the other work, too. [16:17] abentley, yeah, true. [16:18] deryck: But if you prefer, I'll do the rollback. [16:19] abentley, land the real fix. let's make sure we understand from rockstar how to easily qa this when it lands, so we don't lose time waiting on qa for it. [16:20] deryck, abentley, I bet the easiest way to QA it would be checking to see if a private branch's page has merge proposals in the Branch Merges section, when it really does have merge proposals. [16:20] rockstar: probably. At this point, I haven't confirmed that. [16:22] abentley, so I need to step away for 5 minutes, sorry. but I'm fine to go ahead with landing the proper fix. [16:22] deryck: ack. [16:24] abentley: fancy a lazr.batchnav review? https://code.launchpad.net/~adeuring/lazr.batchnavigator/slicing-error-for-too-short-last-backwards-batch/+merge/72745 [16:24] adeuring: handing an incident. [16:24] ah, ok === beuno is now known as beuno-lunch [16:25] * adeuring needs to learn to read the channel topic more carefully [16:25] StevenK: are you up for a review? https://code.launchpad.net/~adeuring/lazr.batchnavigator/slicing-error-for-too-short-last-backwards-batch/+merge/72745 [16:28] rockstar: is there a bug? [16:28] rockstar: I mean has the bug been reported? [16:30] mrevell: Congrats! [16:30] Hey thanks nigelb :) I still have the bug on my to-do list :) [16:31] mrevell: sure, I've been busy well, so I didn't want to poke you and not have time myself :) [16:32] abentley, no. [16:36] abentley, are you filing a bug for the incident? [16:36] deryck: did: bug 833147 [16:36] <_mup_> Bug #833147: landing_candidates does not show proposals for private branches < https://launchpad.net/bugs/833147 > [16:38] abentley, cool. added the regression tag, just to help with the tracking of this sort of thing. [16:39] deryck: also updated incident report. [16:40] abentley, thanks! I'll handle all the successes and problems sections, and make sure it's all complete. [16:41] deryck: The fix landed as r13779 [16:41] ok, cool. [16:42] mrevell: Oh, btw, for that bug. I was thinking if we could have a heading in there. Helps a lot! [16:42] nigelb, Could you add that as a comment please? [16:42] sure, still needs userting though. [16:48] mrevell: \o/ [16:49] :) [16:49] any losas around, we've been waiting on a bug import for a while... https://answers.launchpad.net/launchpad/+question/168463 [16:49] mrevell: also, ironically, I'm not subscribed to the bug ^-^ [16:57] deryck: I am going on lunch. [16:58] abentley, cool. Thanks for taking care of that this morning! [16:58] deryck: you're welcome. === deryck changed the topic of #launchpad-dev to: deployment is a mess right now, be cautious or avoid landings until fixed | https://dev.launchpad.net/ | On call reviewer: StevenK, jtv | Critical bugs: 238 - 0:[#######=]:256 [17:16] congratulations mrevell === beuno-lunch is now known as beuno === almaisan-away is now known as al-maisan [17:54] Project devel build #990: STILL FAILING in 5 hr 59 min: https://lpci.wedontsleep.org/job/devel/990/ === abentley changed the topic of #launchpad-dev to: deployment is a mess right now, be cautious or avoid landings until fixed | https://dev.launchpad.net/ | On call reviewer: abentley | Critical bugs: 238 - 0:[#######=]:256 [18:09] thanks james_w :) [18:22] Project db-devel build #820: STILL FAILING in 6 hr 10 min: https://lpci.wedontsleep.org/job/db-devel/820/ === salgado-lunch is now known as salgado === al-maisan is now known as almaisan-away [18:40] *yawn* [18:43] jelmer: hi; don't look at me for updating lp-tools in debian, EWAYTOOBUSY just now :( [18:44] jelmer: you can add yourself to uploaders though :) [18:45] lifeless: is it hard to fix bug 520413? [18:45] <_mup_> Bug #520413: All changes by user must be revertable < https://launchpad.net/bugs/520413 > [18:45] well, at least the bits that you mentioned in there. [18:45] nigelb: glad you qualified it :) [18:45] hiding the comments by a user when their account is disabled [18:47] nigelb: so, as tom berger said there are few journals of changes within Launchpad [18:47] there may be enough for *bugs* in particular with the bugactivity log. [18:47] I think its a multi-part issue [18:47] lifeless: no no, I'm only looking at your comment, not my bug :) [18:47] oh [18:47] easy [18:48] minimally - and perhaps too minimal (might not get past 'is it a good idea' :P) just check the authors status as part of querying for bug comments to show [18:49] hmm, can I *try* to do it? :) [18:49] thats probably too minimal because folk that were legitimate users of LP may well get suspend when only a few of their artifacts are a problem. [18:49] hello folks [18:49] matsubara: OOPS-2062AZ69 shows a past week count of 13. can you tell me the ids for those related OOPS? [18:49] hi reed [18:50] lifeless: hm, this means introducing a new field into the db along with deactive. [18:50] deactive and hide interacation. [18:50] or some such [18:50] nigelb: less minimal, extend the account status/standing enum to have 'dude was a spammer' and exclude that status specifically - and then if we believe a user was nothing but a spambot we set it to that, so less automatic but also less false-positives [18:51] this involves doing a db migration. [18:51] nigelb: I don't believe it needs a new field. its entangled with being inactive. [18:51] wondering if you've evaluated integrating this zope3 based http://www.groupserver.org/groupserver/ into LP [18:51] a new state rather [18:51] hm, intereting. [18:51] *interesting [18:51] I should dig into the code to figure out if I can even read the bits that are relevant [18:51] nigelb: so it needs: teach the code about the new enum value but don't set it [18:51] nigelb: then once thats deployed add code to set it. [18:52] ooo, so change the display bits first [18:52] and then change the enum bits. [18:52] do one in devel and one in db-devel? [18:52] nigelb: close enough :) - change the things that -read- the enum first. Land. Deploy. Then change the things that -set- it. [18:53] nigelb: otherwise you have a race condition where the new value could be read by old code that does not expect it. [18:53] Right, makes sense :) [18:53] reed: I don't believe anyone has looked at it [18:55] thanks lifeless... now my next question is: where can I find the roadmap for the LP mailing list future development? [18:55] reed: it looks like 50-60% overlap with our existing code, but direct assumptions about layout and schema that would make running it as an integrated service non trivial [18:55] reed: things that are planned in the short term are at dev.launchpad.net/LEP [18:55] reed: things that are far enough out that noone is directly planning when they will work on them just live as bugs [18:56] reed: https://bugs.launchpad.net/launchpad/+bugs?field.tag=mailing-lists [18:56] got it ... thank you [18:56] reed: also zope 5 - aieeee. [18:56] lifeless: g'morning [18:56] We seem to get almost 24 hour coverage with just .au and .nz alone.. [18:57] well, .nz is +12 ;) [18:57] I see lifeless almost every hour that I glance in here :P [18:57] Or wgrant. [18:57] did you not know that they are bots? [18:58] controlled by each other? :P [18:58] bac, , , , , , , , , , , , , [18:58] matsubara: thanks! [18:59] you're welcome [18:59] lifeless: wrt lptools; thanks, I know you're busy. Happy to add do an upload myself though. [19:36] reed: I see you've met our friend lifeless [19:37] mtaylor: that sentence is sooo meta ... [19:37] reed: I'm good like that [19:37] meeting lifeless friend [19:38] lifeless: reed is working with me over in openstack-land [19:38] hi lifeless, i'm the new community manager [19:41] cool [19:41] mtaylor: reed: I'm just OTP at the moment, you'll have my attention soon [19:42] lifeless: no worries, nothing urgent [20:12] mtaylor: reed: ok, am here :) [20:18] qastaging.lp.net timesout [20:18] and suggets I visit the homepage [20:18] which of course, is qastating.lp.net, which what timed out in the first place! [20:19] recursion ftw. === matsubara is now known as matsubara-afk === almaisan-away is now known as al-maisan [20:48] jcsackett, ping [20:49] deryck: pong. [20:49] jcsackett, hey hey. where are we at with qa for r13774, "Updates dsp vocab to use the dsp in db in the search query." ? [20:49] oh, we're done and i forgot to update the bug. :-P [20:49] one sec. [20:50] jcsackett, oh happy day, thanks! [20:51] deryck: updated. report should reflect it shortly. :-) [20:51] jcsackett, rockin! Thanks man! [20:51] jcsackett, you're not blocking yet. but soon would have been. [20:56] statik: Just figured out an easier way to access a remote launchpad instance. Use ssh's built-in SOCKS proxy (ssh -D 8080) and then set your browser to use that proxy. Assuming you have the standard mappings of lp.dev to 127.* in your /etc/hosts. [20:57] abentley: thanks! [21:03] lifeless: I think reed was wanting to poke about better mailing list archives === mwhudson__ is now known as mwhudon === mwhudon is now known as mwhudson [21:15] Later on, everyone. === al-maisan is now known as almaisan-away === abentley changed the topic of #launchpad-dev to: deployment is a mess right now, be cautious or avoid landings until fixed | https://dev.launchpad.net/ | On call reviewer: - | Critical bugs: 238 - 0:[#######=]:256 [22:09] mtaylor: sure would love that :) [22:10] reed: mtaylor: for that, I would look at - https://dev.launchpad.net/ArchitectureGuide/ServicesRoadmap [22:10] * reed reading [22:10] reed: mtaylor: specificxally [22:12] lifeless: read that ... any plans to include reply and post via web UI? [22:12] reed: none so far but actually it should be very easy. [22:13] reed: step 1) implement that refactoring above; step 2) add reply/post buttons on the now-live web pages [22:13] you could do step 2 directly with cross-domain links back into LP itself [22:13] if step 2 is particularly urgent [22:14] the basic need is to give users the possibility to pick their fav system between mlist or forum [22:16] ok [22:16] there are many differences between email and forums [22:16] do you want to reconcile all of them, or just provide a web UI to the mailing list ? [22:17] well... a mlist with a great looking web UI to post/reply would be good enough [22:18] I don't like forums particularly, never found one that was pleasant to hang around but I see the value for casual, non committed users [22:19] gmane.org does a decent job (the ui needs work though) [22:19] Morning. [22:20] Are we up to #5 yet? [22:21] No, but qastaging is screwed. [22:21] * wgrant sighs. [22:21] Oh, buildbot is broken. [22:21] Yay [22:21] And has been since 2 hours after I went to bed. [22:24] wgrant: we're up to 7 I believe. [22:26] Sadly only 4. [22:26] With one very hairy bit of QA in the middle. [22:30] Hmm. [22:30] buildbot was broken, but the deployment report is the really broken thing. [22:30] qastaging is on 13779, deployment report still 13775 [22:30] Hasn't updated in 2 hours. [22:31] lifeless: Can you investigate, please? [22:31] is staging down ? [22:31] That doesn't normally affect the stable report. [22:31] And no. [22:32] https://devpad.canonical.com/~lpqateam/qa_reports/logs/output-launchpad-stable.log [22:32] INFO:qa-tagger:Last revision now is 13779. [22:32] WELL [22:32] This is amusing. [22:32] wgrant: looks fine to me [22:32] Wed Aug 24 22:28:40 UTC 2011 Finished updating code [22:32] Last revision deployed to QAStaging is 13779. There are 35 revisions waiting in the queue. [22:33] Thankyou, qastaging, for that absolutely perfect timing :) [22:33] Argh, no --rollback. [22:38] Ah, but production wasn't reverted, so qa-tagger will happily ignore that. [22:42] Yay, green. [23:15] Project devel build #991: STILL FAILING in 5 hr 20 min: https://lpci.wedontsleep.org/job/devel/991/ [23:17] reed: sorry I went quiet on you [23:17] lifeless: np, I think I know what I needed to know [23:18] reed: we'd be delighted to support sensible reply-in-the-web, and anything making the archives nicer would be great. [23:18] reed: we have no specific work queued in this area but its all fairly straight forward if someone wants to patch it. [23:18] lifeless: at the moment I don't know what resources I have, i'll let you know though [23:19] of course [23:19] if you or someone you know wants to work on it, we will mentor [23:19] nigelb here has been doing patches recently, with great success ;) [23:21] tnx [23:22] mtaylor: interesting mail on ppas === almaisan-away is now known as al-maisan [23:22] lifeless: it seems to have stirred up a few shitstorms [23:22] indeed [23:22] btw [23:23] lifeless: especially where I perhaps wasn't clear enough on the "I think PPAs are amazing" front [23:23] If you want callbacks from PPAs, I know whats involved ;) [23:23] lifeless: hehe. [23:23] oh oh === al-maisan is now known as almaisan-away [23:23] lifeless: well, I _do_ want callbacks - but I also want synchronous runs and streaming output :) [23:23] mtaylor: new toy you might like - lp:python-oops / lp:python-oops-wsgi [23:23] ooh. /me goes to look [23:23] mtaylor: we have the latter, the former you're already escalated in priority [23:24] lifeless: you have streaming output in my jenkins? [23:24] mtaylor: no, we have (slightly jerky) 'streaming' in LP build logs :) [23:25] hehe [23:25] thats a shallow fruit to tune FWIW [23:25] * lifeless mixes metaphors [23:25] * mtaylor tunes fruit. picks a seed out from his muffler [23:27] lifeless: I think the main points that are harder to handle are "I need to be able to build packages for debian in a way similar to ppas" (the hard one) and "I'd really like to get things more integrated in to my CI system" (doable, but definitely some dev work- especially with no java launchpad api) [23:30] lifeless: callbacks would make #2 doable - but jenkins isn't really good at the wait and respond to async events for job completion thing ... and jenkins url triggers are COMPLETELY broken when openid SSO mode is enabled (sigh) [23:30] but then we'd still be stuck on #1 [23:42] so url trigger fixing is a bug :) [23:42] uhm [23:42] debian is technically doable. [23:42] One way you could do it that avoids any political aspects would be a custom LP instance just for debian builds [23:42] same API [23:42] etc [23:43] You could take the well-separated Soyuz and run it for Debian PPAs. [23:44] There's already an LP instance for Debian PPAs. [23:44] mtaylor: ^ [23:44] Or you could submit patches? :-) [23:44] well [23:45] Although, PPAs for multiple distributions is going to break a LOT [23:45] its not clear that we *would* build for Debian. But that is a whole-different-discussion. [23:45] StevenK: Only two bits need changing. [23:45] StevenK: The activate form and URLs. [23:45] Everything else works. [23:45] it is a whole different discussion, most of the time I've brought it up over the last 2 years it's been met with "no" [23:45] The issue is mostly builder time. [23:45] I believe. [23:51] Is deployment still a mess? [23:52] Project db-devel build #821: STILL FAILING in 5 hr 29 min: https://lpci.wedontsleep.org/job/db-devel/821/ [23:52] We hope not, but maybe. [23:52] We'll see in a couple of hours when the deployment is done and tested. [23:53] wgrant: Did you have a branch to move the updating into DSPC? [23:53] They are approved. [23:54] I'm guessing they're both used? :-/ [23:54] DSPC² are both used slightly, yes. [23:54] Used slightly? How slightly? [23:55] DistributionSourcePackageCache is used for package searches, DistroSeriesPackageCache for searches and some DSBP and DSPR properties. [23:55] Both are also used to create the package counts for PPAs, but that is easily fixed. [23:56] I looks like DistributionSourcePackageCache might be the easiest to remove [23:56] Right, it can probably be replaced by the new DSP or whatever we end up with. [23:57] DistroSeriesPackageCache is a little harder. [23:57] Bah, I was hoping to just remove DistributionSourcePackageCache :-) [23:57] please do [23:57] its implicated in some seriously poor performing queries. [23:57] of course, you need to get replacement queries as fast first. [23:57] lifeless: Are you sure? It's only ever used in package searches on Distribution. [23:58] wgrant: bug 816870 [23:58] Well, unless you count the 20h update-pkgcache.py as a poor performing query. [23:58] <_mup_> Bug #816870: Distribution:+search (package search) timeouts < https://launchpad.net/bugs/816870 > [23:58] Hah.