| wgrant | qastaging takes forever to update :( | 01:23 |
|---|---|---|
| thumper | poo | 01:35 |
| wgrant | added: lib/lp/archivepublisher/tests/util.py.THIS | 01:53 |
| wgrant | thumper: ^^ | 01:53 |
| thumper | eh? | 01:54 |
| thumper | arse | 01:54 |
| thumper | did that slip through? | 01:54 |
| wgrant | Yes. | 01:54 |
| thumper | poo | 01:54 |
| wgrant | Just landed a couple of minutes ago. | 01:54 |
| thumper | another pp | 01:54 |
| * thumper fixorates | 01:54 | |
| thumper | wgrant: is it breaking things? | 01:54 |
| thumper | it passed ec2 | 01:54 |
| wgrant | thumper: It shouldn't break anything, no. | 01:55 |
| thumper | pqm-sumitting the fix | 02:01 |
| wgrant | Thanks, | 02:01 |
| * wgrant lunches. | 02:21 | |
| wgrant | Has anyone looked at the db-devel failure? | 03:51 |
| lifeless | the real question is, has the failure looked at anyone? | 03:52 |
| wgrant | Ah, it's already running again. | 03:52 |
| wgrant | Yay buildbot. | 03:52 |
| wgrant | lifeless: How do we fix a stale librarian PID issue on buildbot? Get a GSA to remove the file manually? | 04:49 |
| lifeless | yes, checking that: | 05:00 |
| lifeless | - the librarian is not still running | 05:00 |
| lifeless | - *how* it happened - was the machine rebooted? did the librarian crash? was the test killed (and if so with what signal) | 05:00 |
| lifeless | basically gather data so we can permanently fix | 05:01 |
| wgrant | There was a test explosion in the last run. | 05:01 |
| wgrant | zope.testrunner does not seem to rate very highly in the not sucking department. | 05:02 |
| lifeless | go on, be more accurate ;) | 05:02 |
| lifeless | wgrant: please make sure there is a bug on lp, high pri explaining what you figured out | 05:06 |
| lifeless | wgrant: test explosions should always tear down | 05:06 |
| lifeless | etc | 05:06 |
| wgrant | lifeless: But Twisted tests dying with KeyboardInterrupt make me cry. | 05:07 |
| wgrant | stub: Morning. | 05:23 |
| stub | yo | 05:23 |
| wgrant | stub: Can I please have you db-blessing for https://code.launchpad.net/~wgrant/launchpad/die-lucilleconfig-die/+merge/44305? | 05:23 |
| stub | In the name of the Senate and Peoples of Rome | 05:27 |
| wgrant | stub: Do I also need a TA-blessing? | 05:28 |
| stub | Not before landing, no. Request 2 db reviews from me and lifeless. Land after you have the number and one approval. | 05:32 |
| wgrant | I thought thinks might be different at the moment, given the lack of TA. | 05:33 |
| wgrant | things. | 05:33 |
| wgrant | Thanks. | 05:33 |
| stub | I've put in the second review request | 05:33 |
| stub | The two reviews thing is so we can both keep on top of things, but not block devs. | 05:33 |
| stub | Holidays etc. | 05:33 |
| wgrant | Right. | 05:33 |
| === almaisan-away is now known as al-maisan | ||
| poolie | hi | 06:44 |
| poolie | stub, would you be kind enough to sponsor landing of https://code.launchpad.net/~mbp/launchpad/314507-oauth/+merge/44188 for me? | 06:44 |
| poolie | iow to send it to pqm | 06:44 |
| stub | poolie: Has it gone through ec2 test yet? | 06:45 |
| poolie | no | 06:45 |
| poolie | i guess to send it to pqm via ec2 | 06:45 |
| stub | Ok. Doing that now. | 06:45 |
| poolie | thanks | 06:46 |
| stub | Bug #692872 | 07:01 |
| _mup_ | Bug #692872: Test suite fails if previous run did not tear down Librarian fully. <Launchpad itself:New> < https://launchpad.net/bugs/692872 > | 07:01 |
| wgrant | lifeless: Oh? | 07:03 |
| lifeless | wgrant: did you end up landing my librarianfixture branch ? | 07:03 |
| wgrant | lifeless: No. | 07:03 |
| wgrant | I was told that adeuring was working on that. | 07:03 |
| wgrant | But that was nearly two weeks ago. | 07:03 |
| lifeless | right | 07:04 |
| lifeless | well it needs landing | 07:04 |
| lifeless | the cure is worse than the disease | 07:04 |
| lifeless | we need to stop doing expendient things and actually make the foundations sane and reliable | 07:04 |
| wgrant | Sure. | 07:05 |
| wgrant | I'll take a look at that tomorrow. | 07:05 |
| wgrant | Once I work out how I am going to get this regression fix deployed. | 07:05 |
| lifeless | qa the intermediate patches | 07:05 |
| wgrant | Difficult. | 07:06 |
| lifeless | qa isn't restricted to the author | 07:06 |
| wgrant | I need r12112, but r12102 is bad. | 07:06 |
| wgrant | It is needed on cesium, which r12102 does not affect. | 07:06 |
| wgrant | So I need manual approval. | 07:06 |
| wgrant | But the relevant team lead and project lead are on leave. | 07:06 |
| lifeless | whats in 12102 | 07:06 |
| wgrant | A webapp formatter change. | 07:07 |
| lifeless | ok | 07:07 |
| lifeless | +1 | 07:07 |
| lifeless | the losas should remove cesium from nodowntime | 07:07 |
| wgrant | lifeless: Thanks! | 07:07 |
| wgrant | danilos: Should the variants rev I tried to QA last night block a rollout? | 07:09 |
| wgrant | danilos: Or does it just not fix the bug? | 07:09 |
| wgrant | It looks qa-ok to me, but I'd like to be sure. | 07:09 |
| stub | lifeless: Does that branch relateto Bug #692872 ? | 07:30 |
| _mup_ | Bug #692872: Test suite fails if previous run did not tear down Librarian fully. <Launchpad itself:Won't Fix> < https://launchpad.net/bugs/692872 > | 07:30 |
| lifeless | stub: very much so | 07:30 |
| stub | Your comments don't say if your branch fixes the issue, or if we want the issue to continue. | 07:33 |
| lifeless | oh | 07:33 |
| lifeless | gimme a sec to find it | 07:34 |
| stub | I just don't want buildbot to fail if a previous run left librarian crud on well known sockets or in well known files. | 07:34 |
| stub | (which is why we are in testfix atm) | 07:34 |
| wgrant | (Hudson! Hudson!) | 07:35 |
| stub | wgrant: Won't hudson have the same problems? | 07:35 |
| wgrant | stub: It doesn't have the same catastrophic failures, plus it uses Canonicloud and recreates the instances every so often. | 07:36 |
| lifeless | ok | 07:37 |
| lifeless | https://code.launchpad.net/~lifeless/launchpad/librarian/+merge/39013 | 07:37 |
| stub | Recreating the instances seems nice - pita buildbot requires manual intervention. | 07:37 |
| lifeless | stub: it won't have this problem at all | 07:37 |
| lifeless | because of the new instance thing | 07:37 |
| lifeless | so that MP ^ | 07:37 |
| stub | lifeless: Cool. So the bug isn't won't fix, it is pending :) | 07:37 |
| lifeless | well | 07:38 |
| lifeless | let me reread | 07:38 |
| lifeless | right | 07:38 |
| lifeless | the proposed fix is inappropriate | 07:38 |
| stub | Argh... midair collision | 07:43 |
| wgrant | Hmmm. | 07:43 |
| wgrant | Something is broken. | 07:43 |
| spiv | wgrant: Is that something software? That always breaks. | 07:44 |
| stub | 5963 things are broken in LP | 07:44 |
| spiv | It's a wonder anyone ever bothers to use software, really. | 07:44 |
| wgrant | https://code.qastaging.launchpad.net/~wgrant/launchpad | 07:44 |
| wgrant | - Expression: <PathExpr standard:u'menu/mergequeues/render'> | 07:44 |
| wgrant | KeyError: 'mergequeues' | 07:44 |
| wgrant | I don't think that's changed lately :/ | 07:44 |
| stub | lifeless: Anything stopping the librarian branch landing (apart from being in testfix mode because the librarian branch hasn't landed yet)? | 07:45 |
| lifeless | stub: I'd love it if someone would land it | 07:45 |
| stub | I'll stuff it through ec2 then | 07:46 |
| wgrant | It has some test failures. | 07:46 |
| lifeless | I suspect it will have (some) failures / may need a merge with trunk | 07:46 |
| wgrant | Or did last time I looked. | 07:46 |
| wgrant | I don't remember exactly which. Which may indicate that it was catastrophic enough that I was unable to get a complete list. | 07:46 |
| stub | I'll stuff it through ec2 test to get the list. | 07:46 |
| wgrant | Not if it explodes :) | 07:47 |
| stub | That bad huh? | 07:49 |
| lifeless | its altering fairly fundamental stuff | 07:49 |
| wgrant | IIRC yes. | 07:49 |
| wgrant | The testrunner did not end up very happy at all. | 07:49 |
| stub | Maybe it is a post-lunch job then | 07:50 |
| wgrant | stub: There shouldn't be anything quite as terrible as the databasefixture branch. | 07:53 |
| wgrant | Since we now know that everything uses the right config. | 07:53 |
| lifeless | we're pretty close to being able to truely parallelise | 07:53 |
| wgrant | Yup. | 07:53 |
| lifeless | we need to track down the cause of leaks | 07:53 |
| wgrant | Although I ran into some trouble with launchpadlib tests last night. | 07:53 |
| lifeless | something is either not calling cleanups, or the process is being killed hard | 07:53 |
| lifeless | launchpadlib needs to be fixed | 07:54 |
| lifeless | where is that concurrent use bug | 07:54 |
| wgrant | It seems to enjoy always connecting to :8085 | 07:54 |
| wgrant | I don't know how that can work, since AppServerLayer is meant to be dynamic now. | 07:54 |
| wgrant | Oh. | 07:54 |
| wgrant | I guess we don't do custom ports yet, just custom DBs. | 07:54 |
| wgrant | And on Natty it decides to connect to :443 instead, just for fun. | 07:54 |
| lifeless | win | 07:55 |
| wgrant | So I will probably beat it to death with some monkeypatches. | 07:55 |
| lifeless | wait what | 07:55 |
| lifeless | we maintain it | 07:55 |
| wgrant | Hmm, maybe I can get it to use a custom base URI. | 07:55 |
| wgrant | lifeless: True. | 07:56 |
| wgrant | But lazr.restful has at least one 3000-line doctest. | 07:57 |
| wgrant | Which has sort of put me off that stack a bit. | 07:57 |
| lifeless | boom shaka | 07:57 |
| wgrant | Maybe I should try again. | 07:57 |
| wgrant | Also, Python decided it would be amusing to change the default email header wrapping character from \t to ' '. | 07:58 |
| wgrant | This breaks a lot of tests :D | 07:58 |
| lifeless | OTOH the only thing that should be testing serialised forms is emaillib | 07:59 |
| wgrant | lifeless: That is true, but there are a lot of other 'should' and 'should not's in LP. | 08:01 |
| lifeless | wgrant: I think I mean 'cast the serialised form to an object, use a matcher, or delete the test | 08:01 |
| wgrant | eg. you should probably not have a single monolithic 570KLOC Python app that can be logically split into lots of pieces :) | 08:02 |
| lifeless | wgrant: I hold a rather different view on the 'logical split' thing | 08:03 |
| lifeless | wgrant: I grant that there are /some/ pieces we can(and should) split out | 08:03 |
| lifeless | wgrant: but I'm not convinced that there are lots; and certainly the UI->persistence->storage story is all one logical thing | 08:04 |
| wgrant | At least Translations, Buildmaster and most of Soyuz have no business being in with the rest of LP. | 08:04 |
| lifeless | no, yes, no | 08:05 |
| wgrant | The interaction points between those three and the rest are minimal. | 08:05 |
| wgrant | And will remain so. | 08:05 |
| lifeless | to me there are two key tests | 08:06 |
| lifeless | a) could it be written to APIs | 08:06 |
| lifeless | b) would it be free of lockstep changes with the core | 08:07 |
| lifeless | if the answers are yes and yes, then I think a separate piece makes sense | 08:07 |
| lifeless | that isn't the case with translations or soyuz | 08:07 |
| lifeless | and soyuz is (finally) getting reuse and working in better with code | 08:08 |
| lifeless | translations ditto | 08:08 |
| wgrant | I'm not sure that those interactions are significant enough to prevent a split. | 08:09 |
| lifeless | I think they are massive impediments against splitting | 08:10 |
| wgrant | They make it harder than it would be otherwise. | 08:10 |
| lifeless | they are fundamental problems | 08:10 |
| lifeless | that will massively outweigh any benefits from splitting them out, unless the split edge is designed to prevent those problems occuring | 08:13 |
| lifeless | the latter problem in particular would mean terrible deployment and testing issues | 08:19 |
| stub | AttributeError: type object 'BaseLayer' has no attribute 'config' | 08:24 |
| stub | Think that means I should terminate the run :) | 08:24 |
| lifeless | thats fallout from wgrants fixes to databasefixture | 08:25 |
| wgrant | Indeed. | 08:25 |
| stub | I'll look at it after lunch | 08:25 |
| lifeless | wgrant: what was teh change | 08:26 |
| wgrant | lifeless: I was going to tell him, but then he escaped. | 08:28 |
| wgrant | I renamed config to config_fixture. | 08:28 |
| wgrant | Or config_name. | 08:28 |
| wgrant | Because the other one sprung into existence. | 08:28 |
| wgrant | don't remember which was which. | 08:29 |
| poolie | lifeless, would you care to offer an opinion on https://code.edge.launchpad.net/~mbp/launchpad/690021-rlimit/+merge/43733 | 08:40 |
| poolie | ie about setting an rlimit in cron scripts | 08:40 |
| poolie | not an emergency, just feeling a bit stuck | 08:40 |
| poolie | or anyone else for that matter | 08:40 |
| lifeless | poolie: uhm | 08:41 |
| poolie | if you're not here, you don't have to, of course | 08:41 |
| lifeless | ideally we'd give each task an even slice of the machine memory | 08:41 |
| lifeless | which implies knowing the machine memory | 08:41 |
| lifeless | I'd look to features to configure that | 08:41 |
| lifeless | as for where its set | 08:41 |
| lifeless | I don't care whether its in-proc or in-parent | 08:41 |
| poolie | you could do a lot more towards this | 08:41 |
| poolie | we could have a whole lep on the idea of resource caps on jobs | 08:42 |
| poolie | istm perhaps you want a model of "this job should never need more than x GB unless something's wrong; this machine has Y GB; run y/x of them in parallel" | 08:42 |
| poolie | or perhaps for other jobs, you just want to cap them | 08:43 |
| lifeless | that seems liable to fail with too many jobs for the cpus | 08:43 |
| poolie | oh, that too of course | 08:43 |
| poolie | anyhow, ideally it would be an operational knob, not in the code | 08:43 |
| lifeless | there are mulitple dimensions here | 08:43 |
| poolie | the thing for this particular mp is | 08:44 |
| lifeless | one is not using more resources than we have | 08:44 |
| poolie | istm it is a step improvement to at least have a cap on this job as we have on others | 08:45 |
| lifeless | another is autoconfiguration | 08:45 |
| lifeless | poolie: sure | 08:45 |
| lifeless | poolie: why don't you land it? | 08:45 |
| poolie | and no worse than having losas manually kill them, as has happened at least once (maybe only once) | 08:45 |
| poolie | i didn't want to land it over the top of aaron and tim's concerns | 08:46 |
| lifeless | fair enough | 08:47 |
| lifeless | anyhow, it seems to me that you'll want to use a mock | 08:47 |
| lifeless | because setting an rlimit in a test and then butting up against it would be unreliable at best | 08:47 |
| poolie | but i would like to restrict the scope | 08:47 |
| poolie | i agree | 08:47 |
| lifeless | if you're testing with a mock, it really doesn't matter where you set it, does it ? | 08:48 |
| poolie | a mock of what? a monkeypatched setrlimit? | 08:48 |
| lifeless | yah | 08:48 |
| poolie | oh, i mean restrict the scope to smaller than "make a new job system thx" | 08:48 |
| danilos | wgrant, hi, can you please run the rosetta-approve-imports script on dogfood please? I've made all the variant languages visible to see if that case would at least work | 09:17 |
| wgrant | danilos: Does it need a fresh upload? | 09:18 |
| danilos | wgrant, no, we've got plenty of unapproved files left | 09:18 |
| wgrant | danilos: Running. | 09:18 |
| wgrant | It's doing stuff. | 09:19 |
| danilos | wgrant, cool, thanks | 09:19 |
| danilos | wgrant, it puts stuff in arbitrary order so I can't find it in the queue | 09:19 |
| wgrant | danilos: Should I knock all the old ones to some other status? | 09:20 |
| danilos | wgrant, can you please kill the run and clean up all the TIQE entries with status != 5 (needsreview) | 09:20 |
| danilos | wgrant, or that, if it's not a big bother | 09:20 |
| danilos | wgrant, I believe status 3 is deleted so try with that :) | 09:20 |
| wgrant | danilos: That's what I'm doing. | 09:20 |
| wgrant | 1 -> 3 | 09:21 |
| wgrant | That was quick. | 09:21 |
| danilos | wgrant, cool, thanks | 09:21 |
| wgrant | Reapproving. | 09:21 |
| danilos | cool | 09:21 |
| danilos | wgrant, looks good so far, but please let it finish | 09:24 |
| wgrant | Looks good, yeah. | 09:24 |
| wgrant | danilos: It's finished. | 09:26 |
| danilos | wgrant, oh, is it perhaps because of a time-limit? can you run it again please? | 09:27 |
| wgrant | Still lots unapproved. | 09:27 |
| wgrant | Indeed, ran for a few seconds over 5 minutes. | 09:27 |
| wgrant | And now is doing lots more. | 09:27 |
| danilos | wgrant, I've made "sr@ijekavian" hidden to see what will happen with those (if it works properly, they would stay unapproved, if not, they might again be approved in "serbian" directly) | 09:28 |
| wgrant | danilos: The queue is not getting shorter. | 09:29 |
| wgrant | It's doing dozens of transactions per second, and not running for more than 30 seconds or so. | 09:29 |
| danilos | wgrant, right, then it's surprisingly working fine with hidden languages this time around as well | 09:30 |
| wgrant | danilos: Do we want to do another upload of the same file to another series and try it again from the start? | 09:31 |
| danilos | wgrant, you can perhaps kill that run and I can make "sr@ijekavian" visible again so we just make sure it works fine to clean up the queue (well, mostly: sometimes not all files have matching templates) | 09:31 |
| wgrant | Sure. | 09:31 |
| wgrant | It's finished again. | 09:31 |
| danilos | wgrant, not really, I am experiencing some weirdness locally as well | 09:32 |
| wgrant | So unhide. | 09:32 |
| danilos | wgrant, done | 09:32 |
| wgrant | I need to learn this Translations stuff. | 09:32 |
| danilos | wgrant, I want to get down to the local weirdness first, and then when I am certain why is it happening we can try it all over again :) | 09:32 |
| wgrant | danilos: Right. | 09:32 |
| wgrant | So, is it qa-ok? | 09:32 |
| danilos | wgrant, it's all very simple, and you have a head start compared to everybody else | 09:32 |
| wgrant | danilos: It's running much slower this time. | 09:32 |
| wgrant | Which may be a good sign. | 09:32 |
| danilos | wgrant, yeah, it doesn't break stuff and works fine for when languages are not hidden | 09:33 |
| wgrant | Indeed, queue is decreasing in size. | 09:33 |
| danilos | wgrant, I'll file a new bug if I find more problems with it | 09:33 |
| wgrant | danilos: Can you qa-ok the bug so I can request a deployment? | 09:33 |
| danilos | wgrant, I did that already | 09:33 |
| wgrant | Ah, great. | 09:34 |
| wgrant | Thanks. | 09:34 |
| wgrant | mthaddon: Around? | 09:34 |
| mthaddon | wgrant: yup | 09:34 |
| wgrant | mthaddon: We need r12112 deployed to cesium (to fix bug #692114). | 09:41 |
| _mup_ | Bug #692114: Recipe builds require indices for non-main PPA components <qa-ok> <recipe> <Launchpad itself:Fix Committed by wgrant> < https://launchpad.net/bugs/692114 > | 09:41 |
| wgrant | mthaddon: There is one qa-bad rev in front of it, but that only affects the webapp. | 09:41 |
| wgrant | lifeless has given me his blessing. | 09:42 |
| mthaddon | erm, maybe, but that's completely non-standard... | 09:42 |
| wgrant | It was done for Soyuz a couple of weeks ago... r12043, IIRC. | 09:43 |
| lifeless | wgrant: it will need a incident report, I assumed you already knew that :) | 09:43 |
| mthaddon | that doesn't mean we're okay to do it every time... | 09:43 |
| lifeless | mthaddon: will you be at the epic? | 09:44 |
| mthaddon | lifeless: nope - mbarnett will be there though | 09:44 |
| lifeless | ok, I want to go through our exception-handling workflows face to face, can you perhaps make sure he's across all your concerns? | 09:45 |
| danilos | wgrant, fwiw, 12111 is also still "needtesting" | 09:45 |
| wgrant | mthaddon: Ah, OK. | 09:45 |
| wgrant | mthaddon: I was under the impression that it wouldn't be that abnormal. | 09:45 |
| wgrant | I guess I will wait. | 09:46 |
| mthaddon | how critical is it? | 09:46 |
| mthaddon | lifeless: I'm not really sure I understand the scope of the conversation you're after, but we can certainly cover anything over the phone that you need to | 09:47 |
| wgrant | It breaks recipe builds into new PPAs or new series in existing PPAs, which is probably most new recipes. | 09:47 |
| mthaddon | and how long has it been doing this for? | 09:47 |
| mthaddon | and how long do we expect before we can get those revisions qa-ed? | 09:48 |
| lifeless | mthaddon: that would be great | 09:48 |
| * mthaddon nods | 09:48 | |
| wgrant | Well, buildbot is being more agreeable now, so it's not as long as I thought. Assuming that we don't get another qa-bad which immediately pushes us back another 24 hours :/ | 09:48 |
| wgrant | But these look OK so far. | 09:48 |
| lifeless | mthaddon: scope wise - we have cherrypicks, but we're also doing some adhoc things while I've been on leave | 09:49 |
| wgrant | lifeless: Er, don't we not have cherrypicks any more? | 09:49 |
| lifeless | I'd like to consolidate things a bit, and talk cost-of-execution, latency, and approvals/risk analysis | 09:49 |
| mthaddon | yeah, we've not done a cherry pick for a good while now | 09:49 |
| lifeless | wgrant: we certainly do have them in policy | 09:50 |
| wgrant | I thought we'd lost that capability. | 09:50 |
| lifeless | wgrant: we haven't had to do many - but this soyuz working-around-unqaed-things is cherrypick-like | 09:50 |
| lifeless | wgrant: not at all | 09:50 |
| wgrant | lifeless: We need some way to roll out urgent changes that is not blocked by irrelevant bugs in the webapp, each of which blocks us for a day. | 09:51 |
| lifeless | wgrant: perhaps. | 09:51 |
| lifeless | wgrant: I'm not interested in drilling into this now. At the epic I'd be delighted to. | 09:52 |
| wgrant | Sure. | 09:52 |
| wgrant | I will say that the process simplification and frequent rollouts are great. It just seems to border on ridiculous to block another one-line critical fix for days. | 09:53 |
| lifeless | so we're not meant to block for days ever | 09:54 |
| lifeless | if there is something buggered, roll it back immediately. | 09:54 |
| wgrant | But every time we have a qa-bad we block for 24 hours. | 09:54 |
| wgrant | Which then gives us time for another one to slip in. | 09:54 |
| lifeless | things are *meant* to be qa'd the same day they land. | 09:54 |
| wgrant | => infinite chain of pain | 09:54 |
| lifeless | wgrant: how it is taking 24 hours? | 09:54 |
| wgrant | lifeless: The change is landed during the engineer's working day. | 09:55 |
| wgrant | 4 hours of EC2 + 1 hour of PQM + 6 hours of buildbot takes us well after the end of their day. | 09:55 |
| lifeless | and? | 09:55 |
| wgrant | They QA it the following morning. | 09:55 |
| wgrant | Notice that it's bad. | 09:55 |
| wgrant | Send the rollback through EC2. | 09:55 |
| wgrant | Another 11 hours later, it is in stable and can be QAd. | 09:55 |
| lifeless | so, flaw one: don't wait for the engineer to qa it. | 09:55 |
| lifeless | we're strongly encouraged to describe how to qa things in merge proposals. | 09:56 |
| lifeless | 2) rollbacks go straight to pqm, no ec2. | 09:56 |
| wgrant | s/rollback/fix/, then. | 09:57 |
| lifeless | don't fix | 09:57 |
| lifeless | rollback | 09:57 |
| wgrant | Since avoiding rollbacks in DVCS\{darcs} is nice. | 09:57 |
| lifeless | no, its insane. | 09:57 |
| lifeless | rollbacks are how we undo mistakes rapdily. | 09:57 |
| lifeless | there is no opprobobium in having something rolled back | 09:58 |
| lifeless | and it removes all the latency involved in analysing and fixing the issue in the bad commit | 09:59 |
| lifeless | wgrant: if something was broken, and I was working, I'd land a rollback with no hesitation at all | 10:03 |
| lifeless | wgrant: if we *don't* do this, the expected result is a trunk that is broken a lot. | 10:03 |
| wgrant | lifeless: So, it sounds like everyone needs to know three things: 1) QA quickly. 2) QA other people's stuff. 3) Rollback is the first resort, not the last. | 10:04 |
| lifeless | wgrant: makes sense to me | 10:05 |
| lifeless | wgrant: this has been communicated before, but it bears repeating. | 10:05 |
| lifeless | wgrant: I'll be talking about this in my TA report too | 10:05 |
| wgrant | Nobody treats QA as priority 1. | 10:05 |
| wgrant | When in reality it probably should be. | 10:05 |
| lifeless | wgrant: the logic behind this is simple; once a patch is in trunk, its on the critical path to deploy, anywhere. | 10:06 |
| lifeless | The only thing higher priority than QA is fixing a production issue directly. | 10:06 |
| lifeless | wgrant: this is why I was making such a big deal on your first? second? day about qaing when you had a blocked thing | 10:07 |
| danilos | wgrant, fwiw, I figured what the problem is with "sometimes doesn't work": if we have paths stored in pofiles in the DB (due to the previous buggy behavior) it can wrongly take up a wrong pofile early on | 10:18 |
| wgrant | danilos: Ah, great. | 10:18 |
| === lifeless_ is now known as lifeless | ||
| pcjc2 | allenap, deryck: Is there anything you need me to chase RE: Contributor agreement? | 11:04 |
| pcjc2 | (https://code.launchpad.net/~pcjc2/launchpad/allow-empty-comments/+merge/43449_ | 11:04 |
| pcjc2 | ) | 11:04 |
| henninge | danilos: do you know what the problem might have been with bug 487137? | 11:04 |
| _mup_ | Bug #487137: Allow Rosetta admins to create custom language codes <bugjam2010> <lp-translations> <Launchpad itself:In Progress by henninge> < https://launchpad.net/bugs/487137 > | 11:04 |
| henninge | danilos: Adi had a branch ready a month before his last comment on the branch (and unassigning himself from it). | 11:05 |
| danilos | henninge, I was just looking at that and planning to get it landed | 11:06 |
| henninge | danilos: ;) | 11:06 |
| danilos | henninge, yeah :) | 11:06 |
| danilos | henninge, if you want to take it over and go through it to ensure it's all still good, go for it ;) | 11:06 |
| henninge | danilos: but what is Adi's last comment about? Is the branch flawed? | 11:06 |
| danilos | henninge, well, last comment is from Michael, but just because of the status we should basically re-review it now | 11:10 |
| henninge | danilos: sorry, Adi's last comment on the *bug* | 11:11 |
| henninge | that's a month after Michael looked at the mp | 11:11 |
| danilos | henninge, oh, I think it's easy: we can just let project owners administer their own custom language codes | 11:12 |
| henninge | ;-) | 11:13 |
| danilos | henninge, especially since there won't be any per-app teams anymore, I think that's the way to go | 11:13 |
| henninge | good point | 11:13 |
| danilos | henninge, who ever screws it up for themselves, well, they've done it themselves | 11:13 |
| danilos | henninge, fwiw, we should make them able to do translations approval as well | 11:13 |
| henninge | hah! | 11:14 |
| danilos | henninge, it's just that we won't have the people to monitor it anyway, so we should just worry about being able to clean up later | 11:14 |
| henninge | good thing it's just the two of us here ... | 11:14 |
| danilos | henninge, heh | 11:14 |
| danilos | henninge, anyway, is that how Adi's branch is working? (i.e. using TranslationsAdmin privilege directly) | 11:15 |
| danilos | henninge, (if you looked at it) | 11:16 |
| danilos | henninge, MP suggests it is | 11:16 |
| henninge | sorry, was afk | 11:25 |
| lifeless | danilos: I think in general per-app responsibilities will become interrupt-team responsibilities; its not that there is noone to do it, its that its no longer such a static group of people | 11:28 |
| danilos | lifeless, and this is one of responsibilities that it'd be better to hand of to project owners than to hand of to that team, that's all I am saying | 11:29 |
| lifeless | danilos: +1 | 11:29 |
| lifeless | danilos: I thought that perhaps you felt there wouldn't be staff to do it at all, rather than that project owners were a better natural fit | 11:30 |
| lifeless | danilos: so I was trying to clear that up, was all. | 11:30 |
| danilos | lifeless, well, the staff that will remain will not really be up to the task either, it's a complex work and you have to be careful (I am talking about import queue management) because it's easy to mess stuff up | 11:30 |
| lifeless | danilos: is that an ops thing perhaps? | 11:32 |
| danilos | lifeless, ideally, we wanted to transfer that to project owners only when we made it very hard to make mistakes, but I think it's better to let owners make mistakes now and be able to fix it to at least some extent than to have owners depend on a team that might make mistakes as well and depend on them to fix them afterwards | 11:32 |
| danilos | lifeless, "ops" as in operation? yes | 11:32 |
| danilos | operational, that is | 11:32 |
| lifeless | danilos: I love the idea of being able to fix things rather than obsessing about preventing them | 11:32 |
| lifeless | scales better | 11:32 |
| lifeless | feels easier to use | 11:32 |
| danilos | lifeless, the problem is that we can't really fix them atm | 11:32 |
| danilos | lifeless, which is why the permissions are very restricted | 11:33 |
| lifeless | danilos: sure, we have some work to do to get there | 11:33 |
| lifeless | its too late for me to try to understand the details | 11:33 |
| danilos | when I say "really", I mean "make sure the DB is in the best possible state"; it's not hard to make it appear relatively decent to outside users | 11:33 |
| lifeless | would love to do so at the epic perhaps | 11:33 |
| danilos | lifeless, yeah, sounds good, I guess thunderdome is a good place to do that | 11:33 |
| === _thumper_ is now known as thumper | ||
| allenap | pcjc2: I'll follow up on the contributor agreement now. | 11:41 |
| allenap | pcjc2: I think my colleague might have been looking in the wrong place; your name is already on the signed list, and has been since the 17th. I'll land your branch now. Thank you, and sorry for the confusion :) | 11:46 |
| pcjc2 | Thanks! | 11:46 |
| pcjc2 | danilos: +2^n (n large), for letting projects manage their own translation imports - PRETTY PLEASE ;) | 11:47 |
| danilos | pcjc2, if that +2^n translates into someone doing to work for cleaning up the mess that people can make, I'll go all-in on that bid :) | 11:48 |
| danilos | pcjc2, but will do it even if that work is not done to the full extent | 11:49 |
| pcjc2 | Hard to know - would some clean-uprequire a LOSA? | 11:49 |
| pcjc2 | or are you talking about extending LP code to allow more web-access to undo screw-ups? | 11:50 |
| allenap | pcjc2: Is there a bug related to that fix? | 11:52 |
| pcjc2 | no, sorry | 11:52 |
| allenap | pcjc2: Okay, I'll file one, just so we can track QA. | 11:52 |
| pcjc2 | ok, thanks | 11:52 |
| deryck | Morning, all. | 12:06 |
| danilos | pcjc2, it's about extending LP to allow more clean-ups and do some on it's own | 12:13 |
| danilos | pcjc2, for instance, it's totally impossible to remove templates today, and when they are left around they just cause more problems for people | 12:13 |
| danilos | anyone around who can help QA bug 670452, bug 619555 and bug 504080? (trying to get something rolled out, so need to go all the way to 12121) | 12:38 |
| _mup_ | Bug #670452: Hard to find related branches when composing recipe <lp-code> <qa-needstesting> <recipe> <Launchpad itself:Fix Committed by wallyworld> < https://launchpad.net/bugs/670452 > | 12:38 |
| _mup_ | Bug #619555: cronscripts/request_daily_builds.py is not verbose enough on default logging <bugjam2010> <canonical-losa-lp> <lp-code> <qa-needstesting> <recipe> <Launchpad itself:Fix Committed by thumper> < https://launchpad.net/bugs/619555 > | 12:38 |
| _mup_ | Bug #504080: Please put the URL to the merge proposal in the body of the email <bugjam2010> <code-review> <email> <lp-code> <qa-needstesting> <trivial> <Launchpad itself:Fix Committed by abentley> < https://launchpad.net/bugs/504080 > | 12:38 |
| danilos | Ursinha, matsubara-afk: hi, where's the "specification" of all the bug tags that qa-tagger handles (I am wondering how do I tag a bug to indicate that one revision fixes a previous one) | 12:41 |
| Ursinha | danilos, let me find the link for you | 12:42 |
| danilos | Ursinha, I did find https://dev.launchpad.net/QAForContinuousRollouts though only through the mail archive, is there anything more complete? | 12:43 |
| Ursinha | danilos, that's a bit of a mess... here's the page I know: https://dev.launchpad.net/QAProcessContinuousRollouts | 12:46 |
| danilos | Ursinha, right, that one is much better | 12:47 |
| === jelmer__ is now known as jelmer | ||
| danilos | Ursinha, so, I guess the right approach to landing this should have been a rollback and then a full fixed landing | 12:48 |
| Ursinha | danilos, yes, I think so | 12:49 |
| danilos | Ursinha, should we perhaps add a 'fixes-REVNO' tag as well? (just wondering, because if rollback=REVNO was included in this landing I am looking at it would have been sufficient even though it doesn't really roll the change back) | 12:51 |
| Ursinha | danilos, the tag is bad-commit-firstrevno | 12:51 |
| Ursinha | where firstrevno is the defective one you want to rollback | 12:51 |
| danilos | Ursinha, well, look at bug 118284 | 12:52 |
| _mup_ | Bug #118284: URLs ending with a ) aren't linkified properly <bad-commit-12102> <bugjam2010> <lp-web> <qa-ok> <tales> <Launchpad itself:Fix Committed by jcsackett> <Ubuntu:Invalid> < https://launchpad.net/bugs/118284 > | 12:52 |
| danilos | Ursinha, 12102 is a bad-commit, tagged like that, and it was later fixed by 12121 (for the same bug) | 12:52 |
| Ursinha | danilos, fixed or rolledback | 12:53 |
| Ursinha | ? | 12:53 |
| danilos | Ursinha, fixed | 12:53 |
| Ursinha | so the first revision can be rolled out to production? | 12:54 |
| Ursinha | 12102 | 12:54 |
| danilos | Ursinha, so, 12121 is good to go out, but nothing between 12102 and 12121 isn't because 12102 shouldn't go out without 12121 | 12:54 |
| danilos | Ursinha, no | 12:54 |
| danilos | Ursinha, basically, regarding qa-tagger, it should behave exactly like rollback=12102 imo | 12:54 |
| danilos | Ursinha, but I am guessing developers won't use that because it doesn't make much sense | 12:55 |
| Ursinha | right, but now it doesn't | 12:55 |
| Ursinha | yeah | 12:55 |
| Ursinha | I guess the approach is a rollback then a full fixed branch | 12:55 |
| danilos | Ursinha, the difference between this and rollback is that this requires QA, where rollback doesn't | 12:55 |
| Ursinha | I get it | 12:55 |
| danilos | Ursinha, yeah | 12:55 |
| Ursinha | wondering how to proceed now | 12:56 |
| Ursinha | well, if you already qaed the fix, removing the tag is safe | 12:56 |
| Ursinha | right now deployments are blocked due to the bad-commit tag, so when you think that can land, just remove the tag | 12:56 |
| Ursinha | danilos, would you mind filing a bug against qa-tagger about it? | 12:57 |
| danilos | Ursinha, not at all | 13:01 |
| danilos | Ursinha, bug 692978 | 13:06 |
| _mup_ | Bug #692978: No way to mark a revision as fixing another bad revision <qa-tagger:New> < https://launchpad.net/bugs/692978 > | 13:06 |
| Ursinha | danilos, merci | 13:07 |
| === henninge_ is now known as henninge | ||
| === beuno_ is now known as beuno | ||
| abentley | rockstar, https://code.qastaging.launchpad.net/~abentley/bzrtools is giving a KeyError about "mergequeues". Do you know anything about this? http://pastebin.ubuntu.com/546278/ | 14:55 |
| === ]reed[ is now known as [reed] | ||
| rockstar | abentley, no, I don't know what that's about. | 15:15 |
| abentley | rockstar, okay, thanks. | 15:16 |
| rockstar | abentley, I don't know if wallyworld is working on merge queues, but I didn't but anything about mergequeues on the branch index page. | 15:18 |
| Ursinha | any soyuz people around? | 15:29 |
| === jkakar_ is now known as jkakar | ||
| === beuno is now known as beuno-lunch | ||
| allenap | sinzui: Did I accidentally fix bug 56038? | 16:24 |
| _mup_ | Bug #56038: BugField should define some constraints <bugjam2010> <lp-bugs> <trivial> <Launchpad itself:Fix Released by allenap> < https://launchpad.net/bugs/56038 > | 16:24 |
| sinzui | yes | 16:24 |
| allenap | sinzui: \o/ | 16:25 |
| === deryck is now known as deryck[lunch] | ||
| === beuno-lunch is now known as beuno | ||
| === benji is now known as benji-lunch | ||
| === al-maisan is now known as almaisan-away | ||
| === deryck[lunch] is now known as deryck | ||
| === benji-lunch is now known as benji | ||
| === leonardr is now known as leonardr-dentist | ||
| thumper | morning | 19:28 |
| thumper | abentley: s = u'Hello \N{SNOWMAN}' | 20:04 |
| EdwinGrubbs | StevenK, jelmer_, wgrant: can I assign this question to one of you? https://answers.launchpad.net/launchpad/+question/138604 | 20:16 |
| jelmer__ | EdwinGrubbs: sure | 20:17 |
| === almaisan-away is now known as al-maisan | ||
| === jelmer__ is now known as jelmer | ||
| EdwinGrubbs | jelmer: is it possible to build a package from a branch yet, or do you still need to use dput? | 20:25 |
| jelmer | EdwinGrubbs: That's an odd problem (The question) | 20:26 |
| jelmer | EdwinGrubbs: You can build from a branch without a dput but you'll need a recipe, which is probably more work than a dput at this point. | 20:26 |
| abentley | jelmer, I dunno about that-- we have a reasonable default recipe. | 20:27 |
| jelmer | abentley: "bzr bd && dput ../foo.changes" is quicker (and doesn't require a multi-minute recipe build) than clicking create recipe, adjusting the default recipe and requesting a build imho. | 20:28 |
| thumper | EdwinGrubbs: you can point the user to the help page | 20:30 |
| jelmer | abentley: Don't get me wrong, I'm a big fan of recipe builds and I think we're as close as we've ever been to building from branch but it's still not quite as easy as clicking a "Build this revision as a package" button in the web UI. | 20:30 |
| thumper | EdwinGrubbs: https://help.launchpad.net/Packaging/SourceBuilds | 20:31 |
| EdwinGrubbs | thanks | 20:32 |
| === gary_poster_ is now known as gary_poster | ||
| === gary_poster_ is now known as gary_poster | ||
| lifeless | gmb: hi | 21:44 |
| lifeless | gmb: you might like lp:~lifeless/launchpad/persistence | 21:44 |
| lifeless | gmb: I couldn't stop thinking about it, so I wrote down more science fiction. | 21:44 |
| jcsackett | wgrant, you around? | 22:26 |
| wgrant | jcsackett: Indeed. | 22:27 |
| jcsackett | wgrant: just wanted to let you know that the fix on the branch/bug we chatted about yesterday has gone through, but the bad-commit-tag can't be removed b/c there are revisions between that need to be qa'ed. | 22:28 |
| wgrant | jcsackett: Yeah, I saw that... but there's now *another* bad rev before the fix. | 22:28 |
| wgrant | Just for added fun. | 22:28 |
| jcsackett | wgrant: yeah, i just saw that qa-bad; it's weird, i qa'ed that this morning trying to get teh queue cleared out before i hit bugs i had no idea how to qa. thought that one looked good, but apparently someone more familiar with it saw issues i did not. | 22:30 |
| jcsackett | anyway, just wanted to keep you up to date; sorry there is yet further difficulty in getting to your revision. :- | 22:31 |
| wgrant | Breakage happens. | 22:32 |
| wgrant | But our process cannot deal with it :( | 22:32 |
| === al-maisan is now known as almaisan-away | ||
| === leonardr-dentist is now known as leonardr | ||
| poolie | hello | 23:52 |
| poolie | could somebody please send https://code.edge.launchpad.net/~mbp/launchpad/690021-rlimit/+merge/43733 to pqm for me? | 23:52 |
| wgrant | poolie: Has it been through ec2? | 23:52 |
| poolie | no | 23:53 |
| wgrant | Also, thumper just nak'd it. | 23:53 |
| poolie | ok, fine | 23:54 |
| poolie | thumper: so shall i just reject it? | 23:55 |
| poolie | it was just a kind of drive by | 23:55 |
Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!