| wgrant | WTF is ~itiseasy-org, and why has it uploaded several hundred packages overnight? | 04:28 |
|---|---|---|
| wgrant | Like we needed more in the queue... | 04:28 |
| lifeless | wgrant: dunno | 06:08 |
| wgrant | lifeless: Could you land https://code.edge.launchpad.net/~wgrant/launchpad/replace-archiveuploader-doctests-0/+merge/30850? You approved it a week ago. | 06:21 |
| lifeless | kicking it off | 06:27 |
| lifeless | wgrant: you didn't find anyone to do it for a week ? | 06:28 |
| wgrant | lifeless: No, I just forgot about it. | 06:29 |
| lifeless | :) | 06:29 |
| wgrant | Thanks. | 06:30 |
| wgrant | Do we have working feature flag support yet? | 06:31 |
| wgrant | I saw some stuff land. | 06:31 |
| wgrant | And I'd like to disable something with it, if it does indeed exist... | 06:31 |
| lifeless | db-devel only | 06:31 |
| lifeless | could probably port the interface to devel | 06:32 |
| lifeless | with some care | 06:32 |
| wgrant | No need. | 06:32 |
| lifeless | and then your thing would be disabled by it but able to be controlled on staging & next release | 06:32 |
| wgrant | There's only a few days left until the freeze. | 06:32 |
| lifeless | last freeze ever! | 06:33 |
| wgrant | Oh? | 06:33 |
| wgrant | We're moving to the new merge workflow for 10.09? | 06:33 |
| wgrant | Well, what was 10.09. | 06:33 |
| lifeless | I think all the bits are in place | 06:33 |
| lifeless | if not then very very close | 06:34 |
| wgrant | Wow. | 06:34 |
| wgrant | That was swift. | 06:34 |
| * wgrant whines about tree builds taking far too long. | 06:34 | |
| wgrant | 4 minutes and counting... | 06:35 |
| lifeless | wgrant: using ? | 06:35 |
| lifeless | wgrant: well the basic bits are: | 06:35 |
| wgrant | lifeless: The initial make in a new branch. | 06:36 |
| lifeless | - a daily staging [trivialish] | 06:36 |
| lifeless | - a way to disable stuff [feature flags, rough but usable] | 06:36 |
| lifeless | - new QA toolchain/automation [done, I believe] | 06:36 |
| wgrant | Right, I wan't aware that the last bit was even started. | 06:37 |
| wgrant | So the new system will take all the branches in the queue, merge them, test them as a whole, then commit them? | 06:37 |
| lifeless | no | 06:39 |
| lifeless | thats totally different | 06:39 |
| wgrant | Oh. | 06:39 |
| wgrant | So just the new QA workflow, not the new merge workflow? | 06:39 |
| lifeless | thats an optimisation for landing stuff | 06:39 |
| wgrant | Ah. | 06:39 |
| wgrant | I se. | 06:39 |
| lifeless | https://dev.launchpad.net/LEP/ReleaseFeaturesWhenTheyAreDone | 06:39 |
| lifeless | https://dev.launchpad.net/MergeWorkflowDraft specifically | 06:40 |
| lifeless | https://dev.launchpad.net/MergeWorkflowDraft?action=AttachFile&do=view&target=merge-workflow-draft-2.jpeg is the thing we drew | 06:40 |
| lifeless | or maris drew, I should say | 06:40 |
| wgrant | Hm. that looks a lot simpler than the last draft I saw. | 06:41 |
| lifeless | if the last thing you was before the epic, then yes | 06:41 |
| wgrant | Right. | 06:41 |
| lifeless | We revisited the entire proposal there | 06:41 |
| lifeless | we also need a patch to make 'edge' behaviour depend on the vhost, not on the confi | 06:43 |
| lifeless | config | 06:43 |
| lifeless | I don't think anyone has done that yet, would be nice to do. | 06:43 |
| lifeless | though the meaning of edge is going to erode pretty quickly | 06:45 |
| wgrant | Yes. | 06:48 |
| wgrant | I also suppose that the automation of service restarts should make DB upgrades a whole lot quicker. | 06:49 |
| lifeless | oh yeah thats the other key thing, but its necessary for ratcheting up the frequency, not for switching at all | 06:49 |
| wgrant | Yep. | 06:50 |
| lifeless | wgrant: nuts | 11:13 |
| wgrant | lifeless: Um, devel is broken on 2.5? | 15:47 |
| wgrant | Ew. | 15:47 |
| wgrant | I don't see how that happens, unless people aren't ec2ing branches before landing... | 15:50 |
| jelmer | hey wgrant | 16:01 |
| wgrant | Morning jelmer. | 16:03 |
| jelmer | wgrant, it's broken how? | 16:03 |
| wgrant | jelmer: test_sourcepackagerecipe.py doesn't appear to import with_statement. | 16:03 |
| jelmer | ah | 16:04 |
| jelmer | wgrant, not everybody uses ec2, some people run make test locally | 16:04 |
| wgrant | Can someone please EC2 https://code.edge.launchpad.net/~wgrant/launchpad/replace-archiveuploader-doctests-0/+merge/30850? It was caught in the devel breakage last night. | 22:17 |
| wgrant | So, the build farm has currently been unusable for five days. | 22:17 |
| wgrant | I am failing to see how this is not a critical service issue. | 22:17 |
| wgrant | One of Launchpad's most popular features is broken. | 22:18 |
| lifeless | wgrant: kicked it off again | 22:22 |
| lifeless | wgrant: uhm, I think its an important issue. | 22:22 |
| wgrant | It happens time and time again. | 22:22 |
| lifeless | wgrant: builds > builders for extended periods, + starvation events ? | 22:23 |
| wgrant | lifeless: The sustained reduction of the build farm is an issue. | 22:24 |
| wgrant | As well as prioritisation during extended starvation events. | 22:24 |
| wgrant | But new builds will be waiting for nearly four days. | 22:24 |
| lifeless | so, the whole scoring thing is what drives starvation | 22:25 |
| lifeless | its implementation not policy, and because its complex few people can actually see clearly to define a policy | 22:25 |
| wgrant | Scoring doesn't drive starvation. It just means that the dailies build before the realtime packages. | 22:25 |
| wgrant | So it exacerbates the effects. | 22:25 |
| lifeless | I disagree, but this is an assessment of the impact of the design choice; we both agree that the behaviour is undesirable | 22:26 |
| wgrant | Starvation is going to occur whatever the ordering of the builds. | 22:26 |
| wgrant | It's just going to be less noticable if realtime stuff builds first. | 22:26 |
| lifeless | by realtime do you mean 'manually uploaded single packages' ? | 22:27 |
| wgrant | Yes. | 22:27 |
| wgrant | Things that aren't going to be unnoticed if they don't build for a few days. | 22:27 |
| wgrant | ie. not dailies, and not rebuilds. | 22:27 |
| lifeless | sure | 22:27 |
| lifeless | so short term, we can futz with the scoring algorithms | 22:28 |
| lifeless | builder shortages, and inefficient use of builders, exacerbate the situation | 22:28 |
| wgrant | Longer-term: where did all those builders go for two or three days? | 22:28 |
| lifeless | QA | 22:28 |
| lifeless | only a fraction of the PPA buildds are dedicated to LP | 22:28 |
| wgrant | Odd that it takes so long. | 22:28 |
| wgrant | I know, yes. | 22:28 |
| wgrant | But normally they're only gone for a day at most. | 22:29 |
| wgrant | Except when they're taken for mirrors. | 22:29 |
| lifeless | the handover isn't flawless | 22:29 |
| lifeless | so one possibility is a failed handover and they end up in limbo | 22:29 |
| lifeless | a system wide failure in the handover would explain a batch not coming back | 22:30 |
| lifeless | anyhow, with bigjools fix to the sequences nearly here | 22:31 |
| lifeless | your branch is testing | 22:31 |
| wgrant | Thanks. | 22:31 |
| lifeless | s/sequences/sequencer/ we should get much more juice from the buildds, and be in a good position to talk about expanding the farm with less transient resources | 22:31 |
| wgrant | Right. | 22:32 |
| wgrant | Although buildd-manager isn't toooo bad with this few builders. | 22:32 |
| james_w | if someone is bored: https://code.edge.launchpad.net/~james-w/launchpad/buildout-doc/+merge/31465 | 23:26 |
| james_w | wgrant: has a scheduler based on quotas for people/archives been discussed? | 23:27 |
| wgrant | james_w: Discussed? No. | 23:28 |
| wgrant | It gets mentioned occasionally. | 23:28 |
| wgrant | But nothing comes of it. | 23:28 |
| james_w | what do we need to do to get a new scheduling algorithm? | 23:29 |
| wgrant | 1) Adjust findBuildCandidate | 23:30 |
| james_w | is it just a case of trying something, or does there need to be a discussion about what direction to go in? | 23:30 |
| wgrant | 2) Adjust dispatch time estimation | 23:30 |
| wgrant | I don't know. | 23:30 |
| wgrant | 1) is easy. | 23:30 |
| wgrant | 2) is very difficult. | 23:30 |
| wgrant | So it's not as easy as it was two years ago :( | 23:30 |
| wgrant | Although, if the current trend continues we can just replace the estimation with 'give up'. | 23:31 |
| james_w | p.s. I have an ArchiveCollection running through ec2 to catch any issues before review, and I'm currently taking a hatchet to some soyuz tests to remove sampledata uses | 23:32 |
| james_w | yay for a bored weekend | 23:32 |
| wgrant | Ooh, excellent. | 23:32 |
| wgrant | I've been meaning to see how many tests I can get to not use sampledata simply by tweaking STP. | 23:33 |
| james_w | yeah, I'd never seen it before, but I'm mainly stopping things from using that | 23:35 |
| james_w | the factory is just about as good for the uses I have seen so far | 23:35 |
| wgrant | Oh. | 23:36 |
| wgrant | STP is use pretty extensively throughout the test suite (including brand new stuff). | 23:36 |
| james_w | yeah | 23:36 |
| wgrant | It probably should be merged into the factory, but it does lots of stuff that the factory does not. | 23:37 |
| wgrant | I'm also not a fan of the factory's monolithic approach. | 23:38 |
| wgrant | It doesn't seem to be very compatible with the ongoing attempts at modularity. | 23:38 |
| james_w | indeed | 23:42 |
| james_w | I've only gone through about 10 methods so far, and they made little use of the publisher I think | 23:42 |
| wgrant | For a lot of things, makeBinaryPackagePublishingHistory will indeed suffice. | 23:43 |
Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!
