| james_w | Can't the metazcml lazy-instantiate things registered as utilities? | 00:01 |
|---|---|---|
| wgrant | One would think so. | 00:03 |
| wgrant | It might make things not take several ages to start. | 00:03 |
| james_w | It's probably not a huge percentage of the zcml processing time, but not running all that code wouldn't hurt the time | 00:13 |
| james_w | wgrant: https://code.edge.launchpad.net/~james-w/launchpad/no-more-sampledata-0/+merge/31470 <- what do you think? | 00:17 |
| wgrant | james_w: Looks good. | 00:29 |
| wgrant | It even removes the factory's manual BPPH instantiation, which makes me happy (I have a branch removing the rest already). | 00:30 |
| james_w | wgrant: you have a branch to do it for SPPH too? | 00:31 |
| wgrant | Although you should probably do the same with SPPH too. | 00:31 |
| wgrant | james_w: Yes. I got rid of everything except a couple of tests, STP and the factory. | 00:31 |
| james_w | ah | 00:32 |
| wgrant | Because the factory sets extra attributes, and I wasn't sure if rSP was really prettier. | 00:32 |
| james_w | I can do the same change for SPPH, I was just being lazy, and it's going to conflict anyway | 00:32 |
| wgrant | But it seems it is. | 00:32 |
| wgrant | What's it going to conflict with? | 00:33 |
| wgrant | I didn't touch the factory, for the above reason. | 00:33 |
| james_w | sinzui has a branch to return a proxied SPPH | 00:33 |
| wgrant | Ah. | 00:34 |
| james_w | using the method I used in this branch | 00:34 |
| james_w | not a big problem | 00:34 |
| wgrant | I wonder if 'make lint' should whinge about c.l.i and c.l.d imports. | 00:37 |
| wgrant | Probably. | 00:37 |
| wgrant | james_w: I like the look of that ArchiveCollection branch, too. | 01:07 |
| wgrant | But... 'isRequireVirtualized'? | 01:07 |
| james_w | yeah, it sucks | 01:32 |
| james_w | maybe .requiresVirtualization()? | 01:32 |
| wgrant | james_w: Sounds good. | 01:35 |
| mtaylor | it would be really nice if I could edit the target branch in a merge proposal | 02:44 |
| james_w | wgrant: would a tweak to score() to subtract some constant multiplied by the number of builds in the same archive in the last 24 hours do any good, or would it just be piling more on top of a shaky system? | 02:50 |
| james_w | mtaylor: there is some discussion in http://www.mail-archive.com/launchpad-dev@lists.launchpad.net/msg03646.html about that sort operation on merge proposals | 02:51 |
| mtaylor | james_w: excellent | 02:55 |
| wgrant | james_w: bigjools suggested that a couple of days ago. | 03:03 |
| wgrant | It might work. | 03:03 |
| wgrant | But it's definitely piling more on top of a shaky system. | 03:03 |
| james_w | the difficultly with other schemes is not materialising lots of rows for each of a 1000 builds every time you want to find the next build to dispatch, or to score them all | 03:06 |
| james_w | a stored procedure to calculate scores might be an option | 03:06 |
| james_w | otherwise you are looking at caching scores, which brings you back to something like you have now | 03:06 |
| wgrant | james_w: Yes, that's the trouble :( | 03:24 |
| === Ursinha-afk is now known as Ursinha | ||
| wgrant | james_w: Alternatively we could have a materialised queue somewhere other than an RDBMS. | 03:32 |
| wgrant | Like, something that models a queue. | 03:32 |
| james_w | wgrant: that's true | 03:32 |
| james_w | I'd be wary about behaviour if whatever that was went down and had to be repopulated from the DB | 03:33 |
| wgrant | Yes... | 03:34 |
| james_w | wgrant: do you know of something that would work well there? | 03:35 |
| james_w | the few things I know are very simplistic in that aspect, and don't work well with high contention | 03:36 |
| wgrant | I don't, no. | 03:36 |
| lifeless | uhm | 04:09 |
| lifeless | anything that requires O(queue length) is something I'd worry about | 04:09 |
| lifeless | why not simply reduce the score at upload time - I suggested that last week | 04:09 |
| wgrant | Possibly. | 04:11 |
| wgrant | I guess that's reasonably flexible. | 04:12 |
| wgrant | Simple reducing based on the number of pending builds for the archive is probably insufficient. But it might be a start. | 04:12 |
| lifeless | score = current_score_calc - len(target_build_queue) | 04:12 |
| lifeless | right | 04:12 |
| wgrant | I think we should probably throw out everything we currently know about build scores, and work out what we actually want from then. | 04:13 |
| wgrant | 2505 is a bit magical for my liking. | 04:13 |
| lifeless | we should throw out build scores | 04:13 |
| wgrant | Didn't you just say we should reduce them? | 04:14 |
| wgrant | Isn't that inconsistent with throwing them out entirely? | 04:14 |
| lifeless | short term vs long term | 04:15 |
| wgrant | How do you propose to do away with them? | 04:15 |
| lifeless | the problem with build scores is they try to create a single global static sort | 04:15 |
| wgrant | Yep. | 04:15 |
| lifeless | what we actually have is multiple queues vying for a single resource | 04:15 |
| lifeless | modelling that directly, whether in rabbitmq or in postgresql, will be much more successful at giving a good experience | 04:16 |
| lifeless | see for instance the reams of work done on network packet queuing | 04:16 |
| wgrant | Right. | 04:16 |
| james_w | bug 612351 was a bit of a headscratcher | 21:16 |
| _mup_ | Bug #612351: getPackagesetsForUploader uses Store.find, which doesn't return an SQLObjectResultSet <Soyuz:New> <https://launchpad.net/bugs/612351> | 21:16 |
| jelmer | yuck | 21:17 |
| lifeless | morning | 21:24 |
| james_w | morning lifeless | 21:37 |
| lifeless | james_w thanks for that wiki page | 21:50 |
| lifeless | james_w: but perhaps you could re-read the first sentence :) | 21:51 |
| lifeless | or tow | 21:51 |
| lifeless | bah | 21:51 |
| lifeless | or two | 21:51 |
| lifeless | thanks! | 21:52 |
| james_w | fixed, thanks | 21:52 |
| jelmer | 'morning lifeless, poolie, thumper | 22:34 |
| thumper | hi jelmer | 22:34 |
| thumper | how goes the plugin updates? | 22:34 |
| jelmer | thumper: https://code.edge.launchpad.net/~jelmer/launchpad/update-bzr-foreign/+merge/31438 | 22:34 |
| jelmer | thumper: Does that answer your question ? :-) | 22:34 |
| thumper | maybe... | 22:35 |
| wgrant | jelmer: Do you have a moment to approve my latest changes to https://code.edge.launchpad.net/~wgrant/launchpad/refactor-nuf-creation/+merge/30851, or should I ask tomorrow? | 22:36 |
| jelmer | wgrant: Was that your file classification branch? If so, I can have a look now. | 22:37 |
| wgrant | jelmer: Yep, that's the one. | 22:37 |
| jelmer | wgrant: Great, happy to re-review now then (once loggerhead cooperates) | 22:39 |
| wgrant | jelmer: Yay Loggerhead. | 22:39 |
| poolie | hi jelmer, wgrant, thumper | 22:39 |
| thumper | jelmer: do we have a bzr-hg update? | 22:39 |
| wgrant | It normally works after a refresh or two for launchpad branches. | 22:39 |
| wgrant | Morning poolie. | 22:39 |
| jelmer | thumper: No, none of the changes were really significant enough to warrant a round of QA and a release. | 22:40 |
| thumper | jelmer: ack | 22:41 |
| thumper | ah crap | 22:41 |
| thumper | resolving conflicts between stable and db-devel | 22:42 |
| thumper | and there are significant changes in the source package recipe build code | 22:42 |
| thumper | on both devel and db-devel | 22:42 |
| thumper | tests are failing locally | 22:42 |
| * thumper needs to choose what to delete | 22:42 | |
| thumper | ✁☹ | 23:16 |
| lifeless | heh | 23:18 |
| lifeless | we're working to eliminate this source of waste | 23:18 |
| wgrant | If I'd already convinced lamont and bigjools of https://code.edge.launchpad.net/~wgrant/launchpad/multi-arch-builders, buildd admins could just flip most of the excess amd64 builders onto i386 :( | 23:18 |
| jelmer | wgrant: what's the argument against that? Predictability? | 23:19 |
| wgrant | jelmer: I don't know. I will ask about it again tonight. | 23:19 |
| wgrant | Because it would be reallly handy. | 23:19 |
| wgrant | There is the issue that you don't know if the builder can build amd64 or not. | 23:19 |
| wgrant | But that will become a non-issue once LP gains support for dispatching multiple archs to the one builder. | 23:20 |
| === NC|Alaska is now known as NCommander | ||
| wgrant | But that means fixing the queue size estimation algorithms in a way that is not at all obvious. | 23:20 |
| jelmer | Yeah, I can see how that would be an issue. | 23:21 |
| jelmer | Then again, if we'd process things quicker there's less need for proper time estimation... | 23:21 |
| wgrant | Yes... | 23:21 |
| wgrant | We might also be able to take shortcuts if all of the x86 builders are amd64-capable. | 23:21 |
| wgrant | It's going to be very difficult if we can't. | 23:22 |
| wgrant | It's a bit sad that the main barrier to shorter queues is calculating the length of those same queues. | 23:24 |
| lifeless | uhm | 23:24 |
| lifeless | the estimating system is already terrible | 23:24 |
| lifeless | don't worry about making it slightly less perfect if its an overall win | 23:25 |
| wgrant | 'Slightly less perfect' meaning potentially a factor of three off. | 23:25 |
| wgrant | The estimation system is fine, assuming buildd-manager efficiency is 100%. | 23:25 |
| wgrant | Which it's about to get close to. | 23:25 |
| lifeless | wgrant: it requires many more assumptions than that | 23:28 |
| wgrant | That builds take the same amount of time each time, which is somewhat reasonable. | 23:28 |
| lifeless | wgrant: firstly that all buildds are equivalent (false) | 23:28 |
| wgrant | True. | 23:28 |
| wgrant | And that there are no queue jumpers (false) | 23:28 |
| lifeless | wgrant: secondly that package builds don't vary much (mmmm, maybe true, maybe false - see perf issues with lucid etc) | 23:29 |
| lifeless | that too | 23:29 |
| lifeless | and I'm sure there are other variables | 23:29 |
| lifeless | as for a factor of three, union the queues for all, add 1/3 of that to both i386 and amd64, and you should be tolerably bad | 23:30 |
| * thumper pushing conflict testfix | 23:31 | |
| wgrant | i386 queue > 1000 :( | 23:45 |
| lifeless | jkakar: how would you guys feel about __nonzero__ on ResultSet ? | 23:46 |
| thumper | wgrant: really? WTF? | 23:46 |
| wgrant | Soyuz works around the lack of __nonzero__ in a couple of cases already. | 23:46 |
| wgrant | thumper: That's right. | 23:46 |
| thumper | wgrant: where do I see that? | 23:47 |
| wgrant | thumper: https://launchpad.net/builders | 23:47 |
| wgrant | This means that no recipe will build for a few days at least. | 23:47 |
| thumper | wgrant: recipes now have the same score as PPAs :) | 23:49 |
| thumper | wgrant: so they no longer drop to the bottom | 23:49 |
| wgrant | thumper: Ah. | 23:49 |
Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!