| === persia` is now known as persia | ||
| === jamesh__ is now known as jamesh | ||
| * thumper EODs (for now), back tonight | 03:37 | |
| NCommander | Is there a good way to iterate thorugh SourceReleasePackage records? I need to iterate through ALL of them, download the source package, pop out the changelog, upload changelog to librarian, etc. | 03:59 |
|---|---|---|
| NCommander | (or at least, all published ones) | 03:59 |
| wgrant | You'll probably have to use query using Storm directly. | 04:04 |
| wgrant | There's no way to iterate over them all, AFAIK. | 04:04 |
| NCommander | wgrant: ugh .... *groan* | 04:12 |
| NCommander | wgrant: do I have to use Storm in writing migrationtools? | 04:13 |
| * NCommander is not even sure how to attack this problem in a sane manner | 04:13 | |
| StevenK | NCommander: Given the number of packages, is there a sane manner? | 04:13 |
| NCommander | StevenK: I'm honestly not sure, since we're also including PPA packages in this | 04:15 |
| StevenK | NCommander: Do the math -- 9,000 * Dapper, Hardy, Jaunty, Karmic and Lucid times about 40% is still 18,000 | 04:16 |
| NCommander | StevenK: ugh .... | 04:17 |
| NCommander | migration == PAIN | 04:17 |
| NCommander | StevenK: (necessary pain, but still pain) | 04:18 |
| persia | why 9000? | 04:18 |
| NCommander | StevenK: actually, we may want to include every source package so the changelog can be individually queried on a given SRP | 04:18 |
| NCommander | That requires discussion with BjornT, and bigjools | 04:18 |
| wgrant | We don't have the files for lots of SPRs. | 04:19 |
| NCommander | wgrant: we don't? | 04:19 |
| NCommander | I thought ever SPR had at least a few files | 04:19 |
| NCommander | for the source packages anyway | 04:19 |
| wgrant | All PPA sources that have been unpublished for more than a week, and all non-final primary archive versions from obsolete series. | 04:19 |
| NCommander | wgrant: oh, that greatly reduces the numbers I was planning | 04:20 |
| NCommander | wgrant: is there a special flag on SPR that states if there are files? | 04:21 |
| wgrant | NCommander: No. | 04:21 |
| NCommander | i.e., published, superseeded, obsolete, etc. | 04:21 |
| NCommander | ugh | 04:21 |
| * NCommander thought we had a removed from librarian status | 04:22 | |
| wgrant | You need to scan the LFCs of the LFAs of the SPRFs. | 04:22 |
| wgrant | Sorry, the LFCs of the LFAs of the SPRFs of the SPRs. | 04:22 |
| NCommander | wgrant: *blink* | 04:22 |
| NCommander | ow | 04:22 |
| ajmitch | needs more acronyms | 04:22 |
| * wgrant gives ajmitch a DASBPR | 04:22 | |
| mwhudson | i know let's switch launchpad to a nosql database that doesn't support joins! | 04:22 |
| ajmitch | wgrant: it's worrying that I know what that is | 04:23 |
| * NCommander whacks mwhudson :-P | 04:23 | |
| NCommander | wgrant: https://edge.launchpad.net/ubuntu/jaunty/+source/hello/2.2-2 - they seem ot be there | 04:23 |
| wgrant | mwhudson: My NMAF optimisation does a 9 table join. | 04:23 |
| wgrant | NCommander: Jaunty isn't obsolete. | 04:23 |
| NCommander | oh, d'oh | 04:23 |
| NCommander | fail | 04:23 |
| wgrant | It's possible that we only removed obsolete series' binaries. I reviewed the query, but really do not remember. | 04:24 |
| * wgrant checks. | 04:24 | |
| NCommander | wgrant: https://edge.launchpad.net/ubuntu/feisty/+source/hello/2.2-1build1 - seems thats the case | 04:24 |
| NCommander | yeah, the binaries were deleted | 04:24 |
| wgrant | Oh, right, we kept the old primary sources just in case. | 04:25 |
| NCommander | wgrant: that's a LOT of sources :-/ | 04:25 |
| wgrant | Although they were very very close to being accidentally nuked. | 04:25 |
| ajmitch | perhaps to satisfy some legalities? | 04:25 |
| wgrant | ajmitch: And to satisfy the bzr importer. | 04:25 |
| NCommander | ajmitch: we have to provide sources as long as we provide debs, so as long as old-releases exist ... | 04:25 |
| wgrant | NCommander: But old-releases still holds sources, doesn't it? | 04:25 |
| NCommander | wgrant: oooh, bzr importer does pretty close to what I'm going to need to do, where's the source | 04:26 |
| NCommander | wgrant: eh, I'm notsure | 04:26 |
| ajmitch | NCommander: I never knew if you were also required to provide sources for every package version published as well :) | 04:26 |
| wgrant | NCommander: It uses the API and stuff, though. You have DB access. | 04:26 |
| NCommander | ajmitch: legally speaking, we probably do as long as we distributed it on a release CD | 04:27 |
| NCommander | there's only a time limit to how long we provide sources if we use a written offer to get the source | 04:27 |
| NCommander | (at least on GPL'ed sources) | 04:27 |
| NCommander | Other licenses, YMMV | 04:27 |
| NCommander | wgrant: re: bzr importer: but it had to iterate on all SPRs, and if it uses the API, won't be cleaner? | 04:29 |
| wgrant | NCommander: But you need to get the SPR from the DB to set the changelog. | 06:42 |
| NCommander | wgrant: wait, what? | 06:43 |
| wgrant | NCommander: The bzr importer doesn't have DB access. | 06:44 |
| wgrant | It uses launchpadlib. | 06:44 |
| NCommander | wgrant: ugh | 06:44 |
| NCommander | crap | 06:44 |
| wgrant | You can't set the changelog with launchpadlib, so you need DB access. | 06:44 |
| NCommander | *sigh* | 06:44 |
| mrevell | Morning all | 09:07 |
| mwhudson | gmb, allenap: have you seen ParallelLimitedTaskConsumer (in the context of bugwatch updating)? | 09:33 |
| allenap | mwhudson: Yes, I looked at it briefly yesterday after I saw your message on IRC. That works with the job runner doesn't it? | 09:34 |
| mwhudson | allenap: no, it's not at that level | 09:34 |
| allenap | mwhudson: Okay, I'll look again. | 09:34 |
| mwhudson | (the branch puller uses it, and that doesn't even connect to the database) | 09:35 |
| NCommander | mwhudson: your up late :-/ | 09:35 |
| mwhudson | NCommander: a bit yeah | 09:35 |
| mwhudson | NCommander: why don't i look at that branch of yours | 09:35 |
| mwhudson | NCommander: it must be a considerably sillier time of night for you | 09:35 |
| NCommander | mwhudson: can't sleep unfortunately | 09:36 |
| mwhudson | NCommander: :( | 09:36 |
| NCommander | mwhudson: insommia == suck | 09:36 |
| allenap | mwhudson: Would this be a replacement for the ThreadPool stuff I've done in TwistedThreadScheduler? | 09:41 |
| mwhudson | allenap: maybe | 09:41 |
| * mwhudson has to go away sorry | 09:41 | |
| allenap | mwhudson: Thanks for the pointer, bye :) | 09:41 |
| mwhudson | allenap: talk to jml maybe | 09:41 |
| allenap | Okay. | 09:41 |
| === stub1 is now known as stub | ||
| deryck | Morning, all. | 10:06 |
| wgrant | bigjools: Did you inspect any of the data collected by the script yesterday? | 10:07 |
| NCommander | mwhudson: thanks for taking a look at the branch again | 10:08 |
| bigjools | wgrant: no | 10:17 |
| bigjools | well, I looked at a few rows in the table but didn't closely check | 10:18 |
| NCommander | When are data migrations done usually? (i.e., importing changelogs into librarian and updating the SPR record) | 10:34 |
| wgrant | NCommander: That's a special one -- it can be done at any time after the release, and will take a long while. | 10:39 |
| NCommander | wgrant: I wonder if we're looking at hours, days, or weeks :-/ | 10:40 |
| === stub1 is now known as stub | ||
| wgrant | NCommander: We're probably looking at around 200000 sources. | 10:43 |
| NCommander | bigjools, wgrant: so it sounds like to me, to migrate the changelog data, I'm going to need a script that uses Storm to access the DB, access each SRP where changelog = NULL, download the source package, pop out the debian/changelog, upload to librarian, update the SPR and then go on to the next record | 10:44 |
| NCommander | wgrant: that seems low ... | 10:44 |
| jml | allenap, what? | 10:45 |
| bigjools | NCommander: sounds about right to me | 10:45 |
| NCommander | bigjools: I assume at some point we may want to do this for copyright data as well? | 10:46 |
| wgrant | NCommander: There are only ~18000 primary sources per series, multiplied by a couple for the non-final versions, multiplied by say 10 for the number of releases, excluding most of those before Dapper (they weren't in LP). | 10:46 |
| bigjools | NCommander: you can make it faster by using store.execute() | 10:46 |
| wgrant | Then 30000 published PPA sources. | 10:46 |
| NCommander | (I think stub would be very happy to make the database loose several GiB of data) | 10:46 |
| bigjools | I don't care about PPA sources | 10:46 |
| NCommander | bigjools: hrm ... | 10:46 |
| NCommander | bigjools: we don't? I thought part of native sync sources was to let us go PPA -> main archive with a button click | 10:47 |
| allenap | jml: mwhudson mentioned ParallelLimitedTaskConsumer as (maybe) another/better way to solve the problem in checkwatches, of half-way Twistifying it. | 10:47 |
| bigjools | we also need to be very, very wary of the increase in librarian disk space required | 10:47 |
| wgrant | bigjools: The construction time for the few million Storm objects is going to be negligible compared to extraction time. | 10:47 |
| bigjools | I think the losas would quite like an estimate | 10:47 |
| jml | allenap, oh yes | 10:47 |
| stub | bigjools: Its easier to scale the Librarian than scale PG. | 10:47 |
| bigjools | NCommander: well I meant historic PPA sources | 10:47 |
| jml | allenap, do you want to run a bunch of tasks in parallel but with limitations until there's nothing left to do? | 10:47 |
| allenap | jml: We don't have time for it now, but yes, it does look interesting. It's working for now :) | 10:47 |
| NCommander | bigjools: oh, of course. | 10:47 |
| NCommander | bigjools: if we did the entire PPA history, I think the datacentre would probably implode | 10:47 |
| bigjools | wgrant: I am more concerned about exhausting memory | 10:48 |
| wgrant | bigjools: Historic PPA sources don't exist any more, so there's no question of that. | 10:48 |
| allenap | jml: Yes, basically. At the moment, because each task must always run in a thread, I'm just using a ThreadPool. | 10:48 |
| jml | allenap, :( | 10:48 |
| NCommander | stub: oh, so to answer your quesiton, yess, we will have NULLs for changelogs | 10:48 |
| bigjools | wgrant: quite true | 10:48 |
| allenap | jml: But when we get time to switch to non-blocking IO then something PLTC looks more interesting. | 10:48 |
| NCommander | bigjools: it sounds like where I'm sitting we should just attempt a changelog import wherever we have sources available, and make sure the code can handle changelog=NULL | 10:48 |
| allenap | jml: I've not looked at how it's used in LP, only at its implementation, so I'm not 100% sure how it's meant to be used. | 10:49 |
| wgrant | NCommander: I think that's all it's possible to do. | 10:49 |
| bigjools | NCommander: +1 | 10:49 |
| jml | allenap, the implementation is surprisingly hairy for a simple concept. | 10:49 |
| NCommander | bigjools: great, now I need to figure out how to write it :-/ | 10:49 |
| bigjools | NCommander: good luck :) | 10:49 |
| allenap | jml: PLTC or the one in checkwatches? If PLTC, I agree! ;) | 10:50 |
| jml | allenap, concurrency is hard. luckily you're using threads so you can pretend it isn't. | 10:50 |
| NCommander | bigjools: (I think raw_source_changelog will probably land this week in db-devel, I got a stub +1, just waiting on BjornT and the second code review) | 10:50 |
| wgrant | NCommander: Stealing code from archiveuploader will make your job very easy. | 10:50 |
| NCommander | wgrant: which code? | 10:50 |
| allenap | jml: Yeah, that's the approach we're taking for now. Ear defenders and blinkers at the ready. | 10:50 |
| * NCommander notes ripping packages apart isn't that hard | 10:50 | |
| jml | allenap, actually, PLTC is perhaps an ideal candidate for an informative doctest | 10:50 |
| wgrant | NCommander: The code that takes a source package, grabs files into a temp dir (some from the DB), extracts them, and inserts the changelog into the DB. | 10:51 |
| NCommander | wgrant: oh, the code we ripped apart last week :-) | 10:51 |
| wgrant | Most of it, yes. | 10:51 |
| NCommander | wgrant: that wasn't the part I was worried about, I was more worried about iterating through the databsae, but I think thats just due to not knowing much of storm. | 10:51 |
| allenap | jml: You're right. I'm so used to doctests being muddled that I didn't even think to look for one. | 10:51 |
| jml | allenap, it doesn't have one | 10:51 |
| * NCommander isn't even sure how we can test the importer script sanely | 10:52 | |
| bigjools | NCommander: juse use store.execute() and you can write raw SQL. This is better in this case otherwise it will exhaust memory when caching objects. | 10:52 |
| bigjools | this is a real problem in some scripts that iterate a lot of objects | 10:52 |
| bigjools | NCommander: we can test this initally locally on your laptop, then on dogfood | 10:53 |
| NCommander | bigjools: fair enough, but I'm going to need quite a few joins to get all the files. | 10:53 |
| jml | allenap, http://paste.ubuntu.com/399839/ is probably the best example | 10:53 |
| NCommander | bigjools: my laptop has a small SSD. Your going to blow it up | 10:53 |
| bigjools | NCommander: welcome to Soyuz SQL | 10:53 |
| bigjools | NCommander: I have an SSD too. Aren't they great? :) | 10:53 |
| NCommander | bigjools: is that SQL or SOL :-P | 10:53 |
| jml | allenap, the consumer just has an upper limit of parallel tasks and a logger. You tell it to consume from a "source" and it just runs whatever the source gives it until the source is exhausted | 10:53 |
| NCommander | bigjools: I dunno, I didn't want to burn my out within a week | 10:53 |
| NCommander | bigjools: this laptop takes 510 minutes to run the entire LP test suite (although it didn't swap doing it, which was nice) | 10:54 |
| allenap | jml: That's nice. | 10:54 |
| bigjools | that's what guarantees are for :) | 10:54 |
| bigjools | NCommander: 510?! jeez, mine takes about 180 | 10:54 |
| NCommander | bigjools: I dunno why it took so long. | 10:54 |
| bigjools | how much free memory and what cpu? | 10:54 |
| NCommander | bigjools: 8 GiB of RAM + 2.2Ghz dual core processor | 10:54 |
| NCommander | I'm thinking something must have hung for awhile | 10:55 |
| bigjools | yes | 10:55 |
| NCommander | I got more failures than ec2test | 10:55 |
| jml | allenap, the source just has to implement start(task_consumer) and stop(), and to call appropriate methods on the consumer when it has tasks. we've already implemented a polling source. | 10:55 |
| * NCommander will probably setup a karmic instance or UEC on my old laptop | 10:55 | |
| * bigjools needs to go back to working now | 10:55 | |
| NCommander | bigjools: sorry to distract you :-) | 10:55 |
| bigjools | that's my life | 10:55 |
| jml | allenap, the nice thing for us is that other than the source, the rest of the code would be identical even if we had some kind of event-driven infrastructure (e.g. message queues) | 10:56 |
| bigjools | I have a dead desktop today, I'm upset | 10:56 |
| jml | allenap, anyway, I'll let you get to it. | 10:56 |
| NCommander | bigjools: ugh, what happened to it? | 10:56 |
| * NCommander notes he's run LP onhis ia64 desktop before :-) | 10:56 | |
| bigjools | NCommander: it powers up but the monitor doesn't wake up, no bios messages etc. | 10:56 |
| allenap | jml: I'm can't immediately imagine how that'll fit into checkwatches, but it probably will. Thanks for explaining it. | 10:56 |
| jml | allenap, np. | 10:56 |
| NCommander | bigjools: sounds like your mainboard died | 10:56 |
| bigjools | NCommander: I am thinking gfx card actually | 10:57 |
| NCommander | bigjools: try the intergrated graphics? | 10:57 |
| bigjools | it was working last night until I tried to suspend it | 10:57 |
| bigjools | and hung on the way down | 10:57 |
| wgrant | bigjools: lp.services.buildfarm? | 11:15 |
| bigjools | wgrant: possibly | 11:15 |
| bigjools | increases finger strain though :) | 11:15 |
| bigjools | naming is the hardest part of software engineering | 11:16 |
| wgrant | Yes :( | 11:16 |
| jml | funding is the hardest part of software engineering | 11:18 |
| bigjools | fun is the hardest part of software engineering | 11:18 |
| NCommander | bigjools: not with python; just import fun | 11:18 |
| jml | (or maybe I meant concurrency, or dealing with massive applications) | 11:18 |
| bigjools | no, naming stuff is still the hardest :) | 11:19 |
| jml | bigjools, I'll respectfully disagree. | 11:20 |
| bigjools | jml: it's tongue-in-cheek | 11:21 |
| jml | bigjools, :) | 11:21 |
| wgrant | What became of the generalised builder history discussion? | 11:23 |
| bigjools | ongoing AFAIK | 11:24 |
| bigjools | it got a bit stuck on the fact that the translations jobs don't have a build record | 11:25 |
| wgrant | Right. | 11:25 |
| wgrant | And adding one is going to make everything even more horribly duplicated. | 11:26 |
| * wgrant likes http://people.ubuntu.com/~wgrant/launchpad/buildfarm/new-build-model.png as a solution to that. | 11:30 | |
| bigjools | wgrant: yes, we had something similar in mind | 11:43 |
| bigjools | noodles775 and I discussed this | 11:43 |
| wgrant | Ah, good. | 11:43 |
| bigjools | I want to ditch the use of Job | 11:44 |
| bigjools | it's been a royal PITA | 11:44 |
| wgrant | Yep. | 11:44 |
| bigjools | running intel gfx at 1920x1200 really shows up how awful the performance is :/ | 11:49 |
| wgrant | Lucid or otherwise? | 11:49 |
| bigjools | anything | 11:49 |
| bigjools | lucid right now | 11:50 |
| wgrant | Hmm. | 11:50 |
| NCommander | bigjools: I've nevcer had an issue with intel gfx running two screens at about that resolution | 11:50 |
| bigjools | it's fine at laptop resolutions, but on external monitors it's quite sluggish | 11:50 |
| wgrant | My old i915 drives a 1920x1080 TV fine. | 11:51 |
| bigjools | it could of course be kwin... | 11:55 |
| * wgrant is reminded of the oddities in Wellington. | 11:56 | |
| bigjools | but what about the laptop? | 11:56 |
| bigjools | <rimshot> | 11:56 |
| gmb | Does anyone have any idea why I'd be getting psycopg2 import errors on db-devel branches? AFAICT it's installed correctly but LP seems not to see it. Could it be connected with the buildbot failures we saw yesterday? | 12:03 |
| wgrant | jml: Hm, lp/__init__.py isn't exactly an obvious place for that documentation. | 12:06 |
| jml | wgrant, it is if you are browsing API docs, I think. | 12:06 |
| jml | wgrant, such as at http://people.canonical.com/~mwh/canonicalapi/index.html | 12:06 |
| wgrant | Hm, those are nice. | 12:07 |
| jml | wgrant, I reckon we could do a lot worse than emulate Bazaar, and have a doc/hacking/ directory | 12:08 |
| jml | or however they spell it -- it's easy to find when you need it. | 12:08 |
| === mrevell is now known as mrevell-lunch | ||
| jml | I really really need to end this zope testing branch and figure out what the hell to do with these images | 12:11 |
| bigjools | gnargh, testfix mode | 12:11 |
| wgrant | jml: Any idea why http://people.canonical.com/~mwh/canonicalapi/lp.soyuz.interfaces.build.IBuild.html has only IBuildFarmJob linked, and none of the others? | 12:12 |
| wgrant | bigjools: That db_lp failure from four hours ago with me in the blamelist? | 12:12 |
| jml | wgrant, backticks, I think. | 12:12 |
| * bigjools is checking | 12:12 | |
| jml | wgrant, alternatively, it's because the others are backticked but lack sufficient context for pydoctor to find them | 12:13 |
| wgrant | jml: getFileByName' ILibraryFileAlias is backticked. | 12:13 |
| bigjools | wgrant: test_sigusr2 failure? | 12:13 |
| wgrant | And IBuildFarmJob isn't imported in the other file. | 12:13 |
| wgrant | bigjools: No idea. I can't see buildbot failures. | 12:13 |
| bigjools | oh | 12:13 |
| bigjools | looks spurious | 12:14 |
| jml | wgrant, in that case, I don't know. I *do* know that it's a bit of a black art, and that mwhudson is the person to ask. | 12:14 |
| jml | heck, I don't even know if we have a convenience thing in tree to build those docs | 12:15 |
| * wgrant didn't find one. | 12:16 | |
| bigjools | jml: +1 for separate branch but it will be hard until we can split up LP better than it is | 12:18 |
| jml | bigjools, it will be hard. | 12:19 |
| wgrant | That sounds really really hard. | 12:19 |
| bigjools | but totally necessary for a sane development experience | 12:19 |
| jml | bigjools, it will always be hard. so will collaborating on a 350k line tightly-coupled Python app. | 12:20 |
| bigjools | there are some strong arguments to make it more loose | 12:21 |
| bigjools | it also begs the question of why we made lib/lp/xxx in the first place | 12:21 |
| jml | bigjools, to make the structure clearer. | 12:21 |
| noodles775 | step-by-step | 12:22 |
| bigjools | what structure? | 12:22 |
| jml | bigjools, it then turned out that we had less structure than we'd hoped | 12:22 |
| bigjools | exactly :) | 12:22 |
| jml | bigjools, but we had even less of an idea of that or the scope of that when everything was in one big directory | 12:22 |
| jml | bigjools, also, I think that it's almost always a good idea to change your code to match the way you think about the problem space | 12:23 |
| bigjools | no arguments from me there | 12:24 |
| jml | bigjools, in our case, we were talking always about "code" this and "bugs" that and "registry" whatever. the lp apocalypse has actually made our code clearer imo | 12:24 |
| wgrant | But the API constraints make a complete split difficult. | 12:25 |
| wgrant | And the Soyuz<->Registry split is insane. | 12:25 |
| bigjools | I agree, but it's also pointed out the nightmare of tight coupling | 12:25 |
| wgrant | (although I guess most of the Soyuz-related methods in Registry have belonged on IArchive since archive-rework three years ago) | 12:26 |
| bigjools | wgrant: it's a good idea to split it, the problems come because of other reasons | 12:26 |
| bigjools | yes | 12:26 |
| jml | wgrant, API constraints are the biggest technical blocker I can see. | 12:27 |
| jml | wgrant, bigjools: does the buildfarm code actually depend on stuff outside of the buildfarm? | 12:27 |
| wgrant | jml: Only a little. | 12:27 |
| wgrant | I've removed most of them. | 12:27 |
| jml | so it wouldn't be _that_ hard to move to a separate branch | 12:27 |
| wgrant | eg. BuildBase stuff depends on Archive. | 12:28 |
| wgrant | But BuildBase could reasonably move to Soyuz. | 12:28 |
| wgrant | And the builder UI code is still in Soyuz. | 12:28 |
| bigjools | well | 12:28 |
| jml | I mean, apart from the base level of hard involved in doing anything that changes our build & deployment processes. | 12:28 |
| bigjools | there's probably room for another module between the buildfarm and soyuz | 12:28 |
| wgrant | jml: But splitting something out when it depends on the DB? | 12:28 |
| bigjools | whatever "soyuz" is nowadays anyway | 12:29 |
| jml | wgrant, I don't see why that's an issue | 12:29 |
| wgrant | jml: Oh, also, buildmaster is currently tested with Soyuz objects, since it has no concrete job implementations of its own. | 12:29 |
| wgrant | What is the purpose of splitting it into a separate branch? It can't be run independently. | 12:32 |
| bigjools | ideally it should be | 12:33 |
| jml | wgrant, can't it? perhaps I misunderstand what it is. | 12:33 |
| jml | I'd put the bloody buildd slave code in a separate branch first, tbh. | 12:33 |
| wgrant | Ideally it should be. | 12:33 |
| wgrant | But it probably depends on DB details and blah blah blah. | 12:33 |
| wgrant | Although I guess it really needn't. | 12:33 |
| bigjools | the only thing I know about is tachandler.py | 12:34 |
| jml | wgrant, ideally it wouldn't, but that's not _actually_ a blocker | 12:34 |
| bigjools | otherwise it really should be ripped out | 12:34 |
| wgrant | Why is tachandler not somewhere like Twisted? | 12:34 |
| bigjools | nfi | 12:34 |
| jml | wgrant, a few reasons | 12:34 |
| jml | wgrant, in fact, Twisted might have grown better APIs since we wrote tachandler | 12:35 |
| jml | wgrant, it's not code that a lot of Twisted apps feel the need for. | 12:35 |
| wgrant | jml: Well, a few unrelated bits of LP do. What's so special about them? | 12:36 |
| jml | wgrant, tbh, I don't know why we use it. | 12:36 |
| jml | wgrant, other than for integration tests. | 12:36 |
| * jml greps | 12:37 | |
| jml | ReadyService, afaict could be in Twisted and is only really there to make testing easier (icbw of course) | 12:38 |
| jml | and TacTestSetup is essentially an API for manipulating out-of-process Twisted applications | 12:40 |
| jml | wgrant, I guess someone could spend a few hours and end up by getting rid of tachandler | 12:44 |
| jml | maybe I'll take on splitting out the buildd code as my hacking task after I land the zope.testing upgrade and my poppy work. | 12:45 |
| jml | wgrant, there you go, I asked on #twisted. | 12:48 |
| jml | that is progress. | 12:48 |
| wgrant | I should probably get my buildmaster cleanup branches into potentially mergable condition. | 12:51 |
| jml | I should probably do some strategy today :) | 12:54 |
| NCommander | wgrant: jml: any chance you can make the ABORT button work?:-) | 12:57 |
| bigjools | so, removing my gfx card and booting works fine - I just don't get a desktop :) | 12:57 |
| wgrant | bigjools: Ah, easy fix, good. | 12:57 |
| NCommander | bigjools: hope that card is under warrenty? | 12:57 |
| bigjools | well easy, but expensive :() | 12:58 |
| bigjools | no, it's about 18 months old | 12:58 |
| wgrant | NCommander: The slave changes aren't hard. The main difficulty is working out how to transfer the request through the DB. | 12:58 |
| wgrant | Do we have new Aborting and Aborted DB statuses? | 12:58 |
| bigjools | we also need a CANCELLED | 12:59 |
| bigjools | we're abusing SUPERSEDED right now | 12:59 |
| wgrant | Right. | 12:59 |
| bigjools | lamont will love whoever fixes that | 12:59 |
| wgrant | This should probably be thought about in the schema redesign, if that ends up happening. | 13:00 |
| wgrant | Since it applies to all job types. | 13:00 |
| bigjools | yes | 13:00 |
| bigjools | but as much as I hate it, the job status is a separate status to the build status | 13:01 |
| wgrant | +ALTER TABLE ONLY build ADD CONSTRAINT build__archive__fk FOREIGN KEY (archive) REFERENCES archive(id) on delete cascade; | 13:02 |
| * wgrant cries. | 13:02 | |
| bigjools | ha, if I had a hdmi cable my desktop would be working, it has an onboard nforce 750a | 13:02 |
| wgrant | bigjools: It has onboard HDMI but not VGA? | 13:02 |
| bigjools | yep :/ | 13:02 |
| wgrant | WTF | 13:03 |
| bigjools | yep :/ | 13:03 |
| jml | NCommander, abort button? what do I look like? a programmer?! | 13:03 |
| bigjools | wgrant: gar, that one is wrong isn't it. Crapola. | 13:03 |
| wgrant | That is a seriously fucking dangerous DB patch. | 13:03 |
| bigjools | oO | 13:04 |
| wgrant | Oh, no, it does leave a constraint or two that will stop it cascading too far into other archives. Good. | 13:05 |
| wgrant | (If BPPH.binarypackagerelease had been altered to cascade... oh dear) | 13:06 |
| bigjools | wgrant: so, I could make the Build.archive on delete NULL, but that's gonna break lots of shit | 13:10 |
| * bigjools thinks | 13:10 | |
| wgrant | That's what I said last night :) | 13:10 |
| bigjools | so there's a few options | 13:11 |
| wgrant | There is no precedent for doing this sort of thing. | 13:11 |
| bigjools | 1. don't delete the archive | 13:11 |
| bigjools | 2. delete the binaries | 13:11 |
| bigjools | 3, ?? | 13:11 |
| bigjools | 4. profit | 13:12 |
| * bigjools is fed up with people insisting that ppa deletion is easy | 13:12 | |
| wgrant | Deleting the archive means deleting all sorts of stuff. Future copy logging will be difficult, PackageUploads (and thus all upload accountability information and changes files) will have to be removed... | 13:12 |
| wgrant | It's not like a branch, which is one object, pontentially with a non-critical tendril attached to another branch. | 13:13 |
| bigjools | yes | 13:14 |
| bigjools | however I feel very strongly that we should delete them in their entirety if at all possible | 13:14 |
| jml | wgrant, stacking is a critical tendril | 13:15 |
| wgrant | I am in two minds about it. It's difficult and wrong, but also much cleaner. | 13:15 |
| wgrant | jml: But you deny in that case, don't you? | 13:15 |
| jml | wgrant, maybe. | 13:15 |
| bigjools | we could delay the full deletion until the last PPA that uses another's packages disappears | 13:15 |
| * jml really likes the infrastructure abentley set up for deleting branches | 13:15 | |
| bigjools | I'm starving and I really like the fact that my fridge is full of food, bbiab | 13:16 |
| jml | heh heh | 13:17 |
| === mrevell-lunch is now known as mrevell | ||
| stub | You can delete the things its safe to manually, and leave the constraints there to raise exceptions when necessary. | 13:48 |
| kfogel | intellectronica: you saw https://code.edge.launchpad.net/~kfogel/launchpad/78565-bug-comment-link-to-bug/+merge/21896 ? | 14:00 |
| dobey | hi all. i'm having some trouble filtering bugs for rhythmbox-ubuntuone-music-store (Ubuntu). they seem to be getting past my procmailrc filter (though all other Ubuntu bugs are being filtered just fine) | 14:04 |
| maxb | Hi dobey. Perhaps you mean to be in #launchpad ? | 14:05 |
| dobey | but for rhythmbox-ubuntuone-music-store bugs, there seems to be a newline inside the header... perhaps causing procmail to not interpret the header properly? | 14:05 |
| dobey | maxb: no, i don't think so. this seems to be a bug in the launchpad bug mail creator | 14:07 |
| maxb | Please paste the mail headers to a pastebin. | 14:07 |
| bigjools | wgrant: still around | 14:10 |
| bigjools | ? | 14:10 |
| dobey | maxb: http://pastebin.ubuntu.com/399993/ <- this is from one mail that's failing | 14:14 |
| dobey | maxb: and http://pastebin.ubuntu.com/399994/ is from a mail that is successfully filtered | 14:15 |
| maxb | dobey: I believe that's header folding, as specified by rfc2822. | 14:16 |
| maxb | I.e. it's your filter's fault | 14:16 |
| maxb | sorry | 14:16 |
| dobey | * ^X-Launchpad-Bug: distribution=ubuntu; sourcepackage=\/[^;]+ | 14:17 |
| dobey | that's the filter... and it works for the 59 other Ubuntu packages that i've gotten bug mails for | 14:18 |
| dobey | i'm just trying to understand what the difference is | 14:18 |
| intellectronica | kfogel: i did, and for some reason i thought i approved it. it was early in the morning so i was probably confused. r=me and will tick the mp now | 14:18 |
| dobey | and others seem to have 'folded headers' too | 14:18 |
| thekorn | dobey: the "\n " between distribution=... and sourcepackage= | 14:19 |
| dobey | oh right | 14:20 |
| dobey | blah | 14:20 |
| maxb | dobey: A fully compliant filtering solution would apply rfc822 unfolding before applying regexps to header values. | 14:22 |
| maxb | OOI, do db patches ever get renumbered, or is the idea that the DBA only assigns them a real version number when they will be nearly guaranteed to land in the upcoming dev cycle? | 14:34 |
| BjornT | maxb: the db patch numbers aren't version numbers. they are more like identifiers. it doesn't matter of a db patch gets a number and never lands. | 14:41 |
| maxb | But, they do control the relative ordering of patch application | 14:41 |
| maxb | (right?) | 14:41 |
| BjornT | maxb: yes. so in theory it could cause problems not landing them in order, but not really in practice. i can't ever recall that causing a problem. | 14:48 |
| bac | hi leonardr | 15:15 |
| leonardr | hi bac | 15:16 |
| leonardr | i'm on the phone but type away | 15:16 |
| bac | leonardr: when using utilities/ec2 i'm getting the "File name too long" error from httplib2. you did a fix for that in lazr.restfulclient. should it not be in 0.9.12? | 15:17 |
| leonardr | bac, i'd think so | 15:18 |
| bac | leonardr: your fix was on 2010-02-09 so it should've been in 0.9.11 and definitely 0.9.12 | 15:20 |
| leonardr | bac, give me any information about the failure you can | 15:20 |
| bac | leonardr: http://paste.ubuntu.com/400029/ | 15:21 |
| james_w | leonardr: is there a way to test the wadl that is generated for a particular entry from the LP test suite? | 15:28 |
| james_w | leonardr: I have to make a change, but the current tests pass and they use the webservice calls in the story tests, and show that the representation is correct. The bug is that the wadl is incorrect. | 15:29 |
| leonardr | james_w: unfortunately you are #4 in line | 15:33 |
| mars | sinzui, quick question: if I have a bug about a page's HTTP cache headers, should that bug go against launchpad-web or launchpad-foundations? | 15:44 |
| sinzui | foudations | 15:44 |
| === salgado is now known as salgado-lunch | ||
| mars | sinzui, so launchpad-web is for UI only? Not browser mechanics? | 15:45 |
| sinzui | mars: web is for css/markup issues that are common to all applications. web is thus about the site design | 15:45 |
| mars | right | 15:45 |
| sinzui | mars: no mechanics. these bugs should be fixable by designers | 15:45 |
| mars | hmm, good category | 15:46 |
| mars | I was going through the list of bugs tagged CSS, and a number fell into that category. | 15:46 |
| mars | There were three categories IIRC: inconsistent application of the LP style, stuff where the current UI design can't cope with the data, and browser-specific display issues | 15:47 |
| sinzui | yes, I came to that understanding when I triaged the bugs gary_poster was uncertain of. | 15:47 |
| sinzui | I note that users started putting application specific bugs in launchpad-web. I moved them to the specific app | 15:48 |
| === matsubara is now known as matsubara-lunch | ||
| mars | hmm. orthogonal concerns: responsibility, and category | 15:49 |
| bac | leonardr: i see in your safename replacement you limit to 117 (152-32-1) so it is less than 150. but that is just limiting the key portion. httplib2 then adds the cachedir name to it, which in my case is 52 characters | 15:50 |
| bac | leonardr: cacheFullPath = os.path.join(self.cache, self.safe(key)) | 15:50 |
| leonardr | bac: i see. you have a long cachedir name | 16:05 |
| bac | leonardr: not really. it is the default. yours would be longer. (leonardr > bac) | 16:06 |
| bac | leonardr: i'll open a bug | 16:06 |
| leonardr | is it the path name that is limited or the filename? | 16:06 |
| bac | currently on the key is limited | 16:07 |
| leonardr | i meant in the underlying ecryptfs filesystem | 16:07 |
| === deryck is now known as deryck[lunch] | ||
| leonardr | james_w: i'm not sure what you mean by the wadl generated for a particular entry. do you mean the wadl representation of /~leonardr? i suspect you mean something else but i'm not sure what, nor which wadl is incorrect | 16:10 |
| james_w | leonardr: the wadl declaration for ICodeImport is incorrect | 16:14 |
| james_w | it asserts there will be a "branch" parameter, when there is in fact a "branch_link" parameter | 16:15 |
| james_w | I know the code fix, but I don't know how to test it from that angle | 16:15 |
| james_w | I could do a test that ICodeImport.branch is a ReferenceChoice and not a Choice, but that doesn't seem ideal | 16:15 |
| james_w | the issue is that lazr.restful changes its behaviour between the declarations and the marshalling when the attribute is an entry, but not declared with the extra information of a ReferenceChoice | 16:17 |
| * james_w heads for some lunch, any suggestions welcome | 16:19 | |
| leonardr | james_w: ok, i need to think about this | 16:19 |
| bdmurray | How can I know when my branch is rolled out on edge? | 16:37 |
| bigjools | bdmurray: if you know the revision number that it landed with, you can compare against the rNNNNN at the bottom of each edge page | 16:38 |
| bac | leonardr: https://bugs.edge.launchpad.net/lazr.restfulclient/+bug/545197 | 16:38 |
| mup | Bug #545197: Cache file paths are too long for encryptfs <lazr.restfulclient:New> <https://launchpad.net/bugs/545197> | 16:38 |
| bdmurray | bigjools: I was hoping for a notification not something I'd have to check ;-) | 16:38 |
| bigjools | bdmurray: ah :) edge is normally gets rolled out each day | 16:39 |
| bigjools | and please correct my English | 16:39 |
| bigjools | bdmurray: if there was a bug linked to the branch, you'll see its status change when it sees the branch on edge or staging | 16:40 |
| bdmurray | bigjools: okay, awesome that'll work | 16:41 |
| leonardr | bigjools: the problem is a misconfiguration on dogfood | 16:41 |
| leonardr | the service root for the 'api' virtual host needs to have its /beta/ removed | 16:41 |
| bigjools | leonardr: ah ok, where do I edit that? | 16:42 |
| leonardr | bigjools: launchpad-lazr.conf | 16:43 |
| leonardr | [vhost.api].rooturl | 16:43 |
| leonardr | you _also_ need to remove /beta/ from your dogfood root url, or launchpadlib's first request will fail | 16:44 |
| leonardr | if you remove /beta/ from lputils.py but don't change dogfoot, the first request succeeds and the second one fails | 16:44 |
| leonardr | if you fix all this and dogfood credentials still aren't written to disk, let me know | 16:46 |
| * bigjools is trying now | 16:48 | |
| === beuno is now known as beuno-lunch | ||
| leonardr | james_w: ok, i understand what you're asking for | 16:53 |
| === matsubara-lunch is now known as matsubara | ||
| leonardr | if you look at lib/canonical/launchpad/pagetests/webservice/xx-wadl.txt you'll see a test that retrieves the wadl description of launchpad and does minimal tests to it | 16:58 |
| === salgado-lunch is now known as salgado | ||
| leonardr | if you look at lazr.restful/src/lazr/restful/example/base/tests/wadl.txt you'll see a more detailed test that dissects a wadl file | 16:58 |
| leonardr | you could use an xpath selector to grab the part of the wadl file you want to examine | 16:59 |
| leonardr | but you might like this idea better | 17:00 |
| leonardr | the launchpad wadl description is transformed into an html document | 17:00 |
| leonardr | lib/canonical/launchpad/pagetests/webservice/apidoc.txt grabs that html document and prints it out | 17:01 |
| leonardr | you could do something similar to verify that branch_link shows up under code_import | 17:01 |
| leonardr | so you have several options | 17:01 |
| bigjools | leonardr: ok, success on dogfood! Thanks very much. | 17:02 |
| leonardr | yay | 17:02 |
| === deryck[lunch] is now known as deryck | ||
| === beuno-lunch is now known as beuno | ||
| === gary_poster is now known as gary-lunch | ||
| === salgado_ is now known as salgado | ||
| james_w | leonardr: what about ObjectLookupFieldMarshaller checking for field.schema? Would that be too intrusive? | 18:01 |
| james_w | leonardr: the idea being that it would refuse to unmarshall something that wasn't declared as a ReferenceChoice, and so not cause the skew between the wadl and the representation? | 18:02 |
| mrevell | night | 18:06 |
| leonardr | james_w: that might be a good way to stop errors from begin committed, since the normal wadl test would fail | 18:08 |
| leonardr | try it and see if it interferes with existing tests | 18:08 |
| james_w | ok, but I'm not keen on having to deal with eggs and buildout :-) | 18:09 |
| leonardr | james_w: if you just want to try it, change the code in your existing lazr.restful egg | 18:14 |
| leonardr | ~/.buildout/eggs/lazr.restful.*/... | 18:14 |
| leonardr | it'lll take effect immediately | 18:14 |
| james_w | ah, cool, thanks | 18:16 |
| bac | Ursinha: i approved your MP | 18:31 |
| bac | thanks | 18:32 |
| Ursinha | bac: oh, thanks :) | 18:33 |
| cody-somerville | wgrant, ping | 18:49 |
| === gary-lunch is now known as gary_poster | ||
| mwhudson | good morning | 19:01 |
| === EdwinGrubbs_ is now known as EdwinGrubbs | ||
| jml | mwhudson, hi | 19:17 |
| jml | mwhudson, I need an answer for you re https://code.edge.launchpad.net/~jml/launchpad/update-ec2-image/+merge/21804 | 19:17 |
| * mwhudson hadn't noticed it was back in question land | 19:19 | |
| mwhudson | oh, "do we need to support karmic" | 19:19 |
| mwhudson | i guess not | 19:19 |
| mwhudson | jml: replied | 19:19 |
| jml | mwhudson, thanks. :) | 19:20 |
| bac | leonardr: so the 144 limit you discussed with dustin is just for the name portion, not including the path? | 19:23 |
| mwhudson | jml: "someone" should write a twisted success story for launchpad | 19:31 |
| leonardr | bac: right, the path limit is more like 4096 characters | 19:44 |
| bac | leonardr: ok. sorry then for the red herring in the bug report | 19:44 |
| leonardr | np, it worked out | 19:44 |
| === EdwinGrubbs_ is now known as EdwinGrubbs | ||
| didrocks | gmb: flacoste: hey o/ will it be possible to finish the second merge review on exposing ssh key this week (https://code.edge.launchpad.net/~didrocks/launchpad/expose-sshkeys-bug-357235/+merge/20995)? it prevents me from uploading last Quickly into ubuntu | 20:01 |
| james_w | is anyone else seeing a failure in lib/canonical/launchpad/ftests/../doc/webservice-marshallers.txt ? | 20:24 |
| james_w | I'm getting a TypeError on devel | 20:26 |
| === salgado is now known as salgado-brb | ||
| james_w | make clean && make hasn't fixed it, and neither has anything else I have tried | 21:48 |
| james_w | would someone try running "./bin/test -cvvt lib/canonical/launchpad/ftests/../doc/webservice-marshallers.txt" in a (fairly) up to date copy of devel? | 21:49 |
| gary_poster | sinzui: would you mind helping me categorize the following as registry, foundations, or something else? https://bugs.edge.launchpad.net/launchpad-foundations/+bug/231797 It doesn't feel terribly foundations-y to me, but I'll take it if you think it's appropriate. | 22:03 |
| mup | Bug #231797: no sensible way to use debian/watch files with launchpad hosted tarballs <Launchpad Foundations:New> <devscripts (Ubuntu):Invalid> <https://launchpad.net/bugs/231797> | 22:03 |
| === matsubara is now known as matsubara-afk | ||
| wgrant | cody-somerville: Hi. | 22:11 |
| wgrant | james_w: What's the failure? | 22:12 |
| james_w | TypeError: a float is required | 22:14 |
| wgrant | More traceback? | 22:14 |
| james_w | canonical.launchpad.webapp.adapter.set_request_started() isn't being called, or some variation on that | 22:15 |
| wgrant | Hm. You've updated sourcecode and download-cache and everything lately? | 22:15 |
| james_w | and I can't see what is supposed to call LaunchpadBrowserPublication.beforeTraversal | 22:16 |
| james_w | hmm, I haven't done sourcecode | 22:16 |
| wgrant | It'll be somewhere in Zope, probably. | 22:16 |
| james_w | yeah, that's my problem :-) | 22:17 |
| mwhudson | james_w: i get the same | 22:22 |
| james_w | hooray! | 22:22 |
| james_w | well, not really, but I'm glad I'm not alone in this ;-) | 22:23 |
| mwhudson | http://pastebin.ubuntu.com/400246/ | 22:23 |
| james_w | yep | 22:25 |
| * mwhudson tries db-devel too | 22:25 | |
| james_w | updated sourcecode and download-cache, did make clean && make and still see it | 22:28 |
| james_w | also, why isn't buildbot complaining? | 22:29 |
| mwhudson | lucid vs hardy maybe? | 22:30 |
| james_w | that's possible | 22:32 |
| james_w | I haven't upgraded any packages since this test was working, so I think there is a code change associated too | 22:33 |
| mwhudson | fails in db-devel too | 22:34 |
| mwhudson | james_w: i guess email the list and see what is said there | 22:51 |
| === salgado is now known as salgado-afk | ||
| mwhudson | james_w: it fails for me too in production-devel which suggests some environmental change to me | 23:04 |
| mwhudson | and test failures from ec2 | 23:05 |
| james_w | right | 23:05 |
| * mwhudson grumps off to lunch | 23:05 | |
| * maxb runs 'lpnochange python-apt', and is very happy to have scripted these rebuilds | 23:20 | |
| wgrant | Wasn't staging meant to be moved to Lucid reasonably soon? | 23:27 |
| mwhudson | "reasonably soon after release" i think? | 23:30 |
| wgrant | I thought it was meant to be before release. | 23:31 |
| wgrant | But I heard that a few months ago, so things might have changed. | 23:31 |
| maxb | It would seem a bit odd to run a beta distro in the datacentre? | 23:32 |
| lifeless | maxb: why ? | 23:44 |
| maxb | What's the rush? | 23:44 |
| wgrant | Test. | 23:44 |
| wgrant | Make it less disastrous than Hardy. | 23:44 |
| wgrant | That sort of thing. | 23:44 |
| lifeless | can't make changes to lucid after it releases | 23:44 |
| lifeless | can't tell if its suitable for the dc if we don't deploy it | 23:44 |
| wgrant | s/can't/shouldn't/ | 23:44 |
| wgrant | Much better to get the applications running on it semi-production now. | 23:45 |
| lifeless | s/cant'/won't/ then | 23:45 |
| wgrant | lifeless: Hardy proved otherwise. | 23:45 |
| maxb | ?! | 23:46 |
| wgrant | Hardy had far too many SRUs right after release. That showed that there wasn't enough testing, as usual. | 23:47 |
| wgrant | The more testing Lucid gets in real environments before release, the better. | 23:47 |
Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!
