[00:28] is it normal that i have to run 'bzr update' in download-cache as well as update-sourcecode? [00:31] poolie: yes. [00:37] rocketfuel-get does both. [00:39] but it's deprecated, according to some people? [00:39] NAFAIK [00:39] spm: when lpnet is deployed, do you run 'make' in an existing tree, or a clean tree? [00:40] poolie: I've never used it, since I felt it was a bit crappy to be working on bzr support for Launchpad and not actually using bzr to get my code [00:40] poolie: completely clean, the process is something like: [00:41] copy new tree as launchpad-revno-; make build in that; shutdown service; ln -s switchero (launchpad -> newshiny revno); startup [00:41] mwhudson: hi [00:41] mwhudson: re apache mangling; is there a bug for codebrowse? [00:41] lifeless: hi [00:41] poolie: I grossly simplfy, but that's the gist. [00:41] ah no, i don't think so [00:41] let me search a bit [00:41] mwhudson: we'll need one marked as a blocker for becoming an open id consumer [00:42] mwhudson: probably needs a task on foundations [00:42] not because they should do it but because they are doing the openid stuff and need to see it [00:42] spm that's what i thought [00:43] i was just trying to work out bug 680339, but [00:43] <_mup_> Bug #680339: 'Entry' object has no attribute 'markAsDuplicate' [00:43] i suspect it's an interaction thing and not a real bug [00:45] poolie: markAsDuplicate is a mutator. [00:45] It's not exposed as a method; you just set the attribute. [00:48] I thought that changed, depending on the API version used? [00:49] Probably. [00:49] * jml gone again, back slightly later than UK starting time tomorrow morning [00:50] But I really hope not. [00:50] But probably. [00:51] wgrant: it was exposed as a method [00:51] it is now exposed as .duplicate = ... [00:51] wgrant: Would you believe I still can't nail down where the spph is created for an nascent upload [00:51] when i first wrote my client code, the mutator didn't exist [00:52] StevenK: PackageUploadSource.blahblah [00:52] now the method seems to have disappeared from the 1.0 api [00:52] * StevenK declares tea will magically point it out to him [00:52] poolie: Well, did you know that Launchpad likes to make API stability guarantees without actually thinking about whether it can support them? [00:52] i was kind of suspecting that [00:53] the "api stability" thread prompted me to think about it again [00:53] but that turned out to be a somewhat different question [00:55] AIUI the discussion basically went like this: "We need to have a stable API for Ubuntu releases to use." "OK, you can now annotate methods as being for a particular version." "OK, you must now support beta and 1.0 for 5 years, with no warning." [00:55] Now we cannot change our datamodel. [00:56] so [00:56] thats an exaggeration, and you know it :) [00:57] i'm curious where those annotations go [00:57] in the export() call [00:57] see salgados blueprint branch, for instance. [00:57] that exports only on devel (at my request) [00:59] API stability guarantees are good. Simply freezing an entire API version is not. [01:00] agreed [01:00] EBeforeMyTime [01:00] The bits of the 1.0 API that need to be stable for 5 years are probably limited to a couple of methods. [01:00] perhaps [01:03] StevenK: Found it yet? [01:03] wgrant: Nope [01:04] StevenK: It should be in queue.py somewhere. [01:04] * wgrant looks. [01:04] PackageUploadSource.publish [01:04] newSourcePublication [01:05] Ah ha. There we get the SPR, so digging back for the history should be easy [01:07] HA HA HA [01:20] gah [01:20] I have a test that regularly fails in ec2 but passes locally [01:20] s/regularly/always [01:20] has to do with recipe breadcrumbs of all things [01:27] lifeless: you're saying the export decorator controls which versions it appears in [01:27] that seems plausible but markAsDuplicate calls it with no parameters [01:28] lib/canonical/launchpad/rest/configuration.py: last_version_with_mutator_named_operations = "beta" [01:28] So in beta, mutators appear as named operations. [01:28] In 1.0 and devel they do not. === vednis is now known as mars [02:12] wallyworld_: ping [02:12] thumper: hello [02:12] wallyworld_: we should have a chat [02:12] although I might go make a coffee first [02:12] thumper: ok [02:12] sound good? [02:12] thumper: ack [02:24] wallyworld_: bzr+ssh://bazaar.launchpad.net/~thumper/launchpad/recipe-binary-builds r11904 [02:24] TestSourcePackageRecipeView [02:34] ICanHasLinkedBranch(product).branch [02:35] ICanHasLinkedBranch(sourcepackage).branch [02:36] ICanHasLinkedBranch(productseries).branch [02:39] === Top 10 Time Out Counts by Page ID === [02:39] Hard / Soft Page ID [02:39] 260 / 6568 Archive:+index [02:39] 142 / 301 BugTask:+index [02:39] 26 / 290 Distribution:+bugs [02:39] 26 / 126 ProjectGroupSet:CollectionResource:#project_groups [02:39] 10 / 4 Cve:+index [02:39] 9 / 29 Milestone:+index [02:39] 9 / 15 DistroSeries:+queue [02:39] 8 / 107 Archive:+packages [02:39] 8 / 12 DistroSeriesLanguage:+index [02:39] 6 / 3 Person:+bugs [02:41] Difference: !=: [02:41] reference = 'Request builds for cake_recipe\nMaster Chef\nRecipes\ncake_recipe\nRequest builds for cake_recipe\nArchive:\nSecret PPA (chef/ppa)\nDistribution series:\nSecret Squirrel\nHoary\nWarty\nor\nCancel' [02:41] actual = u'Request builds for cake_recipe\nMaster Chef\nRecipes\ncake_recipe\nArchive:\nSecret PPA (chef/ppa)\nDistribution series:\nSecret Squirrel\nHoary\nWarty\nor\nCancel' [02:41] lifeless: aren't you meant to be on leave? [02:41] wallyworld_: I am [02:41] wallyworld_: He's not very good at it. [02:41] lifeless: if you want something to do, fix my bug [02:42] lifeless: it passes for me and wallyworld_ but fails in ec2 [02:42] thumper: I'm not doing work [02:42] lifeless: well fcuk off then and enjoy your break :-) [02:42] lifeless: no, this would be fun :) [02:42] wallyworld_: I am, I've scrastched a bunch of itches in lp:testrepository [02:42] and watched a movie and a half, plus judge john deed [02:42] cleaned up a bit in my office [02:43] and got wow reinstalled. [02:43] lifeless: i still think you need some "downtime" :-) [02:45] lifeless: O.O at the last one [02:45] StevenK: you managed to break free from that? [02:45] ajmitch_: Er, not quite [02:45] StevenK: google for wine + could not fix permissions, or thereabouts - launcher.exe fail. [02:45] let me guess, waiting impatiently for the 7th? [02:45] ajmitch_: Yes :-) [02:46] wallyworld_: thats why I have a long block of leave. It takes me weeks to spin down. [02:46] * wgrant never fell into the WoW trap. [02:46] we can help [02:46] lifeless: Is that what you're currently tripping on? [02:46] Haha [02:46] StevenK: was [02:46] wgrant: be glad :) [02:46] StevenK: thus the reinstall. [02:46] lifeless: Has it downloaded the 10GiB of patches it needs? [02:46] the incremental downloader is damn awesome [02:46] yeah [02:47] its all good, I logged in and checked its working. Still haven't decided if 'm going to get the upgrade or not. [02:47] StevenK: Ah, so crazy Blizzard patches aren't just limited to SC2? When I reinstalled last week it must have reached 100% eight or nine times... [02:47] wgrant: Oh no, WoW is where they perfected that [02:48] wgrant: on wine or windows? [02:48] lifeless: Both. [02:51] ajmitch_: Does that mean you pulled yourself out of it? [02:51] StevenK: well I stopped raiding at least, that has to count for something? [02:51] Haha [03:02] Neat. parallel-test #16 has been building for 2 days 18 hours [03:02] * StevenK kills it with fire [03:02] Project parallel-test build (16): ABORTED in 2 days 18 hr: https://hudson.wedontsleep.org/job/parallel-test/16/ [03:10] ajmitch_: I found getting a new job worked wonders [03:10] lifeless: worked wonders for getting you away from wow for a bit? [03:11] A bit? [03:11] Like 10 months, more like [03:11] moving country helped too [03:11] StevenK: 6 [03:11] StevenK: april [03:13] lifeless: That's 7, nearly 8 :-P [03:13] I would complain about the state of internet connectivity here, but it wasn't much different in .au I guess [03:13] I had a loose heatsink then [03:13] wondered why I was getting thermal dmesgs and low performance [03:14] Project parallel-test build (17): ABORTED in 12 min: https://hudson.wedontsleep.org/job/parallel-test/17/ [03:14] * William Grant: PgTestSetup needs BaseLayer now, so set up BaseLayer and use the PID when checking that it respects LP_TEST_INSTANCE. [03:14] * William Grant: test_mlists now invokes scripts with the correct appserver config. [03:15] * William Grant: Fix restful-cache.txt to not depend on 'testrunner'. [03:15] * William Grant: Fix missed rename. [03:15] * Robert Collins: Merge wgrants fixes. [03:15] * Robert Collins: Refactor db layer db setup/teardown to make the db available to subordinate layers. [03:15] * William Grant: Scripts' -d arguments now properly override the DB name again. [03:15] * William Grant: Fix two missed canonical.lp.dbname references. [03:15] * StevenK blinks. [03:15] I didn't do that [03:16] I did [03:16] the branch it was testing has landed in db-stable [03:16] my librarian branch is the Next Big Thing [03:17] * spm gibbers [03:17] spm: does that mean you need gibbing? [03:17] The librarian branch seems to work. There's just a couple of dozen tests that need to be dehardcoded, and another one or two which have more complex failures. [03:18] lifeless: Should I change it to devel, or you've done it? [03:18] StevenK: i changed it to my librarian branch [03:18] The main issue after that is the SMTP server. [03:19] yes [03:19] bugs filed for that :) [03:19] Ah, great. [03:19] which interested people need to do [03:19] lifeless: needing gibbing? that sounds a touch personal? [03:19] It looks like we might have to write out ZCML... or otherwise hack around with Zope internals. [03:20] spm: hey, you're the gibberer [03:25] bah [03:26] OperationalError: FATAL: database "launchpad_ftest_template" does not exist [03:26] Hm? Where? [03:26] In a parallel run? [03:27] https://hudson.wedontsleep.org/job/parallel-test/18/console [03:27] theres a test somewhere that removes it [03:30] lifeless: That might be fallout from using the same build slave as #17 [03:30] StevenK: no [03:31] I filed a bug on it back a month [03:31] I was hoping it had been caught [03:36] Possibly test_sampledata [03:37] It does a pg_restore [03:48] Project parallel-test build (18): STILL FAILING in 33 min: https://hudson.wedontsleep.org/job/parallel-test/18/ [03:51] That's a bonus, it actually fails === jamesh_ is now known as jamesh [08:32] good morning [08:54] hi abel [09:02] Hi === gmb` is now known as gmb [09:33] bigjools: Morning. [09:34] morning === almaisan-away is now known as al-maisan [09:49] bigjools: What's the current status of the logparser? [09:50] wgrant: I landed the gzip fix but we're stuck on that error I showed you [09:50] Oh, right, forgot about that. [09:50] steve thinks it might be the 32 bit field I am reading in overflowing but that raises the question about how did gzip itself write the field [09:51] Hm? gzip rights the lowest 32 bits of the size. [09:51] Anything higher is stripped. [09:51] indeed [09:51] s/rights/writes/ [09:52] It's a completely irrelevant traceback, but that probably is it. [09:53] it's a Storm traceback [09:53] so it's caused by writing something bad into the db [09:56] I wish we could turn off write caching. [09:56] So we could actually see which line it was. [09:57] hmmm interesting idea [09:57] ... oh. [09:57] there might be a way [09:57] It's a postgres integer. [09:57] Which is 32-bit. [09:57] That would be the problem. [09:57] * bigjools gets coffee to think about it [09:57] Can you alter it to a bigint and see if it works? [09:58] Although that means we're above the 32-bit limit, so the gzip size calculation will be screwed. [09:58] Yya. === cody-somerville_ is now known as cody-somerville === matsubara-afk is now known as matsubara [12:11] Morning, all. [12:12] wgrant: still there? [12:12] howdy deryck [12:13] bigjools: I am. [12:13] gmb, hey, man. I need to settle in a bit this morning and then I'll turn to that troublesome lazr-js tarball. [12:13] https://launchpad.net/~openswan/+archive/openswan-testing/+packages [12:13] observe and wonder [12:13] Uhoh. [12:13] What's broken? [12:13] Why's it not published? [12:14] two i386 builds ... [12:14] Oh. [12:14] one from a different PPA [12:14] WTF. [12:14] Yeah. [12:14] WTF indeed. [12:15] deryck: Sure, no worries. I'm working on other stuff at the moment anyway. [12:15] gmb, ok, cool. [12:15] (I looked at the lazr-js stuff, decided that I didn't have enough brainjuice to comprehend the problem and thought it best to leave it til later) [12:16] deryck: If I can help in any way, though, let me know. [12:16] gmb, sure, thanks! will do. [12:17] bigjools: I recall being scared away by SPR.getBuildByArch a few months ago. [12:18] There was something strange with how it determined the build given published binaries. [12:18] I don't quite remember what, but it sounds relevant. [12:18] Let's see... [12:19] Oh, that's right. [12:19] wgrant: somehow, the copy he did from his personal PPA to the openswan one overlapped with a build in the openswan PPA [12:19] It made me cry because it issues so many damn queries. [12:19] bigjools: The openswan build is newer. [12:19] yes [12:20] Yessssss, I remember this code now. [12:20] Potentially hundreds of queries per build! [12:24] bigjools: Hmm. The creation date on the build appears to be ~10min after publication. [12:25] Oh. [12:25] No. [12:25] 1:50 *before*. [12:25] So it was copied, had the build created, deleted, then copied back. [12:26] the first had rebuild, the second didn't [12:26] Presumably. [12:26] Can you confirm that? [12:27] no :/ [12:27] Hm, the API should be helpful here. [12:27] * wgrant tries. === Odd_Blok1 is now known as Odd_Bloke [12:31] new b-m running as of 30 mins ago BTW [12:31] What's changed? [12:31] the extra pre-build ping [12:32] Oh, right. [12:33] Does it work? [12:33] shipova correctly failed [12:34] needs more load to be certain [12:34] we'll find out when the faily builds are dispatched [12:36] wgrant: so I suspect he deleted that package while it had a build in progress. Will have to play on that and see what DF does. [12:36] Did you deliberately schedule the daily builds so you're asleep? :P [12:36] I didn't schedule anything [12:36] jtv: name an arbitrary upstream project, please. ;-) [12:37] openobject-addons [12:37] If this is for verifying the migration script I'd look for at least: one with multiple series, one that's in ubuntu main, and one that's just very big—ideally of course one that has all. [12:38] Are you going to compare exports from staging (post-migration) to ones from production? [12:38] XPI won't work yet; KDE may possibly also still be broken. [12:39] jtv: as a start I wanted to see how the merge affects upstream translatoins. [12:39] openobject-addons is not linked to an ubuntu-packge, though [12:39] No. [12:39] But it's nice and large. [12:39] Just to make sure I'd also see if there's any effect on the Ubuntu package. [12:40] (For a product that _is_ linked to main packages, of course) [12:40] * bigjools -> fud [13:04] * henninge lunches === Ursinha-` is now known as Ursinha === Ursinha is now known as Ursinha-afk === Ursinha-afk is now known as Ursinha === Ursinha is now known as Guest51647 === Guest51647 is now known as Ursinha-afk === Ursinha-afk is now known as Guest17956 === Guest17956 is now known as Ursinha === beuno_ is now known as beuno === mrevell is now known as mrevell-lunch === NCommand1r is now known as NCommander === matsubara is now known as matsubara-lunch === Ursinha-afk is now known as Ursinha [14:20] wgrant: argh === mrevell-lunch is now known as mrevell === mbarnett` is now known as mbarnett === salgado is now known as salgado-lunch === bac` is now known as bac [15:22] abentley, hi. Do you happen to know what would happen to a bzr checkout of a branch that is upgraded to 2a after the checkout? Would bzr up fix it, or would devs need to blow the branch away and check it out again? === matsubara-lunch is now known as matsubara [15:24] gary_poster: bzr up would not revert it to the previous format. You'd need to upgrade the branch or downgrade the checkout. You can downgrade the checkout by blowing it away and re-checking out, or using "bzr upgrade --FORMAT". [15:24] bzr upgrade should work - we designed it to handle most cases. [15:25] abentley: sorry, I meant the original branch was upgraded, but ok, thanks lifeless, sounds like we can make it work one way or another. [15:25] lifeless: well, he was asking about "bzr up", which is "bzr update" IIRC. [15:26] that's correct abentley [15:26] the story is this: [15:26] I check out the download-cache, which is not 2a [15:27] I update the launchpad branch of download-cache to 2a [15:27] what do I do now to make my local checkout happy? [15:27] gary_poster: "bzr upgrade" in your checkout should do the trick. [15:27] ok awesome, thanks abentley [15:28] trying :-P [15:31] gmb, deryck, did you get the lazr-js version mismatch resolved? [15:32] gary_poster, heh. I literally *just* worked this out. :-) [15:32] heh, ok :-) [15:32] gary_poster, I believe http://pastebin.ubuntu.com/537940/ shows the issue. [15:32] * bigjools dances around the room, I finally sorted out buildd-manager timeouts [15:32] flacoste, hi [15:33] deryck: ah-hah, yes, I agree [15:33] gary_poster, so I'm fixing lazr-js now and will roll a new tarball. [15:34] cool deryck. /me is sympathetic to your many woes with this stuff :-/ [15:34] thanks. :-) [15:35] it's been so amazingly painful, I've quit complaining or worrying about it really [15:35] probably a good idea. :- [15:36] rockstar`, you around? [15:36] anyone know if I can turn off caching in storm so the traceback I get on a store flush is attached to the code that caused it? [15:37] ah nm === salgado-lunch is now known as salgado [15:41] Can I have a volunteer to try upgrading your download cache to see if these instructions work for you? Assuming you have a checkout (which rocketfuel uses), you would do a bzr upgrade followed by a bzr up. It works for me, but I own the LP branch. Could someone verify that it works for them too before I send an email out? [15:41] bigjools: mazel tov! [15:42] abentley: I'm not jewish but thanks :) [15:44] abentley: feel free to push ahead with your recipe changes [15:44] I'm landing another branch later that makes all the built file downloads asynchronous [15:45] bigjools: Cool. Does this mean we can drop this rlimit on recipe memory use? [15:46] abentley: I don't know, we could try and see what happens. The worst is that the builder blows up and the manager detects that and fails it [15:46] bigjools: that sounds appropriate. [15:46] abentley: we need to address the root cause though, is it bzr that eats memory when checking out those big branches? === jam1 is now known as jam [15:48] bigjools: Yes. Though another cause is we're using a version of bzr that is 2x worse in that regard. [15:48] ugh === rockstar` is now known as rockstar [15:56] rockstar, hey, dude. If you can spare a moment, can you review https://code.edge.launchpad.net/~deryck/lazr-js/fix-build-assuming-branch/+merge/42130 ? [15:57] deryck, did you see my email about this? [15:57] rockstar, yeah, just replied. That was my plan, too. [15:58] deryck, okay. === beuno is now known as beuno-lunch [16:22] gary_poster: I am upgrading mine now, I'll let you know [16:22] thanks bigjools [16:24] gary_poster: seems fine here [16:24] great, thank you bigjools [16:25] my pleasure === Ursinha is now known as Ursinha-afk [16:43] gary_poster: can I tap you for some help now please? :) [16:44] bigjools: sure :-) [16:45] gary_poster: http://librarian.dogfood.launchpad.net/57530918/1z7LhiYuLtwxWSJXAxHNvbyksXZ.txt [16:46] that's an error from the apache log parser [16:46] however i've got no idea what's causing it as the trace is not particularly useful :/ [16:46] ah, errors during the commit phase are always fun [16:46] I've tried setting the storm cache size to zero but no difference [16:47] so if you can offer any help here that'd be great :) [16:48] so, bigjools, do I assume correctly that this is some error received under more-or-less unknown circustances, and we don't know how to dupe? [16:49] gary_poster: I can dupe easily [16:49] it's processing the production logs that I copied to dogfood [16:50] ah ok [16:53] bigjools: I'd do the fairly obvious as a first step and try to see what the args to execute are. If I had to guess, you are trying to store a number that is too big for the DB columb [16:53] column [16:53] I'd see what the args are by hacking the egg [16:53] Because I'm evil that way [16:53] yeah, that much is obvious I think. I suspect it's the download counts themselves [16:53] * bigjools breaks an egg [16:55] I presume that DataError is from psycopg [16:55] * gary_poster wonders if the re-raise-an-exception-with-the-original-exception in Py 2.7 (IIRC) would make it easier to have reasonable behavior in situations like this, so that you can actually have a traceback that tells you what is going on [16:56] yeah, think so. there is a DataError in that package [16:57] useful tracebacks? sounds...useful :) [16:58] does LP_DEBUG_SQL_EXTRA or whatever not work here? [17:04] james_w: it depends on whether the error is from psycopg or the pg server [17:05] I'm not sure where it logs, but I might try it if I am desperate, the log will be swamped. [17:17] gary_poster: so, this is what is failing: args = ('UPDATE ParsedApacheLog SET date_last_parsed=%s, bytes_read=%s WHERE ParsedApacheLog.id = %s', ('2010-11-29 17:13:54.534869+00:00', 2623267714L, 802)) [17:18] bigjools: looks big. ;-) have you already looked up the ParchedApacheLog table to see what the constraints are? [17:18] looks like a signed integer overflow [17:19] heh, ParsedApacheLog [17:19] ParchedApacheLog needs water [17:19] :) [17:19] that could be a bigint (?) [17:19] it's a big freaking log file [17:20] * gary_poster will try to find table, but has little Postgres fu atm === benji is now known as benji-lunch [17:22] gary_poster: http://www.postgresql.org/docs/7.4/interactive/datatype.html#DATATYPE-INT [17:23] bigjools: right, and this appears to be an integer. bigint seems to be the way to go if this is not a "rethink the problem" sign [17:25] bigjools: is there a reason to not make it a bigint? [17:25] gary_poster: I can't think of one [17:25] then sounds like a win :-) [17:25] the file it's falling over on is a gzip which is 189M itself [17:26] heh [17:26] so is it worth doing step-back sorts of questions? [17:26] otherwise, +1 on going with a bigint change [17:27] well it's mainly a question of how big the logs get [17:27] and if this is a one-off, we can split the log up [17:27] well, presumably we'd want to do that in an automated way [17:28] why do we care about the precise number of bytes? [17:28] no idea! [17:28] heh [17:29] ah, it's to determine if we've read the whole thing or not, it looks like [17:29] yeah, just saw that too [17:30] hmmm, I think bigint is safer anyway [17:30] I think it will Just Work, no Python changes? [17:30] bigjools: looks that way to me, yes [17:31] okidoki, I'll get it sorted [17:31] cool [17:31] thanks gary_poster [17:31] np, thank you :-) [17:31] dogfood FTW [17:44] oh man, pushing db-devel branches is ball-achingly slow now [17:45] bigjools: setup the script [17:45] lifeless: que? [17:45] I sent a python script to fix it to the dev list [17:45] days ago :) [17:45] seems like I missed that [17:46] in the thread where thumper announce the lp:launchpad change [17:46] needs someone to do a rt and shepard it through [17:46] lifeless: I didn't understand who or where it has to run [17:47] and if it needs something like this running, why don't we just go back to how it was [17:47] because the way it was causes lots of grief we can't fix. [17:47] the performance, we can fix. [17:48] I didn't see much evidence of grief, just a couple of pebkac moments [17:48] anyway ... [17:48] alternatively [17:49] we could stack db-stable on stable [17:49] thumper: ^ [17:49] * lifeless shrugs [17:49] * lifeless does the on leave dance [17:50] lifeless: step away from the laptop === beuno-lunch is now known as beuno [17:55] * lifeless watches the mentalist and hacks on testtools [17:56] I love the roomba, its really quite effective [17:56] 'morning lifeless [17:56]