/srv/irclogs.ubuntu.com/2010/11/29/#launchpad-dev.txt

poolieis it normal that i have to run 'bzr update' in download-cache as well as update-sourcecode?00:28
jmlpoolie: yes.00:31
wgrantrocketfuel-get does both.00:37
pooliebut it's deprecated, according to some people?00:39
wgrantNAFAIK00:39
pooliespm: when lpnet is deployed, do you run 'make' in an existing tree, or a clean tree?00:39
jmlpoolie: I've never used it, since I felt it was a bit crappy to be working on bzr support for Launchpad and not actually using bzr to get my code00:40
spmpoolie: completely clean, the process is something like:00:40
spmcopy new tree as launchpad-revno-<revno>; make build in that; shutdown service; ln -s switchero (launchpad -> newshiny revno); startup00:41
lifelessmwhudson: hi00:41
lifelessmwhudson: re apache mangling; is there a bug for codebrowse?00:41
mwhudsonlifeless: hi00:41
spmpoolie: I grossly simplfy, but that's the gist.00:41
mwhudsonah no, i don't think so00:41
mwhudsonlet me search a bit00:41
lifelessmwhudson: we'll need one marked as a blocker for becoming an open id consumer00:41
lifelessmwhudson: probably needs a task on foundations00:42
lifelessnot because they should do it but because they are doing the openid stuff and need to see it00:42
pooliespm that's what i thought00:42
pooliei was just trying to work out bug 680339, but00:43
_mup_Bug #680339: 'Entry' object has no attribute 'markAsDuplicate' <Launchpad Bugs:New> <https://launchpad.net/bugs/680339>00:43
pooliei suspect it's an interaction thing and not a real bug00:43
wgrantpoolie: markAsDuplicate is a mutator.00:45
wgrantIt's not exposed as a method; you just set the attribute.00:45
StevenKI thought that changed, depending on the API version used?00:48
wgrantProbably.00:49
* jml gone again, back slightly later than UK starting time tomorrow morning00:49
wgrantBut I really hope not.00:50
wgrantBut probably.00:50
pooliewgrant: it was exposed as a method00:51
poolieit is now exposed as .duplicate = ...00:51
StevenKwgrant: Would you believe I still can't nail down where the spph is created for an nascent upload00:51
pooliewhen i first wrote my client code, the mutator didn't exist00:51
wgrantStevenK: PackageUploadSource.blahblah00:52
poolienow the method seems to have disappeared from the 1.0 api00:52
* StevenK declares tea will magically point it out to him00:52
wgrantpoolie: Well, did you know that Launchpad likes to make API stability guarantees without actually thinking about whether it can support them?00:52
pooliei was kind of suspecting that00:52
pooliethe "api stability" thread prompted me to think about it again00:53
pooliebut that turned out to be a somewhat different question00:53
wgrantAIUI the discussion basically went like this: "We need to have a stable API for Ubuntu releases to use." "OK, you can now annotate methods as being for a particular version." "OK, you must now support beta and 1.0 for 5 years, with no warning."00:55
wgrantNow we cannot change our datamodel.00:55
lifelessso00:56
lifelessthats an exaggeration, and you know it :)00:56
pooliei'm curious where those annotations go00:57
lifelessin the export() call00:57
lifelesssee salgados blueprint branch, for instance.00:57
lifelessthat exports only on devel (at my request)00:57
wgrantAPI stability guarantees are good. Simply freezing an entire API version is not.00:59
lifelessagreed01:00
lifelessEBeforeMyTime01:00
wgrantThe bits of the 1.0 API that need to be stable for 5 years are probably limited to a couple of methods.01:00
lifelessperhaps01:00
wgrantStevenK: Found it yet?01:03
StevenKwgrant: Nope01:03
wgrantStevenK: It should be in queue.py somewhere.01:04
* wgrant looks.01:04
wgrantPackageUploadSource.publish01:04
wgrantnewSourcePublication01:04
StevenKAh ha. There we get the SPR, so digging back for the history should be easy01:05
wgrantHA HA HA01:07
thumpergah01:20
thumperI have a test that regularly fails in ec2 but passes locally01:20
thumpers/regularly/always01:20
thumperhas to do with recipe breadcrumbs of all things01:20
poolielifeless: you're saying the export decorator controls which versions it appears in01:27
pooliethat seems plausible but markAsDuplicate calls it with no parameters01:27
wgrantlib/canonical/launchpad/rest/configuration.py:    last_version_with_mutator_named_operations = "beta"01:28
wgrantSo in beta, mutators appear as named operations.01:28
wgrantIn 1.0 and devel they do not.01:28
=== vednis is now known as mars
thumperwallyworld_: ping02:12
wallyworld_thumper: hello02:12
thumperwallyworld_: we should have a chat02:12
thumperalthough I might go make a coffee first02:12
wallyworld_thumper: ok02:12
thumpersound good?02:12
wallyworld_thumper: ack02:12
thumperwallyworld_:  bzr+ssh://bazaar.launchpad.net/~thumper/launchpad/recipe-binary-builds r1190402:24
thumperTestSourcePackageRecipeView02:24
thumperICanHasLinkedBranch(product).branch02:34
thumperICanHasLinkedBranch(sourcepackage).branch02:35
thumperICanHasLinkedBranch(productseries).branch02:36
lifeless=== Top 10 Time Out Counts by Page ID ===02:39
lifeless    Hard / Soft  Page ID02:39
lifeless     260 / 6568  Archive:+index02:39
lifeless     142 /  301  BugTask:+index02:39
lifeless      26 /  290  Distribution:+bugs02:39
lifeless      26 /  126  ProjectGroupSet:CollectionResource:#project_groups02:39
lifeless      10 /    4  Cve:+index02:39
lifeless       9 /   29  Milestone:+index02:39
lifeless       9 /   15  DistroSeries:+queue02:39
lifeless       8 /  107  Archive:+packages02:39
lifeless       8 /   12  DistroSeriesLanguage:+index02:39
lifeless       6 /    3  Person:+bugs02:39
thumperDifference: !=:02:41
thumperreference = 'Request builds for cake_recipe\nMaster Chef\nRecipes\ncake_recipe\nRequest builds for cake_recipe\nArchive:\nSecret PPA (chef/ppa)\nDistribution series:\nSecret Squirrel\nHoary\nWarty\nor\nCancel'02:41
thumperactual = u'Request builds for cake_recipe\nMaster Chef\nRecipes\ncake_recipe\nArchive:\nSecret PPA (chef/ppa)\nDistribution series:\nSecret Squirrel\nHoary\nWarty\nor\nCancel'02:41
wallyworld_lifeless: aren't you meant to be on leave?02:41
lifelesswallyworld_: I am02:41
wgrantwallyworld_: He's not very good at it.02:41
thumperlifeless: if you want something to do, fix my bug02:41
thumperlifeless: it passes for me and wallyworld_ but fails in ec202:42
lifelessthumper: I'm not doing work02:42
wallyworld_lifeless: well fcuk off then and enjoy your break :-)02:42
thumperlifeless: no, this would be fun :)02:42
lifelesswallyworld_: I am, I've scrastched a bunch of itches in lp:testrepository02:42
lifelessand watched a movie and a half, plus judge john deed02:42
lifelesscleaned up a bit in my office02:42
lifelessand got wow reinstalled.02:43
wallyworld_lifeless: i still think you need some "downtime" :-)02:43
StevenKlifeless: O.O at the last one02:45
ajmitch_StevenK: you managed to break free from that?02:45
StevenKajmitch_: Er, not quite02:45
lifelessStevenK: google for wine + could not fix permissions, or thereabouts - launcher.exe fail.02:45
ajmitch_let me guess, waiting impatiently for the 7th?02:45
StevenKajmitch_: Yes :-)02:45
lifelesswallyworld_: thats why I have a long block of leave. It takes me weeks to spin down.02:46
* wgrant never fell into the WoW trap.02:46
lifelesswe can help02:46
StevenKlifeless: Is that what you're currently tripping on?02:46
StevenKHaha02:46
lifelessStevenK: was02:46
ajmitch_wgrant: be glad :)02:46
lifelessStevenK: thus the reinstall.02:46
StevenKlifeless: Has it downloaded the 10GiB of patches it needs?02:46
lifelessthe incremental downloader is damn awesome02:46
lifelessyeah02:46
lifelessits all good, I logged in and checked its working. Still haven't decided if 'm going to get the upgrade or not.02:47
wgrantStevenK: Ah, so crazy Blizzard patches aren't just limited to SC2? When I reinstalled last week it must have reached 100% eight or nine times...02:47
StevenKwgrant: Oh no, WoW is where they perfected that02:47
lifelesswgrant: on wine or windows?02:48
wgrantlifeless: Both.02:48
StevenKajmitch_: Does that mean you pulled yourself out of it?02:51
ajmitch_StevenK: well I stopped raiding at least, that has to count for something?02:51
StevenKHaha02:51
StevenKNeat. parallel-test #16 has been building for 2 days 18 hours03:02
* StevenK kills it with fire03:02
LPCIBotProject parallel-test build (16): ABORTED in 2 days 18 hr: https://hudson.wedontsleep.org/job/parallel-test/16/03:02
lifelessajmitch_: I found getting a new job worked wonders03:10
ajmitch_lifeless: worked wonders for getting you away from wow for a bit?03:10
StevenKA bit?03:11
StevenKLike 10 months, more like03:11
lifelessmoving country helped too03:11
lifelessStevenK: 603:11
lifelessStevenK: april03:11
StevenKlifeless: That's 7, nearly 8 :-P03:13
ajmitch_I would complain about the state of internet connectivity here, but it wasn't much different in .au I guess03:13
lifelessI had a loose heatsink then03:13
lifelesswondered why I was getting thermal dmesgs and low performance03:13
LPCIBotProject parallel-test build (17): ABORTED in 12 min: https://hudson.wedontsleep.org/job/parallel-test/17/03:14
LPCIBot* William Grant: PgTestSetup needs BaseLayer now, so set up BaseLayer and use the PID when checking that it respects LP_TEST_INSTANCE.03:14
LPCIBot* William Grant: test_mlists now invokes scripts with the correct appserver config.03:14
LPCIBot* William Grant: Fix restful-cache.txt to not depend on 'testrunner'.03:15
LPCIBot* William Grant: Fix missed rename.03:15
LPCIBot* Robert Collins: Merge wgrants fixes.03:15
LPCIBot* Robert Collins: Refactor db layer db setup/teardown to make the db available to subordinate layers.03:15
LPCIBot* William Grant: Scripts' -d arguments now properly override the DB name again.03:15
LPCIBot* William Grant: Fix two missed canonical.lp.dbname references.03:15
* StevenK blinks.03:15
StevenKI didn't do that03:15
lifelessI did03:16
lifelessthe branch it was testing has landed in db-stable03:16
lifelessmy librarian branch is the Next Big Thing03:16
* spm gibbers03:17
lifelessspm: does that mean you need gibbing?03:17
wgrantThe librarian branch seems to work. There's just a couple of dozen tests that need to be dehardcoded, and another one or two which have more complex failures.03:17
StevenKlifeless: Should I change it to devel, or you've done it?03:18
lifelessStevenK: i changed it to my librarian branch03:18
wgrantThe main issue after that is the SMTP server.03:18
lifelessyes03:19
lifelessbugs filed for that :)03:19
wgrantAh, great.03:19
lifelesswhich interested people need to do03:19
spmlifeless: needing gibbing? that sounds a touch personal?03:19
wgrantIt looks like we might have to write out ZCML... or otherwise hack around with Zope internals.03:19
lifelessspm: hey, you're the gibberer03:20
lifelessbah03:25
lifelessOperationalError: FATAL:  database "launchpad_ftest_template" does not exist03:26
wgrantHm? Where?03:26
wgrantIn a parallel run?03:26
lifelesshttps://hudson.wedontsleep.org/job/parallel-test/18/console03:27
lifelesstheres a test somewhere that removes it03:27
StevenKlifeless: That might be fallout from using the same build slave as #1703:30
lifelessStevenK: no03:30
lifelessI filed a bug on it back a month03:31
lifelessI was hoping it had been caught03:31
wgrantPossibly test_sampledata03:36
wgrantIt does a pg_restore03:37
LPCIBotProject parallel-test build (18): STILL FAILING in 33 min: https://hudson.wedontsleep.org/job/parallel-test/18/03:48
StevenKThat's a bonus, it actually fails03:51
=== jamesh_ is now known as jamesh
adeuringgood morning08:32
pooliehi abel08:54
mrevellHi09:02
=== gmb` is now known as gmb
wgrantbigjools: Morning.09:33
bigjoolsmorning09:34
=== almaisan-away is now known as al-maisan
wgrantbigjools: What's the current status of the logparser?09:49
bigjoolswgrant: I landed the gzip fix but we're stuck on that error I showed you09:50
wgrantOh, right, forgot about that.09:50
bigjoolssteve thinks it might be the 32 bit field I am reading in overflowing but that raises the question about how did gzip itself write the field09:50
wgrantHm? gzip rights the lowest 32 bits of the size.09:51
wgrantAnything higher is stripped.09:51
bigjoolsindeed09:51
wgrants/rights/writes/09:51
wgrantIt's a completely irrelevant traceback, but that probably is it.09:52
bigjoolsit's a Storm traceback09:53
bigjoolsso it's caused by writing something bad into the db09:53
wgrantI wish we could turn off write caching.09:56
wgrantSo we could actually see which line it was.09:56
bigjoolshmmm interesting idea09:57
wgrant... oh.09:57
bigjoolsthere might be a way09:57
wgrantIt's a postgres integer.09:57
wgrantWhich is 32-bit.09:57
wgrantThat would be the problem.09:57
* bigjools gets coffee to think about it09:57
wgrantCan you alter it to a bigint and see if it works?09:57
wgrantAlthough that means we're above the 32-bit limit, so the gzip size calculation will be screwed.09:58
wgrantYya.09:58
=== cody-somerville_ is now known as cody-somerville
=== matsubara-afk is now known as matsubara
deryckMorning, all.12:11
bigjoolswgrant: still there?12:12
bigjoolshowdy deryck12:12
wgrantbigjools: I am.12:13
deryckgmb, hey, man.  I need to settle in a bit this morning and then I'll turn to that troublesome lazr-js tarball.12:13
bigjoolshttps://launchpad.net/~openswan/+archive/openswan-testing/+packages12:13
bigjoolsobserve and wonder12:13
wgrantUhoh.12:13
wgrantWhat's broken?12:13
wgrantWhy's it not published?12:13
bigjoolstwo i386 builds ...12:14
wgrantOh.12:14
bigjoolsone from a different PPA12:14
wgrantWTF.12:14
wgrantYeah.12:14
bigjoolsWTF indeed.12:14
gmbderyck: Sure, no worries. I'm working on other stuff at the moment anyway.12:15
deryckgmb, ok, cool.12:15
gmb(I looked at the lazr-js stuff, decided that I didn't have enough brainjuice to comprehend the problem and thought it best to leave it til later)12:15
gmbderyck: If I can help in any way, though, let me know.12:16
deryckgmb, sure, thanks!  will do.12:16
wgrantbigjools: I recall being scared away by SPR.getBuildByArch a few months ago.12:17
wgrantThere was something strange with how it determined the build given published binaries.12:18
wgrantI don't quite remember what, but it sounds relevant.12:18
wgrantLet's see...12:18
wgrantOh, that's right.12:19
bigjoolswgrant: somehow, the copy he did from his personal PPA to the openswan one  overlapped with a build in the openswan PPA12:19
wgrantIt made me cry because it issues so many damn queries.12:19
wgrantbigjools: The openswan build is newer.12:19
bigjoolsyes12:19
wgrantYessssss, I remember this code now.12:20
wgrantPotentially hundreds of queries per build!12:20
wgrantbigjools: Hmm. The creation date on the build appears to be ~10min after publication.12:24
wgrantOh.12:25
wgrantNo.12:25
wgrant1:50 *before*.12:25
wgrantSo it was copied, had the build created, deleted, then copied back.12:25
bigjoolsthe first had rebuild, the second didn't12:26
wgrantPresumably.12:26
wgrantCan you confirm that?12:26
bigjoolsno :/12:27
wgrantHm, the API should be helpful here.12:27
* wgrant tries.12:27
=== Odd_Blok1 is now known as Odd_Bloke
bigjoolsnew b-m running as of 30 mins ago BTW12:31
wgrantWhat's changed?12:31
bigjoolsthe extra pre-build ping12:31
wgrantOh, right.12:32
wgrantDoes it work?12:33
bigjoolsshipova correctly failed12:33
bigjoolsneeds more load to be certain12:34
bigjoolswe'll find out when the faily builds are dispatched12:34
bigjoolswgrant: so I suspect he deleted that package while it had a build in progress.  Will have to play on that and see what DF does.12:36
wgrantDid you deliberately schedule the daily builds so you're asleep? :P12:36
bigjoolsI didn't schedule anything12:36
henningejtv: name an arbitrary upstream project, please. ;-)12:36
jtvopenobject-addons12:37
jtvIf this is for verifying the migration script I'd look for at least: one with multiple series, one that's in ubuntu main, and one that's just very big—ideally of course one that has all.12:37
jtvAre you going to compare exports from staging (post-migration) to ones from production?12:38
jtvXPI won't work yet; KDE may possibly also still be broken.12:38
henningejtv: as a start I wanted to see how the merge affects upstream translatoins.12:39
henningeopenobject-addons is not linked to an ubuntu-packge, though12:39
jtvNo.12:39
jtvBut it's nice and large.12:39
jtvJust to make sure I'd also see if there's any effect on the Ubuntu package.12:39
jtv(For a product that _is_ linked to main packages, of course)12:40
* bigjools -> fud12:40
* henninge lunches13:04
=== Ursinha-` is now known as Ursinha
=== Ursinha is now known as Ursinha-afk
=== Ursinha-afk is now known as Ursinha
=== Ursinha is now known as Guest51647
=== Guest51647 is now known as Ursinha-afk
=== Ursinha-afk is now known as Guest17956
=== Guest17956 is now known as Ursinha
=== beuno_ is now known as beuno
=== mrevell is now known as mrevell-lunch
=== NCommand1r is now known as NCommander
=== matsubara is now known as matsubara-lunch
=== Ursinha-afk is now known as Ursinha
bigjoolswgrant: argh14:20
=== mrevell-lunch is now known as mrevell
=== mbarnett` is now known as mbarnett
=== salgado is now known as salgado-lunch
=== bac` is now known as bac
gary_posterabentley, hi.  Do you happen to know what would happen to a bzr checkout of a branch that is upgraded to 2a after the checkout?  Would bzr up fix it, or would devs need to blow the branch away and check it out again?15:22
=== matsubara-lunch is now known as matsubara
abentleygary_poster: bzr up would not revert it to the previous format.  You'd need to upgrade the branch or downgrade the checkout.  You can downgrade the checkout by blowing it away and re-checking out, or using "bzr upgrade --FORMAT".15:24
lifelessbzr upgrade should work - we designed it to handle most cases.15:24
gary_posterabentley: sorry, I meant the original branch was upgraded, but ok, thanks lifeless, sounds like we can make it work one way or another.15:25
abentleylifeless: well, he was asking about "bzr up", which is "bzr update" IIRC.15:25
gary_posterthat's correct abentley15:26
gary_posterthe story is this:15:26
gary_posterI check out the download-cache, which is not 2a15:26
gary_posterI update the launchpad branch of download-cache to 2a15:27
gary_posterwhat do I do now to make my local checkout happy?15:27
abentleygary_poster: "bzr upgrade" in your checkout should do the trick.15:27
gary_posterok awesome, thanks abentley15:27
gary_postertrying :-P15:28
gary_postergmb, deryck, did you get the lazr-js version mismatch resolved?15:31
deryckgary_poster, heh.  I literally *just* worked this out. :-)15:32
gary_posterheh, ok :-)15:32
deryckgary_poster, I believe http://pastebin.ubuntu.com/537940/ shows the issue.15:32
* bigjools dances around the room, I finally sorted out buildd-manager timeouts15:32
Ursinhaflacoste, hi15:32
gary_posterderyck: ah-hah, yes, I agree15:33
deryckgary_poster, so I'm fixing lazr-js now and will roll a new tarball.15:33
gary_postercool deryck.  /me is sympathetic to your many woes with this stuff :-/15:34
deryckthanks. :-)15:34
deryckit's been so amazingly painful, I've quit complaining or worrying about it really15:35
gary_posterprobably a good idea. :-15:35
deryckrockstar`, you around?15:36
bigjoolsanyone know if I can turn off caching in storm so the traceback I get on a store flush is attached to the code that caused it?15:36
bigjoolsah nm15:37
=== salgado-lunch is now known as salgado
gary_posterCan I have a volunteer to try upgrading your download cache to see if these instructions work for you?  Assuming you have a checkout (which rocketfuel uses), you would do a bzr upgrade followed by a bzr up.  It works for me, but I own the LP branch.  Could someone verify that it works for them too before I send an email out?15:41
abentleybigjools: mazel tov!15:41
bigjoolsabentley: I'm not jewish but thanks :)15:42
bigjoolsabentley: feel free to push ahead with your recipe changes15:44
bigjoolsI'm landing another branch later that makes all the built file downloads asynchronous15:44
abentleybigjools: Cool.  Does this mean we can drop this rlimit on recipe memory use?15:45
bigjoolsabentley: I don't know, we could try and see what happens.  The worst is that the builder blows up and the manager detects that and fails it15:46
abentleybigjools: that sounds appropriate.15:46
bigjoolsabentley: we need to address the root cause though, is it bzr that eats memory when checking out those big branches?15:46
=== jam1 is now known as jam
abentleybigjools: Yes.  Though another cause is we're using a version of bzr that is 2x worse in that regard.15:48
bigjoolsugh15:48
=== rockstar` is now known as rockstar
deryckrockstar, hey, dude.  If you can spare a moment, can you review  https://code.edge.launchpad.net/~deryck/lazr-js/fix-build-assuming-branch/+merge/42130 ?15:56
rockstarderyck, did you see my email about this?15:57
deryckrockstar, yeah, just replied.  That was my plan, too.15:57
rockstarderyck, okay.15:58
=== beuno is now known as beuno-lunch
bigjoolsgary_poster: I am upgrading mine now, I'll let you know16:22
gary_posterthanks bigjools16:22
bigjoolsgary_poster: seems fine here16:24
gary_postergreat, thank you bigjools16:24
bigjoolsmy pleasure16:25
=== Ursinha is now known as Ursinha-afk
bigjoolsgary_poster: can I tap you for some help now please? :)16:43
gary_posterbigjools: sure :-)16:44
bigjoolsgary_poster: http://librarian.dogfood.launchpad.net/57530918/1z7LhiYuLtwxWSJXAxHNvbyksXZ.txt16:45
bigjoolsthat's an error from the apache log parser16:46
bigjoolshowever i've got no idea what's causing it as the trace is not particularly useful :/16:46
gary_posterah, errors during the commit phase are always fun16:46
bigjoolsI've tried setting the storm cache size to zero but no difference16:46
bigjoolsso if you can offer any help here that'd be great :)16:47
gary_posterso, bigjools, do I assume correctly that this is some error received under more-or-less unknown circustances, and we don't know how to dupe?16:48
bigjoolsgary_poster: I can dupe easily16:49
bigjoolsit's processing the production logs that I copied to dogfood16:49
gary_posterah ok16:50
gary_posterbigjools: I'd do the fairly obvious as a first step and try to see what the args to execute are.  If I had to guess, you are trying to store a number that is too big for the DB columb16:53
gary_postercolumn16:53
gary_posterI'd see what the args are by hacking the egg16:53
gary_posterBecause I'm evil that way16:53
bigjoolsyeah, that much is obvious I think.  I suspect it's the download counts themselves16:53
* bigjools breaks an egg16:53
bigjoolsI presume that DataError is from psycopg16:55
* gary_poster wonders if the re-raise-an-exception-with-the-original-exception in Py 2.7 (IIRC) would make it easier to have reasonable behavior in situations like this, so that you can actually have a traceback that tells you what is going on16:55
gary_posteryeah, think so.  there is a DataError in that package16:56
bigjoolsuseful tracebacks?  sounds...useful :)16:57
james_wdoes LP_DEBUG_SQL_EXTRA or whatever not work here?16:58
bigjoolsjames_w: it depends on whether the error is from psycopg or the pg server17:04
bigjoolsI'm not sure where it logs, but I might try it if I am desperate, the log will be swamped.17:05
bigjoolsgary_poster: so, this is what is failing: args = ('UPDATE ParsedApacheLog SET date_last_parsed=%s, bytes_read=%s WHERE ParsedApacheLog.id = %s', ('2010-11-29 17:13:54.534869+00:00', 2623267714L, 802))17:17
gary_posterbigjools: looks big. ;-) have you already looked up the ParchedApacheLog table to see what the constraints are?17:18
bigjoolslooks like a signed integer overflow17:18
gary_posterheh, ParsedApacheLog17:19
gary_posterParchedApacheLog needs water17:19
bigjools:)17:19
bigjoolsthat could be a bigint (?)17:19
bigjoolsit's a big freaking log file17:19
* gary_poster will try to find table, but has little Postgres fu atm17:20
=== benji is now known as benji-lunch
bigjoolsgary_poster: http://www.postgresql.org/docs/7.4/interactive/datatype.html#DATATYPE-INT17:22
gary_posterbigjools: right, and this appears to be an integer.  bigint seems to be the way to go if this is not a "rethink the problem" sign17:23
gary_posterbigjools: is there a reason to not make it a bigint?17:25
bigjoolsgary_poster: I can't think of one17:25
gary_posterthen sounds like a win :-)17:25
bigjoolsthe file it's falling over on is a gzip which is 189M itself17:25
gary_posterheh17:26
gary_posterso is it worth doing step-back sorts of questions?17:26
gary_posterotherwise, +1 on going with a bigint change17:26
bigjoolswell it's mainly a question of how big the logs get17:27
bigjoolsand if this is a one-off, we can split the log up17:27
gary_posterwell, presumably we'd want to do that in an automated way17:27
gary_posterwhy do we care about the precise number of bytes?17:28
bigjoolsno idea!17:28
gary_posterheh17:28
gary_posterah, it's to determine if we've read the whole thing or not, it looks like17:29
bigjoolsyeah, just saw that too17:29
bigjoolshmmm, I think bigint is safer anyway17:30
bigjoolsI think it will Just Work, no Python changes?17:30
gary_posterbigjools: looks that way to me, yes17:30
bigjoolsokidoki, I'll get it sorted17:31
gary_postercool17:31
bigjoolsthanks gary_poster17:31
gary_posternp, thank you :-)17:31
bigjoolsdogfood FTW17:31
bigjoolsoh man, pushing db-devel branches is ball-achingly slow now17:44
lifelessbigjools: setup the script17:45
bigjoolslifeless: que?17:45
lifelessI sent a python script to fix it to the dev list17:45
lifelessdays ago :)17:45
bigjoolsseems like I missed that17:45
lifelessin the thread where thumper announce the lp:launchpad change17:46
lifelessneeds someone to do a rt and shepard it through17:46
bigjoolslifeless: I didn't understand who or where it has to run17:46
bigjoolsand if it needs something like this running, why don't we just go back to how it was17:47
lifelessbecause the way it was causes lots of grief we can't fix.17:47
lifelessthe performance, we can fix.17:47
bigjoolsI didn't see much evidence of grief, just a couple of pebkac moments17:48
bigjoolsanyway ...17:48
lifelessalternatively17:48
lifelesswe could stack db-stable on stable17:49
lifelessthumper: ^17:49
* lifeless shrugs17:49
* lifeless does the on leave dance17:49
bigjoolslifeless: step away from the laptop17:50
=== beuno-lunch is now known as beuno
* lifeless watches the mentalist and hacks on testtools17:55
lifelessI love the roomba, its really quite effective17:56
jelmer'morning lifeless17:56
lifelesshi17:56
bigjoolslifeless: I wanted one of those and someone warned me off it!17:57
lifelessbigjools: they were jealous :)17:58
bigjoolslifeless: no, they already had one!17:58
jelmerlifeless: which generation do you have?17:58
jelmermine has a fondness for cables, I can't leave it alone17:58
lifelessjelmer: 4 weeks old17:58
bigjoolsheh17:58
bigjoolslifeless: timeouts in b-m are no more \o/17:59
lifelessit will wind them up, but loose cables are fugly anyway, so just routing them along the wall and putting U restraints in is sufficient17:59
lifelessbigjools: congrats17:59
bigjoolsthat was a *nasty* bug17:59
bigjoolsand I am going to celebrate with a beer now, good ngiht17:59
lifelesscody reckons theres more to it17:59
lifelessI reckon fixed is grand17:59
bigjoolswell if he wants to contribute something concrete then I'll listen ;)17:59
elmobigjools: what was the problem?18:00
bigjoolselmo: the problem is still there, I just worked around it18:00
elmooh, I see18:00
bigjoolsit sends a dummy connection before the real work starts18:00
bigjoolsthen waits for it to either work or time out18:00
cody-somervilleI believe the hypothesis was that Xen guests drop the first packet or something?18:01
bigjoolsI still see the timeouts in the log but we don't chuck the toys out the pram now18:01
bigjoolscody-somerville: that's the behaviour I was seeing, but only a) randomly, b) when we connect *immediately* after the guest is reset18:01
bigjoolsthere's some interesting stuff on google about xen and dropped packets18:02
bigjoolsanyway, I have to run18:02
bigjoolsg'night all18:02
lifelessjml: hi18:08
=== benji-lunch is now known as benji
=== matsubara is now known as matsubara-afk
=== al-maisan is now known as almaisan-away
cr3is there a preferred xml parser in launchpad? minidom, expat, cElementTree, etc.?20:11
thumperabentley: bzr+ssh://bazaar.launchpad.net/~thumper/launchpad/recipe-binary-builds20:18
thumperTestSourcePackageRecipeView20:19
=== salgado is now known as salgado-afk
lifelesscr3: anything-but-xml ?20:29
cr3lifeless: heh, I'd normally agree with you, but I'm helping with another project which uses junit xml files and I usually try to inspire myself from the tools used by launchpad :)20:31
lifelesscr3: outputting or reading?20:32
cr3lifeless: just reading20:32
cr3lifeless: come to think of it, you might already have something in subunit20:32
* cr3 looks20:32
lifelessmmm, I don't htink there is a junit library for python yet - all the code I've seen is like junitxml - output only.20:32
lifelesshowever20:32
lifelessif you are writing parsing support, let me encourage you to submit patches to python-junitxml, I'd be happy to have a parser in there.20:33
cr3lifeless: ugh, I really don't like the print of xml strings in junitxml :(20:33
lifelessif the project is python based they may be using subunit anyway and only doing xml casting for hudson20:33
lifelesscr3: anyhow to answer the qyestion20:34
lifelessusing anything baked into python is going to be tolerable20:35
lifelessthough I'm fairly sure that modelling junit in launchpad would be a mistake ;)20:35
lifelessjml: https://code.launchpad.net/~lifeless/testtools/listtests/+merge/42166 :) - coming after that, load-list.20:36
cr3lifeless: different project with the server team20:36
cr3lifeless: but, for the sake of future considerations, I agree with you. if there's anything junit related, it would be client side20:37
lifelesscr3: Seriously though, if you do write python modelling code for junit, please supply patches to python-junitxml, I'd be very happy to have a reader exist there.20:39
cr3lifeless: sure thing20:41
thumperWARNING:root:Memcache set failed for pt:testrunner:lp/registry/templates.....  WTF??20:47
thumperseeing lots of these20:47
lifelessmemcache isn't running ?20:48
thumperI'm running tests20:51
thumperwhy isn't it running?20:51
lifelessdunno, its just a thought20:52
thumperlifeless: I had that thought too, but not sure why it isn't running20:53
* thumper sighs20:55
thumperlifeless: the output of --list-tests can't be fed back into --load-list20:55
lifelessthumper: doctests?20:59
thumperlifeless: yes, and others20:59
lifelesshttps://bugs.launchpad.net/launchpad-foundations/+bug/68277220:59
_mup_Bug #682772: doctest construction generates duplicate test ids <Launchpad Foundations:New> <https://launchpad.net/bugs/682772>20:59
thumper  test_unknown_baseurl (lp.bugs.tests.test_bugwatch.SFExtractBugTrackerAndBugTest)20:59
thumperis another20:59
lifelessthumper: what else?20:59
thumperlifeless: abentley helped me debug my test issue a little, and it is a test isolation problem20:59
lifelessthat should work, if thats the test id; its an unusual id but no reason it wouldn't work.20:59
thumperso I was trying to bisect20:59
thumperlifeless: it isn't working21:00
lifelessthumper: why doesn't it work ? [also, file a bug :)]21:00
thumperlifeless: I don't know, but it just isn't running them21:00
thumperlifeless: file a bug on what?21:00
lifelesslaunchpad-foundations21:00
lifelesswhomever analyses further (not me, I *am* on leave), can move it to some other project if needed - but its probably something in LP.21:01
* thumper sighs agani21:01
Guest66935lifeless: in bug 638924, you mention that bugs and milestones have existing off-by-default eager loading. I don't know if I'm missing something, but the only eager loading I see for bugs is the indexed_messages attribute, and I don't see any eager loading for milestones.21:02
_mup_Bug #638924: Milestone:+index timeouts with many bugs <pg83> <timeout> <Launchpad Registry:In Progress by edwin-grubbs> <https://launchpad.net/bugs/638924>21:02
=== Guest66935 is now known as EdwinGrubbs
EdwinGrubbshmm, I wonder why I was Guest66935.21:03
jelmerEdwinGrubbs: freenode will rename you when you fail to identify in time21:04
jelmerbac: Hi!21:28
bachi jelmer21:28
jelmerbac: Thanks for handling CHR today. I noticed you assigned the recipe build issues to Soyuz, they should actually be in launchpad-code.21:29
gary_posterbac, you back in the states?21:29
bacjelmer: oh, sorry21:29
bacgary_poster: yep21:29
gary_postercool, bac.  hope it was a good trip21:29
bacgary_poster: yeah, it was.  glad to be home.21:29
jelmerbac: No problem, just thought I'd mention it if there are any more.21:30
gary_posterGreat.21:30
bacjelmer: no more.  thanks for the heads up.21:30
marsthumper, ping21:35
thumpermars: hi21:35
marsHi thumper21:36
marsthumper, I was just looking at r11995 of devel, which just got merged into db-devel, and started dying there21:36
thumpermars: yes...21:36
thumpermars: my question would be "what else changed around that time?"21:37
marsthumper, do you see any way that your change may be related?  The suite has failed in a similar place twice in a row21:37
thumperbecause that revision touches exactly zero to do with the librarian21:37
thumpermars: similar, but not the same21:37
marsyep21:38
marsthumper, thank you for the info21:39
thumpernp21:39
=== jcsackett_ is now known as jcsackett
mars'what else changed' is a good question.  But according to buildbot - nothing else changed? :(21:39
lifelessEdwinGrubbs: hi21:40
lifelessEdwinGrubbs: searchTasks has eager loading support21:40
lifelessEdwinGrubbs: and theres an eager load flag I added when improving blueprints performance a couple of months back21:41
gary_posterderyck: is there a generic "our windmill tests suck" bug?  If there is, I figure you'd know :-) (I would add https://bugs.launchpad.net/launchpad-foundations/+bug/681817 to the list if so?)21:44
_mup_Bug #681817: lp.registry.windmill.tests.test_team_index.TestTeamIndex.test_addmember fails intermittently <Launchpad Foundations:New> <https://launchpad.net/bugs/681817>21:44
EdwinGrubbslifeless: is it prejoins parameter for searchTasks()? That would enable to pull in the BugTask.assignee, but it would be less straightforward to pull out the info from the ValidPersonCache.21:59
lifelessvpc isn't needed22:00
lifelesswhat you need is Person._validity_clauses22:00
lifelessyes, prejoins is it, I think.22:02
lifelessfor bugtasks22:02
lifelesssee Person._validity_queries22:03
lifelessfor a reused code fragment that will handle validity checks22:03
lifelessthat is in fact used from milestones in milestone prejoining22:03
lifeless - grep for it22:04
lifelessread a bit, theres a lot of unrefactored code, you'll need to page it into your head to see how the bits hang together (sadly)22:04
lifelessits generally taken me a couple of hours to find a good place to push on, when making these changes.22:04
deryckgary_poster, no, there isn't a single bug for the issue.  Just individual bugs on trouble tests.22:04
deryckgary_poster, we should probably use a tag22:04
lifelessbut it gets better each time22:04
gary_posterack thanks deryck .  ok, will start with "windmill" ?22:05
* lifeless trolls: windfail22:05
deryckheh22:05
lifelessor perhaps failmill22:05
gary_poster:-P :-)22:05
deryckwindflail22:05
deryckgary_poster, yeah, just windmill for now works22:06
gary_posterok22:06
deryckok, out for my evening.  Until tomorrow all.22:07
wgrantGrarrrrr.22:16
mwhudsonmorning william22:17
lifelesshenninge: hai22:17
lifelessbah22:17
lifelesswgrant: hai22:17
henninge;-)22:17
henningehi wgrant ;)22:18
wgrantBug #68273822:18
_mup_Bug #682738: Split the PPA publisher into public/private processes <ppa> <soyuz-publisher> <Soyuz:Triaged> <https://launchpad.net/bugs/682738>22:18
wgrantGrarrring at the response to that.22:18
lifelessmine or bigjools?22:18
wgrantbigjools.22:19
lifelesswell, say so :)22:20
wgrantAlready did.22:20
lifelessin the bug ?22:20
wgrantYeah.22:20
LPCIBotProject devel build (259): FAILURE in 3 hr 46 min: https://hudson.wedontsleep.org/job/devel/259/22:51
LPCIBot* Launchpad Patch Queue Manager: [r=henninge][ui=none][no-qa] Convert some webservice tests for22:51
LPCIBotblueprints into model tests.22:51
LPCIBot* Launchpad Patch Queue Manager: [r=mwhudson][ui=none][bug=146389] Start exposing ISpecification and22:51
LPCIBotIHasSpecifications attributes on the webservice API.22:51
wgrantHah, ENOSPC22:52
=== flacoste changed the topic of #launchpad-dev to: Launchpad Development Channel | Week 2 of 10.12 | Performance Tuesday | PQM is open | firefighting: db-devel broken | https:/​/​dev.launchpad.net/ | Get the code: https:/​/​dev.launchpad.net/​Getting
flacostewallyworld, thumper: are you guys doing anything special for performance Tue?23:21
thumperflacoste: no, I'm attempting to find out why I have test isolation failures23:21
flacosteyeah, that seems sensible23:22
wallyworldflacoste: not today, i've got a branch to finish and another to start after that23:22
wallyworldwgrant: seems a build upload is stuck, any ideas? https://launchpad.net/~nova-core/+archive/trunk/+build/206963323:34
wallyworldwgrant: it says it finished building over an hour ago23:35
wgrantwallyworld: Ugggh.23:35
wgrantBinary builds aren't meant to get stuck :(23:35
wgrantCan you see if others have been uploading?23:35
wallyworldnot sure how to do that :-)23:35
wallyworldwgrant: can you tell me where to look?23:37
wgrantwallyworld: Can you look for process-upload.py logs from cesium?23:39
wallyworldwgrant: will do23:39
wgrantArgh, the daily builds have just hit.23:39
wgrantbut they seem to be running OK.23:40
wgrantTHere was a binary build uploaded <30 minutes ago.23:40
wgrantSo it's not a general problem.23:40
wgrantAny errors in the p-u log?23:40
wallyworldwgrant: looking....23:40
spmwgrant (who has not yet started with us...): ref https://bugs.launchpad.net/soyuz/+bug/663562 this appears to have bit again on linux 2.6.24-28.81 in hardy. anything I can poke that may give more useful info?23:40
_mup_Bug #663562: duplicate orig for "linux" package in hardy <soyuz-core> <soyuz-security> <Soyuz:Incomplete> <https://launchpad.net/bugs/663562>23:40
wgrantspm: Grar. We'll have to recowboy.23:42
wgrantI was hoping a real Soyuz person would fix the data before we had to do this again.23:42
wgrantspm: Do you need a new patch?23:43
spmthere are no real soyuz persons. they're unreal.23:43
spmwgrant: I suspect so23:43
wgrantspm: http://pastebin.ubuntu.com/516480/ is what I gave mbarnett last time, and it still works fine.23:44
wgrantCowboy it on cocoplum.23:44
wgrantAnd get jdstrand to use unembargo-package.py23:44
spmhookay, ta23:44
wgrantMyes.23:45
spmis that something we apply, get the unembargo'd, then revert? or is safe to leave? or?23:45
wgrantAnd then uncowboy it. Preferably in quick succession.23:45
spmgreat minds.23:45
wgrantNot safe to leave, no.23:45
spmit didn't look it, just looking at the removals23:45
wallyworldwgrant: logs appear to indicate that the ulpoad of the nova ppa went ok. there's some process messages and then "Finished Checking Upload"23:46
wgrantwallyworld: Yay... might need someone with both log access and uploader knowledge to look at it tonight.23:46
wgrantspm: Huh?23:47
wgrantOh, I see.23:47
spmyeah - rm'ing those checks was setting off alarm bells for me23:47
wallyworldwgrant: can i check anything else? this is in response to a question on #launchpad so i'd like to be able to tell them something if possible23:48
spmand that's *before* coffee too23:48
wgrantAs long as no AAs are doing any copying at the moment (it's rare), it should be fine.23:48
wgrantwallyworld: Not entirely sure. If the uploader log looks fine, there's something pretty broken.23:49
wgrantBecause there are no binaries.23:49
wallyworldwgrant: want me to pastebin the log info just so you can eyeball it?23:50
spmwgrant-aaaaaa.patch <== it seems like a suitable name to give this patch23:50
wgrantwallyworld: Probably a good idea.23:50
wgrantspm: Yep.23:50
spmdry-run applies cleanly. I'll do the juju when jdstrand is around again. thanks muchly wgrant!23:51
mbarnetthah, i was totally PMing spm this in a far less intelligible way.   Perhaps i should pay attention to what's going on around me...23:51
spmNEVER23:51
mbarnettokay, good.23:51
* mbarnett goes back to staring blankly at the floor.23:52
spmI'd suggest you're a crazy texan, but you might pull a gun on me23:52
wallyworldwgrant: https://pastebin.canonical.com/40234/23:52
wgrantwallyworld: That's private.23:52
wallyworldwgrant: what's the public url?23:54
wallyworldwgrant: nevermind, found fit23:54
wgrantpaste.ubuntu.com23:54
wallyworldwgrant: http://paste.ubuntu.com/538151/23:55
wgrantwallyworld: Those are different builds (same package, different series). Anything mentioning 2069633?23:58
* wallyworld looks23:58

Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!