/srv/irclogs.ubuntu.com/2010/06/15/#launchpad-dev.txt

krkhanmerges for implementing api calls should be proposed against devel or db-devel?00:20
krkhanwgrant: branching lp:launchpad gives me a copy of db-devel. that's confusing me as to whether i should branch should i target for merge proposal00:23
wgrantkrkhan: If they need DB changes, db-devel.00:27
wgrantOtherwise, devel.00:27
krkhanwgrant: ok, thanks00:28
ajmitchhow should tests be written for extending the API?00:28
wgrantajmitch: See lib/lp/APP/stories/webservice00:28
ajmitchthanks00:29
wgrantYou can use launchpadlib in tests now, though, so try to find a new one that does.00:29
wgrantIt makes everything much prettier.00:29
ajmitch& less likely to write a bad test00:31
krkhanajmitch: if using launchpadlib for testing webservice, "make clean" is required for lplib to recognize new methods. wasted a few hours on that thing00:33
wgrant'make build' should do it.00:33
ajmitchkrkhan: yeah I knew that one, or at least removing & recreating the apidoc target00:33
brycehwgrant, yes, would be nice if it did00:34
ajmitchbut it still thiks the apidoc target is up-to-date00:34
wgrantHmm.00:34
ajmitchkrkhan: I was just looking at that getBugSubscriberPackages() method when I stumbled across what you'd done on it :)00:35
krkhanajmitch: i'm just going to propose a branch for exporting that method. it's useful :)00:37
ajmitchyes, it came up in #ubuntu-devel this morning00:37
wgrantbdmurray: That branch of yours says that it exports suppress_notify... but @call_with declares that it's not exported. I am confused.00:46
=== Ursinha-dinner is now known as Ursinh
=== Ursinh is now known as Ursinha
bdmurraywgrant: well its still usable by launchpad itself but not API clients01:00
wgrantAh, so it was deliberately *un*exported?01:01
wgrantThat makes more sense.01:01
bdmurrayyes, sorry for any confusion01:01
krkhanwhat would be the command to push a personal launchpad devel branch. bzr push lp:~user/launchpad/devel/branchxyz gives permission denied error01:06
wgrantkrkhan: Drop the devel/01:06
krkhanwgrant: dropping devel just pushes a branch with the message "Using default stacking branch /~launchpad-pqm/launchpad/db-devel"01:08
krkhani don't want db-devel01:09
krkhanthat is bzr push lp:~user/launchpad/branchxyz creates a stacking branch referring to /~launchpad-pqm/launchpad/db-devel01:11
wgrantThat's fine.01:11
wgrantStacking just means it shares revisions, so you have to upload less.01:11
wgrantIt has no further meaning.01:11
krkhanbut then the merge proposal is created against db-devel too. i'm confused, i want the proposal to be against devel01:12
wgrantYou need to select devel when creating the merge proposal.01:13
krkhanwgrant: oh okay, i got it. thanks01:14
lajjrHello jml and flacoste05:26
=== almaisan-away is now known as al-maisan
adeuringgood morning08:19
kb9vqfSo which cron script purges deleted pacakges from Librarian?08:23
kb9vqfOr technically I guess from disk?08:23
wgrantexpire-archive-files, then librarian-gc08:24
wgrantAre you also running process-death-row regularly?08:24
wgrantBut expire-archive-files should only remove stuff that's been gone a week, IIRC.08:25
kb9vqfThere it is; I had librarian-gc but not the other two08:25
wgrantp-d-r removes the files from the archive.08:26
wgrante-a-f expires the librarian files once they're removed from the archive disk.08:26
wgrantAnd l-gc removes expired librarian files from disk.08:26
kb9vqfActually it looks like I was only missing p-d-r08:27
kb9vqfThe other two I had found and was running on a cron job08:27
wgrantThe e-a-f stay of execution is configurable.08:29
kb9vqfp-d-r didn't do anything; I still have 1.3gb of deleted packages floating in the ether08:30
kb9vqfIt said it found 0B to delete08:30
wgrantWhat said that?08:30
wgrantp-d-r?08:30
StevenKprocess-death-row08:31
kb9vqfYes08:31
wgrantI know what it expands to -- I was just wondering if that was what said that.08:31
StevenKwgrant: Oh, never mind, too little context, too much context switching08:31
kb9vqfTechnically it said this:08:31
kb9vqf2010-06-15 07:26:43 INFO    Creating lockfile: /var/lock/launchpad-process-death-row.lock08:31
kb9vqf2010-06-15 07:26:52 INFO    Processing http://archive.quickbuild.pearsoncomputing.net/ubuntu08:31
kb9vqf2010-06-15 07:26:52 INFO    Removing 0 files marked for reaping08:31
kb9vqf2010-06-15 07:26:52 INFO    Total bytes freed: 008:31
wgrantkb9vqf: The packages are actually Deleted, not Superseded?08:31
kb9vqfI deleted them manually08:32
wgrantSuperseded packages will only be considered for removal from disk after 24 hours.08:32
wgrantHmm.08:32
wgrantYou've run the publisher since deleting them?08:32
kb9vqfMaybe the counter isn't updated?08:32
kb9vqfYeah, I have the publisher on a 1 minute timer right now08:32
kb9vqfSo I'm sure it ran ;-)08:32
wgrantCan you point me at the PPA in question?08:32
kb9vqfhttps://quickbuild.pearsoncomputing.net/~trinity/+archive/trinity/+packages08:33
kb9vqfThe size should be around a few hundred MB at most08:34
wgrantOh, you are running p-d-r with --ppa, right?08:35
kb9vqfNo!08:35
kb9vqfLet me try that :-)08:35
wgrantAll the publishing scripts need to be called with that.08:35
wgrantmmm, not using proper Debian PPAs yet? :(08:36
kb9vqfOK, I wasn't sure if it was just the ones that were close to poppy (i.e. in the upload->build->publishing chain) or not08:36
kb9vqfNot yet08:36
kb9vqfGot some build issues to work out first08:36
wgrantAnything that needs to touch the archive, basically. Since the PPA and primary archives are on separate machines in production.08:36
kb9vqfThat worked; thanks!08:37
StevenKwgrant: I suspect that isn't the only reason.08:37
wgrantWell, true. The primary archive publisher takes three eternities.08:38
wgrantAnd there are probably other reasons lost in the mysteries of time.08:38
kb9vqfRegarding Debian PPAs, I'm going to get the build system functioning correctly, then I have a couple weeks of other work to do, then I'll come back and try to hack in correct Debian support08:39
kb9vqfThere are a bunch of problems that are really kinda weird and may require significant rewrites08:40
kb9vqfStarting with the fact that the upload processor doesn't find any distroseries not belonging to Ubuntu08:40
wgrantHmmm. I've not had that problem.08:41
wgrantAs long as the PPA's distribution is set correctly, and the upload path is right, there's no reason it shouldn't work.08:41
wgrantAnd I'm pretty sure I tried that, although it was nearly a year ago.08:41
kb9vqfI might have done something wrong but I'm fairly certain I tried debian in the upload path to no effect08:42
kb9vqfI spent most of a day trying to get it to work08:42
wgrantMaybe I should try that tonight.08:42
StevenKkb9vqf: You know https://quickbuild.pearsoncomputing.net/debian doesn't work, right?08:43
StevenKWhereas /ubuntu does, for example08:43
wgrantI suspect the DB was reset 4 hours ago.08:43
kb9vqfWhy yes it was08:43
StevenKHeh08:43
kb9vqfI was hoping no one would notice08:44
kb9vqf:-P08:44
StevenKBecause scorched earth solves everything08:44
kb9vqfWhen I ran out of disk space a bunch of stuff broke08:44
kb9vqfI couldn't get all the pieces back together again, so....08:44
* kb9vqf notes it's a really bad sign when the ubuntu celebrity is suddenly not found08:45
kb9vqfStevenK: https://quickbuild.pearsoncomputing.net/debian works now ;-)08:47
* wgrant resets his DB and tries Debian PPAs again.08:47
wgrantkb9vqf: You're manually setting external dependencies on each PPA to the real Debian archive, I guess?08:54
kb9vqfwgrant: Yes; I set a flag file in the Debian chroot and when it is detected the builder overrides the provided sources.list with sed s/ubuntu/debian/g or similar08:54
kb9vqfThere's a bit more to it but it works pretty well08:54
kb9vqfas hacks go, anyway... :-P08:55
wgrantAhahah. That is horrible :)08:55
kb9vqfBut it's allowing me to proceed to fixing the Trinity build system for the moment08:55
kb9vqfIt will be fixed ASAP :)08:55
wgrantYep.08:56
* kb9vqf does not miss the PPA build queue on broken package resubmission08:58
wgrantHeh.08:58
kb9vqfSo much nicer for figuring out these weird libtool errors ;-)08:58
kb9vqfAlmost makes the past week worth it :-P08:58
wgrantOh lucilleconfig, I hate you so so much.09:09
mrevellMorning09:14
wgrantkb9vqf: So, it's working OK for me. I did get a missing distroseries error, but it was misleading: I'd used a bad upload path, so it went into an error handler which assumed Ubuntu to make error messages nicer.09:24
wgrantSo make sure you're looking at the first error message.09:24
kb9vqfOK, I will try that when I have time.  Thanks for checking it out; my Python skills really stink09:25
kb9vqfCome to think of it I might have actually created the distroseries incorrectly in the database the first time09:26
kb9vqfWe shall see in a couple weeks09:26
wgrantUhoh, the Launchpad Buildbot appears to have become sentient.09:30
wgrantIt's commenting on bugs.09:30
ajmitchthat's a bit worrying09:31
wgrantBug #59352209:31
_mup_Bug #593522: Don't send out real email from staging codehosting <Launchpad Bazaar Integration:Fix Released by canonical-losas> <https://launchpad.net/bugs/593522>09:31
ajmitchNow if only you could get it to fix bugs as well09:32
kb9vqfWell it already appears to have an IQ higher than most Internet users...it speaks in complete sentences ;-)09:33
spmthat's coming version 2. version 3 is when it starts patching itself. live. version 4 is when it invents a new killer language + api's; rewrites itself into that. declares the rest of the world as backwards and hence obsolete; and turns the entire planet into a giant supercomputer. something pity about 42 in there as well I'm sure.09:33
ajmitchspm: great, when can it write tests for LP code as well?09:34
spms/pity/pithy/09:34
spmajmitch: that'd be version 8 based on the above timeline. simple extrapolation.09:34
* ajmitch has been slacking just a little bit & has finally got back to bug 146389, apart from the little issue of writing tests09:35
_mup_Bug #146389: api for blueprint tracker <api> <feature> <Launchpad Blueprints:In Progress by ajmitch> <https://launchpad.net/bugs/146389>09:35
wgrantIt's much less painful with launchpadlib.09:35
thumperajmitch: oh dear09:35
thumperajmitch: I started looking into that09:36
ajmitchyeah, given that I've been testing manually with it as well09:36
thumperajmitch: and ran screaming09:36
ajmitchthumper: great to know!09:36
ajmitchhow far did you get?09:36
thumperajmitch: as I felt that it needed a machette09:36
ajmitchit does, really09:36
thumperajmitch: too much weird shit logic in the wrong place09:36
thumperajmitch: the model really needs an overhaul09:36
wgrantWell.09:36
wgrantIt's Blueprint.09:36
thumperwgrant: agreed09:36
wgrantIt needs a machette. To the whole app.09:36
ajmitchbut I've just been exporting properties & methods, going WTF in a few places09:36
thumperwgrant: with wikis, I'll be better ;-)09:37
thumpermaybe09:37
ajmitchwondering how I make an exported field editable, all that fun stuff09:37
* wgrant throws stuff at buildd-manager.09:37
thumperajmitch: there is special logic in most of the other setter methods09:37
thumperajmitch: which is why I didn't go ahead and make the methods writable09:38
* ajmitch just made a big, big mistake09:38
ajmitchrunning 'bzr viz' on LP09:38
thumperajmitch: recording things like started date, and who did it09:38
thumperfinished date et al.09:38
wgrantajmitch: qlog works OK.09:38
ajmitchthumper: one thing that the distro team appears to use, is splitting a single blueprint into multiple work items using the whiteboard09:39
ajmitchideally that'd be supported better than just a big text mess09:39
ajmitchif someone will give some love to blueprints :)09:39
thumperajmitch: I'll mentor you :)09:39
ajmitchshared drinking sessions for blueprint code? sounds like a plan09:40
* ajmitch thinks that it'll need a lot more thought put into the design09:42
wgrantOr a good rm -r lib/lp/blueprints09:43
wgrantWhy are they separate?09:43
ajmitchseparate from bugs?09:44
wgrantYes.09:44
ajmitchI think because they were at least meant to have features that didn't really map well to the bug workflow09:45
ajmitchsome things like the dependencies between blueprints would be nice to have in bugs09:46
wgrantExactly.09:46
wgrantMost features unique to blueprints would be useful in bugs too.09:46
wgrantOh.09:47
wgrantI hate you Twisted.09:47
wgrantWhy must you hang when attempting to serialise an XML-RPC request containing None...09:47
wgrantmthaddon: So we needn't worry that the DC is going to come and conquer the world?10:16
mthaddonwgrant: the machines are on the march10:17
wgrantkb9vqf: Once I set up lucilleconfig on Debian and its series and configured nominatedarchindep on lenny, and hacked traverse_named_ppa to not care about the distribution, it all works fine.10:27
wgrantbigjools: You don't object strongly to https://code.edge.launchpad.net/~wgrant/launchpad/bug-592935-hide-disabled-ppas/+merge/27411  or https://code.edge.launchpad.net/~wgrant/launchpad/no-buildd-ogre-model/+merge/27410 , do you?12:27
* bigjools OTP, gimme some 12:29
wgrantk12:30
bigjoolswgrant: +1 to both12:52
bigjoolsprovided that the user can still see his own disabled PPAs12:52
wgrantThey can.12:52
wgrantThanks.12:52
bigjoolswe need a better solution for that in the long term, but it'll do for now12:52
bigjoolsthanks for the change12:52
wgrantYep.12:52
wgrantbigjools: Is the log parser CP blocking on that dogfood issue?13:11
=== matsubara-afk is now known as matsubara
=== Ursinha-afk is now known as Ursinha
=== henninge_ is now known as henninge
=== stub1 is now known as stub
=== stub1 is now known as stub
=== stub1 is now known as stub
=== mrevell is now known as mrevell-lunch
=== mrevell-lunch is now known as mrevell
bigjoolswgrant: it's blocking because it doesn't work14:20
didrocksjml: hey, to implement https://blueprints.launchpad.net/ubuntu/+spec/desktop-maverick-quickly as discussed at UDS (for pushing ssh/gpg key, signing CoC and creating a ppa), I will try to connect "the bad way" to launchpad :/ Is there any way to make it less hackish than screenscraping and avoiding breaking the user?14:23
wgrantbigjools: It doesn't?14:23
jmldidrocks, I'm on the phone.14:23
didrocksjml: no pb, will idle there :)14:23
bigjoolswgrant: no, I removed all existing db entries and re-ran it.  Same issue.14:23
bigjoolsso there's a definite code problem14:23
wgrantAh, I didn't know that bit.14:23
wgrantI wonder why it doesn't use fd.tell() instead of working it out manually.14:29
wgrantbigjools: Any chance you can stick something like a "assert parsed_bytes == fd.tell()" after "parsed_bytes += len(line)", delete the ParsedApacheLog for that file, then rerun?14:33
wgrantWould be nice to see where it actually starts going wrong...14:34
bigjoolswgrant: not in the near future, I'm way too busy14:34
wgrantOK.14:34
bigjoolssorry14:34
wgrantNo worries.14:34
=== al-maisan is now known as almaisan-away
krkhanis there a way for me to debug doc tests quickly? while writing unittests i did pdb.set_trace() inside one of the tests and then played around. for doc tests, pdb.set_trace() loses all context14:44
bigjoolskrkhan: go up one in the stack and the context is there14:55
bigjoolsso hit "u" after it breaks14:55
krkhanbigjools: thanks :-D14:57
bigjoolsit used to stop in the right stack frame, I wonder what changed :(14:57
didrocksjml: still on the phone? :)16:28
=== salgado is now known as salgado-afk
jmldidrocks, no, doing performance reviews. :)16:56
jmldidrocks, I have to confess I forgot about your question16:56
didrocksjml: just ping me when you will some free time, ok? ;)16:56
jmldidrocks, sure.16:56
jmldidrocks, actually, looking at your question, can you please just ask it on the launchpad-dev@lists.lp.net mailing list?16:57
jmldidrocks, I'm really not the best person to ask.16:57
didrocksjml: sure, doing it now. Thanks!16:57
didrocksjml: did you book for RMLL, btw?16:58
jmlmaybe16:58
jmlI'm pretty sure I'm speaking there.16:58
didrocksright, on wednesday IIRC :)16:59
jmlI'm dealing with all of that stuff tomorrow.16:59
didrocksgood luck for your reviewing time :)16:59
=== beuno is now known as beuno-lunch
krkhanin a doc test, login(..); team = factory.makeTeam(); response = webservice.get("/~"+team.name); returns a http response with 500 internal server error. any ideas on what i'm doing wrong?17:03
jelmerkrkhan: you might have to commit() the change17:10
krkhanjelmer: login(...); team=factory.makeTeam(); transaction.commit(); response = webservice.get(...); still returns 500 internal server rror17:19
jelmerkrkhan: with what error message in the 500 internal server error?17:20
krkhanjelmer: empty body, but with ('x-lazr-oopsid', 'OOPS-1627T162') in header17:21
=== kiko is now known as kiko-fud
allenapsinzui: Are milestone listing in memcached now? How long for? Is there any way to force a refresh?17:31
sinzuiallenap, 10 minutes active milestones, 3 hours inactive17:32
sinzuithe status are 1 hour17:32
allenapsinzui: bug status?17:32
sinzuiallenap, bug listing are 10 minutes in active milestones, and 3 hours inactive milestones17:33
allenapsinzui: You said "the status are 1 hour". I didn't understand what that meant.17:35
sinzuiThe status and assignment summary above the listings caches for 1 hour17:36
sinzuisee blog.launchpad.net17:36
krkhanfound the fix. had to call logout() before using webservice17:37
mrevellNight all18:01
=== beuno-lunch is now known as beuno
=== Ursinha-afk is now known as Ursinha
=== deryck is now known as deryck[lunch]
=== Ursinha is now known as Ursinha-bbl
=== matsubara is now known as matsubara-afk
=== Ursinha-bbl is now known as GoBrazil
=== deryck[lunch] is now known as deryck
=== kiko-fud is now known as kiko
=== matsubara-afk is now known as matsubara
=== GoBrazil is now known as Ursinha
krkhanhow should i encode get queries which contain newline characters? "var=line1%0Aline2" gives a HTTP 400 Bad Request response21:35
lifelesskrkhan: what variables are you using that you want newlines in ?21:38
lifelessI suspect that we'd be looking for POST form encoding for mutate operations21:39
krkhanlifeless: for a text search. i was using restclient before but using urllib from python works fine22:04
lifelesskrkhan: interesting22:05
lifelessI'm fairly sure that \r and \n will be ignored by the search engine. IMBW.22:05
krkhanlifeless: i was trying to implement my own search method for bug attachments. anyways, python's urllib works fine as far as request is concerned (it doesn't give http 400) but on the server side i still only see the part before the linebreak22:07
krkhanws.op=abc&text=line1%0Aline2 results in text=line1 at the server side22:08
lifelessplease file a bug on that22:08
lifelessit suggests a possible security hole22:08
krkhanlifeless: but i can't be sure this is a launchpad issue. i mean, perhaps get requests aren't supposed to have newlines? (IMBW as well)22:10
lifelesskrkhan: either way ;)22:10
lifelesskrkhan: there is a bug, and we should get to the bottom of it22:10
krkhanlifeless: okay, doing that now. btw, i'm curious as to how this could be a possible security hole22:11
lifelesswell22:12
lifelessimagine that line2 isn't actually lost22:12
lifelesswhat is it doing? does a subsequent request get it? does it enter via an unknown code path?22:12
lifelessyou could use it to craft a link to launchpad that wouldn't do what the user things it would, for instance.22:13
lifelessI'm probably not devious enough, buts it that sort of inconsistent behaviour between a client and a server that leads to many vulnerabilities22:13
krkhanah. i see. i'm a little unsure as to how to present the issue in the bug report. i'm using my own method here and inserting a pdb.set_trace() at the server side to see that the text isn't received properly22:13
lifelessdocument what you know22:14
krkhanokay22:14
lifelessthats all that one can ask22:14
krkhanlifeless: i think it's an issue in httplib2 :) i just tried using launchpadlib to send newline characters and they reached fine22:29
krkhanso the problem was only with httplib2 and restclient22:29
lifelessok22:31
lifelessthanks for digging22:31
krkhannp22:37
=== Ursinha is now known as Ursinha-fud
sinzuiabentley, I think https://lpbuildbot.canonical.com/builders/lp/builds/982/steps/shell_7/logs/summary shows that Warty\nHoary\nSecret Squirrel does not match, but I cannot say the test is broken23:05
=== matsubara is now known as matsubara-afk
abentleysinzui, I think the problem is there's no defined order for the distroseries.23:12
sinzuiah23:13
wgrantsinzui, abentley: I have an approved branch which fixes that.23:34
wgrant(sorting them by version)23:34
wgrantAlso, neither of the two ec2tests that were running last night bothered to email me.23:38
wgrantAlthough one submitted to PQM successfully.23:38

Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!