[00:20] <krkhan> merges for implementing api calls should be proposed against devel or db-devel?
[00:23] <krkhan> wgrant: branching lp:launchpad gives me a copy of db-devel. that's confusing me as to whether i should branch should i target for merge proposal
[00:27] <wgrant> krkhan: If they need DB changes, db-devel.
[00:27] <wgrant> Otherwise, devel.
[00:28] <krkhan> wgrant: ok, thanks
[00:28] <ajmitch> how should tests be written for extending the API?
[00:28] <wgrant> ajmitch: See lib/lp/APP/stories/webservice
[00:29] <ajmitch> thanks
[00:29] <wgrant> You can use launchpadlib in tests now, though, so try to find a new one that does.
[00:29] <wgrant> It makes everything much prettier.
[00:31] <ajmitch> & less likely to write a bad test
[00:33] <krkhan> ajmitch: if using launchpadlib for testing webservice, "make clean" is required for lplib to recognize new methods. wasted a few hours on that thing
[00:33] <wgrant> 'make build' should do it.
[00:33] <ajmitch> krkhan: yeah I knew that one, or at least removing & recreating the apidoc target
[00:34] <bryceh> wgrant, yes, would be nice if it did
[00:34] <ajmitch> but it still thiks the apidoc target is up-to-date
[00:34] <wgrant> Hmm.
[00:35] <ajmitch> krkhan: I was just looking at that getBugSubscriberPackages() method when I stumbled across what you'd done on it :)
[00:37] <krkhan> ajmitch: i'm just going to propose a branch for exporting that method. it's useful :)
[00:37] <ajmitch> yes, it came up in #ubuntu-devel this morning
[00:46] <wgrant> bdmurray: That branch of yours says that it exports suppress_notify... but @call_with declares that it's not exported. I am confused.
[01:00] <bdmurray> wgrant: well its still usable by launchpad itself but not API clients
[01:01] <wgrant> Ah, so it was deliberately *un*exported?
[01:01] <wgrant> That makes more sense.
[01:01] <bdmurray> yes, sorry for any confusion
[01:06] <krkhan> what would be the command to push a personal launchpad devel branch. bzr push lp:~user/launchpad/devel/branchxyz gives permission denied error
[01:06] <wgrant> krkhan: Drop the devel/
[01:08] <krkhan> wgrant: dropping devel just pushes a branch with the message "Using default stacking branch /~launchpad-pqm/launchpad/db-devel"
[01:09] <krkhan> i don't want db-devel
[01:11] <krkhan> that is bzr push lp:~user/launchpad/branchxyz creates a stacking branch referring to /~launchpad-pqm/launchpad/db-devel
[01:11] <wgrant> That's fine.
[01:11] <wgrant> Stacking just means it shares revisions, so you have to upload less.
[01:11] <wgrant> It has no further meaning.
[01:12] <krkhan> but then the merge proposal is created against db-devel too. i'm confused, i want the proposal to be against devel
[01:13] <wgrant> You need to select devel when creating the merge proposal.
[01:14] <krkhan> wgrant: oh okay, i got it. thanks
[05:26] <lajjr> Hello jml and flacoste
[08:19] <adeuring> good morning
[08:23] <kb9vqf> So which cron script purges deleted pacakges from Librarian?
[08:23] <kb9vqf> Or technically I guess from disk?
[08:24] <wgrant> expire-archive-files, then librarian-gc
[08:24] <wgrant> Are you also running process-death-row regularly?
[08:25] <wgrant> But expire-archive-files should only remove stuff that's been gone a week, IIRC.
[08:25] <kb9vqf> There it is; I had librarian-gc but not the other two
[08:26] <wgrant> p-d-r removes the files from the archive.
[08:26] <wgrant> e-a-f expires the librarian files once they're removed from the archive disk.
[08:26] <wgrant> And l-gc removes expired librarian files from disk.
[08:27] <kb9vqf> Actually it looks like I was only missing p-d-r
[08:27] <kb9vqf> The other two I had found and was running on a cron job
[08:29] <wgrant> The e-a-f stay of execution is configurable.
[08:30] <kb9vqf> p-d-r didn't do anything; I still have 1.3gb of deleted packages floating in the ether
[08:30] <kb9vqf> It said it found 0B to delete
[08:30] <wgrant> What said that?
[08:30] <wgrant> p-d-r?
[08:31] <StevenK> process-death-row
[08:31] <kb9vqf> Yes
[08:31] <wgrant> I know what it expands to -- I was just wondering if that was what said that.
[08:31] <StevenK> wgrant: Oh, never mind, too little context, too much context switching
[08:31] <kb9vqf> Technically it said this:
[08:31] <kb9vqf> 2010-06-15 07:26:43 INFO    Creating lockfile: /var/lock/launchpad-process-death-row.lock
[08:31] <kb9vqf> 2010-06-15 07:26:52 INFO    Processing http://archive.quickbuild.pearsoncomputing.net/ubuntu
[08:31] <kb9vqf> 2010-06-15 07:26:52 INFO    Removing 0 files marked for reaping
[08:31] <kb9vqf> 2010-06-15 07:26:52 INFO    Total bytes freed: 0
[08:31] <wgrant> kb9vqf: The packages are actually Deleted, not Superseded?
[08:32] <kb9vqf> I deleted them manually
[08:32] <wgrant> Superseded packages will only be considered for removal from disk after 24 hours.
[08:32] <wgrant> Hmm.
[08:32] <wgrant> You've run the publisher since deleting them?
[08:32] <kb9vqf> Maybe the counter isn't updated?
[08:32] <kb9vqf> Yeah, I have the publisher on a 1 minute timer right now
[08:32] <kb9vqf> So I'm sure it ran ;-)
[08:32] <wgrant> Can you point me at the PPA in question?
[08:33] <kb9vqf> https://quickbuild.pearsoncomputing.net/~trinity/+archive/trinity/+packages
[08:34] <kb9vqf> The size should be around a few hundred MB at most
[08:35] <wgrant> Oh, you are running p-d-r with --ppa, right?
[08:35] <kb9vqf> No!
[08:35] <kb9vqf> Let me try that :-)
[08:35] <wgrant> All the publishing scripts need to be called with that.
[08:36] <wgrant> mmm, not using proper Debian PPAs yet? :(
[08:36] <kb9vqf> OK, I wasn't sure if it was just the ones that were close to poppy (i.e. in the upload->build->publishing chain) or not
[08:36] <kb9vqf> Not yet
[08:36] <kb9vqf> Got some build issues to work out first
[08:36] <wgrant> Anything that needs to touch the archive, basically. Since the PPA and primary archives are on separate machines in production.
[08:37] <kb9vqf> That worked; thanks!
[08:37] <StevenK> wgrant: I suspect that isn't the only reason.
[08:38] <wgrant> Well, true. The primary archive publisher takes three eternities.
[08:38] <wgrant> And there are probably other reasons lost in the mysteries of time.
[08:39] <kb9vqf> Regarding Debian PPAs, I'm going to get the build system functioning correctly, then I have a couple weeks of other work to do, then I'll come back and try to hack in correct Debian support
[08:40] <kb9vqf> There are a bunch of problems that are really kinda weird and may require significant rewrites
[08:40] <kb9vqf> Starting with the fact that the upload processor doesn't find any distroseries not belonging to Ubuntu
[08:41] <wgrant> Hmmm. I've not had that problem.
[08:41] <wgrant> As long as the PPA's distribution is set correctly, and the upload path is right, there's no reason it shouldn't work.
[08:41] <wgrant> And I'm pretty sure I tried that, although it was nearly a year ago.
[08:42] <kb9vqf> I might have done something wrong but I'm fairly certain I tried debian in the upload path to no effect
[08:42] <kb9vqf> I spent most of a day trying to get it to work
[08:42] <wgrant> Maybe I should try that tonight.
[08:43] <StevenK> kb9vqf: You know https://quickbuild.pearsoncomputing.net/debian doesn't work, right?
[08:43] <StevenK> Whereas /ubuntu does, for example
[08:43] <wgrant> I suspect the DB was reset 4 hours ago.
[08:43] <kb9vqf> Why yes it was
[08:43] <StevenK> Heh
[08:44] <kb9vqf> I was hoping no one would notice
[08:44] <kb9vqf> :-P
[08:44] <StevenK> Because scorched earth solves everything
[08:44] <kb9vqf> When I ran out of disk space a bunch of stuff broke
[08:44] <kb9vqf> I couldn't get all the pieces back together again, so....
[08:45]  * kb9vqf notes it's a really bad sign when the ubuntu celebrity is suddenly not found
[08:47] <kb9vqf> StevenK: https://quickbuild.pearsoncomputing.net/debian works now ;-)
[08:47]  * wgrant resets his DB and tries Debian PPAs again.
[08:54] <wgrant> kb9vqf: You're manually setting external dependencies on each PPA to the real Debian archive, I guess?
[08:54] <kb9vqf> wgrant: Yes; I set a flag file in the Debian chroot and when it is detected the builder overrides the provided sources.list with sed s/ubuntu/debian/g or similar
[08:54] <kb9vqf> There's a bit more to it but it works pretty well
[08:55] <kb9vqf> as hacks go, anyway... :-P
[08:55] <wgrant> Ahahah. That is horrible :)
[08:55] <kb9vqf> But it's allowing me to proceed to fixing the Trinity build system for the moment
[08:55] <kb9vqf> It will be fixed ASAP :)
[08:56] <wgrant> Yep.
[08:58]  * kb9vqf does not miss the PPA build queue on broken package resubmission
[08:58] <wgrant> Heh.
[08:58] <kb9vqf> So much nicer for figuring out these weird libtool errors ;-)
[08:58] <kb9vqf> Almost makes the past week worth it :-P
[09:09] <wgrant> Oh lucilleconfig, I hate you so so much.
[09:14] <mrevell> Morning
[09:24] <wgrant> kb9vqf: So, it's working OK for me. I did get a missing distroseries error, but it was misleading: I'd used a bad upload path, so it went into an error handler which assumed Ubuntu to make error messages nicer.
[09:24] <wgrant> So make sure you're looking at the first error message.
[09:25] <kb9vqf> OK, I will try that when I have time.  Thanks for checking it out; my Python skills really stink
[09:26] <kb9vqf> Come to think of it I might have actually created the distroseries incorrectly in the database the first time
[09:26] <kb9vqf> We shall see in a couple weeks
[09:30] <wgrant> Uhoh, the Launchpad Buildbot appears to have become sentient.
[09:30] <wgrant> It's commenting on bugs.
[09:31] <ajmitch> that's a bit worrying
[09:31] <wgrant> Bug #593522
[09:31] <_mup_> Bug #593522: Don't send out real email from staging codehosting <Launchpad Bazaar Integration:Fix Released by canonical-losas> <https://launchpad.net/bugs/593522>
[09:32] <ajmitch> Now if only you could get it to fix bugs as well
[09:33] <kb9vqf> Well it already appears to have an IQ higher than most Internet users...it speaks in complete sentences ;-)
[09:33] <spm> that's coming version 2. version 3 is when it starts patching itself. live. version 4 is when it invents a new killer language + api's; rewrites itself into that. declares the rest of the world as backwards and hence obsolete; and turns the entire planet into a giant supercomputer. something pity about 42 in there as well I'm sure.
[09:34] <ajmitch> spm: great, when can it write tests for LP code as well?
[09:34] <spm> s/pity/pithy/
[09:34] <spm> ajmitch: that'd be version 8 based on the above timeline. simple extrapolation.
[09:35]  * ajmitch has been slacking just a little bit & has finally got back to bug 146389, apart from the little issue of writing tests
[09:35] <_mup_> Bug #146389: api for blueprint tracker <api> <feature> <Launchpad Blueprints:In Progress by ajmitch> <https://launchpad.net/bugs/146389>
[09:35] <wgrant> It's much less painful with launchpadlib.
[09:35] <thumper> ajmitch: oh dear
[09:36] <thumper> ajmitch: I started looking into that
[09:36] <ajmitch> yeah, given that I've been testing manually with it as well
[09:36] <thumper> ajmitch: and ran screaming
[09:36] <ajmitch> thumper: great to know!
[09:36] <ajmitch> how far did you get?
[09:36] <thumper> ajmitch: as I felt that it needed a machette
[09:36] <ajmitch> it does, really
[09:36] <thumper> ajmitch: too much weird shit logic in the wrong place
[09:36] <thumper> ajmitch: the model really needs an overhaul
[09:36] <wgrant> Well.
[09:36] <wgrant> It's Blueprint.
[09:36] <thumper> wgrant: agreed
[09:36] <wgrant> It needs a machette. To the whole app.
[09:36] <ajmitch> but I've just been exporting properties & methods, going WTF in a few places
[09:37] <thumper> wgrant: with wikis, I'll be better ;-)
[09:37] <thumper> maybe
[09:37] <ajmitch> wondering how I make an exported field editable, all that fun stuff
[09:37]  * wgrant throws stuff at buildd-manager.
[09:37] <thumper> ajmitch: there is special logic in most of the other setter methods
[09:38] <thumper> ajmitch: which is why I didn't go ahead and make the methods writable
[09:38]  * ajmitch just made a big, big mistake
[09:38] <ajmitch> running 'bzr viz' on LP
[09:38] <thumper> ajmitch: recording things like started date, and who did it
[09:38] <thumper> finished date et al.
[09:38] <wgrant> ajmitch: qlog works OK.
[09:39] <ajmitch> thumper: one thing that the distro team appears to use, is splitting a single blueprint into multiple work items using the whiteboard
[09:39] <ajmitch> ideally that'd be supported better than just a big text mess
[09:39] <ajmitch> if someone will give some love to blueprints :)
[09:39] <thumper> ajmitch: I'll mentor you :)
[09:40] <ajmitch> shared drinking sessions for blueprint code? sounds like a plan
[09:42]  * ajmitch thinks that it'll need a lot more thought put into the design
[09:43] <wgrant> Or a good rm -r lib/lp/blueprints
[09:43] <wgrant> Why are they separate?
[09:44] <ajmitch> separate from bugs?
[09:44] <wgrant> Yes.
[09:45] <ajmitch> I think because they were at least meant to have features that didn't really map well to the bug workflow
[09:46] <ajmitch> some things like the dependencies between blueprints would be nice to have in bugs
[09:46] <wgrant> Exactly.
[09:46] <wgrant> Most features unique to blueprints would be useful in bugs too.
[09:47] <wgrant> Oh.
[09:47] <wgrant> I hate you Twisted.
[09:47] <wgrant> Why must you hang when attempting to serialise an XML-RPC request containing None...
[10:16] <wgrant> mthaddon: So we needn't worry that the DC is going to come and conquer the world?
[10:17] <mthaddon> wgrant: the machines are on the march
[10:27] <wgrant> kb9vqf: Once I set up lucilleconfig on Debian and its series and configured nominatedarchindep on lenny, and hacked traverse_named_ppa to not care about the distribution, it all works fine.
[12:27] <wgrant> bigjools: You don't object strongly to https://code.edge.launchpad.net/~wgrant/launchpad/bug-592935-hide-disabled-ppas/+merge/27411  or https://code.edge.launchpad.net/~wgrant/launchpad/no-buildd-ogre-model/+merge/27410 , do you?
[12:29]  * bigjools OTP, gimme some 
[12:30] <wgrant> k
[12:52] <bigjools> wgrant: +1 to both
[12:52] <bigjools> provided that the user can still see his own disabled PPAs
[12:52] <wgrant> They can.
[12:52] <wgrant> Thanks.
[12:52] <bigjools> we need a better solution for that in the long term, but it'll do for now
[12:52] <bigjools> thanks for the change
[12:52] <wgrant> Yep.
[13:11] <wgrant> bigjools: Is the log parser CP blocking on that dogfood issue?
[14:20] <bigjools> wgrant: it's blocking because it doesn't work
[14:23] <didrocks> jml: hey, to implement https://blueprints.launchpad.net/ubuntu/+spec/desktop-maverick-quickly as discussed at UDS (for pushing ssh/gpg key, signing CoC and creating a ppa), I will try to connect "the bad way" to launchpad :/ Is there any way to make it less hackish than screenscraping and avoiding breaking the user?
[14:23] <wgrant> bigjools: It doesn't?
[14:23] <jml> didrocks, I'm on the phone.
[14:23] <didrocks> jml: no pb, will idle there :)
[14:23] <bigjools> wgrant: no, I removed all existing db entries and re-ran it.  Same issue.
[14:23] <bigjools> so there's a definite code problem
[14:23] <wgrant> Ah, I didn't know that bit.
[14:29] <wgrant> I wonder why it doesn't use fd.tell() instead of working it out manually.
[14:33] <wgrant> bigjools: Any chance you can stick something like a "assert parsed_bytes == fd.tell()" after "parsed_bytes += len(line)", delete the ParsedApacheLog for that file, then rerun?
[14:34] <wgrant> Would be nice to see where it actually starts going wrong...
[14:34] <bigjools> wgrant: not in the near future, I'm way too busy
[14:34] <wgrant> OK.
[14:34] <bigjools> sorry
[14:34] <wgrant> No worries.
[14:44] <krkhan> is there a way for me to debug doc tests quickly? while writing unittests i did pdb.set_trace() inside one of the tests and then played around. for doc tests, pdb.set_trace() loses all context
[14:55] <bigjools> krkhan: go up one in the stack and the context is there
[14:55] <bigjools> so hit "u" after it breaks
[14:57] <krkhan> bigjools: thanks :-D
[14:57] <bigjools> it used to stop in the right stack frame, I wonder what changed :(
[16:28] <didrocks> jml: still on the phone? :)
[16:56] <jml> didrocks, no, doing performance reviews. :)
[16:56] <jml> didrocks, I have to confess I forgot about your question
[16:56] <didrocks> jml: just ping me when you will some free time, ok? ;)
[16:56] <jml> didrocks, sure.
[16:57] <jml> didrocks, actually, looking at your question, can you please just ask it on the launchpad-dev@lists.lp.net mailing list?
[16:57] <jml> didrocks, I'm really not the best person to ask.
[16:57] <didrocks> jml: sure, doing it now. Thanks!
[16:58] <didrocks> jml: did you book for RMLL, btw?
[16:58] <jml> maybe
[16:58] <jml> I'm pretty sure I'm speaking there.
[16:59] <didrocks> right, on wednesday IIRC :)
[16:59] <jml> I'm dealing with all of that stuff tomorrow.
[16:59] <didrocks> good luck for your reviewing time :)
[17:03] <krkhan> in a doc test, login(..); team = factory.makeTeam(); response = webservice.get("/~"+team.name); returns a http response with 500 internal server error. any ideas on what i'm doing wrong?
[17:10] <jelmer> krkhan: you might have to commit() the change
[17:19] <krkhan> jelmer: login(...); team=factory.makeTeam(); transaction.commit(); response = webservice.get(...); still returns 500 internal server rror
[17:20] <jelmer> krkhan: with what error message in the 500 internal server error?
[17:21] <krkhan> jelmer: empty body, but with ('x-lazr-oopsid', 'OOPS-1627T162') in header
[17:31] <allenap> sinzui: Are milestone listing in memcached now? How long for? Is there any way to force a refresh?
[17:32] <sinzui> allenap, 10 minutes active milestones, 3 hours inactive
[17:32] <sinzui> the status are 1 hour
[17:32] <allenap> sinzui: bug status?
[17:33] <sinzui> allenap, bug listing are 10 minutes in active milestones, and 3 hours inactive milestones
[17:35] <allenap> sinzui: You said "the status are 1 hour". I didn't understand what that meant.
[17:36] <sinzui> The status and assignment summary above the listings caches for 1 hour
[17:36] <sinzui> see blog.launchpad.net
[17:37] <krkhan> found the fix. had to call logout() before using webservice
[18:01] <mrevell> Night all
[21:35] <krkhan> how should i encode get queries which contain newline characters? "var=line1%0Aline2" gives a HTTP 400 Bad Request response
[21:38] <lifeless> krkhan: what variables are you using that you want newlines in ?
[21:39] <lifeless> I suspect that we'd be looking for POST form encoding for mutate operations
[22:04] <krkhan> lifeless: for a text search. i was using restclient before but using urllib from python works fine
[22:05] <lifeless> krkhan: interesting
[22:05] <lifeless> I'm fairly sure that \r and \n will be ignored by the search engine. IMBW.
[22:07] <krkhan> lifeless: i was trying to implement my own search method for bug attachments. anyways, python's urllib works fine as far as request is concerned (it doesn't give http 400) but on the server side i still only see the part before the linebreak
[22:08] <krkhan> ws.op=abc&text=line1%0Aline2 results in text=line1 at the server side
[22:08] <lifeless> please file a bug on that
[22:08] <lifeless> it suggests a possible security hole
[22:10] <krkhan> lifeless: but i can't be sure this is a launchpad issue. i mean, perhaps get requests aren't supposed to have newlines? (IMBW as well)
[22:10] <lifeless> krkhan: either way ;)
[22:10] <lifeless> krkhan: there is a bug, and we should get to the bottom of it
[22:11] <krkhan> lifeless: okay, doing that now. btw, i'm curious as to how this could be a possible security hole
[22:12] <lifeless> well
[22:12] <lifeless> imagine that line2 isn't actually lost
[22:12] <lifeless> what is it doing? does a subsequent request get it? does it enter via an unknown code path?
[22:13] <lifeless> you could use it to craft a link to launchpad that wouldn't do what the user things it would, for instance.
[22:13] <lifeless> I'm probably not devious enough, buts it that sort of inconsistent behaviour between a client and a server that leads to many vulnerabilities
[22:13] <krkhan> ah. i see. i'm a little unsure as to how to present the issue in the bug report. i'm using my own method here and inserting a pdb.set_trace() at the server side to see that the text isn't received properly
[22:14] <lifeless> document what you know
[22:14] <krkhan> okay
[22:14] <lifeless> thats all that one can ask
[22:29] <krkhan> lifeless: i think it's an issue in httplib2 :) i just tried using launchpadlib to send newline characters and they reached fine
[22:29] <krkhan> so the problem was only with httplib2 and restclient
[22:31] <lifeless> ok
[22:31] <lifeless> thanks for digging
[22:37] <krkhan> np
[23:05] <sinzui> abentley, I think https://lpbuildbot.canonical.com/builders/lp/builds/982/steps/shell_7/logs/summary shows that Warty\nHoary\nSecret Squirrel does not match, but I cannot say the test is broken
[23:12] <abentley> sinzui, I think the problem is there's no defined order for the distroseries.
[23:13] <sinzui> ah
[23:34] <wgrant> sinzui, abentley: I have an approved branch which fixes that.
[23:34] <wgrant> (sorting them by version)
[23:38] <wgrant> Also, neither of the two ec2tests that were running last night bothered to email me.
[23:38] <wgrant> Although one submitted to PQM successfully.