[00:00] <lifeless> yes
[00:00] <thumper> can someone else try authorizing a new launchapdlib token?
[00:00] <thumper> I got a client side and server side blowup
[00:00] <lifeless> win
[00:00] <lifeless> OOPS in the response?
[00:01] <wgrant> thumper: Works for me.
[00:01] <wgrant> On Maverick.
[00:01] <thumper> lifeless: https://pastebin.canonical.com/36957/
[00:01] <thumper> lifeless: OOPS-1713EB2534 on server
[00:02] <lifeless> thats...surprising
[00:02] <lifeless> its also surprising to get the backend failed message
[00:03] <wgrant> What exploded this time?
[00:03] <lifeless> orry, there was a problem connecting to the Launchpad server.
[00:03] <wgrant> That's... not an oops, though.
[00:04] <lifeless> I know
[00:04] <lifeless>   Module canonical.launchpad.webapp.publisher, line 297, in __call__
[00:04] <lifeless>     self.initialize()
[00:04] <lifeless>   Module canonical.launchpad.browser.oauth, line 259, in initialize
[00:04] <lifeless>     assert self.token.is_reviewed, (
[00:04] <lifeless> AttributeError: 'NoneType' object has no attribute 'is_reviewed'
[00:05] <wgrant> Special.
[00:05] <lifeless> at least its a fast page
[00:05] <lifeless> 207ms
[00:05] <wgrant> Heh.
[00:05] <lifeless> LLLLEEEEOOOOONNNNNAAAAAARRRRRRDDDDDD
[00:05] <lifeless> thumper: I'll file a bug for this
[00:06]  * maxb wonders what an earth has happened to the kdegames import, which was "Successful", but "This branch has not been imported yet"
[00:06] <maxb> https://code.edge.launchpad.net/~neon/kdegames/trunk
[00:06] <thumper> maxb: not scanned I guess
[00:06] <wgrant> The scanner is broken at the moment, isn't it?
[00:06] <thumper> not fully broken
[00:07] <thumper> only to those that are linked to bugs with distribution source package tasks
[00:07] <lifeless> thumper: the token authorized ok
[00:07] <lifeless> thumper: I think
[00:07] <thumper> lifeless: well the client went on its merry way
[00:07] <lifeless> yeah
[00:07] <lifeless> but the 'show it to the user' page went boom
[00:08] <maxb> If the branch was scheduled for scanner attention, ought it not show they pale yellow "Updating" box?
[00:09] <lifeless> maxb: we've had a scanner malf
[00:09] <lifeless> maxb: may be implicated
[00:09]  * mwhudson afk for a bit
[00:32] <rockstar> thumper, shall we skype about some UI?
[00:36] <thumper> rockstar: still chasing critical issue
[00:37] <rockstar> thumper, okay.
[00:39] <rockstar> thumper, maybe some annotated mockups?
[00:40] <thumper> rockstar: lets talk anyway
[00:40] <rockstar> thumper, okay.
[03:43] <MTecknology> Why is it that Configuration Progress is never complete until you actually enable everything? You can't just say, I don't want to use this.
[03:43] <thumper> MTecknology: you should be able to
[03:43] <thumper> MTecknology: if it doesn't work that way it is a bug
[03:46] <lifeless> thumper: you can't for code hosting yet, I don't think.
[03:47] <MTecknology> oooooh! You can now!
[03:47] <MTecknology> I lied and ignore me :P
[03:47] <MTecknology> lifeless: it seems you can :)
[04:03] <wgrant> MTecknology: Yeah, as of yesterday it should be OK.
[04:20] <wgrant> +configure-translations isn't exactly... intuitive.
[04:20] <wgrant> Is it finished?
[04:21] <MTecknology> kiko: hey, how ya been?
[04:24] <MTecknology> I hate it when I should know exactly where something in LP is... but I can't remember. This is gonna bug me. :P
[04:25] <wgrant> Things do move occasionally.
[04:25] <wgrant> What've you lost?
[04:26] <MTecknology> add a download file
[04:26] <wgrant> You need to do it from within a release.
[04:27] <MTecknology> and that's the part I forgot..
[04:27] <MTecknology> thanks :)
[04:37] <wgrant> lifeless: This new request timeline stuff is pretty awesome.
[04:47] <michaelh1> Hi there.  I have a web app that uses launchpadlib to create tickets.  How can I set the reporter to an arbitrary user?
[04:48] <michaelh1> (on creation that is)
[04:48] <wgrant> The API doesn't support impersonation.
[04:48] <wgrant> It may eventually for some trusted clients.
[04:49] <michaelh1> OK.  I want the reporter to be the same as the person logged into my web app.  The web app is authenticated using OpenID against LP.
[04:49] <michaelh1> ...and there's only a few users.
[04:49] <wgrant> Ah. In that case you should probably get an OAuth token on behalf of the user.
[04:50] <michaelh1> What's a clean way of doing that?  Worst case is I give them a small app that runs on their machine and get them to email me the credentials file...
[04:51] <wgrant> Well, I'm not sure if there's a great web-based workflow yet -- I haven't done it myself.
[04:51] <wgrant> But when you tell launchpadlib to request a token, it gives you an authorisation URL. The user has to visit that.
[04:54] <michaelh1> Does the 'change non-private data' permission allow me to create and modify tickets?
[04:55] <wgrant> If they're public bugs, yes.
[04:55] <wgrant> Private bugs require the Change Private Data permission.
[04:55] <michaelh1> Yip, these would be new public bugs.  What is a private bug?
[04:56] <wgrant> Bugs start private if their project is configured that way, or if they're marked as security related. You can toggle privacy with the 'This bug is public' box in top right of the bug page.
[04:56] <wgrant> Only subscribers can see private bugs.
[05:00] <michaelh1> Thanks for your help.  We only have a couple of bug reporters so the OAuth hack-around will be fine.
[05:00] <MTecknology> wgrant: for default private bugs, it needs to be either security related or have the proprietary thing, right?
[05:01] <wgrant> MTecknology: I believe so.
[05:03] <lifeless> wgrant: thanks
[05:03] <lifeless> wgrant: what prompted that ?
[05:04] <wgrant> lifeless: Just looking at OOPSes locally and seeing all the memcache etc. calls accounted for.
[05:05] <wgrant> Fewer unexplained holes in time is good.
[05:09] <lifeless> yeah
[05:10] <lifeless> also I think sinzui was able to debug a memcache issue more quickly, via it
[05:12] <wgrant> It looks like it.
[05:12]  * StevenK tries to find how one uses lplib via the test suite
[05:12] <wgrant> It makes it a lot less opaque.
[05:14] <MTecknology> I think you guys should switch to Nginx :D
[05:15]  * MTecknology wonders how hard it would be to do that
[05:15] <wgrant> StevenK: Grep around for launchpadlib_for.
[05:16] <wgrant> test_archive_webservice looks reasonable.
[05:18] <wgrant> I should check how that performs.
[05:18] <wgrant> Since it would be nice to use it everywhere.
[05:19] <mwhudson> it's pretty slow i think
[05:19] <wgrant> :(
[05:19] <mwhudson> but don't take my word for it
[05:19] <wgrant> We should be able to cache the processed WADL or something.
[05:20] <thumper> mwhudson: I'm confused
[05:20] <thumper> mwhudson: take a look at https://code.edge.launchpad.net/~neon/kdegames/trunk
[05:21] <mwhudson> i remember debugging some weird interaction with launchpadlib_for that made me very angry
[05:21] <mwhudson> thumper: it *is* friday, maybe a beer will help?
[05:21] <thumper> mwhudson: I'm stopping soon to drink a bottle of wine
[05:21] <thumper> probably pinot noir
[05:21] <thumper> maybe two
[05:21] <thumper> it has been one of those days
[05:22] <mwhudson> thumper: branch puller is stuck?
[05:23] <MTecknology> thumper: all I have for wine - two cheap brands I mixed together
[05:23] <lifeless> wgrant: have you checked prod & edge?
[05:23] <lifeless> wgrant: for tracebacks?
[05:23] <thumper> mwhudson: no it isn't stuck
[05:23] <thumper> mwhudson: it hasn't been pulled
[05:23] <thumper> mwhudson: I've checked the puller logs
[05:23] <thumper> mwhudson: and... there have been no scan jobs
[05:24] <mwhudson> thumper: maaaybe the requestMirror call does to the internal xmlrpc server oopsed?
[05:24] <lifeless> MTecknology: probably not hard; very uninteresting.
[05:24] <lifeless> MTecknology: what would we gain?
[05:24] <mwhudson> thumper: you could try calling requestMirror over the api and seeing if that kicks it into life?
[05:24] <thumper> mwhudson: yeah, I'll try that
[05:24] <thumper> mwhudson: maybe it was a timeout to finishJob ?
[05:25] <MTecknology> lifeless: nginx is super awesome... and for static content kicks apache's ass - I don't know how many of your requests are for static content though
[05:25] <mwhudson> thumper: oh right, yeah
[05:25] <mwhudson> thumper: grep in the arsenic oops dir?
[05:26] <mwhudson> thumper: would be odd to happen to the same branch repeatedly though
[05:27] <thumper> it would
[05:27] <thumper> mirror requested
[05:27]  * mwhudson afk for a sec
[05:28] <lifeless> MTecknology: super awesome really doesn't mean anything to me.
[05:28] <wgrant> lifeless: Yes.
[05:28] <lifeless> we have very few static files - css, the iconset and shortly the wadl. Oh, and the librarian which is only kindof static (its static but with complex url mapping & dynamic mime types and access control
[05:28] <lifeless> wgrant: thanks
[05:29] <lifeless> MTecknology: I have no interest in change for changes sake
[05:29] <wgrant> lifeless: I've only seem them on production for a few minutes a few years ago. They were on beta for a while (when it was called beta), and on dogfood until just a couple of years ago.
[05:30] <mwhudson> thumper: the branch seems to have been pulled now
[05:30] <MTecknology> lifeless: With Apache if you have say mod_php or mod_python enabled, every single request forks a new instance that can handle those requests. Nginx only passes the request to that if it's needed. Then you can have it try to server static content that needs no additional processing which makes it extremely fast, and light. Then you could keep using Apache if preferred for those heavier requests.
[05:31] <mwhudson> MTecknology: the chance of a server that's not in ubuntu main being deployed is .... small
[05:31] <wgrant> MTecknology: Launchpad just uses Apache to serve a few static files and proxy to the Zope appserver.
[05:31] <wgrant> Hm, nginx really still isn't in main?
[05:31] <wgrant> odd.
[05:31] <lifeless> MTecknology: mod_wsgi++, and php -argh.
[05:32] <lifeless> MTecknology: also the prefork worker doesn't fork on every request
[05:32] <mwhudson> thumper: come to think of it, it can't have been finishJob oopsing, because that's what records the success
[05:32] <lifeless> MTecknology: I know that nginx has a lot of cred going around, but someone needs to map it to a /problem/ or /benefit/ that is relevant to us, before anyone is going to put in the time (non trivial) to change.
[05:33] <thumper> mwhudson: hmm...
[05:34] <thumper> mwhudson: WTF could it be then?
[05:34] <MTecknology> lifeless: I guess if you proxy most with wsgi then it would be an insanely easy deployment. It's probably come down to just the ability to serve static content much faster. and in this case maybe the benefit wouldn't outweigh the hassle
[05:34] <MTecknology> Either way I think I should pull the branch and try out some benchmarking.. Could be fun..
[05:34] <lifeless> MTecknology: I think we're using proxypass for everything in fact.
[05:34] <mwhudson> thumper: don't know
[05:34] <thumper> :(
[05:34] <MTecknology> lifeless: even for static?
[05:34] <thumper> and Friday afternoon isn't the time to go looking for it
[05:35] <mwhudson> thumper: it's not in the puller logs at all?
[05:35] <lifeless> MTecknology: benchmarking would be essential. As mwhudson says, being in main is very important too (otherwise the lp team has to maintain the package)
[05:35] <lifeless> MTecknology: for -anything-
[05:35] <thumper> mwhudson: nope
[05:35] <thumper> requestMirror worked
[05:35] <MTecknology> .. so I definitely should peek at that..
[05:35] <thumper> now pulled and scanned
[05:35] <MTecknology> I would like to know why nginx isn't in main yet though :P
[05:35] <mwhudson> thumper: i guess you could try to find the relevant worker logs from pear
[05:36] <MTecknology> I should go find that out too..
[05:36] <thumper> mwhudson: http://launchpadlibrarian.net/55296521/neon-kdegames-trunk.log
[05:36] <MTecknology> lifeless: btw- thanks for giving an honest discussion to it. :)
[05:37] <thumper> mwhudson: earlier one http://launchpadlibrarian.net/55292524/neon-kdegames-trunk.log
[05:37] <mwhudson> thumper: that's the log from the worker -- i meant the log from the worker monitor i guess
[05:37] <thumper> mwhudson: ah...
[05:37] <mwhudson> they don't end up in the librarian, but are kept on pear
[05:38] <mwhudson> dunno if they're synced to devpad, likely not
[05:39] <mwhudson> spm: grep kdegames  in /srv/importd.launchpad.net/production-logs ?
[05:39] <spm> mwhudson: they're on sodium, or should be
[05:40] <mwhudson> i guess i should find out why i can't log in to devpad any more
[05:40] <spm> mwhudson: /x/launchpad.net-logs/production/neumayer
[05:40] <spm> haha, one sec then
[05:40] <mwhudson> thumper can do it...
[05:40] <mwhudson> :)
[05:40] <spm> *snort*
[05:40] <thumper> do what?
[05:41] <mwhudson> thumper: grep kdegames -r /x/launchpad.net-logs/production/pear
[05:41] <mwhudson> on devpad
[05:41]  * thumper is there
[05:41] <thumper> holy crap there are a lot of files in there
[05:42] <spm> that's the "cleaned up" version. should have seen what it was like before.
[05:43] <StevenK> mwhudson: Probably due to your team move
[05:44] <MTecknology> lifeless: so.. Nginx in main, Patch for configs to work perfect, Benchmarks, A good reason to switch, anything else?
[05:44] <mwhudson> StevenK: yeah
[05:45] <lifeless> MTecknology: I think thats it; elmo has previously discussed this in a more pithy form "hell no" :- but if there is a compelling story I don't see why we wouldn't.
[05:45] <lifeless> However, I don't *expect* a compelling story, based on what I know of our problems (today)
[05:45] <MTecknology> Are most of the issues DB side?
[05:46] <thumper> mwhudson: I can't see anything, but I'm feeling like I'm not getting very far
[05:46] <lifeless> MTecknology: I wouldn't like you to spend a lot of time and end up empty handed is all I'm saying
[05:46]  * thumper halts
[05:46] <mwhudson> thumper: :/
[05:47] <lifeless> MTecknology: DB issues are the proximate cause of a good 50% of the perf issues I look at; another 40+ are most closely identified as python - single threadedness, time-to-do-stuff
[05:47] <MTecknology> lifeless: ya um... I tried to add a patch to the php package which added a whole new feature and I fought with it for about a month... The hardest part about this would be performing accurate benchmarking :P
[05:47] <lifeless> another 10% are things like sending mail (200ms per mail)
[05:47] <lifeless> we serve 2.6Million hits a day from the backend servers
[05:48] <MTecknology> really?.. I sould thought it'd be more
[05:48] <lifeless> some larger count from the squid farm (anonymous content only) & apache handles /all/ of that with nearly no load and no maintenance or uptime issues.
[05:48] <MTecknology> Not to say 2.6mil is low.. I just woulda thought it'd be higher
[05:48] <lifeless> let me check
[05:49] <lifeless> 2838269 from lpnet
[05:49] <lifeless> thats excluding:
[05:49] <lifeless>  - librarian
[05:49] <lifeless>  - bazaar.*
[05:49] <lifeless>  - edge
[05:49] <MTecknology> oh.. that sounds more like what my guess would been then.. My guess was ~10mil
[05:49] <lifeless> 99% of our requests, by count, complete in 3.2 seconds
[05:50] <lifeless> edge was only 66K yesterday
[05:50] <MTecknology> oh- with nginx you can upgrade it in place, so you update the package and never lose any request. I don't think apache can do that - but I suppose you have plenty of redundancy too
[05:50] <lifeless> OTOH we had a big chunk of downtime in place
[05:50] <lifeless> MTecknology: yeah, we have L4 proxies in place already
[05:50] <MTecknology> Not many people use edge?
[05:51] <MTecknology> Now I really feel edgy :P
[05:51] <lifeless> on a non-release day : 1681716
[05:51] <lifeless> 1.6M
[05:52] <lifeless> anyhow, we're going to get rid of edge real-soon now
[05:52] <MTecknology> keep everyone looking at the same thing?
[05:53] <lifeless> simplify deployment
[05:53] <lifeless> get fixes to all users more quickly.
[05:53] <MTecknology> oh
[05:54] <MTecknology> alrighty - so I think I got my curiousity satisfied for the day :)
[05:54] <MTecknology> I have one other question.. what is ~buy-something for?
[05:55] <wgrant> The Software Center purchasing feature.
[05:55] <wgrant> Which is very nearly there, I believe.
[05:55] <lifeless> no idea but I'd guess its the agent account for the SCA
[05:55] <MTecknology> oh
[05:55] <lifeless> which is how folk buying debs from ppas via software centre will have their access requests relayed into LP
[05:56] <MTecknology> elachuni seems to be doing a crap load of work
[05:56] <MTecknology> Been a while since I talked to him or metcalfe - they seem to be locked in closets over there..
[05:59] <MTecknology> Aight, it's beddy bye time - I'll ttyal
[05:59] <MTecknology> lifeless: thanks for that chat; wgrant: thanks for the help
[05:59] <wgrant> Night.
[06:09] <lifeless> gnight
[08:39] <lifeless> anyone around can do a small review ?
[08:44] <adeuring> good moring
[09:04] <mrevell> Morning
[09:09] <lifeless> hi mrevell
[09:09] <lifeless> mrevell: btw, it occured to me you might want to do a team-joining interview with me - or not, I'm easy.
[09:12] <mrevell> lifeless, Oh, I thought I'd sent you one already.
[09:12]  * mrevell goes to rifle through the Sent folder.
[09:44] <lifeless> mrevell: heh
[09:52] <wgrant> lifeless: Did you get a response from -security about the restricted librarian thing?
[09:59] <lifeless> yes
[09:59] <lifeless> +1, all things considered
[09:59] <lifeless> we have the SSL certs
[10:00] <lifeless> its with losa to be configured up on staging and lpnet
[10:00] <lifeless> once thats all done we'll arrange to set the feature flag on staging and QA it all
[10:01] <stub> And me... I need to sort the session db!
[10:01] <stub> When are we looking at switching it on?
[10:01] <lifeless> stub: Monday?
[10:01] <stub> k. I'll do it after a coffee.
[10:05] <wgrant> Excellent.
[10:06] <wgrant> Has the domain restriction landed?
[10:06] <wgrant> I forget.
[10:07] <lifeless> wgrant: no, its in the rt for apache to do.
[10:08] <wgrant> Ah.
[10:13] <kiko> MTecknology, pretty good, just fell off the net :)
[12:00] <deryck> Morning, all
[12:49] <jtv> Reason #311 for not using ReST in doctests: bzr conflicts are hard enough to read with all the >>> that I really don't need any more [12:57] <nigelb> heh ;)
[14:00] <MTecknology> kiko: You did... and it made me sad. :(
[14:01] <kiko> MTecknology, it's all good now!
[14:01] <MTecknology> Yay! :)
[15:17] <cr3> what's the difference between the "lp" icon (see blueprint project) and the "pad" icon (see launchpad-project project)?
[16:04] <deryck> rockstar, ping
[16:25] <rockstar> deryck, pong
[16:26] <deryck> rockstar, yo, dude, I have progress on the widget mess.
[16:26] <rockstar> deryck, really?
[16:26] <rockstar> deryck, skype then?
[16:27] <deryck> rockstar, yup.  And it's dumb but simple.  http://pastebin.ubuntu.com/491643/
[16:27] <deryck> yeah, we can skype
[16:27]  * deryck looks for headphones
[16:27] <rockstar> deryck, are you freakin' kidding me?
[16:28] <deryck> nope
[16:28] <deryck> rockstar, do you want to skype, or that un blocks you now?
[16:29] <rockstar> deryck, that makes the events fire?
[16:29] <deryck> rockstar, let's skype to do high bandwidth.
[16:29] <rockstar> deryck, skype doesn't think you're around.
[16:30] <deryck> cause I am only now :-)
[16:30] <deryck> ring me when you want, rockstar
[16:47] <deryck> rockstar, guess I lost you.  I'm done though, if you are.
[16:47] <rockstar> deryck, I think I lost you.
[16:47] <rockstar> Yup.
[16:47] <rockstar> deryck, we were finishing up anyway.  I was just saying that I'm going to consider my wizard work done (for now) when I have two steps.
[16:48] <deryck> rockstar, righ
[16:48] <rockstar> deryck, okay, cool.
[16:48] <deryck> rockstar, so feel free to call on me or gmb if you need help next week.  Or me later today even.
[16:48] <deryck> gmb needs to wizard widget to move forward, obviously.
[18:20] <EdwinGrubbs> matsubara-lunch: ping
[18:24] <matsubara> hi Edwin-afk
[18:24] <matsubara> hmm
[18:24] <matsubara> hi EdwinGrubbs
[18:26] <EdwinGrubbs> matsubara: I just wrote a big reply to robert's comments on oops-tools, but I wanted to see if you could answer what the status is on open sourcing it. gary had implied that you would know more.
[18:27] <matsubara> EdwinGrubbs, I'll reply. We had a conversation about it during the foundations stand up call today.
[18:28] <EdwinGrubbs> matsubara: also, what email lists should I forward the email to in order to get ISD's and U1's input? I'm not sure if I'm even subscribed to their lists, though.
[18:33] <matsubara> EdwinGrubbs, good question. I'm not sure either as I'm not subscribed to their lists. I usually talk to specific people on those teams (U1: Roman Yepishev <rye>, ISD: Anthony Lenton <achuni>)
[21:17] <lifeless> EdwinGrubbs: I've replied ;)