#launchpad-dev 2009-07-20
<wgrant> What happened to 2.2.8?
<Ursinha> wgrant, why?
<wgrant> Ursinha: I see a whole lot of stuff deferred to 2.2.9, but 2.2.8 doesn't exist. There's also no calendar past 2.2.7.
<jml> wgrant, hasn't an email gone out about that?
<wgrant> jml: Not that I've seen.
<jml> ok. I'll hassle some folk.
<wgrant> Nor has there been anything about the open sourcing, which looks to be happening soon given the RC license fixes...
<noodles775> sinzui: can I assign a registry-related question to you?
<noodles775> https://answers.edge.launchpad.net/launchpad/+question/77463
<sinzui> yes
<sinzui> but I think that question is about bugs
<noodles775> Hmm... ok. I'll check with intellectronica
<noodles775> Thanks sinzui
<sinzui> noodles775: the usual rules is this: if the <bug|code|soyuz> was turned off, would the registry still answer the question
<sinzui> noodles775: I think we may want something foundation in the mailnotification that does a sanity check that the user is ACTIVE.
<noodles775> sinzui: yes, that was why I initially thought of registry.
 * sinzui wants that feature there because he can add hooks for mutted and vacation settings
<noodles775> nice!
#launchpad-dev 2009-07-21
<beuno> wgrant, https://code.edge.launchpad.net/launchpad
<flacoste> mwhudson: is it possible that your branch is stuck in PQM?
<lifeless> https://dev.launchpad.net/Getting
<lifeless> wgrant: ^
<mwhudson> flacoste: it's possible, i guess spm is the man who'd know
 * thumper stretches in his new home
<jml> smell that?
<jml> smells like *freenode*
<thumper> heh
<rockstar> :)
<flacoste> good night guys!
<thumper> mwhudson: does loggerhead cause a test run?
<mwhudson> thumper: no
<thumper> good
<mwhudson> it probably causes a make build though
<thumper> right
<thumper> all this excitement and wgrant isn't around :)
<beuno> we should close it again
<thumper> beuno: heh
<rockstar> Oh man, that's terrible...  :)
<jml> haha
<thumper> beuno: your branch listing page is now the top of my stack
<beuno> thumper, awesomeness
<thumper> spm: is pqm actually doing anything?
<spm> thumper: yes. ssh'ing
<spm> about 7 mins ago
<thumper> hi BjornT
<spm> thumper: I had a stop patch in when I did tghe manual merge for flacoste earlier. forgot to remove it. my bad
<thumper> spm: ta
<BjornT> hi thumper
<thumper> BjornT: welcome to the brave new, public, world
<jml> hey guys :)
<deryck> Glad I was up late tonight for the big moment. :)
<rockstar> deryck, :)
<rockstar> deryck, you're CDT, right?
<deryck> rockstar, yup.
<LaserJock> congrats LP devs!
<thumper> LaserJock: thanks
<jml> LaserJock, thanks :)
<BjornT> deryck: i take it you won't be start working in 5 hours, like usual? :)
<deryck> BjornT, no, I will actually. :)
<mwhudson> my loggerhead branch landed at last
<deryck> BjornT, there's always time for an afternoon nap, after work. :)
<LaserJock> it wasn't clear from the announcement, is *all* of LP getting opened or is Soyuz (and Code hosting?) being held back?
<jml> it's all there.
<lifeless> LaserJock: have a look at the code :P
<jml> LaserJock, wysiwyg :)
<jtv> flacoste_afk: any chance of getting this crucial fix into the rollout? https://code.edge.launchpad.net/~jtv/launchpad/bug-401835/+merge/9074
<jtv> flacoste_afk: very last-minute I know, but QA has been a huge bottleneck since May.
<LaserJock> lifeless: well, https://dev.launchpad.net/Getting said it can take a couple hours to get the code, I was hoping for a shortcut :-)
<mwhudson> LaserJock: it's the lot
<beuno> LaserJock, http://bazaar.launchpad.net/~launchpad-pqm/launchpad/devel/changes
<rockstar> beuno, as if the servers weren't already going to take a beating, you link to loggerhead and watch it melt too.  :)
<jml> https://bazaar.launchpad.net/~launchpad-pqm/launchpad/db-devel/files/head%3A/lib/lp/
<jml> even better URL
<beuno> :)
<beuno> stress test!
 * beuno is glad he worked on nice URLs for LH
<radix> grats everyone :)
<jml> radix, thanks :)
<thumper> hi Hobbsee
<Hobbsee> hey thumper!
<Hobbsee> Congratulations all!
<spm> hey Hobbsee, early birthday pressie for you :-)
<beuno> :)
<jml> hi :)
<Hobbsee> spm: indeed!
<LaserJock> wow, just wow
<lifeless> Hobbsee: we look forward to your patches ;)
<ScottK> Congratulations everyone.
<jml> wuuu https://code.edge.launchpad.net/~jml/launchpad/branch-flow-utility
<LaserJock> I can't believe this day has come :-)
<jml> my first public Launchpad branch
<LaserJock> soo much code
<jml> LaserJock, yes. yes there is.
<beuno> LaserJock, and so much history  ;)
 * mwhudson is very tempted to start drinking
<thumper> mwhudson: me too
<thumper> mwhudson: and since I'm going to be cooking dinner in about 15 minutes, I may well just pour a nice glass of pinot noir
<beuno> it's 2am here, anything goes
<spm> thumper: mwhudson: me three, but um, someone needs to make sure servers don't melt :-)
<rockstar> mwhudson, _I_ am tempted to start drinking.  :)
<mwhudson> thumper: problem is, we really need to go shopping... i guess there's always pizza delivery :)
<jml> mwhudson, me too :)
<LaserJock> is it generally going to require running a local instance of LP to work on bugs?
<thumper> LaserJock: it certainly helps
<rockstar> LaserJock, the tools to do that aren't terrible.
<jml> cjwatson, hi :)
<jml> join the party :)
 * cjwatson waves woozily
<rockstar> LaserJock, it's nothing like running the main LP servers.
<thumper> spm: did flacoste_afk's branch land?
<thumper> spm: I can't tell
<LaserJock> rockstar: ok
<thumper> spm: LP says nay
<thumper> LaserJock: it is "make run" :)
<thumper> LaserJock: if you've done the setup
<spm> thumper: which one? I manually merged one. should be a listening to Brazilian Girls something
<ghindo> I heard it was a party in here
<ghindo> Would this be accurate?
<thumper> spm: the one that he submitted just before going to bed
<rockstar> ghindo, as close to an IRC party as you can get.
<thumper> spm: it was in the queue just a few minutes ago
<spm> looking....
<jml> kfogel, fwiw, I'm getting a lot of questions about whether or not soyuz & codehosting are included
<jtv> g'day spm!  got my feature working.  :)
<thumper> it wasn't very explicit was it?
 * beuno blogged about it
<LaserJock> jml: yeah, everybody was told that soyuz and codehosting weren't going to be included
<jml> LaserJock, right.
<kfogel> LaserJock: I'm about to send an announcement explaining that we chnaged our minds
<kfogel> they have been opened
<LaserJock> I'm sure that will give Canonical some extra PR brownie points ;-)
<jacob> soyuz is included?
<mwhudson> jacob: yes
<LaserJock> especially since for a lot of us soyuz is particularly interesting
<jml> kfogel, cool :)
<jacob> mwhudson: wow, wasn't expecting that. cool. :)
<kfogel> LaserJock: let's hope it also gets us some patches, eh? :-)
<cjwatson> extra present for me when going to DebConf, one fewer question to answer :-)
<LaserJock> kfogel: possibly
<LaserJock> cjwatson: exactly
<rockstar> cjwatson, :)
<jml> cjwatson, :D
<LaserJock> I'll have to go through my LP bugs and see if there are any I dare trying to fix myself
<thumper> spm: can you see why it didn't land?
<spm> thumper: running precommit hook: /bin/false
<spm> which tends to um fail
<ScottK> cjwatson: If you see Phil Kern there, the fact that Soyuz got opened my affect his gsoc project ....
<LaserJock> from what I've heard and what little I've seen, I still don't think contributing is going to be a trivial exercise
<thumper> spm: d'uh
<thumper> spm: I need to land a testfix branch to get buildbot to rebuild
<cjwatson> ScottK: what's his gsoc project?
<spm> thumper: hang on. there's a few in here. gimme a bit....
<mthaddon> spm: we usually do that during release-critical to prevent landings to devel and push all landings to db-devel to avoid the amount of time it takes to get through to db-stable
<ScottK> cjwatson: Rewriting the Debian build system.
<jtv> sinzui, can you help me out with this?  I've got some zcml that uses a path_expression that needs to be updated.  The new version needs access to an element higher up in the path... can I do that?
<spm> mthaddon: yeah... bac was falling afould of it earlier. francis did know, so not sure what's up
<mthaddon> thumper: can you resubmit flacoste's branch to db-devel?
<cjwatson> ScottK: mm. I think Debian's pretty attached to its own system, I'd be surprised to see them switch to ...
<cjwatson> ... LP1111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111 ...
<thumper> mthaddon: I could manually merge devel into db-devel
<cjwatson> ... 111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111 ...
<cjwatson> ... 11111111111111111111111111111111111111111111111111111111111111111111111111
<cjwatson> doh. sorry!
<cjwatson> switch to LP, is what I meant to say
<mthaddon> channel spam!! :)
<ScottK> cjwatson: Agreed, but it's always interesting to see how someone else solved the problem.
<ajmitch> someone's overly excited :)
<ScottK> cjwatson: I expect there may be lesson's learned he can now take advantage of.
<mthaddon> thumper: that'll happen anyway as a matter of course (devel -> stable -> db-devel)
<cjwatson> ajmitch: stuck key repeat due to load :-(
<thumper> mthaddon: well it won't
 * ScottK goes to bed ....
 * mwhudson opens a beer
<thumper> mthaddon: as devel isn't building
<thumper> mthaddon: which is what I'm trying to fix
<mthaddon> thumper: oh, I see
<jml> "Launchpad is a Ruby on Rails blogging platform and CMS. It has categories and tags, RSS, a plugin architecture and theme architecture, and more. It's inspired by Wordpress." -- wtf
<jml> the name Launchpad has been taken on ohloh :(
<beuno> LOL
<thumper> hah
<thumper> mthaddon: so... what do we do here?
<mthaddon> thumper: I can temp open it up for you to kick it
<jml> but the url https://www.ohloh.net/p/launchpad is free.
<thumper> mthaddon: that be good
<mthaddon> thumper: you ready to do that now?
<beuno> jml, trigger the import
<beuno> they'll have fun  :)
<jml> beuno, I have to come up with a name first.
<beuno> wonder if they have an up-to-date version of bzr that can handle 2a
<thumper> mthaddon: gimmie a minute
<mthaddon> k
<jml> beuno, interesting question!
<ajmitch> problems with branches already? I'm getting 404 on http://bazaar.launchpad.net/~launchpad-pqm/launchpad/devel/utilities/rocketfuel-setup
<ajmitch> not a standard launchpad page saying that the page isn't found, just a boring "Not found" page
<beuno> I think that the link is wrong
<lifeless> no rev on it
<beuno> no?
<jml> hmm. ohloh gives 'https://code.launchpad.net/myproject/trunk' as an example branch url :\
<beuno> mwhudson, shouldn't there be a /files/?
<ajmitch> https://dev.launchpad.net/Getting will need updated with a working link then
<deryck> See you all back in a few hours. :)
<lifeless> ajmitch: you can edit the wiki apge :)
 * deryck sleeps
<thumper> mthaddon: ready now
<ajmitch> lifeless: I need a working link first :)
<lifeless> getting that
<mwhudson> ajmitch: bzr cat of that url really should work
<spm> ajmitch: that's not a http page - that's a bzr cat to the URL
<mthaddon> thumper: go for it
<lifeless> http://bazaar.launchpad.net/%7Elaunchpad-pqm/launchpad/devel/annotate/head:/utilities/rocketfuel-setup
<jml> beuno, yeah, looks like it doesn't support 2a yet.
<lifeless> http://bazaar.launchpad.net/~launchpad-pqm/launchpad/devel/annotate/head:/utilities/rocketfuel-setup
<lifeless> ajmitch: ^
<mwhudson> lifeless: no
<lifeless> mwhudson: no?
<spm> no! don't send folks via loggerhead!
<ajmitch> spm: ok, it was bzr cat that was failing for me, honest :)
<thumper> mthaddon: ok, it's in the queue
<mwhudson> lifeless: the instructions are for bzr cat <that url>
<jml> now, where did I put that diagram
<lifeless> mwhudson: ah
<mthaddon> thumper: k, will wait til it's through and then turn it back off
<lifeless> I'm sure lh will survive:)
<mwhudson> why it's not working for ajmitch i don't know
<ajmitch> bzr: ERROR: Invalid http response for http://bazaar.launchpad.net/%7Elaunchpad-pqm/launchpad/devel/utilities/rocketfuel-setup/: Unable to handle http code 404: Not Found
<ajmitch> is the real error I got
<mwhudson> yeah, lh is holding up reasonably well
<lifeless> ajmitch: get rid of the trailing /
<mwhudson> ajmitch: what version of bzr are you using?
<ajmitch> 1.16.1
<lifeless> ajmitch: although bzr may be adding that
<beuno> spm, common!  LH hasn't failed in what, 2 days?  3?
<lifeless> ajmitch: intercepting proxy?
<ajmitch> bzr is
<spm> beuno: edge update/restart about 6 hours ago. I win. QED etc :-P
<ajmitch> I'll unset http_proxy & see if it helps
<LaserJock> ajmitch: I get the same
<LaserJock> and I have no proxy
<ajmitch> no difference
<beuno> spm, that's just out of habit, I'm sure
 * mwhudson tries with 1.16.1
<mwhudson> ajmitch: do you have pycurl installed?
<ajmitch> LaserJock: I'm glad it's not just me going mad
<thumper> mthaddon: I have to go and start cooking now, will check back in about 10 minutes
 * mwhudson clutches at straws
<LaserJock> ajmitch: well .... it could be both of us ;-)
<mthaddon> so what's the actual command you're running that's failing?
<ajmitch> mwhudson: I do
<kfogel> wgrant: ping
<lifeless> I think everyone has been underestimating the load we can handle :P
<LaserJock> mthaddon:  bzr cat http://bazaar.launchpad.net/~launchpad-pqm/launchpad/devel/utilities/rocketfuel-setup > rocketfuel-setup
<ajmitch> mthaddon: bzr cat http://bazaar.launchpad.net/~launchpad-pqm/launchpad/devel/utilities/rocketfuel-setup > rocketfuel-setup
<kfogel> lifeless: yes -- habitual paranoia, but so far no problems
<lifeless> ajmitch: it works for me
<mthaddon> me too...
<lifeless> takes its own sweet time (but I'll be spanked if I tell you better url's to use :P)
<mwhudson> bzr 1.16.1 seems to be taking aaages
<lifeless> ajmitch: is it possible you have an intercepting proxy doing the wrong thing
<ajmitch> it's not instantly failing if I try on another box at work, running jaunty
<mwhudson> lifeless: http://bazaar.launchpad.net/~launchpad-pqm/launchpad/devel/annotate/head:/utilities/rocketfuel-setup > rocketfuel-setup isn't going to get you a very useful shell script
<lifeless> mwhudson: lp:
<ajmitch> it's possible, but I'm not aware of any proxy in the way
<LaserJock> it's failing rather quickly for me
<lifeless> ajmitch: do this please
<LaserJock> on Jaunty with bzr 1.16.1
<lifeless> LaserJock: you too
<lifeless> wget -S http://bazaar.launchpad.net/~launchpad-pqm/launchpad/devel/.bzr/repository/pack-names
<ajmitch> ok, I get a failure on jaunty on a computer with a different ISP about branch version at least
<LaserJock> lifeless: that works fine
<mwhudson> 1.16 worked for me
<ajmitch> lifeless: that grabbed a file
<mwhudson> (and i do have an intercepting proxy, thanks voda!)
<lifeless> LaserJock: ajmitch: I want to see the headers you got
<ajmitch> one second
<ajmitch> http://paste.ubuntu.com/223265/
<lifeless> ajmitch: thats from the failing machine?
<ajmitch> yes
<LaserJock> lifeless: you mean this: http://paste.ubuntu.com/223266/ ?
<lifeless> ok, you seem to be getting the right root node then
<lifeless> try running the command again with -Derror -Dtransport
<LaserJock> lifeless: the wget or bzr command?
<lifeless> bzr
<mwhudson> ajmitch, LaserJock: the script we're trying to get is here btw: http://pastebin.ubuntu.com/223269/
<cjwatson> I'm seeing bzr cat fail too
<mwhudson> (in case you want to get the pull running in the background)
<jkakar> Congratulations Launchpad team.  It's great to see this happen. :)
<cjwatson> wget -S: http://paste.ubuntu.com/223270/
<ajmitch> http://paste.ubuntu.com/223271/ for the updated bzr cat output
<cjwatson> -Derror -Dtransport shows a traceback: http://paste.ubuntu.com/223272/
<ajmitch> mwhudson: thanks
<LaserJock> lifeless: http://paste.ubuntu.com/223273/
<cjwatson> I'm on Karmic with bzr 1.17
<cjwatson> err, or rather bzr 1.17~rc1-1
<lifeless> ah
<lifeless> its bzr-svn FAIL
<mwhudson> er ah eh
<mwhudson> bzr-svn!!
<lifeless> run with --no-plugins
<mwhudson> yeah
<ajmitch> aha
<lifeless> mw care to get the karma for filing the bug? :)
<mwhudson> i guess i'll edit the wiki page
<ajmitch> ok, I'm getting some progress fetching stuff now :)
<cjwatson> --no-plugins works
<lifeless> mwhudson: care to get the karma for filing the bug? :)
<mwhudson> lifeless: looking now
<LaserJock> lifeless: --no-plugins works for me as well
<mthaddon> thumper: success
<lifeless> its open_containing failing is all
<jml> rs=me :(
<mwhudson> jml: what?
<mwhudson> oh
<jml> someone just landed a revision with that tag.
<nxvl> hi, is there any ppa where i can get a bazaar that allows me to download the LP codE?
<ajmitch> lifeless: thanks for tracking that down
<mwhudson> nxvl: ~bzr ppa?
<mwhudson> ajmitch: sorry for not testing harder
<nxvl> mwhudson: https://edge.launchpad.net/~bzr/+archive/ppa that one?
<mwhudson> nxvl: yeah
 * mwhudson notices that 1.17 arrived there 17 minutes ago...
<LaserJock> what's User sabdfl for in Host bazaar.launchpad.dev for the ssh config?
<spiv> Heh.
<spiv> convenience of manual testing of the ssh server; 'sabdfl' is one of the users in the sample data.
<LaserJock> ah, i see
<LaserJock> but all this /etc/hosts and ssh config munging is local, right?
<thumper> jml: I just copied what flacoste_afk did :)
<thumper> mthaddon: thanks, I'm EOD now
<mthaddon> thumper: me too :)
<jml> LaserJock, yeah, that's local.
<jml> spiv, and that's obsoleted convenience -- ./utilities/make-lp-user <your-user-name> is much nicer.
 * spiv nods
<thumper> mwhudson: want to copy the using lp codehosting to dev.launchpad.net?
<mwhudson> lifeless: it's a bit odd, open_containing_tree_or_branch doesn't fail with my local apache
<mwhudson> but surely does against bazaar.launchpad.net
<mwhudson> thumper: ok, filing a bzr-svn bug first
<thumper> mwhudson: ok
 * thumper is really away now
<lifeless> mwhudson: fun
<mwhudson> lifeless: https://bugs.edge.launchpad.net/bzr-svn/+bug/402063 anyway
<ubot3> Malone bug 402063 in bzr-svn "bzr-svn breaks open_containing_branch_or_tree against bazaar.launchpad.net" [Undecided,New]
<jml> I hadn't seen that doc before
<mwhudson> jml: you mean the doc i wrote this morning?
<jml> mwhudson, yeah. :)
<jml> mwhudson, I saw the timestamp, but I'd never knew that such a doc was brewing
<mwhudson> jml: thumper kept bugging me about it in standups
 * jml is obviously a terrible listener.
<henninge> I am here ...
<henninge> !
<mwhudson> :)
<mwhudson> jml: now at https://dev.launchpad.net/Code/HowToUseCodehostingLocally, any idea where to link to it from?
<henninge> spm: Can you please clear the translation import queue on staging for me? If jtv does not object ...
 * jtv does not object
<henninge> spm: "delete from translationimportqueueentry"
<spm> henninge: sure. one sec.
<jml> mwhudson, probably from https://dev.launchpad.net/Code for now
<jtv> spm: may take a while to replicate...
<jtv> we're talking what, 150K entries?
<jml> mwhudson, I just added the system diagram there.
<spm> henninge: jtv: DELETE 129367 commit>
<spm> ?
<henninge> spiv: please do.
<henninge> spm: ^
<spm> fail :-)
<spm> henninge: commited
 * jtv plugs ears
 * jtv squats under table
<henninge> %Â§Î©#* auto-completion
<henninge> jtv: all is well
<mwhudson> jml: done
<henninge> spm: thank you
<jtv> henninge: and the one place where you really want auto-completion nobody ever wants to implement it.  I keep asking for tab password completion, but do they listen?
<jml> mwhudson, woot.
<henninge> jtv: Sounds like the next Windows feature for me!
<jtv> henninge: well Windows doesn't have that problem.  "Hi, you typed the wrong password 3 times.  Would you like to create a new one?"
<jtv> lifeless: had a bit of weirdness with my bzr branches if I build my TransformPreview but do not go on to commit anything...  when I unlock the branch it says the lock was broken.
<jtv> lifeless: as if the info file wasn't found for some extraneous reason (never moved into place unless I commit maybe?)
<henninge> spm: can you please run the poimport script on staging for me. It is running from cron every 20 min but for speed I'd appreciate a manual run.
<lifeless> probably there is a transform thing you need to call to tell it you're not recording the result
<jtv> lifeless: any idea who might know?
<henninge> spm: "cronscripts/rosetta-poimport.py"  just that
<lifeless> jtv: abentley; but - show me the backtrace
<jtv> spm: btw is it really 20 minutes?  That's fine, but istrm we had it set up for much fewer runs initially.
<spm> jtv: yah. every 20
<jtv> spm: that's great, thanks.
<jtv> makes QA a lot easier.
<spm> henninge: running
<jtv> lifeless: hang on, I'll reproduce.
<spm> henninge: done. I should have waited.
<jtv> The problem!  The problem!  I'll reproduce _the problem_!
<henninge> spm: thanks. why?
<spm> henninge: normally poimport runs for ages. finished in < 30 secs
<henninge> spm: that's because we cleared the queue first!
<jtv> spm: ages?  Shouldn'tâafter 8 minutes (iirc) it stops taking on jobs and leaves the rest of the queue to the next script run.
<mwhudson> so is the news going to hit slashdot, i wonder...
 * maco hugs all
<spm> jtv: I may be talking based on old info - but I can find out if desirable?
<LaserJock> mwhudson: and digg and LWN and ...
<mwhudson> LaserJock: yeah
<mwhudson> but i think the slashdot effect is still real, all these years on
<jtv> spm: hmm... would be nice to knowâyes please.
<jtv> lifeless: https://pastebin.canonical.com/20127/
<lifeless> jtv: pastebin.ubuntu.com these days, surely :)
<lifeless> jtv: brand new world
<jtv> lifeless: oh, I wasn't aware of that one.  I just type "paste" and let my browser figure out the rest.
<lifeless> sorry, paste.ubuntu.com
<lifeless> well, open :)
<lifeless> so, something is holding a lock on the branch
<jtv> lifeless: I am.
<lifeless> oh actually
<jtv> A write lock.  And when I unlock it, boom.
<lifeless> that message says 'the lock was broken while open'
<lifeless> so perhaps something deleted the branch?
<lifeless> its in your teardown
<lifeless> is it possible you're causing 'rm -rf', 'branch.unlock()' to happen, in that order?
<henninge> spm: I'd like to put staging into read-only mode for a few minutes.
<spm> henninge: ? why not just use the slave?
<henninge> spiv: This is a patch that does that, can you apply it please and restart staging? https://pastebin.canonical.com/20128/
<beuno> already on it's way to digg: http://digg.com/linux_unix/Canonical_releases_complete_source_code_for_Launchpad
<henninge> spm: No, the fix I am QA'ing checks that configuration directive.
<henninge> spm: The bug is specifically about read-only mode.
<henninge> bug 380852
<jtv> lifeless: it's the first teardown I do, afaik.  I could try doing the unlock explicitly at an earlier stage; hang on.
<henninge> bug #380852
<ubot3> Malone bug 380852 in rosetta "translation page shows editable fields while in read only mode" [Medium,Fix committed] https://launchpad.net/bugs/380852
<henninge> ah!
<henninge> spm: is the restart still such a pain with buildout?
<jtv> lifeless: you're right, it happens in the teardown but not if I unlock before then.
<jtv> lifeless: maybe useBzrBranches puts its cleanup in the wrong place?
<spm> henninge: patched and restarted
<henninge> spm: great!
<lifeless> jtv: use addCleanup rather than tearDown
<jtv> lifeless: already made that change in an as-yet unlanded branch, though it wasn't enough to fix the problem.
<henninge> spm: thank you very much! Everything works as expected.
<spm> henninge: revert?
<henninge> spm: Can you just revert or would you like a patch?
<spm> revert is fine
<henninge> spm: ;-)
<henninge> spm: please do
 * henninge is blazing through the test plan here ...
<spm> henninge: is back
<henninge> spm: cool, that was a great help!
<VK7HSE> just in the throws of setting up a local copy of LP here... \o/
<jtv> VK7HSE: I hope that's "throes," not "exception throws" :)
<VK7HSE> yeah something like that !!! ;)
<\sh> what did I read? LP is opensourced? Congrats to everybody involved...good job :)
<jtv> I guess the word is out then
<VK7HSE> https://dev.launchpad.net/Getting
* beuno changed the topic of #launchpad-dev to: Launchpad Development Channel | https://dev.launchpad.net/ |  Please use #launchpad for support. | https://launchpad.net/~launchpad-dev | Get it: http://identi.ca/notice/6403823
* beuno changed the topic of #launchpad-dev to: Launchpad Development Channel | https://dev.launchpad.net/ |  Please use #launchpad for support. | https://launchpad.net/~launchpad-dev | Get it: https://dev.launchpad.net/Getting
<beuno> COPYNPASTE FAIL
<beuno> should go to bed
<mwhudson> failures in buildbot :(
<mwhudson> all seemingly to do with uncollectable garbage :(
<jml> mwhudson, actually no.
<mwhudson> no?
<jml> mwhudson, one of them is to do with the move of bzr to buildout.
<mwhudson> jml: oh "yay" ?
<jml> mwhudson, it ought to be fixed when my branch is finished running through ec2test.
<mwhudson> jml: oh ok
<jml> except that's going to db-devel
<jml> hmm.
<henninge> jtv: any clue how to best QA bug #388473 ?
<ubot3> Malone bug 388473 in rosetta "POTMsgSets on page not in ascending sequence order" [High,Fix committed] https://launchpad.net/bugs/388473
* lifeless changed the topic of #launchpad-dev to: Launchpad Development Channel | https://dev.launchpad.net/ |  Please use #launchpad for support. | https://launchpad.net/~launchpad-dev | Get it: https://dev.launchpad.net/Getting | http://paste.ubuntu.com/
<jtv> henninge: in a case like this I'd just make sure that the page works properly, and let the oops reports show that the bug is gone.
<henninge> jtv: cool, can do that ;)
<jml> mwhudson, and hey, that just finished with ec2test. I'll retarget the branch for devel & land appropriately
<kfogel> mwhudson, jml: yeah, somewhere we need to explain our four trunks thing, in the dev wiki...
<thekorn> congrats everybody!
<kfogel> thekorn: hey there!  Thank you.  Enjoy :-).
<jml> kfogel, yes. but not today :)
<kfogel> jml: indeed
<emilis_info> how can I remove all changes introduced  by rocketfuel-setup?
<emilis_info> I didn't expect it to mess up my Apache+PHP setup
<jml> emilis_info, good question!
<thekorn> wow, and soyus and codehosting too, awesome
<thekorn> you must have a good PR manager, nice bluff ;)
<jml> emilis_info, there's isn't an automatic way of doing it, probably best to look at the script itself.
<emilis_info> heh
<LaserJock> thekorn: that's what I was thinking ;-)
<emilis_info> I think you should recommend installing it on a separate / virtual machine, because it touches so many places it can break something for a lot of people
<noodles775> Morning
<jtv> hi noodles775!
<noodles775> Hiya jtv, gee you've been around early and late lately? QA to finish?
<henninge> jtv: even better, there is  a link in the sequence bug that fails on  production but succeeds on edge - QAed!
<jtv> noodles775: yeah, QA's been a real bottleneck for the past few months.  Finally got a full end-to-end QA on production though, so pretty cool.
<jtv> noodles775: I've worked about 2.5 working days so far this week.  :)
<noodles775> Sheesh
<jtv> henninge: perfect.
<jtv> henninge: that reminds me... I do _not_ plan to do much more today.  I finished at 5 AM and got back to work at 11 AM, and the most crucial thing is done.
<henninge> jtv: you are tough!
<jtv> henninge: would you mind having a very quick standup soon and then doing without me?
<jtv> henninge: I'm stupid.
<henninge> jtv: what is the crucial thing?
<henninge> jtv: I don't mind at all!
<jtv> henninge: there was yet another bzr hiccup.  Tickling yet another part of bzrlib that Launchpad hadn't touched yet.
<henninge> oh, bzrlib
<jtv> yes, when you reference a directory you need to look up or create an id for it.  And first you need to do the same for its parents.
<jtv> But the lookup doesn't work for directories you just created, only for ones that were already in the committed branch.
<henninge> spm: how much longer will you be around?
<jtv> So if you had n files in the same directory d, where d was not in the committed branch, I'd create d n times.
<lifeless> jtv: meep!
<spm> henninge: -22 minutes. I just haven't had a chance to sign off yet. :-)
<jtv> lifeless: hey, this stuff is obvious to you, Here Be Dragons for us.  :)
<henninge> spm: ok, enjoy the evening, then. Thanks for your  help!
<lifeless> jtv: well, you're using tree transforms, which I'm not actually familiar with
<jtv> lifeless: see?  Going Where No Man Has Gone Before.
<jtv> spm: the script timings look good to me.
<jtv> spm: that's good to know.
<noodles775> mwhudson: do you know if there's a general problem with imports at the moment, or have I just mis-reviewed some, like this one:
<noodles775> https://code.edge.launchpad.net/~vcs-imports/cairo-dock/trunk
<noodles775> There seem to be a lot of failures when I was CHR yesterday.
<jml> spm, PQM is now rejecting my branches with "Executing pre-commit hook /bin/false at Tue Jul 21 08:06:41 2009"
<noodles775> jml: I had that yesterday, but it was because I was trying to submit to devel rather than db-devel?
<jml> yeah. I'm deliberately submitting to devel.
<emmajane> congrats on the announcement!!
<jml> emmajane, thanks :)
<popey> \o/ well done
<henninge> jtv: is that ok? <dpm> kyleN: the only thing is that it'll probably have to wait until tomorrow, I've got a meeting in 30 min.
<henninge> <kyleN> ok.
<emmajane> jml: I'll be sure to gloat on your behalf at oscon :)
<henninge> oops, where did that come from ... ???
<jml> :D
<jtv> henninge: paste from old backlog?
<henninge> jtv: probably.
<henninge> jtv: what I meant to paste: https://answers.edge.launchpad.net/rosetta/+faq/619
<mwhudson> noodles775: that's just cscvs being lame abount reconnecting i think
 * mwhudson AFK REALLY
<noodles775> OK, thanks mwhudson
<jtv> mwhudson: bye
<jtv> henninge: looks about right, apart from the tone maybe...  what about it?
<henninge> jtv: ok, i'll change it ... ;-)
<henninge> jtv: you are talking about the en_GB remark, tright?
<jtv> henninge: oh, you just wrote this?  Great.  Few suggestions...
<jtv> That's the main one, yes.
<a_c_m> congrats!
<jtv> Also, don't say "iso," but "ISO 639."  Also note that we prefer two-letter codes over three-letter codes if both are available.
<jtv> a_c_m: thanks
<jtv> henninge: also, there are two "levels" of exceptions to the country-code rule.  I think it would be useful to distinguish them.
<a_c_m> hopefully means it will get some love from the wider community and perhaps spawn something that can give github a run for its money :)
<jtv> henninge: there are the exceptional _languages_ that really are meant to have separate codes: pt_BR, zh_CN, zh_TW.
<jtv> henninge: then there are the exceptional _cases_ where adding a more specific country code may be justified, e.g. es_MX.
<jtv> henninge: and finally there is English.  English is assumed to be US English, and it's assumed to be the source language, so en_GB is justified as an exceptional "case" as above.
<henninge> jtv: but en_GB is active whereas es_MX isn't.
<jtv> henninge: yes, that's another level of distinction: some specialized languages are more desirable than others.
<henninge> jtv: I think for the FAQ I'll just mention the first case and refer to the second as "Other special cases may be allowed on a pre-project basis."
<jtv> henninge: as long as it's clear that the "default" country should definitely not be used, and there are a few exceptions where the country is always included.
<jtv> henninge: I also wouldn't say there's no list of languages when you also point to the page where people can query the list.  Just complement it with a pointer to the ISO-639 lists.
<henninge> ok
<jtv> henninge: finally, you may want to note that we don't support collective language codes.
<jtv> henninge: e.g. "Indo-European (other)."  No point in translating to that!
<henninge> jtv: good point!
<jtv> henninge: you did open a can of worms here: don't forget variants.  :-)  "Minimally supported for imports, but not in the UI."
<jtv> henninge: and after that, we can have that quick call and I can bugger off.  :)
<henninge> jtv: right ... ;-)
<jtv> henninge: just let me know when you're done.
<jml> g'night all.
<jml> happy hacking
<noodles775> 'night jml
<henninge> jtv: I am done: https://answers.edge.launchpad.net/rosetta/+faq/619
<jtv> henninge: don't say "not the same as the language code"âthat suggests that nb_NO and sv_SE are OK (and that ar_AR is not okay for the wrong reason)
<henninge> jtv: ok, I also forgot the language group rule
<jtv> henninge: say something like "the language's home country."
<jtv> henninge: typo: excpetions
<jtv> henninge: also, exceptions for "de_AT" are a very different matter from exceptions for "de_DE."  I'd still keep the two separate: _never_ use de_DE, but _avoid_ de_AT.
 * ddaa notices that the dev.launchpad.net wiki still points to SQLObject in the "External documentation" section
<ddaa> hasn't the code base switched to STORM a while ago?
<jtv> ddaa: yes it did
<jtv> ddaa: this is something for the Foundations folks (flacoste and team)
<jtv> stub, you maybe?
<noodles775> ddaa: parts of the code still use SQLBase via Storms SQLBase support (as a transition)
<henninge> jtv: I'll not mention those, I think that is covered by "please try without them first."
<ddaa> jtv: just drop the link already, before people start being confused and saying "Hey launchpad uses SQLObject, not STORM"
<jtv> henninge: which ones?
<henninge> jtv: _avoid_ using ...
<jtv> ddaa: sorry, my head is exploding in slow-motion, not opening another tab right now!
<jtv> henninge: if you insist...  Got more:
<jtv> The first bullet point needs punctuation.  In fact I'd hoist it out of the bullet-points list and replace the leading line with "Launchpad uses ISO-639 language codes: two-letter ISO-639-1 codes where available (link) and three-letter ISO-639-3 codes for other languages (link).
<jtv> henninge: if you don't like that, then insert a comma before the "or" and remove the full stop after "nds."
<jtv> henninge: In the second bullet point, I'd say "do not include the country code except in a few exceptional cases."  "Append a specific country code" doesn't sound like the sort of thing a non-programmer would suspect themselves of doing.
<jtv> henninge: btw you can resolve the dichotomy between de_DE and de_AT by replacing "NOT it_IT, es_ES but also not fr_BE" with: "NOT it_IT, es_ES, but also avoid fr_BE"
<beeman_nl> hi guys, cheers on open sourcing Launchpad! great thing!
<beeman_nl> when i try to install it on Jaunty (32bit) i get the following error: bzr: ERROR: Unknown repository format: 'Bazaar repository format 2a (needs bzr 1.16 or later)\n'
<beeman_nl> any clues on what that can be?
<LaserJock> beeman_nl: that you need to have bzr version 1.16 or later?
<jmarsden> beeman_nl: https://dev.launchpad.net/Getting     (you need a newer version of bzr)
<BjornT> ddaa: thanks, i fixed the link to point to storm instead
<jtv> henninge: next, are you sure it's safe to say "because they are really considered separate languages"?  We don't want to be showered in email about how different people's local dialects are different.  In this case I'd only say that the country codes are needed for these languages, and not go into the why.
<jtv> henninge: (I also wouldn't want to go into how Cantonese and Mandarin are completely different languages yet they get the exact same language codes, whereas Mandarin (Simplified) and Mandarin (Traditional) are the same language but get different codes :-)
<beeman_nl> LaserJock, jmarsden: excuse me, i did'nt read that well (it's early up here)
<jtv> henninge: ahhh, the page is magically updating as I type this.  :-)
 * henninge edits and save like a madman
<henninge> jtv: done it again
 * jtv chuckles
<henninge> jtv: I think we should leave it at that now. As you said, it is a can of worms.
<jtv> henninge: sure.  It's good to have this documented, so thanks for taking the trouble.
<jtv> henninge: does that mean we can have the call?
<henninge> jtv: call!
<cbx333> Hi guys
<cbx333> anyway to just get the latest revison from the branch instead of the entire history
<cbx333> ??
<bigjools> use a checkout instead of a branch
<cbx333> so do i edit the rocketfuel scripts?
<bigjools> yeah, there's a bzr branch somewhere, just make it a bzr checkout
<cbx333> ok cool
<cbx333> wanted to check it out, but didn't want to branch the whole thing
<cbx333> at the moment it's just a quick test
<bigjools> it's only about 150M to branch
<spiv> bzr currently isn't very optimised for getting just the current revision, unfortunately.
<spiv> It shouldn't be too hard to fix, but it just hasn't bubbled to the top of the todo list yet.
<cbx333> that's cool
<cbx333> bzr: ERROR: Unknown repository format: 'Bazaar repository format 2a (needs bzr 1.16 or later)\n'
<cbx333> hmmm
<cbx333> I'm on 9.04
<cbx333> i should be ok right?
<noodles775> cbx333: bzr --version ?
<spiv> 1.16 is only about a month old; you need bzr from the bzr PPA I think.
<cbx333> ahh doh
<cbx333> ko
<bigjools> 1.16.1 has some essential fixes in it IIRC
<spiv> Yeah, I meant 1.16.1.  1.17 is even better!
<cbx333> hehe
<spiv> (and is already available in the ~bzr PPA)
<bigjools> cool
 * cbx333 grabs
<cbx333> :)
 * gmb rubs his eyes...
<gmb> Why are all these people looking in at me?
<gmb> Aah, that's it. We must be open source now. Bon.
<an7> Anyone tried installing launchpad on Debian? Anyone know if it will definitely not work?
 * cbx333 pokes gmb
<cbx333> wakey wakey
<proppy> an7: is there a debian package yet ?
<an7> proppy, no idea - was looking at doing it from bzr
<proppy> an7: maybe there is a ppa for you to play with
 * silbs checks out the new digs
<gmb> an7: AFAIK, no-one's ever tried installing it on debian.
<gmb> an7: May eat your cat, etc... but give it a shot and let us know :)
<silbs> well done, guys
<bigjools> silbs!
<silbs> bigjools!
<bigjools> silbs, are you thinking of swapping the day job?  You're in a dev channel now :)
<\sh> guys...I understand that all correctly, it comes even with the build infrastructure parts of LP (aka soyuz?)
<silbs> bigjools: I hear this is the place to come if I want to contribute code to LP ;)
<bigjools> silbs: patches welcome! ;)
<gmb> \sh: Yes.
<bigjools> \sh: yes, soyuz is open
<bigjools> \sh: if you have any questions about soyuz, I'm here to help.
<\sh> bigjools: I'll install it tomorrow on my buildserver in our DC...so I'll come and ask ;)
<\sh> bigjools: btw..which possibilities for chroots do you have? lvm+xen+sbuild or also other ways?
<bigjools> \sh: we only handle sbuild right now (well, the custom version of it we have in the tree)
<mpt_> beuno, the Launchpad front page is missing the sentence "All of it." in 48-pt font
<elmo> beuno: and <blink>
<tacone> is it already opensource ? i thought it was postponed.
 * stub hands silbs his hat and goes out for a beer
<gmb> tacone: It's open. It was never postponed, we just wanted to make sure that everything was working, so we didn't want to pin it on a specific date and then let everyone down.
<tacone> k
<tacone> congratulations.
<tacone> thanks for the info, bye
 * jtv afks
<walterl> hi. so is there somewhere/somehow i can just get the code in stead of having to download and install the whole of lp?
<gmb> walterl: I'm not sure I understand the question. If you bzr branch lp:launchpad you're just getting the code. If you're looking for a tarball, no, there isn't one available.
 * silbs gives back stub 's hat, and takes his beer instead
 * stub stabs silbs in the face
<silbs> wow, talk about escalation!
<stub> Mine! Mine!
 * bigjools grabs popcorn and watches
 * silbs passes out beers to everyone ;)
<walterl> gmb: thanks :)
 * cbx333 passes out victory slushies
<lifeless> mwhudson: hey
<lifeless> https://code.edge.launchpad.net/
<lifeless> mouse over mplayer there
<lifeless> read the comment (128 days old)
<lifeless> then click on it
<lifeless> and look at the trunk
<mwhudson> lifeless: that looks wrong
<lifeless> yes, it does
<deryck> Morning, all.
<bigjools> mornign deryck
<bigjools> and morning, too
<\sh> bigjools: another question regarding soyuz and the underlaying infrastructure: you configure/install as well a full fletched archive infrastructure with it, or is it still self implementable (thinking about wanna-build infra etc.)
<bigjools> \sh: I don't understand your question
<\sh> bigjools: do you ship an automatical archive management with the rocketfull setup? or do the admin installs/configure  the archive stuff him/herself?
<bigjools> \sh: it needs to be configured
 * \sh 's testinstalling now @home...let's see where it will take me ;)
<lifeless> \sh: rf-setup gets the source for development ;P - deployment is a _whole nother problem_
<\sh> lifeless: I just wondered if you bring all the stuff with the source tree, and it just needs to be configured to sane defaults for your own infra...(and/or setting up cron jobs etc.) and regarding the explanation on d.l.n/Getting : it starts to deploy at least a minimum ;)
<\sh> looks like that bazaar.launchpad.net is under heavy load ;)
<prusnak> hi
<bigjools> hello
<prusnak> the first command from launchpad get the source howto stalls:
<prusnak> bzr --no-plugins cat http://bazaar.launchpad.net/~launchpad-pqm/launchpad/devel/utilities/rocketfuel-setup > rocketfuel-setup
<prusnak> do you know what is wrong ?
<bigjools> I think the servers/links are a little loaded, someone is looking at it to see if it can be improved
<stub> bigjools: lucille's db connection is being killed about every two hours (long running but idle transaction).
<stub> Last one 4 hours ago, so maybe it is ok now?
<awilkins> I think the LP server for the LP sources is a bit.... hammered at the moment :-)
<markust> Just a short: T H A N K   Y O U ! This is *really* great news. Especially since soyuz and codehosting are included. This is a huge step forwards. I'm just hoping Canonical will make a brazillion with it's services. :-) Keep up the good work!
<bigjools> stub: that's not pleasant, how long has it been happening?
<bigjools> markust: great, I hope you can get involved with the code!
<lifeless> prusnak: it should be better now
<stub> First notification I have was from 14 hours ago. Then 11, 8, 7 and 4 hours ago.
<stub> Killing lucille(11230), 2009-07-21 06:51:54.161602+01:00, 2009-07-21 06:51:54.638500+01:00
<bigjools> stub: do you know which query?
<an7> $ bzr --no-plugins cat http://bazaar.launchpad.net/~launchpad-pqm/launchpad/devel/utilities/rocketfuel-setup
<an7> bzr: ERROR: Invalid http response for http://bazaar.launchpad.net/%7Elaunchpad-pqm/launchpad/devel/.bzr/repository/packs/34bf81ec32ec262a521009fd975bfad1.pack: Missing the Content-Range header in a 206 range response
<an7> anyone seen that?
<stub> bigjools: There would have been no active query - that is why it was killed. The script will have been sitting around doing something non-db related for an hour.
<gmb> an7: bazaar.lp.net is getting hammered and is hiccuping a bit at the moment. Someone's looking at it to see if matters can be improved
<bigjools> oh idle txn, ok
<an7> thanks gmb, will try later
<stub> The oops will be 'terminated by administrator' or similar
<spiv> an7: IIRC that's generally caused by broken proxies
<spiv> an7: especially dodgy intercepting proxies.  If vila is around on #bzr he might be able to help debug it (assuming it isn't transient)
<bigjools> stub: can you see if there's an oops number available please?
<stub> bigjools: There is no oops at this end.
<an7> spiv, I know there is a dodgy transparent proxy where i work - maybe that is causing it
<stub> bigjools: I don't even know what script was at the other end - just that it was connected as the lucille user :)
<bigjools> stub: bleh. I can't do a lot without an oops :(
<bigjools> stub: it's the publisher
<stub> bigjools: Yup. But now when you see the oops, you will know the trigger.
<awilkins> an7: That could be it.. the typical dodgy proxy error is to do with ranges and multipart separators
<spiv> an7: ah, yep, in that case I'd say that's very likely to be the cause.
<stub> bigjools: We can increase that timeout if necessary, but it is better to turn the long running transaction into shorter transactions.
<awilkins> an7: I wouldn't try using bzr+ssh right now though, I think the server is too popular :-)
<stub> bigjools: lucille gets killed occasionally and nobody really notices - just that this time I saw a sequence of them so letting you know in case something is stuck.
<bigjools> stub: yes, but if it normally runs ok then we have another problem lurking
<bigjools> ok cheers
<awilkins> Just a thought - what about a tarball of the launchpad folder which you could spread across more redundant hosting as a "starter for 10" ?
<awilkins> You could even ship it with the trees removed and get a script to check them out
<lifeless> awilkins: no need; we're serving over http anyway
<lifeless> awilkins: we can do multiple backend stuff there; the problem was /not/ pure load
<awilkins> lifeless: Fairy snuff
<lifeless> what happened is that a cache with a bad failure mode (and heck, you can get the source and read it :P) failed
<lifeless> and once failed it won't recover.
<lifeless> we've bandaided it right now.
<awilkins> I'm excited, been waiting for this for a year :-)
<lifeless> :)
<lifeless> spiv: we bounced apache
<lifeless> spiv: its entirely possible it provoked a bunch of oddness
<VK7HSE> Hmm... rocketfuel-setup has bombed out... so I attempted to re run... now it's nagging me that it has "make: *** No rule to make target `install'.  Stop. ERROR: Unable to install apache config appropriately" :(
<VK7HSE> so I didn't get it built! so is this a case of delete the created lp-branches and re run ?
<VK7HSE> yep that's done the trick!
<bigjools> VK7HSE: can you report a bug about that please, we should fix it.
<VK7HSE> bigjools: sorry for such a NOOB question, but against what? as in the place to do so, as I don't want to file in the wrong place...
<bigjools> VK7HSE: file a bug on the launchpad project, thanks!
<VK7HSE> ;)  will do
<\sh> guys...should I really add "sabdfl" as username to my .ssh/config as rf-setup tells me to do ?
<bigjools> \sh it's just for access to the local instance
<bigjools> the sample data doesn't know about too many people
<\sh> bigjools: k :)
 * wgrant wanders in.
<wgrant> So you did it on schedule after all. Very nice.
<bigjools> did you ever doubt us? :)
<VK7HSE> here's Bug #402187  for review...
<ubot3> Malone bug 402187 in launchpad "rocketfuel-setup can't resume if fails to complete" [Undecided,New] https://launchpad.net/bugs/402187
<bigjools> VK7HSE: do you still have the screen scrollback from when it failed?
<bigjools> it would be useful to see exactly where it failed
<VK7HSE> nahh sorry out of the buffer...   I wanted all that to post into the bug...  but silly me re ran the script before thinking...
<bigjools> VK7HSE: can you remember if it had finished pulling the main trunk?
<VK7HSE> but it is whilst creating, Fetching revisions:Inserting stream
<VK7HSE> No I think this is where it bombed out I think my ISP had a hiccup that resulted in the fail connection...
<bigjools> yeah that's unfortunate
<bigjools> as lifeless said somewhere earlier, bzr and other DVCSes don't restart half-fetched branches :(
<VK7HSE> but so far by deleting the lp-branches directory all seem to be going fine... although the through put is woeful!  5-75 kbs
<bigjools> ouch
<VK7HSE> so it will take a while to grab it all!  but hey it will be woth==rth the wait!
<VK7HSE> Gahh try, worth!
<awilkins> The fix would be to make the initial branch an empty one and to pull it
<awilkins> Or to just make the script delete the existing branch if it found it, because the revisions are already present, the overhead of remaking the branch would be minimal
 * wgrant whines about it not running on Karmic.
<wgrant> Does it still need Python 2.4?
<bigjools> yes :(
 * wgrant schroots.
<bigjools> but the good news is that we're working on that, or more to the point, barry is :)
<awilkins> Alas, can't post a merge branch without having pulled it first ;-|
<\sh> wgrant: what? LP is not working on karmic?
<barry> well, i wish!  actually flacoste_lunch and gary are working on that.  i'm just trying to noodge them along :)
<wgrant> \sh: It needs Ancient Python.
<wgrant> And the PPA isn't set up for Karmic.
<bigjools> barry: ah!  I thought you were involved since your previous work on it
<barry> we're not far from python 2.5 or 2.6 though (hopefully the latter)
<awilkins> I knew there was no reason to be bleeding edge :-)
<barry> bigjools: i have several abandoned branches :)
<bigjools> we should write this up on the wiki ...
<intellectronica> is anybody dealing with the tree breakage? this is not a very nice time to be in testfix mode
<intellectronica> looks like many codehosting tests are failing
<bigjools> intellectronica: I didn't even know we were in testfix, was there an lpbot email?
<intellectronica> bigjools: come to think about it, no, i haven't seen an email from the bot. but there's definitely failures. see  the waterfall page
<barry> bigjools: you mean like a faq entry on the dev wiki? <wink>
<bigjools> barry: yes! and in the Getting page
<gmb> Wow. Looks like just about every codehosting test ever has blow up...
<gmb> Oh, no, just the AcceptanceTests.
<bigjools> barry: ah you added it!
<barry> bigjools: yep! you're editing the Getting page?
<bigjools> barry: yep.
 * bigjools high-fives barry
 * barry cancels his edit
<barry> :)
<bigjools> lol
<barry> bigjools: i was going to add a prerequisites section above "getting it"
<bigjools> barry: that's a good idea
<Mez> hey guys, where can I just get the source code ? rather than an install.
<VK7HSE> I wonder how many people are attempting to get the new source ???  average now is 2kbs :(  (looks like it will be a late/early morning!)
<wgrant> Mez: bzr get lp:launchpad, I suppose.
<gmb> Mez: see https://dev.launchpad.net/Getting
 * bigjools is shocked that wgrant hasn't downloaded it already
<Mez> gmb: AFAICS, that only tells you howto do an install, not how to get a copy of the souce.
<bigjools> Mez: bzr [branch|checkout] lp:launchpad
<wgrant> bigjools: My Internet connection is unhappy with these dependencies.
<wgrant> And Codehosting isn't feeding me.
<bigjools> yeah, I set up LP on top of a JeOS image last week
<bigjools> the dependencies are a bigger download than LP itself
<wgrant> I decided I'd stop apt from installing Recommends.
<gmb> Mez: In which case, you can just branch lp:launchpad, as wgrant suggested.
<wgrant> That cut the initial dependency installation from >300MB to 80MB.
<bigjools> wgrant: remind me how to do that ....
<wgrant> bigjools: Drop 'APT::Install-Recommends "false";
<wgrant> ' in /etc/apt.conf.d/something
<bigjools> wgrant: you use karmic right?
<wgrant> bigjools: Yes.
<wgrant> bigjools: But I'm currently installing in a Jaunty chroot.
<bac> good morning everyone
<wgrant> As your PPA doesn't like Karmic much.
<Mez> gmb: yes. I'd already got my answer from wgrant :D
<bigjools> my jaunty installation doesn't have that apt config
<bigjools> yeah, karmic isn't so supported yet
<wgrant> bigjools: Sorry, /etc/apt/apt.conf.d/something
<bigjools> wgrant: yeah I grepped that dir and there's no file with that apt config directive
<Mez> bigjools: I think he means to add it to the file :D
<wgrant> bigjools: Oh, by 'drop in' I mean put in, drop drop from.
<wgrant> Sorry.
<wgrant> Gah.
<wgrant> s/drop drop/not drop/
 * Mez hugs wgrant 
<wgrant> Currently downloading Launchpad at 0KB/s
<bigjools> ah
<bigjools> thanks wgrant
<bigjools> wgrant: do you know if there's a way of retroactively removing stuff only installed by Recommends ?
<wgrant> bigjools: A good question. I've wondered that myself, but never enough to actually go looking hard.
<wgrant> It should be possible, as aptitude and apt-get know what was installed explicitly.
<wgrant> And they can work out what isn't used any more.
<bigjools> I quite like the idea of shrinking my VM images
<awilkins> bigjools: One of the best ways I found of shrinking VM disks is to zero the empty space and repack the disk
<awilkins> bigjools: That works well for VirtualBox, anyway
<wgrant> I hate to think how slow this will be once it makes /.
<bigjools> no kidding
<awilkins> It's encouraging in a way ; it says that many many people are keen to put up their own Launchpad instances
<awilkins> (myself included)
<bigjools> awilkins: I will try that, hopefully it will work with images build by vmbuilder
<wgrant> Or is this obscene slowness because of that fix to make lp:launchpad resolve only to HTTP?
<bigjools> that should speed it up
<al-maisan> hello flacoste, I need an rc for https://code.edge.launchpad.net/~al-maisan/launchpad/apapi-fix/+merge/9089
<deryck> flacoste, I also have a branch I need rc for, when you have time.
<flacoste> al-maisan: looking
<al-maisan> flacoste: thanks!
<VK7HSE> "This will take a while -- maybe a few hours to get everything, depending on your Net connection."    Not wrong !!! 3kbs average !!! :P
 * wgrant just broke the 100KB/s barrier for a second or two.
<flacoste> does anybody knows what is the issue with the devel tests failure?
<gmb> flacoste: All I know is it was codehosting tests. I'm just pulling devel now to see what's going on...
<bac> flacoste: i'm on the blame list but it doesn't appear related to my change.  i'm looking into it too.
<flacoste> bac: i'm probably also on the blame list
<gmb> bac: There was a buildbot mail? I didn't see it...
<intellectronica> i also didn't get a mail from the bot
<VK7HSE> wgrant: maybe if we cross streams it might come down quicker!!! :P
<salgado> gmb, intellectronica, maybe it was sent to launchpad-dev@.  I didn't get that either and just noticed I'm not subscribed to that list
<gmb> salgado: I am, though... I think.
 * gmb checks the lp-dev archive
<gmb> Yep, I'm subscribed...
<intellectronica> no, i'm subscribed to that
<gmb> salgado: Nope. Didn't go there.
<gmb> Oh, wait
<gmb> bac and flacoste will get it because they're on the blame list
<flacoste> gmb: jml sent an email saying that he got it fixed
<flacoste> i don't understand why the list wasn't notified though
<gmb> flacoste: "Bazaar upgrade branch being bounced, test failure in devel" or something else I haven't seen yet?
<intellectronica> gmb: oh, of course
<flacoste> gmb: that one
<gmb> Ah, right.
<intellectronica> flacoste: maybe it's moderated because gary isn't subscribed?
<flacoste> intellectronica: to the launchpad-dev list? possibly
<flacoste> intellectronica: is there an automatic forward?
 * gmb wonders why buildbot doesn't have its own email address, like PQM does...
<flacoste> otherwise, it should still be sent at the old list
<bac> gmb: interesting, the buildbot failure message only went to me, flacoste, and mwhudson
<intellectronica> flacoste: i don't think so
<wgrant> Slashdotted a couple of minutes ago.
<mars> good morning!
<intellectronica> but maybe it was changed in the bot
<intellectronica> wgrant: oh oh
<mars> wgrant, !
<bac> gmb: looking back to other failure messages it looks like the list gets one message and the blamers another
<flacoste> intellectronica: i don't think so
 * wgrant is barely half-way through the branch :(
<intellectronica> maybe we should create a torrent of trunk
<sinzui> barry, bac, Edwin, salgado: stand up in 2 minutes
<mars> intellectronica, interesting idea, whose key would sign it?
<mars> any losas around?
<intellectronica> mars: why do you need to sign it again? the revisions are already signed by pqm
<flacoste> mars: losas still hang on the internal channel it seems
<mars> intellectronica, just a knee-jerk reaction to distributing a tarball over bittorrent
<flacoste> i don't think there is a bzr torrent transport yet
<intellectronica> flacoste: no, i just meant a torrent of a tarball of devel
<wgrant> I imagine that would be a whole lot faster and easier.
<wgrant> Just need to publish instructions on seeding the shared repo with it...
<awilkins> +1 for a torrent of a tarball
<intellectronica> what's a good tracker for something like that?
<gmb> It's a cool idea... is there any reason it'd be infeasible?
<barry> sinzui: cry
<mars> intellectronica, if we put the tarball on our own DC servers, we can all grab a copy directly at max speed and start seeding it - get say 6 seeds, instead of just say yourself :)
<wgrant> Woah. Speed.
<mars> wgrant, good speed, or bad speed?
<wgrant> Like, 200KB/s. 100x better than normal.
<mars> that would be kfogels' doing :)
<wgrant> Broken again :(
<sinzui> EdwinGrubbs:bug  401974                        Create a milestone link icon looks broken
<ubot3> Malone bug 401974 in launchpad-registry "Create a milestone link icon looks broken" [High,Triaged] https://launchpad.net/bugs/401974
<mars> wgrant, what broke?
<wgrant> mars: Speed dropped back to single digits.
<wgrant> It's now fluctuating every couple of minutes between fast and slow.
<mars> ok
<wgrant> Which is better than the first hour, where it was constantly slow.
<mars> really, interesting
<mars> intellectronica, ^ guess that is why a tarball of trunk sounds like a good idea?
<intellectronica> mars: yup. and that's likely to increase as more people catch the news on /.
<flacoste> ok, testfix is being processed by PQM
<intellectronica> yay
<flacoste> db-devel is green
<hazmat> trying to checkout rocketfuel setup .. gives bzr error invalid http response.. expected a boundary.. on a pack file
<flacoste> so we should be all reset very soon
<wgrant> I see that buildbot is still private.
<mars> hazmat, ok, abentley, ping can diagnose that?
<mars> abentley, <hazmat> trying to checkout rocketfuel setup .. gives bzr error invalid http response.. expected a boundary.. on a pack file
<wgrant> Ah, good, I see you already have people on /. accusing you of only doing this because of Chrome OS.
<gmb> intellectronica, mars: If we can find a tracker then all we need to do is create a fresh branch of devel outside our repo, share that and write some instructions about seeding a fresh repo, right?
<al-maisan> hazmat: what's your bzr version BTW?
<hazmat> 1.17 just upgraded
<gmb> s/share that/tar and share that/
<abentley> hazmat: This is usually caused by misbehaving proxy servers.
<intellectronica> gmb: yes, i'm doing that. any ideas for a preferred tracker? something decent and legit, preferably
<gmb> intellectronica: No, I can't think of a legitimate one :)
<hazmat> abentley, quite possible.. i'm on a gov site.. trying from an external machine now..
<mars> intellectronica, ask the ubuntu guys for a tracker
<mars> intellectronica, they must have something for the release images
<intellectronica> mars: good point!
<greg-g> \m/
<mars> :)
<Ursinha> :)
<greg-g> Congrats to the now full-time Open Source devs :)
 * al-maisan bows :)
<al-maisan> greg-g: thank you
<Mez> lmao @ but #393596
<Mez> bug *(
 * Mez just wishes that he had working virtualisation on Karmic :D
<greg-g> al-maisan: thank you all for your hard work on LP over the years
<wgrant> Mez: No KVM?
<al-maisan> :)
<Mez> wgrant: I dont wanna put KVM on a desktop box.
<Mez> (mainly cause I'm too lazy to use it!)
 * Mez tries VMWare
<wgrant> Mez: Strange...
<wgrant> KVM is surely less of an effort/intrusion than VMWare.
<wgrant> It's not like Xen.
<Mez> wgrant: my experience has been that KVM has been a bit b0rked.
<Mez> (aka it decided that it wanted to set my system clock running at 1000 times faster than it should have.
<Mez> Any time I tried to login, it timed out.
<wgrant> Huh
<wgrant> WFM
<Mez> karmic 64bit ?
<wgrant> Yes.
<bigjools> Mez: AMD?
<Mez> FATAL: Error inserting kvm_amd (/lib/modules/2.6.31-3-generic/kernel/arch/x86/kvm/kvm-amd.ko): Operation not supported
<wgrant> Ah.
<wgrant> Intel here.
<bigjools> I have a solution for that, one moment
<Mez> Unable to complete install: 'Couldn't create storage volume 'Launchpad.img': 'cannot create path '/var/lib/libvirt/images/Launchpad.img': Permission denied''
 * gmb hand-cranks an ec2 instance
<gmb> Damn this thing is slow.
<wgrant> Mez: I'm sure you can fix that one.
<wgrant> gmb: Better than production Codehosting.
<Mez> hmm, seems I don't need kvm-amd
<gmb> wgrant: Codehosting is still up, which given the kicking it's taking is quite impressive :)
<mars> wgrant, we're talking with our sysadmins now - hope to have an alternative download soon
<wgrant> gmb: Indeed.
<wgrant> mars: Great.
<wgrant> Though I've so far downloaded 170MB of the supposedly 150MB branch.
<Mez> wgrant: awesomeness :D
<Mez> wgrant: do a lightweight checkout
<mars> wgrant, is that the sourcecode size?  And does it also have to grab the history?
<bigjools> Mez: options kvm_amd npt=0
<wgrant> mars: It of course has to grab the history, but /Getting says the branch is around 150MB. I presume the bzr HTTP transport is just a little inefficient.
<gmb> wgrant: It's ~150MB on disk when all's said and done. bzr also includes all the negotiation. Also, there's some inefficiency there at the moment for some reason.
<wgrant> gmb: Right.
<gmb> wgrant: It's bug 402114, FTR
<ubot3> Malone bug 402114 in bzr "too many http requests with 2a fetches" [Critical,Triaged] https://launchpad.net/bugs/402114
<bac> gmb:  you were discussing the missing buildbot failure messages.  was there any resolution?  i see they aren't going to the old or new list.
<wgrant> gmb: Thanks.
<gmb> bac: I don't remember there being any. flacoste might know
<flacoste> gmb, bac: emails are still sent to launchpad@lists.canonical.com
<gmb> flacoste: Ah... So why didn't it turn up?
<bac> flacoste: but they went missing last night?
<flacoste> bac, gmb: i don't know yet
 * gmb feels better about not knowing
<mars> flacoste, intellectronica, is rf-setup safe to run when you already have the lp-branches and lp-source-deps directory?
<mars> as in, it won'
<mars> 't try and fetch them again?
<flacoste> mars: it might do a pull
<flacoste> but that should be fine since everything will be there
<mars> flacoste, cool
<salgado> wgrant, 150MB is the Launchpad repo.  lp-sourcedeps has an extra half gig of stuff that comes from lp.net, I think
<allenap> gmb: Have you spoken to mars or flacoste about bug 402160?
<ubot3> Malone bug 402160 in malone "Collapsible sections don't expand in IE8" [High,In progress] https://launchpad.net/bugs/402160
<flacoste> nope
<gmb> allenap: Not yet.
<abentley> Mez: It is not a good idea to do a lightweight checkout if you don't have low-latency access to the branch (i.e. it's on your LAN or hard disk).
<gmb> allenap: Go for it if you want to :)
 * gmb is tied up with a bug import atm.
<allenap> flacoste, mars: All collapsible areas in LP are broken in edge for IE[678]. The cause seems to be in setStyle (in YUI), where it tried to assign undefined to a style object.
<allenap> flacoste, mars: I have a quick hackish fix for which there is a diff in the bug report.
<allenap> flacoste, mars: It's a diff against YUI, so I thought I'd check in with you in case you had some better ideas.
<mars> allenap, I looked at your fix - have you traced the stack to see what is setting the style to undefined, and change it to ''?
<mars> allenap, it is either setStyle(undefined) or setStyle(null) that is doing it
<allenap> mars: Can I trace the stack in IE? My blood pressure is already at about 120 bar.
<mars> barry, ^ didn't you encounter something similar in IE?
<barry> mars: i am a windows-free shop
 * barry doesn't have ie
<mars> allenap, does the suite still pass with your fix?
<allenap> mars: I haven't actually run it ;)
<barry> mars: ah, it helps if i read more scrollback
<mars> allenap, as a contingency, I'm +1 on patching YUI, will need flacoste's OK for landing it
<EdwinGrubbs> flacoste: can I get an RC for https://code.edge.launchpad.net/~edwin-grubbs/launchpad/bug-401748-ie8-new-project/+merge/9083    BTW, there is also a lazr-js branch for this bug which will need an RC after it is reviewed.
<barry> allenap: yes, there is an issue with this.  i will try to remember it
<mars> allenap, put an XXX in there :)
<jelmer> https://dev.launchpad.net/Testing seems incomplete, it only mentions running the testsuite on ec2. is there a way to run it locally?
<mars> a big one
<mars> make check?
<allenap> mars: :) Yeah, I have done. I think gmb was going to file a bug against YUI, but for now I've XXXed it to the bug above.
<mars> been months since I ran that :)
<allenap> mars: You mean in the LP tree? I don't think that'll even touch this code though.
<mars> allenap, you can patch the YUI in the tree, then run windmill.  your fix should be in there.  You can verify by looking in Firebug
<mars> to see the JS files pulled in
<mars> allenap, we bundle our own copy of YUI instead of pulling it from the Y! servers
<al-maisan> jelmer: yes, the LP test suite can be run on your local system as well; "make check" will do it
<allenap> mars: I've patched YUI in a lazr-js branch, so that's okay I think. I'll go and run Windmill. Is there a YUI test suite I can easily run too?
<mars> allenap, no, there isn't a suite for PR2
<wgrant> al-maisan: Won't that take many many hours?
<allenap> wgrant: Yes :)
<allenap> mars: Okay.
<wgrant> Somebody might want to document this.
<barry> leonardr: is it true that with lazr.restful POSTs must be to named operations?
<al-maisan> wgrant: it depends on your machine .. but, yes, it will take 2-3 hours
<mars> allenap, ok
<al-maisan> wgrant: there are plans afoot to parallelize it and/or make it run faster in other ways..
<leonardr> barry: yes, that's the only thing we use POST for
<leonardr> the other thing it makes sense to use POST for is constructors, but i haven't implemented that yet
<barry> leonardr: gotcha.  the latter was what i'm looking for, but for now i'm just using a named op of 'new'
<leonardr> barry: that's a good name for now
<mars> hi stub!
<barry> leonardr: cool.  glad it's on the radar
<barry> leonardr: also for factory functions, we always return u'' in the body, right?  so if i wanted to dump the json that comes back from a request, i have to check that len() != 0
<al-maisan> wgrant: https://dev.launchpad.net/FasterTests
<radix> I wonder how hard it would be to replace the root-owned-and-running apache with a user-run twisted.web server for development purposes.
<radix> apache is used for some vhosting and path-mangling, I take it? is there anything more tricky than that?
<leonardr> barry: well, technically if it's sending you an empty string the response type shouldn't be application/json, but we probably do that
<leonardr> so, yeah
<awilkins> Darn, they slashdotted it
<radix> awilkins: kfogel slashdotted it ;-)
<barry> leonardr: ah, good point.  i'll double check the type
<leonardr> but there's a slight possibility that we send you the quoted empty string, ""
<barry> leonardr: i think not, but i'll double check.  thanks
<mars> radix, I don't believe there is much more that that, since on .dev requests are processed by the Zope server after routing
<al-maisan> awilkins: hey, it's all part of the celebration ;)
<mars> radix, cool idea though
<mars> awilkins, we should have a copy of the source tree up on some faster servers in a few minutes.
<radix> mars: cool.
<awilkins> mars: Including the repo or just the sources?
<mars> direct HTTP download, as fast as an ubuntu release
<mars> herb, ^ ?
<herb> including the repo
<awilkins> You don't even need the sources to be in it if it has the repo.. you can just remove the tree and make downloaders check it out again
<Daviey> branching peaking at 1137KB/s \o/
<allenap> mars: The unittests in lazr-js all fail the same in my branch as in toolchain :) As in, not all unittests pass, which is unfortunate, but at least my change has not introduced any new ones.
<mars> allenap, in IE? or Firefox?
<mars> I should hope they pass in firefox :)
<allenap> mars: In FF.
<mars> ah
<mars> allenap, not fun, but at least it is not a regression there
<mars> allenap, tried windmill yet?
<allenap> mars: Yep, and the vast majority do pass.
<mars> ok, that is good
<mars> flacoste, ^ should we RC and push allenap's fix?
<allenap> mars: No Windmill yet. I'm just bringing a couple more generators online before I can risk that.
<mars> flacoste, big day, some people would probably enjoy having it
<mars> allenap, :)
<allenap> mars, flacoste: We need something, because otherwise IE users will be broken entirely tomorrow!
<mars> need to fix up "make jscheck"
<barry> allenap, mars iirc the proper invocation is foo.setStyle('height', null)
<flacoste> allenap, mars: merge proposal, please?
<flacoste> radix: apache is used for mod_rewrite and mod_ssl also
<flacoste> radix: two nice improvements would be to modify our setup so that the dev environment starts a local apache and postrgesl cluster
<allenap> flacoste: Coming up.
<abentley> flacoste: basically, it would be good if multiple instances could run on the same machine, i.e. two test runs.
<abentley> flacoste: or a test run and launchpad.dev
<flacoste> EdwinGrubbs: what is this alert thing?
<flacoste> EdwinGrubbs: we should use Y.log, or something like that
<flacoste> not an alert
<EdwinGrubbs> flacoste: ok
<flacoste> EdwinGrubbs: and can't you use a waits.forElement() or something like that instead of sleep?
<flacoste> abentley: agreed
<EdwinGrubbs> flacoste: yes, I can do that.
<gmb> allenap, mars The other option is that we make Y.lp.activate_collapsibles() defer to the old activateCollapsibles() function for IE users.
<gmb> flacoste: ^^
<radix> flacoste: hm, but is the mod_ssl really necessary for local development?
<gmb> But I think a fix to lazr-js is preferable.
<allenap> gmb: Or not run at all.
<herb> get your launchpad source code: http://people.canonical.com/~herb/
<gmb> allenap: Why not degrade only a little bit first :)
<flacoste> radix: well, it's good for regression testing
<mars> awilkins, wgrant, sourcecode is available at http://people.canonical.com/~herb/
<wgrant> herb: That's with the tree, not just .bzr? (it looks far too big)
<awilkins> Many thnks
<flacoste> radix: spotting insecure warning and such other things
<radix> ah
<allenap> gmb: Then we have two lots of code to kinda maintain.
<wgrant> I'm already 273MB through, so I'll see how it goes..
<gmb> True
<herb> wgrant: with the tree, yes
* herb changed the topic of #launchpad-dev to:  Launchpad Development Channel | https://dev.launchpad.net/ |  Please use #launchpad for support. | https://launchpad.net/~launchpad-dev | Get it: https://dev.launchpad.net/Getting | http://people.canonical.com/~herb/ | http://paste.ubuntu.com/
<mars> I guess we should update /Getting with instructions on the alternative sourcecode download, and how to use it?
<barry> leonardr: ping
<mars> herb, 625K/s - that's nice :)
<herb> mars: heh
<mars> ETA 6 minutes at that speed
<leonardr> barry, hi
<barry> leonardr: hi.  are we trying to keep all the webservice/api doctests in lib/canonical/launchpad/pagetests/webservice/ ?
<barry> leonardr: i'm working on an api to expose message moderation and am wondering where to put the tests
<leonardr> barry: yes, tests of the launchpad web service in particular go into that directory
<barry> leonardr: cool thanks.  btw, you know i'm taking over for rockstar on thursday?
<leonardr> barry: yes
<barry> cool
<mars> oh wow, herb, download speed is fluctuating
<mars> bursty
<sinzui> EdwinGrubbs: are you removing the bug count feature from the +series page? I can do it if you are not
<herb> mars: that's just me putting kinks in the hose at random times. :)
<mars> 9m 38s for the alternative source download
<allenap> flacoste, mars: It seems that EdwinGrubbs found the root cause of that IE bug, and has fixed it in https://code.edge.launchpad.net/~allenap/lazr-js/set-style-fix-for-ie-bug-402160/+merge/9095
<wgrant> How big is lp-source-dependencies?
<bigjools> wgrant: someone said a few hundred meg
<wgrant> (it actually downloads at a reasonably fairly constant speed)
<allenap> EdwinGrubbs: I was chasing that same bug, but didn't find the cause, so I'm really glad you did :)
<awilkins> Blimey, loom is a required plugin?
<sinzui> awilkins: I never use loom
<awilkins> No.. I meant there's a link for it in the launchpad "bzrplugins" folder
<bigjools> it's optional
<sinzui> awilkins: yes, but you do not need to use it. I do not
<awilkins> Heh, why's it not in the optionalbzrplugins/ folder like git then :-)
<sinzui> salgado: ready for our call?
<salgado> sinzui, yes
<beuno> flacoste, hi. I wants an r-c: https://code.edge.launchpad.net/~beuno/launchpad/open-the-world/+merge/9096
<EdwinGrubbs> sinzui: that would be great, since I'm still making changes to land my rc fixes. Do you want me to assign that bug and the sprite bug to you?
<sinzui> EdwinGrubbs: I think you should keep the sprite since you definitely can get an RC to land.
<sinzui> EdwinGrubbs: If my branch has issues, it can still be landed in the reroll
<herb> mars: any luck on updating /Getting to reflect the static files on people.c.c?
<mars> herb, haven't yet
<deryck> flacoste, ping.
<pkern> pkern@asterix:~/launchpad/lp-branches/devel$ make schema
<pkern> Missing ./download-cache.
<pkern> Developers: please run utilities/link-external-sourcecode.
<pkern> That's somehow not helpful.
<pkern> I did strange things I have to admit in order to avoid a re-checkout, but still...
<salgado> sinzui, all translations-related templates have been moved out of lp.registry
<sinzui> \o/
<pkern> Hm, well, the real error is:
<pkern> link-external-sourcecode: error: Parent branch not specified, and could not be discovered.
<bigjools> pkern: try "make" on its own
<sinzui> salgado: blueprints can move anytime next cycle
<pkern> So I confused it too much.
<bigjools> pkern: ah, you need utilities/link-external-sourcecode
<pkern> bigjools: What "parent" do I need to pass to that script?
<bigjools> is that not in the destructions?
<bigjools> pkern: a path to lp-sourcedeps
<bigjools> pkern: I take it you didn't run rf-setup to completion? :)
<bac> salgado: i just QAd bug 397148.  first try timed-out but the next succeeded.  marked it OK/OK
<ubot3> Malone bug 397148 in launchpad-registry "+adminpeoplemerge should be able to merge people without email addresses" [High,Fix committed] https://launchpad.net/bugs/397148
<pkern> bigjools: Indeed it didn't like me because it told me that my checkout and devel diverged and it needs merge.
<salgado> bac, thanks! do you have the OOPS for the time out?
<bac> salgado: sorry, no
<pkern> bigjools: I'll try again.
<bigjools> pkern: in your devel directory, you can try "bzr pull --overwrite"
<pkern> bigjools: Sounds funny.
<pkern> bigjools: Cool.
<EdwinGrubbs> flacoste: can I get an rc for https://code.edge.launchpad.net/~edwin-grubbs/lazr-js/lazrjs-bug-401748-ie8-new-project/+merge/9081
<flacoste> allenap: so EdwinGrubbs above fix is obsoleting what you wanted in?
<flacoste> hi deryck
<flacoste> beuno, EdwinGrubbs: done
<beuno> thanks
<awilkins> Damn, I love my new 10Mbit/s connection
<deryck> flacoste, hi, I'd like to get rc on: https://code.edge.launchpad.net/~deryck/launchpad/correctly-list-dupe-subscription-401779/+merge/9084
<deryck> flacoste, fixes a bug discovered during QA yesterday.
<flacoste> deryck: done
<deryck> flacoste, awesome, thanks!
<mars> herb, please check https://dev.launchpad.net/Getting, "Alternative Sourcecode Download"
<herb> mars: looks good to me.
<abentley> rockstar: chat?
<awilkins> I'm trying to do the whole rocketfuel-setup stage but primed with the revision data by branching herb's archive into lp-branches ; how long should that take? It seems to be stuck in an infinite loop at "Fetching revisions:Get stream source"
<kripken__> Is there a command to get just the last revision of launchpad? Instead of doing a bzr branch lp:launchpad which gets the massive history?
<awilkins> Ah, now it's saying "Insering missing keys"
<awilkins> Done
<mars> kripken__, you can either download the source from http://people.canonical.com/~herb/, or do bzr checkout lp:launchpad
<awilkins> kripken__: You could get the source tarball from the branch page, or get a stacked branch
<MarkusT> "make schema" fails with: The found YUI root isn't valid: /home/[...]/launchpad/lp-branches/devel/lib/canonical/launchpad/icing/yui/3.0.0pr2/build [...] Traceback (most recent call last):
<awilkins> Or get herbs archive which has a full history
<kripken__> thanks guys
<MarkusT>   File "buildmailman.py", line 15, in ?
<MarkusT>     from canonical.config import config
<MarkusT>   File "/home/[...]/launchpad/lp-branches/devel/lib/canonical/config/__init__.py", line 19, in ?
<MarkusT>     import ZConfig
<MarkusT> ImportError: No module named ZConfig
<MarkusT> make: *** [compile] Error 1
<MarkusT> Any idea?
<mars> MarkusT, rocketfuel-setup ran?
<MarkusT> Jup, no problems there
<joey> noodles775: cloak set
<bigjools> MarkusT: what does "make build" do?
<joey> noodles775: you may have to refresh with nickserv to see it
<al-maisan> MarkusT: I believe "ImportError: No module named ZConfig" is due to you not having run utilities/link-external-sourcecode
<bigjools> yeah that's more likely
<mars> hmm
<noodles775> joey: thanks!
<MarkusT> I'm now running setup again and will try to use your advise... just a moment....
<mars> bigjools, al-maisan, that isn't in the /Getting instructions
<mars> should it be?
<joey> noodles775: you can /msg christel and thank her :-)  I just own the cloak rights. :-)
<bigjools> mars: potentially, but you only need it if you shortcut the instructions in the first place
<rockstar> abentley, yeah.
<mars> bigjools, depends, I assume MarkusT was following the instructions
<MarkusT> mars: Yes, i did :-)
<al-maisan> mars: you typically run utilities/link-external-sourcecode after branching a development branch from (db-)devel ..
<al-maisan> i.e. later in the process
<bigjools> well
<bigjools> you need it in trunk as well, so it can be built
<mars> MarkusT, was that error in a branch you just created, or devel/?
<al-maisan> bigjools: I though rf-setup will take care of running utilities/link-external-sourcecode in the trunk ..
<bigjools> yes it does
<MarkusT> Ok, running Rocketfuel-setup again doesn't solve the problem. "Usage: link-external-sourcecode [options] [parent]", what do I need for parent or options? I'm not branching, just trying to get a first working system. Might it be useful to completely delete all files and start over again?
<mars> MarkusT, first, check the sourcecode/ directory in trunk/.  Do you see a collection of links there?
<awilkins> Hmm, I'm getting all these "inconsistent details in skipped record" errors from bzr while it's branching source-deps
<mars> abentley, ^ ?
<mars> flacoste, fixed the text on /Getting - the source-deps aren't in the tarball, just the history
<abentley> mars: This is because launchpad's trunk wasn't reconciled.  I was in the process of reconciling it, when kfogel's announcement came.
<mars> flacoste, ^
<mars> abentley, is that a problem?
<MarkusT> mars: Where's trunk supposed to be?
<wgrant> awilkins: They seem to be coming from bzr-svn and bzr-git.
<mars> MarkusT, ~/launchpad/lp-branches/devel
<wgrant> And those two are taking absolutely ages longer than any of the others.
<MarkusT> mars: Only have a Makefile there
<mars> MarkusT, ok, how about in ~/launchpad/lp-sourcedeps ?
<abentley> mars: It's not a serious problem, but it does mean that people will see errors like that, and when they run bzr check.
<MarkusT> sourcecode is empty, mars. ls
<MarkusT> download-cache	eggs  sourcecode
<awilkins> What I did ; mkdir ~/launchpad ; cd  ~/launchpad ; bzr init-repo --2a lp-branches ; bzr branch <herbs-folder> lp-branches/devel ; rm -rf lp-branches/devel ; ./rocketfuel-setup
<MarkusT> mars: So should I start over? Seems the easiest way .....
<bigjools> MarkusT: run rocketfuel-get and see what it does
<wgrant> Hmm, I wonder if the sourcedep pulling is so slow because it's upgrading to 2a.
<awilkins> Oops, stacktrace
<awilkins> When branching dulwich
<awilkins> SHA1KnitCorrupt
<mars> abentley, ^ ?
<MarkusT> mars: Using saved parent location: http://bazaar.launchpad.net/~launchpad-pqm/launchpad/devel/
<mars> awilkins, did it kill the download?
<awilkins> Does 2a even use Knits?
<MarkusT> No revisions to pull.
<MarkusT> Tree is up to date at revision 53.
<MarkusT> Could not find /usr/local/bin/sourcedeps.conf
<MarkusT> Updating sourcecode dependencies in rocketfuel:
<awilkins> mars: Yes, I think it did
<MarkusT>   NOTE: Each sourcedep may take a while; please be patient.
<MarkusT>   ALSO NOTE: You can ignore any warnings you see below about how
<MarkusT>   'You have not informed bzr of your Launchpad ID ...' etc.
<MarkusT>   See https://bugs.edge.launchpad.net/bzr/+bug/401617 for why.
<ubot3> Malone bug 401617 in bzr "'bzr branch lp:foo' says "You have not informed bzr of your Launchpad ID ..." in anonymous pull (dup-of: 346677)" [Undecided,New]
<ubot3> Malone bug 346677 in bzr "confusing message about launchpad-login, even when doing readonly access to a public branch" [Medium,Confirmed]
<MarkusT>     Sourcedeps: /home/thielmann/launchpad/lp-sourcedeps/sourcecode
<MarkusT>     Checking /home/thielmann/launchpad/lp-sourcedeps/sourcecode/bzr-git
<MarkusT> Couldn't find lp:~launchpad-pqm/bzr-git/devel/, skipping it. [...]
<abentley> mars: I do not know why.
<MarkusT> This goes on for all dependencies....
<mars> abentley, ok, thanks, just checking :)
<awilkins> Why are they all saying "Created new stacked branch referring to file:///home/adrian/launchpad/lp-branches/devel/."
<awilkins> What's devel got to do with the dependencies (in terms of stacking?)
<bigjools> MarkusT: hmmmm it might be because of what it says - "You have not informed bzr of your Launchpad ID"
<mars> bigjools, I think you can ignore those (like the message says :)
<bigjools> mars: I know, but he's getting them for sourcecode checkouts
<MarkusT> mars: I'm just starting over .... See you an a few minutes .....
<mars> MarkusT, ok
<mars> bigjools, that first line that MarkusT posted is odd: No revisions to pull.
<mars> <MarkusT> Tree is up to date at revision 53.
<mars> <MarkusT> Could not find /usr/local/bin/sourcedeps.conf
<bigjools> that's normal
<mars> ok
<bigjools> no revisions tp pull means his devel is up-to-date
<mars> bigjools, and the "Could not find /usr/local/bin/sourcedeps.conf" ?
<bigjools> I get that every time
<mars> ah, ok too then
<bigjools> I think it defaults to somewhere else
<wgrant> Oh.
<wgrant> rocketfuel-get is stupid.
<bigjools> the problem is the "Couldn't find lp:~launchpad-pqm/bzr-git/devel/, skipping it. [...]"
<wgrant> This explains why it's so horrifyingly slow.
<mhall119|work> I'm using Ubuntu 9.04, which has bzr 1.13, can I not clone the launchpad repo?
<bigjools> wgrant: patches accepted
<mars> MarkusT, you can save yourself the download of sourcecode when starting over if you save your ~launchpad/lp-branches directory
<awilkins> This part of rocketfuel-get makes no sense to me (the bit that's doing it) http://pastebin.ubuntu.com/223662/
<wgrant> The detection code for SSO/ShipIt is broken, so it tries to branch everything as if it's meant to share history with the main tree.
<bigjools> YES.  I have wanted to say that for 2 years.
<mars> MarkusT, sorry, I meant he download of the launchpad trunk
<wgrant> So it first branches devel to the dep's dir, then pull --overwrites the remote branch.
<wgrant> Causing an upgrade to 2a.
<wgrant> And a whole lot of copying.
<awilkins> Aha, all three of those comments were about the same thing
<wgrant> bigjools: Patches may come...
<joey> beuno: what ever happened to the front page badge idea for today's FOSS announcement?
<bigjools> wgrant: I am happy to be your sponsor
<wgrant> awilkins: The easy workaround is to preemptively create empty branches for each dependency.
<wgrant> That will speed things up.
<wgrant> And probably stop dulwich from breaking.
<wgrant> But let me fix rocketfuel-get properly..
<awilkins> It's treating everything as optional and trying to stack them on trunk
<mars> mhall119|work, no, you will need to upgrade to Bazaar 1.16.1 at minimum, and even better, Bazaar 1.17
<awilkins> I think it couldn't find sourcedeps.conf
<wgrant> awilkins: Not quite.
<mhall119|work> mars: any chance of that getting into the Jaunty repos?
<mars> mhall119|work, if you are on Ubuntu, you can use the Bazaar PPA to get them: https://launchpad.net/~bzr/+archive/ppa
<wgrant> awilkins: Oh, yes.
<mars> mhall119|work, the latest Bazaar, as a backport?
<awilkins> In fact, it said "can't find sourcedeps.conf on STDOUT"
<awilkins> I put the quote in the wrong place there
<mars> mhall119|work, I don't know, but the latest version will be in Karmic
<mhall119|work> mars: thanks
<flacoste> shit
<flacoste> there is a typo in rocketfuel-get :-(
<wgrant> Yes.
<wgrant> No $ before the optional.
<flacoste> right
<wgrant> That change was made just today, wasn't it?
<flacoste> yesterday
<joey> flacoste: cp rs=joey
<mars> flacoste, bigjools is sponsoring wgrant's fix, I believe?
<mars> joey, :)
<wgrant> mars: Requires signing of the contributors agreement and blah.
<mars> oh
<wgrant> I probably would have worked that out earlier if it wasn't 2:30.
 * wgrant shall retry rocketfuel-get, then go to bed.
<awilkins> wgrant: Is it because $(dirname"$0") returns "."  ?
<joey> flacoste, et al - I'm afk for a dr's appt. Ring my mobile if you should need me.
<wgrant> awilkins: No.
<wgrant> It's a nice missing $ in the 'if [ optional ]'
<wgrant> awilkins: 'optional' is of course always true.
<wgrant> $optional might not be.
<awilkins> wgrant: aha
<flacoste> herb: can you remove the launchpad tar ball from your home directory, please, we'll need to do another one
<herb> flacoste: what do you want it to contain?
<LordMetroid> :)
<flacoste> herb: rocketfuel-get is broken in it
<herb> flacoste: awesome.
<herb> flacoste: what revision does it need?
<flacoste> a yet unlanded one
<flacoste> i'm on it
<herb> ah-ha
<herb> ok
<flacoste> :-)
<pkern> bzr taking ages on bzr-svn is also normal?
<herb> flacoste: they're gone
<wgrant> pkern: That's this bug.
<wgrant> pkern: There's a typo in devel/utilities/rocketfuel-get
<pkern> wgrant: thanks
<wgrant> pkern: I just added the $, then reran lp-branches/devel/utilities/rocketfuel-get
<wgrant> It seems to be branching properly now.
<awilkins> Yes, I concur
<wgrant> So much faster without the 2a conversion...
<bigjools> sweet
<awilkins> No huge errors about "inconsistend details" too
<mars> flacoste, http://paste.ubuntu.com/223675/ ?
<LordMetroid> Hm, maybe one should join in on the development of this software... I love launchpad as it now and would gladly want to make it better
<wgrant> And hopefully no dulwich conversion crash.
<flacoste> mars: yes, that's what i'm submitting now
<awilkins> dulwich already branched OK here
<LordMetroid> Is there any formal process to become a developer or am I just to send in patches?
<awilkins> LordMetroid: The process for Bazaar is presently to submit a branch and merge proposal to launchpad ; I would expect that LP itself would probably have a similar process ?
<mars> LordMetroid, for now: https://dev.launchpad.net/PatchSubmission
<beuno> joey, kfogel  :)
<awilkins> A shame all those branches are conflicting formats
<awilkins> They could share a repo if they were all 2a
<wgrant> But there would be no benefit.
<pkern> And it's nice that it doesn't tell me what' it's doing when looking at bzr-svn...
<samgee> congratulations on making launchpad free software, guys!
<mars> samgee, thanks :)
 * awilkins does `make schema && make run`
 * wgrant is still two sourcedeps away.
<pkern> Yay, it actually continues now.
<awilkins> My ISP might be caching the packs
<awilkins> They have a transparent proxy
<samgee> btw, there's an error in https://dev.launchpad.net/Getting
<wgrant> awilkins: I have 320ms latency to LP.
<awilkins> wgrant: Ouch
<pkern> awilkins: If you actually use http and not bzr+ssh...
<samgee> it says the source tree is 150Mb, while it seems to be 150 MB
<awilkins> pkern: Didn't I see something above about reconfiguring the lp: responder to hand out http:// links for a while?
<awilkins> Darn. "No module named cElementTree"
<sinzui> EdwinGrubbs: how goes your branches?
<flacoste> jtv-afk: all branch submission should go to db-devel
<awilkins> Need to add that to the list of packages that rocketfuel-setup installs for python 2.4
<flacoste> awilkins: actually, it should come from the download-cache
<mars> flacoste, I don't have it in mine...
<flacoste> hmm
<flacoste> right
<flacoste> i thought it was part of elementtree
<flacoste> but probably not
<flacoste> i need to update the launchpad-developer-dependencies again :-(
<mars> flacoste, I think so.  I'm fetching it from /usr/lib/python2.4
<awilkins> At least this stuff is minor teething trouble
<awilkins> I've spent _weeks_ trying to get the build system working on a project I'm on, and it's frickin' Maven - it's supposed to be EASY
<awilkins> (this is a set of projects created by someone else - I'm not so slow as to have my own build system not working on me...)
<flacoste> uploaded a new l-d-d
<wgrant> flacoste: Just adding python-celementtree?
<flacoste> wgrant: yes
<awilkins> Is _pythonpath.py generated?
<flacoste> jtv-afk: stop submitting to devel!
<flacoste> awilkins: it is
<awilkins> flacoste: Cool, then it'll just pick up the change... which file is it, or is it one of the branches that rocketfuel-get fetches?
<flacoste> awilkins: it's a package in the ~launchpad PPA, give it a few minutes to be published
<awilkins> Righto
<beuno> flacoste, any ideas what this means?  http://buildbot.devpad.info/builders/db_lp/builds/550/steps/bzr/logs/stdio
<flacoste> beuno: trouble connecting to codehosting
<sinzui> herb: ping
<awilkins> flacoste: Does celementtree need an amd64 build as well
<beuno> ah, seemed to obvious  :)
<herb> sinzui: pong
<flacoste> awilkins: if you are on amd64, yes
<beuno> flacoste, so how do I kick this off again?
<sinzui> herb: can you make me an admin on staging to test some team merge fixes?
<flacoste> beuno: i need another fix in, so i'll handle it
<mars> the PPA should handle the amd64 build as well, shouldn't it?
<herb> sinzui: I can
<awilkins> mars: Yes, it's just taking it's time, I think
<wgrant> mars: Yes, but there's a large backlog.
<herb> sinzui: done
<wgrant> Somebody should bump the build score.
<wgrant> eg. herb
<beuno> flacoste, danke
<awilkins> The packages for launchpad-dependencies are only listing i386 builds as well
<bigjools> wgrant: build URL?
<wgrant> Although all of those buildds are going to be occupied by their current builds for at least an hour yet, I expect.
<bigjools> I can bump it
<flacoste> awilkins: actually, it doesn't need an amd64 build
<flacoste> it's an indep package
<flacoste> only deps
<flacoste> and i'm not building python-celementtree itself
<flacoste> that's part of universe
<awilkins> flacoste: Aaaah. Ok.
<wgrant> bigjools: https://edge.launchpad.net/~launchpad/+archive/ppa/+build/1127437
<awilkins> There's a thing to fix in LP then :-) - misleading arch tags on package builds
<bigjools> wgrant: sorted
<bigjools> build start: 1 minute
<pkern> * It appears your search path is unconfigured.
<pkern> 	Have you read <https://launchpad.canonical.com/DatabaseSetup>?
<pkern> a) nobody told me and b) even if I would try...
<wgrant> bigjools: It doesn't take into account currently building builds?
<awilkins> Heh, should be Build score : "one MEEELION points"
<bigjools> wgrant: yes, it does
 * wgrant should have checked the code.
<bigjools> :)
<mars> pkern, looking now
<mars> pkern, on which page did you find that link?  So I can re-link it with the new material
<pkern> make[1]: Entering directory `/home/pkern/launchpad/lp-branches/devel/database/schema'
<pkern> mars: There.
<mars> hmmm
<mars> ok, first I'll copy the page
<pkern> I wonder why that search path is needed anyway.  Storm not supporting schemas?
<EdwinGrubbs> flacoste: have you gotten a chance to look at https://code.edge.launchpad.net/~edwin-grubbs/lazr-js/lazrjs-bug-401748-ie8-new-project/+merge/9081
<EdwinGrubbs> sinzui: their progressing pretty well, but I'm kinda derelict on my chr duties.
<mars> pkern, https://dev.launchpad.net/DatabaseSetup
<sinzui> EdwinGrubbs: I will free to start the CHR in 5 minutes
<sinzui> EdwinGrubbs: I'll start with projects and continue until you are available, which may not be at all
<flacoste> EdwinGrubbs: yes, it's approved already
<mars> pkern, but I don't guarantee that that page has an answer for you, I just brought it out into the light
<wgrant> So, I have my DB!
<EdwinGrubbs> flacoste: ok, I didn't see a comment in the mp, and I wanted to make sure that you weren't confused by the two mps with the same bug number.
<wgrant> But I first had to create a postgres superuser for myself.
<wgrant> And change the search path.
<pkern> * If this fails you need to run as the postgresql superuser
<pkern> * eg. sudo -u postgres make create
<pkern> There is no make create in there.
<pkern> Hm, well there is, but in the subdir, heh ;-)
<pkern> mars: It had, yes, thanks.
<pkern> It's only adjusting the search_path it seems, I didn't touch hba.
<flacoste> EdwinGrubbs: no, there was both a lp and a lazr-js branch
 * bigjools needs to run out, bb later
<pkern> Ok, you need to touch hba to allow launchpad_main to connect.
<wgrant> That makes sense.
 * gmb EODs. 
 * wgrant waits for his disk to catch up with postgres.
<mpt> wgrant, are you having fun?
<mars> wgrant, there is a good reason why some of us still use hefty desktop workstations instead of laptops :)
 * barry needs a new hefty desktop workstation :)
<pkern> psycopg2.OperationalError: FATAL:  database "launchpad_dev" does not exist
<wgrant> mpt: It's a bit late, and rocketfuel-setup is buggy, but otherwise it's not bad.
<pkern> psycopg2.OperationalError: FATAL:  database "launchpad_dev" does not exist
<wgrant> pkern: make schema didn't run properly?
<pkern> Hrm.
<wgrant> Mine certainly created launchpad_dev.
<pkern> wgrant: The problem is that now this stuff is owned by postgres I think.
 * wgrant is still waiting for make schema to finish...
<pkern> Well, let's try
<ryanprior> Thanks for choosing AGPLv3. :-)
<pkern> Oh great.
<pkern> psycopg2.ProgrammingError: permission denied for relation pg_database
<pkern> And as postgres
<pkern>     [Errno 13] Permission denied: '/home/pkern/launchpad/lp-branches/devel/eggs/test-easy-install-3674.pth'
<pkern> *sigh*
<mars> pkern, according to database setup, it just needs your db user to be a superuser
<mars> ryanprior, :)
<pkern> mars: It uses random own users to conncet.
<wgrant> pkern: Which are configured during make schema.
<pkern> wgrant: So which one needs to be superuser?  \-:
<wgrant> It started!
<wgrant> I haz a launchpad.dev.
<wgrant> pkern: The one matching your username.
<mars> pkern, "Your default PostgreSQL user has the same name as your login account (if it existed already, and it was not a superuser, drop it first)"
<mars> pkern, fwiw, stub is our database guru
<pkern> Right.
<flacoste> mars: the future is SSD drive, postgres doesn't have any trouble catching up here :-)
<pkern> "drop it first", heh... alter role $user superuser;
<wgrant> Are the usernames and passwords documented somewhere? I know a couple, but do I have to look through the sample data?
<mars> flacoste, nice :)
<rockstar> flacoste, and the future always will be SSD drives.
<flacoste> wgrant: they are in lib/canonical/launchpad/pagetests/README i think
<wgrant> flacoste: README.txt, but yes. Thanks.
<wgrant> Is there a root cert somewhere that I should tell Firefox to trust, or do I just whitelist the cert for each domain?
<flacoste> wgrant: you whitelist each domain
<flacoste> wgrant: i assume this means that you succeeded in running make run?
<wgrant> flacoste: Oh yes.
<wgrant> The webapp all works.
<flacoste> awesome!!!
<pkern> Works here too.
<flacoste> EdwinGrubbs: branches must be landed on db-devel
<wgrant> Once I'd fixed rocketfuel-get, installed python-celementtree, createuser'd, added the postgres search_path and auth stuff, it all worked.
<EdwinGrubbs> crap
<pkern> What code is running on drescher for the archive? :-P
<cody-somerville> gmb, What does this mean? "ConjoinedBugTaskEditError: This task cannot be edited directly, it should be edited through its conjoined_master"
 * pkern didn't need celementtree yet. o_O
<pkern> Or maybe I just had it :-P
<wgrant> I presume there's some docs on the internal wiki about setting up a local Soyuz system.
<wgrant> The Codehosting ones have been made public.
 * rockstar hates at DOM traversal
<wgrant> But not Soyuz.
<wgrant> rockstar: jQuery makes that so much nicer.
<flacoste> wgrant: actually, not really, Soyuz is that very mysterious secret guarded jealously by bigjools
<flacoste> and cprov
<flacoste> :-p
<flacoste> wgrant: seriously, i think nobody runs Soyuz locally
<pkern> bigjools: soyuz, soyuz, soyuz! how!
<flacoste> wgrant: i know they usually do their QA on a dedicated server aka dogfood
<wgrant> flacoste: Right. But presumably they at least run things manually locally...
<flacoste> bigjools: you officially have a request to document how to test Soyuz stuff locally
 * wgrant hears bigjools scream.
<mars> wgrant, that is a good list of roadblocks
 * mars tosses it into a Tomboy note
<barry> wgrant, pkern wait 'til we ask you to review some soyuz code!
<mars> I think we have rf-get in process
<mars> python-celementree is fixed?
<wgrant> mars: The first has a branch, the second has been uploaded.
<mars> don't know what is up with the database setup stuff though
<mars> wgrant, the first also needs a CP
<wgrant> Is the default batchnavigator size lower on launchpad.dev? I notice team membership lists are only 5 per page, not 50.
<matsubara> wgrant, yes
<matsubara> see default_batch_size: 5 in configs/development/launchpad-lazr.conf
<wgrant> matsubara: Aha.
<EdwinGrubbs> sinzui: can we delay our one-on-one call till tomorrow?
<barry> leonardr: all the pprint_*() methods in webservice.py require a json_body, but that seems tedious.  would it make sense to have a method that calls .jsonBody() automatically?
<sinzui> EdwinGrubbs: sure
<leonardr> barry: well iirc there are a number of methods for using different http methods, get, post etc
<leonardr> if you added a jsonbody you'd expand that x2
<leonardr> i guess you could just have a 'get' method, or an argument to the get method, that returned the body instead of the response object
<barry> leonardr: yeah, i see what you mean.  what does webservice.get() return normally?
<barry> leonardr: i wonder if pprint_() could sniff its argument.  ugly, but convenient
<leonardr> barry, it returns an httplib2 response object, or some kind of response object anyway
<wgrant> cody-somerville: A conjoined bugtask is the non-series task for a project which also has a task for its development series on the same bug - that dev series task is the conjoined master. While the dev focus task isn't Won't Fix, the non-series task cannot be altered.
<barry> leonardr: i.e. not a dict, right?  looks like json_body is expected to be a dict (or maybe a duck type having a .items() method)
<leonardr> json_body will always be a dict in lasr_restful applications, at least for now
<leonardr> barry, i have a very obscure python question for you
<barry> sure thing
<barry> leonardr: i'm using a new irc arrangement, so i can't tell if you're pvtmsging me or not ;)
<awilkins> Which bit downloads the packaged dependencies? It's not picking up the cElementTree egg yet
<mars> awilkins, apt-get install launchpad-developer-dependencies
<leonardr> let me try one recipe and see if it works
<leonardr> wow, ok
<leonardr> barry: nm, the thing i tried worked for now
<flacoste> mars, herb: the db-devel branch now contains the rocketfuel-get fix, we could republish another tarball using db-devel
<awilkins> mars: Aha, I was fooled by not seeing it in synaptic...
<barry> leonardr: cool
 * wgrant needs to sleep.
<wgrant> Thanks everyone.
<herb> flacoste: I'll take care of it as soon as I get back from lunch.
<herb> flacoste: what revision?
<EdwinGrubbs> flacoste: I know this seems rediculous, but can I get an RC for this, since it seems silly not to fix it. https://code.edge.launchpad.net/~edwin-grubbs/launchpad/bug-401974-icon-positioning/+merge/9102
<awilkins> Stupid question ; what's the root user on the default install?
<EdwinGrubbs> awilkins: the default install of what?
<awilkins> Launchpad, as produced by getting a development set of Launchpad
<awilkins> There must presumably be a "god" user in the database
<bac> awilkins: when you run a local copy for development at http://launchpad.dev there are a number of users in the database
<awilkins> bac: I agree, I can see them. But to really explore the features you can't just register your own, you need an admin user, yes?
<bac> look at lib/canonical/launchpad/pagetests  -- there is a README .txt and a REFERENCE.txt that list different test users
<EdwinGrubbs> runyaga: Hi, Alan
<awilkins> Excellent, thanks
<runyaga> EdwinGrubbs, how it goes! good job re: cleaning up and releasing launchpad
<awilkins> bac: Maybe that should go in the FAQ?
<mthaddon> awilkins: I think what you're thinking of is logging into an LP instance as a user that's a member of the ~admins team
<runyaga> really great
<bac> awilkins: yes, probably.
<mthaddon> awilkins: although off hand I don't know which of the sample data users are in that team - I guess the README might show that
<bac> awilkins: "Foo Bar" is a an admin for launchpad.dev and the one we most frequently use when testing.
<awilkins> Just looking
<awilkins> Yes, there he is
<awilkins> in README
<awilkins> I don't think I've configured email sending correctly yet, my puny attempts at registering myself failed
<leonardr> barry: if it's one to one, the mailing list should be a property of the team
<leonardr> and it will show up as /~guadamen/mailinglist
<leonardr> that *should* work now
<barry> leonardr: it's always one-to-one... thanks!
<barry> leonardr: i think i just have to export that
<sivang> congrets for the release! I knew this will come
<Ursinha> sinzui, I'd love to understand why ie8 shows that stupid continue button in the middle of the page
<bigjools> pkern, wgrant, flacoste: right, we *rarely* run soyuz locally, and in many cases it's impossible because you need a build farm.  you can run the publisher and the upload processor locally though.
<soren> Those bzr-1.17.tar.{gz,bz2} things at http://people.canonical.com/~herb/... They're not actually bzr, are they? > 160MB seems like a lot.
<sinzui> Ursinha: I will never be able to provide that answer. I so not have any IE browsers.
<soren> I'm guessing they're tared up launchpad trees, but the name suggests otherwise.
<salgado> soren, they could be bzr branches of the bzr project
<pkern> bigjools: How would I run e.g. the publisher, i.e. what's the wrapper to run scripts in the tree?
<herb> soren: it was bzr
<pkern> freenode hickup?
<sinzui> flacoste: may I have your RC for this change to fix the timeouts on +series pages: https://pastebin.canonical.com/20166/
<soren> herb: Oh, wow.
<awilkins> soren: It's bzr with the full history
<herb> soren: I had bzr branched from the bzr-1.17 branch and tarred it up.
<awilkins> soren: e.g. - includes a repository, not a just source
<herb> ok. new tarballs up with the fixed rocketfuel-setup
<herb> flacoste: ^
<beuno> sinzui, did you know about this: https://edge.launchpad.net/bzr/+download
<beuno> empty series showing up
<sinzui> I have not seen that
<sinzui> beuno: I think we can fix that at the same time we link them
<beuno> sinzui, should I assign it to wgrant?
 * beuno giggles
<sinzui> He can fix his own bugs now...
<sinzui> ...except that I suspect I made most of the bugs he reported
<beuno> shhhh
<cody-somerville> that was weird...
<sinzui> beuno: we are both karma whores. We have the most karma in the Launchpad Regisrty project. I enable him to report bugs, and he enables me to fix them
<cody-somerville> I just clicked a link on edge that sent me to code.launchpad.dev
<cody-somerville> the link seems fine now
<Ursinha> kiko, :)
<flacoste> sinzui: was your branch reviewed?
<sinzui> flacoste: yes, by bac
<flacoste> sinzui: do you have the m-p?
<sinzui> flacoste: https://code.launchpad.net/~sinzui/launchpad/series-performance/+merge/9100
 * rockstar hunts for nourishment
<leonel> Just a BIG  THANK YOU  !!
<bac> hi EdwinGrubbs, got a few minutes?
<mars> leonel, :)
<EdwinGrubbs> bac: sure
<bac> EdwinGrubbs: did you see my msgs?
<barry> bac: ping
<bac> hi barry
<pkern> So how will the launchpad wiki be opened?
<flacoste> pkern: it's opened already, dev.launchpad.net
<DoctorMO> Well done, it'
<flacoste> we haven't used the old one for a while now, edits to it have been closed for several months
<pkern> flacoste: Well, there are many references to the old one.
<DoctorMO> Now I can re-gain access to launchpad code :-D
<RichW> This right channel for launchpad crashes?
<pkern> Like https://launchpad.canonical.com/LibrarianTransactions
<flacoste> pkern: yeah, it's been gardened properly
<flacoste> many of the pages have actually been moved, but not the interlinks
<flacoste> some pages have not been moved
<flacoste> but many of these are obsolete anyway
<flacoste> s/it's been/it's not been/
<pkern> \-:
<pkern> Hah, scripts/ftpmaster-tools, oh well.
<rockstar> mthaddon, where can I find the loggerhead logs?
<DoctorMO> Hey herb
<herb> howdy
<mthaddon> rockstar: looks like there's an in progress RT to fix that (missing firewall settings to allow rsyncing)
<mthaddon> rockstar: although I think we're planning to move the machine into the LAN (currently in the DMZ) so it may need to wait til then
<capitandorko> 1
<rockstar> mthaddon, huh.  Well, I'm getting an unhelpful "Internal Server Error" on a user's branch, and I'd like to see why. What's the process for finding out?
<joey> (typo)
<mthaddon> rockstar: lemme check
<rockstar> mthaddon, I 'spect that mwhudson has super powers on the server itself, so it's not something that's been needed before.
<mthaddon> rockstar: bzr error or apache error?
<rockstar> mthaddon, bzr error.
<mthaddon> rockstar: so codehosting not loggerhead?
<rockstar> It's a loggerhead issue.  When I get it in local loggerhead devel, I usually can see the output from the running server in the terminal.
<rockstar> mthaddon, no, it's a problem with loggerhead's server.
<mthaddon> rockstar: ok, I see the debug.log - what should I be looking for?
<rockstar> mthaddon, something mentioning  http://bazaar.launchpad.net/~richies/hypernucleus-server/trunk/annotate/head%3A/hypernucleusserver/config/
<MarkusT1> Allright, I just removed all files and run the launchpad Setup again (according to the launchpad wiki). Still getting the same error, when running "make schema":
<MarkusT1> Traceback (most recent call last):
<MarkusT1>   File "buildmailman.py", line 15, in ?
<MarkusT1>     from canonical.config import config
<MarkusT1>   File "/home/thielmann/launchpad/lp-branches/devel/lib/canonical/config/__init__.py", line 19, in ?
<MarkusT1>     import ZConfig
<mthaddon> rockstar: https://pastebin.canonical.com/20172/
<MarkusT1> ImportError: No module named ZConfig
 * barry runs
<MarkusT1> make: *** [compile] Error 1
<MarkusT1> Any ideas?
<RichW> Missing a python module.
<Ursinha> MarkusT1, do you have the sourcedeps?
<Ursinha> if so, did you run the utilities/link-external-sourcecode script?
<MarkusT1> Ursinha: Could you be more specific? I was just running rf-setup (two times now, including a complete wipe between). "mars" already told me to run utilities/link-externel-sourcecode, but I was unable to figure out what to use for "options" and "parents" (Usage: link-external-sourcecode [options] [parent])
<bigjools> parent is the lp-sourcedeps dir
<Ursinha> MarkusT1, do you see a sourcecode folder inside lp folder?
<MarkusT1> ls -l
<MarkusT1> total 12
<MarkusT1> drwxr-xr-x  4 thielmann thielmann 4096 2009-07-21 19:35 download-cache
<MarkusT1> drwxr-xr-x 61 thielmann thielmann 4096 2009-07-21 19:50 eggs
<MarkusT1> drwxr-xr-x  2 thielmann thielmann 4096 2009-07-21 19:33 sourcecode
<MarkusT1> It's empty
<Ursinha> MarkusT1, use the pastebin, please :)
<Ursinha> it's empty because the links weren't created
<MarkusT1> So, how do I create the links?
<Ursinha> using the script :)
<MarkusT1> Which script?
<Ursinha> utilities/link-externel-sourcecode
<Ursinha> *external
<MarkusT1> So just "utilities/link-external-sourcecode lp-sourcedeps" ?
<bigjools> cd into the devel dir, then run it as "utilities/link-external-sourcecode ../../lp-sourcedeps
<Ursinha> MarkusT1, see, that's why I asked you if you have the sourcedeps
<Ursinha> what bigjools said :)
<MarkusT1> Ursinha: there's zero output and I don't think it did anything at all?
<bigjools> that's normal
<Ursinha> MarkusT1, if there's zero output it did nothing
<Ursinha> bigjools, for me it shows the links
<bigjools> oh no it it's not :)
<Ursinha> lol
<MarkusT1> So what do I do now?
<Ursinha> MarkusT1, what do you have inside lp-sourcedeps
<Ursinha> ?
<MarkusT1> As I said above: download-cache, eggs, sourcecode
<bigjools> what's in the sourcecode dir?
<MarkusT1> bigjools: Nothing
<Ursinha> that's weird
<bigjools> ok, rocketfuel-setup didn't complete fully then, can you run rocketfuel-get
<Ursinha> hm
<Ursinha> yes
<bigjools> it's in utiltities
<No1Viking> Got one question: Does Launchpad take care of version hirstory of source code or do I need to install subversion or something else?
<bigjools> utilities*
<mthaddon> No1Viking: we use bzr for versioning the code
<Ursinha> MarkusT1, please, run utilities/rocketfuel-get for it to finish downloading the dependencies
<awilkins> No1Viking: If you installed your own instance of Launchpad you already have Bazaar
<No1Viking> I have a Hardy server here. Will it configure apache and everything automagically?
<awilkins> MarkusT1: You might also have to update your launchpad-developer-dependencies package
<mthaddon> No1Viking: rocketfuel-setup will do, yes - be sure to read the comments as it's pretty invasive in terms of changes
<awilkins> No1Viking: Nope, you need Jaunty
<MarkusT1> Ursinha: Tried that before, doesn't work: http://pastebin.org/3454
<awilkins> No1Viking: Patches gratefully accepted
 * Ursinha looks
<mthaddon> awilkins: I'd be surprised if it doesn't run on hardy...
<awilkins> mthaddon: Would you have to install the deps manually though?
<MarkusT1> awilkins: apt-get says: launchpad-developer-dependencies is already the newest version.
<MarkusT1> awilkins: Ah, sorry. Needed an update.
<mthaddon> awilkins: I don't think so - there should be a meta-package for hardy as well
<MarkusT1> Will try that.
<Ursinha> bigjools, do you know what's MarkusT1 problem doing the rf-get?
<gmb> awilkins: It runs on hardy just fine - I ran it in a Hardy chroot for quite some time and the servers are running Hardy.
<bigjools> looking into it now
<No1Viking> awilkins, at the site, https://dev.launchpad.net/Getting, it says the following: "Note that right now, Launchpad can only be built and run on Ubuntu (8.04 "Hardy" to 9.04 "Jaunty")."
<awilkins> Ok, fine
<awilkins> Ah, they updated that, I think
<awilkins> It said Jaunty a while back - pages are in flux :-0
<awilkins> Or I'm just going mad
<joey> mthaddon: Interesting. I just got this from sucking down RF get: "Got a 200 response when asking for multiple ranges, does your server at bazaar.launchpad.net:80 support range requests?"
<No1Viking> I hope so, cos I will try to install it tomorrow
<mthaddon> joey: no idea I'm afraid
<joey> mthaddon: yeah, I think that might need to go to jml / thumper / mwhudson
<mwhudson> morning
<No1Viking> Last Q: Will it be possible to have all programming languages in it?
<joey> oh hi mwhudson
<mthaddon> No1Viking: in what sense? You mean to host a project that's written in any language? definitely, yes
<gmb> No1Viking: I don't understand the question.
<awilkins> No1Viking: Files are files.
<bigjools> MarkusT1: what happens if you run: bzr info lp:~launchpad-pqm/bzr-git/devel/
<No1Viking> mthaddon, thanks fÃ¶r the answer
<mthaddon> sure
<bigjools> joey: proxy server in the way?
<joey> bigjools: none. I'm doing -Dhpss to see if I get anything interesting
<bigjools> joey: might be a transparent one
<No1Viking> Good luck all. I'm happy with Ubuntu and I have it here in my house and we're getting it at work too. Thanks for your help all of ya!
<joey> bigjools: never stopped me before :-)
<bigjools> joey: the bzr guys were saying earlier you get that error with bad proxies, anyway
<MarkusT1> awilkins: updated launchpad-developer-dependencies, ran and link-external-sourcecode and rf-get again. Problem persists.
<MarkusT1> bigjools: http://pastebin.org/3457
<mwhudson> joey: what are you trying to do?
<joey> mwhudson: get rocketfuel-get the normal user way via the instructions on the wiki
<mwhudson> joey: what command are you running?
<bigjools> flacoste: can you check MarkusT1's paste,  http://pastebin.org/3454
<joey> mwhudson: as posted on the wiki plus debug "bzr -Dhpss --no-plugins cat http://bazaar.launchpad.net/~launchpad-pqm/launchpad/devel/utilities/rocketfuel-setup > rocketfuel-setup"
<bigjools> I've no idea wtf is wrong
<joey> mwhudson: see https://dev.launchpad.net/Getting
<mwhudson> joey: -Dhpss isn't going to help much for a http url
<mwhudson> joey: you could try -Dhttp though
<joey> the reason I'm doing this, btw, is just to QA the wiki :-)
<joey> mwhudson: I'll give it go
<flacoste> MarkusT1: did you fix rocketfuel-get?
<flacoste> MarkusT1: there is a typo in the script
<flacoste> MarkusT1: at least some versions of it
<MarkusT1> Well, after the first install failed, I deleted everything and downloaded again. I didn't fix anything by myself. Where do I find information about this fix?
<flacoste> MarkusT1: you got the tar ball? or the branch directly?
<MarkusT1> I'm just following the wiki
<MarkusT1> flacoste: https://dev.launchpad.net/Getting
<flacoste> MarkusT1: using "Getting It" or "Alternative Sourcecode Download"?
<flacoste> MarkusT1: you are running rocketfuel-setup, right?
<MarkusT1> flacoste: Getting it. Yes, I'm running rocketfuel-setup. Two times now. (Well, in fact I ran it more often to check if it solves problem "by itself" :-))
<joey> mwhudson: thinking of bigjools's comment, I also removed all my pre-existing locations info.  I still get http://paste.ubuntu.com/223833/ with no "real" error
<joey> bzr 1.17 as well
<flacoste> MarkusT1: ok, hang on, the fix is being commited to devel as we speak
<MarkusT1> flacoste: Wow, great. What should I do once it's commited?
<flacoste> MarkusT1:  once mthaddon gives us the go, you should just cd to ~/launchpad/lp-branches/devel and do a bzr pull
<flacoste> that should update utilities/rocketfuel-get
<flacoste> and you should be able to run rocketfuel-setup once again
<flacoste> (without deleting anything)
<joey> mwhudson: when I got to the URI in the wiki page, I get a not-found from codebrowse.  http://bazaar.launchpad.net/~launchpad-pqm/launchpad/devel/utilities/rocketfuel-setup
<mthaddon> still in progress, shouldn't be too long
<mwhudson> joey: yes, that's expected
<mwhudson> joey: can you paste a bit more of .bzr.log ?
<mthaddon> flacoste: revno 8971 of devel
<flacoste> mthaddon: thanks
<mhall119|work> is bzr branch lp:launchpad supposed to take so long
<mhall119|work> ?
<mthaddon> it's a pretty big branch, yeah
<flacoste> mhall119|work: yes, there is about 150M of data and it's really loaded now
<mhall119|work> wow
<joey> mwhudson: http://paste.ubuntu.com/223842/
<mhall119|work> okay, well I'm at 90M now, so may as well finish
<mhall119|work> is that all code, or what?
<mwhudson> joey: that's really strange
<sinzui> EdwinGrubbs: ping
<EdwinGrubbs> sinzui: pong
<joey> mwhudson: just thought of something... I have some ssh config settings
<joey> mwhudson: let me remove the LP ones
<sinzui> EdwinGrubbs: have you seen any floating radio button when you tested with IE 8? https://answers.edge.launchpad.net/launchpad/+question/77504
<mwhudson> joey: this is over http, ssh really shouldn't matter...
<joey> mwhudson: yeah :-)
<EdwinGrubbs> sinzui: that question doesn't mention radio buttons. It says "for an unknown raison a button", so it's probably what I fixed.
<sinzui> Which bug number is it so that I can link it to the question?
<flacoste> MarkusT1: can you try again?
<joey> mwhudson: I'm making you a new full Dhttp version of bzr.log
<flacoste> MarkusT1: so bzr pull followed by a rocketfuel-setup again
<EdwinGrubbs> sinzui: I linked it and added a comment.
<sinzui> thanks
<MarkusT1> flacoste: mthaddon: You're really great, guys! Pulled the update and now rocketfuel-setup is actually branching the missing files. Will see if "make schema" run's this time. Thank you so much for updating the tools so fast!
<sinzui> EdwinGrubbs: feedback@ emails are the only thing I have not done.
<sinzui> well and I never answered any IRC questions
<Ursinha> MarkusT1, :)
<MarkusT1> Ursinha: Thank you, too! :-)
<EdwinGrubbs> sinzui: I did those already. There was only one email that wasn't one of your project processing emails.
<Ursinha> MarkusT1, no problem :)
<rockstar> mwhudson, ping
<joey> mwhudson: full log http://paste.ubuntu.com/223845/
<EdwinGrubbs> sinzui: it really seems like feedback@ should use RT.
<sinzui> EdwinGrubbs: Then you wont have any CHR work untul next month
<mwhudson> rockstar: hi
<EdwinGrubbs> sinzui: thanks alot
<rockstar> mwhudson, do you have access to loggerhead logs?
<mwhudson> rockstar: yes, but if it's the problem with annotate it's fixed in rocketfuel already
<rockstar> mwhudson, okay, so it'll be out in the rollout?
<joey> flacoste: was the fix to RF related to launchpad-developer-dependencies?
<joey> s/rf/rocketfuel-get
<flacoste> joey: no, that was a separate issue
<joey> flacoste: ok but it's know. cool
<joey> I'll work around it then
<flacoste> joey: what's the issue?
<joey> flacoste: I got "Package `launchpad-developer-dependencies' is not installed and no info is available."  from rf-get but the pkg is there and I can install it by hand.
<mwhudson> rockstar: loggerhead has edge-style rollouts now, around about now i think
<flacoste> joey: that's a new one
<joey> flacoste: I think it's not a blocker, just a confusing situation.  http://paste.ubuntu.com/223857/
<mwhudson> joey: that's really odd
<mwhudson> joey: are there any proxies lurking?
<mwhudson> (i don't see any in the headers so they would have to be pretty evil ones)
<joey> mwhudson: that's what bigjools suggested. I'm direct connect to real-world and afaik there shouldn't be any
<joey> mwhudson: but it is possible something on my untangle box is getting involved
<rockstar> mwhudson, okay, great.
<joey> mwhudson: I have a webfilter :-)
 * joey goes off and tries to whitelist launchpad
<barry> jml: wouldn't it be cooler if bzr-tools-grep did a case-insensitive grep?
<mwhudson> joey: oh, well that's a little suspicious
<thumper> morning
<mwhudson> joey: can you run this command:
<mwhudson> curl --silent --trace-ascii log.1 --range 10-20,20-30 http://bazaar.launchpad.net/~launchpad-pqm/launchpad/devel/.bzr/repository/indices/c4f5f44fe87847349a73c775afac3555.iix > log.2
<Ursinha> sinzui, on ie8 it shows all the boxes, as if there's no hide/show arrow
 * Ursinha finally managed to make vmware work without borking everything else
<joey> mwhudson: done. log.1 is log, log.2 is bin
<sinzui> Ursinha: EdwinGrubbs ^ is IE 8 unsuable then?
<joey> mwhudson:
<joey> http://paste.ubuntu.com/223860/
<Ursinha> sinzui, no no, it's possible to use it because it apparently behaves as if js was disabled
<Ursinha> let me take a screenshot
<Ursinha> awww stupid me
<Ursinha> looking in the wrong field :( fail
<rockstar> thumper, we have a call in 10, right?
<thumper> rockstar: yes we do
<Ursinha> well, but the answer wasn't wrong after all
<mwhudson> joey: i think something must be dicking with your request en route :(
<Misterniark> hi
<sinzui> barry: EdwinGrubbs: Ursinha: can either of you ensure that the the inline edit widget is not shown in IE since it does not work
<rockstar> thumper, okay, hadn't seen you around, just making sure everything was still koshery.
<Ursinha> sinzui, it just doesn't work as expected, but you can type in the languages with text
<barry> Ursinha: what happens?
<Ursinha> sinzui, it's not shown, yes
<thumper> rockstar: I said "morning" :)
<Misterniark> scuse me, but make schema return a error
<thumper> rockstar: probably lost in the noise
<Misterniark> utilities/shhh.py python2.4 bootstrap.py --ez_setup-source=ez_setup.py \
<Misterniark> 		--download-base=download-cache/dist --eggs=eggs
<Misterniark> utilities/shhh.py make -C sourcecode build PYTHON=python2.4 \
<Misterniark> 	    PYTHON_VERSION=2.4 LPCONFIG=development
<Misterniark> make[2]: *** [download-cache] Erreur 1
<Misterniark> make[1]: *** [build] Erreur 2
<barry> Ursinha: do you have a screen shot, or a vm i can (attempt to) attach to?
<Ursinha> barry, let me check again
<barry> Misterniark: your error is way up higher than that, unfortunately
<Misterniark> somebody have had this probleme
<Misterniark> barry: sorry ?
<barry> Misterniark: sorry, i wasn't clear. ;)  look higher up in your output to find the actual error
<thumper> Misterniark: get the entire output and pastebin it
<parolang> If any of the launchpad devs are here, I want to congratulate you on the announcement.  I think this is as good as any of us could have hoped.
<joey> parolang: We're happy you liked it.  Isn't that right kfogel? ;-)
<kfogel> joey: I'm technically away having food, but yes, absolutely right :-).
<Misterniark> barry: thank for your help http://pastebin.com/m79581ae8
<kfogel> joey: (actually, sending the rollout downtime announcement first -- food next :-) )
<joey> mwhudson: I don't seem to get that except on that command. rf-setup seems to have run fine (despite two non-blocking issues) and rf-get is running at full speed.
<joey> mwhudson: heh, spoke too soon
<Ursinha> barry, sinzui, okay, so here's what happens
<Ursinha> barry, sinzui, you click in the exclamation icon, it redirects you to the +edit page
<barry> Misterniark: i'm sorry, but that's not enough to figure out what's going wrong :(  one thing to try is "make SHHH= build" and capture all the output
<barry> Ursinha: so that could mean several things
<barry> well, hmm. it seems like the edit icon isn't hooked up to the javascript
<barry> though i don't know why that would be
<DoctorMO> herb: So did the lp canonical irc channel move here, or is it still live?
<Misterniark> barry: thanks again http://pastebin.com/mc2a00bd
<Ursinha> barry, oh, isn't hooked up to the javascript?
<barry> Ursinha: well, if it were, it would let you edit inline, so ie is doing something dumb here ;)
<abentley> thumper: ping
<barry> Misterniark: did you run utilities/link-external-sourcecode?
<thumper> abentley: pong
<herb> DoctorMO: this is the dev channel for launchpad. some of the canonical-specific operational issues will probably stay on the lp canonical irc channel.
<abentley> thumper: chat?
<Ursinha> barry, this is how ie has behaving for some time with our inline edits
<thumper> abentley: just about to have a scheduled call with rockstar
<abentley> thumper: Okay.
<barry> Ursinha: ah, so it's not a problem specifically with the inline language editor, but a more general problem with lazr-js?
<Misterniark> barry: link-external-sourcecode : http://pastebin.com/m2975bdaf
<Ursinha> barry, maybe. for instance: if you click in the icon to edit the status of a bug, it redirects you to +edit-status page
<barry> Misterniark: something's wrong with your checkout.  did you run rocketfuel-setup, and did it succeed?
<Misterniark> barry: sorry but i dont understand anythink
<Misterniark> barry: yes !
<barry> huh
<barry> weird
<DoctorMO> herb: Ah like the deployment and such, makes sense.
<Misterniark> barry: an error on python -minimal at start
<Ursinha> +editstatus, sorry
<barry> Ursinha: yep, that sounds like a more general lazr-js bug
<Misterniark> barry: but ok after apt-get install -f
<barry> Misterniark: hmm. maybe you're missing some dependencies?
<barry> Misterniark: i haven't seen this problem before :(
<DoctorMO> herb: thanks :-)
<herb> DoctorMO: sure. you're welcome.
<Misterniark> barry: argg
<barry> Misterniark: what is the "python -minimal" error?
<DoctorMO> herb: And well done on the release, I just got the code myself as I've been busy writing this sys-admin course.
<barry> Misterniark: do you have the python-dev package installed?  you'll definitely need that
<herb> DoctorMO: I'm just an ops guy. :) Thank the other folks here for doing the hard work.
<Ursinha> sinzui, barry, so, do you want me to file a bug about it?
<DoctorMO> herb: Well, you all get credit :-D
<sinzui> Ursinha: yes please
<Ursinha> ok, a minute
<Misterniark> http://pastebin.com/d4c5fb583
<barry> Ursinha: yes, i think so.  on lazr-js project
<DoctorMO> herb: Although if you don't feel like you've done enough community work ;-) I'm sure peer reviewing my course would be most welcome :-D
<barry> Misterniark: i have to hand you over to one of the bzr guys at this point.  maybe thumper jml or mwhudson knows what that problem is
<Misterniark> barry: thank you
<Misterniark> barry: sorry for your time
<mwhudson> uhh
<mwhudson> that doesn't look right
 * mwhudson tries for himself
<barry> Misterniark: no worries.  i've been working for almost 14 hours now, so i think i'm done for the day ;)
<herb> DoctorMO: I'm not sure one can do enough community work. But the day job is keeping me busy lately.
<Misterniark> bye barry thank you
<DoctorMO> Just a coincidence that it's busy during this time. Anyway it won't be complete for a few more weeks.
<MarkusT1> OMG, it's actually working! Thanks again, folks! There was just one thing left: I needed to follow https://dev.launchpad.net/DatabaseSetup, since the "make schema" obviously didn't like my psql setup and my user was missing. But nevertheless, IT'S WORKING! Thanks again! :-)
<thumper> jono: morning
<bigjools> MarkusT1: great!
<thumper> jono: boy do we need you here :)
<flacoste> Misterniark: did you run rocketfuel-setup? or at least utilities/link-external-source?
<DoctorMO> thumper: community problems?
<thumper> DoctorMO: well, jono is a funny guy, and we like fun here :)
<jono> thumper, hey, in a meeting now, back soon - happy to help :)
<Misterniark> flacoste: yes , but it seems i have a probleme with python
<flacoste> Misterniark: what problem?
<DoctorMO> thumper: http://raisegrate.deviantart.com/art/Pirate-Penguin-Goes-Shopping-130282723
<Misterniark> i try to reinstall but apt say to me " no probleme buddy ! "
<flacoste> what did you try reinstalling?
<Misterniark> flacoste: http://pastebin.com/d4c5fb583
<flacoste> Misterniark: try going to ~/launchpad/lp-branches/devel and run bzr pull
<flacoste> then try again
<Misterniark> oki i try thank you
<Misterniark> flacoste: No revisions to pull.
<flacoste> Misterniark: what do you have in ~/launchpad/lp-sourcedeps/sourcecode?
<thumper> flacoste: it seems like it is a bzr-git issue, but not sure
<flacoste> well, it might
<flacoste> but i want to make sure it's not an artefact of the rocketfuel-get screw-up
<flacoste> it treated all branches as an optional branch
<flacoste> and stacked them on devel
<Misterniark> flacoste: yes i have
<flacoste> which of course would screw durwhich
<mwhudson> Misterniark: i don't get that error, is there a possibility you have a "transparent" proxy between you and launchpad
<mwhudson> ?
<flacoste> Misterniark: you mean, you have many directories in there?
<flacoste> mwhudson: i suspect it's an artefact from the rocketfuel-get error
<mwhudson> flacoste: oh, i didn't see that
<Misterniark> flacoste: notes and testes
<flacoste> Misterniark: ?
<flacoste> Misterniark: can you pastebin ls ~/launchpad/lp-sourcedeps/sourcecode
<Misterniark> mwhudson : hello , proxy between me and launchpad ???
<mwhudson> Misterniark: yes
<mwhudson> Misterniark: for instance, i have one thanks to my isp
<Misterniark> flacoste: http://pastebin.com/d6d87f01
<mwhudson> (luckily they seem to have fixed it recently, and it doesn't break so often any more...)
<flacoste> Misterniark: you only have bzr-git i assume?
<flacoste> Misterniark: there are no other directories in ls ~/launchpad/lp-sourcedeps/sourcecode
<flacoste> Misterniark: is that right? in that case, simply delete bzr-git and try running rocketfuel-get again
<Misterniark> flacoste: i have some error in output of rocketfuel-get
<flacoste> pastebin?
<Misterniark> http://pastebin.com/d357e5db8
<Misterniark> it so big to have entier output in my termianl
<joey> mwhudson, bigjools -  http://n1.netalyzr.icsi.berkeley.edu  confirmed a transparent proxy somewhere on my line
<mwhudson> joey: "transparent" heh heh heh
<beeman_nl> i get this error when running make schema: psql: FATAL:  Ident authentication failed for user "bbo01"  , i guess it's got something to do with Postgres authentication?
<Ursinha> sinzui, barry-away, bug 402736
<ubot3> Malone bug 402736 in lazr-js "Buttons seem not hooked up to js in IE8" [Undecided,New] https://launchpad.net/bugs/402736
<mwhudson> Misterniark: can you do this: cd /tmp; bzr get lp:~launchpad-pqm/dulwich/devel dulwich
<beeman_nl> i can give the complete output if needed
<mwhudson> beeman_nl: you need to do things to your postgres install
<Ursinha> sinzui, I thought we were disabling js magic on ie8, and that was on purpose
<mwhudson> beeman_nl: have you seen https://dev.launchpad.net/DatabaseSetup ?
<beeman_nl> mwhudson: oke, thanks. it was not on the system before launchpad install, so it's probably not configured
<beeman_nl> mwhudson: nope :) i just ran the commands from https://dev.launchpad.net/Getting
<mwhudson> beeman_nl: i guess we should mention something about this on there then!
<beeman_nl> i'll check the db setup page :)
<Misterniark> mwhudson: and after ?
<mwhudson> Misterniark: did it work?
<mwhudson> Misterniark: if it completed successfully then something a bit strange is going on
<Misterniark> same error
<mwhudson> Misterniark: ah, ok
<Misterniark> i start to zer0 tomorrow
<beeman_nl> mwhudson: it now continues, i ran "launchpad-database-setup $USER" from the utilities directory, so it's a good idea to mention it on that page :)
<Misterniark> so stange
<Misterniark> thank for helping
<Misterniark> its night for me, so good night
<beeman_nl> good night Misterniark
<mwhudson> beeman_nl: https://dev.launchpad.net/Getting?action=diff&rev2=18&rev1=16
<mwhudson> :)
<mwhudson> Misterniark: np, hope it works better for you tomorrow
<beeman_nl> mwhudson: that's great, thanks :)
<Misterniark> mwhudson: thank you
<sinzui> Ursinha: that is what I thought too. but I see there are IE 8 users reporting bugs. Since IE 8 was untested for that feature I asked you to test, I thought it best ti find out
<Ursinha> sinzui, I see
<sinzui> Ursinha: EdwinGrubbs investigate 1 issues yesterday that turned out to be 3. I think that since IE is buggy, we cannot easily turn it off
<Ursinha> hm
<MarkusT1> Might be a dump question, but is there a kind of superuser-login for the local branch? I can't find any references in ./config?
<sinzui> MarkusT1: do you mean you want to login as the test admin?
<MarkusT1> sinzui: Yes
<sinzui> MarkusT1:  foo.bar@canonical.com:test
<mwhudson> âMeasuring programming progress by lines of code is like measuring aircraft-building progress by weight.â <- good quote
<MarkusT1> sinzui: Thanks!
<mwhudson> from bill gates of all people??
<sinzui> MarkusT1: I think it is in one or two README files in the doctest or pagetests directory
<sinzui> I once measured my wife in light years
<sinzui> I think I also gave her weight in liters
<thumper> sinzui: please use admin@canonical.com:test
<thumper> sinzui: using foo.bar is obscure
<thumper> sinzui: which is why I added that "admin@canonical.com" email for Foo Bar
<MarkusT1> sinzui: Yeah, as I said: This might be a dump question. :-)  I'm still struggling with all the files and directories :-)
<sinzui> thumper: I will. I'll give an rs to the person that fixes it
<MarkusT1> God, "dump". I better get some sleep....
<sinzui> MarkusT1: I understand. The directly structure is better, but not good yet
<pkern> ILaunchpadCelebrities...
<Ursinha> sinzui, do you want me to test on ie6 and 7?
<greg-g> thought: shouldn't "Launchpad" be one of the featured projects on the launchpad.net homepage? :)
<sinzui> Ursinha: no, I am sure they are also broken. We need to be more certain that we are not giving IE an AJAX experience, or very certain it works for them
<Ursinha> sinzui, okay
<sinzui> greg-g: It may be when we release in a day
<greg-g> sinzui: cool
<joey> lol
<joey> greg-g: that's awesome
<greg-g> joey: :)
<greg-g> figuted it was something to feature
<joey> I'm going to do that right now. :-)
<joey> ooh bug
<joey> beuno: I think this one is for you.  https://bugs.edge.launchpad.net/launchpad/+bug/402747
<ubot3> Malone bug 402747 in launchpad "The requested URL /@@popup-window was not found on this server." [Undecided,New]
<joey> Ursinha: matsubara - could one of you triage that for me pretty please.  ^^
<joey> flacoste: fyi ^^
<matsubara> joey, sure
<joey> shouldn't be an issue anywhere 'cept there methinks
<joey> greg-g: your wish, my command.  launchpad.net/
<greg-g> joey: tight!
<greg-g> joey: nit-picking now: the 2 columns are now uneven by 2, thus could be exactly even. :)
<joey> greg-g: yeah, filing a bug about that :-)
<joey> greg-g: it's actually somewhat of a hack if you look at the code.
<joey> greg-g: when that has happened before we sometimes take off some of the older featured projects
<greg-g> joey: oh, wow, interesting.
<joey> greg-g: https://bugs.edge.launchpad.net/launchpad/+bug/402749
<ubot3> Malone bug 402749 in launchpad "featured projects list columns are uneven" [Undecided,New]
<sinzui> kfogel: ^ I think that bug is your call
<firefly2442> Has Rosetta (https://launchpad.net/rosetta) been released as part of Launchpad going GPL?
<joey> sinzui: yeah, I don't have access to the code atm (downloading) but I think there's a number there that tells us to split the display.
<joey> sinzui: I think *I* may even be responsible for it.
<kfogel> sinzui: 402749?  I mean, I'm all for having an evenly divided list.  Have no idea where that code is...
<sinzui> joey: The new design may have a fix for that problem. Well I tested the homepage with the new rules and they worked
<joey> oh cool sinzui
<kfogel> firefly2442: yes
<sinzui> kfogel: I believe someone with the right power needs to add or remove a project from one of the lists. I thought that power was conveyed to you
<wgrant> Morning.
<kfogel> sinzui: might well have been, not sure; can check
<kfogel> wgrant: morning!
<joey> kfogel: somewhere near  lib/canonical/launchpad/templates/root-index.pt
<kfogel> *nod*
<joey> iirc
<kfogel> okay, really really food now :-)  back later
<joey> sinzui: yeah it was unbalanced by one before. I added one just now and it's listing :-)
<sinzui> symmetry is overrated
 * pkern is said that LP doesn't ship with a GPG key included.
<pkern> *sad.  Because GPG key import doesn't work out of the box. ;-)
<wgrant> pkern: But you have psql....
<abentley> thumper: bzr+ssh://bazaar.launchpad.net/~abentley/launchpad/notify-oops
<pkern> wgrant: True
<rockstar> thumper, my internets - they eated themselfs
<pkern> wgrant: How do I run something in that custom python2.4 env?
<pkern> wgrant: I can hack sys.path in the script itself or supply a huge PYTHONPATH but both sounds wrong.
<rockstar> pkern, make harness
<rockstar> thumper, are you done, or can you call again?
<wgrant> bin/py, too.
 * pkern chuckles at bin/py
<awilkins> Is LP going to support wiki any time soon?
<awilkins> I saw the Trac migration kit and thought "cool, wiki"
<awilkins> But it's just bugs :-(
<jml> barry-away, re bzr-tools -- I'm not sure, I've relied on the case sensitivity before
<MT-> What kind of requirements does LP need to run decent?
<Ursinha> MT-, check the topic :)
<MT-> Ursinha: ?
<Ursinha> MT-, https://dev.launchpad.net/Getting
<MT-> Ursinha: that doesn't mention requirements though
<mthaddon> awilkins: I believe it's been discussed, but not sure if there are firm plans
<thumper> rockstar: we are done
<rockstar> thumper, okay.  I missed like 10 minutes of that meeting though.  Anything interesting?
<mwhudson> rockstar: it ended a while ago
<mwhudson> rockstar: you really didn't miss a lot
<awilkins> mthaddon: There's a bug ( #240067 ) but it's triaged
<pkern> wgrant: It looks you need to put it into some keyring though.
<rockstar> mwhudson, okay.  I assume just thumper waxing philosophical? :)
<jml> thumper, abentley, rockstar, mwhudson: sorry!
<awilkins> mthaddon: It's a "checklist feature" IMHO
<mthaddon> yep
<mwhudson> jml: morning
<awilkins> mthaddon: Not that it's a commercial product (although I can see a lucrative commercial support market)
<rockstar> jml, morning!
<Ursinha> MT-, so I guess it's missing somehow in the docs, but you can file a bug against launchpad-documentation project mentioning that
<abentley> jml: morning!
<Ursinha> as your first contribution to lp now opensource :)
<rockstar> Dammit, now I can't tab complete jml's name so easily anymore.
<jml> sorry.
<Ursinha> lol
<MT-> Ursinha: :P - will do
<Ursinha> thanks MT-!
<wgrant> pkern: Hm, indeed. What's broken about the web UI adding process?
<jml> rockstar, I've occasionally thought of assuming my old handle of mumak, but the rebranding would cost too much.
<jml> wgrant, good morning.
<wgrant> jml: Morning!
<rockstar> jml, mumak would work well for my tab completion needs at the moment.
<gmb> rockstar: Sadly, when in his mumak guise, jml has a tendency to trample the forces of good.
<MT-> Does Launchpad-FOSS allow us to create private branches?
<jml> gmb, well, I do that more subtly in this guise.
<pkern> wgrant: It takes strange things about keyserver.ubuntu.com
<pkern> *talkes.  Oh crap, it seems to late but I wanted to continue.
<jml> gmb, I guess being a mumak would be farewell to subtlety
<gmb> jml: Maybe you could just switch nicks whenever you're having a bad day, so we know not to make matters worse.
<jml> :)
<jml> "omg, quick, fetch the poncy elven archer"
<mthaddon> MT-: if you were running your own instance you could be admin and decide who gets private branches, yes
<gmb> jml: Exactly. I'm trying to think who might qualify for that role...
<MT-> mwhudson: cool - I think my company could really benefit from this - as well as contribute back to it
<jml> gmb, stub has the hair for it :)
<mthaddon> mthaddon != mwhudson  :)
<gmb> jml: Mind... boggles...
<thumper> [IRunnableJob(job) for job in jobs]
<mwhudson> jml: thank you for that image
<mthaddon> MT-: however, I suspect the overhead of running it in house might be less than the cost of a license to host proprietary branches on launchpad.net
<mthaddon> MT-: er, I mean, might be more...
<MT-> mwhudson: I was looking for requirements to run it. bug 402762
<ubot3> Malone bug 402762 in launchpad-documentation "Docs don't explain system requirements" [Undecided,New] https://launchpad.net/bugs/402762
<mthaddon> MT-: I don't know that you're really going to get much of an answer beyond "it depends" or "as much as you can throw at it"
<mwhudson> MT-: please learn the difference between mthaddon and mwhudson ! :)
<spm> MT-: ??? many of the LP Dev's run it on their laptops. that sounds fine to me for a single user.
<MT-> sorry...
<jml> spm, but we don't do anything _else_ while we're running it on our laptops.
<gmb> I occasionally cook my thighs.
<ajmitch> mwhudson: aren't all the launchpad developers just clones anyway?
<gmb> ajmitch: Only if the experiment went tragically wrong.
<MT-> would it freak out under 2GB RAM? or could it chug along?
 * spm lol's at the continuing confusion between mwhudson and mthaddon. Michael - you need to grow a beard :-)
<MT-> spm: I have one :(
<spm> MT-: that was at mwhudson :-)
<jml> ajmitch, close enough. we're all actually from the South Island of New Zealand, and thus spring from an extremely limited genetic pool :P
<wgrant> spm: Or was it noodles?
<ajmitch> jml: now that is worrying
 * mthaddon realises wgrant really does know everything...
<MT-> I was wondering if I could play with it in a VM if I give the VM 1.5GB RAM and 20GB HD
<firefly2442> Is there a way to browse the Launchpad sourcecode online instead of downloading the whole thing using Bazaar?
<mwhudson> spm: i'm more working on "not shaving" than "growing a beard"
<MT-> wgrant: does your brain ever hurt?
<mthaddon> MT-: you can definitely play with it on a regular laptop, but as for running it as a "production" service, that'd be a different story
<wgrant> The only problem I had on this new laptop is postgres' disk-eating ability.
<pkern> I think launchpad FOSS doesn't set up keyserver.internal or the keyserver in a sane way at all.
<MT-> my regular laptop has 2GB RAM
<MT-> I know some LP servers have over 100GB RAM
<gmb> firefly2442: http://bazaar.launchpad.net/~launchpad-pqm/launchpad/db-devel/files
<mwhudson> if you ahve a 32 bit install, 2 gigs should be plenty for playing around
<mthaddon> MT-: that's similar to most LP devs I'm guessing, and they run a local LP instance for development/testing fine (the 2GB RAM, not the >100GB)
<firefly2442> gmb: ty :)
<MarkusT1> MT-: I'm running it on 2GHz with 1.5 GB. It's not really fast with this setup (I would not want to use it on a regular basis), but I'm still running the debug environment. Not sure if that slows down that much.
<jml> good grief, 80 people in the channel.
<wgrant> My new T400 with 2GiB of RAM runs it fairly nicely.
<pkern> Mapping keyserver.internal to keyserver.ubuntu.com works.  Thankfully it then goes on to send mail to foo.bar@canonical.com (=
<wgrant> pkern: You altered /etc/hosts to point keyserver.internal at keyserver.ubuntu.com
<MT-> can one turn off certain features to save on performance?
<wgrant> ... 's IP address
<pkern> wgrant: Yes.
<MT-> I can't test yet - I'm installing Vista in a VM :(
<jml> jono, hi
<maxb> ooi, why does rocketfuel-setup request that you disable GSSAPIAuthentication ?
<spm> pkern: minor nitpick: launchpad FOSS is launchpad. we use the *same* code on the prod servers.
<mwhudson> maxb: hm, that's probably not required any more
<mwhudson> maxb: there was a bug that added 10s to the time to negotiate a connection with bazaar.launchpad.net
 * MT- quiets excited self. You all rock immensely.
<kiko> ;)
<MT-> kiko: love you too :)
<ajmitch> so can I safely ignore those errors like NoSuchRevision: KnitPackRepository('bzr+ssh://bazaar.launchpad.net/~launchpad/pygettextpo/trunk/.bzr/repository/') has no revision ('Arch-1:carlos.perello@canonical.com--2004%pygettextpo--devel--0--patch-5',) ?
<ajmitch> (from running rocketfuel-get)
<wgrant> ajmitch: Are you using a fixed rocketfuel-get?
<ajmitch> wgrant: no idea, I grabbed this yesterday & have re-run rocketfuel-get a couple of times
<ajmitch> it appears to be up to date
<wgrant> I don't believe I got those errors.
<ajmitch> it wouldn't surprise me if there was a proxy causing issues for me
<mwhudson> ajmitch: it's probably ignorable
<mwhudson> it's not like pygettextpo has changed in years
<ajmitch> such as SHA1KnitCorrupt
<mwhudson> ajmitch: if you have a proxy affecting your bzr+ssh connections, you have all kinds of problems
<ajmitch> mwhudson: one would think so :)
<wgrant> So only launchpad is a hot_product at the moment?
<maxb> aha, thanks. The expresses preference for apache worker mpm over prefork is also interesting
<mwhudson> wgrant: yes
#launchpad-dev 2009-07-22
<jml> ajmitch, those are bzr-ish errors, unlikely to be caused by a proxy
<lifeless> jml: well
<firefly2442> What are the programming languages used in Launchpad?
<jml> lifeless, NoSuchRevision caused by a proxy?
<lifeless> jml: if pack-names gets cached in violation of our request it would cause that symptom
<ajmitch> right, and I'd not really paid attention to the bzr+ssh:// part of that
<lifeless> neither had I:P
<lifeless> bzr+ssh definitely isn't a proxy
<jml> firefly2442, Python. Python. Python. :)
<lifeless> ajmitch: where are you seeing those errors?
<etank> i get this http://paste.ubuntu.com/223935/ when running make schema
<firefly2442> thanks
<ajmitch> lifeless: on checking various  branches in lp-sourcedeps, when running rocketfuel-get
<jml> firefly2442, but you can get the code & look for yourself. :)
<jml> so
<jml> how can we get the ohloh guys to upgrade their version of Bazaar?
<ajmitch> I'm just rerunning it at the moment after killing off the dulwich branch
<mwhudson> etank: there's an instruction in that error message
<wgrant> Ah, you got /p/launchpad. Good.
<mwhudson> jml: don't you have friends at sourceforge now?
<jml> wgrant, but we didn't get the name 'Launchpad'!
<jml> mwhudson, hmm. there's a thought.
<ajmitch> jml: send in the hordes of rabid lawyers?
<lifeless> ajmitch: its an error, or something logged in .bzr.log?
 * jml looks again at his beautiful diagram
<ajmitch> lifeless: an error with a traceback, just waiting on it to reproduce
<mwhudson> hey, it would be really nice if an error halfway through make didn't totally screw your working tree
<ajmitch> I've found it in .bzr.log if you want it on pastebin
<lifeless> sure
<etank> mwhudson: i did that and got 'link-external-sourcecode: error: Parent branch not specified, and could not be discovered.'
<ajmitch> http://paste.ubuntu.com/223939/
<mwhudson> etank: did you run rocketfuel-setup to start with?
<etank> mwhudson: yeah.
<ajmitch> at least dulwich branched without errors after a careful rm -rf :)
<etank> it took quite a few hours to finish
<mwhudson> though i have to admit, i can't get make to work either right now....
<mwhudson> etank: what's in sourcecode in your branch?
<thumper> jml: call?
<lifeless> ajmitch: file a bug please
<jml> thumper, sure. fire away.
<lifeless> ajmitch: it looks like that branch hasn't been upgraded and the converter is running into trouble
<ajmitch> ok
<wgrant> lifeless: So it was trying to convert to 2a?
<lifeless> on the fly yes
<lifeless> _fetch_batch
<lifeless> is part of IDS
<wgrant> That would be because rocketfuel-get was broken earlier, and was creating 2a repos before pulling sourcedeps.
<wgrant> Now it just checks them out, so no conversion is required.
<jml> thumper, try again please.
<mwhudson> ooh
<etank> mwhudson: just whatever rocketfuel-setup put there
<lifeless> wgrant: regardless, it should have succeeded
<mwhudson> etank: can you run python2.4 bootstrap.py --ez_setup-source=ez_setup.py --download-base=download-cache/dist --eggs=eggs
<mwhudson> etank: and see what that says?
 * gmb calls it a night. T'morrow, folks.
<ajmitch> lifeless: what else do you need on https://bugs.edge.launchpad.net/bzr/+bug/402778 ?
<ubot3> Malone bug 402778 in bzr "NoSuchRevision error when branching with rocketfuel-get" [Undecided,New]
<etank> mwhudson: it said nothing
<mwhudson> etank: and does plain 'make' work now?
<nellery> will it ever become possible for community contributors to earn commit rights?
<jml> gmb, g'night.
<mwhudson> nellery: good question!  probably
<mwhudson> nellery: though with a DVCS it's not that much of an issue
<ajmitch> nellery: given that you can commit & push branches already & most likely propose them for review & merging, I'd say yes
<lifeless> nellery: in the sense of committing directly to trunk? no - noone has that right even @ canonical - commits are done by a daemon that runs the test suite
<lifeless> nellery: in the sense of being able to submit something to land; as mwhudson says, probably :)
<ajmitch> there's info on the wiki about the steps to take for that
 * jml grumbles, the point of commit messages is to communicate
<MarkusT1> Is there some kind of Launchpad Roadmap available? I was looking for the wiki integration, but all I found is this blueprint (https://blueprints.edge.launchpad.net/launchpad-registry/+spec/launchpad-wiki). It seems everything it links to has been removed from launchpad. I was just curious if there already is a decision on this topic.
<mwhudson> jml: rather than defeating obscure regular expressions?
<mwhudson> MarkusT1: there's https://dev.launchpad.net/VersionThreeDotO
<kfogel> mwhudson, wgrant: what exactly was the rocketfuel problem?
<mwhudson> kfogel: i don't know exactly
<wgrant> kfogel: Which one? The 2a upgrade one that I referred to?
<kfogel> wgrant: "which one"... not reassuring words :-).  Yes, I meant the 2a one I saw in backscroll.
<MarkusT1> mwhudson: It seems it's quite empty. So I guess: no decision made on that issue.
<spm> mwhudson: obscure regular expressions are cause & effect by an obscure joke by obscure losas
<lifeless> kfogel: the sourccode branches aren't in 2a
<lifeless> kfogel: and for at least one of them converting on the fly errors
<mwhudson> MarkusT1: which issue, sorry?
<lifeless> kfogel: bug 402778
<ubot3> Malone bug 402778 in bzr "NoSuchRevision error when branching with rocketfuel-get" [Undecided,New] https://launchpad.net/bugs/402778
<ajmitch> jml: is this in reference to knowing what pqm is apparantly listening to?
<MarkusT1> mwhudsono: The question whether launchpad.net get's a wiki or not. :-)
<wgrant> kfogel: rocketfuel-get had a brand new bug which treated all of the source dependencies as if they were SSO and ShipIt. So, to save time, it first branched devel into the target directory, then pulled the dep. This caused the deps to get on-the-fly conversions to 2a.
<wgrant> Which is slow and crashy.
<kfogel> lifeless, wgrant: thank you, got it.
<jml> ajmitch, in particular, it's in reference to the heinous practice of saying rs=me or release-critical=me
 * thumper is trying to avoid the firehose that is #launchpad-dev :)
<thumper> MarkusT1: launchpad.net will get a wiki
<ajmitch> thumper: it's only a few people here causing the trouble :)
<lifeless> at least we're past the first hump
<thumper> yes, less spraying around now than before
<lifeless> I meant downloads actually ;P
<thumper> kfogel: should standup minutes go to the launchpad-dev mailing list now?
<wgrant> I keep getting intermittent connection resets when talking to launchpad.dev, and the local XML-RPC server seems to do it consistently when I attempt to resolve lp://dev/ URLs.
<MarkusT1> thumper: Great news! Is there any announcement on that? (Besides you telling me now :->)
<thumper> MarkusT1: there is much work to do for it
<thumper> MarkusT1: it is in the planning phase post 3.0
<mwhudson> thumper: "XXX on leave for next week" probably shouldn't go to a public list
<thumper> MarkusT1: we know what we want to offer, and no-one has done it yet
<thumper> mwhudson: good point
<kfogel> thumper: I think so.  If there's any company confidential stuff, it can go internal of course.
<jml> MarkusT1, https://bugs.edge.launchpad.net/launchpad-project?orderby=-users_affected_count
 * thumper looks at his notes again
<jml> MarkusT1, it's the single most popular bug on the entire Launchpad project :)
<kfogel> thumper: that is, the confidential stuff can be separated out and go internal.  But I think our standup notes should be like bzr's, which are sent publicly AFAIR.
<kfogel> jml: no, that's "cheddar"
<thumper> kfogel: ok, will do
<jml> kfogel, *blink*
<wgrant> pkern: Have you succeeded in running publish-distro yet?
<MarkusT1> As long as it's on the agenda I'm allright. You guys proved very well how you're able to handle this project. I found the bug report before, I just could not find any official comment on it. Thanks for sharing. This is getting better and better :-)
<wgrant> I presume it's just a matter of populating distroseries.lucilleconfig and the librarian properly, then running it in full careful mode.
 * thumper updates his trunks
<thumper> hmm, what's the chance I actually write any code before lunch...
<jml> https://www.ohloh.net/topics/3685?page=1#post_11061
<mwhudson> depends how strong your "ignore stuff" field is
 * mwhudson is coding
<mwhudson> (and gabbing while acceptance tests run...)
<pkern> I wonder how uploadpolicy is wired into the system, because that might screw with my tries to add a distroseries for Debian.
<wgrant> I might just manually process a PPA upload a publish PPAs for now.
<mwhudson> jml: yayness
<wgrant> Since the librarian is decidedly file-less, and I can't be bothered working out how to get gina to work.
<jml> wgrant, spiv has commented more than once in my hearing that the librarian is an ideal candidate for spinning off into a standalone app, fwiw
<kfogel> thumper: barry points out that standup notes can contain vacation times and other personal stuff, and that therefore they shouldn't be public
<kfogel> thumper: I think we're going to need to think about this more carefully; I will ponder and post a proposal, but for now do it internally I guess.  I think really we're going to have to have two versions: one with internal stuff and another with public dev stuff.
<thumper> kfogel: I checked my standup notes for personal details
<thumper> kfogel: I agree we should be careful with what we put in public notes
<etank> mwhudson: no make does not run
<etank> i get the same error
<mwhudson> etank: i can only suggest running the commands make is running one by one (without the ssh.py bit) and seeing what is breaking
<kfogel> thumper: but we will need a system that is documented and can be routinized.  We don't want team leads having to improvise this every time.
<thumper> kfogel: right
<wgrant> What happens with mail in devmode? I expected it to go to /var/tmp/launchpad_mailqueue, but it instead seems to be sent to root.
<mwhudson> well yes, it gets sent to root
<mwhudson> i once found the code that did that, i can't remember where it is though
<bz_g> ping
<mwhudson> oh wow
<mwhudson> from lp.registry.model.person import *\nAttributeError: \'module\' object has no attribute \'generate_nick\'\n'
<mwhudson> not managed that one before
<mwhudson> (it's a circular import, of course)
<bz_g> Did anyone try/manage to install launchpad on Debian (lenny)?
<jml> hmm. I seem to be getting conflicts every time I pull. :\
<jml> I guess that's because a directory change is moving through the trunk branches.
<jml> bz_g, I don't think anyone's tried yet, that I know of.
<mwhudson> it would be nice if different pyflakes warnings came out as different colours in flymake
<mwhudson> that sounds kind of hard to achieve though
<jml> mwhudson, interesting notion.
<jml> mwhudson, have you got pep8.py hooked up to flymake as well?
<mwhudson> jml: no
<jml> mwhudson, I do, and I'm quite liking it, but I do wish the warnings were differently coloured.
<mwhudson> jml: time to read flymake.el a bit i guess!
<jml> mwhudson, good luck.
<jml> Here is your elisp trowel. Use it wisely.
<mwhudson> it has different faces for errors and warnings at leas
<mwhudson> t
<jml> mwhudson, pyflakes doesn't really distinguish between the two though
<mwhudson> right
 * jml sometimes wonders if it should have some more reporting options.
<mwhudson> i guess it's a regexp somewhere in flymake to try to tell the difference
<jml> \o/
<mwhudson> heh
<mwhudson> it's a warning if the string "warning" appears in the output i think
<jml> and an error otherwise?
<mwhudson> ah, mildly more complex than that
<mwhudson> jml: yeah
<wgrant> Hmmmmm. The Soyuz karmaactions are in sampledata/current.sql, but not sampledata/current-dev.sql, so they were missing from my DB. Why would that be?
<spiv> mwhudson: you mean flymake is two-faced? :)
<mwhudson> spiv: zing!
<bz_g> jml: thanks.  Maybe I'll give a go...
<mwhudson> wgrant: the sampledata used for tests is not the sampledata that you get when you run make run and poke around
<mwhudson> wgrant: i don't know which is which any more :)
<jml> we want to reduce the sampledata used for tests
<wgrant> The missing karmaaction was crashing things :(
<wgrant> Yay, a successfully populated and published PPA.
 * mwhudson goes for lunch
<MarkusT1> I tried really hard to find the right documentation, but I still have a question left :-) : How do I actually delete projects and users in lp as the foo.bar admin user? I'm able to set projects inactive, but I can't find the "delete" button? :-/
<wgrant> MarkusT1: Projects are only ever deactivated.
<wgrant> They can't be deleted.
<wgrant> Same with people.
<MarkusT1> wgrant: Ah thanks. So at least I can stop searching :-)
<lajjr> congratz to launchpad-dev team \o/ .
<jml> lajjr, thanks :)
<lajjr> your welcome great source release. I can't wait to look at the code.
 * Snova_ would, but 280 MB is huge for this time of day
<lajjr> I download the tar.gz file and it is done, but the ./rocketfuel-setup started 10 mins ago.
<lajjr> I think that might be awhile yet :P
<wgrant> lajjr: Did you prepopulate the repository with the tar.gz's contents?
<lajjr> yep..
<wgrant> (also, rocketfuel-setup calls rocketfuel-get, which downloads lots more)
<lajjr> it will work right??
<wgrant> What is rocketfuel-setup saying it's doing?
<lajjr> downloading the deps apache etc.
<lajjr> fresh install on a new laptop Jaunty 9.04.
<lajjr> You can use the following commands to manage your Launchpad
<lajjr> development environment:
<lajjr>  rocketfuel-get
<lajjr>     Update your copy of devel and the necessary source
<lajjr>     dependencies, and make sure all source dependencies are properly
<lajjr>     linked in to all the branches you are working on.
<lajjr>  rocketfuel-branch foo
<lajjr>     Create a new branch of LP called "foo" in
<lajjr>     /home/lajjr/launchpad/lp-branches/foo,
<lajjr>     with all the source dependencies properly linked in.
<lajjr>  rocketfuel-status
<lajjr> wow I guess it's done.
<maxb> ooi, is there a summary of what couples Launchpad to a specific python version so much more so than other python code?
<lajjr> yep it just ran a list with that at the top..
<lajjr> rocketfuel etc etc rocketfuel status etc.
<wgrant> lajjr: That doesn't tell you much. What does ~/launchpad/lp-branches/devel/sourcecode have in it?
<jml> thumper, mwhudson, rockstar: there are still a few untested test plan items
 * thumper looks
<lajjr> ok hold on.
<lifeless> maxb: zope
<lajjr> lp-branches lp-sourcedeps
<lajjr> nd the script rocketfuel-setup
<lajjr> s/nd/and
<jml> maxb, we're very close to being able to switch to a more modern python.
<thumper> and we can't wait
<jml> which will be great for those of us who use karmic.
<jml> thumper, indeed :)
<ajmitch> jml: given the sort of people who are probably likely to grab a copy of the launchpad source, running on karmic may be a good thing :)
<jml> ajmitch, yeah
<jml> ajmitch, I'm running in a hardy chroot atm, which is working out ok
<ajmitch> it'd save me trying to dig around in makefiles to get simple things like bin/py
<jml> ajmitch, but I have a screwed up postgres config which is causing difficulties.
<ajmitch> I should probably setup something in kvm anyway
<lifeless> one consideration is that we deploy on LTS
<thumper> spm: is there going to be a staging update before rollout?
<lifeless> so we need to keep working there regardless of the current ubuntu development status
<mwhudson> maxb: i bet most blobs of python code as large as launchpad end up fairly tied to a specific python version
<mwhudson> jml: looking
<jml> mwhudson, thanks.
<jml> it takes more work than one might think to support more than one python version reliably.
<jml> guh.
<mwhudson> it's like 0.0001 units of aggravation per line of code or something
<jml> so distracted that I have a half written item in my todo list :\
<jml> mwhudson, what is the SI unit for aggravation anyway?
<mwhudson> jml: i don't know, i'm sure we can come up with something suitably offensive
<mwhudson> jml: the BSOD ?
<mwhudson> jml: "the openoffice" ?
<jml> hahaha
<lifeless> jml: I dunno, we do 2.4/2.5/2.6 quite easily in bzr
<lifeless> jml: the clippy
<jml> lifeless, I think that's the imperial unit.
 * mwhudson plays the "is my code on staging yet" game
<mwhudson> lifeless: have you seen http://www.ntk.net/doh/options.html ?
<jml> lifeless, I bet you had to change code to support 2.5 and 2.6.
<lifeless> jml: yes, but it was pretty painless, and we don't routinely have trouble with new code
<lifeless> mwhudson: classic, and no, I hadn't
<mwhudson> lifeless: one of the funniest things on the internet i think :)
<mwhudson> Annoy me with that sodding paperclib:
<mwhudson> ( ) all the time
<mwhudson> (*) when i least expect it
<spm> thumper: staging restore: Wed Jul 22 02:17:56 BST 2009 Lock file deleted. Script finished
<thumper> spm: when was that?
<thumper> like half an hour ago?
<spm> yup
<thumper> spm: but staging is revno 8298, and I need 8301 :(
<thumper> spm: why did staging come up with old code?
<mwhudson> erm
<spm> thumper: mainly as the actual code side happened about 12 hoursish ago
<mwhudson> is stable still being merged into db-devel
<mwhudson> ?
<thumper> :(
<thumper> mwhudson: it should be
<thumper> 1 failure on db_lp builder
<thumper> :(
<spm> thumper: I lie. code was updated ~ Tue Jul 21 21:27:44 BST 2009
<thumper> WTF?
<thumper> why is the doc test printing out javascript code??!?!
<mwhudson> lol
<mwhudson> i need to test the revision _after_ what is running on both staging and edge
<thumper> damn sprite!
<thumper> beuno: was that you?
<jml> mwhudson, my script says there's 1 revision in devel not in stable, and 4 in db-devel that aren't in db-stable
<jml> mwhudson, but that stable & db-devel are up to dae
<mwhudson> jml: cool, i was confused for a few minutes, but i've got it figured out now, i think
<lajjr> kernel update caused a reboot. lol
 * jml has a buildout moment
<mwhudson> jml: i made a branch earlier, buildout broke and the easiest thing to do seemed to be to throw away the branch and start again
<jml> :\
<jml> mwhudson, sometime after the release, we need to make sure that sourcecode/bzr never ever gets created an automated process
<jml> mwhudson, there's a great well of hidden errors just waiting to bite us there.
<mwhudson> jml: maybe there should be an engineer devoted to working on these things for an entire cycle!
<mwhudson> jml: we need to be careful we never import the system bzr somehow too
<mwhudson> (god knows how)
<jml> mwhudson, hmm.
<jml> mwhudson, we could make all of launchpad a bzr plugin & use bzr's api thing
<mwhudson> jml: i admit that wasn't a suggestion i was expecting!
<jml> mwhudson, :)
<lifeless> I likes it
 * mwhudson finds canonical.launchpad.scripts.logger.BufferLogger, finds it to be a piece of crap
<jml> :)
<kfogel> beuno: so what happened with your laptop yesterday?  Did you ever get it working again?
<kfogel> sinzui: I changed bug #386797 to invalid; hope that was okay.
<ubot3> Malone bug 386797 in launchpad-foundations "Enable distributed content development" [Undecided,Invalid] https://launchpad.net/bugs/386797
<sinzui> kfogel: not at all
<kfogel> sinzui: not okay?
<kfogel> :-)
<kfogel> sinzui: "not at all" is ambiguous in this context, but I assume you meant "no problem at all"
<sinzui> kfogel: it is okay. Francis would ignore it for a few years. I even suggested that he spend a day Triaging bugs because he could close 50 of them
 * jml instruments
<jml> closing 50 bugs would be nice
<sinzui> Since foundations was the old Launchpad, Francis could easily transfer 50 bugs to each our our porjects
<jml> that would be less nice
<kfogel> sinzui: I've been a bit hands off about triaging bugs, not knowing if I should be making decisions like that.  Should I be a bit more aggressive and assume that mistakes can be corrected?
<sinzui> We should be honest
<sinzui> If we really don't intend to do it, close it
<thumper> jml: how about: BranchCollection.getTeamsWithBranches(person):
<thumper>         """Return the Teams that person is a member of that have branches."""
<kfogel> sinzui: agreed.  It's not like the record of the conversation is lost if we close it anyway.
<sinzui> Every bug in blueprint, launchpad-answers, and launchpad-registry really should be fixed or implemented. I wish that was true of foundations
<spiv> kfogel: right.  The cost of a mistake there is pretty low.  The worst is probably offending the reporter, which is a pretty bad thing to do, but ignoring their report is probably just as bad...
<jml> thumper, what's the question exactly?
<thumper> jml: comments on a BranchCollection method to return a result set of People
<jml> thumper, ahh ok.
<thumper> jml: good or not?
<thumper> jml: it'd use the _getBranchIdQuery
<jml> thumper, my spider sense is tingling
<jml> thumper, why do you want it?
<thumper> jml: mostly, but tweak for owner
<thumper> jml: we need to be able to get the teams that have branches in a given collection for a person quickly
<thumper> jml: one query rather than one query per team
<thumper> jml: beuno is in 86 teams, and the query is taking 150ms ish
<jml> thumper, is this to generate links from the Person branch listing?
<thumper> jml: yes
<jml> thumper, I feel that it's something of a violation of the spirit of the interface, but I don't have any better ideas right now.
<thumper> jml: we already have branch collection returning branches and merge proposals
<thumper> jml: we could add a new utility somewhere
<thumper> jml: not sure what to call it though
<jml> thumper, returning branches is its raison d'etre, and merge proposals are a pretty natural follow on.
<thumper> jml: yeah, I know
<thumper> jml: but not sure how to get at the collection internals without it being a method on the collection itself
<jml> thumper, yeah. me neither.
<jml> thumper, I think just put it in there for now, making a comment about how it doesn't quite fit.
<thumper> jml: ok
 * jml idly wonders why 'reason for being' lacks punch in English.
<spiv> jml: "to be or not to be"?
<thumper> spiv: that isn't punchy either
<jml> instrumenting code :(
<jml> mwhudson, I wonder.
<jml> mwhudson, how would monkey patching and lazr imports interact, do you think?
<mwhudson> jml: lazy imports?
<jml> mwhudson, yes.
<jml> sorry.
<mwhudson> jml: i guess it could get exciting
<mwhudson> jml: does ScopeReplacer implement __setattr__ ?
<mwhudson> jml: i think i guess where you're coming from
<thumper> lazr imports?
<jml> thumper, lazy.
<mwhudson> jml: lp-serve not logging oops?
<thumper> jml: sorry, read that just after hitting enter
<jml> mwhudson, I can't hide anything from you, can I? :)
<mwhudson> jml: lazy imports --- day before release --- monkey patching --- talking about test plans earlier
<mwhudson> jml: it wasn't _that_ hard :)
<mwhudson> oh and this brain wave sensor
 * jml throws away his aluminium foil hat
<jml> I knew it. It *has* to be tin. This one's no good.
<thumper> jml: go get your cowboy hat :)
<jml> mwhudson, I now have solid evidence that the monkeypatched version is not being called.
<jml> mwhudson, what is this ScopeReplacer of which you speak?
<mwhudson> jml: it's an implementation detail of lazy imports
<mwhudson> bzrlib.lazy_import
<jml> oh. bzr-grep usage fail
<jml> mwhudson, it looks like it implements __setattr__ beautifully.
<mwhudson> jml: new theory required then
<jml> dammit. going to have to do this the Right Way.
<thumper> damn storm compilation problems
 * thumper smacks head on thetable
<jml> mwhudson, I think I have a new theory.
<mwhudson> jml: does something somewhere do "from bzrlib.trace import log_exception_quietly" ?
<jml> mwhudson, bzrlib.smart.protocol imports 'log_exception_quietly' directly from bzrlib.trace before it gets patched
<mwhudson> jml: arghlbargl
<jml> mwhudson, ...
<jml> mwhudson, so I can shuffle around some code, probably.
<mwhudson> statement: SELECT Branch.id, Branch.unique_name FROM Branch WHERE false AND Branch.private = false
<jml> mwhudson, or I can abandon this approach and do it the right way by giving bzr the appropriate hooks.
<mwhudson> that's not a very useful query
<jml> no, 'tisn't.
<mwhudson> i wonder why storm is generating it
<mwhudson> jml: i think the time may have come for the right way
<mwhudson> oh ffs
<jml> mwhudson, ok. I'll file bugs appropriately and prepare a patch for bzr
<mwhudson> jml: cool
<jml> mwhudson, the code I've already written in Launchpad was deliberately designed to be easy to switch to hooks
<jml> so I should only have to change one or two functions & probably none of the tests.
<mwhudson> jml: yay
 * jml -> lunch etc.
<mwhudson> http://pastebin.ubuntu.com/224061/ :(
<thumper> mwhudson: eh?
<mwhudson> thumper: circular import
<thumper> mwhudson: I think I know why
<mwhudson> yeah
<thumper> mwhudson: you need to import canonical.launchpad.database first
<mwhudson> import * is evil
<thumper> it blows
<thumper> that it is
<mwhudson> bug 402845
<ubot3> Malone bug 402845 in launchpad-registry "./bin/py -c 'from lp.registry.model import person' fails due to circular import" [Undecided,Triaged] https://launchpad.net/bugs/402845
<mwhudson> thumper: removing import *s from canonical.launchpad.database will help i think
<thumper> mwhudson: yeah, but likely to be a world of hurt
<thumper> mwhudson: it was a PITA when I did the code ones
<thumper> mwhudson: all the page tests :(
<mwhudson> thumper: maybe only a continent of hurt
<thumper> mwhudson: an Australia of hurt
<mwhudson> thumper: want to review https://code.edge.launchpad.net/~mwhudson/launchpad/much-faster-rewrite-map/+merge/9121 ?
<mwhudson> lifeless: you might be interested too
 * thumper looks
<thumper> awaiting diff
<lifeless> mwhudson: not really, unless its different than we discussed
<lifeless> mwhudson: does it nuke the cache?
<mwhudson> fair enough
<mwhudson> lifeless: what do you mean?
<mwhudson> it still caches
<lifeless> how fast is a miss now?
<mwhudson> but cache misses are done via 1 direct query
<mwhudson> lifeless: < 1 ms i hope
<lifeless> thats pretty good
<lifeless> I'd really like to nuke the cache
<lifeless> or get a stream of updates from the db
<lifeless> thats 300K RPS we can serve @ 1ms per query
<lifeless> sorry, 300K Requests/5minutes
<lifeless> only 7 times our peak yesterday :)
<mwhudson> lifeless: i imagine stub would like the database to be able to do something else as well as serve these requests
<mwhudson> i guess we'd only peg a core at worst
<lifeless> at 1ms each we won't even come close to pegging a core
<lifeless> even at yesterdays peak
<mwhudson> lifeless: i think there's a good case for a very short cache, because of ".bzr/smart, .bzr/branch-format, .bzr/branch/format"
<stub> If it is a problem, we can get a dedicated db for this. I'd also consider if memcache is a good solution - we know we need it in other areas, we just need to integrate it all into the test suite so it is easy to use.
<lifeless> mwhudson: I think theres a strong argument for truncating the mapping at .bzr
<lifeless> mwhudson: because everything to the right is bzr internals
<mwhudson> lifeless: what?
<lifeless> the rewriter doesn't need to map foo/.bzr/baz
<lifeless> only foo/
<mwhudson> right
<mwhudson> lifeless: maybe you should look at the code?
<lifeless> hmm, also backup.bzr
<lifeless> and .backup.bzr
<lifeless> mwhudson: perhaps
<lifeless> mwhudson: right now, I am trying to finish fixing iter_changes
<mwhudson> lifeless: also codebrowse!
<stub> I'm still unsure why we need to do substring matching (WHERE foo IN ('a', 'aa', 'aa/a', 'aa/aa') rather than splitting the path on '/' and looking for WHERE foo in ('aa', 'aa/aa')
<mwhudson> lifeless: fair enough
<lifeless> mwhudson: doesn't codebrowse map itself?
<mwhudson> lifeless: only when the request gets to codebrowse...
<mwhudson> stub: we don't
<mwhudson> stub: my code splits on '/'
<stub> ok
<mwhudson> stub: splitting on characters is actually wrong as well as just inefficient
<lifeless> mwhudson: if my looking at the patch will help you, I will
<lifeless> otherwise I trust in review :)
<mwhudson> lifeless: ok
<mwhudson> lifeless: i only suggested you looked (the second time) because you kept asking questions :-p
<lifeless> :)
<thumper> mwhudson: -> #launchpad-reviews
<stub> Getting rid of the cache or using a shared cache like memcache is needed to avoid weirdness when we scale horizontally - you don't want two different requests accessing different branches because they where served by different Apache's.
<stub> I guess we have that problem already though if someone renames a branch.
<mwhudson> thumper: i'm there
 * wgrant juat managed to convince his local Launchpad to accept a PPA upload, publish it, build it, accept the binaries, and publish them.
<spiv> wgrant: woo!
<ajmitch> wgrant: great, how much did that take to setup?
<ajmitch> & have you documented whatever's missing for the rest of us?
<wgrant> ajmitch: I've got notes here.
<wgrant> It took a while, as there are almost no docs around...
<wgrant> And there are some nice obscure bits.
<wgrant> One particularly time-consuming bit was working out why my builds weren't being dispatched. Apparently they will only be dispatched if there is a source published in the archive.
<wgrant> Which doesn't make too much sense.
<lifeless> well
<lifeless> it does, when you consider NEW
<wgrant> But we build from ACCEPTED now.
<wgrant> I have a suspicion as to why it's like that.
<wgrant> But I need to find the bug report...
<lifeless> ok, you're the expert :P
<wgrant> OK, no, it wasn't what I thought.
<Ursinha> just realized today it's exactly one year I joined lp team
<thumper> Ursinha: congrats
<jml> hello again
<Ursinha> thumper, thanks :)
<jml> Ursinha, grats.
<jml> Ursinha, time flies when you're having fun :)
<Ursinha> jml, thanks
<Ursinha> jml, well said
<Ursinha> I'll quote you in my blog post :P
<jml> haha
<jml> oh, that reminds me.
<jml> what's so funny about ultimo.fm?
<Ursinha> jml, lol
<Ursinha> jml, that was a bad translation, because last == ultimo in portuguese
<Ursinha> but last.fm isn't translatable... :)
<jml> Ursinha, I thought it might be something like that :)
<Ursinha> jml, :)
 * thumper goes to make veggie curry
<wgrant> ajmitch: http://williamgrant.id.au/f/1/2009/soyuzness.html
<wgrant> Some of that was done some time afterwards, so there might be issues.
<ajmitch> nice, thanks
<mwhudson> wgrant: wow, very determined of you
<wgrant> mwhudson: It looks harder than it is.
<jml> some of that should probably be turned into patches.
<wgrant> I suspect so.
<wgrant> A fair bit of stupidity is needed because the sample data is prehistoric, but I guess that's not easily fixable due to the tests.
<wgrant> But a lot of that can go into rocketfuel-setup.
<jml> wgrant, the sampledata used by the development config is separate from that used by the testrunner config
<wgrant> jml: Oh.
<wgrant> Useful.
<jml> wgrant, I think if you just get your dev environment set up with the right sampledata & run 'make newsampledata', you should be close to done
<jml> I think
<jml> it's been a while since I've touched the sampledata, as you can tell.
<ajmitch> current-dev.sql?
<jml> yeah
<ajmitch> I only took a brief look in there, so wasn't sure which to use
<wgrant> It seems I should have read database/sampledata/README. It explains all.
<jml> I'll gladly review & merge patches that make our docs better / more discoverable.
<wgrant> I might run make check tonight to see how long it takes on my laptop...
<ajmitch> wgrant: I take it you're doing all this on jaunty?
<wgrant> ajmitch: In a Jaunty chroot, yes.
<wgrant> Since that was easier than convincing LP to work on Karmic, I suspect.
<ajmitch> yes, I've tried a little with it on karmic, didn't get far & didn't really expect to anyway
<wgrant> have you tried it on Jaunty?
<wgrant> I think rocketfuel-setup should Just Work now.
<ajmitch> not yet, I'd rather set it up on a fresh apache & postgres
<wgrant> That's what schroot and KVM are for.
<ajmitch> yep, I just didn't have time last night after waiting for it all to download
<jml> I couldn't get Launchpad running on karmic either, fwiw.
<ajmitch> jml: various packages have had their python 2.4 support removed
<jml> yeah.
<wgrant> I didn't try very hard. Once I saw rocketfuel-setup fail to install launchpad-developer-dependencies from ~launchpad's PPA, I dropped into a chroot.
<wgrant> ajmitch: I'm not sure many more have in karmic than in jaunty.
<ajmitch> probably not many
<jml> well, I tweaked python-support to build for 2.4
<ajmitch> but I got to the point of it failing to do anything with bin/py
<jml> but not all of our deps use python-support
<wgrant> Doing it in a chroot is probably a good idea anyway, as it does like to put stuff everywhere.
<ajmitch> so I noticed
<ajmitch> I'll have to take care not to destroy the current postgres data I have
<jml> another thing we'd like to do is make it so we can run more than one instance of the test suite on the same machine
<jml> partly because it's a laudable goal in itself, partly so we can use more than one core
 * al-maisan has installed the LP development environment in a (hardy) chroot as well; works like a charm :)
<jml> al-maisan, my postgres config is a little dodgy still
<jml> probably my fault.
<al-maisan> jml: aha
 * wgrant now tries to work out how to use gina.
<jml> I think it's because my karmic postgres & hardy postgres want the same resources.
<al-maisan> jml: like ports..?
<jml> al-maisan, yeah, and pid files maybe
 * jml hasn't investigated.
 * mwhudson is EODing soon, shout if you want me to do anything today...
 * al-maisan installed the chroot according to https://wiki.ubuntu.com/DebootstrapChroot and has no issues with postgres
 * ajmitch would suggest supplying beer, but it's probably a little far
<al-maisan> jml: the PID files are kept here:
<al-maisan> CHROOT db-devel $ l /var/run/postgresql/
<al-maisan> 8.3-main.pid  .s.PGSQL.5433=  .s.PGSQL.5433.lock
<al-maisan> .. and the chroot has it's own /var partition
<al-maisan> the config files are here:
<al-maisan> CHROOT db-devel $ l /etc/postgresql/8.3/main/
<al-maisan> environment  pg_hba.conf  pg_ident.conf  postgresql.conf  start.conf
<jml> al-maisan, thanks.
<al-maisan> jml: hope it helps :)
 * jml moves closer to a power point
<stub> Anyone in this time zone who can give me a release-critical?
<jml> stub, no.
<jml> stub, kiko will be probably be awake soonish.
 * jml makes a note to change getAnyPocket to return something other than release
<wgrant> Is any particular HTML documentation generator favoured?
<jml> wgrant, for API docs?
<jml> wgrant, pydoctor for API docs, sphinx for other docs
<wgrant> jml: Right.
<wgrant> I don't see a make target, so I guess just run it manually?
<jml> wgrant, mwhudson wrote pydoctor, and might well have a recipe somewhere.
<jml> wgrant, probably run it manually -- I've never tried on Launchpad.
<jml> wgrant, I'd be very excited to see a make target though. I just added one to GTG.
 * jml pokes around a bit
<jml> there are up-to-date api docs online, just not public (yet)
<mwhudson> i guess i should move that onto people.canonical.com
<jml> mwhudson, is that the same thing as people.ubuntu.com?
<mwhudson> jml: yes
<wgrant> jml, mwhudson: Ah, I see.
<wgrant> They might be useful.
<wgrant> There's a lot of code.
<jml> you noticed :)
<jml> oh that reminds me, I should put bzr-tools.el on the wiki.
<jml> wgrant, 'make tags' can also help a lot for navigating the code
<kfogel> wgrant: have you built Launchpad yet?
<kfogel> (I might be missing backscroll, just got back from afk.)
<wgrant> kfogel: Oh yes, I had the webapp working after a bit over 3 hours.
<wgrant> I've even had Soyuz bits working today.
<noodles775> wgrant: nice!
<wgrant> And I'm currently trying to work out how to run a local keyserver.
<kfogel> wgrant: er, well, you took less time than I did to do that, and I work for Canonical.  Hats off!
<kfogel> hey adeuring
<noodles775> kfogel: but wgrant had already deduced most of the launchpad innards a long time ago ;)
<wgrant> noodles775: I ended up getting a full upload, build, publish working.
<adeuring> hi kfogel
<noodles775> wgrant: cool!
 * wgrant 
<wgrant> Oops.
<wgrant> noodles775: Any idea how I'm meant to run zeca/buildd-master? I've been running them manually with twistd for now - some other daemons are registered in canonical.launchpad.scripts.runlaunchpad, so are runnable with bin/run, but not those.
<jml> noodles775, hi :)
<jml> I just realized that by cunningly getting distracted in the middle of the day I'm in prime-time to get help from soyuz folks for this patch :)
<noodles775> wgrant: we don't (at least, I haven't) run the build master et al locally... for local development we usually just depend on the testing infrastructure (see SoyuzTestPublisher), and then test on dogfood...
<noodles775> wgrant: but cprov will undoubtedly know more.
<noodles775> jml: hi! Which patch?
<wgrant> noodles775: Hm, OK.
<jml> noodles775, bug 345737
<ubot3> Malone bug 345737 in launchpad-code "lp:ubuntu/package should resolve" [High,In progress] https://launchpad.net/bugs/345737
<jml> noodles775, sorry, bug 347768
<ubot3> Malone bug 347768 in launchpad-code "Allow anyone with upload rights to write to a package branch" [High,Triaged] https://launchpad.net/bugs/347768
<noodles775> wgrant might want to look too...
<noodles775> oh, you're already subscribed by default, so have probably already seen it ;)
<gmb> Good morning ladies, gentlemen and variations thereupon.
<jml> I'm not stuck yet. just poking around trying to find how to allow someone to upload to a package.
<wgrant> jml: archivepermission is the magic thing.
<wgrant> And they're manipulated on IArchive, at least through the webservice...
 * wgrant needs to poke around inside the code.
<noodles775> jml: why do you want to allow someone to upload a package? don't you just need to check whether someone has permission to upload a package?
<pkern> wgrant: Ah, thanks for poking there.
<al-maisan> jml: there are 3 types of archive permissions: based on components, source package names and (new!) package sets.
<wgrant> pkern: Where?
<noodles775> ... and I was about to say, al-maisan is the person to speak to there (having just implemented the new package set permissions)
<jml> excellent
<jml> al-maisan, so, my mission is to make it such that if you can upload to a package, then you can write to any branch officially linked to that package.
<noodles775> al-maisan: is it possible for us to abstract away the fact that there are currently 3 different ways? Or perhaps we already do?
<jml> al-maisan, to do this, I need an API for checking to see whether a person can write to a package
<jml> and a way of granting such permissions.
<al-maisan> jml: that API already exists
<jml> (the latter being for unit tests)
<al-maisan> let me find it for you.
<jml> al-maisan, thanks.
<pkern> wgrant: Into the upload rights code. ;-)
<wgrant> pkern: Ah.
<wgrant> pkern: What did you end up doing with Soyuz?
<jml> thumper, why is bug 243135 invalid?
<ubot3> jml: Bug 243135 on http://launchpad.net/bugs/243135 is private
<al-maisan> jml: IArchive.getPermissions() is the method you'd want to use.
<wgrant> al-maisan: Why not IArchive.canUpload?
<jml> or check_permissions(sourcepackage, 'launchpad.Edit') for that matter
<jml> al-maisan, thanks, looking now.
<al-maisan> wgrant: IIRC Archive.canUpload() only operates on components .. but let me look again.
<wgrant> al-maisan: component_or_package, it takes.
<al-maisan> wgrant: canUpload() eventually invokes getPermissions()
<jml> al-maisan, next question, why not check_permissions(sourcepackage, 'launchpad.Edit')?
<al-maisan> jml: let me have a look.
<jml> thanks.
<wgrant> There's no ArchiveSourcePackage, is there?
 * gmb achieves inbox 4, is satisfied
<maxb> Has anyone else struggled to make LP work on jaunty because of apt installing a broken python2.4 installation? It doesn't seem to have installed any of the python-support/central stuff for the new runtime, and the sys.path doesn't include any directories in /usr/lib !
<wgrant> maxb: I haven't seen that - I installed it in a fresh Jaunty chroot last night, and it worked fine (except for the other issues)
<maxb> hrm :-/
<gmb> maxb: How far have you got with trying to run LP? The LP buildout process creates a bin/py executable with the right sys.path. It also builds all the eggs that you need inside the Launchpad tree rather than polluting your site-packages.
<gmb> maxb: Try apt-get reinstall launchpad-developer-dependencies, see if that helps any.
<maxb> No, I think it's actually my system install of python that's broken
<maxb> python2.4 -c 'import sys; print sys.path'  is missing lots of things which should be there
<gmb> Hmm.
<maxb> even after purging and reinstalling the package
<wgrant> maxb: You even purged python2.4-minimal?
<maxb> yup
<wgrant> Huh.
<maxb> my thought
<maxb> meh
<maxb>  * my thoughts exactly :-)
<al-maisan> jml: check_permission() uses checkPermission() zope.security.management .. how does one configure or hook into the latter?
<jml> al-maisan, canonical.launchpad.security is scanned automagically
<jml> hmm. and maybe there's some zcml
 * al-maisan looks at ./lib/canonical/launchpad/security.py
<jml> there's IPackageUpload and IPackageUploadQueue in there
<jml> and launchpad.Append on IArchive
<al-maisan> jml: hmm .. it looks like the piece you'd like to have still needs to be added to lib/canonical/launchpad/security.py .. sorry
<jml> al-maisan, no worries.
<jml> al-maisan, I'd be happy to add it, but I'm not sure exactly what to make it do.
<jml> bigjools, hello
<bigjools>  /away
<bigjools> morning!
<jml> bigjools, we're just talking about fixing bug 347768
<ubot3> Malone bug 347768 in launchpad-code "Allow anyone with upload rights to write to a package branch" [High,In progress] https://launchpad.net/bugs/347768
<bigjools> ok
<al-maisan> morning bigjools
<jml> trying to figure out the best way of asking whether a person has upload rights to a package, and how to grant them said rights (for testing)
<bigjools> there's a few checks done in the upload processor
<bigjools> unfortunately there's no single call that will give you a yes or a no
<jml> bigjools, lp.soyuz.scripts.soyuz_process_upload being the starting point, right?
 * bigjools stabs Firefox and the xorg intel driver
<jml> bigjools, karmic upgraded the intel driver today. :)
<bigjools> jml: no, IArchive.canUpload()
<bigjools> let me refresh my memory and then I can talk more sense
<jml> bigjools, how do I get to that from an ISourcePackage?
<bigjools> and possible reboot
<jml> bigjools, cool, thanksn.
<bigjools> jml: well you need to know which archive you're uploading to
<bigjools> presumably it's based on a distro, do you can use IDistribution.main_archive
<bigjools> but that obviously only covers the case where you want to upload to, well, the main archive :)
<bigjools> argh, fscking FF and xorg both using over 1Gb resident
<wgrant> bigjools: Karmic?
<bigjools> jaunty
<wgrant> Ah. That would be your problem.
<jml> bigjools, that's probably what we want here.
<jml> bigjools, since these are for official package branches for a distroseries, pocket.
<bigjools> I tried the karmic kernel and xorg driver  a month aho and it was no better
<thumper> jml: because the tests aren't there any more :)
 * thumper thinks...
<bigjools> jml: great, so the next thing to work out is whether there are package-specific permisssions, component-specific permissions then packageset permissions
<jml> thumper, yeah. I checked.
<jml> thumper, and at their new location, the test modules have been correctly factored.
<jml> thumper, so it's a little more fix released than invalid.
<jml> thumper, but no big deal.
<al-maisan> bigjools: I understood jml merely wants a yes/no answer..
<jml> yes.
<jml> what al-maisan said :)
<bigjools> that's not possible
<bigjools> like I already said
<jml> ok.
<thumper> jml: changed to make you feel better :)
<jml> thumper, you're a champ. :)
<al-maisan> bigjools: assuming jml knows the destination archive, IArchive.canUpload() should provide the yes/no answer ..
<bigjools> al-maisan: no, it can't
<al-maisan> please explain.
<bigjools> you also need to know the package name, a component name or a packageset
<bigjools> you only don't need those if you have a PPA
<al-maisan> ah, right.
<jml> bigjools, we have the package name, since we have an ISourcePackage.
<bigjools> jml: that's a good start :)  but not enough yet
<bigjools> you also need to know which component it's headed for
<bigjools> and you have to check both
 * jml buckles up and sits tight.
<bigjools> so start with canUpload(user, package_name)
<bigjools> if it works, great
<bigjools> if it doesn't, you need to use canUpload(user, component)
<jml> bigjools, if I know the distroseries & sourcepackagename, can I can derive the component?
<bigjools> yes
<bigjools> you need to work out where it's published
<bigjools> that was where I was going next :)
<bigjools> jml: I'll point you at example code
<jml> how bout I just sit quiet for a bit then? :)
<bigjools> lib/lp/archiveuploader/nascentupload.py
<bigjools> verify_acl() does the upload checks
<al-maisan> bigjools: that's a great pointer!
<bigjools> jml: you need to know which pocket it's being uploaded to as well, you have a suite somewhere IIRC?
 * jml looks
<jml> bigjools, yep.
<BjornT_> jml: you can also look at BugNomination.canApprove(), which does similar check (except that it doesn't check for packagesets yet)
<jml> http://paste.ubuntu.com/224163/
<bigjools> jml: ok so you need to look up the SPPH for that package/pocket
<bigjools> BjornT_: you need to fix that for karmic :)
<jml> that's the test I'm halfway through writing.
<bigjools> jml: distroseries.getPublishedReleases() does that lookup
<BjornT_> bigjools: how about finally adding a method that gives you a yes/no answer? ;) this is at least the second time we implement these checks.
<bigjools> BjornT_: it's not possible
<BjornT_> bigjools: that is, second time outside of soyus
<jml> bigjools, and once I get the SPPH
<BjornT_> bigjools: why not. how can i and jml get a a yes/no answer if it's not possible?
<bigjools> there are three different things to check depending on what data you have
<bigjools> and which distroseries it is
<bigjools> (after karmic)
<bigjools> jml: once you get the spph, you can get spph.component, and pass it to canUpload()
<jml> bigjools, how will I know which element from getPublishedReleases to use?
<BjornT_> bigjools: both jml (i think at least) and i have an ISourcePackage. i.e., we have a source package in a distro series. all information can be derived from that, can't it?
<BjornT_> bigjools: if not, then i'm doing something wrong
<jml> BjornT_, I do have an ISourcePackage.
<bigjools> BjornT_: you need the pocket as well, but after that yes, we could factor it somewhere
<BjornT_> bigjools: how do i know the pocket?
<bigjools> BjornT_: it depends on where the package is being uploaded to
<bigjools> and is data that you need to supply
<BjornT_> bigjools: can the same ISourcePackage be uploaded to multiple pockets?
<bigjools> it can exist in multiple pockets
<bigjools> it can start in say, security, and be copied to updates
<bigjools> wgrant, can you think of an example where something would have a different component in a different pocket?
<BjornT_> bigjools: so in what way should i fix BugNomination.canApprove()? it doesn't use pockets at all
<bigjools> BjornT_: let me check your code
<wgrant> bigjools: It might have been done once, but I don't remember. It was certainly discussed with the archive admins, who talked to one of you guys, who confirmed it was possible with SQL, IIRC. I don't remember if it was actually done.
<jml> I see you use latest_published_component
<al-maisan> NascentUpload.verify_acl() does not seem to use pockets either ..
<jml> BjornT_, ^
<wgrant> But that's such a strange case it's not worth caring about for this, I don't think.
<bigjools> source_package.latest_published_component is perfect
<jml> ok
<bigjools> al-maisan: not in that specific place, because it already looked them up
<bigjools> al-maisan: see getSourceAncestry()
<BjornT_> jml: yes. that's why i'd like a method that accepts an ISourcePackage that does the check for me!
 * al-maisan looks
<BjornT_> s/check/checks/
<jml> BjornT_, I'm inclined to agree.
<bigjools> BjornT_, jml: yes, I can add something to do that
<jml> But I'd like to get to the end of bigjools's explanation :)
<bigjools> aaaaanyway
<jml> bigjools, well, I'm happy to do so, since I'm working on this problem now.
<bigjools> jml: awesome
<jml> check_permission(ISourcePackage, 'launchpad.Edit'), I assume.
<bigjools> jml: no, IArchive.<something>
<bigjools> let me think a but
<bigjools> bit
<bigjools> because we are trying to fix the security adapter, it won't work for lp.Append on IArchive right now
<bigjools> jml: you could change canUpload() so it takes an optional ISourcePackage
<jml> bigjools, AIUI, BjornT_ & I are just going to derive the IArchive instance from the ISourcePackage anyway
<bigjools> jml: I think that's a bad idea
<bigjools> it's tying everything into a single archive
<jml> bigjools, I don't see how we can avoid that.
<BjornT_> bigjools: i don't see how we could get the right archive, though. all we have is a distribution/distroseries and a source package name
<jml> BjornT_, in my case, I also have a pocket.
<jml> and a pocketful of broken dreams
<jml> but I don't plan on using them.
<bigjools> jml, BjornT_: you have to decide which archive you're uploading to, simple fact.  In most cases it's IDistribution.main_archive
<bigjools> but there's no way that should go in the security adapter
<bigjools> hard-coded I mean
<wgrant> Something like latest_published_archive seems to make sense.
<wgrant> Which will be either primary or partner.
<bigjools> I don't think it does
<jml> bigjools, ok. so how about this:
<wgrant> How is it significantly different from latest_published_component?
<jml> bigjools, I add a method / function / whatever,
<jml> bigjools, it takes an ISourcePackage, an IPerson and an optional IArchive
<jml> if IArchive is None, it gets it from source_package.distribution.main_archive or whatever is a sane default.
<jml> then it does the black handwavey magic that I'm slowly beginning to understand
<jml> bigjools, because, in the design of Bugs and Code, there's absolutely know way of knowing the archive beyond that.
<jml> no way.
 * jml was correcting spelling, rather than adding emphasis
<bigjools> :)
<bigjools> so
<jml> basically all I've got to work with is the stuff in this string: 'lp:ubuntu/karmic-backports/openssh'
<bigjools> at some point, you're going to have source package branches and bugs linked to PPAs
<bigjools> you don't right now, so assuming main_archive is fine
<lifeless> I'd say the per team package branch would map to that team/persons PPA
<bigjools> but don't hack that into soyuz code *please*
<lifeless> e.g. my subunit branches
<bigjools> yes, either way there's an archive involved
<jml> bigjools, I was thinking a method on ISourcePackage, actually. that leaves the assumption at quite a high level, and in exactly one place.
<bigjools> wgrant: latest_published_component works because packages are not generally in more than one distro archive at once
<BjornT_> bigjools: it feels like there should be some object representing a package in a PPA (or rather, a package in a specific archive), no?
<wgrant> bigjools: More than one distro component, you mean?
<bigjools> no, archive
<jml> BjornT_, that sounds right to me.
<jml> (or at least a good question)
<al-maisan> for anything that starts with 'lp:ubuntu/..' you'd take the corresponding main_archive; for anything that begins with 'lp:~user/..' you'd fall back to the respective PPA (how about users with multiple PPAs though?)
<bigjools> BjornT_: that's called a publication
<wgrant> bigjools: How's the archive presence relevant to latest_published_component?
<BjornT_> bigjools: isn't that tied to a specific version of the package?
<bigjools> wgrant: because when you're looking for the publications, it needs an archive a well
<bigjools> BjornT_: right
<wgrant> bigjools: Oh, true.
<BjornT_> bigjools: what i'm looking for is something like ISourcePackage, or maybe even IDistributionSourcePackage. maybe they should be extended to know which archive they belong to.
<al-maisan> I should say for 'lp:<distro>/..' one should take the main_archive ..
<bigjools> ISourcePackage should really be named IDistroSeriesSourcePackage !
<bigjools> let me think about it for a moment
<maxb> How do you properly clean a launchpad source tree, including linked sourcedeps?
<bigjools> jml: ok I have a plan
<bigjools> maxb: make clean does all of that apart from removing the links
<bigjools> which I don't think anything does
<jml> bigjools, I'm all ears.
<bigjools> jml: we need ISourcePackage.lastest_published which will return a publication
<bigjools> then you can do spph.component and spph.archive
<bigjools> without duplicating queries
<jml> what kind of object is a publication, an IPublication?
<bigjools> jml: ISourcePackagePublishingHistory
<jml> golly.
<jml> I can see why you said 'publication' the first time :)
<bigjools> lol
<wgrant> At least there isn't still a SecureSourcePackagePublishingHistory still in wide use, although it does seem to still exist :(
<bigjools> jml: and reviewers always say to us "why are you using spph as a variable name?"
<bigjools> wgrant: yes, it needs ripping out
<bigjools> I tried once, and ended up in a world of pain
<bigjools> jml: does it make sense to you?
<jml> bigjools, ummm. let's go through this again from the top
<bigjools> okay
<jml> bigjools, I get the latest_published
<jml> bigjools, then I ...?
<bigjools> can I assume that you have an ISourcePackage already?
<wgrant> (I'm glad DistroArchSeriesBinaryPackageRelease isn't spelt out in full)
<jml> bigjools, yes!
<jml> bigjools, branch.sourcepackage :)
<bigjools> poifekt
<bigjools> so
<bigjools> add a new property to it, latest_published (trivial, just return self._getFirstPublishingHistory
<bigjools> then you have in your hands a component and an archive
<bigjools> this could be encapsulated inside IArchive.canUpload()
<bigjools> if you pass it an ISourcePackage
<bigjools> the only thing canUpload doesn't check is packageset permissions, I think I need to ask al-maisan why he didn't change it to do that ...
<al-maisan> bigjools: it does .. actually.
 * al-maisan checked the sources.
<bigjools> it does?  great!
<bigjools> al-maisan: but I can't see where
<al-maisan> bigjools: what eventually gets called is IArchive.getPermissions() and that does the right thing.
<bigjools> yes, I see it now
<bigjools> fab
<bigjools> jml: so that's it, are you comfortable with it?
<jml> bigjools, comfortable enough to start, I think.
<bigjools> jml: oh actually you can't put it in canUpload
<bigjools> d'oh
<bigjools> but it's still easy
<jml> I was thinking maybe having a canUploadTo on ISourcePackage or something
<jml> something will fall out anyway
<bigjools> jml: canUploadToArchive would be good
<jml> *nod*
<bigjools> anyway I have a call now
<bigjools> catcha later
<jml> and maybe I can do something to stop the duplication between bugs & code here
<bigjools> yes
<jml> bigjools, np. thanks for the help.
<bigjools> jml: np - ping me again if you need more help
<jml> bigjools, will do
<jml> although I think I'm going to just jot down some notes from this conversation and then switch to working on my talk for tomorrow evening.
<jml> mrevell, hello!
<wgrant> Quick visit.
<jml> He's an efficient man.
<jml> anyway, I'm off home.
<bigjools> jml: I can write you a summary
<jml> bigjools, you can if you'd like, but I think I'll be right from the logs.
<jml> bigjools, putting on the mailing list might help more people than just me though :)
<jml> I do have to leave though
<bigjools> bye!
<jml> al-maisan, bigjools, BjornT_: thanks for the help!
<al-maisan> jml: my pleasure :)
<wgrant> bigjools: So, I managed to get lots of Soyuz bits to cooperate today, but it all seems a bit manual and I'm not sure if I've done anything terribly wrong. http://williamgrant.id.au/f/1/2009/soyuzness.html is my current setup.
<bigjools> will check it out in a bit, on a call right now
<wgrant> Sure.
<stub> jtv: So I've realized that for us, a Storm cache where the size is bounded by the number of objects is not optimal. What we care about is the amount of RAM being used. I was thinking of making a generational cache subclass that bumps a generation whenever num_objects > n and process_ram_usage > r.
<jtv> stub: very true.
<pkern> wgrant: Hm cool.  I should've taken Ubuntu as an example rather than hacking it to name it Debian.  Anyway I had something like "Unable to find distroseries: lenny" albeit I created that below /debian/.  I thought that UploadPolicies are not properly set up, did you find that wired somewhere?
<wgrant> pkern: I used a PPA rather than try to convince primary to publish.
<stub> jtv: Unless there is someway of keeping count of the size of individual objects, but process size may still be better for our needs.
<pkern> wgrant: I also tried PPA.
<wgrant> Although I was able to get Ubuntu primary to publish without --careful, I don't trust that.
<wgrant> pkern: I should try a Debian PPA at some point.
<jtv> stub: object size is horrendous in any case, except where __slots__ are used.
<bigjools> stub: did you see the cache change results I posted to that bug?
<stub> bigjools: https://code.edge.launchpad.net/~stub/launchpad/update-storm/+merge/9130
<bigjools> wgrant: we only use --careful when we want to re-publish everything.  it's quite devastating
<pkern> "Must omit the trailing 'ubuntu', this time. "  I love hardcoded stuff. ;-)
<wgrant> bigjools: Right, but I did want to republish everything, as the on-disk archive was empty.
<bigjools> stub: (apologies if you replied I've not read my email yet)
<wgrant> bigjools: But the librarian files aren't there, so it crashes.
<bigjools> wgrant: yes, in the first instance it's what you need
<wgrant> bigjools: Even once I supersede/delete all of the pulibcations.
<bigjools> oh fucking firefox you piece of shit
<wgrant> pkern: I think it probably just uses the distro name, rather than hardcoding it.
<wgrant> bigjools: Firefox killed my laptop a couple of minutes after you complained about it earlier.
<gmb> Moin + Firefox locked up X completely the other day.
 * gmb joins in the two-minute hate
<wgrant> That hasn't happened to me in a month or so (this is Karmic).
<bigjools> wgrant: yes and mine just restarted without my 10 tabs I wanted saved >:(
<wgrant> bigjools: Ahaha.
<pkern> wgrant: It's at least hardcoded in activate-ppa by retrieving only ubuntu through ILaunchpadCelebrities.  And I was delighted that debian is available there, too. :-P
<bigjools> are you going to work on debianb ppas?
<wgrant> pkern: Right, but you should be able to easily change an Ubuntu PPA into a Debian PPA.
<wgrant> I'll try that after dinner.
<pkern> That's true.
<wgrant> (just IArchive.distribution needs touching, AFAICT)
<noodles775> wgrant: if it's easier for you, you can also do a lot of that setup on your link using `make harness`
<wgrant> pkern: Oh, probably archivearch and archivedependency will need touching too.
<wgrant> noodles775: True.
<wgrant> But it wouldn't make it much shorter, AFAICT.
<noodles775> no, not shorter, but easier to reproduce after the database is reset.
<stub> bigjools: I use the sessionsaver plugin which, so far, has never lost my previous session on a crash.
<wgrant> Firefox 3.5 seems to be better about recovering.
<pkern> I like that you need to quit Banshee to do make run :-P
<bigjools> !
<jtv> hi henninge!
<gmb> pkern: If you turn off the local music sharing options in banshee it stops hogging the port that Launchpad wants.
<henninge> Hey jtv!
<gmb> pkern: Took a while to figure that one out...
<mgdm> what port is that?
<gmb> mgdm: I can't remember offhand... It's whichever port you get an "already in use" message about when you try to make run with Banshee running.
<pkern> 80 something
<pkern> 8085
<bigjools> wgrant: "Builds will only be dispatched for archives with published sources" - not quite right, that only happens if the PPA is private.
<bigjools> wgrant: did you get a local buildd working then?
<wgrant> bigjools: Ahh. Why is that? It won't have a .htaccess otherwise?
<wgrant> bigjools: I did.
<wgrant> It was pretty easy.
<bigjools> very cool, I've never tried
<bigjools> because the buildd gets sources from the librarian normally, but we don't do that for private builds
<wgrant> Ah.
<wgrant> I see.
<bigjools> they need to stay sekrit
<wgrant> Right.
<pkern> private builds ):
<wgrant> pkern: Why ):?
<pkern> bigjools: Why weren't you able to just tell the librarian that it should give out that files local-only?
<pkern> And where are oopses stored btw...
<bigjools> pkern: there are two librarians
<wgrant> /var/tmp/lp-err
<bigjools> because they don't have any access control
<bigjools> so one is restricted in the DC
<bigjools> the buildds can't see it, otherwise other builds could steal data
<pkern> Hm, point.
<deryck> Morning, all.
<bigjools> oopses are logged, check the config to find out where
<pkern> Well, you could sort out access for the outside fetching things only and limit connectivity for the build itself.
<bigjools> I don't remember offhand
<noodles775> Hi deryck !
<pkern> bigjools: wgrant is right (:
<pkern> Apart from s/lp-err/lperr/ but well.
<wgrant> Ehem.
<wgrant> I think I found a hole.
<bigjools> gadzooks
 * wgrant pokes around a bit more.
 * pkern giggles.
<wgrant> I'd wondered about how this stuff worked for some time, as it could easily be vulnerable. But looking at it now seems to confirm it.
 * wgrant tries, then files a bug.
<al-maisan> hmm .. such "discoveries" were to be expected ..
<wgrant> Yes.
<wgrant> I meant to look for this particular one earlier.
<bigjools> wgrant: PM me the bug # when you're done please
<stub> jtv: Hmm.... the problem is that for Launchpad I would need to bump a generation when ram usage *for the thread* got too high, and for a generic storm cache when the ram usage by the cache got too high. Otherwise we risk a large object sitting in an idle cache causing other caches to thrash.
<stub> jtv: Unless we share the storage for all of these cache instances, so they have one big cache rather than individual ones.
<stub> jtv: Or perhaps maintain a list of all of the caches, and bump all of them with num_objects > threshold when the ram limit is exceeded.
<stub> I think that would work
<jtv> stub: (sorry, on call)
<jtv> stub: you could have a shared policy on what would make a nice cache size under current memory pressure.
<stub> jtv: But we don't know how many objects would be nice because the memory footprint of each object can be very different.
<stub> jtv: Or do you mean bumping the desired object counts up and down until RAM usage is within bounds?
<jtv> stub: will that matter so much if we tune it dynamically anyway?  Too much memory pressure -> fewer objects please.
<jtv> It'll victimise stores with lots of small objects, true, but how much would that hurt performance?
<bigjools> stub: the issue with the cache that you're waiting for an RC on is mentioned on the CRB page, it would be great if you could manage that :)
<stub> I'm don't see how that is better than just bumping the caches when the memory limit is reached.
<stub> CRB?
<bigjools> https://wiki.canonical.com/Launchpad/CurrentRolloutBlockers
<stub> bigjools: I probably won't be here, so someone will need to land it for me once it is rc'd.
<bigjools> stub: ok, I can do that
<stub> bigjools: kiko has an email in addition to the mp request.
<bigjools> yeah I saw
<bigjools> flacoste can give RCs too
<stub> flacoste is on leave I believe
<bigjools> ah yes
<bigjools> kiko it is
<gmb> Argh.
 * gmb just punched himself in the face with a mug full of tea.
<allenap> gmb: Edwin's bug: https://bugs.edge.launchpad.net/launchpad-registry/+bug/401748
<ubot3> Malone bug 401748 in launchpad-registry "Cannot create new project in IE8" [High,In progress]
<gmb> allenap: Ta#
<bigjools> is that bug on the CRB page?
<bigjools> I seem to be the only person updating it
<wgrant> bigjools: There's a CRB on dev.launchpad.net now too.
<bigjools> only just added
<bigjools> without migrating the old page :/
<bigjools> ffs
<henninge> jtv: ping
<jtv> henninge: ?
<henninge> jtv: Looking at the pofiles in the queue I see that most of the files are probably still in there because the lang code cannot be determined.
<jtv> henninge: hang on, I'm coming.  Just the po files for projects?
<henninge> jtv: yes
<henninge> jtv: almost 400
<henninge> jtv: There is a lot like domain-ll.po
<jtv> :(
<jtv> ah yes
<henninge> jtv: I think that they should be deleted from the queue and the uploaders informed about how to name the files.
<jtv> henninge: instead of deleting, we could just let them sit there for now and email the uploaders to say there's a faster way.
<henninge> jtv: but even then the po files need to be named properly to be automatically approved.
<jtv> henninge: yes, they'd upload new files and discover that it was much faster.
<jtv> henninge: once they've done that, we can delete the old uploads.  This is not a part of the queue we should watch daily anyway, so that gives them time to play.
<henninge> jtv: oh, I thought "faster way" was bzr uploads ...
<henninge> ;-)
<henninge> jtv: yeah, we should do that
<jtv> henninge: in this case, it's fixing the PO names.  They'll have to do that either way.
<noodles> sinzui: have you started a page about updating the new 3.0 templates? If not I can take a look at your branch to get some ideas.
<sinzui> I am writing it today
 * mars gives up on reading the channel backlog
<sinzui> noodles: The work is easy, but there are heading conflicts that may require us to edit every page twice...making the progress report bogus
<henninge> jtv: what I meant.
<jtv> henninge: we're in violent agreement.  Would you like me to write a canned response and put it up on the wiki?
<noodles> sinzui: OK. Let me know if I can take a sneak preview ;)
<jtv> Maybe I'll start a new subpage, "approval issues."
<henninge> jtv: please do, I'll send them out, then.
<jtv> henninge: I mean, auto-approval issues.
<jtv> OK.
<henninge> jtv: or how busy are you?
<jtv> henninge: not very.  Need help?
<henninge> jtv: no, no, just wanted to put something else on you.
<jtv> henninge: go ahead.
<henninge> jtv: I'll move to my office soon, so this is cool
<henninge> jtv: oh, sorry.
<henninge> ;-D
<jtv> henninge: I don't think I follow
<henninge> jtv: no, no, just *did* *not* want to put something else on you.
<jtv> ahhhh :)
<henninge> jtv: ^ what I meant to write
<jtv> No worries, CHR is *incredibly* quiet.  I think everyone decided to look up their questions in the source first.  Or maybe our entire user base is just stuck downloading/browsing the code for the heck of it.
<henninge> jtv: ok, I'll relocate now.
<jtv> ok
<noodles> sinzui: I'm just fiddling with a main-only layout, and noticed that style-3-0 is quite specific when applying styles to elements that have been reset (via the yui reset). For example
<noodles> sinzui: rather than setting any th element as bold, only those within form table tbody are set to bold...
<sinzui> noodles: I think there is a comment that we wanted to restore some of the settings from style.css
<sinzui> noodles: reset is complete, there is no style. So strong way was never bold,
<sinzui> noodles: I expect the style sheet will get a lot of changes as we get more pages using them
<noodles> sinzui: yes, but I'm just confused why style-3-0 would specifically set 'form table tbody th' to font-weight:bold rather than just th.
<noodles> Ah, ok.
<noodles> I'll do some modifications and than perhaps later chat with you on the diff (after reading your doc of course :) ).
<sinzui> noodles: Some ths are not bold on launchpad, so I set a rule that matches how we currently use th bold
<noodles> I see.
<jtv> pqm still failing?  :-(
<sinzui> noodles: we have only two page designs, only one in html, There is a tremendous about of css we will discover
<sinzui> jtv. yes, I cannot restart buildbot. I am hoping for gary to come by
<sinzui> barry, bac, Edwin, salgado: standup in 2 minutes
<barry> Ursinha: ping.  in the registry test plan you mentioned you filed bug 401887, but that doesn't seem to be the right bug number
<ubot3> Malone bug 401887 in launchpad "offer "make announcement" on +announcements" [Undecided,New] https://launchpad.net/bugs/401887
<Ursinha> barry, argh, paste fail
<EdwinGrubbs> sinzui: can you call again, skype wouldn't ring, but I know the rest of the audio works.
<Ursinha> barry, bug 402736
<ubot3> Malone bug 402736 in lazr-js "Buttons seem not hooked up to js in IE8" [Undecided,New] https://launchpad.net/bugs/402736
<Ursinha> I'll fix in the testplan
<Ursinha> barry, done
<maxb> Does anyone else find themselves routinely running make SHHH= ....
<maxb> ?
<barry> Ursinha: thanks!
<mars> maxb, not personally.  When there are startup errors the extra noise is a welcome debugging aide. :)
<Ursinha> barry, sorry for the mess
<mars> maxb, say, for example, when you accidentally run two instances side-by-side
<maxb> mars: that's the point -  SHHH=(empty string here) turns off the output suppression
<barry> Ursinha: no worries
<mars> maxb, ah, my mistake
<maxb> np, i could have been a lot less terse :-)
<barry> Ursinha: though i do not think we need to release critical this bug.  sinzui says that we don't support ie8 officially yet
<maxb> Well that was comparatively painless
<mars> allenap, ping re: the pages you moved to the dev wiki
 * maxb has a launchpad webapp running...... on karmic
<mars> cool
<allenap> mars: Yeah
<mars> maxb, using which Python?
<maxb> 2.5
<mars> barry, ^
<mars> it *is* possible :)
<bigjools> heh, but I bet he hasn't run the test suite :)
<Ursinha> barry, no problem then, if we don't support officially
<maxb> well, no, baby steps...
<sinzui> barry: Ursinha: AN RC is needed if there is broken JS that prevents a user from completing a task. We do not care if the task can be completed via AJAX or HTML.
<barry> folks.  every week we have two meetings for launchpad code reviewers.  the meetings have always been public.  reviewers are required to attend, but anybody else is also welcome to lurk.  the america/euro version starts in 2 minutes in #launchpad-meeting
<noodles> sinzui: A quick question - on the following url, there is a table used to layout "Repository disk usage, Build dependencies" etc.
<barry> reviewers -> #launchpad-meeting
<noodles> https://edge.launchpad.net/~cprov/+archive/ppa
<noodles> sinzui: is that the kind of thing that we want to use a dl for instead, or will we stick with a table for the moment?
<sinzui> noodles: yes, that is a good example
<Knut-HB> hello together, i have a question/problem concerning the local installation of launchpad. After going through the step-by-step-solution (every seemed to go correct) and when  i then run "make schema && make run" in ~/launchpad/lp-branches/devel/ and when i enter "launchpad.dev" in firefox i get a 503. is there a quick solution for this?
<mars> Knut-HB, well, first lets look for a quick diagnosis :)
<sinzui> noodles: That layout may change. The onecolumn layout used that style, but you can see the project information has a new layout in https://devpad.canonical.com/~curtis/LP_projectdetail.png
 * noodles looks
<allenap> mars: re-pong :)
<noodles> sinzui: yes, that looks lovely :), it would be great to get something similar for the archive index.
<mars> allenap, ah, just wondering if you also moved the wiki pages over? and if so, was it easy to find a place to link them from?
<salgado> allenap, is it just me or are the edit icons for inline editing of bug status/importance misbehaving (i.e. only showing up when I hover the mouse over them)?
<wgrant> salgado: That happens to me sometimes too.
<wgrant> And they sometimes take several seconds to disappear.
<mars> Knut-HB, could you kill the server and paste the output of 'make run'?  I want to be sure that the server is in fact up
<salgado> wgrant, is there a bug about that?
<allenap> mars: Yeah, I moved them over to Bugs/ExternalBugTrackers/Mantis and the other to Bugs/Spec/FormattingBugNotifications
<wgrant> salgado: There are a few bugs about that sort of thing, but nothing about that particular issue.
<allenap> mars: I also added most of the intermediate pages. They're looking a bit sparse, but they're there.
<wgrant> Things were changing, so I didn't file one.
<allenap> salgado: Not just you. I think that's a feature. intellectronica?
<noodles> Knut-HB: when you run make run, watch the output and verify you see "[-] daemon ready!"
<intellectronica> salgado: that's a feature
<salgado> wgrant, if I leave the mouse pointer over the place where the link should be, it keeps showing up and disappearing in an infinite loop.  does that happen to you too?
<salgado> intellectronica, but not that (^), I hope?
<salgado> intellectronica, that happens after the first time the icon shows up and I move the mouse pointer away from it
<jtv> sinzui: if gary is at oscon, that doesn't bode well for your pqm restart.  :/
<intellectronica> salgado: no, that sounds pretty bad. let me see if i can reproduce
<wgrant> salgado: That doesn't, but for the first load on each page it takes one Launchpad-long-time to appear, and another to disappear.
<wgrant> It seems to make a request each time.
<salgado> intellectronica, when I move the mouse pointer back to it and leave it there for around two secs, it goes into that loop
<sinzui> jtv: I am prepared to issue a release-critical empty branch to make buildbot work. I will ask for forgiveness after I see staging updated
<intellectronica> salgado: is that locally or on edge?
<salgado> intellectronica, edge
<jtv> sinzui: non-empty branches sure aren't working...
<salgado> intellectronica, it happens only when the pointer is above the place where the icon should be.  if the pointer is above the text, it doesn't happen
<intellectronica> salgado: there's a fix that didn't make it to the nightly because of an unrelated failure. could you please try to reproduce locally?
<sinzui> jtv: right, I will make a white space change to one of our ugly page templates
<salgado> intellectronica, sure
<intellectronica> salgado: thanks
<jtv> sinzui: so what's breaking it that a branch like that would succeed whereas mine fails?
<sinzui> jtv. I do not know. I will find out now
<jtv> sinzui: good luck!
<salgado> intellectronica, it happens on lp.dev too
<salgado> (and I just did a rocketfuel-get)
<intellectronica> salgado: :( can you please file a bug and assign to me?
<intellectronica> salgado: what browser are you using, b.t.w?
<salgado_> <salgado> intellectronica, it happens on lp.dev too
<salgado_> <salgado> (and I just did a rocketfuel-get)
<mars> hi everyone, need some help debugging a sourcecode dependencies build problem
<mars> in a new setup
<mars> Here's the 'make schema' output showing the error: http://paste.ubuntu.com/224383/
<mars> and the sourcecode/ directory has links in it
<mars> BTW, this is in devel/
<mars> wgrant, around?
<wgrant> pkern: I just successfully uploaded things to both Debian primary and PPA here, and can even access the PPA in the web UI along with Ubuntu ones with a few lines of changes. It just took a bit of lucilleconfigging.
<wgrant> mars: Yes.
<wgrant> An odd error.
<mars> wgrant, just wondering if you saw anyone with problems running link-external-sourcecode in devel/
<wgrant> Is that from your machine?
<mars> wgrant, this is Knut-HB's new setup
<wgrant> I saw somebody complaining of the same thing earlier, IIRC.
<mars> Knut-HB, do you have any contents in devel/bin/ ?  Could you paste them please?
<mars> that way I know what has and has not been generated by the buildout scripts
<wgrant> download-cache should live in ~/launchpad/lp-sourcedeps, shouldn't it?
<Knut-HB> mars: http://paste.ubuntu.com/224397/
<mars> wgrant, yes
<mars> Knut-HB, and devel/download-cache is a valid link?
<Knut-HB> when i open "devel/download-cache" i see another folder named "dist". is that correct?
<mars> Knut-HB, btw, you may want to restart apache.  Launchpad will not do that for you
<mars> Knut-HB, it is.  does dist/ contain a number of .tar.gz files?
<intellectronica> salgado: sorry, you may have missed my reply when you disconnected. i asked if you could file a bug with instructions on how to reproduce and assign to me
<intellectronica> i still can't seem to reproduce this locally
<salgado> intellectronica, will do
<intellectronica> salgado: cool, thanks
<Knut-HB> mars, after doing a "sudo /etc/init.d/apache2 start" the apache restarted and inside the "dist"-folder are 178 files, a lot are tar.gz
<salgado> (I missed your reply indeed)
<mars> Knut-HB, good!
<Ursinha> sinzui, barry, question: why in item r8865 needstesting in your testplan it doesn't have the Konqueror items to test?
<mars> Knut-HB, please try "cd devel/", 'make', then 'make schema'
<Knut-HB> ok
<sinzui> Ursinha: I do not know the answer to that one. I did not it. I removed the Konqueror 3 since it is not supported.
<pkern> wgrant: wow
<Knut-HB> after "make" i get several errors, pastebin?
<sinzui> Ursinha: I did note that Konqueror 4 was missing, but did not consider to ask why
<Ursinha> sinzui, I'm adding by default konqueror 3 browser as C graded, because of people using LTS version of Kubuntu (mars can correct me if I'm wrong)
<Ursinha> sinzui, that was removed by hand, since the script is adding all of them automagically when detecting the js tag
<sinzui> Ursinha: The test will always fail, We are not fixing them
<wgrant> pkern: Have you had luck, or do you want some pointing in the right direction, or do you have significantly better things to do with your time?
<sinzui> Ursinha: Do not make k3 take an AJAX test it has no chance of passing. It should only use html
<Knut-HB> mars: this is what i get after "make": http://paste.ubuntu.com/224401/
<Ursinha> sinzui, do we have to guarantee that it will work as disabled? I assumed so
<mars> Knut-HB, ok, hmm
<mars> that is frustrating
<mars> the directory is there, and it is valid
<sinzui> Ursinha: The HTML must work, and that is what need to be tested. AJAX is for A grade browsers only. The K3 tests should verify that the operation still works in HTML (bug subscribe via HTML was lost for a while, so that is a good example of a FAIL that had to be fixed)
<mars> Knut-HB, does the directory `/home/test/launchpad/lp-sourcedeps/sourcecode/cscvs' exist?
<Knut-HB> mars, checking
<mars> Knut-HB, perhaps the sourcecode download broke during transfer
<mars> that has happened to other people
<Ursinha> sinzui, right, they're being added to the testplans to test not just the ajax, but if the feature is still functional even without js
<bigjools> Knut-HB: is your devel branch up to date?
<Knut-HB> i was at the laptop all the time watching the progress and the download didn't break or something like that
<bigjools> "bzr pull" it
<Knut-HB> bigjools: downloaded just yesterday
<bigjools> pull again please
<wgrant> What about running rocketfuel-get?
<bigjools> there was a fix that went in late
<Knut-HB> ok, first the checking
<mars> wgrant, yep, that's next :)
<wgrant> That will update everything and make sure they're checked out properly.
<bigjools> don't run fg-get yet
<sinzui> Ursinha: then I was mistaken in the intent of the inline edit test
<bigjools> rf-get even
<wgrant> Although it won't fix the 2a issue.
<Ursinha> sinzui, at least I thought that the js testing was about making ajax work and not breaking if it doesn't
<bigjools> Knut-HB: when bzr pull finishes, run utilities/rocketfuel-get
<Knut-HB> mars: check, folder is there with contents
<Ursinha> mars, am I understanding that wrong?
<mars> Ursinha, reading back
<Knut-HB> bigjools: where do i have to pull? inside the folder "launchpad"?
<bigjools> Knut-HB: insde the devel dir
<bigjools> that is the trunk branch
<Knut-HB> ok, pulling
<bigjools> there was an important fix to rocketfuel-get done late yesterday
<Knut-HB> ah ok
<mars> sinzui, Ursinha, what are the K3 tests written in?
<sinzui> mars: this is the test plan, not automated testing
<salgado> intellectronica, bug 403066
<ubot3> Malone bug 403066 in malone "Icons to edit a bug's status/importance come and go in an infinite loop" [Undecided,New] https://launchpad.net/bugs/403066
<Knut-HB> bigjools: ok, pulled, what next? rocketfuel-get?
<mars> sinzui, ah, see, that is why I asked :)
<bigjools> Knut-HB: yep
<intellectronica> salgado: thanks
<bigjools> Knut-HB: utilities/rocketfuel-get
<Knut-HB> ok, working
<bigjools> it will take a while
<mars> sinzui, Ursinha, the K3 tests in the test plan should ensure that the functionality is still accessible as HTML in the K3 browser.
<intellectronica> salgado: whoa, finally realised how to reproduce
<sinzui> herb: buildbot appears to be dead I cannot make it restart. I know this is unlikely. but do you know how to kick buildbot?
<intellectronica> this can be very bad for epileptic users
<Knut-HB> bigjools: ok, i'll tell you when it's finished
<salgado> intellectronica, lol
<bigjools> ok thanks
<mars> sinzui, Ursinha, that is what defines "C" Grade: we explicitly test that the site still works on a known problematic platform.
<herb> sinzui: I can certainly try
<Ursinha> mars, so I got that right :)
<mars> sinzui, Ursinha, versus "X" Grade, where we don't test, we just assume it's ok.
<mars> Ursinha, yep.  The K3 test should check that the AJAX is disabled, and that users can still use the site.
<sinzui> mthaddon: herb: buildbot appears to be dead I cannot make it restart. I know this is unlikely. but do you know how to kick buildbot?
<mars> sinzui, herb said he would try
 * sinzui is still frazzled from yesterday
<mars> sinzui, it was buried in the last flurry of messages addressed to you :)
<mars> sinzui, frazzled, but fun, no? :)
<mars> sorry, should make that more Canadian...
<mars> sinzui, frazzled, but fun, eh? :)
<sinzui> No. My day sucked because I have a ton of work to complete and too many interruptions to work on it. I really do not want to release code today since I cannot see that any of our landing fixed bugs
<Knut-HB> bigjools, ok, finished
<Knut-HB> bigjools, what now?
<bigjools> Knut-HB: any errors?
<mars> sinzui, is there any way I could help?  QA perhaps?
<Knut-HB> bigjools, can't find any
<bigjools> Knut-HB: ok that's good - try "make schema" now
<sinzui> mars: you cannot QA because the code never arrived in staging because buildbot had a heart attack
<Knut-HB> bigjools, same errors again =/
<bigjools> gah
<bigjools> ok
<herb> sinzui: The buildmaster appears to have (re)started correctly.
<bigjools> can you run rocketfuel-setup again
<Knut-HB> ok
<mars> sinzui, ah, sorry to hear that.  You enjoy lean: this is something that should go in the "unnecessary waste" category.
<sinzui> herb: \o/
<sinzui> I can deliver beer to your house later this evening
<sinzui> mars: I suppose my frustration is really these moment I feel powerless to fix something
<mars> sinzui, I've been building a list of problem areas in the dev process, and the building and landing processes are definitely in the running for "biggest problem right now"
<mars> sinzui, if the system was perfect, you wouldn't have to fix anything :)
<sinzui> mars: but where would I get my karma from then?
<mars> sinzui, you'll never catch flacoste, give up :)
<sinzui> mars: I can catch flacoste in 1 week. I choose not to use blueprints until they rock
<mars> well, if you fix blueprints, you will close so many bugs that yes, you will be in first place :)
<sinzui> bac: should we have our call?
<mars> bigjools, do you think a series of environment checks before lp startup would help with diagnosis?  make start-check
<bigjools> mars: not a bad idea, but I'd make it a separate target, "make diagnose" or something like that so we can ask for it to be run
<mars> bigjools, ah, that is a good name :)
<Knut-HB> bigjools, ok, rocketfuel-setup has finished, what next? make schema?
<bac> sinzui: yes
<bigjools> Knut-HB: if there were no errors, yes
<ara> bigjools, I got an error of "Package `launchpad-developer-dependencies' is not installed and no info is available." is it only available in jaunty? (I am running karmic)
<Knut-HB> while doing "rocketfuel-setup" i couldn't find any errors but when doing "make schema" i get the usual errors =/
<bigjools> okay, I think rf-setup is confused
<bigjools> let me check something
<Knut-HB> ok
<maxb> ara: Yeah, us karmic users need to be trailblazers right now :-/
<ara> maxb, ok, thanks
<bigjools> ara: yes, jaunty only, LP doesn't yet work in karmic
<bigjools> or you can use hardy
<maxb> :-P
<ara> only *.04 releases? :P
<mars> that's the last LTS release, and what we deploy on
<bigjools> supported platforms :)
<sinzui> bac: https://edge.launchpad.net/launchpad-registry/+milestone/2.2.7
<bigjools> Knut-HB: what is in lp-sourcedeps/download-cache/dist/ ?
<mars> Knut-HB, what happens if you run 'make download-cache' in devel/ ?
<Knut-HB> mars, i'll do it next
<Knut-HB> bigjools, http://paste.ubuntu.com/224424/ this is inside the folder
<Knut-HB> mars, "make: `download-cache' is up to date.
<bigjools> Knut-HB: in devel, do you have a symlink "download-cache" ?
<pkern> wgrant: I'm currently at Debconf and people want me to move Debian autobuilding to a new host and discussing stuff with people.  That's the main reason I get distracted at the moment, so I appreciate the pointers and will look at it later today. \-:
<Knut-HB> bigjools, yes
<bigjools> Knut-HB: at it points to the ../../lp-sourcedeps/download-cache ?
<bigjools> s/at/and/
<Knut-HB> bigjools, yes it does
<bigjools> Knut-HB: does devel have a sourcecode dir?
<bigjools> full of symlinks?
<Knut-HB> bigjools, i'll check that, just have to check something in
<Knut-HB> bigjools, yes, 27 symlinks and one makefile
<bigjools> Knut-HB: ok, if you type "make build" now, in devel, what happens?
<mars> bigjools, drilling down the Makefile target chain? :)
<Knut-HB> bigjools, http://paste.ubuntu.com/224430/
<bigjools> mars: trying different things :)
<mars> Knut-HB, how about 'make compile' ?
<bigjools> Knut-HB: this is bizarre
<mars> I suspect it will throw the same error
<Knut-HB> bigjools, you won't believe me: the same errors :P
<Knut-HB> mars, you just won 1000 points!
<mars> ok, lets keep drilling then
<bigjools> Knut-HB: rm download-cache and rm sourcecode/* then run "utilities/link-external-sourcecode ../../lp-sourcedeps"
<Knut-HB> ok wait
<mars> there should really be a GNU Make option that gets it to spit out the targets....
<Knut-HB> bigjools, you want to see the output?
<bigjools> Knut-HB: no
<bigjools> if it's done, try make build
<Knut-HB> wow Oo one part of three errors less :D
<Knut-HB> "No rule to make target `build'. Stop."
<bigjools> lol
<bigjools> ok just "make"
<Knut-HB> <Knut-HB> "No rule to make target `build'. Stop."
<mars> bigjools, we need a 'make distclean' target :/
<bigjools> Knut-HB: ok try "python2.4 bootstrap.py"
<bigjools> mars: what would that do over make clean?
<mars> bigjools, 'make clean' cleans up the test env.  'make distclean' would kill eggs/, download-cache, apidoc, etc.
<bigjools> apidoc is removed by make clean, but ISWYM
<Knut-HB> bigjools, http://paste.ubuntu.com/224434/
<mars> bigjools, 'make clean' means 'clean up everything from running'.  'make distclean' means 'remove everything that isn't in the source tree, so I can tarball it'
<bigjools> Knut-HB: huh, I think we see your problem
<Knut-HB> bigjools, really? :D
<bigjools> buildout can't get its dependencies
<mars> bigjools, what Knut-HB said: really?
<bigjools> "Link to http://pypi.python.org/simple/zc.buildout/ ***BLOCKED*** by --allow-hosts
<bigjools> I've not seen that before
<mars> bigjools, that is expected - it is supposed to grab them from eggs/
<mars> afk, biab
<mthaddon> kiko: so are you release manager for today?
<mars> bigjools, or from download-cache, I don't remember which exactly
<bigjools> :)
<kiko> mthaddon, indeed I am, how are you?
<mthaddon> kiko: depends on how things are looking for the release :)
<mars> bigjools, Knut-HB, I think it is the 'bin/buildout' target that is failing
<barry> losa ping
<mthaddon> barry: hi
<mars> or was failing, before download-cache got killed
<bigjools> Knut-HB: what is in your devel/bin dir?
<mars> Knut-HB, you can always try 'make -d foo' if you want more info about what make is deciding to do
<barry> mthaddon: hi.  kfogel asked me to coordinate with you about regenerating the mailing list archives with his new style.  he's at oscon
<mthaddon> barry: ok
<barry> mthaddon: cool.  let me make sure the wiki page with the instructions is updated
<Knut-HB> bigjools, http://paste.ubuntu.com/224437/
<barry> mthaddon: actually.  have we rolled out yet?  if not, i think we need to wait until that happens
<mthaddon> barry: rolling out 22:00 UTC
<bigjools> Knut-HB: that looks ok
<barry> mthaddon: cool.  then let's chat again in a few hours, or tomorrow morning.  in the meantime, i'll make sure the wiki has the right instructions
<bigjools> Knut-HB: when you ran link-external-sourcecode, did it print a load of symlink info?
<Knut-HB> yes
<mthaddon> kiko: so are there any more expected items after what's currently in buildbot?
<bigjools> I am totally baffled
<Knut-HB> bigjools, me too :p
<bigjools> Knut-HB: I only have one more suggestion.  Blow away your lp-sourcedeps, and run rocketfuel-get again
<Knut-HB> bigjools, hm ok, i do as you say ;9
<bigjools> sorry, it will download lots of stuff again :/
<Knut-HB> ah, 16 mbit/s connection ;)
<bigjools> woo :)
<Knut-HB> and it's not my system i'm working on anyway, it's a virtual system on a remote system accessed via ssh so... don't worry :D
<Knut-HB> ok, downloading
<kiko> mthaddon, so the rollout is at 2200 UTC?
<mthaddon> kiko: yep
<Knut-HB> bigjools, "Couldn't find lp:~launchpad-pqm/shipit/trunk/, skipping it." is a message i get after rocketfuel-get finished. what now?
<bigjools> Knut-HB: do you get that for any others?
<bigjools> if not, it's ok
<Knut-HB> now i don't think so
<bigjools> ok
<bigjools> no other errors from rf-get?
<Knut-HB> no, couldn't see any
<bigjools> rf-get does a "make" when it finishes
<bigjools> so that's good
<bigjools> let's see if make build works now!
<Knut-HB> now make build made some more stuff but quits with errors
<bigjools>  /o\
<Knut-HB> "no rule to make target `build''"
<bigjools> ok
<bigjools> make schema
<Knut-HB> <Knut-HB> "no rule to make target `build''"
<bigjools> geez
<bigjools> "make" gives the same error I guess
<beuno> mthaddon, could you rename: https://edge.launchpad.net/research38917 to archive-behavior-research?
<Knut-HB> wow, how did you know that? ;)
<bigjools> I am psychic
<Knut-HB> O.O
<bigjools> so
<bigjools> let's try make clean
<Knut-HB> did a lot of "rm -rf" stuff
<bigjools> and make
<mthaddon> beuno: done
<beuno> mthaddon, you rock, Thanks.
<Knut-HB> ok, it's working longer now and so far no errors
<Knut-HB> shhh.py is doing something
<bigjools> \o/
<mthaddon> kiko: so are you expecting any more submissions before the rollout?
<Knut-HB> bigjools, is that good?
<bigjools> Knut-HB: are you back at the prompt?
<Knut-HB> not yet
<Knut-HB> is that good too?
<bigjools> yeah it takes a while
<bigjools> it makes the api docs
<Knut-HB> so chances are high it works when this step is finished?
<bigjools> let's not go that far yet :)
<Knut-HB> hmpf -.- ok :p
<kiko> mthaddon, would you be a love and give me a URL to buildbot as my bookmarks died?
<bigjools> if it finishes ok, then you need "make schema"
<Knut-HB> ok
<bigjools> kiko: buildbot.devpad.info
<mthaddon> what he said :)
<Knut-HB> bigjools, ok, after doing a long time some stuff in the background make finished with "*** No rule to make target `build'. Stop."
<Knut-HB> make schema?
<bigjools> that's not good
<bigjools> your makefile sounds screwed
<Knut-HB> sounds not good
<bigjools> can you find a build target in it?
<bigjools> about line 114
<Knut-HB> where can i find the file?
<bigjools> top of devel
<bigjools> your're in that directory already, right?
<Knut-HB> yep
<Knut-HB> line 114 says the following
<Knut-HB> "build: $(BZR_VERSION_INFO) compile apidoc"
<bigjools> ok try make build, in the devel dir
<Knut-HB> bigjools, "*** No rule to make target `build'. Stop."
<bigjools> !
<Knut-HB> bigjools, but i have to go now =/
<bigjools> make compile
<bigjools> make apidoc
<bigjools> ?
<bigjools> ah ok
<Knut-HB> same errors
<Knut-HB> both compile and apidoc
<bigjools> weird, you just showed me the makefile
<bigjools> make -F Makefile build
<bigjools> -f sorry
<bigjools> lower case
<Knut-HB> error
<bigjools> same one?
<Knut-HB> would it be the best to just delete the whole launchpad-folder and download it again?
<Knut-HB> yes
<bigjools> ok, delete lp-branches and re-run the rocketfuel-setup script
<Knut-HB> i will asap
<bigjools> save it first
<bigjools> it's in lp-branches :)
<Knut-HB> it's all in the log so i won'T forget ;)
<Knut-HB> thanks for the help dude
<bigjools> ok good luck
<Knut-HB> thanks
<kiko> mthaddon, there doesn't seem to be anything critical I've found yet
<kiko> so I think it's on-track
<mthaddon> kiko: that's what I like to hear :)
<Ursinha> ping rockstar
<tfmoraes> Hi all
<mars> hi tfmoraes
<tfmoraes> I have just installed the launchpad
<tfmoraes> hi
<tfmoraes> Is there a default user?
<Ursinha> tfmoraes, foo.bar@canonical.com:test
<Ursinha> afair :)
<tfmoraes> thank you
<tfmoraes> it works!
<mars> tfmoraes, you can find the full sample data user list in lib/canonical/launchpad/pagetests/README.txt
<tfmoraes> Thanks
<Ursinha> tfmoraes, cool :)
<mars> Added the default user and password to the /Getting page
<mars> should make it easier for everyone else
<intellectronica> how do i cause a 404 from python view code? do i raise NotFoundError?
<mars> intellectronica, try that first, it might bubble up to the right spot in the publisher
<matsubara> tfmoraes, foo.bar@canonical is deprecated. use admin@canonical.com instead for an admin user or test@canonical.com for a regular user
<bigjools> mars, add it to the FAQ as well
<mars> intellectronica, if it doesn't work, look at raising it from one of the traversal-associated methods of the view
<mars> bigjools, ah, matsubara was to /says too late :)
<mars> at least the wiki is editable
<bigjools> fancy that
<mars> matsubara, passwords are still 'test'?
<matsubara> mars, yes
<tfmoraes> thanks
 * gmb EODs
<tfmoraes> Other question
<tfmoraes> If someone try to access my self-hosted launchpad
<tfmoraes> It doesn't locate the launchpad.dev
<maxb> It is not accessible other than from the machine it's running on
<tfmoraes> Is there a way?
<maxb> You'd have to dig into the guts and work out how to reconfigure it
<tfmoraes> any tip?
<maxb> no, I've only just convinced it to run in the default configuration myself
<tfmoraes> thanks
<matsubara> tfmoraes, there's a way to make it accessible in your LAN. does it help?
<tfmoraes> yes
 * matsubara looks for docs
<andrea-bs> Hi! Why utilities/rocketfuel-branch does not create stacked branches?
<sinzui> tfmoraes: When I want to QA my .dev instance from another machine on my how network. I edit <Proxy *> Allow from 127.0.0.0/255.0.0.0 in /etc/apache2/sites-enabled/local-launchpad
<sinzui> tfmoraes: you need to add an allow block for you lan
<tfmoraes> to all VirtualHost ?
<matsubara> tfmoraes, http://pastebin.ubuntu.com/224529/
<matsubara> I'll move the corresponding doc from our internal wiki to the public wiki
<matsubara> it just needs some cleanup first
<tfmoraes> thanks again
<joey> jtv: is that you are a bot? :-)
<joey> s/are/or/
<henninge> jtv: a bot
<jtv> joey: why does everyone want to do Turing tests today?
<henninge> joey: a bot
<henninge> ;-)
<joey> lol
<joey> it's just really late for you to be online jtv
<joey> here you go henninge, here's your 20 marks
 * joey laughs.
<jtv> joey: yes, I've been trying to land an RC branch for... a day and something.
 * henninge looks for a bank that will still trade them into Euros.
<joey> jtv: haven't seen you in ages it seems like. Doing well I hope.
<jtv> joey: great, thanks.
<jtv> joey: still getting used to not typing "Rinchen"
<launchpad> this better jtv ?
<launchpad> :-)
<Ursinha> lol!
<jtv> launchpad: no :)
<Rinchen> ok then
<jtv> ah, that's better
 * Rinchen laughs.
<jtv> Yup, just like old times
<Rinchen> nameserv just loves me
<Rinchen> I grabbed launchpad for the day we add IRC two-way comms into LP
<jtv> I usually get messages from the other two brothers, Nick & Chan Serv.
<joey> Yeah. Mine are all linked of course so they leave me alone, generally.
<maxb> Is it notmal to get Python MemoryError exceptions if your devenv is not beefy enough, or have I stumbled across one of the reasons why LP isn't supported on Python 2.5 yet?
<maxb> *normal
<henninge> jtv: EOD for me. You can see my work output in rosetta@ ;-)
<jtv> henninge: just spotted it.  Impressive!
<jtv> Gute Nacht
<henninge> jtv: but there is more in the queu for tomorrow. Some were over a month old, I just hope the delay is not annoying people more than it helps.
<henninge> jtv: TschÃ¼Ã!
 * henninge needs to learn some Dutch phrases ....
<jtv> gah, still no PQM?
<jtv> sinzui, still no luck with pqm/buildbot?
<sinzui> jtv: I did land to PQM, and herb whipped buildbot to work
<jtv> sinzui: my branch is still failing with /bin/false
<jtv> any idea why that might be?
<jtv> I thought it was the same problem you had.
<sinzui> jtv: I did a direct submit to PQM using the --submit-branch option for db-devel
<jtv> sinzui: oh, I've been trying to land on develâalso using pqm-submit.
<sinzui> jtv: devel will fail, only db-devel can be landed
<jtv> why is that?
 * sinzui thinks this is a bug in the process
<sinzui> because we do not have a sanity check in our build/deploy process to no not to deploy changes to edge if only db-devel is being used
<jtv> sinzui: so all this time, "/bin/false" meant "please submit to db-devel instead"?
<abentley> Does anyone know why versions.cfg for stable and db-stable requires bzr 1.17 when bzr+ssh://bazaar.launchpad.net/~launchpad/lp-source-dependencies/trunk/ only has bzr1.17rc2?
<tfmoraes> Other question
<tfmoraes> And the "bzr launchpad-login" ?
<tfmoraes> If I try that, it tries to log in the launchap.net
<thumper> tfmoraes: it doesn't log in, but it does check that the user exists and has ssh keys
<marc0s> hi all
<marc0s> should i care about a NoSuchRevision error branching pygettextpo?
<marc0s> using rocketfuel-setup
<mars> abentley, bzr blame said jml last updated the line, and I see 1.17rc1 here
<maxb> Hi, I'm trying to do a perfectly standard setup on LP on Jaunty, and am getting: ImportError: /home/maxb/launchpad/lp-sourcedeps/eggs/storm-0.14stub_storm_launchpad_288-py2.4-linux-i686.egg/storm/cextensions.so: undefined symbol: PyUnicodeUCS2_Format
<tfmoraes> I have logged in the launchpad with  foo.bar@canonical.com
<tfmoraes> and the username is name16
<mars> marc0s, you can always try rocketfuel-get after rf-setup finishes.  If there is a problem, it will re-fetch it.
<mars> tfmoraes, yep, long time holdover
<maxb> marc0s: IIRC someone said that is triggered by a bug in rocketfuel-get, now fixed. The first thing to try would be to ensure your LP devel branch is up to date, delete the pygettextpo local branch, and try again.
<tfmoraes> then I tried "bzr launchpad-login name16"
<tfmoraes> And it shows bzr: ERROR: The user name name16 is not registered on Launchpad.
<mbt> Just curious, does LP do administration in its UI, or is that all external to the Web app?
<pkern> There is some in the UI.  But not administration of LP itself, AFAICS.
<tfmoraes> But I've registered her ssh keys
<marc0s> thanks mars and maxb i'll try now with rocketfuel-get
<tfmoraes> and the user exists
<abentley> mars: I think you're out of date.  This was done in revno 8970
<mbt> Yeah, I was thinking a "superuser" type of user in LP that could go around and do things like delete projects, etc., though having a way to install a DB with zero data would probably serve a similar purpose.
<mars> abentley, yep 8969 here
<mars> mbt, some of that stuff has to be done in the DB directly
<mars> tfmoraes, are you trying to point bzr launchpad-login at your local LP instance?
<tfmoraes> yes
<mbt> mars: And I assume there are no scripts that do this?  I haven't had a chance to browse the entire source tree yet, but it seems that good 'ol psql is your friend there.
<kiko> mthaddon, did we unjam PQM?
<mthaddon> kiko: was it jammed?
<mars> mbt, you would have to ask the Launchpad Operational System Administrators (a.k.a LOSAs)
<kiko> I thought jtv suggested it was, but maybe he was confused
<mars> tfmoraes, I'm not sure you can do that with the basic plugin.  Might mean a change to the bzr source?  One of the bzr devs in here might know.
<jtv> kiko: it kept failing with "/bin/false", I thought sinzui had the same
<kiko> mthaddon, at any rate I think what's in buildbot is what will go live unless we find anything critical from then until now
<tfmoraes> mars, there is other way to checkout the source code?
<kiko> mthaddon, I've asked other leads to review the OOPS listings
<mthaddon> kiko: ok, cool
<abentley> mars: There's no need to do that.  bzr will use the default user from the ssh config, sabdfl
<mthaddon> kiko: 'Commit message [[release-critical=flacoste][ui=none][r=mwhudson][bug=401835] Fix\n\tcommit-translations-to-branch: repeated path error.] does not match commit_re [(?is)^\\s*(?:\\[(?:release-critical=[^\\]]+)\\]\\s*\\[(?:rs?=[^\\]]+)\\]|\\[rs=buildbot-poller\\])]'
<mars> tfmoraes, from a local LP instance?  not sure
<tfmoraes> mars, yes from local LP
<mars> tfmoraes, abentley might know, as he is on the code hosting team :)
<abentley> mars: ^
<mars> tfmoraes, <abentley> mars: There's no need to do that.  bzr will use the default <abentley> mars: There's no need to do that.  bzr will use the default user from the ssh config, sabdfluser from the ssh config, sabdfl
<mars> yay, thanks xchat
<kiko> mthaddon, that should work, no?
<jtv> no, the regex no longer expects the [ui=none]
<sinzui> mthaddon: herb: buildbot completed a build with RC fixes in it. Can staging be updated so that we can verify them?
<mthaddon> sinzui: does it include DB updates since the last staging update?
<sinzui> mthaddon: I do not think so. /me looks
<sinzui> mthaddon: no db changes. templates and browser code  and images.
<mthaddon> sinzui: revno 8312?
<sinzui> yes
<tfmoraes> mars, I added the name16 in ssh config
<mars> maxb, do you have /usr/include/python2.4/Python.h on your system?
<maxb> Hmm. I'm confused. How can anyone have a working LP devenv on Jaunty when Jaunty's Python is built using UCS4, but one of the LP eggs is built for a Python using UCS2 ?
<tfmoraes> Then I do a bzr branch, but it shows this error "You have not informed bzr of your Launchpad ID, and you must do this to ... "
<maxb> mars: yes
<maxb> tfmoraes: I think you should only be using bzr lp: urls for interacting with the real launchpad. You probably need to use http:// or bzr+ssh:// to talk to your dev instance
<tfmoraes> I will try
<pkern> I wonder how many knobs I need to push to get an uploader listed for a distroseries. o_O
<mars> maxb, how did you find that the egg uses UCS2?  Perhaps I could check my local setup in comparison
<maxb> I assumed that based on the text of the error message: "undefined symbol: PyUnicodeUCS2_Format". I could unzip the egg and run nm on the .so - /me does that now
<mthaddon> sinzui: staging updated
<sinzui> thanks mthaddon
<mthaddon> yah
<maxb> oh gosh, it's not a prebuilt egg, is it. Hrm
<maxb> Gah
<maxb> ok, I know what has happened, I didn't find all the places I needed to clean after accidentally building with the wrong python
<mars> ah, that could do it
<mars> maxb, yep, mine is built with PyUnicodeUCS4
<maxb> Hmm, rm -r lp-sourcedeps/eggs/* seems to have borked things quite a bit
<mars> just kill the eggs/ directory
<mars> it is a make target, 'make' will recreate it
<maxb> I seem to have confused the build, now I get an ImportError for pytz
<mars> do you have eggs/pytz-2009j-py2.4.egg?
<sinzui> EdwinGrubbs: ping
<maxb> well no, because I just deleted everything in eggs/ :-)
<EdwinGrubbs> sinzui: hi
<maxb> I've run "make clean; make" - it's doing stuff
<sinzui> Edwin, can you verify that IE 8 can create a project on staging?
<EdwinGrubbs> sinzui: yes, I already ran that test and marked it on the wiki.
<sinzui> rock
<sinzui> EdwinGrubbs: do you agree the sprite is fixed: https://staging.launchpad.net/launchpad-registry/2.2/+addrelease
<EdwinGrubbs> sinzui: yes, I hadn't seen that staging was updated with the last few revisions, but it looks good.
<sinzui> EdwinGrubbs: it just happened. That is why I am checking.
<sinzui> matsubara: ping
<matsubara> sinzui, hi curtis
<sinzui> matsubara: There are two registry bugs that are still bad, blocked by anther bug we are fixing. I don't think they are an issue because the feature is only used by admins, and the problem will be fix either in a reroll, or Monday when the fix lands on edge
<sinzui> matsubara: So I can move the bugs to the rc fixed area, or move them to a 2.2.8 test plan
<matsubara> sinzui, you mean the adminteammerge bugs?
<sinzui> yes, both of those
<matsubara> sinzui, remember that timeouts on staging have a lower threshold. are those timeouts under 25s?
<matsubara> and what is the other bug you're fixing that will fix both of those?
<kiko> sinzui, they are not rcfixed; they are BAD, I think
<kiko> sinzui, and then the decision is to roll out with those items as BAD
<sinzui> I have not tested them since they are blocked by the other bug. We believe the other bug is a performance issue for many timeout bugs
<sinzui> kiko: understood
<matsubara> sinzui, as kiko suggested leave them as BAD and make a note in the test plan pointing to the other bug that will solve both BADs
<matsubara> that way we'll know what we're still waiting for the re-roll
<sinzui> matsubara: kiko: registry is ready for the rollout
<matsubara> thanks sinzui
<matsubara> Ursinha, ^
<kiko> thanks
<Ursinha> me
<Ursinha> thanks matsubara
<maxb> Is there an open bug for making LP work on Python 2.5? Or any preexisting discussion I can read on the topic?
<kiko> maxb, gary will be working on that as soon as he has zope running from buildout
<maxb> So I should just keep an eye on this channel then?
<sinzui> maxb: look in launchpad-foundations. The issue is a really a story, not a bug, but it is probably in there
<maxb> Doesn't seem to be - oh well
<mars> maxb, nothing yet, but as kiko said, it's coming: https://dev.launchpad.net/VersionThreeDotO/Foundations
<maxb> sure, I'm just curious to follow it as it evolves
<Snova_> Why is rocketfuel-setup installing what appears to be an entire texlive setup?
<mars> texlive?
<Snova_> Well, lots of texlive-* packages, which are probably the reason apt-get is going to download 277 MB when I press y.
<maxb> I had a go at coaxing it to run under Python 2.5 on karmic myself, but am stuck on MemoryErrors being thrown from the depths of Zope/TAL
<mars> Snova_, somewhere in the launchpad-developer-dependencies chain I would guess
<mars> try installing firefox on a server, and you'll be astounded by what it pulls in :)
<EdwinGrubbs> bac: did your fix for the team merge timeout land?
<bac> EdwinGrubbs: no.  i'm not going to submit for this release
<Snova_> Ahah! I think it's python-epydoc; texlive stuff is in the Recommends.
<mars> Snova_, or it could be a build-depends - apt-rdepends shows some stuff there, too
<bac> EdwinGrubbs: delaying until PQM opens doesn't affect you does it?
<EdwinGrubbs> bac: well, it just means that I can't QA my fix for deactivating the members of a team, so it's really not a big deal.
<Snova_> mars: Well, installing it without the recommended packages seems to have gotten around it, though something else might want it later. It looks like it's only used for documentation, though.
<bac> EdwinGrubbs: yeah, i'm in the same boat for +adminteammerge
* mthaddon changed the topic of #launchpad-dev to: Launchpad will be down/in read-only from 22:00 UTC until 23:00 UTC for a code update | Launchpad Development Channel | https://dev.launchpad.net/ |  Please use #launchpad for support. | https://launchpad.net/~launchpad-dev | Get it: https://dev.launchpad.net/Getting | http://people.canonical.com/~herb/ | http://paste.ubuntu.com/
<shriven> hello. I'm trying to run the rocketfuel-setup and I'm getting errors like this:
<shriven> make: *** No rule to make target `install'.  Stop.
<shriven> ERROR: Unable to install apache config appropriately
<shriven> anyone know whats up?
<mars> shriven, could you please put the output of the 'make' error into a pastebin?  That will help us diagnose it
<shriven> sure
<shriven> mars: when you say the "make" error, what command are you envisioning me running to get said error?
<shriven> all I've done so far is ./rocketfuel-setup
<mars> shriven, just the output from rocketfuel-setup so far will do
<shriven> ok
<mars> I'd like to see the last lines, to get an idea about what it was doing
<shriven> http://pastebin.ca/1503603
<shriven> thats the last 10 lines or so
<shriven> I had previously finished the package installation and was just pking around
<shriven> so thats all there is
<shriven> currently the bzr login just sits... I let it try for a few minutes then hit cntrl-c to it
<shriven> and it moved on
<mars> it moved on, then dies with this error later?
<shriven> yea, just a few seconds later
<mars> lets start by checking what rocketfuel-setup is doing at this point
<shriven> sure
<shriven> just ran it again... this time bzr login seemed to suceed, same apache error though
<mwhudson> good morning
<mwhudson> shriven: are you a sudoer on this system?
<shriven> yup
<mwhudson> though it looks like the checkout must have failed
<shriven> its a VM
<shriven> well I don't see it doing any bzr work at all
<dobey> holas wonderful lp hackers
<mwhudson> oh
<dobey> i was wondering... is there any special reason that branch_merge_proposal.createComment() in the API a) requires the subject argument, and b) doesn't seem to behave the same as if i carete a comment via the lp web ui?
<mars> shriven, "sudo make install" in devel/ fails, I'm guessing
<mwhudson> shriven: it looks like getting a copy of the branch may have partially succeeded or something
<mars> shriven, check the root Makefile, you'll see what the 'install' target is doing
<shriven> there is no devel/
<shriven> maybe I missed a step?
<shriven> I thought I combed over it a few times....
<mwhudson> shriven: what files have been created?
<shriven> let me go re-read
<shriven> just rocketfuel
<mwhudson> shriven: ~/launchpad, ~/launchpad/lp-branches, ~/launchpad/lp-branches/devel ??
<shriven> nothing exists below launchpad except rocketfuel
<shriven> this is where I'm starting at, am I mistaken in doing this? https://dev.launchpad.net/Getting#Getting%20It
<shriven> I saw the alternate source code download... I am assuming it really is alternative and rocketfuel will download the source itself?
<mars> shriven, it will
<shriven> yeah, that must be the part that is failing for some reason
<shriven> there really isn't a target for make install
<mars> shriven, yes, odd, rf-setup should have failed on line 368
<mars> cd $LP_TRUNK_NAME
<shriven> yes, I'm just browsing that myself
<shriven> or earlier at 354?
<mars> yep
<shriven> hmmm is it hardcoded somewhere that we must have our launchpad directory at ~/launchpad?
<shriven> I put mine in ~/Applications/launchpad
<shriven> preferring to do this sort of thing there
<mwhudson> shriven: you can create a file ~/.rocketfuel-env.sh that sets some shell variables
<shriven> ah!
<shriven> yes, ~/launchpad/lp-branches has been created
<mwhudson> i think LP_PROJECT_ROOT=~/Applications/launchpad will do what you want
<mars> shriven, line 47+ in rf-setup
<shriven> ok
<shriven> I'll just use the default way
<shriven> thanks guys
<mars> mwhudson, do you remember what the 'bucket' argument is when creating an image?
<mwhudson> mars: is that an argument to ec2-bundle-vol?
<mars> yes
<mwhudson> no, it doesn't look like i passed it
<mwhudson> mars: are you sure you don't mean ec2-upload-bundle ?
<mars> mwhudson, ah, sorry, yes
<mwhudson> then it's the name of the ami, basically
<mwhudson> you want launchpad-ec2test10 or whatever
<mars> right, ok, thanks
 * mwhudson is extremely glad he kept his notes from when he did all the image diddling
<mars> I just did it from the AWS docs
<mars> mwhudson, here, I'll bundle the script I have, so you can take a peek
<mars> it automates image creation
<mwhudson> same here, but it took a while and some hopeful guessing
<mwhudson> ah, awesome
<mars> and instance configuration
 * dobey wonders about this createComment() method
<mars> mwhudson, lp:~mars/lp-dev-utils/add-windmill-image-generator/
<dobey> hrmm
<marc0s> i started over with rocketfuel-setup and i got an ssh key error after apache setup
<marc0s> Host key verification failed.
<marc0s> bzr: ERROR: Connection closed: Unexpected end of message. Please check connectivity and permissions, and report a bug if problems persist.
<marc0s> any clues on that?
<marc0s> i told my real launchpad username to rocketfuel-setup when it asked me for a launchpad username
<mars> marc0s, what was the last line of successful rf-setup output before the failure?
<mars> so I know how far rf-setup go
<mars> got along
<marc0s> mars: http://dpaste.com/70131/
<marc0s> so you have more lines to check
<marc0s> before that, some apache modules were enabled and apache reloaded
<mars> looks like it dies around line 373 maybe?
<mars> no, that can't be right
<mars> line 425+ somewhere
<marc0s> or maybe 361 ?
<marc0s> let me check 425 ...
<mars> marc0s, I would suspect line 485 then - rf-get itself may have died on you
<marc0s> yes, the supermirror one
<marc0s> if i launch rf-get it dies with the same error
<mars> if so, then no worries, you can just run it manually.  The rest of the configuration suceeded
<mars> good! :)
<marc0s> good? :)
<marc0s> how should i continue now?
<mars> that means we have narrowed the error down to a single script, probably ssh authentication
<marc0s> with the make schema && make run stuff ?
<marc0s> ok
<mars> well, not having rocketfuel-get may be a problem, it implies you can't fetch stuff over bzr+ssh.
<mars> marc0s, did you supply an SSH key for your launchpad user account?
<marc0s> not an ssh key from this machine, so i was asking if i should before :)
<mars> I think it will work to run an instance of LP, but you won't be able to check in code, or use rf-get
<mars> does deve/sourcecode have symlinks in it?
<mars> devel/sourcecode
<marc0s> not at all
<marc0s> just a makefile
<mars> hmm
<mars> rocketfuel-get runs 'utilities/link-external-dependencies'
<mars> which populates devel/sourcecode/
<mars> so you won't be able to proceed to 'make schema' without running that, or getting rf-get going.
<marc0s> hmm no good news :P
<marc0s> do you think uploading the ssh key will let me continue the bzr fetching process?
<mars> I'm not sure if it is the key that is the problem, or that it doesn't like bazaar.launchpad.net itself.  You may be able to add that manually to your known_hosts file
<mars> marc0s, on a whim, what happens if you try 'ssh bazaar.launchpad.net'?
<marc0s> i've uploaded the key and it seems to be fetching more stuff
<marc0s> i try
<marc0s> No shells on this server.
<marc0s> and connectoin closed
<mars> ah, nm then :)
<mars> you added the key?  and rf-get is running now?
<marc0s> it seems to, it's downlading :)
<marc0s> hope it finishes before the maintenance downtime scheduled to be in 10 minutes :D
<mars> I don't think that will affect it.  It should be a read-only switchover.
<mars> not full downtime?
<marc0s> ah ok
<salgado> sinzui, is distributionsourcepackagecache.py really meant to be in lp/registry?
<marc0s> i don't know just seen that on the web after uploading the key
<lifeless> how long will the maintenance be?
<lifeless> oh, 1 hour
<jml> hello everybody
<mars> good morning jml
<mwhudson> hello jml
<Shane_Fagan> I was wondering about apturl in launchpad. I was thinking of writing support for it because I would think it would be quite useful for answers. Would anyone be able to offer some guidance?
<salgado> Shane_Fagan, sure!
<Shane_Fagan> Ok so im downloading the source at the moment
<Shane_Fagan> So give me about 15 minutes and then ill start asking a few questions and hopefully get hacking after that
<mars> It would be cool if bzrlib could handle reconciliation between CLI options and CFG file options supplied to a script :/
<mwhudson> mars: lp:~mars/lp-dev-utils/add-windmill-image-generator/ looks fairly nice, if still a bit shell script ish
<mars> You have to manually merge optparse and ConfigParser into a new class to get that to happen with the stdlib
<mars> mwhudson, shell-ish? :)
<mwhudson> mars: now write a version that updates the ec2test-ish
<mwhudson> mars: well, maybe that's not quite right
<mwhudson> mars: copy-and-paste ish from ec2test
<mars> yeah, there is a good reason for that
<mwhudson> i can believe it
<jml> that reminds me
<mars> that reason being that I did not want to risk breaking ec2test at all, and I needed some features to be factored out into smaller bits
<jml> is ec2test ok to be a bzr plugin?
<mars> jml, no one has objected, but gary is away at OSCON - did he have anything to say on the matter?
<mars> he did have plans for the script
<marc0s> i think the downtime was total :) connection closed to bazaar.launchpad.net
<jml> mars, no definitive answer.
<mars> but then, he is also quite pragmatic - if a plugin is better and cleaner, I'm sure he would enjoy doing so
<barry> jml, mwhudson, thumper are you guys up for meeting in 29m?
<mars> jml, how were you going to land the plugin?
<jml> barry, yes
<mars> land/distribute
<barry> jml: fantastic
<jml> mars, simply change ec2test.py to a plugin, ask people to symlink it into ~/.bazaar/plugins
<mars> jml, why not land it beside ec2test.py as ec2test_plugin.py?
<mars> that way we can transition from one to the other
<mars> jml, set-based development :D
<jml> mars, "why not" is because it's accompanied by other refactorings to ec2test
<jml> to make it more module-like
<lifeless> sinzui: hai
<Shane_Fagan> Damn does bzr stop in read only mode too?
<jml> (and because I want to delete the current option parsing)
<mars> jml, ok, that still doesn't prevent you from landing one beside the other, does it?
<lifeless> Shane_Fagan: yes
<Shane_Fagan> Ah crap
<jml> mars, no, it just means more work.
<mars> lifeless, I take it read-only mode interrupts the SSH connections?
<lifeless> mars: we write to the db when the client disconnects, to trigger scanning, public mirroring, etc
<mwhudson> barry: yes
<jml> lifeless, not anymore, we write to db on Branch.unlock)(
<mars> lifeless, ok, so "yes"? :)
<mwhudson> i think codehosting is down during r/o mode at the moment?
<lifeless> jml: sftp :P
<mwhudson> not strictly necessary
<mwhudson> lifeless: the same
<jml> lifeless, same.
<lifeless> ok cool. I don't want to know how you determine unlock for sftp ;P
<jml> lifeless, I won't tell you
<lifeless> mars: I don't see the correspondence to SSH connections in what I said :)
<ajmitch> magic?
<abentley> thumper: http://www.w3schools.com/CSS/pr_text_white-space.asp
 * ajmitch would like bugs to still be visible when in r/o mode
 * jml too!
<mars> time to get the family supper
<mars> good night!
<marc0s> bye mars
<marc0s> thanks! :)
<Shane_Fagan> Is it ok that the install stopped like that or will it cause problems?
<Shane_Fagan> It was in the middle of getting the branch
<Snova_> rocketfuel-setup has failed while branching... I hope this hasn't been asked half a dozen times today already, but http://paste.ubuntu.com/225073/
<Shane_Fagan> It maybe because launchpad is in read only mode
<Shane_Fagan> Im getting this error http://paste.ubuntu.com/225080/
<mwhudson> barry: i need to pop out for a few minutes, maybe 10?
 * jml looks at the pastes
<jml> Snova_, yeah, sorry about that. Launchpad going down interrupted your pull.
<Snova_> jml: Just my luck. :P I'll try again tomorrow, I guess.
<jml> Shane_Fagan, you'll need to add an SSH key to your Launchpad account to get rid of that error, I think.
<jml> Snova_, it'll be up again in less than an hour.
<Shane_Fagan> I have one
<Snova_> jml: Yes, but this connection is rather slow, and it's getting late.
<jml> Shane_Fagan, is your Launchpad account called 'shane'?
<jml> Snova_, ahh, ok. no worries then.
<Shane_Fagan> shanepatrickfagan
<jml> Snova_, better luck tomorrow
<Shane_Fagan> Its worked every other time ive tried it
<jml> Shane_Fagan, Launchpad is trying to use 'shane' to SSH in.
<jml> "bzr: ERROR: The user shane has not registered any SSH keys with Launchpad."
<Shane_Fagan> Ah ill switch and see what happens
<jml> Shane_Fagan, type: 'bzr launchpad-login shanepatrickfagan' and you should be right.
<Shane_Fagan> Nope still the same error
<Shane_Fagan> Ill try it after it comes out of read only mode and see
<thumper> sinzui: ping
<sinzui> Hi thumper
<thumper> sinzui: I've got a question about layers and requests
<thumper> sinzui: I'm trying to write a test for my oops on demand branch
<thumper> sinzui: which needs a current browser request
<thumper> sinzui: how can I easily set one up in a unittest?
<sinzui> hmm. I think I had to subclass a TestRequest to do that
<barry> thumper, jml, mwhudson -> #launchpad-meeting in 2m
<thumper> sinzui: have you done it somewhere?
<thumper> sinzui: the thing is I'm using the from lazr.restful.utils import get_current_browser_request
<thumper> sinzui: in the code I'm testing
<sinzui> thumper: I am looking for an example
<thumper> sinzui: cool, ta
<jkakar> Aargh.
<jkakar> I just lost a huge bug comment because Launchpad is (presumably) in read-only mode and still allows AJAX "in-page" actions to run.
<Ursinha> jkakar, argh
<Ursinha> I'll file a bug about it, if find any
<sinzui> thumper: look at
<sinzui> from canonical.launchpad.layers import setFirstLayer
<sinzui> setFirstLayer(request, 'CodeLayer')
<jkakar> Ursinha: Hmm, I wonder if I'm wrong.
<sinzui> thumper: it is just a wrapper for:
<sinzui> directlyProvides(request, layer, directlyProvidedBy(request))
<intellectronica> jkakar: you mean you opened the collapsing form under the bug tasks table and tried to submit?
<jkakar> Ursinha: I was on a bug page, clicking "Add comment", entered my comment and then, after clicking "Submit", I got the "Launchpad Offline" page... but I'd loaded the page before LP went down.
<jkakar> intellectronica: Nope, I used the collapsing form for adding a comment.
<beuno> jkakar, we need to slap a launchpad.Edit on that link
<beuno> when LP comes back, we need a bug  :)
<jkakar> Ursinha: If I try and load it now I just get the "Launchpad Offline" page.
<jkakar> beuno: Ah, okay.
<intellectronica> jkakar: ah ok, so nothing to do with ajax
<jkakar> intellectronica: Maybe, I'm not sure.
<Ursinha> beuno, I'll file one
<intellectronica> ideally we should disable forms like that in read-only mode
<jkakar> intellectronica: Yes, exactly.
<intellectronica> but in this case, if you loaded the page before LP entered read-only mode that wouldn't have helped
<jkakar> intellectronica: The only thing is I loaded the page before LP went read-only, so how would it know in that case?
<jkakar> intellectronica: I guess it could ping every now and then to see if LP is alive, but that kind of thing is fraught with terribleness.
<intellectronica> with ajax, otoh, we could try, tell you that we couldn't save and ask you to try again later, so there's an opportunity for a better experience
<jkakar> intellectronica: Ah, duh, of course.  Use AJAX for the form submit.
<jkakar> intellectronica: That'd be nice, yeah.
<intellectronica> jkakar: yes. that's coming soon anyway
<thumper> sinzui: how does that help?
<jkakar> intellectronica: Awesome, thanks.
<joey> hey kiko, remember that matrix you put together recently of all the weekly meetings?  If you didn't already think of this, that would be good info to have on the dev wiki for the community devs don't you think?
#launchpad-dev 2009-07-23
<barry> mthaddon: i think i will ping the losas tomorrow to help with the mailing list archive regen.  hope the rollout goes smoothly!
<spm> barry: rollout is now done, and wasn't smooth :-)
<barry> spm: oh noes
<barry> spm: would you like to try converting one mailing list archive with me then?
<jml> spm, /topic or it didn't happen :)
<spm> jml: don't tempt me to make it not happen :-)
<jml> haha
<barry> spm: can you make it unhappen?
<jml> "launchpad unreleased, uptime was ten minutes. have a nice day"
<Shane_Fagan> jml: Its still giving me out http://paste.ubuntu.com/225202/ any ideas?
<spm> barry: technically we did :-(
<barry> jml: that should read: downtime was negative 10 minutes.  i think einstein proved negative downtime is equivalent to positive uptime
<barry> spm: :(
<jml> Shane_Fagan, 'sudo make install'
<barry> leonardr: bug 387487.  i'm going to uninvalidate it because this is exactly my problem
<ubot3> Malone bug 387487 in lazr.restful "Allow a subordinate entry resource under a resource where there would normally be a field" [Undecided,Invalid] https://launchpad.net/bugs/387487
<mwhudson> um
<mwhudson> openid doesn't seem very happy
<mwhudson> spm: OpenID discovery error: HTTP Response status from identity URL host is not 200. Got status 404
<spm> bleh
<mwhudson> https://login.launchpad.net/ is, indeed, 404in
<mwhudson> g
<spm> !!!
<Shane_Fagan> sudo make install what?
<jml> Shane_Fagan, in the Launchpad tree (~/launchpad//lp-branches/devel by default, I think), run 'sudo make install' on the command line.
<Shane_Fagan> Nope still didnt work and that folder is empty
<jml> hmm.
<Shane_Fagan> I think the prob is that launchpad went into read only mode when I was branching the first time and then bzr seg faulted
<jml> Shane_Fagan, I need more context for the error message.
<jml> Shane_Fagan, that's quite likely.
<Shane_Fagan> So this is the second time I tried it
<wgrant> pkern: You just need to add an ArchivePermission to add an uploader - there's even a webservice API to do it.
<Shane_Fagan> Give me a sec and ill pastebin the whole lot
<jml> Shane_Fagan, thanks.
<Shane_Fagan> jml: http://paste.ubuntu.com/225231/
<jml> ahh, I see.
<jml> Shane_Fagan, so this was from your second attempt, right?
<Shane_Fagan> Yep
<Shane_Fagan> And it got past it the first time
<jml> Shane_Fagan, that makes sense.
<jml> Shane_Fagan, the underlying bug is that rocketfuel-setup is pretty bad at recovery...
<Shane_Fagan> Ah
<jml> http://paste.ubuntu.com/225243/
<jml> Shane_Fagan, that's the snippet from rocketfuel-setup
<jml> you see line 12?
<jml> Shane_Fagan, because 'devel/' exists,  the script assumes you've got a full branch of Launchpad
<Shane_Fagan> Ah so can I remove it ?
<jml> Shane_Fagan, yeah, just the devel/ directory
<jml> Shane_Fagan, then try running the script again. it just might work.
<Shane_Fagan> Ill remove lp-branches too
<jml> Shane_Fagan, if you'd like, you probably don't have to though
<jml> and you might save some time by not removing it.
<Shane_Fagan> Ah that worked
<jml> Shane_Fagan, glad to hear it.
<Shane_Fagan> Maybe we can fix the rocketfuel-setup to be a little more flexible
<jml> Shane_Fagan, that would be wonderful.
<jml> Shane_Fagan, I can help you get a patch ready, if you'd like
<Shane_Fagan> Well first I want to hack something up for apturl because I want that for answers
<jml> *nod*
 * Shane_Fagan loves apturl
<jml> I've got to head afk for a bit now.
<Shane_Fagan> Sure thanks for the help
<jml> no worries.
<jml> Shane_Fagan, I look forward to seeing your apturl patch. there are a few Canonical Launchpad devs around for the next few hours, so just ask if you need a hand.
 * jml afk
<thumper> sinzui: still around per chance?
<thumper> jml, mwhudson: do either of you know of a documentation place in the new lp tree for general doctests?  Especially general pagetests?
<thumper> I'm wanting to put some readable docs about the ++oops++ stuff
<mwhudson> thumper: i guess all that stuff is still in canonical/launchpad
<thumper> mwhudson: hmm...
<thumper> mwhudson: that is where I have it now
<thumper> mwhudson: I'd love to get it moved
<mwhudson> thumper: canonical/launchpad/webapp doesn't have a new home yet
<mwhudson> afaik
<thumper> mwhudson: perhaps the build engineer has some work to do :)
<mwhudson> it belongs with that code really i guess
<mwhudson> heh
* spm changed the topic of #launchpad-dev to: Launchpad Development Channel | https://dev.launchpad.net/ |  Please use #launchpad for support. | https://launchpad.net/~launchpad-dev | Get it: https://dev.launchpad.net/Getting | http://people.canonical.com/~herb/ | http://paste.ubuntu.com/
<thumper> oh FFS
<thumper> why are our  pagetests so sucky
<lifeless> because they are doctests?
<gmb> because doing integration tests with sample data is a really bad idea?
<sinzui> because they test the wrong thing
<jml> thumper: I have some ideas about the OOPS stuff
<thumper> d) all of the above!
<thumper> jml: I've got it sorted now
<gmb> thumper: Common denominator: Because.
<jml> but I have to go, because my flatmate's student has turned up 50 mins early!
 * jml relocating
<sinzui> Changing page layouts /could/ break lots of tests. I propose we replace them instead of fix them
<lifeless> sinzui: hey
<sinzui> hi lifeless
<lifeless> sinzui: I would like to see bugs and blueprints be less different
<lifeless> sinzui: as in, I love bugs, but hate blueprints :)
<sinzui> lifeless: issue type?
 * sinzui feels the same
<gmb> sinzui: I agree, by and large, in a "it's one in the morning what the hell am I doing in #launchpad-dev" kind of way.
<gmb> (about the tests, that is)
 * gmb heads to bed.
<Ursinha> good night gmb
<lifeless> sinzui: I'd like to be able to say to a bug, 'depends on other bug'
<lifeless> and, 'I think this is a 5 hour problem'
<lifeless> and finally, get reports with that data
<thumper> lifeless: +1 on those
<lifeless> but thats all - the approval process stuff and so on isn't something we use in any of the projects I contribute to
<thumper> anyone know BeautifulSoup well?
<thumper> I want the second last html comment in the body
<sinzui> thumper: http://www.crummy.com/software/BeautifulSoup/documentation.html
<thumper> sinzui: thanks
<sinzui> lifeless: I don't hack on bugs, I sometimes hack on blueprints on my own time. I l want better dependency handling on both apps. I want  to qualify the dependency as essential, expected, or optional
<lifeless> I'd actually like blueprints to fade away, into a cleaner single app. But I know that isn't a universal viewpoint
<lifeless> hmm, why do you need a type on the dependency?
<sinzui> lifeless: I suggested that on more than one occasion. I was that was not likely to ever happen and that I should instead focus on common behaviour
<sinzui> lifeless: When I plan a big project, I want to say features  100% implemented. We cannot say that about our specs, not even if we break them into atomic parts because the dependency is absolute
 * thumper is almost finished with the oops on demand branch
<sinzui> lifeless: Many stories depend on common features (other stories), but the feature might be essential one and optional in another
<lifeless> sinzui: if its optional in the other, is it really a dep?
<lifeless> sinzui: I don't object to an optional type flag
<sinzui> I want a library of ideas that I can define features from using better dependancies
<lifeless> I just don't really get it :)
<sinzui> lifeless: I was toying with the idea of showing bugs linked by blueprint in an alternate view of the milestone page...I expect to have a week off next month
 * thumper afk for a while
<lifeless> sinzui: that would be interesting. It still causes duplication though, which is my major objection to blueprints
<sinzui> yes. I really want to solve that. I really don't want 4.0 stories to be all in a wiki, but I cannot bring myself to use blueprints
<lifeless> yup
<lifeless> and I don't believe its a matter of polish
<jml> back
<Shane_Fagan> So any canonical devs around I just have to ask a question
<jml> always just ask :)
<jml> questions are free :)
<Shane_Fagan> Oh ok I thought I should ask first
<Shane_Fagan> Ok so I want to handle apt:// just like the way http:// is handled so where should I be looking?
<lifeless> where do you want to handle apt://
<Shane_Fagan> answers
<Shane_Fagan> apturl would be a lot easier than going go sudo apt-get install package
<Shane_Fagan> Because then it takes the scary command line away from new users
<lifeless> do you mean "I want text typed into answers with apt:// urls to get rendered as <a href="...">, like happens for http://"?
<Shane_Fagan> Yep
<Shane_Fagan> Thats exactly what I mean
<wgrant> It uses fmt:email-to-html.
 * wgrant finds that.
<jml> ooh, I changed this code quite recently
<jml> canonical.launchpad.webapp.tales.FormattersAPI.text_to_html and above.
<wgrant> That's a nice regex.
<jml> Shane_Fagan, tests are in lib/canonical/launchpad/doc/displaying-paragraphs-of-text.txt
<Shane_Fagan> Oh good
<jml> './bin/test -cvvt displaying-paragraphs-of-text.txt' will run the tests
 * jml puts on his best Captain Planet voice,
<jml> "The power is yours!"
<Shane_Fagan> Good, its a feature I really want
<Shane_Fagan> jml: do you think it would be useful?
 * wgrant does.
<jml> Shane_Fagan, I reckon so.
<Shane_Fagan> Its something I complained a lot about myself because I couldnt see why it wasnt used
<Shane_Fagan> But I suppose ye have a lot better things to be hacking
<jml> subjectively better, perhaps.
<Shane_Fagan> So anyway ill go and make it. Oh and congrats on the open sourcing
<jml> Shane_Fagan, thanks :)
<wgrant> sinzui: There seems to be a bug in your bug closing script - you apparently just released Launchpad sinzui, not Launchpad 2.2.7.
<sinzui> wgrant: your are correct. I will look into it
<wgrant> The test suite makes my laptop unhappy :(
<mwhudson> the test suite makes everything unhappy
<mwhudson> but laptops particularly, yes
<jml> wgrant, unhappy how? select at least two from: morbid, maudlin, melancholic, stressed, depressed, upset, troubled, vexed
<lifeless> using ec2test
<wgrant> jml: All of the above.
 * wgrant likes the look of the 3.0 project page.
<wgrant> Interestingly, it seems to be returning to the 1.0 project group page style
<mwhudson> lifeless: we haven't granted non-canonicalers access to the ec2test image yet aiui, and it currently still requires access to devpad aiui
<mwhudson> i guess this should all change...
<lifeless> well, all the code to create an image is out there, isn't it? :)
<mwhudson> uh, i _guess_ ...
<wgrant> We also don't have the code.
<lifeless> hell, bzr's ec2test uses stock images
<lifeless> wgrant: you don't?
<jml> lifeless, bzr has fewer dependencies and requires less system configuration
<lifeless> :P
<lifeless> letse play spot the troll
<jml> lifeless, `but I agree, it's probably possible to make an ec2test.py replacement with currently public code.
<lifeless> <- the troll
<wgrant> lifeless: The ec2test stuff is in lp-dev-utils, which is still in the proprietary realm.
<lifeless> ah
 * mwhudson stares at buildbot's estimate for the end of this build
<wgrant> buildbot is also still private :(
<mwhudson> i don't know if there are any good reasons for that
 * wgrant wonders what is so special about the tests in lp.bugs.browser.tests.special
<mwhudson> jml: i don't completely understand your email about soyuz permissions
<mwhudson> (surprise!)
<jml> mwhudson, me neither.
<jml> mwhudson, how /what can I clarify it?
<mwhudson> in particular i don't understand how latest_published helps get a sensible archive
<mwhudson> my understanding is that the archives are: primary, partner and a bunch of ppas
<mwhudson> (is an archive tied to a distro?)
<mwhudson> and primary has a bunch of components (main, universe, etc)
<mwhudson> (ppas sort of pretend everything is in main?)
<wgrant> PPAs override everything to main.
<wgrant> partner only has stuff in parter.
<wgrant> primary can have stuff in anything.
<mwhudson> wgrant: partner is a component as well as an archive?
<wgrant> mwhudson: Yes. It's very broken.
<mwhudson> man, the crack must have been good in 2004
<jml> mwhudson, an Archive has a Distribution
<wgrant> Partner is a new thing.
<mwhudson> true
<mwhudson> well
<mwhudson> new, as in 3 years or so, right?
<wgrant> Two or three years, yes.
<jml> mwhudson, a Distribution also refers to a main_archive, so assume it's one Distribution to many Archives.
<Shane_Fagan> What version of postgres do you need?
<jml> Shane_Fagan, 8.3
<jml> mwhudson, a publication has a component and an archive
<jml> mwhudson, whether the archive is sensible for these purposes is beyond my ability to judge.
<mwhudson> jml: if i ask something about pockets, will i regret it?
<wgrant> bigjools suggested last night that it wasn't sensible, IIRC.
<jml> mwhudson, oh crap, I knew I forgot something.
<mwhudson> jml: it seems to _me_ (and what do i know?) that for an official source package branch, the archive you care about is the primary archive
<jml> wgrant, I derived from:
<jml> <bigjools> add a new property to it, latest_published (trivial, just return self._getFirstPublishingHistory
<jml>  then you have in your hands a component and an archive
<mwhudson> jml: and on the face of it, i don't see why latest_published might not refer to a ppa
<jml> mwhudson, in my relative ignorance, I'm inclined to agree with you.
<mwhudson> bug 347768
<ubot3> Malone bug 347768 in launchpad-code "Allow anyone with upload rights to write to a package branch" [High,In progress] https://launchpad.net/bugs/347768
<wgrant> mwhudson: Because they'll be excluded.
 * jml has only had a chance so far to ask the question and type up the summary.
 * mwhudson reads
<wgrant> It's already done in lots of places already.
<wgrant> ISourcePackage.latest_publishing would use ISourcePackage._getFirstPublishingHistory, which only gets it from the distro's official archives. No PPAs.
 * mwhudson chases things around a big
<mwhudson> bit
<mwhudson> wgrant: i see you are right
<mwhudson> but ehhh
<wgrant> It's all a bit strange.
<jml> I really think that verify_acl should be turned into a function that takes a person ('signer'), an archive and a sourcepackage.
<wgrant> jml: Pocket!
<jml> wgrant, it doesn't use pocket right now.
<mwhudson> isn't there something about the release pocket being read only after, well, the release?
<mwhudson> or is that done at a different level?
<jml> UploadPolicy.checkUpload, apparently
<mwhudson> i wonder if there's a way to kick off a build of the bzr builder without waiting for 22.5 hours
<jml> cprov, hi
<cprov> jml: hi, give me one minute, I need coffee.
<jml> cprov, np
<jml> http://paste.ubuntu.com/225594/ -- diff against db-stable
<jml> but that's got a dependent branch
<jml> I'll split the diff up.
<wgrant> ICanHasLinkedBranch. Nicely done.
<cprov> jml: k
<jml> wgrant, :D
<mwhudson> wgrant: there is also IHasLinkedBranch, which is _entirely_ different
<jml> http://paste.ubuntu.com/225599/ -- that's the new branch, official
<jml> mwhudson, yeah, we need to fix that.
<cprov> jml: right, you already have 'suite_sourcepackage' which is SP (SPN + DS) + Pocket
<jml> http://paste.ubuntu.com/225601/ -- that's the ubuntu-packages branch, which makes lp:ubuntu/package work.
<jml> cprov, yes.
<jml> cprov, I want two things:
<cprov> jml: for upload ACL purposes we theoretically also need IArchive
<jml> a function from (suite_sourcepackage, person) -> boolean, "can this person upload"
<jml> and a way of granting a person upload permissions, to be used in the tests
<jml> I think the second is much more straightforward than the first
<jml> cprov, so where do we get the archive from?
<cprov> jml: okay, having a SomenthingMeaningful.canUpload(person) available is more valuable for testing and doesn't exclude the possibility of it bound to the lp.Append permission on the object later, for UI purposes.
<jml> +1
<cprov> jml: as mwhudson pointed we can safely assume PRIMARY (distribution.main_archive) for all DSP not published in the 'partner' component.
<jml> ok.
<cprov> jml: it's never overridden.
<cprov> jml: I don't see this assumption making ppa-package-branches any harder to implement in the future. Do you ?
 * jml has less than 20 minutes of battery life left
<jml> cprov, I don't.
<cprov> jml: once you have an suitable archive available (or assumed) you can figure out the current component (archive.getPublishedSources(spn, series)[0].component ... please make it elegant :))
<jml> cprov, so, I'm really very strongly thinking of refactoring verify_acl into a function that takes archive, a person (signer), and a SuiteSourcePackage.
<jml> cprov, and raises exceptions instead of calling reject()
<jml> cprov, would this be appropriate to use (probably together with some checkUpload() equiv.)
<jml> for package branch permissions.
 * jml has less than 10 minutes of battery life left
<cprov> jml: we can to that, but it's very likely that it will be something else than the current spread checks.
<jml> why would it be different?
<cprov> jml: in a way we would port nascentupload to use this new thing too
<jml> cprov, right.
<jml> cprov, it would be making verify_acl call a function that embodies all the logic of 'can this person upload to this pocket of this package in this archive'
<cprov> jml: right
<jml> I'm kind of surprised you don't have a function like this already, tbh.
<jml> cprov, anyway, if you think that's a good idea, I'll make it happen.
<cprov> it's definitely a good idea and you can count on our help.
<jml> cool.
<jml> I'm going to move to a power point. back in 10
<wgrant> It would be useful to have on the dev wiki a list of who is in which team, and in which timezone they reside.
<mwhudson> cprov: soyuz, sleep is for the weak standard time
<cprov> mwhudson: I confess sleeping is more tempting at home, but not that much ;)
<jml> back.
<jml> let's try that again.
<jml> cprov, are there any tests for verify_acl?
<cprov> jml: yes, blackbox-style, though
<cprov> jml: lib/lp/archiveuploader/tests/test_uploadprocessor.py
<jml> thanks.
<cprov> jml: one of the advantages of extracting and encapsulating that code would be allowing us to have quick and beauty unittests
<jml> yes, I was just thinking thta.
<jml> this is big enough for another branch, I think.
 * cprov nods
<mwhudson> jml: can i get you to review a branch?
<mwhudson> i'm not quite sure if i've got enough tests, but i guess you can tell me that when you review it :)
<jml> mwhudson, sure thing.
<mwhudson> jml: https://code.launchpad.net/~mwhudson/launchpad/puller-job-scheduling/+merge/9174, you should have mail
 * wgrant has today developed a particularly strong dislike for tests that rely on dict ordering.
<thumper> OMG!
<thumper> we're getting a sysadmin guy in Wellington
<thumper> spm: You'll have someone to hit up locally now :)
<ajmitch> oh fun, more kiwis
<jml> wgrant, you haven't found them in Launchpad, I hope.
<lifeless> \o/
<lifeless> thumper: who is it/
<thumper> lifeless: Paul Collins, starting 10 Aug
<lifeless> excellent name :)
<mwhudson> thumper: heh, you take longer to get through your email than me
<wgrant> jml: I have. It's not directly printing a dict, but rather generating a list of bug notification emails based on dict order.
<jml> :(
<spm> thumper: aye
<thumper> mwhudson: :)
<jml> wgrant, I'll gladly review a patch to fix that.
<spiv> wgrant: any chance of a patch to fix it? :)
<wgrant> I attempted to change the dict keys (previously strings) to Storm objects, and they hash differently each time...
 * wgrant will attempt to fix this.
<lifeless> wgrant: compare two dicts
<wgrant> But it's a very very long doctest.
<lifeless> wgrant: or use sorted() on both sides
<wgrant> lifeless: Right, I plan to use sorted.
<wgrant> Then I have to reorder a thousand lines of doctest.
<lifeless> welcome to doctest
<lifeless> have I told you what I think of them?
<mwhudson> wgrant: you are aware of the "shame" motivation for not open sourcing, i presume? :)
<wgrant> I think I saw you complaining about them in here recently.
<wgrant> mwhudson: Heh.
<lifeless> wgrant: oh, I don't complain, unless I'm forced to touch em.
<wgrant> Some of the Soyuz doctests were good explanations.
<spiv> Heh.
<wgrant> But bugnotification-sending.txt is ... not.
<jml> we do have a helper for testing dicts in doctests
<jml> I think
<jml> I've forgotten what it's called.
<ajmitch> it's not that helpful then?
<wgrant> It's not a dict when the doctest gets hold of it.
<wgrant> That would be too easy.
<wgrant> It's a list of strings which happens to have been created by iterating through a dict.
<mwhudson> wgrant: lib/lp/code/doc/branch.txt is particularly terrible
<ajmitch> wgrant: this is the part about email_notifications in there?
<wgrant> ajmitch: There's a lot more than one of those, but yes.
<jml> ajmitch, I don't write many doctests.
<thumper> mwhudson: are you suggesting wgrant cleans up our crappy tests?
<mwhudson> not really
 * jml back to package-permission-love
<jml> cprov, any thoughts on which module this code should live in?
<ajmitch> it's interesting that https://code.edge.launchpad.net/~launchpad-pqm/launchpad/devel disappears from the default view as it's set as merged now
<jml> ajmitch, oh, that's easy enough to fix
 * jml fixes
<thumper> grr
<ajmitch> which sort of makes sense, but I was briefly wondering why it disappeared  :)
<thumper> jml: can you create a devel series for launchpad?
<jml> thumper, I think so.
<thumper> jml: I looked but couldn't figure out how
<thumper> that way lp:launchpad/devel will work
<thumper> ajmitch: it is the branch scanner being clever :)
<thumper> ajmitch: it doesn't update series branches
<cprov> jml: I guess archiveuploader is the right place.
<jml> cprov, thanks.
<jml> thumper, https://edge.launchpad.net/launchpad/devel
<thumper> jml: just out of interest, where is the link to create a new series?
<jml> thumper, https://edge.launchpad.net/launchpad "Register a series"
<thumper> also I noticed that konqueror formats code review comments like FF3.5, with a non-wrapping proportional font
<jml> which python has class decorators?
<wgrant> 2.6?
 * jml tests
<mwhudson> spm: a question
<spm> mwhudson: an answer
<jml> yeah 2.6
<mwhudson> spm: i'd like to produce some data inside the internal xmlrpc server that i'd like to end up on a graph
<spm> mwhudson: 42
<mwhudson> spm: any idea how to do that?
<spm> mwhudson: sure - point us at a way of getting the numbers, or numberise?
<mwhudson> spm: of course, i should simply dump the data into our separate-from-main-database *cough*
<mwhudson> spm: i'm thinking more where to store the data
<spm> oh
<spm> is this something we'd ping on a regular basis?
<mwhudson> mmf maybe it can wait until we use the job system properly, then we can pull it from the database
 * jml is curious
<mwhudson> spm: to be specific
<mwhudson> spm: there's this method, acquireBranchToPull, that returns a branch that's waiting to be pulled
<mwhudson> spm: i'd like to log somewhere how long it's been waiting
<mwhudson> right now there's not really anywhere in the db to log this
<spm> mwhudson: hmmm
<mwhudson> i think using a BranchJob probably really is the right answer
<mwhudson> jml: is this making sense to you?
<spm> is this logged anywhere? request/start/fin times?
<jml> mwhudson, yeah.
<mwhudson> spm: currently, no
<jml> mwhudson, 'how long it's been waiting for' is interesting, but it's not ultimately the value we'd like to optimize.
<mwhudson> jml: which is what?  how long from request until completion?
<jml> mwhudson, for hosted branches, it's how long from push finished until pulled&scanned.
<mwhudson> jml: that's less under our control though
<mwhudson> jml: for example, jelmer's open office branch was always going to take ages
<mwhudson> jml: from my pov as an integrator, i think all i can do is squeeze on the time spent not working
<jml> mwhudson, well, I don't mean to say it's either / or.
<mwhudson> jml: ok
<spm> jml: to throw the question back at you - if you know that inro (push fin'ed to pull/scan) what action(s) would you expect to take given that info?
<spm> s/inro/info/
<jml> spm: if that were all the information I had, I'd ask for more numbers :)
<spm> jml: heh
<jml> spm: but it is the number that we ultimately care about
<spm> is a serious question tho - the difference between reporting vs analysis. the numbers are useless in and of themselves in a report. it's the actions those numbers drive that matters. Else it's so much "Gee Whiz"
<jml> spm: in this case, because there are so many parts to the system, and because the design for quite a few of those parts is changing, it makes sense to have an overall thing that measures what we value
<spm> certainly. so we can? add more servers? speed up the code?
<jml> spm: we know the problem fairly well, so we are unlikely to do a local optimization that causes a global sub-optimization, but it's still worth being careful of it
<spm> both :-D
<mwhudson> spm: a particular knob to tune is how many branches we pull in parallel
<jml> spm: see the impact of bzr upgrades, see the impact of branches moving to a new format, see how database changes affect the scanner etc.
<spm> coolio. I wasn't objecting btw, just asking in a way to help drive a foxus on what you *really* need, vs want. if you ken. :-)
<thumper> damn firebug!
<jml> spm: I understand.
<spm> foxus. sigh. focus :-)
<thumper> it lies!
<jml> spm: I've been wanting to do it for a very long time :)
<thumper> firebug tells me it is using white-space: pre-wrap
<thumper> but konq view source tells me no
<thumper> (and the source code itself)
<spm> thumper: "computer say no"?
<jml> spm: the principle I'm working from is "measure what we value".
<spm> jml: for sure! I try and hit things like that with the "So What?" question. This value goes up. So what? What do we do differently? Goes down? same. Is *great* for alerts. We get an alert on codebrowse for example - do we ignore or do an action. if we ignore, why have the alert in the first place.
<jml> right :)
<spm> so back to the original. if we want to know how many branches to pull in ||, can we get that closer to the branchscanner itself? proxy it via load? type of thing.
<spm> would throughput of all branches scanned be better? vs time/delay?
<spm> or just buy honking 64 core boxes with PT's of ram and bugger it all :-P
<mwhudson> argh
<mwhudson> we seem to have two interfaces called IBranchPuller :(
<jml> mwhudson, sorry
<jml> mwhudson, that's my fault.
<mwhudson> oh well
<jml> spm: I don't understand "get that closer to the scanner"
<spm> jml: heh. sure. what I meant was, is it better to look down in the weeds - at a single process (vs daemon) vs the holistic start to finsih.
<jml> spm: we want to combine the puller and the scanner, to reduce the amount of dead time.
<spm> right
<jml> spm: we ought to do both!
<jml> spm: we know we are done when the overall number is sufficiently low.
<spm> soudns fair to me - cause that way you'll know if holistic is better throughput etc; and detail is more efficient.
<jml> (of course almost everywhere we're saying 'number' here, we mean 'distribution')
<spm> :-)
<spm> so... have a start of holistic timestamp logged in a table; end of same; and maybe reuse same table(s) (normalised) to get the detail for individual jobs?
<spm> the branch scan took X-Y seconds; the entire start to finish request took A-B seconds?
<mwhudson> the main problem with the scanner is the fucking branchrevision table
<jml> yeah, something like that
<spm> the former would be useful for code performance improvement feedback
<mwhudson> unfortunately, we need to do design to fix this :(
<spm> mwhudson: nah. we're open source now. a solution will magically arrive for us :-P
<mwhudson> spm: hah
<jml> spm: you've seen https://dev.launchpad.net/Code?action=AttachFile&do=view&target=codehosting.png right?
<spm> jml: verra briefly, yes. it didn't seem complicated enough tho
<jml> :)
<jml> spm: I left out the Cthulhu's
<spm> for shame
<jml> (ever since then I've been cursed with superfluous apostrophe's)
<spm> doesn't lh talk direct to xmlrpc?
<jml> spm: http://paste.ubuntu.com/225869/
<wgrant> Nyargh.
<mwhudson> spm: yes
<mwhudson> spm: apache does too
<wgrant> Can I get doctest to avoid doing its diffing thing and just give me the raw output, or do I have to run the test manually myself?
<mwhudson> well, for the moment
<spm> mwhudson: :-)
<mwhudson> soon it will talk to the db
<spm> mwhudson: apache?
<mwhudson> spm: branch-rewrite.py
<mwhudson> spm: it's almost apache
<spm> ah
<jml> I have the svg if you'd like to update it.
<jml> wgrant, that, I don't know.
<wgrant> Because it makes an awful mess of the diff.
<jml> wgrant, the version of zope.testrunner we use is a little obsure.
<lifeless> !
<lifeless> stop trolling for understatement of the day award
<wgrant> Oh.
<wgrant> '...' doesn't match '<BLANKLINE>'
<wgrant> It would seem.
<thumper> wgrant: well, it depends
<wgrant> That would have been quick to work out if it hadn't been intelligent and tried to make a minimal diff.
 * jml gets up to pay more rent on the table.
<jml> gah signing
<thumper> jml: coffee?
<jml> thumper, red bull, actually.
<thumper> heh
<thumper> how do I request a QA review for browser compatibility?
<mwhudson> i see ursula's connection gave up at an appropriate time :)
<jml> haha
<cprov> g'night, guys.
<wgrant> I need to make a change to the test sampledata. Can I just patch current.sql directly, without having to create a patch?
<mwhudson> wgrant: yes, in fact you certainly shouldn't create a patch
<mwhudson> (as patches get applied to prod)
<BjornT_> wgrant: you also shouldn't patch current.sql manually. you should do the changes using the web ui, or some other api, and then run 'make newsampledata'
<wgrant> BjornT_: I would normally (after reading the README), but this was just changing a false to a true.
<wgrant> Or is there another reason?
<mwhudson> spm: got a few minutes?
<thumper> jml: is bug 390563 fixed for us now?
<ubot3> Malone bug 390563 in bzr "absent factory exception from smart server when streaming 2a stacked branches" [Critical,In progress] https://launchpad.net/bugs/390563
<BjornT_> wgrant: sometimes changing one value requires changing other values as well (think of the case where the attribute is a property and does some more than just setting the attribute to True). therefore it's a good policy to always use make newsampledata, so that you don't risk creating inconsistent data
<wgrant> BjornT_: OK, thanks.
<jml> thumper, don't know, sorry.
<lifeless> thumper: its as fixed as it was before
<lifeless> thumper: which is, there is a bandaid in bzr, and you've deployed the bandaid. Theres a larger fix coming, but you shouldn't need it as it covers edge cases only.
<lifeless> thumper: that bug *is not the one to do with bundles*
<lifeless> I've just retitled it to make it clearer
<lifeless> https://bugs.edge.launchpad.net/bzr/+bug/390563
<ubot3> Malone bug 390563 in bzr "absent factory exception from smart server when streaming 2a stacked branches" [Critical,In progress]
<lifeless> https://bugs.edge.launchpad.net/bzr/+bug/390563
<lifeless> https://bugs.edge.launchpad.net/bzr/+bug/390563
<lifeless> cmon ubot3
<lifeless> https://bugs.edge.launchpad.net/bzr/+bug/390563
<spiv> I assume ubot3 tries to spam the same info more than once every few minutes?
<lifeless> https://bugs.edge.launchpad.net/bzr/+bug/390563
<spiv> tries *not* to, rather...
<ubot3> Malone bug 390563 in bzr "absent factory exception from smart server when streaming 2a stacked branches" [Critical,In progress]
<lifeless> its also reading from an out of date replica :(
<spiv> Looks like about a 60s timeout :)
<lifeless> or something
<spiv> <not-ubot3> Malone bug 390563 in bzr "overly large delta creation when fetching from 2a repositories" [Critical,In progress]   :P
<lifeless> :)
<mwhudson> thumper: branch mail is different!
<mwhudson> i guess you knew that
 * wgrant pushes a branch.
<wgrant> Actually, I should probably run the whole test suite first...
<mwhudson> wgrant: i'm sure a launchpad dev will run it through ec2test for you
<mwhudson> (until we work out how to get other people able to use it)
<wgrant> I'll just run the bugs suite here. It won't take too long.
<ajmitch> how much space is required to play around with launchpad on a laptop? I'd rather not delete too much :)
<mwhudson> sounds optimistic, but ok :)
<wgrant> mwhudson: I ran it before.
<wgrant> And I need to have lunch now.
<mwhudson> ajmitch: nothing much beyond the space for the source
<ajmitch> alright, I can hopefully rsync from my desktop then
<ajmitch> & try it out on my hardy laptop, this should be interesting
<mwhudson> spm: can you run select count(*) from branchjob, job where job_type = 5 and branchjob.job = job.id group by job.scheduled_start - job.date_created; for me if you get the chance?
<mwhudson> spiv: what's the status of network deltas for 2a?
<spiv> mwhudson: waiting for review, I think it's ok to merge, but we'll find out what the review says... :)
<mwhudson> spiv: cool
 * mwhudson EODs
<spm> mwhudson: 2 rows. 664 and 186
<mwhudson> spm: i guess it would have been more useful to say
<mwhudson> select job.scheduled_start - job.date_created, count(*) from branchjob, job where job_type = 5 and branchjob.job = job.id group by job.scheduled_start - job.date_created;
<mwhudson> spm:  but good
<spm> mwhudson: unknown for the 664, 7 days for 186
<mwhudson> huh strange, should be 0 for the 664
<mwhudson> select job.scheduled_start, job.date_created from branchjob, job where job_type = 5 and branchjob.job = job.id limit 25;
<mwhudson> spm: try that?
<spm> weird. start dates are oft null
<spm> mwhudson: https://pastebin.canonical.com/20226/
<mwhudson> oh right
<mwhudson> that actually makes sense
<mwhudson> spm: update job set scheduled_start = date_created + '7 days'::interval where job.id in (select job.id from job, branchjob where job.id = branchjob.job and branchjob.job_type = 5) and job.scheduled_start is NULL;
<mwhudson> spm: should update 664
<spm> mwhudson: errr... in prod?
<mwhudson> spm: yes, i should get approval i guess :)
<spm> please :-)
<mwhudson> or we can just wait a week
<mwhudson> anyway, i really need to eod
<spm> your call :-)
<mwhudson> i'll think about it tomorrow :)
<spm> no worries, g'night
 * thumper EODs too
<thumper> mwhudson: branch email may still need tweaking
<thumper> mwhudson: I'm not sure we should show superseded ones
<thumper> mwhudson: and perhaps an extra blank line between merge authors and proposals
<spm> night thumper
<spm> jtv: re that db user translationstobranch - is master only access enough? or do you need the slaves as well?
<jtv> spm: slaves as well
<spm> kk
<jtv> spm: I deliberately used slave stores as a way to limit the risk of accidental rightsâbut nowadays scripts actually seem to map slave stores to slave dbs.
<jtv> may help with scalability too, of course. :)
<spm> hell yes - on the latter. the former to for that matter :-)
<lifeless> network deltas are something we probably want to cherrypick
<wgrant> Does anybody feel like ec2testing a branch for me?
<jml> wgrant, sure.
<wgrant> jml: lp:~wgrant/launchpad/team-verbose-bugnotifications-bug-253788
<jml> wgrant, against devel or db-devel?
<wgrant> jml: devel
<jml> wgrant, what email address do you want the results sent to?
<wgrant> jml: me@williamgrant.id.au
<wgrant> How long will it take?
<jml> wgrant, 20-30mins until I know the tests are running for certain, then 2-3hrs.
<jml> I recently changed the default instance type (it used to take 4.5hrs), but I haven't done any rigorous timings.
<wgrant> jml: Great. Thanks.
<wgrant> Is it reasonable to create a separate branch for a trivial fix, or should it be thrown in with some other related one?
<spm> wgrant: i'd suggest perfectly reasonable - we used to do similar with our configs
<thumper> jml: got a contributor agreement?
<jml> thumper, I don't need one to run tests, do I?
<thumper> no
<thumper> just to land it :)
<thumper> night all
<wgrant> Night thumper.
<jml> wgrant, so, if you want to get your patch landed, you have to speak with my manager, thumper, who'll arrange a contributor agreement :P
<wgrant> I've signed one for LAZR, but I suppose I need another.
<jml> thumper, g'night :)
<mwhudson> i don't think the agreements are per-project
<mwhudson> might be wrong there
<stub> -o, --no_save Do not save shutdown state
<stub> jml: What does that mean?
<mwhudson> stub: you want -o
<mwhudson> though with tac files it probably doesn't make any difference?
<wgrant> -y implies -o
<wgrant> So you don't need it.
<mwhudson> tap files
<jml> stub, back when tap files (pickled application objects) were all the rage, twistd used to save the state of the server as a pickle when it finished.
<mwhudson> what a good idea they were
<lifeless> \\\not///
<noodles775> Morning
<wgrant> Morning noodles775.
<noodles775> Hiya wgrant :)
<jml> wgrant, ok, those tests are definitely running
<wgrant> jml: Excellent. Thanks.
<jml> wgrant, you should get an email later this evening
<jml> wgrant, np
 * jml is off for the evening
<noodles775> wgrant: is that your first branch being tested? :)
<noodles775> Night jml
<wgrant> noodles775: It is!
<noodles775> wgrant: Excellent stuff!
<wgrant> The diff is much larger than I expected (due to test changes), but it worked eventually.
 * noodles775 looks for the MP
<wgrant> No MP yet.
<wgrant> https://code.edge.launchpad.net/~wgrant/launchpad/team-verbose-bugnotifications-bug-253788
<noodles775> Looks like a nice clean chang!
<noodles775> wgrant: have you had a chance yet to see the test factory? I noticed you've got some new sample data there (I didn't check, perhaps it was necessary?)
<wgrant> Apart from the 840 lines of test changes...
<wgrant> Test factory?
<noodles775> wgrant: gee, I must have missed that revision (I was just checking revs through the web ui)
<noodles775> wgrant: yeah, it's great! take a look at lp.testing.factory, you can use factory.makePerson etc.
<noodles775> which means not having to update sample data all the time (and tests aren't dependent on sample data changes etc.)
<wgrant> noodles775: Ah, useful.
<wgrant> What defines whether I can write to a particular table in a test?
<noodles775> wgrant: basically who you're logged in as... so in some tests you'll see "login('foo.bar@canonical.com')" to get admin access...
 * noodles775 looks for a good eg
<noodles775> wgrant: lp/soyuz/browser/tests/archivesubscription-views.txt for a doc-test eg.
<BjornT_> noodles775: actually, not really ;) that defines whether you can write to an object, not to a table. whether you can write to a table depends on which db user the test uses.
<wgrant> Right, in this case it was a postgres permission denied.
<wgrant> Nothing Zopey.
<noodles775> BjornT_: Ah, misunderstood...
<wgrant> What controls which user it runs as? Some tests must be able to write to person.
<noodles775> BjornT_, wgrant : in which case, an example that I know of where I had to change db users is lp.registry.tests.test_distributionsourcepackage...
<noodles775> See the call there to switchDbUser('karma')
<wgrant> Ah, I see.
<wgrant> And DB user privileges are defined in security.cfg?
 * BjornT_ would be more helpful if he wasn't enjoying his breakfast
<noodles775> wgrant: sorry, lib/canonical/config/schema-lazr.conf matches the actual db usernames to those seen in switchDbUser().
<wgrant> noodles775: Ah, thanks.
<wgrant> http://www.canonical.com/contributors says I need to send my contributor agreement to kiko - is that right?
<noodles775> wgrant: Yup, I guess so.
<wgrant> I wonder if I do have to do it again...
<noodles775> wgrant: just check with kiko later.
<wgrant> noodles775: A good idea.
<adeuring> morning
<noodles775> Morning adeuring ;)
<adeuring> hi noodles775 ;)
<gmb> Morning folks.
<Knut-HB> morning
<noodles775> Morning gmb and Knut-HB
<mrevell> Howdy open sourcers :)
<wgrant> Evening mrevell.
 * wgrant just took his first bug.
<mrevell> woo, cool
<gmb> Hullo mrevell. Welcome back to a new, freedom-loving fold.
<mrevell> gmb: Howdy
<henninge> mrevell: Congratulations!
<henninge> mrevell: and Welcome to the club ... ;-)
<mrevell> thanks :)
<bigjools> goooooooooood morning open sourcerers
<noodles775> :) Hi bigjools
<bigjools> Knut-HB: hello - how did your re-download go?
<allenap> Morning mrevell!
<mrevell> yo allenap
<mrevell> Man alive, I have some email to get through.
<Knut-HB> bigjools, didn't do it by now. i just connected to the system via two ssh-connections
<bigjools> Knut-HB: ok, let me know.  I was having dreams about this problem last night!
 * allenap hi-5s wgrant for doing bug 253788.
<ubot3> Malone bug 253788 in malone "Bug mail should use my verbose_bugnotifications, not the team's" [Medium,In progress] https://launchpad.net/bugs/253788
<Knut-HB> bigjools, ok, re-running rocketfuel-setup
<BjornT_> wgrant: nice! did you talk with someone about the implementation, before starting it?
<bigjools> BjornT_: do you have any LP API scripts to mark bugs fix released?
<BjornT_> bigjools: it's not mine, but i use lp:~launchpad/lp-qa-tools/bug-editor
<bigjools> BjornT_: okidoki, thanks.  Saves me writing one :)
<bigjools> BjornT_: was that script written by Diogo "karma thief" Matsubara?
 * noodles775 releases his bugs quickly before bigjools runs the script
<bigjools> heh
<BjornT_> bigjools: yeah, i think so :)
<Knut-HB> bigjools, ok, rocketfuel-setup finished with no errors
<bigjools> Knut-HB: ok let's take a deep breath and try make schema
<bigjools> (you did blow away lp-branches first before rf-setup?)
<Knut-HB> bigjools, i renamed it
<bigjools> ok
<Knut-HB> bigjools, "test@test-desktop:~/launchpad/lp-branches/devel$ make schema" like this?
<bigjools> yep
<Knut-HB> bigjools, "utilities/shhh.py LPCONFIG=development /home/test/launchpad/lp-branches/devel/bin/py -t buildmailman.py" this line is new, i didn't see it yesterday
<bigjools> looking good
 * ajmitch has an issue with that bin/py just not existing
<bigjools> ajmitch: buildout makes it, you need to run "make" or "make build" I forget which
<ajmitch> bigjools: it runs buildout, but bin remains stubbornly free of anything but buildout
<ajmitch> however I am trying this on hardy at the moment
<bigjools> ok make clean, then make
<ajmitch> oh nice, an error about having zope.interface already
<ajmitch> http://paste.ubuntu.com/226529/
<bigjools> ummm
<ajmitch> just running rocketfuel-get to make sure I've got the latest - lp-branches & lp-sourcedeps were copied over from another computer due to issues fetching them
<bigjools> ajmitch: ok, see how it goes.  Can you try "rocketfuel-branch testbranch" and see what happens if it still fails.
<ajmitch> ok, will be a couple of minutes, bzr from here to london is not fast
<ajmitch> (still waiting on rf-get)
<Knut-HB> bigjools, looks like no error was produced this time :D
<bigjools> Knut-HB: \o/
<Knut-HB> make run?
<bigjools> Knut-HB: ok "make run" and profit
<bigjools> ajmitch: yeah there's a bzr bug about it issuing too many requests
<ajmitch> NZ to UK is about as bad as it gets for latency :)
<bigjools> yeah
<bigjools> unless I am using VOIP to Brazil ;)
<ajmitch> I imagine that'd require patience
<Knut-HB> bigjools, http://harrius.gmxhome.de/launchpad.png does this look good? ;)
<bigjools> Knut-HB: not really, it's text :)
<bigjools> but congrats, you are running LP now
<Knut-HB> bigjools, i know, it would just be a real pain in the ass when starting a firefox over two ssh-connections ;)
<Knut-HB> bigjools, thanks for the help :)
<bigjools> Knut-HB: you're welcome, glad we got there in the end!
<bigjools> Knut-HB: although can you port forward over ssh and run FF locally?
<noodles775> Well done Knut-HB :)
<Knut-HB> bigjools, dunno, would have to ask
<bigjools> Knut-HB: something like ssh -L 80:127.0.0.88:8080 <host>
<bigjools> and then browse to localhost:8080
<bigjools> dunno if that would work, never tried it
<ajmitch> bigjools: http://paste.ubuntu.com/226565/ for rocketfuel-branch testbranch
<ajmitch> could I be missing something important from the setup steps on hardy
<bigjools> no idea :/
<ajmitch> ah well
<ajmitch> I stalled at the same place on karmic, no bin/py
<bigjools> I am guessing your sourcedeps are not intact
<bigjools> but rf-get is supposed to fix that
<bigjools> what happens if you do "make bin/py"
<ajmitch> no target, the target is /home/ajmitch/launchpad/lp-branches/devel/bin/py
<bigjools> actually try: make SHHH="" so we can see some errors
<ajmitch> which runs buildout as before, but still doesn't build anything :)
<bigjools> someone changed the default to using shhh.py everywhere and I hate it
<ajmitch> the same error as earlier about zope.interface
<ajmitch> buildout version conflicts are not fun
<bigjools> can you send an email to the launchad-dev list
<bigjools> someone who knows more than me about this can help
<ajmitch> ok
<ajmitch> let me make sure I don't have stray packages installed that should be conflicted with
<bigjools> did you install launchpad-developer-dependencies?
<ajmitch> I did
<ajmitch> it may be a problem of extra packages rather than missing ones
 * ajmitch checked & has the zope3 package installed form hardy here
<ajmitch> removing & retrying
<bigjools> ah
<ajmitch> hm, no difference there
<deryck> Morning.
<bigjools> morning deryck
<maxb> Is there somewhere where I can *usefully* register my annoyance at shhh.py ?
<allenap> maxb: https://bugs.launchpad.net/launchpad/+filebug :)
<awilkins> I see so many people with difficulties getting LP to run locally.. I must be lucky
<awilkins> A few basic teething troubles with rocketfuel, but otherwise, things started up nicely
<awilkins> I don't do any web dev really, so I didn't have many of the dependencies installed ; maybe that has something to do with it. I wonder how easy an empty Jaunty VM is to set it up on.
<noodles775> Good to hear awilkins !
<maxb> I really had no problems once I stopped shooting myself in the foot with a /usr/local installation of python that I'd forgotten about :-)
<maxb> s/no problems/no problems not attributable to being running Karmic/ :-)
<Darkwing> Hello and good morning.
<mrevell> Hello Darkwing
<Darkwing> Hi mrevell
<mrevell> jtv: thanks for marking my 2.2.7 Translations bugs as fixed released
<jtv> mrevell: well given the revision numbers, they had to be.  :)  How are things over there?
<mrevell> jtv: good thanks :)
<jtv> Are you back already?  Or are you just popping in?
<mrevell> jtv: Back today
<Daviey> Hi.. Should i be able to upload a new mugshot on a local dev launchpad?
<gmb> Daviey: I would've thought so.
<gmb> Daviey: In fact, yes.
<gmb> I don't know why I was being fuzzy about it.
<Daviey> gmb: hmm.. i blame the northern influence :)
<gmb> Daviey: No, that just leads me to think that everything's bad and it's going to rain all the time.
<gmb> Daviey: I have been known to look at a blue sky and say "Aye, we'll pay for this"
<Daviey> heh.. :).  So.. i must be doing something else wrong.. It seems to work other than that tho :/
<gmb> Daviey: What's the problem you're having?
<bigjools> gmb: but it does rain all the time oop north?
<gmb> bigjools: No, just most of it.
<Daviey> gmb: i'll try and get a useful log
 * gmb is baffled by this "oop" nonsense. It's "up." Or "uhp" if you want to try writing it phonetically.
<gmb> book, OTOH, is pronounced "bewk"
<bigjools> gmb: maybe were you come from, but it's oop here ;)
<gmb> Daviey: Oh. I'm seeing an error, too. That's weird.
<gmb> Let me try to debug it.
<Daviey> gmb: http://erk.daviey.com/lp_log.txt <-- that is a log from submit, to returning to ~$user page
<gmb> Daviey: Doesn't really tell me anything. What's the error message you're seeing on the page.
<Daviey> gmb: that is it.. no error
<Daviey> seesm to upload fine.
<gmb> Oh.
<gmb> That's suboptimal.
<Daviey> the url on the ~home is:
<Daviey> https://launchpad.dev:58080/95/mugshot.jpg
<noodles775> hi cprov-zzz :)
<Daviey> http://erk.daviey.com/get_mugshot.txt <-- then freezes
<Daviey> gmb: ^^
<gmb> Daviey: Hmm, so I can see the same behaviour... the image never seems to turn up.
<Daviey> exactamundo
<gmb> Daviey: The best person to talk to is one of the registry team: sinzui and his cohorts. sinzui will be around in a couple of hours; I suggest talking to him about it.
<Daviey> gmb: thanks
 * gmb -> lunch
<wgrant> BjornT_: I didn't this time, because it looked small enough at the start that it would take just a few minutes, and there were no Bugs people around. Had I known I'd end up spending longer on it after running into the test issues, I probably would have discussed it first.
<mrevell-lunch> bigjools: Thanks for fixing the broken link in the tour
<BjornT_> wgrant: yeah. it's good to talk it over, though. our code base contain old code that doesn't conform with current standards, so it can be unclear how to do certain things. we usually have pre-implementation calls, to reduce the risk of having to redo it at review time.
<bigjools> mrevell: thank noodles775!
<mrevell> ah thanks noodles775 :)
<noodles775> mrevell: np! :)
<wgrant> BjornT_: Right, but I didn't think there'd be much to redo. Then things changed.
<gmb> wgrant: Welcome to Launchpad development!
<gmb> That's exactly what *I* always say.
<BjornT_> wgrant: things always change :)
<wgrant> Heh.
<gmb> And that's what he always says...
<deryck> the first rule of fight club is.... oh, wait.... right, launchpad we're talking about.  Yes, things always change. :)
 * wgrant makes a change to the branch, then considers requesting a review.
 * deryck is on the third version of a branch even with pre-imp discussions.
<wgrant> I didn't do it cleanly at the start because I was scared I'd break the tests while reordering things.
<jtv> sinzui, you guru, can you help me out with a zope issue?
 * sinzui sips coffee
 * jtv waits respectfully while sinzui sips
<sinzui> jtv: I maybe able to help, but I do not feel like a guru today
 * jtv goes slowly
<jtv> I've got an obsolete path_expression in zcml that's becoming impossible because of a schema change.
<jtv> It's using a simple attribute, but that needs to become a method call with an argument to be provided by an object "higher up in the http directory structure."
<jtv> But I suspect that's not going to fly in zcml.
<jtv> Concretely, in lib/lp/translations/browser/configure.zcml, we've got a path_expression="${potmsgset/sequence}"
<jtv> but the sequence number should now only be available as potmsgset.getSequence(potemplate).
<sinzui> I see
<jtv> This path is applied "inside" a POFile; the POFile would have a reference to the right potemplate.
 * sinzui think jtv is in a new world here
<jtv> sinzui: I don't remember teleporting...
<sinzui> jtv: I need to think about this. I did some changes to openid once that did some tricky work, but your need looks beyond my practical experience
<jtv> oh
<jtv> seems like I sure know how to pick 'em :)
<wgrant> BjornT_: So, um, can you have a bit of a glance over what I've done and see if it's terribly wrong?
<gmb> wgrant: If you want someone to take a look at your code, I'm between tasks ATM.
<sinzui> jtv: I recalling writing an adapter for openid  path_expression so that the express was simple.
<jtv> sinzui: maybe an adapter is overkill though...  ISTRM there's some relatively straightforward way of solving this with a class in the right place.
<sinzui> jtv: but my hack is now in SSO, not launchpad, so I need to poke in some other trees to see if I have an example
<wgrant> gmb: https://code.edge.launchpad.net/~wgrant/launchpad/team-verbose-bugnotifications-bug-253788 is the branch. Beware of the more than 800 lines of test reorderings - the previous order relied on string ordering in a dict.
<gmb> Ouch.
<BjornT_> wgrant: i'll let gmb look at it
<wgrant> (they are now sorted by emailaddress)
<gmb> Ah, a nice big diff. Should keep me occupied til this test run finishes...
<jtv> sinzui: ah, a Navigation subclass maybe?
<gmb> wgrant: Is this ready for merging in your opinion (pending review)?
<wgrant> gmb: There are two new bits in bugnotification-sending.txt, both mentioning salgado. The rest is just reordering.
<wgrant> gmb: I believe so.
<sinzui> jtv: I recall I needs an adapter because I was not sure if we were working the an account or a person.
<gmb> wgrant: Cool. Can you propose it for merging please? It's easier to work with a merge proposal at this stage.
<wgrant> gmb: True. Will do.
<cprov> gmb: shouldn't we move review conversations to #launchpad-reviews ?
<gmb> cprov: Yes. But I ahven't started yet :)
 * wgrant didn't know that was public too.
<gmb> wgrant: Oh, we're all open source and freedom loving hippies now.
<jtv> sinzui: in this case I think it always has to be a POFile we're inside.  There already is a POFileNavigation though, with a "traverse" method that looks suspiciously similar to what I want to do.
<cprov> gmb: ehe, no I'm not blaming you, I'm the OCR today and I haven't 'started' either.
<gmb> wgrant: Besides, it's a bit mean to poke fun at your branch behind your back.
<gmb> Even for me.
<wgrant> Now, this is branched off devel. Is there any problem with proposing the merge into db-devel, since devel has been merged there?
<barry> leonardr: when you're around... ping
<BjornT_> wgrant: i'm looking a bit at your patch anyway, though :) i don't think you need those test re-orderings
<gmb> wgrant: No, there's no problem with that.
<leonardr> barry, hi
<jtv> sinzui: Navigation.usedfor is the interface you're "drilling down" into, right?  Maybe the ZCML is stopping the class that's already there from being used.
<barry> leonardr: hi!  up for a fun day of reviewing?  if it's quiet maybe we can talk about the lazr.restful thing after our standups?
<wgrant> BjornT_: The easiest way to do it meant rekeying the dict, and relying on dict ordering is sevil anyway...
<sinzui> jtv: correct
<leonardr> oh yeah, i forgot that -reviews moved
<BjornT_> wgrant: well, maybe you do. let me take a closer look at it....
<wgrant> BjornT_: I've almost proposed a merge, which will make the diff easier to get to.
<sinzui> jtv:  I found an old diff of how I made a url from an object. I need to find the magic bits and put it in a pastebin for you
<jtv> sinzui: that'd be great, thanks.
<gmb> Hmm. `self.i_know_this_is_an_openid_security_issue_input`.
<Daviey> sinzui: Did you see the issue i was discussing with gmb earlier?
<BjornT_> wgrant: hmm. i thought that recipient was a dict-like object, and not a dict. it's odd that that test hasn't failed intermittently before. anyway, i'll let gmb continue looking at it :)
<sinzui> jtv: I am not sure I can help. My implementation had the path_expression call a property on the object. The property had access to global objects. The property negotiated the rootsite, then called another method to create the url.
<mars> good morning
<sinzui> jtv: So in your case the potmsgset needs another way to retrieve the potemplate.
<wgrant> BjornT_: The object in the notification might not be, but the object used in that certainly is a dict - see bugnotification.py:40.
<jtv> sinzui: so that parameter messes things up for me... worst case, have to introduce a class representing the inputs to that method.
<sinzui> jtv:The Navigation object is for deconstructing a url into objects during traversal. path_expression is for constructing a url from objects using canonical_url()
<jtv> sinzui: ah thanks, I keep getting that wrong.
<sinzui> barry, bac, Edwin, salgado: standup in 2 minutes
<mars> ah
<mars> uhm
<mars> BjornT_, ping?
<jtv> sinzui: the more I dig into this, the less I like what I find...  Looks like we're storing a reference to the containing POFile in a TranslationMessage model object.  Trouble is, nowadays, a TranslationMessage isn't exclusively in one POFile.  Moving this to the browser code may not be easy.
<sinzui> jtv: understood.
<jtv> sinzui: note I'm not saying it shouldn't be done.  :)  I may be missing something, but this looks like an obscure bug to me.
<jtv> Time to freshen up my mental model of the store.  Two requests in threads of the same appserver can "simultaneously" (apart from the GIL) access the same model object, right?
<sp00nyG> Hello?
<bac> sinzui: are we doing a 2:00 mtg?
<sinzui> bac: I want to say yes, but I think it depends on my next meeting
<mars> hi sp00nyG
<sp00nyG> mars: Hi :) I'm testing Colloquy on iPhone, wanted to see if it worked
<sp00nyG> Btw your launchpad install instructions are great
<sp00nyG> Got it up and running on a VM yesterday
<mars> cool, glad to hear it
<henninge> sinzui: Are deleted series supposed to show up like this? https://translations.edge.launchpad.net/democracy/2.5/+pots/democracyplayer
<sinzui> yep
<henninge> sinzui: "see the same template in ..."
<henninge> sinzui: ok, looks weird though ...
<salgado> sinzui, I'm looking at OOPS-1299F1625 (the one on +participation).  I think I know what the problem is
<ubot3> https://devpad.canonical.com/~jamesh/oops.cgi/1299F1625
<mars> intellectronica, ping?
<sinzui> henninge: we cannot really delete series because of the translations. If we could disconnect the translations, we could do a true delete. We move them to obsolete junk, which is a place holders of undeleteable objects
<sinzui> salgado: thanks
<sinzui> henninge: if you can offer a way to disconnect the translations from the series, I will make delete really work
<BjornT_> sinzui, beuno, bigjools, kiko, joey/matsubara: TL call in 8 minutes
<henninge> sinzui: I see.
<bigjools> k
<noodles775> sinzui: later after your meetings, I've got two questions regarding the style-3-0. (1) http://pastebin.ubuntu.com/227000/ and (2) see implementation details of https://code.launchpad.net/~michael.nelson/launchpad/archive-3-0/+merge/9180
<henninge> jtv: what happened to the message sharing blog post???
 * jtv looks
<jtv> henninge: hmm... I don't see it any more.  Luckily I sent a much more detailed story to kfogel.
<henninge> mrevell: didn't you like what danilo wrote about message sharing ??? ;-)
 * henninge has not read the post yet, so ...
<mrevell> henninge: I've just seen the draft and I'll read it now
<henninge> mrevell: cool ;-)
<sinzui> noodles775: I will send you draft of the conversion notes I have
<henninge> mrevell: yeah, he did not use the holding post
<jtv> mrevell: I also created a placeholder for committing translations to bazaar branches, but that seems to be gone now.
<BjornT_> beuno, kiko, matsubara, joey: are you joining the call?
<noodles775> Thanks sinzui - although I think I've already gotten them (via Julian)? Or maybe it's a different version?
<matsubara> BjornT_, call?
<sinzui> noodles775: If you got them a few hours ago, then you you have them
<BjornT_> matsubara: team lead call. not sure if you are standing in for joey again, or not
<noodles775> Subject: Draft UI 3.0 implementation notes
<noodles775> Date: Thursday 23 July 2009
<noodles775> woops. Sorry, that was meant to be a pm.
<matsubara> BjornT_, I can join. joey doesn't seem to be around atm
<barry> losa ping
<henninge> jtv: I can see your blog post among the drafts, but it is unpublished.
<jtv> henninge: yes, I sent the full text to karl.  Who is at OSCon now.  mrevell, I don't suppose you've heard about the translations export to branches?
<mrevell> jtv: I haven't, no. I'm catching up on the old internal team list now
<jtv> mrevell: this is something that's not operational in the original 2.2.7 rollout yet, but will be.  We'll need an announcement.
<mrevell> jtv: Okay. Would tomorrow morning UTC be okay for us to catch up?
<jtv> mrevell: perfect, thanks.
<mrevell> thanks
<mrevell> jtv: Let's say 10.00 UTC?
<jtv> mrevell: any chance of an hour earlier?
<mrevell> jtv: sure, no problem
<jtv> mrevell: splendid, thanks!
<mrevell> :)
<gmb> Christ, 22 test failures. This could get tedious.
<noodles775> beuno: I was just chatting with sinzui, and we wanted to ask you whether it is ok to update pages to the new template and later implement the redesign, like this:
<noodles775> https://code.edge.launchpad.net/~michael.nelson/launchpad/archive-3-0/+merge/9180
<sinzui> noodles775: beuno has a branch that makes some adjustments to the CSS.
 * noodles775 checks
<sinzui> noodles775: We need to stop writing CSS that uses attributes because it encourages each page to have its own rules. we want a common class that can be shared between pages
<noodles775> sinzui: yes I agree totally that that should be the end goal, I just can't see how we can get there immediately.
<noodles775> sinzui: either (1) beuno controls all the general shared styles - which will take time - and in the mean time we create temporary exceptions (as commented in the MP), or (2) we all tweak the general styles.
<sinzui> #archive-details-table -> .detail-table. If we are going to use that presentation, it should be shared
<sinzui> noodles775: if we are saying that we will fix this later, the conversion report will be a lie
 * noodles775 checks the context of that style
<matsubara> Ursinha, stub, herb, flacoste_afk, rockstar, bigjools, henninge, sinzui, intellectronica: meeting in 20min in #launchpad-meeting
<Ursinha> matsubara, yes sir
<sinzui> noodles775: lots of pages were move the the onecolumn layout without the 0.0 and 1.0 markup and classes being fixed. We want to empty style.css of all rules by adding reusable rules to style-3-0.css
<matsubara> mars, can you sit in for flacoste_afk ^?
<herb> matsubara: thanks for the reminder
<mars> matsubara, sure
<matsubara> abentley, can you sit in for rockstar ^?
<noodles775> sinzui: I don't think so, but that is what I wanted to discuss (hence the comment in the CSS section there about the temporary rules)
<abentley> matsubara: Yes, he asked me to.
<noodles775> sinzui: yes, I agree... it would be great to having style-3-0 with all reusable css, but I'm wanting to discuss the best way to get there.
<matsubara> thanks abentley, mars
<sinzui> noodles775: understood
<barry> herb: hi.  would you have time today to work with me on regenerating the mailing list archives?
<herb> barry: possibly. I have to finish up something that is time sensitive, but I don't think it should take too long.
<barry> herb: cool.  i'm in no rush, just ping me when you're free
<herb> barry: ok
<kiko> BjornT_, sorry, got stuck in another call! are you still on?
<sinzui> noodles775: The biggest blocker from landing changes is that there are two conflicting designs for headings. One says we need to restore the structural-header slot, the user says we must continue to delete it.
<noodles775> sinzui: is there a safe solution I can go with for the moment that won't break the report (I didn't understand that point in your email)
<sinzui> no
<noodles775> :/
<sinzui> only the locationless pages that do not have tabs could land
<noodles775> OK.
<sinzui> silbs has been pressing for a universal private presentation for months. We need this solved now since your page can show a private artefact
<noodles775> Something like a statically positioned el (that somehow doesn't get in the way - ouch)
<BjornT_> kiko: no, we've finished already; i sent out the notes
 * gmb runs notfound-traversals.txt and goes to get some tea, possibly build a house.
<wgrant> kiko: Apparently I need to talk to you about the contributor agreement. If I've already submitted one for LAZR, do I need to submit another for Launchpad?
<noodles775> sinzui: will you be sending an email to lp-dev sometime soon? If so I'll wait for that and reply there, otherwise I'm keen to start the discussion on options for transitioning the CSS.
<sinzui> Yes. I am writing an open issue email now that includes a previous discussion about the side portlets
<beuno> sinzui, hi
<beuno> I hear we need to talk
<noodles775> Great! Thanks sinzui
<sinzui> beuno: yes. I am preparing a list of issues that have been raised in several conversation
<beuno> sinzui, good, let me know
<Daviey> sinzui: Can you look at bug #403561 please? :)
<ubot3> Malone bug 403561 in launchpad "Mugshot upload/retrieval fails on development launchpad" [Undecided,New] https://launchpad.net/bugs/403561
<sinzui> Daviey: I will
<Daviey> thnaks
<matsubara> stub, meeting
<matsubara> intellectronica, meeting
<mars> I am this // close to switching mail clients right now.  Thunderbird does *not* know how to filter mail.
<sinzui> beuno: are you available for a call?
<beuno> sinzui, I am. Give me 3' to finish this email and I'll be on skype
<sinzui> fab
<beuno> sinzui, onlineing
<Ursinha> hey mars, I found one very old and already fix released ExpatError bug, bug 44871
<ubot3> Malone bug 44871 in launchpad-foundations "xmlrpc should return appropriate response for a GET" [Medium,Fix released] https://launchpad.net/bugs/44871
 * gmb => AFK for a while; back later
<Ursinha> mars, but since it's not related, I've filed 403606
<bac> Hi Kiko.  Can I get an RC for https://code.edge.launchpad.net/~bac/launchpad/bug-399964-teamvocab/+merge/9195 which has been reviewed by Leonard and mentored by Barry?
<barry> losa ping
<mthaddon> barry: hi
<mthaddon> barry: mailing list templates?
<barry> mthaddon: yep
<mthaddon> barry: ok, how do we do this thing?
<bac> sinzui: Is Kiko the release-manager now?
<sinzui> bac: he is
<barry> mthaddon: https://wiki.canonical.com/ImportingMailingListArchives
<barry> mthaddon: i don't know which lists have been converted yet though. seems like at least two have.  we can experiment with the haibunku lists
<mthaddon> barry: er, we have to do this for every list?
<barry> mthaddon: we do unfortunately.  how many lists do we have?  i'm guessing you'd like me to try to automate this, right? ;)
<mthaddon> barry: well, which part do we have to do? just the second part?
<bac> salgado, sinzui: what controls the start-up of the librarian as to whether it is http or https for launchpad.dev?  or should it answer on both?
<mthaddon> barry: I can write a wrapper script that loops over the directories (I hope)
<mthaddon> barry: we have 762 lists
<barry> mthaddon: the first one i think because we're not adding any new mbox file
<barry> mthaddon: let's try with the haibunku list first
<salgado> bac, I know there's a use_https config variable for the librarian, but I'm not sure that's what's used by the server
<mthaddon> barry: okay
<barry> mthaddon: what i don't know is whether you have to clear out the existing html, but let's try doing that first without that step
<sinzui> bac: launchpad-lazr.conf sets http verses https. I think you need to look at feeds and private librarian for examples
<bac> salgado: it doesn't appear to be.  it controls how we construct URLs to the librarian but not start-up, AFAICT
<bac> sinzui: i was looking at bug 403561 while i waited on my review.  mugshots are served up find over http but the generated URL is https
<ubot3> Malone bug 403561 in launchpad "Mugshot upload/retrieval fails on development launchpad" [Undecided,New] https://launchpad.net/bugs/403561
<mthaddon> barry: https://pastebin.canonical.com/20244/ ?
<bac> s/find/fine
<sinzui> bac: interesting
<barry> mthaddon: possibly s/team-name/team_name ?
<bac> sinzui: +download files are smart and look at X-SCHEME to decide http vs. https but other users of the librarian don't
<mthaddon> barry: https://pastebin.canonical.com/20245/
<bac> sinzui: but this problem is only for launchpad.dev...
<sinzui> bac: mthaddon worked with Francis to put a proxy in front of librarian calls
<barry> mthaddon: +1
<mthaddon> barry: Unknown option: definevr
<sinzui> bac: I was certain that this was a dev-only issue when the bug ware reported
<bac> barry: i have a request to make a mailing list private.  can you remind me how to get that done?
<bac> sinzui: that is true.  but it was interesting to me so i looked at it.
<barry> bac: the team has to be private before you create the ml
<sinzui> bac: I do not think the production config should have broken dev
<bac> sinzui: i don't know that dev has recently been broken.  it may have never worked, at least not since dev moved to https.
<barry> mthaddon: s/definevr/definevar/
<barry> mthaddon: i'll fix the wiki page
<mthaddon> barry: ok, that's been run
<mthaddon> barry: doesn't seem to have done anything :(
<barry> mthaddon: nope
<barry> mthaddon: let me read the mhonarc manpage
<mthaddon> barry: k
<sinzui> bac: I think there may be a conflict between the image formatter and the config. I think the image formatter should know to use the correct protocol when talking to the  librarian. The issue may be that there are two and it is safer to always talk https and let a proxy work out the details
<sinzui> bac: change download_url: http://launchpad.dev:58080/ to https
<sinzui> in development/launchpad-lazr.conf
<bac> sinzui: i tried that.  doesn't help.
<bac> since use_https is True for lp.dev all URLS for the librarian come out as https
<sinzui> bac: right, but setting the local librarian to also use https should fix the issue
<bac> sinzui: that variable doesn't seem to affect how it gets started.  that was the first thing i tried and it didn't work.  it also exposed a bug in that it tried generating "httpss://" schemes!
<barry> mthaddon: can you add the -reconvert option to that mhonarc command?
<sinzui> bac: does
<sinzui> utilities/lsconf.py -s librarian configs/development/launchpad-lazr.conf
<sinzui> confirm that the server will start with https?
<mthaddon> barry: does it matter where?
<barry> mthaddon: it probably doesn't as long as it doesn't split an argument.  safest is at the front
<mthaddon> barry: done and rerun - doesn't seem to have done anything
<barry> mthaddon: huh.  okay, i guess i have to experiment now :/
<barry> mthaddon: thanks, i'll ping you again soonish
<bac> sinzui: lsconf.py does show the urls to be https.  but looking at the runlaunchpad.py starter script it doesn't look like it pays any attention to those values.
<mthaddon> barry: sure
<bac> barry: are you still in #launchpad-code?
<barry> bac: yes, but my irc client doesn't always make it obvious i have a message waiting for me
<sinzui> bac: did you verify everything works when vhost.use_https: False is set?
<bac> sinzui: no
<bac> sinzui: not yet
<sinzui> beuno: I have characterised the Involvement as creative actions. The but that is not the case in the translations link or the user participlation links
<sinzui> beuno: Is the menu application activity? if so can I have more than one link for an application? Join this Team, Contact this Team?
<kiko> wgrant, I don't think so -- your agreement should cover all your contributions to the projects identified, but let me double-check with karl
<kiko> sinzui, does bac need anything?
<bac> kiko:  I asked for an RC
<kiko> bac: can I see it?
<bac> kiko:  sure:   https://code.edge.launchpad.net/~bac/launchpad/bug-399964-teamvocab/+merge/9195
<beuno> sinzui, hm
<bac> kiko: note the new SQL that is generated has been run on production and was sufficiently fast.
<beuno> sinzui, join this team could very well be the main action for a team
<sinzui> yes it can
<bac> kiko:  also, this is the bug that is prevently two of the other registry branches from working
<bac> by the non-sense word "prevently" i meant "preventing"
<beuno> sinzui, I think we should only have 1 main action per page. Some may not have it at all, just page-wide actions that can't be placed within reason inline
<sinzui> beuno: I am certain most pages will not have a menu
<beuno> sinzui, I think we have: 1 main action (ie download), maybe up to 3 extra actions (just regular links), or neither
<sinzui> beuno: I am looking that the involvement and participation menus, which have 5 links
<beuno> sinzui, ah, right
<sinzui> the Involvment links are easy because they are buttons now
<kiko> bac, heh, nice freudian :)
<beuno> sinzui, you're asking what happens to those within, say, a bug page?
<sinzui> yes.
<beuno> what a great question
<beuno> I'm not 100% sure, but this may be a good opportunity to leave all the cross-application actions
<beuno> and flex that "cross application muscle"
<bac> kiko: i think it's a cool new word.  i just need to figure out what it means and start using it more
<sinzui> beuno: okay, this sounds like a promising perspective.
<beuno> sinzui, maybe I'm on crack (and, if we do that, we should make sure we don't show the ones that the user said they don't use"
<sinzui> beuno: I don't think these links will change then. We have 10 links defined in the mockup, and we may choose to not show some if the project does not use it
<beuno> sinzui, I'm inclined to think that way, yes. I'm sure there's a lot of great things we could do with that, but I don't want to try and solve all the problems in the world right now
<sinzui> beuno: translations and blueprints really require the owner to enable. bugs and code and answers can be contrbuted by 3rd parties I think
<beuno> sinzui, but we have these magic knobs where users tell us what they use Launchpad for
<beuno> barry is going to help me actually mean something!
<sinzui> beuno: where a minority of projects tell us. many are clearly using code, bugs, and translations, yet the page says they do not
<beuno> sinzui, yes. We need to fix that so by default, if you create a project in Launchpad, you actually use us
<beuno> and, if you said you don't, make it incredibly obvious how to flip that switch
<sinzui> beuno: the menu rules see pretty clear. I can create portlets that JDRT so no engineer thnks about it
<beuno> sinzui, awesomness
<bac> kiko:  i need to step out for a bit.  you have any questions about that RC branch before i go?
<barry> beuno: yes!
<kiko> bac, well, one
<kiko> did you check and update all the callsites of _privateTeamQueryAndTables?
<kiko> and why not use a verb in that method name, too long?
<bac> kiko:  yes, i updated all of the call sites (i'll double check)
<bac> kiko: i can verberize it if you want
<kiko> bac, I don't really care so much, it's damned long as it is
<bac> kiko:  ok, i've got it running on ec2 now so i can submit it in a couple of hours if you're satisfied.
<kiko> bac, request an RC there and I'll approve
<bac> kiko: done.  thanks.
 * bac -> late lunch
<marc0s> hi all
<marc0s> i've already setup launchpad and i think its working quite well.. in localhost, i'm trying now to make it available to the other hosts of the lan, so i want to have apache listening in my eth0 ip address, the problem is that i'm getting HTTP 302 loops and i can't see the problem
<marc0s> what i've done is to replace 127.0.0.88 with my local ip addr and 172.0.0.99 for the ssl host with another ip addr that is assigned to eth0:1
<marc0s> should i change anything else?
<matsubara> andrea-bs, hi, I noticed that you started working on adding comments to blueprints. did you have a pre-implementation call with anyone?
<andrea-bs> matsubara, no, I didn't
<matsubara> andrea-bs, it'd probably help if you have so you won't waste time
<andrea-bs> matsubara, how can I have a pre-imp call?
<matsubara> sinzui, can you help andrea-bs with his fix for bug 49698?
<ubot3> Malone bug 49698 in blueprint "specifications should allow comments" [Low,In progress] https://launchpad.net/bugs/49698
<salgado> marc0s, do you see any errors on the console where you started (make run) launchpad?
<sinzui> matsubara: I have already commented, outlined our expectation, and offered my help,
<matsubara> sinzui, great. thanks!
<marc0s> salgado: i think the request is not reaching launchpad
<sinzui> andrea-bs: you rock
<andrea-bs> sinzui, launchpad rocks :)
<marc0s> just a bunch of 302 messages in the apache log
<andrea-bs> thanks matsubara and sinzui
<mars> marc0s, did you edit /etc/hosts ?
<marc0s> mars: sure
<marc0s> i can paste the log output in dpaste or sth if you want to look at them
<marc0s> with the /etc/hosts and the local-launchpad vhost
<marc0s> http://dpaste.com/70576/
<mars> marc0s, bind VirtualHost 443 to '*' instead of the local IP
<marc0s> mars: now i get a warning about _default_ being overlapped by the one defined in local-launchpad
<marc0s> and the behavior is the same ...
<marc0s> oh wait
<marc0s> i changed the two :443 vhost, not doing so in the bazaar one did the trick! :)
<marc0s> thanks mars :)
<herb> barry: it's not looking good. I think I'm going to be tied up for the rest of the afternoon.
<herb> barry: how pressing is this?
<herb> barry: can we do it first thing tomorrow?
<barry> herb: no worries.  i'm still working on some local tests and a helper script, but i'm ocr today, so it's been slow going
<barry> herb: let's do it tomorrow (and hopefully it will be a trivial thing ;)
<herb> barry: sounds like a plan
<mars> fun, I think I found a bug in pdb
<mars> hmm, pdb doesn't like it when you mask its "args" command with the parameters "def foo(*args, **kwds)"
<barry> herb: i have a script that will make this easy now in lp-dev-utils.  we can do it tomorrow
<herb> barry: excellent
<Toba_> helllo oscon!
<thumper> morning
<Ursinha> morning thumper
<mars> thumper, have a sec to help with a basic Python unicode question?
<thumper> mars: sure
<mars> thumper, looking at https://lp-oops.canonical.com/oops.py/?oopsid=OOPS-1300E2074, the UnicodeDecodeError at the end
<ubot3> https://devpad.canonical.com/~jamesh/oops.cgi/1300E2074
<mars> ubot3, wrong URL?
<ubot3> Factoid wrong url? not found
<mars> :)
<mars> thumper, I'm looking at the string substitution in login.py
<mars> and I *think* that one of the strings going into that string substitution, or cgi.escape(), is unicode
<thumper> mars: what line?
<thumper> mars: that would explain the error :)
<mars> thumper, line 381
<thumper> which login.py?
<mars> thumper, that would explain it, but I can't get the console to reproduce the error
<mars> lib/canonical/launchpad/webapp/loging.py
<thumper> looking
<thumper> mars: I expect that cgi.escape is not expecting a unicode string
<thumper> mars: what happens if we tell the form it is in UTF8 and encode the strings before cgi.escape?
<mars> thumper, ok, trying to reproduce that in the console is a pain, thoughj
<thumper> console?
<thumper> which console?
<mwhudson> i expect that cgi.FieldStorage shouldn't be returning a unicode string from items()
<mars> Python console
<mwhudson> the error will be from the '%s'%(u'\xe9',) part
<mwhudson> er
 * mars tries that
<mwhudson> or nog
<mwhudson> not
<mwhudson> no
<mwhudson> '%s %s'%(u'x', '\xe9') will do it
<mars> I tried that exact string in the Python interpreter, it didn't error out
<thumper> mwhudson: why will the second do it?
<mwhudson> yeah, i was wrong
<mwhudson> thumper: because the presence of the unicode string upcasts the result to unicode
<mwhudson> and then the non-ascii plain string can't be coerced into unicode
<mars> it works because you are somehow mixing the string encodings with the intperpolator?
<mars> ha
<mars> and that works
<mars> ah, now I remember the strange inverted logic for the UnicodeError messages - they are named the inverse of what you think they are doing :)
<mwhudson> decode means moving towards unicode, yes :)
<mars> yeah, that's the trick :)
<mars> ok, so the problem is right there, in the line
<mars> and the source of the evil string is field.title I think?
<mars> https://lp-oops.canonical.com/oops.py/?oopsid=OOPS-1300E2074#requestvars
<ubot3> https://devpad.canonical.com/~jamesh/oops.cgi/1300E2074
<mwhudson> looks like it
<mars> so the answer is to pre-decode everything
<mwhudson> COME ON launchpad.dev, start up sometime soon
<mwhudson> maybe
<mars> actually
<mars> you probably want to decode ASAP
<mars> you know, at the edges
<mars> so we would do so in iter_form_items(), because we can't do it in the Zope form object itself
<mwhudson> however
<mwhudson> i can't reproduce this
<mwhudson> i.e. log out in launchpad.dev and go to https://bugs.launchpad.dev/bugs/+filebug/+login?field.title=%E0%AC%87
<mars> hmm
<mars> might be another field?
<mars> the user name?
<mwhudson> which is a character in some indian script and i get valid html back
<mars> according to the request variables, there is only one field being submitted: field.title
<mwhudson> yeah
<mwhudson> ah
<mars> or was that the list of variables coming into this page?
<mwhudson> using _that_ field.title repros the bug
<mars> ?
<mwhudson> try it
<mwhudson> https://bugs.launchpad.dev/bugs/+filebug/+login?field.title=package+uml-utilities+20070815-1.1ubuntu2+failed+to+install%2Fupgrade%3A+el+subproc%E9s+post-installation+script+retorn%E0+el+codi+d%27eixida+d%27error+1
<mwhudson> (after logging out)
<mwhudson> i don't know why my fancier example succeeded
<mwhudson> (maybe because it has codepoints >= 128, but if that's really true we should take up crying as a hobby)
 * mwhudson cries
<mars> :)
<mars> mwhudson, I don't see the difference, perhaps it is the string structure?  Mixing %27 with %E9?
<mwhudson> https://bugs.launchpad.dev/bugs/+filebug/+login?field.title=%C4%80
<mwhudson> succeeds
<mars> mwhudson, https://bugs.launchpad.dev/bugs/+filebug/+login?field.title=aaa%C4bbb%80ccc
<mwhudson> hmm, so does https://bugs.launchpad.dev/bugs/+filebug/+login?field.title=%C3%BF
<mwhudson> it seems that things that urlencode to two chars are ok, maybe?
<mars> mwhudson, is it possible that the %F00 characters are being translated into \ or something odd like that?
<mars> maybe try everything in the string
<mars> %3A%E9%27%2F
<mars> what is %2F?
<mwhudson> %2f is /
<mwhudson> oh wtf
<mwhudson> you can't even do 'print self.request' in the breaking cases :)
<mwhudson> um
<mwhudson> i think when you type unicode into ff's location bar, it gets utf-8 encoded, then url quoted and this works
<mwhudson> but these oopses have the "funny foreign characters" latin-1 encoded
<mwhudson> and things break
<mars> where do you see that?
<mars> ah, not in the request, just with what was done to the strings
<mwhudson> right
<mars> at the last place I worked, I wrote a script to make sense of user-uploaded files, extracted from corporate databases, spreadsheets, CSV files, etc.
<mars> I had to double-decode, or decode-encode, or encode, just to get the %F00 characters to go away
<mwhudson> so many pieces of code get this wrong
<mars> Ursinha, I've updated the bug description with the steps to reproduce, and the suggested fix
<mars> Ursinha, looking at the second one now
<Ursinha> mars, thanks!!! :)
<Ursinha> mars, do you think it's too hard to fix?
<mars> Ursinha, one or two lines maybe?  Plus the test
<mars> which is the fun part :/
<Ursinha> mars, hm..
<Ursinha> mars, because it's really annoying and it's been there for ages
<mars> mwhudson, here's a fun one: print('\x03ba3\x03b2')
<mwhudson> my font doesn't seem to have that glyph
<mars> ok, \u03 is... \u0003 ?
<mars> or \u0300
<wgrant> kiko: That's what I thought the wording indicated, but the bit where you have to send it to the project lead as well suggests otherwise.
<mars> mwhudson, found it, someone was searching in a PPA for Îº3Î²
<mars> which is print(u'\u03ba3\u03b2')
<Snova_> When I type "make run", how do I access it? https://localhost ? http://localhost shows the default Apache "It works" page.
<mars> Snova_, http://launchpad.dev
<Snova_> Oh yeah...
<Snova_> Cool! Thanks :)
<mars> Ursinha, it looks like filtering PPAs for any unicode string causes the OOPS.  Something is causing the unicode to trickle down the callstack in a nasty way.
<mars> Ursinha, do you have a bug for that?
<mwhudson> wgrant: congratulations on the first approved third party merge proposal
<mwhudson> thumper, jml, abentley?
<thumper> yep
<thumper> mwhudson: jml is on leave today
<abentley> mwhudson: hi
<mars> mwhudson, thanks for the help with the Unicode issue
<mars> thumper, you too :)
<wgrant> mwhudson: Thanks. When does the tree normally reopen?
<mwhudson> wgrant: monday
<Snova_> How do I register with it? I can't send email, it's currently throwing Connection Refused exceptions in smtplib.
<wgrant> I think you might need a local MTA, as it tries to send to root@localhost.
<wgrant> Snova_: But you can log in with an existing user or use utilities/make-lp-user to get in without an MTA.
<Snova_> Ah, I found the latter script, but it seemed to be hanging.
<wgrant> It'll take a few seconds to start.
<Snova_> It works now :)
<Snova_> How do I push to a launchpad.dev branch?
<thumper> Snova_: there is a link on the dev wiki
<Snova_> Hm, should have looked there first
<thumper> Snova_: linked from http://dev.launchpad.net/Code I think
<thumper> https://dev.launchpad.net/Code/HowToUseCodehostingLocally
<thumper> mwhudson: what's the status of the deleting branches spec?
#launchpad-dev 2009-07-24
<mwhudson> thumper: it's all implemented but as up until the rollout, branches deleted on production would create deletion jobs scheduled for right now, it's not running yet
<thumper> mwhudson: so what should we mark the spec?
<thumper> done?
<mwhudson> thumper: i guess the "deployment" status it's in is about right
<mwhudson> thumper: move it to 2.2.8 and i'll set it to implemented when the cronscript is set up?
<thumper> ok
<mthaddon> anyone able to review https://code.edge.launchpad.net/~mthaddon/lp-production-configs/bug-287304-staging-logrotate/+merge/9097 for me?
<pkern> All those private branches ;-)
<wgrant> Indeed :(
<wgrant> Fortunately all new ones are public.
<Chauncellor> Congrats on the open source! :)
<Chauncellor> I'm very happy
<mwhudson> pkern: a lot of the old branches are stacked on old-format versions of trunk, and as they're mostly merged anyway, they're probably not very exciting
<mwhudson> and yes, you're not getting to see our production config :)
<mwhudson> mthaddon: done
<mthaddon> thx mwhudson
<blueyed> Is Karmic supported soon as a dev platform? I've gotten various errors while doing the rocketfuel-setup.
<lifeless> thumper: I think you may be confused about bug 390563
<ubot3> Malone bug 390563 in bzr "overly large delta creation when fetching from 2a repositories" [Critical,In progress] https://launchpad.net/bugs/390563
<wgrant> blueyed: LP needs Python 2.4, which Karmic doesn't love. A port to newer Pythons is in progress.
<thumper> lifeless: probably
<lifeless> thumper: its not affecting lp any more, not since you cherrypicked the first stage fix weeks ago
<thumper> lifeless: ok, so fix released for us then
<lifeless> there will be a follow on patch, but its corner cases, not common, and you can just wait for a release for that.
 * mwhudson wonder if there's anything we can really do to make 'make schema' less of a hog
<wgrant> mwhudson: Please please please.
<Daviey> I was _more_ suprised how much the bzr co process took more tbh
<Daviey> On both RAM and cpu
<lifeless> thats a bug
<lifeless> and filed as such
<lifeless> (in bzr)
<Daviey> oh :)
<lifeless> bzr does much better streaming, but the expected load was high so we made lp checkouts use http for the release period
<lifeless> mwhudson: speaking of which, should we disable the hot spot now?
<mwhudson> i guess so
<wgrant> The HTTP checkout really didn't work too well.
<lifeless> wgrant: it did
<lifeless> wgrant: we had a single component in the DC fail, but that was the same as failed saturday so was fast to fix
<wgrant> It lightened the load, sure.
<wgrant> But I mean for the users.
<lifeless> wgrant: and after that we spiked up to *time times* our normal request rate
<lifeless> s/time/ten/
<wgrant> It's reasonable that you did it.
<wgrant> Not bad.
<lifeless> now, many of those are probably due to the [major] bugs in bzr to do with 2a and http
<lifeless> *I* think we would have handled it on bzr+ssh with no problems
<lifeless> but, that wasn't load tested,and we had load tested http
 * wgrant will be interested to see how swift a checkout is over bzr+ssh.
<mwhudson> it took me ~an hour last time i did it
<Daviey> wget + tarball would be even quicker :)
<mwhudson> thumper: after all that, i'm not sure stub's concerns are valid
<thumper> mwhudson: really?
<lifeless> mwhudson: an hour? that seems long
<mwhudson> (but i've beefed up my tests)
<thumper> mwhudson: stub's concerns seemed entirely reasonable to me, and I'd be kinda surprised if they were off
<mwhudson> lifeless: i was getting ~40 kbytes/s
<thumper> I think that the tests may have problems duplicating multiple database connections
<mwhudson> thumper: i'm surprised too
<mwhudson> thumper: the default isolation level is READ COMMITTED for scripts though
<mwhudson> thumper: and given that i don't write to the db, i fail to see how that is different from AUTOCOMMIT
<mwhudson> the storm cache issue i'm less clear on, but i can't make anything break because of it
<thumper> hmm
<mwhudson> thumper: actually, it's clear the storm cache doesn't matter
<thumper> I'm going to have to defer to stub on the connection stuff
<mwhudson> thumper: we never create a Branch object
<thumper> ah ha
<thumper> cunning plans
<mwhudson> well
<mwhudson> lucky
<mwhudson> also
<mwhudson> 2009-07-24 00:06:32 INFO    '/~sabdfl/applets/wabble/' -> 'http://localhost:8080/~sabdfl/applets/wabble/' (0.001703s, cache: MISS)
<mwhudson> 2009-07-24 00:06:39 INFO    '/~sabdfl/applets/wabble/' -> 'http://localhost:8080/~sabdfl/applets/wabble/' (0.000180s, cache: HIT)
<mwhudson> so not sub ms for hits, but not bad
<mwhudson> grr, not sub ms for misses
<mwhudson> (will be a bit slower in the dc i guess)
<thumper> mwhudson: still better than 100 or 500ms
<mwhudson> thumper: yes, definitely
<mwhudson> also, in main.log:
<mwhudson> [2009-07-24 12:06:32.534 NZST] branch-rewrite@launchpad_dev LOG:  statement: SELECT Branch.id, Branch.unique_name FROM Branch WHERE Branch.unique_name IN (E'~sabdfl', E'~sabdfl/applets', E'~sabdfl/applets/wabble', E'~sabdfl/applets/wabble/') AND Branch.private = false
<mwhudson> which is nice an minimal, i think
<lifeless> mwhudson: so thats 320kbits, whats your link capable of?
<mwhudson> lifeless: what?
<mwhudson> oh
<mwhudson> most isps in nz don't limit you, it's determined by distance to exchange
<mwhudson> i get 1 megabyte/s to nz.archive.ubuntu.com on good days
<mwhudson> never internationally though
<lifeless> so
<lifeless> not today
<lifeless> but we need to debug this
<lifeless> the python DVCS eval had issues too, and we didn't figure it out
<mwhudson> lifeless: maybe i'm just pessimistic, i don't really expect to get much more than that to london
<mwhudson> (on good days i do, but not every day)
<lifeless> mwhudson: it streams
<mwhudson> lifeless: i meant, over any link
<mwhudson> not bzr+ssh specicif
<mwhudson> spm: guess what!  jelmer rebased dulwich again :)
<lifeless> mwhudson: We should find out why you get that behaviou
<lifeless> r
<mwhudson> ok
<mwhudson> not now though
<spm> mwhudson: do you think it'd be abuse of power if I suspend jelmer's launchpad account? just asking. no reason. hypothetically.
<lifeless> jelmer: ^ :P
<mwhudson> launchpad/bzr.dev builder seems to be working, yay
 * mwhudson lunches
<jelmer> spm: (-:
<spm> jelmer: I was joking? :-P
<matsubara> bug 666
<ubot3> Malone bug 666 in malone "can't file a bug on Ubuntu" [Medium,Invalid] https://launchpad.net/bugs/666
<mup> Bug #666: can't file a bug on Ubuntu <Launchpad Bugs:Invalid> <https://launchpad.net/bugs/666>
<mup> Bug #666: can't file a bug on Ubuntu <Launchpad Bugs:Invalid> <https://launchpad.net/bugs/666>
<mup> Bug #666: can't file a bug on Ubuntu <Launchpad Bugs:Invalid> <https://launchpad.net/bugs/666>
<thumper> we shouldn't have two bots here :)
<mwhudson> haha
<matsubara> yeah, I vote for keeping mup and let ubot3 go
<mars> mwhudson, we are on AMI launchpad-ec2test11 now?
<mwhudson> mars: i think so
 * mars watches the script blow up again
<mars> we really need a library for this stuff
<EdwinGrubbs> jml: ping
<mars> EdwinGrubbs, I think he is on leave today
<EdwinGrubbs> mars: that's sad
<EdwinGrubbs> thumper: ping
<thumper> EdwinGrubbs: hi
<mwhudson> yes, jml shouldn't be allowed to go on leave
<EdwinGrubbs> thumper: hi, I was wondering if you knew if people use the feature of the BranchPopupWidget where it will create a mirrored branch if you enter in a url with the hostname?
<thumper> EdwinGrubbs: it was asked for, and implemented
<thumper> personally I'm not a big fan of it
<EdwinGrubbs> thumper: I'm in the process of converting all instances of the PopupWidget to use the ajax Picker, and I wanted to verify that I had to keep that functionality.
<thumper> EdwinGrubbs: yes, I'd say keep it for now
<mars> thumper, btw, found a bug in the Popup widget that links branches to bugs
<mars> thumper, it returns a launchpad.dev URL from the AJAX request
<thumper> mars: please file a bug :)
<mars> besides that, works beautifully :)
<EdwinGrubbs> thumper: do you know if there are any existing tests that exercise that feature? All the +linkbranch pagetests that I found just pass in the identifier for an existing launchpad branch.
<thumper> EdwinGrubbs: the code team is very fond of unittests :)
<thumper> EdwinGrubbs: I'm certain it'll be tested somewhere
<mars> ironic that I just timed out three times in a row trying to view jml's branches page
<mars> thumper, bug 403839
<ubot3> Malone bug 403839 in launchpad-code ""Link to a bug report" feature returns a launchpad.dev URL" [Undecided,New] https://launchpad.net/bugs/403839
<mup> Bug #403839: "Link to a bug report" feature returns a launchpad.dev URL <ui> <Launchpad Bazaar Integration:New> <https://launchpad.net/bugs/403839>
<mup> Bug #403839: "Link to a bug report" feature returns a launchpad.dev URL <ui> <Launchpad Bazaar Integration:New> <https://launchpad.net/bugs/403839>
<mup> Bug #403839: "Link to a bug report" feature returns a launchpad.dev URL <ui> <Launchpad Bazaar Integration:New> <https://launchpad.net/bugs/403839>
<mars> eeewww
<mars> bot barf
<thumper> heh
<thumper> mars: ta
 * wgrant boggles.
<wgrant> RC changes aren't having much luck this release.
<wgrant> db-devel r8318 has another extra unintended change.
<mars> another one?
<wgrant> Yes.
<wgrant> rocketfuel-get is broken again.
<mars> odd, db-devel isn't listed on https://code.edge.launchpad.net/launchpad
<mars> ah, it's just the mainline
<wgrant> Right.
<mars> as are you - that is two this release
<wgrant> Yes. That doesn't normally happen, does it?
<mars> thumper, ^ should we rc-unbreak rocketfuel-get?
<mars> wgrant, no, this is quite odd
<wgrant> I can't find an MP for that branch, so I am wondering if it showed up in the diff or not.
<ajmitch> great, so I managed to popular bin/ by removing the i18n section in buildout.cfg which was giving me version conflicts on hardy
<ajmitch> how is rocketfuel-get broken today?
<wgrant> ajmitch: Same as last time.
<mars> the same glitch as before snuck into a release-critical landing
<wgrant> The fix was accidentally RC-reverted.
 * ajmitch doesn't know what broke it last time
<thumper> what's broken?
 * wgrant kicks loggerhead.
<wgrant> Ah, there we go.
<mars> thumper, rocketfuel-get, http://bazaar.launchpad.net/~launchpad-pqm/launchpad/db-devel/revision/8318
<wgrant> thumper: Observe http://bazaar.launchpad.net/~launchpad-pqm/launchpad/db-devel/revision/8318
<thumper> how'd that happen?
<mars> no idea
<mwhudson> wgrant: woops
<mars> that is the second one I've seen this release though
 * thumper shakes his head
<thumper> I'm pleased I don't use the rocketfuel scripts :)
<mars> both issues came in via db-devel rc landings
<mars> which is quite strange
<mwhudson> that's reverting the fix from http://bazaar.launchpad.net/~launchpad-pqm/launchpad/db-devel/revision/8304
<wgrant> mwhudson: Right.
<mwhudson> i guess bong merge conflict resolution again?
<mars> mwhudson, ?
<mwhudson> mars: someone merged from trunk, got conflicts in rocketfuel-get and resolved them incorrectly?
<mwhudson> it's a theory
<mars> ah
<mars> the other one killed some app links in sourcecode/, but it appears the test suite somehow missed it.
<ajmitch> victory, I have launchpad running on the laptop even with a hacked buildout
<wgrant> ajmitch: Why did it need hacking?
<ajmitch> because there was some version conflict in getting z3c.recipe.i18n, so I thought I'd see how far I'd get by disabling that
<ajmitch> to my surprise I've got it running
<thumper> ajmitch: did you have a copy of lp-launchpad-dependancies as ./download-cache?
<ajmitch> thumper: yes
<thumper> hmm...
<mwhudson> wee, basically all the codehosting tests fail with bzr.dev
<thumper> mwhudson: I saw that there were some failures
<thumper> mwhudson: what's changed this time?
<mwhudson> 50 failures, 19 errors
<mwhudson> not sure yet
<mwhudson> argh, of course it's the acceptance tests so the failures are useless
<mwhudson> oh
<mwhudson> argh!
<ajmitch> thumper: I won't pretend to know why buildout failed, apart from that I'm running on hardy, not jaunty
<ajmitch> & that I'd rsynced branches from my desktop previously
<mwhudson> because these tests weren't run out of an actual egg, there is no file at /home/buildbot/slaves/launchpad/bzr/build/bzr.dev/EGG-INFO/scripts/bzr
<mwhudson> grump
<thumper> :(
<lifeless> eggs are such a heinous invention
<wgrant> The new license headers in current(-dev).sql have to be added manually and make 'make lint' whine. Should newsampledata be fixed to add them, or should they be removed?
<mars> lifeless, so you are a pip fan then?
<lifeless> mars: I'm a deb an
<lifeless> *fan*
<lifeless> deb and source
<lifeless> either install normally
 * thumper hurls
<thumper> I've just seen a truely (sp?) ugly page
<thumper> and the worst bit is - I think I wrote it
<thumper> :(
<wgrant> There are a few around.
<mwhudson> thumper: truly
<thumper> mwhudson: I was tossing up whether or not to add the "e"
<thumper> but neither looked right to me
<thumper> I just suck at spelling
<mwhudson> english is ridiculous
<mwhudson> thumper: want to review a branch that will work better at this?
<thumper> better at what?
<mwhudson> testing against bzr.dev
<thumper> mwhudson: sure
<thumper> here's an unrelated question
<thumper> when "fixing" a bug on a page
<thumper> and you notice something "wrong"
<thumper> how hard should I resist fixing the "wrongness"?
<mwhudson> almost always better to do it in another branch, i'd say
<lifeless> thumper: if you have time for yak shaving, shave the yak
<lifeless> thumper: I'd do it in a separate thing to be reviewed, to be nice to the reviewers
<thumper> hmm...
<thumper> ok
<mwhudson> thumper: https://code.edge.launchpad.net/~mwhudson/launchpad/bzr.dev-help/+merge/9234
<lifeless> how do you unassign via email
<spm> "dear losas, pls to be doing XYZ. kthxbye" ???
<wgrant>   assignee nobody
<thumper> hah
<thumper> thanks wgrant
<lifeless> wow
<lifeless> I so want to create an account nobody now
<ajmitch> it's not a reserved name?
<lifeless> wgrant: you're sure about this?
<lifeless> https://help.launchpad.net/Bugs/EmailInterface needs an update if True
<wgrant> lifeless: It's there...
<lifeless> oh ugh
<wgrant> There was a Person named nobody ages ago.
<wgrant> But it got renamed.
<lifeless> there are two sections on assignee
<lifeless> ><
<lifeless> https://help.launchpad.net/Bugs/EmailInterface#Assigning%20and%20targeting%20the%20bug <- seems very confused
<wgrant> lifeless: It's shown in the reference, but not the short introduction. Seems reasonable.
<lifeless> wgrant: the short intro links to 'affects' not 'assignee'
<lifeless> which makes one think that the short intro is all there is
<lifeless> thanks though
 * thumper just deleted two page templates \\o/
<thumper> mwhudson: want to review a branch where I clean up the codereviewcomment +fragment and +macro views to use services/comment +render?
<thumper> mwhudson: it is pretty boring
<thumper> mwhudson: good for a friday afternoon :)
<thumper> mwhudson: I'm just running all the branches stories to make sure I didn't break anything
<thumper> mwhudson: tested locally :)
<mwhudson> thumper: boring sounds good
<thumper> oh FFS
<thumper> _someone_ keeps playing with sample data
<thumper> grr
<mwhudson> conflicts?
<thumper> no, just a failure since I've not done "make schema" in a while and someone added Sample Person to another team
<mwhudson> ah
<mwhudson> that's sucky
 * wgrant finds a bug in the malone email interface, thanks to lifeless' "assignee nobody" email.
<lifeless> ?
<wgrant> lifeless: It didn't announce in the email or activity log that you had added a launchpad-code task. Very odd.
<lifeless> heh
<lifeless> I _really_ want a 'change what this task affects'
<lifeless> rather than adding a task
<lifeless> if you do that, Jelmer and I will be extremely happy
<sanxiyn> Maybe a bit late, but congrats on open source!
<spiv> sanxiyn: it's never too late to be loved!
<wgrant> lifeless: That doesn't seem too hard. I wonder if it would be better to have a command that takes two args, or a command to follow the 'affects' that reassigns the current task.
<sanxiyn> Now I can stop evangelizing how evil Launchpad is and how bad Canonical is. Thanks.
<sanxiyn> (Been doing that for last couple years.)
<lifeless> wgrant: reaffect <new product.
<lifeless> or someting
<ajmitch> affects instead
<wgrant> lifeless: retarget?
<lifeless> yeah
<wgrant> Although I guess that's not the UI term.
<lifeless> it should be:P
<wgrant> Indeed.
<wgrant> It's in the webservice API.
<thumper> mwhudson: I'm just pushing the branch up now
<thumper> mwhudson: although I'm about to EOD
<thumper> mwhudson: shall I bother to assign the review to you?
<thumper> mwhudson: or just wait till Monday
<mwhudson> thumper: oh assign it to me
<mwhudson> thumper: it might wait until monday, but i'll probably slog through it
<thumper> bzr sending now
<thumper> mwhudson: sent
<thumper> sanxiyn: so you like Launchpad now?
<sanxiyn> thumper: I should try it, and then I would be able to tell :)
<sanxiyn> thumper: I like e.g. tracking of bugs in remote bug trackers. (That's why I am a contributor to Debian's bts-link.)
<thumper> :)
<thumper> bts?
 * thumper doesn't know much about debian
<sanxiyn> thumper: http://bts-link.alioth.debian.org/
<wgrant> The BTS is the Debian Bug Tracking System
<thumper> ah
<sanxiyn> And bts-link is a component of that system that can track remote Bugzilla, Trac, GForge, RT, whatever.
<thumper> anyway...
 * thumper EODs
<wgrant> noodles775: Hi.
<noodles775> Hi wgrant :), Congrats on getting your MP approved!
<wgrant> noodles775: Thanks.
<wgrant> noodles775: Are there any designs for the new DSP page with its version listing yet? Bug #144620 would probably be fixable in the redesign.
<ubot3> Malone bug 144620 in soyuz "Some displayed sourcepackagerelease changes files don't have attribution" [High,Triaged] https://launchpad.net/bugs/144620
<mup> Bug #144620: Some displayed sourcepackagerelease changes files don't have attribution <Soyuz:Triaged> <https://launchpad.net/bugs/144620>
<mup> Bug #144620: Some displayed sourcepackagerelease changes files don't have attribution <Soyuz:Triaged> <https://launchpad.net/bugs/144620>
<mup> Bug #144620: Some displayed sourcepackagerelease changes files don't have attribution <Soyuz:Triaged> <https://launchpad.net/bugs/144620>
<wgrant> Oh dear.
<noodles775> wgrant: Only what's in those images attached to the blueprint... I think bigjools is keen for us to work on the DSP refactoring asap...
<noodles775> wgrant: I'm just looking at the new 3-0 style refactoring (and having a go at applying it to the archive-index)
<wgrant> noodles775: I also don't really like the elimination of the D(S)BP(R) pages. They're very useful for looking at the sometimes complicated histories.
<noodles775> wgrant: The idea (which may not be clearly expressed) is that the pages would still exist, but not be needed when the user has JS on.
<wgrant> noodles775: Ah, I see. That is fine, then.
<noodles775> wgrant: ie. that info would be available on the DSP page (after drilling down), but would still need to be there for non-js browsers.
<noodles775> Cool!
<wgrant> Some of the info isn't appropriate for the DSP page.
<wgrant> (eg. BPRs with the same name but from a different source)
<noodles775> Why would a BPR that wasn't from that source be displayed on the page?
<noodles775> ah, I see
<wgrant> noodles775: It wouldn't, which is a good reason for the DASBP page to exist. Otherwise I can't find the history of what the name mapped to.
<noodles775> wgrant: Hmm... it'd be great to capture these as stories on the spec, could you add one?
 * noodles775 checks the (practically empty) spec
<wgrant> noodles775: Under Use Cases?
<noodles775> wgrant: yep
<noodles775> Thanks!
<noodles775> wgrant: just add any positive or negative use cases, we can evaluate them when we've collected lots.
<wgrant> noodles775: Sure.
<adeuring> good morning
<noodles775> Morgan adeuring
<adeuring> hi noodles775!
<mrevell> Morning :)
<bigjools> morning all
<wgrant> Morning bigjools.
<bigjools> hey wgrant
<henninge> Hi!
<henninge> what is that page again that shows all of the used icons on launchpad?
<wgrant>  /+graphics
<henninge> wgrant: cheers
<jtv1> mrevell: morning!
<mrevell> hey jtv1!
<mrevell> jtv: I'll just get a cup of tea, if that's ok
<bigjools> thunderstorm alert here, I might go netless any time
<mrevell> bigjools: really?
<jtv> mrevell: sure, I'll have one too thanks
<bigjools> mrevell: yes, my power goes out usually
<mrevell> bigjools: Wow but I'd still rather live in Kempsey, heh
<bigjools> mrevell: the house next door is up for sale :)
<mrevell> bigjools: interesting -- PPA and Soyuz help whenever I want it :)
<bigjools> mrevell: I have guard dogs, barbed wire and feral children
<mrevell> :) I bet I have more dogs
<bigjools> i used to be able to say I had more children :)
<mrevell> heh
<mrevell> jtv: Skype or IRC sir?
<jtv> mrevell: I think irc for this one, since it's all in writing... good way to throw notes at each other :)
<mrevell> jtv: sounds great
<jtv> mrevell: I sent you an email with the text I have.
<mrevell> jtv: I have it here, I'll read it then PM you
<jtv> mrevell: perfect
 * jtv reaches for coffee
<deryck> Morning, all.
<bigjools> I'm really sad that my new Dell Mini 10v, with Ubuntu pre-installed, has got a key with a Windows logo on it :/
<noodles775> bigjools: wierd?
<bigjools> crass
<noodles775> Maybe canonical could provide Dell with similar keys?
<andrea-bs> adeuring, Hi! May I ask you a question?
<adeuring> andrea-bs: sure :)
<andrea-bs> adeuring, thanks for your review for my branch "blueprint-whiteboard-diffs"; what should I put instead of 72 for get_unified_diff()?
<adeuring> andrea-bs: nothing ;) As BjÃ¶rn pointed out, my question about the diff formatting was a a bug in the review.
<andrea-bs> adeuring, oh, sorry... I should read all the comments before asking :)
<adeuring> andrea-bs: no prblem ;)
<wgrant> Who owns canonical.zeca?
<bigjools> good question, cprov-afk  do you know?
<cprov-afk> wgrant: foundations, I guess
<cprov-afk> wgrant: ideas ?
<cprov-afk> wgrant: or bugs ?
<stub> Isn't that the GPG thingy used by Soyuz?
<stub> I don't think anyone in foundations has ever messed with it...
<wgrant> cprov-afk: I needed a keyserver.launchpad.dev, so I patched it to work properly. Previously IGPGHandler would be unable to re-get keys from it. http://williamgrant.id.au/f/1/2009/zeca-get-key-id.diff is the ugly patch, and I wonder what to do.
<stub> Or knows enough Portuguese to even understand what the name means :-P
<wgrant> cprov-afk: it works fine for tests, because the IGPGHandler survives for the whole test.
<wgrant> But that patch makes it work for real stuff as well.
<cprov-afk> wgrant: the patch is fine, why do you need zeca on a production-like environment ?
<wgrant> cprov-afk: I needed a local keyserver to verify signatures (process-upload, CoC), and zeca is close enough that it was easier than setting up a proper one.
<cprov-afk> wgrant: right, but it would only work for a fixed set of keys
<wgrant> cprov-afk: No. It accepts uploads.
 * mrevell is off to grab some lunch
<cprov-afk> wgrant: jeez, it does !! I wrote it...
<wgrant> cprov-afk: Haha.
<wgrant> It has been a while, I imagine.
<cprov-afk> wgrant: the patch is okay. Doesn't hurt, really, but you need tests. I can help you in one hour or so.
<wgrant> cprov-afk: I was going to write tests, but I couldn't work out how to run the existing ones.
<cprov-afk> wgrant: `./bin/test -vv -t canonical.zeca`
<wgrant> cprov-afk: Ah. I missed the -t - lp.* don't need it :(
<wgrant> cprov-afk: Thanks.
<cprov-afk> wgrant: np
<bigjools> cprov-afk: jeez that is an old file
 * gmb => afk for a bit.
<bigjools-afk> wgrant: your config change is affecting the tests for some reason, tests that refer to keyserver.ubuntu.com are failing
<wgrant> bigjools_: Oh, blah. I see testrunner's conf extends development's.
<bigjools_> right
 * wgrant overrides it in testrunner's conf.
<wgrant> bigjools_: Which tests were they?
<bigjools_> I wish I had a complete list, I just had a power outage after running the test suite for 2.5 hours >:(
<bigjools_> but grep is your friend
<bigjools_> gpghandler.txt and foaf/xx-person-rdf.txt
<bigjools_> at least
<bigjools_> and xx-ubuntu-ppas.txt
<wgrant> Yep. Rerunning tests mentioning keyserver.ubuntu.com now...
<bigjools_> ta
<bigjools> wgrant: I am going to lunch, comment on the MP when you're done
<wgrant> bigjools: Pushing now.
<wgrant> bigjools: All those tests pass now.
<maxb> rocketfuel-setup configures a "merge_target" line in ~/.bazaar/locations.conf - I cannot find any mention of this configuration item in the Bazaar User Reference, nor in google. Is it a typo?
<allenap> maxb: I think it's for the lpreview plugin. Which I don't use and don't really remember anything about, but it's probably safe if you leave it or remove it.
<bac> good morning sinzui.  when you get a chance could you look at https://answers.edge.launchpad.net/launchpad-registry/+faq/567.  the last sentence of the second paragraph is confusing to the point i don't know what you're saying.
<sinzui> yes that is awkward
<sinzui> bac: Did I fix it?
<sinzui> reload the page
 * bac looking
<bac> sinzui: yes that is better.  i understand what you are saying but not why.  the trusted team is set up to not get any email.  are you saying the team will miss legitimate, non-bug email sent to the team?
<bac> maxb: i do use the lpreview plugin and i have my merge_target set the same as my submit_branch.
<sinzui> I am saying that launchpad developers are idiots. They made mailing lists impossible to to use, then they turned subscriptions into teams. Setting up this recipe requires EVERY user to get his subscription setting right. The project owner most certainly get complaints.
<bac> sinzui: ah, i see.
<sinzui> bac, Should I put that into the FAQ too?
<bac> sinzui: was the way i characterized it a suitable subset of what you really meant?
<bac> i think, perhaps, we should not include your real version.  :)
<sinzui> bac: I am sorry, this recipe is a little different than I recall. since we are not relying on a list, I think the sentences can be dropped
<bac> ok
 * sinzui updated the FAQ again and drinks more coffee
<bac> sinzui: i wish we could grow an option for the team to just not get email rather than providing a legitimate but unused one.
<sinzui> bac: we need to take a hammer to teams and subscriptions to ensure their are no ambiguities in how they are used. Each may exist for access-control or communication, but neither works well if it is used for both purposes.
<thekorn> how can I reset the database of my launchpad instance to the default values?
<wgrant> thekorn: make schema
<thekorn> ok, I thought there is something simpler than this
<salgado> thekorn, there is
<salgado> thekorn, createdb -E UNICODE --template=launchpad_dev_template launchpad_dev
<salgado> first you'll need a dropdb launchpad_dev, though
<thekorn> ok, cool, noted for the next time
<bac> salgado: hmm, that doesn't look simpler...
<salgado> bac, good point, but it's a hundred times faster. :)
<thekorn> it depends on the definition of "simpler"
<bac> oh, well,yeah.
<mrevell> abentley: Hi, I see you were the last person to edit https://dev.launchpad.net/DatetimeUsageGuide -- there's a similar but longer document here -- https://launchpad.canonical.com/DatetimeUsageGuide ---- is there anything on the old launchpad.c.c wiki version that should be on the dev.lp.net version, in your view?
<mars> good morning launchpad!
<abentley> mrevell: I'm not sure how this happened.  I think the canonical.com version is more useful.
<mrevell> thanks abentley
<mars> thekorn, if you are curious and want to play with the db, check database/schema/Makefile
<mars> there are targets in there for doing things like creating the launchpad_empty database, and 'make diagram'
<BjornT_> mars: can i get your opinion on this patch? https://pastebin.canonical.com/20291/
<barry> abentley, mrevell i haven't looked closely, but the c.c version should be merged into the l.n version
<BjornT_> mars: or tell me who's the best person to review it
<sinzui> barry, bac, Edwin, salgado: standup in 2 minutes
<BjornT_> mars: it's to prevent the bug page from temporarily showing the 'subscribe someone else' picker when loading, causing the beginning of the page to be focused
<mars> BjornT_, hmm, I'd refer you to sinzui or noodles775
<BjornT_> mars: ok, no worries
<mars> BjornT_, that is an interesting way to fix it then
<BjornT_> sinzui, noodles775: can you take a look at this patch? https://pastebin.canonical.com/20291/
 * noodles775 looks (just on phone)
<mars> is that even a valid value for 'visible'?
<BjornT_> mars: i think it should be ok, since there's a hide() further down, so it should be hidden. i assumed 'visible' should be a boolean, shouldn't it?
<sinzui> visibility: <visible|hidden>
<sinzui> BjornT_: Are you using an unsupported value to take advantage of a CSS engine's weeknes?
<BjornT_> sinzui: i'm not passing a CSS value there. it's an JS attribute
<mars> oh!
<sinzui> BjornT_: I see, thank you for clarifying this
<mars> BjornT_, sorry, it didn't click in.  You'll have to test it
<mars> at the Berlin sprint, we found some strange interactions with the 'visible' attribute
<mars> if it works, then cool
<bac> hi mrevell
<mrevell> hi there bac
<bac> mrevell: if you don't need my help on feedback moderation any more could you remove me from the owners?
<BjornT_> mars: it doesn't seem to be causing any problems. do we use the picker anywhere else, apart from on the bug page?
<BjornT_> sinzui: ^^^
<mrevell> bac sure thing, gimme a mo
<noodles775> BjornT_: I've used the person picker for archive subscriptions (/~user/+ppa/name/+archivesubscriptions)
<noodles775> BjornT_: I'll send you a script that will set one up for you.
<BjornT_> noodles775: cool
<mrevell> bac sorted
<gmb> Ooh, pretty.
 * gmb just discovered bin/test -c by accident
<mars> gmb, ever tried '-D' ?
<gmb> mars: No, what happens then? Does it Dance?
<bac> gmb:  wow, -c gives me limegreen on my blue background.  u-g-l-y
<mars> no, it just dumps you into a pdb console at the exact point where the test raised an exception or failure.
<gmb> Nice.
<gmb> Except it's a pagetest, so maybe not so much.
<gmb> bac: Eww.
 * gmb has a traditional black background with white text as standard.
<mars> a bit more annoying for pagetests.  You need to up-frame a bit
<mars> 'u<ret><ret><ret>' at the prompt
<gmb> BjornT_: Question: w.r.t removing +filebug-advanced (or at least making it redirect to +filebug) there are tests that ensure that clicking on 'Advanced bug reporting options' on a +filebug page will maintain any blob tokens passed to +filebug. Those tests are obviously not relevant now, but are there any cases in which a blob token would be submitted to +filebug-advanced? Should I worry about tokens when redirecting to +filebug?
<BjornT_> gmb: no, that shouldn't be an issue. afaik, nothing is linking to +filebug-advanced passing along a blob token
<gmb> BjornT_: Cool.
<BjornT_> noodles775: did you send that script?
 * gmb like excising tests
<noodles775> BjornT_: sorry, not yet (just replying to an email). I'll have it in 3 mins :)
<BjornT_> noodles775: ok, no worries :)
<noodles775> BjornT_: http://pastebin.ubuntu.com/230555/
<sinzui> kiko: jml added it some time ago. We did not have a rule to link lp: to a bug, so there was no conflict for him to see
<bac> hi salgado
<kiko> sinzui, that's fine, as long as it actually works
<kiko> sinzui, and I think there's no conflict -- if it's lp:3213 then it's a bug; if it's lp:foo then it's a project branch
<salgado> hi bac
<sinzui> kiko: right. I was going to file a bug to the affect that the linker needs more specific rule
<bac> hi kiko:  the branh i landed yesterday with your RC accidentally reverted a small fix to rocketfuel-get.  you can see it here:  http://bazaar.launchpad.net/~launchpad-pqm/launchpad/db-stable/revision/8318
<bac> kiko:  do you want me to correct that in db-devel now or wait until PQM opens and fix it in devel?
<BjornT_> noodles775: thanks. the ppa page still works :) can i get an r=you for that patch?
<bac> matsubara: when are we doing the re-roll-out?
<noodles775> BjornT_: So it's just the lp widget using visible=False as a default right? And you've grepped/tested any other uses of the widget in LP? If so, r=me
<noodles775> BjornT_: actually, why does the default for the widget need to be set to false? Why can't the bugs page that uses it create the widget passing in false? (maybe I missed that before?)
<kiko> sinzui, sounds perfect
<ursula_> sinzui, hi
<barry> losa ping
<sinzui> hi ursula_
<kiko> bac: why did your branch revert that?!
<BjornT_> noodles775: well, we would basically need to pass it as false everywhere it's used, no?
<Ursinha> sinzui, I'm not able to access +milestones page, it's timing out consistently on edge and lpnet
<matsubara> bac, don't know yet
<bac> kiko:  i did a merge before submitting and it must've come from the wrong branch
<Ursinha> sinzui, I just filed a bug for it, bug 404120, could you take a look, please?
<ubot3> Malone bug 404120 in launchpad-registry "Timeout in +milestones page" [Undecided,New] https://launchpad.net/bugs/404120
<mup> Bug #404120: Timeout in +milestones page <timeout> <Launchpad Registry:New> <https://launchpad.net/bugs/404120>
<mup> Bug #404120: Timeout in +milestones page <timeout> <Launchpad Registry:New> <https://launchpad.net/bugs/404120>
<mup> Bug #404120: Timeout in +milestones page <timeout> <Launchpad Registry:New> <https://launchpad.net/bugs/404120>
<Ursinha> hahaha
<Ursinha> someone need to tell ubot3 to leave
<matsubara> sinzui, can you fix that timeout for the re-roll?
<sinzui> Ursinha: matsubara: no
<kiko> bac, hmph, please read your diffs carefully before you submit, that's pretty bad
<kiko> Ursinha, matsubara: does anybody else have an RC branch to land?
<matsubara> kiko, it'd be nice to have that timeout fixed ^
<sinzui> matsubara: Ursinha: well, I cannot fix the page, but I can look into not showing bug counts. like we did on the +series page. They are expensive to get since there is a privacy check for each one
<kiko> sinzui, you could load the bug counts via ajax like the bugs page does, maybe even reusing that code?
<matsubara> mthaddon, when's the deadline to submit code for the re-roll?
<sinzui> kiko: indeed that is why I said no at first. I engineered the templates to let me do it via AJAX., but I need more time to write a script that uses it
<kiko> mthaddon, I don't think we will re-roll at all unless I see good reason
<noodles775> BjornT_: I thought most call-sites created it with visible=false for that reason... I'm just not sure whether there is a YUI standard default for visible.
 * noodles775 looks
<kiko> matsubara, ^^
<matsubara> kiko, are we not going to re-roll 8314..8319?
<sinzui> kiko: I will stop what I am doing and looking to an immediate solution to the page and create a bug about the long term solution
<kiko> sinzui, just remove the information for now
<BjornT_> noodles775: well, the ppa page has the same issue as the bug page. i can't see it shown, but it grabs focus, so that you can't use Page Up and Page Down to scroll
<kiko> matsubara, if mthaddon wants to, but then I'd rather just land sinzui's branch and re-roll
<kiko> sinzui, if you are going to prepare a branch, please include the patch that bac reverted
<matsubara> kiko, sounds like a good plan
<kiko> sinzui, just show it to me and I'll r/rc
<sinzui> okay
<kiko> mthaddon, do you feel like trying to update the librarian?
<BjornT_> noodles775: and considering hide() is called in create(), i don't see a reason not to pass in visible: false
<noodles775> BjornT_: +1, if that's the case!
<mars> bigjools, Ursinha, ping, is it a known issue that filtering the PPA package list using unicode characters causes an OOPS?
<Ursinha> mars, not in that specific field, afaik, but will check for a bug #
<bigjools> not heard of that
<BjornT_> noodles775: thanks! i've grepped for other uses, and it's ok.
<noodles775> BjornT_: great :)
<mars> bigjools, OOPS-1300A556 and OOPS-1300G2888 both come from the same thing
<ubot3> https://devpad.canonical.com/~jamesh/oops.cgi/1300A556
<ubot3> https://devpad.canonical.com/~jamesh/oops.cgi/1300G2888
<mars> and I know too little about storm to tackle that error efficiently
<mars> someone with better knowledge about how storm and BatchNavigator interact would be able to make quick progress
<BjornT_> kiko: what do you think of fixing bug 392138 and bug 404088 for the re-rollout?
<mup> Bug #392138: Broken keyboard navigation on bug pages <Launchpad Bugs:In Progress by bjornt> <https://launchpad.net/bugs/392138>
<ubot3> Malone bug 392138 in malone "Broken keyboard navigation on bug pages" [High,In progress] https://launchpad.net/bugs/392138
<mup> Bug #404088: Overlay temporarily shown when loading the bug page <Launchpad Bugs:In Progress by bjornt> <https://launchpad.net/bugs/404088>
<mars> Ursinha, you'll probably kill a number of OOPses on search pages by fixing that issue
<mup> Bug #392138: Broken keyboard navigation on bug pages <Launchpad Bugs:In Progress by bjornt> <https://launchpad.net/bugs/392138>
<ubot3> Malone bug 404088 in malone "Overlay temporarily shown when loading the bug page" [Medium,In progress] https://launchpad.net/bugs/404088
<mup> Bug #392138: Broken keyboard navigation on bug pages <Launchpad Bugs:In Progress by bjornt> <https://launchpad.net/bugs/392138>
<mup> Bug #404088: Overlay temporarily shown when loading the bug page <Launchpad Bugs:In Progress by bjornt> <https://launchpad.net/bugs/404088>
<mup> Bug #404088: Overlay temporarily shown when loading the bug page <Launchpad Bugs:In Progress by bjornt> <https://launchpad.net/bugs/404088>
<bigjools> botfight
<mars> more bot barf!
<BjornT_> kiko: the fix is simple, but it's js, so it might be a bit risky anyway: https://pastebin.canonical.com/20291/
<Ursinha> gee
<Ursinha> ubot3, owner
<ubot3> This bot is owned by jussi01 - Questions about ubottu should be asked in #ubuntu-bots
<Ursinha> can we remove ubot3 from here?
<bigjools> Ursinha: can you file a bug about that unicode issue please, if there isn't one
<Ursinha> bigjools, doing now
<Ursinha> bigjools, do you know how to fix that?
<bigjools> no
<bigjools> but I have a suspicion
<abentley> kiko: I've introduced a bug for Revision emails where the merge proposal has an outstanding review request, that causes them to oops.  I have a two-line fix.  Can I get an RC? https://pastebin.canonical.com/20301/
<kiko> BjornT_, abentley: looking
<Ursinha> bigjools, bug 404144
<ubot3> Malone bug 404144 in soyuz "UnicodeDecodeError when using unicode chars filtering PPA packages" [Undecided,New] https://launchpad.net/bugs/404144
<mup> Bug #404144: UnicodeDecodeError when using unicode chars filtering PPA packages <oops> <Soyuz:New> <https://launchpad.net/bugs/404144>
<mup> Bug #404144: UnicodeDecodeError when using unicode chars filtering PPA packages <oops> <Soyuz:New> <https://launchpad.net/bugs/404144>
<mup> Bug #404144: UnicodeDecodeError when using unicode chars filtering PPA packages <oops> <Soyuz:New> <https://launchpad.net/bugs/404144>
<Ursinha> aww
<bigjools> please get him to get rid of ubot3
<mars> who's channel op?
<bigjools> or mup, I  don't care which
<mars> mup is easier
<mars> at least ubot3 does OOPS reports
<bigjools> thanks for filing it Ursula
<kiko> BjornT_, how bad is the problem? if that breaks things further on lpnet, will it be worse than the current situation?
<bigjools> mars: with the wrong url!
<matsubara> isn't it a dupe of 44919?
<matsubara> Ursinha, ^
<mars> bigjools, good point
<kiko> abentley, yes, rc=kiko for that, but I'd try and land that close to sinzui's patch to avoid having to do multiple buildbot runs
<Ursinha> matsubara, I don't think so, but will check again
<Ursinha> bug 44919
<ubot3> Malone bug 44919 in launchpad-foundations "UnicodeDecodeError while POSTing forms with non-ascii characters." [High,Triaged] https://launchpad.net/bugs/44919
<mup> Bug #44919: UnicodeDecodeError while POSTing forms with non-ascii characters. <infrastructure> <oops> <Launchpad Foundations:Triaged by bjornt> <https://launchpad.net/bugs/44919>
<mup> Bug #44919: UnicodeDecodeError while POSTing forms with non-ascii characters. <infrastructure> <oops> <Launchpad Foundations:Triaged by bjornt> <https://launchpad.net/bugs/44919>
<mup> Bug #44919: UnicodeDecodeError while POSTing forms with non-ascii characters. <infrastructure> <oops> <Launchpad Foundations:Triaged by bjornt> <https://launchpad.net/bugs/44919>
<sinzui> kiko: I am close to a patch. It breaks a few tests that I need to reconcile
<abentley> kiko: I haven't run the test suite against it yet.
<kiko> abentley, coordinate with sinzui
<kiko> sinzui, abentley: thanks a lot for handling these, you guys rock
<mars> matsubara, I'm not sure if that is the same issue
<abentley> kiko, np
 * mars check the stack trace
<Ursinha> matsubara, the problem seems a little different
<mars> yes, that is different
<abentley> sinzui: I'm running the full test suite against my branch, started a minute ago.  I'll let you know when I'm done.
<BjornT_> kiko: it's an annoyance, but nothing serious. with javascript even simple fixes can break things badly, so i'm inclined to land it when pqm opens up. i just wanted to see if you had another opinion.
<mars> matsubara, the issue we were looking at is Storm + BatchNavigator + Unicode = boom!
<sinzui> bac: can you advise me about how to restore the rocketfuel-get change that was reverted
<bac> sinzui: undo this: https://pastebin.canonical.com/20303/
<bac> sinzui: in other words, restore line 8
<sinzui> bac: thank you
<matsubara> mars, ok
<mars> matsubara, and this appears to be a problem with the user's requested language encoding - in the other OOPSes, the browser does accept en-us
<bac> sinzui: thanks to you for including it
<kiko> BjornT_, no, I agree with you that it's better to put it on edge and let users cry
<mars> sinzui, would you or salgado have a moment to fix -61171?
<mars> it is a one or two line fix
<mars> but the test is beyond me
<salgado> mars, is that a bug?
<mars> salgado, yes, I don't want the bots to get it
<salgado> why? I do
<mars> bug 61171
<ubot3> mars: Bug 61171 on http://launchpad.net/bugs/61171 is private
<salgado> oh, bot*s*
<Ursinha> hm, mup ignored the bug because it's private?
<mars> perhaps, but I don't think that is a scalable solution to bot-fights :)
<Ursinha> :)
<mars> salgado, mwhudson managed to reproduce the bug yesterday, but for some reason you have to match the OOPsing string specifically
<mars> so we couldn't come up with a general test case
<mars> salgado, would you even bother with a test case here?  Or would you just fix it to get rid of the OOPS?
<Ursinha> mars, salgado, do you think it can make it for the re-roll?
<mars> Ursinha, tbh, not if I wrote it - that is why I asked salgado and sinzui
<salgado> Ursinha, mars, I'm OCR today
<mars> salgado, I'm CHR :)
<Ursinha> uhoh
<Ursinha> well, ok then
<Ursinha> salgado, can you fix that on Monday then?
<salgado> Ursinha, I haven't had a look at the bug yet, so I have no idea how much work is involved
<Ursinha> salgado, let me change the question then, can you take a look at the bug on Monday? :)
<salgado> hell, yeah.  that's for sure
<Ursinha> thanks salgado :)
<Ursinha> and mars for investigating that :)
<allenap> kiko: About that lp:~allenap/launchpad/bugs-without-tasks-bug-403182 branch I asked you about this morning, Bjorn noticed a few other cases that needed to be taken care of. They're done and adeuring has reviewed them. A diff from this morning to now <https://pastebin.canonical.com/20297/>. Full diff against db-devel <https://pastebin.canonical.com/20309/>. It's still a small code change. Please could you take a look and give your blessing, then I'll
<allenap> get it into db-devel asap?
<kiko> allenap, is the last hunk in handlers.py a sort of assertion -- in other words, is that the only place where we actually decide to create the bug and commit?
<kiko> allenap, my question is whether or not an assertion or oops would make sense
<allenap> kiko: Off the bottom of that diff the same check is done, before notify()ing the bug_event. I just did the same thing.
<allenap> kiko: It's better to check then the user gets an email saying what they did wrong, rather than an assertion which just blows up.
<kiko> allenap, sure. but I think you're telling me that there is no single location that we can add this check to, which suggests you're right in wanting to refactor this code. am I right?
<kiko> allenap, I'm saying r/rc=kiko regardless
<kiko> allenap, coordinate with sinzui and abentley so we don't get stuck waiting for two buildbot runs
<allenap> kiko: Yeah, that's it.
<allenap> kiko: Thanks, I'll talk to them.
<kiko> allenap, i raised my eyebrow to the change in the second file in the patch, but it's harmless and correct
<allenap> kiko: :)
<sinzui> kiko: I need to make another change. my fix need to be more comprehensive than just project-groups. I need another 30 minutes
<mrevell> Is kfogel due online today or is he still OSConing?
<kiko> the latter I believe
<allenap> sinzui: When your change is ready to submit (to db-devel I assume), could you merge my lp:~allenap/launchpad/bugs-without-tasks-bug-403182 branch so that it all goes through the same buildbot run?
<allenap> abentley: ^ is your change ready too?
<sinzui> allenap: I do not use buildbot
<sinzui> allenap: sorry
<sinzui> I am an idoiot
<allenap> sinzui: You read ec2test? :)
<abentley> allenap: My change has not passed a test run yet.
<sinzui> allenap: I do not use ec2test.
<abentley> sinzui: me neither.
<allenap> abentley: kiko mentioned (in another channel) that we should just get our changes into db-devel together so that they have a buildbot run together. If it succeeds, re-roll, else we get to pick up the pieces for a CP on Monday.
<sinzui> allenap: if your branch is tested, all is cool
<abentley> allenap: I saw that.
<allenap> sinzui: I'm testing a limited subset of it locally.
<allenap> abentley: Ah, do you mean that it fails a test run and you're working to fix it, or are you waiting for a run to complete?
<allenap> Where "it" is "your branch".
<abentley> allenap: I am waiting for a run to complete.
<allenap> Okay.
<abentley> allenap: I just restarted the test because salgado asked for some tweaks.
<allenap> abentley, sinzui: I'm running as much of the test suite as I can locally, but I have to go in about 30 minutes. Could whoever of you ends up submitting to db-devel please include my branch? Don't worry if it's problematic. It's not totally critical, and we have to do some clean-up anyway so a CP on Monday will be fine.
<abentley> allenap: I feel uncomfortable landing code that has not been fully tested.
<sinzui> allenap: I hope to start testing in 15 minutes
<allenap> abentley: Okay, that's fine, it'll have to go in on Monday.
<abentley> allenap: I could merge your branch and sinzui's now, and restart the tests.
<abentley> sinzui: Or is your branch still being written?
<sinzui> I am still writing it
<allenap> abentley: I would love it if you could run the tests for my branch and land it assuming it's good. I really really hope it's good.
 * sinzui is struggling to keep the changes to a template and one group of tests
<abentley> allenap: Okay, URL?
<allenap> abentley: lp:~allenap/launchpad/bugs-without-tasks-bug-403182
<sinzui> kiko: This is my fix for the +milestone pages and the rocketfuel fix: https://pastebin.canonical.com/20322/
<sinzui> kiko: I should add that the template, view , and test has not changed since I landed the update to +milestone page last week. We know this is the only test that must change.
 * sinzui prepares to start a test run
<sinzui> allenap: that is a pretty deep change. Did you run all the tests for lp.bugs?
<allenap> sinzui: Yes, they all pass. Why is it so deep?
<sinzui> It is lower than pagetests. That is all I meant. I would test everything from zopeless to pagetests
 * sinzui cannot remember if bugs has pagetests that test email
<sinzui> allenap: I'll merge your branch into mine and play it on my hardy box.
<allenap> sinzui: Okay, thanks. abentley is also running the full test suite with it merged too. Thanks guys :) I have to go now, but I'll check back later (probably much later).
<sinzui> allenap: abentley: then I think I just need to verify my changes
<abentley> sinzui: Well, I haven't started running allenap's tests, because I'm still downloading his branch.
<allenap> abentley: Really? Its diff against db-devel is a bit over 200 lines.
<abentley> allenap: I've downloaded 160 M so far..
<allenap> Unfortunately I really have to go now :(
<allenap> abentley: That's not right! Abandon it, it can wait for Monday. Get your stuff in.
<sinzui> abentley: okay. I will merge his and play lp.bugs
<abentley> sinzui: roger.
 * maxb views https://dev.launchpad.net/Releases/2009Calendar and wonders why the version numbers jump immediately from 3.0 to 3.1.x
<bigjools> bye all, have a nice weekend
<mrevell> Okay guys,have a good weekend
<sinzui> barry: ping
<bac> hi sinzui
<sinzui> hi
<bac> sinzui: the problem stu is having is b/c he is trying to add a private membership team (~canonical) to a public team, which we disallow via the ValidTeamMemberVocab
<sinzui> bac: as I thought
<sinzui> did you update the team
<bac> sinzui: i can relax that constraint to allow private teams to join public
<bac> sinzui: update which team?  we do not allow PMT or Private teams to join others as it is now.
<sinzui> Why not change all the teams involved to private teams as we planed
<bac> so converting ~canonical to private now won't help
<sinzui> bac: then we need to land a fix as we talked about. Gustavo is also looking for this fix
<bac> sinzui: agreed
<sinzui> I think all the users who want this are edge users so we can settle this in good pace
<sinzui> bac: I did not see a bug for this, though we have discussed it. Create a bug if you cannot find one
<barry> losa ping
<mthaddon> barry: hi
<barry> mthaddon: hi.  i have a script that will help us regenerate all the mailing list archives.  are you up for giving it a try?
<mthaddon> barry: sure
<barry> mthaddon: cool.  we can try one or two lists first, and then the whole shebang
<barry> mthaddon: can you grab the lp-dev-utils branch?
<mthaddon> barry: lp:lp-dev-utils ?
<barry> mthaddon: lp:~launchpad/lp-dev-utils/trunk
<barry> mthaddon: once you grab that branch, look for the archiveregen.py script
<mars> btw, why does lp-dev-utils not have a trunk/ set?
<barry> mars: dunno.  i guess nobody ever set it.  jfdi man! :)
 * mars shrugs
<barry> mthaddon: we need to copy that script onto forster and then ping me for a test run
<mars> ok
<mthaddon> barry: ok, I have it on forster
<barry> mthaddon: double check the paths at the top of the file, and comment out the second BASE_PATH setting
 * barry is not positive the first one is correct
<mthaddon> barry: done
<mthaddon> barry: it is the right path
<barry> mthaddon: cool, now we'll convert first the haibunku list and see how it goes
<barry> mthaddon: thanks
<barry> so...
<barry> mthaddon: ./archiveregen.py haibunku
<mthaddon> barry: some success I think?
<mars> hmm, usability FAIL for setting the default branch for lp-dev-utils, exactly as bzr says I should
<mars> the word 'Default branch' doesn't appear anywhere in the UI from what I can see
<barry> mthaddon: total success i think :)
<mars> let alone something to convey the idea that I might set it
<mthaddon> barry: ./archiveregen.py --all ?
<barry> mthaddon: let's cross our fingers and jfdi
<mars> beuno, ^ any thoughts on the 'Default branch' thing?
<barry> mars: :(
<beuno> mars, I lack context
<beuno> even after reading the back log
<beuno> want to fill  me in?
<mars> beuno, I try this in the UI:
<mars> $ bzr co lp:lp-dev-utils
<mars> bzr: ERROR: Invalid url supplied to transport: "lp:lp-dev-utils": Launchpad Developer Utilities has no default branch.
<mars> err, at the command line
<barry> mars: i just set "code for this branch is devleoped in lp"  dunno if that helps
<mars> beuno, so I go to the lp-dev-utils code page, see that it has a 'trunk/', and a development focus, but nothing to say what is wrong with bzr lp:
<beuno> mars, barry, I assume that project is privatae
<beuno> so lp: doesn't resolve
<barry> it is private
<beuno> exactly what *used* to happen with Launchpad  :)
<beuno> lp: uses a public xmlrpc to figure that out
<mars> then why does it work if you give it the full branch URL?
<beuno> so it means no lp: shorcuts for private projects
<mars> ah, so lp:foo-bar doesn't work
<mars> but a full lp: url does
<mars> because it just has to substitute the transport?
<beuno> mars, because that doesn't do any guessing, it just converts "lp:" into "http://bazaar.launchpad.net/"
<mars> ok
<beuno> yes, and uses ssh if you have you lp=login set as well
<beuno> it doesn't do anything server-side
<beuno> I'm not saying this is the right solution
<beuno> just filing you in on it
<barry> mthaddon: still running?
<mthaddon> barry: yep
<barry> coolio
<mthaddon> barry: done
<barry> mthaddon: fab.  let me see if i can find a mailing list to look at
<barry> (that wasn't already converted)
<barry> mythbuntu-documentation looks good
<barry> mthaddon: i think we're good.  thanks!
<mthaddon> barry: cool
<barry> sinzui: would you like to talk about this lazr.restful test?
<sinzui> yes
<barry> skype?
<barry> sinzui: skype?
<sinzui> barry: yes. I am ready
<sinzui> kiko: My test run of my branch + allenap's + bac's rocketfuel patch is complete. Do you want me to land my branch with abentley's
<kiko> sinzui, the branches can land separately as long as they all fit into the same BB run
<sinzui> understood
<kiko> abentley, yours is ready too right?
<abentley> kiko: Yes.
<kiko> sinzui, abentley: let's land them?
<abentley> kiko: Okay, submitting now.
 * sinzui submits
<EdwinGrubbs> bac: ping
<bac> EdwinGrubbs: hi
<EdwinGrubbs> bac: did you see that email from salgado to us about rocketfuel-get?
<bac> EdwinGrubbs: yes.  that's "bac's rocketfuel patch" referenced in sinzui's message up there ^.
<bac> EdwinGrubbs: it was my blunder as i got confused moving between devel and db-devel
<sinzui> EdwinGrubbs: bac: I am waiting for a success message from pqm now
<EdwinGrubbs> bac: ok, I wanted to make sure that I was the only one ignoring that email.
<bac> EdwinGrubbs: i'm not sure why you were included in the mess.  collateral damage.
<salgado> bac, I CCed EdwinGrubbs because he was in the list of authors of that revision.  such list looks like a new feature of LP
<bac> salgado: yeah, but i'm not sure why he was.
<bac> salgado: 8318 was all mine:  http://bazaar.launchpad.net/~launchpad-pqm/launchpad/db-devel/revision/8318
<bac> i'm also confused as to why i don't get db-devel  landing messages even though i'm subscribed to the branch
<salgado> bac, are you subscribed to revision notification on that branch?  I've had problems with that too, but it was because my subscription was not properly configured
<BjornT_> bac: you merged in some of edwin's revisions in r8318, thus it wasn't all yours
<bac> BjornT_: ah, that's right.
<bac> salgado: i just checked.  i have 'branch and revision notifications' turned on, send entire diff, and email all changes.
<maxb> Hmm. Would you expect len() on a storm ResultSet to return 4294967295 ?
<sinzui> abentley: my branch is in
<abentley> sinzui: Mine is in PQM./
<maxb> Hi - I'm trying to figure out why calling len() on an object which *claims* to be a storm ResultSet is returning 2**32-1 despite the fact that a storm ResultSet doesn't implement len
<maxb> Can anyone suggest if the LP code is likely to have thrown a proxy around it, and how I might detect that? repr(x) and x.__class__ just say storm.store.ResultSet
<sidnei> maxb: try from zope.proxy import isProxy
 * maxb tries
<maxb> yup, it's a zope proxy
<maxb> thanks
<maxb> bingo, looks like it's the int vs. Py_ssize_t issue as you said in #storm
<sidnei> maxb: yeah, the proxy is written in C, so the version being used by launchpad probaby does not contain  the fix for that
<sidnei> maxb: http://pypi.python.org/pypi/zope.proxy
<sidnei> maxb: the version launchpad is using is roughly what was shipped as zope.proxy 3.4.0
<maxb> Aha.
<sidnei> maxb: luckily, gary is working on making launchpad use the eggified version of the zope.* dependencies, so it should be trivial to upgrade that after the fact. or you could submit a patch to the zope3 branch launchpad is using with that fix.
<abentley> kiko: I got conflicts because my branch was not based on db-devel, so I missed the landing.
<barry> sinzui: ping
<sinzui> hello
<kiko> abentley, it's okay, fix it up and land it for CPing
<barry> sinzui: hi.  stepping back, i have a very simple test of just the traverseName() behavior that i need.  yay for simpler unit tests.
<abentley> kiko: Okay.
<barry> sinzui: how'd you like to review the branch?
<sinzui> I would
<barry> sinzui: cool, thanks.  i'll push and mp it to you
<barry> sinzui: well, if code stops timing out on me ;)
 * barry cries
<barry> sinzui: https://code.edge.launchpad.net/~barry/lazr.restful/387487-subordinate/+merge/9263
 * sinzui waits for diff
<sinzui> run launchpad run
<barry> launchpad sez: i think i can, i think i can...
<bac> abentley: i messed up too b/c my branch was based on devel but i had to submit to db-devel.  what is the correct way to make that happen?
<BBHoss> hey is there a serious problem with the bzr server?  I've been trying for the past three days to setup launchpad for my project (locally, on my server) and it always locks up about halfway through with the message "Fetching revisions:Inserting stream ", after a day it times out.  Is this just capacity or something else?
<sinzui> BBHoss: If there was a problem, it was not that serious. several users have completed setup and are submitting patches.
<BBHoss> sinzui: hmm, well i keep trying to run it, its locked up right now, just sitting there
<sinzui> BBHoss: I think there is a network issue between you and the server. I Just pulled devel and db-devel for updates. I got changes pretty quickly. I also ran a rocketfuel-get on a machine that was two weeks stale.
<sinzui> BBHoss: The initial pull was about 40 minutes when the repo was updated to 2a
<mars> BBHoss, you installed bzr 1.16.1+?
<BBHoss> mars: Bazaar (bzr) 1.17
<BBHoss> sinzui: hmm this is in a datacenter
<BBHoss> i'll try installing my ssh keys to see if it helps
<mars> BBHoss, there are a few alternative ways to get the branch then
<mars> BBHoss, you can try 'cd ~/launchpad/lp-branches'
<mars> then "bzr branch nosmart+http://bazaar.launchpad.net/~launchpad-pqm/launchpad/db-devel"
<BBHoss> alright
<BBHoss> i'll try that after adding the ssh keys
<BBHoss> when it asks me for the username, what does it want, my email address?
<mars> Alternatively, you can grab the source tarball posted on the /Getting page; untar to launchpad/devel; cd launchpad/devel; bzr pull; cd ~; rocketfuel-setup again
<mars> it wants your Launchpad username
<BBHoss> got it
<mars> 90% sure, I haven't logged out of bzr lp in over a year :)
<sinzui> mars: how do you logout?
<mars> sinzui, you probably have to tell bzr launchpad-login to use a different username
<BBHoss> it seems to be downloading really slow
 * sinzui nods
<BBHoss> like 5kB/s
<mars> BBHoss, which transport method?
<BBHoss> most sites i download at at lest 3MB/s
<BBHoss> this is using the rocketfuel script still
<mars> bzr+ssh?
<mars> ok
<BBHoss> if it fails again i'll try the ways you suggested
<BBHoss> every once it a while it jumps to 800kB/s
<sinzui> BBHoss: I think I normally see about 200kB/s
<BBHoss> i know about the buffer limitation to ssh, i hate it
<BBHoss> but its not even getting close to that speed
#launchpad-dev 2009-07-25
<ardk> Hi, I'm trying to install launchpad on a new server (9,04). When I run "make schema" all stop. I don't have py. launchpad/lp-branches/devel/bin/py: not found.
<ardk> In the directory bin, I only have buildout. Thanks for any help
<BBHoss> mars or sinzui: could it possibly be a memory issue? http://pasternak.superalloy.nl/pastes/2487
<mars> ardk, could you please paste the output of running 'make' in ~/launchpad/lp-branches/devel ?
<sinzui> BBHoss: maybe. I think so
<ardk> utilities/shhh.py python2.4 bootstrap.py --ez_setup-source=ez_setup.py \
<ardk> 		--download-base=download-cache/dist --eggs=eggs
<ardk> utilities/shhh.py ./bin/buildout configuration:instance_name=development
<ardk> utilities/shhh.py make -C sourcecode build PYTHON=python2.4 \
<ardk> 	    PYTHON_VERSION=2.4 LPCONFIG=development
<ardk> utilities/shhh.py LPCONFIG=development /home/ardk/launchpad/lp-branches/devel/bin/py -t buildmailman.py
<ardk> make: *** [compile] Error 127
<sinzui> BBHoss: launchpad requires about 2G to develop with.
<mars> ardk, could you please use http://paste.ubuntu.com ?
<BBHoss> sinzui: i just want to play around with it, can it not just use swap?
<mars> ardk, it keeps the channel traffic down
<sinzui> BBHoss: I do not now the details of downloading and memory with the python process. But BBHoss will barely run in 1G of memory
<mars> BBHoss, I think you would have to ask a bzr guru if that is causing an error.  As for running a LP instance on 256MB of RAM
<BBHoss> hmm yeah
<BBHoss> i know it would be slow, just wanted to know what it would do
<ardk> Sorry, http://paste.ubuntu.com/231802/plain/
<sinzui> BBHoss: I did get Launchpad to run on 512 two years ago. I could run the test suite or all the services. I could browse and use bugs and answers.
<mars> sinzui, could run the test suite?  Doesn't that need a gig or so?
<sinzui> mars no. I thought I was an idiot and could not setup Launchpad
<mars> ardk, rocketfuel-setup ran to completion, without errors?
<sinzui> mars: Once I bought a new computer the next week
<ardk> Yes, no error with rocketfuel, and no error with ./utilities/launchpad-database-setup $USER
<sinzui> mars: and the one nice affect of the GIL is that on a two core machine, you can still work while the test suite is running
<mars> ardk, try "$ bin/buildout" in launchpad/devel?
<mars> sinzui, FWIW, I'll try to finish off my CHR tomorrow
<sinzui> mars: I'll do the projects...I usually do about 30 on the weekend so the person on Monday does not cry
<ardk> mars: an error whith buildout http://paste.ubuntu.com/231823/plain/
<mars> sinzui, I'll see what I can do on the projects, too
<mars> ardk, ok, try 'rocketfuel-get' then.  It looks like your devel/sourcecode directory may be broken, or possibly devel/download-cache
<mars> I should have suggested rocketfuel-get first
<mars> that is turning out to be the primary indicator of a working devel setup
<ardk> mars: I'm trying rocketfuel-get thanks
<ardk> mars: Exactly the same errors after rocketfuel-get
<mars> ardk, could you please paste the full output?
<ardk> mars: the output of rockefuel-get http://paste.ubuntu.com/231841/plain/
<BBHoss> i dont see why bzr wouldn't be able to handle the small memory, wget seems to get it fine
<mars> ardk, ok, so ~/launchpad/lp-sourcedeps/sourcecode has links, and ~/launchpad/lp-branches/devel/sourcecode has links in it?
<mars> ardk, and ~/launchpad/lp-branches/devel/download-cache is full of .tar.gz files?
<ardk> mars: yes , and in download-cache I have a subdir "dist" full of files .tar.gz
<mars> ok
<mars> that's as it should be
<mars> ardk, and there is an 'eggs/' directory?
<ardk> mars: Yes, list of eggs http://paste.ubuntu.com/231862/plain/
<mars> ardk, ok, I have 118 .egg files in mine
<mars> ardk, I have to leave now, maybe try 'python bootstrap.py', see what happens.  Otherwise, hang around here, or ask on the dev list: launchpad-dev@lists.launchpad.net
<mars> good luck!
<ardk> Thanks
<mars> ardk, oops, typo, make sure that is 'python2.4 bootstrap.py'
<mars> otherwise you'll have a mess on your hands, untangling the 2.5 files from the 2.4 ones.
<ardk> mars: Yes I've corriged, and I have a problem with lazr.smtptest 1.1
<maxb> Launchpad seems to contain both a monolithic zope tree and various zope modules - can anyone explain how that works?
<maxb> I'm trying to investigate zope.proxy misbehaviour, but it's not clear which zope.proxy is actually in use
<wgrant> maxb: It seems to be the one in the monolithic Zope.
<wgrant> Which makes little sense.
<wgrant> In fact it looks like all of the eggs are shadowed.
<sidnei> wgrant: are you referring to that zope.proxy issue? i think i missed the context of that
<wgrant> sidnei: Ah, yes, you were away while he said it. maxb is unsure as to which zope.proxy is in use in Launchpad.
<wgrant> There is a 3.5 egg, and what I presume to be a 3.4 from the monolithic tarball.
<wgrant> But the monolithic zope 3.4 is in lib/, which is always earlier in sys.path AFAICT.
<sidnei> wgrant: that's correct. we should use the 3.5 egg as soon as gary merges his branch
<wgrant> sidnei: So those eggs are just sitting there doing nothing at the moment?
<sidnei> wgrant: yes, at the moment. i also suggested patching the zope 3.4 from lib with the fix from the 3.5 egg
<sidnei> in the meantime that is
<sidnei> wgrant: i believe this is the branch gary is working on: https://code.edge.launchpad.net/~gary/launchpad/zbuildout
<sidnei> wgrant: it's only pending some test failures afaict to be merged
 * wgrant mutters something about private branches.
<sidnei> wgrant: oh, drats. let me see if i can toggle that bit
<wgrant> sidnei: You probably can't.
<sidnei> i wonder if bzr does remote branching
<wgrant> It's unlikely a non-admin can now that the security policy is relaxed.
 * sidnei watches as something happens with https://code.edge.launchpad.net/~sidnei/launchpad/zbuildout
<wgrant> sidnei: Aha, thanks.
<sidnei> still pushing, but seems like it might be done soon
<jml> sinzui, actually, I added the rule to link lp:12345 to a bug as well. there ought to be no conflict due to [0-9] and [a-z] having no intersection.
 * wgrant discovers why /var/tmp/fatsam is called that.
<maxb> Well, the interesting thing is that even applying the zope.proxy fix from upstream, I still seem to experience the bug
<wgrant> maxb: Is the bug causing any issues?
<maxb> It's causing MemoryErrors all over the place for me
<wgrant> Oh.
<wgrant> Of course.
<wgrant> Stupid Python.
<wgrant> That's why Storm doesn't define it in the first place.
<maxb> Well, really I can't blame it for having difficulty instantiating a 4 billion element list
<wgrant> Perhaps.
<maxb> The annoying thing is that if I manually instantiate a zope.proxy.ProxyBase(object()), it seems happy to return correct results
<wgrant> From 'make harness'?
<maxb> I've not come across that one, I'm just working with "make run", or a standalone script in the style of utilities/make-lp-user
<wgrant> Ah.
<wgrant> make harness will get you an interactive Python with Zope configured and lots of useful objects.
<wgrant> I believe there's also a make iharness, which is probably even more useful.
<maxb> The other somewhat perplexing thing is that I'm now managing to get a zope.security.interfaces.ForbiddenAttribute: ('__len__', <storm.store.ResultSet object at 0x611e090>)
<wgrant> That's good.
<wgrant> That's what the proxy is there for.
<maxb> when the only change from code that didn't do that was to delete some operations frm my minimal test script
<wgrant> Hmm, actually.
<maxb> Get a storm resultset, print len(rs) -> 2**32-1, transaction.commit() results in ForbiddenAttribute.
<maxb> However, add "for i in rs: pass" before the commit, and no ForbiddenAttribute happens
<wgrant> Special...
<maxb> hehe, yes, "special"
 * wgrant looks with bemusement at build__datecreated__key
<wgrant> That makes no sense.
<cody-somerville> wgrant, Are you pleased to finally have access to lp source code? :)
<wgrant> cody-somerville: I am!
<maxb> aha, the same 64bitness bug exists in zope.security.proxy as well as zope.proxy
<cody-somerville> wgrant, I was told that I'm the only person outside of the launchpad team to contribute code to launchpad in a couple of years. Hopefully that'll change real quick now! :)
<wgrant> cody-somerville: I've already two approved branches that will land once devel reopens on Monday.
<cody-somerville> wgrant, awesomeness
<cody-somerville> wgrant, I'm sure you'll help pave the way for all sorts of folks to be able to contribute effectively
 * wgrant hates gina's config system.
<cody-somerville> zconfig or... ?
<wgrant> lazr.config. But it's not a good fit.
<cody-somerville> Yea
<cody-somerville> I examined lazr.config for one of my internal projects and I decided to go with a different solution
<wgrant> lazr.config is good. Just not for this.
<maxb> Hmm, no vcs-import for zope.security
<wgrant> It would have been part of the main Zope 3 import until recently.
<maxb> OK, woo, now I actually have a functioning webapp running on karmic :-)
<wgrant> Which Python?
<maxb> 2.5
<wgrant> Aha.
<wgrant> Using all the default deps except zope.security?
<maxb> I just grabbed and applied patches of the relevant revisions from zope.security and zope.proxy to the sourcecode/zope tree
<wgrant> Is this the first time you've had the webapp running?
<maxb> No, but before now it tended to OOPS a lot
<wgrant> I count that as a yes.
<maxb> Even an OOPSing webapp was proof I was on the right track :-)
<wgrant> It's nice running a local Launchpad - it's so fast!
<maxb> On my good laptop, yes
<maxb> Not so much on my netbook!
<wgrant> Probably not.
<wgrant> maxb: Found anything yet that doesn't like 2.5?
 * wgrant needs an SSD
<maxb> I suppose I should now figure out how to run the testsuite
<wgrant> maxb: make check
<wgrant> Making sure you have good cooling.
<wgrant> And many, many hours.
<wgrant> I should run the full test suite here at some point - I've so far only run the bugs tests in full, which took 50 minutes.
<maxb> hmm. possibly a laptop sitting on a bed is not the best environment for long duration high CPU work
<wgrant> A lot of it's high-disk, as the database has to be replaced after each test.
<wgrant> I'm not sure if I was reading things properly, but the output of the lp.bugs tests suggested that 30 of the 50 minutes were spent doing DB setup.
<maxb> Does anyone know when the dev wiki is going to be opened for editing? I'd like to scribble some notes on my experiences of getting LP to work with Python 2.5
<cprov> good morning, folks
<wgrant> Morning cprov.
<sidnei> hey wgrant, pushing that branch finished at some point during the night: https://code.edge.launchpad.net/~sidnei/launchpad/zbuildout
<maxb> sidnei: am I simply not driving bazaar properly, or is there litte interesting in that branch yet?
<sidnei> maxb: what do you mean?
<maxb> Unless I'm failinf to use bzr properly, it seems to contain a group of changes not related to the buildsystem, and a one-line fixup
<maxb> bzr log -n0 -r ancestor:../devel..
<sidnei> maxb: that command doesn't show much indeed. use diff instead of log, you will see that there's a lot of changes
<sidnei> maxb: it looks quite unpolished though, there's quite some code that has been commented out, presumably because it doesn't pass the tests yet
<sidnei> maxb: i think using a patched sourcecode/zope3 is a better approach until gary makes more progress
<timut> hi all
<timut> i'm installing launchpad on hardy 8.04, anybody have a list of dependency or lib packages prerequisite before install?
<maxb> are you following https://dev.launchpad.net/Getting ?
<maxb> dependencies are dealt with as part of rocketfuel-setup
<timut> maxb: yups, but i have problem with bzr and python-pyrex
<timut> Setting up python-pyrex (0.9.6.4-1ubuntu1) ...
<timut> pycentral: pycentral pkginstall: not overwriting local files
<timut> pycentral pkginstall: not overwriting local files
<timut> dpkg: error processing python-pyrex (--configure):
<timut>  subprocess post-installation script returned error exit status 1
<timut> Errors were encountered while processing:
<timut>  python-pyrex
<timut> E: Sub-process /usr/bin/dpkg returned an error code (1)
<maxb> timut: I would suggest temporarily moving aside anything including pyrex in its name from the /usr/lib/python2.4/site-packages/ directory and retrying the package setup with dpkg --configure -a
#launchpad-dev 2009-07-26
<timut> maxb: thanks
<timut> it's helping me
<wgrant> How do I get something bin/harness but with security working? I want, for example, to be able to switch users interactively and see how access to objects changes.
<BjornT_> wgrant: use getUtility(IFooSet) to get the initial object. when you use getUtility the object will be security wrapped, and each object you access through it will also have a security proxy.
<wgrant> BjornT_: I know hot to get the objects, but I can't seem to switch to a user that doesn't have any permissions.
<wgrant> BjornT_: canonical.launchpad.ftests.login doesn't seem to work.
<wgrant> Well, it runs, but my mortal user still has launchpad.Admin on everything.
<BjornT_> wgrant: oh. it probably runs with the permissive security policy...
<wgrant> BjornT_: Right. Any idea how I can change that?
<wgrant> It does look like c.l.ftests.login is configuring the participation properly, so that must be it.
<wgrant> Aha.
 * wgrant tries something.
<BjornT_> wgrant: the following might work: setSecurityPolicy(LaunchpadSecurityPolicy)
<wgrant> Yep.
<wgrant> A grep showed me that.
<wgrant> Thanks.
<wgrant> (needed to import setSecurityPolicy from zope.security.management)
<AdamDV> <3 Launchpad
<AdamDV> Anyone here?
<AdamDV> I just got an error :/
<AdamDV> bzr: ERROR: no such option: --2a
<AdamDV> ERROR: Unable to set up local LP repository
<andrea-bs> AdamDV, you need bzr version 1.9 or higher
<AdamDV> I just installed 1.17..
<AdamDV> This morning, because on my first rocketfuel attempt I got the 'bzr version not high enough' error
<AdamDV> The script has been runing successfully for like 30 minutes
<AdamDV> andrea-bs: should I use the nightly build instead?
<andrea-bs> AdamDV, no, you shouldn't: 1.17 is the right version (I was wrong about 1.9)
<andrea-bs> AdamDV, how did you install bzr 1.17?
<AdamDV> I downloaded it from the ppa
<AdamDV> cd Downloads
<AdamDV> sudo dpkg -i bzr*
<andrea-bs> what does ``bzr version`` tell you?
<AdamDV> HOwever, when I opened the deb in GDebi, the Install betton was greyed out...
<AdamDV> It says 1.13 :/
<AdamDV> I'm installing again, andrea-bs
<andrea-bs> maybe dpkg has encountered some errors while installing
<AdamDV> Now it says version 1.17
<andrea-bs> it should work now
<AdamDV> Is i possible that rocketfuel-setup would at some point downgrade bzr?
<andrea-bs> no, I don't think so :)
<AdamDV> Well now I see
<AdamDV> 'Making local branch of launchpad truck'
<andrea-bs> however, I strongly suggest you to add the Bazaar PPA to your sources.list, so you'll get the updates
<AdamDV> Alright
<AdamDV> Does the install just carry on if postfix and everything have already been instaled?
<AdamDV> So that I wouldn't have to wait the 30 min again?
<AdamDV> (I'd assume so)
<andrea-bs> all the required deb packages are not re-installed if they are already present
<AdamDV> Ah, ok.
<AdamDV> Now I get this:
<AdamDV> make: *** No rule to make target `install'.  Stop.
<AdamDV> ERROR: Unable to install apache config appropriately
<AdamDV> Anyone here?
<AdamDV> I keep getting this:
<AdamDV> bzr: ERROR: Connection error: Couldn't resolve host 'bazaar.launchpad.net' [Errno -2] Name or service not known
<AdamDV> ERROR: Unable to create local copy of Rocketfuel trunk
<elmo> AdamDV: does 'ping -c 1 bazaar.launchpad.net' work from the command line?
<AdamDV> yes
<AdamDV> It works for awhile, maybe 20 minutes, then I get that.
<AdamDV> elmo: does launchpad.tar.gz have the lp-branches folder?
<elmo> AdamDV: can you install the dnstracer package, and pastebin the output of 'dnstracer -c bazaar.launchpad.net'?
<AdamDV> sure
<elmo> AdamDV: I've no idea offhand, sorry - it's a full copy of the LP tree is all I know
<AdamDV> I can't install dnstracer.
<AdamDV> It requires bzr 1.14 <
<AdamDV> And I have 1.17, which is requireed by LP.
<elmo> heh, dnstracer doesn't, but something else you've got installed does
<AdamDV> I haven't installed anything, I think the rocketfuel-setup script asks for bzr 1.17 for somestuff and 1.14 for others.
<AdamDV> I've run into a problem like that twice with rocketfuel.
<AdamDV> anyway, going for a coffee, I'll mess with it when I get back.
<AdamDV> Aright, I just downloaded launchpad.tar.gz from herbs repo.
<AdamDV> Its got a bit of a different name scheme, but I hink It'll work.
<AdamDV> *think
<AdamDV> Alrighty
 * AdamDV hopes that this time it will work
<mars> mwhudson, around yet?
<mwhudson> mars: yep
<mars> hi mwhudson, and good morning!
<mars> I have a question about a mail that just arrived on the feedback list
 * mwhudson looks
<mwhudson> mars: the freebie trial one?
<mars> mwhudson, "my email was published"
<mwhudson> oh sigh
<mwhudson> why can't people just give up on the 'hide my email' approach to spam prevention?
<mars> mwhudson, would you be able to write a reply?  I'm not sure if there is a real bug there or not.
<mars> and I don't know our policy in that regard, either
<mwhudson> there's no bug
<mwhudson> i'll write a reply
* mwhudson changed the topic of #launchpad-dev to: Launchpad Development Channel | Week 0 | https://dev.launchpad.net/ |  Please use #launchpad for support. | https://launchpad.net/~launchpad-dev | Get it: https://dev.launchpad.net/Getting | http://people.canonical.com/~herb/ | http://paste.ubuntu.com/
<thumper> morning
<thumper> mars: I was just looking at that too
<mars> mwhudson, many thanks
<thumper> mwhudson: call?
<mwhudson> thumper: yeah, was just thinking that
<mwhudson> thumper: couple sec
 * mwhudson stabs synergy
<mwhudson> my two laptops can't ping each other??
<thumper> :(
<mwhudson> oh actually, my router had shuffled ip addresses around
<mwhudson> thumper: so, yes, call?
<thumper> yep
<mwhudson> skype thinks you're offline of course
<thumper> spm: buildbot.devpad.info
<thumper> spm: is not responding...
<wgrant> Not resolving.
<spm> which was going to be my 1st problem....
<wgrant> Is there any reason it can't be public?
<spm> thumper: I 'spect we're going to have to wait for Gary on that
<spm> dig +trace suggests the ns servers are fine, but something is nicely borked with the domain itself
<spm> whee. no SOA record even. totally borked.
<thumper> spm: who is responsible for merging db-stable back into devel?
<thumper> spm: and db-devel's last revision isn't in db-stable :(
<spm> whee
<mwhudson> devpad.info is joey's domain isn't it?
<mwhudson> spm: happy new week!
<spm> mwhudson: so I've been updated to believe, yes.
<spm> i wonder what's up with that domain - it doesn't appear to expire for another 6+ weeks...
<thumper> spm: are we re-rolling production?
<mwhudson> spm: do you know why the re-rollout didn't happen ?
<mwhudson> (it's almost like thumper and i are talking to each other)
<spm> !!! I've been here 20 minutes. How much email do you two think I've read? :-)
<ubot3> spm: Error: I am only a bot, please don't think I'm intelligent :)
 * spm brb - updates to install
<spm> back...
<thumper> https://lp-oops.canonical.com/oops.py/?oopsid=1302EB113
<mwhudson> !irc
<ubot3> A list of official Ubuntu IRC channels, as well as IRC clients for Ubuntu, can be found at https://help.ubuntu.com/community/InternetRelayChat - For a general list of !freenode channels, see http://freenode.net/faq.shtml#channellist - See also !Guidelines
#launchpad-dev 2010-07-26
<lifeless> spm: hi
<lifeless> spm: I'm sure you have a bunch of firefighting to do [ubunet cron, *cough*]
<lifeless> spm: but I wonder, as its 1am here, if you can do me a solid first
<spm> lifeless: yo; sup?
<lifeless> spm: which is, to deploy lp:~lifeless/launchpad/malone so I can see if it does make dup checking faster
<lifeless> on staging
<spm> oh sure, app server I assume?
<lifeless> yeah
<spm> yoikes that's a big patch. it seemed to have a merge fail; but it scrolled out of my buffer. trying again to a file
<lifeless> spm: it should be tiny
<lifeless> spm: https://code.edge.launchpad.net/~lifeless/launchpad/malone/+merge/30904,
<spm> bzr merge --preview  lp:~lifeless/launchpad/malone ?
<spm> just take the latest version then of it?
<lifeless> spm: https://code.edge.launchpad.net/~lifeless/launchpad/malone/+merge/30904/+preview-diff/+files/preview.diff perhaps
<spm> that works
<lifeless> spm: its probably other devel stuff naffing staging up, or something
<spm> nod
<spm> damn. 4/4 hunks failed
<lifeless> which ones
<lifeless> oh, all
<spm> :-)
<lifeless> erm
<spm> 1am, you said earlier. noted. :-)
<lifeless> on which files
<lifeless> there is much more than 4 hunks in there
<spm> http://paste.ubuntu.com/469077/
<spm> yeah - that was just the most obvious
<lifeless> spm: can you revert; then pastebin bzr missing lp:~lifeless/launchpad/malone ?
<spm> preview - i always preview/dry-run first
<spm> sure sure
<spm> lifeless: http://paste.ubuntu.com/469078/
<lifeless> odd
<lifeless> 11189 should be included
<lifeless> can you grep slow_ lib/canonical/database/nl_search.py ?
<spm> 0 results found
<lifeless> ok
<lifeless> merge -c 11199 lp:~lifeless/launchpad/malone ;
<lifeless> merge --force -c -1 lp:~lifeless/launchpad/malone
<lifeless> should behave better
<spm> oki
<lifeless> its in place ?
<spm> is that bzr, svn or git merge?
<spm> not yet
<lifeless> bzr
<spm> 1am. trolling is too easy. sigh.
<lifeless> 121
<spm> qed
<spm> 1st went in easy
<lifeless> vidi vici veni
<spm> on a preview; live doing now...
<spm> Civil Engingeering Students Association (@ Qld Uni) CESA: We Came. We Swore. We Concreted.
<spm> coolio, both applied; restarting
<spm> lifeless: restarted with new shiny ponies
<lifeless> hmm, you sure ?
<spm> yup :-)
<spm> this is the app server right? not code*?
<spm> start time: Mon Jul 26 00:24:52 2010 (BST natch)
<lifeless> booyah
<lifeless> wgrant: try this page https://bugs.staging.launchpad.net/ubuntu/+filebug
<lifeless> with e.g. eclipse get unmet dependency error
<lifeless> its hitting disk a lot still
<lifeless> but once off disk its hell snappy for me
<lifeless> mwhudson: ^
<lifeless> spm: what do you think
<wgrant> Hm, not too bad.
<spm> whimper. can't login to staging I think...
<wgrant> But the results... I don't know.
<spm> No I lie. I'm in. wooo!
<lifeless> wgrant: they are missing the prefilter step that was doing millions of comparisons to filter out terms; will reinstate that in a bit
<lifeless> wgrant: other than that its ranked precisely the same way
<lifeless> [terribly]
<spm> seemed snappy to me
<lifeless> k
<lifeless> bombs away, I say
<wgrant> lifeless: Isn't it returning a subset of the original results, though?
<lifeless> yes
<lifeless> but ranking is strictly additive given the same terms as the rank function
<lifeless> so 3 terms >> 1 term anyhow
<lifeless> and we're not [yet] using heat or anything
<lifeless> night
<wgrant> Right.
<wgrant> I guess you try the one-missing query, and then jump to two-missing, three-missing etc. queries if there aren't enough results.
<wgrant> s/you/you could/
<wgrant> spm: Some more corrupt builds appear to have sprung up :(
<wgrant> If you have a moment, could you run 'SELECT buildqueue.id, builder, lastscore, buildqueue.job, job_type, processor, build FROM buildqueue JOIN job ON job.id = buildqueue.job JOIN buildpackagejob ON buildpackagejob.job = job.id WHERE buildqueue.virtualized = false AND buildqueue.processor = 1 AND job.status = 0' for me?
<wgrant> There are somehow 13 corrupt builds in the i386 non-virt queue, for a total of 40 seconds of buildd time, and they don't appear to actually be in the queue.
<spm> *blink*. 0 rows?
<wgrant> Bah.
<wgrant> SELECT buildqueue.id, builder, lastscore, buildqueue.job, job_type, processor FROM buildqueue WHERE buildqueue.virtualized = false AND buildqueue.processor = 1;
<wgrant> They must be even more broken than the last lot.
<spm> we don't break by halves round here
<spm>    id    | builder | lastscore |   job   | job_type | processor
<spm> ---------+---------+-----------+---------+----------+-----------
<spm>  3233211 |         |     12510 | 1941857 |        1 |         1
<spm> (1 row)
<spm> wgrant: ^^ that last score seems... curious
<wgrant> Just means it's in a private archive.
<spm> is that like the 'score' we'd override a build? ahh kk.
<wgrant> So there's something odd going on here, and that's not it.
<wgrant> (https://edge.launchpad.net/builders -- see the i386 non-virt build queue. that query should have returned everything in it. :()
<spm> looks
<wgrant> Hmm. What if you say 'virtualised IS NULL' instead of 'virtualized = false'?
<spm> non-virt the offical builders yes/no?
<wgrant> Official, yes.
<spm> non-virt *is* the offical builders yes/no? yes, ta
<spm> 13 rows :-)
<spm> jobtype=4, processor=1, lastscore=1000
<spm> so curious that the = false gets something at all.
<spm> recognising that sql has 3 binary states - on, off, null :-/
<wgrant> It is somewhat odd, since it's not appearing in the queue, and is not building. But that's not the main issue.
<wgrant> Thanks for your assistance.
<spm> np
<wgrant> Odd that they're not being dispatched. But they're translations jobs, so who knows what evil they are up to.
<spm> I know people who would know what evil they're up to.....
<wgrant> There are another 18 properly virtual jobs floating around inappropriately, but it's just about impossible to track them down until the queue is empty.
<wgrant> Filed bug #609904, but that doesn't explain why the builds exist in the first place.
<_mup_> Bug #609904: BuilderSet.getBuildQueueSizes needs to consider virtualized=NULL as false <Soyuz:New> <https://launchpad.net/bugs/609904>
<wgrant> jtv: Hi. There are 13 pending translation template build farm jobs, and they're not being dispatched. Do you know why that would be?
<jtv> wgrant: hi.  No idea...  it's only TTBJs?
<wgrant> jtv: There are some other non-dispatching builds around. But they're corrupt -- I'm not sure if the TTBJs are.
<jtv> wgrant: corrupt in what way?
<wgrant> jtv: The status of the PackageBuild is wrong.
<wgrant> But you only have a single status, so that's not the case.
<jtv> wgrant: it's not one of the recent changes and us not having a Build?
<jtv> Maybe the bad build status is a result of the same thing that's causing these jobs not to be dispatched?
<wgrant> jtv: Unlikely. I saw some TTBJs building not even 24 hours ago, so it's not a general problem.
<wgrant> I wonder if the branchjob is missing or something.
<jtv> A branchjob and a job are all we have, no?
<wgrant> And a buildqueue.
<jtv> Ah right
<jtv> that too
<wgrant> The buildqueue and job are present.
<wgrant> Not sure about the branchjob.
<wgrant> I don't trust the recent branch deletion changes, so I'll blame them for now :)
<jtv> Deleting a branch might cause this...
<jtv> The recent branch changes would cause both the BJ and the BQ to be cleaned up.
<jtv> How old are these broken jobs?
<jtv> The old behaviour (still running on production afaik) would cause an incomplete cleanup.
<wgrant> SELECT buildqueue.id, builder, lastscore, buildqueue.job, buildqueue.job_type, processor, job.date_created, job.status, branch, branchjob.job_type, json_data FROM buildqueue JOIN job ON job.id = buildqueue.job LEFT JOIN branchjob ON branchjob.job = job.id WHERE buildqueue.virtualized IS NULL AND buildqueue.processor = 1;
<wgrant> spm: ^^
<spm> yup
<jtv> wgrant: no rows
<spm> hrm that json data for those is interesting; all over the place.
<wgrant> Those two statements conflict....
<jtv> I ran mine on staging.
<wgrant> Ah.
<wgrant> staging has buildqueue cleared.
<wgrant> Because it has a buildfarm now.
<spm> jtv: https://pastebin.canonical.com/35029/ I think I can post that publicly, but can you confirm or deny?
<wgrant> There are no private TTBJs yet, so it /should/ be safe... but...
<spm> yeah, juts wanted to be doubly sure
<spm> the sus ones are all public
<spm> wgrant: http://paste.ubuntu.com/469098/
<wgrant> Hmm.
<wgrant> So they all have BranchJobs, are not deleted, and all very fresh.
<wgrant> I don't know what's going on, unless it's trying to dispatch them but failing.
<jtv> And maybe re-dispatching them?
<jtv> wgrant, spm: it's suspicious that these are all so recent...  might it be a dispatch priority issue?
<spm> jtv: recent as in the last 24ish hours?
<jtv> spm: yes
<jtv> Did we CP anything that could have done this?
<spm> jtv: and btw, good morning! you're up early? :-)
<jtv> spm: amsterdam
<spm> 'up early' still rings true :-)
<jtv> up late.  :)
<spm> heh
<jtv> met up with team members & an ex team member today... a good portion of the Prague attendance went to Holland next it seems.
<spm> jtv: no CP's to the build side since the 9th, so unlikely?
<spm> ahh cool!
<jtv> Some for holidays, others for the GNU meeting, yet more for Guadec.
<jtv> No CPs...  then either it's operational or it's something weird on edge.
<jtv> The branches I checked out weren't deleted, so I doubt it's related to branch deletion.
<wgrant> jtv: Yeah, it's a bit odd. I guess someone needs to check buildd-manager logs at some point :/
<spm> edge? edge gets updated regularly - so CP?
<jtv> :(
<spm> try again; so CP isn't as unlikely an option- it's just the normal state of affairs.
<wgrant> Hmm. I guess it's possible it's priority.
<jtv> spm: yes, it's just that edge is less likely to have an effect on the build farm
<wgrant> But that would imply that the queue hasn't been empty for 24 hours.
<jtv> ...and yet we have builders sitting idle.
<wgrant> I guess I'll just hope they disappear over the day.
<jtv> Maybe something's bottlenecking on uploads?
<jtv> wgrant: is the builder marked as idle while its uploads are being processed?
<wgrant> jtv: No.
<jtv> it'd be nice to know what dispatch queries are going out right now...
<wgrant> It's normal at the moment for lots of builders to be idle, because buildd-manager sucks.
<wgrant> but... buildd-manager is far worse than normal.
<wgrant> It's redispatching the same set over and over.
<wgrant> Every minute or so.
<wgrant> lamont alluded to this yesterday, saying there was a rogue build causing it to abort most cycles.
<wgrant> I'll talk to Julian tonight, I suppose.
<wgrant> (if you click on a few of the current virtual builds, you'll see they started <30s ago.
<wgrant> Then if you refresh in another 30 seconds, they'll still be <30s ago.
<wgrant> Anyway, uni.
<wgrant> Probably not much we can do now.
<jtv> If the builds keep getting re-dispatched, that's a familiar symptom of something...
<jtv> ...slaves breaking protocol?
<jtv> as you say, uni.
<rockstar> build farm busy
<jtv> rockstar: would a busy build farm do that?  That'd be a pretty major and obvious bug.
<jtv> (IIRC a job gets marked as in-progress _and committed_ before actual processing starts)
<rockstar> jtv, well, that was supposed to go to another channel, but yes, it seems the build farm is busy right now.
<jtv> rockstar: from what wgrant has been finding out, it appears that some jobs are being eternally re-dispatched
<rockstar> jtv, oh wait, you thought I was following your context.
<jtv> Yes, I'm weird that way
<jtv> :-)
<rockstar> jtv, no, I haven't been reading the backchat.
<thumper> rockstar: hey
<thumper> rockstar: you home again?
<thumper> rockstar: and back to normal?
<thumper> jtv... weird... surely not
<jtv> preposterous
<rockstar> thumper, well, "normal" is objective, but yeah, I'm home.
<thumper> rockstar: objective or subjective?
<thumper> pedantry rules
<jtv> thumper: he means that he's normally objective.
<spm> heya rockstar
<wgrant> buildd-manager is still screwed :(
<wgrant> spm: No obvious errors in the log?
<spm> orsum. looking.
<wgrant> It should try to dispatch lots of builds, then encounter some error. Every <1min, probably.
<spm> exceptions.AssertionError: amd64 build of eclipse 3.4.1-0ubuntu2 in ubuntu karmic RELEASE (1213758) can not be built for pocket RELEASE: invalid pocket due to the series status of karmic. ??
<spm> not every minute, but surely not far off.
<wgrant> That would do it.
<wgrant> That's... interesting.
<spm> it's a rather impressive stack trace too
<spm> been going for ages too
<wgrant> Bug #575165
<_mup_> Bug #575165: Buildd-manager erroneously checks COPY archives for release pocket upload permissions when dispatching <buildd-manager> <soyuz-upload> <trivial> <Soyuz:Triaged by julian-edwards> <https://launchpad.net/bugs/575165>
<wgrant> But that archive is disabled.
<wgrant> So the build shouldn't be active :/
<thumper> :(
<wgrant> I guess we could just manually suspend it.
<wgrant> Why it is not already suspended I do not know.
<wgrant> thumper: Is the porting of SPRBs from BuildBase to PackageBuild going to happen this cycle?
<thumper> wgrant: I believe so
<wgrant> Excellent.
<thumper> wgrant: abentley has it mostly done I think
<thumper> he was stuck though on where the builds get set to fully built
<thumper> at least that is where it was on my Friday
<thumper> he may have solved that now
<wgrant> Yeah, there's the wonderfully-named 'findBuild' method which actually sets it to fully built...
<thumper> WTF?
<thumper> that does not sound like a mutating function
<wgrant> No.
<thumper> fix plz?
<wgrant> Heh. Maybe once the build farm code has settled down and we've removed the four excess tables and several additional classes...
<mwhudson> wow, recipe builds really don't have much priority do they?
<wgrant> What is their priority?
<wgrant> I forget.
<wgrant> Ah, 900.
<wgrant> Yeah, normal PPA builds are 2510 or so.
<wgrant> I need to convince everyone that build scores are stupid and need to die.
<mwhudson> i'm just experiencing the "been expected to start in 1 hour for several hours" thing
<wgrant> Well, that's not due to the score.
<wgrant> That's due to everything being fucked.
<mwhudson> i see
<wgrant> The ETA takes the score into account.
<thumper> wgrant: you have a +1 from me to kill scores
<thumper> but I don't hold much weight there
<thumper> what we need is a way to prioritise some builds
<thumper> and scores are an approximation for that
<spm> if team==losa then do-losa-builds-first elseif team==ubuntu-security then do security-builds else hahahaha fi ?
<thumper> spm: something like that
<wgrant> thumper: A pretty ineffective approximation.
<thumper> mwhudson: I'm deleting most of doc/branch-merge-proposals.txt
<thumper> mwhudson: I'm rewriting it to not use sample data
<thumper> mwhudson: and in my rewrite I realised it is a big pile of turd
<wgrant> There seems to be a bit of that going around lately.
<thumper> sample data should be avoided in all tests
<thumper> whereever possible
<thumper> which is almost everywhere
<wgrant> Yeah.
<mwhudson> thumper: hooray?
<thumper> mwhudson: I'm actually writing documentation
<thumper> the level of executability in the doc is very small
<thumper> I'm trying to focus on the documentation being meaningful
<spm> thumper: !?!? bit excessive isn't it? *meaningful*!?!?! srsly?
<thumper> yeah
<thumper> what was I thinking
<spm> haha
 * thumper replaces the doctest with "Read the freaking code bitch"
<spm> damn. that's almost too easy to take out of context. I shall pass. this time.
<thumper> spm: heh
<thumper> lifeless: hi, back in NZ?
<lifeless> nope
<thumper> lifeless: EU still??
<lifeless> I fly out from here in 12 hours
<lifeless> GNU Hackers Meeting was on
<thumper> how'd that go?
<lifeless> pretty good
<lifeless> I need to write it up but <so tired>
<lifeless> thumper: did you try the staging search perf ?
<thumper> no
<thumper> but I read your writeup
<thumper> and commented on the merge proposal
<lifeless> yeah
<lifeless> its because of conflation
<lifeless> thumper: https://bugs.staging.launchpad.net/ubuntu/+filebug
<lifeless> sadly it failed an obscure teset
<lifeless> so I need to update some more tests with the same 'it sucks, sorry, fast-first' dialogue.
 * thumper nods
<lifeless> thumper: I'm interested in perf for you, on that url.
<lifeless> 'folk in nz are like folk in china'
<thumper> what do you want me to search on?
<lifeless> whatever you like
<thumper> lifeless: "bzr transfers too much data" gave me a response in about 5 seconds, but none of them really relevant
<lifeless> thumper: item 0 for me is bug 82305
<_mup_> Bug #82305: push and pull on bound branches use too much network <Bazaar:Confirmed> <bzr (Ubuntu):Confirmed> <https://launchpad.net/bugs/82305>
<lifeless> which is pretty relevant
<thumper> well... yes
<lifeless> remember that this is in-ubuntu rather than in-bzr context - its not searching the bzr bug db
<thumper> I suppose
<lifeless> https://bugs.edge.launchpad.net/ubuntu/+filebug
<lifeless> same search
<lifeless> doesn't return any bzr bugs
 * thumper has to go collect people
<lifeless> ciao
<lifeless> and well production simply won't answer
<adeuring> good morning
<lifeless> stub: hi
<stub> Happy Asaha Bucha Day.
<lifeless> oh nice
<lifeless> you're on leave for it ?
<stub> Yer. Just ticking off a few things from my todo list first though.
<lifeless> cool
<lifeless> well I'm travelling tonight
<lifeless> I've done an audit of the timeout bugs in malone
<lifeless> and there are a few which would really benefit from you casting an eye over some query<->index stuff
<lifeless> if you had time tomorrow to just have an eyeball - just the malone stuff tagged 'timeout'
<stub> Ok.
<lifeless> theres things like a union plan which takes 11 minutes to run !
<mrevell> Morning
 * flacoste is away: Gone away for now
 * thumper takes a look at the merge conflict
<jml> thumper, thanks
<jml> thumper, I was about to, since I last touched branch.txt
<thumper> jml: it seems to be that we just need to delete that block again
<thumper> sound right?
<jml> thumper, haven't got that far yet
<jml> thumper, still pulling db-devel :)
<thumper> jml: but in the branch that you merged into devel you just deleted chunks of branch.txt?
<jml> thumper, pretty much.
<jml> (note that it doesn't use branch sampledata any more)
<jml> thumper, yeah, delete it.
<jml> thumper, all of that has been moved to unit tests
<thumper> jml: the difference on db-devel is that someone reformatted the lines to the right length
 * thumper deletes
<jml> thumper, thanks.
<jml> thumper, btw, I'm still progressively working through the code tests getting rid of their dependency on branch sampledata
<thumper> jml: you should see my merge proposal that is up
<jml> thumper, I did!
<thumper> I tried to focus on useful documentation
<thumper> rather than a test
<jml> thumper, cool.
<jml> thumper, I'll review that in a moment.
 * thumper leaves the office to drink a cuppa
<lifeless> adeuring: what does having sec proxies returned from the lpfactory do for us ?
<adeuring> lifeless: I think the tests should work with the same objects as we use in real code.
<adeuring> and we should be able to detect for example permission problems
<lifeless> adeuring: thanks
<thumper> jml: yes it is still running, just not under a special layer
<jml> thumper, ahh, thanks.
<thumper> gah, my email client is getting my emails :(
<bigjools> jml: Hope you're feeling better this week because I added you to the MP for the buildd-manager branch last week, it might make you feel sick again.
<jml> bigjools, thanks :)
<bigjools> jml: I do feel bad and apologise profusely.  However, not *too* bad since I had to suffer when changing the damn thing :)
<bigjools> some of the tests are bloody awful, so I hope you have suggestions on how to make them look better.
<lifeless> bigjools: can you describe the awfulness ?
<lifeless> bigjools: like - overly coupled, too sensitive, not sensitive enough, visually long, hidden setup, ?
<bigjools> lifeless: hideous setup
<bigjools> see lib/lp/buildmaster/tests/test_manager.py, look for TestSlaveScanner.setUp()
<bigjools> it stubs out half the code its testing
<bigjools> and then selectively re-enables it in some tests!
<bigjools> and it probably falls foul of all those other things you listed
<lifeless> so, if different tests need different amounts stubbed
<lifeless> I think I'd tackle that by:
<lifeless> moving the setup to a helper function
<lifeless> delete the setUp() on the test class
<lifeless> at the top of each test call the helper with parameters appropriate to the test being run
<bigjools> the tests need re-writing, they're rubbish
<lifeless> it will still be hideous, but less so.
<lifeless> and it will be a cheap change to make
<bigjools> sort of - I'd need to work out exactly what each test is depending on
<bigjools> I would not be surprised if it's masking bugs
<wgrant> bigjools: Evening.
<wgrant> Are you aware of the rogue karmic rebuild which is killing everything?
<wgrant> (or was over the weekend; not sure if it's been fixed in the last few hours)
<bigjools> wgrant: sigh, thought it had disappeared
<bigjools> how the *hell* is it even there
<wgrant> bigjools: Yeah, it should surely have been suspended...
<bigjools> I wonder if someone pocket-copied it
<wgrant> bigjools: Within a copy archive?
<wgrant> Unlikely.
<lifeless> -> stuff
<bigjools> how do you know it's a copy archive build?
<wgrant> I thought it was build 1213758
<bigjools> how do you know that?
<bigjools> I can see it in the logs, but...you?
<wgrant> spm checked the logs.
<bigjools> ah
<wgrant> I'm not that good, sadly.
<bigjools> this is a test archive from 20090909
<bigjools> I know the bug for this
<wgrant> Yep.
<bigjools> but the archive is disabled...
<bigjools> wtf
<wgrant> Bug #575165
<wgrant> Right.
<_mup_> Bug #575165: Buildd-manager erroneously checks COPY archives for release pocket upload permissions when dispatching <buildd-manager> <soyuz-upload> <trivial> <Soyuz:Triaged by julian-edwards> <https://launchpad.net/bugs/575165>
<wgrant> But was it disabled before build suspension was implemented?
<wgrant> Probably.
<bigjools> grar
<bigjools> why has it appeared now though?
<wgrant> I don't quite know.
<wgrant> Maybe it depwaited.
<wgrant> So wasn't suspended.
<wgrant> And then somehow got retried.
<bigjools> ok, I'm going to re-purge disabled copy archive builds
<bigjools> FWIW http://pastebin.ubuntu.com/469264/
<wgrant> 'Purge' meaning suspend, or really delete?
<wgrant> bigjools: Any progress with PPA log parsing?
<bigjools> supersede
<bigjools> no - but good reminder :)
<wgrant> Heh. That query looks good, anyway.
<bigjools> that query has been used too many times already :/
<wgrant> bigjools: How I look forward to the great purge next month...
<bigjools> wgrant: que?
<wgrant> bigjools: Once SPRB and TTBJ are ported to the new BuildFarmJob infrastructure, we get to delete lots of classes and DB tables.
<bigjools> ah
<wgrant> And remove the opportunities for inconsistencies.
<bigjools> yay
<wgrant> bigjools: https://code.edge.launchpad.net/~wgrant/launchpad/ddeb-domination
<wgrant> How does that approach seem?
<bigjools> looking ...
<bigjools> wgrant: will it look like a deb dominates a dedeb?
<bigjools> ddeb, even
<bigjools> wgrant: and will the ddeb publisher notice that a ddeb was superseded by a different publisher?
<bigjools> BTW, a WIP MP would be useful to see a diff :)
<bigjools> wgrant: how is performance affected on the query changed in domination.py
<bigjools> (these are all questions that you don't need to answer right now, I am just checking your approach)
<wgrant> bigjools: Remember that BPPH.supersededby is a BinaryPackageBuild. For this sort of reason.
<wgrant> Hmm, fair point on that query. Not sure about that.
<wgrant> As for the publisher noticing the superseded stuff, it works fine apart from the bug I filed yesterday.
<wgrant> And that's a restricted case that we can probably ignore, but I wanted to record.
<bigjools> wgrant: ok - I was going by one of your commit comments where you implied that ddebs live in the same archive - they don't.
<wgrant> Ah, right, yes. That's for the non-primary case, where they do live in the same archive.
<wgrant> Do we know when Lucid is happening yet?
<bigjools> RSN
<rockstar> bigjools, is the build farm overloaded right now?
<bigjools> I can say yes without even looking :)
<wgrant> It seems to have just been hit with a wave of dh7 retries.
<wgrant> But that should clear soon.
<bigjools> rockstar: https://edge.launchpad.net/builders <-- self service
<rockstar> bigjools, I knew there was a URL, I just never remember what that URL is.  :)
<bigjools> rockstar: unfortunately I don't have the same amnesia issue
<rockstar> bigjools, yup.  That's why I asked you.  :)
 * flacoste is back.
<lifeless> \o/ my librarian patch has landed
<lifeless> mthaddon: I'm hopping on a plane now, but I'll try to get the configs branch up for you
<mthaddon> cool
<lifeless> fast search will have to wait for me to get to NZ I think
<lifeless> mthaddon: the appservers - please tell me they are not using a single oops dir, with the same oops prefix, across different processes
<mthaddon> lifeless: I think you should be able to tell me that :) but I'm pretty sure they use different OOPS prefixes
<lifeless> in which case its ok
<lifeless> mthaddon: do you have enough data to move forward on rt 40480 ?
<lifeless> mthaddon: tell me a bout librarian and librarian2 ?
<mthaddon> lifeless: erm, so are you saying that X frontends doesn't expose us to bzr locking any more than 1 frontend?
<lifeless> mthaddon: right
<mthaddon> lifeless: I don't think we're blocked on you in that case (although testing this theory would be nice, and we'll certainly need to do this on staging first)
<lifeless> mthaddon: right
<lifeless> mthaddon: lp:~lifeless/lp-production-configs/librarian - don't merge till we do a deployment to the librarian of whats currently out on edge, cause the schema stuff is lock-step
<lifeless> mthaddon: I only see a config for 1 librarian though - public and internal instances-of, but only one process for each role. Is that right ?
<mthaddon> lifeless: you didn't see the librarian2 folder?
<lifeless> I did
<lifeless> that appears to be the private librarian
<mthaddon> nope
<lifeless> not a redundant public librarian
<lifeless> ok, cool
<lifeless> anyhow, I've put the change into both parts
<lifeless> I must run and get security checked etc now.
<mthaddon> since it's not proposed for merging yet, we'll leave it til that happens
<lifeless> please do grab spiv/mwhudson/jml if the codehosting thing gets hairy - I know all of them have looked at the entire vertical stack
 * lifeless fades into the sunset 
<jml> sinzui, hi
<sinzui> hi jml
<jml> sinzui, want to mumble?
 * lifeless is in da queue
<leonardr> thumper, are you around to talk about your web service question?
<leonardr> otherwise i will send an email
<lifeless> leonardr: its EEEEEARLY for him - I suggest a mail :)
<lifeless> leonardr: 3:30am specifically.
<leonardr> lifeless, sure, just checking
<rockstar> abentley, we seem to be having some sort of issue with the scanner not setting branch merge proposals to "Merged"
<abentley> rockstar, okay, can you be more specific?
<rockstar> abentley, no.  I'm trying to find more details now.
<abentley> rockstar, you don't have an example where this didn't happen or anything?
<rockstar> abentley, actually, I can be a bit more specific.  The folks in #tarmac are complaining that Tarmac keeps trying to land things after its merged them already.
<abentley> rockstar, could it be a race condition?
<rockstar> abentley, I'm not prepared to rule anything out until I see an example proposal.
<mtaylor> https://code.edge.launchpad.net/~mordred/swift/fix-pep8/+merge/30894
<rockstar> abentley, https://code.edge.launchpad.net/~mordred/swift/fix-pep8/+merge/30894
<rockstar> mtaylor, :)
<mtaylor> beat you to it
<mtaylor> :)
<abentley> mtaylor, that branch hasn't been fully merged.
<mtaylor> no?
<rockstar> mtaylor, oh, I know what is.
<abentley> mtaylor, see the "unmerged revisions" heading?
<rockstar> mtaylor, you pushed new revisions AFTER it was approved, didn't you?
<mtaylor> rockstar: yes. it does seem so
<rockstar> mtaylor, yup, so the approved revision id isn't the tip of the branch.
<rockstar> abentley, thanks for your help.
<abentley> rockstar, np.
<mtaylor> rockstar: ok. but the merge proposal should still be marked as merged, no? the thing that was approved happened
<mtaylor> I mean- something should certainly be in some state here
<abentley> mtaylor, I don't know.  Certainly I didn't think so when I wrote it.
<abentley> mtaylor, the fact that there are outstanding changes definitely seems like something we should represent.
<abentley> mtaylor, because we wouldn't want those changes to get lost.
<mtaylor> yeah - it's in a weird limbo state right now ... it's still approved -so I can't really approve the new changes easily
<abentley> mtaylor, commonly in LP code reviews, we'll say "approved, if you make these changes", and so a branch in this state isn't really merged for us.
<abentley> mtaylor, we don't really consider what state the proposal is in when we mark it merged.  It could be rejected, approved, but when tip goes into target, we consider it merged.
<rockstar> mtaylor, also, we don't want the proposal to disappear in cases where not all the revisions are merged.  That would lead the user to think something incorrect.
 * rockstar sees abentley has already said this
<mtaylor> yeah - I understand all of what you're both saying...
<rockstar> mtaylor, it helps when we're saying the same thing.  :)
<abentley> mtaylor, it's hard in the Launchpad workflow to get into this state, because our landing tool always lands the tip.
<mtaylor> I'm just saying that the system does not currently lead me to think any action needs to be taken, or what action I should take
<rockstar> abentley, with PQM you mean?
<abentley> rockstar, yes.
<rockstar> mtaylor, yeah, that's a Tarmac bug more than a Launchpad bug.  I think Launchpad is acting as it should.
<abentley> mtaylor, It seems to have gotten your attention, though.
<mtaylor> abentley: only because of the failed tarmac job
<abentley> mtaylor, you don't use +activereviews?
<mtaylor> but no, I don't think it's just a tarmac bug - because I think tarmac is doing the right thing here
<mtaylor> I think just merging the tip rather than merging the approved revision is a terrible idea
<mtaylor> as there is no workflow support to say that the new revisions are actually good
<abentley> mtaylor, you're welcome to your opinion, but I don't share it.  See above.
<mtaylor> abentley: you don't think new revisions need code review?
<abentley> mtaylor, not always.
<rockstar> mtaylor, new revisions need code review, yes, but I think that's a user's responsibility.  Launchpad shouldn't have to know about everyone's different workflows.
<abentley> mtaylor, we trust our contributors to not make gross changes.
<abentley> mtaylor, and we would find it a pain to have to do another roundtrip for the trivial changes we sometimes request in a review.
<mtaylor> ok. that's all fine, we can differ in our opinion here - just tell me, in this state, how do I approve new revisions
<rockstar> mtaylor, just unset Approved and reset it.
<rockstar> mtaylor, we could probably do better in that regard though.
<mtaylor> <troll>how about not setting the status to Approved until the changes have been made?</troll>
<mtaylor> rockstar: ok. I'll just do that. seems clunky though
<rockstar> mtaylor, in my case, that's what I do.  If I ask for changes, I'll vote "approve" but not set it to Approved.
<abentley> mtaylor, "We would find it a pain to have to do another roundtrip for the trivial changes we sometimes request in a review".
<mtaylor> hehe
<rockstar> mtaylor, if you file a bug about the "clunkiness" of re-approved, I'm sure we can triage it.
<mtaylor> rockstar: have I mentioned that we need merge queues
<rockstar> mtaylor, yeah, I think I was the one that put that earworm in your head.  :)
<rockstar> mtaylor, see #tarmac for a tarmac solution.
<mtaylor> the _real_ problem here of course being that we are encoding two different pieces of information with the same status - and abentley and I choose to overload that system in different ways
<abentley> rockstar, you wrote a song about merge queues?
<mtaylor> zomg. I so want to hear rockstar's song about merge queues
<rockstar> abentley, earworms don't have to be songs.  :)
<rockstar> Although I guess now I need to write a song about merge queues...  :)
 * rockstar buckles under peer pressure
<abentley> rockstar, http://www.wordspy.com/words/earworm.asp
<abentley> rockstar, send me your song about merge queues, and I will lay down some flute, cowbell, tambourine or vox-- your choice.
<mtaylor> jml: see... we're writing songs about it now..
<rockstar> abentley, all of the above, but always more cowbell.
<rockstar> abentley, I regret to inform you that the build estimation code that you worked so hard on is being trampled on by builders being overwhelmed at the last second...  :)
<bdmurray> leonardr: how could I address robert's concerns in https://code.edge.launchpad.net/~brian-murray/launchpad/bug-320596/+merge/30690?
<leonardr> bdmurray, checking
<jml> ciao
<leonardr> bdmurray, ok, long story short, to maintain backwards compatibility you need to have two calls to @operation_parameters
<leonardr> this should work:
<leonardr> @operation_parameters(parameters_for_devel)
<leonardr> @operation_for_version('devel')
<leonardr> @operation_parameters(parameters_for_old_versions)
<leonardr> you should define a list containing the parameters, then copy it and change only the parameter you want to change, so that you don't have two copies of that huge list
<leonardr> add a doctest to the stories/webservice directory of malone, illustrating that in 1.0 the default value is True, but that in devel the default value is False
<leonardr> does this make sense?
<leonardr> (be sure to put @operation_for_version('devel') on top of all of your current annotations--it's the dividing line between 1.0 and devel
<bdmurray> the parameters for devel would be very search parameter?
<bdmurray> every that is
<leonardr> bdmurray: yes, you would pass in slightly different lists for each @operation_parameters call
<bdmurray> leonardr: okay thanks
<gary_poster> bac: hey.  didn't you have a "run the tests that failed on ec2" command?  If so, does it still work?  If so, what is it? :-)
<gary_poster> ah!  bin/retest looks promising...
<gary_poster> except that the ec2 output doesn't have the kind of information we need, anymore, it seems :-/
<bac> gary_poster: :(
<bac> gary_poster: but you can use testr to do it
<gary_poster> bac, ?
<bac> gary_poster: read robert's email from last week
<gary_poster> ok
<gary_poster> ("oh," Gary thinks, "Robert's one email from last week!  Right!" :-) )
<gary_poster> bac, AFAICT that only helps if you have run tests locally, not if you ran test on ec2 that you want to rerun now
 * gary_poster found the email
<benji> the person that develops a working knowledge management system that gathers data from email will make a mint
<bac> gary_poster: i thought you could unzip the attachment that came in the failure message and pipe it into testr load, or something similar
 * gary_poster looks at docs again
<bac> gary_poster: i admit i have not tried it personally
<bac> gary_poster: perhaps as a non-expert you should write down what actually works and how to do it...assuming you are successful
<gary_poster> bac, https://dev.launchpad.net/Testing has what has been written so far.  I see ``testr load < testrun`` in testr --help but the relationship between that and the ec2 email output is less than obvious.  I'll play around for a second, and, yes, write something down if it seems valuable
<gary_poster> LOL, "testr load < ~/apidoc.log.gz" didn't seem useful :-)  Maybe I'm supposed to uncompress it...
<gary_poster> For those following along at home, uncompressing did in fact seem to allow testr to load the subunit file.  The resulting output was a bit concerning:  id: 0 tests: 8956 failures: 5 skips: 23
<gary_poster> i.e., what about the skips?
<gary_poster> Then there's the concern that Gavin raised on the list
<gary_poster> abentley: hey.  If I get the output "id: 0 tests: 8956 failures: 5 skips: 23" after loading in a subunit ile to testr, what is the source of the "skipped" line?  How do I see what the skipped tests are?  How do I run them, if I need to?
<abentley> gary_poster, I believe there are no skips in the Launchpad test suite.  testresources counts failure to tear down layers as skips.
<gary_poster> abentley: ah, great, so, for the short term at least that means that the information about skipped stuff is ignorable?
<abentley> gary_poster, yes.
<gary_poster> cool thanks
<mtaylor> rockstar: here's a UI thought for you
 * rockstar listens
<mtaylor> rockstar: I just had someone be confused as to why merge requests and bug reports are separate things
<rockstar> mtaylor, yes, I've heard this joke before.
<rockstar> :)
<mtaylor> rockstar: so it just got my brain trying to wrap its head a) around the urge and b) around whether or not there is something useful in the request
<rockstar> mtaylor, so, this is why we have branch links.
<mtaylor> rockstar: the reference was that in JIRA, a patch is attached to a bug and then discussion ensues
<mtaylor> I think the main thing here had to do with discussion around a thing
<rockstar> mtaylor, I do know the landscape team was doing their code reviews on the bug report.
<rockstar> (I think that is crazy)
<mtaylor> me too... but I'm wondering if there isn't some way to express some sort of tighter coupling or some sort of information sharing that isn't something we think is on crack
<mtaylor> rockstar: the specific initial comment was "It's just annoying that e.g. bug reports are separate from merge requests, and the blueprints don't have a good discussion platform built in... Just feels like I have to repeat everything two or three times to keep launchpad happy..."
<rockstar> mtaylor, well, yeah, I've been thinking about ways to display more bug content in the merge proposal page, but I'm currently more focused on the branch page redesign currently.
<mtaylor> rockstar: I believe I had a thought a while back about being able to pre-populate a merge request description from a blueprint,
<mtaylor> rockstar: essentially thinking that all branches should either be related to a bug or a blueprint -but that in many cases something could be generated automatically in one directoin or another
<rockstar> mtaylor, that would require people other than Drizzle, Ubuntu, and Wikkid (because thumper is a karma whore) use blueprints.
 * thumper likes karma
<mtaylor> rockstar: well, once I fix them, I hope that they will
<thumper> at least I don't game the system though
<thumper> mtaylor: are you going to work on blueprints soon then?
<mtaylor> thumper: yes
<rockstar> mtaylor, my hat's off to you.
<thumper> mtaylor: how soon?
<mtaylor> rockstar: you can add openstack to the blueprint using list :)_
<mtaylor> thumper: soon
<thumper> real soon now?
<rockstar> mtaylor, I might have recruited someone to help you with that.
<mtaylor> rockstar: excellent
<rockstar> mtaylor, the downside is that he's 15.  :)
<thumper> mtaylor: I'd happily have pre-impl calls about blueprints
<thumper> mtaylor: and review changes
<thumper> mtaylor: I'm sure sinzui would too
<mtaylor> thumper: great. well, I _really_ need to write my list of things down
<thumper> jelmer: could I get you to send me an email with the known good revisions of the bzr plugins?
<thumper> jelmer: also specifically if they'll work with 2.1 and 2.2 at the same time
<jelmer> thumper: what sort of timeframe do you have in mind for landing these?
<thumper> jelmer: if they are simple, then we could just drop them in ASAP
<rockstar> mtaylor, I know this awesome new site that allows your to write a todo list of things.  The call the items "bugs" and it really helps me do my work...  :)
<thumper> jelmer: all we need to do is merge them into the launchpad branches, and update the config file
<thumper> rockstar: but you can't order them :P
<mtaylor> rockstar: hrm. ... relative cost of opening a bug in launchpad vs. writing a list in a file in vi...
<rockstar> thumper, sure you can.  It's called "importance"
 * thumper coughs 
<thumper> that is pretty rough ordering
<mtaylor> rockstar: have I mentioned that the bug opening cost in lp is still pretty high?
<rockstar> mtaylor, blame lifeless.  That's his job.  :)
<mtaylor> rockstar: oh, I will
<jelmer> thumper: can I perhaps just pqm-submit to the various lp:~launchpad-pqm/ branches and then have you review the update to sourcedeps.conf ?
<james_w> as an alternative to workitems in blueprints, we were pondering last week using bugs
<james_w> and providing an API client that allows you to just write a list of things and it will open all the bugs for you
<jelmer> thumper: still there?
<thumper> jelmer: yep
<jelmer> thumper: Did you see my question earlier?
<thumper> jelmer: sorry, rebooted due to updates
<thumper> jelmer: yes, and yes
<jelmer> thumper: ok, I'll that then - thanks
 * thumper searches for more coffee
<jelmer> thumper: Any plans yet to have bzr-svn/bzr-git in the PPA and built as daily builds from lp:~launchpad-pqm/bzr-{git,svn}/devel rather than checkout out manually in sourcecode.conf ? :-)
<rockstar> jelmer, that would be awesome, and I would like that.
<thumper> jelmer: we need a way to control the plugins, and add them as we need them
<thumper> jelmer: so as long as we can work out how to do that...
<thumper> jelmer: should be fine
<rockstar> Argh, timeouts on staging make rockstar angry.
<rockstar> You wouldn't like me when I'm angry.
<jelmer> thumper: what do you mean by add exactly? when to load them, when to update their version?
<thumper> jelmer: we don't want them on all the time
<jelmer> thumper, isn't that an issue now as well though?
<thumper> jelmer: no
<james_w> OOPS-1668ED4702
<james_w> no bot?
<benji> james_w, you might like this: I added a quick search to my Firefox so I could type/paste "OOPS 1668ED4702" in the address bar and be taken to it
<james_w> good idea
<james_w> could someone confirm that https://lp-oops.canonical.com/oops.py/?oopsid=OOPS-1668ED4702 looks to be https://bugs.edge.launchpad.net/launchpad-code/+bug/518337 ?
<_mup_> Bug #518337: timeout oopses on person page <oops> <Launchpad Bazaar Integration:Triaged> <https://launchpad.net/bugs/518337>
<benji> made one for "bug LP_BUG_NUMBER" too
<james_w> I have that one, plus a bunch of others, but never thought of doing it for oopses
 * flacoste is away: Gone away for now
#launchpad-dev 2010-07-27
<thumper> I hope I didn't make lazr.enum's too restrictive...
<thumper> \o/
<mtaylor> thumper: bah
<thumper> whazzup?
<mtaylor> thumper: you know, hanging out
<mtaylor> had a spirited discussion with rockstar on #tarmac today about merge queues
<thumper> mtaylor: just feeling bah? or looking at code bah
<thumper> there is a #tarmac?
<mtaylor> oh - I was saying bah to "I hope I didn't make lazr.enum's too restrictive"
<mtaylor> thumper: yup. you should come hang out
<thumper> mtaylor: there already
<thumper> mtaylor: lazr.enum's are cool (IMO)
<thumper> I'm biased though as I wrote it
<mtaylor> hehe
<thumper> it uses a lot of metaclass magic
<thumper> luckily I didn't shoot myself in the foot
<thumper> I wanted to add an extra class method
<thumper> and it works fine
<mtaylor> excellent
<mtaylor> metaclass magic is fun
<thumper> to avoid my work getting too cumbersome, I'm about to pipeline it
<thumper> and land bits as I go
<thumper> I'm just missing a handy reviewer, that's all
<thumper> mtaylor: what is ETA on starting blueprint hacking?
<thumper> mtaylor: I've got a good starting list for you
<mtaylor> soon. real soon now
<mtaylor> I've got a good starting list for me too!
<mtaylor> thumper: I've got a few items I need to clear from my queue
<mtaylor> then it's on to lp hacking - although I need to do the CoC abstraction first
<thumper> mtaylor: feel free to ping me on any ideas, email or IRC
<mtaylor> thumper: but I welcome any blueprint things you need done... I think making "request feedback" actually do something may be first on my list :)
<thumper> mtaylor: some refactoring needs to be done internally on the properties to handle the automagical way that started and completed is set
<thumper> mtaylor: in order to have meaningful exposure of the attributes through the API
<thumper> mtaylor: this then leads to the ability to use ajax to set the various status values
<mtaylor> thumper: mmm. status values
<mtaylor> thumper: so - did you see the bug I filed about inline code reviews?
<mtaylor> thumper: so - did you see the bug I filed about inline code reviews?
<mtaylor> ag. netsplit not back here
<mtaylor> thumper: so - did you see the bug I filed about inline code reviews?
<mtaylor> also - this lies:
<mtaylor> https://code.edge.launchpad.net/~mordred/+recipe/trunk-nightlies
<lifeless-cn> turns out, there is a firewall in china
<lifeless-cn> who'd a thunk it
<lifeless-cn> and its boarding time
<lifeless-cn> ciao
<thumper> mtaylor: yes I did see your bug about inline code reviews
<thumper> mtaylor: and I said so last time too :)
<mtaylor> thumper: sorry. netsplits
<mtaylor> :)
<thumper> :(
<thumper> FFS
<thumper> devel is broke
 * thumper EODs
<mtaylor> oh. you should let devel break
<mtaylor> shouldn't
<mtaylor> is what I meant to say
<adeuring> good morning
<mrevell> Howdy
<wgrant> bigjools: I wonder if the publisher should rename PPAs once it's deleted them.
<wgrant> That would at least enable users to recreate them, until we have a better way.
<wgrant> (rather like with account deactivation)
<bigjools> mebbe
 * flacoste is back.
<jml`> if I wanted to start fiddling around with using sphinx to generate documentation for Launchpad...
<jml`> is there somewhere I should start rather than google?
<mtaylor> jml`: are you a learn by example or a learn by explanation type of guy?
<jml`> mtaylor, both.
<mtaylor> jml`: http://sphinx.pocoo.org/ is where all the doco is - feel free to grab lp:swift for an example project generating docs with sphinx
<jml`> mtaylor, thanks.
<mtaylor> jml`: my pleasure!
<jml`> do you know if anyone has already done some work on this for LP?
<mtaylor> I do not
<mtaylor> but I think it's a Very Good Ideaâ¢
<mtaylor> I've got a hudson job currently generating and publishing our docs on every push ... seems to be working out really well
<mtaylor> of course - and I don't know why I didn't think of this earlier - there should be a launchpad thing similar to translations that will watch a branch, generate docs from sphinx if they have them and publish them to docs.launchpad.net/project or something
<mtaylor> :)
<mtaylor> rockstar: d00d. when I'm creating a project, it would be so nice if there was a button somewhere which said "I'm hosting this on launchpad and I want to do everything there" instead of having to go through and enable each launchpad feature individually
<rockstar> mtaylor, that's a sinzui suggestion.  :)
<mtaylor> sinzui: d00d. when I'm creating a project, it would be so nice if there was a button somewhere which said "I'm hosting this on launchpad and I want to do everything there" instead of having to go through and enable each launchpad feature individually
<rockstar> mtaylor, I'd like if I could create a project on launchpad by just pushing code to lp:~rockstar/<project-name>/trunk
<mtaylor> that would also be cool
<sinzui> mtaylor, We have considered that. That is a rare case. Do you know what everything really entails with translations. Do you really want to use blueprints...less than 1% of project do.
<mtaylor> sinzui: sure - but perhaps if there was an expert button somewhere ... I keep creating projects because I've made the commitment to use launchpad for everything I do
<mtaylor> but it seems I pay the cost for _not_ being a n00b
<mtaylor> in that I have to run through 700 steps to set up something when I know from the beginning that I want everything :)
<mars> bac, ping
<bac> hi mars
<rockstar> mars, ping
<mars> Hi rockstar
<rockstar> mars, why are the code windmill tests looking for http://code.launchpad.dev:8085/+yui3-unittest/lp/code/javascript/tests/test_productseries-setbranch.html ?
<rockstar> (It's 404ing on my yui3.1 branch, and I'm trying to figure out why)
<mars> rockstar, just a guess: someone added YUI unit tests to the code suite (as has been done for bugs and registry)
<mars> there is a file for it in the windmill/ directory
<mars> rockstar, lp.code.windmill.tests.test_yuitests
<rockstar> mars, yes, I have a traceback here.
<mars> First place I would look
<rockstar> mars, before I go digging through code, do you have any idea why this would be 404ing?
<rockstar> mars, I've only changed frontend code, so I have to idea why this would 404.
<mars> rockstar, no idea, sinzui and brad wrote that I believe.  But could I see the traceback?
<sinzui> rockstar, do you know that +yui-unittest is a handler only registered for the testrunner?
<rockstar> sinzui, yeah, I looked at the diff after I did it (you know, 'cause I use version control) and it seemed kosher.
<sinzui> rockstar, and I doubt something that small would have been caught by a reviewer.
<rockstar> sinzui, yeah.  This is why tests is awesome.
<rockstar> Unfortunately, it's a windmill test that saved me this time, so I need to go eat my socks.
<mars> sinzui, thank you for fixing devel - now I can finally land my branch from yesterday!
<mars> rockstar, ping
<rockstar> mars, hi.
<mars> Hi rockstar, just a heads up that I am going to land my lazr-js 1.0 branch: lp:~mars/launchpad/update-lazr-js-to-1.0
<mars> Now that devel is fixed
<rockstar> mars, does that have yup 3.1.1 in it?
<rockstar> s/yup/yui/
<mars> rockstar, no.  It bumps the lazr-js version number and fixes the windmill test suite to work with your CSS Overlays
<rockstar> mars, hrm.
<rockstar> mars, I guess go ahead and do that.
<rockstar> mars, I seem to fighting velocity of devel right now landing yui 3.1.1
<mars> ok
<mars> I just need to verify that devel fixes the busted tests, then I'll lp-land it
 * rockstar goes to buy an air conditioning unit instead of baking
#launchpad-dev 2010-07-28
<wgrant> mars: Could you land that branch for me, please?
<lifeless> wgrant: have you fixed the typo
<wgrant> lifeless: See my reply...
<wgrant> It was already fixed.
<lifeless> hah
<wgrant> It was in a removed line.
<lifeless> ok
<lifeless> see, jetlag :P
<wgrant> Heh.
 * wgrant -> uni
<lifeless> its on its way
<mars> thanks lifeless
<mtaylor> lifeless: is there a way to track how many times a branch has been pulled
<lifeless> do you perhaps mean
<lifeless> 'is there a way for a branch owner to know its popularity' ?
<lifeless> cause thats a very different question
<mtaylor> uh - I dunno... I was just asked "hey is there a way to track how many time a branch has been downloaded/pulled?"
<thumper> mtaylor: nope
<mtaylor> thumper: k. cool
<thumper> mtaylor: subscriptions may indicate others interest though
<thumper> mtaylor: if I'm really interested in a branch, I'll subscribe
<thumper> mtaylor: but I often grab branches I've just a passing interest in
<thumper> mtaylor: that isn't so useful to the branch owner
<thumper> mtaylor: have you talked to sinzui about your CoC generalisation?
<mtaylor> thumper: we spoke briefly a few months aog
<mtaylor> ago
<lifeless> mtaylor: so tracking pull is hard for a number of reasons
<lifeless> mtaylor: only one of which 'its distributed, fool'
<lifeless> mtaylor: other reasons are - http gets != pulls, requests to a stacked-on don't indicate pulls but use the same network verbs, and so forth
<mtaylor> lifeless: hrm. fair enough
<lifeless> mtaylor: not impossible, just not done
<lifeless> well, I mean - its not impossible to gather the specific data we /have/. Tracking consumers of a branch is impossible in the broader sense ;)
<mtaylor> well yes
<thumper> mwhudson: got a minute for a call?
<mwhudson> thumper: not really :(
<mwhudson> thumper: will do in a bit
<thumper> after lunch maybe?
<mwhudson> thumper: ok
<lifeless> thumper: can I help ?
<thumper> lifeless: no
<lifeless> ok :)
<thumper> lifeless: your biggest problem being that your aren't mwhudson :)
<lifeless> thumper: heh - I wasn't offended or anything :).
<thumper> lifeless: testr load finished with-> id: 1 tests: 8993 failures: 22 skips: 23
<thumper> lifeless: but testr failing shows one failure
<lifeless> thumper: btw, if you want a weekly call with me or something-like-that, let me know. I'm inclined to just focus on good team wide dialogs fo rnow.
<lifeless> thumper: there is an unimplemented(shock, horror!) bit of testr that should detect deleted / renamed tests.
<lifeless> thumper: your test run 0 had the failure in it, and the failure is one for a test id not reported in your test run 1.
<lifeless> thumper: for now, rm .testrepository/failing
<thumper> and reload?
<lifeless> thumper: yup
<lifeless> oh, hangon - see, this is me just getting off a plain :)
<lifeless> 22 failures - that really should be reported by 'testr failing' :)
<lifeless> this is from a ec2 land response ?
<thumper> hmm...
<thumper> yes
<thumper> same result
<lifeless> forward me the stream?
 * thumper deletes .testrepository
<thumper> same with a clean testr init
<thumper> :(
<lifeless> I'd like to look at the stream, if I may
<lifeless> I'm not alert enough to do a sensible series-of-questions-to-debug
<thumper> lifeless: email sent
 * thumper thinks that perhaps devel is failing again :(
<thumper> ERROR: lp.soyuz.browser.tests.test_archive_webservice (subunit.RemotedTestCase)
<lifeless> thumper: its the same test every time
<thumper> lifeless: yeah
<thumper> I was just looking
<lifeless> thumper: zcat ~/Desktop/Downloads/merge-directive-handling-no-request-mirror.log.gz | subunit-filter --no-passthrough --no-skip | subunit-ls | uniq
<thumper> lifeless: added to .bashrc:
<thumper> function lp-failing
<thumper> {
<thumper>   zcat $1 | subunit-filter --no-passthrough --no-skip | subunit-ls | uniq
<thumper> }
<lifeless> thumper: heh
<lifeless> wgrant: what is 'lucille'
<thumper> gah
<thumper> WTF?
<thumper> ââ¹
<thumper> ââ¹
<thumper> ââ¹
<thumper> ââ¹
<thumper> ââ¹
<thumper> lifeless: that failing test assumes python 2.6
<thumper> and doesn't import the with statement
<thumper> which is why ec2 fails for it
<thumper> but the builder passes
<lifeless> noice
<lifeless> by which I mean 'garh'
<thumper> lifeless: rs from you to add the import?
<thumper> lifeless: I'll land the "fix" now
<lifeless> doit
<lifeless> do we have a 'trivial' tag ?
<thumper> lifeless: not any more
<thumper> but rs (rubberstamp) is often considered good enough
<thumper> we do expect to have more than one person look at things
<thumper> even if just in concept
 * thumper files a bug to make done
<lifeless> hmmm
<lifeless> we should really consider rolling that back
<lifeless> peer review is an important thing for helping make things great, but something like this - bah
<wgrant> lifeless: lucille was archivepublisher.
<lifeless> hah
<lifeless> we still call the db user lucille
<wgrant> We do.
<wgrant> And there's still lucilleconfig.
<wgrant> I may one day unleash my fury on and destroy lucilleconfig.
<lifeless> bbiab, post-travel fug is winning
 * thumper -> lunch
<mwhudson> thumper: i guess you don't want to chat then
<thumper> mwhudson: I'm back now
<mwhudson> thumper: how long is the call likely to take?
<thumper> mwhudson: not long
<mwhudson> thumper: i'll call your landline?
<thumper> mwhudson: you have no skype?
<mwhudson> thumper: i can see if that's still working
<mwhudson> i'm in a cafe
<thumper> ok
<mwhudson> so a headset is probably actually a good ida
<wgrant> lifeless: Ah, thanks, I didn't know you were ec2testing that.
<wgrant> I wonder why buildd-manager doesn't log an OOPS and suspend builds when the dispatcher crashes.
<lifeless> it just needs the same love I gave librarian, I suspect
<wgrant> Also, universities need to die.
<wgrant> Seriously.
<spm> from the perspective of having been nearly 20 years since I got my degree, I'm inclined to agree. Although... The Qld Uni Great Court in Summer time... ahhh. memories.
<thumper> lifeless: still with us?
<lifeless> ish
<lifeless> 'sup ?
<thumper> testr run --failing fails to identify that one of the failing ones is fixed
<thumper> testr failing still shows it
<lifeless> check that it is run - subunit-ls < .testrepository/$id | grep testname
<lifeless> its possible the loader is doing something funky
<thumper> $ testr run --failingrunning: xvfb-run ./bin/test --subunit --load-list /home/tim/src/launchpad/stop-mirroring-hosted/failing.list| testr load
<thumper> $ testr failing --subunit | subunit-ls  -> now gives a shorter list
<lifeless> check that it is run - subunit-ls < .testrepository/$id | grep testname
<thumper> but it is still showing lp.soyuz.browser.tests.test_archive_webservice
<thumper> lifeless: I don't grok your last comment
<lifeless> each test run generates a new file .testrepository/ID
<lifeless> where ID is printed at the end of the run
<thumper> yes
<lifeless> that has in it all the tests that were run
<thumper> running: xvfb-run ./bin/test --subunit --load-list /home/tim/src/launchpad/stop-mirroring-hosted/failing.list| testr load
<lifeless> the most likely reason for testr run --failing to not see a failing test as fixed is if the test runner (zope.testing) did not run it.
<thumper> that is what it says in response to testr run --failing
<lifeless> yes
<lifeless> then it should say
<lifeless> id: FOO tests run: ....
<thumper> id: 3 tests: 18 failures: 6 skips: 1
<lifeless> right
<lifeless> so
<thumper> which I weirdly read as "18 failures" "6 skips"
<lifeless> subunit-ls < .testrepository/3 | grep lp.soyuz.browser.tests.test_archive_webservice
<thumper> not there
<lifeless> si
<lifeless> so
<lifeless> its not being run
<thumper> why is it telling me it is failing then?
<lifeless> this suggests its not in your branch
<thumper> testr failing ?
<lifeless> testr keeps a record of all seen tests ever
<thumper> but it is no longer failing
<thumper> :(
<lifeless> if you bzr switch or whatever between branches while a test is failing, and the test is not in the new branch
<lifeless> then testr will still think its failing
<lifeless> thumper: its *not being run*
<lifeless> thumper: that means whether its failing or not is *unknown*
<lifeless> thumper: my suggestion, when you bzr switch, nuke .testrepository
<thumper> I don't switch
<lifeless> thumper: try bin/test -t lp.soyuz.browser.tests.test_archive_webservice
<thumper> that passes
<lifeless> but does it run the test
<lifeless> see
<lifeless> that string looks like a module name to me
<lifeless> and thats how zope reports a failed import
<thumper> what happened was:
<thumper> I loaded the repository with the output from ec2
<thumper> my branch didn't have an up to date devel
<thumper> so it didn't have the failing soyuz test
<thumper> until I merged devel
<lifeless> so as far as testr is concerned, a test has been deleted, which as I explained earlier, testr doesn't *yet* have a heuristic to guess at.
<thumper> and committed
<thumper> now it is stuct
<thumper> but it is there now...
<lifeless> right
<lifeless> so ignore that one test until you've fixed all the issues
<lifeless> then delete .testrepository/failing
 * thumper looks skyward
<lifeless> patches accepted; its a very raw tool. I, and others, find it very useful, but its still rraw.
<lifeless> thumper: do you see what is going on ?
<thumper> I think so
<mtaylor> /msg lifeless oh, and hi, btw
<mtaylor> heh. guess people know my secrect message to lifeless now
 * wgrant puts it in the blackmail file.
<mwhudson> scandal
<poolie> hi mtaylor, mwhudson
<mtaylor> hi poolie
<poolie> lifeless, i worked on the flags thing on the plane, so now it only looks up scopes as needed
<mwhudson> hi poolie
<adeuring> oodmorning
<wgrant> bigjools: Morning.
<bigjools> morning
<wgrant> Are you aware of the chaos this morning?
 * bigjools sighs heavily
<bigjools> no
<wgrant> Bug #610687 caused the build farm to grind to a halt for many hours. We've disabled the archive in question (I think you might have an email about that), but it needs manual cleanup before it can be reenabled.
<_mup_> Bug #610687: Delayed copies do not respect PPA component override <canonical-losa-lp> <Soyuz:New> <https://launchpad.net/bugs/610687>
<wgrant> Basically, we had a build attempting to dispatch with a component of 'non-free'. This made things rather unhappy indeed.
 * bigjools sees the email
<spm> 3 emails to be precise, but yes.
 * bigjools sees why it happened and is very angry
<spm> wgrant: pls to be ware. bigjools has a '.procmailrc' - From: spm@c.c >> /dev/null, so emails can be a tad hit'n'miss
<bigjools> spm who?
<spm> hahaha
<wgrant> (somewhat amusingly, I was on Monday looking at the code throughout Soyuz that creates publications, and noting how badly factored it was. one of my planned cleanups would have accidentally fixed this...)
<bigjools> wgrant: don't let me stop you :)
<wgrant> I need to fix it for ddeb copies, so it will be happening soon, yes.
<wgrant> bigjools: Is cleanup just a matter of DELETEing the publications and builds, or did some stuff actually make it to disk?
<bigjools> wgrant: I've not analysed it yet, I've got a load of stuff to get done this morning before I can look.
<bigjools> it can wait if it's not breaking anything *right now* :)
<wgrant> Well, as long as Brian doesn't reenable it.
<bigjools> he's been warned no to
<bigjools> not
<wgrant> Ah, good.
 * wgrant is tempted to take over the world, just to impose a global ban on shitty CS courses.
<jelmer> heh
<jelmer> wgrant: What are they having you do?
<wgrant> jelmer: Well, it's so far been an excellent waste of 2.5 years, though I held some hope that the final semester project (4-5 people per team, 12 weeks in length) might hold some interesting or at least vaguely challenging material. Today, project assignments came out: most of the other teams' projects have some interesting visualisation, data processing or HID components. We were landed with a basic Web 2.0 app, remarkable only in the fact ...
<wgrant> ... that someone could think it was even close to comparable with any of the others.
<wgrant> So there goes any hope of some value in this course.
<wgrant> Yay.
<jelmer> :-/
<jelmer> That sounds quite familiar. We had a 8 to 10 person final semester project (though I did it before my final semester) and quite a few teams ended up doing simple web services too.
<jelmer> The participation in a relatively large team was still quite educational though.
<wgrant> There was a 10-12 person project at the end of the course, but that has been cut in the new version.
<maxb> I think I learnt about 1 weeks worth of useful stuff from my uni course. And an immense amount from open source hacking whilst at uni :-)
<wgrant> Heh, yes.
<jelmer> heh, indeed
<mwhudson> i learnt stuff during my degree
<mwhudson> my phd on the other hand...
<mwhudson> (my degree wasn't in cos, that probably helped)
<mwhudson> *cs
<wgrant> The most useful thing I've learnt from the degree is to not do or trust degrees.
 * jelmer thought some of the more theoretical CS courses were very useful
<wgrant> True, the few theoretical ones have been somewhat educational.
<StevenK> wgrant: I learnt some useful things at uni. Most of which wasn't taught ...
<wgrant> StevenK: Heh.
<jelmer> EdwinGrubbs: Hi
<henninge> Hi!
<henninge> Did I miss the discussion on moving the registration slot between the title and the bread crumbs or was there no discussion?
<maxb> It's a little jarring
<wgrant> I like it for branches and some other stuff.
<wgrant> But for projects it doesn't really work, because it's not important information.
<wgrant> The old location wasn't good, though.
<EdwinGrubbs> jelmer: hi
<lifeless> morning all ye non-jetlaggers
<bigjools> hello lifeless
<lifeless> hi bigjools
<jml> lifeless, hi
<lifeless> hi
<lifeless> henninge: ping
<lifeless> henninge: your buildd manager logging patch : upstream twisted's log class supports what you want directly; I'm confused why you seem to have reimplemented it : or am I missing something ?
<mars> jml or lifeless, around?  Would either of you feel comfortable doing a Unix build system pre-implementation call?  I am trying to figure out how much effort we should put into cleaning up the process tree
<jml> mars, otp right now
<mars> k
<jml> mars, happy to in a little while
<lifeless> mars: sure
<henninge> lifeless: Hi
<lifeless> hi henninge; I'm really excited to see the twisted log rotation stuff getting fixed for us - we've a lot of daemons (buildd,librarian, codehosting) that all need it
<lifeless> henninge: I'm just hoping we don't need to maintain code [and thus tests] for it
<henninge> lifeless: I don't see how this could be done any other way in twisted
<henninge> lifeless: It does not re-open logs on SIGUSR1 but rotates them (in its own way).
<lifeless> henninge: just use a LogFile(foo, bar, rotateLength=0)
<lifeless> ?
<henninge> lifeless: that's what I do, don't I?
<lifeless> hmm, let me re-read
<lifeless> henninge: ok
<lifeless> otp for a sec
<rockstar> bigjools, dogfood fall down go boom.
<bigjools> rockstar: will look later, OTP
<henninge> lifeless: I'll be finishing off now. I replied on the MP. ;)
<lifeless> yeah thanks
<lifeless> ok, I think I'm going to pull on this librarian thing a bit, it should make a big difference
<mtaylor> bac: ah... so perhaps it's because of the commercial subscription ?
<bac> mtaylor: that's what i'm investigating
<bac> if so, it should be easy to fix.  this problem has been infested with red herrings
<mtaylor> hehe
<mtaylor> fish are difficult to catch
<bac> slippery
<mtaylor> yup
<bac> i'll let you know what i find
<jml> thumper, are you guys planning on doing anything with https://code.edge.launchpad.net/~jml/launchpad/private-branch-lookup-bug-261609
<thumper> yes
<jml> thumper, it's been stale for a while now, and I'm really keen to get it off my branch list.
<thumper> jml: I was finishing off another tech-debt branch, this is next
<jml> thumper, cool, thanks.
<james_w> the code users of the job system raise SQLObjectNotFound in a couple of places, is there something that should be used instead, or is this still the idiom?
<james_w> it's raised in a get @classmethod on a Storm object
<james_w> abentley: any idea? ^
<abentley> james_w, I'm not aware of something that should be used instead.
<james_w> abentley: is get() raising an exception a required thing? It seems to me that the only callers convert the exception to None, so the work of converting None to the exception is not needed.
<abentley> james_w, I don't know if it's a required thing, but I would expect that retrieving an object that doesn't exist is usually a serious error.
<lifeless> jml: am I on th eplanet ?
<lifeless> jml: I don't think I am... can I be added ?
<james_w> lifeless: is it a requirement to get a review from both you and stub for db patches?
<lifeless> yes to deploy, no to merge
<james_w> ok, good
<jml> lifeless, sure. email mrevell.
<lifeless> jml: you know, if lp knew about blogs, the planet could be auto maintained ;P
<jml> lifeless, patches tolerated.
<lifeless> haha
<lifeless> I only get to code if I'm a *good boy*
<lifeless> matsubara: hi
<matsubara> lifeless, hey
<lifeless> I put a patch up for getting bug numbers in the timeout section
<lifeless> but I can't bootstrap properly to test it
<lifeless> did you have any thoughts on it
<matsubara> lifeless, I didn't get an email notification about  the merge proposal
<lifeless> matsubara: I put it in the bug
<matsubara> lifeless, looking
<Ursinha> lifeless, bug 607776 is testable or not?
<_mup_> Bug #607776: +filebug-show-similar FTI query timeouts <qa-needstesting> <timeout> <Launchpad Bugs:In Progress by lifeless> <https://launchpad.net/bugs/607776>
<benji> lifeless: if you have a working buildout you can use it to bootstrap a new buildout cd path-to-new-project; path-to-working-project/bin/buildout bootstrap
<lifeless> Ursinha: no
<benji> unfortunately, having a bootstrap.py script in a buildout is an attractive nuisance
<lifeless> Ursinha: I have a follow on branch that drops the time by ~ 75%
<Ursinha> lifeless, cool
<Ursinha> that's cool
<lifeless> benji: it seems the oops tools want stuff that isn't in the dep cache
<Ursinha> lifeless, will remove the qa-needstesting tag for now
<lifeless> Ursinha: thanks, wasn't sure if I should / shouldn't
<benji> can't help you there :)
<lifeless> benji: :/
<matsubara> lifeless, what's the error?
<Ursinha> lifeless, that's the incremental change, if it's not testable, doesn't make sense to keep the tag there
<lifeless> Ursinha: kk
<lifeless> matsubara: bootstrapping?
<benji> maybe you need to bzr up the dependencies (~/launchpad/lp-branches/devel/download-cache
<benji> )
<matsubara> lifeless, I'm not sure the patch is going to do what you want. My initial impression to the bug report is that the static summary (https://devpad.canonical.com/~lpqateam/oops-
<matsubara> summaries/lpnet-2010-07-20.html#time-outs) was generated before the bug was linked to the oops
<matsubara> lifeless, yes, what dependency oops-tools is asking that's not in the cache?
<lifeless>   Getting distribution for 'zc.buildout==1.4.1'.
<lifeless> Error: Couldn't find a distribution for 'zc.buildout==1.4.1'.
<lifeless> matsubara: when I traced the code, it seemed like it was simply that the bug wasn't being set for the reporter
<abentley> james_w, I'm looking at the build emails, and the ordering of items looks like it could be improved.  I'd put the buildstate high.
<lifeless> matsubara: you could try it and see ;P
<matsubara> lifeless, you want this cache: bzr+ssh://bazaar.launchpad.net/~launchpad/lazr-source-dependencies/download-cache/ not the LP one
<benji> indeed, zc.buildout 1.4.1 doesn't appear to be in the centrally-managed cache; I suspect 1.4.0 would work for you
<lifeless> matsubara: thanks
<james_w> abentley: yeah, that would be a good idea
<lifeless> pulling it
<lifeless> matsubara: was that in README.txt and I was simply blind ?
<abentley> james_w, in your bug, you seem inconsistent about whether the component should be included.
<matsubara> lifeless, yep, it's explained there and the make command should fetch the right cache. not sure why it didn't work for you
<james_w> abentley: bug number?
<abentley> james_w, 603606
<abentley> james_w, or what is RELEASE?  the pocket?
<james_w> abentley: pocket, yes
<abentley> james_w, and that's not listed in binary build email body, just subject.  I see.
<james_w> abentley: the pocket will be important once it is possible to target non-release pockets, but for now there is only one choice, so I say put it in there
<lifeless> matsubara: my bad!
<james_w> abentley: I'm not particularly keen on the binary build email, but it's better than what was there for recipes IMO, and I think they should at least be largely consistent. If you want to improve both, that would be even better
<abentley> james_w, what about distroseries?
<james_w> abentley: that's a requirement IMO, it's especially important with multi-series builds.
<abentley> james_w, err, we don't have multi-series builds.
<james_w> well, one recipe being dispatched to multiple series at the same time
<james_w> if I wake up to a daily build failure, I want to be able to work out which series failed, without having to query them each in turn
<abentley> james_w, well, you can click though and get all the information, and more.
<abentley> james_w, but you didn't ask for distroseries.
<abentley> james_w, do you prefer pocket and distroseries separate, or do you want "suite"?
<james_w> abentley: the series is in the subject as well
<james_w> abentley: I don't mind
<james_w> I would prefer it was the same as binary builds
<abentley> james_w, binary builds don't list the pocket or distroseries in the message body at all.
<james_w> indeed
<abentley> james_w, so you're saying it shouldn't be included?
<james_w> abentley: I want it there somewhere. I don't have particularly strong preferences on where.
<james_w> abentley: are you redesigning binary build failure messages too?
<lifeless> matsubara: now it wants distribute 0.6.12
<abentley> james_w, no, not my domain.
<lifeless> matsubara: with that new cache
<james_w> abentley: in that case, I would prefer that the recipe build failures just follow the binary build failures.
<abentley> james_w, seems like a lowest-common-denominator approach to me.
<james_w> abentley: why?
<abentley> james_w, because you don't especially like binary build mails, but you want me to make source build mails just as bad.
<james_w> abentley: they are better than what we currently have for recipes, and I value consistency. If you see ways to improve the binary build mails then I encourage you to improve those as well.
<abentley> james_w, so you're basically asking me to make recipe build mails no better than binary build mails.
<james_w> abentley: I'm asking you to make them equally as good, yes. I'd love for you to make them both excellent.
<abentley> james_w, It's not my job to make binary build mails better, but perhaps if I make recipe build mails better, soyuz will see and follow suit.
<james_w> abentley: I don't see why it isn't your job
<abentley> james_w, I am a member of the code team, not soyuz.
<james_w> and I don't think that is an approach that has worked particularly well in the past
<matsubara> lifeless, hmm I think I saw that distribute problem before with gary_poster and was unable to reproduce locally. would you mind filing a bug on oops-tools so I can investigate this further later on?
<abentley> james_w, I'm not going to go trampling though another team's code for no especially good reason.  That will just upset them.
<james_w> abentley: I think that characterisation isn't helpful. I would imagine the soyuz team would be overjoyed at someone looking at this area of the project and improving it.
<james_w> I doubt anyone is particularly attached to the status quo
<james_w> fwiw, upload acceptance emails were redesigned about a year ago, with great success. They are now for more readable and usable than before.
<abentley> james_w, improvement is very much in the eye of the beholder.  I would be very upset if someone made this kind of change to my code without consulting me.
<lifeless> matsubara: sure
<james_w> abentley: then consult them?
<bigjools> abentley: BFNs are crap, fee l free to fix them :)
<abentley> james_w, it's simpler to just ask them to do it instead.  They have the experience with the code and the domain knowledge to do a better job.
<mars> lifeless, ping, any chance you could point to some example code that uses testtools.clone_test_with_new_id() ?
<lifeless> testscenarios
<lifeless> (apt-get install python-testscenarios, or grab it from pypi, or lp:testscenarios)
<mars> thanks, looking at the code now
<lifeless> mars: start with the readme
<mars> lifeless, hmm, I'm getting no text
<lifeless> what do you mean ?
<mars> http://bazaar.launchpad.net/~lifeless/testscenarios/trunk/annotate/head:/doc/example.py?file_id=lib-20090307095458-ntl18b730pwjf0qy-10 is empty
<mars> for instance
<mars> does that mean codebrowse is going to die? :(
<lifeless> I don't know what it means
<lifeless> http://bazaar.launchpad.net/~lifeless/testscenarios/trunk/revision/1#doc/example.py
<lifeless> clicking on doc/example.py there worked
<mars> lifeless, neat, that is a useful tool.  Parameterization made easy
<lifeless> :)
<mwhudson> morninh
<mtaylor> morning lifeless
<lifeless> hiya
<bdmurray> I just got oops-1670ea4372 when reporting a bug and the traceback did not look to my untrained eye
<lifeless> look what?
<bdmurray> look good?
<lifeless> https://lp-oops.canonical.com/oops.py/?oopsid=1670EA4372 looks sane (but sad) to me. otp for now, chat in a sec
<lifeless> bdmurray: ok, off the call
<lifeless> bdmurray: that timeout looks legit
 * wgrant looks at the build queues with dismay.
<wgrant> barry!!!!!!!!
<wgrant> Oh, no, it's not all py27stack4
<wgrant> Then what is it...
<wgrant> .....
<wgrant> py27stack5.
<jelmer> 'evening
<wgrant> Can we at lest score that archive down a few hundred points, or something like that?
<mwhudson> jelmer: i wanted to ask you something a minute ago but now can't remember what :-)
<poolie> hi lifeless, wgrant, mwhudson
<poolie> thanks for the review lifeless
<wgrant> Morning poolie.
<lifeless> poolie: my pleasure
<poolie> how should i write tests for the page macros?
<poolie> preferably not as doctests
<lifeless> you get a browser instance
<lifeless> and use some template that uses the macros so you get a rendered view
 * wgrant whispers 'really bad user experience', and points at #launchpad and the build queue which is currently being eaten by an unimportant rebuild.
<lifeless> wgrant: the magic phrase is 'losa ping'
<wgrant> Well, no, there are underlying issues.
<poolie> wgrant: i was thinking about proposing an "i care about this" button that can only be pressed by a human (not in the api) that adds X to the build score
<poolie> this will of course only work if people don't press it for everything
<wgrant> The problem at the moment is that people do partial archive rebuilds without telling anyone, so they are scored like normal PPA builds.
<wgrant> (well, that shouldn't be a problem, except our queue discipline is pathetic)
<lifeless> perhaps subtract the queue length for a PPA from uploads to the PPA
<bigjools> who is rebuilding?
<wgrant> barry.
<wgrant> I thought it was py27stack4.
 * bigjools scores it down
<wgrant> But I was wrong; there's only a few there.
<wgrant> Now it's py27stack5.
<bigjools> I asked him to use rebuild archives
<wgrant> If he doesn't want to do that, he could at least get someone to add a score delta to the archive first.
<wgrant> Scoring them down to <900 might be nice, otherwise recipe builds won't happen.
<lifeless> whats a rebuild archive ?
<bigjools> I scored everything -100
<wgrant> Haha. Even better.
<bigjools> that'll free the farm up
<wgrant> Thanks.
<bigjools> when I say -100 I mean a delta
<bigjools> not absolute :)
<wgrant> lifeless: ArchivePurpose.COPY. Doesn't accept source uploads, and has a forced build score of -10.
<wgrant> Oh :(
<wgrant> So recipe builds are screwed for the next couple of days. I guess that's not tooo bad.
<bigjools> hmmm
<wgrant> Also, will that affect existing builds?
<bigjools> yes
<bigjools> anything needsbuild
<wgrant> But I thought queuebuilder didn't run any more.
<bigjools> oh arse
<bigjools> I need to manually rescore them
<bigjools> well, it's only a 4 hour wait
<wgrant> Assuming perfect efficiency.
<bigjools> if it were days...
<wgrant> It's lots of short builds.
<wgrant> True.
<wgrant> But it'll be more like 24 hours.
<wgrant> Once you take into account the continuing pileup of builds, and the fact that they're short builds, which buildd-manager really doesn't handle well.
<bigjools> it will, soon
<wgrant> Indeed.
<wgrant> That will be good.
<bigjools> I should fix that score delta stuff so it applies the delta on the query
<bigjools> then it'll work on existing builds
<wgrant> That sounds confusing.
<wgrant> But maybe better.
 * bigjools shoos a fox away
<bigjools> anyway it's seriously antisocial to upload 500+ packages for rebuild in a normal PPA
<bigjools> g'night
<wgrant> Night.
<poolie> are developer docs in the tree ever built into web pages people can read?
<poolie> like for bzr's?
<lifeless> there is apidoc.launchpad.dev now
<wgrant> How does one create code imports these days?
<wgrant> I can't see a way to do it outside +setbranch.
<jelmer> wgrant, some project code pages don't have a link, but https://code.launchpad.net/ has a link to create imports for all projects
<wgrant> Ah, +new-import
<wgrant> I was using +newimport :(
<thumper> the theory was...
<thumper> that we wouldn't show "new import" for a project that uses launchpad codehosting
<thumper> however it seems its all gone to pot
<wgrant> It seems to just not show the link if there is a dev focus.
<wgrant> AFAICT.
<jelmer> thumper: I can think of some (pehaps not common, but possible) situations where an import would be useful even if the trunk is hosted on Launchpad.
<thumper> jelmer: like?
<jelmer> thumper: dulwich is maintained in bzr, but I keep a mirror of it in git and some external contributors host their branches on github
<jelmer> (I did emphasize it wasn't a common situation :-)
#launchpad-dev 2010-07-29
<thumper> FFS
<thumper> devel build failed
<wgrant> thumper: Doesn't he probably not want a shared repo?
<wgrant> Sounds like he might just want a branch.
 * thumper nods
<mtaylor> w00t!
<mtaylor> don't know who fixed the swift mailing list situation ... but thanks
<mtaylor> sinzui: was that you?
<sinzui> mtaylor, me and Chex
<sinzui> I am waiting on my test email to complete
<mtaylor> sinzui: well thank you to both of you
<Chex> mtaylor: spm did a lion share of work previously, too.
 * mtaylor hands Chex, sinzui  and spm beers
<spm> mtaylor: heh, glad it's sorted.
<Chex> mtaylor: glad to help, sinzui did a lot of the work today, I just saw the outstanding answer and jumped in to move it along
<mtaylor> spm: now we've just got that can't see launchpad.net/swift thing left, but I think bac may have figured that one out and filed a bug
<spm> mtaylor: that's not because I had a hissy fit of frustration the other day and dropped those tables  from the DB?
<mtaylor> it's almost like I'm going to have nothing more to bug anyone aboud
<mtaylor> about
<mtaylor> spm: hehe
<sinzui> I have not gotten a copy of my test email, and I do not see the archive
<sinzui> mtaylor, do you see a moderation link on https://edge.launchpad.net/~swift to verify my missing message is stuck?
<mtaylor> sinzui: I see a moderation link, but I see no messages in the queue
<sinzui> :(
<sinzui> I think the new list was not fully created because of the stalled proc and our deft axe-like hacks to make this work
<sinzui> I propose a rapid deactivate, purge, and request of a new list. I think this will take a total of 10 minutes
<mtaylor> sinzui: do you need me to do that? or was that a back-end operation you're suggesting?
<sinzui> I just did it all
<sinzui> I am sending another test
<sinzui> I see an archive before I even start this time :)
<sinzui> Sweet
<sinzui> mtaylor. I think you have a list. And I just learned the minimum total tear down and setup of a list in the ui is 3 minutes
<mtaylor> sinzui: w00t!
 * thumper lunchinates
<poolie> thumper, spm, do you want a hand with bug 610297
<_mup_> Bug #610297: stable codeline is failing to build - compile error <canonical-losa-lp> <Launchpad Foundations:Fix Released by jelmer> <https://launchpad.net/bugs/610297>
<poolie> ah apparently not
<spm> :-)
<jelmer> 'morning poolie, spm
<jelmer> poolie: Did you have a good flight back to Sydney?
<spm> hey jelmer! how goes!
<jelmer> Very well, thanks! :-)
<poolie> hi jelmer, it was ok
<lifeless> bdmurray: https://bugs.edge.launchpad.net/malone/+bug/611115
<_mup_> Bug #611115: timeout: bug notifications are calculated in-request <timeout> <Launchpad Bugs:Triaged> <https://launchpad.net/bugs/611115>
<bac> mtaylor: the fix is in
<spm> bac: that's a really easily misconstrued statement. just sayin'. ;-)
<bac> spm: really?
<bac> spm: you employing some aussie linguistic jujitsu?
<lifeless> only for australians
<spm> bac: hrm. possibly local slang? 'the fix is in' is another way of saying 'we've done some illegal stuff to achieve X'
<lifeless> spm: its aussie vernacular
<bac> spm: ok.  well rest assured it was all legal, reviewed, and landing soon
<spm> heh
<bac> and look, you ran off wgrant.  launchpad productivity plummets
<spm> on the grounds that he's only 7-8 hours drive away; i make no comment in case he's the revenge seeking type
<lifeless> e.g. australian ?
<lifeless> :P
<spm> ....
<spm> ha. good def'n here of 'the fix is in' http://answers.google.com/answers/threadview?id=65441 fwiw
<lifeless> ah right
<lifeless> so less aussie than I thought
<wgrant_> bac: That was actually only one of me.
<wgrant_> (my server at home apparently reset itself just as I was SSHing in... this is concerning)
<wgrant> Is devel broken at the moment?
<wgrant> I have a test failure which seems to be caused by the registration slot changes.
<thumper> wgrant: it is possible
<thumper> wgrant: there have been many commits since the last successful run
<wgrant> Eep.
<wgrant> But the registration slot is on edge...
<lifeless> this is odd
<wgrant> xx-copy-packages.txt is the failure that I see.
<mwhudson> wgrant: should the fact that there are 13 idle i386 builders be bothering me?
<mwhudson> ppa builders, that is
<wgrant> Maybe.
<wgrant> If it's for more than a few minutes.
 * wgrant watches.
<mwhudson> ah a bunch just got dispatched
<wgrant> You just caught it at a bad time, yeah.
<mwhudson> so it was probably just process-upload-ing openoffice or something
<wgrant> Heh.
<wgrant> Each p-u invocation takes like 15 seconds.
<wgrant> So if it is processing a couple of dozen uploads, it can take a while to get around to the next cycle.
<wgrant> Even if those uploads are small.
<mtaylor> rockstar: around?
<mtaylor> so - now I'm seeing work-in-progress merge requests on the +activereviews page ...
<mtaylor> oh. even weirder
<mtaylor> +activereviews for the branch shows work-in-progress - +activereviews for the project does _not_ show work in progress
<mtaylor> just thought I'd point that out in case no body noticed
<thumper> lifeless: quick question
<lifeless> shoot
<thumper> mtaylor: yes it does
<thumper> mtaylor: by design
<thumper> lifeless: the librarian bug you're fixing, will this stop the timeouts we are seeing on merge proposal pages?
<thumper> mtaylor: work in progress is of interest in the personal namespace
<lifeless> maybe. Point me at such a timeout OOPS
<thumper> mtaylor: branches are in the personal namespace
<thumper> lifeless: they are in the daily oops reports
 * thumper has a quick look
<thumper> lifeless: OOPS-1670O1242
 * thumper looks at _mup_
<lifeless> works in #launchpad :P
<lifeless> yes, I would expect that one to be fixed
<lifeless> we'll allocate a token
<thumper> \o/
<lifeless> and the client will grab the content directly from the librarian
<thumper> um...
<thumper> that'll require a coding change won't it?
<thumper> it isn't a transparent fix
<thumper> because we format the diff
<thumper> the formatted diff isn't stored
<lifeless> ok
<lifeless> so no, that one won't be fixed.
<thumper> :(
 * thumper afk
<lifeless> Heres what will be fixed - places where we just proxy the content to the client.
<lifeless> like private build logs
<lifeless> thumper: the separate 'client does not time out fast' bug will help with those pages though - but it will just mean they fail fast rather than fail slow.
<wgrant> lifeless: Hum, so the restricted librarian is going to visible externally?
<wgrant> +be
<lifeless> wgrant: not precisely
<lifeless> wgrant: we'll accept requests on the public librarian for secured content if there is a Front-end-https: on header and it has an access token that is live in the session db
<lifeless> wgrant: see my WIP branch / bug discussion
<wgrant> Ah, sounds reasonable.
<wgrant> Well.
<wgrant> Sort of reasonable.
<wgrant> As long as you use the right Content-Disposition...
<lifeless> can you suggeset some improvements ?
<lifeless> How is c-d different for this case than other librarian objects?
<wgrant> The public librarian doesn't use Content-Disposition: attachment.
<wgrant> So JS is executed.
<lifeless> right
<wgrant> So if you serve anything from the HTTPS librarian without Content-Disposition: attachment, you're fucked, because anyone can steal your access tokens.
<lifeless> how is this *different* in terms of requirements/risks/constraints ?
<lifeless> no
<lifeless> the token is per url
<lifeless> we can't use the oauth tokens for the reason you give
<wgrant> Relying on cross-window security seems a bit odd.
<lifeless> even if we set c-d they might exploit a browser bug etc
<wgrant> They might, yes, but then the entire Internet is fucked. Not just us.
<lifeless> wgrant: so still, that seems like an argument for setting it on every librarian file
<lifeless> how is it unique for secured content
<wgrant> lifeless: Because there's nothing to be stolen on the public librarian.
<wgrant> That's why it lives on launchpadlibrarian.net nowadays: so there's no cookie.
<lifeless> the secured still will also live there
<lifeless> and also have no cookie
<wgrant> But there is private information.
<wgrant> In the URLs.
<lifeless> go on
<lifeless> I'm moderately fucked from jetlag, so you're going to have to treat me like a 2-year old and spell it out :)
<wgrant> I'm aware of strong cross-domain security rules.
<wgrant> I don't know about cross-window restrictions, though.
<lifeless> what do you mean cross-window restrictions
<wgrant> The security of your system relies on the fact that a page executing in the context of a domain cannot determine the URLs of any other windows open for that domain.
<wgrant> I do not know that to be the case.
<lifeless> or of urls in the history
<wgrant> That too.
<lifeless> so if you're saying 'please set cd:attachment on secured urls' - sure, I'm fine doing that, I don't see that we would want js scripts in the librarian anyhow for secured content
<lifeless> but it seems to me that we're already at risk from such things - browser history containing product urls etc which are private
<wgrant> Nobody else has managed to do that without being harrassed for days.
<lifeless> please expand
<wgrant> U1 was pretty badly broken in this respect.
<wgrant> Took a bit of poking to get it fixed.
<wgrant> But I don't see your point about private URLs.
<lifeless> I don't know what 'that' or 'this' is bound to.
<wgrant> The serving user web pages on authenticated domains thing.
<wgrant> The only untrusted content that LP currently serves up is on ppa.launchpad.net and launchpadlibrarian.net.
<wgrant> ppa.launchpad.net is HTTP, so can't access cookies.
<wgrant> launchpadlibrarian.net is on another domain, and contains no private content, so it can't access cookies or private content.
<lifeless> right
<lifeless> now I'm proposing to add secured content to ll.net by using the url to carry the token
<lifeless> it can't access cookies - wrong domain
<wgrant> But it's not clear if it can access other windows or history.
<lifeless> right
<lifeless> if we add c-d to the headers, will that break things like emblems, if they are secured ?
<mtaylor> thumper: interesting
<wgrant> lifeless: Not sure.
<lifeless> wgrant: so, I'm not clear if you're saying 'stop this the plan is flawed', or 'the plan needs tweaking', or 'this axiom needs researching'
<lifeless> wgrant: perhaps you're being clear and I'm just too freakishly tired
<wgrant> lifeless: Probably "this axiom needs researching"
<lifeless> wgrant: ok, cool - I can keep plodding forward.
<lifeless> wgrant: if you wanted to research a little, to parallelise efforts, that would be rad.
<lifeless> wgrant: or, if there is a good mitigation strategy we can use regardless, lets do that.
<wgrant> lifeless: Content-Disposition is the way to do it, I believe.
<lifeless> we could, in principle, wire dns up to the librarian
<lifeless> and do hash.launchpadlibrarian.net
<lifeless> but that seems a little overkill
<lifeless> wgrant: do you think it would be safe to whitelist file types like gif, svg, txt etc ?
<wgrant> lifeless: IE does some pretty strange stuff. So you'd have to be careful.
<lifeless> so the vector you're describing is:
<lifeless> Attacker puts a bad file in the librarian
<lifeless> Victim looks at that file, and has either in that windows history, or in another window, the url for a secured file on the librarian
<lifeless> Attackers script downloads the secured file (because its within the timeout/use count for the token) and sends it off (how?) to another site
<wgrant> It's more practical to just transmit the URL.
<wgrant> Either through redirection, or a form post.
<lifeless> so they set self.url = badsite.net/?location=secured_content_url
<wgrant> window.location, but yes, exactly.
<lifeless> blah, self.location =  ...
<lifeless> righto
<lifeless> one thing we can do
<lifeless> is look for a referrer which must exactly match the canonical url in LP
<lifeless> [note that this is hard today because we don't have a backlink on LFA, I'm speculating]
<lifeless> gesturing... handwaving, thats what I mean, handwaving.
<wgrant> If you add more referrer sniffing, I will do bad things.
<lifeless> heh
<wgrant> (yes, I did suggest it for the CSRF fix, but only because I knew the hole had no chance of being fixed within years otherwise)
<lifeless> now, the referrer may be no more secure
<wgrant> I'd hoped that the user complaints would force Foundations to fix it properly.
<wgrant> But I was wrong.
<lifeless> so that might be pointless
<lifeless> c-d will break inline display of diffs, logs, etc - which is annoying.
<lifeless> security over pretty though
<wgrant> Right.
<lifeless> theres been some content-sniffed spec stuff recently
<lifeless> probably want to review that
<wgrant> Hmm.
<wgrant> I wish things like getUtility(IComponentSet)['main'] would cache.
<lifeless> ugh
<wgrant> Hm?
<lifeless> I wish such things didn't really exist. Or something.
<wgrant> Well, yes.
<wgrant> I'm currently trying to refactor some publication stuff.
<wgrant> But I can't do it really cleanly, since there were apparently timeouts caused by too many getUtility(IComponentSet)['main'] calls.
<lifeless> heh
<lifeless> I hear objects provide a nice context for remembering things
<wgrant> I guess we could just assume it never needs to be invalidated.
<wgrant> Ugly and wrong, but it would never be a problem ever.
<wgrant> And then my code can be all nice and clean.
<wgrant> lifeless: Am I allowed to do that?
<lifeless> do what
<wgrant> Make it cache lookups, even though someone could in theory delete and recreate the component.
<lifeless> within a transaction, yes.
<lifeless> as long as a delete & recreate in the transaction will be handled
<wgrant> Is there a way to do that?
<lifeless> have you considered providing a stateful proxy in your code
<lifeless> wgrant: you can hook into certain events in storm, but I think it would be ugly.
<wgrant> It would be very ugly, yes.
<wgrant> Especially for a situation that will never occur.
<lifeless> never is a long time
<lifeless> have you considered providing a stateful proxy in your code?
<wgrant> How do you mean?
<lifeless> class mycomponentset: def get_component(self, component_name): if component_name not in self.components: self.components[component_name] = getUtility(IComponentSet)[component_name]; reuturn self.components[component_name]
<wgrant> I was hoping to be able to do that in ComponentSet itself.
<lifeless> the context for such a cache is ill defined at the component level
<lifeless> AIUI
<wgrant> Well. Components are currently immutable.
<wgrant> Like SourcePackageNames and BinaryPackageNames.
<wgrant> They are just a string.
<lifeless> update on the name is prevented ?
<wgrant> No. But it's never done, because it would break the world.
<lifeless> as long as you really clearly document it then
<wgrant> And while it's *possible*, it seems bad to make the code significantly uglier to take into account that case.
<lifeless> be sure not to leak objects across transactions
<lifeless> you'll want to hook into end request to clear your cache
<wgrant> Yeah, that's the bit I'm not quite sure about.
<lifeless> and if you're used outside web contexts you'll need to provide an API for clearing the cache and audit existing code to comply
<lifeless> global caches are a pain
<lifeless> for an example of hooking into end request see profile or feature flags
<spiv> I'm slightly surprised there isn't some sort of @cacheduringtransaction decorator.
<wgrant> It would be handy.
<spiv> I suppose the tricky bit is telling the decorator which store to use.
<spiv> (There is @cachedproperty, but it caches indefinitely, although you could invalidate it by hand fairly easily...)
<wgrant> Right, but it doesn't help for attribute accesses.
<wgrant> Ah.
<wgrant> I could use a celebrity.
<lifeless> really?
<wgrant> That sounds evil, but they have the rules that I want.
<spiv> That's true.
<lifeless> cause I need a hero
<StevenK> lifeless: Why? When will you return the last one you borrowed?
<lifeless> :)
<lifeless> wgrant: http://code.google.com/p/browsersec/wiki/Part2#Same-origin_policy - 'The behavior for URLs that do not meaningfully have a host name associated with them (e.g., file://) is not defined, causing some browsers to permit locally saved files to access every document on the disk or on the web;he behavior for URLs that do not meaningfully have a host name associated with them (e.g., file://) is not defined, causing some browsers
<lifeless> bwah c&p fail
<spm> lifeless: is np. simply went intot the tl;dr bucket anyway ;-)
<mwhudson> lifeless: that's a fairly depressing document
<lifeless> yes
<mwhudson> A long-standing security flaw in Microsoft Internet Explorer 6 permits stray newline characters to appear in some XMLHttpRequest fields, permitting arbitrary headers (such as Host) to be injected into outgoing requests. This behavior needs to be accounted for in any scenarios where a considerable population of legacy MSIE6 users is expected.
<mwhudson> aaaaaaaaaaa
<lifeless> wgrant: see 'Gaps in DOM access control' on that page
<lifeless> wgrant: so interestingly, *any* page can get any url
<lifeless> wgrant: for many browsers; so urls that we consider private, like private team names, are obtainable (though requires both guessing and a named window AFAICT)
<mtaylor> " where a considerable population of legacy MSIE6 users is expected. " ... that give me nightmares
<wgrant> mtaylor: That sadly accounts for a significant portion of Launchpad's privacy userbase, I believe.
<lifeless> rumoured, not confirmed
<adeuring> good morning
<mrevell> Hello
<poolie> hello europa
<lifeless> thats a whole new tz :)
<poolie> you still here?
<poolie> what's the cleanest way to get my docs into something visible on the web?
<poolie> put them in a module docstring?
<wgrant> Probably.
<lifeless> yeah, for now, docstrings
<wgrant> Can someone else run xx-copy-packages.txt? It fails for me in devel.
<wgrant> But stable is up to date.
<wgrant> Even though the topic says buildbot is broken.
<wgrant> I am confuse.
<bigjools> it failed for me too
<bigjools> meant to mention that 2 days ago
<wgrant> So... buildbot is fail-deadly?
* jml changed the topic of #launchpad-dev to: Launchpad Development Channel | Week 2  of 10.08 | PQM is open | https://dev.launchpad.net/ | Get the code: https://dev.launchpad.net/Getting | On-call review in irc://irc.freenode.net/#launchpad-reviews | Use http://paste.ubuntu.com/ for pastes
<lifeless> jml: can we keep the bb summary please, so its obvious where to put updates to it?
* jml changed the topic of #launchpad-dev to: Launchpad Development Channel | Week 2  of 10.08 | PQM is open | firefighting: - | https://dev.launchpad.net/ | Get the code: https://dev.launchpad.net/Getting | On-call review in irc://irc.freenode.net/#launchpad-reviews | Use http://paste.ubuntu.com/ for pastes
<wgrant> bigjools: Hi.
<bigjools> hello
<wgrant> bigjools: I'm currently working on DDEB copies. The binary copy logic is currently in PublishingSet.copyBinariesTo rather than BPPH.copyTo, originally for two performance improvements.
<wgrant> One of those was to retrieve the insecure records all in one go. That's clearly no longer an issue.
<wgrant> The other is to only issue one query for the PPA component override (since getUtility(IComponentSet)['main'] doesn't cache).
<wgrant> It seems like that second optimisation could be implemented a lot better by making ComponentSet cache responses.
<bigjools> yeah I agree
<bigjools> components simply don't change much
<wgrant> Exactly. They *could*, but they never will.
<bigjools> for Ubuntu indeed,  but when we start hosting more distros, they might
<wgrant> Well, in that case some will be added.
<wgrant> But existing ones will never change or be removed...
<bigjools> yep
<wgrant> So it seems reasonable to have ComponentSet just cache positive responses within a transaction.
<bigjools> +1
<wgrant> Great.
<wgrant> Not sure how best to go about it, though... unless I go the celebrity route and store the ID at first request, and then rely on Storm's cache.
<wgrant> I'm not sure there are any other utilities that do a similar thing.
<stub> jtv: What happened to the work to drop BranchRevision.id ?
<jtv> stub: John went on with that...  I pushed him some to keep going.
<jtv> We landed the stormification of the queries.
<stub> Cool. I'm getting concerned meeps from the losas about disk space, and that should reclaim maybe 15GB
<jtv> psiew
<jtv> (that's a sound, not an acronym)
 * wgrant blinks.
<wgrant> 15GB from removing the integer primary key of that table?
<jtv> jam: see above... getting rid of BranchRevision.id is increasingly urgent.
<stub> The index itself is about 12.9GB, plus id storage, minus bloat = <mumble>
<wgrant> Ah.
<jtv> stub: if this is an append-only table as it seems to be, I'm not expecting much bloat that vacuums wouldn't keep under control.
<stub> yer, there isn't
<jtv> The id storage is presumably 4GB for every G rows.  There's also another index that contains id, so add that.  Yeah, this would be a nice cleanup.
<jtv> stub: I'd say the savings would be closer to 20GB.
<jml> today is not working out well for me
<jml> I may have to edit Python code to soothe my frazzled nerves.
<deryck> Hi, all.
<jelmer> 'morning Deryck
<jml> lifeless, ping
<jml> gary_poster, hello
<jml> gary_poster, I have an idea lurking in the back of my mind to move some of the more strategic docs that I'm writing to the Launchpad tree, as well as some of the developer documentation
<jml> gary_poster, but I think I'd only want to do that if the docs were also browseable on the web
<jml> gary_poster, technology-wise, should I be using the apidoc book for this, or some kind of sphinx thing?
<gary_poster> jml, I'll summarize pros and cons in 15
<jml> gary_poster, thanks
<jam> hi jml
<jam> lifeless likely won't be on for another 4-6hours
<gary_poster> jml: Sphinx: great if you need links between pages or more sophisticated rendering possibilities--closer to a website or a book.  apidoc: great if you need quick ReST processing that's integrated with something we already have, and can be loosely organized into a tree.
<gary_poster> downside of both: getting them on the web is a manual process, and likely to stay that way at least for awhile.  (I was vaguely imagining a buildbot/hudson thing that would do this eventually for the apidoc.)  It's very slow for the computer to generate the static apidoc and upload it, but no big deal for the person pushing the keys.
<gary_poster> For the kind of docs I *expect* you'll be writing, I'd lean towards apidoc, especially if I can shunt off the extraneous stuff (like ZODB) so the docs valuable to us are easier to see
<jml> jam: thanks.
<deryck> adeuring, hi.  So I'm trying to catch up here on the proxied (or not) factory objects discussion.  Do we have resolution on that?  i.e. are you still working on it, or has the work been abandoned now?
<gary_poster> I need to wade through that discussion :-/
<jml> gary_poster, thank you, that's helpful.
<adeuring> deryck: i've abandoned it for now. it is easy to pick up once we know how to proceed with (un)proxied objects from the factroy
<gary_poster> cool np
<jml> gary_poster, although you missed that sphinx looks prettier than apidoc :)
<gary_poster> jml: very true :-)
<deryck> adeuring, is bdmurray unblocked on the work he wants to do?  Or has a work around at least for the test problem he had?
<adeuring> deryck: he's making some progress. sorry forgot the details
<deryck> adeuring, so the card for you and bug 610000 is abandoned?
<_mup_> Bug #610000: add security proxies to the objects returned by LaunchpadObjectFactory.makeMergeDirective() and LaunchpadObjectFactory.makeMilestone()  <Launchpad Foundations:In Progress by adeuring> <https://launchpad.net/bugs/610000>
<adeuring> right
<deryck> adeuring, can you move it to trash on the board, and add a comment why you're abandoning the work?
<adeuring> ok, will do
<adeuring> deryck: do we have a trash lane on the kanban board?
<deryck> adeuring, yeah, if you right click the card, move to --> archive --> trash.  You'll need to then expand archive to find it and comment, or else comment before you move it.
<adeuring> deryck: thanks, found it now
<deryck> cool
<deryck> *sigh* can't even spell my own name today
<mtaylor> gary_poster: fwiw, I've got a hudson job that build sphinx doc for our openstack projects and publishes them to web on every trunk push
<gary_poster> mtaylor: sweet, sounds nice :-)
<mtaylor> gary_poster: yes. I gives me pleasure
<mtaylor> it
<mtaylor> not I
<mtaylor> gah
<gary_poster> lol
<deryck> gary_poster, hi.  Do you have 5 minutes to chat?
<gary_poster> deryck, on call, and booked.  in hour, maybe?
<deryck> gary_poster, sure.  sounds good.  No hurry.
<deryck> Just trying to figure out why my changes last week caused test failures and had to be reverted, and I think you might have some idea. :)
<gary_poster> heh ok :-)
<bdmurray> I've landed a branch on edge to cache a bug target's bug tags portlet but I'm not seeing cache comment in the source for the page
<bdmurray> gary_poster: is there somebody who could help me sort this out?
<jml> bdmurray, are you sure it's been rolled out?
<bdmurray> jml: pretty sure - it is revision number 11195
<bdmurray> speaking of it'd be neat if 'r11249' linked to the code branch
<jml> that does give you a certain ground for confidence
<jml> bdmurray, yeah, that would be neat.
 * jml hacks a little
<rockstar> bigjools, ping
<bigjools> rockstar: is it quick, I'm about to EOD?
<rockstar> bigjools, just wondering if there's a factory method for simulating the creation of all objects that would be created to have a package in a PPA.
<bigjools> rockstar: yes, the SoyuzTestPublisher
<rockstar> bigjools, okay, I'll take a look at that.
<rockstar> bigjools, have a good night.
<jml> bdmurray, reckon it should link to http://bazaar.launchpad.net/~launchpad-pqm/launchpad/stable/files/11249 ?
<bigjools> rockstar: there's a getPubSource() which creates a package and publishes it
<bigjools> rockstar: there's a few gotchas with it, see some example tests first (grep is your fwiend)
<gary_poster> bdmurray: code only adds comments "if not config.launchpad.is_lpnet and not config.launchpad.is_edge:"
<rockstar> bigjools, okay.  I need to allow that to grow the ability of using a sourcepackagerecipe to do so.
<bigjools> rockstar: sounds great
<gary_poster> bdmurray: so it would be visible in staging...if staging were up.
<bigjools> good night everyone
<gary_poster> night
<gary_poster> (on behalf of everyone)
<bdmurray> jml: Personally I'd want to see https://code.edge.launchpad.net/~launchpad-pqm/launchpad/devel so I could see all the recent revisions
<bdmurray> jml: to know what change recently not just the latest revision
<jml> *nod*
<bdmurray> gary_poster: ah, that helps thanks!  is there an eta for staging being up?
<gary_poster> deryck[lunch]: if you were not lunching I would have five minutes to talk :-)
<jml> except I'll do stable, not devel.
<gary_poster> bdmurray: losas are firefighting to get restore working.  dunno when that will be back, but they are on it.
<mthaddon> staging is up, it's just the update is busted
<jml> bdmurray, https://code.edge.launchpad.net/~jml/launchpad/link-to-lp-revision/+merge/31310 fwiw
<bdmurray> jml: cool!
<gary_poster> bdmurray: staging is on r9589, which == r 11252 of edge.  should be plenty for r11195.
<gary_poster> thanks for the clarification mthaddon
<mthaddon> gary_poster: actually, it's on 9565 :(
<gary_poster> mthaddon: oh, the footer on https://staging.launchpad.net/ lies? :-(
<mthaddon> gary_poster: it's displaying the wrong revno because I did a "bzr revert -r 9565"
<gary_poster> ah!
<mthaddon> gary_poster: what would I need to do to get it to display the right revno?
<gary_poster> bdmurray: you still might be in luck.  db-stable r9565 includes stable r11195, just barely
<bdmurray> gary_poster: well the pages I'd like to view are oops'ing on me :-(
<gary_poster> bdmurray: :-(
<gary_poster> mthaddon: I *think* the number comes from bzr-version-info.py, in which case ``rm bzr-version-info.py && make bzr-version-info.py`` and a restart ought to do the trick.
<gary_poster> need to go on call
<jml> mthaddon, scripts/update-bzr-version-info.sh
<maxb> Ugh. Ugh ugh ugh. Apparently the validation on code import creation is more lax than that of code import editing, so I can't mark https://code.edge.launchpad.net/~registry/tidy/trunk as invalid without getting told to fix the cvsroot
<sinzui> maxb, yes, I was suspicious of that when I registered it
<sinzui> I was going to track that down later today
<maxb> oh *you* registered it? You know it's totally bogus, right?
<sinzui> Yes. I was shocked that it went through
<maxb> Mind if I write "sinzui will delete this when done debugging it" in the branch whiteboard?
<sinzui> please do it
<deryck> gary_poster, thanks anyway. Sorry I wasn't around.  I'll just catch lifeless later when he's on, or get you tomorrow about it.
<gary_poster> deryck: cool
<sinzui> maxb, +setbranch on a series will not accept a valid cvs url. It *requires* me to use http or https. I will report the bug
 * jml frowns
<jml> mars, hi
<mars> jml, on a call, will be available soon
<jml> mars, ta
<jelmer> sinzui: There is an open bug about that somewhere
<jelmer> sinzui: I hit it too a while ago.
<sinzui> I could not find the bug, I hope thumper can dup it
<jml> jam: your branch https://code.edge.launchpad.net/~jameinel/launchpad/lp-serve-cleanup/+merge/30542 is approved to land. want me to land it for you?
<mars> jml, I'm free now, what's up?
<jml> mars, just going through my outstanding lp branches and saw different-ec2-mail
<jml> mars, are you going to carry on with it or shall I?
<mars> jml, yeah, we just discussed that - if possible, I need to work on merge workflow before I go on holidays
<mars> If you are able to pick it up, that would help
<jml> mars, ok, I'll do that then.
<jml> mars, Waste sucks.
<lifeless> mornink
<mars> jml, yes :(   But it's not waste yet!
<lifeless> deryck: hi
<mars> just a little stale, no green fuzz
<jml> lifeless, hi
<lifeless> hi jml
<deryck> lifeless, hi.
<jml> mars, it's wasteful in the lean sense right? unlanded branches == inventory; time between starting & finishing that doesn't progress toward goal == waste
<deryck> lifeless, I really want to chat about why exactly my work caused the db-devel test run to fail, given it passed two runs already.  But I'm brain dead from clearing email backlog and at my EOD.
<mars> jml, true.  It got stuck in a queue here - by that count, I had a bunch of other stuff that needed to land first
<lifeless> deryck: AFAIK no one has dug deep enough to answer
<jml> mars, oh yeah, understood.
<deryck> lifeless, ah, well, that's answer enough.  I shall have to do so if I want to get my stuff landed. :-)
<jml> mars, likewise, I haven't had any LP hacking time until now
<lifeless> deryck: oh there are two things:
<lifeless> deryck: a) Why did it get so far
<lifeless> deryck: b) what was going wrong
<lifeless> deryck: we know b
<lifeless> deryck: storm cache bug + triggers
<deryck> ok.  I don't know that I grok b yet.  But I can re-read the mails.
<mars> jml, speaking of backlog, I'll file the ++profile++ view bug TODO list now
<jml> mars, good idea :)
<mars> (and waste some time trying to dig up the bug number)
<mars> I end up using my branches list page an aweful lot now
<deryck> I am almost my Inbox's master.  Until tomorrow everyone....
<mars> lifeless, did we already have a bug for profiling individual requests?
<mars> I thought we did
<lifeless> yes
<lifeless> http://www.google.com/search?sourceid=chrome&client=ubuntu&channel=cs&ie=UTF-8&q=launchpad+profile+decorator
<lifeless> first hit
<lifeless> ^ now thats search
<mars> ha
<lifeless> bug 598289 <-
<_mup_> Bug #598289: support a profile decorator for use in staging and development environments <performance> <Launchpad Foundations:Triaged by mars> <https://launchpad.net/bugs/598289>
<mars> it's also "I have trained through years of use to hit Google with keywords more likely to produce the result I want"
<mars> aka "I am a Professional Google Basher"
<lifeless> mars: haha!
<lifeless> my google fu is strong
<mars> I'm assigned to fix that bug.  That's embarrassing.
<jml> as the summer sun bids London its blushing farewell, so to must I bid farewell to you
<jml> g'night all
<lifeless> ciao
<mars> night jml
<mars> lifeless, btw, I added you as a reviewer on the test_on_merge.py fixes, along with mwh.  I hope that gives us some good coverage.
<lifeless> fingers crossed ;)
<jam> jml: landing https://code.edge.launchpad.net/~jameinel/launchpad/lp-serve-cleanup/+merge/30542 would be very nice. I don't think I've set anything up for actually landing code.
<lifeless> jam: ec2land
<lifeless> in bin
<jam> lifeless: I have a strong feeling that requires more than just running a single script
<jam> such as.... having an ec2 account
<jam> etc
<jam> but I can give it a shot
<lifeless> it should be all documented on the wiki
<james_w> jam: do you have PQM access?
<jam> james_w: not if it requires anything resembling configuration
<james_w> jam: it requires your GPG key being allowed to commit to LP branches IIUC
<jam> (meaning I've never been set up to contribute to lp, etc.)
<jam> consider me as a community member wrt launchpad code
<james_w> just don't want you setting all this up and waiting 3 hours only to be told that you don't have permission to do it :-)
<jam> james_w: right, and I imagine that would happen. Not to mention the patch mentioned is a comment only change
<lifeless> that would be painful
 * mtaylor trolls: it should not be painful to contribute code
<mwhudson> jam: John Arbash Meinel <jameinel@falco-lucid>  ? :)
<mwhudson> jam: would you like me to land the branch?
<jam> mwhudson: sure. Though I did at least correct my whoami since then :)
<jam> that would be very kind of you
<mwhudson> ok
<Ursinha> lifeless, hi
<lifeless> hi
<Ursinha> lifeless, I made some changes in the bzr-pqm plugin, but only in the lp-land part
<Ursinha> is that correct to say that the changes, even if only applicable to launchpad process, belong upstream?
<lifeless> sure
<mwhudson> jam: the tests are running
<Ursinha> lifeless, cool
<lifeless> I wasn't aware folk still used it :)
<jam> mwhudson: thanks
<Ursinha> lifeless, what's the process to have it upstream?
<lifeless> ping jam
<jam> I guess I'll find out sometime tomorrow if it lands :)
<Ursinha> hahahaha
<jam> pong lifeless
<mwhudson> jam: yeah
<Ursinha> jam, so :)
<lifeless> jam: I was telling Ursinha to ping you :)
<Ursinha> jam, what do I have to do to have my branch merged? propose to merge to bzr-pqm trunk on lp?
<jam> Ursinha: sounds reasonable to me
<Ursinha> jam, cool, I'll do that then :) thanks
<jam> Ursinha: go ahead and ping me here with the merge proposal url, just in case there are problems with the reviewer setting of bzr-pqm
<Ursinha> jam, sure, thanks
 * thumper gets another coffee
 * rockstar  hates at pagetests
 * benji hesitates before rebooting his server with 997 days uptime.
<lifeless> benji: nooo
<lifeless> 3 more days man
<benji> lol
<lifeless> 3 more days
<benji> LOL
 * rockstar is with lifeless
<rockstar> It's not often you get 1000 days uptime.
<rockstar> (Mostly because you should be rebooting more often to get those kernel updates)
<benji> yeah, this thing is running 7.04 and not even the latest kernel
<wgrant> Erm, 7.04 has been unsupported for almost two years...
<lifeless> heh
<lifeless> so - suspend it
<lifeless> wait three days
<lifeless> resume and reboot
#launchpad-dev 2010-07-30
<mtaylor> lifeless: it's running 7.04... probably no suspend :)
<mtaylor> benji: I'm with lifeless and rockstar... you can't reboot that bad boy right now
<benji> ok, bowing to popular opinion he gets a stay of execution until next week
<benji> we'll have a small memorial service with light refreshments to follow
<lifeless> and whoever said peer pressure didn't work
<mtaylor> thumper: the commit count on https://code.edge.launchpad.net/nova ... that's counting unmerged revs? cause in a branch if I run  'bzr log -n0 | grep revno | wc -l' I get 527
<mtaylor> thumper: so I'm just curious what the number on the site means
 * thumper looks
<thumper> 1171  commits by 22  people in the last month
<thumper> that bit?
<mtaylor> yup
<thumper> counts all commits into branches related to that project
<thumper> in the last 30 days
<thumper> not just mainline commits
<mtaylor> does it de-dup?
<thumper> yep
<thumper> distinct revisions
<mtaylor> cool
<mtaylor> thanks
<thumper> np
<wgrant> Hrmph.
<wgrant> The python-debian upgrade overnight seems to have blown everything up.
<wgrant> Starting the test suite fails with a DeprecationWarning.
<wgrant> Commenting it out makes it work fine.
<wgrant> But I don't see why a DeprecationWarning would be fatal...
<wgrant> Ah, I wonder if that latest bzr-builder-related revision fixes it.
<wgrant> How do people manage to file Ubuntu bugs against malone using apport?
<lifeless> hmm
<lifeless> is it just malone?
<lifeless> could be a buggy apport script pointing at malone directly somewhere
<wgrant> lifeless: It's just a single accidental bug.
<wgrant> But it's amazing that people still manage it.
 * mwhudson lunches
<lifeless> wgrant: theres been a few I thought
<wgrant> lifeless: Not normally filed with apport.
<wgrant> Fuuuck.
<wgrant> The upload processor was just really delayed.
<wgrant> So I now have the same package accepted twice into the same PPA.
<wgrant> Somehow the cocoplum and germanium upload processors managed to run at just the same time.
<wgrant> After not running for an extended period.
<wgrant> Now my PPA is going to be all broken :(
<wgrant> ooooohdear.
<StevenK> wgrant: What have we told you about breaking the world? :-P
<wgrant> At least it's only the staging PPA, not the production one...
<wgrant> The result of these builds could be, um, interesting.
<wgrant> https://edge.launchpad.net/~unimelb-ivle/+archive/staging/+packages
<wgrant> Start 2010-08-01 (2505) What's this?
<wgrant> Grrrrr.
<wgrant> Can I convince someone to give me a score bump?
<wgrant> https://edge.launchpad.net/~unimelb-ivle/+archive/staging/+build/1896715 is very sad and would love a bump.
<wgrant> Oh.
<wgrant> There are no builders.
<wgrant> Grr.
<wgrant> Why must the builders disappear when the queue is days long?
<wgrant> It really makes PPAs a whole lot less useful.
<wgrant> When you have to wait 48 hours for a build.
<wgrant> Something is seriously wrong with the upload processors. I just got a rejection for an upload that I did an hour ago.
<lifeless> losa ping
<lifeless> ^
<thumper> mwhudson: ping
<mwhudson> thumper: pong
<thumper> mwhudson: can I have a pre-impl thing with you?
<thumper> although thinking it through, I think I almost have it :)
<mwhudson> thumper: heh, ok, skype?
<thumper> yeah
<mwhudson> going online...
<wgrant> Blah.
<wgrant> Just got another three rejections.
<wgrant> 1.5 hours later.
<lifeless> wgrant: are they valid & slow, or invalid rejections ?
<wgrant> lifeless: They're valid, but really really slow.
<wgrant> Heh.
<wgrant> Actually, the rejection reason is broken, because the PPA is now broken. but they should have been rejected anyway.
 * wgrant sometimes wonders how Launchpad hopes to be taken seriously.
<mwhudson> wgrant: surprisingly, there exists software that sucks much more than launchpad
<wgrant> mwhudson: Yes, but other stuff generally doesn't make me regret moving to it for deployments every time I try to make a release.
<wgrant> And other stuff doesn't have 3 day build queues.
<mwhudson> yeah, the build queue is pretty insane currently
<wgrant> And often is.
<wgrant> And nobody seems to care.
<thumper> wgrant: we do care
<thumper> wgrant: perhaps we should show you our scars where we cut ourselves each time a user complains
<wgrant> thumper: The build queue has reached the 2-3 day mark at least once every couple of weeks, lately.
<wgrant> That makes one of Launchpad's big features just about useless.
<wgrant> A couple of years ago, a bad PPA build wait might be a couple of hours. Now it's often 24 times that.
<magcius> Does anybody even work on Loggerhead anymore?
<thumper> magcius: yes, but not an active LP dev
<magcius> thumper: argh. There are several UX issues with it that are so annoying.
<magcius> thumper: is there a slated replacement or similar for the code browsing?
<thumper> magcius: more a fixing than replacement
<thumper> magcius: what is it that annoys you so much?
<magcius> thumper: the lack of left margin for code, especially annoying with Python
<magcius> thumper: and the blame lines are not monospaced and centered, which makes it even more annoying
<thumper> magcius: can you give me a concrete example?
<thumper> magcius: a link maybe?
<magcius> thumper: http://bazaar.launchpad.net/~loggerhead-team/loggerhead/trunk-rich/annotate/head%3A/loggerhead/apps/config.py
<magcius> thumper: (the URLs are my second complaint)
<magcius> thumper: if you scroll down a bit, it's easy to lose your place in the indentation
<magcius> thumper: also, if I want to go up to the directory that that is in
<magcius> thumper: I can't just edit the URL to remove the "config.py", it gives an error
<magcius> thumper: the 'head:' that is encoded is annoying when passing around, and the URL is needlessly long.
<magcius> thumper: I logged these: #569355 and #569358
<_mup_> Bug #569355: Left margin in code view. <loggerhead:New> <https://launchpad.net/bugs/569355>
<_mup_> Bug #569358: Code viewer URLs have problems <loggerhead:New> <https://launchpad.net/bugs/569358>
<magcius> After a month or two, they're still both "New"
<thumper> magcius: I'm sure loggerhead could use some keen devs :)
<magcius> thumper: it looks to be a Django app.
<thumper> magcius: I think it is pretty vanilla wsgi
<lifeless> its pastedeploy specifically, which is pretty stock wsgi
<magcius> oh hey, it's broken too
<magcius> http://bazaar.launchpad.net/~loggerhead-team/loggerhead/trunk-rich/annotate/head%3A/loggerhead/apps/config.py
<magcius> if you click on one of the parent directories at the top, it takes you to a weird URL where none of the files from then on work
<magcius> like http://bazaar.launchpad.net/~loggerhead-team/loggerhead/trunk-rich/files/428?file_id=apps-20080618010429-n87ewcyimdwsx90g-3
<lifeless> mwhudson: ping; I've just done a microreview about string formatting ;)
<lifeless> mwhudson: if you want to discuss in realtime, let me know.
<mwhudson> lifeless: you have? where?
<lifeless> https://code.edge.launchpad.net/~mwhudson/launchpad/vostok-traverse-distro/+merge/31241
<mwhudson> lifeless: branch_type is an enum
<mwhudson> or value from an enum, really i guess
<lifeless> mwhudson: _always_, reliably, even when its gone wrong ?
<mwhudson> lifeless: i think so
<mwhudson> lifeless: however
<mwhudson> lifeless: this isn't my change and i'm happy to revert it
<lifeless> It seems to me that the reflex of 'this can go wrong' is one worth making easy to apply
<mwhudson> yeah, i agree i think
<mwhudson> hey!
<mwhudson> someone review this: https://code.edge.launchpad.net/~mwhudson/launchpad/kill-launchbag.site/+merge/31348
<StevenK> O hai! thumper, mwhudson: Could one (or both, I don't mind) have a look at an MP that I've pushed up that uses the job infrastructure?
<StevenK> https://code.edge.launchpad.net/~stevenk/launchpad/db-add-ifp-job/+merge/31349
<lifeless> mwhudson: can you let whomever made that change know about this discussion ?
<mwhudson> lifeless: sure
<StevenK> lifeless: You too, if you wouldn't mind, but you tend to stalk new MPs anyway
<lifeless> StevenK: I stalk everyones
<lifeless> StevenK: I'm subscribed to the feed
<lifeless> StevenK: but I'll pass today, today is jetlad day #2.
<lifeless> StevenK: I will look monday
<StevenK> lifeless: You'll need to do a DB review in any case, but Monday is cool
<mwhudson> StevenK: + -- The particular type of foo job
<StevenK> Oh, damn it
<mwhudson> do you need the job_type anyway?
<StevenK> I knew I should have grep'd the diff for 'foo'
<mwhudson> i can't really imagine InitialiseDistroSeriesJob containing other kinds of job, somehow...
<StevenK> mwhudson: TBH, I was following the wiki page -- so I'm not certain. I can forsee the need to change it later to add the ability to, for example, do a scorched-earth rebuild.
<mwhudson> ok
<StevenK> mwhudson: So is that a use-case for job_type, or that can be fed in via the JSON metadata?
<mwhudson> StevenK: i guess it depends on details
<mwhudson> StevenK: the point of job_type not being in the json is that you can query it
<mwhudson> would you have a different cron job for that kind of ifp job?
<StevenK> Nope, they should be handled by one cron job
<mwhudson> then i don't think you need a job_type at this stage
<mwhudson> StevenK: similarly, i'm a bit suspicious of the json_data
<StevenK> mwhudson: Well, I think there is a need (although, not right now) to feed in some more data, so leaving it in sounds fair.
<StevenK> mwhudson: However, I agree re: death to job_type, and have added to that my list
<lifeless> StevenK: json is json
<lifeless> why have two fields?
<lifeless> assuming I read the chatter here correctly
<wgrant> Isn't someone else working on copy archive population jobs?
<wgrant> Isn't that pretty similar to i-f-p?
<mwhudson> wgrant: it was james_w
 * StevenK reads through james' MP
<StevenK> That's for an entire archive, whereas my MP covers distroseries
<wgrant> StevenK: They seem very, very similar.
<wgrant> populate-archive just happens to not copy binaries.
 * StevenK will talk to Julian
<thumper> StevenK: and while you're at it, please another blood-letting for the lack of builders making wgrant's build take two days
<StevenK> thumper: I would, but I value my ears
 * StevenK heads out to the bank
<mwhudson> thumper: did you know that the view returned by test_traverse() is security proxied?
<thumper> mwhudson: nope
<mwhudson> or at least in this case
<wgrant> I am confusede.
<StevenK> Is that Ye Olde English confused?
<wgrant> It's "I'm doing too many things to state my confusion" :P
<wgrant> I don't understand why dailies have priority to destroy the build farm even more than it is already.
<wgrant> I would have thought they would be the last things, apart from rebuilds, that should be built in a situation of the worst resource starvation we've seen in a looong time.
<wgrant> The queue is three days long.
<wgrant> It will take more than that to clear.
<StevenK> wgrant: So, I had already admitted I had privledges, and would bump builds. Am I supposed to reply back with "Your builds aren't important enough, sod off" ?
<wgrant> I think there needs to be a policy.
<wgrant> Because this starvation happens frequently.
<wgrant> And long-running new crack builds are surely not the things that need to be prioritised.
<StevenK> Alpha 3 is around the corner, which is where I suspect the builders have gone.
<StevenK> wgrant: Which still ends up with people being told "Your builds aren't important enough, sod off"; but they are, to *them*
<wgrant> StevenK: I don't see how dailies could possibly be important.
<wgrant> They are dailies.
<wgrant> They happen daily.
<StevenK> They attempt to happen daily
<wgrant> Nobody is going to be pissed off if one delivery of the new daily crack is delayed by 24 hours.
<wgrant> People *are* going to be pissed off if their first builds in weeks take five days.
<StevenK> Except, say, fta, who is well-documented about his feelings of the build farm
<wgrant> See, I have a build which will take less than a minute.
<wgrant> And it won't be done for five days. Because the dailies are prioritised.
<wgrant> When there could be hundreds of other builds done in the time a single daily takes.
<StevenK> But I can't bump that one
<StevenK> Sadly
 * wgrant now has to advise the admins to use a repository elsewhere, because the PPA can't be populated with the new release within a week.
<StevenK> wgrant: I can bump builds if you ask, but I was under the impression recipe builds can't be bumped
<wgrant> StevenK: A score bump for https://edge.launchpad.net/~unimelb-ivle/+archive/staging/+build/1896715 would be most appreciated.
<wgrant> Otherwise, I'll make my arch-indep packages arch-dep, and just let it build on amd64 in a few hours...
<StevenK> wgrant: Bumped; with a score above the other builds I bumped
<wgrant> StevenK: Thankyou muchly.
<StevenK> Can you even buy beer yet?
<wgrant> Hopefully the other builders will reappear at some point.
 * StevenK cackles
<wgrant> Not in the US :P
<StevenK> I wasn't in the US, last I checked :-)
<wgrant> Tarue.
<wgrant> Hmm.
<wgrant> Double form submission.
<wgrant> I clicked the button once, but it submitted twice.
<wgrant> That's a bit scary.
<StevenK> Browser bug? :-P
<wgrant> Possible...
 * StevenK is also distracted, arguing with Telstra over a bill :-/
<wgrant> KILL THEM.
<StevenK> Haha
<wgrant> At least with NBN that might all go away...
<wgrant> Telstra will become irrelevant and fade away :)
<wgrant> Or so we can hope...
<adeuring> good morning
<mrevell> Ni hao
<wgrant> bigjools: I wonder if current_source_publication should be latest_source_publication.
<bigjools> wgrant: it could be an additional method
<wgrant> Ah, good idea.
<bigjools> current_ is very valid
<deryck> Morning, all.
<jml> pqm could be much better.
<jml> deryck, good morning
<mrevell> Thanks for the blog post deryck
<deryck> mrevell, glad to do it.  Now to actually get that bug fixed. :-)
<mrevell> :)
<wgrant> bigjools: Any chance of a score bump for https://edge.launchpad.net/~unimelb-ivle/+archive/staging/+packages?
<bigjools> wgrant: still around?
<wgrant> bigjools: Indeedily.
<bigjools> wgrant: seen /builders for nonvirt i386?
<bigjools> 149 jobs (eight minutes)
<wgrant> It's translations jobs.
<bigjools> I think not - and nothing was getting dispatched earlier
<wgrant> They appear in the wrong queue, and their estimated duration is screwed.
<bigjools> still not
<wgrant> Oh?
<bigjools> ah
<bigjools> TTBJs?
<wgrant> Translations jobs.
<wgrant> Yes.
<wgrant> I filed a bug about the incorrect queue earlier this week.
<wgrant> And they have a low score, so they'll sit there until the i386 virt queue clears.
<wgrant> I don't know of a bug for the bad duration estimate.
<bigjools> wgrant: are they also in the PPA queue length?
<wgrant> bigjools: No.
<bigjools> ah
<bigjools> wow, I wonder why there;s so many
<wgrant> It's not that many.
<wgrant> it's a couple of days' worth, at least...
<bigjools> 150!
<bigjools> we need to score them up a bit
<bigjools> they're quick
<bigjools> I'm also toying with the idea of scoring people's builds -1 for each outstanding build they have
<wgrant> Possibly.
<wgrant> I think we need to rethink scheduling.
<wgrant> Since the current system obvious doesn't work.
<wgrant> At all.
<wgrant> +ly
<bigjools> well, it does work
<bigjools> the problem is that there;s more work than the builders can handle
<bigjools> we're due 7 more PPA builders  soon-ish
<wgrant> You've been saying that since Wellington :P
<wgrant> But the problem also manifests itself when the daily wave appears.
 * bigjools can't comment on why they're not instaled
<wgrant> The dailies drown out other builds for a few hours.
<bigjools> yes
<bigjools> and if we score those down, someone else complains
<bigjools> *shrug*
<wgrant> If they don't build for a while, sure.
<wgrant> But there should be no issue with a wait of a few hours.
<wgrant> Which allows a much more realtime experience for the people who actually upload things.
<bigjools> so the next piece of work will be to add some knobs and dials to bias scores on the fly
<wgrant> Hmm. I'm not sure that a more automated solution could not be devised.
<wgrant> But first, sleep.
<wgrant> Did they just steal samarium too?
<StevenK> Unlikely
<james_w> hi StevenK
<StevenK> james_w: O hai!
<james_w> StevenK: I'll take a look today at how much the two job types can be coalesced
<james_w> I'm pretty sure we don't need two db tables at least
<StevenK> james_w: That sounds great. We could probably get away with 2 job types
<james_w> I'm not sure what's so different that they can't share some of the same code as well
<wgrant> I think they can share an awful lot of code.
<wgrant> Populating a copy archive is just a subset of i-f-p.
 * bigjools high fives james_w, StevenK and wgrant
<StevenK> wgrant: samarium is back
<jelmer> sinzui, hi
<sinzui> hi jelmer
<jelmer> sinzui: The PPA appears to be lacking a copy of pocket-lint for hardy, is that intentional?
<maxb> I mentioned this on the mailing list
 * maxb finds thread
<sinzui> jelmer. Sort of. I think all developers are on lucid or maverick, and pocket-lint is only used by developers
<sinzui> Amu I wrong
<sinzui> ?
<sinzui> am I wrong?
<jelmer> sinzui: The ec2 image is still hardy-based so it's impossible to update at the moment.
<sinzui> bugger
<maxb> sinzui: We should not preclude developing on the distro that production uses, nor should launchpad-developer-dependencies be uninstallable in any ppa series, IMO
<sinzui> well I do not know if pocket-lint works on hardy. We could copy it and see if it builds
<jelmer> sinzui: I'll try
<sinzui> has 0.5 been built yet?
<jelmer> no, the PPA has 0.4 at the moment
<sinzui> jelmer, I think it will work since python 2.5 was backported for lp. pocket-lint includes its external deps in a contrib dir.
<sinzui> has anyone put any thought about what for do about LaunchpadObjectFactory.makeDistributionSourcePackage returns an unproxied object
<sinzui> It is simple class that differs to database classes. I cannot image how to get a proxied version of a DSP
<bigjools> sinzui: construct it from proxied objects?
<bigjools> s/from/inside/
<sinzui> I am look for that actually
<sinzui> I think distribution.getSourcePackage() is the only way to get a proxies object
<rockstar> mars, so what's the deal with the lazr-js branches?
<jml> deryck, hi
<deryck> jml, hi.  on call right now.
<jml> deryck, np. I'll use email to achieve my nefarious purpose
<jml> sinzui, ProxyFactory
<sinzui> jml: distribution.getSourcePackage() a nice one line change
<jml> sinzui, cool.
<jml> fwiw, my branch that turns the print outs into Python warnings has landed
<jml> if it sucks, revert it & send me an email.
 * rockstar lunches
<james_w> so, we were asked to make sure that it is possible to delete all artefacts of copy archives from the librarian. Could anyone point me to any code that I could read about the cleanup process and how it is decided what to delete?
<bigjools> james_w: it's not easy
<bigjools> james_w: start at the publishing records, then cascade down through builds, librarian files and spr/bpr if they're not published elsewhere
<bigjools> we added some ON DELETE CASCADEs to the model but some of it's dangerous
<james_w> bigjools: so it's merely that the archive is deleted, and that cascades down to delete the librarian files at the end of the chain?
<bigjools> james_w: in an ideal world yes, but remember that stuff was copied from other archives so a lot of the files may be shared
<bigjools> your binaries won't in the case of copy archives of course, but if we apply this work to PPAs as well then it will
<james_w> bigjools: right, I'm just trying to understand the process first
<bigjools> james_w: the db cascade only goes so far, it stops at the stuff that might be shared
<lifeless> moin
<james_w> bigjools: so we need another process, as we can't rely on the database to do this for us
<bigjools> james_w: yes - we've implemented some of it in the publisher, which is the right place to do this
<bigjools> hello lifeless, working on the weekend
<bigjools> james_w: there's an archive status of DELETING and DELETED
<bigjools> james_w: the former is set by the webapp request and the publisher picks that up, deletes the repo, and sets status to DELETED
<james_w> bigjools: the publisher deletes the published files and the librarian files too?
<bigjools> james_w: no, just the repo area right now
<bigjools> there's a separate process that deletes librarian files
<bigjools> we need to extend it to pick up copy archive files
<james_w> ok, and where is that process?
<bigjools> cronscripts/expire-archive-files.py
<bigjools> james_w: it picks up superseded and deleted publications
<bigjools> james_w: you can just make a one line change to that script :)
<james_w> and deleting a copy archive will delete the publications from that archive?
<bigjools> yes
<bigjools> well
<bigjools> actually, if you delete the publishing records then the librarian files will be orphaned and the GC will get them
<bigjools> so don't worry about that expiration script
<james_w> so we want to make deletion of copy archives delete the binary publishing records for that archive?
<james_w> (and the associated builds presumably)
<bigjools> and source publications
<bigjools> and any binarypackagereleasefile / sourcepackagereleasefile
<bigjools> that is not published elsewhere
<rockstar> mars, ping
<james_w> bigjools: so it already deletes all sources, we should just make it do the same for the other objects?
<james_w> well requestDeletion() of all published sources at least
<bigjools> james_w: when I say delete, I mean remove the database row, not mark the publication deleted
<james_w> ok
<bigjools> we want to completely blow any trace of this archive away
<james_w> but still in Archive.delete?
<bigjools> no
<bigjools> you have to do it in the publisher
<james_w> why?
<james_w> just so it's only done when it goes DELETING->DELETED?
<bigjools> yes
<bigjools> it would take too long to do in a webapp request
<bigjools> we might even need to move that bit of code that marks all the source publications as deleted into the handler in the publisher, I suspect it'll take too long for copy archives
<james_w> heh
<james_w> you can't actually delete copy archives right now any way
<james_w> so what is actually done right now?
<bigjools> all the publications are marked as deleted and the repository is removed
<james_w> I guess it's because they aren't published
<james_w> the repository is not removed via the standard mechanism that PPAs are, so I don't want to change something that isn't actually used.
<bigjools> I don't understand what you mean
<james_w> if copy archives are set to DELETING then they are not set to DELETED by the publisher as it stands. So either there is some other mechanism used, or we just have a bunch of copy archives in the DELETING state.
<bigjools> oh - I don't know if anyone set those as deleted yet
<james_w> well, they go inaccessible from the web
<bigjools> they get disabled
<james_w> so maybe they are just disabled?
<james_w> right
<bigjools> jelmer is fixing that so we can still see disabled copy archives :)
<james_w> so we want to change it such that an admin can go to +edit and delete them?
<james_w> at which point the publisher will remove them from disk if they have been published and set them to DELETED.
<bigjools> right, so if you see the bottom of lib/lp/soyuz/scripts/publishdistro.py
<james_w> and in addition extend it such that if it is a copy archive all publications are deleted, and other things such as SPRs are deleted if they are only references by this archive?
<bigjools> it only considers PPAs before calling publisher.deleteArchive
<james_w> right, that's what I was saying a minute ago
<bigjools> yes :)
<bigjools> sorry it's late and I am slow :)
<james_w> should that be the same for all archives, or just copy archives?
<bigjools> should be the same for copy archives too
<bigjools> so
<bigjools> I would take a look at the foreign key chain off the publishing records and you'll see quite quickly what you need to delete :)
<james_w> no, I mean, should the deletion of publications and SPRs etc. happen for all DELETING archives?
<james_w> or just copy archives?
<bigjools> yes, we need to blow them away entirely
<bigjools> we do a half-assed job at the moment so that people can rename their accounts
<sinzui> I have a test that does
<sinzui> naked_productseries.datecreated = test_date
<james_w> ok, so no ArchivePurpose check.
<sinzui> to test a value that cannot be written via the interfaces
<james_w> bigjools: thanks, I'll show you some code tomorrow so you can verify the direction
<bigjools> not tomorrow you won't :)
<bigjools> james_w: the only trick part is where packages are published in more than one archive
<bigjools> tricky even
<bigjools> once you get past that it's easy
<james_w> bigjools: I'll show you tomorrow, but you won't look :-)
<bigjools> heh
<bigjools> I'm about to upgrade my hardy server to lucid, I might not be looking at anything
<barry> sinzui: ping
<sinzui> hi barry
<barry> sinzui: hi!  say, i'm trying to set up another branch mirror but lp isn't happy.  could you double check this for me?  https://code.edge.launchpad.net/~python-dev/python/2.7
<barry> sinzui: i can svn ls http://svn.python.org/projects/python/branches/release27-maint
<james_w> barry: you want an import not a mirror. Mirror is for native bzr branches, import for foreign VCS.
<barry> d'oh!
<sinzui> ah! I was wonder why the page looks so strange
<lifeless> we should eliminate that difference
<lifeless> its implementation not intent
<barry> james_w, sinzui: thanks.  i just did an approved import request
<barry> lifeless: yes, or add import to the list of options in "register a branch" and explain the difference ;)
<lifeless> barry: file a bug pleasE?
<barry> lifeless: already started... :)
<sinzui> I believe there is already several bug about "Register a branch" and a discussion to remove it
<barry> lifeless, sinzui: you are the triagemasters, please do your magic: https://bugs.edge.launchpad.net/launchpad/+bug/611837
<_mup_> Bug #611837: Confusions between import and mirror branches <Launchpad itself:New> <https://launchpad.net/bugs/611837>
<barry> thanks guys!
<sinzui> I am sure that is a dup I wonder if it would have shown up as a dup if the bug was reported against launchpad-code
<james_w> bigjools: still around? How is the uploadpolicy context set for uploads from users. Apparently all the process-upload calls in lp-production-configs use the buildd policy.
<bigjools> james_w: it uses the insecure policy for general uploads, it's set in the crontab entry
<james_w> ah, crontabs, thanks
<barry> sinzui: possibly!
<sinzui> The +setbranch form shows a unified view of all the nonsense. It does a better job of explaining the difference
<sinzui> That view is for registration *add* setting it to a series, so it is not always the right link to use
<james_w> oh crap, a car crash of concerns
<james_w> did jtv's GenericCollection ever land?
<lifeless> yes
<james_w> hum
<wgrant> sinzui: ProxyFactory lets you proxy any proxiable object. You don't need to get it from anywhere special.
<sinzui> wgrant, thanks. I have applied that to the objects I cared about
<james_w> hi wgrant
<wgrant> Hi james_w.
<james_w> so, deleting an archive disables it, which means that the publisher tends to ignore it
<wgrant> That's correct.
<james_w> so we have to teach the publisher to consider disabled archives in the DELETING state
<wgrant> I thought that was already the case.
<wgrant> There is a special DELETING path in there...
<james_w> however, the methods it uses for finding archives aren't very suitable for this
<james_w> wgrant: I'm extending it beyond PPAs, it works fine for them
<wgrant> Ah, right.
<wgrant> Copy archives are going to be materialised soon?
<james_w> yeah
<james_w> well, it's more about librarian, but yes, that too
<james_w> I've been asked to make the publisher delete artefacts in the db from deleted archives
<wgrant> Ah...
<wgrant> It would be nice if LP permissions didn't suck.
<wgrant> Once you do that, anyone with upload rights can completely destroy the archive and all its history irretrievably.
<james_w> that applies to PRIMARY too?
<lifeless> that sounds undesirable
<wgrant> No. It's special.
<wgrant> But anyone in ~ubuntu-drivers could.
<james_w> right
<wgrant> Which is far too large a group of people, but that causes other more gravely concerning issues.
<james_w> still not particularly desirable in general
<wgrant> No.
<james_w> we could limit this to COPY archives for now
<wgrant> That sounds good.
<james_w> I got the impression that this is wanted for PPA as well so that they can be properly deleted
<james_w> is that right?
<wgrant> I cannot support PPAs' inclusion in it until it's more restricted.
<wgrant> It is, yes.
<james_w> what would a good interface for the general case of this look like?
<james_w> a way to query deleted archives and purge them?
<wgrant> Possibly.
<wgrant> It's unobvious, though, due to the various criteria that need to be used to select the archives.
<james_w> DELETE just refers to the published state right now?
<wgrant> Not just purpose, but privacy.
<wgrant> Hm?
<james_w> deleting an archive just removes the published files, the archive is still in the db, disabled?
<james_w> I assumed that the purging of the archives would be an admin-controlled task.
<lifeless> mmm
<lifeless> really we want the $-distro admins, not the lp-instance admins
<wgrant> Right. The publications are Deleted, then the archive status is set to DELETING, then the archive is purged from archive disk, then the status is set to DELETED.
<wgrant> lifeless: Distro doesn't care about PPAs.
<lifeless> wgrant: true
<james_w> so with some surgery something in DELETED could be resurrected
<lifeless> for ppas why not immediate delete? or time-based ?
<wgrant> james_w: Exactly.
<wgrant> Or its latest contents at least downloaded.
<james_w> though the publications would all be deleted too, so it would be a convoluted process
<wgrant> Right. The librarian files would disappear in a week, too.
<james_w> right
<james_w> so perhaps we have another state PURGED that has a delay of a week too, and removes the DB rows for the publications etc. as well?
<wgrant> Possibly. But I don't really see the pressing need to delete that stuff.
<lifeless> wgrant: dead data costs to keep around; its not free.
<lifeless> wgrant: neither in a dollars sense, nor a performance sense.
<wgrant> True.
<lifeless> if the users does not want it
<lifeless> and we don't have a legal obligation to keep it
<lifeless> why would we incur that overhead?
<james_w> COPY archives is what we have been asked to implement this for, Julian said that I should do it for everything
<james_w> I'm fine with putting the infrastructure there and using it for COPY archives, as it will be easy to use it for other types later if desired
<wgrant> james_w: It doesn't look like any filtering is done on copy archives.
<james_w> wgrant: where?
<wgrant> james_w: lib/lp/soyuz/scripts/publishdistro.py
<james_w> no, it's getArchivesForDistribution that is the problem
<wgrant> So you probably just have to change the archivepurpose guard around the deleteArchive call.
<wgrant> Oh.
<james_w> it only lets you see enabled archives unless you pass in an admin
<wgrant> Ah.
<wgrant> Or you say exclude_disabled=False...
<james_w> nope
<james_w> you have to say that *and* pass in an admin
<wgrant> Oh.
<wgrant> Indeed.
<wgrant> How stupid.
<james_w> indeed
<james_w> I can't see an easy way to unthread it without checking all callers
<james_w> hence my question about GenericCollection, I wondered if implementing an ArchiveCollection would make sense, but that's a bit large for me to take on right now
<wgrant> I think that would be a good idea. But it's probably too big for a small change like this.
<james_w> http://paste.ubuntu.com/471263/ is the horrible hack that I am using for now
<wgrant> james_w: You could call it filter_visible or something like that.
<james_w> yeah, that's a better name
<wgrant> james_w: But the callsites for the method number only three, so it's not too bad if you have to change it in other ways.
<james_w> good to know
<james_w> not sure how I would change it other than this or going for a collection
<james_w> anyway, I think that's enough for this week, I'll tackle the hard bit of this on Monday
<james_w> and a nice illustration of the perils of purging deleted archives just now :-)
<wgrant> Sounds good. I think the Collection pattern needs to be used more widely.
<wgrant> In that case I think purging (or just renaming) would have fixed it.
<wgrant> He doesn't want the content -- just to be able to upload to it again.
<james_w> thanks for your help
<wgrant> np
 * jelmer laughs reading the summary for bug 235939
<_mup_> Bug #235939: X-Launchpad-Message-Rationale header for code import emails too human friendly <code-import> <email> <Launchpad Bazaar Integration:Triaged> <https://launchpad.net/bugs/235939>
#launchpad-dev 2010-07-31
<wgrant> WTF is ~itiseasy-org, and why has it uploaded several hundred packages overnight?
<wgrant> Like we needed more in the queue...
<lifeless> wgrant: dunno
<wgrant> lifeless: Could you land https://code.edge.launchpad.net/~wgrant/launchpad/replace-archiveuploader-doctests-0/+merge/30850? You approved it a week ago.
<lifeless> kicking it off
<lifeless> wgrant: you didn't find anyone to do it for a week ?
<wgrant> lifeless: No, I just forgot about it.
<lifeless> :)
<wgrant> Thanks.
<wgrant> Do we have working feature flag support yet?
<wgrant> I saw some stuff land.
<wgrant> And I'd like to disable something with it, if it does indeed exist...
<lifeless> db-devel only
<lifeless> could probably port the interface to devel
<lifeless> with some care
<wgrant> No need.
<lifeless> and then your thing would be disabled by it but able to be controlled on staging & next release
<wgrant> There's only a few days left until the freeze.
<lifeless> last freeze ever!
<wgrant> Oh?
<wgrant> We're moving to the new merge workflow for 10.09?
<wgrant> Well, what was 10.09.
<lifeless> I think all the bits are in place
<lifeless> if not then very very close
<wgrant> Wow.
<wgrant> That was swift.
 * wgrant whines about tree builds taking far too long.
<wgrant> 4 minutes and counting...
<lifeless> wgrant: using ?
<lifeless> wgrant: well the basic bits are:
<wgrant> lifeless: The initial make in a new branch.
<lifeless>  - a daily staging [trivialish]
<lifeless>  - a way to disable stuff [feature flags, rough but usable]
<lifeless>  - new QA toolchain/automation [done, I believe]
<wgrant> Right, I wan't aware that the last bit was even started.
<wgrant> So the new system will take all the branches in the queue, merge them, test them as a whole, then commit them?
<lifeless> no
<lifeless> thats totally different
<wgrant> Oh.
<wgrant> So just the new QA workflow, not the new merge workflow?
<lifeless> thats an optimisation for landing stuff
<wgrant> Ah.
<wgrant> I se.
<lifeless> https://dev.launchpad.net/LEP/ReleaseFeaturesWhenTheyAreDone
<lifeless> https://dev.launchpad.net/MergeWorkflowDraft specifically
<lifeless> https://dev.launchpad.net/MergeWorkflowDraft?action=AttachFile&do=view&target=merge-workflow-draft-2.jpeg is the thing we drew
<lifeless> or maris drew, I should say
<wgrant> Hm. that looks a lot simpler than the last draft I saw.
<lifeless> if the last thing you was before the epic, then yes
<wgrant> Right.
<lifeless> We revisited the entire proposal there
<lifeless> we also need a patch to make 'edge' behaviour depend on the vhost, not on the confi
<lifeless> config
<lifeless> I don't think anyone has done that yet, would be nice to do.
<lifeless> though the meaning of edge is going to erode pretty quickly
<wgrant> Yes.
<wgrant> I also suppose that the automation of service restarts should make DB upgrades a whole lot quicker.
<lifeless> oh yeah thats the other key thing, but its necessary for ratcheting up the frequency, not for switching at all
<wgrant> Yep.
<lifeless> wgrant: nuts
<wgrant> lifeless: Um, devel is broken on 2.5?
<wgrant> Ew.
<wgrant> I don't see how that happens, unless people aren't ec2ing branches before landing...
<jelmer> hey wgrant
<wgrant> Morning jelmer.
<jelmer> wgrant, it's broken how?
<wgrant> jelmer: test_sourcepackagerecipe.py doesn't appear to import with_statement.
<jelmer> ah
<jelmer> wgrant, not everybody uses ec2, some people run make test locally
<wgrant> Can someone please EC2 https://code.edge.launchpad.net/~wgrant/launchpad/replace-archiveuploader-doctests-0/+merge/30850? It was caught in the devel breakage last night.
<wgrant> So, the build farm has currently been unusable for five days.
<wgrant> I am failing to see how this is not a critical service issue.
<wgrant> One of Launchpad's most popular features is broken.
<lifeless> wgrant: kicked it off again
<lifeless> wgrant: uhm, I think its an important issue.
<wgrant> It happens time and time again.
<lifeless> wgrant: builds > builders for extended periods, + starvation events ?
<wgrant> lifeless: The sustained reduction of the build farm is an issue.
<wgrant> As well as prioritisation during extended starvation events.
<wgrant> But new builds will be waiting for nearly four days.
<lifeless> so, the whole scoring thing is what drives starvation
<lifeless> its implementation not policy, and because its complex few people can actually see clearly to define a policy
<wgrant> Scoring doesn't drive starvation. It just means that the dailies build before the realtime packages.
<wgrant> So it exacerbates the effects.
<lifeless> I disagree, but this is an assessment of the impact of the design choice; we both agree that the behaviour is undesirable
<wgrant> Starvation is going to occur whatever the ordering of the builds.
<wgrant> It's just going to be less noticable if realtime stuff builds first.
<lifeless> by realtime do you mean 'manually uploaded single packages' ?
<wgrant> Yes.
<wgrant> Things that aren't going to be unnoticed if they don't build for a few days.
<wgrant> ie. not dailies, and not rebuilds.
<lifeless> sure
<lifeless> so short term, we can futz with the scoring algorithms
<lifeless> builder shortages, and inefficient use of builders, exacerbate the situation
<wgrant> Longer-term: where did all those builders go for two or three days?
<lifeless> QA
<lifeless> only a fraction of the PPA buildds are dedicated to LP
<wgrant> Odd that it takes so long.
<wgrant> I know, yes.
<wgrant> But normally they're only gone for a day at most.
<wgrant> Except when they're taken for mirrors.
<lifeless> the handover isn't flawless
<lifeless> so one possibility is a failed handover and they end up in limbo
<lifeless> a system wide failure in the handover would explain a batch not coming back
<lifeless> anyhow, with bigjools fix to the sequences nearly here
<lifeless> your branch is testing
<wgrant> Thanks.
<lifeless> s/sequences/sequencer/ we should get much more juice from the buildds, and be in a good position to talk about expanding the farm with less transient resources
<wgrant> Right.
<wgrant> Although buildd-manager isn't toooo bad with this few builders.
<james_w> if someone is bored: https://code.edge.launchpad.net/~james-w/launchpad/buildout-doc/+merge/31465
<james_w> wgrant: has a scheduler based on quotas for people/archives been discussed?
<wgrant> james_w: Discussed? No.
<wgrant> It gets mentioned occasionally.
<wgrant> But nothing comes of it.
<james_w> what do we need to do to get a new scheduling algorithm?
<wgrant> 1) Adjust findBuildCandidate
<james_w> is it just a case of trying something, or does there need to be a discussion about what direction to go in?
<wgrant> 2) Adjust dispatch time estimation
<wgrant> I don't know.
<wgrant> 1) is easy.
<wgrant> 2) is very difficult.
<wgrant> So it's not as easy as it was two years ago :(
<wgrant> Although, if the current trend continues we can just replace the estimation with 'give up'.
<james_w> p.s. I have an ArchiveCollection running through ec2 to catch any issues before review, and I'm currently taking a hatchet to some soyuz tests to remove sampledata uses
<james_w> yay for a bored weekend
<wgrant> Ooh, excellent.
<wgrant> I've been meaning to see how many tests I can get to not use sampledata simply by tweaking STP.
<james_w> yeah, I'd never seen it before, but I'm mainly stopping things from using that
<james_w> the factory is just about as good for the uses I have seen so far
<wgrant> Oh.
<wgrant> STP is use pretty extensively throughout the test suite (including brand new stuff).
<james_w> yeah
<wgrant> It probably should be merged into the factory, but it does lots of stuff that the factory does not.
<wgrant> I'm also not a fan of the factory's monolithic approach.
<wgrant> It doesn't seem to be very compatible with the ongoing attempts at modularity.
<james_w> indeed
<james_w> I've only gone through about 10 methods so far, and they made little use of the publisher I think
<wgrant> For a lot of things, makeBinaryPackagePublishingHistory will indeed suffice.
#launchpad-dev 2010-08-01
<james_w> Can't the metazcml lazy-instantiate things registered as utilities?
<wgrant> One would think so.
<wgrant> It might make things not take several ages to start.
<james_w> It's probably not a huge percentage of the zcml processing time, but not running all that code wouldn't hurt the time
<james_w> wgrant: https://code.edge.launchpad.net/~james-w/launchpad/no-more-sampledata-0/+merge/31470 <- what do you think?
<wgrant> james_w: Looks good.
<wgrant> It even removes the factory's manual BPPH instantiation, which makes me happy (I have a branch removing the rest already).
<james_w> wgrant: you have a branch to do it for SPPH too?
<wgrant> Although you should probably do the same with SPPH too.
<wgrant> james_w: Yes. I got rid of everything except a couple of tests, STP and the factory.
<james_w> ah
<wgrant> Because the factory sets extra attributes, and I wasn't sure if rSP was really prettier.
<james_w> I can do the same change for SPPH, I was just being lazy, and it's going to conflict anyway
<wgrant> But it seems it is.
<wgrant> What's it going to conflict with?
<wgrant> I didn't touch the factory, for the above reason.
<james_w> sinzui has a branch to return a proxied SPPH
<wgrant> Ah.
<james_w> using the method I used in this branch
<james_w> not a big problem
<wgrant> I wonder if 'make lint' should whinge about c.l.i and c.l.d imports.
<wgrant> Probably.
<wgrant> james_w: I like the look of that ArchiveCollection branch, too.
<wgrant> But... 'isRequireVirtualized'?
<james_w> yeah, it sucks
<james_w> maybe .requiresVirtualization()?
<wgrant> james_w: Sounds good.
<mtaylor> it would be really nice if I could edit the target branch in a merge proposal
<james_w> wgrant: would a tweak to score() to subtract some constant multiplied by the number of builds in the same archive in the last 24 hours do any good, or would it just be piling more on top of a shaky system?
<james_w> mtaylor: there is some discussion in http://www.mail-archive.com/launchpad-dev@lists.launchpad.net/msg03646.html about that sort operation on merge proposals
<mtaylor> james_w: excellent
<wgrant> james_w: bigjools suggested that a couple of days ago.
<wgrant> It might work.
<wgrant> But it's definitely piling more on top of a shaky system.
<james_w> the difficultly with other schemes is not materialising lots of rows for each of a 1000 builds every time you want to find the next build to dispatch, or to score them all
<james_w> a stored procedure to calculate scores might be an option
<james_w> otherwise you are looking at caching scores, which brings you back to something like you have now
<wgrant> james_w: Yes, that's the trouble :(
<wgrant> james_w: Alternatively we could have a materialised queue somewhere other than an RDBMS.
<wgrant> Like, something that models a queue.
<james_w> wgrant: that's true
<james_w> I'd be wary about behaviour if whatever that was went down and had to be repopulated from the DB
<wgrant> Yes...
<james_w> wgrant: do you know of something that would work well there?
<james_w> the few things I know are very simplistic in that aspect, and don't work well with high contention
<wgrant> I don't, no.
<lifeless> uhm
<lifeless> anything that requires O(queue length) is something I'd worry about
<lifeless> why not simply reduce the score at upload time - I suggested that last week
<wgrant> Possibly.
<wgrant> I guess that's reasonably flexible.
<wgrant> Simple reducing based on the number of pending builds for the archive is probably insufficient. But it might be a start.
<lifeless> score = current_score_calc - len(target_build_queue)
<lifeless> right
<wgrant> I think we should probably throw out everything we currently know about build scores, and work out what we actually want from then.
<wgrant> 2505 is a bit magical for my liking.
<lifeless> we should throw out build scores
<wgrant> Didn't you just say we should reduce them?
<wgrant> Isn't that inconsistent with throwing them out entirely?
<lifeless> short term vs long term
<wgrant> How do you propose to do away with them?
<lifeless> the problem with build scores is they try to create a single global static sort
<wgrant> Yep.
<lifeless> what we actually have is multiple queues vying for a single resource
<lifeless> modelling that directly, whether in rabbitmq or in postgresql, will be much more successful at giving a good experience
<lifeless> see for instance the reams of work done on network packet queuing
<wgrant> Right.
<james_w> bug 612351 was a bit of a headscratcher
<_mup_> Bug #612351: getPackagesetsForUploader uses Store.find, which doesn't return an SQLObjectResultSet <Soyuz:New> <https://launchpad.net/bugs/612351>
<jelmer> yuck
<lifeless> morning
<james_w> morning lifeless
<lifeless>  james_w thanks for that wiki page
<lifeless> james_w: but perhaps you could re-read the first sentence :)
<lifeless> or tow
<lifeless> bah
<lifeless> or two
<lifeless> thanks!
<james_w> fixed, thanks
<jelmer> 'morning lifeless, poolie, thumper
<thumper> hi jelmer
<thumper> how goes the plugin updates?
<jelmer> thumper: https://code.edge.launchpad.net/~jelmer/launchpad/update-bzr-foreign/+merge/31438
<jelmer> thumper: Does that answer your question ? :-)
<thumper> maybe...
<wgrant> jelmer: Do you have a moment to approve my latest changes to https://code.edge.launchpad.net/~wgrant/launchpad/refactor-nuf-creation/+merge/30851, or should I ask tomorrow?
<jelmer> wgrant: Was that your file classification branch? If so, I can have a look now.
<wgrant> jelmer: Yep, that's the one.
<jelmer> wgrant: Great, happy to re-review now then (once loggerhead cooperates)
<wgrant> jelmer: Yay Loggerhead.
<poolie> hi jelmer, wgrant, thumper
<thumper> jelmer: do we have a bzr-hg update?
<wgrant> It normally works after a refresh or two for launchpad branches.
<wgrant> Morning poolie.
<jelmer> thumper: No, none of the changes were really significant enough to warrant a round of QA and a release.
<thumper> jelmer: ack
<thumper> ah crap
<thumper> resolving conflicts between stable and db-devel
<thumper> and there are significant changes in the source package recipe build code
<thumper> on both devel and db-devel
<thumper> tests are failing locally
 * thumper needs to choose what to delete
<thumper> ââ¹
<lifeless> heh
<lifeless> we're working to eliminate this source of waste
<wgrant> If I'd already convinced lamont and bigjools of https://code.edge.launchpad.net/~wgrant/launchpad/multi-arch-builders, buildd admins could just flip most of the excess amd64 builders onto i386 :(
<jelmer> wgrant: what's the argument against that? Predictability?
<wgrant> jelmer: I don't know. I will ask about it again tonight.
<wgrant> Because it would be reallly handy.
<wgrant> There is the issue that you don't know if the builder can build amd64 or not.
<wgrant> But that will become a non-issue once LP gains support for dispatching multiple archs to the one builder.
<wgrant> But that means fixing the queue size estimation algorithms in a way that is not at all obvious.
<jelmer> Yeah, I can see how that would be an issue.
<jelmer> Then again, if we'd process things quicker there's less need for proper time estimation...
<wgrant> Yes...
<wgrant> We might also be able to take shortcuts if all of the x86 builders are amd64-capable.
<wgrant> It's going to be very difficult if we can't.
<wgrant> It's a bit sad that the main barrier to shorter queues is calculating the length of those same queues.
<lifeless> uhm
<lifeless> the estimating system is already terrible
<lifeless> don't worry about making it slightly less perfect if its an overall win
<wgrant> 'Slightly less perfect' meaning potentially a factor of three off.
<wgrant> The estimation system is fine, assuming buildd-manager efficiency is 100%.
<wgrant> Which it's about to get close to.
<lifeless> wgrant: it requires many more assumptions than that
<wgrant> That builds take the same amount of time each time, which is somewhat reasonable.
<lifeless> wgrant: firstly that all buildds are equivalent (false)
<wgrant> True.
<wgrant> And that there are no queue jumpers (false)
<lifeless> wgrant: secondly that package builds don't vary much (mmmm, maybe true, maybe false - see perf issues with lucid etc)
<lifeless> that too
<lifeless> and I'm sure there are other variables
<lifeless> as for a factor of three, union the queues for all, add 1/3 of that to both i386 and amd64, and you should be tolerably bad
 * thumper pushing conflict testfix
<wgrant> i386 queue > 1000 :(
<lifeless> jkakar: how would you guys feel about __nonzero__ on ResultSet ?
<thumper> wgrant: really? WTF?
<wgrant> Soyuz works around the lack of __nonzero__ in a couple of cases already.
<wgrant> thumper: That's right.
<thumper> wgrant: where do I see that?
<wgrant> thumper: https://launchpad.net/builders
<wgrant> This means that no recipe will build for a few days at least.
<thumper> wgrant: recipes now have the same score as PPAs :)
<thumper> wgrant: so they no longer drop to the bottom
<wgrant> thumper: Ah.
#launchpad-dev 2011-07-25
<wgrant> lifeless: Can we defer bugheat like bugsummary?
<wgrant> (no, it shouldn't be expensive, but it is, so it should GTFO)
<lifeless> wgrant: we can do something
<lifeless> however its inline in the rows being edited usually
<lifeless> or is there a project row being updated?
<wgrant> It's mostly the max_heat, I think.
<lifeless> yes, we should
<lifeless> garbo it up
<wgrant> Let me just add that to the critical queue... oh wait
<wgrant> :(
<lifeless> if it causes timeouts its already in there
<wgrant> Ooh shiny.
<wgrant> https://qastaging.launchpad.net/ubuntu/oneiric/+localpackagediffs?batch=10
<lifeless> ?
<wgrant> The packagesets column.
<lifeless> nice
<lifeless>  OOPS-2032QASTAGING19
<wgrant> 'This is the first version of the web service ever published. Its end-of-life date is April 2011, the same as the Ubuntu release "Karmic Koala".'
<lifeless> wgrant: 809786 timed out for me
<lifeless> when I filtered by core
<wgrant> lifeless: The page times out a lot anyway.
<wgrant> Without packageset filtering.
<wgrant> (drop the ?batch=10 and try to have it render)
<wgrant> Putting &batch=10 on the packageset-filtered URL works fine.
<lifeless> hmm, it should have preserved the batch size
<lifeless> I have closed the tab, but perhaps thats it
<lifeless> headdesk. There are too many ways to send mail to teams.
<spm> which is why procmail was invented. so as to auto delete the vast majority of email that LP sends, because there's no other way to manage it.
<StevenK> Harsh
<StevenK> But true
<lifeless> I got sufficiently pissed off with some mail - 815623
<lifeless> bug 815623
<_mup_> Bug #815623: Mail notifications sent to team admins on joins / leaves to open teams <Launchpad itself:Triaged> < https://launchpad.net/bugs/815623 >
<lifeless> grah
<lifeless> the way deactivated memberships are special cased for joining can be surprising ... and tis buggy
<nigelb> \o/ Fix released :)
<lifeless> man, that took way too long.
<nigelb> btw, New Zealand has snow?
 * nigelb wwas surprised of a friend stuck because of snow
<lifeless> http://www.stuff.co.nz/national/5334310/Wintry-blast-brings-country-to-a-standstill
<nigelb> woah.
<lifeless> can has review ? https://code.launchpad.net/~lifeless/launchpad/bug-815623/+merge/69021
<lifeless> diff is up
<lifeless> wgrant: ^ ?
* wgrant changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: - | Critical bugs: 239 - 0:[#######=]:256
 * wgrant looks.
<lifeless> or stevenk ^ ?
<wgrant> lifeless: I think we should talk to teams like LoCos first.
<wgrant> The LoCo Council has been furious with us a few times lately for making team policy changes like this.
<lifeless> this seems entirely different to the things they were (reasonably) unhappy with
<wgrant> Yes, but still.
<lifeless> but sure
<lifeless> whats their contact ?
<wgrant> There's loco-council@lists.u.c, but pinging czajkowski or popey directly might also work better.
<wgrant> BugTask.target is worse than Soyuz.
<StevenK> Haha
<lifeless> wgrant: anyhow, you can review the change; I'll hold of landing for loco response
<lifeless> s/you can/can you
<wgrant> Sure.
<lifeless> whats left on my todo list....
<wgrant> lifeless: DOne.
 * wgrant lunches.
<lifeless> quick supermarket run
<wgrant> I am fairly sure that nobody who ever touched anything related to BugTask.target knew about the concepts of encapsulation or layering.
<lifeless> are you creating a BugTaskTarget table ?
<wgrant> No.
<wgrant> That would be slow.
<wgrant> But any time non-model code puts its fingers near the target key attributes, they are being cut off.
<lifeless> actually
<lifeless> stub and I think it would be fast
<lifeless> it would shrink the task table size
<lifeless> make its constraints easier to read
<wgrant> Well, it will be very possible to do in a few branches.
<lifeless>  / evaluate
<wgrant> "in a few branches" == "once I've finished this current stack of refactoring", that is.
<lifeless> we'd probably want CTE's for the dimensions
<lifeless> cool
<wgrant> lifeless: I'm worried how slow it would be to query for, eg, all bugs in Ubuntu.
<wgrant> Because you'd need all DSP targets.
<wgrant> And conceivably all SP targets too.
<wgrant> CTEs may work.
<wgrant> But maybe not.
<lifeless> wgrant: that is indeed a factor; however...
<lifeless> there are 20K * small-int targets
<wgrant> Yeah.
<lifeless> this is a small number when you're dealing with table scans already
<lifeless> and a moderate number otherwise.
<wgrant> Anyway, my branch to remove access to sourcepackagename/product/productseries/distribution/distroseries from the security declarations came back with ~50 failures, most of which just need to be changed to use transitionToTarget instead.
<lifeless> awesome
<wgrant> Some of the other failures reveal about four or five branches of further necessary refactorings, but we're getting there...
<wgrant>     def maybeAddNotificationOrTeleport(self):
<wgrant> So it might add a notification or possibly jump somewhere else?
<wgrant> Yay
<lifeless> rotfl
<lifeless> wgrant: hi
<lifeless> wgrant: SPRB's - you were involved with the current impl right ? the wellington sprint ?
<wgrant> lifeless: Yes.
<lifeless> in the -way-back- plans for no more source packages
<wgrant> I did various bits, then grabbed everyone else's bits and mashed them together at the end into something that almost worked with only about 10 breakage points along the way...
<lifeless> we were going to record a manifest describing each build
<wgrant> That was an amusing afternoon.
<wgrant> Ah, yes.
<lifeless> sprdata seems to match that design
<wgrant> We send it back.
<wgrant> I forget if we store it.
<wgrant> Let me check.
<lifeless> but we're not recording a version per build
<lifeless> we only have 1300 in prod
<lifeless> I want to know if thats something we planned to do when we need it
<lifeless> or if the actual intent changed
<lifeless> https://code.launchpad.net/~jelmer/launchpad/bfbia-db/+merge/68990 for context
<wgrant> jelmer and I discussed this a bit in Dallas.
<wgrant> But not details like this.
<wgrant> IIRC we already send the manifest back, but don't do anything with it.
<wgrant> THe intent was to parse it back into a SourcePackageRecipeData, and store it in SourcePackageRecipeBuild.manifest (which already exists).
<lifeless> ok
<lifeless> wgrant: so in short - original plan unchanged, but bits not implemented
<wgrant> Yes.
<lifeless> HI STUB
<lifeless> *cough*
<lifeless> Hi stub
 * stub rubs his ears
<lifeless> stub: hey, so -0 patches; am I right that we no longer use them at all in the new world ?
<lifeless> stub: I've optimistically updated the schema process page to say that
<stub> So previously, -0 where the ones being run during a full rollout. But now we are not going to have full rollouts (db or code, not both at the same time)
<lifeless> yeah
<lifeless> and -0 requires code and db to be in sync
<stub> We might as well just pull out the schema version detection code, or tune it for the new world.
<lifeless> in that the code says a -0 in the db the code doesn't know is boom, and vice versa
<stub> Can you think of a better rule? How about 'if it is in my tree but not applied to the db, fail'.
<lifeless> stub: well, we still have, in extreme cases, synced deployments - at least in principle, until we're past the first few of these deploys and can be sure we've got the kinks out
<stub> Or just switch it off entirely.
<lifeless> stub: I believe that 'if it is in my tree but not applied to the db, fail' is what -non-0 patches enforce
<stub> ok. so for now, no -0 patches. But we should fix that, as leaving it in there unused could bite us.
<lifeless> stub: so i guess in a few weeks or a month, we should make it the same as the -non-0 rule.
<stub> Sure. I don't see the point of supporting '-0 patches cause things to explode' :-)
<lifeless> me neither :)
 * stub opens a bug
<lifeless> stub: so you just need to stop and start pgbouncer for the tests right ?
<stub> lifeless: what is the secondary fastdowntime tag?
<lifeless> -later IIRC
<lifeless> its linked from the LEP
<stub> lifeless: Yes, that covers it.
<lifeless> https://dev.launchpad.net/LEP/FastDowntime -> https://bugs.launchpad.net/launchpad-project/+bugs?field.tag=fastdowntime-later
<lifeless> hmm
<lifeless> my less-mail change may hide some mails we care about - transitions to-from admin of members.
<lifeless> still, less of a wart than what we have today, IMNSHO
<wgrant> Oh, blah, that counts as joining too, doesn't it.
<lifeless> wgrant: setStatus too
<lifeless> wgrant: I think
<jtv> hi lifeless, hi wgrant
<wgrant> Evening jtv.
<wgrant> jtv: generate-contents-files may have just finished.
<wgrant> jtv: For the first time.
<jtv> wgrant: still morning here :)
<jtv> It must have taken a while because it didn't run for ages.
<wgrant> Not so much.
<wgrant> It started slightly under three hours ago.
<jtv> Oh
<wgrant> So not too much slower.
<jtv> Phew.   I thought you were saying it had been running all weekend.
<wgrant> It failed to run over the weekend, because it didn't have permissions to do the move.
<jtv> !
<jtv> So you spotted that and fixed it?  I am in your debt.
<wgrant> Ah, no, it's still going.
<wgrant> Must be nearly there.
<jtv> excitedâ¦
<jtv> morning rvba!
<rvba> Morning jtv, morning all!
<wgrant> Oh, it's not done powerpc yet.
<wgrant> So it's got a while left.
<wgrant> It was just being very quiet :(
<jtv> More things are LaunchpadCronScripts now, and weird things happen to logging when one of those instantiates another.
<jtv> That's something I'm looking into.
<wgrant> I think it's just apt-ftparchive being itself.
<wgrant> However, there is one significant bug in the new script.
<wgrant> Killing archivepublisher(5321) from launchpad_prod_3:
<wgrant> query: <IDLE> in transaction
<wgrant> backend start: 2011-07-25 04:02:12.038005+00:00
<wgrant> query start: 2011-07-25 04:02:12.207793+00:00
<wgrant> age: 0:57:49.467450
<wgrant> It seems to be continuing OK, but it remains to be seen how badly it blows up at the end.
<lifeless> meep
<wgrant> If it blows up it won't escape from its own little /srv/launchpad.net/ubuntu-archive/contents-generation world, so I'm not too concerned about it leaving the archive in a bad state.
<wgrant> A more interesting risk is that it might generate different output for a frozen suite.
<jtv> So apt-ftparchive is taking long enough that its transaction gets reaped?
<lifeless> uhm, please tell me you're not holding a transaction ope while you run apt-ftparchive ?
<wgrant> Yes, but what lifeless said.
<wgrant> No need for a transaction, and it takes hours so it's silly.
<wgrant> So any time at all is long enough.
<lifeless> this runs as a different user to the publisher, right ?
<wgrant> No.
<lifeless> ok, so this needs to be critical then, because its going to break downtime deploys (of any sort)
<lifeless> losas know that quiescing things (time period) in advance is enough, and time period is not (IIRC) 1 hour
<lifeless> assuming that this is a transaction-open-around-ftparchive
<wgrant> Er, you know about the 24h translations scripts, right? :)
<spm> when did they suddenly speed up to 24h?
<jtv> wgrant: I was just going to ask what lifeless asked.  :)
<lifeless> wgrant: not on the whitelist to abort
<jtv> That script has been tweaked to avoid holding transactions open.
<lifeless> wgrant: they'll get slaughtered, and we've had no response beyond the publisher to the requests asking for things to whitelist, both on-list and to team-leads
<jtv> How do we get certainty about what stage the script was in when it lost its transaction?
<spm> quite a few times aiui. it use to be a minor bane of our existence. but haven't seen any woes from it for ... well years.
<lifeless> jtv: yes, that script is fine
 * lifeless wants to move all scripts to internal API clients
<wgrant> 2011-07-25 04:58:39 WARNING  dists/maverick/restricted/binary-amd64/: 28 files 112MB 0s
<wgrant> 2011-07-25 05:00:58 WARNING  dists/maverick/universe/binary-amd64/: 24080 files 24.7GB 2min 18s
<wgrant> 2011-07-25 05:01:03 WARNING  dists/maverick/multiverse/binary-amd64/: 700 files 2871MB 4s
<wgrant> It was kill at 05:00 UTC
<wgrant> It was during a-f.
<wgrant> killed.
<jtv> OK, so we need to make sure there's no transaction open at that point.
<jtv> It may have been the ORM reloading objects.
<jtv> That won't allocate a full transaction in modern-day postgres, but I'm not sure our scripts would know that.
<jtv> I'm filing a bug.
<jtv> BTW since it's monday morning: this is generate-contents-files, right?  Or is it publish-distro (or publish-ftpmaster running publish-distro)?
<wgrant> generate-contents-files
<jtv> bug 815725
<_mup_> Bug #815725: Long-running transaction in generate-contents-files <Launchpad itself:In Progress by jtv> < https://launchpad.net/bugs/815725 >
* wgrant changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: - | Critical bugs: 240 - 0:[#######=]:256
<lifeless> jtv: thanks!
<jtv> wgrant: there are no transaction boundaries at all in that script, yet I haven't found any traces yet of it running in auto-commit previously.
<wgrant> jtv: That script didn't exist three months ago.
<wgrant> It's never run before.
<lifeless> jtv: wgrant: is there any reason we can't change the db user for this script as well ?
<wgrant> Well.
<wgrant> It was a shell script.
<wgrant> lifeless: Maybe.
<wgrant> lifeless: Just needs work to find out DB users.
<lifeless> what I want is stub's whitelist to get no false positives
<jtv> Ah, drat, this was a shell script.
<wgrant> We really need something like User-Agent.
<lifeless> things we need to abort deploys on need to be precisely and accurated identified by the whitelist code
<adeuring> good morning
<jtv> We can probably change that without any problemsâ¦  focusing on the other thing right now though.
<jtv> hi adeuring
<adeuring> hi jtv!
<stub> jtv: might need to focus sooner than later if you don't want your script killed every few days
<jtv> stub: thanks for distracting me from just that.  :)
<wgrant> Is archivepublisher whitelisted for reapability?
<jtv> It does a lot of commits, so might not need it.
<stub> wgrant: it will be - bigjools flagged that. So rollout will abort until that stuff has been shut down manually.
<lifeless> stub: actually its the other way around; this script runs as something we aren't willing to interrupt, so we need to move it out of the way or cause deploys to refuse to run
<jtv> wgrant: can you think of any reason why generate-contents-files shouldn't run against the read-only store?
<wgrant> jtv: There is no read-only store.
<wgrant> Which could be an issue.
<wgrant> (read-only mode has gone away)
<jtv> How is there no read-only store?
<stub> ISlaveStore(DbClass) is the read-only store
<lifeless> jtv: FYI we'll be interrupting transactions on the slaves too, not just the master
<wgrant> Yes, but that doesn't help deployments.
<jtv> wgrant: read-only store, not read-only mode.
<stub> And using it makes things go faster which helps everything
<jtv> Not thinking of deployments, thinking of not getting the script killed.
<wgrant> It's still going to be killed.
<lifeless> jtv: it still will, same rules
<jtv> But much easier to deal with.
<wgrant> Oh?
<jtv> Database changes are the real bastard.
<wgrant> There should be a decorator/contextmanager around somewhere to ensure a lack of transaction.
<jtv> That'd be nice.
<wgrant> checkwatches uses one, but it's Twisted.
<lifeless> abently wrote it I believe
<wgrant> That'
<wgrant> s right.
<wgrant> For upgrade-brances
<wgrant> TransactionFreeOperation or something
<stub> There is a database policy we use to guarantee no db access is being made.
<jtv> Anyway, to repeat the question: does anyone know of any reason why this script shouldn't run against the read-only store?
<wgrant> As long as it's up to date, no, that's fine.
<lifeless> jtv: unless it changes things, running against a fresh slave should be fine.
<stub> (and database policies are context managers)
<jtv> lifeless: I'm asking wgrant whether he can think of anything it changes.
<wgrant> It shouldn't.
<wgrant> It will probably need to eventually.
<wgrant> But right now it doesn't.
<wgrant> Or at least shouldn't.
<lifeless> it should make API calls anyway
<wgrant> Eventually, yes.
<lifeless> if it doesn't change anything now, there is no good reason for it to ever.
<lifeless> (via the db)
<jtv> stub: is SlaveOnlyDatabasePolicy what I need?  (What I care about really is a read-only store to catch db changes, not necessarily an actual slave database).
<stub> jtv: Sounds like that is what you want.
<stub> We use DatabaseBlockedPolicy to confirm no db access at all for pages we need to work when the db is unavailable.
<jtv> OK, I'll do that first.  If I understand correctly, that'll keep the reaper off its back _for the time being_, so we can land it separately, and then feel much more confident changing transactionality later.  Right?
<stub> Can also be used for narrowing down long-transaction issues
<jtv> Which is what this is.
<lifeless> jtv: I think that still takes out a transaction
<lifeless> jtv: so that it gets consistent reads
<lifeless> jtv: which is why wgrant and I were saying it won't help.
<jtv> Yes it does.
<lifeless> because it will still get reaped.
<jtv> Oh, I thought you were saying the reaper currently only kills master transactions.
<wgrant> The regular reaper may well only kill master transactions.
<wgrant> The deployment reaper kills everything.
<lifeless> jtv: we're starting fastdowntime deployments very soon - stub has made fantastic progress, it may be as early as wednesday.
<stub> We have a reaper installed on hackberry too now
<lifeless> jtv: and that will nuke all connections on all replicas
<wgrant> lifeless: I think we should JFDI and watch what breaks.
<lifeless> stub: choke as well ?
<jtv> Okay, then I'll have to do both right now.
<lifeless> wgrant: oh, we will
<stub> lifeless: chokecherry too
<lifeless> wgrant: I'm mainly worried about this causing a fastdowntime to abort if it happens to have a transaction open at just the wrong time
<wgrant> lifeless: Yeah.
<lifeless> maybe we should change the db user for the archivepublisher
<stub> jtv: So I'm not sure what the original problem is, but the fallback is we whitelist the db user your script is connecting as. This will block rollouts until the script has been shut down manually. Update Bug #809123 if we need this.
<_mup_> Bug #809123: we cannot deploy DB schema changes live <fastdowntime> <qa-untestable> <Launchpad itself:In Progress by stub> < https://launchpad.net/bugs/809123 >
<jtv> stub: I suspect this will be a rollout blocker regardless, since it works on the filesystem â wgrant will know.
<lifeless> this works in a staging area
<lifeless> its not a blocker
<wgrant> As lifeless says, it's OK.
<lifeless> whats the cron time for this beastie ?
<wgrant> 04:02
<stub> lifeless: it will not abort the fastdowntime deployment. We will check the whitelist, and if no whitelisted connections, we shut down pgbouncer. If a script managed to sneak in a connection between the two steps, it will die.
<lifeless> wgrant: do you think it will take 4 hours routinely ?
<wgrant> lifeless: Rarely more than 2.5, I expect.
<wgrant> lifeless: But we will see.
<lifeless> ok, we're probably safe then.
<wgrant> stub: Well, it will abort it, just before we are down.
<lifeless> stub: this script we're talking about runs as the same user as the archivepublisher which is whitelisted
<lifeless> stub: which is why it could abort the deploy; if it had a non-idle connection at the whitelist check time
<lifeless> stub: (or do you check for -any- connection ?)
<stub> ok. I'd call that blocking the deploy, not aborting it. I expect all the whitelisted systems would be shutdown manually before kicking off the update. The checks are just there to confirm that all that stuff really has disconnected.
<lifeless> stub: agreed
<lifeless> stub: so I'm thinking we should make this script be on a different user; or move the whitelisted script to a dedicated just-for-it-user
<stub> (I'm considering an abort as aborting mid-way, which is a problem as we might be partially updated)
<lifeless> stub: and make that a clear policy for anything that gets whitelisted
<stub> yes, it should be a different user per existing policies :-)
<stub> Every separate script should be connecting as a unique user already.
<lifeless> jtv: wgrant: bigjools: how many different scripts connect as archivepublisher?
<stub> Everywhere that is not happening is a bug.
<wgrant> lifeless: Let's not go there.
<stub> (and they know it ;) )
<jtv> lifeless: afaik, lots.
<wgrant> lifeless: But at least 14.
<jtv> But that's a matter for archeology.
<lifeless> ok
<jtv> Predates the policy, I think.  :)
<lifeless> so, action item here is ot move the actual fragile script to its own user.
<stub> jtv: Can you point me at the script?
<jtv> stub: cronscripts/generate-contents-files.py
<jtv> lifeless: "not" move or "to" move?
<lifeless> jtv: *to* move.
<lifeless> finger-fail.
<stub> jtv: so quick fix just hard code a new user in there, and add the new user in security.cfg to be an alias to archivepublisher.
<jtv> OK, I'll include that.
<stub> (4 lines, not including whitespace)
<lifeless> stub: jtv: I suggest doing the new user in the archivepublisher script, not this one.
<jtv> Is this really an either-or choice?
<lifeless> given there are so many scripts on the same user already.
<lifeless> jtv: not at all.
<lifeless> rephrasing
<lifeless> stub: jtv: I suggest doing a new user in the archivepublisher script, becuase thats the one we care about for deploys.
<bigjools> which archivepublisher script?
<jtv> Hi bigjools
<bigjools> (morning!)
<wgrant> publish-distro and probably process-death-row need care.
<jtv> lifeless: this is beginning to sound like a very different problem from the one I'm currently dealing with â can we do it as a separate bug (though presumably critical)?
<wgrant> p-d-r sometimes takes a while to run, because it doesn't look for PPAs that need work: it just looks at all of them.
<lifeless> jtv: yes
<bigjools> what is the problem?
<lifeless> bigjools: the contents generation script takes more than 1 hour to run
<StevenK> Uh, it has for a while?
<lifeless> bigjools: the pre-deploy suspension of crontabs is done one hour before
<wgrant> That seems premature and disruptive, but OK.
<lifeless> StevenK: i didn't say 'now takes' :P...
<bigjools> you can just kill it, it won't break anything
<lifeless> bigjools: the contents generation script runs as the same db user as the archivepublisher, which needs to be whitelisted
<lifeless> bigjools: so, the deploy script can't tell the difference
<bigjools> wtf does it need a DB user?
<bigjools> it used to be a shell script
<StevenK> I have work in progress as a in-my-spare-time project to move contents generation from cocoplum's disk to the DB
<lifeless> bigjools: you'll need to talk to jtv and wgrant for that question.
<wgrant> bigjools: So we can tell which script it is, so we don't stop deploys for it.
<wgrant> bigjools: At present it is indistinguishable from publish-distro.
<lifeless> bigjools: but do you see the issue ? only the things that can't be interrupted can use a db user which is whitelisted.
<bigjools> wgrant: no, I mean *why* does it need to touch the DB at all
<bigjools> lifeless: yes
<wgrant> bigjools: It used to use lp-query-distro and stuff.
<bigjools> gah
<wgrant> bigjools: Now it is Python.
<lifeless> bigjools: so I'm proposing we move the must-not-interrupt stuff to a new dedicated user which we whitelist.
<jtv> bigjools: it always got bits and bobs from the DB, such as configs.
<wgrant> It's just it used lots of short scripts, which also ran as archivepublisher.
<bigjools> right
<mrevell> Hi
<bigjools> morning mrevell
<jtv> hi mrevell
<StevenK> So it probably needs to grab configs and then delete the store
<lifeless> bigjools: do you see any gotchas or issues with doing that ?
<bigjools> lifeless: +1
<bigjools> at the worst, we just inherit DB permissions in the cfg
<StevenK> Since the rest of it does not require DB access
<bigjools> adding a new user is trivial
<jtv> There's slightly more than just configs, but I'm currently making the script grab data first, then commit & apply a "no DB" policy.
<lifeless> ok, can someone that knows which scripts are relevant, file a bug for this? as jtv says its a different issue from the generation script having too-long transactions.
<bigjools> jtv: it might be worth re-writing it to shell out to the short-running scripts that need db access
<jtv> And wait for ZCML to be parsed for each?
<bigjools> no bigdeal
<bigjools> compared to the hour-long txn :)
<jtv> I've already got that isolated here.
<wgrant> Please no.
<wgrant> Maybe add XML-RPC calls.
<wgrant> But no shelling out :(
<jtv> Harder to test, too.
<jtv> Anyway, I think I have this solved for apt-ftparchive.
<wgrant> Not to mention disgusting and side-stepping the user issue, as those scripts still connect as archivepublisher.
<bigjools> alternatively disconnect from the DB
<jtv> What other parts of the script might need the same treatment?
<wgrant> bigjools: That's what's happening.
<wgrant> jtv: That's the only long-running bit, right?
<jtv> I'm not 100% sure, though it looks like it.
<wgrant> jtv: The rest is just trivial setup and some moves that might take a second or two.
<jtv> There's some "cp" going on as well, butâ¦
<jtv> Then what I have should do the trick.
<wgrant> I thought they were mvs, but maybe not.
 * lifeless tags wgrant to file the bug
<jtv> Puts the whole script in read-only DB access, and gives the sensitive part no db access (or transaction) at all.
<wgrant> lifeless: :( OK
<wgrant> lifeless: High?
<wgrant> Or Critical?
<wgrant> Tempting to go to critical.
<wgrant> I want to have an excuse to find a unicode explosion to put in /topioc
<lifeless> critical
<stub> https://code.launchpad.net/~stub/launchpad/db-cleanups/+merge/69038  to switch three cronscripts from connecting as the archivepublisher db user.
<wgrant> Bug #815753
<_mup_> Bug #815753: process-accepted, publish-distro, process-death-row and generate-contents-files should use their own DB users <fastdowntime> <Launchpad itself:Triaged> < https://launchpad.net/bugs/815753 >
<lifeless> do we still use _pythonpath ?
<lifeless> (also imports with side effects? ueeeeeep)
<wgrant> Yes.
<stub> I can't recall if buildout made that irrelevant or not.
<poolie> ultra-teeny review, anyone? https://code.launchpad.net/~mbp/launchpad/721166-test-gc-warnings/+merge/69040
<wgrant> It should for buildout-generated scripts.
<wgrant> But not for scripts/ and cronscripts/
<lifeless> stub: for you  - https://code.launchpad.net/~lifeless/python-pgbouncer/start-stop/+merge/69041
<StevenK> poolie: Uh? When?
<lifeless> stub: looks like you're missing some scripts if that bug is correct?
<StevenK> poolie: I keep seeing the lockwarner garbage on Jenkins
<poolie> hm, do you
<poolie> causing those tests to fail?
<bigjools> lifeless, stub: I am a bit worried about long transactions in our derived distros scripts, the initdistroseries job can take hours to run.  While that should be quicker, it holds a txn open. :(  But it could possibly take a very long time and I need a complete backout if it goes wrong.  What can we do?
<stub> lifeless: Probably. I just did the ones I could find.
<StevenK> poolie: Oh, that was _lock_actions
<poolie> StevenK: which bzr does it have?
<StevenK> https://lpci.wedontsleep.org/job/db-devel/lastFailedBuild/
<poolie> oh, complaining about tests exiting with files locked?
<lifeless> bigjools: use a schema which allows multiple commits as you do the work and can be backed out by deleting data if you fail.
<stub> bigjools: It holds the transaction open to allow it to rollback, or does it hold the transaction open to keep a clean snapshot of the data, or both?
<poolie> in particular if the test has already failed, bzr will complain it didn't unlock things
<lifeless> bigjools: the current schema /may/ support this, or may not.
<poolie> arguably it should have more of a sense of perspective
<bigjools> stub: to roll back
<StevenK> poolie: I have no idea, seven failures that keep cropping up.
<bigjools> stub: well actually both
<bigjools> lifeless: it does not :(
<poolie> StevenK: but not in ec2 for some reason?
<lifeless> bigjools: why doesn't it? I mean, AFAICT you have enough info to delete all the stuff afterwards
<lifeless> bigjools: using the job to record your state machine state
<stub> bigjools: So one approach is to store a snapshot of the data you need in a holding area, store the results in a holding area, and on completion in a single transaction pour the results into the final location and purge.
<bigjools> lifeless: what if there's a failure that brings the script down hard?
<poolie> StevenK: i think at least filing a bug with what data we do have would be worthwhile
<bigjools> stub: right
<bigjools> temp table maybe?
<poolie> i would guess those tests need to run some bzr test framework setup or inherit from a bzr class and they're not
<lifeless> bigjools: so lets say we have the job row
<StevenK> poolie: So, we have three different methods for running the test suite, and all 3 have different failures
<poolie> or, if they're failing only on wedontsleep, i would wonder if a dependency is out of date
<StevenK> poolie: They being buildbot, jenkins and ec2
<lifeless> bigjools: on startup, do a commit to it saying 'in progress' - update the json dict
<poolie> i'm pretty sure they're unrelated to what i'm changing here though
<stub> bigjools: A real table is better - with luck, we won't be able to rely on temp tables persisting across transactions any more.
<lifeless> bigjools: on completion, you remove the job or whatever.
<bigjools> stub, lifeless: ok this is the job that writes to SPPH and BPPH.
<stub> bigjools: or a file on disk even :-)
<bigjools> using the packagecloner or packagecopier
<bigjools> remember that the former is already used when opening new ubuntu series
<lifeless> bigjools: if you die hard in the nmiddle, then the job runner will see the job showing 'in progress' and can initiate recovery - looking up the new series, then deleting all the SPPH and BPPH entries for it.
<bigjools> lifeless: we must not leave incomplete SPPH and BPPH lying around, ever
<lifeless> bigjools: why not ?
<bigjools> because other parts of the code use their presence as an indicator
<lifeless> bigjools: you've got a series marked 'uninitialised'
<lifeless> bigjools: that other code should'nt be looking at anything in that series yet.
<bigjools> lifeless: nearly -  we've got an uninitialised series marker as initialised
<StevenK> poolie: The build slaves would install the latest bzr from lucid that they can
<bigjools> s/marker/marked/
<cjwatson> jtv: step 13 should be kept.  it doesn't have to involve running code on cocoplum (we could compare them remotely instead), but as an operational matter I don't want Ubuntu people opening new releases without sanity-checking the resulting Packages files.  This shouldn't have to be something that LP people worry about
<lifeless> bigjools: so my point is to keep it marked uninitialised until the job completes.
<bigjools> lifeless: it's a lot of very old legacy code all over the place
<bigjools> I can't easily change this behaviour
<poolie> StevenK: if it's really 'the latest from lucid' i'm a bit surprised more things don't fail
<bigjools> otherwise I'd love to do that
<jtv> cjwatson: thanks.  The surrounding steps will become automatic, so it'll become another matter of "wait for the right script to run."
<poolie> since lp itself uses the bzr from maverick-updates
<poolie> (or something similar)
<poolie> no, correction, natty-updates
<lifeless> bigjools: I don't understand how anything would look at SPPH/BPPH rows for a series that isn't live yet.
<lifeless> bigjools: can you clue me in a bit more ?
<poolie> shouldn't you be using some ppa at least?
<lifeless> poolie: there are several different bzr's in use
<bigjools> lifeless: the UI, the publisher for starters.
<StevenK> poolie: Er, sorry. I do use the bzr ppa
<poolie> the lucid version of ppa:bzr ?
<poolie> that should be close enough
<lifeless> poolie: LP hosting uses a copy in sourcecode/, devs use the ppa, recipe builds use the one from the distro its running on IIRC
<cjwatson> jtv: yep
<cjwatson> jtv: that much is fine, certainly
<poolie> i know
<lifeless> bigjools: the UI queries across all series for a distro unconditionally ?
<bigjools> lifeless: I think doing what stub said makes sense - if I have a copy of SPPH/BPPH somewhere and write into those using the tuner, then I can mass-copy quickly later
<poolie> my point is trying to run lp's bzr-integration test using a version of bzr very different from that normally used is likely to generate noise results
<poolie> but apparently we're not doing that
<cjwatson> could somebody review https://launchpad.net/~cjwatson/+archive/launchpad for merging into the Launchpad PPA?
<cjwatson> dependencies for dpkg-xz-support and multiarch-translations
<cjwatson> (or tell me I need to ask somewhere other than IRC ...)
<stub> bigjools: Hopefully you wouldn't need the whole SPPH/BPPH - they are rather large and will take time and energy to build a copy.
<lifeless> bigjools: what stub suggests will work as well, but I don't follow why my suggestion won't - I mean I cam imagine some buggy code thats not honouring the active flag etc, but nothing /fatal/ surely?
<bigjools> lifeless: not exactly - but various pages use the presence of packages to trigger different bits of info display, etc.  It may work, it may not.  The publisher will also start trying to publish an incomplete series which is a nightmare because it requires manual intervention to remove all the files and let it run again.
<jtv> cjwatson: ahhh, and step 15 already says to do that so I can move it behind that.  Getting pretty bare, that part.  (Sorry, can't help with review just now; working on Critical).
<bigjools> lifeless: there is no "active" flag
<cjwatson> jtv: it can indeed
<jtv> cjwatson: you may want to look up the reviewer schedule on dev.launchpad.net, to see who's supposed to be on call.
<lifeless> bigjools: so the publisher should check the initialised flag, which would be atomic in my proposal, and that should be pretty easy to add.
<lifeless> bigjools: i don't think we'd care a *lot* if the UI is a little messed up as the new series populate.s
<cjwatson> henninge: ^- I guess that's you?
<lifeless> bigjools: copying the 60K rows or so that will be needed will still be pretty slow I fear.
<bigjools> lifeless: actually we might be ok with the publisher because it waits for the series to be moved from experimental to frozen before it does the careful publication run
<bigjools> lifeless: 60k is overestimating a lot.  It'll be nearer 4k.
<bigjools> or even 1k
<henninge> cjwatson: I am
<poolie> StevenK: so...?
<stub> I hope that fixing the existing high level bugs (eg. publishing uninitialized distributions) is easier than implementing an unnecessarily convoluted workaround.
<henninge> I mean "it is"
<lifeless> bigjools: when we open a new Ubuntu ?
<cjwatson> henninge: either way :-)
<henninge> ;-)
<henninge> cjwatson: let me have a look at it
<lifeless> bigjools: 20K source, some K binary-all, some K binary-any ?
* henninge changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: henninge | Critical bugs: 240 - 0:[#######=]:256
<bigjools> lifeless: yeah that one will be larger :(
<bigjools> but it only takes 10 minutes
<StevenK> poolie: Sorry, too much discussion in here, and you're not hilighting me
<cjwatson> henninge: it amounts to some backports of xz support plus a backport of LongDescription support in apt-ftparchive; there's also some stuff in lucid-updates which we'll get without fuss
<poolie> StevenK: so can i get a +1?
<lifeless> bigjools: so the worst case we know today is 10 minutes ?
<henninge> cjwatson: um ... this is not a code review, is it?
<bigjools> lifeless: no, there's another scenario I tested on Friday with DF and it took 4 hours :(
<lifeless> bigjools: what was that scenario doing ?
<cjwatson> henninge: no, I don't know the process for getting things into the Launchpad PPA; bigjools told me the other day that I should stage stuff in a PPA and then you (plural) could review it and copy it in
<bigjools> inheriting a new distro from 2 parent distros, it was only about 100 sources so we have a performance bug somewhere, but still ...
<bigjools> we can fix that, but it's still likely to take a long time.
<henninge> cjwatson: I am sorry I don't know the process either nor how to reviw it ...
<bigjools> lifeless: how quickly can PG copy 60k rows?
<jtv> bigjools: on a sidenote, did you also confirm our suspected problem with "unique" packages?
<bigjools> jtv: I forget which problem that is
<cjwatson> henninge: OK, should I send mail somewhere?
<henninge> cjwatson: the launchpad-dev list would be a good start.
<jtv> bigjools: where we had lots of packages in +uniquepackages, and we hypothesized that it was because they get DSDs created if they're missing from _any_ parent rather than just if they're missing from _all_parents.
<cjwatson> henninge: OK, will do, thanks
<henninge> cjwatson: sorry to not be of any more help.
<bigjools> jtv: ah right, I've not looked at the UI yet!
<bigjools> jtv: I shall run the populate script now
<jtv> cjwatson: I've updated the release procedure.  Simple cut&paste job
<jtv> Hmm actually it's still savingâ¦
<jtv> Done.
<jtv> cjwatson: would you mind checking for blunders on my part?
<jtv> bigjools: just updating the Ubuntu release procedure to get rid of the manual initial careful publisher runs.
<bigjools> jtv: cool
<poolie> henninge: can you review my 0-line mp ?
<poolie> https://code.launchpad.net/~mbp/launchpad/721166-test-gc-warnings/+merge/69040
<henninge> poolie: I am on it right now.
<henninge> poolie: let me run the test on my machine, too, just as a second check.
<poolie> thanks
<lifeless> bigjools: reasonably quickly, limiting factor can be how its copied ;)
<lifeless> as in select into vs insert*60K vs insert values (60K items)
<bigjools> lifeless: jtv reckons a minute
<bigjools> lifeless: yeah would be select into if we have a "staging" SPPH/BPPH
<jtv> very rough guess, so grain of salt etc
<bigjools> is that ok for the length of one txn?
<lifeless> anyhow, as stub says, '20:56 < stub> I hope that fixing the existing high level bugs (eg. publishing uninitialized distributions) is easier than implementing an unnecessarily convoluted workaround.
<lifeless> '
<bigjools> it is not
<bigjools> basically because I have no idea where all these assumptions in the code are being done
<lifeless> bigjools: 1 min would be tolerable, particularly on new data with references to things we don't delete
<bigjools> this is my preferred route, by far.
<lifeless> bigjools: do you think you could identify things that will have destructive side effects? (a smaller subset than all-places-which-assume-all-series-are-valid)
<bigjools> because it means we can just add a loop tuner to the cloner/copier.  the latter will be a PITA though
<bigjools> lifeless: I can think of a few.  But I'm not certain I have them all.
<lifeless> can has review ? https://code.launchpad.net/~lifeless/python-pgbouncer/start-stop/+merge/69041
<jtv> lifeless: I'll trade you this one.  https://code.launchpad.net/~jtv/launchpad/bug-815725/+merge/69056
<bigjools> wgrant: can you think of any critical bits of code, apart from the publisher, that will do the wrong thing if we have SPPH/BPPHes from a failed initseries run?
<wgrant> bigjools: Very, very amusing things will happen in the publisher and other places that are difficult to avoid.
<wgrant> bigjools: eg. stuff that checks if there are any pubs left.
<jtv> Hmm no Launchpad, I did not claim that review an hour ago.  Did you mean "sometime in the past hour, actually the past minute"?
<wgrant> bigjools: There's a bit of that around, and it's going to be hard and unobvious to exclude uninitialised series from that.
<bigjools> wgrant: yes my thoughts exactly
<wgrant> It's certainly the best solution. But it is difficult and potentially disastrous.
<wgrant> Bug #814643
<_mup_> Bug #814643: Don't use the cloner for FIRST initializations because it only considers  the release pocket. <derivation> <Launchpad itself:In Progress by rvb> < https://launchpad.net/bugs/814643 >
<wgrant> Wouldn't it be best to fix the cloner?
<wgrant> The copier will take almost literally forever.
<cjwatson> jtv: looks fine at a brief glance
<jtv> thanks
<wgrant> bigjools, rvba: ^^
<bigjools> wgrant: the cloner needs to due
<bigjools> die
<wgrant> bigjools: Maybe once the copier isn't a steaming pile.
<wgrant> It is slow and full of special cases.
<cjwatson> jtv: in any event I'll review the changes to the process before executing it, so you guys don't need to ask about each single change
<wgrant> The cloner is trustworthy.
<bigjools> hahahaha
<wgrant> You cannot deny it is far simpler and more verifiable :)
<bigjools> wgrant: rvba raised this point, to be fair
<bigjools> wgrant: but unless someone initializes the world, the copier is fine
<jtv> cjwatson: otoh there's a lot of value for us to know sooner what we did wrong.  Having to come back to something much later adds significantly to the cost for me.
<wgrant> bigjools: You're suggesting that nobody is going to want a full copy of Uubntu?
<bigjools> wgrant: correct
<bigjools> most people will be doing overlays
<wgrant> And those that don't will block things for several hours :/
<wgrant> Also, convincing the copier to copy into staging tables?
<wgrant> Impossible to do consistently, unless you rewrite it.
<rvba> wgrant: bigjools Do we have a way to evaluate how bad the copier will be for a full copy of Ubuntu?
<bigjools> wgrant: I don't think it's that melodromatic
<wgrant> We could easily test.
<bigjools> testing is the only way
<wgrant> We've never tried to copy a whole distribution before. I have my doubts that the copier will yield a usable archive, but it might.
<bigjools> wgrant: also how do we make packagecopier use the looptuner
<wgrant> packagecopier can't.
<wgrant> Something that wraps it has to.
<bigjools> yeah
<bigjools> so
<bigjools> when I was testing the cloner on mawson, a single INSERT took 2 hours
<wgrant> Ahhhhhhhhhhhhhhhhhhhh the races here are amazing.
<wgrant> Kill us all now.
<bigjools> initializing 300 packages from 2 parents took 4 hours :/
<wgrant> Yeah.
<bigjools> the copier was quicker!
<cjwatson> jtv: you're editing our process, if you did it wrong then I'll just edit it again :-)
<cjwatson> jtv: I'm at a conference this week and on holiday next week.  Blocking on me is a really bad plan.
<jtv> Okay, if you insist.  Next time, no warning!
<wgrant> bigjools: Violating all these assumptions is going be fun :/
<bigjools> wgrant: AAAAAAAAAAAAAAAAAAAAAAARRRRRRRRRRRRRGGGGGGGG
<wgrant> Hm?
<bigjools> just this whole thing
<wgrant> Ah.
<bigjools> I am seriously fed up with it
<wgrant> This situation is so perilous that even I cannot be overdramatically negative about it.
<bigjools> it's not perilous.  The problem is dealing with long transactions.
<wgrant> All the solutions are perilous.
<bigjools> I still think writing into staging tables in batches is the way to go
<bigjools> easy to do and not at all perilous
<wgrant> Staging table => you'll need to rewrite tonnes of stuff to check within the tables, and you'll open up hours of race conditions in everything.
<wgrant> Rolling back by deleting => you'll need to track down everything that might deal with an uninitialised series and kill it, and you'll still open up hours of race conditions in everything.
<bigjools> yes :/
<wgrant> Either way you have loads of code to rewrite buggily.
<bigjools> I dunno about rewriting loads of stuff, we just make the checker optionally use the staging table
<wgrant> And race conditions that probably won't expose themselves until they wipe out an archive or something :/
<wgrant> bigjools: And what about GC and other consistency checks that run outside the copier?
<bigjools> what races are you thinking of/
<wgrant> Well, assuming we do expiry at some point for primary archives, which is probable...
<bigjools> bear in mind that we're only doing this at init time
<wgrant> The expirer will need to check the staging tables.
<bigjools> staging tables would only have data when initialising
<wgrant> Yes.
<bigjools> nothing else will be happening until init is finished
<wgrant> So I start an initialisation, and in the ensuing hours the origin has more uploads, and some of its files expire.
<bigjools> ah the source
<wgrant> You can't initialise from extra parents except when initialising a new series, right?
<wgrant> s/new series/new distribution/
<bigjools> right
<wgrant> That mostly eliminates copychecker races.
<bigjools> yup
<wgrant> However, I think batched insertion into the real tables is probably easier to get right.
<bigjools> *cry*
<wgrant> I'm trying to think of stuff other than deathrow that would need to be suspended for partial archives.
<wgrant> Well, deathrow and the publisher.
<jtv> lifeless: are you reviewing that branch I offered to trade you?
<bigjools> wgrant: yes, me too.
<cjwatson> jtv: I've subscribed to NRCP now, so I'll be notified of changes without you guys having to explicitly ask me
<wgrant> bigjools: AFAICR no DB GC is done during the operation, except condemning publications.
<jtv> Can't we just have a "poison" state for distroseries?
<wgrant> So it's only that and FS GC that is worrying, really.
<jtv> thanks cjwatson â bigjools: Colin now subscribes to the Ubuntu release-process page so he'll notice automatically when we change it â no general need to coordinate first.
<bigjools> wgrant: everything in the publisher needs to stop until initialised.  Also the buildd-manager, the pkgcache, recipe branches?,  uploads, packagediffs, +initseries itself
<bigjools> jtv: step 10 needs to change, i-f-p is dead
<wgrant> bigjools: buildd-manager?
<wgrant> Oh, if we create builds early, true.
<bigjools> wgrant: we don't want it building for partially init...
<bigjools> this is why I wanted staging tables, but yeah, they present issues too.
<wgrant> pkgcache will sort itself out after 24 hours.
<bigjools> still a waste of processing time
 * bigjools has a thought
<bigjools> archive.enabled?
<bigjools> already stops a lot of stuff running
<wgrant> Something like that, perhaps.
<wgrant> But I don't think that flag itself is a good choice.
<wgrant> Smells of tech debt.
<bigjools> explain?
<bigjools> AFAICS it does exactly what we want
<bigjools> IDS can enable it on successful completion
<wgrant> It also does other stuff like revoking access to the archive. But possibly.
<wgrant> And we need to go further than it presently does, I suspect.
<bigjools> revoking access?
<wgrant> Only the owner can see the archive.
<wgrant> On the API and web UI.
<bigjools> hmm
<bigjools> does that work outside of PPAs?
<wgrant> Not sure.
<stub> <aside>So SPPH/BPPH both end in 'history', so putting stuff in there that doesn't yet exist is rather silly. So the staging area metaphor isn't that silly. We publish stuff from 'foo' to SPPH/BPPH and move the records over at the same time?</aside>
<wgrant> stub: Yes, if all of Soyuz didn't rely on [SB]PPH being consistent and complete.
<stub> And do we need to snapshot SPPH/BPPH, or can we get an effective point-in-time snapshot by just filtering on a timestamp?
<wgrant> It already makes invalid assumptions, but these are only for the length of a webapp transaction.
<wgrant> stub: Neither.
<stub> yer, just musing on long term stuff. I don't know enough about the problem to actually help ;)
<wgrant> stub: The hours-long initialisation process may create records in the staging table, and the source is, for example, garbage-collected for being unreferenced before the rows get moved across to the real table.
<wgrant> And now your distribution is broken.
<bigjools> wgrant: well we can fix the GC to look in staging as well
<wgrant> All of them?
<stub> Foreign key constraints can enforce that too between the holding area and the real tables.
<wgrant> Hahahahahahahahahahahahahahahahahahahahaahh
<stub> Which would require the snapshot of the rows you need from SPPH/BPPH
<bigjools> I think for the first time ever, I am faced with a choice here and the one to pick is really not clear
<jtv> henninge: hi!  Got a review for you, if you have time: https://code.launchpad.net/~jtv/launchpad/bug-815725/+merge/69056
<wgrant> bigjools: Both are terrible. But my preference is to go with handling partially initialised series.
<wgrant> Only by a little bit.
<bigjools> wgrant: I can see the attraction of that
<wgrant> But I think it will require less reworking.
<bigjools> given the 2 big problems with staging
<wgrant> And any problems relating to partial initialisation are probably going to be clearer than the races around the staging area.
<bigjools> yes
<bigjools> so we loop tune batches to the copier, which is fine
<bigjools> and we'd need to do the same to the cloner, because that single insert of 2h is concerning.  Maybe it is missing a key
<stub> Can a partially initialized distroseries break other distroseries, or just itself (and any descendants)?
<wgrant> stub: Others as well.
<wgrant> Only in the same archive, I think.
<bigjools> ?
<wgrant> But still others.
<wgrant> Well, maybe not break them.
<bigjools> in what ways are you thinking of?
<wgrant> First thing that jumps to mind is publication condemnation.
<stub> I'm wondering if you can ignore any breakage for now, because you can just reset anything that might be broken once the initialization is complete.
<wgrant> New series is partially initialised.
<wgrant> Old series has package superseded. Publisher sees it's still in new series, doesn't condemn it.
<bigjools> we can catch exceptions at the top level and just clean out all publications
<wgrant> Initialisation fails, is rolled back.
<bigjools> (in that series)
<wgrant> Pool file is now orphaned.
<bigjools> how does that orphan it?
<stub> Can it be initialized into a new, separate archive, then once initialized we switch it to the real archive and trash the temp archive?
<bigjools> stub: not really, same race conditions as using a staging table
<wgrant> stub: That has only slightly fewer consistency issues as the staging table.
<wgrant> s/as/than/
<bigjools> wgrant: how does that orphan it?
<wgrant> Ah, sorry.
<wgrant> It might not... I forget exactly how the two ends of this work.
<bigjools> I figured it'd just pick it up on the next run
<stub> Can we put in a big semaphore (global or on the archive) that just stops the processes that can't cope with a partially initialized distroseries from running until the initialization is done?
<wgrant> But there is something there that allows a removal to proceed if there are other publications left in the archive.
<wgrant> stub: Yes.
<wgrant> stub: Identifying those processes is the issue.
<wgrant> But that is the plan.
<bigjools> yup
<wgrant> bigjools: Ah, it might be removing sources that are still referenced by binaries.
<bigjools> long transactions are looking great :)
<wgrant> If it's still published anywhere else in the archive, a source publication can be removed even if it still has published binaries.
<bigjools> I am starting to think that the soyuz db model is wrong
<wgrant> Starting!?
<bigjools> sorry, sarcasm doesn't transfer well over irc
<cody-somerville> lol
<bigjools> wgrant: ok so that would be an unfortunate situation :(
<bigjools> but
<wgrant> Yes, but if we just suspend the publisher for such an archive it should be OK.
<bigjools> right
<wgrant> Just there are lots of places we might need to do this.
<bigjools> that was the plan anyway
<stub> Initialization will take maybe 5 hours, but how often will that be happening?
<bigjools> it should not take that long
<wgrant> We can use our well-defined interfaces and crisp layering to block inappropriate access at obvious points.
<bigjools> for most cases
 * bigjools loses mouthful of coffee
<wgrant> Sorry, I've been working on BugTask.target today.
<wgrant> BugTask.target and surrounding code is worse than Soyuz.
<bigjools> heh, that bad
<jtv> This is going to be interesting.
<bigjools> stub: so we have a problem in the cloner, an INSERT took 2 hours at least on mawson and it was only copying about 200 rows, so I'm wondering if it hit a lock
<stub> bigjools: not surprising when you have all these big arsed transactions running!
<bigjools> stub: that was the only one runn ing
<stub> bigjools: You would need to tell Storm to spit out its query log
<bigjools> how can I find out if there's a lock?
<bigjools> ok
<bigjools> LP_DEBUG_SQL?
<wgrant> bigjools: I also see similar very slow inserts on process-upload for a source.
<stub> yer, something like that
<wgrant> Every time.
<bigjools> hmm
<wgrant> That may be a cheaper way to see what's going on.
 * wgrant tries, unless you are already doing stuff.
<bigjools> I'll do it again
<stub> slow inserts might happen if there are missing indexes on columns being referenced
<bigjools> trivial to re-create on mawson
<wgrant> stub: I considered that, but we should be seeing terrible issues in more places in that case.
<stub> but 2 hours for 200 inserts is silly even if it is doing full tables scans of related tables
<wgrant> stub: Also, all the fkeys are primary.
<lifeless> jtv: no, I was (and am) AFK
<wgrant> As you would hope.
<stub> so inserts should not block except to confirm that referenced rows exist and unique constraints on the table can be satisfied.
<wgrant> Right, which is why this makes no sense and I gave up.
<lifeless> an update on the referenced rows will cause the insert to block
<lifeless> until the referenced row update commits / rollsback
<bigjools> not sure how anything can reference it
<bigjools> stub: check the last query, that's the one that takes 2 hours.  http://pastebin.ubuntu.com/651690/
<stub> If you are inserting a row with a .owner, the corresponding record needs to be looked up in Person. If that row has been updated by another transaction, it may or may not exist so be has to wait until that transaction is committed or rolled back.
<bigjools> pg_stat_activity says that's the only query running
 * wgrant narrows eyes.
<wgrant> ed       pts/3    chinstrap:S.0    28Feb11  6.00s  0.57s  0.23s /bin/bash
<wgrant> That wasn't there 5 minutes ago.
<wgrant> Yet 28Feb11...
 * wgrant swats mawson for lying.
<bigjools> screen
<stub> bigjools: So that select inside the INSERT, when I run it, is fast and yields 0 results.
<bigjools> ?
<bigjools> stub: on mawson?
<stub> bigjools: on production
<bigjools> not surprised you get 0 on prod :)
<stub> query plan seems relatively sane anyway
<bigjools> ok
<bigjools> have you got access to mawson?
<stub> there are some nested loops which could make bad estimates cause ugly runtime
<lifeless> jtv: I will review it later if noone else has, can't now sorry!
<stub> bigjools: no access to mawson
<StevenK> stub doesn't need mawson, he has production!
<bigjools> different DB may produce different query plan
<bigjools> ARGH
<bigjools> it has a crazy plan
<bigjools> stub: http://pastebin.ubuntu.com/651693/
<bigjools> 2 seq scans on very big tables
<bigjools> I've got some deja vu about this
<stub> that would do it
<bigjools> next question is, why
<lifeless> beyond fits-in-memory, the bigger the table the smaller the % of data access needed for a table scan to be more efficient.
<lifeless> (with a limit when you get below an expected one-row per page of access
<lifeless> jtv: what was that branch again ?
<bigjools> lifeless: that seems odd, given it takes 2 hours
<jtv> lifeless: https://code.launchpad.net/~jtv/launchpad/bug-815725/+merge/69056
<lifeless> bigjools: well, its a rule of thumb that I'm fairly sure the stats based query cost estimators will be running into
<stub> bigjools: How does http://paste.ubuntu.com/651697/ look on mawson?
<lifeless> bigjools: rows per page * rows expected * random-io-cost vs pages-in-table * sequential-io-cost
<bigjools> stub: better! http://pastebin.ubuntu.com/651698/
<stub> Man, that is the most disgusting thing to come out of my nose in ages.
<jtv> Could possibly be eased a tiny bit further by moving the SPN out of the query and passing ids instead.
<bigjools> but you need to get the IDs from somewhere
<jtv> lifeless: rows per page * rows expected?  That seems odd.
<stub> maybe, but what bigjools got seems fine
<lifeless> jtv: blah, 2318 here
<stub> bigjools: hmm... you have a temp_ index in there.
<lifeless> jtv: pages = rows expected / rows-per-page
<jtv> lifeless: that sounds more like it
<bigjools> stub: grrrr!
<stub> bigjools: I don't have that on production, so either you or someone created or you got a backup made when I was testing stuff.
 * bigjools wants to stab whoever does that
<stub> In which case, might want to drop that index to confirm the query
<stub> temp_ is normally me. It might have existed when the backup you used was kicked off.
<bigjools> ah
<bigjools> dropping it ...
<jtv> stub: I've also created them sometimes.
<stub> So its his fault :-)
<henninge> jtv: r=me
<jtv> Always is.
<jtv> thanks henninge
<jtv> lifeless: it's already done!
<wgrant> jtv: http://archive.ubuntu.com/ubuntu/dists/oneiric/
<wgrant> Success
<wgrant> And http://archive.ubuntu.com/ubuntu/dists/natty/ still has the old ones, so the output is correct.
<jtv> wgrant: great, thanks!  Was this despite the reaped transaction?
<wgrant> Yes.
<jtv> Heh
<wgrant> 2011-07-25 09:56:38 WARNING Done. 802GB in 679023 archives. Took 5h 53min 54s
<jtv> Not really what I'd call a warning, butâ¦
<stub> ;)
* wgrant changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: henninge | Critical bugs: 239 - 0:[#######=]:256
<stub> 802GB. How do we squeeze that onto a CD again?
<lifeless> we don't :P
<bigjools> 679023 archives.
<bigjools> O_o
<stub> Insert disk 1 of 1146 to continue
<bigjools> I bet you just divided 640MB into 802GB didn't you?
<stub> Well I'm not going to volunteer to bloody bzip2 it all
<bigjools> I'm chuckling at the fact you did it rather than make up the number of disks :)
<stub> 700MB :-P
<bigjools> overburn :)
<stub> Glad we canned shipit anyway
<jtv> bigjools: besides being obviously insane, do those numbers look sane?
<bigjools> jtv: NFI
<stub> The numbers are just used to draw graphs. You expect them to mean something too?
<allenap> henninge: Fancy a review? https://code.launchpad.net/~allenap/launchpad/localpackagediffs-show-changed-by-bug-798865/+merge/69072
<henninge> allenap: sure ;)
<allenap> Thanks.
<wgrant> jtv, bigjools: It and the old script generate all supported suites each run, even frozen ones.
<wgrant> jtv, bigjools: It then copies over any Contents files that differ.
<stub> bigjools: So you are ok optimizing the sucky query using my simplification?
<lifeless> jtv: ah well, you get two for the price of one :P
<wgrant> Since http://archive.ubuntu.com/ubuntu/dists/natty/ shows mtimes in April, the output is identical to the old script.
<bigjools> stub: it looks quite different, but is it a drop-in replacement for the SELECT inside the original?>
<stub> bigjools: Think so, yes.
 * bigjools wonders how long it can take to drop an index
<jtv> That long-transaction fix is landing, by the way
<lifeless> bigjools: it should be very fast... unless you have a read (or write) transaction open that has read from it
<stub> bigjools: should be fast
 * bigjools stops DF
<lifeless> bigjools: if they've read from it, they have a lock on the relation, and the drop cannot complete
<jtv> no other transaction that could be using that index?
<stub> even then, dropping the index in one transaction won't affect running queries using their old copy from MVCC
<lifeless> stub: no, but it it will block the drop, and the thing blocking the drop will block other readers trying to determine what indexes are available
<stub> What is the index? If it is unique it might be needed for integrity checks
<lifeless> stub: AIUI
<bigjools> it's frustrating that make stop depends on build :(
<stub> Just port the damn thing to Cassandra.
<lifeless> should get pgbouncer in there
<lifeless> then you can use stubs fastdowntime magic to shovel stuff through quickly
 * bigjools brings out the query killer
<bigjools> stub: with that index dropped, I still see 2 seq scans
<jtv-eat> So maybe that index is just simply a good idea.
<stub> bigjools: Run 'analyze binarypackagepublishinghistory;' and check the query again just in case
<bigjools> ok
<stub> But probably will want the index (archive, distroarchseries, pocket) according to the plan from before.
<henninge> allenap: why are the asserts in the two implementations of test_diff_row_shows_spr_creator different?
<henninge> allenap: I assume its two ways of getting to the name?
<bigjools> stub: ok, down to one seq scan on binarypackagebuild now
<bigjools> stub: ah I can't read, still 2 :(
<stub> bigjools: status might be useful in the index too.... (archive, distroarchseries, status, pocket) or similar. I recall testing being done around here before - wgrant?
<allenap> henninge: Good spot. I meant to change the one with find(".//a") to use text_content() like the others.
<henninge> allenap: why is text_content() the better choice?
 * stub wonders if some indexes haven't been landed
<bigjools> stub: since last db restore on mawson?
<henninge> allenap: the test relies on the words "moments ago by" to be present on the page but that is not really related to the test, is it?
<allenap> henninge: I think it's easier to demonstrate that the useful information has been conveyed.
<stub> bigjools: it isn't in the tree or on production, but I have a vague impression indexes on these tables with distroarchseries went through review
<henninge> allenap: yes, that is true.
<henninge> allenap: but it is a bit more fragile, too. Just a thought, though.
<henninge> allenap: otherwise r=me
<allenap> henninge: That column shows the date, so it does rely on that display, but the tests are named in terms of the creator. No other tests exist for this part of the table so perhaps I should rename the tests to be more generic; to say that they're tests for the "Last changed" column.
<henninge> or that ...
<stub> maybe I'm thinking of buildfarmjob stuff - that is what google is picking up
<rvba> henninge: thanks for the review BTW :)
<bigjools> you missed archive from your changed query
<stub> whoops
<stub> nah, it is in there
<stub>     AND bpph.pocket = 0 and bpph.archive = 26028
<bigjools> I still can't read
<allenap> Thanks henninge :)
<jtv-eat> stub: it was a partial index, for statuses 1 and 2 only.
<jtv-eat> the temp index, that is.
<stub> i see. I don't know what those statuses are, so not sure if it matches bigjool's full use case or just that 1 example query
<jtv-eat> ask him.  :)
 * jtv-eat _really_ goes afk
* benji changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: henninge, benji | Critical bugs: 239 - 0:[#######=]:256
<bigjools> stub: 1 and 2 are pending/published.  it does match the use case.
<stub> bigjools-afk: So we need an index CREATE INDEX bpph__archive__distroarchseries__pocket__idx ON BinaryPackagePublishingHistory(archive, distroarchseries,pocket) WHERE status IN (1,2);
<stub> maybe the equivalent on SPPH too
<lifeless> blargh, late
<lifeless> mrevell: I wonder if a survey would help bug 815623
<_mup_> Bug #815623: Mail notifications sent to team admins on joins / leaves to open teams <Launchpad itself:Triaged> < https://launchpad.net/bugs/815623 >
<lifeless> night all
<bigjools> stub: can you remember what actually needs that index?  I can't!
<stub> bigjools: The query we were just looking at!
<bigjools> stub: well your new one runs just fine
<stub> I thought you got full scans again once you dropped the temp_ index?
<bigjools> stub: with the old query
<stub> If not, don't worry about it
<stub> k
<bigjools> the new one is all good
<bigjools> runs in seconds
<stub> false alarm then :)
<bigjools> yarp
<deryck> Morning, all.
<cr3> hi folks, I just noticed the warning "<tag> has not been used in <project> before". I understand this might help normalize the cloud of tags in launchpad but I'd like to know if this was inspired from somewhere
<bac> cr3: not sure who had the idea or if it was borrowed.
<StevenK> lifeless: I agree with Chuck -- make it configurable
<deryck> adeuring, henninge -- coming for the standup.... just trying to find my headphones.
<adeuring> ok
 * henninge goes searching for his
<sinzui> jcsackett, do you have time to mumble?
<jcsackett> sinzui: yes.
<sinzui> mrevell, do you have any finding from the person/target picker tests to share with me?
<mrevell> sinzui, Yes, almost. I'm working on it now.
<sinzui> thanks
<bigjools> if I am reading correctly, does the "team:" scope on feature flags also work with a single person?
<sinzui> bigjools, it will work
<sinzui> a person is always a member of himself except when engineers give losas bad sql deletes to run
<bigjools> yeah that's what I figured, thanks sinzui
* henninge changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: benji | Critical bugs: 239 - 0:[#######=]:256
<adeuring> benji: could you have a lookk at this MP: https://code.launchpad.net/~adeuring/lazr.batchnavigator/lazr.batchnavigator-urlencode-batch-params/+merge/69118 ?
<benji> adeuring: sure
<adeuring> thanks!
<benji> adeuring: all done, the branch looks good
<adeuring> benji: thanks!
<nigelb> 972
<nigelb> err, sorry
<jtv> Is rocketfuel-get broken for anyone else?
<jtv> I can't "make" in my branch because: Couldn't find a distribution for 'zope.publisher==3.12.0'.
<jtv> rocketfuel-get (after "bzr pull" and with the usual "utilities/link-external-sourcecode ../devel ; make" incantation) doesn't fix it.
<jtv> Ahem: *followed by* "bzr pull" etc.
<jtv> "Could not find /usr/local/bin/sourcedeps.conf" â wonder where that comes from.
<sinzui> jcsackett, I may fall off net. A vicious thunderstorm is starting.
<jcsackett> sinzui: thanks for the headsup.
<LPCIBot> Yippie, build fixed!
<LPCIBot> Project db-devel build #751: FIXED in 5 hr 58 min: https://lpci.wedontsleep.org/job/db-devel/751/
<lifeless> flacoste: https://pastebin.canonical.com/50103/
<lifeless> flacoste: https://code.launchpad.net/~jtv/lp-production-configs/config-810989/+merge/68867
<flacoste> lifeless: http://leanandkanban.wordpress.com/2011/07/15/demings-14-points/
* benji changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: - | Critical bugs: 239 - 0:[#######=]:256
<lifeless> flacoste: https://code.launchpad.net/~bcsaller/ensemble/config-options-deploy
<mtaylor> lifeless: any chance you know of anyone with a script to migrate issues from a github repo to a launchpad project?
<lifeless> no, but there is a xml format we use for imports
<mtaylor> or anyone else on channel ... ^^^
<lifeless> so you just need to write a github -> bug import xml script
<mtaylor> oh goodie. I love xml. so if I export to an xml thing, I can hand it to someone and they'll appear in launchpad?
<lifeless> yup
<mtaylor> great. got docs on the xml format?
<lifeless> no, we make you guess.
<mtaylor> SWEET
<lifeless> https://help.launchpad.net/Bugs/ImportFormat
<mwhudson> just bash <>&; keys until it works
<mtaylor> and no one had written an exporter for github yet?
 * mtaylor cries
<poolie> hi mtaylor
<mtaylor> hi poolie !
<lifeless> mtaylor: well, I haven't heard of one, particularly for their new tracker.
 * mtaylor imagines that poolie is about to tell me that he has one ...
<poolie> sorry no
<poolie> you want to sync bugs from github to lp, or just to sync them down locally?
<lifeless> migrate
<wgrant> D:
<wgrant> +        do_names_check = Version(distroseries.version) < Version('11.10')
<mwhudson> declarative!
<sinzui> wgrant, StevenK: mumble?
<StevenK> sinzui: http://pastebin.ubuntu.com/652040/
<wgrant> Anyone want to review https://code.launchpad.net/~wgrant/launchpad/sensible-validate_target/+merge/69175?
<sinzui> StevenK, https://code.launchpad.net/~sinzui/launchpad/dsp-vocab-contracts/+merge/69183
<sinzui> wgrant, I can review that in about 2 hours
<wgrant> sinzui: Thanks.
* wgrant changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: wgrant | Critical bugs: 238 - 0:[#######=]:256
<lifeless> sinzui: hi
<lifeless> sinzui: I wanted your thoughts on my bug about open team membership change notifications
* lifeless changed the topic of #launchpad-dev to: Performance Tuesday | https://dev.launchpad.net/ | On call reviewer: wgrant | Critical bugs: 238 - 0:[#######=]:256
#launchpad-dev 2011-07-26
<StevenK> sinzui: It is not an error if the DSP has no publishing_history
<sinzui> StevenK, Really? How can it be legitimate if it has not even gotten to the pending state. How do we ensure that https://launchpad.net/ubuntu/+source/html5-browser (ubuntu/html5-browser) is not accepted as an Ubuntu package?
<wgrant> My latest branch touches on that for bug targetting.
<wgrant> But only in terms of erroring, not restricting vocab results.
<sinzui> lifeless, I never want to see an open-team membership notification. I was tempted this weekend to hack a fix for team mail notification so that I can have a rule to delete them
<sinzui> I will see that in a few minutes then
<StevenK> sinzui: Hmm. You may be right. I will investigate today.
<lifeless> sinzui: https://bugs.launchpad.net/launchpad/+bug/815623
<_mup_> Bug #815623: Mail notifications sent to team admins on joins / leaves to open teams <Launchpad itself:Triaged> < https://launchpad.net/bugs/815623 >
<lifeless> sinzui: I have a branch that nukes the notifications
<lifeless> sinzui: it has one wart
<sinzui> rs=me
<StevenK> But it looks like some poeple want the notification
<lifeless> sinzui: it squelches notification about folk that become / leave the admins of the team, which is still actually restricted; I think thats undesirable but not a Big Deal
<sinzui> lifeless, yes. that is a bad
<lifeless> sinzui: so I'd like your input on two things
<lifeless> sinzui: does the admin changes not being notified matter ?
<lifeless> sinzui: -and-
<sinzui> lifeless, this wart is one if underlying issue team notifications were not an easy fix.
<lifeless> sinzui: do we care about the folk that care about these spam messages ?
<lifeless> sinzui: I can fix it to retain admin notifications by checking the status field of the teammembership change, I suspect.
<lifeless> sinzui: but I don't know if we need to do that, or if we can file a high bug to come back and reinstate just those messages.
<sinzui> lifeless, I am not aware of a team admin that wanted the those notifications. I would not want to put a switch in. I think this class of issue is really structural subscriptions for team, which is a request of sorts in our bug tracker already
<lifeless> sinzui: so you support - squelch the notifications; when we do explicit subscriptions to team changes, then we can default a subscription for non-open team admins, default to no subscription for open team admins, and moderation etc will still be automatic ?
<sinzui> lifeless, yes
<lifeless> do you happen to know the bug # for subscriptions-to-team-changes ?
<sinzui> I can find it quickly
<sinzui> lifeless, bug 507515
<_mup_> Bug #507515: want to subscribe to changes in a team <email> <lp-registry> <teams> <Launchpad itself:Triaged> < https://launchpad.net/bugs/507515 >
<lifeless> thanks
<sinzui> lifeless, you may be fixing the root cause of bug 285414
<_mup_> Bug #285414: There should be an option so that team admins don't receive notifications <email> <feature> <lp-registry> <teams> <Launchpad itself:Triaged> < https://launchpad.net/bugs/285414 >
<lifeless> I will comment to this effect in the bug, file a bug about no admin notifications on open teams, and land my branch.
<lifeless> I think I am
<StevenK> lifeless: I just think landing your branch will lead to more bugs along the lines of "I used to be notified, now I'm not."
<lifeless> StevenK: which we'll dup onto bug 507515
<_mup_> Bug #507515: want to subscribe to changes in a team <email> <lp-registry> <teams> <Launchpad itself:Triaged> < https://launchpad.net/bugs/507515 >
<StevenK> Fairy nuff
<lifeless> sinzui: can admins make someone else an admin ?
<wgrant> No.
<sinzui> no. I can find the bug.
<lifeless> thats ok
<lifeless> it means that this notification change w.r.t. admins is ok
<lifeless> because there is also no action the admins can take
<lifeless> in fact, we shouldn't be notifying admins in that case, we should be notifying the owner.
<wgrant> They can demote others. But that's it.
<lifeless> sinzui: https://bugs.launchpad.net/launchpad/+bug/816196
<_mup_> Bug #816196: new / deactivated admins of teams are not notified to the owner <Launchpad itself:Triaged> < https://launchpad.net/bugs/816196 >
<lifeless> sinzui: and https://bugs.launchpad.net/launchpad/+bug/815623/comments/8
<_mup_> Bug #815623: Mail notifications sent to team admins on joins / leaves to open teams <Launchpad itself:Triaged> < https://launchpad.net/bugs/815623 >
<sinzui> thanks
<lifeless> sinzui: you might want to add tags to the new bug, I'm not sure what it should have
<sinzui> lifeless, done: this is my short list of team email issues: https://bugs.launchpad.net/launchpad/+bugs?field.tag=teams+email&field.tags_combinator=ALL
<lifeless> cool
<lifeless> sinzui: surely we've fixed https://bugs.launchpad.net/launchpad/+bug/120037 as a drive-by by now ?
<_mup_> Bug #120037: Action reported in email as done by owner rather than administrator <email> <lp-registry> <teams> <Launchpad itself:Triaged> < https://launchpad.net/bugs/120037 >
<poolie> lifeless: hi
<lifeless> hi poolie
<poolie> why use a pretty printer at all, rather than just having recommendations for code style?
<lifeless> poolie: I thought I covered that earlier in the thread; because its easier for contributors.
<poolie> hm maybe i misunderstood
<poolie> i think bouncing changes because they're not pep8 clean raises the bar
<sinzui> lifeless, maybe. I thought there were one or two emails that were hard coded to admin
<poolie> would using a pretty printer make things easier?
<lifeless> poolie: we could eliminate all code review formatting discussion
<lifeless> poolie: that seems easier to me.
<poolie> if it happens at pqm post-commit, and if the contributor uses 200-character wide lines, reviewers will still complain
<poolie> or tell them to run the formatter themselves
<poolie> oh, i guess that was mentioned
<lifeless> poolie: are you against the use of a pretty printer ?
<poolie> only slightly, because it seems to have some risk of breaking things and i don't think it gains very much
<poolie> i am definitely in favor of making it easy to contribute, and in particular avoiding unnecessary rejections
<sinzui> lifeless, I am not sure that bug is fixed. There are rules to fallback to an historic membership.reviewed_by.
<sinzui> lifeless, NM. I think this issue was in TM.setStatus() and enforce its use now. I marked the bug released.
<lifeless> sinzui: \o/ cheap bug fixes ftw
<lifeless> does the API show hidden messages ?
<mwhudson> i wonder how many hours a year the buildds spend "Building database of manual pages ..."
<mwhudson> it's about 25% of the pbuilder run time on my machine...
<lifeless> wgrant: StevenK: either of you know why we do this - https://lp-oops.canonical.com/oops.py/?oopsid=OOPS-2032CD17#longstatements
<lifeless> the packagebuildjob query
<lifeless> buildfarmjob
<lifeless> I meant
<wgrant> lifeless: To show the failed/successful build counts, I guess.
<lifeless> wgrant: I'm unclear what you mean in https://bugs.launchpad.net/launchpad/+bug/711073/comments/3
<_mup_> Bug #711073: Archive:+packages (Archive:+index) timeouts <timeout> <Launchpad itself:Triaged> < https://launchpad.net/bugs/711073 >
<lifeless> are you saying 'the queries in this bug are fixed but others are a problem'
<lifeless> or
<lifeless> ???
<wgrant> lifeless: +index and +packages were 99% < 5s.
<wgrant> Not fantastic, but not terrible.
<lifeless> ok
<lifeless> and bug 758258 is in-progress with no assignee ?
<_mup_> Bug #758258: buildfarmjob schema is inefficient for reporting <timeout> <Launchpad itself:In Progress> < https://launchpad.net/bugs/758258 >
<wgrant> Fixed.
<lifeless> wgrant: did you land the new columns & garbo jobs?
<wgrant> No. Was going to do it soon before the DB deploy to minimise conflicts, but then got unexpectedly moved to Disclosure when Yellow was prematurely moved to maintenance.
<wgrant> There's only a couple of code changes needed in addition to the DB changes.
<wgrant> Most of them landed two months ago.
<lifeless> so, new world order, schema and code separate ;)
<wgrant> Yes.
<wgrant> I never planned to land them together.
<lifeless> bug https://bugs.launchpad.net/launchpad/+bug/816233
<_mup_> Bug #816233: Archive:+packages timeouts in buildfarmjob status summaries <timeout> <Launchpad itself:Triaged> < https://launchpad.net/bugs/816233 >
<wgrant> They did not need to be.
<wgrant> https://code.launchpad.net/~wgrant/launchpad/flatten-sprb
<wgrant> Actually flattens SPRB and BPB.
<wgrant> And mostly works.
 * wgrant links it to the bug.
<lifeless> bah
<lifeless> edge needs to begone
<wgrant> Ah, already is linked.
<wgrant> Yes.
<wgrant> We should do that.
<wgrant> Or just hit people.
<lifeless> losa are chewing through things now
<lifeless> which reminds me
<lifeless> hmm
<lifeless> jtv: around ? I'm guessing too early.
<lifeless> ahha
<lifeless> stub: I've figured out one of our timeouts... due to a unique constraint.
<wgrant> Oh?
<lifeless> https://bugs.launchpad.net/launchpad/+bug/816235/comments/1
<_mup_> Bug #816235: Bug:EntryResource:linkBranch timeouts <timeout> <Launchpad itself:Triaged> < https://launchpad.net/bugs/816235 >
<lifeless> (I've added a second comment there now)
<lifeless> poolie: hi
<poolie> hi hi
<lifeless> in bug 721166 yesterday
<lifeless> you pushed a branch
<lifeless> and linked it to the bug
<_mup_> Bug #721166: Tests sometimes fail on EC2 due to _LockWarner garbage <build-infrastructure> <spurious-test-failure> <Launchpad itself:In Progress by mbp> < https://launchpad.net/bugs/721166 >
<lifeless> the link timed out on you
<lifeless> did you retry it, or had you done a commit --fixes lp:721166 ?
<poolie> istr getting a javascript error while establishing the link
<poolie> with no details
<lifeless> it was OOPS-2032F24
<poolie> why, did it leave debris behind?
<poolie> hell orobot?
<lifeless> I'm trying to confirm a theory behind the cause
<lifeless> if I'm right, we may have an explanation for a wide range of sporadic failures, and more data on why backend scripts with long (>1 second) transactions are bad)
<lifeless> the other possibility is that the scanner updates the branch row... that may be it
<poolie> i actually wondered whether to mention it and then decided that "i got a js error with no oops" was boring
<poolie> ok; can i do anything to help settle it?
<lifeless> I've got it sussed I think
<lifeless> https://bugs.launchpad.net/launchpad/+bug/816235
<_mup_> Bug #816235: Bug:EntryResource:linkBranch timeouts due to branch scanner transaction length <timeout> <Launchpad itself:Triaged> < https://launchpad.net/bugs/816235 >
<poolie> it reminds me a bit of some other bug i filed
<poolie> i forget the number
<poolie> where what should have been a trivial action timed out after 9 seconds
<lifeless> yes
<lifeless> I've been of the opinion that backend scripts are a problem for a while
<lifeless> but 'contention' was never a very satisfying explanation
<poolie> hm
<poolie> i can't actually find that
<lifeless> and I can hardly expect to get a cultural change to something that isn't quite as easy on the basis of something that isn't satisfying :)
<poolie> is there {waves hands} a way to run the branch scanner at a different isolation level or something?
<lifeless> wouldn't help
<lifeless> this is because we have foreign keys
<lifeless> the appserver commit cannot progress until it obtains a lock preventing anyone deleting or changing the id on the branch row.
<lifeless> pg does row locks, I don't think it does row,column locks
<lifeless> there are some design things we can do
<lifeless> putting status-tracking data in separate tables, for instance
<lifeless> but the biggest is short transactions
<wgrant> UbuntuSourcePackageNameWidget :(
<wgrant> Can I add a precommit hook forbidding any diffs matching [Uu]buntu?
<lifeless> no
<lifeless> you need a cultural change I think
<lifeless> or rather, yes, if you give me the db one for pqm at the same time ;)
<wgrant> Heh
<poolie> anyhow, the things i saw, of should-be-tiny transactions timing out, would be consistent with that kind of long running lock holding transaction
<poolie> perhaps also with other things like a thrashing db, but in that case i presume we'd notice
<lifeless> indeed
<wgrant> Anybody up for reviewing two ~100 line branches?
<lifeless> sure
<wgrant> https://code.launchpad.net/~wgrant/launchpad/transitionToTarget-validate_target/+merge/69189 and https://code.launchpad.net/~wgrant/launchpad/launchpadtargetwidget-prefix-respect/+merge/69187
<stub> lifeless: another one for the 'long transactions are evil' bucket.
<lifeless> yes, exactly
<lifeless> that and DRI :P
<stub> lifeless: not sure why the branch scanner needs long transactions. It could probably run in autocommit mode even, provided it remembered to insert rows in earliest -> latest revision order.
<wgrant> It already partially commits sometimes.
<wgrant> Because if the initial scan fails it needs manual cleanup.
<lifeless> it probably updates the last_scanned field too early
<lifeless> or something
<lifeless> stub: how hard would it be to make the transaction reaper kill *updating* transactions over 2.5 seconds long ?
<wgrant> Goodbye bug modifications.
<stub> lifeless: We could kill transactions that have been trying to execute an UPDATE for longer than 2.5-3.0 seconds (0.5 seconds range, as the field we need is only updated every 500ms). Not sure if that would help anything though.
<stub> lifeless: We can't kill transactions that issued an update earlier in their transaction and might be blocking others.
<lifeless> stub: because idle in transaction might be a non-mutating request ?
<stub> yes
<lifeless> stub: we could introspect pg_locks ? <evil face>
<stub> Better off doing all this with query timeouts.
<lifeless> yeah
<lifeless> was really just speculating
<stub> Less chance of blowing off your whole leg.
<lifeless> see -dev for some rambling about this :)
<poolie> oh, i remember
<poolie> lifeless: the thing i was thinking of was the failure of the ssh server to talk to the internal xmlrpc
<poolie> xmlrpc was failing to look up a branch (iirc)
<lifeless> stub: francis is excited by our progress :)
<poolie> ah, it's not a readonly query though
<lifeless> which bug ?
<poolie> https://bugs.launchpad.net/bugs/294726
<_mup_> Bug #294726: "ProtocolError for xmlrpc.lp.internal:8097/branchfilesystem" when pushing a branch <lp-code> <Launchpad itself:Triaged> < https://launchpad.net/bugs/294726 >
<poolie> i replied
<poolie> especially #4
<poolie> lifeless: these are interesting cases because of course a person is quite likely to want to link a branch, or push it again, at the time it's being scanned
<lifeless> yup
<wgrant> Or create an MP...
<wgrant> That's where I see it most.
<poolie> right
<poolie> also these seem like fairly interesting things to put into a message queue or into a non sql database
<poolie> though, probably that's not the best short term step
<wgrant> Launchpad has very few good short term steps :)
<stub> lifeless: Bug is the other table with similar issues to Branch. Want to move heat and heat_last_updated to a separate table, probably the other timestamps and caches too.
<lifeless> yup
<lifeless> (I agree, I haven't thought deeply about where or how.
<lifeless> e.g. should different fields be in separate tables
<lifeless> or a k:v table we get row locks on
<lifeless> or ...
<poolie> i saw your comment reccently about "generally lp doesn't want ...options"
<poolie> which i agree with in many ways
<poolie> however it was in the context of notifications
<poolie> and i do think a fb or g+ style "do/don't tell me about X action" per user would be ok, don't you?
<lifeless> perhaps
<lifeless> however, we have a bug asking for reasonably comprehensive subscriptions to teams
<lifeless> which is worth doing directly I think, rather than an awkward one off in a corner
<poolie> what does "subscription to teams" mean?
<poolie> telling you about changes to memberships?
<lifeless> and possibly other metadata
<lifeless> stub: 0.0.2 of pgbouncer is in the download cache for you
<stub> ta
<lifeless> stub: I have to run, but I'd love to know whats left to do a no-op downtime run ?
<lifeless> stub: I am guessing docs and landing last nights branch ?
<stub> lifeless: We need to push out configs to make everything go via pgbouncer.
<stub> lifeless: Apart from that, we can test now. Last nights branch is an improvement, but not necessary. And docs I prefer to write after I've confirmed everything works rather than having to adjust them for reality later :-)
<lifeless> \o/
<stub> pgbouncer configs is the slow step
<lifeless> I've blogged about this, as promised
<lifeless> so users won't be shocked
<wgrant> Does this mean we need downtime for all downtime services in the next couple of days?
<wgrant> (librarian being the main problem)
<lifeless> ah the uploader ? yes we do
<stub> Think librarian actually reconnects happily - did on staging anyway when I ran this stuff.
<wgrant> We restarted it two weeks ago without hiccups, but still, something to consider.
<lifeless> stub: wgrant is referring to migrating it to use pgbouncer
<lifeless> stub: uploads to it are not load balanced at the moment
<stub> Ahh, yes. But outages will be minor (test connectivity first, then bounce).
<stub> Is the upload port going via haproxy?
<wgrant> Not yet.
<wgrant> Firewall rules are unclear.
<stub> it shouldn't be a problem being generous with access to the upload port. Not much you can do apart from try and fill up our disk.
<StevenK> There is no try
<StevenK> They *can*
<lifeless> uploads go via appservers
<lifeless> so an interrupted librarian results in an appserver OOPS
<lifeless> its not a big deal
<wgrant> Uh.
<wgrant> Yes, let's pretend that scripts don't exist :)
<lifeless> they can be queisced prior to the librarian bounce
<wgrant> True.
<lifeless> stub: sounds like the config update is the critical path thing then; if you need anything from me on it, let me know, otherwise its in your court
<wgrant> So, codehosting, ftpmaster, librarian, mailman are the downtime services I know of.
<wgrant> But mailman doesn't have DB access, I suppose.
<jtv> hi lifelessâalmost 9AM here now.  What's up?
<rvba> Good morning.
<jtv> good morning rvba!
<jtv> StevenK, wgrant: mind if I update dogfood and run generate-contents-files?
<wgrant> Please do.
<jtv> rvba: dogfood conflict in package cloner.  Were you running an experimental branch there?
<henninge> danilos: Hi! ;)
<jtv> morning, former teammates
* wgrant changed the topic of #launchpad-dev to: Performance Tuesday | https://dev.launchpad.net/ | On call reviewer: - | Critical bugs: 238 - 0:[#######=]:256
<henninge> Hi jtv!
<henninge> jtv: What do you think should be done to fix bug 504062?
<_mup_> Bug #504062: External suggestion "Used in" links to disabled template <404> <lp-translations> <oops> <trivial> <ui> <Launchpad itself:Triaged> < https://launchpad.net/bugs/504062 >
 * jtv looks
<henninge> jtv: Should non-admins not see the suggestion at all or just don't get a link to the template?
<rvba> jtv: sorry, was afk ... I might know what the problem is ...
<rvba> jtv: how can I take a look? (I don't have access to DF)
<jtv> rvba: ah.  Hang on, have to look at something else now.
<poolie> spiv: re your question on bug 813349, people filed a couple of nit type bugs
<_mup_> Bug #813349: hard to get from mp to per-revision diffs <code-review> <javascript> <qa-ok> <ui> <Launchpad itself:Fix Committed by spiv> < https://launchpad.net/bugs/813349 >
<poolie> like your link colour is wrong
<poolie> personally i wouldn't block on them
<jtv> henninge: istm the reason for showing suggestions from disabled templates is mostly obsolete thanks to sharing.
<jtv> henninge: we could try getting some data â how often are those translations both different and useful?
<jtv> If the answer is "rarely," then the solution is simple.
<jtv> (Well, apart perhaps from lag between the time the template is disabled and the next update of the suggestive-templates cache).
<jtv> rvba, here's the conflict: http://paste.ubuntu.com/652250/
<jtv> henninge: if it turns out we're getting real benefit from disabled templates, then we'd have to worry about disabling links.  I even think Zope would do that for us if this were a permission error rather than a 404, but at a performance cost.
<adeuring> good morning
<henninge> HI adeuring!
<adeuring> hi henninge!
<rvba> jtv: It's Julian SQL optimisation from yesterday. It looks to me like he tried it manually on DF, then he got it properly landed because (I think) all the code that causes the conflict is there already.
<henninge> jtv: Hm, I'd like to keep this simple ...
<jtv> rvba: in that case, I should be able to revert & pull.  That's great.
<rvba> jtv: I think so yes.
<jtv> henninge: I sympathize, but if it were truly simple maybe this bug would've been fixed by now.  :)
<henninge> jtv: Before I read this I would have expected translations from disabled templates to not show up
<jtv> Yes, I thought they were excluded from the suggestive-templates cache.  Have you been able to reproduce the problem?
<henninge> jtv: on qastaging, yes. But I am not sure about the result being cached.
<henninge> jtv: how long does stuff live in that cache?
<spiv> poolie: hmm, it'd be nice to have those bugs linked to 813349 perhaps, but probably not important if they're not blockers
<henninge> also, I never worked with the cache before.
<jtv> henninge: the suggestive templates cache is rewritten in its entirelyâ¦ I think daily at the moment, but may be much more frequently.  We could afford to do it much more often than that.
<spiv> poolie: my main concern is that the bug doesn't languish indefinitely as Fix Committed.
<jtv> henninge: it's just a database table containing the ids of POTemplates that can be used to get suggestions from.  And it does indeed exclude disabled templates.
<jtv> (Just checked)
<poolie> spiv so fortunately 'new bugs' should find them
<jtv> henninge: so whenever a template is disabled, this problem can exist until the cache is rewritten.
<henninge> ah!
<jtv> henninge: (Also, I see it hasn't been updated for usage enums)
<jtv> henninge: The bug description therefore would seem a little out of date.
<henninge> yes
<henninge> jtv: so the problem is really just about how to deal with an invalid cache.
<jtv> Yes, it's cache invalidation â one of the two things that are hard.
<henninge> ;)
<jtv> We could afford to update the cache whenever anyone deactivates a template.
<jtv> IIRC it takes only a fraction of a second to find all those templates; it's just that we were doing it multiple times for every translations page.
<jtv> If the suggestions queries also join in the POTemplate, we could just add a redundant iscurrent check.
<poolie> spiv if only there was automatic connection between related bugs :)
<henninge> jtv: yes, those were the two things I was considering right now.
<jtv> Alas, the query doesn't include POTemplate.
<henninge> jtv: I just wasn't sure how expensive the joining would be.
<jtv> And this is velly velly pelformance-snsitive.
<henninge> yup
<jtv> So best not add that join.
<henninge> jtv: but removing that entry from the cache should be simple.
<jtv> What's really weird by the way is that the code in _getExternalTranslationMessages seems to fetch POTMsgSet ids and call them "pots."
<poolie> spiv, added
<jtv> Maybe that's just someone who wasn't familiar with the conventional abbreviations.
<jtv> henninge: it looks like someone's made a mess of that code.  There's also an over-long line of SQL, with the keywords not capitalized.  That smells of someone just trying something out and accidentally landing it as soon as it worked, without review.
<spiv> poolie: thanks
<jtv> henninge: ah no, it's lifeless code.
<henninge> jtv: how is that different?
<jtv> This is hard to say diplomatically.
<jtv> In Launchpad we have traditionally focused on producing code that's easy to read.
<jtv> Robert leans more towards keeping it easy to write.
<henninge> ...
<jtv> So I often struggle reading his code.
<jtv> For that reason I don't think it saves any time, but opinion seems divided on the subject.
<henninge> ...
<jtv> Still, you would have expected a "make lint."
<henninge> very much so
 * spiv offers hennige a â¦, for variety
<henninge> thanks spiv ;)
<jtv> Also, try <compose>..
<spiv> jtv: how do you think I typed it? :)
<jtv> spiv: I do apologize.  :)  I only checked Henning's.  :)
<spiv> Or perhaps I should say: âº
<jtv> I â¥ <compose>
<henninge> â¦
<jtv> <compose>âºÂ¹
 * henninge learnt something
<jtv> "You've taken your first step into a larger worldâ¦"
<henninge> Â¹Â²Â³Â¼Â½
<henninge> I already know those â¦
<henninge> s/know/knew/
 * spiv offers jtv <compose><"
<jtv> Oooh!  Thank you!
<jtv> You know what's going to happen now, don't you?
<jtv> âQuote-mania.â
<spiv> Â¡Por supuesto! ;)
<jtv> wÃ¸Ã±ÄÃªÅfÃ¼Å
<jtv> morning bigjools
<bigjools> morgen
<jtv> bigjools: I'm just messing around on dogfood right now, so beware
<bigjools> jtv: eek :)
<jtv> Ah-HAaah.  No "idle in transaction" no more for generate-contents-files.
<jtv> Deployments to cocoplum do take downtime, right?
<wgrant> Yes. We'll probably need one in the next day or so for config updates.
<wgrant> stub: Any idea on the timing of that?
<stub> wgrant: nup
<stub> asap, but out of my hands now (apart from nagging and whining)
<jtv> hloeung: about https://code.launchpad.net/~jtv/lp-production-configs/config-810989/+merge/68867 â shall we just merge & deploy it then?
<mrevell> Hi
<jtv> hi mrevell
<jtv> wgrant: any idea whether staging and qastaging have the ubuntu-contents dir?
<jtv> In other words, whether we ever run generate-contents-files on them (whether from cron or for testing)?
<wgrant> jtv: They should not.
<jtv> Then I'll not worry about it.
<wgrant> AFAIK nobody has ever been foolish enough to run a publisher on them.
<jtv> huwshimi: thanks for the icon by the way.  I'll just bung it into l/c/l/images and update the code to use it.
<huwshimi> jtv: Great, no problems.
<henninge> jtv: so, should I put the removal into the admin view code (which AFAIK is the only place where templates can be disabled) or in the model code (i.e. introduce a setter).
<henninge> jtv: I'd prefer model code I think
<jtv> Absolutely.
<henninge> cool, great to see we agree â¦
<jtv> At this point it may possibly be worth reviving the utility.
<henninge> jtv: which utility?
<jtv> I originally wrote a utility for this cache, but IIRC the reviewer thought it was too much.
<jtv> Up to you to draw the line of where it deserves its own utility though.
<henninge> jtv: oh yes, I noticed there is no model code for it.
<jtv> Exactly.
<jtv> If you want to add that, just be sure to define the template id as the primary key.
<jtv> I don't think we really need it though.
<henninge> I don't either ...
<henninge> quick question
<henninge> getUtility(IStoreSelector).get(MAIN_STORE, DEFAULT_FLAVOR)
<henninge> is now equivalent to
<henninge> IMasterStore(something)
* gmb changed the topic of #launchpad-dev to: Performance Tuesday | https://dev.launchpad.net/ | On call reviewer: gmb | Critical bugs: 238 - 0:[#######=]:256
<henninge> right?
<henninge> oh, I don't have "something" here ...
<jtv> henninge: no, default flavor is IStore(something).  May be a class.
<jtv> IMasterStore is the master flavor.
<henninge> I see
<jelmer> gmb: Thanks for the re-review and ec2 submission
<gmb> np
<lifeless> bigjools: you know how I said 10m was ok for a backend script the other day ?
<bigjools> yeeaasssss?
<lifeless> I was wrong; see my performance-tuesday mail from today :)
 * bigjools reaches for razor blades
 * bigjools is not sure whether to use them on himself or lifeless
<lifeless> jtv: I pinged you when I was looking at some locking / concurrency/contention innards of postgresql
<lifeless> jtv: for a teddy bear, but I sorted itout
<wgrant> lifeless: Well, it should be OK for some things :)
<bigjools> lifeless: I'd say that doing txns in less than 2.5 seconds is lala land
<bigjools> unless the magic performance fairy visits the packagecopier
<bigjools> and somehow we get divine intervention in the soyuz schema and all the code gets rewritten to avoid races and contentions
<wgrant> Yes.
<wgrant> Some sorts of transactions are OK.
<jtv> Teddy bear..?
<lifeless> jtv: someone to talk about a problem with
<lifeless> bigjools: so, if we can't get to 2.5 seconds consistently, we can never get to 5 seconds reliably if there are any common things being changed.
<lifeless> bigjools: its not something we can wave a wand to have happen, but its definitely doable.
<bigjools> lifeless: I am OTP but see our conversation this time yesterday
<bigjools> for all the scary reasons why this is hard
<bigjools> tl;dr = consistency issues
<lifeless> bigjools: sure, so the key question is how we can reduce the risk / find ways to assess it.
<bigjools> yup
<bigjools> we kinda ruled out using staging tables
<bigjools> which means fixing the rest of the code.  And we're not sure what needs fixing.
<lifeless> the first thing that occurs to me is divide and conquer:
<lifeless> break the problem code (whatever it is) into a few categories:
<lifeless>  - readonly code that we can tolerate stupidity in - things with ephemeral output like queuing decisions or web ui
<lifeless>  - readonly code that must be accurate like archive publication and contents generation
<lifeless>  - writing code (which we assume we cannot tolerate stupidity in)
<lifeless> then, look for queries that reference distroseries - probably a few hundred tops that do that; toss them into each of those categories as much as possible, and things that are ok we can just move on.
<bigjools> yeah, not saying it's impossible, just a tedious and error-prone job
<bigjools> I'm interested in re-assessing our model to prevent this crap
<bigjools> we need a discussion at the next T'Dome
<bigjools> the basic issue is that the model doesn't prevent the inconsistency
<lifeless> thats a pretty core ingredient :)
<bigjools> lifeless: yes - there's another timebomb as well, I'll tell you about it in -ops
<jtv> gmb: review?  https://code.launchpad.net/~jtv/launchpad/bug-800573/+merge/69240
<gmb> jtv: Sure
<jtv> thanks
<gmb> 'Huw "Emo" Wilkins'? Nice :)
<gmb> jtv: I've come to expect fairly weighty branches from you. Nothing like breaking stereotypes, eh? Approved.
<jtv> gmb: a lot of that is just plain bad luck, I'm afraid.
<jtv> Not all.
<gmb> :)
<jtv> Maybe you're just my reviewer of last resort, the one I end up with when everyone else has turned me down.
<jtv> gmb: thanks by the way.  :)
<gmb> np :)
<jtv> bigjools, cjwatson: is the -commercial pocket completely and utterly gone now that dapper is eol?  Can we drop the publisher provisions (the commercial-compat.sh script)?
<jtv> wgrant also said to talk to the releases team, IIRC â if so, what's the right communication channel for that?
<cjwatson> hmm
<cjwatson> yes, I think it is
 * gmb lunches
<wgrant> Dapper is officially EOL, but someone may be playing OEM-like tricks with it...
<bigjools> jtv: talk to skaet
<henninge> jtv: any idea why I might get this? ForbiddenAttribute: ('__call__', <lp.testing.fakemethod.FakeMethod object at 0xdc8d74c>)
<cjwatson> jtv: you should also check with Brian Thomason (iamfuzz), since he does 99% of the work on partner
<cjwatson> if he's no longer uploading anything to dapper-commercial, then it can die
<cjwatson> I expect he'll in fact be more directly up to date on this than skaet will
<cjwatson> bigjools: you asked me to prepare that staging PPA with changes intended for the Launchpad PPA (https://lists.launchpad.net/launchpad-dev/msg07686.html) - are you also able to review it?
<bigjools> cjwatson: someone on the maintenance cycle should review it, I'm kinda busy.  Perhaps gmb?
<jtv> henninge: fakemethod behind a proxy, I think.
<jtv> bigjools: thanks
<henninge> jtv: yes
<henninge> jtv: I was patching a utility. bad idea
<jtv> rather
<jtv> cjwatson: thanks
<jtv> henninge: I guess you're probably already cleaning this up, but just so we don't forget: bug 816366
<_mup_> Bug #816366: Misleading code in POTMsgSet <tech-debt> <Launchpad itself:Triaged> < https://launchpad.net/bugs/816366 >
<henninge> jtv: http://paste.ubuntu.com/652366/
<jtv> henninge: great, thanks.  Phe!
<jtv> Phew.
<lifeless> night y'all
<gmb> sinzui: Can you take a look at, and perhaps give a cogent response to https://answers.launchpad.net/launchpad/+question/165200?
<gmb> henninge, danilos, jtv: Can one of you good fellers take a look at https://answers.launchpad.net/launchpad/+question/165699 please?
<danilos> gmb, on it
<gmb> Thanks
<danilos> gmb, btw, can you take a look at https://code.launchpad.net/~danilo/launchpad/bug-800123/+merge/69264 when you find some time?
<gmb> Sure
<danilos> gmb, thanks
<gmb> danilos: r=me
<danilos> gmb, ta
<deryck> adeuring, henninge-lunch -- sorry I'm late, coming to standup in a couple minutes.
<jtv> gmb: still reviewing?  If so â https://code.launchpad.net/~jtv/launchpad/bug-814141/+merge/69285
<gmb> jtv: Sure.
<jtv> thx
<deryck> bac, hi there.
<bac> hi deryck
<deryck> bac, I was going to do a nodowntime deploy and if the qa page is correct, your fix for bug 788685 is blocking other qa-ok revs....
<_mup_> Bug #788685: Enable translating selected Ubuntu universe packages in Launchpad <escalated> <oem-services> <qa-needstesting> <Launchpad itself:Fix Committed by bac> <OEM Priority Project:Invalid> <Ubuntu Translations:Triaged> <pkgbinarymangler (Ubuntu):Fix Released by pitti> < https://launchpad.net/bugs/788685 >
<deryck> bac, any chance of qa'ing that today?
<bac> deryck: i'll try.  it isn't straightforward
<deryck> bac, I assumed it wasn't.  do you need a former translations persons help?
<bac> deryck: luckily i have one.  :)
<deryck> bac, right, I knew danilos was on your team. :-)  I didn't know if he was still around to help. :)
<deryck> was going to offer my translation guy :)
<bac> deryck: i may take you up.  will have a go now.
<bac> bigjools: ping
<deryck> bac, ok, cool.  Thanks!
<bigjools> bac: hello
<bac> bigjools: i need to QA bug 788685.  danilos suggested yesterday we do it on dogfood and "ask soyuz people about re-uploading (re-publishing or something) existing ubuntu packages on dogfood"
<_mup_> Bug #788685: Enable translating selected Ubuntu universe packages in Launchpad <escalated> <oem-services> <qa-needstesting> <Launchpad itself:Fix Committed by bac> <OEM Priority Project:Invalid> <Ubuntu Translations:Triaged> <pkgbinarymangler (Ubuntu):Fix Released by pitti> < https://launchpad.net/bugs/788685 >
<bac> bigjools: would you have a moment or two and does that make sense to you?
<bigjools> bac: you can upload to DF and I can run the upload processor
<bigjools> you just need to "apt-get source" on an existing universe package and bump its version
<bigjools> build the package then dput to DF
<bac> bigjools: do i just ask a losa to deploy my branch onto DF?
<bigjools> bac: no you ask me :)
<bigjools> is it landed?
<bac> bigjools: excellent
<bac> bigjools: yeah, it is
<bac> bigjools: r13510
<bigjools> bac: is it on db-devel (we run DF off db-devel)
<bac> no
<bigjools> if not I'll merge the branch
<bigjools> donde esta?
<bac> bigjools:  lp:~bac/launchpad/bug-788685
<bigjools> bac: ah it was already on db-devel it seems.
<bac> oh righty
<bigjools> bac: so go ahead and upload, ping me when it needs processing
<bac> bigjools: about that...
<bac> bigjools: i need to get a source that has translations, right?
<bigjools> yes
<bigjools> bac: and you need the XS-Ubuntu-Use-Langpack: yes header in debian/control
<bac> yes, of course i do.  i was just thinking that
<bigjools> and also we need to check that the DF buildds chroots have got the new pkgbinarymangler
<sinzui> matsubara, mumble or skype?
<matsubara> sinzui, mumble
<henninge> gmb: Hi!
<gmb> Hi henninge
<henninge> gmb: Can you please do a review for me?
<henninge> https://code.launchpad.net/~henninge/launchpad/bug-504062-external-suggestions/+merge/69296
<gmb> Sure
<gmb> henninge: Approved
<henninge> gmb: thanks
* gmb changed the topic of #launchpad-dev to: Performance Tuesday | https://dev.launchpad.net/ | On call reviewer: - | Critical bugs: 238 - 0:[#######=]:256
<henninge> gary_poster: Hi!
<gary_poster> hey henninge :-) what's up?
<henninge> gary_poster: I am looking at bug 481512
<_mup_> Bug #481512: race condition when rotating logs <canonical-losa-lp> <escalated> <lp-foundations> <Launchpad itself:Triaged> < https://launchpad.net/bugs/481512 >
<gary_poster> henninge, cool
<henninge> gary_poster: but I am not familiar with the logging system in LP.
<henninge> gary_poster: I hoped that you could give me any input you have on that bug.
<gary_poster> henninge, we send a signal which is supposed to rotate the logs.  I found some pertinent code recently; lemme see if I can find it.
<henninge> ah
<gary_poster> henninge, still looking, but lib/canonical/launchpad/webapp/sigusr2.py
 * henninge looks
<henninge> ok, basic stuff
<henninge> there is a test case for it
<gary_poster> henninge, I've looked for more--I thought I saw some custom log handlers but the only ones I can find now are for tests and for scripts (and a simple one that adds custom log levels).  So, I'm afraid ZConfig might be the place to look for that.
<gary_poster> Author of that log rotate code in ZConfig is Fred Drake, but I don't think knowing that helps us any :-)
<gary_poster> so that's it henninge
<henninge> ;-)
<henninge> gary_poster: thank, that's a start
<henninge> s
<gary_poster> cool henninge, thanks for looking at that one
<henninge> gary_poster: it was escalated, so somebody has to do it ... ;-)
<gary_poster> :-)
<bac> deryck[lunch]: making progress
<bigjools> good night all
<sinzui> jcsackett, do you have time to mumble
<jcsackett> sinzui: sure.
<jcsackett> sinzui: i'm on mumble now.
<LPCIBot> Project devel build #920: FAILURE in 5 hr 39 min: https://lpci.wedontsleep.org/job/devel/920/
<lifeless> sinzui: hi
<deryck> Later everyone.
<sinzui> hi lifeless
<lifeless> sinzui: hi; I've forwarded you a mailling list test failure
<sinzui> I saw
<lifeless> do we use the --notifications option ?
<lifeless> and do we want it to override this quieter behaviour ?
<sinzui> The list import is an interesting case. I think we can drop it though since
<sinzui> lifeless, I really do not know about --notifications
<sinzui> I was not certain what was going on in the archive email failure. Do we add users to teams when they are subscribed to archives?
<lifeless> sinzui: the test made an open team
<sinzui> I really do understand that case then
<lifeless> sinzui: so it then purged the mails that adding someone to the team generated. (factory.makeTeam makes OPEN teams)
<lifeless> sinzui: e.g. the archive failure was trivial and you can ignore it.
<lifeless> sinzui: the comment above the failing line said (paraphrased) 'ignore the mails about joining the team'
<sinzui> okay
<lifeless> sinzui: so, I'm trying to change the test_mlists test to use a moderated team
<allenap> Night all.
<sinzui> lifeless, we have run The mailing list import script twice
<sinzui> lifeless, it is run by a losa to migrate a list. it create and find users from the email address in the foreign list id
<sinzui> lifeless, I think the emails are spam in this case. When the import is done, the team admin is going to inspect the  team subscription page and ask for the log of the import
<sinzui> s/id/DB/
<lifeless> sinzui: this fixes the test: http://paste.ubuntu.com/652682/
<lifeless> sinzui: but if you want, I could rip out the --notifications feature entirely.
<sinzui> lifeless, keep the test chnge
<sinzui> change
<sinzui> I do not think this script will be used until it can be used via the UI
<lifeless> ok, so that pastebin is ok with you ?
<sinzui> yep. +1
<lifeless> I will land this change then.
<sinzui> StevenK, mumble?
<sinzui> StevenK, http://pastebin.ubuntu.com/652721/
* StevenK changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: StevenK | Critical bugs: 238 - 0:[#######=]:256
* wgrant changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: StevenK | Critical bugs: 240 - 0:[#######=]:256
<wgrant> These "Job ran too long" scanner OOPSes really suck.
<wgrant> sinzui: https://code.launchpad.net/~wgrant/launchpad/transitionToTarget-series-sourcepackage, if you have time.
#launchpad-dev 2011-07-27
<StevenK> wgrant: I'm reviewing a branch of allenap's. Isn't there an already defined ACTIVE_PUBLISHING_STATES that is PENDING, PUBLISHED already?
<wgrant> StevenK: It's active_publishing_status
<poolie> hi all
<poolie> can someone confirm that it's possible for ubuntu developers to rebind the lp:ubuntu/oneiric/foo alias onto a different branch?
<poolie> they do that through the web ui?
<StevenK> wgrant: Ah! Thanks!
<wgrant> poolie: They can do it through the API.
<poolie> ah i see
<poolie> is there a tool for that?
<wgrant> It was restricted to ~ubuntu-branches until like a month ago
<poolie> or a web page describing it?
<wgrant> No.
<wgrant> It's not a supported workflow.
<wgrant> The only reason it's no longer restricted to two people is to remove the celebrity.
<wgrant> You would need to talk to the UDD people about whether that should be supported.
<poolie> i'm thinking in particular about https://lists.ubuntu.com/archives/ubuntu-devel/2011-July/033777.html
<wgrant> poolie: That's a debian-only branch.
<wgrant> Last I heard the policy was that official package branches must be complete.
<wgrant> Regardless, this is not a Launchpad issue.
<wgrant> This is UDD policy.
<poolie> :/
<lifeless> so the reason for completeness is reproducability
<lifeless> debian only branches are entirely lacking there
<poolie> i don't think "not a launchpad issue" is a very productive attitude
<wgrant> Hm?
<wgrant> How not?
<wgrant> This is a policy that was set by james_w and co.
<poolie> if people are having trouble manipulating ubuntu source, it at least touches on lp
<wgrant> Launchpad doesn't set UDD policy...
<poolie> it may not be something that ends up being changed in lp
<wgrant> Also, everybody that went near UDD has since left Launchpad.
<wgrant> bzr or whoever replaced james_w is probably better.
<poolie> anyhow, i will talk about it with luke
<poolie> i'm not saying "~launchpad must solve this"
<poolie> only that i think it's reasonable to ask about it here
<wgrant> We can answer technical questions about our dodgy implementation of package branches.
<wgrant> But most of this stuff is UDD policy.
<wgrant> Which has never had much to do with the LP team.
<wgrant> So we are not exactly well-placed to answer questions.
<poolie> so, as far as the narrow question
<poolie> it is theoretically possible to rebind ubuntu branch aliases, but this should never be done?
<poolie> (i realize this won't be actually useful in this case)
<wgrant> It is possible (the importer does it).
<lifeless> it should be done if the official branch is being entirely replaced
<lifeless> but it would be extremely unusual and need notificatoin of some sort to anyone with pending merges, checkouts,e tc
<wgrant> The relevant API call is source_package.setBranch.
<lifeless> uhm, thats too strong
<wgrant> Anybody with upload privileges should be able to do it.
<wgrant> But, as lifeless says.
<lifeless> its expected that when someone starts working with UDD, that the package they are workig on may involve their higher-fidelity branch replacing the importers branch.
<poolie> right
<lifeless> but all the branches need to be the same as what the importer creates, or it will stomp over them
<lifeless> e.g. full rather than debian only
<poolie> though, probably they could just push overwrite the importer's branch, rather than changing the name
<poolie> maybe not
<lifeless> right
<lifeless> a reason to rebind is to get to a branch in the ubuntu namespace
<lifeless> rather than in /~ubuntu-branches/ubuntu-branches/... or whatever the workaround before distro namespaces existed was
<wgrant> Huh.
<wgrant> PackageBugSupervisor
 * wgrant demolishes.
<StevenK> wgrant: There is a table too
<wgrant> Yes.
<wgrant> I fear I may need to drop the foreign key in an initial branch, drop the code in a second, and move the table away in a third.
<wgrant> So this can wait until we have the process sorted out :/
<lifeless> vat is dis ?
<wgrant> Table that has been obsolete since like 2007.
<wgrant> It's empty.
<lifeless> oo
<lifeless> clean out da cwap
<wgrant> I think I need to drop the foreign key before I can delete the code :(
<wgrant> Because of person merges.
<lifeless> yes
<wgrant> Tests will blow up if they don't handle all foreign keys.
<lifeless> thats right
<wgrant> Which means DB, code, DB branches.
<lifeless> yes
<wgrant> Hopefully next week we will have fastdowntime :)
<lifeless> or
<lifeless> feature flag
<lifeless> but that won't help if you're dropping a column in a used table
<lifeless> if there are no columns in used tables
<lifeless> you should be able to FF it, and add the flag value in the DB patch
<lifeless> giving you the less disruptive
<lifeless> code DB code
<lifeless> sequence
<wgrant> That's not a terrible idea. But it's more complicated and involves new code, when DB changes should be cheap soon.
<lifeless> its one less downtime window
<lifeless> which is not to be sneezed at
<LPCIBot> Project db-devel build #756: FAILURE in 5 hr 31 min: https://lpci.wedontsleep.org/job/db-devel/756/
<poolie> huwshimi: have a look at bug 816757?
<_mup_> Bug #816757: people didn't find the "download diff" link <code-review> <confusing-ui> <ui> <Launchpad itself:New> < https://launchpad.net/bugs/816757 >
<huwshimi> poolie: Interesting.
<poolie> i'm interested in the particular case but also about how to deal with "Foo is confusing" bugs generally
<poolie> i think this particular one is fairly close to being a hard-edged testable bug
<lifeless> wgrant: anyhow, please consider it - making db changes cheaper still doesn't make them free (until we're < small num (e.g. 200ms) on all web requests an can just slide them in without disruption)
<lifeless> wgrant: 3 /    0  Bug:EntryResource:newMessage
<huwshimi>  poolie: This is interesting in that it seems like the download diff link is in a fairly good place (it's in the first place I thought to look when I went to an mp after reading the bug report -  I don't think I've ever tried to download a diff before).
<huwshimi> poolie: There could be a number of reasons why people don't notice the feature. It could be as simple as people want to download the diff once they've read it, so the download diff link has scrolled off the page, or I could take a guess at other places that would be more obvious.
<huwshimi> poolie: However, that is a not a good approach. The only really good way to figure it out would be with data (otherwise how can I test the change I'm making has actually resulted in people using the link more). What I would like to do is to a/b test it. That way I can quantitatively determine which position/style/wording results in the most use.
<poolie> i would have guessed ctrl-f 'download'
<poolie> i agree about a test
<poolie> i would be thrilled to see someone get data out of logs
<huwshimi> poolie: Often people don't think to look for something. Often people only use feature's if they're obvious.
<poolie> ah, i also think perhaps having more of a system of what goes where might help
<huwshimi> poolie: That's why I think the current location is good. It has a number of controls to do with the current block of content in a header (in this case the content is a diff, we do a similar thing with comments).
<poolie> for instance this is a bit like other "downloadable attachments" which in some cases are in portlets
<poolie> often those things on the shaded background are fairly uninteresting controls, though
<poolie> but, this is just random speculation
<LPCIBot> Project devel build #921: STILL FAILING in 5 hr 40 min: https://lpci.wedontsleep.org/job/devel/921/
<wgrant> BugTask conjoinment ararrrrrrrgh
<wgrant> Perhaps it was all a cruel joke.
<StevenK> wgrant: Still?
<wgrant> lifeless: Should bug #809719 be closed now we've upgraded/
<_mup_> Bug #809719: current lazr.batchnavigator version requires high OFFSET queries <qa-ok> <timeout> <Launchpad itself:Fix Committed by lifeless> < https://launchpad.net/bugs/809719 >
<lifeless> yes
<wgrant> Thanks.
<sinzui> wgrant, r=me
<wgrant> sinzui: Thanks!
<StevenK> Hmmmm. I think the lazr.restfulclient is going to force a new release of launchpadlib too
<nigelb> No one's updating https://dev.launchpad.net/Contributions ?
<lifeless> every now and then
<nigelb> I wonder if I can drop a cron on my box.
<StevenK> Bah, bzr grep doesn't support -A
<wgrant> I should land my patches and run it on devpad.
<wgrant> Bah, now it crashes with a Unicode error.
 * wgrant blames rvba.
<wgrant> nigelb: Updated.
<wgrant> Ah, it was lool, not rvba.
<wgrant> How dare people have non-ASCII names.
<nigelb> wgrant: ah, thanks.
 * StevenK renames wgrant to Å´Ã­llÃ­Ã¡m Ä ÅÃ¡ÅÅ¥
<nigelb> beat me to it ;)
<spiv> wgrant: impressive that poolie is at both #6 and #12 of the second list.
<wgrant> Yeah.
<wgrant> I'm trying to fix that up.
<nigelb> I keep forgetting poolie isn't Launchpad per se.
<wgrant> The last update should have.
<wgrant> But the missing trailing > seems to be breaking something.
<wgrant> poolie has like 5 identities, some of them with typos :)
<nigelb> lol
<poolie> i prodded ohloh a while ago to help get their bzr-format scans going
<poolie> i wonder if they work yet
<wgrant> It's failed probably 10 times now :/
<wgrant> poolie is now promoted from 6 + 12 to 3.
<poolie> hooray
<poolie> i wonder if i ever set my identity wrongly in bzr, or if it got mangled somewhere in lp?
<poolie> istr seeing other duplicates on that list
<wgrant> It turns out the typo was actually in my mapping.
<wgrant> There is the full 'Martin Pool <mpb@c.c>', 'mpb@c.c', and 'mpb@sf.n'
<wgrant> The 'Martin Pool <mpb@c.c' was my fault.
<nigelb> heh, I wondered why 6 + 12 = 3 ;)
<spiv> wgrant: Now it's james_w that's being robbed :P
<nigelb> hmm, need to get 6 more landings to be in top-5
<poolie> wgrant: sadly you'll need to evict yourself from that list
<poolie> or is it time limited?
<nigelb> poolie: but those are pre-canocical days right?
<wgrant> spiv: For exactly the same reason. I must have copied it :(
<poolie> ah, apparently it is
<nigelb> I see only 2010
<wgrant> poolie: That's only my pre-Canonical commits.
<wgrant> Everything after I joined is with my canonical.com address, which magically gets excluded.
<wgrant> But maybe I should remove myself entirely.
<nigelb> nah
<nigelb> its nice to see what you did *before* joining
<wgrant> Gives you something to aspire to :P
<nigelb> Yes, that too.
<poolie> yes exactly
<spiv> wgrant: I suggest grepping for <[^>]*$ :P
<nigelb> 128 to go to beat wgrant :P
<poolie> i kind of want to see the scoreboard for ~canonical-launchpad too though
<spiv> wgrant: I see at least one other unmatched <> pair in those lists.
<poolie> and whether non-canonical wgrant beats them
<wgrant> poolie: bzr stats
<wgrant> Actually, I guess that doesn't quite do it.
<wgrant> Since that's number of commits, not number of merges.
<wgrant> poolie: I may do that tonight and see what happens.
<nigelb> poolie: heh, that should be interesting
<nigelb> I'd like to see if the pre-Canonical wgrant beats the Canonical wgrant.
<wgrant> spiv: Luke?
<spiv> Yeah.
<wgrant> That's a real bzr whoami typo.
<wgrant> Maybe I will map it.
<spiv> Ah, heh.
<nigelb> btw, bugzilla will probably soonish have a generic Android app. For triaging bugs.
<nigelb> (Full disclosure: I'm writing it)
<poolie> nice
<wgrant> Hm.
<poolie> heh
<wgrant> james_w didn't get any extra merges.
<wgrant> I guess all the linaro address commits were in branches with canonical commits too.
<wgrant> spiv: Any more issues?
<poolie> nigelb: for any site in particular?
<poolie> or, user
<nigelb> poolie: bugzilla.mozilla.org in particular.
<poolie> ok
<poolie> this is so people can triage on the train or something?
<nigelb> exactly :)
<nigelb> there are a few people triaging on commute
<nigelb> or having to do something quick on commute
<spiv> wgrant: hmm, I guess I have some commits that ought to be eligible, but I'm still a member of various launchpad-related teams so it's probably not a surprise that the script excludes me.
<nigelb> spiv: er, you aren't Launchpad team now?
<spiv> nigelb: bzr
<wgrant> spiv: Indeed.
<wgrant> spiv: There's no date-based restrictions.
<nigelb> Ah
<wgrant> spiv: I suspect you are on the Launchpad list because you were Launchpad years ago.
<wgrant> Many years ago.
<spiv> wgrant: â¦and still heckling!
<wgrant> For now :)
<nigelb> heh.
<StevenK> How do I make a launchpadlib release?
<wgrant> There's not a document in the branch?
<wgrant> https://dev.launchpad.net/HackingLazrLibraries#Releases
<wgrant> https://lists.launchpad.net/launchpad-dev/msg02076.html
<StevenK> Bleh. 1.9.8 is the latest, but we're still using 1.9.3
<lifeless> also
<lifeless> dev.launchpad.net/ReleaseChecklist
 * StevenK confused
<StevenK> release series?
<wgrant> StevenK: Presumably the Launchpad series name.
<wgrant> You could check the code.
<StevenK> Which seems to be trunk
<StevenK> lifeless: Can you upload launchpadlib 1.9.9 to PyPi?
<lifeless> whats your pypi account /
<StevenK> I don't have one
<wgrant> It does OpenID, so you can fix that quickly.
<StevenK> lifeless: The usual suspect. IE, 'stevenk'
<lifeless> ok, you can now.
<StevenK> Hm
<StevenK> Server response (401): basic auth failed
<StevenK> :-(
<lifeless> StevenK: you may need a regular account
<lifeless> https://dev.launchpad.net/Downtime - updated for new world order, could use a set of eyeballs
<StevenK> lifeless: Sorted out anyway
<lifeless> what was t ?
<StevenK> I used the web interface to do it
<lifeless> poolie: gentle reminder - please triage bugs you file on LP; no need for another teammember to do it (unless you're genuinely unsure that it is a bug or how it fits in current work plans)
<poolie> yeah
<poolie> in this specific case i was talking to people about what to do with it
<poolie> didn't finish the transaction
<wgrant> The janitor took it half-way...
<lifeless> someone me too'd it
<wgrant> Yeah.
<poolie> done
<StevenK> How the heck do I express 100 AS rank in Storm?
<wgrant> Try SQL('100 AS rank')
<LPCIBot> Project db-devel build #757: STILL FAILING in 5 hr 34 min: https://lpci.wedontsleep.org/job/db-devel/757/
<StevenK> But Storm has Alias which does AS?
<wgrant> I'm not sure if that works for non-classes.
<StevenK> Neither
<jtv> StevenK: that may be the best code review I've ever had.
<StevenK> Haha
<StevenK> Was it as good for you as it was for me? :-)
<wgrant> Emphatic condonement of the removal of commercial-compat, or have you been secretly working on something similarly glorious?
<StevenK> wgrant: The former.
<wgrant> jtv: Given that you have both dogfood access and Translations expertise, could you perhaps be convinced to QA bug #788685?
<_mup_> Bug #788685: Enable translating selected Ubuntu universe packages in Launchpad <escalated> <oem-services> <qa-needstesting> <Launchpad itself:Fix Committed by bac> <OEM Priority Project:Invalid> <Ubuntu Translations:Triaged> <pkgbinarymangler (Ubuntu):Fix Released by pitti> < https://launchpad.net/bugs/788685 >
<jtv> I can try.  First I'll have to take some time to figure out just what's changed.
<jtv> StevenK, wgrant: by the way, mind if I update dogfood at this point?
<wgrant> jtv: Not at all.
<wgrant> The change is fairly simple, if slightly nauseating.
<adeuring> good morning
<jtv> hi adeuring
<adeuring> hi jtv!
<mrevell> Hello
<LPCIBot> Project devel build #922: STILL FAILING in 5 hr 43 min: https://lpci.wedontsleep.org/job/devel/922/
<allenap> StevenK: Thanks for the review :)
<StevenK> allenap: No fair not looking for active_publishing_status :-P
<allenap> StevenK: I knew I could rely on splendid chaps such as your good self :)
<gmb> StevenK: Are you still around and on-call?
* StevenK changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: - | Critical bugs: 240 - 0:[#######=]:256
<henninge> danilos: hey, are you around?
<lifeless> henninge: gl
<henninge> gl?
<lifeless> henninge: good luck
<henninge> ah! ;-)
<henninge> lifeless: have you been trying to talk to him about QA for bug 788685?
<_mup_> Bug #788685: Enable translating selected Ubuntu universe packages in Launchpad <escalated> <oem-services> <qa-needstesting> <Launchpad itself:Fix Committed by bac> <OEM Priority Project:Invalid> <Ubuntu Translations:Triaged> <pkgbinarymangler (Ubuntu):Fix Released by pitti> < https://launchpad.net/bugs/788685 >
<lifeless> henninge: no; I was referring to your bombshell from last night :)
<henninge> lifeless: oh! ;-)
<henninge> lifeless: thank you ;)
<henninge> rvba: Hi!
<rvba> henninge: Hi!
<henninge> rvba: Could you please do QA for bug 815751?
<_mup_> Bug #815751: In initialize_series, the package copier copy method should be called with all the packages from pockets RELEASED, UPDATES, SECURITY. <derivation> <qa-needstesting> <Launchpad itself:Fix Committed by rvb> < https://launchpad.net/bugs/815751 >
<rvba> henninge: sure, I'll do that.
<henninge> rvba: I am trying to get a deployment together. Thanks. ;-)
<wgrant> henninge: Thanks!
<henninge> wgrant: can I have your opinion on bug 788685
<_mup_> Bug #788685: Enable translating selected Ubuntu universe packages in Launchpad <escalated> <oem-services> <qa-needstesting> <Launchpad itself:Fix Committed by bac> <OEM Priority Project:Invalid> <Ubuntu Translations:Triaged> <pkgbinarymangler (Ubuntu):Fix Released by pitti> < https://launchpad.net/bugs/788685 >
<henninge> ?
<gmb> StevenK: Point taken :)
<StevenK> gmb: :-)
<henninge> wgrant: QAing it involves dogfood and such and I know bac was working on figuring that out yesterday.
<wgrant> For this sort of thing I think a QA strategy should be devised before landing.
<henninge> yes
<wgrant> There is not one?
<henninge> bac asked me about it yesterday but I could not help him since it was mainly about uploading a source package.
<henninge> I havn't done that either.
<henninge> So I guess there is none.
<henninge> bigjools was helping him, maybe he knows more?
 * bigjools otp, yes I know about it
<henninge> still otp? that's what bac told me yesterday, too .... ;-)
<henninge> wgrant: I'll sort it out with bigjools when he gets otp (the other otp)
<wgrant> Thanks.
<wgrant> I'll probably still be pingable if you need me.
<danilos> henninge, yeah
<danilos> henninge, I know bac was talking to bigjools yesterday-ish
<danilos> (and only know I realize after reading the full backlog that you've already come to the same point :)
<danilos> henninge, I've also got an email from bac
<danilos> henninge, he was stuck looking for a package with translations, and I know of one (epiphany-browser)
<henninge> danilos: can you take care of the qa, or is that too much to ask?
<bigjools> henninge, danilos: hello
<bigjools> so, we need a package in universe
<bigjools> and you need to add a special header to debian/control
<bigjools> bac tried it with one package but it FTBFS
<bigjools> so a simple package would be good
<danilos> henninge, sure, I'll take it on
<henninge> danilos: thanks
 * henninge reboots computer
<danilos> bigjools, I've uploaded epiphany-browser package to dogfood, can you remind me what the next step is? will it get rejected because I am not the ubuntu guy?
<bigjools> danilos: yeah, I'll add you to core-dev
<danilos> bigjools, thanks
<bigjools> danilos: grar, there's a million danilos!
<danilos> bigjools, I am only "danilo" on LP, and you can always look for "ÐÐ°Ð½Ð¸Ð»Ð¾" :)
<bigjools> No items matched "ÐÐ°Ð½Ð¸Ð»Ð¾".
<bigjools> :)
<danilos> bad, bad search
<bigjools> I can't even do ~danilo
<wgrant> bigjools: You can also search for 'danilos' and it will always return a username match as the first result.
<bigjools> 17 pages of results
<bigjools> aha fluked it
<wgrant> Er, 'danilo' is the username, but 'danilos' should still get OK IRC nick results.
<danilos> I am in, thanks bigjools :)
<bigjools> danilos: just processing the upload
<wgrant> bigjools: 'danilo' returns danilos as the first result... did it not for you?
<bigjools> nope
<wgrant> danilos returns him as the third result.
<bigjools> yep
<bigjools> danilos: 2011-07-27 11:14:48 INFO    Upload was rejected:
<bigjools> 2011-07-27 11:14:48 INFO        Unable to find epiphany-browser_3.0.4.orig.tar.bz2 in upload or distribution.
<danilos> bigjools, what's the dput option to force the orig.tar.gz to be included as well?
<danilos> or was that a debuild option?
<bigjools> debuild -S -sa
<danilos> bigjools, I should up the version number again?
<bigjools> no
<danilos> ok
<bigjools> dput -f though
<danilos> bigjools, yep, using that already, all set now, please try the upload processor again
<bigjools> accepted
<jml> does "Related bugs" also show bugs that affect me?
<poolie> jml i believe it does
<danilos> bigjools, I should have probably used the latest oneiric epiphany package from dogfood and not from production, this might fail to build as well
<jml> poolie: thanks.
<poolie> i think it is "related in any way"
<poolie> imbw
<bigjools> danilos: we'll see!
<wgrant> jml: I don't believe it does.
<jml> since I'm working across quite a few projects now, really feeling the need for some more aggregate views.
<wgrant> jml: I don't know of any way to find bugs that affect you.
<jml> oh no
<jml> a conflict of beliefs.
<jml> crap. I'm going to have to either try science or consult an authority.
<wgrant> It's just an aggregation of the other 4 or 5 views.
<wgrant> commented/reported/assigned/whatever
<wgrant> There is a simple path of deduction which allows you to work this out :)
<danilos> bigjools, yeah :) I'll also prepare a natty package to confirm that one doesn't get translations extracted and uploaded
<jml> wgrant: need observations to make deductions.
<wgrant> jml: Well, there are some generally applicable observations, I suppose.
<wgrant> But anyway, code.
<poolie> he's right, i'm wrong
<poolie> i think i'm right that related means 'any relation' but lp won't tell you about affecting bugs
<wgrant> It's just the union of the other categories in that menu.
<jml> that's unfortunate
<poolie> right
<poolie> there's a bug for it
<jml> I've marked bugs as affecting me in the hope that I'd be able to recover that information later
<wgrant> Many things are unfortunate, but it seems that people only work this out once they leave Launchpad management :)
<jml> wgrant: don't be a dick.
<wgrant> It's *possible* there's an API call.
<wgrant> I think someone might have added one at some point.
<wgrant> Nope, only HWDB's deviceDriverOwnersAffectedByBugs :(
<jml> poolie: https://bugs.launchpad.net/launchpad/+bug/323000 fwiw, it's one of the top 10 hot bugs.
<_mup_> Bug #323000: Make "this bug affects me too" bugs visible from user profile <lp-bugs> <profile> <Launchpad itself:Triaged> < https://launchpad.net/bugs/323000 >
<poolie> yeah, i voted for it
<poolie> i would like it alot
<jml> well, it's down in my yak stack, whatever that counts for.
 * jml has to relocate.
<danilos> bigjools, ok, the build finished successfully, I remember us needing to run upload processor again (or something) to get the translations.tar.gz to be pushed through to the translations import queue
<danilos> bigjools, I do see the translations.tar.gz on https://dogfood.launchpad.net/ubuntu/oneiric/+queue fwiw
<bigjools> danilos: then it worked I guess
<danilos> bigjools, right, I'd like to confirm it shows up in the translations queue, because it still doesn't
<bigjools> danilos: accept the upload then
<danilos> bigjools, how do I do that?
<henninge> gary_poster: Hi!
<bigjools> danilos: you're in the wrong team, I'll do it :)
<danilos> bigjools, heh, people keep telling me that
<bigjools> danilos: ok accepted
<gary_poster> henninge, hey!
<gary_poster> I have another conversation going henninge, just so you know, but no problem as long as you forgive me if it takes me a bit to reply sometimes :-)
<danilos> bigjools, do we run the upload-processor again?
<bigjools> danilos: I need to run process-accepted
<bigjools> running now
<danilos> bigjools, ah, ok, thanks
<danilos> this is so much fun :)
<bigjools> 2011-07-27 11:57:24 DEBUG   Publishing custom epiphany-browser, epiphany-browser_3.0.4-1ubuntu2_i386_translations.tar.gz, epiphany-browser_3.0.4-1ubuntu2_static_translations.tar.gz to ubuntu/oneiric
<bigjools> danilos: all done
<henninge> gary_poster: that's ok. I just had a quick question on how LP is running and I guess you are not the only one who could answer that.
<danilos> bigjools, excellent, it worked; now I'd like to test that we haven't broken anything for natty (i.e. universe packages still can't get translations on), but I got an "upload rejected" email
<henninge> gary_poster: actually, I just thought I'll try something else first before asking, so nm me atm. ;-)
<danilos> bigjools, actually, "upload rejected" email is because I tried the main archive ;)
<danilos> bigjools, so, uploaded to dogfood, and we'll need to go through the same dance again for natty: upload processor, wait for build, and if translations.tar.gz doesn't show up, we are good, if it does, we'll have to check that it doesn't end up in the queue
<gary_poster> henninge :-) ok.  looks like other conversation is mostly handled atm, so feel free to ping if you want to later
<bigjools> danilos: ok
<LPCIBot> Project db-devel build #758: STILL FAILING in 5 hr 37 min: https://lpci.wedontsleep.org/job/db-devel/758/
<danilos> bigjools, can you please run the upload processor then?
<bigjools> danilos: 2011-07-27 12:01:41 DEBUG   Not permitted to upload to the RELEASE pocket in a series in the 'CURRENT' state.
<bigjools> target -updates
<danilos> bigjools, ah, natty-updates?
<henninge> wgrant: we had some revisions for bug 80902 land and that reverted r13514 back to needstesting.
<_mup_> Bug #80902: Can't target bug report from project to distribution, or vice versa <bugs> <disclosure> <javascript> <linaro> <lp-bugs> <project-picker> <qa-needstesting> <questions> <ubuntu-qa> <Launchpad itself:In Progress by launchpad-teal-squad> < https://launchpad.net/bugs/80902 >
<bigjools> danilos: yes
<henninge> henninge: is that revision ok to be deployed? How can I properly mark it as such without removing needstesting from the others?
<danilos> bigjools, done
<wgrant> henninge: Fixed.
<bigjools> danilos: it should be building
<danilos> bigjools, I don't see it happening on https://dogfood.launchpad.net/builders :/
<gmb> danilos, Are you OCRing today?
<danilos> gmb, oh right, I should be doing that today as well :/
<gmb> Yeah, no rest for the weary.
* danilos changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: danilos | Critical bugs: 240 - 0:[#######=]:256
<gmb> danilos: If you don't have the brainjuice, I'll happily ping some otther schmuck^Wreviewer
<danilos> gmb, I do, I do, but it's best to wait until after the call
<gmb> danilos: Okidoke.
<bigjools> danilos: sorry it was stuck in UNAPPROVED
 * danilos nods in "while of course" style, hoping bigjools knows how to get it out of unapproved state :)
<bigjools> danilos: it should be building :)
<danilos> bigjools, still don't see it :/
<bigjools> argh
<bigjools> my fault
<bigjools> danilos: now?
<danilos> bigjools, nope
<danilos> I am looking at https://dogfood.launchpad.net/builders, should I look somewhere else?
<bigjools> the build page would be a good start :)
<gmb> danilos: So, https://code.launchpad.net/~gmb/lazr.restful/accessor-for/+merge/69416 is the branch, when you've got a second. I think you owe me now ;)
<danilos> gmb, sure thing
<danilos> bigjools, you mean https://dogfood.launchpad.net/ubuntu/+source/epiphany-browser/2.30.6-1ubuntu6?
<wallyworld__> sinzui: ping
<danilos> i386 is in 'needs building' state, but I don't know what needs to happen for it to be picked up (that was buildd-manager or whatever the name is now, right)
<danilos> gmb, just before I dive in deeper, is the change to versions.cfg really required or just a drive-by fix?
<gmb> danilos: It's required to get it to build these days.
<danilos> gmb, ah, ok, just checking before I dive in deeper
<gmb> danilos: It's not a requirement for my change, per se, but it would have been hard to test otherwise :)
<henninge> wgrant: thanks
<danilos> gmb, btw, have you tried ensuring all classes are new-style-classes before using super()? (eg. define them as "class Something(object):" instead of just "class Something:") - not sure that'd help, but just an idea
<gmb> danilos: Yes, I tried that. Didn't help, unfortunately.
<danilos> gmb, ack
<henninge> gary_poster: ping
<gary_poster> henninge, yo
<henninge> gary_poster: from this I see that LP is running in different threads/processes ...
<henninge> https://pastebin.canonical.com/24724/
<henninge> that is where the race condition comes from.
<gary_poster> henninge, threads, yes
<danilos> gmb, btw, that versioning of accessors (lines 218-253), what does it exactly do? (the latter part seems clear, but why do we need the 'ensure only one accessor' part)
<henninge> gary_poster: that means they share memory?
<henninge> gary_poster: I am jumping ahead here.
<henninge> gary_poster: I am just trying to figure out what is going on.
<gmb> danilos: It's to avoid collisions. If you have an implicit accessor and an explicit one for the same version, which one are you supposed to call to get the value?
<gmb> (I cargo culted that, but that's what I took its intention to be)
<danilos> gmb, also, I have no suggestions on how to do it, but it seems as if it'd be very nice to have tests for that code
<danilos> gmb, oh, ok
<henninge> gary_poster: so this is the same on productio?. On process (that gets signalled for log rotation) running in multiple threads. On process per appserver instance.
<danilos> gmb, implicit = a regular exported attribute?
<gmb> danilos: Ah, good point. There are similar tests for the mutator code, so I'll add some to cover the only-one-accessor-per-version stuff.
<henninge> gary_poster: and each process with its own set of log files.
<gary_poster> henninge, I am 95% sure they are threads, though production reduces number of threads serving users to 1 at a time.  That still means we have at least two threads: main thread, with asyncore loop (similar to Twisted loop); and worker thread, actually figuring out what to serve
<gmb> danilos: Or @accessor_for without @operation_for_version
<gary_poster> henninge, each process typically has its own log files
<danilos> gmb, cool (don't forget the test for defined in version N gets included in N+1 if it's not too hard :)
<gary_poster> hennige, Python's stdlib logging module is supposed to be thread safe
<gary_poster> henninge, so is ZConfig, but that's certainly in less usage
<gmb> danilos: Sure. I'm sure leonard wrote something that I can rip off ;)
<danilos> gmb, excellent :)
<henninge> gary_poster: yes, loghandler simply does a close -open sequence with no coordination what so ever.
<henninge> no thread safety there.
<gary_poster> henninge, "On process (that gets signalled for log rotation) running in multiple threads. On process per appserver instance." If I magically transmute "On" to "One" then that is correct
<henninge> what I meant.
<gary_poster> I figured, was being silly :-)
<danilos> bigjools, btw, nothing is happening with https://dogfood.launchpad.net/ubuntu/+source/epiphany-browser/2.30.6-1ubuntu6/+build/2492859 still? got any ideas?
<bigjools> danilos: no, and sorry I can't help right now I'm trying to debug a problem
<henninge> gary_poster: I am not familar with threads in python.
<gary_poster> henninge: close/open sequence: logging module is supposed to be friendly to that, I thought--unless you mean it is a raw file close?
<henninge> gary_poster: that is in ZConfig
<danilos> bigjools, np, please ping me when you have some time
<henninge> gary_poster: ZConfig.component.logger.loghandler
<gary_poster> henninge, right, but I mean, it gets the log files themselves--the file handles--and closes them directly?  OK looking
<bigjools> danilos: I just checked the manager log and the builder is failing for some reason
<bigjools> probably no chroot available
<henninge> gary_poster: AFAIUI, yes.
<henninge> gary_poster: yes, "self.stream"
<henninge> gary_poster: looks like a standard file handle
<deryck> Morning, all.
<adeuring> morning deryck
<danilos> bigjools, can we perhaps get back to this later when you are done with that debugging, or is it more of a general problem?
<gary_poster> henninge, sideways thought: have you checked stdlib logging module to see if they (now?) have a solution for logrotation themselves?
<gary_poster> morning deryck
<danilos> gmb, btw, r=me as per our discussion
<henninge> gary_poster: have not, no.
<gmb> danilos: Excellent, thanks.
<henninge> Hi deryck! ;)
<henninge> gary_poster: so you think it might be best to replace the ZConfig logging stuff?
<henninge> s/best/an idea/
<gary_poster> henninge, if I were looking at this problem that would certainly be an option on the table.  Moving to something more maintained would be a win.  But looking at that file you were pointing to...
<gary_poster> henninge, I don't think closeFiles is dealing with file objects
<gary_poster> henninge, the reason why is in "reopenFiles"
<gary_poster> it calls h.reopen()
<henninge> gary_poster: I  was looking at FileHandler.reopen.
<gary_poster> reopen is not on a Python file object
<henninge> that is a few lines down.
<gary_poster> henninge, ah you might be right there.  Not that this would be necessary, but have you pdb'd in to confirm, by chance?  or verified in logging module?
<gary_poster> verifying in logging module docs might be a minimal reasonable solution :-)
<gary_poster> henninge, so if your guess is right, then I see two paths to a solution so far
<henninge> gary_poster: i just looked into the logging module and self.stream is the stream passed in.
<henninge> which in this case is simple "open" call.
<gary_poster> 1) confer with Fred Drake and see if you can get a ZConfig change in, or maybe do it ourselves locally and then try to push it upstream.  A simple solution there would be to remove self.stream.close(): Python will close it when it cleans up trash.  Meanwhile, if a thread is still writing to it, it can.
<gary_poster> henninge, case closed then
<gary_poster> 2) see if logging module has an alternative.
<gary_poster> option 1 sounds so easy I would be inclined to try that
<gary_poster> coordinating with upstream can be annoying, I grant you :-)
<henninge> ;-)
<henninge> gary_poster: thanks, I'll look at both (since the first one is so easy) ;)
<gary_poster> We could make a local change, see if it helps, and if it does *then* confirm with upstream and try to get a change.  Extra bonus for you: you are gone by then ;-)
<gary_poster> ok cool henninge
<henninge> gary_poster: it really takes that long?
<gary_poster> henninge, what is "it" in that sentence? :-) I meant proving to ourselves that we have fixed the problem.  Since it is intermittent, that might take a little bit.
<henninge> oh, ok
<gary_poster> henninge, coordinating with upstream depends on the upstream.  Fred is a good guy, but busy, and his company no longer really cares about open source afaict
<henninge> "it" was referring to "coordinating with upstream"
<gary_poster> gtcha
<gary_poster> if he agrees, could be fast
<gary_poster> I could mae a release
<gary_poster> if he doesn't agree....
<gary_poster> <shrug> :-)
<gary_poster> having "proof" that it helps I thought might be a way of getting him to agree more easily
<henninge> gary_poster: So, I don't think this can (or should) be pressed into a test of any sort, since race conditions are hard to reproduce reliably.
<henninge> gary_poster: that means, if I go for option 1 it is just a matter of removing the line and see if it helps in production, right?
<bac> bigjools: when you have time, can you do your thing to get this building?  https://dogfood.launchpad.net/ubuntu/+source/epiphany-browser/2.30.6-1ubuntu6/+build/2492859  (it is the upload danilos did)
<gary_poster> henninge, you could probably make a test that showed that the original file was not closed after rotating.  If you do that at a low-enough level then that would be easy.  "easy"? :-) but that would be a ZConfig test.  That would be the kind of thing to write to have it go upstream.
<gary_poster> I definitely agree that we do not need a LP test for this change.
<bigjools> bac: the builder is failing, I don't know why
<bac> oh.  :(
<bigjools> well in fact it seems to have fallen over entirely
<wgrant> bigjools: The chroot has expired
<wgrant> http://librarian.dogfood.launchpad.net/70572775/chroot-ubuntu-natty-i386.tar.bz2 -> 404
<bigjools> ferraz is also not responding
<wgrant> That could also be a problem.
<wgrant> Working for me...
<wgrant> So, upload a new natty/i386 chroot, I suspect.
<rvba> henninge: sorry, I'm having trouble with my QA ...
<rvba> henninge: It's going to take a little bit longer than what I thought ...
<henninge> rvba: I am in the stand-up call atm, I can see if I can help you afterwards.
<rvba> henninge: I'm trying to work it out with the help of Julian ...
<bac> abentley: i thought you could specify a pre-req branch with 'bzr lp-propose' but it isn't documented.  can you do that?
<abentley> bac: It auto-detects pre-req branches from a pipeline.  Specifying a pre-req from the commandline is a missing feature.
<bac> abentley: ok, i'm not using pipelines, so i'll not use it
<abentley> bac: sure.
<abentley> bac: just merged and pushed.
<deryck> adeuring, abentley, henninge -- forgot to mention that I'll be switching offices in about 45 minutes and will be offline for 20-25 minutes.
<abentley> deryck: okay.
<rvba> henninge: qa-ok. Endlich.
<henninge> rvba: Super!
<henninge> rvba: I just came off the call ...
<rvba> henninge: Perfect timing :)
<wallyworld__> sinzui: you there?
<sinzui> I am
<wallyworld__> sinzui: i have a failing test i can't figure out without more digging and perhaps you can see the issue
<wallyworld__> it's a web service test
<sinzui> okay
<wallyworld__> take a look at the diff in this mp: https://code.launchpad.net/~wallyworld/launchpad/question-subscribe-webservice/+merge/69433
<wallyworld__> it's the test_subscribe()
<wallyworld__> the question.subscribe(person) call fails
<wallyworld__> with error: TypeError: Could not serialize object <security proxied lp.answers.model.questionsubscription.QuestionSubscription instance at 0xa847c90> to JSON.
<wallyworld__> i have modelled the web service stuff for IQuestionSubscription on IBugSubscription
<wallyworld__> but i can't see what else i need to do to resolve the issue
<wallyworld__> i think there needs to be a json adaptor of some sort
<wallyworld__> but if there does, i can't see where it's been set up for IBugSubscription
<sinzui> ah
<LPCIBot> Project devel build #923: STILL FAILING in 5 hr 51 min: https://lpci.wedontsleep.org/job/devel/923/
<sinzui> wallyworld__, I am unfamiliar with the publish_web_link=False arg of  export_as_webservice_entry
<wallyworld__> i figured out at one point what it did but can't recall exactly now
<wallyworld__> it fails with and without
<sinzui> wallyworld__, I have not used that, but your error does imply a url is wanted
<sinzui> wallyworld__, I added urls to a few question objects a few months ago to make export work
<wallyworld__> i thoought it was trying to turn a QuestionSubscription into a json data object to serialise the result of a subscribe() call, so it is not clear to me atm why a url is needed
<wallyworld__> and also bub.subscribe() works
<wallyworld__> and it is set up the same way
<wallyworld__> bug.subscribe()
<sinzui> wallyworld__, removeAnswerContact is already a method on the target. Do you intent the question method to call the QT method?
<wallyworld__> yes. the subscription portlet invokes apis on the context object ie question
<sinzui> wallyworld__, creating a url is easy and we can try it to see if this solves the problem
<henninge> bac: Hi!
<wallyworld__> ok
 * sinzui tries to write the zcml without caffeine
<bac> hi henninge
<henninge> bac: any eta on that QA?
<henninge> I saw danilo and bigjools where helping?
<bac> henninge: there is a problem on dogfood (scroll up to see wgrant's comments about natty chroot) that i'll need to get assistance from bigjools to sort out.  so, no eta.
<bigjools> I am busy, and it needs someone to upload a natty i386 chroot to DF
<bac> bigjools: who can do that?
<bigjools> bac: it needs a losa to get one from production, put it somehwere in the DC where I can get it and then I can upload it
<bigjools> bac: scripts/ftpmaster-tools/manage-chroot.py
<bac> bigjools: i can facilitate the first part while your snowed under
<henninge> bac: thanks for the heads up
<sinzui> wallyworld__, try http://pastebin.ubuntu.com/653123/
 * wallyworld__ looks
<sinzui> wallyworld__, the attribute_to_parent value could also be the id you added to IQuestionSubscription
<wallyworld__> i'll try the pastebin as is....
<sinzui> wallyworld__, when this is complete will the subscriber portlet load after the page is loaded?
<wallyworld__> sinzui: you are a genius. that works. i don't fully understand why (my zope foo is not fully develped)
<wallyworld__> i need to add a bit more glue before it works, but it will be the same as bugs - the portlet will load after the page has loaded, yes
<sinzui> wallyworld__, I am not a genius. I just did not know about that arg you used. I wrote and export must of answers code you are working with. make me the reviewer since I have read most of this change and It looks good
<wallyworld__> sinzui: so, next branch, i need to extend the javascript portlet stuff to allow each subscription entry to specify the web service api to use to "unsubscribe" and then it's pretty much done
<sinzui> wallyworld__, you may accidentally fix a timeout bug doing this. the subscriber code can be expensive to use and I did not factor that out of the page load two months ago
<wallyworld__> cool, one less critical :-)
<wallyworld__> with the bugs portlet, the unsubscribe method will always be "unsubscribe", for questions, it will either be "unsubscribe" or "removeAnswerContext"
<wallyworld__> sinzui: i'll finish the mp and then need sleep. btw, i will likely miss the standup since i have to go and pick up a new car from the dealer
<sinzui> understood. Your working overtime right now
<sinzui> wallyworld__, bug 736003 is the one I was thinking of. I believe this *wont* be affected by your work since it is a question target page.
<_mup_> Bug #736003: Distribution:+questions timeouts <questions> <timeout> <Launchpad itself:Triaged> < https://launchpad.net/bugs/736003 >
<wallyworld__> ah, too bad
<wallyworld__> sinzui: mp is now ready for review, thanks for the help. i hate leaving stuff unfinished so am glad to get it done.
<sinzui> fab
<bac> benji: could you have a look at https://code.launchpad.net/~bac/launchpad/getnewcache/+merge/69476 ?
<benji> bac: sure
<bac> bigjools: mbarnett has prepared a chroot to be uploaded to dogfood:  devpad:~mbarnett/tmp/chroot-ubuntu-natty-i386.tar.bz2
<benji> bac: I'm done with https://code.launchpad.net/~bac/launchpad/getnewcache/+merge/69476
<bac> thanks benji
<henninge> gary_poster: that looks very promising. We could do away with SIGUSR2 handling alltogether.
<henninge> http://docs.python.org/library/logging.handlers.html#watchedfilehandler
<gary_poster> henninge, +1
<gary_poster> henninge, that will probably be more work that the other approach, and it might be tricky because I think ZConfig creates files
<gary_poster> I mean, is in charge of our log files
<gary_poster> but I would prefer it in the long haul
<gary_poster> & I suspect others would too
<henninge> gary_poster: ok, I will see how they'd work together.
<gary_poster> cool
<henninge> thanks  for the info
<henninge> gary_poster: the other thing I'd worry about is the extra load that each emit creates by doing the checks on the file.
<bac> benji: uh, thanks for catching that little bit of overexuberance in the page template.  clearly i hadn't intended to check that in
<gary_poster> henninge, probably not an issue.  any change other than "remove stream.close" will need approval and coordination from losas; if you were concerned about performance then you could check with Robert too, but unless that emit coe is doing a lot more than I expect, I think it should be fine.
<henninge> gary_poster: I just found that we have our own WatchedFileHandler implementation in our tree ... ;-)
<gary_poster> henninge, heh
<henninge> gary_poster: it's identical to upstream, so that was merged into logging.
<gary_poster> ah ok
<henninge> gary_poster: but it also has this comment:
<gary_poster> or vice versa
<henninge>             # BACKPORT: Doing a stat on every log seems unwise. A signal
<henninge>             # handling log handler might be better.
<henninge> ;-)
 * henninge goes to look who wrote that comment
<gary_poster> it may have been a backport from 2.6
<henninge> ah, ok
<henninge> yeah, reading module docstring helps
<deryck> adeuring, abentley, henninge -- I may drop from IRC as I reboot to recover my system.
<deryck> Not sure yet, but if I disappear, that's where I've gone. ;)
<abentley> deryck: okay.
<henninge> gl
<adeuring> deryck: sounds familiar to me today ;)
<deryck> adeuring: :)
<deryck> thanks, henninge
 * henninge learned gl from lifeless today
<sinzui> jcsackett, do you have time to mumble
* danilos changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: - | Critical bugs: 240 - 0:[#######=]:256
<henninge> gary_poster: so, I decided to go with the one-line change.
<gary_poster> hrnninge, sounds safest to me :-)
<henninge> gary_poster: but this change is not in our tree, I guess I need to create an egg/package?
<gary_poster> henninge, right.  Easiest story: Get the most recent ZConfig from our download cache, unpack it, make your change, and then follow instructions in "Work with Unreleased or Forked Packages" section of doc/buildout.txt.  unpacking the ZConfig from download cache takes the place of the svn checkout.
<gary_poster> A nicer approach would be to check out svn
<gary_poster> push to LP
<gary_poster> (or import it)
<gary_poster> then make a branch
<gary_poster> make your changes
<gary_poster> push to LP
<gary_poster> follow instructions
<henninge> ok, thanks
<henninge> gary_poster: we are not using the latest version btw.
<gary_poster> henninge, ah interesting.  You could see if latest version has change :-)
<henninge> gary_poster: yeah, looking at that now
 * gary_poster doubts it
<henninge> it says it just adds "SMTP auth for email logging"
<henninge> but I am looking at the diff now
<gary_poster> there ya go
<gary_poster> cool
<bigjools> bac, danilos: the build is now starting.
<bigjools> sorry for the delay
<henninge> gary_poster: nope, no other change. So 2.7.1 is just as good for us.
<gary_poster> cool henninge
<henninge> I am EOD now, I wil "follow instructions" tomorrow ...
<gary_poster> henninge, if you have done the work to verify that the newer version is not a problem for us, upgrading would not be a bad idea
<henninge> gary_poster: thanks for your help
<gary_poster> then next person does not have to do it
<henninge> I am fine with that, too.
<gary_poster> your very welcome henninge
<gary_poster> "you're," even
<henninge> ;)
<henninge> gary_poster: zconfig is already on LP
<gary_poster> cool
<bigjools> gary_poster: about that bug
<gary_poster> bigjools, hey!
<bigjools> gary_poster: it's indicative of data corruption
<gary_poster> (I was explicitly trying to not bother you today, and thus emailing)
<gary_poster> ah ok
<bigjools> gary_poster: it's possible for something to not have a BQ when it's finished building, but it must have one if it's NEEDSBUILD or BUILDING
<bigjools> gary_poster: well I am inbetween interruptions :)
<gary_poster> heh
<bigjools> gary_poster: so I would not paper over the crack and work out why the data is corrupted
<gary_poster> bigjools, ack.  I'll report as such on the bug, thanks.  If I look at https://launchpad.net/~chromium-daily/+archive/dev/+builds?build_state=building then it *looks* like those are all older than the current ones (14.0...)
<gary_poster> ah ha yes
<gary_poster> those are the bad ones
<gary_poster> ok thanks!
<bigjools> gary_poster: it's not easy to work this out, unfortunately
<bigjools> gary_poster: I have some SQL that fixes these if you want it
<gary_poster> because we don't know how those three would have gotten in that state, right?
<bigjools> right
<gary_poster> bigjools, it
<bigjools> it's a bug somewhere else in the code - or it could be someone poking SQL
<bigjools> I've not seen it happen for a long time
<gary_poster> 's what you want, I'd say.  I'm happy to run the SQL
<gary_poster> (arrange it to be run :-) )
<bigjools> ok
<gary_poster> we can put it in the bug report
<bigjools> gary_poster: http://pastebin.ubuntu.com/653198/
<gary_poster> and then not close the bug until we figure out something to do to actually provide some diagnostics as to how this might happen
<bigjools> replace the XXX with the build ID
<gary_poster> bigjools, do you have any easy/soyuz-novice thoughts on what we could do to add diagnostics so we can figure out how this would happen in the future?
<gary_poster> (or simply notes I can put in the bug)
<bigjools> gary_poster: I would trawl the build manager logs first
<bigjools> remember my first rule of soyuz debugging? :)
<gary_poster> heh
<gary_poster> ok
<bigjools> having said that, I don't know how many days of logs we keep for the manager
<bigjools> it's quite verbose, so they get rotated often
<gary_poster> ack
<jml> just running some tests, got this: http://paste.ubuntu.com/653218/
<jml> might be new since oneiric upgrade
<mtaylor> hey lovely people!
<mtaylor> https://help.launchpad.net/Bugs/ImportFormat says that I should file a question to get bugs imported - but there is no file attachment on questions that I can see...
<mtaylor> how should I attach the file?
<jml> I don't know.
<mtaylor> yay!
<sinzui> mtaylor, you point to a URL where the the xml is
<mtaylor> sinzui: ah.
<sinzui> mtaylor, someone will run cleanup script over the xml and test it on staging for you to review. there may be a report of of off ignored items that will require manual changes to the xml
<mtaylor> sinzui: excellent. what happens with user ids that aren't valid launchpad user ids?
<sinzui> I think profiles are created that users can claim
<mtaylor> ok. but if I reference valid launchpad accounts all works as expected, yeah?
<sinzui> yep
<mtaylor> cool
<mtaylor> I think I shall endeavor to make a username mapping before I actually give you a file then
<jml> ok
<jml> I'm trying to set up a lucid chroot to work around that rabbit timeout problem
<jml> so I can submit patches to Launchpad
<jml> but I'm told that, " launchpad-dependencies: Depends: python2.6 (>= 2.6.7-2ubuntu1) but 2.6.5-1ubuntu6 is to be installed or python-profiler but it is not installable
<jml> "
<jml> so, how do I get the correct version of Python?
<jml> any help would be much appreciated.
<maxb> jml: Enable multiverse
<jml> maxb: really? wow. ok. thanks.
<jml> maxb: why would a Python be in multiverse?
<maxb> python-profiler's licence is a bit odd, IIRC
<jml> ahh.
<jml> maxb: great. it's installing. thanks.
<jml> now rabbitmq won't install
<jml> gotta kill the outside one
<sinzui> jml, python-profiler is not required on oneiric
<jml> sinzui: I can't run the tests on oneiric, http://paste.ubuntu.com/653218/
<jml> sinzui: I've spent the last hour setting up a lucid chroot instead.
 * sinzui looks at rabbit
<sinzui> jml, I get the same error
<jml> sinzui: it's nice to feel like part of a crowd
<jml> now I get this in my chroot
<jml> OperationalError: could not connect to server: No such file or directory
<jml> 	Is the server running locally and accepting
<jml> 	connections on Unix domain socket "/var/run/postgresql/.s.PGSQL.5432"?
<jml> http://paste.ubuntu.com/653266/
<jml> maybe that's ... I'll run database-setup & check ports.
<sinzui> I would remove the rabbit layer to proceed. Does your test really need it? I avoid the page test layer since it starts lots of services that are rarely needed by an individual test
<sinzui> for example, the google layer is used by 2 tests in our suite
<jml> sinzui: hmm.
<jml> sinzui: it's PageTestLayer. I'm writing a webservice test.
<sinzui> :(
<sinzui> I hate that upper level
<jml> sinzui: yeah, me too. I don't really have a clear vision of an alternative, though.
<jml> at least, not for this.
<sinzui> I am trying a hack to the timeout
<sinzui> jml, I get: /usr/lib/rabbitmq/bin/rabbitmq-server: 63: cannot create /var/log/rabbitmq/rabbit@solstice.log.1: Permission denied
<jml> sinzui: you're running oneiric too?
<sinzui> yes
<sinzui> I think I hacked the actual run command baddly
<LPCIBot> Project db-devel build #759: STILL FAILING in 5 hr 42 min: https://lpci.wedontsleep.org/job/db-devel/759/
<sinzui> jml, a pdb in _spawn implies rabbitmq did start. I saw four beam.smp process for it
<sinzui> This might be a case were we are not reading $HOME/.erlang.cookie
<sinzui> jml, looks like the cookie format changed. The server is up
<jml> sinzui: ahh. cool.
<bac> deryck: bug 788685 is finally marked qa-ok
<_mup_> Bug #788685: Enable translating selected Ubuntu universe packages in Launchpad <escalated> <oem-services> <qa-ok> <Launchpad itself:Fix Committed by bac> <OEM Priority Project:Invalid> <Ubuntu Translations:Triaged> <pkgbinarymangler (Ubuntu):Fix Released by pitti> < https://launchpad.net/bugs/788685 >
<deryck> bac: excellent!
<bac> that frees up many branches as deployable
<deryck> bac: ok, great.  I'll see about doing a nodowntime deploy before my day is out.  calls coming up here shortly now.
<bac> the conch goes to wallyworld
* abentley changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: abentley | Critical bugs: 240 - 0:[#######=]:256
<sinzui> bac, wallyworld__'s bugs are now qa-ok
<bac> sinzui: great
<sinzui> jml, I really do not know how to fix rabbitmq. I see it running, but our fixture is not connecting to the nodes it started :(
<jml> sinzui: :\
<jml> sinzui: for the time being, I'm ok with running in lucid.
<sinzui> okay
<jml> ungh
<jml> does whatever linter LP is using now not understand __all__ for re-exports?
<jml> just use pyflakes. life will be better.
<lifeless> micahg: please use the literal OOPS code, not the urls (there is a bug in ooptools)
<lifeless> sinzui: you think we should just <elided> ? (bug 736012)
<_mup_> Bug #736012: Person:+mailinglist-moderate timeout <mailing-lists> <timeout> <Launchpad itself:Triaged> < https://launchpad.net/bugs/736012 >
<sinzui> lifeless, 1. we could write a script to remove the spam messages on the because we believe that this timeout was because of historic data
<jml> is there a way to do check_permission but to pass in the user?
<sinzui> and/or two change the batch size to a smaller number because we believe some list owners are negligent
<sinzui> jml, no, but that would be a nice to have
<bigjools> bac: https://dogfood.launchpad.net/ubuntu/natty/+source/epiphany-browser/2.30.6-1ubuntu6
<bac> bigjools: and nothing shows up here: https://translations.dogfood.launchpad.net/ubuntu/natty/+source/epiphany-browser/+imports
<bigjools> bac: does it show up there for oneiric?
<bac> bigjools: https://translations.dogfood.launchpad.net/ubuntu/oneiric/+source/epiphany-browser/+imports  -- indeedy
<bigjools> bac: qa-ok
<bac> yep
<bac> bigjools: thanks for your dogfoodness
<bigjools> my pleasure
<bac> (my mail client insisted on changing that to 'dogwood' every time i typed it.)
<lifeless> sinzui: my question was because your comment on the bug was incomplete :)
<sinzui> lifeless, my thoughts are still incomplete.
<lifeless> sinzui: fair enough
<benji> I'm trying to do a bug import (for the first time) and don't see how to do the actual import to staging (reading https://dev.launchpad.net/PolicyAndProcess/BugImportHowto).  Anyone have a hint?
<lifeless> have a losa run the script
<lifeless> bigjools: hi
<lifeless> https://bugs.launchpad.net/launchpad/+bug/816870 quick q for you
 * bigjools is not really here
<_mup_> Bug #816870: Distribution:+search (package search) timeouts <Launchpad itself:Triaged> < https://launchpad.net/bugs/816870 >
<lifeless> why is spr in that query ?
 * bigjools looks
<lifeless> oh, because it wants a distribution spc
<deryck> lifeless: hey.  Trying to get a nodowntime request going.  I udpate this:
<deryck> https://wiki.canonical.com/InformationInfrastructure/OSA/LaunchpadProductionStatus
<bigjools> in a not-kind-of-here-way
<lifeless> bigjools: thanks!
<deryck> lifeless: is this right?  And what is the date column for?  Date requested or deployed?
<lifeless> deryck: see under 'requested stable deploments'
<deryck> lifeless: I don't follow.  Should what I added go under "requested" section?
<deryck> or are you directing me at some info there?
<bigjools> lifeless: it's making sure that it can match a binary name that's for a source in the distro
<LPCIBot> Project devel build #924: STILL FAILING in 5 hr 47 min: https://lpci.wedontsleep.org/job/devel/924/
<lifeless> bigjools: thanks
<lifeless> bigjools: its a little inefficient :P
<bigjools> lifeless: welcome to the soyuz data model
<lifeless> deryck: the 'requested stable deployments' section has no date in it.
<lifeless> deryck: so I think you were looking at the wrong part of the page
<bigjools> efficiency was an after thought
<deryck> lifeless: ah, gotchas now.  thanks.
<lifeless> deryck: you have a request to make - you should follow the in-page docs at the 'requested stable deployments' section.
<deryck> lifeless: think I got it now.  Do I ping a losa, or just adding to the wiki page is enough?
<thedac> deryck: nodowntime deploy?
<deryck> thedac: yes, indeed.
<thedac> deryck: If the wiki is updated I will get rolling on it
<deryck> thedac: awesome, thanks!
<lifeless> deryck: a ping is usual
<deryck> lifeless: gotcha.  Thanks for the hand holding. :)
<lifeless> deryck: since you want to be around for the end, in case it goes boom
<lifeless> deryck: and that means timeliness matters
<deryck> ah, ok.
<deryck> maybe I shouldn't have started one when I'm about to jump on a call then ;)
<thedac> deryck: Do we need to hold off?
<lifeless> deryck: it usually doesn't go boom
<lifeless> thedac: no, its fine
<thedac> ok
<thedac> deryck: confirm revno: 13478
<deryck> thedac: was just kidding mostly
<lifeless> deryck: doing at your EOD would be ill advised though :)
<deryck> gotcha :)
<lifeless> thedac: that looks wrong
<deryck> thedac: no, it's r13535
<lifeless> we're running 13503 atm
<thedac> huh, is this out of date then: https://wiki.canonical.com/InformationInfrastructure/OSA/LaunchpadProductionStatus
<lifeless> thedac: see the 'roll out to' column
<lifeless> thedac: and the requested-by column
<lifeless> thedac: it may be out of date, but derycks request is sane ;)
<benji> my CHR time's up and I didn't even get to questions
 * bac CHRs
<benji> the bug import?  That was indeed a question, but I got to it from the "bug import questions" link, I expect that there are other open questions
<benji> pff, wrong hchan
<jml> allenap: finally put up a fix for your *actual* testtools bug, as opposed to the one I thought you meant to file :)
<micahg> lifeless: sorry, will do from now on
<abentley> jml: I'm liking ExpectedException in general, but I'm thinking it could be generalized to take a Matcher as a parameter.  But assertThat only works in test cases, so maybe it should be TestCase.exceptionMatches ?
<jml> abentley: that's what the patch does
<abentley> jml: was not aware of a patch.
<jml> abentley: https://code.launchpad.net/~jml/testtools/expected-exception-791889/+merge/69546
<jml> it just raises AssertionError, rather than calling TestCase.fail
<jml> which is almost always the same thing
<henninge> anybody about to request a deploy?
<abentley> jml: cool.
<henninge> if not, I will now
<jml> hmm. I should update docs & NEWS
<henninge> ah, reloading LPS page helps ...
<henninge> deryck: thanks for taking care of the deployment.
<henninge> well, the deplyoment *request* ;-)
<deryck> henninge: no worries.  It kind of all came together.
<deryck> thedac: hi again.  Did the deploy go okay?  Or still in progress?
<thedac> deryck: still in progress
<deryck> thedac: ok.  thanks.
<thedac> I think we are still just pushing code around at this point in the deploy script
<wgrant> jml: python-profiler was the one part of the standard library that wasn't in the normal Python packages, because it had a non-free license. Disney bought the company and they only convinced them to relicense it this year. In oneiric and above it's in the core python2.* package.
<deryck> I started a deploy with thedac couple hours ago but am now past EOD and need to go.  bad timing on my part. :)
<deryck> Can I hand off to someone, to help thedac if he needs it?
<wgrant> Sure.
<wgrant> thedac: Where are we at?
<deryck> wgrant: thanks so much!
<wgrant> (thanks for organising the deploy, deryck!)
<thedac> wgrant: using deployment-manager it is cranking along
<wgrant> lpnet is updated.
<wgrant> So we are nearly there.
<deryck> wgrant, np!  I'll still claim it as my first deploy, even though I'm handing off. ;)
<thedac> Just deployed  to germanium-scripts. It is hard for me to tell exactly how much more we have to go. But I think we are close
<wgrant> The webapp is updated, so it's hardly handed off :)
<deryck> :)
<deryck> ok, until tomorrow then, everyone.....
<thedac> wgrant: fyi, at this point the script is just doing cleanup. We should be fully deployed. I will update wiki and email when the script completes
<wgrant> Thanks.
<wgrant> thedac: Was loganberry's scripts target being deployed at 21:45?
<wgrant> thedac: We have an odd piece of cronspam which suggests that either something rather terrible has happened, or the symlink was switched right as a cronjob started at 21:45.
<thedac> wgrant: that seems likely. Let me check my scrollback to see if I have a time stamp
<wgrant> It seems to be happy now, so I assume that was it, but best to check if we can.
<thedac> wgrant that looks like that was it. The time stamp from a few actions prior was at 21:44. I don't have an exact one for loganberry but that has to be it
<wgrant> Thanks.
<lifeless> win!
<lifeless> so thats what, a hundred or so deploys before hitting the race we suspect might exist
<wgrant> Yup.
<wgrant> I think that's pretty good.
<wgrant> lifeless: So, is there a good reason that mailman is still in our tree?
<lifeless> wgrant: yes, noone has done the work to make a SOA project out of it
<wgrant> That's not a good reason.
<lifeless> wgrant: .e.g a network test fake of the xmlrpc service LP offers mailman, and vice versa
<lifeless> wgrant: theres no intent to keep it in-tree.
<lifeless> wgrant: if that helps.
<wgrant> Good, good.
<wgrant> It seems like pretty low-hanging fruit.
<wgrant> And annoying fruit.
<lifeless> I need to blog about this notification change.
<lifeless> grah.
<wgrant> 'Copied from debian sid in Primary Archive for Debian GNU/Linux' woo
<lifeless> is that in prod ?
<wgrant> Yes.
<wgrant> Of course, an extra 1000 packages also got synced...
<wgrant> But, er, baby steps.
<lifeless> by mistake ?
<wgrant> By bug.
<wgrant> Multiple bugs.
<wgrant> Only 10 of them actually had Ubuntu changes.
<wgrant> So 883 of the erroneous syncs hopefully didn't break too much.
<wgrant> jelmer: Around?
<wgrant> jelmer: https://pastebin.canonical.com/50371/ <- I'm reverting your publisher-use-debian2
<sinzui> StevenK, mumble?
<wgrant> gaaaaaaaaa just missed buildbot by 30 seconds
<wgrant> 20 seconds, in fact.
<LPCIBot> Project db-devel build #760: STILL FAILING in 5 hr 44 min: https://lpci.wedontsleep.org/job/db-devel/760/
#launchpad-dev 2011-07-28
<jelmer> wgrant: ok :(
<wgrant> Yes :(
<lifeless> wgrant: community-contributions; can you sudo to lpqateam ?
<lifeless> if so, you could run it from that service accounts crontab
<wgrant> lifeless: I can't.
<wgrant> I don't think many can.
<lifeless> its available on request, if you want it
<jelmer> wgrant: I think we should just drop that branch; it was a nice refactoring, but it's taking more time than it's worth
<wgrant> jelmer: Yeah ;/
<wgrant> :/
<LPCIBot> Project devel build #925: STILL FAILING in 5 hr 44 min: https://lpci.wedontsleep.org/job/devel/925/
<LPCIBot> Project db-devel build #761: STILL FAILING in 5 hr 38 min: https://lpci.wedontsleep.org/job/db-devel/761/
<lifeless> stub: \o/
<stub> *\o/*       *_o_*      *\o/*
<lifeless> :)
<lifeless> want to see if the internets are working and do a catchup call ?
<stub> lifeless: sure. gimme a tick for the caffine to kick in.
<lifeless> wgrant: do you know -why- or ui gets broken by having series tasks w/no non-series 'parent' task ?
<lifeless> wgrant: like, was it oversight? deliberate? ...?
<wgrant> lifeless: It seems to be completely designed around always having a non-series task. I don't see how it would behave without one, without a complete redesign.
<wgrant> As series tasks have only the series name.
<wgrant> Not the Distribution/Product/DSP
<lifeless> the original wasn't like this
<wgrant> As in most cases they would just be massively duplicating information.
<wgrant> Mmmm, it hasn't been changed much since 2006's release management rework.
<wgrant> Which introduced conjoinment.
<lifeless> I'm old skool
<wgrant> I didn't really use series tasks before then.
<rvba> Hi StevenK.
<wgrant> lifeless: So, until we redesign the UI and or model...
<lifeless> I may be wrong, but it seems shallow to me.
<wgrant> It requires a major redesign of a piece of UI which was thoroughly designed in 2006.
<lifeless> set a variable when looping over tasks for 'current pillar' [a lie, tell me the right name] and if it doesn't match and the next task is for a subordinate, emit a pretty line describing it.
<rvba> StevenK: I'd like to have your opinion about a change I'm making to lib/lp/soyuz/model/distroseriesdifferencejob.py. I don't need a full review from you but I'd be glad if you could take a look at lines [267 - 286]. In particular, I'm not sure I understand the old code (that I'm removing) properly because it looks to me that DSDJs with parent_series=the parent derived_series=its grandchildren could be created ... https://code.launchpad.net/~rv
<rvba> b/launchpad/dsd-creation-multiple-parents-bug-815775/+merge/69407
<rvba> StevenK: https://code.launchpad.net/~rvb/launchpad/dsd-creation-multiple-parents-bug-815775/+merge/69407
<lifeless> wgrant: I'm sure we could do a major redesign, I don't see that one is needed.
<wgrant> lifeless: How do you propose to fix it?
<lifeless> up 2 lines.
<wgrant> I'm not sure it's worth it, given it will hopefully all be redesigned soon.
<wgrant> Oh.
<wgrant> Could do, I guess.
<lifeless> thats what the bug boils down to isn't it? 'series task on its lonesome is missing context description'
<StevenK> rvba: You only need one loop: for series in derived_series.getParentSeries() + derived_series.getDerivedSeries():
<lifeless> other than conjoined tasks we don't have special business rules or cross-task conditional logic.
<StevenK> rvba: I thought may_require_job() did the child stuff?
<StevenK> -> out
<rvba> StevenK: Thanks.
<wgrant> lifeless: Yeah, true. It's just a bit ew.
<wgrant> But this possibly makes it a bit less ew.
<lifeless> I think this is a shallow oversight in that earlier work, rather than a big issue.
<wgrant> Possibly.
<wgrant> Not sure how to make it clear that it's not a real task.
<wgrant> But I guess we'll find out.
<lifeless> well it is a real task isn't it ?
<lifeless> the only real special logic needs to be conjoined - when the user sees one task, we need to move two.
<wgrant> Do you propose to show the project/distro/DSP in the series task, or as a ghost parent?
<wgrant> I assumed the latter.
<lifeless> oh, I see what you mean
<lifeless> I was proposing a line, like a task line, but with no status importance assignee milestone fields
<wgrant> That's what I thought.
<wgrant> Just need to make it clear it's not a real task.
<lifeless> sure
<wgrant> Conjoinment already confuses people with fake tasks.
<lifeless> huwshimi may have some ideas. We could do what conjoinment does - no worse.
<LPCIBot> Project devel build #926: STILL FAILING in 5 hr 45 min: https://lpci.wedontsleep.org/job/devel/926/
<lifeless> stub: caffeinated ?
<stub> lifeless: for now :)
<adeuring> good morning
<wgrant> lifeless: It turns out it's rather easy to do that in the existing code. http://people.canonical.com/~wgrant/launchpad/ghost-task.png, but it, er, needs some UI work.
<wgrant> Not sure what to say.
<wgrant> Since I don't think we're allowed to talk about tasks.
<lifeless> wgrant: I think it would be fine with no blah blah blah
<wgrant> Possibly.
<wgrant> I guess it's going to be fairly rare.
<jtv> Reviewer in the house?  https://code.launchpad.net/~jtv/launchpad/bug-816833/+merge/69597
<lifeless> wgrant: yeah
<wgrant> But it would be nice if it were clear that it wasn't really a task and you could get status/importance tracking by clicking a link below.
<mrevell> Hello
<jtv> hi mrevell
<mrevell> Hallo jtv
<jml> could someone please review https://code.launchpad.net/~jml/launchpad/create-private-ppa-814567/+merge/69539
<james_w> jml, is that going to stay a commercial admin-only operation?
<jml> james_w: bigjools often expresses a desire to change it.
<jml> james_w: although I've not actually heard an outline of an alternative.
<jml> james_w: why do you ask?
<bigjools> We need a PPA admin celeb
<james_w> if this is for appdevelopers then we're not going to make them commercial admins
<bigjools> there's a software center celeb for that
<bigjools> it has permission to make private PPAs
<james_w> but that doesn't necessarily fit the desired workflow though
<bigjools> right
<bigjools> the workflows are ill defined
<jml> this is a specific instance of a more general entitlement problem
<jml> permission to create private X, and the means by which those permissions are acquired
<bigjools> jml: in your branch did you consider refactoring the check for commercial admin, somehow?
<bigjools> I'm not sure how I'd do it tbh :(
<jml> bigjools: the only thing that came to mind was writing a version of check_permission that took user as a parameter, rather than getting it from the session.
<bigjools> jml: I'm just wary of duplicating the check, that's all.  Been there, picked up the pieces :(
<jml> bigjools: yeah. I can put a note in the new code & the zcml if you'd like, pointing one to the other.
<bigjools> jml: fair enough, thanks
<jml> bigjools: clear enough? http://pastebin.ubuntu.com/653641/
<bigjools> jml: perfect
<jml> bigjools: I've pushed that change up. Could you please land the branch for me?
 * jml no longer has permissions to do so.
<bigjools> jml: !
<bigjools> wth
<gmb> allenap: Are you OCRing today?
<allenap> gmb: Yes.
* allenap changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: allenap | Critical bugs: 240 - 0:[#######=]:256
<bigjools> jml: set a commit msg
<jml> bigjools: I left ~canonical-launchpad. lifeless & flacoste have yet to re-establish the broader launchpad committer policy.
<allenap> gmb: I've just done one of yours.
<bigjools> jml: I am sure we have other internal committers
<gmb> allenap: Ah, magico. Thanks.
<lifeless> bigjools: they are in ~launchpad
<lifeless> bigjools: jml jumped, he was not pushed :P
<jml> yeah.
<jml> wanted to be clean.
<allenap> gmb: Fwiw, I think that would have been fine as a self-review.
<bigjools> dirty :)
<jml> & also to not get sucked back into LP stuff too much, thus doing  a disservice to my new overlords
<gmb> allenap: Agreed. But I always forget I can do those until after I've asked.
<bigjools> jml: your branch is hitting ec2 now, thanks for your contribution :)
<jml> my pleasure :)
<henninge> anybody with buildout foo around?
<gmb> henninge: I hesitate to say this, but I have a little. What's up.
<henninge> gmb: I am trying to get my own version of ZConfig build.
<lifeless> hackinglazrlibraries ?
<rvba> allenap: may I add that to you queue? https://code.launchpad.net/~rvb/launchpad/dsd-creation-multiple-parents-bug-815775/+merge/69407
<rvba> *your
<henninge> gmb: I am following the instructions in doc/buildout.txt
<allenap> rvba: Certainly :)
<rvba> allenap: Thanks ;)
<henninge> gmb: so, I have the modified source tree and did "setup.py egg_info sdist" which gave me a tarball in dist
<henninge> gmb: I put that tarball into the download-cache, updated version.cfg and ran bin/buildout.
<henninge> gmb: that gives me this error:
<henninge> Getting distribution for 'ZConfig==2.9.1dev-20110728'.
<henninge> error: NEWS.txt: No such file or directory
<henninge> An error occurred when trying to install ZConfig 2.9.1dev-20110728. Look above this message for any errors that were output by easy_install.
<gmb> henninge: Just for clarity, did you put the tarball in download-cache or download-cache/dist?
<wgrant> henninge: How did you get the version suffix?
<henninge> gmb: so two possible questions: Why does NEWS.txt not get included in the tarball?
<wgrant> henninge: You didn't rename the tarball manually?
<henninge> gmb: in dist
<henninge> wgrant: no, sorry, I used "-d" on setup.py
<wgrant> Ah, good.
<wgrant> -d, or -b?
 * henninge checks
<henninge> wgrant: the one that does it automatically
<henninge> -d
<wgrant> Right.
<gmb> henninge: Lickily for you, the Oracle of Melbourne has arrived.
<gmb> Luckily, even.
<henninge> second possible question: Why does buildout expect NEWS.txt to be in there?
<henninge> gmb: ;-)
<wgrant> henninge: It must be in the manifest of the tarball.
<wgrant> jtv, allenap, danilos: Need any help with QA? There's a germanium cowboy that I'd like to squash ASAP.
<wgrant> henninge: Is this tarball in lp-sourcedeps yet?
<bigjools> Lickily sounded eerily appropriate
<jtv> I need to run publish-ftpmaster.
<allenap> wgrant: I'll take a look at mine.
<henninge> wgrant: yes, it is
<wgrant> Let me see.
<bigjools> jtv!  Let's do it
<wgrant> jtv, allenap: Thanks.
<wgrant> jtv: I guess you should also test cron.publish-ftpmaster, or maybe just pretend that everything will be OK.
<wgrant> Since it's not exactly a controversial change.
<jtv> Yes, good point â I can run the existing cron.publish-ftpmaster.
<jtv> That's assuming I can.
<wgrant> Worth a try.
<allenap> wgrant: Done.
<wgrant> But not worth a vast quantity of effort if it proves non-trivial.
<jtv> wgrant: I'd still like to be able to run the new script separately later today though, and get verifiable results.
<wgrant> True.
<danilos> wgrant, I need help with CHR, will get to QA asap :)
<wgrant> henninge: I'm not sure ZConfig is set up well enough that you can use setup.py to build a tarball.
<wgrant> henninge: I suspect you may need to hack the version and tar it up manually.
<wgrant> Since setup.py references NEWS.txt in the cwd without asking for it to be installed.
<wgrant> Hmm, but 2.7.1 still has egg_info...
<wgrant> You may need to invoke gary or similar, I fear.
<wgrant> ZConfig is a little strange.
<henninge> wgrant: gary sent me down this path ... ;-)
<wgrant> I wonder if adding NEWS.txt to MANIFEST.in would work.
<henninge> let me see what I get when I build from the 2.7.1 source
<allenap> wgrant: Actually, I need to spend some more time looking at r13538.
<danilos> wgrant, my rev is qa-ok
<wgrant> danilos: Thanks.
<jtv> wgrant: bigjools just told me we can test the existing publish-ftpmaster script now.
<lifeless> henninge: you've tracked the rotation segfault to zconfig ?
<henninge> lifeless: yes, it simply closes and opens the files with no logging whatsoever.
<henninge> s/logging/locking/
<lifeless> \o/
<henninge> gary suggested a simple fix to simply not close the file. GC will do that ones all processes are done with it.
<henninge> s/ones/once/
<wgrant> I hope upstream won't accept that. It's not very portable.
<henninge> why not?
<henninge> portable accross python version?
<wgrant> CPython's reference counting is an implementation detail. Relying on GC for managing external resources like that isn't a great idea.
<wgrant> eg. will behave differently in Jython and PyPy.
<henninge> I see what you mean
<lifeless> also on windows
<lifeless> on windows you can't replace open files under all conditions
<lifeless> (not that I care about windows)
<lifeless> henninge: do you mean 'threads' or 'processes' ?
<henninge> lifeless: it's threads AFAIUI
<henninge> it should not be too hard to implement proper locking
<henninge> the logging module does that, too.
<henninge> There is also WatchedFileHandler but it does a "stat" call on the log file each time it logs a line which seems a bad idea, too.
<lifeless> henninge: if I may suggest, use a single thread for writes; hand off things to log to that thread using a threading.queue, and that one thread can receive a message in the queue to do rotations.
<lifeless> henninge: or something-like-that
<lifeless> e.g. rotate() would look something like: insert the 'rotate now message', with a callback to notify on completion; bind that to a temporary threading event, and you're done, more or less
<henninge> lifeless: I had been thinking along those lines at first, too, but hoped for a simpler solution ...
<henninge> oh, that is a nice idea, too.
<jtv> bigjools: can I just run cron.publish-ftpmaster on dogfood then?
<jtv> And allenapâ¦ thanks!
<bigjools> jtv: doit
<jtv> And given that you're speaking French todayâ¦
<jtv> "must"
<bigjools> :)
<bigjools> jtv: allez
<danilos> henninge, hi, will you have a few minutes to discuss translations sharing today? dpm has filed a bunch of questions about it not working for different projects, and he suggested that you've already done some investigation
<henninge> danilos: I have not, no.
<henninge> danilos: I just told him that it would need some and that it looks like there are more bugs hidden there.#
<danilos> henninge, right, fair enough, I'll see if I can get some dedicated time to look into it tomorrow-ish then
<danilos> mrevell, hi, do you think you'd have time to help me with announcing the import queue management change and perhaps even start on the documentation?
<mrevell> danilos, Yes, certainly. Tomorrow would be best for me but if it needs to be today then I can rearrange a couple of things.
<danilos> mrevell, it'd be nice for us to at least come up with an announcement today since it's probably going to go out in a nodowntime deployment soon
<mrevell> danilos, Okay, no problem.
<mrevell> danilos, I can talk now.
<danilos> mrevell, excellent
 * danilos shuts the music down and switches to headset
<danilos> mrevell, how about skype? :)
<henninge> danilos: thanks
 * mrevell engages the Skypotron
<jtv> bigjools, wgrant: cron.publish-ftpmaster isn't really supposed to complete within the minute, is it?
<bigjools> depends how much work there is.  got a log?
<bigjools> but generally no :)
<jtv> rsync: change_dir "/srv/launchpad.net/ppa/ubuntu-partner/dists" failed: No such file or directory (2)
<bigjools> heh
<jtv> log: http://paste.ubuntu.com/653691/
<bigjools> jtv: just make the dir
<bigjools> seems like the PPA repo purge I did blew away DF's partner archive
<jtv> /srv/launchpad.net/ppa/ubuntu-partner is emptyâ¦ mkdir /srv/launchpad.net/ppa/ubuntu-partner/dists?
<bigjools> yes
<wgrant> Erm, /srv/launchpad.net/ppa?
<wgrant> Oh.
<wgrant> You need to hack the script.
<bigjools> publisher doesn't need it but that script does
<wgrant> I remember this from last time.
<wgrant> It checks if the config is right or something.
<wgrant> And if not uses /srv/launchpad.net/ppa, because why not.
<bigjools> that directory is served on Apache, it's useful for testing
<wgrant> if [ "$LPCONFIG" = "$PRODUCTION_CONFIG" ]; then ARCHIVEROOT_PARTNER=/srv/launchpad.net/ubuntu-archive/ubuntu-partner
<wgrant> Ah.
<bigjools> the script is fine
<wgrant> Fine, perhaps. But still crazy :)
<bigjools> except rsync hates it when dists is missing
<jtv> So we'll give it one.
<bigjools> yes
<jtv> (haven't mentioned it but it's running)
<LPCIBot> Project db-devel build #762: STILL FAILING in 5 hr 48 min: https://lpci.wedontsleep.org/job/db-devel/762/
<cjohnston> mrevell: ping
<mrevell> Hi cjohnston
<cjohnston> Want to setup a time?
<mrevell> cjohnston, Yes please. How does 14.00 UTC sound?
<cjohnston> ~3 hours from now?
<cjohnston> I guess 2 hours 45 minutes from now
<mrevell> cjohnston, Yeah.
<cjohnston> Sounds good
<mrevell> Great, thanks cjohnston, speak to you later.
<bigjools> StevenK: the notification code you wrote always seems to use changed-by as the From: address which I think  is wrong and should be blamer, do you agree?
<StevenK> bigjools: Hmmmm. For which bit?
<bigjools> StevenK: it calls fetch_information which always returns from_addr that way
<bigjools> the syncs we did yesterday had "From:" as the Debian uploader
<StevenK> bigjools: Yes, but the notification to changed-by, or the announce?
<bigjools> StevenK: both
<StevenK> Hmmmmm.
<wgrant> This is difficult.
<StevenK> Agreed
<wgrant> For the announce list we really want the author of the change.
<wgrant> For an Ubuntu upload that's Changed-By.
<wgrant> For a sync, that's the requester... not necessarily the blamer, but perhaps we can just use the blamer there.
<wgrant> We can't just unconditionally use the blamer.
<bigjools> ok
<bigjools> the old code used katie as a sign it was a sync
<wgrant> Yes :/
<bigjools> feck sake
<bigjools> in fact wasn't that the chnaged-by?
<StevenK> If it's from PCJ, the requester is the blamer.
<bigjools> indeed
<wgrant> bigjools: katie was for autosyncs.
<wgrant> It's in Changed-By in that case.
<bigjools> right
<wgrant> For manual syncs, katie is not involved.
<bigjools> is sync-source mangling anything ?
<StevenK> For manual syncs, Changed-By was the requester
<StevenK> I *think*
<bigjools> exactly
<wgrant> bigjools: Yes.
<wgrant> sync-source puts the requester in Changed-By.
<bigjools> https://launchpad.net/debian/sid/+source/gedit-plugins/3.0.5-1
<bigjools> I think Gina has a bug, can you spot it? :)
<wgrant> Yes.
<wgrant> Also, it's using Debian URLs in emails.
<bigjools> yeah I know
<wgrant> I guess it's asking for the SPR's canonical_url, rather than the DSPR, perhaps.
<StevenK> It does use the SPR
<wgrant> bigjools: Oh, you're using changelog_entry in notify()?
<wgrant> That's not really suitable.
<bigjools> when you say "you're" ....
<wgrant> We need to parse changelogs.
<wgrant> "you're" == red
<bigjools> == StevenK :)
<StevenK> Not my code any more
<bigjools> ha
<bigjools> it's all our code but since you wrote it you don't get to escape
<wgrant> I waved StevenK's code through with the understanding that it was an initial refactoring to be tested and made suitable later :(
<wgrant> Since there was lots of refactoring :(
<StevenK> And so far, we've discovered that the one missue of it was due to PCJ suckage
<StevenK> bigjools: I don't get to fix it either
<bigjools> StevenK: dude, I am just asking for some info, not a fix
<StevenK> Which you have -- sync-source does Wierd Shit[tm]
<bigjools> so, the code as it is works fine, except for syncs where we want From: to be the blamer
<bigjools> yes?
<wgrant> bigjools: I believe so.
<bigjools> thanks
<bigjools> easy enough to fix
<wgrant> Let's hope.
<wgrant> It's faaaairly well-factored now.
<wgrant> And even somewhat tested.
<bigjools> so I can either pass in an override for from_addr or an is_sync bool.  I'm not sure the code itself can work out this.
<StevenK> Pass in the override
<StevenK> Simplest, easiest way
<StevenK> And won't effect the existing tests
<wgrant> It shouldn't know about syncs, so is_sync is out.
<wgrant> Possibly a from_blamer, or some similar flag like that, though...
<wgrant> Rather than having to calculate the address yourself.
<bigjools> well it should pass in a Person
<StevenK> From may not be a Person
<bigjools> it should be
<bigjools> so that the preferred email stuff is all dealt with inside the notify code
<StevenK> notify() doesn't take a from_addr
<bigjools> it will do in 5 minutes
<StevenK> Ew
<StevenK> notify() has too many arguments already :-)
<bigjools> shit happens
<wgrant> Perhaps a trust_changed_by=True argument, or something.
<wgrant> Otherwise you are going to have to inspect the SPR's people.
<bigjools> eh?
<bigjools> the from_person will default to None
<wgrant> It just feels fairly dirty for the callsite to have to get the people itself.
<bigjools> I disagree on that - it's just an override and used in special cases, like syncing.  We already know the person.
<bigjools> and this is feeling like bikeshedding
<wgrant> It is not quite bikeshedding, but close, true.
<bigjools> we already pass blamer, I don't see the problem with knowing the person who sends the email
<wgrant> Yeah, maybe.
<wgrant> I'm just worried that this could end up turning into a bigger mess than it was before :)
<wgrant> Still, at least it won't have katie celebrities this time.
<bigjools> yeah, when sync-source stops getting used we can throw that bit of code away
<bigjools> thank you for your input gentlemen
<wgrant> Killing celebrities is a favourite pasttime of mine.
<bigjools> hmm interesting, the email to blamer would end up being the config.uploader.default_sender_address)
<bigjools> from the*
<wgrant> That's how it tends to be, yes.
<bigjools> guess that's ok
<wgrant> 'Launchpad PPA' or 'Ubuntu Installer' have historically sent the non-announcement emails.
<bigjools> indeed
<wgrant> jtv: cron.publish-ftpmaster's not yet blown up?
<jtv> No, sorry
<jtv> 2011-04-18 10:59:36 INFO    Done with ubuntu natty.
<jtv> .0.1-1
<jtv> Getting ancestry for 3depict - 0.0.2-1
<jtv> Getting ancestry for gdc-4.4 - 1.063-4.4.5-2ubuntu1
<jtv> Getting ancestry for gdc-4.4 - 1.063-4.4.5-2
<jtv> Actually, maybe it's blown up and just not told anybody.
<jtv> It hasn't produced any new output for a while.
<henninge> gary_poster: Hi!
<wgrant> jtv: I would not put that past it.
<jtv> Hmm can't find the process.
<jtv> So that sounds dead.
<gary_poster> sinzui, I need to qa that Mailman's XMLRPC is still working after a bugfix I made.  AFAICT Mailman doesn't work at all on qastaging--when I try to interact at all with Mailman lists things blow up.  Am I missing something?  How would you recommend I qa this kind of thing?
<wgrant> But what is that?
<wgrant> germinate?
<gary_poster> hey henninge
<wgrant> LP doesn't print that sort of message.
<wgrant> gary_poster: qastaging doesn't have mailman.
<wgrant> gary_poster: But staging does, and got the rev a while ago.
<gary_poster> wgrant, I was afraid of that
<gary_poster> oh ok excellent
<gary_poster> thanks wgrant
<wgrant> gary_poster: And successfully resynced, and I created a new list and it was created after it resynced.
<wgrant> So it seems to be OK.
<gary_poster> wgrant, heh, you rock!
<gary_poster> thanks also for the email about the OOPS wgrant
<wgrant> Sorry I didn't chase that up earlier.
<gary_poster> np at all
<henninge> gary_poster: I am almost there but I have a problem with building the egg using the egg_info command.
<gary_poster> and, bigjools, thanks for your email.  I'll rerun that query after I get a few other things handled
<bigjools> yup
<jtv> Pretty much all the runtime seems to be 20 minutes of just this.
<gary_poster> henninge ok (egg_info just modifies egg data; sdist makes the sdist)
<henninge> gary_poster: it does not include the file NEWS.txt althogh that is mention in setup.cfg. Any idea why?
<henninge> gary_poster: ok, then it's sdist
<henninge> I did not realized those were two chained commads
<wgrant> It looks like you need to manually tar rather than sdist, but I hope gary has more insight :)
<gary_poster> henninge, yes, reasonably confident.  it is because setuptools has special magic for svn and not for any other rcs.  packages that rely on this don't have a MANIFEST.in and so break in this kind of situation. :-(
<gary_poster> so, options...
<gary_poster> 1) probably a manual tar would work, guessing at the right name for things...you might have to look for magic files.  Others might know about that; I've never done it.  That's because I do option 2:
<henninge> ;)
<gary_poster> 2) get the branch into svn.  I could commit it for you into Zope's svn if you'd like.  Then it would work for you.  I also sometimes do option 3:
<gary_poster> 3) Write your own MANIFEST.in. You could quite likely steal the one from one of our lazr packages and it might work fine.  MANIFEST.in is quite underdocumented though, I'm afraid.
<gary_poster> henninge, option 4: toss it to me to make it my problem :-P
<gary_poster> I like the other options better, but I'm willing to play with that option :-)
<henninge> gary_poster: option 2) sounds fine since I want to submit it upstream anyway.
<henninge> gary_poster: can you get it into svn for me? Here is the lp branch:
<gary_poster> henninge sure
<henninge> https://code.launchpad.net/~henninge/zconfig/bug-481512-reopen-logs
<henninge> gary_poster, wgrant: I implemented proper locking, btw, which was not that heard because the underlying logging.StreamHandler already has it, just gotta use it.
<wgrant> henninge: Ah, great. Better than relying on GC implementation details.
<gary_poster> cool
<jtv> wgrant: what was that you said about manually tarring rather than sdisting?  Complete Greek to me, and I was always terrible at Greek.
<jtv> (Not entirely undue to the fact that our books, in stark contrast to our Latin and German books and for reasons that never became entirely clear, consistently put the cases in order 1-4-2-3)
<henninge> gary_poster: so, I hear back from you once my branch is in svn or do you need anything else from me?
<gary_poster> henninge, I am doing it now.
<henninge> cool, thanks.
<henninge> ;-)
<wgrant> gary_poster: We had to roll germanium back 12ish hours ago. I reverted the problematic revision in r13542, so it'd be great if someone in your squad could arrange a deployment of at least that rev once we are QA'd.
<gary_poster> wgrant, ack, will do
<gary_poster> henninge: ``svn co svn://svn.zope.org/repos/main/ZConfig/branches/LP-481512-reopen-logs`` is good to go
<wgrant> gary_poster: Thanks.
<henninge> gary_poster: thanks
<henninge> gary_poster: that worked ;-)
<gary_poster> henninge, cool :-)
<gmb> abentley: Can you answer cjwatson's follow-up reply on this question? https://answers.launchpad.net/launchpad/+question/164657
<deryck> Morning, all.
<henninge> gary_poster: https://code.launchpad.net/~henninge/launchpad/bug-481512-reopen-logs/+merge/69643
<henninge> Hi deryk!
<henninge> Hi deryck! ;)
<gary_poster> henninge, approved
<henninge> gary_poster: thanks
* jcsackett changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: allenap, jcsackett | Critical bugs: 240 - 0:[#######=]:256
<henninge> gary_poster: ec2 test run makes no sense on a branch like that.
<gary_poster> bigjools, on #launchpad GTRsdk is trying to upload thunderbird 6, and the orig source upload hangs at the last kilobyte.  Should he just be patient or is this indicative of something wrong, or do you have any other idea?  Also, is there anyone else I can bother about this right now?  I know you are busy.
<gary_poster> henninge, eh, I think it does
<gary_poster> adding a dependency is risky
<henninge> um, ....
<henninge> too late
<gary_poster> henninge ok
<henninge> but right
<henninge> I was only thinking in term of changing that file
<gary_poster> yeah
<henninge> I'll start a test run in parallel
<bigjools> gary_poster: it's a bug in a hardware router we think, this problem is happening in two completely different FTP server implementations we've done now.  He can work around it by using sftp instead.
<gary_poster> many thanks bigjools
<bigjools> gary_poster: I'll explain to him
<gary_poster> thanks again :-)
<abentley> allenap, jcsackett: could you please review https://code.launchpad.net/~abentley/launchpad/json-serialization/+merge/69519 https://code.launchpad.net/~abentley/launchpad/reload-cache/+merge/69527 and https://code.launchpad.net/~abentley/launchpad/translations-sharing/+merge/69531 ?  They are a series.
<allenap> abentley: Sure, I was just looking at the first one...
<abentley> allenap: thanks!
<deryck> http://lpqateam.canonical.com/
<abentley> bac: I've got my changes up for review now.  When the last in the series, "translations-sharing", is approved, I plan to land it.  This will effectively merge all prerequisites, including getnewcache.
<bac> abentley: ok, great!
<abentley> bac: So you don't need to do anything, but if you have any outstanding getnewcache revisions, please push 'em.
<bac> abentley: latest have been pushed
<bac> abentley: please let me know when it lands, in case i miss it
<abentley> bac: Sure thing.
<jcsackett> allenap: can i add https://code.launchpad.net/~jcsackett/launchpad/decouple-privacy-notifications/+merge/69174 and https://code.launchpad.net/~jcsackett/launchpad/extend-privacy-notification-to-comments/+merge/69655 to your queue? i can take over anything else that pops up in the review queue today since i figure that would take you to EOD.
<jelmer> jcsackett: in that case, can I perhaps add something to your queue?
<jelmer> https://code.launchpad.net/~jelmer/launchpad/upgrade-stderr/+merge/69225
<abentley> jelmer: Heya.  Noticed you touched a baz-import bug.  Is it relevant to you?
<allenap> jcsackett: I'm doing one of abentley's branches right now, but I'll probably have time for one of those at least before closing if you take abentley's other branch.
<jelmer> abentley: no, I was mainly just looking at all the New bugs on http://bugs.launchpad.net/bazaar/+bugs
<abentley> jelmer: cool, cause I have no plans to work on baz-import ever again :-)
<jelmer> abentley: heh, ok :)
<jcsackett> allenap: i can certainly take abentley's remaining branch.
<abentley> jelmer: funny thing is, baz is no longer in Ubuntu, but tla is.
<jelmer> abentley: I guess tla is still used (and maintained) by somebody in Debian
<jelmer> abentley: ah, actually. tla is orphaned, so there is a good chance it will disappear soon too
<bigjools> jcsackett: helleau.  Can I add this to your queue please: https://code.launchpad.net/~julian-edwards/launchpad/sync-email-from-addr-bug-817102/+merge/69659
<jcsackett> bigjools: absolutely, i'll take a look when i finish abentley's branch.
<bigjools> jcsackett: cheers, it's not big (~100)
<jcsackett> bigjools: cool.
<abentley> jcsackett: thanks.
<jcsackett> abentley: thank you for the work.
<jcsackett> bigjools: looks like jtv is reviewing your branch?
<jtv> jcsackett: looks like that, yes
<bigjools> jcsackett: help me.  help me now.
<abentley> jcsackett: np.  I wrote the original code, so I was pretty comfortable cleaning it up.
<jcsackett> bigjools: i think you're in fine hands. :-)
<bigjools> jcsackett: Didn't know he was going to pick it up, sorry.
<jcsackett> bigjools: no worries at all. it's a pleasant surprise to turn to a task and find someone else doing it. :-)
<bigjools> jcsackett: if only life was more like that
<bigjools> I fear that your gain is my loss though :)
 * jcsackett laughs.
<jtv> bigjools: I _told_ you I could review it.  And I did.
<bigjools> jtv: I figured you were busy with the publisher ...
<jelmer> jcsackett: if there is a free spot in your queue, is there any chance you could have a look at https://code.launchpad.net/~jelmer/launchpad/upgrade-stderr/+merge/69225 ?
<jtv> bigjools: the publisher is busy.  It seems to be getting on fine without my further help.
<jtv> Anyway it didn't hurt much did it?
<bigjools> jtv: I dunno, how many nits do I have in your review? :)
<jtv> Go look.
<jtv> Come on.
<jtv> I know you want to.
<jcsackett> jelmer: looking now.
<jcsackett> jelmer: so, if i read your comment right, we only have ~60-100 imports that would be upgraded?
<bigjools> jtv: bloody hell :)
<bigjools> jtv: thanks
<jtv> This is what restraint looks like.
<bigjools> lol
<bigjools> jtv: yes, lots of setup :(  Welcome to Soyuz.
<jelmer> jcsackett: yes, and those are all disabled at the moment because of the regression this branch fixes
<jelmer> jcsackett: so we could re-enable them gradually
<jcsackett> jelmer: ok.
<abentley> allenap: I'm happy to add comments on the tests, but there are separate tests for the contents already: test_wrap_resource_nested_mapping, test_wrap_resource_nested_array, test_wrap_resource_null.  The "creates" tests are just to show that the array or mapping is a new object, not reused.
<jcsackett> jelmer: to my understanding, this looks good, so r=me.
<allenap> abentley: Yeah, I understand; the itemsAreSame() assertion would be repetition for the sake of clarity. If a comment makes it clear enough then stick with that.
<abentley> allenap: I think that repetition would make the purpose of the test less clear.
<jelmer> jcsackett: thanks!
<allenap> Okay, cool.
<jtv> I just introduced my father to Zombo.com.  It was time.
<gary_poster> abentley, deryck, we have three remaining non-assigned open LP questions, and they are all code related.  could I assign them to abentley to either act on, or give another CHR person an idea of what to do?
<deryck> gary_poster: I'm fine with that, if abentley doesn't mind, of course. :)
<gary_poster> cool, thank you deryck.  waiting for abentley's blessing, curse, or alternate suggestion then
<abentley> gary_poster: I see two questions.  "	Please remove my comment" doesn't look code related.
<gary_poster> abentley, it is a MP comment
<abentley> gary_poster: Oh, okay.
<abentley> bac: landed.
<abentley> gary_poster: I've taken those questions.
<bac> abentley: great
* allenap changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: jcsackett | Critical bugs: 240 - 0:[#######=]:256
<jcsackett> allenap: thanks for the review; that actually explained an intermittent failure i *thought* i had fixed, but clearly is still intermittent. but now i have some idea what to do.
<gary_poster> thank you abentley
<henninge> gary_poster: test suite passed, so nothing to worry about ;-)
<gary_poster> :-) cool henninge
<henninge> bye, back on Monday
<LPCIBot> Project db-devel build #763: STILL FAILING in 6 hr 14 min: https://lpci.wedontsleep.org/job/db-devel/763/
<mtaylor> so - with bug imports ... I got things rejected because I didn't have an email address associted with the bug
<mtaylor> if I don't HAVE an email address from my source - would it be better to just include a bogus email address? what if I _do_ have that person's launchpad id and it doesn't match the email address in their profile?
<abentley> jcsackett: I can has review? https://code.launchpad.net/~abentley/launchpad/mark-dup-empty/+merge/69704
<jcsackett> abentley: you can haz.
<jcsackett> abentley: you can also has r=me.
<abentley> jcsackett: kthxbye
<benji> jcsackett: do you have a moment to look at a small JS branch? https://code.launchpad.net/~benji/launchpad/bug-pre-search-2/+merge/69710
<jcsackett> benji: looking at it now.
<benji> cool
<jderose> mrevell: jkakar said you're the one to ping about getting a project group for novacut :)
<jkakar> Hiya. :)
<mrevell> Hi jderose! Would you mind dropping an email to feedback@launchpad.net? I'm not officially here at the moment. Sorry to be a pain.
<jkakar> mrevell: You're going to learn to hate me for sending people your way all the time. ;b
<mrevell> jderose, Also, I wanted to interview you about Novacut for the Launchpad blog some time, if that's cool with you.
<mrevell> jkakar, Hey, not at all :)
<jkakar> :)
<jderose> mrevell: oh, would be awesome to do interview, thanks! :)
<jderose> mrevell: i'll drop an email, sure, np ;)
<mrevell> jderose, It looks like an interesting project. Also, I saw one of you was in Longmont, CO. I visited there three years ago for work :) Okay, thanks for the email, one of us will reply over the next 24 hours.
<jderose> no rush :)
<cr3> how is the python-twisted-web dependency pulled in the launchpad development environment? I really can't find how it gets pulled from apt-cache rdepends --recursive python-twisted-web even though launchpad-developer-dependencies is listed :(
<LPCIBot> Project devel build #927: STILL FAILING in 6 hr 40 min: https://lpci.wedontsleep.org/job/devel/927/
<jcsackett> benji: sorry for the delay, encountered some issues with my computer.
<jcsackett> benji: this all looks good, but i do have one question. is it necessary to have the option to pass in a bug_id? it doesn't look like that option is used anywhere.
<benji> jcsackett: sorry, I was on a call; being able to pass in the bug ID makes testing easier
<jcsackett> benji: dig.
<jcsackett> benji: r=me.
<benji> jcsackett: cool, thanks
<deryck> Later on, everyone.
* jcsackett changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: - | Critical bugs: 240 - 0:[#######=]:256
<poolie> ** Branch linked: lp:~benji/launchpad/bug-pre-search-2
<poolie> oh you legend
<benji> heh
<poolie> that will be a nice time saver
<poolie> we could do so much more there though
<poolie> through a kind of time and motion study around choosing a bug through to proposing a fix
<benji> yeah that'd be nice (and nice in general); Huw had several good ideas about ways to further improve that task too
<sinzui> wgrant, mumble?
<StevenK> sinzui: http://pastebin.ubuntu.com/654051/
* wallyworld changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: wallyworld* (jtv) | Critical bugs: 240 - 0:[#######=]:256
#launchpad-dev 2011-07-29
<LPCIBot> Project db-devel build #764: STILL FAILING in 6 hr 2 min: https://lpci.wedontsleep.org/job/db-devel/764/
<wallyworld> jtv: you back on board yet?
<poolie> wallyworld: hello, can you review https://code.launchpad.net/~xaav/loggerhead/export-tarball/+merge/66408
<poolie> and merge it if appropriate
<wallyworld> poolie: sure
<wallyworld> that one was marked as needing info for a while i think
<poolie> it stalled for a bit
<poolie> if people push and don't ask for more review it's easy for things to get stuck
<poolie> which is https://bugs.launchpad.net/launchpad/+bug/745469 i guess
<_mup_> Bug #745469: please send notifications when merge page has been updated with new revisions <code-review> <Launchpad itself:Triaged> < https://launchpad.net/bugs/745469 >
<wallyworld> poolie: in the above case, i normally add a comment to the mp explaining what i did and why and how it relates to any previous request for change
<wallyworld> this causes an email to be sent
<poolie> yup
<poolie> it's not obvious that you ought to do this though
<poolie> ought/need
<wallyworld> too true
<lifeless> fixing up aaron's incremental diff thing will address that - and get sensible incremental diffs with merges from trunk (which the loggerhead based diff thing does not do)
<LPCIBot> Project devel build #928: STILL FAILING in 5 hr 37 min: https://lpci.wedontsleep.org/job/devel/928/
<wallyworld> lifeless: can you +1 this? i'm not 100% familar with loggerhead etc but it seems reasonable and has had a lot of input already.... https://code.launchpad.net/~xaav/loggerhead/export-tarball/+merge/66408
<poolie> thanks wallyworld
<poolie> to merge it, when you're ready, you should be able to just merge into lp:loggerhead
<poolie> and i guess it would be good to deploy that to lp too
<wallyworld> poolie: np. i'll clean up the lint first
<wallyworld> poolie: i may need to ask for help deploying - i've had nothing to do with loggerhead so far :-)
<poolie> np
<poolie> i'm getting "lib/canonical/launchpad/icing/yui/yui/yui.js" in my lp checkout
<poolie> oh, a broken symlink or something, isn't it
<wallyworld> poolie: likely. a make clean will fix it
<wallyworld> will || should
<lifeless> are there any tests of the personmerge view other than the doctest ?
<LPCIBot> Project db-devel build #765: STILL FAILING in 5 hr 40 min: https://lpci.wedontsleep.org/job/db-devel/765/
<lifeless> hahaha
<wgrant> ?
<lifeless> -> ops
<lifeless> wgrant: are there unit tests for person.merge() ?
<wgrant> I doubt it. But let's see..
<lifeless> I can't see any in test_person
<wgrant> There is a _do_merge in test_person.
<wgrant> Is that not relevant?
<wgrant> I haven't actually seen anything beyond its existence.
<lifeless> nothing in there looking for notifications that I can see
<lifeless> which is what I'm touching
<wgrant> :/
<lifeless>  and \o/
<lifeless> can has review ?
<lifeless> https://code.launchpad.net/~lifeless/launchpad/bug-761874/+merge/69748
<lifeless> wgrant: ^ or wallyworld ^
<lifeless> wallyworld: I will look at the loggerhead thing monday; it requires some pagination
<wallyworld> lifeless: ok, thanks. will look at your mp
<lifeless> wallyworld: deployment is the usual thing - update the version we use, and qa that once that commit hits qa-staging
<wallyworld> ok
* wallyworld changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: - | Critical bugs: 240 - 0:[#######=]:256
<poolie> cheerio wallyworld
<poolie> what's the idiomatic way to html-escape things in lp python code?
<wgrant> Don't do it.
<wallyworld> poolie: i'll do that mp on monday when it gets +1ed
<wgrant> poolie: What are you trying to do?
<wgrant> Escaping is usually the wrong thing.
<wallyworld> wgrant: you want to mentor: ://code.launchpad.net/~lifeless/launchpad/bug-761874/+merge/69748 ?
<poolie> wgrant: test_scope_documentation_displayed is checking that the feature scopes are in the browser.contents
<poolie> it's failing to find them, i think because the new one i've added contains angle brackets
<wgrant> Ah, so this is in a test?
<wgrant> Just use cgi.escape.
<poolie> heh :) ok
<LPCIBot> Project devel build #929: STILL FAILING in 5 hr 34 min: https://lpci.wedontsleep.org/job/devel/929/
<poolie> i agree with not having random escaping and unescaping in  production code
<adeuring> good morning
<poolie> hi abel
<poolie> could someone do a review for me, on https://code.launchpad.net/~mbp/launchpad/mail-scope/+merge/60281
<adeuring> poolie: I'll look
<poolie> thanks
<lifeless> wallyworld: still here ?
<lifeless> wallyworld: you asked for create to mention delete, but I do that in my patch already.
<lifeless> wallyworld: do you mean you want the interface to talk about implementation ?
 * lifeless loads the question
* adeuring changed the topic of #launchpad-dev to:  https://dev.launchpad.net/ | On call reviewer: adeuring | Critical bugs: 240 - 0:[#######=]:256
<adeuring> lifeless: could you have a look at this mp: https://code.launchpad.net/~adeuring/launchpad/bug-739052/+merge/69625
<lifeless> adeuring: hahha, on friday night no less.
<lifeless> adeuring: great stuff
<adeuring> lifeless: thanks!
<bigjools> morning
<poolie> o/
<mrevell> hallo there bigjools
<poolie> ok, good night
<lifeless> adeuring: reviewed
<adeuring> lifeless: thanks!
<lifeless> adeuring: no probs
<bigjools> wgrant: need a chat with you
<bigjools> adeuring: morning!  I have a lovely branch for you if you can take it please?
<adeuring> bigjools: sure
<bigjools> adeuring: https://code.launchpad.net/~julian-edwards/launchpad/utf-codes-in-emails-bug-817106/+merge/69758
<wgrant> bigjools: Not entirely here, but what's up?
<bigjools> wgrant: I wanted to talk about the no-builds-in-progress check in the IDSJ, and whether/how we should fix the packagecopier
<bigjools> I am very concerned
<wgrant> Your concern is well-placed.
<bigjools> wgrant: so I'd rather mumble/skype.  If now is bad, next week is ok.
<wgrant> bigjools: Next week is probably best, sorry.
<bigjools> not a problem
<bigjools> thanks
<adeuring> bigjools: the diff on the MP page shows a minor merge conflict
<bigjools> adeuring: ah I'll fix that, thanks
<bigjools> adeuring: conflict fixed, just waiting for the branch scanner
<adeuring> bigjools: thanks
<adeuring> bigjools: r=me
<bigjools> adeuring: thank you sir.  Did the unicode stuff look ok?  I figured you'd have more experience of that than me :)
<adeuring> bigjools: as I wrote in the MP, we used something like u'Lo\u2345c' and avoided the "coding: utf-8" line at the top
<bigjools> ah ok
<adeuring> bigjools: but
<adeuring> I thin k we can really switch to the way you did it
<bigjools> I took inspiration from a different python file in our tree
<adeuring> bigjools: even better then :)
<bigjools> LoÃ¯c looks better than Lo\u2345c huh? :)
<adeuring> there is a small risk that somebody uses an editor which does not understnd utf-8
<adeuring> but I am not aware of any such editors...
<bigjools> yeah - I had issues with Vim but then it seems all ok with it today :/
<adeuring> bigjools: and yeah, this '\u3456' looks really ugly and is sometimes hard to understand
<bigjools> indeed
<jml> lifeless: can you please merge: lp:~jml/python-fixtures/misc-fixes
<adeuring> I think Python supports also something like "LATIN_CAPITAL_A_WITHDIERESIS" with some escape mechanism but that's not much better than \u1234
<bigjools> heh
<bigjools> it is a total minefield
<adeuring> bigjools: not such a big pitfall as some Qt unicode -> string conversion method (can't remember which one exactly) It gave you ISO8859-1 strings by default. Was quite annoying for any language spoken that did not originate in western europe
<bigjools> adeuring: nice.
<cjwatson> https://dev.launchpad.net/LaunchpadPpa says "The current development series [...] (currently: natty)" and "The current release [...] (currently: maverick)"
<cjwatson> should that be changed to oneiric and natty respectively now?
<wgrant> Indeed. It does work on oneiric.
<cjwatson> launchpad-dependencies has versions 0.95~oneiric1, 0.95~natty1, 0.95~lucid1.  lp:meta-lp-deps only has 0.95.  Is that done by hand?  How do you deal with different versions of dependencies in different releases?
<bigjools> jtv: the expander sections on the queue syncs look nicer now
<cjwatson> oh, hm, https://code.launchpad.net/~launchpad/+recipe/meta-lp-deps-on-demand
<jtv> bigjools: Thanks.  Actually expanding does look more appropriate on an expander, does it not?
<bigjools> jtv: generally :)
<jtv> BTW it looks like the other expanders don't get the colspan quite right.
<jtv> My eye keeps expecting a link where the archive name is.  Is there anything to link to?
<wgrant> cjwatson: Right, that recipe is used. The deps are identical between series these days, though, so the series-specific sources are unnecessary.
<cjwatson> I guess I can use substvars to force the right versions of apt/python-apt
<cjwatson> cleaner than series-specific sources
<wgrant> Slightly.
<wgrant> maverick doesn't have a sufficient version?
<cjwatson> no
<wgrant> That is inconvenient.
<cjwatson> bigjools has copied stuff into the PPA now (including maverick), but I assume that that won't necessarily get upgraded without forcing versions in lp-deps
<wgrant> Sometimes we force it, sometimes we don't...
<cjwatson> Should I just not bother?
<wgrant> Probably not. Only the two buildbots + cocoplum really need it.
<wgrant> And we can ensure that those are correct easily.
<wgrant> And it avoids the substvar mess.
<cjwatson> it's mildly inconvenient that cocoplum doesn't seem to have the Launchpad PPA in sources.list
<wgrant> Right, the DC machines use CAT.
<wgrant> We generally import stuff from the PPA into CAT.
<cjwatson> so I should RT that?  it's just for apt/lucid
<wgrant> Not sure. LOSAs ^^?
 * mthaddon catches up
<mthaddon> yeah, an RT asking for it to be uploaded to the -cat repos and installed would be good
<cjwatson> OK
<mthaddon> cjwatson: unf we can't install directly from PPAs on servers in the DC as that would basically mean a compromise of LP would root all DC servers
<mthaddon> s/would/could/
<wgrant> Erm...
<wgrant> You know what controls the Ubuntu archive, yeah? :)
<mthaddon> well, that's true, but it's a slightly different case
<wgrant> The security of the primary archive is less tested, yes.
<cjwatson> I'd normally use the ubuntu-platform@ RT address, but I assume I should use the launchpad@ address in this case ...
<cjwatson> wgrant: assuming that the buildbots are up to date or will update, then https://code.launchpad.net/~cjwatson/launchpad/dpkg-xz-support-619152 could be landed now, I believe
<wgrant> cjwatson: Has it been through ec2 lately?
<cjwatson> No
<bigjools> wgrant: can you help the dude in #launchpad?
<cjwatson> mthaddon: done, RT#47110
<_mup_> Bug #47110: [UNMETDEPS] php4-yaz links against old libmysqlclient <php4-yaz (Ubuntu):Fix Released> < https://launchpad.net/bugs/47110 >
<mthaddon> thx
<mthaddon> cjwatson: do you know if this has been tested on staging/dogfood?
<allenap> adeuring: Would you be able to review a mostly JavaScript branch for me? https://code.launchpad.net/~allenap/launchpad/localpackagediffs-filter-by-person-or-team-bug-798873-ui/+merge/69766
<adeuring> allenap: sure
<allenap> Thanks :)
<allenap> adeuring: I am going to get some lunch now but I'll try to watch for pings. Is that okay? If not, leave it for later when I'm back.
<adeuring> allenap: then let me ytart my lunch break too before I start to look at the mp :)
<allenap> adeuring: Cool :)
<LPCIBot> Project db-devel build #766: STILL FAILING in 5 hr 43 min: https://lpci.wedontsleep.org/job/db-devel/766/
<bigjools> """Gina's changelog parser and muncher for great justice"""
<jml> am thinking of doing work to address / mitigate https://bugs.launchpad.net/launchpad/+bug/814725
<_mup_> Bug #814725: No feedback between upload & acceptance/rejection <Launchpad itself:Triaged> < https://launchpad.net/bugs/814725 >
<jml> james_w suggested adding an API method for upload
<jml> any thoughts from folk here? bigjools? wgrant? StevenK?
<bigjools> jml: it's not a trivial problem
<wgrant> Until we have a message queue.
<bigjools> an API for upload is not going to work, it'll time out
<wgrant> Until then, best not to think about it :)
<bigjools> we need to decide what feedback people would be happy with
<jml> bigjools: so the timeout starts from 'request received'?
<jml> or rather, request started
<jml> rather than, say, request finished
<wgrant> jml: The processing itself can easily take many seconds.
<bigjools> jml: to make sure we're talking about the same thing, is that a webservice API?
<bigjools> s/seconds/minutes/
<jml> bigjools: yes.
<jml> why minutes?
<bigjools> jml: ok, that will never work.  we need a different transport
<bigjools> jml: the upload processor has to unpack the package
<bigjools> that can take a loooong time
<jml> why does it need to do that?
<bigjools> to verify its contents
<wgrant> It should never take minutes, but it can have to un-xz and un-bz2 a few hundred megabytes :/
<bigjools> "should" :)
<bigjools> jml: at a stretch we could make poppy poke something into the database to say "pending uploads +=1" and then make the upload processor decrement that.  Then the UI can show something.
<bigjools> not sure of the ramifications of that though
<jml> bigjools: well, there are other checks that are fast to do that the user would be well served to be told about sooner
<jml> e.g. permission checks
<bigjools> jml: yes, they are all done first
<wgrant> I have considered deferring the extraction to a job.
<jml> bigjools: yeah, but currently asynchronously
<wgrant> We could possibly get away with removing a couple of the pedantic checks on the extracted package.
<bigjools> jml: I think it doesn't try to unpack unless all the other stuff passes
<jml> I can't see any mention of tarfile in archiveuploader
<wgrant> And then do the file retrieval async.
<wgrant> jml: Hahah
<wgrant> dpkg-source
<bigjools> anyway - there's a load of bugs which means it'll just OOPS
<wgrant> Not that easy :)
<jml> wgrant: ah, ok. got it.
 * bigjools has deja vu
<wgrant> archiveuploader is a bit special.
<wgrant> Not gina-special.
<wgrant> But still fairly special.
 * bigjools found a horrible bug in gina
<bigjools> it doesn't help when the doctest that checks for output format doesn't have a -NORMALIZE_WHITESPACE :/
<jml> afaict, the verification here is just for 'single directory', 'changelog' and 'copyright'
<jml> I guess dpkg-source might do verification that the code implicitly relies on.
<wgrant> bigjools: While you're there, want to fix archiveuploader's changelog_entry generation?
<bigjools> what aspect?
<wgrant> jml: Right. dpkg-source does a lot of checks.
<jml> why not check the first by listing the tarball contents, and get the other two by just extracting those files?
<wgrant> jml: But if we accept an invalid package, it will just fail to build, so it's not that bad.
<wgrant> Extracting those files how?
<wgrant> We'd have to be able to unpack all the source formats.
<wgrant> Untar + diff
<wgrant> Or untar + untar
<wgrant> Or whatever they come up with next.
<bigjools> accepting invalid packages to the build farm is nuts, don't do that
<wgrant> bigjools: They are rare and will just end up FAILEDTOBUILD. It's not *that* bad.
<bigjools> it's build time wasted
<jml> wgrant: hmm. I see. I guess you'd have to patch upstream to provide the feature.
<bigjools> disk space wasted
<bigjools> and crufty
<jml> bigjools: every successful check is wasted CPU too
<bigjools> reject early, reject often
<bigjools> jml: no, it's just getting deferred
<bigjools> the later you reject in a pipeline that has a lot of steps, the harder it is to produce a decent error message
<jml> Certainly failing early is good.
<jml> But if you are doing an expensive check early that is also implicitly done in later steps, you are penalizing successful cases to help the bad ones.
<bigjools> in this case I disagree; you're just pushing the penalty to later on
<wgrant> Perhaps we should get some data.
<jml> but it's actually making it take longer, isn't it? Or is the output of dpkg-source in the upload check saved so that the build farm doesn't need to run it?
<wgrant> I suspect it's very rare that dpkg-source fails.
<jml> wgrant: yeah, that'd be interesting to see.
<jml> funnel graphs
<wgrant> No, all the buildds have to rerun it.
<bigjools> so assuming most uploads are actually OK, then you're right.  But yeah we need data on that.
<bigjools> I still think we need to check this stuff :)
<wgrant> I'm also not quite sure on how a webservice API would work here.
<bigjools> it wouldn't
<wgrant> I guess you'd upload files to the librarian and then fire off an MQ-based job.
<wgrant> It's doable, and should be done eventually :)
<bigjools> please not the librarian
<wgrant> They would have a <24h expiry.
* bac changed the topic of #launchpad-dev to:  https://dev.launchpad.net/ | On call reviewer: adeuring, bac | Critical bugs: 240 - 0:[#######=]:256
<bac> hola abel
<jml> still probably need a  MQ though.
<bigjools> the way we do it is fine.  We can make poppy send a message to something to kick off processing
<bigjools> I'd like to parallelise per archive (up to a max thread count)
<wgrant> Indeed.
<wgrant> It would be nice if Python did threads :/
<bigjools> heh
<wgrant> Maybe by the time we have a MQ-based jobrunner we will have moved to PyPy.
 * bigjools can't believe this changelog parsing bug has been in Gina so long
<jml> well
<bigjools> wgrant: haha
<jml> we'll see what I actually need to get my thing done.
<bigjools> jml: in lieu of a message, another solution would be to have some sort of pending upload indicator in IArchive which poppy increments and the upload processor decremements
<wgrant> jml: Recipes!
<jml> wgrant: I asked for in-page updates for async jobs for every feature LP did since January 2010.  Including recipes.
<bigjools> jml: when we finish DDs, we'll be finishing *that* :)
<jml> bigjools: looking forward to it.
<bigjools> hellyes
<wgrant> Bug #628738
<_mup_> Bug #628738: SourcePackageRelease.changelog_entry now incorrectly generated <lp-soyuz> <soyuz-upload> <Launchpad itself:Triaged> < https://launchpad.net/bugs/628738 >
<wgrant> bigjools: That's the bug.
<wgrant> (also not caught because of NORMALIZE_WHITESPACE, IIRC)
<bigjools> yay
<bigjools> different problem to the one I am fixing though
<wgrant> doctests, yay
<wgrant> Yeah.
<bigjools> there's nothing like testable documentation
<bigjools> and our doctests are nothing like testable documentation
<jml> except maybe burgers with turds in them.
 * bigjools blinks
<adeuring> allenap: r=me
<allenap> Thanks adeuring :)
<bigjools> wgrant: do you know if iron is in NDT?
<wgrant> bigjools: It is.
<wgrant> mcmurdo too.
<bigjools> good
<bigjools> I love the way that someone thought that list.append(string) included a newline
<wgrant> ftpmaster, ppa, librarian, librarian2, mailman, codehosting are not in nodowntime.
<wgrant> Everything else is.
<wgrant> mailman might be as of 12 hours ago. I forget.
<wgrant> And codehosting will be on monday.
<wgrant> Shall be good.
<bigjools> woo
<cjwatson> mthaddon: TTBOMK it has not
<mthaddon> cjohnston: ok, I've added a note to the ticket that we should do that first, so we'll make it so
<cjwatson> great, thanks
<jml> cjwatson: that's a new acronym for me.
<jml> excellent. no more learning required from me for the rest of the day.
 * cjwatson prescribes jml alcohol
<cjwatson> mthaddon: do I need to take that backlogged message seriously, or is it automatic irrelevancy?
<mthaddon> cjwatson: it basically just means it hasn't been directly assigned to someone yet, but I've given in a priority, and if that needs bumping up it can be done at any time
<mthaddon> s/given in/given it/
<jml> cjwatson: with prescriptions like that, perhaps there's more to be said for private medical practitioners than I had previously thought!
<cjwatson> mthaddon: ok, cool
<deryck> Morning, all.
<jtv> Hi bac!  Free for a review?  I've got this: https://code.launchpad.net/~jtv/launchpad/bug-818032/+merge/69791
<bac> jtv: sure
<jtv> Thanks.
<bac> jtv: did you have any pre-implementation discussions about this bug and branch?
<jtv> bac: yes, hence the "pre-implementation notes" :)
<jtv> With Julian, to be precise.
<bac> jtv: ok.  since you didn't list any one in that section i wanted to find out.
<jtv> bac: in retrospect I see I wasn't very helpful in that section, sorry.
<bac> jtv: looks good.  thanks!
<jtv> thank you.
<abentley> adeuring or bac: could you please review https://code.launchpad.net/~abentley/launchpad/fix-lpviews/+merge/69800 ?
<bac> abentley: i will
<adeuring> bac: you bebat me :)
<abentley> bac: I have fixed the lint.
<bac> i did bebat you!
<bac> abentley: done.  good catch.
<abentley> bac: thanks.
<bac> i didn't realize we had pages using the zope default
<jcsackett> sinzui: i see that you are working on 800361. i just unassigned myself from that in lp; shall i assign you?
<sinzui> jcsackett, I suppose
<jcsackett> sinzui: is there a reason i shouldn't? i only unassigned myself b/c i'm not currently working further on it after UI tests and thought someone else might have the time to do it. which seemed to be you.
<sinzui> jcsackett, I have been very distracted the last two days and have made no progress. If I am distracted today, I will give up the work to wallyworld. This is critical path work and I not certain I should ever be on the critical path
<jcsackett> sinzui: ok. leave it till monday and decide then?
<sinzui> jcsackett, I have been working to unblock others. I am participating in the product strategist interviews
<jcsackett> sinzui: aaah.
<sinzui> jcsackett, do you have time to mumble?
<jcsackett> sinzui: sure.
<gary_poster> bigjools, is there anyone with a somewhat overlapping timezone to mine whom I can bother about a relatively simple soyuz question (https://answers.launchpad.net/launchpad/+question/166346 fwiw)?  I'm sorry I've been bothering you a lot lately.
<bigjools> gary_poster: that doesn't look like a soyuz question
<gary_poster> oh
<bigjools> project release stuff isn;t it?
 * gary_poster feels dumber...
<gary_poster> ok
<gary_poster> bac...do you have any idea about https://answers.launchpad.net/launchpad/+question/166346 ?
<bigjools> gary_poster: oh and I think the answer is "no" anyway :(  You've got 2 Aussies, and me.
<gary_poster> bigjools, heh, ok, thanks
<bac> sorry, gary your message didn't get highlighted.  maybe my client doesn't like "bac..."
<bac> gary_poster: i can't answer that question off the top of my head.  i'm not familiar with how that pattern is used.  i can look into it, though.
<gary_poster> thanks bac.  maybe on your CHR rotation.  I'm not pursuing it
* adeuring changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer:  bac | Critical bugs: 240 - 0:[#######=]:256
<jml> mrevell: did you just put Welsh on the Launchpad blog?
<dobey> hey guys
<dobey> i got OOPS-2036MPJ10 from lp trying to generate a diff for my branch
<dobey> i think it might be a bug in bzr or something
<jelmer> hi dobey
<jelmer> hmm, that OOPS hasn't synced yet
<dobey> ok
<dobey> jelmer: it probably says something akin to this, anyway: http://pastebin.ubuntu.com/654553/
<LPCIBot> Project db-devel build #767: STILL FAILING in 5 hr 38 min: https://lpci.wedontsleep.org/job/db-devel/767/
<jelmer> dobey: that's a known bug
<jelmer> a 5 digit bug even
<jelmer> bug 82555
<_mup_> Bug #82555: Merging to an empty branch doesn't work <merge> <Bazaar:Confirmed> < https://launchpad.net/bugs/82555 >
<dobey> wow
<jelmer> dobey: while you're here... :)
<jelmer> dobey: did you see my mps against lptools?
<dobey> yeah, but i haven't had time to look at them. i guess poolie hasn't either
<jelmer> there's no hurry
<dobey> that bug is interestingly odd
<dobey> since it is possible to merge to an empty branch, but weird things can happen
<dobey> if i just merge to the empty branch and commit straight away, it sort of 'rights' itself
<dobey> so, this is interesting...
<dobey> jelmer: http://pastebin.ubuntu.com/654630/ <- notice the revnos that are listed as negative :)
<dobey> and log -n0 shows them as 1..5
<jelmer> dobey: yeah, there seem to be multiple issues here
<jelmer> dobey: this sort of thing is tricky because of the way we do revno assignment, as you can see :)
<dobey> sure
<gmb> bac: I have a branch that needs reviewing, if you've got time: https://code.launchpad.net/~gmb/launchpad/remove-unused-potmsgsets/+merge/69841
<bac> gmb: i do have time.
<gmb> Thanks
<gmb> Woah.
<gmb> bac: Hang on. Bad diff.
<gmb> bac: Should have submitted against db-devel. https://code.launchpad.net/~gmb/launchpad/remove-unused-potmsgsets/+merge/69842
<bac> gmb: review done with a couple of suggestions
<gmb> bac: Thanks
<deryck> abentley: I'll be ready for our call in about 3 minutes.  Meet you in mumble then?
<abentley> deryck: sounds good.
<LPCIBot> Project devel build #930: STILL FAILING in 6 hr 1 min: https://lpci.wedontsleep.org/job/devel/930/
<abentley> gary_poster: ping
<gary_poster> abentley, hi
<abentley> gary_poster: I'd like to talk about the jsoncache.  Do you have a couple of minutes?
<gary_poster> abentley, sure, in 5 ok?
<abentley> gary_poster: sure.
<gary_poster> cool, abentley, I'll ping
<gary_poster> abentey, thanks, ready
<gary_poster> I'd prefer skype
<gary_poster> abentley ping :-)
 * gary_poster has EoD in 5
<abentley> gary_poster: So that the Thunderdome, you expressed concern about the idea of reloading the whole JSONRequestCache as a means of updating pages.
<gary_poster> yeah
<gary_poster> at least as one to be applied as a blanket approach
<abentley> gary_poster: Was the concern about the size of the download or the expense of calculating it server-side?
<gary_poster> probably fine in some cases
<gary_poster> abentley, expense
<abentley> gary_poster: I think you suggested using ETAGs as a remedy to avoid retrieving already-known data?  But wouldn't that still be expensive server-side?
<gary_poster> abentley, I don't remember suggesting ETAGs for this problem, but who knows. :-)  My preferred solutions might be long poll pushes, and/or MVC in client-side code (which in isolation would only address part of the "curentness" problem, but would be better than what we have).
<abentley> gary_poster: I don't understand how MVC would address it.  The +sharing-details page wants to use it, and IS MVC.
<abentley> gary_poster: Anyhow, I guess we can pick this up next week.
<gary_poster> abentley, ack.  I'm glad that we have the mechanism, and my concerns/thoughts maybe can go under the category of "future work".
<lifeless> gmb: still her e?
<abentley> gary_poster: Anything I come up with will be opt-in, at least initially.  But If there opportunities to support other use cases better, I don't want to miss them.
<gary_poster> abentley, ack.  You might want to talk to wgrant; he has some thoughts about the long poll that might be nicely integrated in the future
<abentley> gary_poster: Yes, we've chatted about that.  He was my roommate at the Thunderdome.
<gary_poster> abentley, heh, ok
<gary_poster> cool
 * gary_poster needs to run
<gary_poster> have a good weekend abentley, all
<abentley> gary_poster: you too.
<abentley> lifeless: IIRC, we have a mechanism for a web client to control what page a form submission redirects us to.  Do you know what it is?
<lifeless> I'm not aware of that except in the special case of the login / logout handshakes
<abentley> lifeless: cool, thanks.
<lifeless> I believe those cases are special-cased by the page being submitted to
<abentley> lifeless: XHR automatically follows redirects, so submitting a form via AJAX redirects to a useless page, which is a pointless roundtrip.  I'd like to either avoid the redirect or redirect to a useful page.
<lifeless> abentley: is this the new ++ thing, or existing form submissions ?
<abentley> lifeless: existing form submissions, but the "useful page" would be the ++model++.
<lifeless> can you tell that its an ajax xhr request on the server?
<lifeless> abentley: also, if you're tuning this, wouldn't it be better not to redirect at all ?
<lifeless> the redirect exists for web clients to end up with nice urls is all.
<lifeless> (ah, as you say above 'avoid the redirect') +1
<abentley> lifeless: Yes, it would be better to not redirect at all, but that would have to be decided server-side.
<abentley> lifeless: Even better would be to return the data we actually want.
<abentley> lifeless: I don't know whether there's a generic way to detect an XHR client, but I'm sure we could set a header or something.
<lifeless> I think having a 'this is xhr' flag and then calculating and returning -only- the needed data would be ideal.
<lifeless> it would avoid the redirect, and avoid the concerns around ++model++ doing too much work on big pages (e.g. BugTask:+index w/bug 1)
<_mup_> Bug #1: Microsoft has a majority market share <iso-testing> <ubuntu> <Clubdistro:Confirmed> <Computer Science Ubuntu:Confirmed for compscibuntu-bugs> <dylan.NET.Reflection:Invalid> <dylan.NET:Invalid> <EasyPeasy Overview:Invalid by ramvi> <elementary OS:In Progress by elementaryproject> <GenOS:In Progress by gen-os> <GNOME Screensaver:Won't Fix> <Ichthux:Invalid by raphink> <JAK LINUX:Invalid> <LibreOffice:In Progress by bjoern-michaelsen> <Th
* bac changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer:  - | Critical bugs: 240 - 0:[#######=]:256
<lifeless> http://activitystrea.ms/
<abentley> lifeless: "returning -only- the needed data", where that's a subset of the ++model++ would be quite a trick.
<lifeless> abentley: are we blocked on deploys until we fix all these views ?
<abentley> lifeless: The only known-broken view is fixed in r13560.
<lifeless> abentley: I'm a little worried that there are others we don't know about, and we'll find out the hard way.
<lifeless> abentley: is it possible to make the code gracefully handle the absence of the method (and perhaps log / write a soft oops) when that happens ?
<abentley> lifeless: In the words of Robert Collins, "Untested code is broken code".
<abentley> lifeless: Writing a soft oops would be fine with me.
<lifeless> I agree that untested code is broken code. We know now that we have (at least) one untested view.
<lifeless> If we deploy as-is and find more untested views exist, we'll have to halt all deployments, rollback the deploy - moderately expensive shenanigans
<abentley> lifeless: That's not the only way to handle a regression.  You can also just fix it.
<abentley> lifeless: these fixes are trivial, e.g. a line of zcml or adding a parent class.
<abentley> lifeless: this morning I searched the oopses for any more affected views and didn't find any.  I can search again.
<lifeless> reverting the deployment takes minutes; the fixing pipeline takes (minimal) 6 hours
<lifeless> doing a cowboy on the spot requires an incidentreport and analysis - its something we want to avoid as much as possible (perhaps too much, but thats a separate discussion)
<lifeless> Lynne and I are popping out, so I have to go
<lifeless> sorry
<abentley> lifeless: I just re-synced qastaging oopses and didn't find any more broken views.
#launchpad-dev 2011-07-30
<LPCIBot> Project db-devel build #768: STILL FAILING in 6 hr 12 min: https://lpci.wedontsleep.org/job/db-devel/768/
<LPCIBot> Project devel build #931: STILL FAILING in 5 hr 36 min: https://lpci.wedontsleep.org/job/devel/931/
<LPCIBot> Project db-devel build #769: STILL FAILING in 5 hr 45 min: https://lpci.wedontsleep.org/job/db-devel/769/
<LPCIBot> Project devel build #932: STILL FAILING in 5 hr 32 min: https://lpci.wedontsleep.org/job/devel/932/
<lifeless> http://theprogrammersparadox.blogspot.com/2007/12/nature-of-simple.html is interesting too
<nigelb> I'm trying to figure out how to go the right view when looking at a template.
<nigelb> Pointers?
<lifeless> zcml
<LPCIBot> Project db-devel build #770: STILL FAILING in 5 hr 54 min: https://lpci.wedontsleep.org/job/db-devel/770/
<nigelb> lifeless: hrm, the configure.zcml file?
<lifeless> files
<lifeless> e.g. lp/registry/configure.zcml maps objects -> template+view
<nigelb> lifeless: aha, thanks!
<LPCIBot> Project devel build #933: STILL FAILING in 5 hr 59 min: https://lpci.wedontsleep.org/job/devel/933/
<LPCIBot> Project db-devel build #771: STILL FAILING in 5 hr 49 min: https://lpci.wedontsleep.org/job/db-devel/771/
<LPCIBot> Project devel build #934: STILL FAILING in 5 hr 38 min: https://lpci.wedontsleep.org/job/devel/934/
<LPCIBot> Project db-devel build #772: STILL FAILING in 5 hr 47 min: https://lpci.wedontsleep.org/job/db-devel/772/
#launchpad-dev 2011-07-31
<LPCIBot> Project db-devel build #773: STILL FAILING in 5 hr 47 min: https://lpci.wedontsleep.org/job/db-devel/773/
<nigelb> I finally understand zope better than before.
<lifeless> nigelb: have your eyes imploded?
<nigelb> lifeless: Not yet. I assume that's the rite of passage.
<StevenK> Eyes bleeding tends to do it, too
<nigelb> I picked up one of those JS + Python bugs.
<nigelb> Fun so far. I had to read the text I hate on how to fix it thrice to find out where to start.
<nigelb> *have
<lifeless> heh :)
<nigelb> Utter respect to people who have to do it everyday.
<nigelb> And furthermore for wgrant for figuring out *before* he joined Canonical.
<nigelb> btw, there's a short cut to not running make run everytime, what was that again?
<lifeless> no idea
<lifeless> -> evening stuff. ciao.
<nigelb> laters!
<wgrant> nigelb: You'll see it run 'bin/run -r librarian -i development' or so.
<wgrant> nigelb: That's the magical command.
<wgrant> Actually, it runs -r librarian,googletestservice,memcache or so, but you can make it quicker by just doing -r librarian.
<wgrant> If you need to change CSS, 'make csscombine' before bin/run. For JS, make jsbuil.
<wgrant> +d
<nigelb> \o/
<nigelb> Thanks!
<nigelb> Should save it this time!
<wgrant> Indeed.
<wgrant> Means you can restart in ~7s rather than 90s.
<wgrant> (FWIW, I knew Zope 3 well before LP)
<nigelb> wgrant: Ah. No wonder.
<lifeless> wgrant: fwiw its 40s for me (make run)
<wgrant> Hmm.
<nigelb> Is Australia/NZ awake yet?
<lifeless> no
<lifeless> :P
<nigelb> ha
<nigelb> I guess I need to wait a bit more for wallyworld to be on IRC.
<lifeless> its 0630 for him now I think
<nigelb> gah, timezones :|
<poolie> can i send https://code.launchpad.net/~cjwatson/launchpad/dpkg-xz-support-619152/+merge/32868 to ec2 for colin?
<poolie> s//is it safe to
<lifeless> poolie: I'm not sure; are all the deps available in the relevant production archives etc ?
<poolie> it's in lucid-updates as far as i can tell
<poolie> lp does get that, doesn't it?
<poolie> anyhow it's in ec2 now
<wgrant> poolie: Not to land, I hope.
<wgrant> poolie: buildbot is not upgraded yet.
<wgrant> Oh, different branch.
<wgrant> Not sure about that one... maybe.
<wgrant> But the apt one is definitely not upgraded everywhere yet.
#launchpad-dev 2012-07-23
<StevenK> I thought I could click on 'Private: Some' on a +sharing page to remove access. :-(
<wgrant> StevenK: You can
<StevenK> wgrant: Except it doesn't work on qas
<wgrant> StevenK: Hm, but the FF is enabled...
<wgrant> Oh
<wgrant> StevenK: You're trying on launchpad?
<wgrant> flacoste owns that on qas, rather than ~launchpad
<wgrant> So you can't edit it
<StevenK> Ah
<wgrant> webops: could you please reassign https://launchpad.net/launchpad from ~flacoste to ~launchpad?
<wgrant> Bah
<wgrant> https://qastaging.launchpad.net/launchpad
<StevenK> Hahahaha
<hloeung> wgrant: "No items matched "~launchpad"
<wgrant> hloeung: Drop the ~
<hloeung> wgrant: done
<wgrant> Indeed, thanks.
<wgrant> StevenK: Try again?
<StevenK> Right, now it has edit icons and such
<StevenK> Before, the text were links, which gave the impression
<StevenK> wallyworld: You haz QA
<StevenK> Not that it matters
<wallyworld> yes, will do it soon
<StevenK> wallyworld, wgrant: Either of you want to review https://code.launchpad.net/~stevenk/launchpad/auditor-for-packageupload/+merge/116180 ?
<wgrant> StevenK: à² _à²  à² _à² 
<wgrant> AuditorClient.recieve is awful
<wgrant> One doesn't compose query strings using string formatting operations.
<wgrant> Also, consider using lazr.uri to compose the rest of the URL
<wgrant> >>> u = lazr.uri.URI(scheme='http', host='foo', port=1234, path='/')
<wgrant> >>> u.append('log/').replace(query='somequery')
<wgrant> URI('http://foo:1234/log/?somequery')
<wgrant> Also, datetime.now() is local time, which is wrong
<StevenK> wgrant: I didn't know about lazr.uri, but it doesn't seem like it will help much, given the URLs for auditor are simple.
<StevenK> wgrant: If not string formatting operations, then what?
<wgrant> >>> urllib.urlencode([('correct=encoding', 'is&good?')])
<wgrant> 'correct%3Dencoding=is%26good%3F'
<StevenK> wgrant: http://pastebin.ubuntu.com/1105753/
<wgrant> Improved.
<StevenK> But still not good? :-P
<StevenK> wgrant: Diff updated, makes use of urlencode rather than '&'.join() and datetime.utcnow()
<wgrant> StevenK: Have you discussed all this with lifeless?
<StevenK> wgrant: He just gave his 2 cents on the MP itself
<StevenK> So I'll be refactoring after lunch
<StevenK> wgrant: But thanks for the hints about making use of urlencode.
<wgrant> StevenK: In general if you're using string formatting to produce a machine-readable string in a format that you didn't design yourself you probably have a terrible bug
<lifeless> intel should quit the wifi chip business.
<StevenK> Or perhaps you should switch to Ethernet
<lifeless> yeah, no.
<lifeless> I'd rather whinge than fix the problem :)
<lifeless> Actually, cables around home are problematic when you have a 11month old.
<lifeless> +2 cats with cable fethishes.
<StevenK> Haha
<StevenK> Yes, I can remember having a cat with a cable fetish
<lifeless> Anyhow, I see my name in backscroll.
<StevenK> Pretty much every cable that the TV/DVD player/PS2 used had bite marks/exposed wiring on it
<StevenK> Except for power cables, she never bit those for some reason
<lifeless> so ...
<lifeless> whats 'this' under
<lifeless> 14:29 < StevenK> wgrant: Diff updated, makes use of urlencode rather than '&'.join() and datetime.utcnow()
<lifeless> 14:46 < wgrant> StevenK: Have you discussed all this with lifeless?
<lifeless> StevenK: ?
<StevenK> [12:57] < StevenK> wgrant: He just gave his 2 cents on the MP itself
<StevenK> [12:57] < StevenK> So I'll be refactoring after lunch
<StevenK> lifeless: ^
<wgrant> wallyworld: What do you feel is the best way to customise vocab content? Sometimes we pass a context into the vocab constructor, but a couple of weeks ago I used a different method (constructing a SimpleVocabulary using a subset of the terms from the original vocab) in BranchEditView.setUpWidgets, since it's more a case of the view knowing which information types to show.
<wgrant> I'm not sure it's great to push that knowledge down to the vocab itself
<wgrant> Since InformationTypeVocabulary really shouldn't know about branches or bugs
<wgrant> But I want others' opinions.
<wallyworld> wgrant: hmmm. the way i've used it based purely on the context
<wallyworld> but i've also seen a SImpleVocab constructed
<wgrant> wallyworld: The context being the pillar?
<wallyworld> yes
<wgrant> That works sometimes
<wgrant> But then you have artifact-specific restrictions
<wgrant> eg. a branch can't change to Private Security if it's stacked on a Proprietary branch
<wgrant> A bug can't be Proprietary if it affects 10 projects
<wallyworld> so we need to do the construction at the point where we have the info to make a decision
<wallyworld> so maybe a method on the moel
<wallyworld> which the view can call to get the vocabl to use
<wallyworld> or even a method on the service
<wgrant> Yeah, that's sort of the design I went for with Branch.information_type
<wgrant> There's Branch.getAllowedInformationTypes
<wallyworld> yes, that sounds like something reasonable
<wgrant> And BranchEditView.getInformationTypesToShow reduces that further
<wgrant> And BranchEditView.setUpWidgets turns getInformationTypesToShow into a vocab
<wgrant> It seems to work pretty well
<wgrant> But I need to do something similar for bugs now
<wallyworld> so the view is responsible for presentation logic, the model for model logic etc
<wgrant> Yep
<StevenK> wgrant: FTR, I've never liked the pattern of teaching the vocab stuff
<wgrant> getAllowedInformationTypes is used by transitionToInformationType to verify that the type is legal
<wallyworld> and the model/service apis can also be used to validate for when called over the ws api
<wgrant> getInformationTypesToShow filters that list down to remove unlikely types from the UI
<wgrant> I mostly just need to work out how to integrate this with the JS choicesources, but I think it should be easy enough
<wallyworld> for the choicesources, just put stuff in the json cache
<wgrant> Yeah
<wallyworld> and then js can construct it
<wgrant> I'm just hoping they don't rely on having a registered vocab
<wgrant> But I don't think they do
<wallyworld> the choice sources widget?
<wallyworld> no
<wgrant> Yeah
<wallyworld> you just pass an array of items
<wgrant> Ah, InformationTypeVocabulary isn't even registered in the first place, excellent.
<StevenK> wgrant: If you want to rip out the context stuff out of that vocab you would be my hero
<lifeless> StevenK: Ah. So I didn't see that - see under iwlwifi fail :)
<lifeless> StevenK: As for the MP, I didn't review it, just glanced at it and saw the client was inlined which seemed strange to me.
<StevenK> lifeless: Yeah, and I'm refactoring it out again
<StevenK> Well, half of it out
<StevenK> Some of it is LP specific
<lifeless> the config bits?
<lifeless> they are sure
<StevenK> lifeless: And object_to_enterpriseid() and object_to_enterpriseid() calls
<StevenK> Sigh, enterpriseid_to_object() even
<lifeless> right, thats a layer thing though I suspect ;)
<wgrant> I guess vocabs really want a filter() method that returns a subset.
<StevenK> TypeError: must be type, not classobj you make me very sad
<wgrant> You fail at new-style classes.
<StevenK> Obviously
<StevenK> wgrant: I thought it was just __metaclass__ = type and move on
<wgrant> Yes.
<StevenK> Hmmm, the egg didn't change.
<adeuring> good mornig
<czajkowski> StevenK: morning any reason why your branch has never landed? https://bugs.launchpad.net/launchpad/+bug/790535
<_mup_> Bug #790535: Distribution:+ppas timeouts <timeout> <Launchpad itself:Triaged> < https://launchpad.net/bugs/790535 >
<rick_h_> ccccccbgjgvchvctjfjjeluutgulchrvkjjhtftctfcc
<rick_h_> ccccccbgjgvcugtgelehbubljkrlledibbfhgrjulcdd
<rick_h_> ccccccbgjgvccbtrielunjgkdgvkhldctjkdvbkbfjhl
<rick_h_> bah, sorry
<StevenK> czajkowski: Because it's not the right fix.
<czajkowski> rick_h_: I know the feeling
<rick_h_> czajkowski: monday monday
<StevenK> I think the fix is to delete Distribution:+ppas for being a crime against humanity.
<czajkowski> StevenK: well possibly
<rick_h_> adeuring: morning, did you get a chance to catch up on the branch from thurs/friday?
<adeuring> rick_h_: ummm. somewhat.
<rick_h_> it's listed as needing qa and deryck rolled it back in 15661
<rick_h_> I really want to try to get the report a bug link landed, 15657, so wanted to bug you on that to see if we can clear it up to try to get a NDT
<adeuring> rick_h_: in that case, I'd prefer a rollback. The problem with the buildbot failure: the test call apply_async() -- when celeryd is at least suipposed not to run -- and a subsequent check if there is a message for the task queued fails: No messages at all.
<rick_h_> adeuring: right, the rollback is in the queue from deryck on friday
<adeuring> i habe no clue how this can happen...
<rick_h_> but is it ok then to just qa yours as qa bad for now?
<rick_h_> and try to get through to the rollback for a NDT?
<adeuring> rick_h_no, deryck just disabled the test
<rick_h_> ah, doh ok then
<rick_h_> sorry, didn't realize that.
<wgrant> rick_h_, adeuring: Bug #1027103
<_mup_> Bug #1027103: TestRunMissingJobs.test_run_missing_ready_does_not_return_results disabled due to spurious failure <spurious-test-failure> <test-system> <Launchpad itself:Triaged> < https://launchpad.net/bugs/1027103 >
<wgrant> Please revert ASAP if it needs it
<rick_h_> wgrant: just saw the MP email on that one
<rick_h_> wgrant: so in process
* bac changed the topic of #launchpad-dev to: http://dev.launchpad.net/Â | On call reviewer: - | Firefighting: - | Critical bugs: 4.0*10^2
* benji changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: benji | Firefighting: - | Critical bugs: 4.0*10^2
<benji> gary_poster: coming
<wgrant> adeuring: Hm, did the rollback get bounced by PQM?
<adeuring> wgrant: its running thourgh ec2 right now.
<wgrant> adeuring: Oh, no point or time for that
<wgrant> lp-land
<wgrant> We can't deploy until it's landed, so even if there was a chance of failure we still lose nothing by landing it now.
<adeuring> ok, i'll try it...
<deryck> Morning, all.
<czajkowski> deryck: ello
<deryck> adeuring, https://plus.google.com/hangouts/_/ceb51247bd8993e84395ae3e71619d1a725b8063?authuser=0&hl=en
<deryck> wgrant, hi there.  Was there something actually qa-bad about adeuring's branch or are you reverting as a matter of conscious?  or to get the deploy going, or something else?
<wgrant> deryck: Ah, adeuring said here that it should be reverted.
<deryck> wgrant, ah, ok.  Fair enough.  Thanks!
<czajkowski> gary_poster: what make/model headset do you have btw ?
<gary_poster> czajkowski, Sennheiser...um...looking...
<gary_poster> czajkowski, not sure, but Sennheiser PC 151 looks like newer version (mine does not have noise canceling microphone but looks very similar and the price of about $50 USD seems to match my memory)
<czajkowski> gary_poster: cheers
<czajkowski> adding to my wishlist
<gary_poster> :-) cool
<czajkowski> gary_poster: you're very clear on calls
<gary_poster> yay czajkowski :-)
<mgz> he also enunciates well which helps no doubt
<gary_poster> heh
<czajkowski> this is rather true, I however speak way too fast and live over a flight path, not a good combination
<gary_poster> it's all the many years of singing training
<wgrant> Over a flight path? Impressive.
<gary_poster> heh
 * czajkowski peers at wgrant 
<czajkowski> wgrant: lots of planes fly over the house reguarly, hows that mr. pedant :)
<gary_poster> lol
<wgrant> Aw
<wgrant> And there I was thinking that it was cool that you lived in the stratosphere.
<czajkowski> wgrant: one day we're going to meet and I'm going to be in throwing distance of you!
<gary_poster> he moves quickly
<gary_poster> probably can dodge well
<gary_poster> no actual experience from throwing end though
<Bert_2> Hi, I'm editing all the launchpad templatefiles to comply with the requirements of canonical I change all icons and the name Launchpad, now am I allowed to have launchpad in the css and new imagefilenames, or do I need to change that as well ?
<Bert_2> en thandige is datk ook ni kan zien of een file edits nodig heeft of ni omda zo goed als elke file i18n:domain="launchpad" bevat
<Bert_2> sorry, wrong channel for that last (dutch) message
<deryck> Bert_2, I don't honestly know, but we could find out.  I'm sure the intent is to protect branding of the launchpad site.  So I don't think anyone will care if this is css class names and such internal to the site.
<czajkowski> mrevell: do yo happen to know re Bert_2 request
<mrevell> I expect deryck is correct.
<mrevell> But IANAL. Bert_2 If you email help@launchpad.net we could forward it to our legal team.
<Bert_2> perfect, I'll email to help@launchpad.net then
<Bert_2> thx a lot, let's hope the new logo, icons, and name will be enough, cause manually editing each template file is already taking a lot of time :p
<Bert_2> k, you've got mail ;)
<sinzui> benji: do you have time to review https://code.launchpad.net/~sinzui/launchpad/adapt-subscription-to-pillar/+merge/116095
<benji> sinzui: sure, if after lunch is fine
<sinzui> It is, thank you
<benji> sinzui: your branch looks good
<sinzui> thanks benji
<benji> my pleasure
<rick_h_> benji: couple of reviews your way please. Pre-landing bugfix with JS gardening, no real changes, just cleaning up some LoC to get a credit for the bugfix coming later.
<rick_h_> https://code.launchpad.net/~rharding/launchpad/tab_client/+merge/116329 and https://code.launchpad.net/~rharding/launchpad/garden_client/+merge/116330
<benji> rick_h_: cool, I'll take a look
<rick_h_> ty much!
<allenap> benji: Thanks for the review :)
<benji> my pleasure
<lifeless> o/
<allenap> o/ lifeless
<abentley> deryck: even though I have bugs.dynamic_bug_listings.enabled, I still don't see the sorting losenges and config wheel.  Any idea what I need to do?
<deryck> abentley, hmmm, weird.  thinking....
<deryck> abentley, I wonder if some other flag is conflicting and needs turning off.  Something off in production by default but on in dev?
<abentley> deryck: err, seeing "ReferenceError: YUI is not defined"
<deryck> abentley, ah, js build issue then.
<abentley> deryck, rick_h_: Did make clean; make run, still getting ReferenceError: YUI is not defined.
<rick_h_> abentley: not sure. You have build/js/yui? combo load is on by default now so should be looking in there
<deryck> abentley, I would think make run should call  make jsbuild, but might be worth trying a make clean_js && make jsbuild
<rick_h_> abentley: check the network and see if it has a combo load call and if so what came back from that
<abentley> rick_h_: I do have build/js/yui.  It has a yui-3.3.0 symlink that is broken.
<rick_h_> abentley: ok, so make clean_buildout and make again
 * rick_h_ dbl checks clean_buildout
<rick_h_> abentley: yea, that should hopefully fix your symlink? Should be linked from the dep cache
<abentley> rick_h_: The link is still broken.  It points at ../../../../build/js/yui-3.3.0, but that doesn't exist.  ../../../../build doesn't exist.
<abentley> ../../../../ is the directory containing my launchpad branch.
<rick_h_> abentley: so you don't have build in the root LP branch dir? along with bin, lib, etc?
<abentley> rick_h_: I do.
<abentley> rick_h_: It's looking one parent directory above that.
<abentley> rick_h_: My working tree is at ~/launchpad/colo, but it's looking for build in ~/launchpad.
<rick_h_> abentley: looking for what originally creates the build directory itself, sec
<rick_h_> that's buildout that creates the dir and sets up the YUI 3.3 symlink that the combo-rootdir then links to build/js/yui
<rick_h_> abentley: check out the [yui] block in buildout.cfg and see which command isn't working right that causes you to not have the right path
<rick_h_> abentley: you still in dev mode? In production mode it setups up /srv/launchpad.net for the build dir
<abentley> rick_h_: still in dev mode AFAIK.
<rick_h_> abentley: yea, so check out the buildout install yui and see if it's erroring on something in your env and if that's correct then combo-rootdir comes in and symlinks to the build dir specified which is just 'build/dir' from the calling point
<abentley> rick_h_: I don't really know what to look at.
<rick_h_> abentley: so in the buildout [yui] block should be a series of commands that extract yui from the download-cache and setup the point for the symlink into build/js/yui-3.3.0
<abentley> rick_h_: I have that, and a yui symlink that links to yui-3.3.0.
<rick_h_> ok, and that's an ok symlink?
<abentley> rick_h_: Yes.
<rick_h_> abentley: then run ./bin/combo-rootdir build/js and it should get you a good symlink into the build/js/ directory
<abentley> rick_h_: The broken symlink is Inside yui-3.3.0, and is named "yui-3.3.0".
<rick_h_> hmmm, wtf
<rick_h_> your buildout.cfg is still pulling download-cache/dist/yui-${:yui_version}.tar.gz right?
<abentley> rick_h_: http://pastebin.ubuntu.com/1107124/
<rick_h_> abentley: ok, I see something strange
<rick_h_> so in build/js/yui-3.3.0 I see all of YUI and a symlink that's ../../../../build/js/yui-3.3.0 i'm not sure where it came from
<abentley> rick_h_: So if you see it, it's not what's breaking me.
<rick_h_> right
<abentley> rick_h_: In the network view, I see a 404 for https://bugs.launchpad.dev/+combo/?yui/yui/yui-min.js&lp/meta.js&yui/loader/loader-min.js
<rick_h_> do you have the build/js/lp dir?
<abentley> rick_h_: yes.
<rick_h_> is the meta.js in there?
<abentley> rick_h_: yes.
<rick_h_> if so can you get just https://bugs.launchpad.dev/+combo/?lp/meta.js
<rick_h_> without a 404?
<abentley> rick_h_: No.
<rick_h_> ok, so you don't have convoy setup in apache then
<abentley> rick_h_: http://pastebin.ubuntu.com/1107142/
<rick_h_> right, the apache config should be updated to not proxy +combo urls and instead serve via convoy
<rick_h_> abentley: look in configs/development/local-launchpad-apache and diff against your apache config
<abentley> rick_h_: I just ran "make install", and that's giving me plausible results.
<abentley> rick_h_: Yes, all looks good now.
<rick_h_> abentley: ok awesome
* benji changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: - | Firefighting: - | Critical bugs: 4.0*10^2
<Ergo^> evening
<Ergo^> hello, ive seen launchpadblog entry about career opportunity pointing here so i can see what you guys do, so i've decided to pop in
<Ergo^> hi czajkowski
<czajkowski> Ergo^: hiya, which role?
<Ergo^> czajkowski, http://blog.launchpad.net/were-hiring/were-hiring-software-engineers-to-work-on-cloud-projects-in-canonical
<czajkowski> Ergo^: ah that one.
<czajkowski> Ergo^: well if you have questions many of the team are on here, gary_poster might also be able to help
<czajkowski> or you can apply via the link in the post
<Ergo^> ah, ok, i was thinking i would fit the role well, when do you guys plan to hire someone ?
<Ergo^> my firstborn is supposed to be born this week so im not sure its right time to apply ;-)
<Ergo^> gary_poster, hello gary, what pythonic web frameworks do you guys mostly use?
<czajkowski> Ergo^: soon, he may not be around as it's possibly coming to his EOD
<czajkowski> Ergo^: where are you based?
<Ergo^> czajkowski, Poland, so its midnight for me
<czajkowski> ah most of hte EU folks are well past EOD
<czajkowski> sorry
<Ergo^> czajkowski, no problem
<Ergo^> i might try tommorow
<czajkowski> earlier in the day might be bette
<czajkowski> r
<czajkowski> good night
<Ergo^> night!
<rick_h_> doh, just missed him. I know Ergo^ from the pyramid/pylons irc world
#launchpad-dev 2012-07-24
<nigelb> 26
<nigelb> grr
<StevenK> wgrant: I have private bug with nothing, with subscriber and with structural subscriber, asserting on what getBugNotificationRecipients() returns. I'm obviously missing a bunch of scenarios
<wgrant> There are three implicit subscriber cases: structural subscriptions, dupe subscriptions, and assignees.
<wgrant> I'm not sure if dupe assignees and dupe structsubs also count.
<StevenK> wgrant: So assignee sounds a good plan -- I've not thought about dupes yet
<StevenK> wgrant: Blink.
<StevenK> wgrant: IBug.markAsDuplicate() recalculates heat, but so does IBug._markAsDuplicate() which the former calls.
<wgrant> StevenK: markAsDuplicate recalculates the heat of the new master, _markAsDuplicate recalculates the heat of the old.
<StevenK> Ah
<StevenK> wgrant: http://pastebin.ubuntu.com/1107437/
<wgrant> StevenK: A good start, indeed.
<StevenK> wgrant: Do you think it's worthwhile expanding out to public bugs?
<wgrant> StevenK: Certainly
<wgrant> StevenK: It is to be a thorough set of tests for getBugNotificationRecipients
<wgrant> It's hardly comprehensive or thorough if it doesn't do public bugs too.
<lifeless> gary_poster: nice mail :)
<StevenK> wgrant: http://pastebin.ubuntu.com/1107562/  bugwatch is up next, and then subscribed to question linked to bug
<StevenK> Then I might be done
<StevenK> wgrant: So now that hilarity has passed, bugwatches?
<wgrant> StevenK: How are watches involved?
<StevenK> wgrant: sinzui implicates them
<wgrant> Hm
<wgrant> That's odd.
<StevenK> wgrant: I don't know how they work. :-(
<StevenK> Well, I know they link to an external bugtracker and checkwatches does something to some watches and then hangs.
<StevenK> But can a watch have subscribers or impact on getBugNotificationRecipients()?
<wgrant> No
<wgrant> I don't believe they're relevant here at all.
<StevenK> I shall ignore them, then
 * StevenK moves onto working on how to staple a question and bug together
<StevenK> wallyworld, wgrant: https://code.launchpad.net/~stevenk/launchpad/bugnotification-testcases/+merge/116402
<wgrant> StevenK: Looking
<wgrant> StevenK: https://code.launchpad.net/~wgrant/launchpad/faster-checkCanBeDeleted/+merge/116404
<StevenK> wgrant: r=me, looks great
<wgrant> Thanks.
<StevenK> wgrant: What about my review? :-)
<wgrant> StevenK: Looking
<wgrant> What's + def test_public_bug_with_duplicate_same_pillar_subscriber(self):?
<StevenK> wgrant: Both bugs are against the same pillar
<wgrant> Yes, but why?
<wgrant> How's it relevant as a separate case?
<StevenK> The private case will be different, and I did the same cases for public bugs
<StevenK> I can condense the two public duplicate tests if you wish
<wgrant> I don't see how the private case is different.
<wgrant> It's matters whether the subscriber has access
<wgrant> Not whether it's a different pillar.
<StevenK> wgrant: It will be -- give an APG against the pillar
<wgrant> StevenK: Und?
<wgrant> That's the case where there's an APG against the pillar
<wgrant> Not the case where they're the same pillar
<StevenK> wgrant: Right, so condense the four down to test?
<StevenK> Down to two, rather
<wgrant> Right
<wgrant> Test things that should be tested because they make a difference.
<StevenK> I split it out due to sinzui's mail
<wgrant> Also, do the tests pass?
<StevenK> They do
<wgrant> Huh
<wgrant> How does the question one work...
<wgrant> Oh god
<wgrant> Question.linkBug subscribes the question owner
<wgrant> What could possibly go wrong.
<StevenK> WCPGW, right?
<StevenK> :-)
<wgrant> Implicit access grants are bad, m'kay.
<StevenK> Yes, I'm not sure the two question tests add value
<wgrant> They don't.
<StevenK> Bin them too?
<wgrant> Yes
<wgrant> Let me read sinzui's mail
<wgrant> Ah
<wgrant> So he
<wgrant> probably speaks of the bug->question notification forwards
<wgrant> I'm not quite sure how they work
<wgrant> But they're separate from this
<wgrant> So don't need to be considered here, at least not yet.
<wgrant> StevenK: Are you sure that all of xx-initial-bug-contacts.txt is duplicated?
<StevenK> wgrant: It was sinzui's suggestion that it be binned since much of the test will fail to be true in a few weeks with the bug supervisor and security team playing
<wgrant> StevenK: Sure, but it covers a whole lot of stuff that's only barely related to what you're adding tests for
<wgrant> Like setting the bug supervisor
<wgrant> And the security contact being subscribed by default
<StevenK> Bug supervisor is also dying horribly too?
<StevenK> Security contact is also changing after we can filter on information type
<wgrant> StevenK: Bug supervisor is not dying.
<wgrant> Its role is changing.
<wgrant> Security contact will probably die entirely.
<wgrant> We will still need to be able to set the bug supervisor.
<StevenK> I can shuffle that part into a browser test easily
<wgrant> Rogjt
<wgrant> Right
<wgrant> But you can't just delete it
<wgrant> Without confirming that it's been replaced, or replacing it yourself
<StevenK> lib/lp/bugs/browser/tests/test_bugsupervisor.py:class TestBugSupervisorEditView(TestCaseWithFactory):
<StevenK> Does that count?
<wgrant> If it tests the same stuff, sure.
<wgrant> Oops
<StevenK> wgrant: Diff updated, I've added new tests for the Change bug supervisor link
<StevenK> And I'm still 164 lines in credit
<wgrant> StevenK: Thanks
<StevenK> wgrant: Thanks. Did you see the branch I just landed?
<wgrant> Aha
<wgrant> I think only two of those were mine
<wgrant> Yay
<wgrant> Only one actual bug in bugsummary maintenance
<wgrant> And one in the rebuild script
<wgrant> StevenK: https://code.launchpad.net/~wgrant/launchpad/bug-1028279/+merge/116411
<wgrant> StevenK: (yes, this is very similar to your thing which broke everything, but I like to live dangerously)
<adeuring> good morning
<Ergo^> morning
<Ergo^> gary_poster, hello, im looking for some info on the position
<Ergo^> http://blog.launchpad.net/were-hiring/were-hiring-software-engineers-to-work-on-cloud-projects-in-canonical
<Ergo^> czajkowski, said i should be asking you about it
<lifeless> or gmb
<czajkowski> gmb: ping
<lifeless> gary will be asleep right now (I hope!)
<Ergo^> haha ok :]
<czajkowski> Ergo^: mornign more of the developers are online now
<czajkowski> and in fact gmb is on hols
<czajkowski> wgrant: might you be able to answer some of Ergo^ questions on lp dev please, he's thinking of applying for one of the roles
<wgrant> I'm trying to not be here :)
<wgrant> 'cause it's getting late
<czajkowski> I know but I cant find any EU folks up atm
<wgrant> gmb can't be too far away
<Ergo^> wgrant, carry on, i can ask later ;-)
<czajkowski> adeuring: allenap ping are either of you about ?
<jpds> wgrant: From where you are?
 * adeuring is looking
<wgrant> Heh
<czajkowski> jpds: AU
<czajkowski> although he never sleeps
<czajkowski> adeuring: thanks
<czajkowski> Ergo^: adeuring might be able to answer some of your questions
<czajkowski> Ergo^: thanks for coming and asking though
<adeuring> Ergo^: I think you should wait for gmb if you have more specific questions about the job.
<adeuring> But we can of course anwser more generic questions about Canonical ;)
<Ergo^> well, i would be interested to what i would work on and what kind of skill level would be expected for the position
<Ergo^> adeuring, i've never worked for a IT organization that would have many employees, so im not sure what i would ask about :-)
<adeuring> Ergo^: I don't know much what the the work is about aside from the general "cloudy" stuff... Regarding an org with "many" employees: I myself do not notice that much about the number of people employed by Canonical. You will work i a relatively small team, I assume. four or five people probably.
<Ergo^> czajkowski, can i pm you ?
<czajkowski> Ergo^: sure
<allenap> Ergo^: gmb will be able to tell you more about the team's first project, but, like adeuring, there isn't the feel of "big company IT". We try to produce the best code we can, in a professional way; I have learnt *so* much about how to produce large pieces of software from working here. We're rigorous about testing, and we try to make decisions based on evidence and measurement. We don't always get it right, but we tend to see mistakes a
<allenap> s an opportunity to learn rather than to blame.
<lifeless> Ergo^: the new squad will be working on ceilometer initially.
<lifeless> allenap: +1
<czajkowski> lifeless: go sleep!
<lifeless> one word: Cynthia.
<allenap> Ha!
<czajkowski> lifeless: awww
<Ergo^> ok, well i work in environment where we dont write tests at all - they move more like early facebook - the boss doesnt want to budget for tests
<Ergo^> but theres no problem to write tests
<Ergo^> i would love to move to environment where i can learn new things
<Ergo^> lifeless, so ceilometer - what kind of web stack will be used for the project? pylons/pyramid? something home brewed ?
<lifeless> have a poke around yourself - https://launchpad.net/ceilometer
<Ergo^> oh it uses webob
<Ergo^> nice :]
<Ergo^> and sqlalchemy, even nicer
<Ergo^> i was expecting it would be using sqlstorm
<rick_h_> Ergo^: hey howdy
<Ergo^> ohai rick_h_
<rick_h_> Ergo^: you were hacking on some of the min-webhelpers stuff right?
<Ergo^> rick_h_, yeah ive made html.grid stuff
<rick_h_> Ergo^: heh, not going to see much of the usual stack around :/
<rick_h_> but really good dev practices, some stuff to learn for sure
<Ergo^> rick_h_, well i would love that because i wont learn new things the place im at atm
<Ergo^> different focus
<rick_h_> Ergo^: definitely apply if you might be interested and chat with the guys. They're a really good group of smart peeps.
<Ergo^> i would be, question is if they would be :P
<Ergo^> rick_h_, well even if its not stack im used to it has some parts i know and love, so thats a good start
<rick_h_> right, the launchpad stack is kind of strange, but you're allowed to learn it. I didn't know anything more than some articles of zope stuff
<rick_h_> but most other things are django based
 * Ergo^ knows nothing about django
<rick_h_> heh, it's 'easy' :P
<Ergo^> i would expect pylons/pyramid based wsgi apps, since i would expect its more flexible for this kind of environment
<rick_h_> you'd think...but no. I've not won a battle on that front yet though there are some of us trying
<Ergo^> rick_h_, i would expect mramm to help ;-)
<rick_h_> hah, I'll have to ask him what his team's up to on that end I guess.
<lifeless> go
<lifeless> they are all about go
<Ergo^> yeah he mentioned he writes in go right now last time ive asked
<rick_h_> ah, he's over there
<Ergo^> well, the ive browsed the code for ceilometer, it seems understandable and nice for me :]
<rick_h_> czajkowski: looks like the report a bug link failure is fixed yay
<Ergo^> rick_h_, when do you plan to add a human friendly interface to launchpad ? ;-)
<rick_h_> Ergo^: heh, that's huw's job :P
<Bert_2> Hi, I have setup an instance of launchpad with an actual domain (not launchpad.dev) and a clean db using wgrant's bootstrap-db-from-scratch and have added users using his user creationscript, now nor the admin nor regular users seem to be able to login into the subdomains of my instance (code, answers, blueprints), when I login with any account it redirects back to the page I came from but I'm still not logged in, does anyone know what'
<Bert_2> s wrong ? Did I misconfigure something ?
<Ergo^> gmb, ping
<deryck> Good morning, all.
<abentley> deryck: morning.
<abentley> deryck: http://www.goodreads.com/quotes/14559-good-morning-said-bilbo-and-he-meant-it-the-sun
<deryck> abentley, heh.  Great passage.
<deryck> I knew there was a reason I normally typed only "morning" :)
<abentley> deryck: Hehe.
<deryck> adeuring, https://plus.google.com/hangouts/_/5c02c8b06ff0d06aaf9314c93315d217f5459977?authuser=0&hl=en
<Ergo^> gmb, ping
<czajkowski> Ergo^: he's not here. and won't be for two weeks
<Ergo^> ah :]
<Ergo^> rick_h_, so you say i should apply ? ;-)
<rick_h_> Ergo^: give it a go and talk with them.
<deryck> Ergo^ is a friend of rick_h_?
<abentley> deryck: Currently, the mustache_model and the mustache_template from the server can be used by the javascript verbatim to render an identical result to the server-side rendering.
<abentley> deryck: This is because the mustache_model includes control variables that affect the rendering, even though they're not logically part of the model.
<deryck> abentley, ok.  are you wanting to change this where only the template can be rendered?  Or something else?
<abentley> deryck: With the new pystache, we could remove all control variables from the mustache_model, which would be cleaner, but would break this property that the JS can use the model verbatim.
<deryck> abentley, is there a downside to not using the model verbatim?
<abentley> deryck: The downside is it's a bit more work to get an identical rendering between client-side and server-side rendering.
<abentley> deryck: (but it's work we already do)
<deryck> abentley, ah, I think I see.  We just do it via variables in the model, where as we need to specify those values at the call site now. right?
<abentley> deryck: Yes, we need to inject the control variables at the call site, if they're not in the data sent by the server.
<deryck> abentley, sounds fine to me.  Seems cleaner on both ends to me.  i.e. model is cleaner and rendering is explicit.
<abentley> deryck: Cool.
<Ergo^> heh i think i should install ubuntu :/
<rick_h_> doh, forgot to put myself as ocr
* rick_h_ changed the topic of #launchpad-dev to:  http://dev.launchpad.net/ | On call reviewer: rick_h | Firefighting: - | Critical bugs: 4.0*10^2
<rick_h_> and since I am... jcsackett can I bug you please? https://code.launchpad.net/~rharding/launchpad/lpclient_fix/+merge/116338
<jcsackett> rick_h_: sure.
<rick_h_> jcsackett: ty much kind sir
<jcsackett> rick_h_: sorry, forgot to ping you when i finished reviewing your branch.
<jcsackett> it is r=me.
<rick_h_72> Thanks it's off to ec2. thanks again!
<cody-somerville> Why are some versions listed twice under 'Releases in Ubuntu'? https://launchpad.net/ubuntu/precise/+source/thunderbird
<Bert_2> wgrant: are you here ?
<abentley> rick_h_: still around?
<rick_h_> abentley: yep, what's up?
<abentley> rick_h_: Could you please review https://code.launchpad.net/~abentley/launchpad/restore-queue-test/+merge/116534 ?
<rick_h_> abentley: loading up now
<abentley> rick_h_: Thanks.
<rick_h_> abentley: what's task_init meant to do?
<rick_h_> abentley: nvm, looking
<abentley> rick_h_: It sets up a launchpad scripting environment.
<rick_h_> abentley: couple of questions (side notes) r=me ty much
<abentley> rick_h_: Thanks.'
<rick_h_> jcsackett: ping
<rick_h_> jcsackett: in your MP it says the 'helper method is removed' which I'm reading as userCanAccessUserData as being removed but it's not?
<Bert_2> Does anyone know why I'm perfectly able to log in on the main section of my launchpad instance but not on the other sections like code and answers ? I don't get any errors but after supplying info to openid I just don't get logged in, any suggestions ?
* rick_h_ changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: - | Firefighting: - | Critical bugs: 4.0*10^2
<sinzui> wgrant: StevenK: jcsackett: wallyworld: this is the non-published link of what I did today: http://youtu.be/Vq6VsKuFY_o
<wgrant> Bert_2: Did you set cookie_domains to include your new domain?
<wgrant> Bert_2: That controls how wildcardy the cookies are
<sinzui> wallyworld: you are proposing to fix 3 or more of thses bugs https://bugs.launchpad.net/launchpad/+bugs?field.tag=bug-links
<sinzui> wallyworld: http://blog.launchpad.net/general/bug-linking-part-2 "Linking existing bugs"
<Bert_2> wgrant: where can I set the cookie_domains ?
<wgrant> Bert_2: the development config inherits the default value from lib/lp/services/config/schema-lazr.conf. You can override it in your own config.
<wgrant> # Session cookies being sent to a subdomain of the parent
<wgrant> # domains listed here will be sent to the parent domain,
<wgrant> # allowing sessions to be shared between vhosts.
<wgrant> # Domains should not start with a leading '.'.
<wgrant> cookie_domains: demo.launchpad.net, staging.launchpad.net, launchpad.net, launchpad.dev
<Bert_2> so I shouldn't edit it inside schema-lazr.conf ?
<wgrant> Probably not. The configs form an inheritance hierarchy, so you can just override the key in yours
<Bert_2> so I would override from launchpad-lazr.conf in my custom configsdir ?
<wgrant> Right
<Bert_2> awesome, thx a ton :D
<wgrant> np
<sinzui> wallyworld: StevenK, wgrant: this is the one I was thinking of https://bugs.launchpad.net/launchpad/+bug/1004248
<_mup_> Bug #1004248: hook to force page refresh on context url changing is not working <javascript> <regression> <ui> <Launchpad itself:In Progress by rharding> < https://launchpad.net/bugs/1004248 >
<rick_h_> sinzui: something up with that?
<sinzui> rick_h_: I was asking if a cache issue when a bug was marked a dupe related to that bug
<sinzui> The conclusion was no
<rick_h_> sinzui: ah ok cool. Wanted to make sure there wasn't anything up since I put that to ec2 today
<Bert_2> wgrant: I just noticed that your bootstrap-db-from-scratch's bootstrap-lp-db script is missing the bug-watch-updater Celebrity, is that possible ?
<Bert_2> (I have an oops page if you need more info)
<wgrant> Bert_2: Yeah, if you look in the script where it creates the celebrities, you'll see comments instead of makeTeam/makePerson calls for some of them. I mainly just created the essential/interesting ones.
<wgrant> You can alter the script or create the team manually to resolve it.
<wgrant>     bazaar_experts = factory.makeTeam(
<wgrant>         name='bazaar-experts', displayname='Bazaar Experts', owner=admins,
<wgrant>         subscription_policy=TeamSubscriptionPolicy.RESTRICTED)
<wgrant>     # bug_importer
<wgrant>     # bug_watch_updater
<wgrant>     buildd_admin = factory.makeTeam(
<wgrant>         name='launchpad-buildd-admins',
<wgrant>         displayname='Launchpad Builder Administrators', owner=admins,
<wgrant>         subscription_policy=TeamSubscriptionPolicy.RESTRICTED)
<wgrant> Some of the missing ones may be more essential now than they were when I wrote it.
<wgrant> But the intention was that it'd get the web UI to work enough that you could create the rest if you needed them.
<Bert_2> I see, so I'm missing the bug-watch-updater person then ?
<Bert_2> it's apparently required to file a bug ;)
<wgrant> Yeah. You can run the makePerson/makeTeam calls in bin/harness to fix that up
<wgrant> In bin/harness, say eg. "factory.makePerson(name='bug-watch-updater', displayname='Bug Watch Updater', email='bugwatch@example.com')" then "transaction.commit()"
<Bert_2> if you're interested: http://pastebin.com/yFzyDNCU it was on a call to bugs.[maindomain]/[project]/+filebug
<wgrant> Ah right
<wgrant> That is indeed a relatively recent change.
<Bert_2> wgrant: why should I add it to bin/harness and not eg. make a utilities file for it and just execute that ? (just wondering)
<wgrant> Well, you can update the script to do it too, but if it's a one-off it's usually easier to do it in bin/harness
<wgrant> rick_h_: awk: cannot open build/js/yui-3.3.0/yui/yui.js (No such file or directory)
<wgrant> rick_h_: Any idea on that build error?
<wgrant> $ ls -l build/js/yui-3.3.0/
<wgrant> total 0
<wgrant> lrwxrwxrwx 1 wgrant wgrant 30 2012-07-25 09:49 yui-3.3.0 -> ../../../../build/js/yui-3.3.0
<wgrant> It's a recursive symlink..
<wgrant> rick_h_: Also, your yui buildout changes still use a predictable /tmp name
<wgrant> StevenK: Can you build a fresh tree?
<rick_h_> wgrant: yes, sorry. While we talked about it, webops didn't have feedback on that part and so only changed that which was required.
<StevenK> wgrant: My devel build a few minutes ago worked
<rick_h_> wgrant: I can make a follow up that will adjust that dir.
<StevenK> wgrant: I can clean and then do a fresh make?
<wgrant> StevenK: Try make clean_buildout build
<rick_h_> we were tossing stuff at the wall at the time
<wgrant> Heh
<StevenK> rick_h_: May I suggest you not do that with our Makefile?
<rick_h_> wgrant: added a card to move that dir in the morning
<Bert_2> wgrant: so bin/harness is meant for one-off scripts ?
<StevenK> wgrant: make clean_buildout build worked for me
<wgrant> StevenK: Hmm
<wgrant> Bert_2: It just gives you a Launchpaddy Python console
<rick_h_> StevenK: heh, the sticking to the wall was guessing what issue my change had on qas not our makefile...and it's buildout.cfg now makefile :P
<Bert_2> wgrant: I see
<StevenK> rick_h_: Even worse, because buildout loves giving obtuse error messages.
<rick_h_> StevenK: so think it's ok, will adjust the path to keep things inside of the build dir vs /tmp
<Bert_2> wgrant: just so you know, same goes for bug-importer, apparently
<wgrant> Yup
<wgrant> StevenK, rick_h_: Hm, even 'make clean build' breaks
<wgrant> Oh
<wgrant> I don't have unzip installed
<wgrant> It's nice that buildout doesn't actually notice or display the error
<StevenK> [09:54] < StevenK> rick_h_: Even worse, because buildout loves giving obtuse error messages.
<StevenK> wgrant: ^
<wgrant> StevenK: It also loves not giving any error message at all, though
<wgrant> Obtuse errors are fine :)
<Bert_2> Does anyone know what the bugtask tasks and nominations table is ?
#launchpad-dev 2012-07-25
<wgrant> Bert_2: It's the table at the top of the bug page that shows status/importance/etc.
<Bert_2> wgrant: mmm, on submitting a filled in bug my instance now errors on that
<wgrant> What's the error?
<wgrant> Could be another missing celebrity breaking a permission check
<Bert_2> here's the full oopsinfo: http://pastebin.com/dzEKFBYd
<Bert_2> it seems to be the case I cannot view bugdetails, only the table of bugs
<Bert_2> the oops I gave you is shown on displaying a specific bug
<wgrant> Ah, I think I know what that is.
<wgrant> Give me a sec
<Bert_2> take your time ;)
<wgrant> Bert_2: On https://launchpad.dev/ubuntu/+edit (or the equivalent for your domain), select 'Bugs in this project are tracked in Launchpad'
<Bert_2> wgrant: did that, and it fixed it :O
<Bert_2> so, why was it not working and why is it now ?
<wgrant> If you click on the expander icon at the left of the task table, you'll see that the Distribution widget defaults to Ubuntu
<wgrant> That's a bit of a hack -- it's hardcoded
<wgrant> The widget is configured to only show distros that track their bugs in Launchpad, so the default value of Ubuntu was illegal.
<Bert_2> the Target:Distribution:Ubuntu thing you mean ?
<wgrant> Yeah
<Bert_2> oh, I see
<Bert_2> thanks a lot man, I'm pretty sure I couldn't have found that by myself :p
<wgrant> Yeah, it was pretty obscure, but I was able to guess it right away as I've had the misfortune of running into that code before.
<wgrant> I'm fixing the script to add those essential celebrities and set that Ubuntu bit
<Bert_2> wgrant: awesome, that'll help a lot of people out in the future
<Bert_2> now as we'll be using bugs, answers and bluepints I'll let you know if I run into any more issues so the script is fully up to date ;)
<wgrant> Yup
<Bert_2> awesome :D
<Bert_2> wgrant: thanks again for your help, now I'm off to bed, bye
<wgrant> Bert_2: Night.
<StevenK> wgrant: http://pastebin.ubuntu.com/1109290/
<StevenK> wgrant: I wouldn't expect subscriber to be there for the new tests -- so I'm guessing I have something backward
<wgrant> StevenK: Will look soon
<StevenK> wgrant: Is it soon yet? :-)
<spm> StevenK: it is "Too Sooooon!"
<wgrant> StevenK: +        subscribers = BugSubscriberSet().union(
<wgrant> +            self.structural_subscribers,
<wgrant> +            self.all_pillar_owners_without_bug_supervisors,
<wgrant> +            self.all_assignees)
<wgrant> StevenK: None of those seem to be filtered
<StevenK> wgrant: I thought self.structural_subscribers would be? Although the structuralsubscription model code is messy at best
<wgrant> StevenK: Right, but the others aren't...
<StevenK> self.all_pillar_owners_without_bug_supervisors probably does not require filtering
<StevenK> self.all_assignees is currently causing me grief
<wgrant> It requires filtering
<wgrant> To /dev/null
<wgrant> That's the thing we said we were going to delete
<wgrant> On the basis that it doesn't make sense
<StevenK> Ah
<wgrant> And things that don't make any senses probably shouldn't exists
 * StevenK deletes it
<wgrant> -plurals
 * wgrant stabs the branch scanner
<wgrant> StevenK: Separate branch
<StevenK> :-(
<wgrant> Probably the same branch as the bug supervisor default sub removal
<StevenK> Which is *after* this branch. Bah.
<wgrant> Nah
<wgrant> Can be before this branch
<wgrant> Bug supervisor default structsub removal, that is
<wgrant> Not the default private_bugs direct subscription
<StevenK> wgrant: http://pastebin.ubuntu.com/1109408/ Ignoring the printf debugging, it seems like forbidden_recipients_filter and forbidden_subscribers are backwards
<wgrant> StevenK: What suggests that?
<StevenK> wgrant: My printf debugging does
<wgrant> Specific examples?
<wgrant> Ah
<wgrant> Well, yes, forbidden_subscribers is backwards
<wgrant> So I'd suggest making it not backwards
<StevenK> So forbidden_recipients_filter is backward?
<wgrant> forbidden_recipients_filter returns Storm conditions that restrict the returned subscriptions to those which have access
<StevenK> Oh, damn it, that is still keyed on BugSubscription
<StevenK> I need both BugSubscription and StructuralSubscription
<StevenK> So no wonder it returns [] :-(
<StevenK> wgrant: http://pastebin.ubuntu.com/1109443/ is the StructuralSubscription query
<wgrant> StevenK: Does it work?
<wgrant> lifeless: Is bzrlib's Graph.find_difference meant to be slow?
<wgrant> lifeless: It's taking upwards of 5 minutes to calculate for one of my LP branches. It was about 5000 mainline revs behind devel, and it chokes on the merge I did yesterday.
<lifeless> it uses _find_border_ancestors
<lifeless> which notes: This will scale with the number of uncommon ancestors.
<lifeless> wgrant: what is it doing? Can you lsprof it ?
<wgrant> lifeless: My smaller test cases are 5x faster after a pack. Trying the original case again now
<wgrant>      1039155            0      2.3077      2.3077   <method 'extend' of 'list' objects>
<wgrant> blink
<wgrant> That takes 11s unprofiled, and it's b.repository.get_graph().find_difference('william.grant@canonical.com-20120717002424-zaimmc2rnew8s0qp', 'launchpad@pqm.canonical.com-20120724215633-t9eizm9m7wlb8nwo') -- the former rev being a merge from devel a week ago, so it's only a week of revisions.
<wgrant> After packing the full 14-month diff takes 150s.
<wgrant> I'm not quite sure why calculating a 277-node graph difference requires more than a million list extends...
<lifeless> I don't, at this remove, remember which apis are fast and which are not.
<lifeless> I *think* this is meant to be fast
<StevenK> wgrant: No, which is why I'm pasting it
<wgrant> And 476s for the full diff on an unpacked repo.
<wgrant> StevenK: What doesn't it do?
<wgrant> StevenK: Note that that is finding forbidden subscribers
<StevenK> wgrant: I thought Not(Or(*get_bug_privacy...) would filter out forbiddens
<wgrant> StevenK: get_bug_privacy_filter filters bugs based on privacy
<wgrant> You use it when you want bugs that are visible to someone
<wgrant> Read it :)
<wgrant> The first clause it returns is information_type.is_in(PUBLIC_INFORMATION_TYPES)
<StevenK> wgrant: Right, so it looks like the filtering in the depths of structural subscriptions was backward
<StevenK> IntegrityError: duplicate key value violates unique constraint "accesspolicy__product__type__key"
<StevenK> DETAIL:  Key (product, type)=(27, 4) already exists.
<StevenK> :-(
<StevenK> wgrant, wallyworld: https://code.launchpad.net/~stevenk/launchpad/one-linkcve/+merge/116582
<wallyworld> StevenK: so you no longer need @export_operation_as('linkCVE')
<StevenK> Oh, duh
<wallyworld> other than that looks ok i think
<StevenK> wallyworld: And it's -15 now :-)
<StevenK> wallyworld: That change has pushed
<wallyworld> done
<StevenK> wallyworld: Thanks
<wallyworld> np
<StevenK> wgrant: Do you have a suggestion how to squeeze filtering into BugSubsciptionInfo.duplicate_subscriptions ? I've been thinking about using filter_forbidden_subscriptions, but the multiple dupes thing is tripping me up.
<adeuring> good morning
* frankban changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: frankban* (rvba) | Firefighting: - | Critical bugs: 4.0*10^2
<jml> has the font rendering on Launchpad changed recently?
<czajkowski> jml: yes I think huw as working on that
<czajkowski> dont know if it landed though
<jml> czajkowski: as in, "it's a bug and huw is fixing it", or as in, "yes, and it's a work in progress", or as in, "yes. tremble at our newly attained beauty?"
<jml> certainly looks nicer to me :)
<czajkowski> he was working on it
<czajkowski> it's not a bug
<czajkowski> and I think the code was landed and rolled out
<czajkowski> so yes praise our new beauty!
<jml> czajkowski: :)
<wgrant> stub: The sampledata changes are not noise. There are two new product columns that I added last week, and one removed from bugsummary.
<wgrant> But they were all changes made by patches, so nothing to review.
<stub> wgrant: cool.
<Ergo^> morning
<Ergo^> gmb, ping
<czajkowski> Ergo^: as I said yesterday, already, he's offline for 2 weeks.
<Ergo^> yes, jsut noticed he showed online now ;-)
<czajkowski> yes a lot of us run screens and while we're connected we may not always be here
<jelmer> czajkowski: to be fair, some launchpadders seem to have trouble with the concept of "holidays" :P
<czajkowski> jelmer: yes you and wgrant do, gmb doesn't :)
<Ergo^> seems my xchat client shows "away" statuses, with delay
<czajkowski> mpt: I have a bug for you!
<czajkowski> payback!
<mpt> czajkowski, mmmm, delicious
<czajkowski> mpt: mind if I show it to you
<czajkowski> much easier than me attempting to explain
<czajkowski> .c
<mpt> I am waiting with bated breath
<wgrant> wallyworld, rick_h_: I think your JS changes may have conflicted, as I have a failure on ec2.
<wgrant> actual    = ["Failure in /var/launchpad/test/lib/lp/bugs/javascript/tests/test_duplicates.html.lp.bugs.duplicates_tests.test_duplicate_form_submission_success: Unexpected error: Result of expression 'uri' [undefined] is not an object."]
<wallyworld> hmm. let me check
<wgrant> That's the only failure
<wallyworld> buildbot seems happy so far but i think the yui tests are run near the end
<wgrant> Yeah, this was in the last 5 minutes of the test suite
<wallyworld> bollocks
<wgrant> I have another instance 15 minutes behind that hasn't failed yet
<wgrant> Ah
<wgrant> Now it has
<wgrant> In the last two minutes
<wallyworld> i'll have to pull devel and take a look
<wallyworld> but i have something i need to attend to first
<rick_h_> wgrant: looking, yea bet it's something with the lp.client changes
<rick_h_> I renamed lp_original_uri to just uri looking at wallyworld's change to see if something there didn't agree since tests passed on my end.
<wallyworld> i didn't make any lp client changes
<rick_h_> wallyworld: right I did, and wonder if you used it in any way that broke
<rick_h_> I'm pulling and updating, will run the tests and see
<wallyworld> ok, thanks
<wallyworld> i used a couple of 'standard' methods
<rick_h_> right, nothing jumped out at me when I reviewed it yesterday, but will look
<Ergo^> http://www.happyprog.com/tdgotchi/?repost=true
<rick_h_> wallyworld: wgrant one word fix, going to just submit straight to pqm to keep from killing buildbot all day if that's cool
<wallyworld> rick_h_: go for it, thanks!
<wallyworld> rick_h_: the current build will fail to you will need to force it when it happens
<rick_h_> wallyworld: will do, thanks for the reminder. Will keep an eye on it
<wallyworld> np, thanks for fixing
* abentley changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: frankban* (rvba), abentley | Firefighting: - | Critical bugs: 4.0*10^2
<rick_h_> deryck: ping, got a quick sec?
<deryck> rick_h_, sure.  what's up?
<rick_h_> deryck: helping popey help someone with an issue using ubuntu-bug and sent me a screenshot: http://uploads.mitechie.com/lp/lp_bug_screenshot.png anything look strange that would toss an error you know of off the top of your head?
<rick_h_> deryck: wondering if () in the summary or something?
<czajkowski> rick_h_: we give rather usless error help don't we
<rick_h_> czajkowski: :) could be a bit better I suppose
<deryck> rick_h_, no, it shouldn't choke over the parens.
<deryck> rick_h_, there's some required field or something not displayed.  I wonder if something in extra options was set incorrectly somehow?
<deryck> rick_h_, or some field with bad data.
<rick_h_> deryck: ok, I'll see if I can get some more info and maybe talk him into submitting it manually to make sure that works
<deryck> rick_h_, or maybe he already filed a bug against the token and it's no longer valid? The random number thing in the url is what links to the apport filed data.  Maybe something there.
<rick_h_> deryck: yea, this was an ubuntu-bug ... filing
<Ergo^> rick_h_, change that message to Error: Success, that will teach them
<rick_h_> deryck: ok cool, looks like a bad tag name: rc-6.0.0-0ubuntu5~test1
<deryck> oh dang, standup time.
<deryck> rick_h_, abentley, adeuring -- coming, sorry.  forgot time.
<rick_h_> deryck: rgr
<deryck> adeuring, https://plus.google.com/hangouts/_/6e08b518c64d571c487a6e9da883b81733756c2d?authuser=0&hl=en
<abentley> adeuring: Why does test_run_missing_ready_does_not_return_results call a script instead of running the code in-process?
<adeuring> abentley: I can't remember exactly... Might be related to config dteails.
<rick_h_> abentley: can you peek at https://code.launchpad.net/~rharding/launchpad/no_more_tmp/+merge/116663 when you get a sec, short one
<abentley> rick_h_: You're creating a directory named ${buildout:yui-directory}/yui-${:yui_version}.zip ?
<rick_h_> abentley: yea, because I copy out to yui-${:yui_version}
<rick_h_> it was either that or create a tmp directory in there to extract/copy out of
<rick_h_> and that just seemed like creating a dir for no good reason
<abentley> Does this avoid creating a directory? It sounds like it would just be a different name if you created a tmp directory.
<rick_h_> abentley: right, this avoids creating the directory and extracting to it vs just extracting straight to yui-version.zip as a dir
<rick_h_> it's extracting the yui-version.zip from source-deps into build/js/ and then pulling only the build (no tests, docs, etc) from that zip into build/js/yui-3.3.0
<abentley> rick_h_: I find extensions in directory names surprising.  Would ${buildout:yui-directory}/yui-${:yui_version}-extracted or ${buildout:yui-directory}/yui-${:yui_version}-tmp or something work for you?
<rick_h_> abentley: sure thing, all the same to me.
<rick_h_> will update, thanks
<abentley> rick_h_: r=me.
<sinzui> mrevell: do you have time to talk
<mrevell> sinzui, I think I'm just about to going into a meeting . I'm in the new London office.
<sinzui> okay. If you have time today. ping me for a short talk about my anxieties.
 * czajkowski hugs sinzui 
<czajkowski> sinzui: read the warthogs mail
<czajkowski> and did laugh
<czajkowski> worst holiday from hell?
<sinzui> My in-laws invited me to the same beach for another week of near-death-experiences at the end of August
<czajkowski> sinzui: will you be going ?
<sinzui> I am uncertain. My wife and kids will
<jcsackett> sinzui: avoid the beach bugs.
<sinzui> czajkowski: : this is the non-published link of what I did yesterday. I gave up trying to get the flashes out of the video since it is a known openshot tansition bug: http://youtu.be/Vq6VsKuFY_o
<sinzui> jcsackett: since Anne did not select the house, there is no guarantee there will be internet.
<czajkowski> sinzui: cheers
<sinzui> I guess I could take the car and drive 2.5 hours to get to that Chinese restaurant
<czajkowski> sinzui: or... you could tkae a holiday
<czajkowski> dear purple squad you need to learn how to take off your holidays properly
<jcsackett> czajkowski: in addition to his many allergies, sinzui has a fatal allergy to not working on launchpad.
<czajkowski> jcsackett: so does wgrant
<sinzui> My meditation is TDD. Zen and the Art of Test Driven Development. I don't have to work on Lp, but I need to do something.
<mrevell> sinzui, Hey, I have 15 minutes now or longer than that in 90 minutes. Which works best for you?
<czajkowski> sinzui: arduinos ?
<jcsackett> oh, yeah. we should get him started on that. i need to know more people doing arduino hacking.
<jcsackett> sinzui: can i grab you after you're done talking to mrevell?
<sinzui> mrevell: now would be fine
<mrevell> sinzui, I'll grab a room
<sinzui> mrevell: https://plus.google.com/hangouts/_/410f6d1a701063146f8bb2bc46e26dfa04cd2539?authuser=0&hl=en when you are ready
<deryck> rick_h_, hey, not sure if it's inviting you:  https://plus.google.com/hangouts/_/73a8173804e27710a1abf877e26aefe3848f70d3?authuser=0&hl=en
<abentley> frankban: Could you please review https://code.launchpad.net/~abentley/launchpad/enable-tests/+merge/116678 ?
<frankban> sure abentley
<sinzui> mrevell: http://youtu.be/Vq6VsKuFY_o
<deryck> rick_h_, abentley, adeuring -- I'm playing with G+ events to get a named hangout for our standup, so forgive if you get event invite spammed now.
<abentley> deryck: np
<abentley> sinzui: do you have a minute to talk about https://bugs.launchpad.net/launchpad/+bug/918745 ?
<_mup_> Bug #918745: PersonMergeJob sends emails to people who do not care <email> <merge-deactivate> <teams> <Launchpad itself:Triaged> < https://launchpad.net/bugs/918745 >
<sinzui> I do
<abentley> sinzui: I don't understand your comment.  Why would the target of the merge care if the job failed, unless the target of the merge requested the job?
<sinzui> abentley: this happens when the user deletes the team, they do not care about it, and there is nothing they can do to fix it.
<sinzui> abentley: we can send the email to the person who requested the deletion. It will delay us from fixing the data in the team
<sinzui> abentley: we fixed the most common cause last week. We do not allow people to delete teams with private branches because they cannot be given to any other team
<abentley> sinzui: If this error is not meant to be sent to users, I think we should let the normal oops mechanism handle it.
<sinzui> abentley: that is fine. the user will just create a question for us to look up and then fix the data as we do when ~registry gets the email
<sinzui> Your change is good because it tells the user the delete failed
<abentley> sinzui: If we want to send email to a celebrity, that could make sense.  But is the target guaranteed to be ~registry, or could it be a normal user/team?
<sinzui> abentley: The email went tot the target of the merge. We get the oopses where ~registry is missing the proper BVP. Other teams get them when the change is a merge. Remember that delete is always a merge into ~registry because there are undeletable artrefacts.
<abentley> sinzui: Okay, I think I'll go ahead with the change.
<sinzui> okay
<sinzui> abentley: you want to send the email to the user that scheduled the job?
<abentley> sinzui: Yes.
<sinzui> fab
<abentley> rvba: I don't know what value is best, either.  In most cases, we don't hit the sleep at all.
<abentley> rvba: So maybe it doesn't matter.
<rvba> abentley: ok then.
<abentley> rvba: But in cases where we race, I guess it doesn't hurt to do a fraction of a second.  The processor has nothing better to do :-)
<rvba> abentley: that was my point :)
<abentley> rvba: Okay, I'll change the sleep to (0.1) and the range to 100.  Cool?
<rvba> abentley: cool.
<abentley> rvba: (while I didn't write sleep(1) it in *this* branch, I wrote it yesterday in another branch :-)
<Bert_2> Hi, blueprintslists in my instance of launchpad come in pages of 5, launchpad.net has pages of 20, I seem to be unable to find any mentioning of a limit in the templates, does anyone know where this number is set ?
<sinzui> development launchpad defaults to 5 items per list. This allows us to see batching work with our small sampledata size
<Bert_2> sinzui: and how can it be changed, cause I can't really seem to find a place to change the 5 to 20 or more
<sinzui> it is defined in the schema.
<Bert_2> schema-lazr.conf ?
<sinzui> It can be redefined in  configs/development/launchpad-lazr.conf
<Bert_2> what is the option called, cause I don't see any option with a 5 in schema-lazr.conf that have to do with it, I think ?
* frankban changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: abentley | Firefighting: - | Critical bugs: 4.0*10^2
<sinzui> I don't know the name of the key, but it is defined in schema-lazr.conf. Search for 5
<Bert_2> I just did
<Bert_2> the only ones with just five are announcement_batch_size and bugnotification_interval
<Bert_2> and consecutive_failure_limit
<Bert_2> those don't seem to have to do with blueprints :S
<sinzui> As I said, this is about batching, not applications
<sinzui> Change the value in configs/development/launchpad-lazr.conf as I suggested
<Bert_2> so announcement_batch_size is the one ?
<Bert_2> sinzui: I'm not using development
<Bert_2> I have my own configdir
<sinzui> Bert_2: This channel is for launchpad development. There no channel for support, the code is unsupported
<Bert_2> sinzui: I have supplied multiple additions to the bootstrap-db-from-scratch script, I'm not just here to get support
<Bert_2> got it, thx anyway sinzui
<rick_h_> abentley: celery test failed on buildbot can you dbl check please?
<abentley> rick_h_: sure.
<abentley> rick_h_: The fix in r15684 was broken.  It's corrected in r15688.
<jono> is there an issue with LP code hosting - I am trying to set trunk to my branch and it is rejecting it
<rick_h_> abentley ok, will force another round
<jono> the branch is lp:~jonobacon/ubuntu-accomplishments-system/validation-service
<abentley> rick_h_: thanks for letting me know, though.
<rick_h_> abentley: started up aonther build then, look slike 15688 was merged so here we go again wheee :)
<sinzui> lifeless: do you have a few minutes to discuss/listen to a story about the monsters in the forest of sharing
<lifeless> I do
<lifeless> sinzui: skype? G+ ?
<sinzui> skype I think
<lifeless> signed in .... now
<wgrant> abentley: I'm rolling back the celery changes, as in 15688 both tests have failed
<wallyworld> sinzui: we finished the standup. all good, no-one is blocked etc
<cjohnston> hello
<czajkowski> sinzui:  can you talk to cjohnston he has some questions over your recent blog post about project maintainers and private buts
<czajkowski> *bugs
<czajkowski> man I am knackered
 * cjohnston thinks its bed time ;-) 
<cjohnston> and that may go for me too
<cjohnston> lol
<sinzui> wgrant: StevenK: look at this
<sinzui> https://bugs.launchpad.net/launchpad/+bug/1029093
<_mup_> Bug #1029093: Can't change the information type of a bug that has dupes filed on deactivated project  <disclosure> <sharing> <Launchpad itself:Triaged> < https://launchpad.net/bugs/1029093 >
<czajkowski> cjohnston: I suggest asking the question :)
<cjohnston> I was waiting to see if he responded (or atleast now that I see that he is here) ;-)
<cjohnston> sinzui: I'm a little confused about marking bugs private.. From the help pages for LP, it says that projects can set all bugs by default to be private. I tried implementing this setting and wasn't allowed to because I don't have a commercial licence.. if nothing else, the text is confusing
<cjohnston> https://help.launchpad.net/Bugs/PrivateBugs#Private_bugs
<wgrant> cjohnston: Yeah, all the documentation around privacy is pretty terrible.
<wgrant> We're almost done completely reworking it (including replacing the default private bugs checkbox), after which the docs can be rewritten
<cjohnston> wgrant: quick pm?
<wgrant> cjohnston: Sure
<StevenK> wgrant: http://pastebin.ubuntu.com/1111033/ is the dupe query
<czajkowski> wgrant: cjohnston is an unsual area, the project is for canonical but he's a contractor
<wgrant> czajkowski: Yeah
<wgrant> cjohnston: Might be best to discuss this in #launchpad-ops on irc.c.c
<czajkowski> indeed
#launchpad-dev 2012-07-26
<StevenK> wgrant: Bleh, I can't pass arguments to a property :-(
<lifeless> sure you can if it curries
<StevenK> If it what?
<wgrant> StevenK: Why are you trying to pass arguments to a property?
<spm> chicken mango curries. of course.
<wgrant> StevenK: Did you get around to removing the default maintainer subscription in a separate branch, or should I do that now?
<StevenK> wgrant: I haven't, no.
<StevenK> wgrant: forbidden_recipients_filter is a property, but for all_assignees it needs to not link against BugSubscription
<wgrant> StevenK: Well
<wgrant> StevenK: It could not be a property.
<StevenK> Indeed
<wallyworld> great, looks like my credit card has been hacked :-(
<cody-somerville> I love chicken mango curry
<bigjools> wallyworld: oops. Dodgy card reader in a restaurant?
<wallyworld> not sure. i got unauthorised charges from Crazy Domains (who I have a domain name with) and EuroDns (who I have not heard of before)
<bigjools> I'd be looking at incompetence rather than malicious hacking then
<wallyworld> yeah, you're probably right
<wallyworld> but i'm still pissed off and if it were just Crazy Domains I'd be less unsure. it's happened before to me with charges from India worth $1000s
<bigjools> dodgy call centres :)
<lifeless> wallyworld: stop calling the sex lines then ?
<wallyworld> lifeless: i will if you will
<lifeless> I'm not the one with credit card issues :P
<lifeless> but sure, why not :)
<wallyworld> ha
<StevenK> I just had a glance at my credit card statement - Steam, Woolworths, pizza resturant, Steam, Steam, Woolworths ...
<StevenK> Damn Summer sale
<wallyworld> don't know where you get time to play all those games
<StevenK> wallyworld: Not all of the Steam purchases were me, Sarah is a secondary card holder
<wallyworld> no wonder you don't have kids yet
<StevenK> Ouch
<wallyworld> sorry :-P
<wgrant> Bah
<wgrant> Project groups, we meet again
<wgrant> And you ruin all my plans, again
<StevenK> Hahaha
<StevenK> wallyworld_, wgrant: https://code.launchpad.net/~stevenk/launchpad/structsub-private-bugs-redux/+merge/116798
<wallyworld_> StevenK: probably best that wgrant looks at that one
<wallyworld_> i don't feel i have the necessary background knowledge not to miss anything
<StevenK> Hahaha
<StevenK> WCPGW
<StevenK> Don't answer that :-)
<wallyworld_> StevenK: yes, the potential for leaking info scares me in this case
<wgrant> StevenK: Will look shortly
<wgrant> StevenK: Are you trying to poison me?
<StevenK> wgrant: With which bit?
<wgrant> 261	Not(In(BugSubscription.person_id,
<wgrant> 262	- Select(BugMute.person_id, BugMute.bug_id == self.bug.id))))
<wgrant> 263	+ Select(BugMute.person_id, BugMute.bug_id == self.bug.id))))
<wgrant> 288	+ filters = [BugSubscription.bug_notification_level >= self.level,
<wgrant> 289	+ BugSubscription.bug_id == Bug.id,
<wgrant> 290	+ Bug.duplicateof == self.bug,
<wgrant> The indentation doesn't really come through here
<wgrant> But it's near-fatal
<StevenK> wgrant: 260 and such are not my fault
<wgrant> The case at 260 was a grey area, but you took it into a red one :)
<wgrant> It's now indented by three (3) spaces
<wgrant> And it's less indented than the previous line of the expression, which is at the same level of nesting
<StevenK> I've reverted that bit
<StevenK> wgrant: http://pastebin.ubuntu.com/1111457/
<wgrant> I think I love you.
<StevenK> Now now
<wgrant> StevenK: I'll finish the review tomorrow, if that's OK
<wgrant> We can't deploy it before Monday, anyway
<wgrant> Since deploying another bugmail privacy change on Friday afternoon comes up as "fuckno" on the 0-10 scale of good deployment ideas.
<StevenK> wgrant: I'll commit and push that one bit
<wgrant> Thanks.
<wgrant> Since I'm sure you recall what happened in March...
<StevenK> I try not to.
<wgrant> Speaking of deploying.
<wgrant> Let's deploy.
<wgrant> StevenK: I can't see any glaring issues, but I'll want to go over it in more detail and test a few things.
<wgrant> Particularly the test changes.
<wgrant> But it looks good so far
<StevenK> wgrant: I'm wary about not filtering direct subscriptions, but doing so breaks the world.
<StevenK> wgrant: You probably want to claim the review lest sinzui swoop in and review it while you sleep.
<wgrant> I was just thinking that.
<adeuring> good morning
<mpt> Thanks to whoever fixed the "Report a bug" link :-)
<StevenK> mpt: Rick did.
<jml> \o/
* jcsackett changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: jcsackett | Firefighting: - | Critical bugs: 4.0*10^2
<adeuring> jcsackett: could you please review this MP: https://code.launchpad.net/~adeuring/launchpad/bug-1020443-2/+merge/116876 ? (Follow-up for the one you reviewed last week.)
<jcsackett> adeuring: looking now.
<adeuring> thanks!
<deryck> adeuring, hey there.  Good for a call now?
<adeuring> deryck: sure. mumble or skype?
<deryck> adeuring, I think G+ is working now.  Try:  http://tinyurl.com/orange-standup
<adeuring> ok
<jcsackett> adeuring: r=me, but there are two small issues you'll want to double check before landing.
<jcsackett> they are in the comment on the MP.
<adeuring> jcsackett: thanks!
<adeuring> jcsackett: good spot, these two issues!
<abentley> jcsackett: Could you please review https://code.launchpad.net/~abentley/lp-dev-utils/ec2-errors/+merge/116891 ?
<jcsackett> abentley: when you say user error, what do you mean?
<jcsackett> what sort of user error in ec2 is this concerned with?
<abentley> jcsackett: I mean the user made an error.  The one I ran into was requesting a test run when I hadn't pushed my changes, i.e. PublicBranchOutOfDate
<jcsackett> so, with this, if you call ec2 with out pushing changes, it would no longer exit?
<abentley> jcsackett: Sure it would.  See the sys.exit on 21?
<jcsackett> ah, my mistake, i misread indentation.
<jcsackett> r=me, abentley.
<abentley> jcsackett: ty.
<dobey> how often do oops get synced?
<czajkowski> dobey: what do you mean
<dobey> the lp-oops page is telling me it can't find an oops id from a few hours ago
<czajkowski> dobey: matsubara-afk might know more
<czajkowski> he'll be back shortly
<sinzui> jcsackett: do you have a few minutes
<sinzui> well minutes to talk
<jcsackett> sinzui: sure.
<sinzui> jcsackett: I started a hangout, My phone thinks it is calling you
<lifeless> sinzui: hi
<lifeless> sinzui: little late in checking in, sorry.
<sinzui> lifeless: I have nothing to report at the moment. I offered to review how hwe was targeting bugs to multiple pillars to find an alternate bug linking strategy.
<lifeless> kk
<sinzui> I am decided to try to fix a bug, but now find that i am totally disgusted by zope schema, widgets, templates, and Lp form layout. I just want to run away screaming
<lifeless> welcome to my world! Want to join TA? :)
<sinzui> I think I will pass today. I think deleting the archaic title attribute on product will at least let test forms quickly
<lifeless> dobey: use the new oops system.
<lifeless> dobey: and update your bookmarks.
<lifeless> dobey: the automatic links LP makes should work fine.
<lifeless> czajkowski: see my mail to canonical-launchpad a few weeks back for context.
<czajkowski> i pointed that out
<czajkowski> lifeless i tend to star yiur mails. usually stuff i need to know or reference :)
<lifeless> heh :)
<dobey> lifeless: what's the new system? i don't see a link in the html body
<lifeless> dobey: what html body ?
<dobey> lifeless: the one for the 503 from the api server that gets logged. there's an x-lazr-oopsid in the http headers though
<lifeless> dobey: perhaps a little more context will help, I may be telling your redundant or incorrect things.
<lifeless> dobey: ah righto, so - if you were to file a bug on LP.
<lifeless> you'd get the OOPS id linkified to link through to oops.canonical.com
<dobey> we're seeing a lot of 503 errors in one of our tarmac instances (not sure why others aren't getting it)
<lifeless> dobey: most oopses are delivered in realtime, event based, no cron involved.
<dobey> ah ok
<lifeless> oops.c.c is the new multi-team server
<lifeless> it has ca, lp, ops oopses and more planned
<dobey> maybe lp-oops should redirect?
<lifeless> it has unmigrated oopses
<lifeless> we'll turn it off once they lose all relevancy.
<dobey> ah
<lifeless> this was announced, but apparently not on a wide enough net.
<dobey> status: Doomed
<dobey> nice. :)
<dobey> looks like a db timeout poking landing_candidates for this one branch :-/
<lifeless> whats the oops ID ?
<dobey> 61c47ad2db8192800bc7f2df619a7c64
<lifeless> dobey: thanks. Future ref - the OOPS- is part of the id (distinguishes it from a mere hash :))
<lifeless> the oops.c.c web UI has special code to try for OOPS-$1 if a lookup fails.
<lifeless> dobey: https://oops.canonical.com/?oopsid=OOPS-61c47ad2db8192800bc7f2df619a7c64#repeatedstatements is the interesting bit to me: 177 separate branch lookups.
<dobey> right
<lifeless> the long sql statements output is bong there.
 * lifeless files a bug
<lifeless> bug 1029637
<_mup_> Bug #1029637: incorrect 'long sql statements' in an oops <python-oops-tools:Triaged> < https://launchpad.net/bugs/1029637 >
<lifeless> dobey: anyhow, the landing candidates collection isn't eager loaded for security rules.
<lifeless> So its going to perform terribly slowly.
<lifeless> dobey: this may be a regression due to privacy, or it may be that privacy has had no impact and your data set has just tipped over the edge.
<lifeless> I can't tell.
<dobey> yeah, not sure. though we do have a fairly large data set of course :)
<lifeless> dobey: bug 1029642 for you.
<_mup_> Bug #1029642: ScopedCollection:CollectionResource:#branch_merge_proposal-page-resource (landing candidates) dying from late evaluation of security rules <Launchpad itself:Triaged> < https://launchpad.net/bugs/1029642 >
<lifeless> dobey: its only dealing with one batch AFAICT - 75 rows.
<dobey> thanks
<lifeless> I suspect its more a regression that not, but I'd need to check the code quite closely to actually tell.
<lifeless> regardless, its critical either way.
<dobey> yeah, it's preventing branches from landing
<lifeless> when did it last work ?
<dobey> yesterday or day before i think; though not sure. i have seen a couple 503s happen on and off, but not sure if they're all related
<dobey> i take that back
<dobey> it *just* worked
<dobey> but it seems we get intermittent 503 errors; much more frequently today it seems
* jcsackett changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: - | Firefighting: - | Critical bugs: 4.0*10^2
<YokoZar> sinzui: poke :)  (courtesy of czajkowski)
<sinzui> hi YokoZar
<YokoZar> I do have a question actually
<YokoZar> Is it possible to have a private project / series?  Or just private bugs/PPAs?
<sinzui> YokoZar: the latter with the addition of branches
<sinzui> YokoZar: Private-projects is in planning now Work will start in a few weeks
<YokoZar> sinzui: in the meantime I suppose I can invent some cool code names
<sinzui> YokoZar: yes, that is good practice. avoid dates if your dates can be linked to other public data about what you are doing
<sinzui> YokoZar: Every project will have the option of going truly private when the private-project feature enters beta
<YokoZar> Cool
<YokoZar> sinzui: is renaming a series possible?
<sinzui> yes, the maintainer or series release manager can rename a series using the Change details link on the series page
<sinzui> name == the name in the url and is used on other pages
<sinzui> wgrant: https://bugs.launchpad.net/launchpad/+bug/615604
<_mup_> Bug #615604: +filebug on project group asks for product too early <filebug> <lp-bugs> <Launchpad itself:Triaged> < https://launchpad.net/bugs/615604 >
<lifeless> http://hueniverse.com/2012/07/oauth-2-0-and-the-road-to-hell/
<sinzui> I sympathise
#launchpad-dev 2012-07-27
<wallyworld> wgrant: StevenK: local search removal was "inadvertent" http://www.androidcentral.com/international-galaxy-s3-local-search-removal-inadvertent-says-samsung-patch-coming
<StevenK> Hah
<wallyworld> at least the UK courts have some sense. i can't wait to see those Apple "We are sorry" ads they will be forced to run
<wgrant> :)
<StevenK> wallyworld: Have you read The Rainmaker by John Grisham?
<wallyworld> StevenK: no. what;s it about?
<StevenK> A lawyer, effectively.
<wallyworld> of course, stuoid question :-)
<StevenK> wallyworld: I can just draw a few parallels between the court battle in The Rainmaker and Samsung v Apple in the US and the UK
<wallyworld> i for the life of me will never understand how Apple has got away with all their shit in the US. i mean wtf. how did things get so broken
<wallyworld> they must have photos of judges or something
<StevenK> Or the judges are Apple fans.
<StevenK> "The American company MUST win, the Koreans obviously stole everything." etc
<wallyworld> yeah :-( But their Korean CEO wasn't quoted as saying "Good artists copy, great artists steal" or whatever it was
<StevenK> Oh my god, can I delete lib/lp/bugs/doc/bugnotification-sending.txt for being utterly disgusting?
<wallyworld> only if you convert to unit tests :-P
<StevenK> wallyworld: Read through it and then say that with a straight face.
<wallyworld> but then how could I push your buttons
<wgrant> StevenK: I think that was the first LP test I touched
<wgrant> So it has a special place in my heart.
<wgrant> Kill.
<StevenK> wgrant: I've stopped it failing at least.
<StevenK> 12 files changed, 21 insertions(+), 299 deletions(-)
<wgrant> Yeah
<wgrant> I changed most of that test
<wgrant> Well, reordered
<wgrant> Since my first LP change caused the order to no longer be undefined
<StevenK> 1407 lib/lp/bugs/doc/bugnotification-sending.txt
<StevenK> Holy crap
<wgrant> $ bzr cdiff -c8976 'lib/lp/bugs/doc/bugnotification-sending.txt' | wc -l
<wgrant> 1060
<StevenK> wgrant: So I can bzr rm it from the tree for making peoples lives miserable?
<wgrant> Heh
<StevenK> wallyworld, wgrant: https://code.launchpad.net/~stevenk/launchpad/no-implicit-maintainer-sub/+merge/116991
<wallyworld> looking
<wgrant> StevenK: +21? What is this treachery?
<StevenK> I've got a locksmith turning up in about five
<wallyworld> StevenK: couldn't you open the handcuffs? you still attached to the bed?
<StevenK> wallyworld: Hah
<StevenK> wallyworld: The deadbolt on the front door stopped working on Tuesday. The bolt is fully retracted and no amount of fiddling has made it come out.
<wgrant> StevenK: 521	-Foo Bar and Sample Person got subscribed to the bug. Foo Bar got
<wgrant> 522	-subscribed "implicitly", because he's the product owner.
 * wallyworld bites his tongue
<wgrant> 523	+Foo Bar got subscribed to the bug.
<wgrant> StevenK: Was the original text the wrong way around?
<StevenK> wgrant: I have NFI
<StevenK> I made a guess
<StevenK> wallyworld: Oh?
<wgrant> The code changes look correct, but that test change seems the wrong way around. I guess if it passes it doesn't matter...
<wallyworld> StevenK: don't worry. my mind is full of things which are best left unsaid
<StevenK> Is foo.bar Foo Bar or Sample Person?
<wallyworld> should that line say "Sample Person ..."
<wallyworld> since it previously said Foo Bar was the one implicitly subscribed
<wgrant> StevenK: Hm. I am confuse
<wgrant> StevenK: I think that code might be separate...
<wgrant> StevenK: Observe that the same logic is present in get_also_notified_subscribers
<wgrant> Maybe that's just to add the reason.
<StevenK> if pillar.owner in also_notified_subscribers:
<wgrant> Yeah
<wgrant> So kill that block
<wgrant> And addRegistrant
<StevenK> Right
<wgrant> And addRegistrant's tests
<StevenK> MOAR KILLING
<wgrant>     def addMaintainer(self, person):
<wgrant>         """Registers a maintainer of a bugtask's pillar of this bug."""
<wgrant> Looks suspicious as well, but I don't think it's actually used
<wgrant> wtf
<wgrant> wtf
<wgrant> wtf
<wgrant> lib/lp/bugs/subscribers/bug.py
<wgrant> search for addMaintainer
<wgrant> I've often wondered why that strangeness happened
<wgrant> Now I know
<wgrant> I'll a card for reworking that
<wgrant> Or a bug
<wgrant> It's separate code from what you're working on
<wgrant> But should be reexamined in light of private structsubs
 * StevenK watches the four bugnotificationrecipients.txt fall over in a heap of fail
<wgrant> wallyworld, StevenK: Do you see any reason to keep the maintainer/bugsupervisor section of add_bug_change_notifications? I'm pretty sure it becomes irrelevant once structsubs work
<wallyworld> wgrant: i think it can go
<wgrant> StevenK: addSecurityContact isn't used or tested
<wgrant> addBugSupervisor is used, but only in the condemned code like addMaintainer is
<StevenK> wgrant: I'm to kill addMaintainer, or is that for later?
<wgrant> StevenK: This branch is big enough. Kill addSecurityContact since it's unused, and we'll investigate addMaintainer/addBugSupervisor's single callsite later
<wgrant> "later" may mean after lunch
<StevenK> wgrant: add{Registrant,SecurityContact}()'s killing has been pushed
<wgrant> BugNotificationRecipients is going to be like 2 lines by the time you're done
<StevenK> This sounds like an excellent plan
<StevenK> wgrant, wallyworld: Diff updated
<wgrant> StevenK: Thanks
<wgrant> I've verified that the suspicious changes were just a bug in the old test.
<wgrant> r=me
<StevenK> This is my surprised face.
<StevenK> wgrant: no-qa or is there a bug for this?
<wgrant> Now now
<wgrant> Doctests are people too
<wallyworld> this? :O
<wgrant> StevenK: There may not be a bug, but there should be
<StevenK> Let me file one and create a card
<wgrant> ie. if there isn't, file
<wgrant> I thought there was one, but I can't find it. So file a new one :)
<wgrant> StevenK: Bug #483521 is relevant to the next phase of this
<_mup_> Bug #483521: Allow the project owner to appoint other people as the bug supervisor <lp-bugs> <supervisor> <Launchpad itself:Triaged> < https://launchpad.net/bugs/483521 >
<StevenK> Bug #113825 could also be
<_mup_> Bug #113825: Null bug supervisor is confusing <confusing-ui> <disclosure> <lp-bugs> <sharing> <Launchpad itself:Triaged> < https://launchpad.net/bugs/113825 >
<wgrant> StevenK: Sort of
<wgrant> But not directly.
 * StevenK files a new bug
<StevenK> wgrant: Bug #1029725
<_mup_> Bug #1029725: Pillar owners should not be implicity notified about bugs <bugs> <disclosure> <Launchpad itself:Triaged> < https://launchpad.net/bugs/1029725 >
<wallyworld> wgrant: as far as i can tell, there's no api to get a bug by id honouring the privacy rules. i can add a bug_id parameter to BugTaskSearchParams. do you have an issue with this?
<wgrant> StevenK: Bug #1029724
<_mup_> Bug #1029724: Retargeting a private bug notifies maintainer and bug supervisor without subscriptions <disclosure> <email> <sharing> <Launchpad itself:Triaged> < https://launchpad.net/bugs/1029724 >
<wgrant> wallyworld: bug= should work
<wgrant> It's meant to cope with an ID too, I'm pretty sure
<wallyworld> ok, i didn't realise that supported taking an id
<wallyworld> will check
<wgrant>     standard_args = {                                                                                                                                    |                self.distroseries = distroseries
<wgrant>         BugTaskFlat.bug: params.bug,
<StevenK> wgrant: Hm.
<wgrant> It'll compare it with the Storm Reference
<StevenK> wgrant: Both of them?
<wgrant> Which should work for an ID
<wgrant> StevenK: ?
<StevenK> wgrant: Oh, is the 724 the upcoming work?
<wgrant> StevenK: Yeah, 724 is the next bit
<wgrant> 724 is about removing the implicit, hardcoded, un-opt-outable subscription
<wgrant> 521 is part of removing the automatic structural subscription given to bugsupervisors
 * StevenK notes Coding has been wallyworld'd
<wallyworld> StevenK: i have a branch which fixes 4 bugs
<wgrant> Hm
<wgrant> I think that's all of it
<wgrant> After fixing those two, you'll only be notified about a project's bugs if you ask to be notified about a project's bugs :)
<wgrant> A revolutionary concept, I know
<StevenK> wgrant: Tossing it at ec2.
<wgrant> Great.
 * wgrant wanders off for a while.
<StevenK> wgrant: Can haz pre-impl? Or a review?
<wgrant> As for preimp... you basically just need to copy the status/importance stuff
<wgrant> It's identical
<StevenK> wgrant: Where does that get stuffed into the DB?
<wgrant>     TABLE "bugsubscriptionfilterstatus" CONSTRAINT "bugsubscriptionfilterstatus_filter_fkey" FOREIGN KEY (filter) REFERENCES bugsubscriptionfilter(id)
<StevenK> Right
<StevenK> Rargh, it's a new table, not even a column :-(
<wgrant> Right
<wgrant> Since it's many-to-many
<StevenK> wgrant: Uhhhh. lib/lp/archiveuploader/tests/nascentupload-closing-bugs.txt fails with the death, murder and destruction we did this morning.
<wgrant> StevenK: Failure is expected, but in what manner?
<StevenK> wgrant: http://pastebin.ubuntu.com/1113149/
<wgrant> StevenK: Right, is there anything odd in the slightest about that?
<wgrant> Seems completely expected
<wgrant> You removed a notification mechanism
<wgrant> And now some notifications aren't happening.
<lifeless> wgrant: StevenK: https://code.launchpad.net/~lifeless/python-oops-tools/report-recipient/+merge/116999 if you want less mail.
<wgrant> lifeless: r=me
<StevenK> wgrant: Review?
<wgrant> StevenK: I'm trying to break it
<wgrant> It's not going very well
<StevenK> Hahaha
<StevenK> I guess this means I am smart enough to write it
<wgrant> I need more monitors
<StevenK> Hah
<StevenK> wgrant: I have the DB patch up
<wgrant> StevenK: Is it as immense as I expect?
<StevenK> +24
<wgrant> Looks good
<wgrant> StevenK: How's no-implicit-maintainer-sub doing in EC2?
<StevenK> no-implicit-maintainer-sub => devel       [FAILED]   (up for 3:19:34) i-96b9b5ee
<StevenK> nascentupload-closing-bugs.txt is the only failure
<wgrant> Excellent
<StevenK> So I guess I can lp-land it in about an hour
<wgrant> StevenK: r not quite equals me
<StevenK> wgrant: Oh? Why?
<StevenK> wgrant: assignees behaves differently depending if there is a bugtask or not, I wanted to support both.
<wgrant> StevenK: Which bit's that about?
<StevenK> Why have you split the subquery out into a separate local? Might as well inline it, or at least not shadow a builtin. Also, did you consider inlining the subsequent filter into load_people's where argument?
<StevenK> I considered the Or/True thing for whatever I rename forbidden_recipients_filter to, so I'll do that.
<wgrant> Ah
<StevenK> wgrant: no-implicit-maintainer-sub is in r15699
<wgrant> So I saw
<poolie> lifeless, hi
<poolie> hi all
<wgrant> Hi poolie.
<poolie> well done on the privacy and other recent changes
<spm> hey poolie
<lifeless> poolie: hi
<StevenK> poolie: O hai
 * StevenK steals r15700
<adeuring> good morning
<czajkowski> hey adeuring
<adeuring> hi czajkowski!
<jml> gary_poster, benji (others welcome): we're looking at switching our dependency management to buildout and were wondering if you could help us get started.
* adeuring changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: adeuring | Firefighting: - | Critical bugs: 4.0*10^2
<czajkowski> jcsackett: ping me when you're online please :)
<benji> jml: I am about to be AFK for an hour and then I have calls and interviews for much of the day (similar for Gary) but I would love to help you with buildout if I can.
<jml> benji: thanks!
<jml> benji: https://code.launchpad.net/~james-w/pkgme-service-python/buildout/+merge/116980 has a summary of where we're at
<benji> cool, I'll take a look when I have a spare moment; is there anything in particular you need added next or are you looking more for suggestions with what you have?
<jml> benji: wondering if there are known workarounds for https://bugs.launchpad.net/zc.buildout/+bug/1029715 and possibly some direction in fixing it.
<_mup_> Bug #1029715: bootstrap.py fails if there is a system-wide install of the desired buildout version <Buildout:New> < https://launchpad.net/bugs/1029715 >
<jml> benji: also confirmation of our approach in sharing eggs
<benji> sounds good, I'll take a look when I can
<jml> benji: and of course we'd welcome any feedback from you & gary since you've got way more experience with this and will probably have suggestions on things we wouldn't think to ask about.
<jml> benji: thanks.
<Brooklyn> How can I configure launchpad to send notifications(email)?  Any hints?
<gary_poster> jml, hi.  I will be more than happy to leave that to benji's capable hands, but also will be happy to help later if needed.
<sinzui> jcsackett: can you investigate why comment hiding is broken on a bug https://bugs.launchpad.net/launchpad/+bug/1029668
<_mup_> Bug #1029668: Hide Comment works sometimes, doesn't work other times <comments> <disclosure> <information-type> <javascript> <regression> <Launchpad itself:Triaged> < https://launchpad.net/bugs/1029668 >
<jcsackett> sinzui: looking.
<sinzui> jcsackett: maybe comment hiding does not work with the comments that are loaded in the sceond phase
<jcsackett> sinzui: i think it has been fixed by my last comment js branch, which has landed.
<jcsackett> sinzui: open the same bug on qastaging and see if you can reproduce. i cannot, though i can on production.
<sinzui> jcsackett: the low number comments can be hidden, but I cannot hide the high number comments. I think the division is the comments that are loaded by the timer are inserted into the dom, but they are not wired for comments
<jcsackett> sinzui: right, and that isn't an issue in the refactor i did, as it just puts a delegate listener on the container for all comments.
<sinzui> jcsackett: I agree that delegates fixed the issue.
<jcsackett> sinzui: fantastic.
<sinzui> jcsackett: claim the bug, mark it fix committed, and a card in the ready lane
<jcsackett> will do.
<czajkowski> jcsackett: ignore ping that was the bug I was looking for you for.
<abentley> adeuring: Could you please review https://code.launchpad.net/~abentley/launchpad/re-roll-r15692/+merge/117070 ?
<adeuring> abentley: sure
<adeuring> abentley: r=me
<abentley> adeuring: Thanks!
<czajkowski> benji: sorry I go by the online calendar :)
<benji> czajkowski: as you should; I am sorry I didn't get my leave in earlier
<abentley> sinzui: http://people.canonical.com/~curtis/lp-milestone/report.html is in the iso-8859-1 encoding, but contains what appears to be a utf-8-encoded character after "In progress and fix committed", etc.
<sinzui> that sucks.
<sinzui> I will fix that today
<sinzui> thank you abentley
<abentley> sinzui: np
<czajkowski> benji: no bother I try and sent out it out late UK time on a Friday
<sinzui> jcsackett: I like your proposal to keep the bug info in one column. I think the misalignment of ids is acceptable and will not happen often. You may have noticed that when we moved the related bug to the MP summary, we got phantom space caused by attempts to handle unwanted table columns...your code might be reusable to fix the MP page
<jcsackett> sinzui: awesome, thanks.
<jcsackett> sinzui: do you have time to look at the finished branch for me? https://code.launchpad.net/~jcsackett/launchpad/misaligned-details/+merge/117118
<sinzui> I will do that now
<jcsackett> thanks.
<sinzui> jcsackett: I don't think class="amount" is needed
<jcsackett> sinzui: no? i'll remove it and check how it looks.
<jcsackett> i think that was just cribbed from other layouts.
<sinzui> jcsackett: yes, it means right align the number in the table column
<sinzui> is class="icon left" really doing something? I think it want to float an icon, but we are using a sprite
<sinzui> oh, jcsackett you may need to add action-icon to the sprite so that older browsers see it
<jcsackett> sinzui: not sure. it was "icon right" which had the sprite sort of misaligned with the edge of the table.
<jcsackett> "icon left" fixed that.
<sinzui> Older browser do not show empty spans, so we need the action-icon, and that turns the span into an inline-block which will change the float behaviour.
<sinzui> jcsackett: okay, I understand. we might need it, check that it is still needed after action-icon. I don't think anything should be floated, but we might have to if something else is making other parts float
<jcsackett> sinzui: dig.
<sinzui> jcsackett:  the sprite could also be on the bug id so we do not need the action-icon and we have less markup to manage
<jcsackett> sinzui: ah, nice idea.
<sinzui> oh, and that makes the table item rules from bugs and branch almost the same
<jcsackett> sinzui: hm. adding sprite onto the bug id span and removing the enclosing span ends up with everything floating a bit too far right from the edge of the table.
<jcsackett> getting you a screenshot.
<jcsackett> sinzui: http://people.canonical.com/~jc/images/misaligned-with-new-markup.png
<sinzui> jcsackett: with the removal of 3 columns, we have ambigous colspans being used by bugs and branches. We just want one column for the td
<sinzui> jcsackett: when I hack the html in https://qastaging.launchpad.net/launchpad/+sharing/abentley to do what what you tried, I see the bug sprites finally line up with the branch sprites.
<sinzui> jcsackett: I see the issue, it is not just the colspan, it is the extra cols in colgroups
<sinzui> we do not need the first two with your change
<jcsackett> sinzui: ok. so i'll remove the extra col stuff and ping you when i've got that all working.
<jcsackett> sinzui: fixes pushed up. sorry for the delay, my znc server has crashed.
<sinzui> np
<sinzui> jcsackett: move {{bug_id}} from amount into the sprite
<sinzui> then delete the amount span
<sinzui> and most importantly, does it look good after you make the change?
<jcsackett> sinzui: it looked perfect post change. http://people.canonical.com/~jc/images/simplified-markup.png
<sinzui> fab.
<sinzui> Then r=me
#launchpad-dev 2012-07-29
<Nour_al_Imen> Hi
<lifeless> Nour_al_Imen: hi?
<Nour_al_Imen> Hi lifeless !
<Nour_al_Imen> I want to know please what are the tasks of the LP team.
<Nour_al_Imen> Those who works and are not simply volunteers ...
<Nour_al_Imen> lifeless
<lifeless> There isn't a precise central list, you would need to ask them all individually. Why do you want to know?
<Nour_al_Imen> I have read the job offer software engineers
<Nour_al_Imen> And I wanted to have more details
<Nour_al_Imen> lifeless
<lifeless> ah
<lifeless> so, the usual stuff: design solutions to problems, write code to implement the solutions, fix bugs, write docs, do releases of the components you're working on
<lifeless> check email :)
<Nour_al_Imen> Which e-mail ? :)
<lifeless> your own?
<Nour_al_Imen> Ah I see :)
<Nour_al_Imen> Thx for your answer...
<lifeless> Nour_al_Imen: you should apply! there is time in the interview process to ask lots of questions of the interviewers, learn all you want to.
<Nour_al_Imen> Ok I'll do it ...Thx for your tips lifeless
<nigelb> argh, I've forgotten to apply.
 * nigelb does so
<Nour_al_Imen> **/is wondering what is he waiting for to apply :))
<nigelb> Now I realize why I forgot. My server died, taking my things to do wiki down along with it :)
<lifeless> nigelb: its not replicated? shame :)
<nigelb> lifeless: It's just a blog + irssi + wiki. Wasn't much of a sysadmin when I set it up :P
<nigelb> I realized when I configured nginx today that I did a lot of fail configuring it last time ;)
<nigelb> One of these days, I'm going to write a job application site That Does Not Suckâ¢
<nigelb> I wish we had pretty graphs for PPA downloads.
<nigelb> Wait, no. I know the answer. "We accept patches"
<lifeless> we do
<wgrant> This celery stuff is not having much luck, is it...
<wgrant> wallyworld_: Thanks.
<wallyworld_> np
<czajkowski> morning
<wgrant> Do I want to revert the celery stuff for the third or fourth time?
<lifeless> its proken again ?
<wgrant> Well
<wgrant> It's not completely broken
<wgrant> So I won't revert it
<wgrant> But one of the related tests failed on db-devel like the old days
<wgrant> But it wasn't touched directly by the recent changes
<wgrant> But reverting this revision has become a fixture of my day lately...
<lifeless> you haven't cronned it yet?
<lifeless> -> can't be that bad then :)
<wgrant> Heh
<czajkowski> yet..
#launchpad-dev 2013-07-22
<jml> wgrant: welcome to the _real_ timezone
<wgrant> Morning jml
<lifeless> lol
<StevenK> wgrant: https://code.launchpad.net/~stevenk/launchpad/drs-for-getsourcesfordeletion/+merge/176149
<StevenK> wgrant: http://pastebin.ubuntu.com/5900221/
<StevenK> wgrant: http://pastebin.ubuntu.com/5900867/
#launchpad-dev 2013-07-23
<cjwatson> wgrant: Could I have a DB patch number for Distribution.development_series_alias, following up on our conversation last night?
<cjwatson> StevenK: https://bugs.launchpad.net/launchpad/+bug/1170120
<_mup_> Bug #1170120: Package diff generation is unnecessarily unresponsive <package-diff> <performance> <Launchpad itself:In Progress by cjwatson> <https://launchpad.net/bugs/1170120>
<cjwatson> feel free to steal assignee
<Ursinha> wgrant, https://code.launchpad.net/~ursinha/launchpad/convert-translationuploads-to-job/+merge/176415
<cjwatson> Ursinha: sort of https://bugs.launchpad.net/launchpad/+bug/180218, kind of, not really entirely
<_mup_> Bug #180218: override mismatch race needs to be fixed <boobytrap> <lp-soyuz> <soyuz-publish> <soyuz-upload> <Launchpad itself:Triaged> <https://launchpad.net/bugs/180218>
<cjwatson> (sorry)
<Ursinha> no problem :)
<cjwatson> Ursinha: the problem I described makes the occurrences of that underlying problem much more common, basically - we could avoid most of the problems with the easier approach and defer having to deal with the hard problem
#launchpad-dev 2013-07-24
<cjwatson> Ursinha: https://code.launchpad.net/~cjwatson/launchpad/refactor-cron-germinate/+merge/84624
<wgrant> cjwatson: eg. I do 'import xmlrpclib; print xmlrpclib.ServerProxy("http://foo.buildd:8221/rpc").status()'
<dlpenguinlover> Hello all, I'm having some trouble running the launchpad instance. Every time i have attempted to run rocketfuel-setup, some point along the way, bzr fails and says "bzr: error: No WorkingTree exists for "file:///home/david/launchpad/lp-branches/devel/.bzr/checkout"/ ERROR: Your trunk branch in /home/david/launchpad/lp-branches/devel is corrupt. Please delete /home/david/launchpad/lp-branches/devel and run rocketfuel-se
<dlpenguinlover> s
<dlpenguinlover> I want to have to redownload all 400+ mbs of data as I have a somewhat slow internet connection. Suggestions?
<dlpenguinlover> *don't want
#launchpad-dev 2013-07-25
<cjwatson> StevenK: sorry, it's in archivepublisher, not archiveuploader, because custom uploads
<cjwatson> StevenK: it appears to actually be in each CU implementation
<StevenK> cjwatson: Yes, I just determined that, but thanks.
<cjwatson> wgrant: The semantics of BuilderStatus.ABORTED on the master side seem a bit different from what we want from BuildStatus.ABORTED.  BuilderStatus.ABORTED ends up in BuildFarmJobOld.jobAborted, which sets the status to BuildStatus.NEEDSBUILD; I think we want BuildStatus.ABORTED to end up in BuildStatus.CANCELLED.
<cjwatson> But maybe that means that (a) we want it to be called CANCELLED on the slave side, (b) perhaps this isn't quite so compatible with the rescue case after all?
<wgrant> cjwatson: It depends on the origin state
<wgrant> BUILDING + ABORTED -> NEEDSBUILD
<wgrant> CANCELLING + ABORTED -> CANCELLED
<cjwatson> IOW it should depend on that, not depends on that right now?
<cjwatson> (Since I don't see that in the code)
<wgrant> Yeah
<wgrant> That's how it probably should work
<cjwatson> Mkay
<wgrant> The slave doesn't need to know why it's being cancelled
<wgrant> That's purely a master-side policy decision
<wgrant> The slave may be being rescued, or it may be being cancelled.
<wgrant> They are identical from the slave POV.
<cjwatson> OK.  Bit confusing for BuildStatus.ABORTED not to necessarily be jobAborted, but I suppose there are only so many verbs.
<wgrant> Well
<wgrant> jobAborted is never called atm.
<wgrant> Because ABORTED is only ever hit after a rescue
<wgrant> In which case the job is no longer assigned to the builder, so there's no job to call jobAborted on.
<wgrant> An ABORTED slave for a BUILDING build should probably be an OOPS
<wgrant> Because I can't see how that could occur
<cjwatson> It's quite possible it is an OOPS right now, then
<wgrant> jobAborted might not even make sense.
<wgrant> The job is cancelled
<wgrant> handleStatus_ABORTED might call jobCancelled when status was CANCELLING, and either set it back to NEEDSBUILD or OOPS and set to FAILEDTOBUILD for any other status
<cjwatson> wgrant: https://code.launchpad.net/~cjwatson/launchpad/buildstatus-aborted/+merge/176990
<wgrant> cjwatson: I think that looks fairly sane
<wgrant> I'll have to think about it
<wgrant> Well
<wgrant> And try it
<wgrant> Until it breaks
<StevenK> cjwatson, wgrant: https://code.launchpad.net/~stevenk/launchpad/no-dead-custom-uploads/+merge/176995
<cjwatson> StevenK: the setComponents description in customupload.py should probably be more generic
<cjwatson> StevenK: and don't you want target_series.enabled_architectures instead of .architectures?
<StevenK> -        """Set self.version and self.arch (if applicable)."""
<StevenK> +        """Set instance variables based on decomposing the filename."""
<cjwatson> Seems fair
<StevenK> cjwatson: That's a question, do we want to copy custom uploads for arches which are disabled?
<cjwatson> I don't think so
<StevenK> That's more a question for you, or infinity
<cjwatson> That was the raring(?)/armel case
<StevenK> Right. This also gives us grace to disable them from the get go, and then yay, no custom uploads on first publish
<cjwatson> Once it's disabled, I think we want to pretend it doesn't exist, it's just that we can't delete it easily
<wgrant> Well, the raring/armel case would still have worked
<wgrant> Because this is for the target arch check
<wgrant> But yes
<wgrant> enabled_architectures should be the default
<cjwatson> https://code.launchpad.net/~cjwatson/launchpad-buildd/fix-abort/+merge/177003  *phew*
<cjwatson> wgrant: Is there a plausible way to get a callback in buildd-manager when a build goes to ABORTED (for the bit during cancellation that needs to wait for that), or do I just have to poll?
<wgrant> cjwatson: You have to poll. I'd suggest using the separate handleStatus_ABORTED and just working off the state
<wgrant> Rather than doing it as part of the cancellation function
<cjwatson> Hm, yeah, that makes sense.
<wgrant> Unless you want to rewrite  buildd-manager's state machine
<cjwatson> No :-)
<wgrant> Which somehow seems unlikely..
<cjwatson> Still need a timeout which I cancel, but I guess I can manage that
<wgrant> Yeah, I'm not quite sure how to do the timeout.
<wgrant> Possibly store is as a local attribute in  buildd-manager
<wgrant> It means that a restart will reset the timeout, but meh
<wgrant> Really doesn't matter
<wgrant> And is probably cleaner
<cjwatson> Why not just reactor.callLater?
<wgrant> A good point.
<wgrant> Well
<wgrant> A restart would stop that from working
<cjwatson> Though I still need to store the Deferred somewhere
<wgrant> eg. I start cancelling a build, then buildd-manager restarts
<wgrant> If you just naively use callLater then it will never time out
<wgrant> So, I guess, ideally it would see slavestatus == BUILDING and masterstatus == CANCELLING, and call abort() and set a timeout
<wgrant> If it seeis slavestatus == ABORTING and masterstatus == CANCELLING, it might have to ensure that a timeout is set.
<cjwatson> Maybe the simplest answer is just to check each time SlaveScanner notices a masterstatus == CANCELLING build, since it's polling every 15 seconds anyway
<cjwatson> It could easily just keep count of a multiple of that
<cjwatson> Not very Twisted, but
<wgrant> Yeah, indeed
<cjwatson> If we were really trying hard then we could add a DB column on Builder for the timeout
<wgrant> That would be trying altogether too hard.
#launchpad-dev 2013-07-26
<cjwatson> wgrant: Do you have any remaining concerns about https://code.launchpad.net/~cjwatson/launchpad/db-series-alias/+merge/176344 ?
<cjwatson> wgrant: http://anonscm.debian.org/gitweb/?p=buildd-tools/sbuild.git
<cjwatson> wgrant: git diff release/sbuild-0.64.0 origin/buildd-0.64
#launchpad-dev 2014-07-24
<wgrant> cjwatson: Found any issues for RTM on DF?
<cjwatson> wgrant: Nothing yet.  Getting livefs builds working was a bit exciting, but not really for any reasons connected to Launchpad.
<cjwatson> wgrant: Just set lp.distributions["ubuntu-rtm"].development_series_alias to "devel" to match Ubuntu
<cjwatson> Might possibly be worth having that as part of the init procedure
<wgrant> cjwatson: Noted.
<mvo_> hi, I keep getting a timeout error when doing: w3m -dump https://launchpad.net/ubuntu/+archivemirrors-rss (latest oops-id is OOPS-c3c091162163ac61751c1a003d5c8fdf). is there a workaround or anything so that I can get a cached copy? update-manager needs this to update its internal mirror list
<wgrant> Hmm
<wgrant> I thought I increased that timeout to like 20s last release.
<wgrant> Surely it can't be timing out still...
<wgrant> Oh, that was +cdmirrors-rss.
<wgrant> I'll have a look at archivemirrors after dinner.
<wgrant> mvo_: ^^
<stub> wgrant: Shall I do the LFC DB patch now, or are you on it?
<wgrant> stub: I haven't started yet, so feel free.
<stub> ta
<wgrant> Thanks.
<wgrant> It's not terribly complicated, hopefully.
<wgrant> Pretty simple except for the pg_attribute hackery.
<wgrant> stub: While you're there, might be worth making LFC.sha256 NOT NULL as well. The only reason it's not already is that I didn't care enough to hack pg_attribute, but we have to for the PK so we might as well kill two birds.
<stub> wgrant: Should I just reuse the sha256 as the PK?
<stub> Bah. stupid.
<wgrant> stub: No, that makes it impossible to split.
<stub> Have you timed just using ALTER TABLE for the NOT NULLs? I'd rather avoid hacking the system catalog if I can.
<wgrant> I tried that with a table that I forget and ended up with three minutes of downtime, even when I thought it was very hot.
<wgrant> And LFC is not small.
<wgrant> It's even reasonably wide due to the hashes.
<wgrant> From http://www.postgresql.org/docs/9.3/static/catalog-pg-attribute.html:
<wgrant> "This represents a not-null constraint. It is possible to change this column to enable or disable the constraint."
<wgrant> That sounds like an invitation to me.
<mvo_> thanks a lot wgrant
<wgrant> mvo_: "fixed"
<cjwatson> wgrant: Is it possible to create non-Ubuntu PPAs yet?  I see that createPPA doesn't take a distribution= kwarg
<wgrant> cjwatson: dogfood has two unlanded branches merged: overrides and createPPA(distribution=)
<cjwatson> Ah, I should look at DF's apidoc
<cjwatson> Oh, argh, PPA names are unique regardless of distribution, aren't they?
<wgrant> They shouldn't be, but I probably haven't hacked createPPA hard enough, fixing.
<wgrant> I still need to do some rearrangements on haetae before they can be ununique on prod, though.
<cjwatson> And I have deleted https://dogfood.paddev.net/~ci-train-ppa-service/+archive/ubuntu/landing-000 due to abject confusion, bah
<wgrant> Heh.
<cjwatson> How do I go about undeleting that?  I guess I need to run something to fully process the deletion?
<wgrant> Undelete or recreate, I suppose.
<wgrant> I'd just set the status back to 0.
<wgrant> But the PPA publisher will set it to DELETED and rename it out of the way so you can recreate it.
<cjwatson> Oh right
<cjwatson> I'll flip the status
<cjwatson> ok, undeleted, thanks
<cjwatson> let me know when I can try createPPA again
<wgrant> cjwatson: Fixed.
<cjwatson> Thanks, looks good.  Error message in the case of a clash could possibly be improved now.
<cjwatson> I wonder if we should put the distribution name in the rendering of PPAs on Person:+index.
<cjwatson> https://dogfood.paddev.net/~ci-train-ppa-service looks somewhat confusing.
<wgrant> I've thought about that on occasion.
<wgrant> My most favourable proposal is to just split them up into sections by distribution.
<cjwatson> Mm, yeah.
<cjwatson> That's nicer than "(Ubuntu)" x a zillion
<wgrant> Also just pulled a fix not obliterate all similarly named PPAs when you delete one of them.
<cjwatson> Hah
<cjwatson> similarly> same name different distro?
<wgrant> Right.
<wgrant> Deletion killed /wgrant/ppa rather than /wgrant/ppa/ubuntu, for good reason (.htaccess, .htpasswd live in /wgrant/ppa rather than /wgrant/ppa/ubuntu)
<wgrant> But haetae's Apache config is fixed, so I can push that all down a level now.
<wgrant> The ArchiveDependency UI should be terribly broken, but I think non-Ubuntu PPAs otherwise work.
<cjwatson> We'll find out soon, I imagine, as sil2100 is working on getting citrain working
<wgrant> Oh, there's no non-Ubuntu PPA publisher cronned atm, should fix that.
<wgrant> In fact I don't think there's a PPA publisher cronned at all?
<cjwatson> I don't think so, no.
<wgrant> cjwatson: Both are publishing at */1 now. I've set publish=false on most Ubuntu PPAs so it should be quick.
<cjwatson> Cool, thanks.
<wgrant> cjwatson: Do we want something like the archive reference as well, perhaps?
<wgrant> I don't know how the ddeb fetcher works, and I guess it's not worth making it too much more reliable at this point.
<cjwatson> wgrant: pitti tells me that it otherwise works; in fact he's ignoring Distribution right now and deriving from Release, but he probably shouldn't be
<cjwatson> Translations might care though.
<wgrant> cjwatson: The translations sbuild hack hasn't been using in very nearly eight years
<wgrant> ...
<wgrant> used
<wgrant> I suspect it's not even triggered any more.
<cjwatson> Ah, good
<cjwatson> So it's just to make ddebs a little less horrible, and because --archive=ubuntu in the config file is disgusting
<wgrant> Quite.
<cjwatson> (That option being called --archive is disgusting, too, but)
<wgrant> cjwatson: I'm initialising some more distros on DF for QA, so don't expect the derived primary publisher to be particularly responsive at all times.
<cjwatson> wgrant: ta
<wgrant> Hm
<wgrant> Creating a full copy of precise amd64 and i386 with packagecopier doesn't work.
<wgrant> CannotCopy(u'libunwind 0.99-0.3ubuntu1 in precise (binaries conflicting with the existing ones)',)
<wgrant> I wonder why.
<wgrant> Perhaps the primary archive is inconsistent.
#launchpad-dev 2014-07-25
<stub> wgrant: new rev of the db patch up, https://pastebin.canonical.com/114208/ contains the before/after schema diff from a dev db.
<stub> I need a new pastebin bookmark. Lost colorization somewhere.
<wgrant> stub: I don't speak 1970
<wgrant> But let me see.
<stub> fab
<wgrant> stub: That looks pretty sensible, I think.
<stub> I'll approve and land it now then and we can start the ball rolling next week. Its all standard until we get to the last step.
<stub> Things to confirm on staging are if adding the new column with a FK reference needs to do an unnecessary scan, and similarly that final primary key.
<stub> wgrant: Or do you want to do a proper review first?
<wgrant> stub: I know those two things don't require full scans; I've done them each several times. So we're pretty safe except for the pg_attribute hackery, but that looks fine.
<wgrant> stub: I'll have another look at the patch on Monday before you awake, if you don't mind. Just to ensure it makes sense with a fresh mind.
<wgrant> cprov, cjwatson: Thanks for the reviews.
<cprov> wgrant: sorry for letting them slip until today
<wgrant> cprov: No worries. Could you please also look at https://code.launchpad.net/~wgrant/launchpad/htaccess-in-archiveroot/+merge/225783 and https://code.launchpad.net/~wgrant/launchpad/delete-archiveroot/+merge/225797 soonish? I expect to get moving on migrating production early next week, if possible.
<cprov> wgrant: will do
<cjohnston> wgrant: in Malta we discussed increasing the scan timeout by a minute or two to see if that would help with LP branches... did you have thoughts on the possibility of actually implementing that?
#launchpad-dev 2015-07-20
<blr> wgrant: hmm, which modal had the multi-step process, I thought it was 'link a related branch' on +bug, but that doesn't seem to be it
<blr> I must be going mad, I was looking at it yesterday...
<wgrant> blr: +sharing is multi-step
<wgrant> blr: BugTask:+index's branch picker is highly customised.
<wgrant> Well, not highly.
<wgrant> But quite.
<blr> wgrant: ah thanks, yes it was sharing.
<satmandu> Quick question: I need to rebuild a package on my ppa for armh. armh builds have already been approved. How do I get an uploaded package to be rebuilt for armh?
<satmandu> it seems that ubuntu-build might be the correct tool to use for this, but I can't figure out the right syntax for working with ppa builds
<cjwatson> satmandu: If armhf was turned on after you uploaded the package in question, you'll need to upload it again.
<cjwatson> That is, bump the version in debian/changelog but make no other changes.
<satmandu> that's what I thought.  ok thanks
<cjwatson> I could check if you told us which PPA.
<cjwatson> Oh, https://launchpad.net/~satadru-umich/+archive/ubuntu/sernet-samba/+packages I guess
<satmandu> ppa:satadru-umich/sernet-samba
<satmandu> yup that's it
<cjwatson> Yes, bump version and reupload.  Future uploads will have armhf builds created by default.
<satmandu> ok will do. Thanks for the help!
<satmandu> Can I ask a build question?
<satmandu> my amd64 build took 16 min
<satmandu> but the armhf build seems to be stuck.
<satmandu> I want to be a good ubuntu citizen...
<satmandu> is there anything I'm doing wrong?
<satmandu> amd64 build: https://launchpad.net/~satadru-umich/+archive/ubuntu/sernet-samba/+build/7672673
<satmandu> current armh build: https://launchpad.net/builders/lgw01-53
<cjwatson> satmandu: armhf builds run via qemu-user-static.  It's not very robust, and some things (particularly, but not limited to, just about anything involving threading) just plain don't work.
<satmandu> Hmmm. ok. But the only way to store builds as a ppa is to have them compiled through the qemu-user-static, right? I can't build the packages locally and then upload them?
<cjwatson> Correct.
<satmandu> ok :)
<cjwatson> We'll be bringing proper virtualised ARM builds online in the next couple of months or so.
<satmandu> understood.
<cjwatson> That should make many things better.
<satmandu> Does the i386 build process often cause permissions issues during build? Getting a crash here: https://launchpadlibrarian.net/212115290/buildlog_ubuntu-vivid-i386.samba_99%3A4.1.19-11_BUILDING.txt.gz
<cjwatson> satmandu: The difference between amd64 and i386 here is that (for >= vivid) amd64 builds the architecture-independent components as well.  You can reproduce this locally using sbuild -A vs. sbuild --no-arch-all.
<cjwatson> Also, that isn't a permissions issue; read the error message closely.
<satmandu> the directory isn't being created...
<cjwatson> Right, that's not permissions. :-)
<satmandu> I think it's confused about wanting a /tmp/ folder
<satmandu> but it works ok on amd64
<cjwatson> Your build is probably only creating it in the architecture-independent case.
<satmandu> right
<cjwatson> I don't see how /tmp/ has anything to do with it.
<satmandu> just say /var/lib folders earlier as /tmp/var/lib etc etc...
<satmandu> I'm just going to try these builds locally then
<cjwatson> You mean debian/tmp/var/lib
<satmandu> to see if they work
<satmandu> yes
<satmandu> Is there any way to see why this is stuck before I cancel it? https://launchpad.net/~satadru-umich/+archive/ubuntu/sernet-samba/+build/7673738
<cjwatson> satmandu: Not really, no.
<cjwatson> Pretty sure your builds are just doomed on armhf for now :-(
<satmandu> so it would seem
<cjwatson> satmandu: sernet-samba-common is Architecture: all - this means that it is only built in builds that build architecture-independent packages.
<cjwatson> satmandu: So debian/sernet-samba-common/<anything> won't have been created by other bits of the packaging process for architecture-dependent-only builds, such as i386.
<cjwatson> satmandu: The correct fix with modern-ish versions of debhelper, such as that in vivid, is to change "override_dh_fixperms" to "override_dh_fixperms-indep", since you only need to override the arch-indep behaviour there.
<satmandu> noted
<satmandu> Is there a good way to build debs locally from the dsc & changes files to replicate what launchpad is doing?
<satmandu> good = best practices
<satmandu> aside from this: https://help.ubuntu.com/community/UpdatingADeb
<cjwatson> satmandu: https://wiki.ubuntu.com/SimpleSbuild
<blr> morning
<cjwatson> evening
<blr> how's things cjwatson?
<cjwatson> I think I have most of the initial pieces for snap building put together, just got sidetracked into fixing https://bugs.launchpad.net/launchpad/+bug/1424672
<mup> Bug #1424672: LiveFS builds cancelled before they start sort above other builds in history <confusing-ui> <soyuz-build> <Launchpad itself:In Progress by cjwatson> <https://launchpad.net/bugs/1424672>
<cjwatson> (since the livefs and snap code are *cough* suspiciously similar, so I didn't want to import that bug)
<blr> cjwatson: hoping to get onto the squid authentication service soon, trying to get the 2-stage clientside ref picker sorted.
 * cjwatson nods
<blr> discussed oauth/bearer token auth vs basic auth with wgrant yesterday, he seemed to think basic should suffice
<cjwatson> I didn't totally follow, but the arguments sounded plausible :)
<beuno> internal auth?  we use basic auth everywhere I think, except for the things where it's easier to use tokens because they already terminate sso
<satmandu> cjwatson: I'm trying one last armhf build.. which appears to be going albeit slowly. (The compile is running much faster on my raspberry pi sitting next to me.)
<satmandu> So slow that I should cancel it to not be a drain on the build servers?
<satmandu> https://launchpad.net/~satadru-umich/+archive/ubuntu/sernet-samba/+build/7674252
<cjwatson> satmandu: We have plenty of build resources at the moment.  If it actually works then it's fine.
<satmandu> cjwatson: Thanks.
<satmandu> cjwatson: Does this look like a hung compile to you? https://launchpad.net/builders/lgw01-54
<satmandu> I can't debug it any further from my end.
<wgrant> satmandu: qemu-user-static is quite slow. You pretty much have to watch the build log to see if it changes.
<cjwatson> satmandu: Yeah, basically no way to tell.  Leave it for a few hours; if it's idle with no output for too long then the builder should kill it anyway.
<satmandu> Understood. Thanks again.
<carstenh> Hi, loggerhead fails to display tabs as tabs in a diff.  Tabs displayed as tabs: http://paste.ubuntu.com/11908410/ - tabs displayed as spaces: http://bazaar.launchpad.net/~ubuntu-core-dev/command-not-found/ubuntu/revision/190/zsh_command_not_found
<carstenh> In case you ever wondered why cron logs three useless spaces every hour to syslog ... someone mixed up tabs with spaces in /etc/crontab, so this is not only an only academicially relevant issue
<carstenh> And your paste service is suboptimal, correctly pasting a tab-idented file from a terminal into a browser is complicated and error-prone (at least I failed in doing so, I inserted an additional space).  A way to upload a file and a way to download w/o logging in would be helpful (paste.debian.net is way better at this), it's also ugly in lynx (but paste.debian.net is even more ugly in lynx).  Good night
<cjwatson> pastebinit supports non-browser upload to paste.u.c
<cjwatson> the login requirement is annoying but it mitigated some quite serious attacks aiui (I don't remember the details - it isn't a Launchpad service)
<carstenh> Sounds good, such a tool is a great addition, but no replacement for an upload button in the html page.
<cjwatson> we don't maintain it
<cjwatson> loggerhead should be easily enough fixable, but could you please file a bug so we don't forget?
<carstenh> Ok, I think the tab vs spaces issue in loggerhead is still valid
<cjwatson> irc doesn't make a good bug tracker
<carstenh> I'm goging to bed soonish, you could just paste what I wrote into a bug report since you are more familiar with launchpad :)
<cjwatson> I'm going to bed *now*, and on phone so no easy paste
<cjwatson> so no, not really ...
<carstenh> Ok, I'll try to remember this when I login into launchpad next time :)
#launchpad-dev 2015-07-21
<blr> hmm all the yui tests seem to be missing sam/test.css, that appears to have disappeared at some point.
<blr> the disclosure js is... dense.
<wgrant> blr: Have you worked it out?
<cjwatson> wgrant: Blast, I accidentally sent mail-footer-pref off to PQM before noticing your comment on it.  I'll do a follow-up branch.
<cjwatson> One day I will remember to merge db-devel before trying to land dependent code branches.
<wgrant> cjwatson: Was that the "email" vs "e-mail" thing? Not critical, since we have both spellings all over, but it isn't 1980 any more.
<cjwatson> wgrant: even Wiktionary only thinks it's "becoming a standardized usage" rather than actually standard, but I'm not intending to make a fight of it :-)  https://code.launchpad.net/~cjwatson/launchpad/email-spelling/+merge/265374
<wgrant> Hm, I haven't seen "e-mail" outside US news sources and a few websites from the 00s in quite some time.
<wgrant> Anyway, thanks for making it consistent :)
<cjwatson> wgrant: I updated both https://code.launchpad.net/~cjwatson/launchpad/send-bug-notifications-oops/+merge/264814 and https://code.launchpad.net/~cjwatson/launchpad/basemailer-oops/+merge/264856 in line with your comments on the former (I want to try to keep them in sync, as hopefully I'll manage to get s-b-n using basemailer eventually).  Could you re-review?
<cjwatson> Hopefully the control flow in the former is a bit clearer now, even though I think it was already correct earlier.
<blr> morning
<cjwatson> wgrant: I updated both https://code.launchpad.net/~cjwatson/launchpad/send-bug-notifications-oops/+merge/264814 and https://code.launchpad.net/~cjwatson/launchpad/basemailer-oops/+merge/264856 in line with your comments on the former (I want to try to keep them in sync, as hopefully I'll manage to get s-b-n using basemailer eventually).  Could you re-review?
<cjwatson> Hopefully the control flow in the former is a bit clearer now, even though I think it was already correct earlier.
#launchpad-dev 2015-07-23
<cjwatson> wgrant: https://code.launchpad.net/~cjwatson/launchpad-buildd/complex-build-deps/+merge/265650
<cjwatson> hopefully actually right this time
<wgrant> cjwatson: Aha, will go through it tomorrow.
<cjwatson> ta
<blr> morning
<rpadovani> blr, where do you live if I can ask? Every time you appear in this channel in the morning for me it's time to start to go to bed :D
<blr> rpadovani: New Zealand, yourself?
<rpadovani> blr, Italy :-)
<blr> ah lovely! would very much like to visit one day
<rpadovani> indeed it's a great place for turism - a bit less lovely to live in
<mwhudson> wgrant: good morning
<wgrant> mwhudson: Hi
<mwhudson> wgrant: i'm wanting to do some test rebuilds in prep for go 1.5 and so would like a ppa devirted
<wgrant> mwhudson: Sure
<mwhudson> wgrant: just create the ppa and ping you?
<wgrant> Yep
<mwhudson> wgrant: https://launchpad.net/~mwhudson/+archive/ubuntu/go1.5-rebuild-tests
<wgrant> mwhudson: Which architectures?
<mwhudson> wgrant: is 'all' a resonable answer?
<wgrant> It is.
<wgrant> It is done.
<mwhudson> yay thanks
<mwhudson> wgrant: do you know if there are scripts to help with this sort of thing?
<mwhudson> my plan so far as it went was (a) upload a golang 1.5 snapshot package (b) grep-dctrl -FBuild-Depends golang-go.tools  -sPackage /var/lib/apt/lists/*Sources (c) no-change upload  of everything that (b) spat out
<wgrant> mwhudson: Not that I know of, but depending on what you're doing you might be able to get away with uploading the new toolchain, waiting for it to build, then using an API script to copy the entire reverse dependency set in.
<wgrant> Not much point in a no-change upload if you're using a sacrificial PPA just for testing.
<wgrant> A copy is easier :)
<mwhudson> ah true
<mwhudson> and yes, that should work fine, enabling shared libraries will need a bit more cleverness to build in some kind of dependency-related order but i'm not poking that bear yet
<mwhudson> wgrant: do you have a link to such an api script?
<wgrant> mwhudson: lp.archives.getByReference('~mwhudson/ubuntu/go1.5-rebuild-tests').copyPackages(from_archive=lp.archives.getByReference('ubuntu'), from_series='wily', to_series='wily', to_pocket='Release', source_names=['blah', 'blah', 'blah'])
<wgrant> mwhudson: Or use the copy-package script from lp:ubuntu-archive-tools
<mwhudson> wgrant: thanks
<mwhudson> wgrant: will it fall over if i pass like 100 source_names in one go?
<mwhudson> oh yeah, i guess i need something wily to run the grep-dctrl in...
<wgrant> It's fairly asynchronous, so it works with reasonably large batches.
<mwhudson> cool
<wgrant> I'm not sure if 100 will work, but it may well.
<wgrant> Consider chdist for the grep-dctrl thing
<mwhudson> oh nifty
#launchpad-dev 2015-07-24
<mwhudson> rebuild testing disrupted by finding that go itself is busted on arm today
<wgrant> Hah, unhelpful.
<mwhudson> reasonably simple fix though, luckily
#launchpad-dev 2015-07-26
<Pali> hello, launchpad has some problems with importing from git repository: https://code.launchpad.net/~sjr/beignet/master
<Pali> last successfull import was in Februar
<Pali> I can clone that git repository locally, so problem should be on launchpad...
<Pali> can somebody look at it?
<StevenK> 2015-07-26 09:15:20 INFO    The repository you are fetching from contains submodules, which are not yet supported.
<Pali> StevenK: what I can do?
<Pali> how can tell launchpad to ignore that git submodules?
<blr> morning
<wgrant> Morning blr
<blr> how was your weekend wgrant?
<wgrant> blr: Pretty good.
<wgrant> How's the proxy stuff looking?
<blr> wgrant: just about to push a bit more up, service is looking ok, need to add the requests script.
<blr> and need some view tests
#launchpad-dev 2016-07-28
 * mpt goes to report a Launchpad bug, and discovers that he already reported it in December 2010
#launchpad-dev 2017-07-24
<sil2100> Morning! Another LP API question - I would like to easily get a count of overall people with main_archive upload rights, possibly counting core-devs, MOTUs and per-package uploaders
<sil2100> Wanted to use the main_archive.getAllPermissions() for that possibly, but even though I get a set of archive permissions (over 700) those are not accessible for me
<sil2100> I can't read their contents
<sil2100> What would be the best and easiest way to achieve that?
<wgrant> sil2100: What do you mean when you say they are inaccessible?
<sil2100> wgrant: strange thing, I get a collection, can check its length but if I try to access its individual archive_permission objects it says it's out of bounds, e.g. main_archive.getAllPermissions()[0]
<wgrant> sil2100: Ah, what if you log in?
<sil2100> wgrant: is this not available anonymously? Since that would be my requirement
<wgrant> sil2100: It's very easy to accidentally export something that's available to all authenticated users but not anonymous ones.
<wgrant> It's also easy to fix.
<sil2100> hm, since otherwise it'll be quite hard for me to get the number of all uploaders to Ubuntu, hmm
<wgrant> Sure
<wgrant> I'm just saying you should do that now to a) verify that it's the problem and b) have a workaround until we can fix it in LP.
<sil2100> wgrant: confirmed, when logged in it works
<wgrant> sil2100: Great. Can you file a bug?
<sil2100> Sure thing
<sil2100> https://bugs.launchpad.net/launchpadlib/+bug/1705996
<mup> Bug #1705996: Unable to access archive_permission for archive when not logged in <launchpadlib :New> <https://launchpad.net/bugs/1705996>
<sil2100> Thanks!
<sil2100> wgrant: while thinking about it, can I actually get a real number of *actual* users that have upload rights to an archive?
<wgrant> sil2100: You'd need to expand all the team participants and exclude robots and such.
<sil2100> wgrant: thanks
<sil2100> wgrant: one (hopefully) last question: is there a way I could check a given user's team memberships? I don't see that straight away from the docs - I'd be mostly interested in the data of 'if this user part of canonical', since the ~canonical team seems to be private
<sil2100> And I can't access it when not logged in
<wgrant> sil2100: It's deliberately not possible to determine someone's Canonical employment status anonymously.
<blahdeblah> And membership in that team doesn't guarantee Canonical employment, either.
<blahdeblah> (Most of those are corner cases that usually aren't relevant, though.)
<sil2100> oh!
<sil2100> Indeed, I didn't know that when I'm not logged in I can't see my canonical membership
<sil2100> Ok, today is a day where things that seemed easy just get a bit more complicated ;)
<sil2100> I'll just have to do some fancy guess-work
<skay> hey, I was wondering what needs to happen for lp:948857. I don't know how to do that, so I made a junk repo to experiment on. https://code.launchpad.net/~codersquid/+junk/oops-datedir-no-pruning
<skay> I've not used zope.interface, but when working in Java I've used interfaces and dependency injection. so, I imagine it is a similar principle. I just haven't done it in python before
<cjwatson> Well, I mean I guess an abstract interface is the ideally correct way to do it, but that code isn't using zope.interface (or indeed zope.component) more than trivially at the moment so it may not be worth it.  Perhaps just try / except ImportError: instead?
<cjwatson> We'd need to review in a bit more detail but your "hack" doesn't seem obviously wrong.
<skay> cjwatson: that is what I did in my junk branch. I made a wheel for my own use.
<cjwatson> Don't use junk branches when a project already exists though, as they mean you won't be able to propose a merge ...
<skay> I wasn't sure I'd ever propose one since it is hacky
<skay> mine is hacky
<skay> if you want it, I'll make a real branch
<cjwatson> Doesn't seem overly hacky :)
<cjwatson> You probably want a bit more effort into telling the user they don't have the right stuff to use that feature.
<skay> would you want a separate branch for changing setup.py to use setuptools instead of distutils.core? that part is unrelated to making prune optional.
<skay> it was just for personal use. seems like I shouldn't mix things if someone's going to review
<cjwatson> I don't really know this codebase particularly well, but on general principles I think that ought to be a separate branch
<skay> cjwatson and anyone, I made https://code.launchpad.net/~codersquid/python-oops-datedir-repo/optional-prune-dependencies/+merge/327978 for making prune optional, but this is not ready to land since I couldn't get tests to run
<skay> see my comment. I followed the README bit on development and I'm missing a step somewhere
<skay> I haven't used testr before, so perhaps I've missed something obvious
#launchpad-dev 2017-07-25
<skay> cjwatson: the readme in python-oops-datedir-repo is missing a step for testing. is the launchpad-dev still using the same build/testing scheme for launchpad projects? if so, could you point me to a more fleshed out readme?
<skay> cjwatson: but if not, is there a good repo I can use as a reference to get this one up to date so that it is consistent with what your team does?
<skay> otherwise I can run around going weeeeeee and use some trendy testing tool like python unittest or pytest or something :p :)
<skay> (this is a really minor bug, but we use this repo for an internal django repo that I'm finally upgrading and I'm taking the opportunity to trim our dependencies down)
<skay> pipdeptree is pretty awesome for helping map out transitive dependencies
<cjwatson> skay: Sorry, I've never looked at this codebase significantly, and the various repos that use buildout are all a bit of a mess in different ways.  If I were you I'd just get it to the point where it has a more or less standard setup.py and requirements.txt such that it's possible to set up a suitable virtualenv with pip, and env/bin/python -m unittest discover -v or something built on top of that
<cjwatson> skay: Most of our repositories use buildout for historical reasons, but it's no longer what we prefer
<skay> ack
<cjwatson> skay: I guess turnip (which is in git) is the most modern build system we have
#launchpad-dev 2019-07-23
<SpecialK|Canon> To what extent do people feel https://launchpad.readthedocs.io/en/latest/ is up-to-date and accurate?
<SpecialK|Canon> (I've no reason to believe it isn't, but I just noticed the 2011 at the bottom, and docs are easy to bitrot etc.)
<cjwatson> Definitely looks like something isn't being automatically updated there.  https://launchpad.readthedocs.io/en/latest/buildout.html was removed in 2017 ...
<cjwatson> (or should've been)
<cjwatson> Ah, build is failing says readthedocs
<cjwatson> "An unexpected error occurred"   thanks
<cjwatson> Trying another build
<cjwatson> (and if that works I'll set up a webhook to do it automatically, which apparently isn't the case at the moment)
<SpecialK|Canon> It landed, thanks :)
<cjwatson> Oh, bother, that's why I hadn't set up a webhook
<cjwatson> The branch is owned by ~launchpad-pqm
<cjwatson> Might be easier to do this after git migration
<cjwatson> SpecialK|Canon: https://code.launchpad.net/~cjwatson/launchpad/readthedocs-copyright-2019/+merge/370480
#launchpad-dev 2019-07-24
<SpecialK|Canon> Where can I learn more about the librarian please? https://launchpadlibrarian.net/ is a bit light, and LP search isn't turning up much for me
<cjwatson> SpecialK|Canon: doctesta delenda sunt, but https://bazaar.launchpad.net/+branch/launchpad/view/head:/lib/lp/services/librarian/doc/librarian.txt may actually be the best overview
<cjwatson> tomwardill: ^- may be useful to you too
<SpecialK|Canon> cjwatson: cheers :)
<cjwatson> Also please admire my Latin conjugation because for some reason I put actual effort into that :P
<blahdeblah> cjwatson: I am so jealous that you actually know Latin.
 * tomwardill assumes that it is correct
 * blahdeblah curses short-sighted .au and .us education systems that neglected it during his childhood
<cjwatson> I took it up to GCSE (exams you take at about 16 in the UK, although I took Latin a year later for reasons), though VERY rusty
<stub> I did Latin in AU, but remember next to nothing.
<StevenK> bleDepnds on the school, one of my friends did 6 years of Latin in an .au high school
<SpecialK|Canon> cjwatson: nice - I was always a bit weak on the grammar
<cjwatson> I went to a Catholic grammar school, which I imagine had something to do with it
<SpecialK|Canon> (but I have more Catullus wedged in my brain than I'd ideally choose...)
<SpecialK|Canon> and I think I could regurgitate most of "oculi omnium in te sperant domine..." if pushed
<SpecialK|Canon> Could someone grant doismellburning edit rights on the dev.lp wiki please?
<SpecialK|Canon> cjwatson: ^ perhaps please?
<cjwatson> SpecialK|Canon: Should be done - you'll probably have to log out of that wiki and back in, and make sure SSO tells it about your new ~launchpad-doc membership
<SpecialK|Canon> cjwatson: cheers :)
<SpecialK|Canon> https://dev.launchpad.net/ArchitectureGuide/Git boom thanks
<cjwatson> Cheers
<SpecialK|Canon> tomwardill: ^ :)
 * tomwardill adds to todo list
<tomwardill> just after 'why must you keep breaking, tests?'
<cjwatson> tomwardill: It remains true that permissions support isn't in the smart HTTP/SSH layer though
<tomwardill> ah, true
<cjwatson> tomwardill: That's in the storage backend - the place to expand would be "can invoke a callback ...", which is a passing reference to the hook RPC system
#launchpad-dev 2019-07-25
<tomwardill> couple of sentences added about using the RPC hook system for per branch/ref permissions
<cjwatson> LGTM
<SpecialK|Canon> where's the definition of LOSA? I've seen it around and it looks like a sysadmin of some sort but https://dev.launchpad.net/FrontPage?action=fullsearch&titlesearch=0&value=losa&context=180 has not helped me find a canonical def
<cjwatson> Launchpad/Landscape Operational System Administrator.  Nowadays probably wants to say SRE or something instead
<SpecialK|Canon> And in concrete terms that nowadays resolves to "someone from IS"?
<cjwatson> Right
<SpecialK|Canon> Thanks :)
<Spads> It used to be that we separated people who managed deployments from general SRE work
<cjwatson> Used to be a subteam just for us, is no longer
<cjwatson> (hence #webops vs. #is)
<Spads> but these days everything is juju and mojo so there's no maintenance of that division
<Spads> yeah
<Spads> LOSA started as Launchpad Operations SysAdmin, then Launchpad/Landscape OSA and then L* and then WebOps
<Spads> it's vestigial now
<Spads> "IS Staff" should be sufficiently general
<SpecialK|Canon> Yep, but it's still liberally referenced on dev.lp so I'm adding a page for it :)
<SpecialK|Canon> does IS have any kind of public presence?
<SpecialK|Canon> well https://dev.launchpad.net/LOSA is a (trivial) thing now
<SpecialK|Canon> (and thanks all for the context!)
<Spads> SpecialK|Canon: rt@ubuntu.com reaches our ticket system which we try to stay on top of
<SpecialK|Canon> Spads: thanks! I think I'll hold off adding any kind of public contact method until I can clean up some more stuff
<Spads> ok cool
<Spads> I think in the days of the LOSAs they had a more prominent presence in these channels
#launchpad-dev 2020-07-20
<SpecialK|Canon> https://code.launchpad.net/~doismellburning/launchpad/+git/launchpad/+merge/387647
<cjwatson> SpecialK|Canon: Needs other changes - let me know if my comment isn't clear
<SpecialK|Canon> cjwatson: blah, no, cheers, perfectly clear - I should have actually read the script, it just looked like someone accidentally omitted -e because of the amount of blind attempts
<SpecialK|Canon> *of unchecked attempts at stuff
#launchpad-dev 2020-07-21
<Laney> cjwatson: I just saw you've been working on Built-Using support... is that close to being ready at all? The new proposed-migration is now enforcing this but it's a little bit odd to do so without LP actually keeping the sources around. Steve grumbled about it a bit and forced qemu in over a complaint, and I'm wondering what we should do here really. I think in principle enforcing BU is a good idea...
<cjwatson> Laney: Making progress but it'll probably be a little while yet
<cjwatson> Maybe a month or so if there are no more serious roadblocks?  Wild guess though
<Laney> nod
<Laney> I guess we should turn the policy off until then
<cjwatson> The basic idea is that if sources referenced by Built-Using aren't published in their own right then they'll stick around (or if necessary be copied in) with a new "Referenced" publishing status for as long as necessary
<cjwatson> But it's a new publishing status, significant dominator work, guards around copying and deletion, etc. - not straightforward.  And the last time we tried to turn part of the work in progress on, we discovered a significant roadblock with kernel workflow that forced a rethink
<Laney> Yeah, I've not thought through all of the implications but I can imagine there are many
<Laney> I think it only really matters for the release team when it entangles things in -proposed
<cjwatson> The interaction with -proposed is a bit complex in itself.  I think it will likely involve publications being temporarily copied into -proposed in a Referenced status in order to satisfy Built-Using (this is less confusing than trying to do cross-pocket domination IMO).  Hopefully proposed-migration will just ignore those because they'll be duplicates of publications in the release pocket
#launchpad-dev 2020-07-22
<kkeithley> s390x question: I just did builds of glusterfs-8 in https://launchpad.net/~gluster.   Even though automake is not an explicit requires in control, it was installed.  But 10min later doing glusterfs-7 builds automake is not found and the build fails
<kkeithley> previous glusterfs-7 builds worked on s390x
<kkeithley> but I'm probably in the wrong place for this
<cjwatson> kkeithley: Can you provide a link to the failed build?
<cjwatson> Oh, maybe https://launchpadlibrarian.net/489648063/buildlog_ubuntu-groovy-s390x.glusterfs_7.7-ubuntu1~groovy2_BUILDING.txt.gz?
<kkeithley> yes
<cjwatson> kkeithley: The error is specifically that automake-1.13 is not found
<cjwatson> Get:54 http://ftpmaster.internal/ubuntu groovy/main s390x automake all 1:1.16.2-3ubuntu1 [548 kB]
<cjwatson> kkeithley: This is probably because the source package ships with autoconf/automake output that somebody generated on an old system with automake 1.13.  The usual fix is to use dh-autoreconf to regenerate the build system with modern tools before doing anything else.  I see your package build-depends on dh-autoreconf, but maybe it doesn't actually use it?
<kkeithley> yes, I'm building the .dsc on a focal box
<cjwatson> It's best to make it irrelevant what series you built it in.
<cjwatson> *on
<kkeithley> er, the source_changes
<cjwatson> And indeed, I see that your glusterfs-8 build log mentions running dh_autoreconf, but your glusterfs-7 build log does not.
<cjwatson> That's where I would start investigating.
<cjwatson> It may be automatic depending on debhelper compat version.
<kkeithley> hmm.. okay. I'm not a debian packager by any stretch. let me see what I can figure out.
<kkeithley> thanks for the hint
<cjwatson> Ah yes - your glusterfs-7 package uses debhelper compat level 9 (see debian/compat), while your glusterfs-8 package uses debhelper compat level 11.
<cjwatson> "man debhelper" documents that dh_autoreconf is run by default as of compat level 10.
<kkeithley> indeed
<cjwatson> But the least invasive fix is probably to leave the compat level alone, and instead add "--with autoreconf" (or add autoreconf to an existing dh --with) when using compat <10.
<kkeithley> okay.  fwiw I inherited this job years ago and it has usually run smoothly.  Except when it doesn't. :-)
<cjwatson> Changing the compat level changes various other things, and if you aren't familiar with them it's probably best not to get into that when you're just trying to fix this one issue.
<kkeithley> noted.
<cjwatson> You might need to explicitly add dh-autoreconf to Build-Depends when doing that, too.
<kkeithley> okay
