#launchpad-dev 2010-05-24
 * thumper afk for a bit
<poolie> hi all
<poolie> how do i run a single launchpad test?  i thought there was a test.py script
<mwhudson> poolie: ./bin/test is that script now
<poolie> that doesn't seem to exist
<poolie> ah it's now a generated file
<poolie> or maybe it has been for a long time
<poolie> mwhudson, when I try to run make i get a complaint about missing download-cache
<poolie> and when i run link-external-sourcecode, that complains the egg directory is missing
<poolie> how do i fix that?
<mwhudson> poolie: have you read ./utilities/rocketfuel-setup?
<mwhudson> or even run it, if you're adventurous
<poolie> i thought i had but perhaps not
<poolie> i might try rocketfuel-update
<mwhudson> well, if you don't have download-cache either you never set up this tree fully or it's a very long time since you last used it
<poolie> ah ok
<poolie> i have it in another branch
<poolie> i guess if i'd used rocketfuel-branch or something it would have been copied?
<mwhudson> oh you should be able to use link-external-sourcecode then
<poolie> another question if i may
<poolie> bzr-svn doesn't seem to be in launchpad-dependencies
<mwhudson> maybe you need to give it an argument like ./utilities/link-external-sourcecode ../trunk
<poolie> shouldn't it be?
<poolie> oh ok
<mwhudson> poolie: no, we use a branch of bzr-svn in sourcecode
<mwhudson> we tend to use a newer version than anything packaged (or released, for that matter)
<poolie> ok
<poolie> ok, and now i have that directory, but i get 'no such file or directory' on an egg when i run make and it runs bootstrap.py
<poolie> is there something else i need to run first?
<mwhudson> not that i can think of
<mwhudson> poolie: pastebin the error?
<poolie> http://pastebin.ubuntu.com/438700/
<mwhudson> poolie: maybe bzr update download-cache?
<poolie> ah, i didn't know that was a branch
<mwhudson> yeah it is, for all that that isn't a very good idea
<poolie> mwhudson, that seems to be running now, thanks
<poolie> mwhudson, and is bzr-builder supposed to come from a branch too? i get an import error about that
<mwhudson> poolie: ./utilities/update-sourcecode
<poolie> k
<poolie> jeez you wouldn't want to be in a hurry
<poolie> spm, mwhudson, is this outage known
<poolie> yay, my test fails
<poolie> ok good night
<lifeless> night
<deryck> Morning, all.
<wgrant> deryck: Why not have a root +filebug which just has a unified project/distro search widget and redirects to the right +filebug?
<wgrant> That will redirect to the right Ubuntu wiki page, removes the widget complexity, and removes the bad Ubuntu default.
<wgrant> And makes everyone happy.
<deryck> wgrant, I suppose that's a better version of what was there before.  But is that really *that* much quicker than just searching for the project?
<wgrant> deryck: You can't search for projects.
<wgrant> At least without using the generic search widget on the top of another page.
<wgrant> Yes, it would be awesome if everybody just landed on the project page.
<wgrant> That's clearly how the current UI was designed.
<wgrant> But it's not how things happen.
<wgrant> And this sort of page would be generically useful, across several applications.
<wgrant> (and the unified project/distro search widget itself even more widely applicable)
<deryck> wgrant, then let's open a new bug about that suggestion.  The original bug was about one-click search.
<wgrant> deryck: I do not see the difference. The workflow here is 1) Go to bugs.launchpad.net. 2) Click "Report a bug". 3) Enter search terms. 4) Click on match. This jumps into the workflow that is currently reached with 1) Go to launchpad.net/fooproject. 2) Click "Report a bug".
<deryck> wgrant, ok, I thought you were saying something different.  I would not be in favor of that.  That isn't too far off what was there before.  People still have to know what project to file against, and if they don't, they will file against Ubuntu.
<wgrant> deryck: Why would they file against Ubuntu?
<wgrant> They did so before because it was the default.
<wgrant> If you ignored the form, you got Ubuntu.
<wgrant> With my design (based rather strongly on the design of the AJAX pickers), there is no default.
<wgrant> And if somebody does enter Ubuntu, they will simply be redirected to the usual wiki page.
<deryck> wgrant, ah, I see what you mean now.  I don't think technically we could do a text search like what's on the current dupe search across all bugs and not timeout 100% of the time.  But I'm fine to entertain the idea of the feature.
<wgrant> deryck: The plan was not to do a dupe search at that stage.
<wgrant> Users would select the project or distro, sending them to the normal +filebug for that target.
<deryck> wgrant, so why is this any better than just searching from bugs.launchpad.net now?  because it has the semblance of a guided filebug workflow?
<wgrant> deryck: You can't search from bugs.launchpad.net.
<wgrant> And yes, a guided workflow is approximately an awful lot easier.
<wgrant> If you don't provide guided workflows, you end up like Launchpad.
<deryck> wgrant, you can't search?  there is a big "Search bug reports" form at the top of bugs.lp.net, no?
<wgrant> deryck: The desired functionality is to search for *projects*, not bugs.
<deryck> wgrant, are you personally planning to work on this?
<wgrant> deryck: No.
<wgrant> I'm just saying that Won't Fixing every bug about it is silly.
<wgrant> Because it's a valid concern.
<wgrant> And I've seen it confuse lots of users.
<wgrant> And once the widely useful widget exists, this view is trivial.
<deryck> wgrant, ok, first, you're being a bit too dramatic.  I haven't Won't Fix'ed *every* bug, I've marked one bug this way.
<wgrant> deryck: There have been one or two before.
<deryck> wgrant, that I have marked won't fix?
<wgrant> Bug #162271 comes to mind.
<mup> Bug #162271: search results page doesn't offer link to report a bug <ui> <Launchpad Bugs:Won't Fix> <https://launchpad.net/bugs/162271>
<wgrant> And it comes up on IRC every so often.
<deryck> wgrant, fine.  Update the bug with your suggestions and I'll mark it low.  You win.
<wgrant> Since you seem to feel strongly that I am wrong, I shall not.
<deryck> wgrant, I don't feel that your suggestion is bad.  I feel that this bug is not worth fixing.  I marked the bug as such, and you're challenging my call on that bug.  I can only mark the bug in front of me.  Had you opened a different bug making your suggestion, I doubt very seriously I would have marked it won't fix....
<wgrant> Hmmmmm.
<deryck> wgrant, and I'm saying now if you update the bug with your suggestions, I'll change my mind on the bug.
<wgrant> "Not worth fixing": by that rationale, shouldn't most Low bugs need Won't Fxing?
<wgrant> I understand Won't Fix as a policy decision that a change is inappropriate.
<wgrant> That is how it tends to be used normally.
<wgrant> Although not its original intention.
<deryck> wgrant, so my language is not accurate.  I mean "should not be fixed."  by "worth," I only meant the fix suggested (i.e. renable reporting from the top-level) is not worth the pain it causes.
<deryck> wgrant, I marked it won't fix because if someone submitted a patch to re-enable reporting from the top-level page, we would not accept it.
<deryck> wgrant, and your alternate suggestion is fine, but it's a feature request, not a bug.  And trying to salvage the bug into a feature request is fair enough, and maybe that's why I used the phrase "not worth it" thinking about the whole spectrum of this bug to feature discussion.
<wgrant> But not all solutions to that bug revive the Ubuntu issue. That was indeed a big issue, but it is easy to leave resolved.
<deryck> wgrant, so why not propose an alternate solution in the bug report, rather than taking me to task from my call on this bug?
<wgrant> deryck: I'd not intended to take you to task; I started by merely asking why other potential solutions were not considered. It seemed at the time to be a better idea to check before polluting a nice clean bug on which you had made a policy decision.
<deryck> wgrant, if changing the status of a bug turns it from clean to dirty then we might as well quit reporting bugs.
<deryck> or triaging them
<deryck> or working on launchpad altogether
<wgrant> deryck: I knew it was probably to end with the idea being crushed and the bug flipped back to Won't Fix. I imagined that a couple of lines of discussion here would remove the need for such useless operations. As can be seen, I was quite wrong.
 * jml out to lunch
<adiroiban> leonardr: hi. Do you know how could I debug the lazr.restful when the error  is  ComponentLookupError ? http://paste.ubuntu.com/438961/
<leonardr> adiroiban: it looks like the web service is trying to serialize a field where the field object doesn't exist?
<leonardr> as if you had put foo = exported(None) in your interface instead of eg. foo = exported(Text)
<leonardr> does that make any sense?
<leonardr> if i were you i'd catch the exception in _unmarshallField and see what field it is
<adiroiban> leonardr: thanks. but is there a way for finding how which field can not be serialized?
<leonardr> adiroiban -^
<adiroiban> leonardr: thanks. so there is no other way of finding out which field is raising this error?
<leonardr> adiroiban: ordinarily you would get a clue from the first part of the component lookup--that would be the field object
<leonardr> but here the field object is None
<adiroiban> thanks!
<adiroiban> leonardr: it looks like the error was caused by an exported(List) field http://paste.ubuntu.com/438968/
<adiroiban> Do you think it would make sense to have this patch http://paste.ubuntu.com/438969/ ?
<leonardr> adiroiban: when publishing a list through the web service, you need to use Collection, not List
<leonardr> let me find you an example
<leonardr>     bugtasks = exported(
<leonardr>         CollectionField(
<leonardr>             title=_('BugTasks on this bug, sorted upstream, then '
<leonardr>                     'ubuntu, then other distroseriess.'),
<leonardr>             value_type=Reference(schema=IBugTask),
<leonardr>             readonly=True),
<leonardr>         exported_as='bug_tasks')
<leonardr> from lib/lp/bugs/interfaces/bug.py
<leonardr> lazr.restful.fields.CollectionField
<leonardr> the semantics of a List field are undefined in the web service
<leonardr> same thing for Object -- if you want to relate one object to another you need to use Reference
<adiroiban> yes. and replace IChoice with IReferenceChoice ... but I was puzzled by that âNoneâ field name
<leonardr> i think lazr.restful may have been looking up a deserializer for the .schema
<leonardr> since oyu didn't have a .schema that was None
<adiroiban> and I was asking if you think that lazr.restfull could be improved and give a better error message
<leonardr> sure it can
<leonardr> ah, sorry
<leonardr> i was looking at the wrong paste
<mtaylor> thumper: hey, up yet?
<thumper> mtaylor: hey
<thumper> mtaylor: up but need some food
<thumper> mtaylor: a somewhat frustrating morning
<mtaylor> thumper: no prob... I'm here all day :)
<mtaylor> oh, that's no good
<mtaylor> you should eat! eating makes it better
 * thumper nods
<mtaylor> (so does coffee for me, but not everyone is as obsessed as I am)
 * thumper goes to make some bacon and egg muffins with espresso
<mtaylor> mmm
<thumper> mtaylor: you've got me one handed for a few minutes while I eat
<thumper> whazzup?
<lifeless> nom nom nomification
<mtaylor> thumper: so, an unnamed organization I know of is going to be hosting some open source development on launchpad
<mtaylor> thumper: and amongst the things entailed in that process are going to be a CLA
<thumper> mtaylor: ok
<thumper> CLA?
<mtaylor> contributor license agreement
<mtaylor> which is one of those "need to sign this form before you can contribute" sorts of thing
 * thumper nods
<mtaylor> I was trying to figure out if/how that process could be integrated into launchpad
<thumper> well, it is something we've been talking about in the past
<mtaylor> which made me think of the (currently one-off hardcoded) ubuntu code of conduct
<thumper> but I'm not entirely up with the play
<thumper> ask sinzui
<lifeless> o
<lifeless> so
<lifeless> lp has the CoC for Ubuntu
<sinzui> mtaylor, CoC is a flatfile with a table mapping the version signed
<lifeless> I *think* that a generalisation of that, to make multiple cocs available might be a reasonable thing
<mtaylor> lifeless: that was the real question - I'm happy to write the code for that, if it's something that people might be interested in
<lifeless> but I *don't know* if the concept of scaling to 1-per-project would be expected ;)
<lifeless> sinzui: ^
<lifeless> or even N per project, because Ubuntu related projects have several cocs to choose from
<sinzui> mtaylor, The model classes assume that all CoC are Ubuntu. I considered adding a naming convention test to permit multiple CoCs for canonical projects.
<lifeless> (oroginal, rev 2, leadership)
<sinzui> mtaylor, I think we need a real DB class to manage file content and versions if we let any project have a CoC
<mtaylor> sinzui: yes. I believe adding them as flat-files with mapping would be the wrong way
<lifeless> sinzui: is there any reason, in principle, not to accept a patch that does that to your standard ?
<lifeless> rephrasing:- Would a patch that generalises this and makes it editable via the web UI for all open source projects be accepted [if of appropriate quality]
<sinzui> lifeless, mtaylor we would love patches to improve CoC to support more than Ubuntu
<lifeless> mtaylor: there you go
<mtaylor> sinzui: awesome! I will write up some thoughts on how I might go about that and see what you think
<sinzui> thanks
<mtaylor> sinzui: how soon would I need to have code reviewed and accepted for it to land in a running launchpad by early july?
<sinzui> mtaylor, I think 2010-06-18
<mtaylor> sinzui: thanks
#launchpad-dev 2010-05-25
<thumper> where'd my damn code go....
<thumper> :-(
<poolie> thumper, ?
<thumper> installing repoze.what causes Launchpad not to build
<thumper> no freaking idea why
<thumper> from the import site part of our scripts
<thumper> bin/py to get an interactive interpreter
<thumper> then "import site" blows up
<poolie> ouch.  i'm glad it's not my fault though :)
<thumper> apt-get remove python-repoze.what fixed it
<thumper> for now
<thumper> but we should look into it
 * thumper files a bug on foundations
 * thumper afk to get Maia and have lunch
 * thumper goes to make dinner, back to talk to some europeans
<adeuring> good morning
<bigjools> good morning
<jelmer> 'morning bigjools
<bigjools> hey jelmer
<mrevell> Hello!
<jelmer> 'morning Matt
<bigjools> jelmer: hey you're not on the internal server
<deryck> Morning, all.
<adeuring> morning deryck
<SmartViking> Launchpad power
<SmartViking> Hi everyone! :D
<SmartViking> No one want to say hi to the lonely viking :(
<SmartViking> sidnei hi! :D
<sidnei> hi
<SmartViking> Why are people in this channel if they dont talk? :)
<mars> rockstar, ping
<rockstar> mars, pong
<mars> hi rockstar, did you have some time to chat today about the windmill tests?  You said on Friday that you may have a local reproduction of the hang.
<rockstar> mars, I may have some time, but not for the next few hours.
<rockstar> mars, the basic idea is to run windmill without a $DISPLAY
<mars> rockstar, that's fine, I'll be around :)
<mars> rockstar, interesting
<rockstar> I suspect that whatever fake X11 server we're running is broken.
<mars> maybe.  I did not check for an xvfb.log on the hung servers.  That would have been useful.
<mars> rockstar, unfortunately the test_on_merge.py issues obliterated the crime scene - before this, we would not have been able to see any X server issues, because the server was being forcibly killed
<mars> rockstar, when I reproduce this on EC2, that will be one of the first things I look for then.  That is a good hypothesis.
<rockstar> mars, I know for a fact that it hangs here, and it looks exactly like it looked on ec2.
<mars> rockstar, did you just unset $DISPLAY to get it to happen?
<rockstar> mars, I run launchpad in a chroot, so I have to set $DISPLAY to run the windmill tests.  It was just pure coincidence that I forgot to set it.
<mars> rockstar, ok, thanks for the insight
<deryck> BjornT, ping.
<mrevell> Night all :)
<Guest99246> hello how do you remove everything installed by rocketfuel (launchpad's source)
<matthew_> does any body know how to remove it?
<sinzui> deryck, ping
<deryck> hi sinzui.  Was just about to log out for EOD, though.
<sinzui> deryck, I extracted a mixin class from bug supervisor that will be used in 3 different modules...
<sinzui> deryck, Do you want this mixin in lp.bugs.brower.__init__ or lp.bugs.browser.bugroles
<deryck> hmmm, bugroles kind of seems nice.
<deryck> sinzui, yeah, let's put it in bugroles
<sinzui> I will create it and move the mix there
<deryck> sinzui, thanks!  Looking forward to this refactor.
<thumper> sinzui: ping
<krkhan> in a locally running launchpad, i'm getting the 'unknown consumer' error on all REST reqs. do i have to add known consumers somewhere in lp?
<bryceh> what happened to lib/canonical/launchpad/doc/security-teams.txt ?  It seems to no longer be in the current tree?
<sinzui> hi thumper
<thumper> sinzui: hey, got some time for a call?
<sinzui> I do not for the next hour. I am interviewing in 9 minutes
<sinzui> do you want my rs to remove something I despise?
<thumper> heh
<thumper> project groups?
<sinzui> They are on my list of papercuts
<sinzui> thumper, do you want the go work or go away
<thumper> sinzui: I don't get that last statement
<sinzui> Users use PGs to show affiliations. Drivers use PGs to manage permissions and see milestones. The problem is that since the user controls the linking, the driver has not control over what he controls. eg any user can add their project to Launchpad-Project, forcing the drivers to see unwanted milestones
<thumper> sinzui: I want project groups to be entirely different
<thumper> sinzui: I want project groups to be controlled by the project group creator, easier to create, and they can add whatever projects they want to their group
<thumper> sinzui: it allows a user to provide a unified view over projects they care about
<sinzui> ...and create series and milestones that propogate to the projects!
<thumper> in some cases maybe
<thumper> it it just happens to be a group of projects that I depend on, then no
<sinzui> landscape and OEM have asked for that several times. So has mrevel
<sinzui> l
<wgrant> If you need your series and milestones to propogate, you probably want components rather than projects.
<sinzui> Yes I want components. I am not sure I will ever get them
<thumper> I want components
<thumper> I'd like launchpad to use launchpad like we tell others to
<wgrant> YES.
<thumper> if we can't dogfood it for what we want,
<thumper> we should fix it
<sinzui> lets start a list of papercuts and pimp that for the epix
<thumper> sinzui: sounds good to me
<thumper> sinzui: I'm heading out briefly(ish) for a coffee, but I'd love a talk - I have some sekrit plans
#launchpad-dev 2010-05-26
<lifeless> mwhudson: ping; how do I tell if a given loggerhead rev is is production?
<lifeless> thumper: https://bugs.edge.launchpad.net/loggerhead/+bug/518689 - does that need to be private? Seems like no to me. Web crawlers and users can hit that easily enough
<mwhudson> lifeless: production only lags lp:~launchpad-pqm/loggerhead/devel by at most 24 hours
<lifeless> and if someone were to dos it we can filter
<lifeless> mwhudson: its auto rolled out ?
<mwhudson> lifeless: if you need more precision than that, ask someone who can log into guava i guess (that includes me)
<lifeless> mwhudson: oh, I see, I need to do missing between the two
<mwhudson> lifeless: yes
<lifeless> mwhudson: is lp-pqm's branch a series ?
<mwhudson> lifeless: no, maybe it should be though
<thumper> lifeless: Logging as a sec vuln as this is a rather easy way to currently DOS codebrowse.
<thumper> lifeless: that was from the bug comment
<lifeless> yeah
<lifeless> I'm saying I disagree
<lifeless> using loggerhead is an easy way to DOS is
<lifeless> s/is/it
<mwhudson> wasn't that the pygments problem?
<thumper> lifeless: ok, fair enough
<lifeless> I'm trying to decide between
<lifeless> a) public or b) subscribe max
<thumper> mwhudson: I think it may have been
<lifeless> mwhudson: yes, but root cause suggests thread pool limits are to blame too
<thumper> mwhudson: do we have a max pygments limit yet?
<lifeless> huh
<lifeless> I can't reject https://code.edge.launchpad.net/~loggerhead-team/loggerhead/trunk-rich/+merge/26010
<mwhudson> lifeless: man, the thread pool stuff is so messed up, please fix that
<lifeless> which is silly, as I created it.
<mwhudson> thumper: yes, i think so
<lifeless> is there a bug open about the permissions there?
<thumper> lifeless: it is a known issue
<lifeless> shall I file a bug ?
<thumper> lifeless: and somewhat releated to the proposal status
<thumper> and that we don't have a withdrawn
<lifeless> yes, I can see the place in the code
<thumper> why not delete it?
<thumper> does it have any real value?
<lifeless> well, I wanted to be able to point ppl to it :P thats ok, deleted it
<thumper> lifeless: lets subscribe max for now
<thumper> what is his lp id?
<mwhudson> mkanat i think
<thumper> subscribed
<lifeless> thumper: already openned it based on your ok, fair enough able
<lifeless> *above*
<lifeless> mwhudson: the paste guts seem clear enough, it knows how many total threads, it should be able to have a hard cap.
<mwhudson> lifeless: but it has this strange concept of 'hung' threads and if threads are 'hung' it spawns more
<lifeless> yes
<lifeless> the specific check is
<lifeless> spawn_if_under
<lifeless> + hung threads don't count as workers
<mwhudson> lifeless: probably the limits are way too generous, we should kill threads more aggressively
<lifeless> that too, but killing threads is likely to show up locking bugs - mutex ownership issues, etc.
<lifeless> losa
<lifeless> how many requests/sec does loggerhead suffer?
<mwhudson> the limits are set in scripts/start-loggerhead.py
<mwhudson> in the launchpad tree
<lifeless> grah
<mwhudson> lifeless: https://lpstats.canonical.com/graphs/CodebrowseHTTPResponses/
<lifeless> thats backend?
<lifeless> man
<lifeless> I was going to book plane tickets First Thing today
<mwhudson> lifeless: um, it's extracted from the apache logs on the frontend
<lifeless> theres no squid yet?
<lifeless> if there was, we'd need a separate graph.
<mwhudson> no
<mwhudson> well, at least if there is, noone's told me about it
<poolie> lifeless, i believe the critical incident policy tells you who's meant to follow up etc
<lifeless> hmm, lpstats taking -forever-
<lifeless> mwhudson: I can't see that graph
<lifeless> mwhudson: can you just tell me ?
<mwhudson> lifeless: "less than 1 req/s" would seem to be the summary
<lifeless> would measuring in minutes be better ?
<lifeless> 'd like a >1 figure, so I can do math.
<mwhudson> lifeless: between 80 and 180 per 5 minutes
<mwhudson> so that's, errrrrrrr, 16-36 per minute?
<lifeless> 1/4-1/2 per second
<mwhudson> the lack of robots.txt pushed it to ~1 per second and everything fell over
<lifeless> I have a theory
<lifeless> actually, robots.txt hit a bad url on many branches
<lifeless> and that pushed it over
<lifeless> http://trac.pythonpaste.org/pythonpaste/ticket/416
<mwhudson> yeah, annotate of files with deep history would be a good one to pound if you want to take codebrowse out
<mwhudson> lifeless: huh!
<mwhudson> lifeless: the thread killing does work at least sometimes
<lifeless> so I'd rather say 'lh handled 4 times the load fine, except that we have a bug on some urls'
<lifeless> mwhudson: possibly with low-valued threadids, or something.
<mwhudson> this is another thing we should do, of course: change the default view from the filelisting to be a simple listing, not annotate
<lifeless> mwhudson: is there a bug open on that ?
<lifeless> if now, can you -please- open
<lifeless> also, are you happy with all of johns work landing on LP ?
<mwhudson> yes, but we should try to pre-seed the history.db for a few select large branches first
<mwhudson> like launchpad, mysql, the kernel
<lifeless> I'm mainly checking we don't need a seperate 'for lp' branch for Max
<mwhudson> lifeless: t
<mwhudson> grr
<mwhudson> https://bugs.edge.launchpad.net/loggerhead/+bug/568148
<mup> Bug #568148: Default view for a file should be its content <performance> <loggerhead:Confirmed> <https://launchpad.net/bugs/568148>
<lifeless> thumper: ping
<thumper> lifeless: pong (although leaving the office to eat)
<lifeless> thumper: hi
<lifeless> so
<thumper> lifeless: leave a message and I'll get back to you as soon as possible :)
<lifeless> I had a bug asking about getting allist of broken-due-to-upgraded-trunk branches
<lifeless> you've closed the bug
<lifeless> should I file a fresh dedicated one ?
<thumper> yes
<lifeless> ok will do
<zyga> hmm
<zyga> https://blueprints.edge.launchpad.net/ubuntu-arm/+spec/arm-m-validation-dashboard
<zyga> (it was there a moment ago, now it's an OOPS)
<zyga> OOPS-1607ED337
<zyga> can anyone help with this please?
<zyga> bah, ok - the spec was renamed, sorry for causing noise
<zyga> (still - the search did link to the wrong place)
<mrevell> Morning
<maxb> Ursinha-afk: Have you considered running your bugbot as a separate LP person, so that it is obvious that its actions are automated?
<deryck> Morning, all.
<wgrant> bigjools: AAAAAAA. https://edge.launchpad.net/ubuntu/+source/vowpal-wabbit/4.1+20100420-1/+build/1755930 is scary on several levels.
<bigjools> yes, we've seen it
<maxb> "special" :-)
<bigjools> I thought we'd fixed this by stopping queue-builder :/
<wgrant> We must have fixed at least 5 of this sort of bug in the last six months.
<wgrant> At least it will become impossible soon.
<bigjools> well "fixed" is a strong word here :)
<wgrant> For that particular one, yes.
<bigjools> wgrant: so
<bigjools> it's the result of a give-back
<bigjools> why the builder does that instead of failing the build is somewhat odd
<zyga> is there any SQL query that can extract a small subset of a big text field in a more efficient way (preferably without having to copy the whole field form the db to the client)
<wgrant> bigjools: Yeah, I worked that out quite a while ago.
<zyga> assuming I know the start/end offsets (in bytes/characters)
<wgrant> It's the 'Illegal instruction' in the log.
<wgrant> I forgot that we just fixed it so that it doesn't break b-m; the display is still all broken.
<bigjools> I need to kill that build anyway
<bigjools> I think that problem should also be a failed build, not a give-back
<wgrant> Maybe it expects that some builders might be able to build some stuff.
<wgrant> (like, say, armel)
<wgrant> But that's an utterly stupid way of arranging that.
<wgrant> So, yes, a quick deletion of that line will fix things.
<wgrant> I can't think of any practical legit uses.
<deryck> sinzui, please feel free to remove the bugs test, if the same condition is tested in test_securitycontact.py
<sinzui> deryck, I did. I unconsciously knew it was a duplicate. It was a simple verification of the form label and that the form works. I wonder why it passed on ec2?
<deryck> yeah, that seems odd that it would pass there.  not sure why.
<Ursinha> bigjools, hi
<bigjools> Ursinha: heyu
<bigjools> or something that might be spelled better
<Ursinha> bigjools, :)
<Ursinha> bigjools, I have an oops and think you can help: https://lp-oops.canonical.com/oops.py/?oopsid=OOPS-1600E1649
<bigjools> Ursinha: that's the same as the one you told me about a while ago
<bigjools> Ursinha: noodles775 is looking into it, feel free to remind him :)
<Ursinha> hehe
<Ursinha> hahahahaha
<Ursinha> bigjools, he heard you
<bigjools> Ursinha: actually didn't you already file a bug about it?  I think he commented on there
<Ursinha> bigjools, I filed a bug that is a NotOneError
<bigjools> it's basically down to a previous bug where sun-java6 ended up in partner and main at the same time
<Ursinha> just found it
<Ursinha> bug 580181
<mup> Bug #580181:  DistributionSourcePackageRelease page still oopsing with NotOneError <oops> <Soyuz:Triaged> <https://launchpad.net/bugs/580181>
<Ursinha> bigjools, OOPS-1600E1649 is a LocationError
<Ursinha> are they the same?
<bigjools> Ursinha: it's essentially the same underlying problem
<bigjools> if you could reference this oops on that bug it would be helpful
<Ursinha> bigjools, surely
<bigjools> "Past week count: 686"
<bigjools> eek
<Ursinha> bigjools, if noodles785 could fix it, it would be helpful as well :)
<bigjools> :)
<Ursinha> bigjools, yesterday's count: 701
<bigjools> hmm no referrer
<noodles785> bigjools, Ursinha: I haven't been looking into it... I just did some initial investigation, but can look into it tomorrow.
<Ursinha> noodles785, that would be appreciated, thanks :)
<bigjools> noodles785: cheers
<Ursinha> bigjools, thanks :)
<bigjools> Ursinha: any time
<mrevell> Righto peoples, I'm off. Ta ra.
<Alkini> how active is the search for a Launchpad Web Engineer (http://webapps.ubuntu.com/employment/canonical_LPWE/)?
<beuno> Alkini, what do you mean?
<Alkini> has the position just been sitting around for six months? or are a dozen people a day being considered?
<beuno> Alkini, it's been opened up recently
<beuno> and some people are being interviewed
<beuno> not super sure what you're getting at :)
<Alkini> I just applied today and don't know much beyond the "you might not hear back from us for three weeks" email so I just thought I'd ask :-)
<beuno> well, it takes a day or two for the CV to be passed on from HR
<beuno> and then it depends on the availability of the team lead
<Alkini> sure, totally understandbale; I didn't mean to be impatient, just curious
<beuno> I'm sure it's a position eager to be filled, so it'll probably be sooner than later
<Alkini> alright, cool
<Alkini> it's, presumably, the same story for the software engineer reporting to the launchpad code team lead? (http://webapps.ubuntu.com/employment/canonical_LP-SEC/)
<beuno> well, different position, different people involved, but same HR process
<Alkini> right
<Alkini> you're on the ubuntu one team these days?
<beuno> Alkini, yes I am, it's been a good 3 or 4 months now
<beuno> still love Launchpad though  :)
<Alkini> heh
#launchpad-dev 2010-05-27
<thumper> I have a feeling that this branch is going to be large
<lifeless> this==?
<thumper> this == the branch that adds subscribed_by to BranchSubscription
 * thumper wants to delete the branch-notifications test
<thumper> however not just yet
<poolie> lifeless, feed-pqm would be nicer in gtk i think, where you can directly point to the thing you want
<poolie> is this what you were heading for?
<lifeless> poolie: not so much
<lifeless> lptools has a clickable list already
<lifeless> if you're heading in that direction I *highly* encourage you to build on that
<henninge> Hello! ;)
<henninge> Can packaging links be removed again?
<wgrant> henninge: there are at least delete links on https://launchpad.net/ubuntu/+source/SOMEPACKAGE
<wgrant> I'm not sure about the other end.
<wgrant> noodles775: Morning.
<noodles775> Hi wgrant :)
<wgrant> noodles775: Could you please re-EC2 lp:~wgrant/launchpad/bug-558905-get-archive? It failed a week or so ago now in the MailmanLayer breakage, and I forgot to deal with it.
<noodles775> wgrant: sure. Sending now.
<wgrant> Thanks.
<henninge> wgrant: thanks, the relevant method I was looking for is in PackagingUtil.
<wgrant> henninge: Ah, code-wise, I see. Sorry.
<henninge> wgrant: no, no, don't be. I found the code through the view which I found through your link ... ;-)
<henninge> didn't know about PackaginUtil before
<mrevell> Morning
<wgrant> Have we lost half the test suite again?
<wgrant> That was a sub-3-hour EC2 run.
<deryck> wgrant, windmill tests are not run in ec2 now.
<deryck> and something else.  mailman tests maybe?
<maxb> I don't think the mailman ones ever were (recently)
<wgrant> MailmanLayer hasn't ever been run, except for a few hours two weeks ago.
<wgrant> But the Windmill thing is a good point.
<krkhan> has anyone here communicated with a locally running launchpad over rest?
<wgrant> krkhan: yes. Drop the 'api.' from the domain you're using.
<wgrant> edge.launchpad.net -> launchpad.dev
<wgrant> api.edge.launchpad.net -> api.launchpad.dev
<krkhan> wgrant: thanks, let me try
<krkhan> wgrant: worked like a charm :-) thanks again
<wgrant> krkhan: Excellent.
<BjornT> matsubara: ping
<matsubara> BjornT, hi
<BjornT> matsubara: i'm trying to get the oops-tools running locally. i've followed the instructions in the readme.txt, but when i go to /oops, i get ProgrammingError: relation "oops_oops" does not exist
<matsubara> BjornT, did you run bin/django syncdb?
<BjornT> matsubara: yes. the output looked a bit odd, though. let me get you a paste...
<jelmer_> Are there any plans to switch to python 2.6 in the near future ? I vaguely recall seeing something but don't remember where.
<matsubara> jelmer_, gary_poster is working on that
<gary_poster> jelmer_, matsubara, yeah, at least in a supervisory capacity.  here are the plans
<BjornT> matsubara: nm. i guess i should do what the output tells me to do (although, it said to use manage.py, which doesn't exist)
<gary_poster> Well, don't know how much detail you want :-)
<BjornT> matsubara: fwiw, the output was this: http://paste.ubuntu.com/440408/
<gary_poster> The summary is that we plan to switch to Python 2.6 and Lucid for 10.06, the release after this one.
<matsubara> BjornT, when it says manage.py use bin/django instead.
<gary_poster> jelmer/jelmer_: ^^^
<matsubara> BjornT, bin/django is a buildout wrapper for manager.py
<jelmer_> gary_poster: ah - thanks!
<matsubara> BjornT, right, now you just bin/django migrate and then you should be good
<gary_poster> np :-)
<BjornT> matsubara: now, how do i actually use it? :)
<matsubara> BjornT, do you think the README.txt could be improved? Would you add the missing steps to it and provide a diff?
<matsubara> BjornT, make run
<BjornT> matsubara: yes, i plan to add what's missing, if anything (e.g. the migrate part)
<BjornT> matsubara: right, it's running. and then?
<matsubara> BjornT, you probably want to add some sample data to it, so `make load_sample_data` first and then `make run`
<BjornT> matsubara: well, i want to load the oopses in /var/tmp/lperr
<matsubara> BjornT, then you can point your browser to localhost:8000/oops
<matsubara> BjornT, if you know the OOPS id, you can enter it in the search oops field, click the button and it should give you the OOPS page
<matsubara> BjornT, or do you want a summary of all oopses in /var/tmp/lperr?
<BjornT> matsubara: i do know the oops id. it doesn't find any oops with that id
<BjornT> matsubara: getting a summary would be useful as well
<matsubara> BjornT, can you make sure /var/tmp/lperr is in src/oopstools/settings.py ?
<matsubara> BjornT, oh nm
<matsubara> it probably is and I know why you're not finding those oopses
<matsubara> s/you're/oops-tools/
<matsubara> BjornT, can you try bin/update_db and see if loads the oopses you want?
<BjornT> matsubara: tried that already: Loaded 0 OOPS into the database.
<matsubara> BjornT, what are the dirs you have in /var/tmp/lperr?
<BjornT> matsubara: 2010-05-19 and 2010-05-26
<matsubara> BjornT, can you try this patch: https://pastebin.canonical.com/32723/ and then bin/update_db again (also check that you do /var/tmp/lperr in settings.py, it should be but it doesn't hurt to double check)?
<adiroiban> any idea why I get this error â We only know about GET, HEAD, and POSTâ when trying to save an entry using launchpadlib ? the +apidoc page for that entry specify that it support PATCH method and entry.lp_save() is using PATCH for updating that entry.
<BjornT> matsubara: ok, that work (after changing datetime.datetime to datetime.date)
<matsubara> BjornT, cool
<matsubara> BjornT, to generate the summary you can: bin/analyse_error_reports -pX --html=<path-to-html-summary> --text=<path-to-text-summary> --from=2010-05-18 --db-summary
<matsubara> BjornT, I'm not sure the summary/ view would work for X prefixed oopses
<gmb> sinzui, Can I get a preemptive RC for https://code.edge.launchpad.net/~gmb/launchpad/stored-proc-for-bug-heat-bug-582195/+merge/26064 please? It has been db-approved by BjornT and stub
<gmb> sinzui, It's part one of a two-part fix for bug heat calculation taking too damn long.
 * sinzui knows your pain
<sinzui> gmb, you have my rc
<gmb> sinzui, Thank you.
<Ursinha> Chex, gary_poster, rockstar, bigjools, danilos, sinzui: production meeting on #launchpad-meeting @ Freenode in 12 minutes
<gary_poster> thank you
<danilos> Ursinha, can you move it for 17 mins please?
<Ursinha> 17 mins?
<danilos> Ursinha, oh, it doesn't work like that? ok then, in 11 minutes
<Ursinha> danilos, if everyone else agree with that, sure
<danilos> Ursinha, no, no, j/k
<Ursinha> danilos, .... ok _boss_, sure
<Ursinha> :P
<adiroiban> henninge, danilos : Hi. Any idea why shortlist is used here http://paste.ubuntu.com/440458/?
<danilos> adiroiban, because it's a smarter thing to use than to use list() which may produce gobbles of information
<adiroiban> danilos: but if the list is bigger than 300 elements
<danilos> adiroiban, I am not sure why are we not returning a resultset though, but I'd have to read more code for that
<adiroiban> I get a warning
<danilos> adiroiban, yes, it will traceback
<adiroiban> is that ok ?
<danilos> adiroiban, that's the idea
<lifeless> edge is on a go-slow
<lifeless> anyone else noticed ?
<lifeless> not timing out, but it must be damn close to it
<adiroiban> danilos: I have create 4000 templates and using the API, I only get a warning (no traceback) and the batch of 300 templates is fast in the API
<adiroiban> danilos: +templates from the browser is taking 5 minutes to render them
<danilos> adiroiban, +templates in the browser is broken, one of the reasons is your introduction of using non-cached data with getPOTMsgSets().count(), but it has other problems as well
<danilos> adiroiban, anyway, I have a meeting now, sorry
<adiroiban> danilos: np. thanks!
<danilos> adiroiban, anyway, have you checked that somebody without permissions can't really touch potemplates?
<adiroiban> danilos: yes
<danilos> adiroiban, cool, then I'll have another look at your branch tomorrow; 300 seems a reasonable limit for productseries, but I'll have to check why we are not returning a resultset there instead
<adiroiban> danilos: the limit in shorlist is only âinformativeâ
<adiroiban> the full result is returned as a list
<adiroiban> and a warning is issued if the limit is exceded
<adiroiban> also the +templates page is not using getTranslationTemplates, but rather callling getUtilit(IPOTemplateSet).getSubset() directly
<adiroiban> right now the API is the only user for the getTranslationTemplates method
<danilos> adiroiban, ah, right
<danilos> adiroiban, we should probably use a hardlimit there then
<danilos> adiroiban, that's very weird
<danilos> adiroiban, all of that should be fixed
<adiroiban> danilos: getTranslationTemplates is defined in many places, but only used in lp/registry/templates/distroseries-packaging.pt
<adiroiban> and in that template, it is used in a very strange way
<adiroiban> assigned to a variable in a tal:define
<adiroiban> but that variable is not used later
<danilos> adiroiban, it's used in a distroseries copy, and we should probably move the +templates page to it as well
<sinzui> adiroiban, I discovered that two months ago...
<adiroiban> danilos: ok. maybe we can add pagination to +templates and by doing so avoid the timeout
<danilos> adiroiban, I'd like to try to make it work without pagination first: with pagination, we'll have to hack sorting in and I'd rather avoid that
<sinzui> adiroiban, danilos. Edwin is working on a model fix that will remove my need for the method
<adiroiban> sinzui. ok. thanks!
<adiroiban> danilos: regarding the soft/hard limit on shortlist. Do we realy need it?
<danilos> adiroiban, yes, because we know we'll time out if we try to process that many, so we can either: 1) ensure we won't time out with unlimited number of entries 2) introduce batching 3) introduce a hard-limit so we see in the OOPSes that our users have started needing it
<danilos> 1) is very hard to do, but we can probably raise the limit to a few thousand (thus satisfying ubuntu as well)
<adiroiban> danilos: in distroseries, the softlimit is 2000
<adiroiban> on for productseries and sourcepackage it is set to 300
<danilos> adiroiban, but that page has never worked, and it has gotten only worse
<danilos> adiroiban, all of those sound like sane limits to me (and they should be hardlimits at that)
<adiroiban> danilos: ok. but from what I think, with or without the hard limit, the page will crash anyways if it tries to get a big number of templates
<danilos> adiroiban, yes, and with one approach we'll be using a lot of memory and cpu cycles on our servers (thus exposing a possible, albeit unlikely, DoS attack vector), and in another we'll be giving an error to the user instantly
<danilos> adiroiban, when it comes to "choose one", I know which one I am choosing :)
<adiroiban> :)
<adiroiban> danilos: but, then we need to change the implementation of shortlist
<adiroiban> danilos: sorry. nope. just use softlimit instead of hardlimit
<sinzui> danilos, adiroiban: We are adding a sum of templates to distrosourcepackage that is cached so that we will not need to do a lot of work for getTranslationTemplates()
<mrevell> Righto, friends, I'm offski
<jelmer_> are there any known issues with oopses not being cleaned up properly when tearing down tests?
<matthew_> how do you remove launchpad from your computer (rocketfuel) ?
<jelmer_> matthew_: I don't think we have anything that does that at the moment.
<matthew_> do you know what dependencies that it installes then i can remove them manually?
<jelmer_> matthew_: for the packages at least you can deinstall launchpad-dependencies, launchpad-developer-dependencies, launchpad-database-dependencies and launchpad-soyuz-dependencies
<matthew_> and is all the downloaded items just in /home/matthew/launchpad?
<matthew_> ok removed thoese and then did a sudo apt-get autoremove
<matthew_> *those
<jelmer_> matthew_: in ~/launchpad and it installs some scripts (all starting with rocketfuel- I believe) in /usr/local/bin/, adds some entries to /etc/hosts and adds some files to /etc/apache2 IIRC
<matthew_> thanks
<Ursinha> rockstar, bug 586467
<mup> Bug #586467: upgrade_branches script failing  <Launchpad Bazaar Integration:New> <https://launchpad.net/bugs/586467>
<rockstar> Ursinha, ta
<mars> abentley, ping, did using xvfb and 'make check' work for you as a substitute for VNC?  Your mail does say whether switching got you back up and running again or not.
<abentley> mars, yes it works for me.
<mars> abentley, great.  May have been a known issue in Karmic.  IIRC they fixed xvfb in August or September.
<lifeless> mars: what was xvfb doing ?
<abentley> mars, I am running Lucid.
<mars> abentley, didn't you say you switched to VNC to get around a bug in Xvfb/Karmic that was stopping you?
<abentley> mars, when I was using Karmic, I switched to VNC.  When I switched to Lucid, I continued using VNC.
<mars> lifeless, don't remember, just that a fix landed in the repos literally minutes before one of our developers needed it :)
<mars> abentley, ah, ok.
<gmb> sinzui, Can I get a pre-emptive RC for this https://code.edge.launchpad.net/~gmb/launchpad/hook-up-stored-proc-boom-bug-585803/+merge/26191 please? It's the second part of project-make-bug-heat-not-suck.
<sinzui> gmb, you have my rc. since buildbot threw a wobbly you may still need to use it with PQM submit if ec2 is faster than buildbot
<gmb> sinzui, Yeah, that's why I asked now :)
<gmb> Thanks
<bryceh> I got this error message, am I interpreting it correctly as trying to say, "Your branch could not be landed because PQM is turned off now"?
<bryceh> []
<bryceh> Command failed!
<bryceh> All lines of log output:'Commit message [[r=intellectronica][ui=none][bug=373508,\n\t455203] Flesh out docs
<bryceh> for interfaces/bugs.py and fix various\n\tspelling errors.] does not match commit_re
<bryceh> [(?is)^\\s*\\[testfix\\]\\s*\\[(?:release-critical=[^\\]]+|rs?=[^\\]]+)\\]]'
<thumper> bryceh: no, it needs a testfix
<thumper> which means there is a broken test on trunk
<thumper> (or buildbot had some other error)
<bdmurray> Where can I find the value of config.launchpad.default_batch_size ?
<bryceh> thumper, hum odd, I got a "Test results: SUCCESS" at the same time as the above
<bryceh> and ec2 test has passed ok on this branch previously to that
<thumper> bryceh: ec2 isn't running all the tests right now
<bryceh> oh
<bryceh> I assumed it was running all the tests...  Is there a way to run all the tests locally or something so I can see what the problem is?  The above error doesn't really give me enough clue as to what's wrong.
<bryceh> maybe I should just wait and try again after PQM is back on
<jelmer> bryceh: hi
<bryceh> hi jelmer
<jelmer> bryceh: I saw you did some work on Launchpad support for gtg
<bryceh> yes
<jelmer> bryceh: What were you working on exactly? I'm looking for a way to integrate my assigned bugs into gtg
<bryceh> jelmer, two things
<jelmer> mwhudson, thumper: hi
<thumper> jelmer: hi
<jelmer> thumper: What's the policy for landing things on ~launchpad-pqm/{dulwich,bzr-git,bzr-svn,subvertpy}/devel exactly?
<thumper> jelmer: relatively lax given that we need to adjust the revnos in the launchpad branch
<thumper> jelmer: if you have something that will work, land it
<thumper> jelmer: the justification comes in the LP branch change
<jelmer> thumper: ok - thanks!
<jelmer> thumper: I'll try and land as part of the releases processes, that way we get the benefit of the QA that happens before a bzr-* release.
#launchpad-dev 2010-05-28
<poolie> jelmer, what's the eta for testing daily builds for real?
<poolie> by which i mean on production launchpad?
<jelmer> poolie: basically we're waiting for IS to open some ports
<jelmer> poolie: That would make it possible to use on edge/staging. I'm not sure when it'll be enabled on production, but I imagine it would be soon afterwards unless we find things that need to be fixed.
<sinzui> thumper, I am starting the next review...
<thumper> sinzui: ta
<sinzui> I am concerned about ec2. I see a failure that must be from my branch, but ec2 sent me a success message
<wgrant> Well, Windmill is switched off in EC2 at the moment, so that was always going to happen.
<sinzui> This happened last Thursday too. my confidence is shaken
<sinzui> I think lib/lp/blueprints/tests/../stories/standalone/xx-personviews.txt and lib/lp/soyuz/tests/../stories/soyuz/xx-distributionsourcepackagerelease-pages.tx are broken in trunk
<sinzui> The last one was run by StevenK before he sent it off to  PQM. it had unicode in it which cause ec2 to barf
<rockstar> Is there a way I can run a single test in ec2?  It's one that fails with the GPGmeError on my local box.
<wgrant> Erm, what was the failure in the latter file?
<wgrant> That looks like it could be one of mine.
<wgrant> rockstar: What's the gpgme error?
<wgrant> Is it the one with the one-liner fix?
<sinzui> I want a thunder dome form test runners, pqm vs ec2 vs buildbot. 3 runners enter one leaves
<rockstar> wgrant, I don't remember specifically, but it's a known issue (one that abentley spent some time trying to sort out)
<lifeless> sinzui: you forgot hudson
<lifeless> 2 in 1 out, 2 in 1 out, 2 in 1 out :)
<lifeless> thumper: I'd like a link to activereviews on https://edge.launchpad.net/lmirror/trunk
<lifeless> thumper: what do you think?
<rockstar> lifeless, no tournament, battle royale!  :)
<sinzui> wgrant, the fault was smart quotes pasted into actual test. the test was not loaded run because of a decode error. StevenK ellipsisised the quotes and ran the test locally before submitting it.
<wgrant> sinzui: Yeah, and it is now broken because of my change last night.
<sinzui> I am preparing a fix for the two tests...if the error really is just the tests
<wgrant> So it wasn't running before?
<sinzui> right, it did not run.
<sinzui> I have seen this before when I used to hack a lot in answers
<thumper> lifeless: in the code for this series ?
<lifeless> yeah
<thumper> lifeless: just targetting the series branch?
<lifeless> yeah
<wgrant> sinzui: http://pastebin.ubuntu.com/440648/
<thumper> sure
<lifeless> one less click, one step closer to series-are-just-big-branches
<sinzui> wgrant, regardless of who changed it and who fixed it. we are getting mixed message from test runs about whether is it broken
<wgrant> sinzui: are we? StevenK probably fired the test off before my thing landed.
<thumper> lifeless: in which case we should add a codebrowse link too
<lifeless> thumper: YES
<lifeless> thumper: that would make many people happy
<wgrant> It's been in devel only 13 hours.
<thumper> lifeless: maybe not many, maybe just a few
<lifeless> thumper: maybe
<lifeless> thumper: Lots of people get lost finding codebrowse
<thumper> lifeless: I think many people would be happy if we had a bigger brighter "see the code here" on the front page
<lifeless> starting at lp.net/PROJECT
<poolie> hi
<lifeless> thumper: yes
 * thumper agrees
<wgrant> Well, that's because everywhere other than LP has codebrowse integrated into the app.
<thumper> wgrant: that's true
<poolie> if i write to a python logger from inside launchpad's mail processor, will that just turn up in a log file in product?
<poolie> *production
<thumper> wgrant: I'd love to have loggerhead more integrated, but it isn't up to it just yet
<lifeless> poolie: I think it has its own system
<lifeless> you raise an event
<poolie> i see some other code uses it...
<sinzui> wgrant the error is different now, and this command certainly confirms the unicode is gone:  iconv -f ascii lib/lp/soyuz/stories/soyuz/xx-distributionsourcepackagerelease-pages.txt > /dev/null
<thumper> poolie:  what are you after?
<thumper> poolie: and yes, writing to a logger should end up in the process-mail logfile
<wgrant> sinzui: The diff I pasted fixes it in an up-to-date devel.
<poolie> i want to log the results of the dkim check
<sinzui> wgrant, I just got this which matches the error I see in buildbot: http://pastebin.ubuntu.com/440653/
<wgrant> sinzui: That's what my diff is for.
<wgrant> I broke that test last night.
<sinzui> ah!
<sinzui> sorry, I was promoted over my abilities years ago.
<lifeless> darn
<lifeless> I forgot my second battery
<lifeless> ah well, not like I'll need it on a cross-tasman hop :P
<sinzui> wgrant, thank you. I committed that fix to my branch
<wgrant> sinzui: Thanks, and sorry.
<wgrant> What caused someone to notice and reenable it?
<wgrant> Remarkably bad timing.
<lifeless> ok, later - ciao
<sinzui> thumper, I am starting your review again after dealing with the buildbot distraction.
<thumper> sinzui: the second review?
<thumper> sinzui: I'm working on the branch subscription one right now
<lifeless> I like the iwlagn driver
<lifeless> lets me change mac addresses *and doesn't filter its own traffic*
<wgrant> Its MAC spoofing functionality was very handy at LCA.
<wgrant> Since the hotel granted 30 minutes free Internet connectivity per day.
<lifeless> yes
<lifeless> this waiting area is 30/4 hours, which is == for me :)
<lifeless> n+1, associate again.
<wgrant> Yep.
<mwhudson> :)
<thumper> sinzui: where is the code that does the magic user.in_admin et al?
<thumper> sinzui: for the security adapters
<thumper> sinzui: I'd love to move the code specific adapters out of that file
<sinzui> .me thinks
<sinzui> something "roles"
<thumper> sinzui: although I do recall that the security file is "special"
<thumper> sinzui: and processed by a script in a special way
<thumper> sinzui: to register the security adapters
<thumper> personally I'd prefer a metaclass
<wgrant> It's not that special. There's just an <authorizations> ZCML directive referencing it.
<thumper> wgrant: oh, is that all?
 * thumper tries to move some code ones :)
<wgrant> Yeah, it looks like it should work fine if you have another one.
<wgrant> Since it just registers adapters.
<thumper> hmm...
<thumper> perhaps not in this branch...
<thumper> later.
 * thumper must not get distracted
<wgrant> But yes, a good idea to chop that damn file up.
 * thumper collects children
<sinzui> thumper, you have my RC for the second branch.
<poolie> are the python logging logs recorded anywhere special during a unit test?
<thumper> sinzui: ta
<poolie> thumper, is there any way to get into pdb inside a launchpad unit test?
<thumper> -D
<poolie> it looks like stdin is redirected if i just call set_trace
<thumper> poolie: sometimes it is
<thumper> it depends on the layer
<thumper> and if it is a subprocess test
<thumper> not always easy to do
<poolie> ok thanks
<thumper> bin/test -vvt some_test -D *should* drop into the debugger on a fail
<poolie> thanks
<poolie> also do you know off hand why things are marked with IWeaklyAuthenticatedPrinciple
<poolie> rather than being marked if they're strongly authenticated
<poolie> it doesn't really matter, it just seems backwards
<poolie> so i wondered if there was a reason
<poolie> thumper, my test, after parsing the email, gets
<poolie> ForbiddenAttribute: ('email', <Person at 0xe73280c name16 (Foo Bar)>)
<thumper> preferredemail.email
<thumper> sinzui: being bitten by two things
<thumper> sinzui: You should not import BranchSubscriptionEdit from canonical.launchpad.security
<thumper> sinzui: and ForbiddenAttribute: ('in_admin', <Person at 0xca7e950 ...
<thumper> :(
<thumper> too much special casing
<sinzui> yuck
 * thumper thinks
 * thumper could avoid the first by making lp.code.security and putting the class in there with the configuration change
<thumper> and then not using the "special in_...." methods
<thumper> AAARRRRGGGHHHHHHHHHHH!!!!!!!!!!!!!!!!!!!!!
<thumper> canonical.launchpad.security line 452
<sinzui> odd, I am already there
<wgrant> thumper: So you get a different object if you're in another module?
<poolie> thanks thumper that fixed it
<thumper> wgrant: which question?
<wgrant> Or are you grabbing another person manually and not adapting to IPersonRoles?
<wgrant> The ForbiddenAttribute.
<thumper> the IPersonRoles thing will fix the second issue
<poolie> actually wgrant (hi) do you know why we have IWeaklyAuthenticatedPrincipal rather than IStronglyAuthenticatedPrincipal?
<wgrant> poolie: No idea, sorry.
<thumper> yes
<thumper> poolie: it is because the email code was written second
<poolie> ok... and?
<thumper> poolie: and I'm guessing that all the existing code wasn't wanted to be updated to check for strongly authenticated
<poolie> mm i can imagine that
<poolie> i just wondered if it was easier for them to assert the absence of IWeakly... than to check for IStrongly
<poolie> it's just curiousity
 * _thumper_ cleans as visitors coming, back tonight
 * sinzui find bed
* StevenK changed the topic of #launchpad-dev to: Launchpad Development Channel | Week 3 of 10.05 | PQM is open | https://dev.launchpad.net/ | Get the code: https://dev.launchpad.net/Getting | On-call review in irc://irc.freenode.net/#launchpad-reviews | Use http://paste.ubuntu.com/ for pastes
<StevenK> (I think ...)
* thumper changed the topic of #launchpad-dev to: Launchpad Development Channel | Week 3 of 10.05 | PQM is in release-critical mode | https://dev.launchpad.net/ | Get the code: https://dev.launchpad.net/Getting | On-call review in irc://irc.freenode.net/#launchpad-reviews | Use http://paste.ubuntu.com/ for pastes
<StevenK> thumper: Oh, duh. Thanks. :-)
<thumper> np
<BjornT> poolie: IWeaklyAuthenticatedPrincipal is really old code. iirc, it's really a start of a rework of the security system, that didn't get finished. the idea was to have both IWeaklyAuthenticatedPrincipal and IStronglyAuthenticatedPrincipal, but the latter never got added. the end result would be to have different security adapters for the two
<poolie> ah thanks BjornT
<adeuring> good morning
<mrevell> Morning
<krkhan> i'm getting the "nonce used already" error no matter what value i choose for the nonce. i've even tried FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF as the value
<krkhan> (for the api reqs)
<krkhan> is there a way for clearing the nonce records
<jml> krkhan: look inside ~/.launchpadlib/, there's probably a file you can delete
 * jml going offline to get some work done
<maxb> If krkhan is choosing nonces, this presumably means not using launchpadlib
<james_w> no, the error is misleading I think. I've seen that before, but I can't remember the cause now
<krkhan> i wasn't using launchpadlib. i wanted to clear the cache at launchpad server's side. anyways, i have disabled nonce checking for the time being as a workaround
<krkhan> i have another question though. i'm currently using the cycle of "make run; change code; ctrl-c; make run again" to play with lp's code. make run takes around 30 seconds to launch each time and it's a bit cumbersome. is it possible to modify lp's code while it's running?
<james_w> krkhan: were you using lp.me by any chance? Or accessing anything else that will result in a redirect?
<james_w> if you get a redirect then you need to re-sign the resulting request, not send the same headers to the new URL
<mars> gmb, ping, there is something odd going on here.
<mars> gmb, I ran bzr branch -r 9415 lp:~gmb/launchpad/hook-up-stored-proc-boom-bug-585803
<gmb> mars, Ah, the phrase I love to hear Friday of week 3
<mars> Good!  We are on the same page :)
<mars> gmb, so I ran that test locally, r9415 as the SUCCESS mail says, and the test passes?  Does it work for you too?
<gmb> mars, let me just try.
<mars> going to try it via 'make check'
<gmb> K
<mars> passes 'make check'
<mars> make check TESTOPTS='-t xx-temporary-blob-storage_txt'
<mars> so either a test isolation failure + testrunner fault, or a test environment factor + testrunner fault
<gmb> mars, Weird.
<gmb> mars, But it *should* fail
<gmb> Ah,... or should it
<gmb> Hmm.
<krkhan> james_w: i was requesting a non-existent page actually. i guess that was the root of all evil
<mars> gmb, your mail says "a stray line of output I'd left in a pagetest", but it looks like the doctest failure is producing "{}" when it shouldn't?
<mars> gmb, that means the error is that "{}" does not appear in the doctest when it should, correct?
<gmb> mars, What I meant was that I'd left 'job.metadata' in the test, so it outputted {}, which I hadn't accounted for
<gmb> So by right, it *should* have failed.
<gmb> mars, Ah, but, it fails locally
<mars> gmb, ok, so why is job.metadata not producing that same output here.
<gmb> mars, I have *no* idea.
<mars> gmb, it fails for you locally?  What command?
<gmb> bin/test -cvvt xx-temporary-blob-storage
<mars> wtf, it failed this time
<gmb> (!)
<mars> argh, my mistake, I was testing "xx-temporary-blob-storage_txt" instead of "xx-temporary-blob-storage"
<mars> gmb, ok, thanks!  I can reproduce locally.  Now to figure out why it reports success.
<gmb> mars, Best of luck :)
<gmb> let me know if I can help further
<mars> gmb, ok.  I'll be out for a while, probably up to your EOD, so we may have to pick this up on Monday
<mars> make that Tuesday, Monday is a holiday there
<gmb> mars, It's a national holiday Monday here too, so no worries :)
<rockstar> Is anyone working on the db_lp failure?
<jelmer> rockstar: I think curtis said he was going to look into it
<noodles775> jelmer, rockstar, sinzui: I think there's some confusion there... sinzui it looks like you replied to the db_lp failure, but meant to reply to the lp failure (in your email)?
<rockstar> noodles775, I'm handling the db_lp one.
<sinzui> rockstar, db_lp is fixed in devel
<noodles775> rockstar: great. Just note that it was already fixed in devel with r10935, so you might just want to merge the change from there.
<rockstar> sinzui, ah, okay.  So it WASN'T caused by my branch then (which landed directly into devel)
<sinzui> rockstar, This is dumbfounding: https://lpbuildbot.canonical.com/builders/lp/builds/931/steps/shell_7/logs/summary
<rockstar> Er, landed directly into db_devel I mean.
<rockstar> sinzui, what's dumfounding about that?
<sinzui> ^ the error we see is because there is an actual error in how we setup oops reporting. That is hiding the real error that caused the oops
<nigelb> bryceh: poke
<rockstar> sinzui, this is the one I was looking at: https://lpbuildbot.canonical.com/builders/db_lp/builds/830/steps/shell_7/logs/summary
<sinzui> yes I fixed that with wgrant
<rockstar> sinzui, okay, I'll leave it alone then.  As I started investigating, I started to realize it probably wasn't my fault anyway, but still was happy to fix it.
<noodles775> rockstar: it might be worth checking your ec2 results when your branch landed (see gmb's 'email "Er, this isn't a SUCCESS").
<bryceh> nigelb, yep
<nigelb> bryceh: wondering if we could interface lp api to talk to reportbug for easy forward to debian
<nigelb> something that can pull bug title and description and give to reportbug if we give it the lp bug number
<nigelb> if I start off a branch, can help me out?
<bryceh> jelmer, btw sorry last night in the middle of our conversation Verizon decided I didn't need internet and switched it off for a few hours.
<rockstar> noodles775, well, my output seems truncated, but there's no failures logged.
<bryceh> nigelb, sure
<nigelb> bryceh: awesome, thanks :)
<sinzui> devel is the one I am out of my depth. I cannot see the real error. I think the test env is not setup right for reporting codeimportworker oopses. apparently the section is missing
<jelmer> bryceh: ah :-)
<bryceh> nigelb, the lp side for that should be pretty straightforward; I've got most of that in arsenal now
<jelmer> bryceh: I figured it was something like that
<nigelb> bryceh: just pulling from launchpadlib should be enough right?
<bryceh> nigelb, yeah
<nigelb> bryceh: I'll give it a go i about an hour, I'll poke if I have something working :)
<nigelb> (or not working)
<bryceh> jelmer, anyway, dunno if you got everything I wrote.  In short, pulling bugs into gtg as tasks isn't implemented but with the latest backends work in theory it should be possible to do now
<bryceh> jelmer, I've got plans to do it, just haven't gotten time yet
<bryceh> nigelb, sounds good, I've got a meeting at the hour but will be around before and after that
<nigelb> bryceh: cool :)
<jelmer> bryceh: ah, cool
<jelmer> bryceh: are you planning on landing this in mainline or as a separate plugin?
<bryceh> jelmer, it would be a plugin which would be included in bzr trunk
<bryceh> jelmer, although during development would be in a branch of course
<bryceh> jelmer, btw I'm interested in hearing ideas of how it should work, use cases, etc.
<jelmer> bryceh: for me personally, I'd be interested in having a way to automatically add all bugs that are assigned to me to gtg, inheriting tags from lp
<jelmer> bryceh: as well as all my merge proposals that are work-in-progress
<bryceh> jelmer, mm
<bryceh> jelmer, some questions that have come up in relation to this...  would you want the import to be read-only or read-write (e.g. if you close the imported bug task or change tags, should it cause some change on lp or not)?  Would you want to make sub-tasks off of bug tasks (LP doesn't support sub-items so we'd have to figure out some alternate way to track it in gtg)?  Do you want to make notes on the bug task in gtg?
<jelmer> bryceh: I hadn't thought that far ahead. read-only would be very useful by itself, and a nice first milestone but read-write would certainly also be nice.
 * bryceh nods
<bryceh> yeah that's what I'll be aiming for first
<nigelb> bryceh: how do I query if a bug is private?
<bryceh> nigelb, if [ bug.private ]
<bryceh> or if you're using arsenal, if [ arbug.bug.private ]
<nigelb> I'm directly querying
<nigelb> I'll file a bug to tweak the apidocs soon enough.  Its not clear this is how I go about it
<nigelb> bryceh: anyway to expose the pcakage name of a bug via api? I see only bug_tasks_collection_link
<maxb> nigelb: A bug is not associated with a package directly. A bug has bug tasks. The bug tasks are associated with projects, distributions, or packages.
<nigelb> maxb: oh, thats tough :D
<maxb> no, that's good :-)
<bryceh> nigelb, yeah it gets complicated
<nigelb> bryceh: I got everything working except the plugging to reportbug
<nigelb> playing with subprocess right now :x
<bryceh> nigelb, what you have to do is given the bug #, look at its bug_tasks.  Exclude ones which are closed or that are external watches.
<nigelb> nah, I found easier way
<bryceh> oh?
<nigelb> First line of report is usually "Binary package hint: <package-name>"
<nigelb> not perfect, but works :D
<maxb> eww, that's horrid
<maxb> You really don't want to do that
<maxb> iterate the bugtasks, find the ones which are ubuntu packages, and take it from that
<nigelb> the problem is that package name might be wrong, so I'm only giving a hint and then user has to manually choose
<nigelb> anywy, I'm stuck up against a brick wall called subprocess.open
<nigelb> bryceh: help!
<bryceh> ok
<bryceh> yeah, I would not suggest relying on the Binary package hint text, that is going to be wrong a large percentage of the time
<bryceh> yeah like I suggested before, go through the bug_tasks and exclude what you can
<nigelb> bryceh: trust me, that's the least of my worries right now
<nigelb> bryceh: https://code.launchpad.net/~nigelbabu/ubuntu-review-overview/report-debian
<bryceh> ok what's the worry?
<nigelb> if you can get this beast working, it would be great
<nigelb> I can't get python to pass the arguments to reportbug
<nigelb> bryceh: ok, now the arguments are passing correctly, but still #fail
<bryceh> what command line are you using to run it?
<nigelb> python report-but -b 33333
<nigelb> s/bug/bug
<bryceh> ahh process handling
<nigelb> um, whats wrong?
<nigelb> bryceh: ok, now its partly working :)
<nigelb> yay to mentors :)
<bryceh>     p = subprocess.Popen(args)
<bryceh>     p.wait()
<bryceh> or perhaps p.communicate() is better than p.wait() depending on how you intend to use it
<bryceh> nigelb, anyway, other folk are probably better skilled than me at python to help with process handling stuff, but if you get stuck on launchpadlib-isms feel free to ping me further
<nigelb> ok :)
<nigelb> thanks :)
<mars> nigelb, are you trying to interact with the subprocess using the current script, or just call it?
<bryceh> nigelb, lp:~bryceharrington/ubuntu-review-overview/report-debian
<nigelb> mars: try to interact with the current script
<mars> ok, yes, you'll need .communicate() then to send data to the subprocess' Standard Input, and to read it's Standard Output
<nigelb> ah
<jml> g'night all
<bryceh> I'm trying to find where the package-requests table is in my launchpad.dev environment... could anyone suggest a path to get to this page?
<bryceh> the title of the page is "Copy archive contents" fwiw
<maxb> Does anyone understand *why* OAuth uses split request/access tokens, rather than just having one token, that gets activated?
<maxb> (Purely out of curiosity)
<nigelb> mars: I'm getting wierd errors
<nigelb> the lp_bug.description becomes the name of the attachment instead of the content :x
<nigelb> bryceh, mars: any thoughts on ^ ?
<bryceh> right that's correct
<bryceh> I have a snippet for getting the content, hang on
<bryceh> 	for a in bug.attachments:
<bryceh>             # Retrieve content
<bryceh>             hosted_file = a.data
<bryceh>             hosted_file_buffer = hosted_file.open()
<bryceh>             content = hosted_file_buffer.read()
<nigelb> hang on
<nigelb> whats ^ for?
<bryceh> a.message has additional info about the attachment
<nigelb> I'm not talking about LP attachment
<bryceh>             author = a.message.owner.name
<bryceh> nigelb, then maybe you should clarify
<nigelb> I meant the description of the bug that I pass to reportbug ends up being the name of the attachment to a debian bug instead of an actual bug
<bryceh> oh, I don't really know much about reportbug
<nigelb> does arsenel talk to debian bts?
<bryceh> nigelb, probably best you save me for launchpadlib questions specifically ;-)
<bryceh> nigelb, nope
<bryceh> nigelb, bugzilla.freedesktop.org
<bryceh> but doesn't do attachments (yet)
<nigelb> bryceh: hm, probably I need to write something to talk to debian bts then
<nigelb> this is getting me nowhere otherwise
<nigelb> bryceh: I'll try to write something to talk to bts and perhaps you can add to arsenel too :)
<bryceh> nigelb, ok that would be great :-)
<mars> bryceh, ping, could you please tell me the *exact* command you used to execute your test?
<mars> bryceh, and the location of the ec2 script that was executed?  (was it in the same branch, or somewhere else like trunk/?)
<bryceh> mars, ./utility/ec2 test
<bryceh> mars, yeah same branch
<mars> bryceh, ok, thanks.  I'm writing a reply to your mail.  Must be past EOD where you are?
<bryceh> mars, yeah getting close
<mars> ok.  After I get the mail out, feel free to write back whenever you have the time.
<bryceh> ok will do
<maxb> Ursinha: Hello. Have you considered registering a special Launchpad person to run your QA bot as? It would make notifications/bug histories somewhat more obvious to the uninitiated
<Ursinha> maxb, yes, I've considered but haven't executed the idea
<maxb> well, +1 :-)
 * Ursinha appends on her TODO list 
<Ursinha> :)
#launchpad-dev 2010-05-29
<wgrant> rockstar: Hi. Was there a corresponding launchpad-buildd change for your sourcepackagename-dropping branch?
<wgrant> It still looks for package_name in the dict passed through from the master, so it's broken at the moment.
<wgrant> Also, argh, our tests really suck.
<wgrant> So, recipe builds are currently broken in several ways in several places.
<wgrant> Some of them are crashes, some of them just do something that I cannot understand.
<james_w> hi wgrant
<wgrant> Morning james_w.
<wgrant> Does anyone know if QA has caught the breakage?
<james_w> I know that it is not being rolled out to production
<james_w> in some manner or other
<wgrant> But it's in db-devel...
<james_w> https://code.edge.launchpad.net/~abentley/launchpad/disable-recipe-builds/+merge/26322
<wgrant> Hm, well, that doesn't really indicate that it's known. But I guess it works around it.
<james_w> yeah, I'd file the bugs anyway
<james_w> though I think you of all people doesn't need telling to file bugs :-)
<wgrant> james_w: Also, when did "ARM Archive Branching" become "rewrite Launchpad"?
<james_w> yeah, it's a bit unfortunate
<james_w> we aim to reuse what we can
<wgrant> It seems like you're trying to work around Launchpad rather than fixing it.
<wgrant> Which may mean rewriting it, I don't know.
<wgrant> But writing another completely independent but also completely duplicated thing seems... odd.
<wgrant> Ah, it is so OEMs can run their own archives locally. I see.
<wgrant> It still seems pretty silly to not fix Launchpad instead.
<cody-somerville> People have been trying to fix Launchpad for years :P
<wgrant> Have they?
<wgrant> It doesn't seem like anybody has tried anything revolutionary.
<cody-somerville> Too difficult to do in Launchpad.
<wgrant> Well, inventing a new archive system to run along Soyuz forever is not the solution!
<wgrant> Rip Soyuz out, sure...
<wgrant> s/along/alongside/
<cody-somerville> Maybe this new thing will replace Soyuz at some point.
<rockstar> wgrant, no, if there wasn't a test for it, I didn't fix it.
<rockstar> wgrant, there are configs that block recipe availability in production.
<rockstar> So it is not being rolled out to production.
<rockstar> We still have yet to actually run a build through the production system, because there are infrastructure changes that couldn't be made in time.
<wgrant> rockstar: So we're waiting for 10.06, or doing lots of cherrypicks?
<rockstar> wgrant, well, probably a little of both.
<wgrant> Bug #587109, bug #587110, and bug #587113
<mup> Bug #587109: Needs to cope with not receiving package_name from the master <Launchpad Auto Build System:New> <https://launchpad.net/bugs/587109>
<mup> Bug #587110: Need fmt:link for SourcePackageRecipeBuild <Launchpad Bazaar Integration:New> <https://launchpad.net/bugs/587110>
<mup> Bug #587113: BuildBase result handling broken <Soyuz:New> <https://launchpad.net/bugs/587113>
<wgrant> 587110 is the only one that is likely to affect anyone apart from cesium and the slaves.
<wgrant> And there's also something going horribly wrong with updating the status of SPRBs.
<nigelb> bryceh: I ran a script to subscribe ubuntu-reviewers to bug with patches that are not kernel or doc team bugs, but some bugs were never counted.  If you've got the time can you help figure out why?
<thumper> jelmer: I was bitten again by two import related bugs:
<thumper> jelmer: the corrupted config file
<thumper> jelmer: and the locked db with two imports from the same host
 * thumper goes to make breakfast
<jelmer_> thumper: both are waiting for changes in bzrlib
<jelmer_> thumper: the locked db one can be worked around by installing python-tdb
<thumper> jelmer: waiting for a change to land in bzrlib? or a change in bzrlib to land in LP?
<lifeless> thumper: the config one is a bzrlib issue, but noone is working on it AFAIK
<lifeless> as it has, till now, only affected bzr-svn
<lifeless> the bug explains what is needed
<jelmer> thumper: waiting for a change to land in bzrlib
#launchpad-dev 2010-05-30
<thumper> morning
<jelmer> 'morning thumper
<lifeless> hi thumper
 * thumper has printed out things to think about and heading for coffee
#launchpad-dev 2011-05-23
<wgrant> wgrant@mawson:~/lp$ bzr branch lp:launchpad/devel
<wgrant> You have not informed bzr of your Launchpad ID, and you must do this to
<wgrant> write to Launchpad or access private data.  See "bzr help launchpad-login".
<wgrant> bzr: ERROR: Invalid url supplied to transport: "bzr+ssh://bazaar.launchpad.net/+branch/launchpad/devel": no supported schemes
<wgrant> That's not meant to happen.
<wgrant> Interesting.
<wgrant> lp:launchpad works fine.
<wgrant> Oh, there is no devel series.
<wgrant> Still, unobvious error :(
<StevenK> File a nastygram?
<wgrant> wallyworld__: So, what are you up to on our first day of featureness?
<wallyworld__> wgrant: digesting all the material, mockups etc. working on the picker display bug. musing about horrible query used to populate picker
<wgrant> wallyworld__: Yeah, I was going to look at that vocab and try to work out what we can do to it.
<wallyworld__> you?
<wgrant> Since it needs a bit of work.
<wgrant> And vaguely pondering UI.
<wallyworld__> should be an interesting sprint
<huwshimi> wgrant, wallyworld__: Have you guys started working on disclosure stuff now?
<wgrant> huwshimi: Yup.
<huwshimi> wgrant: nice
<wgrant> We'll soon see about that.
<huwshimi> wgrant: Whether it's nice?
<wgrant> Right, it is probably going to be a bit of a mess, at least to start :)
<huwshimi> wgrant: And you won't have as much opportunity to contribute to the pie
<wgrant> huwshimi: Indeed.
* lifeless changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: - | https://code.launchpad.net/launchpad-project/+activereviews | Critical bugs:213 - 0:[#####=__]:256
* lifeless changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: - | https://code.launchpad.net/launchpad-project/+activereviews | Critical bugs:213 - 0:[######=_]:256
<lifeless> wgrant: so, critical bugs & bar charts
<lifeless> wgrant: jmls challenge is at best a proxy for flacostes goal.
<lifeless> wgrant: I'm charting flacostes goal: which was no open criticals
<lifeless> gnuoy: your irc client is bouncing. Can has fix?
<wgrant> lifeless: I thought it was to eliminate the backlog.
<wgrant> But OK.
<lifeless> wgrant: if we are at 5 (the current escalated count) come dublin, I'm --sure-- everyone will be ecstatic and pies will happen
<wgrant> Heh
<lifeless> I'll be happy if we're in the 10-20 range
<wgrant> It is just somewhat demotivating to have the countdown regress when there's nothing we can do about it.
<lifeless> uhm
<lifeless> I hate to demotivate
<lifeless> but francis is not letting every escalation through
<wgrant> Ah, I see.
<lifeless> and
<wgrant> That's not been clear previously.
<lifeless> he even unescalated one the other day
<wgrant> I saw that.
<lifeless> the target-to-project one
<lifeless> concretely, we (the team) control what ones we're willing to escalate
<lifeless> hmmm
<lifeless> can I be bother kickbanning gn
<lifeless> uoy
<wgrant> Well, we have a room full of pingable people.
<wgrant> LOSA ping :P
<mbarnett> heya wgrant.
<lifeless> wgrant: whats bug 78436 really about ?
<_mup_> Bug #78436: Careful publishing approach doesn't work <lp-soyuz> <soyuz-publish> <Launchpad itself:Triaged> < https://launchpad.net/bugs/78436 >
<wgrant> mbarnett: gnuoy was bouncing, but appears to have stopped now.
<wgrant> lifeless: Do you understand what a careful publishing run is?
<lifeless> wgrant: never heard of it until the summary in that bug
 * mbarnett likes when things stop
<StevenK> mbarnett: No fair enclosing gnuoy's laptop in a Faraday cage.
<wgrant> lifeless: The -C, -P, -A, -D options to publish-distro cause its various stages to become "careful". That means they run completely from scratch -- they don't just process pending data like a normal run.
<wgrant> lifeless: For something the size of the primary archive this can take a Long Time.
<wgrant> eg. -P causes all the package files to be republished, and took nearly 24 hours to do maverick on DF.
<wgrant> I think the MemoryError is long-fixed.
<lifeless> do you mean the Packages files or deb & sources ?
<lifeless> because, the former - mmm, should be as-is :P
<wgrant> Oh, and -A (careful index generation) can't regenerate byte-identical indices for released pockets, so it's not really useful.
<wgrant> deb & sources
<wgrant> The actual packages.
<wgrant> should be as-is?
<lifeless> wgrant: btw I did some lmirror testing the other day :)
<wgrant> Oh?
<lifeless> wgrant: I mean we recreate Packages.,* every run, no?
<wgrant> lifeless: Not for all suites, but yes.
<lifeless> lmirror - I think its at 'do some real world testing' stage
<lifeless> I need to discuss w/ elmo again I think
<wgrant> You said that a year ago :)
<wgrant> But yes.
<wgrant> I think so.
<lifeless> wgrant: I did, but now I have twox1TB disks for testing
<lifeless> only problem is my link is -fail- atm
<wgrant> Ah, excellent.
<lifeless> also python slow
<wgrant> PyPy fast :)
<lifeless> so so slow
<wgrant> er
<lifeless> 25% faster overall, and 400% at the tokenisation
<lifeless> 400% slower that is
<lifeless> ok, I'm going to nuke that memoryerror bug
<wgrant> Erk.
<wgrant> But didn't you have a faster generator-based method?
<lifeless> that was slower
<wgrant> Wow.
<wgrant> OK.
<lifeless> https://bugs.pypy.org/issue718
<lifeless> 300%, 4times
<wgrant> Self-signed cert ew
<wgrant> Bad cn, too.
<lifeless> so an interesting question is
<lifeless> what auth for internal services
<lifeless> host based? basic? oauth?
 * lifeless doesn't really fancy having oauth overhead in every microservice.
<lifeless> not to mention the bootstrap complexity for stuff the SSO wants to reference.
<wgrant> I think bsic.
<wgrant> +a
<wgrant> I guess host-based could work, but it has certainly fallen out of favour in the last couple of decades.
<lifeless> I'm inclined to say hosts + basic (must pass both checks)
<wgrant> Yeah.
<lifeless> belt and braces
<wgrant> I imagine lots of these will be available far more widely than the present librarian, so, yeah. Probably a good thing.
<lifeless> I can imagine support oauth eventually, optionally
<lifeless> so hosts + basic, or oauth from anywhere
<lifeless> but I'm willing to have to retrofit than pay up front
<wgrant> Authentication on these primarily-private services should be easy to fix.
<wgrant> zomg
<wgrant> non-Rabbit lucid_db_lp failure!
<StevenK> Say it ain't so!
<lifeless> woo
<LPCIBot> Project db-devel build #569: ABORTED in 3 hr 9 min: https://lpci.wedontsleep.org/job/db-devel/569/
<StevenK> Oh, stop that Jenkins
<StevenK> Using a slave that I'm trying to destroy is not cool
<LPCIBot> Project windmill-db-devel build #301: STILL FAILING in 1 min 21 sec: https://lpci.wedontsleep.org/job/windmill-db-devel/301/
<nigelb> Hi! I'm trying to figure out how to fix bug 90628.  Do I create a new property and use that?
<_mup_> Bug #90628: Subscribers to a spec are randomly arranged <easy> <lp-blueprints> <ui> <Launchpad itself:Triaged> < https://launchpad.net/bugs/90628 >
<lifeless> nigelb: so, I think we show all subscribers
<lifeless> nigelb: in which case just sort the list in the view is simplest and not unreasonable
<nigelb> hrm, I tried to figure out the flow of control and got lost ^-^
<lifeless> ok, its fugly but starting to come together https://launchpad.net/bugs/90628
<_mup_> Bug #90628: Subscribers to a spec are randomly arranged <easy> <lp-blueprints> <ui> <Launchpad itself:Triaged> < https://launchpad.net/bugs/90628 >
<lifeless> bahhh
<lifeless> https://dev.launchpad.net/ArchitectureGuide/ServicesRoadmap
<nigelb> heh
<lifeless> nigelb: so - SpecificationView
<lifeless> lib/lp/blueprints/templates/specification-portlet-subscribers.pt
<lifeless> which uses context/subscriptions
<lifeless> which is on model/specification
<lifeless> as a multiple join
<lifeless> one was is a new property
<lifeless> I'd be inclined to move the existing join to _subscriptions
<lifeless> and make a cachedproperty which eager loads the subscribers and sorts appropriately.
<nigelb> the one from line 185 in the model?
<lifeless> yes
<nigelb> okay, I am getting used to my way around LP codebase.
<lifeless> wgrant: did you get an oops for bug 786804
<_mup_> Bug #786804: branch scanner fails with MemoryError leaving branch page in intermediate state <oops> <Launchpad itself:Triaged> < https://launchpad.net/bugs/786804 >
<nigelb> lifeless: ok, I'll try to figure this out and ask again if I'm lost.
<wgrant> lifeless: There wasn't one.
<lifeless> wgrant: ><
<wgrant> http://launchpadlibrarian.net/72030392/lWzH7ofhPZuORhdAbLi89zRxIV9.txt
<lifeless> wgrant: isn't it meant to oops ?
<wgrant> Oh.
<wgrant> There is one, on the next line.
<wgrant> 2011-05-19 17:42:47 INFO    Job resulted in OOPS: OOPS-1965SMS9
<lifeless> thanks
<StevenK> The OOPS prefix is SMS? Fail.
<wgrant> Supermirror scanner.
<wgrant> Not that fail.
<wgrant> Just old :)
<nigelb> lifeless: should the join be moved from SQLBase to StormBase?
* henninge changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: henninge | https://code.launchpad.net/launchpad-project/+activereviews | Critical bugs:213 - 0:[######=_]:256
<jtv> huwshimi, hi!  Can I trouble you for some UI expertise?
<huwshimi> jtv: Of course!
<LPCIBot> Project windmill-db-devel build #302: STILL FAILING in 1 hr 8 min: https://lpci.wedontsleep.org/job/windmill-db-devel/302/
<jtv> Great.  It's about this page: https://dogfood.launchpad.net/ubuntu/oneiric/+localpackagediffs
<jtv> huwshimi: I'm just setting you up with the privileges you need to see the pertinent parts.
<jtv> huwshimi: if you have those privileges, you should see a checkbox to the left of each entry.
<jtv> That checkbox lets the user select specific entries in order to synchronize them.
<jtv> But that doesn't make sense for ones that are already being synchronized.
<huwshimi> jtv: Hmmm... don't have any checkboxes
<LPCIBot> Project windmill-devel build #118: STILL FAILING in 1 hr 9 min: https://lpci.wedontsleep.org/job/windmill-devel/118/
<StevenK> huwshimi: Are you logged in?
<jtv> Try reloading.  You should just have received the privileges to get them.
<jtv> Oh, yes, and what Steve said.  This is dogfood, a special test server.
<huwshimi> StevenK: Thanks :)
<huwshimi> jtv: Still no checkboxes
<jtv> You do see a table with green package names with "details" widgets, right?
<jtv> StevenK: might he need anything more than anoint-dogfood-admin?
<StevenK> I don't think so
<huwshimi> jtv: yeah
<StevenK> Perhaps you annoited the wrong user?
<wgrant> Erm, let's not anoint-dogfood-admin without extreme caution and consideration, please :)
<jtv> anoint-dogfood-admin huwshimi
<jtv> Looks right to me.
<wgrant> Indeed, that is correct.
<wgrant> Do admins have launchpad.Append, though?
<wgrant> Wait, it doesn't use launchpad.Append any more...
<wgrant> needs an archivepermission.
 * wgrant fixes.
<wgrant> (by removing from ~admins and adding to ~ubuntu-core-dev0
<jtv> Thanks.
<wgrant> If the picker loads...
<wgrant> huwshimi: Should work now.
<huwshimi> wgrant, jtv: Got it
<jtv> Great.
<jtv> Now, to see how I'm signaling that a package is synchronizing (which means there's no point in requesting synchronization), look for "Synchronizing"
<huwshimi> jtv: Yep
<jtv> You'll note two things: the checkbox is disabled, and there's a "(Synchronizing)" in italics next to the package name.
<huwshimi> jtv: Yep
<jtv> Those are the changes I wanted your feedback on.  Very simple, really.
<jtv> But I wanted to be sure that I'm not causing any distress anywhere by just slapping a bit of extra text into that column.
<jtv> (Disabling the checkbox seems like a bit of a no-brainer in retrospect, but I played with putting a "please wait" icon in its place)
<huwshimi> jtv: I think that looks alright. The loading icon is something that should be used if the action happens in a short period of time (and an ajax event that will update the page)
<jtv> Yes, that's one of the reasons why I ended up not going with that.  There's no polling as yet.
<huwshimi> jtv: yeah
<huwshimi> jtv: You could have some kind of static icon symbolizing that it is synchronizing
<jtv> (Another reason is that we don't seem to _have_ a "please wait" icon :)
<huwshimi> jtv: What is the time scale for the sync
<huwshimi> >
<huwshimi> *?
<jtv> A minute perhaps.
<StevenK> I doubt it
<jtv> Possibly much less.
<StevenK> If it's done via a job, maybe a little bit longer than a minute
<jtv> The job was to run every minute I believe; though rabbit will make it much faster.
<jtv> An icon is an interesting idea, but will it be clear enough?
<jtv> Mind you, this UI is only for a select few experts.  But still, the less expertise I add to the requirements the better.
<huwshimi> jtv: At the very least I would consider adding an ellipsis after "synchronizing" as it is not a permanent state
<jtv> Would that work well with the parentheses?
<jtv> (Synchronizing...)
<huwshimi> jtv: The icon would be an addition, not  replacement
<jtv> oic
<huwshimi> jtv: I think English allows you to do that
<jtv> Well yes, it's probably allowed â just asking whether it's nice. :-)
<huwshimi> jtv: Yeah, I'm not sure
<jtv> Let me try it.
<jtv> huwshimi: try reloading the page?
<jtv> (And I just updated it to use &hellip;
<jtv> )
<huwshimi> jtv: Hmm...
<jtv> Doesn't quite look right, does it?
<jtv> What if I dropped the parentheses?
<jtv> foo â synchronizingâ¦
<huwshimi> jtv: You could try dropping the parentheses, and maybe adding a bit more space between the name and the sync
<jtv> But no emdash?
<huwshimi> jtv: Yeah I think so
 * jtv fiddles
<jtv> Oh, and drop the capital letter?
<huwshimi> jtv: I'm not sure about that
<jtv> huwshimi: try now
<huwshimi> jtv: OK, maybe also drop the italics and make it a bit grey?
<huwshimi> jtv: Actually we do have an icon
<huwshimi> jtv: It might be good to add that before the "synchronizing..."
<huwshimi> jtv: It's a little clock, let me find it
<jtv> That's the one I was trying to find as well.
<jtv> It's animated though.
<jtv> huwshimi: hmm... I worry that an icon there may be a bit distracting.
<jtv> I've got the space-separated grey text now though.
<jtv> Looks nice and discreet.
<huwshimi> jtv: Ugh, the demo text is so distracting
<wgrant> /@@/processing?
<StevenK> henninge: Hi! Do you mind reviewing a slightly oversized branch?
<StevenK> henninge: (833 lines)
<jtv> Thanks wgrant.  I guess animating sprites wasn't a good idea.
<wgrant> jtv: Quite possibly not.
<henninge> StevenK: Sure, I'll do it after I finish jtv's branch.
<jtv> henninge: just a mentoring request :)
<wgrant> StevenK: Why the change in nascentupload-announcements?
<henninge> jtv: I know
<huwshimi> jtv: Actually the one I was looking for turns out to be the milestone icon
<huwshimi> jtv: So nevermind that
<jtv> wgrant: /@@/processing isn't working for me I think
<wgrant> https://launchpad.net/@@/processing works for me...
<StevenK> wgrant: The first one is just a e-mail change, it doesn't actually send the annoucement -- and I didn't want to special case "Don't set the announce list if dry_run is True"
<jtv> Hmm
<StevenK> wgrant: The second one is due to notify() hooking into notify_spr_less() which does far less logging
<jtv> Ah, I got the syntax all wrong.
<jtv> Been a while since I used non-sprite images!
<jtv> huwshimi: got it with the wait icon now
<jtv> â¦and now the spaces are on the right side of the icon
<LPCIBot> Project windmill-db-devel build #303: STILL FAILING in 45 min: https://lpci.wedontsleep.org/job/windmill-db-devel/303/
<jtv> huwshimi: the icon is a bit distracting, isn't it?
<lifeless> nigelb: generally do just one thing
<nigelb> okay
<nigelb> I'm getting so many OOPS because I'm doing it wrong.
<lifeless> nigelb: so yes, we should move the class including the joins, to storm base. But either do that, or the sort. Don't do both (a change doubled is a risk quadrupled)
<nigelb> ah, okay. I'll stick with the sort
<lifeless> nigelb: please feel free to chat here and say 'help, X happened' :)
<nigelb> okay :)
<huwshimi> jtv: So there will be a trade-off between how obvious the notification needs to be and how often it will be shown
<jtv> I suspect it needn't be very obvious.
<huwshimi> jtv: It will be a short term thing, so maybe a little bit more distracting is ok
<huwshimi> jtv: In that case it might be fine without it.
<huwshimi> jtv: This might be a situation where we can release the feature and keep an eye out for feedback
<huwshimi> jtv: And iterate when we need to
<jtv> Yes... I suspect people won't care much either way.
<jtv> For me it's more an explanation for the "hey why can't I sync this?" moment
<huwshimi> jtv: OK, sure
<jtv> Then I'll go with what he have: spaces, grey text, ellipsis; no icon, no upper-case, no parentheses.
<jtv> Thanks for the help!
<henninge> jtv: r=me
<jtv> thanks henninge!
<StevenK> henninge: Do you need the URL for my MP?
<henninge> StevenK: usually no
<henninge> ;)
<henninge> got it
<lifeless> foodingk
<wgrant> henninge, StevenK: I have reviewed that MP, but henninge's opinion is also useful given that I don't really exist yet.
<StevenK> wgrant: :-(
<StevenK> wgrant: I explained the two test changes on IRC
<adeuring> good morning
<jtv> hi adeuring
<adeuring> hi jtv!
<wgrant> StevenK: Yeah, but they were in the textbox already and I don't really trust it.
<huwshimi> jtv: No problems :)
<huwshimi> jtv: Anytime
<StevenK> wgrant: You don't trust the textbox, or the changes I made?
<wgrant> StevenK: That you're not invalidating part of the test.
<wgrant> You probably aren't.
<wgrant> But still, I wanted to record my concern.
<StevenK> wgrant: The first test I had to change is a crook anyway. "If dry_run is specified, we don't send a mail. But oh, let's look for the entire thing, verbatim, in the logs."
<wgrant> StevenK: I'd expect to see something there, like the "Would have sent a mail" bit.
<StevenK> And the subject bit is dangerous?
<StevenK> Er? The old code didn't use packageupload.displayname?
<wgrant> 287	-        # Weed out duplicate name entries.
<wgrant> 288	-        names = ', '.join(set(packageupload.displayname.split(', ')))
<wgrant> 298	-        subject = '[%s/%s] %s %s (%s)' % (
<wgrant> 299	-            packageupload.distroseries.distribution.name, suite, names,
<wgrant> 300	-            packageupload.displayversion, message.STATUS)
<StevenK> Ah
<mrevell> Hello
<jtv> hi mrevell!
 * henninge relocates
<jtv> wallyworld__: reviewed your branch.  A few changes to make.
<jtv> allenap: do we have a job runner for IPlainPackageCopyJob?
<allenap> jtv: It gets run by cronscripts/process-job-source-groups.py
<allenap> jtv: It runs on loganberry every 5 minutes.
<jtv> Can I run it on dogfood?
<bigjools> wgrant: hello
<bigjools> wgrant: can you follow up on Steve's MP that you started to review, I will mentor you.  I'm happy with it myself now.
<jtv> henninge: can I trouble you for another one?  https://code.launchpad.net/~jtv/launchpad/db-bug-786822/+merge/61924
<jtv> allenap: and another question.  When a PlainPackageCopyJob raises CannotCopy, do we know reliably which package the error was for?
 * allenap looks
<allenap> jtv: Only if we parse the exception content.
<jtv> :( :( :(
<jtv> May be a good idea to change that then.
<jtv> So that we may accurately report errors.
<jtv> I think I'll go grab a bite first though.
<allenap> jtv: Yeah, good idea :)
<LPCIBot> Project windmill-db-devel build #304: STILL FAILING in 1 hr 9 min: https://lpci.wedontsleep.org/job/windmill-db-devel/304/
<wgrant> bigjools: Looking.
<lifeless> mpt: hi
<mpt> hi lifeless
<lifeless> mpt: book you might like (or even love) - the innovators solution
<bigjools> wgrant: thanks
<lifeless> jml: around?
<mpt> lifeless, oh yes, I saw your tweet about that
<lifeless> mpt: cool; evan's post about ~ubuntu-metrics reminded me you would probably get a lot out of (or vigorously agree and say you know it already :P) it
<lifeless> bigjools: when you say you're not happy about the feature flag do you mean 'its not complete' or 'it shouldn't be flagged'. I don't care which you mean, but I can't *tell* which you mean :)
<bigjools> lifeless: I was too brief after discussing it with steve, he knows what I mean :)
<mpt> lifeless, found a copy on the Millbank bookshelves :-)
<wgrant> bigjools: Done.
<LPCIBot> Project windmill-devel build #119: STILL FAILING in 1 hr 9 min: https://lpci.wedontsleep.org/job/windmill-devel/119/
<lifeless> mpt: hah! nice.
<bigjools> StevenK: you have your answers, go forth and fix0rate
<lifeless> allenap: bugs fixed
<allenap> lifeless: Awesome :)
<lifeless> rev 11
<lifeless> bah
<lifeless> 151
<lifeless> or wait for the ppa to pick it up
<stub> What was the query we wanted to denormalize Revision.revision_date to  BranchRevision.revision_date for?
<lifeless> merge proposal page IIRC
<lifeless> its in the bugs tagged timeout
<stub> Why did it want Revision.revision_date instead of using BranchRevision.sequence to filter or order?
<lifeless> You'll need to have a nosy
<lifeless> I have paged it out
<stub> Bug #778393 isn't it
<_mup_> Bug #778393: Branch:+index timeouts <dba> <timeout> <Launchpad itself:Triaged> < https://launchpad.net/bugs/778393 >
<stub> I see - Bug #742916
<_mup_> Bug #742916: BranchMergeProposal:+index timeouts - slow query plan <timeout> <Launchpad itself:Triaged> < https://launchpad.net/bugs/742916 >
<LPCIBot> Project windmill-db-devel build #305: STILL FAILING in 48 min: https://lpci.wedontsleep.org/job/windmill-db-devel/305/
<StevenK> wgrant: You will cope with bpn_archtag?
<StevenK> wgrant: And how do you want confirmation about partner?
<wgrant> StevenK: I guess QA on DF would be fine.
<lifeless> hey allenap
<lifeless> btw, you've invalidated the test
<allenap> lifeless: Explain?
<lifeless> the hunk @@ -28,10 +27,7 @@
<lifeless> had explicit connection logic
<stub> lifeless: I think the query is broken anyway. It is trying to display the set of revisions that were reviewed, between two timestamps. That timestamp is the timestamp when the revision first entered Launchpad - it isn't the timestamp that it landed on the branch being reviewed.
<lifeless> now its calling a method that could in principle be lazy
<lifeless> so its not valid anymore
<allenap> lifeless: Okay, I'll revert that bit. Are you happy with the rest?
<stub> c/'first entered launchpad'/'system timestamp when the revision was committed to its original branch'
<allenap> I mean, it's just stuff I twiddled as I read the code.
<lifeless> allenap: the other functional thing is 'getConnection' - its basically noise
<lifeless> allenap: because the parameter list including things like require etc will need to be reflected in it
<lifeless> allenap: which is why I had and then deleted a get_connection method
<lifeless> allenap: so I'd delete that method
<LPCIBot> Project db-devel build #570: FAILURE in 5 hr 37 min: https://lpci.wedontsleep.org/job/db-devel/570/
<lifeless> allenap: you might want to have it glued in with the lazr.config settings for rabbit
<stub> Am I mistaken that Revision.revision_date is untrusted data, originating from a remote system timestamp?
<lifeless> allenap: if you pull in my rabbit branch to get the layer
<lifeless> stub: it is untrusted, yes
<lifeless> allenap: other than that its all fine; some minor could-be-done different stuff (e.g. I was going PEP 8 on the methods, leaving stuff that hadn't been migrated uncommented so that when we fix it up the diff is saner)
<allenap> lifeless: Is there a reason why ExportRabbitServer and RunRabbitServer are split into two?
<lifeless> allenap: but nothing to sweat about any which way
<lifeless> allenap: I'm glad you asked. Yes, there is
<lifeless> allenap: they do different things
<lifeless> the exporter exports enough to run rabbitctl or rabbit server or both
<lifeless> you often want to run the former separate from the latter
<lifeless> allenap: I don't object to it being camelcase, though it is hard on the eyes
<allenap> lifeless: I'm happy to go PEP8. Our coding standards and actual practice are a bit schizophrenic on the matter.
<lifeless> oh, one other thing - def start()
<lifeless> allenap: that looks like its not a public method
<lifeless> allenap: so its odd for it to be missing the _
<allenap> lifeless: I guess stop() isn't either?
<lifeless> indeed
<lifeless> that was oversight
<lifeless> I would change stop to _stop
<lifeless> I can't see the bugfix you mention
<lifeless> oh
<lifeless> also
<lifeless> fq_nodename is a method because its not cheap to call
<allenap> lifeless: Re. getConnection, that's just a rename from get_connection. Do you suggest I ditch that altogether and use amqp.Connection everywhere instead, rename it back, or move it elsewhere (ExportRabbitServer doesn't seem like the right place for it now anyway).
<lifeless> allenap:
<lifeless> +    def getConnection(self):
<lifeless> +        return self.server.getConnection()
<allenap> lifeless: Is gethostname() really that expensive?
<lifeless> allenap: that one isn't a rename
<allenap> lifeless: Okay, yeah, I'll ditch that.
<allenap> lifeless: The bugfix is actually in the prerequisite branch. This is a bit of a muddle :-/
<allenap> lifeless: It wasn't sleeping when calling check_running during start-up.
<allenap> lifeless: Also, the reason I went camelCase is because of Fixture.setUp().
<lifeless> yeah, thats in the fixture protocol, a bit sadly
<lifeless> its camelcase because of TestCase.setUp
<allenap> Yeah, I assumed as much.
<lifeless> but, FWIW, I use PEP8 everywhere else in the fixtures library
<lifeless> anyhow, this is ok as is, everything is changable if needed :P
<lifeless> I'm glad you're digging into it
<lifeless> I think you should do what you think is best
<lifeless> the only thing that really matters is the test actually showing what its testing
<allenap> lifeless: Thanks, and I'll try not to make so many cock-ups in future :)
<LPCIBot> Project windmill-db-devel build #306: STILL FAILING in 46 min: https://lpci.wedontsleep.org/job/windmill-db-devel/306/
<lifeless> allenap: hah no worries :)
<lifeless> allenap: have you had a chance to look at the layer patch?
<allenap> lifeless: Briefly, that's all, but I have it on the list.
<lifeless> cool
<stub> jtv-eat: http://paste.ubuntu.com/611779/ is the diff of a launchpad schema dump before and after applying my current 'Drop BranchRevision.id' patch.  It all looks healthy so far.
<stub> jtv-eat: https://code.launchpad.net/~stub/launchpad/drop-branchrevision-id/+merge/55480 has the db patch at the bottom, not much simpler. The existing UNIQUE constraint is a big help, not a hindrance.
<bigjools> anyone else getting bzrlib.errors.ReadOnlyError when doing "bzr up" in download-cache?
<wgrant> Sure it was bzr up and not bzr pull?
<bigjools> yes
<bigjools> rf-get is falling over and I've not changed that ;)
<jml> lifeless: sorry.
<StevenK> bigjools: henninge was
<henninge> yes, I was.
<bigjools> did you file a bug henninge?
<henninge> I forgot but I can still do it.
<bigjools> ok
<bigjools> I have a fresh crash file
<henninge> bigjools: I don't  ;)
<henninge> bug I can make one
<bigjools> well apport says bzr is not a genuine Ubuntu package here :)
<henninge> StevenK: Sory about your MP, btw. I just don't feel up to it atm.
<StevenK> henninge: It's perfectly fine, wgrant has already written me a novel
<henninge> weired, I cannot reproduce it on this computer
 * henninge tries laptop again
<henninge> s/weired/weird/
<jtv> stub: ah yes, you don't have to invert the dependency like we do when we start with an index, right?  (Going from memory; hope to look deeper later)
<stub> jtv: Yup. The pg_depends stuff seems identical for a unique and a primary key index, so just changing the type in pg_constraint seems to work fine.
<stub> jtv: Claim that MP review if you want to double check the work.
 * jtv wonders if unique and primary-key constraints wouldn't be easier if they merely enforced the existence of a suitable index.
<stub> I've just pushed updated sampledata and set the MP status to needs review.
<jtv> stub: err ok
<stub> jtv: I think that would help, yes.
<jtv> Just going through review fin/ack for Ian, and checking up on one of my own.
<stub> jtv: Sure. No need to do it now. I do want it on staging though, so if you can't check it over tomorrow I'll land it r=stub and we can inspect what happens on real data.
<jtv> But yes, I should go over it definitely and decide whether I need plastic surgery, spare passport etc.
<jtv> stub: maybe want to walk me through it IRL?  Just an idea.
<stub> Sure.
<nigelb> okay, take 3 at fixing the specification ordering bug.
<jml> :)
<nigelb> okay, what is subscription and subscribers in lp/blueprint/model/specifications.py
<nigelb> I can't figure out how they're different.
<nigelb> so, this is what life less told me earlier today on fixing this.
<nigelb> http://pastebin.ca/2067555
* benji changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: henninge, benji | https://code.launchpad.net/launchpad-project/+activereviews | Critical bugs:213 - 0:[######=_]:256
<nigelb> can I just move the query onto specificationsubscription.py and will it Just Work?
<jml> nigelb: 'subscriptions' is a result set of SpecificationSubscription objects
<jml> nigelb: 'subscribers' is a result set of Person objects
<nigelb> jml: ah.
<jml> ffs
<jml> my mobile phone operator needs to check my age :\
<nigelb> did they give you free texts for a month?
<nigelb> :P
<nigelb> okay, I find to find the person who tagged the bug I'm working easy :/
<nigelb> This is my third attempt and I'm going no where :(
<jml> nigelb: I can't view that paste. sorry.
<jml> nigelb: what are you trying to do?
<nigelb> jml: I'm trying to fix bug 90628
<_mup_> Bug #90628: Subscribers to a spec are randomly arranged <easy> <lp-blueprints> <ui> <Launchpad itself:Triaged> < https://launchpad.net/bugs/90628 >
<jml> nigelb: how are you trying to do it?
<nigelb> http://irclogs.ubuntu.com/2011/05/23/%23launchpad-dev.html#t06:14
<nigelb> there ^^ that way.
<jml> nigelb: thanks.
<jml> nigelb: so, you are trying to order the subscribers on the web page by changing the ordering of Specification.subscriptions?
<nigelb> jml: yeah, and failing of course :)
<jml> nigelb: do you understand what lifeless meant by "cached property"?
<nigelb> they way the subscription is done seems vastly different from how the sprint was.
<nigelb> isnt the one with a @cachedproperty wrap?
<nigelb> *isn't it
<jml> nigelb: exactly.
<jml> nigelb: lifeless was saying, "rename ISpecification.subscriptions to ISpecification._subscriptions"
<jtv> bigjools: did you say that the Version(x) < Version(y) is not the right way to compare versions in a DSD?
<jml> nigelb: and then add a new property called "subscriptions"
<nigelb> oh
<jml> nigelb: this new property should just return _subscriptions, but sort it first.
<nigelb> jml: okay, let me see if I can get that bit.
<benji> gary_poster: I didn't see that one; I made one specifically for the bug;
<benji> I'm also in the wrong chan.
<gary_poster> :-)
<nigelb> Is there an easier way to clear cache other than stopping and starting LP?
<lifeless> jml: sorry?
<wgrant> nigelb: Which cache? The code cache?
<nigelb> wgrant: well, I changed the code, isn't reflecting on refresh, I presume its cached
<lifeless> jml: you might like to cast your eye over the 'wiki' lep. I suspect its a little gung ho without enough thought about the end product & what we want out of it
<wgrant> nigelb: Right, the appserver doesn't reload code, because in Python that doesn't really work.
<lifeless> jml: other than that I was going to propose a catchup at your convenience
<nigelb> wgrant: so restart LP? :)
<wgrant> nigelb: I tend to run 'bin/run -r librarian -i development' directly, since make run takes forever.
<wgrant> Right.
<nigelb> oh
<nigelb> I don't have to do make run every time? :O
<lifeless> wgrant: s/doesn't really work/really doesn't work/
<nigelb> (and yeah, it does take forever)
<wgrant> nigelb: Our Makefile sort of sucks, so it rebuilds far more than necessary.
<nigelb> \o/ It works!
<nigelb> lifeless, jml: Thanks :)
<wgrant> nigelb: Run 'make jsbuild' if you make JS changes, make something else if you make CSS changes, otherwise just use bin/run.
<nigelb> oh, wait. I need to write tests.
<lifeless> nigelb: there will be tests already
<lifeless> nigelb: you just need to update htem
<nigelb> whee, so just run all the tests, something will fail, fix them :D
<nigelb> hrm, I don't see any tests testing the sort order.
<lifeless> bin/test -vvt blueprints
<lifeless> I bet something will fail
<jml> lifeless: yeah.
<jml> lifeless: I'm unwell today, so maybe tomorrow morning UK time?
<lifeless> sure
<lifeless> get well soon
<nigelb> I guess it will take a while. Tests running.
<lifeless> night y'all
<wgrant> nigelb: blueprint doesn't have that many tests.
<wgrant> Which may be a good or a bad thing :)
<nigelb> night lif
<nigelb> night lifeless
<nigelb> wgrant: heh
<nigelb> wgrant: I fear doctests.
<wgrant> Everybody does.
<nigelb> I'm glad I have company.
<nigelb> Oh, well. Something did fail.
<nigelb> Of course, doctests.
<bigjools> jtv: right, you need to use apt_pkg.VersionCompare()
<jtv> bigjools: touchÃ©âI already implemented it.  :)
<deryck> Hi adeuring and henninge-lunch.  Running just a minute or two late for our standup.
<deryck> Morning everyone else! :-)
<jtv> bigjools: the hard part for tracking errors, I think, is finding the DSD that matches a PlainPackageCopyJob.  After all some PPCJs may not be related to DSDs.
<jtv> hi deryck
<bigjools> jtv: gavin added extra columns to the PCJ table to support querying.  I'd get all jobs and match what you can.
<jtv> Yes, I suppose.  It's not a very critical path, since it's an error case in an offline process.
<bigjools> jtv: when I say get all jobs, get all jobs for that archive/distroseries, then sift the json_data
<jtv> Yes I wasn't thinking of getting _all_ all jobs, thank you :)
<bigjools> jtv: just clarifying :)
<henninge-lunch> deryck: stand up over?
<deryck> henninge, no, still in progress
<henninge> deryck: wating for my laptop to boot
<henninge> oh, standby
<bigjools> henninge: did you report that bug?
<henninge> bigjools: no yet, sorry. otp now
<bigjools> henninge: I'll do it
<henninge> bigjools: ok
<henninge> bigjools: we had noticed that the failing branch had the same parent branch with a different url
<bigjools> henninge: https://code.launchpad.net/~launchpad/lp-source-dependencies/trunk ?
<henninge> bigjools: whereas the new, working checkout does not have a parent branch at all.
<henninge> yeah
<bigjools> henninge: what new checkout?
<henninge> bigjools: http://paste.ubuntu.com/611718/
<henninge> bigjools: I worked around it by creating a new checkout of lp-source-dependencies/trunk
<henninge> the paste ^ is the broken one.
<bigjools> henninge: my broken one has no parent
<henninge> so that is not related ...
<gary_poster> deryck, hi!  When you get a chance, I'd love to chat about CHR to get hints and clues
<henninge> bigjools: what's your bzr version
<bigjools> 2.3.3
<henninge> me, too.
<henninge> The bug does not appear to be in 2.3.1
<deryck> gary_poster, sure.  Just finishing standup.  Let me close out notes and then we can chat anytime.
<gary_poster> cool thanks a lot deryck
<henninge> bigjools: I have that on my desktop machine and a copy of the "broken" branch and it's not failing.
<bigjools> henninge: https://bugs.launchpad.net/bzr/+bug/786980
<_mup_> Bug #786980: bzr: ERROR: bzrlib.errors.ReadOnlyError: A write attempt was made in a read only transaction <Bazaar:New> < https://launchpad.net/bugs/786980 >
<henninge> bigjools: thanks, I'll comment there
<bigjools> henninge: oh well :/
<bigjools> I'll make a new checkout then
<deryck> adeuring, bug 739075 is still InProgress but you're done on that, right?
<_mup_> Bug #739075: Person:+questions timeouts <qa-ok> <question> <timeout> <users> <Launchpad itself:In Progress by adeuring> < https://launchpad.net/bugs/739075 >
<adeuring> deryck: yes, that should be "fix committed", if not fix released
<deryck> adeuring, awesome, thanks
<deryck> adeuring, it's db-stable stuff, so committed not released.
<adeuring> well, it's not "fix released": requires a DB schema change...
<jtv> allenap, bigjools (in alphabetical order): errors from package copy jobs need to appear to come from a Launchpad user.  Is there a user identity that properly represents the package copier?
<bigjools> jtv: no, we need to get the requesting user from somewhere
<jtv> bigjools: I thought it'd be more intuitively accurate to have "Launchpad" make the comment, and also help prevent confusion over manually pasted messages vs. automatically generated ones.
<bigjools> jtv: it would be better to show who caused the problem, then you can talk to them.,
<bigjools> we can make it look obviously auto-generated in other ways
<jtv> Yes, that'd be nice too.  But showing them as the actual person making the comment seems a bit too much of a good thing to me.
<jtv> We can, but AFAICS that means adding schema.
<jtv> (Not saying that's a problem, just a thing to keep in mind)
<bigjools> jtv: well there's a "launchpad" celeb
<bigjools> oh wait that's a project
<bigjools> the mild-mannered launchpad janitor?
<jtv> The janitor cleans things up.  Not the person people would look to as a spokesperson for industrial processes.
<deryck> gary_poster, I can chat now.  Or whenever you like from here on out.
<gary_poster> deryck, now would be great thanks
<gary_poster> skype or mumble?
<deryck> gary_poster, lets mumble!
<bigjools> henninge: ok so there's a workaround that doesn't require a re-checkout
<bigjools> use bzr switch lp:lp-source-dependencies
<nigelb> bah, doctests giving me random failures :(
<StevenK> nigelb: Such as?
<nigelb> StevenK: part of the test which exploded http://paste.ubuntu.com/611854/
<sinzui> jcsackett: do you have time to mumble?
<jcsackett> sinzui: i was just about to ask you that very question.
<StevenK> nigelb: "The joys of doctests."
<nigelb> StevenK: Hah. I know!
<StevenK> TALES does not love you, given that traceback
<jcsackett> sinzui: can you hear me?
<jcsackett> i cannot hear you, though i see that you are talking.
<sinzui> I ead you
<sinzui> heard
<nigelb> StevenK: how do I fix that? :(
<StevenK> nigelb: It looks like context/@@+portlet-subscribers is the issue
<StevenK> And given you changed subscribers ...
<nigelb> yeah, I did
<nigelb> let met get you my branch's diff on LP
<nigelb> *me
<nigelb> StevenK: My diff that caused the explosion :) http://bazaar.launchpad.net/~nigelbabu/launchpad/90628-spec-sub/revision/13102
<deryck> henninge, you're not in a hurry for your review, right?  Since you're OCR.  I'm just burning down a couple open tasks and then will turn to it, if that's ok.
<henninge> deryck: go, burn'em!
 * deryck lights a fire
<LPCIBot> Project windmill-db-devel build #307: STILL FAILING in 1 hr 17 min: https://lpci.wedontsleep.org/job/windmill-db-devel/307/
<deryck> sinzui, even though I'm a crybaby about project review, I do give you mad props from the new interface....
<deryck> sinzui, it's a million-hundred-thousand-and-two times better than it used to be. :-)
<sinzui> deryck: You should withhold props. I realised after I landed the UI that the license_approved field is smart and will change the project_reviewed field to true, but you do not see that in the UI :(
<deryck> sinzui, ah!  That is nicer still!  So we only have to mark the license approved in the web UI?
<deryck> sinzui, but I guess the list won't change for us to know what's left until that's fixed.
<deryck> still it's a nice improvement for ye days of old.
<deryck> s/for/from/
<sinzui> The list will change after you reload, the item does fall off the list. I think I need to add a callback to update the state the of other field in the ui.
<deryck> ah, ok
<deryck> sinzui, that's good to know thanks!
<sinzui> deryck: I chose to not remove the entry from the list after I changed the status because I was unsure of how to find the project if I make a mistake. Maybe we do not make mistakes often enough to care
<sinzui> nigelb, most tests is stories are rubbish.Most tests think they are testing the view and model code. A story test should be written from a user's perspective. It is an acceptance tests that demonstrates the user can accomplish a simple task and verify how he knows it was completed. I tend to delete failing stories. Sometime I do not need to write a replacement test in browser/tests because there already is one.
<nigelb> sinzui: At this point, I'm still not sure how my code caused something to fail :(
<sinzui> nigelb: this error is a string indication that the test is bad. A test of the model or the view would clearly state what is wrong and you would know how to fix it. tales hides the error most of the time. Can you paste a diff of your branch, or have you pushed it so that I can take a look?
<nigelb> I
<nigelb> I've pushed it
<nigelb> https://code.launchpad.net/~nigelbabu/launchpad/90628-spec-sub
<sinzui> nigelb: You have a simple branch. I do not see what broke the page yet. Maybe there is something else looking at the value. I think we need to look at browser/specification at the "Unsubscribe" action
 * sinzui does
 * nigelb looks too
<sinzui> wow, this does not use LaunchpadForm
<nigelb> yeah, I don't think blueprints have had much love :)
<sinzui> nigelb: their is a contradiction introduced by Specification.unsubscribe in the model
<sinzui> That method iterates over the .subscriptions, and mutates it. We have two choices
<sinzui> 1. clear the property cache after we change subscriptions or
<nigelb> not cache the property at all?
<nigelb> I hope (2) isn't rewrite unsubscribe :)
<sinzui> 2. Use a different way to get the subscription. The method is guessing at the subscription, and loading a lot of data what will not be used
<sinzui> let me check if there is already a method that gets the sub
<sinzui> There is not, so we choose option 1
<nigelb> aha
<nigelb> clear_property_cache ?
<sinzui> nigelb:  at a minimum, we want to
<sinzui> from lp.services.propertycache import get_property_cache
<sinzui> del get_property_cache(self).subscriptions
<nigelb> okay
<sinzui> nigelb: but we could just remove the sub so we do not reload the users. maybe...
<sinzui> get_property_cache(self).subscriptions.remove(sub)
<sinzui> SpecificationSubscription.delete(sub.id)
<nigelb> sinzui: that would delete the property entirely?
<sinzui> nigelb:  ^ I have not done this before, but It is worth a try because it means there is no performance hit. In fact, you are making Lp faster!
<nigelb> or remove that particular subscription?
<nigelb> in which case, I'll gladly try this!
<wgrant> bac: Hi.
<sinzui> nigelb: The first form deletes the cached value. The second gets the cached value (a list) and removes the found subscription from it
<nigelb> sinzui: so, the second form needs to be done along with what's existing right?
<sinzui> nigelb: yes. we are just teaching Specification.unsubscribe() about your change
<nigelb> okay, let me run the tests again :)
<nigelb> sinzui: Does this sort of difficulty happen often?
<sinzui> nigelb: I see that Specification.subscribe() is implicitly accessing .subscriptions() using a .subscription(person) We may need to delete the cache or add the user to the list (which means resorting it)
<nigelb> sinzui: ah, yes/
<sinzui> nigelb: yes, but the test issue is no, Blueprints was not actively developed between 2007 and 2011. There is a lot of bad code in it
<nigelb> sinzui: True :(
<nigelb> If it weren't for the fact that its going away, I would have loved to help remove some of the b0rk.
<sinzui> nigelb: I believe a lot of the code rules will be combined in a future issue tracker. So there should be no guilt is making the code better.
<nigelb> sinzui: In which case, I'm happy to fix whatever I can. Though its big of an overhead because I have to get someone to help :)
<LPCIBot> Project windmill-db-devel build #308: STILL FAILING in 50 min: https://lpci.wedontsleep.org/job/windmill-db-devel/308/
<wgrant> bac: You marked bug #777789 qa-bad... It doesn't seem like a really risky change, except for the Y.Node.create('<div />') thing. If it really is bad, could you please roll it back ASAP?
<_mup_> Bug #777789: "Other subscriptions" description of direct team subscription makes no sense <qa-bad> <story-better-bug-notification> <Launchpad itself:Fix Committed by bac> < https://launchpad.net/bugs/777789 >
<benji> henninge: would you like for me to review https://code.launchpad.net/~henninge/launchpad/bug-740208-leak-email-bug-title/+merge/61784?
<nigelb> sinzui: okay, the tests did pass :)
<nigelb> sinzui: Now I guess I need to figure out the subscribe() bit :)
<sinzui> nigelb: excellent
<nigelb> sinzui: How do I fix the subscribe() where it again modifies the list?
 * sinzui thinks
<sinzui> nigelb: how did you fix the unsubscribe case?
<nigelb> sinzui: I removed it from the cache as well.
<nigelb> so update the cache and the *real* one and then resort both?
<sinzui> okay. then I think we can do the same at # since no previous subscription existed, create and return a new one
<sinzui> nigelb: we want to append the sub to the cached list, then resort on displayname
<sinzui> Then we call notify()
<LPCIBot> Yippie, build fixed!
<LPCIBot> Project db-devel build #571: FIXED in 5 hr 15 min: https://lpci.wedontsleep.org/job/db-devel/571/
<nigelb> sinzui: does this look right? http://pad.ubuntu-uk.org/VX7IyBpAbD
<sinzui> nigelb: I am not sure you are mutating the cached list. Are you? I an sure you are working with the real item when I read
<sinzui> get_property_cache(self).subscriptions
<nigelb> ah
<nigelb> sinzui: is the self.subscription = correct? or is there a more correct call to update the item in cache?
<sinzui> nigelb: you approach might work because propertycache is returning a mutable instance. Well it probably is since I can change a cached person object.
<LPCIBot> Project windmill-db-devel build #309: STILL FAILING in 49 min: https://lpci.wedontsleep.org/job/windmill-db-devel/309/
<sinzui> bigjools: I think you need to do
<sinzui> get_property_cache(self).subscriptions =  sorted(subscriptions, key=lambda sub: sub.person.displayname)
<nigelb> aha
<nigelb> fixed that.
<sinzui> nigelb: self.subscription is a cached property so I do not think it can really be assigned without a setter explicitly defined.
<nigelb> ahh.
<nigelb> Does it look right now though?
<sinzui> yes, it does
<nigelb> :)
<nigelb> That definitely wasn't an "easy" bug.
<sinzui> nigelb: no bug is really easy without 3 months of experience working with Lp's byzantince code. I do think you could repeat your change to fix other bugs in less than a day, so this class of bug is not easy
<nigelb> sinzui: heh. I agree.  Unless its just text fixing, everything else needs a fair bit of delving.  I'm a bit more confident now.
 * bigjools morphs into nigelb
<nigelb> haha
<nigelb> that was a wrong ping :p
<bigjools> it's ok I'm used to Curtis' typos :)
<nigelb> This one's the most satisfying bug fix. I spent quite a bit of time trying to fix this :)
<LPCIBot> Project windmill-db-devel build #310: STILL FAILING in 51 min: https://lpci.wedontsleep.org/job/windmill-db-devel/310/
<deryck> sinzui, hi.  Could I get some time with you to do a pre-imp call on a bug?
<sinzui> I can talk now
<deryck> excellent.  I'll jump to mumble.
<nigelbabu> benji: Hi, could you review https://code.launchpad.net/~nigelbabu/launchpad/90628-spec-sub/+merge/62003 for me? :)
<sinzui> nigelbabu: why doesn't the diff show a change to subscribe()?
<nigelbabu> sinzui: woah, let me look!
<nigelbabu> sinzui: heh, did someone break that preview? :P
<nigelbabu> because its there on my branch http://bazaar.launchpad.net/~nigelbabu/launchpad/90628-spec-sub/revision/13102
<nigelbabu> Its showing a diff of last commit to branch vs trunk
<nigelbabu> instead of entire branch vs trunk.
<nigelbabu> wait.
<nigelbabu> its showing last commit to branch vs its previous commit :(
<nigelbabu> sinzui: ^^
<sinzui> nigelbabu: did you not commit and push changes toe the scrubscribe() method. I saw it on the collaborative notepad
<LPCIBot> Project windmill-devel build #120: STILL FAILING in 1 hr 10 min: https://lpci.wedontsleep.org/job/windmill-devel/120/
<gary_poster> In https://answers.launchpad.net/launchpad/+question/158714 , someone is asking to recreate an old PPA from when they left LP (they have now returned) .  Can anyone help with that?  I don't even know if it is possible.
<benji> nigelbabu: sure I'll review it for you, let me know when you get the merge proposal straightened out
<nigelbabu> benji: yup :)
<nigelbabu> sinzui: okay, now I'm confused. I commited everything and I see the code locally.
<nigelbabu> I did a bzr status and it didn't return anything
<LPCIBot> Project windmill-db-devel build #311: STILL FAILING in 50 min: https://lpci.wedontsleep.org/job/windmill-db-devel/311/
<nigelbabu> bah, I didn't save the file :/
<sinzui> nigelbabu: but did you push the branch to lp:~nigelbabu/launchpad/90628-spec-su
<sinzui> bzr st -r branch:lp:~nigelbabu/launchpad/90628-spec-sub
<nigelbabu> sinzui: I hadn't done the unsubscribe, my mistake.
<nigelbabu> well, I didn't save the file, so it didn't show up in bzr st :\
<nigelbabu> sinzui: IRobot++ :)
<sinzui> No one will follow up on my suggestion to break IPerson is role/app interfaces. ITranslator, IQuestioner, IBugger
<nigelbabu> haha
<nigelbabu> IBugger lol
<sinzui> nigelbabu: did specification-notifications.txt really verify you could subscribe someone twice?
<nigelbabu> sinzui: didn't get you
<nigelbabu> sinzui: until I removed the two instances of test@canonical.com, the test wouldn't pass.
<nigelbabu> poolie: woah, thanks for featuring cjohnston and me on the blog :)
<sinzui> nigelbabu: I think you  may have fixed another bug :)
<nigelbabu> sinzui: with the sort?
<nigelbabu> woah
<sinzui> A user cannot/should not  be able to subscribe twice, yet the test you changed appears to say a user could be subscribed twice :(
<cjohnston> go nigelbabu
<sinzui> Since the code clearly did not allow it, I think the defect is in sample data. That is why we prefer to test using a factory to create only the objects under test.
<nigelbabu> hrm, do you want me to write new unit tests?
<nigelbabu> I could potentially write something under browser/test
<nigelbabu> sinzui: anything more you want me to do on that branch before I ask for it to be reviewed? :)
<sinzui> nigelbabu: I am not going to burden you with extra tests. I may write one. I am reading lib/lp/blueprints/doc/specification-notifications.txt and I am perplexed by the test@canonical.com addition (Sample Person). Your change looks like what should have always been the case. Why were we really sending Sample Person  two emails? Where they the same email?
<nigelbabu> sinzui: heh, indeed.
 * sinzui looks for existing bug
<nigelbabu> sinzui: and why hasn't anyone complained yet
<nigelbabu> we really need to fix mails from blueprints.  Its quite chatty like bugs.
<nigelbabu> switching nicks.
<nigelb> o/
<sinzui> nigelb: The test indicates sample person was not subscribed. Then we was, and he got 2 emails. Yet your change is made *before* notify where you ensure the user subscription is created, and the list of subscribers is sane. I am going to add a sanity check to the existing test to search how many notifications.
<nigelb> sinzui: okay, will you be making your change on top of mine?
<LPCIBot> Project windmill-devel build #121: STILL FAILING in 46 min: https://lpci.wedontsleep.org/job/windmill-devel/121/
<sinzui> nigelb: I think something is broken. The doctest does not do what it thinks it dos. It does not commit the transaction from the previous block and verify that Sample Person got an email stating that he was subscribed. I can see that two messages were sent to test@, the first because the the subscription two blocks before, then because of the spec change. your test change implies the subscribe email was not sent
 * sinzui crafts a test fix to verfiy the expected behaviour
<nigelb> sinzui: so, my change should not have been necessary?
<sinzui> nigelb: You have already made the change to the second chunk. Make the first set of changes to verify a subscription email was sent: http://pastebin.ubuntu.com/611986/
<nigelb> sinzui: okay!
<sinzui> I see that the code that manages email notification is not in blueprints. That is why I could not see what was going on
<nigelb> ahh
<nigelb> I'm not sure if I'm picking progressively harder LP bugs, but it sure does seem that way :)
<nigelb> sinzui: pushed :)
<sinzui> nigelb: did the test pass with the change?
<nigelb> sinzui: woops, didn't run them.  gimme 5 minutes :)
<sinzui> The change you made looks correct
* lifeless changed the topic of #launchpad-dev to: Performance Tuesday! | https://dev.launchpad.net/ | On call reviewer: henninge, benji | https://code.launchpad.net/launchpad-project/+activereviews | Critical bugs:214 - 0:[######=_]:256
<lifeless> sinzui:     "specificationsubscription_spec_person_uniq" UNIQUE, btree (specification, person)
<lifeless> sinzui: even the sample data can't have someone subbed twice
<sinzui> hm
<sinzui> lifeless: fab, So we just need to learn if we lost a subscription  with the change.
<lifeless> sinzui: so, disclosure eh?
<jcsackett> henninge or benji: can i throw https://code.launchpad.net/~jcsackett/launchpad/dont-expose-domains/+merge/62023 on the review queue?
<benji> jcsackett: Sure. I can look at it right now in fact.
<jcsackett> benji: great, thanks!
* benji changed the topic of #launchpad-dev to: Performance Tuesday! | https://dev.launchpad.net/ | On call reviewer: benji | https://code.launchpad.net/launchpad-project/+activereviews | Critical bugs:214 - 0:[######=_]:256
<nigelb> sinzui: bah, 3 tests failed :(
<nigelb> let me echo it into a file and upload somewhere
<sinzui> You ran -m lp.blueprints?
 * sinzui can see the doctest in question passed
<lifeless> deryck: gary_poster: sinzui: -extremely- rough as yet - https://dev.launchpad.net/ArchitectureGuide/ServicesRoadmap
<nigelb> sinzui: no, I ran bin/test --vvt blueprints
<LPCIBot> Project windmill-devel build #122: STILL FAILING in 46 min: https://lpci.wedontsleep.org/job/windmill-devel/122/
<gary_poster> lifeless, I expect you've been considering where Python 3 might fit into this as well?
<deryck> lifeless, thanks for the link.
<deryck> lifeless, on first read, looks good.
<sinzui> nigelb: that is about the same as -m lp.blueprints. It may have run more
<sinzui> nigelb: what were the failing tests. I merged your changed and I see that doc/specification-notifications passes
<nigelb> sinzui: specification.txt fails
<nigelb> and sprint-meeting-export.txt
<gary_poster> lifeless, curious in regards to rest bits about "optimises for things that don't really affect us".  I would guess that some do, and some don't.  Can I assume that we at least will follow standard and basic "GETs don't write" no matter what the protocol?
<lifeless> gary_poster: hell yes
<gary_poster> ok, good
<gary_poster> lifeless, and did you already reject the github xmlrpc competitor I mentioned?  It's fine, I just saw it mentioned, but I thought it was worth making sure that you had seen it.  Trying to find it again
<lifeless> gary_poster: I had a look, I think I disliked something - I
<gary_poster> lifeless, then that's good enough for me
<lifeless> 'll have another look now and either document that or add it in
<sinzui> nigelb: okay. good. This means the cache was not made yet. We only want to update the cache is it was made!.
<gary_poster> eh, maybe so noone else bothers with it
 * sinzui reads code
<lifeless> gary_poster: yeah
 * sinzui run test again
 * sinzui has a fix
<lifeless> gary_poster: oh, I remember
<lifeless> its erlang binary wire encoding
<sinzui> nigelb: I am prepare a patch for you. It incorporates my change as well as a number of lint fixed from by unposted review.
 * nigelb hugs sinzui
<lifeless> gary_poster: which is a bit esoteric for something folk may have to debug from time to time
<gary_poster> lifeless, yeah.  Of interest if (a) they have thought something through that we ought to; or (b) we ever care about erlang, but I hear you.  I don't think I saw a reply to my Python 3 question: do you expect these services to be in Python 3?
<lifeless> gary_poster: on the restful side, it optimises to make the path visible and have the ability to cache, and neither of those really make a difference to us.
<lifeless> gary_poster: If all things are equal, I think restful is nicer than xmlrpc
<lifeless> just cause I'm an http junkie
<gary_poster> making path visible and giving ability to cache *and* giving another opportunity to divide up services by boxes
<gary_poster> (different resources by different machines)
<lifeless> gary_poster: python3; uhm, if we can, sure. Python3 in lucid is 3.1, but we'd really only have the standard library available to us.
<lifeless> gary_poster: yes, but we're already dividing the services up
<lifeless> gary_poster: so we don't need a single url space for heterogeneous services internally
<gary_poster> And we'll never have another bottleneck?  Rephrased: taking easy steps for future proofing makes sense to me
<nigelb> Tests with failures:
<nigelb>    lib/lp/blueprints/doc/specification.txt
<nigelb>    lib/lp/blueprints/doc/sprint-meeting-export.txt
<nigelb>    lib/lp/blueprints/stories/standalone/subscribing.txt
<nigelb> sinzui: ^^
<gary_poster> I don't mean a single space for heterogeneous services, actually lifeless
<gary_poster> I mean...
<lifeless> gary_poster: we're starting out with horizontal scaling built-in for each microservice
<gary_poster> like, resources beginning with identifiers A-M are handled by box X
<gary_poster> or resource foo is a biggie
<gary_poster> it is handled by special box Y
<sinzui> nigelb: apply this patch to add a guard, wrap long lines, and remove trailing whitespace: http://pastebin.ubuntu.com/612018/
<lifeless> gary_poster: I think for most of our internal services doing that would be an antipattern
<nigelb> sinzui: tests fail for lint? :O
<gary_poster> lifeless, using what mechanism (and why does it make it unnecessary to consider others)?
<gary_poster> (horizontal scaling)
<lifeless> gary_poster: haproxy to some N instances, where N >= 2
<gary_poster> lifeless, that's a common pattern...as is the one I'm describing.  They both are valuable for different sorts of problems
<lifeless> agreed
<sinzui> nigelb: they will no fail for lint, but we have a coding style. Some merges get screwed up if we do not follow the recommended style, such as alphabetised imports
<gary_poster> and I don't see any convincing evidence that we would have only a single kind of problem :-)
<nigelb> sinzui: wah, I didn't know about the alphabetized imports
<lifeless> gary_poster: I think we have many kinds of problems; we're selecting a -default- not a -required-
<lifeless> gary_poster: uhm, to me, that means picking the cheapest-to-deploy-and-use 80% fit
<sinzui> nigelb: and I did not fix line 50 in you Mp's diff, those two imports should be on single lines so that subsequent changes do not inflate the diff
<lifeless> gary_poster: so what I'm saying isn't that the restful design principles are invalid - they are valid
<sinzui> nigelb: ps, You are a much better engineer after today
<nigelb> sinzui: you mean on individual lines?
<nigelb> sinzui: hehe
<lifeless> gary_poster: I'm saying that it optimises - by doing the things we're discussing - for specific cases, most of which won't affect us most of the time
<sinzui> nigelb: yes
<nigelb> aha, /me fix0rs
<gary_poster> lifeless, cheapest to deploy and use 80% fit: that can be primary guide, but "supporting best practices" is a reasonable secondary guide
<gary_poster> lifeless, so the convincing argument get me to be quiet about this is whether or not it really is that much harder to build a REST service--or at least
<benji> jcsackett: hi, I added some comments to https://code.launchpad.net/~jcsackett/launchpad/dont-expose-domains/+merge/62023 for you to consider.
<gary_poster> a service that is more REST than XMLRPC
<lifeless> gary_poster: I have no desire to get you to be quiet :)
<gary_poster> lifeless, lol, I have a desire to be quiet ;-)
<jcsackett> benji: dig.
<lifeless> gary_poster: so building a REST service requires url routing that is object identity aware; building an RPC service requires routing that is function aware
<gary_poster> perhaps a better phrasing would be, "...a way to convince me is to determine whether..."
<lifeless> gary_poster: ditto consumption
<gary_poster> true
<lifeless> gary_poster: one has a hierarchy that has to be designed - and many toolkits don't offer great migration support (by which I mean after you make a mistake, what then)
<benji> Everyone enjoys the melodic bass tones that emanate from gary.
<lifeless> gary_poster: RPC on the other hand is essentially a freeform space, which on the downside is basically freeform
<gary_poster> benji :-P :-)
<gary_poster> lifeless, if XMLRPC is freeform, that means it doesn't have migration support either
<gary_poster> (and I would agree with that)
<gary_poster> If instead you begin with a simple pattern like lazr.restful has
<gary_poster> with a first element that is in essence a namespace
<lifeless> gary_poster: it doesn't, but it doesn't need migration support anywhere near as much because of two things: you can trivially bind a function to two names (giving you migration), and the function name is not impacted or altered by data model.
<gary_poster> then you have the building blocks for a nice migration story
<gary_poster> lifeless, I'm not clear on how the RESTful namespace doesn't give you similar power
<lifeless> perhaps I misunderstand, but AFAICT you'd be cloning all your functions across
<lifeless> so something like /1/object/query_op?params
<gary_poster> (where "1" is the namespace?
<gary_poster> )
<lifeless> would migrate as /2/object/differentname?params
<lifeless> which would work for many things because you'd be able to register the new thing
<lifeless> gary_poster: I'm sure something can be worked out - but this is kindof my point - its a very heavy hammer vs migrating individual methods, and thats driven by the structure (which is its blessing and its curse)
<lifeless> flacoste: are we on now ?
<benji> as a reformed Windows developer I have bad memories of APIs having a "Foo" function and adding a "FooEx" function in order to change the signature for a later version of Windows
<gary_poster> lifeless, I didn't follow so much...
<sinzui> nigelb: I approved your branch. I am landing it now
<gary_poster> lifeless, we can continue at later date :-)
<nigelb> sinzui: woah, thanks :)
<lifeless> gary_poster: sure ;)
<nigelb> sinzui: I was still trying to run the tests again before poking you to land :)
<sinzui> okay. I need to reconfigure to land anyway. Lp assumes that Lp engineers only hack on the Lp project
<lifeless> gary_poster: here is a concrete example of the sort of complexity rest adds - is a 404 a missing object or a bad method name? [python itself has this complexity, so it is toleratable but  still confusing]
* benji changed the topic of #launchpad-dev to: Performance Tuesday! | https://dev.launchpad.net/ | On call reviewer: - | https://code.launchpad.net/launchpad-project/+activereviews | Critical bugs:214 - 0:[######=_]:256
<gary_poster> lifeless, (niggle where I might be wrong but I think I'm right: with "true" REST at least AIUI, you don't have a method name in the URL...but you might have specialized objects and a 404 inherently doesn't tell you where you went wrong specifically with the path and so your underlying point is still valid ISTM) XMLRPC doesn't buy you anything there, lifeless.
<gary_poster> REST isn't adding complexity, but offering a solution equivalent in my mind to what an RPC offers.  RPC let's you send a payload back that you have to know how to interpret; and so can REST.  Perhaps you could even argue that REST has a more regular and yet flexible language for communicating errors and what to do with them (with hypertext) but decisions still need to be made on how to interpret the error response, no
<gary_poster> But again, there's no XMLRPC win.
<gary_poster> Maybe there's a Google protocol win there, or whatever the Facebook equivalent is; I haven't looked at those.  I suspect it is easier to argue against XMLRPC than some of the newer options.
<gary_poster> (Well, I've looked at them, but not lately, and I've forgotten the error passing mechanisms)
* benji changed the topic of #launchpad-dev to: Performance Tuesday! | https://dev.launchpad.net/ | On call reviewer: benji | https://code.launchpad.net/launchpad-project/+activereviews | Critical bugs:214 - 0:[######=_]:256
<lifeless> benji: (re FooEx - yes, mistakes Will Happen) :)
<benji> gary_poster: are you referring to Protocol Buffers? or maybe Google Data Protocol? <shudder>
<benji> the former is just a data format (like JSON) as far as I know so it doesn't have an error channel, and the latter should die in a fire (it [ab]uses Atom as the data format)
<gary_poster> benji, let's see the Facebook thing is Thrift...
<benji> I'll have to take a look at Thrift.
<gary_poster> benji, and for Google I was thinking of protocol buffers, yeah
<gary_poster> which I see you are right, is just a data format, as opposed to thrift, which is more of a full-flledged rpc
<lifeless> thrift is apparently fffugly
<lifeless> according to the cassandra folk, which are trying to make thrift diaf
<benji> lifeless: the wire format?
<lifeless> benji: the whole kit and caboodle
<benji> heh
<gary_poster> heh, ok
<lifeless> http://code.google.com/p/thrift-protobuf-compare/wiki/Benchmarking
<lifeless> is an interesting read
<lifeless> gary_poster: FWIW I'd be entirely happy with us making early mistakes here - we're aiming for such narrow services migrating one to a new protocol should be low cost
<lifeless> gary_poster: not saying I *want* to make mistakes
<gary_poster> link is interesting, yes.  never heard of kryo...or many others
<lifeless> gary_poster: just that I think the cost is easily absorbed
<benji> gary_poster: there are several RPC mechanisms based on protocol buffers: http://code.google.com/p/protobuf/wiki/ThirdPartyAddOns#RPC_Implementations
<gary_poster> want to make mistakes: heh :-) .  lifeless, that's fair enough, but IME, the "I should have had a V8" moment (did you all have that ancient ad campaign?) with the XMLRPC approach was pretty far down the line for us, when expenses were much higher than they could have been, for following what is so often described as best practice anyway.
<gary_poster> benji, which is the one that should diaf? :-)
<nigelb> sinzui: Thank you so much for helping me out today. Even though LP is hard, its nice you folks are all so helpful :)
<LPCIBot> Project windmill-devel build #123: STILL FAILING in 46 min: https://lpci.wedontsleep.org/job/windmill-devel/123/
<benji> gary_poster: all of those are innocent until immolated; I was referring to http://code.google.com/apis/gdata/
<gary_poster> benji, lol, ok cool
<sinzui> nigelb: Your welcome. It is always nice to see contributors completing a goal
<gary_poster> one is developed on LP, so obviously superior ;-)
<benji> obviously the bad part is that none of those are "official"
<benji> heh
<nigelb> sinzui: :)
<lifeless> which one is developed on LP ?
<gary_poster> lifeless, Twisted based: https://launchpad.net/txprotobuf/
<benji> gary_poster: last commit was 3 years ago
<gary_poster> bah :-)
<gary_poster> that's because the twisted stuff is rock solid! :-)
<rockstar> gary_poster, you should ask lucio about protobuf and Ubuntu One.
<gary_poster> rockstar, can you give a one sentence preview? :-)
<benji> gary_poster: the only ones with Python bindings that have any sort of recent activity apear to be http://zeroc.com/ice.html and http://code.google.com/p/protobuf-socket-rpc/
<rockstar> gary_poster, not much more than what vds just told me (he's sitting right in front of me): It's slow, had a lot of problems, lucio would know more.
<gary_poster> benji, I left the first when it started looking commercial...did I leave too soon?
<benji> (and ICE doesn't list Ubuntu as supported)
<gary_poster> fair enough, thanks rockstar.  lifeless, I expect you saw that, and quite possibly knew that already :-)
<benji> gary_poster: it is commercial (and GPL)
<rockstar> gary_poster, thrift is my serializable protocol of choice right now.
<gary_poster> ah
<gary_poster> rockstar, ack
<gary_poster> rockstar, lifeless was saying that cassandra wanted it to diaf
<rockstar> thrift?
<gary_poster> yeah
<rockstar> Huh. I assume the Casandra people are much smarter than I.
<gary_poster> <shrug> :-)
<jcsackett> benji: i've pushed up changes on my MP and replied to your comments.
<benji> I assume we want a synchronous protocol (meaning that protobuf+rabbitmq wouldn't be attractive).
<benji> jcsackett: cool, I'll look now
<gary_poster> benji, I assume so as well.  AIUI, we're talking about building pages
<lifeless> gary_poster: I did :)
<gary_poster> :-)
<deryck> sinzui, hi.  Mind doing an easy review for me, based on this morning's call?
<sinzui> okay
<lifeless> gary_poster: currently slow in the cons, based on talking with lucio :)
<gary_poster> lifeless, heh, cool
<deryck> sinzui, https://code.launchpad.net/~deryck/launchpad/bug-activity-do-not-snapshot-680131/+merge/62035
 * gary_poster running away
<lifeless> ciao
<gary_poster> bye!
<lifeless> flacoste: yo?
<deryck> sinzui, thanks!
<sinzui> deryck: r=me
<deryck> sinzui, many thanks!
<benji> jcsackett: I think I'm confused.  The code to add prefixes to generated names is still there and doesn't seem to be tested anywhere else; shouldn't we test it in test_nickname.py?
<jcsackett> benji: it is tested; in the collision test use up the nick with the suffix, and try the nick generation again to trigger prefix.
<jcsackett> that was the copied code you saw earlier--me getting ready to set up that part of the test and forgetting it. now it's there.
<jcsackett> the rest of the doctest deals with it using more and more of the email domain.
<benji> ah! I'll look at that then.  Thanks.
<benji> jcsackett: I'm done with https://code.launchpad.net/~jcsackett/launchpad/dont-expose-domains/+merge/62023.
* benji changed the topic of #launchpad-dev to: Performance Tuesday! | https://dev.launchpad.net/ | On call reviewer: - | https://code.launchpad.net/launchpad-project/+activereviews | Critical bugs:214 - 0:[######=_]:256
<jcsackett> thanks benji.
<benji> my pleasure
<nigelb> night folks, hopefully I should have a mail with the landing results when I wake up :)
<lifeless> sinzui: I've tagged bug 633 as disclosure
<lifeless> sinzui: this may be incorrect, so I'm letting you know ;)
<sinzui> I hate you
<lifeless> sinzui: I love you too
<sinzui> I was using that tag to ask my team to fix them. I wont now
<lifeless> sinzui: well thats why I'm mentioning it
<sinzui> I think I need a new tag that only jml can assign
<lifeless> sinzui: you can use disclosure for that
<lifeless> what tag should I use here though?
 * sinzui is not working a a 13 month feature again
<lifeless> the bug is about accidental information disclosure when someone subscribes the wrong person
<lifeless> so your trusted picker work will reduce the incident rate (but not eliminate it)
<sinzui> lifeless: We used privacy or private before. I added disclosure to the bug that were very relevant to features stakeholders were requesting
<lifeless> I'll change it to privacy
<sinzui> I wonder what the overlap is?
<lifeless> well
<lifeless> if this bug were fixed a stakeholder would be a little bit happier right now :P
<sinzui> oh
<sinzui> lifeless:  you are going to laugh at me. I thought you wrote that you added the 'disclosure' tag to 633 bugs, or 10% of all Lp bugs
<lifeless> sinzui: HAHAHHAHAHHAHAHAHAHAHAHAHA
<lifeless> sinzui: your comments make a lot more sense now
<sinzui> lifeless: we have several dupes of 633. We absolutely are fixing this.
<lifeless> sinzui: I know you have dupes about accidental subscriptions
<lifeless> sinzui: this is about dealing with them after they happen (and they still will), I wasn't sure if that was on your plate
<sinzui> I Changed Answers API to collect subscribed_by  two weeks ago, and secretly changed the permission so that LOSAs can remove answer contacts
<lifeless> how do you get the email address for someone suspended ?
<sinzui> staging
<sinzui> I am not joking
<lifeless> is there a bug for this?
<sinzui> No, There was a bug, but we had a partial fix so it was closed
<lifeless> do we cc feedback@ or anything on mails to hacked-account-spammers ?
<jcsackett> i haven't. wiki doesn't say anything about it. doesn't seem like a bad idea though.
<sinzui> We have not this year, we used to last year and it was felt that we should track this in answers when we can
<lifeless> ah
<lifeless> so I should have insisted on a Question
<sinzui> Of course a suspended user cannot use answers, so we end up asking the question. Which is one reason ~registry got permission to just fix the account
<lifeless> right, I've cc'd feedback
<lifeless> because there was no Question
<sinzui> fab
<lifeless> bac: hi
<sinzui> benji: was bug 315563 solved by muting or filters?
<_mup_> Bug #315563: can't unsubscribe from team-related bug-activity emails <lp-bugs> <Launchpad itself:Triaged> < https://launchpad.net/bugs/315563 >
<lifeless> sinzui: no
<lifeless> sinzui: assignee != subscription I suspect. IMBW
<lifeless> sinzui: the data model permits muting a *subscription* but its flexible enough to treat assignment as a form of subscription
<sinzui> That's right, the assignee is added the the subscribers list if he is not explicitly subscribed
<lifeless> exactly. But its only a presentation as a subscriber. Its not modelled as such lower down. It probably should be.,
<benji> sinzui: muting, where applicable.  It isn't possible to mute a team subscription if the team has a contact email address because we (LP) isn't in charge of parceling out the individual emails and so can't squelch the ones to people with mutes
<sinzui> benji: thanks.
<lifeless> benji: am I wrong? you can mute an assignment ?
<benji> lifeless: unfortunately I don't know; I'm inclined to say you can't.
#launchpad-dev 2011-05-24
<lifeless> sinzui: you will laugh and cry at bug 618356
<_mup_> Bug #618356: Person-picker takes a very long time consistently <disclosure> <lp-blueprints> <pg83> <sprints> <timeout> <Launchpad itself:Triaged by launchpad-teal-squad> < https://launchpad.net/bugs/618356 >
<lifeless> sinzui: full text queries without a full text index are slow.
<sinzui> lifeless: we are discussing this right now actually. Why do we do a fti searching for a person? I think we care about displayname, not homepage_content
<lifeless> sinzui: the simplest thing would be to drop the fti lookup
<lifeless> sinzui: but if we do /any/ fti lookups on person we will need that index
<lifeless> sinzui: anyhow, its in your squads court; I just dug into the query from curiosity; I hope that helped.
<sinzui> lifeless: It was a very timely query. Thank you very much
<wallyworld_> huwshimi: http://people.canonical.com/~ianb/person-picker-extra-detail.png
<huwshimi> Ugh, my wifi is flaky
<LPCIBot> Project windmill-db-devel build #312: STILL FAILING in 1 hr 10 min: https://lpci.wedontsleep.org/job/windmill-db-devel/312/
<wgrant> lifeless: How does it not explain the cause?
<wgrant> It says there are already binaries published in the archive.
<wgrant> It is less than completely transparent.
<wgrant> But it *does* explain the cause for anybody who knows anything about Debian archives in the last 10 years.
<lifeless> wgrant: its sufficiently opaque to confuse doko.
<wgrant> True.
<wgrant> Why does an Icelandic volcano always erupt like a month before we go to Europe? :/
<wgrant> Die, git, die.
<wgrant> WedonotneedaForkbuttondammit
<lifeless> man, users get *so* confused
<lifeless> bug/158702
<wgrant> Yes.
<wgrant> But this user has a bit of karma.
<wgrant> It's not the normal confused brand new user case.
<spiv> Ah, heh.
<lifeless> no
<lifeless> its a confused established user
<spiv> I think they meant question 158072?
<spiv> Er, 158702.
<lifeless> wow
<lifeless> bug queries with structuralsubscriptions in the are -really- slow
<lifeless> still running
<lifeless> still running
<lifeless> !
<StevenK> lifeless: When doing the initial work for populate-spr-changelogs, the initial query took 65 *minutes* on DF
<LPCIBot> Project windmill-devel build #124: STILL FAILING in 1 hr 19 min: https://lpci.wedontsleep.org/job/windmill-devel/124/
<lifeless> heh
<lifeless> this is up at 45 now
<lifeless> you can see why its timing out in prod
<lifeless>  Aggregate  (cost=816866.73..816866.74 rows=1 width=0)
<mwhudson> that's quite a lot of milliseconds
<lifeless> thats cost, not ms
<lifeless> running it the time was past  45 minutes when I nuked it
<lifeless> cost may be ms, but its actually not comparable between pg instances unless - all- the config knobs are identical (including CPU)
<mwhudson> ah
<lifeless> plans are evaluated by cost
<lifeless> the cheapest is chosen
<lifeless> the knobs say things like 'a block of random IO costs X'
<lifeless> and an in-memory sort of width 500 length 200 costs Y
<LPCIBot> Project windmill-devel build #125: STILL FAILING in 50 min: https://lpci.wedontsleep.org/job/windmill-devel/125/
<sinzui> wgrant: ping
<wgrant> sinzui: Hi.
<sinzui> wgrant: I want to talk about bug 561380
<_mup_> Bug #561380: /builders page is oopsing <boobytrap> <disclosure> <lp-soyuz> <oops> <Launchpad itself:Triaged> < https://launchpad.net/bugs/561380 >
<wgrant> I'm not sure it requires any discussion, but OK :)
<wgrant> The fix is very clear and simple.
<lifeless> delete the page yeah ?
<sinzui> wgrant: Since the template is just reporting current state; I propose PackageBuildFormatterAPI return either nothing, , state thee is nothing to see, or the link the build obfuscating the private team+ppa,
<wgrant> sinzui: We already have facilities to show private BPBs.
<wgrant> We just need to extend that to SPRBs.
<wgrant> lifeless: Why?
<lifeless> wgrant: I have a massive headache; my sense of humour gets distorted at such times
<sinzui> I do not see the formatters for that.
<wgrant> lifeless: I suspected it may have been an attempt at humour, but you are not unknown for making radical proposals.
<wgrant> sinzui: Let me see.
<wgrant> Oh, I see.
<wgrant> This is different from the SPRB bug.
<wgrant> So, yes, Julian's analysis is correct.
<wgrant> But hmm.
<wgrant> I think this may be fixed.
<sinzui> wgrant: PackageBuildFormatterAPI think it is checking that the user can see it:
<sinzui> if not check_permission('launchpad.View', build):
<sinzui>             return 'private source'
<wgrant> IPerson's launchpad.View adapter allows commercial admins to see the person.
<sinzui> But that guard predates the bug reports
<wgrant> s/the person/private teams/
<wgrant> Sure, in this case he can view the build because he can view the archive because he's a commercial admin.
<wgrant> But the archive is owned by a private team.
<wgrant> Which he can see (at least now).
<wgrant> I guess the adapters have changed since PMTs were removed.
<wgrant> But let's see.
<wgrant> Heh.
<wgrant> Yes.
<wgrant> You fixed this in August.
<wgrant> 11327.2.1 "Allow commercial admins to view private teams."
<sinzui> Yes, but was bigjools the victim here?
<wgrant> "You are logged in as Julian Edwards."
<wgrant> Unless we have another Julian Edwards.
<sinzui> He has also been a commercial admin for more that a year, so how did he see it
<wgrant> hmm?
<wgrant> This was filed in last April.
<wgrant> When he could see all private PPAs due to his commercial adminity.
<wgrant> But couldn't see private teams.
<sinzui> Sorry
<sinzui> I have no ind
<sinzui> mind
<wgrant> No ind!
 * sinzui shuts up
<wgrant> This sounds disastrous.
 * wgrant closes ze bug.
<sinzui> ViewBinaryPackageBuild looks sensible too
<wgrant> Yeah.
<wgrant> SPRBs into private archives may still be problematic.
<wgrant> But BPBs are fine.
<wgrant> (assuming the security adapters aren't insane, and in this case they were -- access to a subordinate object was granted without having access all the way up to the root)
<wgrant> jtv: The conventional way to avoid this discussion is just to use ~janitor.
<wgrant> jtv: I think that may be prudent here.
<jtv> wgrant: been over that.  No.
<wgrant> :(
 * jtv relocates
<wgrant> lifeless: Any ideas on that maybe-bad rev?
<wgrant> I've tested it in a few browsers and it seems to work OK.
<wgrant> And bac did not respond to my cries last night.
<wgrant> And the feature is as-yet unreleased.
<wgrant> So I think I shall qa-ok it and the two other things and organise a deployment.
<wgrant> Hm.
<wgrant> Although we are LOSAless.
<wgrant> So I guess it doesn't matter all that much.
<wgrant> lifeless: You were saying earlier that person.fti was unindexed, but I see a gist index on it.
<wgrant> sinzui: Still around?
<sinzui> yes
<lifeless> wgrant: where are you looking
<wgrant> sinzui: I am attempting to install Balsamiq Mockups. They have am amd64 .deb, but I cannot locate an amd64 build of Adobe AIR. Am I missing something here, or must I hack the i386 one?
<lifeless> wgrant: revert the bad rev
<wgrant> lifeless: Huh, you're right, it's not on the production DB.
<wgrant> But is on dev.
<StevenK> wgrant: http://kb2.adobe.com/cps/521/cpsid_52132.html
<wgrant> lifeless: :(
<sinzui> yuck.
<lifeless> wgrant: the different between theory and practice
<wgrant> StevenK: BURN
<sinzui> I am using i386
<StevenK> sinzui: The 90s called, they want their archtag back
<wgrant> i386 VM time.
<lifeless> its ironic to be using proprietary software to build open source software
<lifeless> by ironic I mean saddening
<wgrant> lifeless: *cough splutter* Launchpad *cough cough*
<sinzui> StevenK: I gave up on 64 when I realised that the apps were only using more memory  and not providing more performance
<StevenK> wgrant: By various hacks that I don't want to recall, I got AIR to install on amd64
<wgrant> StevenK: Yes, but I probably don't want Adobe crap outside a VM anyway.
<jtv> wgrant: at least we fixed Launchpad.
<StevenK> sinzui: But it's a complete waste if you have more than 4GiB RAM
<wgrant> jtv: This is true.
<sinzui> not soon enough for me. I was tired if having deadlines with a broken environemnt
<jtv> StevenK, wgrant: mind if I kick df for an upgrade?
<StevenK> YES
<wgrant> I believe StevenK is currently touching it in bad ways.
<jtv> wgrant: mind if I kick StevenK for an upgrade?
<StevenK> I'm using it, hands off
<jtv> Well keep the door open just in case.
<jtv> Or do you mean "I'm using it.  Hands off."?
<StevenK> I updated the code before lunch, so about 2 hours ago
<jtv> Looks like that was still well before my overnight landing percolated through.  :(
<lifeless> wgrant: so bac's rev
<lifeless> wgrant: I think it should be reverted
<lifeless> wgrant: i said as much in the bug
<wgrant> I shall revert it.
<wgrant> You did.
<jtv> Ahhh what luck it was merely the version number that was out of dateâmy branch is in there.
<jtv> StevenK: mind if I sync a few oneiric packages on df?
<StevenK> jtv: Yes
<jtv> hmmmgrr
<StevenK> I'm playing with the package copier
<jtv> Ah!
<jtv> Then maybe you can just tell me what I need to know.
<jtv> In fact, looks like (once I got past the timeouts just now) I've found an example of just the display I was trying to reproduce.  Thanks
<wgrant> Gnargh testfix.
<wgrant> lifeless: Can we disable the rabbit test pls?
<StevenK> s/test //
<lifeless> wgrant: gavin has an improvement he is/has landed
<wgrant> Stil fails.
<wgrant> +l
<lifeless> wgrant: its possible that the failure is due to that improvement, or that the improvement hasn't reached bb
<wgrant> No, this is clearly a failure with the new code.
<wgrant> Hard to tell if it's the same failure that we've hit 20 times in the last week.
<wgrant> But concealed by the improvement.
<lifeless> so sure - critical bug for disabled test, etc
<wgrant> You know it's failing probably near half the buildbot runs, right?
<lifeless> wgrant: I just agreed to disabling it!
<wgrant> Yes, but your suggestion that the improvement caused the failure suggests that it hadn't already been failing every second run for weeks.
<lifeless> wgrant: I didn't realise it was that flaky
<wgrant> It's actually only 1/3. But yeah.
<wgrant> still not great.
<lifeless>                      ->  Bitmap Index Scan on bugtask__distribution_sourcepackage__heat__idx  (cost=0.00..567.17 rows=17172 width=0) (actual time=141.166..141.166 rows=503359 loops=164)
<lifeless> repeated 146 times
<wgrant> Fun.
<lifeless> -> structural subscription scaling fail
<lifeless> 70M rows processed to get 2.5K results
<wgrant> Hmm.
<wgrant> Not sure how best to disable this.
<wgrant> Since it's the only test in the class.
<wgrant> And the only class in the file.
<wgrant> I guess I should rename the file.
<StevenK> Yeah, rename the file
<lifeless> hih
<lifeless> just rename the test
<wgrant> No, then the testrunner bitches that there are no tests in the class.
<wgrant> :(
<lifeless> well
<lifeless> thats a stupid freaking test runner then isn't it
<lifeless> does it support skips ?
<wgrant> I don't think so.
<lifeless> actually
<lifeless> testtools does
<lifeless> just do a skip
<lifeless> self.skip('disabled')
<wgrant> @skip?
<lifeless> see what happens
<wgrant> Heh.
<wgrant> OK.
<lifeless> or that; 6 billion ways to spell it
<wgrant> It appears to succeed, but at least does not fail.
<wgrant> Thanks.
<lifeless> not the plan your doctor ordered:
<lifeless>  Nested Loop  (cost=2.76..17359385328490.43 rows=247357031677542 width=4)
<wgrant> .....
<lifeless> exactly
<StevenK> Is that 247 trillion?
<lifeless> who knows
<StevenK> Looks like it. I don't think we have that many rows in our entire db
<lifeless> structsubs
<lifeless> win
<lifeless> got it down to 25 seconds so far
<wgrant> 247 trillion, yeah. Awesome.
<lifeless> 25 seconds
<lifeless> bah
<lifeless> 15
<lifeless> server team has 146 subscriptions
<lifeless> the live plan is evaluating *each one* with 5 bugtask table index bitmap scans
<lifeless> sorry, 164 subs
<wgrant> Adobe crap safely contained in a VM :)
<wgrant> I like how the AIR download page tells you you'll probably need to turn off antivirus software.
<lifeless> win!
<lifeless> hah, this subscription query is broken anyhow
<wgrant> Oh?
<lifeless> actually a sub limit
<lifeless> can't do series + package
<wgrant> Hm.
<wgrant> ValidPersonOrTeamVocabulary is stupid.
<StevenK> jtv: Did you copy alien-arena-data into oneiric-updates?
<jtv> StevenK: yesterday (though I didn't get to specify a pocket)
<StevenK> jtv: It's component is non-free
<wgrant> Because copies don't do overrides.
<StevenK> Yet
<wgrant> ValidPersonOrTeamVocabulary is pretty nice.
<jtv> StevenK: did I cause a problem?
<StevenK> Yes
<jtv> I'm sorry about that... any way in which it could have been prevented?
<wgrant> It seems to take the 100 most relevant results, then order them by displayname instead of relevance.
<wgrant> How useful.
<StevenK> jtv: By not doing that
<jtv> *And* get my Q/A done?
<StevenK> By undoing it
<StevenK> :-)
 * StevenK re-publishes -updates
<wgrant> Or by copying stuff that's in Debian main.
<wgrant> And not contrib/non-free.
<wgrant> That's the critical bit.
<jtv> Why is it specifically contrib and non-free that are a problem?
<wgrant> Because they're not in Ubuntu.
<StevenK> Because Ubuntu doesn't have those components
<wgrant> The other Debian component is main, which Ubuntu has too.
<lifeless> wgrant: win!
<jtv> Then presumably the copy simply went into main?
<wgrant> jtv: No, because that needs overrides.
<wgrant> Copies don't do overrides.
<jtv> So then what happened instead?
<wgrant> It went into non-free.
<jtv> So it created a component?
<StevenK> And alien-arena is in contrib
<wgrant> it created an invalid publication.
<wgrant> Because that component is not publishable in that archive.
<jtv> I suppose that will be fixed at some point though?
 * StevenK re-publishes -updates AGAIN
<wgrant> lifeless: It also seems to retrieve the top N private teams separately, then merges the two and drops all the FTI info.
<wgrant> That whole function makes my head hurt.
<StevenK> jtv: Copies need overrides, yes. I'm QAing it now
<StevenK> Which this mess didn't help at all
<wgrant> QAing the first step of it.
<jtv> Very sorry about that; I'll coordinate with the others next time I copy a package.
<StevenK> Or pick a package that is in Debian main, not contrib or non-free, like wgrant said
<wgrant> Proprietary software destroys the world, again!
<StevenK> Bwaha
<jtv> StevenK: I don't think that'll help as much: this time it's a matter of figuring out what Debian component it's in (which I guess I'd find out on packages.debian.org?) but once this particular bug is fixed, it'll be something else.
<StevenK> jtv: rmadison is your friend
<jtv> It may say so on youface or whatever your kids use these days, but I'm not aware of the lady in question.
<StevenK> % rmadison -u debian -s sid alien-arena
<StevenK>  alien-arena | 7.51-2 | sid/contrib | source, amd64, armel, hurd-i386, i386, ia64, kfreebsd-amd64, kfreebsd-i386, mips, mipsel, powerpc, s390, sparc
<wgrant> Ah, alpha/hppa are actually dead now?
 * jtv sheds tear for alpha
<wgrant> Or at least on ports.
<StevenK> They've moved
<LPCIBot> Project windmill-devel build #126: STILL FAILING in 1 hr 9 min: https://lpci.wedontsleep.org/job/windmill-devel/126/
<jtv> Hi stub!  Want to try the Ubuntu CafÃ© for going over that patch?
<stub> Urgh.
<stub> Still booting.
<stub> Where is it btw?
<wgrant> The BranchRevision.id doompatch?
<stub> The footgun, yeah.
<wgrant> It looked simple enough, but zomgscary.
<stub> With luck, it will give our toenails a lovely trim.
<StevenK> And not our toes?
<jtv> Ha ha, StevenK, always the optimist
 * jtv relocates
<lifeless> oh man, that was a tough one.
<lifeless> wgrant: if you're interested - https://bugs.launchpad.net/launchpad/+bug/787294/comments/11
<_mup_> Bug #787294: Person:+patches timeouts <timeout> <Launchpad itself:Triaged> < https://launchpad.net/bugs/787294 >
<lifeless> my headache really didn't help - hard to think well about problems with a pounding going on
<adeuring> good morning
<allenap> wgrant, lifeless: We could try increasing the time-out. I didn't change that.
<allenap> Re. the RabbitMQ fixture test.
<wgrant> Hmm.
<allenap> wgrant: The time-out is only 5 seconds right now, which is fairly tight. I think we should put it up to 15 seconds and see what happens.
<mrevell> Guten morgen
<LPCIBot> Project windmill-devel build #127: STILL FAILING in 1 hr 8 min: https://lpci.wedontsleep.org/job/windmill-devel/127/
* gmb changed the topic of #launchpad-dev to: Performance Tuesday! | https://dev.launchpad.net/ | On call reviewer: gmb | https://code.launchpad.net/launchpad-project/+activereviews | Critical bugs:214 - 0:[######=_]:256
<LPCIBot> Project windmill-devel build #128: STILL FAILING in 1 hr 17 min: https://lpci.wedontsleep.org/job/windmill-devel/128/
<lifeless> oh yay
<lifeless> linkin invites sent to lp bugs
<nigelb> heh
<nigelb> how long does a fix take to land from qa-staging to production?
<nigelb> (after being verified)
<bigjools> nigelb: depends how quickly someone presses the button
<nigelb> bigjools: Ah, so just wait some more I guess :)
<bigjools> and what other things need QA
<StevenK> nigelb: Which revision?
<nigelb> StevenK: "Fixed in stable r13094"
<StevenK> Ah, you're stuck behind r13092 which is qa-bad
<StevenK> There was a revert happening at some point
<StevenK> gmb needs to QA r13096
<StevenK> And allenap needs to QA r13101
<nigelb> ah, that's 13108, revert 13092
<StevenK> Right, it likely hasn't hit qastaging yet
<StevenK> Ah
<StevenK> It has
<StevenK> My deployment report was old
<allenap> StevenK: I /think/ I've done that one.
<nigelb> I was looking at http://bazaar.launchpad.net/~launchpad-pqm/launchpad/stable/changes
<StevenK> nigelb: And you need to QA r13106 :-)
<nigelb> StevenK: that landed a few hours back, didn't get time :D
<StevenK> "[testfix][rs=wgrant][rollback=13092] Disable TestRabbitFixture.test_start_check_shutdown. Its days of destroying the test suite are over."
 * StevenK cackles
<nigelb> haha
<StevenK> nigelb: So, to answer your question: "It depends"
<nigelb> Right, so I think I'm one of the people blocking it at this point :)
<StevenK> There is a bunch of QA to do
<nigelb> :)
<nigelb> I guess I have to wait until 13108 to get QA'd because of 13092.
<StevenK> r13108 won't get QA'd, it's a rollback
<StevenK> If you mean up to r13108, that's different.
<jml> lifeless: hi
<nigelb> I meant upto.
<nigelb> Bah, my rev isn't seeming to work.
<StevenK> nigelb: Right, then yes, you do.
<nigelb> How do I know something is in qastaging for sure?
<nigelb> wait, PQM got done with it a minute back :/
<lifeless> jml: hi
<lifeless> nigelb: when its on qastaging the tagger will notice and tag the bug
<bigjools> nigelb: check the revision number that it landed on
<bigjools> compare with the one at the bottom of every page
<nigelb> lifeless: ah, yes. Its tagged.
<nigelb> bigjools: hrm, that seems to indicate its landed. But it isn't working. *whee* QA fail.
<bigjools> nigelb: we only flag qa-bad if it should block rollout
<nigelb> bigjools: Its not doing anything harmful, just needs more fixing.
<nigelb> hrm, okay.  This is really random.
<nigelb> https://blueprints.qastaging.launchpad.net/ubuntu/+spec/test-blueprint-mark/
<nigelb> I don't even know what order its getting sorted :(
<nigelb> Its working "mostly" except for the adicarlo name.  I suspect it has something to do with python's sort function.
<StevenK> You probably should have used .order_by() on a ResultSet?
<nigelb> I used sort because that's what's used in the sprints too.
<spiv> nigelb: Python's sort is entirely sensible, except that the default ordering for objects that don't define how they should be compared is arbitrary (i.e. based on memory address)
<nigelb> Bah, its a problem with using sort(), it clubs all the capitals first and the small letters later.
<jml> lifeless: hello. I'm at a cafe, but am willing to try some sort of voice comms if you're up for it
<lifeless> that sounds great. pick your poison
<lifeless> sky.net?
<spiv> nigelb: Well, that's how str/unicode's sort order is defined :)
<spiv> nigelb: sort(seq, key=unicode.lower)
<spiv> nigelb: or as StevenK says use order_by on the result set
<LPCIBot> Project windmill-db-devel build #313: STILL FAILING in 1 hr 10 min: https://lpci.wedontsleep.org/job/windmill-db-devel/313/
<nigelb> order_by ?
<nigelb> isn't that a query thing?
<jml> lifeless: ok.
<spiv> nigelb: yes.  The data originates in a db query after all.
<nigelb> spiv: yeah, but its a cached property, so we end up having to mutate that list every now and then when someone subscribes newly.
<bigjools> that's probably the most efficient place to do it
<nigelb> https://code.launchpad.net/~nigelbabu/launchpad/90628-spec-sub/+merge/62003
<nigelb> ^^ that's the branch with the merge changes for this one
<jml> lifeless: hello can you hear me speaking?
<jml> lifeless: I can hear you
<jml> lifeless: my mic isn't coming up in sound config, even though it's correctly configured.
<spiv> nigelb: well I guess just add .lower() to the sort key func, e.g. key=lambda sub: sub.person.displayname.lower()
<nigelb> spiv: aha, I shall test that when I get to my computer running LP and propose merge again :)
<nigelb> this means I should add a test case for this.
<spiv> nigelb: I'm so glad you said that :)
<bigjools> gmb: I have a branch waiting for your perusal if you're reviewing please?
<gmb> bigjools: Sure. I'll be free to look in about ten minutes
<bigjools> cheers, no rush
<nigelb> spiv: yay, it works locally. So I need to fix and push them both.
<nigelb> Sorry for breaking it :)
<jml> lifeless: http://blog.launchpad.net/general/a-cream-pie-in-the-face
<gmb> bigjools: r=me
<bigjools> gmb: cheers
<bigjools> pie chart - rofl
 * gmb -> lunch
<wallyworld_> benji: i've checked a mp your way. it's a redo of a recent bug fix you did to fix an issue with the picker widget. the original fix broken some windmill tests. the detail is in the mp. thanks in advance for looking :-)
<benji> wallyworld_: sure, I'll take a look
<wgrant> wallyworld_: Doesn't that make the names ambiguous?
<wallyworld_> wgrant: no. the field names are unique and hence the ids are too. uniqueness has never ever been an issue afaik
<wallyworld_> the real issue in the bug report was the invalid chars
<wgrant> wallyworld_: It looks to me like that just suppresses invalid characters.
<wgrant> If there are two names differing only by invalid characters, they are ambiguous.
<wallyworld_> yes, that's all that's needed
<wallyworld_> as i said, uniqueness has never been an issue, unless i am totally wrong :-)
<wgrant> I don't see how it isn't an issue now...
<wallyworld_> wgrant: i don't think tat's a real issue?
<wallyworld_> we've never had bugs due to uniqueness
<wallyworld_> or lack thereof
<wallyworld_> in node ids for the picker widgets
<wallyworld_> unless i'm missing something
<wgrant> The original change was made because some package name had a . in it.
<wgrant> Which means that package names are used in IDs.
<wallyworld_> it was a "+"
<wgrant> Or that.
<wgrant> One of them
<wgrant> Even so.
<wgrant> You're now string that out.
<wgrant> Which means that a package named 'foo' and a package named 'foo+' will conflict.
<wallyworld_> the package was gtk+
<wallyworld_> will that be an issue in practice?
<wallyworld_> i could replace the "+" with another char
<wgrant> Can we not go for "probably won't break in practice if we don't have an unlucky package combination", please? :)
<wallyworld_> i could get hit by lightning too, but the chances are not worth worring about :-)
<wgrant> Yeah, but this is easily avoidable and *will* break at some point.
<wallyworld_> but anyway, if i replace the invalid chars with a  "-" or something, that should suffice
<wgrant> Can't we urlencode it or something?
<wallyworld_> probably
<wgrant> That's likely to be less ambiguous.
<wgrant> Replacing them all with a constant character is still ambiguous.
<benji> wgrant: my original fix was to urlsafe_b64encode it
<wgrant> RIght.
<wallyworld_> the main point is that for tests, we need to be able to determine the id from the field name
<wgrant> But that's a bit too opaque.
<benji> yep, I see that now
<wgrant> I don't have a problem with it, but it is slightly awkward for tests.
<wallyworld_> more than slightly :-)
<wgrant> The original code was probably written when someone thought along the lines of "pfft, who is going to put a + in a package name"
<wgrant> We should make our code correct.
<wgrant> Not working in most cases that we can conceive.
<benji> I propose we replace the non-ID-safe characters with string equivalents plus the escaping char; so "gtk+" would become "gtk-plus-" and "foo-bar" would become "foo-dash-bar"
<wallyworld_> benji: that sounds reasonable. it won't affect the existing tests because we don't put such chars in there in the first place for testing
<benji> a definate plus ;)
<wallyworld_> and the node ids are still easily discoverable from the field names
<wgrant> wallyworld_: You can't identify the nodes in the tests some other way?
<wallyworld_> not easily
<wgrant> Attempting our own encoding method is somewhat fraught with peril.
<benji> how about field names?  the base64ification was only for IDs
<wallyworld_> and when one views page source, b64 "gibberish" is pretty yucky to look at
<wgrant> You could even reuse the tiny base63-encoding helper.
<wgrant> In the tests.
<wgrant> When generating field names.
<wgrant> That way we have correctness and not too terrible tests.
<wallyworld_> yik, why go to all that bother
<wallyworld_> yuk
<wgrant> Because it's correct and reasonably clean.
<wgrant> Ugly source.
<wallyworld_> i like benji's idea :-)
<wgrant> But cleaner than implementing our own dodgy encoder.
<wgrant> Only [A-Za-z0-9_:.-] are permitted in ids.
<wgrant> So you're going to have to manually encode a lot of stuff.
<wallyworld_> benji: wrt field names, if they're dynamically generated from some data then we may still have an issue there
<wallyworld_> wgrant: we could just replace with the ascii code number then
<benji> wallyworld_: try this one on for size: if the ID has no disallowed characters then we do nothing, if it does we base64 encode it; that way all the tests work and we still cover all the bases
<wallyworld_> that may work :-)
<wallyworld_> benji: it's not just current tests, it's also new ones that need to be easy to write :-)
<wallyworld_> if ^%^@!%^%^@! windmill were not turned off then we wouldn't be having this conversation :-(
<benji> right, but I don't think we'll be doing too many wonky things in tests, so it won't happen very often -- back to your lightning argument ;)
<wallyworld_> actually, i think it's the other way around. in tests, we have simple field names and we need to be able to easily know what the corresponding "show widget" element is called
<wallyworld_> for some tests
<wallyworld_> that need to poke that node
<wallyworld_> benji: i'm tired now so i'll very likely rework the mp tomorrow morning
<benji> wallyworld_: have a good rest, re. "the other way around.": I might have miscomunicated, I intended the tests to never need to encode and only encode in real life when neccesary
<wgrant> wallyworld_: Sorry to pedantic, but I don't think it's a really great idea to, after fixing a bug which was caused by code not handling the whole input domain, replace the good fix with one that is known not to handle the whole input domain, and presume it will be good because users won't do crazy things.
<wallyworld_> benji: yes. i think we're both on the same page
<wgrant> If you are relying on users not doing crazy things, you probably have a bug.
<wgrant> And someone will find it.
<wallyworld_> wgrant: can't argue your points. it's been a good discussion thanks. i'm happy we've converged on a solution that's easily to implement that achieves the goals
<wgrant> Yup.
<benji> wgrant: the idea was to not base64 encode unless there is an invalid character in the ID, that way the tests will be fine and we'll still cover the real-world corner cases
<wgrant> Sounds reasonable.
<henninge> adeuring: Hallo! ;)
<henninge> adeuring: my laptop is blocked branching LP trunk ...
<henninge> adeuring: I still have to install mumble
<adeuring> henninge: ok...
<henninge> adeuring: oh, works
 * henninge tried to use software center but that did not start
<LPCIBot> Project windmill-db-devel build #314: STILL FAILING in 1 hr 9 min: https://lpci.wedontsleep.org/job/windmill-db-devel/314/
<deryck> Morning, all.
<deryck> abentley, adeuring, henninge -- you guys done with standup?
<abentley> deryck: morning, yes we are.
<deryck> ok, cool.
<henninge> Hi deryck!
<deryck> sorry about running late.
<deryck> Thanks for going ahead with it.
<deryck> henninge, bug 785637 is still marked in progress, but the linked branch is merged.  Has it passed buildbot yet?
<henninge> n
<henninge> o
<henninge> deryck: it's in there now
<deryck> o
<deryck> k
<deryck> ;)
<henninge> ;)
<jcsackett> wallyworld: it
<jcsackett> it's sort of shocking to wake up to so many cards moved across kanban. :-P
<jml> :)
<bigjools> I hope they were all pie-critical?
<deryck> abentley, ready when you are.
<abentley> deryck: Coming...
<deryck> np
<jcsackett> bigjools: alas, we're on feature work, which doesn't hit the pie-critical queue.
<jcsackett> or at least, doesn't hit it very often.
<bigjools> a crying shame
<LPCIBot> Project windmill-devel build #129: STILL FAILING in 1 hr 18 min: https://lpci.wedontsleep.org/job/windmill-devel/129/
<sinzui> jcsackett: do you have time to mumble?
<jcsackett> sinzui: i do.
 * jcsackett fires up mumble
<nigelb> sinzui: Well, yesterday's sort is not qa-ok per se.
<nigelb> I need to do a .lower() of the element.
<LPCIBot> Project windmill-db-devel build #315: STILL FAILING in 45 min: https://lpci.wedontsleep.org/job/windmill-db-devel/315/
<sinzui> nigelb: yes, you can do that. qa-ok means it is safe to roll out. It will not break. So you can report an new bug and show me the fix.
<nigelb> sinzui: awesome, I'll do that :)
<sinzui> nigelb: did you see alejandra in the listing?
<nigelb> sinzui: which one?
<sinzui> In my qa, I saw alejandra was the last user listed because she was the only lowercase display name
<nigelb> sinzui: ah, I did a qa with some other starting with lowercase 'a'
<LPCIBot> Project windmill-devel build #130: STILL FAILING in 50 min: https://lpci.wedontsleep.org/job/windmill-devel/130/
<bigjools> it'd be really nice if bug triagers tagged bugs please
<sinzui> benji: I may have stepped in your review of https://code.launchpad.net/~wallyworld/launchpad/better-popup-show-widget-name/+merge/62113 . The branch appears to reimplement FormattersAPI.css_id
<benji> sinzui: no worries; that review was paused after some discussion here about better ways to accomplish the desired result; your contribution looks valuable too
<sinzui> bigjools: +1. I constantly revisit recently triaged bugs
<bigjools> sinzui: me too
<bigjools> to tag them :)
<benji> sinzui: I don't understand how css_id avoids collisions.  What if you had two strings for IDification "foo@bar" and "foo&bar"; would they not both end up as "foo-bar"?
<sinzui> benji: it does not. You need to think, as is done in registry.browser.product.ProductView to create widgets for a listing or projects
<sinzui> Project names are unique so the engineer is responsible for choosing the namespace. Since the id is used in testing and scripts. The engineer should choose a name we can read and want to maintain
<benji> sinzui: I meant even given a namespace prefix you could still have the same ID generated.  E.g., I choose "PREFIX-" as the prefix and there is a project named "foo@bar" and another named "foo&bar" and I want to represent both on the same page, they both will have IDs "PREFIX-foo-bar".
<sinzui> benji: I do not think "foo@bar" and "foo&bar" are real cases in Lp name  (unique id) rules.but I can image we could support transliteration instead of substution
<abentley> gmb: Could you please review https://code.launchpad.net/~abentley/launchpad/handle-concurrent-lint/+merge/62139 and https://code.launchpad.net/~abentley/launchpad/retry-job/+merge/62145 ?
<gary_poster> sinzui, I have a user who is a spammer (all 10 comments are spam).  I marked comments as spam.  Should I deactivate the user next?
<gary_poster> Or is there some other procedure
<sinzui> gary_poster: If the user has zero karma, yes. If the user has karma, you should contact him and explain that we will suspend him if he does  not confirm he has taken control of his computer, browser, and email account.
<gary_poster> sinzui, 0 karma.  Cool, thanks.
<sinzui> gary_poster: https://wiki.canonical.com/Launchpad/DealingWithSpam#Dealing with spam
<sinzui> \o/ an easy one
<gary_poster> bigjools, sinzui: triaging bugs: the only tags I know if that are particularly important are critical bugs that need to be tagged regression, timeout, oops.  Are there other important tags/actions?  If so, is this documented?
<gary_poster> sinzui, I thought we had a page, but should have checked on the internal wiki.  Thanks.
<LPCIBot> Project db-devel build #575: FAILURE in 6 hr 1 min: https://lpci.wedontsleep.org/job/db-devel/575/
<bigjools> gary_poster: I have a bunch of standard soyuz-related ones that I can educate maintenance teams about if you'd like
<bigjools> ideally on a wiki page for sure
<bigjools> hey, we used to have one of those :)
<sinzui> gary_poster: I add tags that describe the objects in play and the kind of issue eg: ppa+email, teams-ui
<sinzui> the tag widget is very good at suggesting
<gary_poster> bigjools...I guess.  CHR docs are already quite long in parts, and I'm encouraging my squad to do things to make things shorter (like Danilo is going to make the translations stuff much easier hopefully)
<bigjools> sinzui: it is indeed
<bigjools> gary_poster: I agree with making things easier - this is why I really recommend the effort of tagging up front as it reduces pain later
<gary_poster> sinzui, bigjools, ok...we can make an effort.  It feels pretty ad hoc
<gary_poster> ok, I'll hold further comments till after we try it a while :-)
<sinzui> gary_poster: it is because the user did not know how to describe the issue into domains and uses
<abentley> gary_poster: does https://dev.launchpad.net/BugTriage help?
<gary_poster> sinzui: ...yeah, that seems to describe my general concern...
<gary_poster> abentley, I don't see the word "tag" on that page, which is what I'm talking about
<abentley> gary_poster: True, but "oops" "regression" and "timeout" are there...
<sinzui> gary_poster: I was working with question emails a few weeks ago. I search for questions+email and located all the bugs related to the code I was in. I fix 4 critical bugs and 7 low bugs. The low bugs would not have been fixed if the bugs were not properly tagged
<gary_poster> abentley, yeah, thanks.  those are the three I'm most comfortable with, and that our squad is working with already.
<gary_poster> sinzui, I'm not questioning value, but process.  But really, let's table this discussion until my squad has given it a try for a while.
<sinzui> gary_poster: remember that you want to tag bugs my team create a disclosure to ensure my team works them. otherwise you get extra work :)
<gary_poster> heh
<sinzui> s/a/as/
<abentley> gary_poster: I've added the word "tagged" to the page :-)
<gary_poster> abentley, lol, cool, thanks
<bac> gmb: time for a wee review?  https://code.launchpad.net/~bac/launchpad/bug-777789-2/+merge/62150
<gmb> bac: Sure
<gmb> bac: r=me
<bac> gmb: thanks
<LPCIBot> Project windmill-devel build #131: STILL FAILING in 1 hr 11 min: https://lpci.wedontsleep.org/job/windmill-devel/131/
<LPCIBot> Project windmill-db-devel build #316: STILL FAILING in 1 hr 16 min: https://lpci.wedontsleep.org/job/windmill-db-devel/316/
<jcsackett> sinzui: http://people.canonical.com/~jc/images/lp-id-screenshots/
<jcsackett> i'm working on something better on the profile page. the current inline versions are terrible.
<jcsackett> but i realized it was getting on towards noon so i thought i'd ping you with the versions i have.
<jcsackett> sinzui: after seeing them rendered it, doing it with the "Launchpad Id:" thing looks terrible.
<sinzui> jcsackett:  would the example be clearer if there were two comments from two different Jonathans?
<jcsackett> sinzui: on the pickers shots?
<sinzui> and comments. I have seen two Toms posting messages in a bug conversation
<sinzui> I think we should not submit the Launchpad-Id example. It really is more annoying than explict
<jcsackett> sinzui: sure, i can set those up. and i'll just kill the Launchpad-Id, as i concur. it's just terrible.
<sinzui> jcsackett: and the picker examples were be more convincing of the problem we are solving if the users had hidden email addresses
<jcsackett> sinzui: dig.
<sinzui> jcsackett: I had not thought to put the Lp-id in the profile page title. I think users may expect that though. I look forward to user feedback
<jcsackett> sinzui: it seems we should probably standardize where we can if we go down this road.
<sinzui> jcsackett: that is the intent of this  excercise
 * sinzui is reporting a bug about IRC nicks at this moment
<henninge> gmb: Hi! Here is an easy one to send you home with .... ;)
<henninge> https://code.launchpad.net/~henninge/launchpad/bug-683406-xss-ppa/+merge/62158
<gmb> henninge: Righto!
<jcsackett> sinzui: updated http://people.canonical.com/~jc/images/lp-id-screenshots/
<sinzui> jcsackett: that looks good to test
<jcsackett> sinzui: cool. email the link to mrevell (who has now been pinged, if he's listening...)
<jcsackett> >
 * jcsackett fails at typing.
<jcsackett> just assume that last bit was a question. :-P
<mrevell> um
 * mrevell reads up a bit
<sinzui> mrevell: we want to show Launchpad-id in comments and in the person picker so user are not confused by similar user display names. We do not know how to represent the Launchpad Id
<mrevell> Ahhh
<mrevell> cool
<mrevell> Wow, that rocks.
<jcsackett> i can email you a zip of these images if you would prefer, mrevell.
<mrevell> jcsackett, Nah, the link on people.c.c is fine. Thanks. Sorry I was a bit slow off the mark.
<jcsackett> mrevell: all good. :-)
<jcsackett> sinzui: you wanted to mumble a bit again post-mockups, yeah?
* gmb changed the topic of #launchpad-dev to: Performance Tuesday! | https://dev.launchpad.net/ | On call reviewer:  - | https://code.launchpad.net/launchpad-project/+activereviews | Critical bugs:214 - 0:[######=_]:256
<LPCIBot> Project windmill-db-devel build #317: STILL FAILING in 51 min: https://lpci.wedontsleep.org/job/windmill-db-devel/317/
<jcsackett> sinzui: ready to chat again when you are.
<sinzui> jcsackett: I was just going to try to configure sip
<jcsackett> sinzui: cool. i think i have sip setup. at least, i am told that i do.
<sinzui> jcsackett: I have credentials I tested 4 years ago
<sinzui> jcsackett: http://www.youtube.com/watch?v=r711rZ4M5K0 . The fish joke stars about 1 minute 25 into the scene, but the last part is missing. Mike returns empty handed and asks "What's this fish doing in my bed", and the rest ask, "What fish?"
<jcsackett> that's excellent.
<deryck> lifeless, ping
<LPCIBot> Project windmill-db-devel build #318: STILL FAILING in 1 hr 11 min: https://lpci.wedontsleep.org/job/windmill-db-devel/318/
<bac> hi sinzui, do you know anything about bug 490767?
<_mup_> Bug #490767: Cannot post a comment on an existing bug with Konqueror. <comments> <javascript> <lp-bugs> <Launchpad itself:Triaged> < https://launchpad.net/bugs/490767 >
<bac> lifeless mentions an email thread but i cannot find it
<sinzui> bac I think it was escalated without being confirmed it is still a problem. Check the age of the bug against the fact Konqueror got a major upgrade.
<sinzui> There are no dupes or other affected users either
<bac> sinzui: right, i cannot reproduce it with current konq in lucid or natty
 * bac closes
<sinzui> Konqueror now uses webkit
<bac> i'm just curious as to what new info caused it to be escalated
<sinzui> bigjool, thumper, and flacoste are also kubuntu users
<sinzui> lifeless reconnised the description as a script fault
<sinzui> s/reconnised/recognised/
<sinzui> I would close it, or at the very least mark it incomplete
<sinzui> bac: this bug may also be fixed for the same reasons: bug 457784
<_mup_> Bug #457784: When bugs reporting tries to report bug in Kubuntu,  using Konqueror, "I am affected by this bug" popup does not works <filebug> <javascript> <lp-bugs> <ui> <Launchpad itself:Triaged> <kde4libs (Ubuntu):Invalid> < https://launchpad.net/bugs/457784 >
<deryck> sinzui, your team is doing the disclosure story now, right?
<sinzui> bac: and If you are using konqueror right now, you can check bug 546813 and close it if it was fixed by one of our rounds to ensure empty anchors are not used
<_mup_> Bug #546813: Bugtask table "Affects" edit icon missing in Konqueror <css> <lp-bugs> <trivial> <Launchpad itself:Triaged> < https://launchpad.net/bugs/546813 >
<sinzui> deryck: yes
<deryck> ok, thanks
<bac> sinzui: will do
<deryck> sinzui, I think bug 419531 should be downgraded to high.  I'm commenting as such in the bug now.
<_mup_> Bug #419531: project name picker search / vocabulary is hard/impossible to use (too many results on exact searches) <disclosure> <regression> <vocabulary> <Launchpad itself:Triaged> < https://launchpad.net/bugs/419531 >
<deryck> can it really be a regression if it was opened two years ago? ;)
<sinzui> bac: bug 255294 is surprising if it is true.
<_mup_> Bug #255294: Links with target=help don't render on Konqueror <javascript> <lp-web> <Launchpad itself:Triaged> < https://launchpad.net/bugs/255294 >
<flacoste> sinzui: i'm a Ubuntu user who uses a lot of KDE apps (but I'm no longer a Kubuntu user)
<flacoste> and i don't use konqueror
<flacoste> bac: i'm not sure Critical was the right priority for this
<flacoste> bac: konqueror isn't a supported browser
<flacoste> JS errors are critical, sure
<sinzui> deryck: I agree. The project picker is better than it was tree years ago. It is not a regression. It just continues to suck
<bac> flacoste: good to know
<flacoste> but if it only affects a low-following browser, i'm not sure Critical is the right priority there
<flacoste> we often discussed that we don't have the resources to test/support all browsers
<deryck> sinzui, exactly!  And while sucking sucks, it doesn't make it critical. :-)
<sinzui> The timeout will stop the user from selecting a project in some forms unless you do the trick in the work around
<flacoste> konqueror is 0.22% of our audience
<sinzui> deryck: the timeout makes it critical. disclosure makes it high. My team will officially start on it in 2 weeks, but I believe one or two members of the squad will address the performance issue next week
<flacoste> IE is 5.4%
<flacoste> and Opera even beats it at 3.20%
<deryck> flacoste, I also think we should downgrade bug 490767 to HIGH or LOW since we don't support Konq.
<_mup_> Bug #490767: Cannot post a comment on an existing bug with Konqueror. <comments> <javascript> <lp-bugs> <Launchpad itself:Invalid> < https://launchpad.net/bugs/490767 >
<flacoste> deryck: it is marked invalid
<deryck> oh, heh
<deryck> flacoste, sorry, I had it open to ask about and never refreshed. :-)
<deryck> I see bac is working over all the konq bugs.
<deryck> I'm just trolling for new work and getting bugs off the list however I can! :-)
<sinzui> bac also closed an IRC nick bug today, but landed the work last August. bac rocks.
<bac> deryck: apparently if you launch konq you get to touch all the bugs!  :)
 * flacoste is amazed that 25% of our visits come from Windows client
<bac> sinzui: what IRC nick bug?
<deryck> sinzui, should we file a new bug about the timeout as critical?  it really is two wildly, separate issues.
<flacoste> beats Mac users who are 4.25%
<bac> flacoste: and most of those work on launchpad
<bac> s/those/them
<sinzui> flacoste: Lp has spectacular Google fu
<sinzui> bac: the bug was about contradictory instructions when adding an irc nick, and the presentation of that nick on the profile page
<sinzui> We no longer ask for irc:// nor present it
<sinzui> deryck: The picker timeout is ultimately caused by a missing ranking, and an improper use of fti. The impelmentation causes bad UI and timeouts. So my squad will fix both
<deryck> sinzui, gotcha.  thanks for supporting my downgrading it. :-)
<sinzui> This action will also permit my squad to contribute to the critical count
<abentley> deryck: boo.  I hoped when you touched bug 419531 that you were going to fix it. :-(
<_mup_> Bug #419531: project name picker search / vocabulary is hard/impossible to use (too many results on exact searches) <disclosure> <regression> <vocabulary> <Launchpad itself:Triaged> < https://launchpad.net/bugs/419531 >
<deryck> abentley, not me, sorry.  but sinzui's team is.  So some hope there, no?
<abentley> deryck: Yes, some hope.
<lifeless> deryck: hi
<deryck> lifeless, hi.  See my scrollback discussion with sinzui or look at what I said/did on bug 419531.
<_mup_> Bug #419531: project name picker search / vocabulary is hard/impossible to use (too many results on exact searches) <disclosure> <regression> <vocabulary> <Launchpad itself:Triaged> < https://launchpad.net/bugs/419531 >
<deryck> lifeless, just wanted to argue that bug should be high, but I just changed it.  and can certainly discuss if you disagree.
<flacoste> lifeless: sorry to have stood you up yesterday, but it was a public holiday in Canada
<lifeless> deryck: its a regression
<lifeless> flacoste: ah! no worries
<deryck> lifeless, for some value of the word "regression," no? :)
<flacoste> lifeless: in what way it's a regression?
<flacoste> i don't think this ever worked
<flacoste> the regression tag was added 2 years after the initial bug
<deryck> flacoste, I think he means in a world before pickers you didn't get this error.
<lifeless> flacoste: the UI used to be a text field plus 'choose'
<lifeless> flacoste: in that world you could select 'launchpad' as the project
<lifeless> flacoste: now the UI is a popup
<flacoste> ah right
<lifeless> flacoste: and its impossible to select launchpad as the project
<flacoste> i forgot about that
<lifeless> how is that *not* a regression
<lifeless> deryck: I don't see it needing feature level work: oh theres tonnes that could be done but special casing an exact match is < 4 hours work
<lifeless> deryck: anyhow, the current rule we have is 'tagged regression -> critical'
<deryck> lifeless, maybe "needing feature level work" is the wrong phrase.  it will be fixed by sinzui's team doing feature work.  or it is part of the disclosure feature anyway.  or some other phrasing.
<lifeless> deryck: so lets finish discussing whether its a regression or not.
<flacoste> deryck: that doesn't lower it's priority nor remove the regression tag :-)
<deryck> right
<deryck> flacoste, right.  but I don't consider it a regression, that is my main point.
<lifeless> Do you agree that something users could do before, and can't now?
<deryck> but I knew regression purists would disagree and was looking for more ammo ;)
<flacoste> deryck: it's been a long-standing regerssion
<lifeless> deryck: And further, do you agree that it wasn't an *intentional* restriction in functionality?
<flacoste> because lifeless is right, the regression doesn't come from the pop-up itself, that vocabulary has always been broken
<flacoste> but before that, the picker was optional
<flacoste> now it's not
<flacoste> before the picker was used to pre-fill a text field
<flacoste> now the text-field is gone
<deryck> sure, but that's by design.
<deryck> that's all I'm saying.  it's not the same kind of regression.
<flacoste> it is
<deryck> it's a two year old bug
<flacoste> the bug report is actually too vague
<deryck> no, I understand the problem perfectly.
<flacoste> the regression is that on some specific pages, you could enter a team/project name
<flacoste> in case the picker failed you
<flacoste> you can't anymore
<deryck> I know, I remember :-)
<flacoste> it's really a different bug
<lifeless> well
<flacoste> so yes, that specific bug report isn't a regression per se
<lifeless> ok, now I'm lost
<flacoste> but there is another one (unfiled) that is
<lifeless> flacoste: step me through this?
<deryck> if it's this complicated, it can't be a regression.  or at least it doesn't matter.
<deryck> :-)
<lifeless> deryck: why do you want this to not be a regression? What does it help you with ?
<lifeless> flacoste: also, would you like to do a call today?
<deryck> lifeless, I fear the precedent of js changes behavior, you now can't do what you did before, and it's a critical bug.
<flacoste> lifeless: i have a call with gary in 10 minutes, i could do after that, but we could have more time tomorrow as i don't have any in the afternoon
<deryck> I just don't like the word regression for this.  maybe it's just me
* lifeless changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer:  - | https://code.launchpad.net/launchpad-project/+activereviews | Critical bugs:214 - 0:[######=_]:256
<flacoste> lifeless: the picker was actually always broken, that's not a regression
<lifeless> deryck: so I think if we deliberately change the UI to be nicer that the old UI going is *not* a bug in any regard.
<flacoste> the regression is that in some case, we remove the possibility of not using the picker
<lifeless> deryck: but if an existing bug prevents using the nicer UI to accomplish stuff, we've essentially removed a workaround and have to consider the combination of events a regression
<flacoste> but fyi, on forms, the picker is still optional
<flacoste> it's on inline-edit forms (new design) that it's impossible to use
<deryck> flacoste, but we're not going to fix it so you can get an input element.  we're committed to removing the input element as an option.
<flacoste> deryck: agreed
<flacoste> but i don't think we removed any form yet
<deryck> so the picker is bugged, not regressed.
<flacoste> yes
<lifeless> deryck: thats fixable in two ways: restore the old UI (fugly) or fix the new UI to permit the old workflow
<flacoste> we have been constant there :-)
<flacoste> lifeless: actually, i'm pretty sure that all the boomerang pages still exist
<flacoste> with the old way of selecting things (text field + picker)
<lifeless> flacoste: even in shiny shiny like recipes?
<flacoste> they might not be linked to though
<lifeless> flacoste: right
<deryck> lifeless, but we're not going to fix the ui to allow the old workflow, here.  and going back is not an option for these js workflows.  and you can still right-click your way to a real page.
<flacoste> if it's shinny new, it's not a regression ;-)
<deryck> lifeless, therefore, I don't see us calling this a regression.
<lifeless> so why do I want to call this a regression
<lifeless> because I can't see our users considering it anything else
<deryck> you have a pedantic streak in you ;) :)
<flacoste> deryck: i think he's striving for clarity :-)
<deryck> I meant that in fun, not insulting way
<lifeless> if we change something and the result doesn't work, *and* we didn't intend to make things not work
<lifeless> then I think users would say that we've regressed in functionality
<deryck> maybe so.  but to me that's not the issue.  not what they would *say*.  I do care that they're blocked.  and I do think this should have been fixed well before now.
<lifeless> if we explicitly said 'we don't care about folk searching for exact matches', it would be entirely different
<lifeless> deryck: so if this is tagged regression and we had the regression-jumps-queue policy two years ago, it would have been fixed well before now.
<deryck> sure.
<lifeless> deryck: are you worried that we have other regressions sitting in our 6K database?
<deryck> but even today, I wouldn't call this a regression
<deryck> partly, yes.  and partly, I fear the slippery slope.  especially when you start saying "a user would say this is a regression, so it is"
<deryck> it's really that ^^ mostly
<deryck> in that case, lp is nothing but one big regression, and for a certain class of user that is completely true and valid.
<bac> sinzui: regarding bug 255294, the help pop-up does appear now and is readable, which is good.
<_mup_> Bug #255294: Links with target=help don't render on Konqueror <javascript> <lp-web> <Launchpad itself:Triaged> < https://launchpad.net/bugs/255294 >
<lifeless> I don't think thats true
<lifeless> deryck: we can always make an explicit choice when changing something
<lifeless> deryck: but unexpected consequences *are* regressions I think; we can say 'actually thats a consequence we like, WontFix'
<bac> sinzui: unfortunately, most of the sprites do not appear!
<allenap> lifeless: Hi. By any (remote) chance do you have some time in the next 30 minutes to chat about Rabbit? If not, some time tomorrow or later in the week?
<lifeless> deryck: or 'the use case didn't work before it just looked like it did', and mark it low
<lifeless> allenap: sure do
<bac> sinzui: so on a project page, the help icon is not shown but if you know where it *should* be you can make the help pop-up pop.
<sinzui> bac: okay, The real defect bug 521219 and that is really Lp making bad markup. It affects web kit to some degree too
<_mup_> Bug #521219: Konqueror does not display sprite for empty element <css> <lp-web> <trivial> <Launchpad itself:Triaged> < https://launchpad.net/bugs/521219 >
<lifeless> deryck: what I don't understand is how 'could do X -> cannot do Y' can be not-a-regression
<deryck> lifeless, but every bug is an unexpected consequence.
<lifeless> deryck: I understand that at no point did we revert out a patch
<allenap> lifeless: Awesome :) Mumble, Skype...?
<lifeless> allenap: in ~ 15 minutes, skype.
<sinzui> bac: oh, update the bug then. The issue may also be a sprite on empty markup
<allenap> lifeless: Cool, thanks.
<lifeless> deryck: yes, but not all unexpected consequences prevent doing things
<bac> sinzui: ok.  i'll close 255294
<LPCIBot> Project windmill-db-devel build #319: STILL FAILING in 46 min: https://lpci.wedontsleep.org/job/windmill-db-devel/319/
<deryck> lifeless, I guess I don't see users being prevented here.  you can click through to a real form if you want in every case.
<deryck> it's a search or picker bug.  a new bug.  with unintend consequences.  the ui changed.  it didn't regress.
<lifeless> deryck: I assert that without user testing its impossible to agree or not agree
<deryck> fair enough.
<deryck> lifeless, may I approach this another way?
<deryck> do you have the time?
<lifeless> of course
<lifeless> this is kindof my job :)
<deryck> maybe a better way is to say, "I don't think this is a queue jumping bug"
<deryck> even if it happened today.
<deryck> it shouldn't sit around for two years, and it shouldn't be released with crappy basic search not working, but I just don't think it's enough to be critical.
<lifeless> mmm
<lifeless> so lets say then that we had no criticals
<lifeless> that is, that while we have a huge number of bugs, every use case that we have implemented is working
<lifeless> things we haven't implemented are working, but thats fine we'll get to them
<lifeless> now, purple squad come along and ajaxify the PPA package copy screen
<lifeless> there is a bug in their js and copies from security ppas to the main archive security pocket don't work
<sinzui> deryck: I have been saying that for many years. we cannot have more criticals or highs then we can plan to fix without interruption. The fact that we let failed to be honest about the importance of bugs, and worked on other bugs put us into this mess
<lifeless> and they removed all visible links to the non-ajax UI
<lifeless> deryck: when its released we find out from the ubuntu security team immediately when they go 'zomg cannot unembargo xyz disclosure'
<deryck> the use of "security" here makes this more important than current issue, I think.
<lifeless> deryck: what would we do ?
<lifeless> deryck: thats deliberate
<deryck> I think it's critical because of "security" not because of removing old ui.
<deryck> and it's not a regression
<lifeless> we have existing bugs affecting the ubuntu security team that are only high.
<lifeless> bugs where they cannot do things
<lifeless> [this bit isn't fictional, its real]
<deryck> ok, so maybe I'm reading too much into the word "security" and picturing ubuntu cannot make a security update.
<lifeless> deryck: that is the scenario
<lifeless> deryck: something they could do and now they can't, and its something very important
<deryck> but I'd call it critical for that reason.  the impact.  I don't consider removing a web 1.0 link in favor of js a ui regression.
<lifeless> deryck: I think we'd rollback the UI via its feature flag until fixed
<deryck> sure
<deryck> agreed
<lifeless> deryck: but the bug does not say 'put the 1.0 link back'
<deryck> right
<deryck> but there is no way this picker search bug is near the impact of that security bug
<lifeless> deryck: the bug says 'trying to do X, which I could before, and I get Y (an error)'
<deryck> no, "can't make an Ubuntu security udpate" vs. "can't search for the project I want" are worlds apart in impact.
<lifeless> deryck: so I'm arguing that things of that pattern, unless we wanted that result, are regressions.
<lifeless> our users, the people we write Launchpad for, don't care about the implementation choices of our system, they care about the things they can do with it.
<lifeless> any discussion about regression that talks implementation rather than doable-things is (IMO) missing the point
<deryck> sure, we're agreed about that.  I think in practice this naming of things -- "regression" or not -- is really not that important to our users.
<lifeless> I think its vitally important to our users because its a queue jumping mechanism (for good or bad)
<deryck> sure, I think I'm talking about doable things.
<lifeless> deryck: but you're not: you're talking about implementation - about replacing the widgets we used
<deryck> in order to try to define a regression.
<deryck> how about this....
<deryck> if we change the UI and introduce a bug, that is not a regression to me.
<deryck> no matter what you could do before.
<lifeless> so for me its clearly conditional.
<lifeless> the conditions being:
<lifeless>  - you could do X before
<lifeless>  - we didn't intend to disable X
<deryck> but we did!
<lifeless>  - with a grey are for X's that were only accidentally possible in the first place
<lifeless> deryck: we wanted to stop people selecting launchpad when asked to choose a project?
<deryck> no
<lifeless> then clearly we didn't intend to disable X
<deryck> so how are people assigning a bug to launchpad today?
<lifeless> I don't know :). Certainly not through the 'normal' UI
<deryck> I would describe this as: we introduce a new ui, search is broken for that ui
<deryck> people can still right-click their way to the old ui and enter names directly.  it's not ideal, but you can do it that way if you want.
<deryck> that's why I speak of impact.
<lifeless> but do they know that?
<deryck> not every Launchpad user, no.  of course not.  but I think we have fairly web savy users.  clearly they put up with a lot from us so far. :-)
<deryck> this is not to excuse the current behavior.
<deryck> I just fail to see how introducing a new ui with bugs is a regression.
<deryck> I don't think we'll ever agree about this.
<lifeless> I'd say lets talkin Dublin
<lifeless> but I won't be there
<deryck> yeah, and I don't particularly enjoy talking about things where we aren't likely to get much consensus.
<lifeless> I intensely dislike the 'introduce new thing and its bugs get carte blanche' slope
<deryck> sure, and I don't intend to argue that.
<lifeless> I totally get that a new thing may be worse in some ways than an old thing
<lifeless> thats inevitable
<deryck> I would prefer we never released this widget, and I hope that the current review process with jml's team makes this go away.
<lifeless> deryck: right! its essentially tech debt from a prior project that didn't get to the standard it needed to be consumer ready
<deryck> but bugs are inevitable.  and I think we dealing with a new UI, it's easier to judge impact, then the semantics of whether or not it's clearly a regression.
<deryck> lifeless, yes, agreed.
<lifeless> all our critical are either stakeholder escalations, things changing on us (e.g. browser support ones) or things we've done to ourselves potentially far in the past
<deryck> sure.
<deryck> I don't see what that has to do with whether or not this sort of bug is a regression and therefore critical.
<lifeless> in terms of judging impact
<lifeless> I worry
<lifeless> I think if we want an impact-based assessment for queue jumping, we should remove regression as a queue jumper - to be honest about how we assess things
<deryck> And I would be ok with that. :-)
<deryck> I think "regression" is an attempt to had objective assessment to something that always has to be subjectively assessed.
<deryck> I think regressions are seldom clear cut for a web app.
<deryck> we don't do releases after all
<lifeless> anyhow, the old UI is broken these days too
<lifeless> go here
<lifeless> https://bugs.qastaging.launchpad.net/launchpad/+bug/419531/+editstatus
<_mup_> Bug #419531: project name picker search / vocabulary is hard/impossible to use (too many results on exact searches) <disclosure> <regression> <vocabulary> <Launchpad itself:Triaged> < https://launchpad.net/bugs/419531 >
<lifeless> type bazaar into the input box for project
<deryck> timeout and oops is easy to objectively assess.  functionality for a web app is not.
<lifeless> click on save changes
<lifeless> 'There are 2 errors in the data you entered. Please fix them and try again.
<lifeless> '
<lifeless> 'Enter a project name'
<lifeless> with bazaar sitting there in plain text
<deryck> hmmm, I didn't get that error.
<allenap> lifeless: I see you're still busy. Would you have time in the morning, around 8pm your time?
<deryck> but I know the one you speak of
<lifeless> allenap: no, because 6am phone call for TL meeting
<deryck> allenap, sorry I sidelined lifeless from you.
<lifeless> allenap: lets call now
<allenap> lifeless: Okay.
<allenap> Ring when you're ready.
<allenap> deryck: No worries :) He's in demand!
<lifeless> deryck: I'm going to put this back to critical, because the OLD UI has been broken too; I'm not trying to close off the conversation - but I need to given allen a hand
<deryck> lifeless, thanks for the chat.  sorry to be so stubborn on this.  but well, I just am on ui judgements. :-)
<lifeless> deryck: strong opinions are good - we learn from such things
<deryck> lifeless, while I strongly disagree that this is critical, I respect that you're the TA and can make that call. :-)
<lifeless> allenap: whats your skype id ?
<allenap> lifeless: gavinpanella
<lifeless> ah, I see it
<lifeless> allenap: https://dev.launchpad.net/ArchitectureGuide/Services
<thumper> sinzui: you should drop me off the kubuntu users, and add wallyworld_ :-)
<sinzui> I think we might want to survey ourselves at the next thunderdome. OSes, distros, browsers, editors
<sinzui> The non vim/emacs can sit at the children's table with me
<LPCIBot> Yippie, build fixed!
<LPCIBot> Project db-devel build #576: FIXED in 5 hr 24 min: https://lpci.wedontsleep.org/job/db-devel/576/
<Ursinha> OOPS-1970CG450
<jcsackett> sinzui: how would your survey handle those of us who regularly use two or more things in a given category?
<sinzui> one moment
<sinzui> jcsackett: this is what I collected from staging: http://pastebin.ubuntu.com/612449/
<jcsackett> huh. i'm surprised we don't have more people with two irc nicks, honestly.
<sinzui> I may hunt down the person with 24 nicks
<deryck> 24!  wow.
<deryck> they're not domain names
<jcsackett> it's possible it's someone registering all the variations of their nicks. e.g. jcsackett, jcsackett-afk, jcsackett|afk, and so on.
<sinzui> jcsackett: I think freenode is the secret here. Lp/ubuntu/debian  has a long history with one network. If I am use one of those, I may only need to list that one
<jcsackett> still crazy, mind you.
<jcsackett> sinzui: that makes sense.
<sinzui> jcsackett: Lp has a small issue in that we collection irc nick as a contact address, so listing just the freenode is correct.
<sinzui> I do not list the rizon net since that is for manga for example
<jcsackett> sinzui: yeah, i have several networks not listed here, b/c they're not contextually needed.
<jcsackett> including rizon, come to think of it, though i don't use that one much.
<sinzui> jcsackett: deryck: https://launchpad.net/~ionutjula
<sinzui> I wish dot plan would make a comeback
<jcsackett> wow. it really is mostly separate networks.
<sinzui> This looks like a misconception that Lp is a geek dating site
<sinzui> I might get this: http://www.redbubble.com/people/zerobriant/art/7186582-omg-they-killed-rory-posters
<mwhudson> holy doodoo what happened to all the code imports overnight?
<sinzui> jcsackett: I really think we should just drop 'network' from the formatting. just use "pting on irc.freenode.net".
<jcsackett> sinzui: that seems reasonable to me. the network is very clearly implied.
<jcsackett> and only people who would understand that are going to care about irc nicks as ID anyway.
<lifeless> flacoste: hi
<lifeless> flacoste: ping me when you are done
<lifeless> ?
<sinzui> jcsackett: I agree. That is the salient point
<lifeless> sinzui: irc nick isn't usable as a trustable address
<lifeless> sinzui: we can't validate them
<lifeless> sinzui: if this is about the trustable picker subproject
<sinzui> lifeless: we know, but users check multiple data points when search for a user
<jcsackett> lifeless: it's not remotely the only thing used.
<jcsackett> but people search on ircnicks in the picker.
<flacoste> lifeless: done
<lifeless> jcsackett: sinzui: its not got a UNIQUE on it
<lifeless> you might consider searching on it but not showing it. I dunno :)
<sinzui> No WAY
<sinzui> that is why users thing we are stupid
<lifeless> ah, ok
<sinzui> We allow you to search for information then not show why it matched
<jcsackett> lifeless: part of the reason people don't trust the picker is b/c they search on "sinzui" and get this other guy and don't know why he came up.
<lifeless> we could show what matched (irc/name/fulltext/email) without showing that data
<lifeless> I don't mean to interfere here :) just riffing
<jcsackett> we're at the riffing stage, so this is fine. :-)
<jcsackett> well, partially riffing.
<sinzui> We have an open bug about IPersonSet.findPerson() not supporting nick. That method has diverged from the vocabulary. I think we should consider unifying them. There will be less code to audit
<jcsackett> works for me.
<lifeless> flacoste: http://queue.acm.org/detail.cfm?id=1394128
<lifeless> 'BASE is diametrically opposed to ACID. Where ACID is pessimistic and forces consistency at the end of every operation, BASE is optimistic and accepts that the database consistency will be in a state of flux.'
<sinzui> wgrant: wallyworld_,  I will be 15 minutes late to the stand up meeting
<wallyworld_> ok
#launchpad-dev 2011-05-25
<sinzui> wgrant, mumble?
<LPCIBot> Project windmill-db-devel build #320: STILL FAILING in 1 hr 9 min: https://lpci.wedontsleep.org/job/windmill-db-devel/320/
<abentley> lifeless: re your service-based Launchpad idea, it would be really nice to eliminate branch scanning and just have a service to query the branches.
 * wallyworld_ needs caffeine stat!
 * thumper passes wallyworld_ the IV
<lifeless> abentley: In principle yes
<lifeless> abentley: I wonder if bzr is quite fast enough yet
<lifeless> abentley: I think there are some things we'd want to have caches for - e.g. 'show me branches that have new work' might want a fast graph query
<mwhudson> a nice thing about a SOA is that you can make that change independent of more or less everything else
<lifeless> yes indeed!
<LPCIBot> Project windmill-devel build #132: STILL FAILING in 1 hr 10 min: https://lpci.wedontsleep.org/job/windmill-devel/132/
<abentley> lifeless: I think that it probably is fast enough, considering there's no invocation time.  Fast graph traversal is a service you were already going to provide, right?
<abentley> lifeless: by invocation, I mean bzrlib import cost.
<lifeless> abentley: right; I think it depends on the particular bit of bzrlib we're talking about
<lifeless> abentley: things like extracting a file can still do substantial IO, grabbing a revision should be pretty quick
<lifeless> abentley: and it will depend on what we're doing whether a particular thing works / doesn't.
<lifeless> abentley: where it will, I think using the branches directly is a good idea.
<abentley> lifeless: The data we currently retrieve from scans doesn't include files, though.  It's mostly revision ids and a revision graph.
<lifeless> abentley: given the complexity of bzrlib, memory footprint etc, I'd be inclined to have a webservice - say loggerhead :) - providing this, and it be backed by the raw branches
<abentley> lifeless: Yes, this is what I meant by "a service to query the branches".  I didn't mean using the bzr protocol.
<lifeless> abentley: we may still need scanning to populate a graph cache for graph queries, but it could stop being relevant for viewing the recent commits in the web UI
<lifeless> abentley: cool
<lifeless> abentley: did you know loggerhead has a json API already?
<abentley> lifeless: You did mention it before, but I hadn't thought about it.
<wgrant> How complete is it?
<lifeless> topically, gustavo wants an API to query branch revision tips for principia
<abentley> lifeless: providing that service via loggerhead makes sense, *iff* we have a well-behaved, performant loggerhead.
<lifeless> wants a collection of (branch url, revid)
<lifeless> wgrant: its minimal; easy to extend though
<wgrant> True.
<lifeless> wgrant: we'd need to add something for private branch lookups if appservers do them
<wgrant> Hm.
<wgrant> Looks like staging updates are working fine, but the code isn't being synced beforehand?
<lifeless> scripts/update-bzr-version-info.sh
<lifeless> Creating bzr-version-info.py at revno 10574
<lifeless> I bet its the config-manager change on carob
<lifeless> spm!
<wgrant> successful-updates.txt shows it updated to 10574 for the second time several hours after a new rev was blessed.
<wgrant> So I'm pretty sure that's it.
<wgrant> sinzui: Sorry I missed the call this morning.
<wgrant> sinzui: Did you make any progress on the picker issue?
<wgrant> I think we basically need to completely rethink it before we can make any progress.
<lifeless> wgrant: we had nothing blessed for 5 days ?
<wgrant> lifeless: 10578 was only bleesed May 22 23:32
<lifeless> wgrant: which is still 3 days ago
<wgrant> Sure. We've had stuff blessed since.
<wgrant> But potentially only a single update has failed to update the code.
<wgrant> Since the 2011:05:23 13:21 update could have started before 23:40
<lifeless> the update on the 24th
<lifeless> saw rev 10574
<lifeless> on sourcherry
<wgrant> Right.
<wgrant> And that's the only failed update we have evidence of.
<lifeless> cody-somerville: ping
<lifeless> bug 787765 - can you tell me the figure you see under '79 queries/external actions issued in 0.85 seconds' on the page that was/is failing ?
<_mup_> Bug #787765: ProductSeries:+index timeouts <oem-services> <timeout> <Launchpad itself:Triaged> < https://launchpad.net/bugs/787765 >
<lifeless> cody-somerville: and can you reproduce it on qastaging
<cody-somerville> lifeless, under?
<lifeless> cody-somerville: what that text shows for you :)
<cody-somerville> ah
<cody-somerville> '4 queries/external actions issued in 0.14 seconds'
<lifeless> cody-somerville: timeout though, right ?
<cody-somerville> lifeless, yup
<cody-somerville> lifeless, And it does load on qastaging.
<lifeless> cody-somerville: can you get a profile on qastaging ?
<cody-somerville> oh wait, my bad. Was loading wrong milestone on qastaging.
<cody-somerville> It timesout on qastaging as well
<lifeless> cody-somerville: can you add me as a driver on qastaging?
<cody-somerville> lifeless, Sure. FYI, the project page is timing out on qastaging.launchpad.net as well but not every time.
<cody-somerville> (https://lp-oops.canonical.com/oops.py/?oopsid=OOPS-1971QASTAGING19)
<cody-somerville> lifeless, Done. Added 'lifeless' to oem-solutions-releng team on qastaging.
<lifeless> 16            0      7.5976      0.1703   lp.bugs.browser.structuralsubscription:431(expose_user_administered_teams_to_js
<lifeless> cody-somerville: you're in lots of teams aren't you ;)
* StevenK changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: StevenK | https://code.launchpad.net/launchpad-project/+activereviews | Critical bugs:214 - 0:[######=_]:256
<cody-somerville> lifeless, 225
<cody-somerville> (indirect + direct)
<lifeless> do you know how many you have admin access over ?
<cody-somerville> lifeless, Administrator of 15, Owner of 17.
<cody-somerville> (cut, grep, and wc for the win!)
<lifeless> heh
<lifeless> that probably fails the indirect test
<lifeless> anyhow
<cody-somerville> ah, true, I could have used grep to instead of wc
<cody-somerville> *to count
<cody-somerville> And could have used a regular expression instead of cut to make sure I was looking at the right field
 * StevenK hands cody-somerville a 'pointless pipe' award
<LPCIBot> Project windmill-db-devel build #321: STILL FAILING in 1 hr 9 min: https://lpci.wedontsleep.org/job/windmill-db-devel/321/
<wgrant> wallyworld_: How's your picker stuff going?
<wallyworld_> wgrant: been tied up doing other stuff - revising popup field name branch, doing a feature flag branch for disclosure, seperating out a separate branch for the irc nick formatter etc
<wgrant> wallyworld_: Ah, right.
<wgrant> wallyworld_: It seems to me that we're not going to get very far by treating the search/rank/affiliation issues separately.
<lifeless> wow, slow
<lifeless> 190530            0      3.2106      1.3876   +storm.properties:51(__get__)
<wgrant> We have a set of requirements and seem to be jumping into designing these three things separately.
<wgrant> Without actually really thinking about how they will work together.
<lifeless> cody-somerville: so this is easy fixed
<wallyworld_> wgrant: did you know there's a FormattersAPI class which is supposed to handle escaping of invalid html ids. but it doesn't ensure uniqueness
<lifeless> cody-somerville: care to put the patch up?
<wgrant> wallyworld_: I didn't!
<wgrant> wallyworld_: That's somewhat unpleasant.
<lifeless> its probably the root cause for a half dozen timeouts
<wallyworld_> wgrant: curtis mentioned it in reviewing the mp
<wallyworld_> wgrant: with the picker stuff, i think we can do alliliation separately from search/rank. affiliation will be expensive, but we will only need to do it for each batch of 6 or whatever results
<wallyworld_> potentially expensive
<wgrant> wallyworld_: I don't think that's the case.
<wgrant> wallyworld_: We need to be able to filter by affiliation, or at least rank by it.
<wgrant> Search for "william" on a Launchpad bugtask.
<wgrant> That's clearly wrong.
<wallyworld_> hmmm. that will complicate things for sure
<wgrant> Exactly.
<cody-somerville> lifeless, Ugh, sure! :-)
<wgrant> The search at the moment is useless.
<wgrant> It finds the 100 most relevant results, then orders them by displayname.
<wallyworld_> we will need to consider denormalising the data model in that case to make affiliation easy to get
<wgrant> Possibly, but we first need to consider what we *want*.
<wallyworld_> agreed
<wgrant> Do we want ranking, do we want filtering, or what?
<wallyworld_> no question there
<wallyworld_> wgrant: that's what the product team needs to feed us - requirements :-)
<lifeless> cody-somerville: we're doing 64K __eq__ calls
<wgrant> wallyworld_: Are those requirements, or design?
<StevenK> Can haz review from someone?
<wgrant> wallyworld_: The only real requirement is that it returns "good" results. :)
<StevenK> +13/-0
<lifeless> wgrant: wallyworld_: can I be of assistance?
<wallyworld_> requirements. ie what  we need to deliver as opposed to how to do it
<wgrant> wallyworld_: This is part-way between requirements and design, IMO.
<wallyworld_> lifeless: we are pondering the new search behaviour of the picker
<wgrant> Because currently it is useless unless you type the full username or displayname.
<wallyworld_> wgrant: not sure i agree :-) i think we need to be clear about what we need to deliver, not how to do it
<wgrant> wallyworld_: You have a branch to stop using Person.fti, right?
<wallyworld_> wgrant: yes, in ec2 now
<wgrant> wallyworld_: Have you tested how that works in practice?
<wgrant> wallyworld_: I think it will eliminate displayname matching.
<wgrant> Which would be a very bad thing.
<wallyworld_> wgrant: well, i ran it locally and tried a couple of searches with and without the fti removed and they appear to behave the same
<wgrant> wallyworld_: I don't see where it does a displayname search.
<wgrant> Apart from fti (which is name, displayname)
<wallyworld_> i can't recall if i checked display name matching. i'll have another look
<wallyworld_> to be sure
<wgrant> But the query is obscene, so it's a bit hard to tell :/
 * wgrant tries.
<cody-somerville> lifeless, I've got to take the garbage out and then I need to head to bed but I'll push a branch with a patch tomorrow morning if someone else hasn't gotten around to it. Kudos for looking into this. :-)
<lifeless> cody-somerville: I've finished capturing my learnings
<lifeless> cody-somerville: if you  need a hand, just shout
 * cody-somerville nods.
<wgrant> wallyworld_: So, back to affiliation for a sec: yes, it's easy to just show the affiliation of the current six, but that doesn't help if I have to scroll through 10 non-deterministic batches to find the right person :/
<wgrant> (yes, it is actually non-deterministic, it seems :/)
<lifeless> wgrant: wallyworld_: is there a LEP ?
<wgrant> TrustedPickers
<wallyworld_> wgrant: yes, i agree that we may want to sort by affilliation. that is not something i was previously considering
<wgrant> https://dev.launchpad.net/LEP/TrustedPickers
<wgrant> Ahh, https://dev.launchpad.net/LEP/TrustedPickersUseCases doesn't seem to be linked from anywhere.
<wgrant> But is useful.
<wgrant> It has a usecase like the one I raised
<wgrant> StevenK: Can I revert your changes on DF?
<wgrant> Looks like it's just that merge from yesterday.
<lifeless> what defines affilate?
* jtv changed the topic of #launchpad-dev to:  https://dev.launchpad.net/ | On call reviewer: StevenK, jtv | https://code.launchpad.net/launchpad-project/+activereviews | Critical bugs:214 - 0:[######=_]:256
<lifeless> I'm worried about the use of irc handles, maybe I'm cracked
<wgrant> lifeless: Probably membership in a team related to the project.
<wgrant> But then we may well also want a Canonical affiliation, so that falls apart.
<StevenK> wgrant: Sure
<wallyworld_> lifeless: getFeatureFlag. i want to add a variant which returns a user specified default value if features.getFlag() returns None. i'd like to add this to lp.services.features.__init__. but i think you disagree about doing that there?
<StevenK> jtv: Can haz review?
<jtv> StevenK: when I finish my current one, sure
<lifeless> wallyworld_: why do you want to do that ?
<lifeless> wallyworld_: what problem are you solving?
<wgrant> StevenK: Thanks.
<wallyworld_> lifeless: i have an int feature flag and i don't want to do what was done in archive.py
<wallyworld_>         limit = getFeatureFlag(FEATURE_FLAG_MAX_SYNCHRONOUS_SYNCS)
<wallyworld_>         try:
<wallyworld_>             limit = int(limit)
<wallyworld_>         except:
<wallyworld_>             limit = 100
<lifeless> wallyworld_: you'll still need that code
<lifeless> wallyworld_: because crap might be entered in
<wallyworld_> not if i can specify a default as in myflag = getFeatureFlag('flagname', 1)
<lifeless> wallyworld_: I'm totally happy with folk layering convenience functions; I'm unhappy with folk embedding schemas in it
<lifeless> wallyworld_: thats not sufficient to remove the try:except:
<wallyworld_> sure, i'd do the try except inside the getFeatureFlag
<wallyworld_> and maybe name the function getIntFeatureFlag
<lifeless> wallyworld_: I think thats a bad idea because it hides the ability to tell whats going on, and the casting you want isn't related to feature flags
<lifeless> wallyworld_: I think you want a safe_int(value, default=0) function
<wallyworld_> hmmm. i think it is because we specify the type of feature flag - boolean, string, int etc - so there should be relevant methods to get flags cast to the expected type
<wallyworld_> safe_int works for me
<wgrant> wallyworld_: That type is just documentation.
<lifeless> well, the type is crufty and I'm not 100% on board with it being there
<lifeless> but I support folk doing what they think makes sense ;)
<wallyworld_> not sure why you dislike it. it serves a good purpose
<wallyworld_> can i add safe_int to features __init__ ?
<lifeless> wallyworld_: I don't think it belongs in features
<lifeless> theres probably an implementation in the tree already
<lifeless> in a util or misc or helpers or some such module alrady
<lifeless> wallyworld_: i dislike it because typing drives the requirements -way- up for very little win.
<wallyworld_> fair enough. will look. i wasn't going to call it safe_int if it were in feaures init :-)
<wallyworld_> i can't yet bring myself agree with that :-)
<lifeless> wallyworld_: I'm totally in favour of the documentation we have
<wallyworld_> i like the doco too. but if it makes sense to define a flag as an int, we should easily be able to treat it as such
<wallyworld_> s/treat/use
<wgrant> wallyworld_: You may want to kill that ec2 instance before it lands.
<wgrant> wallyworld_: Try searching for "william grant" in the assignee picker on https://bugs.dogfood.launchpad.net/launchpad/+bug/1234
<_mup_> Bug #1234: Gina is an unmaintainable mess of command line options, environment variables and shell scripts <lp-foundations> <Launchpad itself:Fix Released by debonzi> < https://launchpad.net/bugs/1234 >
<wallyworld_> wgrant: sure. i had retested yet. was on my todo :-) display name matching broken then i assume
<wallyworld_> ?
<wgrant> wallyworld_: Yeah :(
<wgrant> I'm not sure why it returns what it does, but it's certainly not right!
<wallyworld_> wgrant: bollocks. i'll look at modifying  the query to specifcally match displayname
<wgrant> wallyworld_: We probably still need FTI over that.
<wgrant> wallyworld_: And Person.fti is already over just name and displayname.
<wgrant> So we probably want an index on Person.fti instead.
<wgrant> (which dev and DF already have, but production doesn't)
<wallyworld_> really?
<wallyworld_> why not production?
<wgrant> Because production lacks it.
<wgrant> It only exists on DF because I created it yesterday totest :)
<wgrant> No idea why :/
<lifeless> we can add that now, losas are around
<wgrant> True.
<wallyworld_> are we sure it wouldn't be better just to say (pseudo)  'xxxx where name like ? or displayname like ?'
<lifeless> wgrant: how big is it on df ?
<wgrant> lifeless: Most will only have three terms, so it should be small, but let's see.
<lifeless> \di+ person_fti
<lifeless> wallyworld_: there is no index on display name
<wallyworld_> doesn't seem worth the cost of creating and maintaining a fti over just two simple string fields?
<wgrant> 49MB
<lifeless> wallyworld_: fred bloggs should match fred J bloggs
<wgrant> wallyworld_: But how do you propose to do a full-text search of displayname?
<wgrant> Without an FTI index?
<wgrant> Right, what lifeless said.
<lifeless> wgrant: from the perspective of a user doing a search
<lifeless> wallyworld_: and foo%bar style searches are terribly slow
<wgrant> lifeless: Hm?
<wallyworld_> fair enough.
<lifeless> wgrant: wildcard searches are not indexed
<lifeless> wgrant: literal prefixes sometimes use indices
<lifeless> ok on the LEP
<lifeless> Must point 1
<lifeless> I think is over broad
<lifeless> I would say that the information which lead to a *hit* must be shown
<lifeless> which is not the same as showing all the possible things we search on on every hit
<lifeless> secondly, point 2: will require a new cached data structure
<wgrant> Agreed
<lifeless> wgrant: agreed on 1 or 2  or both ?
<wgrant> I don't agree that 2 needs a new datastructure, unless we want to search/rank on it, which we probably do.
<lifeless> wgrant: ' previous bug assignee'
<lifeless> wgrant: + Ubuntu
<wgrant> Ah, missed that.
<wgrant> Ubuntu?
<lifeless> Ubuntu scope searches
<wgrant> What about them?
<lifeless> 400K bugs to consult to find previous bug assignees
<wgrant> Right.
<wgrant> I hadn't considered the assignee case.
<lifeless> I suspect we want to expose whatever thing we have an let folk delete inappropriate things
<lifeless> but thats a different discussion
<wallyworld_> lifeless: perhaps a case for a new service :-)
<lifeless> perhaps
<lifeless> I think it would fit well in a services model
<wgrant> lifeless: Right, if we have assignees then we very probably want to permit manual administration.
<wallyworld_> yep, especially since there will likely need to be a (denormalised) data model to go with it
<wgrant> lifeless: Shall we hijack a LOSA to 'CREATE INDEX person_fti ON person USING gist (fti);'?
<wallyworld_> lifeless: btw, found intOrZero() and friends in canonical.lp.helpers.py
<lifeless> wallyworld_: hahahahaha :>
<wgrant> canonical.lp buuuuuuuuuurn
<StevenK> I've been trying to remove that, with little to no success
<wallyworld_> so how confident are we that creating that fti index will solve the picker timeouts?
<wallyworld_> StevenK: well, seems like i'm about to add another bit of code that uses it :-)
<wgrant> wallyworld_: The queries execute in <3s on DF, and that's DF.
<StevenK> wallyworld_: DIE
<wgrant> wallyworld_: There's no other significant explanation for its slowness.
<lifeless> wallyworld_: there may be other causes of slow, but we know that that is a cause
<wallyworld_> besides all the unions etc
<StevenK> jtv: Still reviewing?
<jtv> Yes
<lifeless> wallyworld_: unions are not prima facie causes of slowness
<wgrant> wallyworld_: Unions aren't slow unless the queries are slow.
<StevenK> jtv: How large is this branch? :-)
<jtv> StevenK: it's not the size that matters
<jtv> it's the distractions of people going "hey jtv are you still reviewing that branch?"
<wgrant> The emailaddress search seems to be the slowest now.
<wallyworld_> i agree about unions not being slow per se. i was more commenting on the number of queries. and when you consider our batcher does a count(*) and then a select it's doubly bad
<wgrant> Yes, our batcher is stupid :(
<lifeless> I really must finish that upgrade
<wgrant> Yeah.
<wallyworld_> and in postgres a count(*) is just as bad as a select i'm told so in effect we are executing our queries twice
<wgrant> The FTI is 100ms on DF. Still slow, but not timeoutfully so.
<lifeless> wallyworld_: yes, thats true.
<wgrant> MVCC FTW
<wallyworld_> i wonder if we can't fix that?
<lifeless> its mostly fixed but not followed through on
<wallyworld_> ah, the non exact count stuff?
<lifeless> three things
<lifeless> using query estimates - function is in the tree
<wallyworld_> can you point to the function?
<lifeless> making 'last' links cheap (batchnav fixed, need to merge into the tree and start updating collections to use it)
<lifeless> and lastly making arbitrary offsets cheap (same code as point 2)
<lifeless> oh
<lifeless> and turning off estimates entirely for collections where we can do that
<lifeless> wallyworld_: its in lib/canonical/database I think
<wallyworld_> ta
<lifeless> estimateRowCount
<lifeless> basically parses EXPLAIN
<wallyworld_> that's nice assuming it works well
<lifeless> so one thing you can do is estimate and if the count is < 100 execute the count
<lifeless> on the assumption we've appropriate indices for small datasets to be answered well
<lifeless> wgrant: wallyworld_: on the LEP then
<wgrant> I was about to say.
<lifeless> what should I do with my feedback - leave it with you, add to the LEP ?
<wallyworld_> i haven't digested the comments yet. we can add them and discuss with the product team etc
<wgrant> It seems slightly bad that we have a picker deliverable in 1.5 weeks but we don't actually know what we're doing.
<wallyworld_> lifeless: with the ff int thing, even if i use intOrZero(getFeatureFlag('foo')), that will be used all over the place and will pollute the code with stuff StevenK is trying to get rid of. sure wish i could just add a single helper to features __init__ and use that....
<lifeless> wallyworld_: what stuff is stevenk trying to get rid of ?
<wallyworld_> the helpers.py i think he said
<lifeless> wallyworld_: and why do you need to use it all over the place?
<lifeless> wallyworld_: totally unrelated efforts
<spiv> wallyworld_: canonical.lp as a place I think rather than helpers.py specifically, I think
<wallyworld_> because anywhere we want to see the value if the ff in order hide a block of code, we need to do that test
<lifeless> wallyworld_: I'm confused, if its an int you're not using it for hiding stuff
<wallyworld_> spiv: ah ok. makes more sense
<lifeless> wallyworld_: for hiding stuff just use
<lifeless> if getFeatureFlag('foo'):
<wallyworld_> lifeless: we will be. we will be incrementing the int each time a 2 week sprint is done
<wallyworld_> so pickers first
<lifeless> please start from the beginning
<wallyworld_> we say "if ffvalue == 1" then show the picker stuff
<wgrant> This sounds like a very strange use of flags.
<lifeless> what does this do for you that 'if getFeatureFlag('newpicker'): does not ?
<wgrant> Exactly.
<wallyworld_> ok. so we want to use the one feature flag value
<wgrant> Why?
<lifeless> why ?
<wallyworld_> feature flag name i mean
<lifeless> why?
<wgrant> What benefit does that provide?
<wgrant> Besides no fewer LOSA requests and less flexibility?
<wallyworld_> so that we don't have to keep adding new flags each 2 weeks
<lifeless> wallyworld_: flags are free
<wgrant> Adding a new flag is not expensive.
<lifeless> wallyworld_: /by design/
<lifeless> wallyworld_: you don't even need to document them.
<wgrant> You just use them.
<lifeless> wallyworld_: and for ephemeral ones I wouldn't bother documenting.
<wgrant> And then ask an admin to add them.
<wgrant> You'd have to ask an admin to change the int anyway.
<wgrant> So no admin time saved.
<wgrant> No coding time saved.
<wgrant> Much flexibility lost.
<wgrant> Transparency lost.
<lifeless> and you'd be stuffed if you were not ready to flick the switch for one part but were for another
<jtv> StevenK: moving on to yoursâ¦ it's the copy-into-derived-series branch you want reviewed?
<wallyworld_> wgrant: we can talk tomorrow at teh standup. sinzuiand i discussed it today and sort of thought using an enum style flag was going to be ok
<lifeless> wallyworld_: using a flag as an enum is fine
<lifeless> wallyworld_: I don't think this is an enum situation though, do you ?
<lifeless> wallyworld_: what problem are you trying to solve?
<wgrant> It sounds like you're trying to put a whole lot of features as different stages on a single flag.
<wallyworld_> not having to change flags.py each time we incrementally deliver a new disclosure feature
<wgrant> You don't have to change it, but changing it is roughly two lines.
<lifeless> wallyworld_: so you don't ever change flags.py when you deliver a feature
<lifeless> wallyworld_: you change flags.py when you *start* a feature.
<lifeless> wallyworld_: *if* you chooes to change it.
<wallyworld_> lifeless: yes agreed. and since we are starting a new feature of disclosure each 2 weeks, that's a lot of editing to flags.py
<lifeless> wallyworld_: I don't understand that
<lifeless> wallyworld_: its *literally* 2 lines, with no tests needed.
<wgrant> *And* it's optional, but encouraged.
<wallyworld_> lifeless: and if every sqaud added two lines each time, it will bloat. but i guess we remove old entries when a feature is delivered
<lifeless> wallyworld_: yes
<lifeless> wallyworld_: just imagine the help text you'd need to describe 8 weeks of work loaded onto one flag
<lifeless> '1 means pickers, 2 means bug unsubscribe on mistakes, 3 means ...'
<lifeless> wallyworld_: I wanted the help text to be federated across the codebase
<lifeless> wallyworld_: but that got lost in translation
<wallyworld_> ok. you win :-)
<lifeless> wallyworld_: KISS wins
<wallyworld_> i didn't realise adding to flags.py was optional but i sort of think it should be mandatory
<lifeless> hell no
<lifeless> :)
<lifeless> the -whole- point of these things is to be ultra lightweight
<wallyworld_> why? it adds necessary doco
<wallyworld_> so devs can know wtf is going on
<lifeless> it adds doco
<lifeless> necessary is a matter of perspective
<lifeless> not all things need prose to explain them.
<lifeless> things that will only last a few days certainly don't
<wallyworld_> so how else can one quickly get an overview of what ff are used? i like being able to go lp.net/+feature-info
<wallyworld_> sure, ones that only last a few days re different
<lifeless> so +feature-rules shows the live rules
<lifeless> +feature-info includes all features evaluated during the appservers lifetime
<wgrant> wallyworld_: Flags that have been queried but are undocumented are listed there.
<jtv> StevenK: you're done
<lifeless> wallyworld_: wgrant: on the LEP : I'll leave my feedback with the two of you. Does it help at all with your planning, or are you still in a state of stuck ?
<wgrant> lifeless: I think we're going to be stuck for at least another couple of days.
<wgrant> lifeless: We have been given usecases and that's it.
<wallyworld_> lifeless: i've got enough to do today. we need to talk to curtis though
<lifeless> wgrant: I'd be delighted to help - as a sounding board or whatever - if you like.
<lifeless> sinzui is still doing bug triage though :) so I know he's arund
<sinzui> Am I needed?
<wallyworld_> wgrant: do we have the business rules for deciding affillation documented? that would be a good start
<lifeless> sinzui: so, bug 696973 - what are we doing to satisfy the multi-tenant case?
<_mup_> Bug #696973: There are no roles that can see all private artifacts in (public or private) projects <disclosure> <projects> <teams> <users> <Launchpad itself:Triaged> < https://launchpad.net/bugs/696973 >
<lifeless> sinzui: (two proprietary organisations working on one public project)
<sinzui> If we had business rules this feature would have been created in 2007
<wallyworld_> sinzui: we sort of need them because htey will drive the design decisons we need to make
<wgrant> wallyworld_: We don't even have requirements for affiliation.
<wallyworld_> eg do we need to sort of affilliation is a big issue and will affect to what extent we need a bdifferent data model
<sinzui> lifeless: I believe that rule is supposition. Every person I interviewed assumed that the owners of the project had access to all private bugs. We want to make the rules work like the stakeholders assume they do
<wallyworld_> the stuff i've done so far is geared towards displaying affilliation info
<wgrant> wallyworld_: eg. we clearly want to show affiliation with the project, but we might also want to show other organisational affiliation (eg. to identify Canonical employees). And we might want to filter by it, or rank by it, or just show it in search results. And what about on comments? Do we show all of them? Just the ones for the relevant tasks? Nothing at all? And how do we distinguish types of affiliation? Ubuntu developer vs. bugsquad ...
<lifeless> sinzui: branch privacy is explicitly built to support the rule
<wgrant> ... person vs whatever.
<lifeless> sinzui: I don't care if we have it or not, but I do care if we half-have-it
<lifeless> sinzui: because it will make implementing 696973 impossible.
<wgrant> sinzui: I don't think Ubuntu desires that.
<wallyworld_> wgrant: yes, important questions
<wgrant> sinzui: Ubuntu's owners shouldn't be able to see security bugs by default.
<lifeless> sinzui: we want to deliver the stakeholders needs, which is different
<wgrant> sinzui: They are embargoed.
<sinzui> They can choose to speak against it then So sar they have not!
<wgrant> sinzui: Erm, have we told them? :)
<wgrant> sinzui: The pillar-owners-can-sort-of-see-all-bugs-except-it-doesn't-actually-work appeared one day and even I didn't notice.
<wgrant> *And* it doesn't actually work.
<lifeless> sinzui: whats the right way for me to help refine this? I can think of at least one other approach that doesn't break multi tenancy
<lifeless> sinzui: and would deliver what the stakeholders want, reliably.
<wgrant> I think we want something pretty much like branch visibility policies.
<wgrant> Except a little less unpolished.
<wgrant> We need to have multiple sets of visibility rules.
<sinzui> The crux of poolie's bug was private bugs. And I really do mean everyone I spoke to in landscape, OEM, HWE, U1, Ubunyu, linaro thought their private bugs were getting to the developers...They want  the bugs available to the developers...that is why they were reported
<lifeless> sinzui: yes, I agree
<wgrant> sinzui: Who are "the developers"?
<wgrant> HWE needs private bugs in Ubuntu.
<sinzui> PS. No one understand private branches except the authors of the code. User literally cannot use them without an admin and an interpreter.
<wgrant> Does that means that we can have no community Ubuntu developers?
<sinzui> Stakeholder believe the are the owners of the project registered in Lp
<lifeless> and for some projects they are
<sinzui> Maybe the LP contributes to the communication problem but not delivering the correct messages to the right people
<wgrant> There are bugs which are to be worked on by "the developers". But there are others that aren't.
<lifeless> so there are a few horribly intersecting things
<lifeless>  - shared workspaces
<lifeless>  - unique workspace names
<sinzui> Our stakeholders do not report bugs on public projects if they do not want someone from that project to work on it!
<lifeless>  - branding
<wgrant> sinzui: Why?
<wgrant> sinzui: That sounds like it's because our privacy is terrible.
<wgrant> sinzui: Not any real reason.
<wgrant> sinzui: And that is clearly false.
<wgrant> sinzui: Security bugs in Ubuntu.
<sinzui> Because They beleive that if the bug on on that project, the owners can see it. This is a simple model
<wgrant> sinzui: Are embargoed from everyone but the security team.
<lifeless> one way we can address this is to ditch the shared workspace model
<lifeless> and say 'a workspace is for one group/organisation/team'
<lifeless> and provide a way to aggregate workspaces that are working on the same codebase
<lifeless> this would greatly help OEM/HWE I believe
<wgrant> It is upsetting that we are working on this feature and supposedly have a schedule but really have no idea what we are doing.
<lifeless> one workspace per client, aggregate all the workspaces to see overall OEM work
<wgrant> Right.
<lifeless> each workspace would be a distro/project as appropriate
<wgrant> That's already sort of what they do.
<sinzui> This sounds like an effort to make something even more complicated than what was rejected by our stakeholders. We are not being asked to design a system with more rules, or to solve the worlds problems.
<wgrant> But in workaroundish ways.
<lifeless> this fits -very- well with the derived distro works.
<wgrant> sinzui: What was rejected?
<wgrant> sinzui: And who are the stakeholders?
<lifeless> sinzui: this actually fits very tightly in with what you are proposing (owners can see everything)
<wgrant> And was the issue communicated clearly?
<sinzui> Please read the lep for names
<wgrant> We have had major misunderstandings before.
<lifeless> sinzui: I very strongly believe in building simple things that work well
<sinzui> Yes, like how every thinks the can change the bug supervisor role.
<wgrant> sinzui: I see no Ubuntu stakeholder there.
<sinzui> If someone blogged about that we would be hunted down and clubbed like baby deals
<lifeless> sinzui: I fear that we are in a half-way house at the moment with some very complex stuff (like branch privacy) mixed in with naive stuff (like inaccessible private bugs)
<sinzui> s/deals/seals/
<lifeless> sinzui: we need to decide what the overall shape will be so that we can head towards it
<lifeless> sinzui: if you've decided that we're not doing multitenancy
<lifeless> sinzui: thats fine - I respect that
<sinzui> lifeless: I agreed, and can only provide a UI that our stake holders can use. The UI does describe a simpler model than we have built in the past
<lifeless> sinzui: but it implies we need to change the multitenancy things and address issues like  name squatting when two folk would currently multi-tenant successfully.
<lifeless> sinzui: the alternative is to multi-tenant correctly.
<sinzui> I am relying on the team to do the implementation. The concerns everyone brings up are legitimate, but I think the weight of many users voices are being ignored for reasons that need revalidation
<lifeless> sinzui: s/the/an/
<lifeless> sinzui: I don't understand the ignoring comment - I think everyone here is taking users confusion into account
<wgrant> I agree that branch privacy is currently pretty and terribly confusing, but we have learnt from it. I think we know how to do multi-tenancy fairly well.
<wgrant> s/pretty/pretty bad/
<lifeless> wgrant: I don't know that we do. I'm sure we can for private tenants (bug supervisor per tenant etc), but what about public bugs ?
<sinzui> wgrant: stakeholders will not benefit if we do not fix it. Working on bug and branch privacy is not in scope
<wgrant> Erm.
<wgrant> what is in scope, then?
<lifeless> sinzui: In which case I think its misscoped. I will raise this with francis & jml.
<wgrant> How can we fix project privacy without fixing bug and branch privacy?
<sinzui> We are working on project/distro privacy and allowing projects to state who they trust so that they do no need to manage subscriptions
<wgrant> I don't think fixing the privacy of root objects is useful or practical unless we fix the privacy of the underlying artifacts too.
<lifeless> sinzui: this is squarely in the space of tenancy as a concept. I'm hugely excited about this work.
<wgrant> We are going to have a fourth privacy mechanism, and it will be an unprecedented mess.
<sinzui> I was hoping to abandon it. Since we are not we are supporting those stories for legacy purposes. The principle means to control access will be on the porject
<lifeless> that means we're dropping multi tenancy
<wgrant> And that is a big change.
<wgrant> Of the Launchpad model.
<lifeless> which I'm ok with, but we should make sure we resource it properly
<wgrant> I thought Launchpad was about keeping project communities together.
<wgrant> Not forcing fragmentation because two parties want to have their own private artifacts.
<sinzui> well I thought you were arguing other otherwise wgrant. If I reported a private bug on a public project that the owner of the project could no see, I am not collaborating. Our stakeholder do want those bugs seen
<lifeless> wgrant: the way we do this is not necessarily the best
<sinzui> wgrant: lifeless. There has been no conversation about transition from private to public. Private teams cannot do that. I assume that a commercial story will require that
<lifeless> sinzui: I assume its a desired feature too.
<wgrant> sinzui: Teams can work partly in private and partly in public.
<wgrant> lifeless: Sure, but not doing it at all is also probably not the best.
<wgrant> So, we don't know how privacy is going to work (we don't even know what's in scope), we don't know how pickers are going to work (we don't even have requirements for some parts)...
<lifeless> sinzui: it must be late for you
<lifeless> sinzui: perhaps you me and francis can talk after the TL meeting tomorrow ?
<sinzui> Yes. I am in bed and my turned out the lights
<sinzui> oh...
<wgrant> Heh
<sinzui> about shared and private workspaces. That may overlap with the request for 3 and 3 level nested projects
<lifeless> it may indeed
<sinzui> OEM wants projects that represent factories nested below a project that uses the parts. This is ugly though, because factory a cannot know about factory b
<lifeless> yes
<sinzui> I blew a gasket thinking about it
<lifeless> this is a form of multitenancy
<lifeless> its precisely the thing that the bug we're talking about will break - or if we do either aggregation of workspaces or stay with multitenancy it will enable it
<sinzui> lifeless: The TL meeting is my only meeting tomorrow. I am available to talk most of the day
<lifeless> sinzui: I would like to have some time with you and francis tomorrow on voice to tease things out, after the TL meeting
<sinzui> wgrant: I do not believe we are going to get requirement that could work with agile development. Building the UI first to understand what stakeholders will accept allows us to discover what is needed in the months that are a lotted.
<wgrant> sinzui: Sure, we can't spec it out fully from the start.
<lifeless> I'm +1 on building the UI first, but we do need to have an idea of what it will look like
<wgrant> But we really have very little idea what it's going to look like and again lifeless beats me.
<sinzui> wgrant: The one consistent rule is the members of canonical must have access to all canonicals projects, the bug and branches too, without managing individual subscriptions
<wgrant> sinzui: That is false.
<sinzui> We have pictures that were we recieved
<wgrant> Simple counterexample: Ubuntu security bugs are not to be seen by all Canonical employees,.
<sinzui> I thought we aggeed in the past that security != private
<wgrant> It is true for most things, but there are exceptions.
<wgrant> Perhaps.
<wgrant> But special-casing security seems like it may be overcomplex.
<wgrant> We may see.
<lifeless> other organisations may choose to fund security work in the ubuntu space
<lifeless> multitenancy would suggest showing the entity that will get the bug rather than a plethora of checkboxes
<lifeless> multiple workspaces would suggest that other organisations get their own workspace
<lifeless> jml: if you can be around after the TL call that would be great too
<wgrant> lifeless: Precisely!
<wgrant> We had a bug filed this morning that ubuntu-security is not subscribed to some security bugs.
<wgrant> I imagine that private projects would have a couple of visibility policies.
<wgrant> eg. "Canonical" and "Security"
<wgrant> Private artifacts would have one or the other.
<lifeless> yes, I sketched something like this in prague
<wgrant> No subscription fiddling, no security hardcoding.
<wgrant> Better for everyone.
<wgrant> And it is, to me, the obvious solution.
<wgrant> Oh, we have designs around this?
<lifeless> I do
<wgrant> I'd not heard of this.
<lifeless> or was it texas
<lifeless> perhaps texas
<sinzui> prague
<lifeless> but they are merely options
<lifeless> wgrant: I don't have answers for public multitenancy yet
<lifeless> wgrant: github doesn't multitenant
<wgrant> lifeless: Is public multitenancy a problem?
<lifeless> wgrant: its inconsistent
<wgrant> Yeah, I looked at GitHub yesterday.
<wgrant> It is an interesting model.
<sinzui> I was just doing UI at the time. I went to UDS N and told everyone that while I was working on the UI, I would not be involved in the implementation. Then we switched to squads and I am leading the implentation
<wgrant> lifeless: What's inconsistent about it?
<wgrant> lifeless: I'm not saying there isn't anything.
<wgrant> I just want to know what you think.
<lifeless> wgrant: there is one bug supervisor and one bugtask status
<lifeless> wgrant: but potentially many organisations with differing triage and prioritisation rules
<wgrant> Ah, right.
<wgrant> Yes.
<wgrant> That is slightly more awkward.
<wgrant> It should be considered here.
<wgrant> But I don't think it really requires resolution.
<wgrant> But we should not design against it :)
<lifeless> for example
<lifeless> bazaar had/has another company doing consulting and contracting
<wgrant> Yeah
<lifeless> they would benefit from being able to say 'this matters to me'
<lifeless> and in principle each user has their own prioritisation rules
<lifeless> I'd be quite happy if users could mark all the bugs they care about critical and it didn't impact the LP team.
<lifeless> For instance.
<lifeless> This could either be a genuine discriminator
<lifeless> or terrible
<wgrant> Right, and splitting workspaces will not solve this.
<lifeless> well it would
<wgrant> MMm, I guess.
<wgrant> But not in really useful ways.
<lifeless> there are two issues there
<lifeless> name squatting
<lifeless> and aggregation
<lifeless> if folk can fork the workspace easily and aggregate only what they trust easily
<lifeless> then naming and the passion around that becomes the sole issue
<lifeless> this is roughly the same as the single-blessed-tenant for public tenancy vs private multitenancy
<lifeless> I mean - the two approaches have some common challenges
<lifeless> but will tradeoff on things they do better
<lifeless> the naming/branding thing is actually very important
<wgrant> That's one reason I am reluctant to not multi-tenant.
<lifeless> it may be a driving factor to choose one approach over the other
<lifeless> however I don't know what side of the equation it sits on :)
<wgrant> Anyway, I guess discussing this further here is pointless, so I shall await the outcome of your discussion with Francis and co tomorrow.
<StevenK> wgrant: Do you want to look at refactor-notification again?
<StevenK> wgrant: I've not done some of what you mentioned, however I think I've addressed most of your concerns.
<wgrant> StevenK: Let me have a look.
<wgrant> StevenK: Have you run the delayed copy tests over this?
<wgrant> I'm pretty sure the announcement tests will fail.
<wgrant> Although they may be crap enough that they don't.
 * StevenK does so
<StevenK> wgrant: 88 tests, 0 failures and 0 errors
<wgrant> Yay.
<wgrant> spr.package_upload is not what you want.
<wgrant> So the tests should fail :(
<StevenK> Is that a sarcastic yay I see?
<wgrant> Never!
<StevenK> ITYM "Always!"
<wgrant> Hm.
<wgrant> Why does AJAX "Subscribe someone else" on a bug check whether I can unsubscribe the subscription before it creates it?
<wgrant> * * Check if the current user can unsubscribe the person * being subscribed. * * This must be done in JavaScript, since the subscription * hasn't completed yet, and so, can_be_unsubscribed_by_user * cannot be used.
<wgrant> But why?
<lifeless> thats surprising
<wgrant> It's an extra 5 requsets :(
<lifeless> it may be about teams
<wgrant> Howso?
<wgrant> It should just be able to get can_be_unsubscribed from the subscription once it's created, AFAICT...
<wgrant> It basically reimplements inTeam in JS :/
<wgrant> Buggily, too!
<wgrant> Perhaps I should remove it and see what tests fail.
<StevenK> wgrant: Did you get distracted? :-)
<StevenK> Like I did ...
<wgrant> StevenK: I was waiting for your responses on spr.package_upload.
<StevenK> wgrant: I had no idea it was a question. It seems to work?
<wgrant> StevenK: It's the wrong one.
<wgrant> For delayed copies, at least.
<StevenK> So I change it back to taking a packageupload and getting the spr from the PU?
<wgrant> I don't know.
<wgrant> Possibly optionally taking a PackageUpload.
<wgrant> But then what about binaries?
 * StevenK tosses the entire soyuz tests at this branch
<StevenK> Let's see what blows up
<wgrant> I suggested progressive refactoring to avoid this sort of dead-end. But that doesn't work so well if the tests don't exercise the stuff that is going to break :/
<thumper> does anyone know about poolie's script for making a kanban board?
<wgrant> lp:kanban?
 * thumper looks
<StevenK> wgrant: So I get questioned either way. Crumbs, this code sucks.
<wgrant> StevenK: It sucks, and it's not clear how to solve this particular bit. We need to handle cases without an SPR, and cases without a PackageUpload.
<wgrant> Fun for all!
<wgrant> StevenK: That's why I suggsted an exploratory refactoring to minimise dependencies on SPR without changing behaviour.
<wgrant> But we need to work this out somehow.
<wgrant> Hmmm.
<wgrant> s/SPR/PU/
<StevenK> wgrant: Which I attempted, and then you said I've gone too far. I've dialed it back a bit and it's still wrong.
<StevenK> lib/lp/soyuz/doc/distroseriesqueue-notify.txt is utterly broken
<wgrant> StevenK: The fact that you've gone too far is not a problem. The problem is that you've broken functionality!
<wgrant> And your reversion is broken because SPR.package_upload is stupid and legacy :(
<wgrant> Well, possibly more because delayed copies are stupid and hackish :(
<wgrant> But we still have the binary upload problem which makes me cry.
<wgrant> StevenK: Perhaps we announce on a set of packages.
<wgrant> May have zero or one SPRs, zero or more BPRs, and zero or more baby-eating custom uploads.
<StevenK> And then my explodes trying to keep track of changes files and content for each of those
<StevenK> s/my explodes/my head explodes/
<wgrant> Sounds like it already may have :(
<wgrant> Right. But each announcement has a single changes file.
<wgrant> Which you can possibly pass in for now.
<wgrant> Remember that changes files will be optional soon.
<wgrant> Out of necessity.
<wgrant> So the changes file stuff does not have to be pretty; it is going to be aggressively mutilated in a week or two.
<StevenK> By whom?
<wgrant> undef
<StevenK> PERL!
<wgrant> But it's needed for copies from Debian.
<wgrant> So it has to be done.
<wgrant> So it is left as an exercise to the Julian, and can be disregarded by us as a given.
<wgrant> So.
<wgrant> it looks like we are looking at sets again.
<wgrant> Sets of SPRs and BPRs and pedovores.
<wgrant> What still uses the changesfile....
<wgrant> Apart from too much.
<wgrant> Hm.
<wgrant> So we still parse Changed-By and Maintainer and Changes out of it.
<wgrant> hmmmm.
<wgrant> Difficult.
<wgrant> :(
<StevenK> Yes.
<wgrant> The first two are easily replaced with looking at SPR. The last needs to be replaced anyway.
<wgrant> So we can continue passing the content around for now, I think.
<wgrant> And remove it eventually.
<wgrant> But does that mean we can just use the string?
<wgrant> And not a file object?
<StevenK> The reason we use the file object is due to attaching the actual file to the mail
<wgrant> Hm. We can't do that with a string?
<StevenK> Ordering
<wgrant> Even if we can't, let's just write that to a temporary file or stringio or something.
<wgrant> Huh?
<StevenK> Oh, string
<StevenK> Not dict
<wgrant> String.
<wgrant> Right, not parsed.
<wgrant> Although I guess we need the parsed version too, is that what we parse around niow?
<wgrant> s/parse/pass/
<StevenK> Yes, we pass around a dict and a file-like object
<StevenK>         for build in packageupload.builds:
<StevenK>     AttributeError: 'NoneType' object has no attribute 'builds'
<StevenK> For crying out loud
<wgrant> Ah.
<wgrant> sad.
<StevenK> This is all broken
<wgrant> Do we use the dict outside assemble_body?
<wgrant> Or can we just parse it in there?
<wgrant> And just pass the string around.
<StevenK> Yes, in notify_spr_less()
<wgrant> That doesn't count.
<wgrant> It's a separate beast.
<StevenK> _getRecipients wants the parsed dict too
<wgrant> Why? :/
<wgrant> Ah
<wgrant> Is that used by notify_spr_less too?
<StevenK> No
<wgrant> Hrmph.
<StevenK> It is tied pretty tightly to packageupload, so I ignored it
<wgrant> Anyway, let's try not to change that yet.
<wgrant> We can continue passing around both the dict and file versions, I guess.
<StevenK> Total: 1101 tests, 10 failures, 5 errors in 29 minutes 58.207 seconds.
<StevenK> Those 15 are mostly doctests, so it's worse
<wgrant> That's what I was hoping for.
<wgrant> Failures, not worse.
<StevenK> lp.soyuz.tests.test_packageupload.PackageUploadTestCase.test_realiseUpload_for_delayed_copies
<StevenK> Damn it, I hate it when you're right.
<StevenK> :-P
<wgrant> :(
 * StevenK looks at the failure and goes cross-eyed
<wgrant> StevenK: No point looking at the failure yet.
<wgrant> Until you've fixed it to use the right BPRs and PUCs.
<StevenK> So, stop tossing around SPR and go back to PU?
<wgrant> Then we should have just about nearly almost slightly restored the original behaviour.
<wgrant> And there will be reasonable failures.
<wgrant> Given how far you've gone already, I think just take optional sets of BPRs and PUCs.
<wgrant> In addition to the SPR.
<wgrant> Don't take a PU.
<wgrant> You don't need it.
<wgrant> You just need the subordinate releases/uploads and the changesfile.
<StevenK> You said the opposite for the subject
<wgrant> We can sort of sourceless notifications after that.
<wgrant> Hm?
<StevenK> packageupload.displayname has behaviour we might still need
<wgrant> Argh, forgot that bit.
<wgrant> We need to reimplement that.
<wgrant> Let me look at it.
<wgrant> Ah.
<wgrant> StevenK: You'll see you undo about half of what it does anyway.
<wgrant> StevenK: I think just grab the names yourself from the packages you're given.
<wgrant> Ignore the (delayed) bit.
<wgrant> Which I filed a bug about years ago anyway.
<StevenK> wgrant: If spr.package_upload is wrong, does that mean packageupload.sourcepackagerelease is also wrong?
<wgrant> StevenK: No. PackageUpload.sourcepackagerelease is fine.
<wgrant> But an SPR may have multiple PackageUploads.
<wgrant> spr.package_upload is the original one.
<wgrant> Which isn't the right one in the case of a delayed copy.
<wgrant> StevenK: Do we use the PU for anything else?
<wgrant> StevenK: Apart from getting the BPRs and PUCs, and displayname?
<StevenK> _buildSummary, but I just ripped that out
<wgrant> Great.
<StevenK> Perhaps I should write a PUS to string function
<wgrant> Hm?
<wgrant> You're not going near PUS here...
<wgrant> At least I hope not.
<wgrant> PUC, sure.
<wgrant> Because we have no other way to represent custom uploads.
<wgrant> But not PUS.
<StevenK> wgrant: A lot of the new code I've written takes an action as a string
<wgrant> Oh.
<wgrant> PackageUploadStatus.
<StevenK> PUS being PackageUploadStatus
<wgrant> Not PackageUploadSource.
<wgrant> Right!
<wgrant> But not quite.
<wgrant> Since you need 'announcement' as a separate thing.
<wgrant> Or maybe not.
<StevenK> I do, but that's only called once
<wgrant> If you can separate that out somehow, by all means use PUStatus instead.
<wgrant> Much cleaner.
<StevenK> Er? I'm advocating *not* using it
<wgrant> Oh, you want a map (no need for a function) to turn a PUS into a string to pass into notify?
<wgrant> Mm.
<wgrant> I guess.
<wgrant> Doesn't make much difference.
<StevenK> Not really
<StevenK> Right, assemble_body still uses spr.package_upload
<StevenK> And calculate_subject
<StevenK> That's it
<wgrant> What do they use them for?
<wgrant> s/them/it/
<StevenK> assemble_body wants the signing key
<StevenK> calculate_subject wants displayname
<wgrant> signing key is OK for now. calculate_subject doesn't really want displayname... it's exactly 6 lines to reimplement the part of it that it requires.
<StevenK> So calculate_subject also wants builds and customfiles?
<wgrant> BPRs and PUCs, but yes, it will need to know about them.
<wgrant> StevenK: I think we may actually have a reasonable solution here!
<StevenK> It doesn't look reasonable yet
<wgrant> Well, a reasonable *vision* of a solution.
<StevenK> wgrant: Did you want spr.package_upload.signing_key usage to die too?
<wgrant> More than my "refactor mercilessly and see what falls out" started with.
<wgrant> StevenK: I think that's OK for now.
<wgrant> StevenK: Eventually it will be the requestor instead.
<StevenK> Yes
<wgrant> StevenK: But for now all we have is the SPR signer.
<wgrant> And that's on the original package_upload.
<wgrant> So spr.package_upload.signingkey is OK.
<StevenK> wgrant: Commit and push?
<wgrant> Ideally run a couple of the failing tests first.
<wgrant> But then, sure :)
<StevenK> Right, one is because I can't code
<StevenK> Another is due to the exact thing I just questioned
<wgrant> :O
<wgrant> Heh.
<wgrant> Which? signingkey?
<StevenK>        if spr.package_upload.signing_key is not None:
<StevenK>     AttributeError: 'NoneType' object has no attribute 'signing_key'
<wgrant> Guard it with a spr.package_upload None check.
<wgrant> And see what happens.
<wgrant> I suspect the test is dodgy.
<wgrant> Not creating a valid initial PackageUpload.
<wgrant> Or not invalidating a cache.
<wgrant> Or something like that.
<StevenK> Oh, BAH
<StevenK> set.add(string) is not helpful!
<StevenK>     + '[ubuntu/breezy-autotest] a, e, l, n, p, t 0.99.6-1 (New)'
<wgrant> Heh. That should work.
<wgrant> But set(string) won't.
<StevenK> Ah
<StevenK> Which is what I do exactly
<StevenK> :-(
<StevenK> set([string]) makes me sad
<wgrant> We'll move to 2.7 within a few months.
<wgrant> Where you can just do {string}
<StevenK> Grumble. I think PackageUpload.sourcepackagerelease is also None for dist-upgrader uploads
<wgrant> It is :D
<wgrant> Don't worry about sourceless uploads right now.
<wgrant> Get delayed copy and mixed upload announcements working first.
<wgrant> Let me glance over that, then we can strategise on sourceless.
<jtv> Hey everyone, an American just told me I had to learn about sarcasm!
<wallyworld_> jtv: noooo, really? surely you jest
<wgrant> StevenK: How's it going?
<StevenK> wgrant: test_realiseUpload_for_delayed_copies sucks
<wgrant> Surely not?
<StevenK> Hah
<jtv> wallyworld_: see, there, what you said there?  I can spot sarcasm!
<StevenK> wgrant: Right, delayed_copies fixed
<wgrant> StevenK: Excellent!
<wgrant> StevenK: Hopefully this only leaves sourceless uploads broken.
<StevenK> wgrant: I dunno. :-)
<StevenK> wgrant: http://pastebin.ubuntu.com/612607/ are the failing tests
<wgrant> Ye of even less faith than me, if that is possible.
<StevenK> wgrant: I've fixed test_realiseUpload_for_delayed_copies
<wgrant> OK, that's not too bad.
<StevenK> wgrant: What else do you think I should look at before pushing?
<wgrant> You might want to have a look at TestQueueTool next.
<wgrant> Since it's likely to be more understandable than the doctests.
<wgrant> I hope.;
 * StevenK peers at it
<StevenK> Not sure if I've broken nascentupload-mumble.txt either
<wallyworld_> StevenK: wgrant: you guys are watching SOO tonight, right?
<wgrant> Mumble integration in archiveuploader sounds like a critical bug.
<wgrant> SOO?
<StevenK> State of Origin
<wgrant> I have heard of it but can't remember to which sport it relates.
<wallyworld_> wgrant: sad that you have to ask :-)
<wgrant> Possibly rugby.
<StevenK> Thugby league
<wallyworld_> heathens!
<wgrant> Victorians :P
<wgrant> AFL is all the rage here.
<wallyworld_> and you call yourselves australian
<wgrant> Although I hate it with passion :)
<wgrant> No, I'm Canadian !
<StevenK> I don't really like the thought of watching grown men grop each other for 80 minutes
<wallyworld_> grop? is that a kinky version of grep?
<wallyworld_> or maybe grok?
<StevenK> groupe
<StevenK> Sigh
<wgrant> GroupÃ©?
<StevenK> grope
<wallyworld_> ah well, i won't bother talking about the gaem tomorrow then
<StevenK> wgrant: 4 of the 5 errors are fixed by side-effect, the other is due to the test tossing in stringio
<StevenK> wgrant: (For TestQueueTool)
<wgrant> StevenK: Great, we're getting there :)
<wgrant> wallyworld_: Did you see that ludicrous display last night?
<wallyworld_> wgrant: no, didn't watch much tv last night
<wallyworld_> what happened?
<wgrant> Bah.
<wgrant> Does nobody watch The IT Crowd?
<wallyworld_> wgrant: yes, but i think i've seen them all
<StevenK> I've seen up until the end of season 4
<wgrant> "Did you see that ludicrous display last night?" is the opening of the responses they use from bluffball.co.uk.
<wgrant> To cope with colleagues discussing the game the following day :)
<wallyworld_> hey, how do i use addCleanup() in a doc test? ie how do i get a reference to the test case being run?
<lifeless> wallyworld_: you don't
<thumper> wallyworld_: you don't
<StevenK> wgrant: Commit and push, or fix more?
<thumper> wallyworld_: unless someone fixed it
<wgrant> StevenK: Commit and push and then fix more!
<lifeless> wallyworld_: However it is possible to add a cleanup facility to the globals that our doctests have
<StevenK> wgrant: I don't like that idea. I think it's your turn. :-P
<lifeless> wallyworld_: this is yet another fail of doctests
<wallyworld_> so i want to use a feature flag. i can set it up but i need to be able to discard it afterwards
<StevenK> wallyworld_: with FeatureFixture?
<wgrant> wallyworld_: If you must use it in a doctest, can you use a FeatureFixture with a with block?
<wallyworld_> StevenK: yes
<lifeless> wallyworld_: or
<StevenK> wgrant: distroseriesqueue-views.txt
<StevenK> Sigh
<lifeless> feature = FeatureFixture(...)
<wallyworld_> wgrant: well, the stuff that needs it is spread throughout
<StevenK> wgrant: distroseriesqueue-views.txt also fixed by side effect
<lifeless> feature.setUp()
<wgrant> wallyworld_: :(
<lifeless> ...
<lifeless> feature.cleanUp()
<wgrant> StevenK: Excellent!
<wgrant> StevenK: Do you use testr?
 * StevenK runs soyuz-set-of-uploads, but that crap fails if the wind is blowing the wrong way
<wallyworld_> lifeless: that will work. i was hoping not to have to spread that stuff from one end to the other
<StevenK> wgrant: I do not
<wgrant> StevenK: It's handy for stuff like this.
<wgrant> testr run --failing, done!
<LPCIBot> Project db-devel build #578: FAILURE in 5 hr 16 min: https://lpci.wedontsleep.org/job/db-devel/578/
<wgrant> zomg, it works.
<wgrant> Got rid of that hideous pre-subscription inTeam reimplementation.
<StevenK> wgrant: Pushed
<wgrant> StevenK: Great. Will check in a sec.
<wgrant> StevenK: Is anything left failing except for sourceless uploads?
<wgrant> Also, will my eyes survive the diff?
<wgrant> Our JS infrastructure is fairly nice.
<StevenK> wgrant: Running -vvm soyuz again ...
<wgrant> See you in 50 minutes :(
<StevenK> 29, on this machine
<wgrant> Hmm.
<wgrant> Oh, right 50 was with archiveuploader and archivepublisher too.
* danilos changed the topic of #launchpad-dev to:  https://dev.launchpad.net/ | On call reviewer: danilos | https://code.launchpad.net/launchpad-project/+activereviews | Critical bugs:214 - 0:[######=_]:256
<danilos> oh, jtv, StevenK: I've removed you from the topic as OCRs, are you still "on duty"? (I thought that was from yesterday)
<jtv> danilos: I still am, on request.
<StevenK> danilos: I am, for another 35-ish minutes
<jtv> (But have done about all the backlog I was going to)
 * StevenK drops two from the backlog
<danilos> StevenK, I assume you need not go back on OCR anymore then, and as for you jtv, please re-add yourself to the topic if appropriate :)
<jtv> danilos: you broke it, you fix it.  Or just take over OCR.  Your pick.  :)
<danilos> jtv, heh, ok, ok
* danilos changed the topic of #launchpad-dev to:  https://dev.launchpad.net/ | On call reviewer: jtv, danilos | https://code.launchpad.net/launchpad-project/+activereviews | Critical bugs:214 - 0:[######=_]:256
<StevenK> danilos: *shrug* :-)
<StevenK> I attempted to review benji's drop feature flag branch, but my eyes glazed over at about line 600
<jtv> And I limited myself to one backlog review per person, on grounds of fairness (as in, it's not fair to an engineer to give the poor bastard multiple jtv reviews on the same day)
<StevenK> Haha
<wgrant> StevenK: 'for build in builds' should be 'for bpr in bprs', right?
<StevenK> Probably
<wgrant> "build.build" is slightly nonsensible.
<StevenK> wgrant: Fixed locally
<wgrant> StevenK: Thanks. So packageupload is entirely gone now (apart from spr.packageupload.signingkey), and it appears to work for everything except sourceless uploads?
<StevenK> wgrant: def notify() depends on it
<StevenK> wgrant: And _sendSuccessNotification
<wgrant> That's rather antisocial of them.
<StevenK> send_mail() needs to be weaned off, but that is simple, since it expands all of them from packageupload anyway
<wgrant> Ah, right, I was hoping they'd all just be passed down from notify().
<wgrant> Is that easy enough?
<wgrant> It's a lot of arguments, but meh.
<StevenK> It can worked out inside _sendSuccessNotification or inside notify
<wgrant> We need to pull it up high enough that we can send notifications without a PU.
<wgrant> I don't know how high that is.
<StevenK> wgrant: Currently, notify()
<wgrant> You'd best take it all the way to the top, then :(
<StevenK> I can't easily wean notify off PU
<wgrant> Why not?
<StevenK> _send{Success,Rejection}Notification is fine
<StevenK> wgrant: Because nearly everything notify() does is in relation to packageupload
<StevenK> wgrant: http://pastebin.ubuntu.com/612620/
<wgrant> Ah.
<adeuring> good morning
<StevenK> wgrant: I've pushed it all up to def notify()
<wgrant> StevenK: Great.
<StevenK> _sendSuccess and such need to grow an action, now
<wgrant> StevenK: So, that's enough refactoring for now... we just have to get sourceless announcements working. What crashes?
<wgrant> Or that.
<StevenK> Since it wants the PackageUploadStatus
<wgrant> Right.
<wgrant> That's all that's keeping packageupload anywhere other than notify()?
<StevenK> I *think* so
<wgrant> Great.
<wgrant> This is a really good first step.
<StevenK> wgrant: Almost
<StevenK> packageupload.isAutoSyncUpload(changed_by_email=changes['Changed-By']):
<wgrant> Can that be a class method?
<wgrant> No.
<wgrant> Still, easy to move across.
<wgrant> But what a dirty, dirty piece of code :(
<StevenK> Can't we just kill it and replace it with if not bprs and spr and changed_by == katie?
<wgrant> That was my intention.
<StevenK> But yes, makes me vomit
<wgrant> And check signingkey and stuff, yeah.
<wgrant> That's the only place it's used, except for tests.
<wgrant> (yes, it's tested!)
<wgrant> ... but only in distroseriesqueue-translations.txt.
<wgrant> Yay
<StevenK> Haha
<StevenK> Fail
<StevenK> If not signing_key, from_addr is None. Otherwise, it's Changed-By.
<StevenK> Shoot me now
<StevenK> wgrant: Actually, it's *worse*
<StevenK> No, it isn't.
<StevenK> distroseriesqueue-translations.txt's isAutoSync... is a re-implementation
<wgrant> Hahahaha
<StevenK> It's not bad, so I'll leave it be
<StevenK> It's in the interface :-(
<wgrant> Is that a problem?
<StevenK> Checking if it is
<StevenK> wgrant: I was worried it was exported
<wgrant> PU is exported rather minimally, fortunately.
<StevenK> Indeed
<StevenK> I think I've run out of steam, but with one grave hack, packageupload is contained in notify() only
<wgrant> Excellent.
<StevenK> Sigh, and _getRecipients
<wgrant> Hmm.
<wgrant> This is slightly entertaining.
<wgrant> "Subscribe someone else" makes four AJAX requests to work out where to put the spinner, then three more to perform the action once the spinner is there.
<StevenK> Haha
<wgrant> This means there's more lag between the picker closing the and the spinner appearing than there would be if the spinner was just not used at all.
<StevenK> wgrant: Do you want to claw your eyes out with this diff?
<wgrant> StevenK: With pleasure.
<wgrant> I think I will instead place a static spinner on the "Subscribe someone else" link.
<mrevell> Hello devotrons
<lifeless> wgrant: ---hah---
<wgrant> lifeless: Rather.
<henninge> oh, jtv left us :(
<henninge> danilos: Hi!
<henninge> danilos: Can I bother you for a very quick review, please?
<henninge> https://code.launchpad.net/~henninge/launchpad/bug-753387-ajax-bug-subscription/+merge/62256
<danilos> henninge, if you must
<henninge> danilos: suffer it to be, I pray thee
<danilos> ;)
<henninge> :)
<danilos> henninge, btw, have you thought of using the actual feature flag check? that makes it so much easier to find it when removing the feature flag
<henninge> danilos: oh, we have that in js?
 * henninge reads up on feature flags
<lifeless> we don't
<henninge> ah
<lifeless> we generally export a value to the js for the js to consult
<lifeless> or export custom js based on the flag
<lifeless> usually the former
<danilos> henninge, we do have it in that particular JS file
<henninge> ;-)
<henninge> danilos: use_advanced_subscriptions
<henninge> danilos: sorry, I was blind
<danilos> henninge, nothing a pair of glasses won't fix :)
<henninge> otoh, it always makes sense to check is something is null before you use it
<wgrant> I've just moving that same code around, and was wondering why that isn't all one function.
<danilos> wgrant, henninge: fwiw, we expect a lot of that code to be refactored with the one final big bug our team is still working on
<jml> lifeless: I'm not around for the TL call
<danilos> henninge, as for the null check, sure
<wgrant> danilos: Which bug is that?
<danilos> henninge, also, note that I'd move the null check to update_mute_after_subscription_change() then to encapsulate it properly
<wgrant> I'm refactoring the subscribe someone else stuff at the moment.
<wgrant> Which is hopefully mostly unrelated.
<danilos> wgrant: it's not, it's very much related :(
<danilos> wgrant: bug 772754
<_mup_> Bug #772754: After better-bug-notification changes, list of bug subscribers is confusing <story-better-bug-notification> <Launchpad itself:In Progress by gary> < https://launchpad.net/bugs/772754 >
<lifeless> jml: ok
<danilos> wgrant: basically, we are changing the subscribers list in a big way, and that means that these links are going to change as well
<danilos> wgrant: mock-up we are going for: https://launchpadlibrarian.net/71552495/all-in-one.png
<henninge> danilos: that was actually my first solution
<henninge> danilos: but I thought "Well, usually there is no reason to do that"
<henninge> and I'd think it is the caller's responsibility to provide correct parameters, is it not?
<wgrant> danilos: How can you separate them like that when the "Other bug subscribers" might include teams of which the user is a member?
<wgrant> danilos: Anyway, I think my refactor is pretty much necessary for that. And I hope to be done with it tomorrow.
<wgrant> danilos: (at the moment it tries to predict where the new subscriber node will end up, which looks like it'll be just about impossible)
<wgrant> But if you are working on this imminently, I can drop this and leave you to it.
<danilos> wgrant: yeah, we're already in the process of changing that all
<wgrant> Ah, OK.
<danilos> henninge, r=me, your call on this, you've got good arguments, and I am not strongly tied to any of the sides
<wgrant> It seems odd that the feature was declared done on Monday with a massive change still underway...
<jml> was it?
<danilos> wgrant: I agree
<henninge> danilos: I'use the feature flag check and leave out the null check.
<henninge> ll
<lifeless> jml: the squad roster was changed
<lifeless> jml: which is slightly different, but closely related
<jml> lifeless: oh ok.
<wgrant> It was also intended to release on Monday.
<lifeless> jml: as you aren't around for the tl meeting, can we have a brief call in ~20 minutes time ?
<wgrant> And it was only decided on Monday that that would not happen...
<wgrant> :/
<jml> wgrant: don't frown. agility!
<lifeless> jml: new stuff
<danilos> jml, well, kind of, we left the subscribers list (bug 772754) for maintenance as well, and the "restore subscription on unmute" will only be released with db-deploy
<_mup_> Bug #772754: After better-bug-notification changes, list of bug subscribers is confusing <story-better-bug-notification> <Launchpad itself:In Progress by gary> < https://launchpad.net/bugs/772754 >
<wgrant> jml: Heh, maybe.
<danilos> I do concede that this didn't require full team to work on this one remaining bug, so switching to maintenance was a practical solution
<jml> but, in case you were curious, the whole "moving from feature squad to maintenance" thing is a bit of a mystery to me. flacoste has ideas that I don't quite follow.
<jml> lifeless: ok. I've got to arrange some stuff, might be a little longer.
<danilos> henninge, if you use the use_advanced_subscriptions check, perhaps you should wrap the get_mute_subscription() call in it as well
<lifeless> jml: a little longer is fine
<danilos> henninge, also, note that you might see a merge conflict with db-devel where entire get_mute_subscription was removed
<henninge> :-P
<henninge> danilos: err
<lifeless> btw
<henninge> danilos: I did wrap
<lifeless> expose_user_administered_teams_to_js is causing timeouts
<danilos> henninge, cool, I haven't checked the diff, just responding to your comment, seeing now that you did :)
<lifeless> (to anyone that recognises that function name)
<danilos> lifeless, sounds familiar to me
<danilos> is that +subscriptions page perhaps?
<lifeless> danilos: its being called into from milestone view initialisation, and it causes about 5K sqlobject object comparisons (quadratic on number of teams)
<lifeless> danilos: which turns out to be moderately inefficient, and when multiplied by the number of milestones series pages show (and others may show) it becomes pretty huge
<henninge> danilos: do I have your r= for merging into db-devel as well?
<henninge> danilos: I just tried out the conflict and would like to fix it now and foget about it ;-)
<lifeless> jml: that said, because the tl call is crazy early, sooner is better ;)
<lifeless> allenap: jml: bigjools: https://dev.launchpad.net/LEP/NotificationService
<lifeless> flacoste: ^
<allenap> lifeless: Cool.
<danilos> henninge, yes, you do :)
<lifeless> allenap: I've just updated the servicesrequirements page, if you were already reading it
<allenap> otp
<henninge> danilos: thanks a lot
<henninge> danilos: for reference, this is the resolved diff against db-devel.
<henninge> http://paste.ubuntu.com/612645/
<danilos> henninge, cool, thanks
<lifeless> stub: hi
<lifeless> stub: I wonder, branchsummary, could I ask you to do the sqlobject/storm bits, enough that it can have some tests wrapped around it?
<jtv> allenap: somewhat sad newsâwe'll have to stick to single-package PlainPackageCopyJobs for the time being.
<jtv> Which means that the "collate by archive" code that both of us wrote in our own ways go out the window.
<bigjools> it may come in useful later
<wgrant> Is this for PackageUpload integration?
<wgrant> Or is there some other consideration?
<wgrant> eg. error reporting
<allenap> jtv: :_(
<jtv> wgrant: yes, it's basically error reporting & handling.
<jtv> Not related to PackageUpload work.
<jtv> bigjools: I doubt it'll be useful to keep any of that codeâit's basically 3 lines.
<jtv> And while I'm looking at thisâ¦ can we really only copy from a series' main archive?
<LPCIBot> Project windmill-devel build #133: STILL FAILING in 1 hr 10 min: https://lpci.wedontsleep.org/job/windmill-devel/133/
<wgrant> :(
<jtv> Oh, to _and from_ main archives.
<wgrant> jtv: Doesn't PackageCopyJob store the source and target archive?
<wgrant> Possibly only in the JSON.
<wgrant> But still.
<jtv> wgrant: store, yes.
<jtv> But those values get set to the respective main archives for the source and target distroseries.
 * jtv mumbles about forgetting non-primary archives altogether and modeling "partner" as an additional distroseries parent instead
<wgrant> Are we considering partner here?
<wgrant> It is meant to be demolished soon.
<bigjools> partner can go to hell
<stub> lifeless: EPARSE
<wgrant> :)
<wgrant> I am glad we agree.
<wgrant> So it doesn't matter.
<bigjools> partner has no place in DDs
<wgrant> If a derivative wants partner, they are stupid.
<wgrant> Right.
<bigjools> I'm simply not supporting it so they have no option
<wgrant> Heh, good policy.
 * stub tries to recall BranchSummary
<wgrant> bigjools: I guess creating a new distro doesn't even create the archive, does it... that's handy.
<wgrant> May 534 be the only purpose 4 archive that ever exists :)
<bigjools> with any luck
<lifeless> stub: bug 758587
<_mup_> Bug #758587: summarising bugs is expensive <timeout> <Launchpad itself:Triaged> < https://launchpad.net/bugs/758587 >
<wgrant> bigjools: I might point out that rabbit isn't usable in the test suite yet :/
<wgrant> bigjools: It was disabled yesterday, reenabled yesterday, and is failing again today but I haven't disabled it again yet.
<wgrant> allenap: Did you see that? :(
<allenap> wgrant: No, I haven't seen that yet :-/
<wgrant> It's failed twice.
<wgrant> On lucid_lp
<stub> lifeless: Ahh... confused me by saying 'BranchSummary', not BugSummary
<stub> lifeless: sure
<lifeless> hhhha 0- yes
<lifeless> thanks!
<lifeless> gnight all
<jtv> I guess we're not actually putting anything in Job.requester yet?
<jtv> bigjools: Job.requester isn't hooked up yet.  :(
<bigjools> jtv: cock
<jtv> See how useful a Launchpad Person would be sometimes?  Even as a stopgap?
<bigjools> jtv: did you see how much work it would be to add?
<bigjools> jtv: I'd not be averse to using the Janitor
<jtv> Probably not a lot of workâ¦ just a matter of adding it to the interface, the model class, & the utility.  Just not sure whether that might interfere with abentley's ongoing Job work.
<jtv> The Janitor as a bug reporter?  Heaven help us.
<bigjools> jtv: it's just going to be adding comments on DSDs ...
<bigjools> it already adds comments on bugs elsewhere
<jtv> But only along the lines of "you left this open overnight so I threw it away," no?
<bigjools> jtv: no, when uploads close bugs, for example
<jtv> Well it's a bit of a metaphorical stretch but that to me is still "you left this open so I threw it away."
<bigjools> it's nothing like that!
<jtv> "It was starting to smell funny."
<bigjools> :)
<jtv> Although... true, an upload is different from a regular time-based closing.
<jtv> So we've already got the Janitor filling in for that Launchpad persona I suggested.
<jtv> Great.
<jtv> Maybe I'll just make the problem worse to drive the point home.
<bigjools> jtv: the janitor is not actually *doing* anything, it's a fake persona that we just stamp onto things
<bigjools> perhaps that was the point that nobody understood in your email?
<jtv> Yes, perhaps.
<jtv> I thought the examples of "be the owner of things" and "be the author of messages" would exemplify that, but I suppose they didn't.
<jtv> (As opposed to "create things" and "post messages").
<jml> henninge: I notice that https://code.launchpad.net/~chrisjohnston/launchpad/197793/+merge/61053 hasn't landed. Any special reason?
<jml> also https://code.launchpad.net/~chrisjohnston/launchpad/483373/+merge/61042
<henninge> jml: actually, three of his branches havn't landed.
<henninge> jml: test failures, simple to fix
<henninge> jml: I emailed him to ask if he wanted to fix these himself
<henninge> jml: if don't here back from himi I'll JFDI myself
<jml> henninge: thanks!
<henninge> I consolidated them in one branch to land: lp:~henninge/launchpad/land-chrisjohnston-187013-197793-483373
<StevenK> wgrant: packageupload ripped out of _getRecipients
<wgrant> StevenK: !
<wgrant> Excellent.
<wgrant> So it's in notify() only now?
<StevenK> I think so. I'll check once I fix this pyflakes issue
<StevenK> wgrant: Yes, it's just notify()
<wgrant> Excellent.
<wgrant> So now we just need to deal with sourcelessness.
<wgrant> ... and test fallout.
<StevenK> wgrant: I've been thinking about this while cooking dinner. We can probably just re-implement notify() as notification() with spr, bprs and customfiles
<wgrant> StevenK: If you call it that your apartment will collapse.
<wgrant> But yes, something along those lines.
<StevenK> wgrant: notification() was what we agreed to before? :-)
<StevenK> wgrant: And then the packageupload gubbins can be held inside queue.py
<wgrant> notification() isn't a verb.
<wgrant> But yeah.
<StevenK> Oh well, we just fix notify()
<wgrant> Yes.
<wgrant> Fix notify() to not take a PU, moving that into PU.notify.
<jtv> Why, oh why does LaunchpadScriptLayer.switchDbConfig sabotage my database connection?
<jtv> "Connection is closed."
<stub> If it was deliberate, maybe to ensure a rollback?
<jtv> Well the one test for this does switchDbConfig, then opens a cursor and does a query,.
<jtv> So I suspect it's something that broke when we went to Storm.
<jtv> "Test that [â¦] we end up connected as the right user."
<jtv> It actually queries current_user.
<jtv> I try commit / switchDbConfig / access-the-db and it fails in that last step.
<jtv> switchDbConfig is implemented as a call to reconnect_stores
<jtv> â¦which actually accesses the database through the main store.
<jtv> I tried passing the same config name as the layer test did, but no change.
<wgrant> You know that switchDbConfig is called once in the whole codebase, right?
<wgrant> What are you doing with it?
<wgrant> Everyone tends to use switchDbUser normally...
<LPCIBot> Project windmill-devel build #134: STILL FAILING in 1 hr 9 min: https://lpci.wedontsleep.org/job/windmill-devel/134/
<abentley> henninge: bug #419531
<_mup_> Bug #419531: project name picker search / vocabulary is hard/impossible to use (too many results on exact searches) <disclosure> <regression> <vocabulary> <Launchpad itself:Triaged> < https://launchpad.net/bugs/419531 >
* jtv changed the topic of #launchpad-dev to:  https://dev.launchpad.net/ | On call reviewer: danilos | https://code.launchpad.net/launchpad-project/+activereviews | Critical bugs:214 - 0:[######=_]:256
<jcsackett> jtv: you still having a problem with the switchDb stuff?
<jtv> jcsackett: no, I dropped it.
<benji___> danilos: if you're looking for a somewhat big branch to review, I have just the thing for you: https://code.launchpad.net/~benji/launchpad/bug-784575-flags/+merge/62175
<jcsackett> jtv: dig. i was about to suggest the helpers in lp.testing.storm but i just realized those are for switchDbUser, not *Config, so no help anyway.
<danilos> benji, ha, sounds just like the thing I've been missing in my life today :)
<benji> heh
<danilos> benji, though it's measly 2627 lines, not up to my standards :) I'll take it, but it's going to be the last one for today then; also, you've got a conflict
<benji> danilos: thanks; I'll look at the conflict now
<danilos> thank you
* danilos changed the topic of #launchpad-dev to:  https://dev.launchpad.net/ | On call reviewer: - | https://code.launchpad.net/launchpad-project/+activereviews | Critical bugs:214 - 0:[######=_]:256
<danilos> benji, you might want to prepare a merged branch with db-devel as well since that one has more changes to the code you are touching, so it will raise conflicts when your branch hits stable and is attempted to be merged to db-devel (saying because I've seen the changed code :)
<LPCIBot> Project windmill-db-devel build #322: STILL FAILING in 1 hr 9 min: https://lpci.wedontsleep.org/job/windmill-db-devel/322/
<danilos> benji, lines 112, any reason not to make this an actual property defined on the class level? (i.e. field_names = ['bug_notification_level'])
<benji> danilos: well, we wouldn't want a list as a class attribute (shared mutables are bug magnets), but we could make it a tuple -- assuming no one tries to append to it
<danilos> benji, sure, a tuple would be nicer
<benji> if you would, note that in your review and I'll see if the tuple works
<danilos> benji, sure, though do note that we've mostly used lists for these in a bunch of LaunchpadFormViews
<benji> that's unfortunate; mutable class attributes have cost the universe many an hour
<danilos> benji, perhaps worth filing a tech-debt bug and mailing the -dev list to warn of the problem?
<benji> could be, I'll think about doing that
<danilos> thanks
<LPCIBot> Project devel build #749: FAILURE in 5 hr 34 min: https://lpci.wedontsleep.org/job/devel/749/
<jcsackett> sinzui: i am trying to qa bug 228355, and it occurs to me that qastaging uses the same signup service as lpnet; i'd rather not create a throwaway account on the service to test the change. thoughts?
<_mup_> Bug #228355: Unique name reveals hidden email address <disclosure> <lp-registry> <qa-needstesting> <Launchpad itself:Fix Committed by jcsackett> < https://launchpad.net/bugs/228355 >
<sinzui> hmm
<wgrant> I have a couple of test accounts on prod SSO.
<wgrant> At least one of which I haven't used on qastaging lately.
<sinzui> Isn't this about one that has never been used
<wgrant> Well, I have two SSO accounts that don't have LP people on qastaging. So they will do.
<sinzui> yes, thanks
<wgrant> (I created them back when, er, experimenting with U1)
<wgrant> https://qastaging.launchpad.net/~me+ubuntuonetest
<wgrant> That looks qa-ok.
<danilos> benji, around lines 1224-1254, you remove a test that I believe is still useful to keep around, even if it needs to update for the new +subscribe page (eg. selecting appropriate radio box)
 * benji looks
<danilos> I am mostly concerned because it's related to private bugs
<benji> I don't think so.  My reading of that test is that it's about the unsubscribe and then redirect behavior, which the new code doesn't have.
<sinzui> thanks wgrant
<jcsackett> thanks, wgrant.
<danilos> benji, uhm, ok, but what happens with the new code when someone unsubscribes from the private bug using +subscribe page?
<benji> do you mean the +subscription page?  or are you really taking about +subscribe?
<benji> my understanding is that +susbcribe will be phased out
<danilos> I mean +subscribe, it's there for non-JS browsers, we are not breaking it even if we were not focusing on it
<benji> if you think it's worth testing, I'll rejigger the test to use the new UI
<danilos> benji, the test was actually testing +subscribe page for these things, with advanced features, it offers slightly different options, but still works for non-JS browsers
<benji> right
<danilos> benji, ok, I guess it's best to check with gary when he comes back if we want to keep +subscribe working for non-JS browsers (I assume we do), and then reinstate the tests if true (that includes most of xx-bug-personal-subscriptions.txt that you are removing as well, imo)
<danilos> benji, also, this is about Bug:+subscribe, not Product:+subscribe page I think, I am not sure about the latter
<LPCIBot> Project windmill-devel build #135: STILL FAILING in 46 min: https://lpci.wedontsleep.org/job/windmill-devel/135/
<danilos> benji, other than the +subscribe situation which I am unsure about, my other big concern with the branch is keeping the feature flag in the JS, and removing a third feature flag conditional in one place
<danilos> benji, thus, needs-fixing for now
<benji> it sounds like you found some good stuff, I'll take a look in a while
<LPCIBot> Project windmill-db-devel build #323: STILL FAILING in 1 hr 9 min: https://lpci.wedontsleep.org/job/windmill-db-devel/323/
<jcsackett> sinzui: chat?
<sinzui> yes
<jcsackett> i believe i have the empathy stuff all set up right.
<sinzui> I am told you are not available? were you calling me?
<jcsackett> i was trying to.
<sinzui> try again
<jcsackett> huh. empathy just crashed.
<deryck> Hi, everyone.
<henninge> deryck! ;-)
<LPCIBot> Project windmill-devel build #136: STILL FAILING in 46 min: https://lpci.wedontsleep.org/job/windmill-devel/136/
<gary_poster> danilos, benji, yeah, until told otherwise by LP leadership, I suspect we should keep previously-working non-JS functionality around.  I'm thinking of Bug:+subscribe here.
<benji> gary_poster: the discussion was more about the tests for the new +susbcribe (the one that exists with the feature flag on/removed), I removed some tests for the old +subscribe and need to verify that the new +subscribe has annalogous tests and (I suppose) implement them if not
<benji> (I haven't (intentionally) removed any functionality.)
<gary_poster> benji, huh.  i'm not sure i'm on the right track then'
<gary_poster> gimme a few and we can have a call, to make sure, if that is alright
<benji> that's ok, I wasn't on the right track when Danilo and were discussing it earlier :)  I've since found enlightenment.
<benji> sure
<cjohnston> henninge: ping
<henninge> Hi cjohnston!
<cjohnston> Hey! Just checking the status of my MPs
<cjohnston> ;-)
<henninge> cjohnston: The tests failed, I forwarded the email.
<henninge> cjohnston: I just wanted to give you a chance to fix the tests yourself but I can do that, too.
<henninge> cjohnston: they are simple to fix
<cjohnston> When did you send the email
<henninge> cjohnston: yesterday, 9:37 UTC
<cjohnston> found ti
<cjohnston> it
<cjohnston> was hidden
<jml> henninge: out of curiosity, did you run the tests with 'ec2 land'?
<henninge> jml: yes, I believe so
<jml> henninge: I ask because it's supposed to email the patch author with the test results as well the lander.
<jml> henninge: if it's not, maybe it's buggy.
<henninge> jml: I created a new branch an merged three of cjohnston's branches and landed that branch
<jml> ahhh
<jml> that'll do it.
<cjohnston> jml just wants me to get more mail since I want to throw the pie :-P
<jml> cjohnston: heh
<jml> cjohnston: it's actually that I have a deep and abiding belief in automating the processing of information using computers
<jml> call me crazy
<cjohnston> ;-)
<deryck> henninge, well done on plowing through some bugs the last couple days.
<henninge> deryck: ;-)
<cjohnston> henninge: I'm not completely sure how to fix this.. Got time to help me learn?
<henninge> cjohnston: sure
<henninge> well, actually, time is the problem
<cjohnston> Want to try tomorrow?
<cjohnston> I think your nearing your EOD?
<henninge> cjohnston: I am.
<henninge> cjohnston: yes, let's do it tomorrow, if that's ok.
<cjohnston> Sure.. Do you have a time frame that may be good? Its morning over here, so I'd just like to make sure I'm about.
<henninge> cjohnston: what's your timezone?
 * henninge is at UTC+2
<cjohnston> -4
<henninge> oh, not too bad
<cjohnston> anytime after 1130 UTC should be good for me.
<henninge> cjohnston: you should just ping me whenever you come online
<henninge> ok
<cjohnston> Ok.. Sounds great. Thanks
<henninge> cjohnston: actually, I might be at lunch at 1130, make it 1200. See you tomorrow ;)
<henninge> UTC
<jml> g'night all
<bigjools> nn from me too
<nigelb> I'm curious, what's the difference in the two options on the survey?
<nigelb> jml: ^^
<LPCIBot> Project windmill-devel build #137: STILL FAILING in 1 hr 8 min: https://lpci.wedontsleep.org/job/windmill-devel/137/
<LPCIBot> Yippie, build fixed!
<LPCIBot> Project db-devel build #579: FIXED in 5 hr 42 min: https://lpci.wedontsleep.org/job/db-devel/579/
* abentley changed the topic of #launchpad-dev to:  https://dev.launchpad.net/ | On call reviewer: abentley | https://code.launchpad.net/launchpad-project/+activereviews | Critical bugs:214 - 0:[######=_]:256
<LPCIBot> Project windmill-db-devel build #324: STILL FAILING in 1 hr 16 min: https://lpci.wedontsleep.org/job/windmill-db-devel/324/
<cr3> if I get married, I'm registering my marriage as a project on launchpad so that we can track bugs, write blueprints, etc.
<LPCIBot> Yippie, build fixed!
<LPCIBot> Project devel build #750: FIXED in 5 hr 22 min: https://lpci.wedontsleep.org/job/devel/750/
<bac> cr3: remember we don't have private projects yet, so this will all have to be open source.
<cr3> bac: I'll make sure the project is under a GPL license so that others getting married have to do the same
<beuno> I'd be worried about people forking the project with GPL
<LPCIBot> Project windmill-db-devel build #325: STILL FAILING in 47 min: https://lpci.wedontsleep.org/job/windmill-db-devel/325/
<lifeless> cody-somerville: hi :)
<lifeless> flacoste: btw - https://dev.launchpad.net/LEP/NotificationService#preview
<cody-somerville> lifeless, Hi. :) Besides creating a new branch, I haven't had a chance to start tackling the bug we discussed last night - have been unfortunately been unexpectedly preoccupied today with other matters. I'm still hoping to get around to it today however.
<LPCIBot> Project windmill-devel build #138: STILL FAILING in 1 hr 9 min: https://lpci.wedontsleep.org/job/windmill-devel/138/
<lifeless> cody-somerville: no worries
<lifeless> cody-somerville: its rather low hanging, so I might stab at it myself.
<lifeless> I have one more (phew) email from tuesday to send
<jkakar> Does Launchpad use PostgreSQL partitions for large tables?
<lifeless> no
<jkakar> Ah, okay, I've been wondering at what point it makes sense to think about it.
<lifeless> selena would be a good person to ask, I think
<lifeless> she did a 'managing TB's' talk at pgcon last week
<cjohnston> Is LP slow for anyone else or is it just me
<cjohnston> I'm up to over a minute trying to load a bug report
<lifeless> which bug
<cjohnston> https://bugs.launchpad.net/bugs/787914
<_mup_> Bug #787914: Bottom of "Why use Ubuntu?" has "download 11.04" with an 10.10 logo. <Ubuntu Website:New> < https://launchpad.net/bugs/787914 >
<lifeless> loaded in 3 seconds for me
<lifeless> 0.91 seconds render time
<cjohnston> must be me.. what abot loading the attachment?
<sinzui> It is slow for me, still laoding
<sinzui> and still loading
<cjohnston> ok.. so im not the only one
 * sinzui thinks a server is going belly up
<lifeless> attachment loaded
<sinzui> still loading
<cjohnston> It's seemed slow to me for a day or two now, but this is bad.
<lifeless> while thats slow
<lifeless> please check -
<lifeless> ping launchpad.net
<lifeless> we want no packet loss over 10 pings
<cjohnston> From eth0.chenet.canonical.com (91.189.88.133) icmp_seq=25 Destination Host Unreachable
<lifeless> ahha
<sinzui> I can get to qastaging very quickly, but neither browser is loading lpnet pages
<lifeless> 223 works
<lifeless> 222 does not
<lifeless> goig to our ops channel to grab a network guy
<lifeless> sinzui: could you perhaps tweet on identi.ca? I'll start escalation stuff
<lifeless> I have to go out soon for a medical appointment
<sinzui> okay
<cjohnston> I feel better that I'm not imagining things
<LPCIBot> Project windmill-devel build #139: STILL FAILING in 46 min: https://lpci.wedontsleep.org/job/windmill-devel/139/
<lifeless> when I put 91.189.89.222 launchpad.net into my hosts
<lifeless> I cannot browse the site
<lifeless> its failing fast now
<lifeless> should be happier now
<lifeless> sinzui: front end is back
<sinzui> thanks
<cjohnston> lifeless: sinzui 0% packetloss in 14 packets
<cjohnston> this ones from nutmeg
<sinzui> deryck: gary_poster: If one of you can confirm that we really do not need to store wiki names any more, we can close about 6 bugs: bug 186660 is the root issue
<_mup_> Bug #186660: Launchpad shouldn't store wiki names <lp-registry> <users> <Launchpad itself:Triaged> < https://launchpad.net/bugs/186660 >
<gary_poster> sinzui, cool.  deryck, I'm happy to promote that one on my team, or do you want to call it?
<deryck> gary_poster, you can take that one since I got the downtime action earlier. ;)
<gary_poster> deryck, +1 :-)
<lifeless> gary_poster: hi
<gary_poster> lifeless, hi
<lifeless> gary_poster: I need a very quick code check with you
<gary_poster> ok lifeless
<lifeless> gary_poster: can you do voice? its near your EOD so I want to be as prompt as possible
<gary_poster> sure; lemme close a door :-)_
<lifeless> say when
<gary_poster> lifeless, I assume you'd prefer mumble. I'm in yellow one-on-one; join me or drag me
<gary_poster> Skype is also fine
<lifeless> gary_poster: mumble is terrible for me from NZ :)
<lifeless> skype is better, sadly.
<lifeless> gary_poster: bug 787765
<_mup_> Bug #787765: ProductSeries:+index timeouts <oem-services> <regression> <timeout> <Launchpad itself:Triaged> < https://launchpad.net/bugs/787765 >
<cjohnston> lifeless: I'm getting between 5 and 11% packet loss on 222 again
<NCommander> StevenK or wgrant: if you have an opportunity, I'd like you to review  my test case for arch any/all skew. (I got sidetracked on working it this and last week due to specs)
#launchpad-dev 2011-05-26
<sinzui> wgrant: mumble
<poolie> thumper: hi
<thumper> hi poolie
<poolie> thumper: did you find out about kanbans?
<thumper> poolie: yeah... although it seemed to have missed out one of the bugs and I don't know why
<poolie> (just reading my scrollback from my holiday)
<poolie> neither do i :)
<poolie> which bug? and what mode of kanban were you using?
<LPCIBot> Project windmill-devel build #140: STILL FAILING in 1 hr 18 min: https://lpci.wedontsleep.org/job/windmill-devel/140/
<lifeless> cjohnston: I suspect your link - I see none
<poolie> o/ lifeless
<StevenK> wgrant: I've pushed all of packageupload out of notify()
<StevenK> Now to fix the tests
<wgrant> StevenK: Great!
<lifeless> hi poolie
<StevenK> wgrant: I had a thought about binary notification too -- We can call notify_spr_less() if all of spr, bprs and customfiles are None. Or we can just fix model/queue to call notify_spr_less() directly
<wgrant> StevenK: Possibly.
<wgrant> StevenK: What in the main function still needs the SPR?
<StevenK> wgrant: def notify()?
<wgrant> Right.
<wgrant> It's difficult because the SPR has lots of the data we need.
<wgrant> eg. Changed-By and Maintainer.
<StevenK> wgrant: _buildUploadedFilesLists, _buildSummary, and _sendNotification
<wgrant> Which BPR only has normally because we get it from the changes file...
<wgrant> but we can possibly get that from the linked SPR.
<jtv> Any reviewers in town?
<poolie> hi wgrant, stevenk, jtv
<poolie> bac, hi?
<jtv> hi poolie
<StevenK> poolie: Hai
<jtv> and hi StevenK, while we're at it :)
<StevenK> NCommander: If you show me a diff or a WIP MP, happy to look it over.
<StevenK> wgrant: Right, I have nascentupload-announcements.txt passing -- except for the notify_spr_less() call
<StevenK> wgrant: Since notify_spr_less() wants the file-object, so it can get the filename
<jtv> I haven't touched JS in a while and need to get back into it.  Can anyone help with some Ajax interaction issues?
<jtv> Two questions right off:
<jtv> 1. When the client waits for stuff to happen on the server, have we got anything better than dumb regular polling yet?
<jtv> 2. What's the current pattern for re-rendering a piece of TAL once stuff happens?
<jtv> thumper: I know you're not here, but any spontaneous pearls of wisdom would be most welcome.  :)
<jtv> lifeless maybe?
<thumper> jtv: for what?
<jtv> thumper: see above
<jtv> "I haven't touched JS in a while" etc.
<thumper> jtv: I don't think so
<jtv> booo!
<thumper> jtv: I believe there is a plan for long-poll
<thumper> but nothing is implemented yet AFAIK
<jtv> Ah yes, plans.  Don't we love those.  :)
<thumper> :)
<wgrant> jtv: All we have is dumb regular polling :(
<wgrant> You can ask for an HTML representation from lazr.restful.
<wgrant> But in general I think it's better to alter the DOM yourself.
<wgrant> What exactly are you wanting to do>
<jtv> Re-render DSD entries on +localpackagediffs
<jtv> â¦because syncing completes, for instance.
<jtv> I think there could be so much to that (especially as changes progress) that it makes more sense to re-render the TAL than to try and mimick the same logic in JS.
<jtv> I suppose I could move more into the view so that more of that logic is still done server-side and transferred as data, but then I fear I may end up with, in essence, a custom representation of the HTML.
<jtv> wallyworld_: wasn't "4+1" a Vietcong methodology?  Approach quickly, strike quickly, disengage quickly, retreat quickly; prepare slowly.
<wallyworld_> jtv: maybe it was. perhaps the software world copied :-)
<jtv> Butâ¦ butâ¦ copying is a _bad_ thing isn't it?
<jtv> At least in Chinese-influenced cultures, everything catchy needs at least one number in it.  And there aren't that many catchy numbers.
<jtv> "One [x], one [y]."  A thousand points of light.  The Four Precepts.  And so on.
<wgrant> wallyworld_: How goes QA for bug #302097?
<_mup_> Bug #302097: Trying to access rss feeds of a private bug OOPSes <easy> <feeds> <lp-bugs> <oem-services> <oops> <qa-needstesting> <Launchpad itself:Fix Committed by wallyworld> < https://launchpad.net/bugs/302097 >
<StevenK> wgrant: Seems I can't just call notify_spr_less() in do_reject() :-(
<wgrant> StevenK: Oh?
<StevenK> wgrant: So for nascentupload-announcements.txt at least, there are 2 rejections -- one with full content, and one with an incorrect .changes filename
<wallyworld_> wgrant: waiting for staging to be updated
<wgrant> StevenK: Could you QA bug #787904?
<_mup_> Bug #787904: Copying a package into a derived series fails <derivation> <qa-needstesting> <Launchpad itself:Fix Committed by stevenk> < https://launchpad.net/bugs/787904 >
<StevenK> Oh, it is
<StevenK> I already know the fix is fine
<wgrant> Ah, great. Please mark it as such.
<StevenK> wgrant: Done
<wgrant> Thanks.
<StevenK> wgrant: Sorry, I tossed it at ec2 and meant to do so when the qa tagger nagged me
<wgrant> wallyworld_: I think I just QA'd it on qastaging.
<wgrant> wallyworld_: Needs feeds.qastaging added to /etc/hosts, and has to be accessed over HTTPS, but it works.
<StevenK> Are you not sure? :-)
<wallyworld_> wgrant: i was told rss feeds don't work on qastaging and that staging is the system to use
<StevenK> wgrant: Would an RT to get feeds.qastaging added to DNS be a good step?
<wgrant> wallyworld_: It's not in DNS and needs an Apache reconfig to be complete, but feeds.qastaging.launchpad.net is accessible if you poke it hard enough.
<wgrant> StevenK: Probably.
<wallyworld_> ok :-)
<wgrant> "The requested bug is private. Feeds do not serve private bugs."
<wgrant> Looks reasonable.
<wallyworld_> i think so. better than a yucky error page
<StevenK> Hm, ENOLIFELESS
<wgrant> netsplit
<StevenK> For over ten minutes? Go go Freenode
<StevenK>  lib/lp/archiveuploader/tests/nascentupload-announcements.txt
<StevenK>   Ran 1 tests with 0 failures and 0 errors in 9.389 seconds.
<StevenK> \o/
<wgrant> How'd you manage that?
<StevenK> By doing evil things
<wgrant> That goes without saying.
<StevenK> changes['_filename'] = changes_file_object.name
<StevenK> ^ That, mostly
<wgrant> OK, not that evil.
<wgrant> :(
<StevenK> The only thing that cares about the filename is notify_spr_less
<StevenK> And I can't call it from do_reject, since it has no idea about the callstack
<StevenK> Anyway, running -vvm soyuz over it now
<wgrant> Great.
<wgrant> Then commit, push, and I will examine it and work out what we can do next.
<wgrant> To denauseatify it.
<StevenK> After the tests or during?
<wgrant> After.
<wallyworld_> wgrant: hard hard is it to get another field added to a fti and where do i look to see how an existing fti is defined?
<wallyworld_> how hard
<wgrant> wallyworld_: database/schema/fti.py
<wgrant> Reindexing everything is harder.
<wgrant> What are you looking to add?
<wallyworld_> person . irc nick
<wgrant> That's a bit harder.
<wgrant> Since that's in a separate table.
<wallyworld_> of course. silly me
<wgrant> Is the join too expensive?
<wgrant> There are not many IRC nicks.
<wgrant> emailaddress is the one that needs optimisation, if anything odes.
<jtv> Does that need FTI?
<jtv> I mean, IRC nicks?
<wallyworld_> i'm just thinking out loud
<wallyworld_> jtv: no, i said something stuoid without thinking
<jtv> no worries then â we all do that
<jtv> Who's up for a JS client-server pre-imp call?
<NCommander> StevenK: http://paste.ubuntu.com/613055/ (just realized I forgot to put the paste link)
<StevenK> NCommander: Indeed you did
<StevenK> NCommander: Making a BPR should make builds for you
<wallyworld_> jtv: what are you looking to do?
<jtv> wallyworld_: just otp back in a sec
<wallyworld_> ack
<StevenK> NCommander: It looks good -- your imports are all over the place, so I'd suggest running utilities/format-imports over it. You have some spelling errors, and every comment should be a complete sentence -- you're missing some full stops.
<StevenK> NCommander: Eight lines of self.assertEquals is mind-numbling, can I suggest a for loop?
<StevenK> wgrant: ^
<wgrant> There may be a matcher for that.
<StevenK> NCommander: Oh, and you import pdb without using it
<wgrant> Or it may be in james_w's unlanded branch.
<james_w> yeah, I think that's in more-matchers
<james_w> land it! :-)
<jtv> wallyworld_: back.
<StevenK> NCommander: However, if that test passes and you have the publisher changes finalized, put up an MP!
<jtv> Here's what I want to do: you've got a page displaying a bunch of items with somewhat complicated state, and I want live updates for that state.
<StevenK> james_w: Link me the MP?
<wallyworld_> the page stays open for a while?
<jtv> Possibly, yes.
<wallyworld_> so you want a long poll which we don't currently do :-(
<jtv> Ideally, yes.
<jtv> But I have what I hope is a pretty efficient poll with some mitigation measures for the load problem.
<wallyworld_> i *think* that the option for now would be to set up a js timer and poll yourself
<jtv> Right.
<wallyworld_> every so often
<StevenK> NCommander: But I'd suggest making the changes I suggest first, since any reviewer is likely to pick up on the exact same points.
<james_w> https://code.launchpad.net/~james-w/launchpad/more-matchers/+merge/32057
<jtv> wallyworld_: and possibly less often for large numbers of objects.
<wallyworld_> and i think yuo has some apis which support that
<wallyworld_> yui
<lifeless> wallyworld_: wgrant: that feeds fix
<lifeless> wallyworld_: wgrant: I bet I can still make it oops
<wallyworld_> jtv: you can say "execute this emthod after x seconds"
<lifeless> without url hacking
<james_w> PublishedStateIs along with the matcher that loops over the input and you'd have a test that would make lifeless proud :-)
<jtv> wallyworld_: yes, I've seen that.
<wallyworld_> lifeless: fire away
<wgrant> lifeless: Probably.
<wgrant> I don't care about whether the fix is good.
<wgrant> I care about whether it breaks stuff.
<lifeless> wallyworld_: open two tabs on a public bug. In one make it private. In the other follow the rss link.
<wallyworld_> jtv: i think you'd just requeue the method after each invocation
<wgrant> lifeless: You failed.
<lifeless> wgrant: yes, its deployable.
<wgrant> lifeless: It redirects.
<wallyworld_> lifeless: i accounted for that case :-P
<jtv> wallyworld_: yesâthat avoids piling up overlapping poll attempts.
<lifeless> wallyworld_: excellent
<StevenK> james_w: I suspect that branch needs devel merged in
<wallyworld_> lifeless: you don't need two pages. just do an inline js edit of prvacy
<lifeless> wallyworld_: I only remembered the work around the link rel=...
<lifeless> so, cool.
<jtv> wallyworld_: I was thinking: render each item on the page with a "status cookie" that encodes everything about the item that might change and require it to be re-rendered.  To poll, the client sends its item ids with their cookies; the server responds with HTML and fresh cookies for any items that have changed.
<wallyworld_> lifeless: thanks for asking though. best to be thorough
<jtv> wallyworld_: The cookie would be computed in one place only, so nobody else needs to care what goes into it.
<lifeless> wallyworld_: indeed!
<wallyworld_> jtv: sounds like a good solution. could you use an etag like lazr restful?
<wallyworld_> jtv: that essentially is the same problem
<lifeless> sounds heinous http://www.theregister.co.uk/2011/05/25/intel_hybrid_cloud_appup_service/
<jtv> wallyworld_: I had no idea lazr.restful did etags!  Yes, that'd be an ideal place for the cookie to go.
<wallyworld_> jtv: i *think* lazr restful has js for computing the hash/cookie you could perhaps look to reuse too
<StevenK> james_w: No fair having 5 dependant branches deep
<lifeless> wallyworld_: jtv: etag can only talk about one object at a time
<lifeless> which drives lots of round trips
<jtv> wallyworld_: wouldn't that by nature have to be based on the HTML?
<lifeless> whats the problem with just doing the UI in js ?
<lifeless> if the user doesn't have js, they won't get updates anyway
<jtv> But they want to see the same items.
<wallyworld_> lifeless: i didn't realise that etag limitation
<lifeless> wallyworld_: etags apply to an entity from a url
<jtv> And I stupidly thought wallyworld_ was saying lazr.restful had it deeper down where you could do it per item.  :)
<jtv> So scratch etags.
<lifeless> jtv: the recipes UI does updates using the html representation
<wallyworld_> jtv: lifeless: i wasn't sure about how it was implemented. just pointing out it had them in some form in case they were useful for this, but not so
<wallyworld_> sorry :-)
 * lifeless doesn't understand the problem yet
<wallyworld_> lifeless: jtv wants a long poll type scenario to asynchronously update a page i think
<jtv> Yes, items on a page.
<lifeless> oh, I get that bit
<lifeless> I don't understand why the existing answers don't work here
<jtv> I asked earlier about existing answers.
<wallyworld_> we don't currently have anything like that atm in lp do we?
<lifeless> we do page updates in js all the time
<StevenK> wgrant: http://pastebin.ubuntu.com/613059/
<lifeless> pickers, subscriptions, recipes, bugs
<wallyworld_> lifeless: yes, but that;s in response to a user action
<lifeless> wallyworld_: so?
<wallyworld_> not a server side event
<wallyworld_> this is a different scenario afaiu
<lifeless> wallyworld_: fire a background poll, it notifies when an object is updated, call into the local js
<jtv> lifeless: that's not a very helpful description.
<wallyworld_> lifeless: yes, that's the solution that is being canvased
<StevenK> wgrant: A lot of the failures are due to tests tossing in StringIO for changes_file_object, which doesn't have a .name
<wgrant> StevenK: :(
<wallyworld_> we are discussing the implementaion options
<lifeless> wallyworld_: jtv: what I'm saying is that finding out about the change, and updating the UI in response, are largely separate problems
<jtv> Which is why I posed them as separate questions from the start.
<jtv> lifeless: You may want to read about 2 hours back, search for your name.
<jtv> We're not confused on that point.
<lifeless> ok
<wallyworld_> sure. we are more discussing the "finding out about the changes" bit i think
<StevenK> wgrant: And getattr(changes_file_object, 'name') makes me cry
<wallyworld_> i think jtv has what seems like a reasonable solution
<jtv> I was sketching out the solution I had in mind, based on the answers I got earlier about existing solutions.
<jtv> The answers were: no, we have nothing better than polling yet; and even though we generally prefer to manipulate the DOM directly for object changes, it is possible to request HTML from lazr.restful.
<lifeless> jtv: so, I'm confused again
<lifeless> you seem to be describing a sort of complex page-state-diff algorithm.
<jtv> Not particularly complex, when you get right down to it; just "send me any updates" but with a single roundtrip.
<lifeless> Whats the concrete use case here? Is it really 'any part of the page has changed', or is it 'show the status of jobs for this DSD' ?
<jtv> It's showing the status of a DSDâwhich may include the status of the DSD itself, but also different types of jobs that may be in progress for it.
<lifeless> some thoughts:
<lifeless>  - detecting a change generally involves as much as work as rendering the object again
<lifeless>   - the server knows what would be on the page so why would the client supply stuff to check?
<jtv> Why does the server know what would be on the page?
<lifeless> because the server renders pages
<jtv> It knows _when_ it renders the page.  But after that, it blissfully forgets.
<lifeless> yes, But it can redetermine that trivially as needed
<jtv> Excuse me?
<jtv> How can it do that?
<lifeless> render the page?
<jtv> That's a different page.
<jtv> It's got different DSDs on it.
<jtv> That's not the page the user is looking at.
<wallyworld_> if there are 100 clients connected at different times, how can the server know what's on each and every client? that's not scalable
<jtv> Exactly.
<lifeless> wallyworld_: so, in the future, ajax subscription does *exactly that*
<lifeless> wallyworld_: but thats not today.
<lifeless> jtv: why does it have different DSDs ?
<wallyworld_> even in that case though, the server would just push out new state for a domain object and it wouldn't know what's already been rendered
<jtv> Because it's a filtered view.  DSDs change status.
<wallyworld_> it would be up to each client to "do the right thing"
<jtv> A DSD gets resolved, it falls off the page.
<jtv> A package change may push a new DSD onto the page.
<wallyworld_> lifeless: storm doesn't support sql case statements does it?
<lifeless> wallyworld_: sure does
<wallyworld_> oh cool.
 * wallyworld_ looks for it
<lifeless> jtv: then perhaps just request that the dsds present get re-rendered ?
<NCommander> StevenK: I don't have changes for it yet, I got tied up withspecs so I only finished the test suite today to see if its sane, as I'm unsure if this is the proper solution. I kinda felt it mightbe clearly to maintain publishing for SUPERSEDED items ...
<jtv> lifeless: that's going to be very similar, except with unnecessary rendering and a bigger payload.  Where's the gain?
<StevenK> NCommander: Hopefully, the test fails, then?
<lifeless> jtv: its simpler
<lifeless> jtv: you could just return domain objects, but I got the impression you didn't want to do the ui in js
<jtv> That's right, I didn't want to do the UI in JS.
<lifeless> though domain objects are often *more* expensive to generate than rendering a page.
<jtv> I don't think rending all items takes away much complexity: instead of keeping track of {id: cookie} we'd need to keep track of [id], and that's about all I see changing.
<NCommander> StevenK: indeed since everything expect the i386 binary should still be PUBLISHED
<jtv> *rendering
<wallyworld_> lifeless: sorry. i can't see any native support. i know i can inject the required sql into the find(), i was looking for a built in Expr subclass
<StevenK> wgrant: The first 14 errors fixed
<wgrant> StevenK: That was easy.
<StevenK> 15
<StevenK> test_realiseUpload_for_delayed_copies is still broken
<wgrant> :(
<wgrant> What is its complaint?
<lifeless> jtv: it means you don't need a cookie, generation of which is usually expensive [in all similar systems I've sen]
<StevenK> wgrant: It was expecting an annoucement mail
<wgrant> Hmm
<StevenK> wgrant: Let me commit and push
<lifeless> wallyworld_: hmm, sorry :(. Would you like some pointers on adding a case statement expr ?
<StevenK> % bzr di | wc -l
<StevenK> 413
<StevenK> I am such a bad person
<jtv> lifeless: even if generating the cookie is as expensive as rendering an item, big win.
<wallyworld_> lifeless: i should be ok to do that. i'm not sure if i need it. just experimenting at the moment
<jtv> lifeless: the cookies can be generated in batch.
<lifeless> jtv: rendering can be done in a batch too?
<jtv> Are you asking or saying?
<lifeless> both I guess
<jtv> What I'm saying is, I don't see why generating the cookie and in most cases doing no rendering at all and sending basically no data across the wire is going to be anywhere near as expensive as re-rendering all items on the page and sending them to the client.
<jtv> Also, I think saw IE running JS once.
<lifeless> i/win 21
<jtv> There's no longer a monitor attached to that computer, so I don't know how well it does.  But not replacing items on a page that don't need touching may also be a plus.
<lifeless> so, I've fixed something like 10+ timeouts related to generating API objects now
<jtv> How does that follow?
<lifeless> the API definition for an object includes grabbing *all* the adacent objects and calculating their urls
<jtv> Good point: another reason to do it the way I had in mind I guess.
<lifeless> the reason I suggested rendering is that rendering is *targeted*
<jtv> They're going to have to do something with the internet here, so I'll be offline for a bit.
<lifeless> k
<lifeless> pick this up in a bit
<wgrant> Hm.
<wgrant> Did someone break PQM?
<wgrant> Nobody fixed the conflict, but it has stopped complaining.
<wgrant> ah, buildbot failure.
<wgrant> "Set changed size during iteration"
<wgrant> WYay
<wgrant> wallyworld_: Found any remarkable insights from your ranking experiments?
<wallyworld_> wgrant: there is a relatively simple change that will improve the ordering of results while not changing the query too much. the query is quite complex so i'm loath to mess with it too much. i'm also experimenting a bit with a few other things
<wgrant> wallyworld_: You mean actually using the rank instead of throwing it away and ordering by displayname?
<StevenK> wgrant: The diff should be updated.
<wallyworld_> wgrant: yes, in a nutshell
<wgrant> wallyworld_: Good luck doing that while not changing the query too much :(
<wallyworld_> exact matches to name (lp id) > exact matches to displayname > exact matches to irc nick > partial matches to email
<wgrant> What's an exact match to displayname?
<wgrant> as in a full string match?
<wallyworld_> wgrant: i've got a solution which i expect wiol work but i'm messing with other things before i implement
<wallyworld_> yes.
<wgrant> wallyworld_: We also need to consider affiliation in ranking, or maybe just filtering... and I'm not sure how trusted affiliation wants to be.
<wgrant> Hmm.
<wgrant> I'd prefer an FTI rank over a string match.
<wgrant> Otherwise you have to get it just right.
<wallyworld_> wgrant: we discussed on the call this morning not doing affilliation in rankings first up
<lifeless> if you're checking displayname be sure to add a db index for it
<wallyworld_> will do
<wallyworld_> wgrant: wrt the fti match vs string match. the fti match is still being done too. but if the user does type in an exact match, that should take precedence
<lifeless> trust fti to do that
<wgrant> wallyworld_: But the FTI rank is not taken into account.
<wgrant> Right.
<wgrant> If we take FTI rank into account, we'll get full matches and partial matches, in the right order.
<wallyworld_> lifeless: the fti will do it but the ranking will be off
<wallyworld_> we want exact matches ranked higher
<stub> lifeless: "BugTask.sourcepackagename = ss4.sourcepackagename OR ss4.sourcepackagename IS NULL" is better written as "BugTask.sourcepackagename IS NOT DISTINCT FROM ss4.sourcepackagename"
<lifeless> stub: doh of course... thanks
<wgrant> wallyworld_: FTI rank should do that.
<wallyworld_> wgrant: not when fti rank is 0..1 and we use 100 for exact string match on name and 10 for irc nick
<wgrant> wallyworld_: Better to tweak those than work around FTI, surely.
<wallyworld_> i could look at adjusting the other raqnking values currently in use
<wallyworld_> but there must have been a reason for doing it the way it is
<wgrant> You've been around Launchpad for long enough to know that's not necessarily true :)
<wallyworld_> does our local db on our laptops have fti enabled?
<wgrant> Yes.
<wallyworld_> cool.
<wgrant> launchpad_dogfood=# SELECT Person.name, Person.displayname FROM person ORDER BY rank(fti, ftq('william grant')) DESC LIMIT 10;
<wgrant>       name       |         displayname
<wgrant> -----------------+------------------------------
<wgrant>  me-williamgrant | William Grant
<wgrant>  wgrant          | William Grant
<wgrant>  me+testing      | William Grant (test account)
<wgrant>  shuilongzi      | Basil Bakhtin
<wgrant> We just have to balance them with exact name matches.
<wgrant> Those results aren't bad.
<wgrant> And IRC nick matches.
<wallyworld_> wgrant: wrt the exidting ranking. i realise that it may not be "correct" but i do want to try and understand why it was done the way it was in case there's some corner casse or something i'm missing
<stub> Those constants were just a guess that have never been tweaked IIRC
<stub> Just there to get all exact matches above fti fuzzy matches
<wallyworld_> stub: makes sense. wonder why if fti ranks from 0..1 that string matches used 100 though
<wallyworld_> if fti gives a ranking of 1 for exact match, then wou;dn't we have used 1 for for explicit exact matches?
<stub> wallyworld_: Exact match in a particular field. fti match does stemming, and searches other fields too probably, but will give the same score for 'wgrant' as it would 'wgrants'
<stub> And miss you entirely if your name happens to be a stopword.
<stub> Try a search for 'Ng' for instance...
<stub> fti is setup for english word searches, and names don't really fit that.
<wallyworld_> ok. thank for the explanation. sounds like we need to explicitly do any exact string matches we require. and there's none for displayanme yet
 * wallyworld_ has to go to soccer training. back later
<stub> wallyworld_: We likely need an index created to do exact matches fast and case insensitive. Are exact matches on name useful though? Or do you really want a substring match (slow without further work...)
<wallyworld_> stub: i think they are useful. if you know you want to find "Joe Bloggs" you should be able to type that. you may not know his lp nick or email
<wallyworld_> i don't think we need substring. fti will cover that? just an index for exact match
<wallyworld_> for displayname
<wallyworld_> i'm assuming there is already an index for person.name
 * wallyworld_ really has to go now
<wgrant> stub: Oh yeah, person.fti doesn't have an index on production.
<wgrant> stub: We discovered this yesterday.
<StevenK> wgrant: New list: http://pastebin.ubuntu.com/613073/
<wgrant> StevenK: What's up with soyuz-set-of-uploads?
<StevenK> wgrant: distroseriesqueue.txt
<StevenK> Sigh
<StevenK> wgrant: distroseriesqueue.txt is complaining that spr.name is None for an announcement mail
<wgrant> :(
<wgrant> spr.name is None, or spr is None?
<StevenK> The latter, sorry
<wgrant> Right.
<wgrant> We probably have to infer it from the BPR or something.
<StevenK> So, if spr: name = spr.name, if bprs: bprs....
<StevenK> wgrant: The soyuz-set-of-uploads failure is odd: http://pastebin.ubuntu.com/613074/
<wgrant> StevenK: Why's it not emailing name16 any more? :(
<StevenK> It should be?
<wgrant> It was before.
<StevenK> wgrant: distroseriesqueue.txt fixed
<StevenK> Ah, it's a bug in notify_spr_less
<stub> wgrant: whoops
<wgrant> StevenK: Your diff is now Full HD.
 * stub wonders how that happened
<wgrant> stub: Rather...
<StevenK> wgrant: Oh?
<wgrant> stub: It's all of 50MB when freshly created on DF.
<wgrant> StevenK: It is 1080 lines.
<wgrant> Getting a little on the large side,...
<lifeless> \o/ I feel validated, I fixed a bug this week
<stub> fti.py is supposed to maintain all that
<StevenK> wgrant: Haha
<lifeless> StevenK: yo
<lifeless> bah
<lifeless> stub: yo
<StevenK> Ha
<stub> lifeless: yo
<lifeless> call?
<stub> sure
<lifeless> sky.net?
<wgrant> 2011-05-26 05:57:06 INFO    2208-62-0 applied just now in 0.3 seconds
<wgrant> Success!
<poolie> would it be too snarky to edit "Most developers will be happy with [[API/launchpadlib|launchpadlib]],"? :}
<lifeless> poolie: ECONTEXT
<StevenK> wgrant: 62 is the patch o' doom?
<wgrant> Yes.
<poolie> the help.l.n/API page
<poolie> i was actually going there to add something more objectively useful
<poolie> which is the tip about string parameters being encoded as json strings
<lifeless> stub: ohhai?
<lifeless> zing!
<StevenK> lifeless: I think that's your answer ...
<StevenK> wgrant: soyuz-set-of-uploads fixed
<StevenK> It's still horridly slow
<LPCIBot> Project db-devel build #581: FAILURE in 7 min 6 sec: https://lpci.wedontsleep.org/job/db-devel/581/
<StevenK> Hmmm
<StevenK> bzr: ERROR: http://bazaar.launchpad.net/~launchpad-pqm/launchpad/db-devel/.bzr/repository/packs/c953fdd6233a6989afbd047b37b58c4b.pack is redirected to https://launchpad.net
<StevenK> Again
<wgrant> :(
<wgrant> StevenK: How'd you fix it?
<StevenK> wgrant: soyuz-set-of-uploads? It wasn't even checking the blamer.
<wgrant> Ah
<wgrant> That might do it.
<StevenK> [16:21] < StevenK> Ah, it's a bug in notify_spr_less
<wgrant> 191 open critical bugs in launchpad :)
<StevenK> Then fix the topic? :-)
<wgrant> launchpad-project is still 209 :(
* wgrant changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: - | https://code.launchpad.net/launchpad-project/+activereviews | Critical bugs:209 - 0:[######=_]:256
<lifeless> stub: https://dev.launchpad.net/ArchitectureGuide/Services
<wgrant> StevenK: What's still failing?
<StevenK> wgrant: Everything else on that list aside from the two I've mentioned.
<jtv> lifeless: we were discussing that DSD page update thingy.
<StevenK> wgrant: Are you still looking it over, or do you want to fly to Sydney to kick me in the teeth?
<wgrant> StevenK: A hybrid.
<StevenK> wgrant: Oh, so you're still looking it over *and* you want to fly to Sydney to kick me?
<wgrant> Right.
<lifeless> jtv: hi, yes we were.
<lifeless> jtv: I have an email I've been trying to get to for a day now, which I want to finish as a priority
<jtv> OK
<lifeless> jtv: can we raincheck for a little more?
<jtv> sure
<StevenK> wgrant: distroseriesqueue-dist-upgrader.txt fixed. With a one-line patch. :-/
<wgrant> StevenK: Oh?
<wgrant> How?
<wgrant> I expected it to be harder.
<StevenK> wgrant: Adding 'MAINTAINER': '', to the information dict
<wgrant> Heh
<StevenK> It fixed distroseriesqueue-debian-installer.txt too
<wgrant> And translations?
<wgrant> Or ddtp or whatever it's called.
<StevenK> ddtp-tarball looks like whitespace changes
<StevenK> wgrant: ddtp-tarball fixed with http://pastebin.ubuntu.com/613092/
<wgrant> Ahh, - is the default version, I guess.
<StevenK> Ah
<wgrant> You might want to fix that by s/''/'-'/ instead.
<StevenK> I can fix that
<wgrant> I was going to chastise you for using '' anyway.
<StevenK> wgrant: That fixed it, good catch.
<wgrant> poolie: Bug #778437 is already released.
<_mup_> Bug #778437: Recipe build success sends emails, please stop doing that <email> <qa-ok> <recipe> <Launchpad itself:Fix Released by mbp> < https://launchpad.net/bugs/778437 >
<wgrant> poolie: Does it not seem to be?
<wgrant> StevenK: Does that change dist-upgrade/debian-installer at all?
<wgrant> Is test_realiseUpload_for_delayed_copies still broken?
<wgrant> And what about xx-queue-pages.txt?
<StevenK> wgrant: Both still pass with the '-' change -- perhaps they don't check the subject.
<wgrant> Hm.
<wgrant> Would not surprise me :(
<StevenK> Haha
<StevenK> I believe test_realiseUpload_for_delayed_copies is still broken
<StevenK> xx-queue-pages I'm not sure about
<StevenK> wgrant: I think that gets us to test_realiseUpload_for_delayed_copies, distroseriesqueue-notify.txt and xx-queue-pages.txt as broken
<wgrant> StevenK: Ah, yeah, look at PackageUpload.displayversion.
<wgrant> Uses source version, then binary version, then -.
<StevenK> Ah yes
<StevenK> wgrant: Which of the three do you want me to dig into next?
<wgrant> StevenK: Whichever has the least obscure error message.
<poolie> wgrant: ooh, that's great
<wgrant> Which may well be test_realiseUpload_for_delayed_copies, but possibly not because screw delayed copies.
<poolie> i have the mail filtered
<wgrant> Ah, heh.
<StevenK> wgrant: Haha
<wgrant> poolie: But yeah, it was released like 7 hours after it landed.
<wgrant> Where by 7 I mean 9.
<wgrant> Because I can't count.
<StevenK> wgrant: xx-queue-pages: http://pastebin.ubuntu.com/613096/
<poolie> i don't have a totally reliable model of which bug state and which branch landing corresponds to 'actually running'
<poolie> wgrant: so you qa'd it? thanks
<wgrant> poolie: Fix Released means "actually running", unless somebody makes a mistake.
<StevenK> wgrant: The bottom one is fixable, once I figure out where that comment is getting injected
<wgrant> And it's normally lifeless or I, and we are fairly good at that these days.
<StevenK> wgrant: The top one concerns me
<poolie> aj
<poolie> i mean ok
<wgrant> StevenK: The top one is concerning mostly because more tests did not fail.
<adeuring> good morning
<wgrant> StevenK: It's a reasonable change. But more should have broken.
<StevenK> wgrant: Beh, the bottom one is because the older code set "Rejected by archive administrator." as the reason if there was none.
<LPCIBot> Project windmill-db-devel build #326: STILL FAILING in 1 hr 7 min: https://lpci.wedontsleep.org/job/windmill-db-devel/326/
<wgrant> StevenK: Ah. You should probably do so too.
<wgrant> StevenK: Eventually that will be gone, but let's preserve behaviour for now.
<wgrant> Also, wow our tests are terrible.
<poolie> night all
<wgrant> Night poolie.
<StevenK> wgrant: Fixed that -- should I correct the subject in the doctest?
<wgrant> StevenK: I'm not sure. Why did it change?
<wgrant> Did none of the other tests of the ".* rejected" subject change?
<wgrant> It seems odd that a single test found it.
<StevenK> wgrant: So, rejected messages are interesting -- notify_spr_less() will set the subject as "<changesfile> rejected", but notify() will set it as in calculate_subject()
<wgrant> Right.
<wgrant> That's what I thought.
<wgrant> Sounds reasonable.
<wgrant> For now, at least.
<wgrant> So change the test.
<StevenK> And interestingly, the test is rejecting alsa-utils
<StevenK> But the test is coded to expect the subject is 'netapplet-1.0.0.tar.gz rejected'
<StevenK> So, fail.
<wgrant> Possibly sampledata fail.
<wgrant> Wait.
<wgrant> *tar.gz* rejected?
<wgrant> How does that make sense :(
<StevenK> It doesn't!
<StevenK> xx-queue-pages.txt fixed
<StevenK> wgrant: distroseriesqueue-notify.txt suffers from the same brain damage
<wgrant> StevenK: Yay sampledata.
<StevenK> wgrant: -notify sorted out, that leaves test_realiseUpload_for_delayed_copies
<StevenK> wgrant: Which is strange. It seems to only want an annoucement mail and no other
<wgrant> StevenK: That's right.
<wgrant> StevenK: Since there is no requester.
<StevenK> And I don't want to put a delayed copy hack into notification
<wgrant> StevenK: Notifying the signer is probably wrong.
<wgrant> Check what the old code did.
<wgrant> Are you still using spr.packageupload.signingkey?
<StevenK> Yes
<wgrant> That's probably it.
<StevenK> Er, no
<wgrant> :(
<wgrant> You're using blamer now?
<StevenK> Everything is
<wgrant> OK.
<wgrant> So whatever is passing in blamer might be wrong.
<wgrant> this is just a guess; I may be entirely wrong here.
<StevenK> PU.notify() sets it to self.signing_key.owner if signing_key isn't None
<wgrant> Hmm. So it should be None for delayed copies still.
<StevenK> Let's find out
<wgrant> Work out why it's sending the extra email.
<wgrant> Then work out why it wasn't sent before.
<StevenK> It's a unit test, so I can trace easier
<wgrant> doctests rule the world!
 * StevenK hangs himself
<StevenK> wgrant: (Pdb) p self.signing_key
<StevenK> <GPGKey at 0xf332950>
<StevenK> :-(
<wgrant> StevenK: .. on a delayedcopy?
<StevenK> (Pdb) p self.is_delayed_copy
<StevenK> True
<mrevell> Morning
<StevenK> wgrant: So, yes
<wgrant> StevenK: I don't think that's valid.
<wgrant> How does it create the delayedcopy?
<StevenK> _autotest.distribution)
<StevenK> Rargh
<wgrant> And can you check the code that creates them in reality?
<wgrant> To see if it sets it?
<wgrant> Also, brb.
<StevenK> wgrant: The test passes in self.test_publisher.person.gpg_keys[0] explicity
<StevenK> *explicitly
<bigjools> StevenK: mumble
<StevenK> wgrant: Right, looks like the package copier sets the key to None, but a lot of tests set it to a key
<wgrant> StevenK: Kill them.
<wgrant> StevenK: Actually, probably best to work out why it worked before.
<StevenK> wgrant: I have no idea :-(
<wgrant> StevenK: You'd best work that out :/
<bigjools> wgrant: don't we have code that stops account merges when someone has a PPA?
<wgrant> bigjools: Yes.
<wgrant> bigjools: It worked when I tried it last week, too.
<wgrant> NFI what happened there.
<bigjools> :/
<bigjools> there will be an orphan repo
<bigjools> which is a bit of a disaster if a new on gets created
<wgrant> It's merged on DF.
<wgrant> Let's see...
<wgrant> It's disabled...
<wgrant> status == 2
<wgrant> The PPA is deleted.
<wgrant> So the merge was OK.
<bigjools> ah
<wgrant> But the sub link shouldn't be shown.
<bigjools> ok
<StevenK> wgrant: Fixed, I think.
<wgrant> StevenK: How?
<StevenK> wgrant: Was a bug in notify()
<wgrant> bigjools: Can you tell sladen so I can pretend to nearly keep to core hours? :)
<bigjools> yeah PMing him now
<StevenK> wgrant: The old code said "If pocket is SECURITY and there is no source", my code said "If pocket is SECURITY and there is source" ...
<wgrant> StevenK: aaaaaaaa
<wgrant> Pocket-specific hacks make me cry.
<wgrant> And/or murderous.
* allenap changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: allenap | https://code.launchpad.net/launchpad-project/+activereviews | Critical bugs:209 - 0:[######=_]:256
<wgrant> But yes, I know the one you mean.
<jtv> ooh allenap, I have one for you!
<allenap> jtv: Send it over :)
<jtv> As soon as the #$%*&@ page will load.
<jtv> allenap: https://code.launchpad.net/~jtv/launchpad/db-bug-786970/+merge/62292
<LPCIBot> Project windmill-devel build #141: STILL FAILING in 1 hr 12 min: https://lpci.wedontsleep.org/job/windmill-devel/141/
<jtv> Hmm it's still showing the conflicts.  But that should update any moment now.
<StevenK> wgrant: So I think all tests pass now.
<wgrant> StevenK: -vvt soyuz -t archiveuploader -t archivepublisher
<StevenK> wgrant: I don't really want to tie up my machine for 50 minutes
<wgrant> Yeah.
<wgrant> ec2 it.
<bigjools> and the rest
<StevenK> bigjools: Speak for yourself, my machine runs -vmm soyuz in 29 minutes.
<StevenK> -vvm, even
<bigjools> StevenK: and archivepublisher, archiveuploader, builddmaster ? :)
<StevenK> wgrant didn't say buildmaster
<wgrant> buildmaster's not so relevant here.
<bigjools> it'd be a good idea to run it though
<wgrant> But still useful.
<jtv> bigjools: should the UI allow the user to request a sync for a DSD that also has a DSDJob pending?  Probably not what you _want_, but don't know whether we should be difficult about it.
<bigjools> it generates emails
<wgrant> There may be one or two relevant tests.
<StevenK> Anyway, ec2 is off
<jtv> allenap: my diff's updated.  Conflicts are gone.
<bigjools> jtv: it's a valid thing to do so don't prevent it
<allenap> jtv: Cool. I've claimed it.
<StevenK> wgrant: If it passes ec2, what's the next step?
<jtv> thanks allenap
<bigjools> jtv: which of course means there may be more than one DSDJ for the package
<wgrant> StevenK: I will rereview it first thing tomorrow morning.
<jtv> bigjools: yes, makes sense (otherwise parent updates could become a DoS condition :) though it introduces a slight asymmetry in the logic.
<wgrant> StevenK: And attempt to restrain myself from asking you to split it.
<StevenK> Hah
 * bigjools starts looking at what info +queue and the queue tool need out of PackageCopyJob to work
<wgrant> bigjools: Surely you wouldn't be insinuating that we don't have a nice clear, simple interface between the queue UI and the data model :)
<bigjools> :/
<bigjools> the queue page is already a nightmare.  this will make it a zombie nightmare
<StevenK> And pitti was asking us yesterday to make it quicker
<StevenK> :-)
<wgrant> At least delayed copies are nearing their end.
<bigjools> yay
<bigjools> PCJ is the new delayed copy :)
<wgrant> Except slightly less ugly.
 * StevenK shoots self, allenap 
<jml> Do we have a policy template (distinct from the process template, which I know we have)
<jml> StevenK: tch tch. such violence is unnecessary.
<allenap> StevenK: Why?
<StevenK> allenap: [19:20] < bigjools> PCJ is the new delayed copy :) And PCJs are your fault :-)
<StevenK> And delayed copies make me want to hurt people.
<wgrant> bigjools: One day we will properly unify them.
<wgrant> bigjools: But until then PCJ is a much less bad hack.
 * bigjools is not responsible for delayed copies!
<wgrant> I don't appreciate your lies.
<allenap> StevenK: I blame Jelmer :)
<bigjools> a certain Brazilian did them
<StevenK> allenap: Actually, it's my fault! I told you how to do it quickly!
<wgrant> I can understand why they were done, given time pressures and reuploading... but doesn't make me like them any more.
<allenap> StevenK: Ah ha, that's why you were proposing to shoot yourself!
<StevenK> I've been reading delayed copies code today, I'm suicidal
<wgrant> StevenK: It's OK to think about delayed copies if you are doing it to remove them.
<StevenK> Bleh, branch is now 1116 :-(
<StevenK> It still removes more than it adds back
<wgrant> At least you've done what LCD manufacturers have failed to do.
<StevenK> What's that?
<wgrant> Exceed the 1080p you were stuck at for a while.
<StevenK> "lol"
 * bigjools strokes his 1920x1200 24-incher
 * StevenK stabs his eyes
<StevenK> I've forgotten how to check resolution
<wgrant> xrandr
<StevenK> But I think this LCD is only 22 inches
<StevenK> 1920x1080
<bigjools> wgrant: so, the plan was to create a PU at the same time as a PCJ, right?
<wgrant> bigjools: I think that is best for now.
 * allenap gingerly pets his 2560x1600 30-incher.
<bigjools> or did we talk about making the PCJ create the PU?
<wgrant> allenap: !!
<wgrant> allenap: I want one.
 * bigjools drives to allenap's house with a lock pick and a shotgun
<wgrant> 24" @ 1920x1080 is almost painfully low DPI.
<wgrant> bigjools: How are PCJs normally created now?
<StevenK> allenap: Which make and model?
<bigjools> wgrant: in the browser code
<StevenK> Hm, I wonder if X can spit out the EDID information
<allenap> It's an Apple thingy, I'm not sure they sell 30" models any more. It's 4 years old.
<wgrant> StevenK: xrandr --prop
<StevenK> You said the magic A word that makes me lose interest.
<allenap> Hehe :)
<wgrant> bigjools: And the real permission checks are done at job execution time?
<StevenK> wgrant: EDID isn't the make of model of the display?
<bigjools> wgrant: yeah it would have to, as a backstop
<wgrant> StevenK: It contains that information, plus the supported modes.
<wgrant> bigjools: This is going to be ugly for a while :(
<wgrant> Not sure how to do this efficiently...
<bigjools> wgrant: s/for a while/
<wgrant> bigjools: It can be cleaned up as tech-debt eventually.
<wgrant> EVENTUALLY
<wgrant> This is some tech-debt I am happy to tolerate for now, though.
<wgrant> bigjools: We are moving to one PCJ per package, right?
<bigjools> the form button is intelligently presented depending on whether you have permission but there's nothing to prevent url hacking to POST something it seems
<wgrant> Right.
<wgrant> That's sensible enough.
<bigjools> wgrant: yes, one per package to make error reporting easier
<bigjools> also so that mass-sync completes :)
<wgrant> Yeah.
<wgrant> So.
<wgrant> What we want, I guess, is a quick bit of hackery to work out whether to create an UNAPPROVED or DONE packageupload.
<wgrant> Since the autoapproval logic will have to be batched.
<wgrant> And is currently lurking in the depths of Archiveuploader the Great.
<bigjools> I already did the policy stuff
<bigjools> just need to hook it into the PCJ
<wgrant> Does it work?
<bigjools> ...
<bigjools> it's just the policy object
<bigjools> it's not *used* yet
<wgrant> Yes, but that doesn't mean it is completely working.
<wgrant> eg. overrides aren't *quite* done yet.
<bigjools> they're not done at all, until something uses the code in PCJ
<bigjools> which sort of tells me that we probably need to create the PU in the PCJ
<wgrant> The underlying logic is incompletely implemented.
<bigjools> what logic?
<wgrant> The override policies.
<wgrant> No multiverse override, no way to handle newness, etc.
<wgrant> Hee hee newness.
<wgrant> We still have absolutely no way to handle that.
<wgrant> But we'll see.
<wgrant> And/or die.
<jml> wgrant: newness is actually a perfectly cromulent word
<wgrant> jml: It's also a good way to describe the demise of several of the grand plans.
<bigjools> the newness has to come from the ancestry check
<wgrant> But that is yet to be seen.
<wgrant> bigjools: Sure, that's how we identify it.
<wgrant> But we have no way to resolve it.
<bigjools> that can now be hooked into the PCJ
<bigjools> eh?
<wgrant> You know how queue overrides are done, right?
<bigjools> ...
<wgrant> Queue overrides are required for NEWing.
<wgrant> And queue overrides involve mutating the SPR and BPRs.
<jtv> Grrr how do I get privileges to see +localpackagediffs on my local machine again?
<henninge> Can somebody give me some help on vocabulary lookup?
<wgrant> This doesn't work quite so well in a copying world.
<bigjools> since I wrote a lot of the original code I do have some idea, yes
<wgrant> jtv: The FF name changed yesterday. Check /+feature-info for hints.
<bigjools> no, we don't ever, ever mutate SPR or BPR
<wgrant> bigjools: orly.
<wgrant> Let me find it for you.
<bigjools> if it's happening somewhere it's a bug
<wgrant> No, it's just terribly ufly.
<wgrant> ugly.
<bigjools> I changed some code once that did that, so it creates the publication properly
<wgrant> lololol
<wgrant> bigjools: PackageUpload.overrideSource.
<wgrant> Calls SourcePackageRelease.override.
<wgrant> Which you may not want to view without an empty stomach.
<jtv> wgrant: ah great, somebody did something about the "-" issue
<henninge> I have a lookup code like this in InlineEditPickerWidget: http://paste.ubuntu.com/613120/
<henninge> The exported field is an "owner" from IHasOwner. Where is that vocabulary defined?
<bigjools> wgrant: too late
<wgrant> henninge: That's not a vocab.
<wgrant> henninge: You'd normally use a variant of ValidPersonOrTeam for that.
<henninge> wgrant: I don't understand. What is not a vocabulary?
<wgrant> henninge: IHasOwner.
<wgrant> Or owner, even.
<bigjools> wgrant: so those overrides are only manual overrides
<henninge> FTR, this is not code I wrode, I am just trying to understand existing code.
<henninge> s/wrode/wrote/
<henninge> wgrant: The widget gets instantiated like this: http://paste.ubuntu.com/613126/
<henninge> wgrant: ISourcePackageRecipe inherits from IHasOwner
<henninge> it does not define anything else for owner
<henninge> wgrant: never mind, there it is in the interface
 * henninge was blind again
<wgrant> bigjools: Right, but you need manual overrides when you're NEWing.
<bigjools> wgrant: we can make the PCJ code DTRT for non-NEW stuff as it won't hit the queue, but anything that is held and overridden will be a problem :(
<henninge> UserTeamsParticipationPlusSelf
<bigjools> wgrant: this is a nightmare.
<wgrant> bigjools: Yes, that's what I said :)
<wgrant> The demise of several of the grand plans.
<bigjools> wgrant: I said it was a zombie nightmare :)
<bigjools> we'll have to have different logic in the manual overrides if it's a PCJ
<wgrant> Yes, but we have nowhere to put them.
<bigjools> good point.
<bigjools> shit
<wgrant> Rather.
<bigjools> we could dupe the SPR
<wgrant> You're a bad person.
<bigjools> :)
<bigjools> it's no different to what happens when someone uploads manually
<bigjools> but with no publishing records, wtf else can we do
<wgrant> We'd have to store JSON overrides on the PCJ or something.
<bigjools> hmmm
<lifeless> did you just tell data integrity to go f**k itself bob?
<bigjools> that means fixing process-accepted
<wgrant> bigjools: Why is p-a affected?
<wgrant> lifeless: Yes.
<wgrant> lifeless: The proper fix for this is too big to consider now.
<bigjools> because it creates publications, it'd need to read the json blob with the override
<wgrant> lifeless: But it's not that hard for one person to retrofit in a week or so.
<wgrant> I don't think.
<wgrant> bigjools: No... PCJ creates the publications.
<wgrant> bigjools: This is not delayed copies.
<wgrant> bigjools: Your mind is tainted.
<bigjools> not really
<wgrant> p-a only creates publications for builds and delayed copies.
<wgrant> PCJ will create these publications.
<bigjools> ok so this is easy, as I first though.  You're the one dropping poison in my head
<bigjools> thought*
<wgrant> You still need some new way to store overrides on PCJ.
<bigjools> nope
<wgrant> Oh?
<wgrant> How do you propose to do it?
<henninge> lifeless: ping
<lifeless> henninge: hi
<henninge> lifeless: Hi, I'd like to discuss my options for fixing bug 778129
<_mup_> Bug #778129: cannot change owner for recipe - picker shows 'undefined' <regression> <Launchpad itself:In Progress by henninge> < https://launchpad.net/bugs/778129 >
<lifeless> shoot
<henninge> lifeless: it seems that only few people are affected, namely those that have a lot of team participations.
<henninge> The simplest solution would be to increase the number of batches
<henninge> like from 20 to 30.
<lifeless> I thought the problem was that 'big' was being assessed in two different ways
<lifeless> like, there is one metric that defines 'huge'
<henninge> The vocoabulary for this does not provide IHugeVocabulary and therefore no search box is displayed.
<lifeless> and another that defines 'too many batches'
<henninge> right, "huge" is by providing the interface
<lifeless> so, can we provide the interface?
<henninge> "too many batches" is a fixed literal in the js
<henninge> we could but then everybody gets it, even if they are only in three teams ...
<henninge> I mean , the search box.
<lifeless> so
<lifeless> perhaps a short + long term fix.
<henninge> The most complex solution would be to first peek how many entries there are and only present the searchbox if it exeeds a limit.
<henninge> yes, so my short term fix would be to increase the number of batches
<lifeless> how many items per page ?
<henninge> 6
<henninge> i could increase that instead
<lifeless> we need to show 230 items
<lifeless> 38 pages
<henninge> why 230?
<lifeless> cody
<henninge> ;-)
<henninge> ok
<lifeless> thats without a safety margin
<lifeless> one thing I don't understand
<lifeless> why don't we grab the first page of the vocab, *not* now a total batch, link to 'next' (not 2, 3, 4 etc) and if the first page of the vocab is filled up offer the serch box
<lifeless> that seems to me like it would scale low (< 7 items would show a simple page), scale medium (click next a couple of times) and scale indefinitely (folk can search)
<henninge> that would be my long-term solution.
<lifeless> I don't know how much work this would be
<henninge> The vocab needs to provide IHugeVocabulary because that gives search capability.
<lifeless> yes, thats pretty easy
<henninge> I know, just saying
<lifeless> yeah, agreed.
<lifeless> anyhow
<henninge> lifeless: I will fix this bug with the short-term solution and file a bug for the complex solution.
<lifeless> I have no particular view on whether we should invest in the long term solution right now.
<lifeless> I do think that the inability to use the form is the crucial thing to address
<henninge> I think it may also tie in with the broken search that was discussed
<henninge> right
<henninge> any form that uses this vocab
<lifeless> its a little odd that the vocab is restricted as it is
<lifeless> we used to be able to hand off objects to any user
<lifeless> but perhaps I'm out of date
<lifeless> certainly teams one is directly related too is a common case
<henninge> yes, I think so
<lifeless> thinking slightly bigger picture showing just some sensible teams and allowing a search of either 'my other teams' or 'all people in launchpad'
<lifeless> would let things be more flexible
<lifeless> e.g. the way I found this bug I was handing a recipe off to a new maintainer
<lifeless> there was no expectation that there be a team in common
<henninge> so this really should just be "any team or person"
<henninge> ?
<lifeless> so another way you could fix this is to change the vocab
<henninge> good point
<lifeless> I would consult with sinzui, as master of all abandonded objects
<wgrant> In this case it's critical that it be limited to teams of which you are a member.
<wgrant> It's used for privilege checking for daily builds.
<lifeless> wgrant: how so?
<lifeless> huh?
<wgrant> lifeless: The requester of a daily build is the recipe owner.
<wgrant> The requester is used for privilege checking on the archive.
<lifeless> do you mean 'the only valid owners of the recipe are folk that can upload to the archive' ?
<wgrant> Launchpad impersonates the recipe owner when scheduling daily builds.
<wgrant> So setting the recipe owner is delegating impersonation rights.
<wgrant> So it must only be done by a member of the team selected.
<lifeless> hang on
<lifeless> the dots aren't joining
<lifeless> I get why we can't use anyone
<StevenK> wgrant: archiveuploader tests are broken :-(
<wgrant> StevenK: TYS
<wgrant> StevenK: But howso?
<lifeless> but both henninge and I can upload to the lp ppa
<lifeless> why can't I make a recipe and hand it over to him ?
<wgrant> lifeless: Because that would be impersonating him.
<StevenK> wgrant: Wrong number of recipents
<wgrant> StevenK: Yay
<StevenK> reference = ['Foo Bar <foo.bar@canonical.com>', 'Foo Bar <foo.bar@canonical.com>']
<StevenK> actual = ['Foo Bar <foo.bar@canonical.com>']
<wgrant> Hahaha
<lifeless> wgrant: who does it impersonate when the owner is a team ?
<wgrant> lifeless: You have an anthropomorphised team.
<lifeless> wgrant: then I'm missing the issue
<lifeless> if the impersonation is checking permissions
<wgrant> Hmm?
<wgrant> If I set the daily build archive of a recipe I own to be the OEM primary archive, it will fail because I can't upload to the OEM primary archive.
<lifeless> and the selection of owner is limited to entities with upload on the archive, and I have upload permission as well.
<StevenK> wgrant: You think those are buggy tests?
<lifeless> wgrant: right, it shouldn't let you set it to that archive.
<wgrant> lifeless: Mm, I guess......
<wgrant> StevenK: I'd work out why it wants a duplicate.
<lifeless> wgrant: I'm spotting an inconsistency here is my point:
<wgrant> StevenK: You probably do have a bug in your code.
<wgrant> StevenK: But the test is also clearly crap.
<lifeless>  - the teams I'm in are not the same as 'folk with upload access to the archive'
<lifeless>  - because those teams are not me, and only *some* of them may grant upload access to the archive
<lifeless>  - so limiting it to those teams does not make sense
<lifeless>  - limiting the selection to entities that *do* have upload access to the archive would make sense, but to prevent privilege escalation I must also have said access.
<lifeless> right now, I can select a broken team as easily as a working one.
<wgrant> We'd need to think about how restricting those would work.
<wgrant> It may make some moves impossible.
<lifeless> I put it to you the right vocab is ICanUploadToArchiveButEmptyIfICannot
<wgrant> Needs thought.
<wgrant> Probably.
<lifeless> it would make a move impossible where there isn't a common archive that the source and target can both upload to
<lifeless> to do that move would need a handshake rather than a simple assignment
<lifeless> henninge: does anything else use the vocab ?
<lifeless> henninge: and have you followed this chatter?
<henninge> at first I did ...
 * henninge reads rest
<lifeless> wgrant: we could say 'anyone with upload if I have upload, or any team I'm in if I don't have upload' - but thats just an ugly way of permitting a hand-shake to do arbitrary moves.
<lifeless> wgrant: I think at the point you want to change trust boundaries like that its better to say to folk 'copy the recipe text across and delete the old one'
<henninge> let me check on the vocab
<gmb> allenap: Hola. Can you take a look at https://code.launchpad.net/~gmb/launchpad/conjoined-oops-bug-106338/+merge/62456 for me?
<lifeless> brb
<allenap> gmb: Sure. I'm looking at a branch for Jeroen right now, but I'll do your's right after that.
<gmb> Ok. No rush.
<allenap> gmb: I have a sneaking suspicion that the apostrophe was unwarranted.
<gmb> allenap: I'll forgive you THIS ONCE.
<allenap> Thank you :)
<henninge> lifeless: it's also used for Branch and BranchMergeQueue
<henninge> lifeless: anyhhow, I think that discussion is about  a different bug then what I am trying to fix here.
<henninge> I will go with the short-term/long-term solution that we discussed.
 * henninge has to change locations now
<bigjools> wgrant: you've got mail
<wgrant> Forboding.
<wgrant> Is it like 100MB or something?
<mwhudson> that's kind of a redundant thing to say to a canonical employee
<wgrant> Hm, only 616KB.
<wgrant> Why did it take 2 minutes to download? :/
<wgrant> bigjools: Why "in different distro"?
<wgrant> bigjools: PPA -> primary copies should be queued too.
<wgrant> bigjools: Hm, so the job will run twice?
<cjohnston> henninge-lunch: I'm around when you get back.
<bigjools> wgrant: yes it needs to run twice
<bigjools> otherwise webapp requests need to start checking policy and ancestry
<wgrant> Ah, I didn't read your prose.
<wgrant> That makes sense.
<wgrant> I was hoping I could think of a way around that.
<wgrant> But I cannot right now.
<wgrant> Ew @ json_data, but it will work.
<wgrant> doit.
<lifeless> jml: I think talking at the very end of my day gets into rambling conversations.
<jtv> allenap: I'll be away now, so will have to follow up on that review tomorrow.
<lifeless> jtv: sorry
<lifeless> jtv: I just cleared my plate !
<lifeless> jtv: but its really late.
<lifeless> jtv: lets talk your morning?
<jml> lifeless: heh
<jml> lifeless: certainly is the case with me.
<jtv> lifeless: never mind, it's not needed now!  Will explain another time if desired.
<lifeless> sure
<allenap> jtv: Okay, have a good evening. Fwiw, I'll be finished in less than 5 minutes.
<jtv> Ooh@
<jtv> @
<jtv> !
<jtv> such temptation.
<jtv> But that's how it always goes.  "Just this one more, darling"
<jtv> And then there's yet another bug to fix, another branch to land.
<jml> any maint squad folk around to deal w/ doko's problem?
<lifeless> jml: I'm not sure I've made sense in my latest mail; be generous reading it :)
<lifeless> also python 3 upstream attitude about bytes vs unicode really is starting to depress me
<jml> what's their attitude?
<lifeless> it seems to be 'its appropriate for it to be hard to work with bytes, because bytes are not unicode/text'
<jml> lifeless: it does make sense, thanks. still thinking about the answer to the "where to capture?" question
<lifeless> the latter I agree with, the conclusion bogus
<lifeless> jml: [back to bytes]
<lifeless> ok gnight
<wgrant> allenap: Ah, nice, we have the actual rabbit log this time!
<LPCIBot> Project db-devel build #582: STILL FAILING in 4 hr 58 min: https://lpci.wedontsleep.org/job/db-devel/582/
<henninge-lunch>  /nick henninge
<henninge> Hi cjohnston!
<allenap> wgrant: Yes, at last :)
<cjohnston> hey henninge
<henninge> cjohnston: hang on, I am preparing something
<cjohnston> yup
<jml> mrevell: strikethrough doesn't work on the dev wiki. where should I file the bug?
<henninge> cjohnston: are you familiar with doctests and how to read them or would you like me to explain?
<henninge> cjohnston: also, we can do this in voice, if you prefer that ;-)
<cjohnston> That would be fine.
<cjohnston> I'm guessing at it, but I think I'm doign ok
<henninge> which of the two? ;-)
<cjohnston> explain and voice ;-)
<henninge> cjohnston: I only have our internal Mumble server set up, the other option is skype.
<LPCIBot> Project windmill-devel build #142: STILL FAILING in 1 hr 9 min: https://lpci.wedontsleep.org/job/windmill-devel/142/
<henninge> cjohnston: one thing I saw that may be confusing is that one of the tests is run four times.
<henninge> that's a bug in our test suite
<cjohnston> ok
<wgrant> henninge: Are you sure? It's not run once for each implementation of an interface?
<henninge> wgrant: it's a doctest
<henninge> bugnotificationrecipients.txt
<henninge> wgrant: http://paste.ubuntu.com/613205/
<wgrant> henninge: Ah, it's run in four different DB users.
<henninge> ah, ok
<henninge> I remembered jtv reporting tests being run multiple times
<henninge> anyway
<henninge> cjohnston: we'll only have to deal with it once, though.
<cjohnston> ok.. cool
<henninge> cjohnston: but first I have this: http://paste.ubuntu.com/613206/
<henninge> that's the output of the other two failing doctests. I pasted it so we have linen numbers to talk about.
<cjohnston> ok
<henninge> cjohnston: doctest will output a lot of context that is not really failing but look like it.
<cjohnston> my thought is  ^^^^^^^^^^^^ is what we are looking at?
<henninge> in this case, the actual failure is on lines 87-89
<cjohnston> ok.. so I was in the right area atleast
<henninge> the other differences it points out is where it matched '...' to '--' on the page.
<henninge> do you have a local copy of the branch?
<henninge> well, you should
<cjohnston> yes
<henninge> so, in lib/lp/bugs/doc/bugnotification-sending.txt line 68 is where this starts
<henninge> that's what line 24 in the diff tells us
<henninge> scrolling down, line 123 is where the text needs to be adapted to the new wording
<cjohnston> so just s/xx/xx to make it match?
<henninge> right
<henninge> doctests don't care about line breaks, so you can fit it nicely in the 78 columns limit
<cjohnston> ok.. That one is done.
<henninge> cool, next one
<henninge> line 323
<henninge> (line 101 of the diff)
<cjohnston> yup
<henninge> you are aware of the meaning of '...'
<henninge> ?
<henninge> It just maches any text
<cjohnston> no
<cjohnston> ok
<henninge> line 105 of the diff shows you what the test saw, starts with a '+'
<henninge> lines 106-114 show you what it expected
<cjohnston> right.
<henninge> that's from the doctest line 324...
<henninge> The first one starts with [...
<henninge> so there it eats any text it finds until the next text fits
<cjohnston> http://paste.ubuntu.com/613218/
<henninge> For this array literal it means ignoring the first elements until the '-- ' element.
<gmb> allenap: Thanks for t'review. I'll make the change you suggest.
<henninge> cjohnston: perfect!
<henninge> ... and here I am explaining
<henninge> ;-)
<cjohnston> well.. learning what the stuff means is more helpful than just being able to s/xx/xx
<henninge> right
<henninge> ;)
<cjohnston> so the next issue is 152 of the diff?
<henninge> yup
<cjohnston> bugnotification-sending.txt 437
<henninge> right
<mrevell> jml, https://bugs.launchpad.net/launchpad-dev-moin-theme/
<henninge> cjohnston: bug line 162 of the diff also says that it is missing the unsubscribe footer.
<henninge> hm
<cjohnston> I don't understand what 173/175 means
<henninge> cjohnston: there is a ... that is replaced by an 18
<henninge> it's not a failure although the output makes it look like one
<cjohnston> ok
<cjohnston> so the test is bug 18 I guess?
<_mup_> Bug #18: RFE: I'd like to be CC:d automatically to bugs I report <feature> <lp-bugs> <Launchpad itself:Fix Released by bradb> < https://launchpad.net/bugs/18 >
<henninge> the test does not care about the bug number, that's why the ... is there
<henninge> but when the test is run it happens to be 18, yes.
<cjohnston> ok
<henninge> line 184 is the actual failure
<cjohnston> right.
<henninge> and line 196 is ok, just a trailing ... to leave the dummy text out of the test.
<cjohnston> ok
<cjohnston> the next one I'm kinda condused about.. starting at 201
<henninge> yeah, I was just wondering waht's going on there
<jml> mrevell: thanks.
<henninge> cjohnston: I am not quite sure but it seems the test runner is getting mixed up a bit.
<henninge> I would let this one be for now and go to the next one.
<cjohnston> ok
<henninge> once we fix all places that we can spot, we can re-run the test and see if it fits now.
<henninge> the next one is line 242 of the diff
<henninge> cjohnston: I mean, you know what you changed and it should not affect anything else.
<henninge> Just look for where those phrases appear and adapt them in the test.
<henninge> line 272
<henninge> line 322
<cjohnston> I'm up to 400
<cjohnston> ;-)
<henninge> cool
<henninge> 419, actually ;)
<cjohnston> there isnt anything wrong with 416 is there?
<henninge> no, it appears on 421
<henninge> it's just different line wrapping in the test and the actual output.
<henninge> once all the text fits, the line wrapping is ignored, as it should be.
<cjohnston> I almost wonder if it should be "which is subscribed to this bug report."
<bigjools> would anyone care to have a pre-imp chat with me about a change to the Job system?
<bigjools> just the man
<henninge> cjohnston, bigjools: we are having our team call now ;)
<bigjools> abentley: good morning.  When you have a moment can I have a chat with you about a change I would like to make the Job system?
<cjohnston> henninge: I'm down to line 1263 looking for anything else. how do I fix those? just edit unsubscribe to Edit Subscription?
<abentley> bigjools: sure.  I'll be free in a few minutes.
<bigjools> abentley: thanks.  shouldn't take long
<henninge> cjohnston: re-run the test when you are done to see if anything else fails
<henninge> bin/test -vvt bugnotification-sending.txt
<cjohnston> henninge: I have no idea how/where to do that
<henninge> cjohnston: run the test?
<henninge> cjohnston: is that what you mean?
<cjohnston> yes
<henninge> do you have a working launchpad setup or were you just working on the branches?
<jml> off for lunch
<cjohnston> just working on branches
<cjohnston> I thought just editing text wouldn't be a big deal.. I guess I was wrong. ;-) lol
<cjohnston> https://code.launchpad.net/~chrisjohnston/launchpad/land-chrisjohnston-187013-197793-483373
<henninge> yeah, Ill run the test for you, np.
<cjohnston> Do I want to know what all of those things that got added in my branch are?
<henninge> what things?
<cjohnston> if you look at that branch theres 20 files added
<cjohnston> http://bazaar.launchpad.net/~chrisjohnston/launchpad/land-chrisjohnston-187013-197793-483373/revision/13047
<henninge> what did you do to get this branch?
<cjohnston> I just merged in your branch made the changes and pushed back
<henninge> right
* jcsackett changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: allenap, jcsackett | https://code.launchpad.net/launchpad-project/+activereviews | Critical bugs:209 - 0:[######=_]:256
<henninge> my branch is based on a current (at the time) devel, so merging my branch pulls in all the changes that have been merged since you branched devel.
<StevenK> wgrant: ec2 returned -- it is only the seven archiveuploader tests.
<wgrant> StevenK: Good news.
<henninge> cjohnston: You can ignore those but you should have committed after merging.
<henninge> bzr merge lp:...
<henninge> bzr commit -m"Merged such-and-such"
<cjohnston> ok
<henninge> or in this case it actually would have been best to just create a new branch from my branch.
<henninge> bzr branch lp:~henninge/launchpad/land-chrisjohnston-187013-197793-483373
<henninge> if you could do that and then manually copy the changes to the doctest into that branch, I think it would be cleaner.
<cjohnston> working on it
<henninge> I mean, just copy the doctest file you changed.
<cjohnston> yup
<henninge> and then do "push --overwrite" if you want to continue using the branch name.
<cjohnston> ok
<abentley> bigjools: I'm free to chat.
<bigjools> abentley: great, mumble?
<abentley> bigjools: sure.
<cjohnston> henninge: https://code.launchpad.net/~chrisjohnston/launchpad/land-chrisjohnston-187013-197793-483373
<henninge> cjohnston: still failing but less. Let me see what might be going wrong.
 * henninge aims at deryck ....
<henninge> ... POW! http://paste.ubuntu.com/613269/
<deryck> henninge, on call, just a minute
<henninge> deryck: nm, you're dead
<henninge> cjohnston: that ^^ is the reason why the unsubscribe link is missing.
<bigjools> abentley: should the new exception handler do anything with incomplete_jobs?
<abentley> bigjools: Yes, when a job is suspended, it should be added to incomplete_jobs.
<bigjools> ok thanks
<cjohnston> henninge: I'm not sure I understand whats wrong
<henninge> cjohnston: that code is searching for "direct subscriber"  in order to determine if it should add the unsubscribe link.
<cjohnston> ok
<henninge> like the XXX correctly points out, that is not a good way because it is fragile.
<henninge> like we saw now - change the wording, and it breaks.
<cjohnston> ya.. So maybe if 'subscribed to the bug' and not 'member of' ?
 * jcsackett is starting to hate his computer.
<bigjools> abentley: when you're ready -> https://code.launchpad.net/~julian-edwards/launchpad/suspend-running-jobs-bug-788612/+merge/62494
<abentley> bigjools: ack
<bigjools> thanks
<cjohnston> henninge: https://code.launchpad.net/~chrisjohnston/launchpad/land-chrisjohnston-187013-197793-483373
<henninge> cjohnston: yup, that's the fix
<cjohnston> :-)
<henninge> cjohnston: there is still a failure  in line 178 http://paste.ubuntu.com/613283/
<henninge> an lingering " a"
<henninge> s/an/a/
<cjohnston> pushed
<henninge> cool, bugnotification-sending.txt is passing now! ;-)
<henninge> cjohnston: here is how "subscribing.txt" is failing
<jcsackett> sinzui: chat?
<henninge> http://paste.ubuntu.com/613288/
<sinzui> jcsackett: yes
<cjohnston> henninge: s/Unsubscribe/Edit Subscription?
<henninge> cjohnston: right! I was trying to remember what might have changed here ... ;-)
<cjohnston> I'm not sure totally how to read this one
<deryck> henninge, hi.  ready for our call when you are.
<henninge> deryck: one moment
<deryck> henninge, np
<cjohnston> >>> unsubit = browser.getControl(name='unsubscribe')  what are the requirements for that? is 'Edit Subscription' valid?
<cjohnston> wait... that should be unsubscribe
<henninge> cjohnston: well, the remaining failures may just be results of the initial failure. That is one of the problems with doctests.
<henninge> let's see if the test passes with that first fix ...
<cjohnston> pushed
<henninge> it's "Update Subscription"
<cjohnston> I changed it, you would think I would know that.. lol
<cjohnston> done
<henninge> cjohnston: there is one more getLink('Unsubscribe')
<henninge> cjohnston: I have to be otp now.
<cjohnston> ok
<cjohnston> henninge: your talking about line 105? The way the text reads is that is the button that you click to unsubscribe no?
<henninge> cjohnston: ok, back
<cjohnston> :-)
<henninge> line 105 is clicking on the link, then the test browser goes the next page and finds the actual unsubscribe button.
<henninge> so line 105 needs to be updated for the test to pass
<cjohnston> ok
<jcsackett> deryck: would you have an opportunity to talk javascript picker with me by any chance?
<deryck> jcsackett, I've been on calls non-stop since I logged on.  Can we wait 30 minutes to and hour and let me wrap a couple loose ends first?
<cjohnston> done henninge
<deryck> if you're blocked and it's critical, I can manage the time.
<jcsackett> i'm semi-blocked, but i can certainly wait 30 minutes to give you a rest.
<henninge> cjohnston: passing ;-)
<gary_poster> allenap, you around?
<jcsackett> i'll ping you again in a bit then, deryck. thanks in advance.
<allenap> gary_poster: Yeah, hi.
<allenap> Got a branch for me?
<deryck> jcsackett, thanks.  and it's not rest.  :-)  I need to finish somethings from the calls before I loose the thought or note. ;)
<gary_poster> hey allenap.  No, this is more in line of "picking malone dev's brain." :-) Do you happen to know anything about how we process incoming mail for comments?  In particular, do you know if we honor SPF records when we are processing mail?  Context: someone has had their account suspended because the account is being used to send spam.
<gary_poster> They say they do not use web mail, and only use kubuntu, and have SPF records for their domain, so I am trying to see if I can give a scenario for this happening other than "your box or your webserver have probably been hacked, sorry."
<henninge> cjohnston: here is the output for bugnotificationrecipients.txt http://paste.ubuntu.com/613310/
<allenap> gary_poster: I have no knowledge that we check SPF in Launchpad. Martin Pool is probably the most versed in Launchpad's mail code right now, but I suspect SPF would be a question for the LOSAs or IS.
<gary_poster> ok cool, thanks allenap
<henninge> cjohnston: Thats all just replacements of the wording, towards the end of each section.
<henninge> easy ;)
<henninge> cjohnston: and finally http://paste.ubuntu.com/613313/
<henninge> The error is on line 22. It's searching for "duplicate bug (17)" but now it's "duplicate bug report (17)"
<henninge> line 20 of the diff tells you that it fails on line 62 of the test
<abentley> bigjools: r=me.  I would call it "SuspendJob", not "SuspendJobError", because it's one of those rare exceptions that doesn't indicate an error.
<henninge> cjohnston: EOD for me. Just push what you got and email me a reminder, I will take care of it tomorrow.
<bigjools> abentley: yeah I thought about that one, I just wanted it to "look" like an exception
<cjohnston> Thanks henninge
<cjohnston> Have a good night!
<bigjools> abentley: I'll change it, thanks
<bigjools> abentley: although, SuspendJob looks like a type of Job :)
<abentley> bigjools: SuspendJobException?  Anyhow, no big deal.
<bigjools> yeah that's better
<jml> good grief
<jml> thunder
<jml> in London
<timrc> Noticed the topic in #launchpad-reviews.  Is this a permanent change?  If so, can someone please change #launchpad-reviews to #launchpad-dev in Step1, https://dev.launchpad.net/PatchSubmission
<henninge> cjohnston: you have a good day ;-)
<cjohnston> Thanks :-)
<timrc> bigjools (+ friends), I'm looking at #724740... before I talk about my implementation idea, I've noticed (what seems to be) an inconsistency with how we set buildd_secret... in lib/lp/soyuz/model/archive.py we use create_unique_token_for_table(20, Archive.buildd_secret) whereas in lib/lp/soyuz/browser/archive.py we use "create_token(16)... should that not be create_token(20) or visa-versa ?
<_mup_> Bug #724740: setting a ppa private cannot be done over the API <api> <oem-services> <ppa> <Launchpad itself:Triaged> < https://launchpad.net/bugs/724740 >
<bigjools> timrc: yeah they could be the same, it doesn't really matter though.
<timrc> bigjools, ah ok
<bigjools> it's just the length of the auto-generated password
<bigjools> and it's not about size, it's about what you do with it :)
<timrc> bigjools, lol
<timrc> allenap, jcsackett: do you have some time today to discuss #724740 ?
<_mup_> Bug #724740: setting a ppa private cannot be done over the API <api> <oem-services> <ppa> <Launchpad itself:Triaged> < https://launchpad.net/bugs/724740 >
<jcsackett> timrc: the ppa machinery is well out of my experience, i'm afriad. you would be better served with someone else. not sure about allenap, but i think he's approaching the end of his day.
<jcsackett> deryck: had enough time to collate your thoughts from earlier calls?
<timrc> jcsackett, okay :) so who are your ppa gurus and do you guys rotate the ocr responsibility?
<timrc> is there a schedule that I can view so I know when to nag
<deryck> jcsackett, yes.  5 more minutes.  I'm only marveling at my new found net fame.  But need to grab a coke.
<bigjools> jcsackett: you don;t need to know anything about PPAs, it's about setting a property over the API that we don't want all and sundry to do
<jcsackett> bigjools: ah, i see.
<jcsackett> timrc: i probably can help with that, but i'm about to be otp and then out for a bit. i can ping you later to talk about API stuff, if you like.
<allenap> timrc: Sorry, I have almost no PPA knowledge, and I will be afk in a short while too.
<jcsackett> timrc: off the top of my head, i believe benji might be able to help you out if you need assistance ASAP, though i don't know about his current availability.
<timrc> the change as I see it basically making the private attribute on IArchivePublic read-only, creating a mutator for it (e.g. setPrivate), and then placing the validator code into this mutator along with the logic to set the private attribute and the buildd_secret if it does not exist
<jcsackett> perhaps ASAP isn't quite right, given that. :-P
<timrc> (and the ppa was set to private)
<bigjools> timrc: that sounds ok to me, what would the validator look like?
<bigjools> ie. who can do it? :)
<deryck> jcsackett, drag me where you want or meet me in orange.
<jcsackett> deryck: mumble is apparently rejecting my password. one sec.
<deryck> jcsackett, np
<timrc> bigjools, the validator is the exact same validator that's currently used on the property
<bigjools> timrc: ok, and the zope security will ensure it's only commercial admins who can do it
<timrc>  bigjools, e.g: _validate_archive_privacy
<timrc> bigjools, ok, well I'm going to assign the bug to me and proceed with the implementation if that's okay with you
<bigjools> timrc: sounds great
<bigjools> thanks for contributing
<jml> I'm going offline for a bit. Will see you all tomorrow, most likely.
<bigjools> circular imports BLOW
<jml> yeah, it's a glitch in Python.
<bigjools> or, I am being a moron
<jml> Hmm. I've had the same issue.
<jml> What version are you using?
<bigjools> nothing to see here, move along
<jml> loggerhead         14
<jml> launchpad-buildd    3
<jml> launchpadlib        2
<jml> lazr.delegates      1
<jml> oops-tools          1
<jml> bye for real.
<bigjools> cheerio
<LPCIBot> Yippie, build fixed!
<LPCIBot> Project db-devel build #583: FIXED in 4 hr 36 min: https://lpci.wedontsleep.org/job/db-devel/583/
<LPCIBot> Project windmill-db-devel build #327: STILL FAILING in 1 hr 7 min: https://lpci.wedontsleep.org/job/windmill-db-devel/327/
<jkakar> It's sad that price is coming as a reason to use GitHub over Launchpad (for private projects).
<jkakar> It'd be nice if Launchpad just set the prices to the same as GitHub.
<bigjools> can anyone think of any SQLObject model classes that have a FK to a StormBase model class?
<sinzui> bigjools: Nothing comes to mind
<bigjools> well I am trying to do just that and getting:
<bigjools> PropertyPathError: Path 'PackageCopyJob.<primary key>' matches no known property.
<bigjools> where PackageCopyJob is the FK table
<abentley> I mean https://code.launchpad.net/~abentley/launchpad/handle-concurrent/+merge/62511
<abentley> sinzui: I am looking at bug #595166.  ISTM that the root issue is that we expect process-mail to have new messages as its input, not existing ones.  Shouldn't it discard messages that are dupes?
<_mup_> Bug #595166: IntegrityError raised filing a bug using the email interface <easy> <email> <lp-bugs> <oops> <Launchpad itself:Triaged> < https://launchpad.net/bugs/595166 >
<abentley> deryck[lunch]: pre-implementation call when you're back?
<jcsackett> abentley: sorry for the delay, looking at your branch now.
<abentley> jcsackett: s'okay.
<sinzui> abentley: I am re reading.this to remember is this really is a spam issue
<abentley> sinzui: I don't see this being a spam issue unless an identical spam message gets sent to bugs twice.  Non-identical messages with the same msgid produce different messages.
<sinzui> abentley: I think we really do want to discard messages from dupes
<abentley> sinzui: cool, thanks.
<sinzui> abentley: I investigate three cases. All were different messages with differnt spam links from different people, but the message ids were the sam
<sinzui> same
<abentley> sinzui: Was the original message spam, or had the content changed?
<sinzui> yes the orginal was spam.
<abentley> sinzui: cool.
<sinzui> no client/process should reused an id, so if someone stole another users legitimate message, we want to reject that oo
<sinzui> too
<benji> anyone have pointers to how mailing lists work in LP?  I have a user that isn't getting emails he sends to the list
<benji> perhaps user's don't get their own messages?
<sinzui> benji: I can help
<sinzui> benji: 1. verify lp shows he is subscribed.
<benji> sinzui: https://launchpad.net/~syncany-team/+mailing-list-subscribers shows him as subscribed
<sinzui> benji: 2. verify he is sending with a registered email address (lp rejects those it does not know about)
<benji> he is
<sinzui> benji: 3. is the message in the archive? This is dodgy because there can be a backlog
<benji> an unrelated CHR question: is hiding non-spam comments at the commentor's request ok? (he was confused when he commented and doesn't want to spread confusion)
<sinzui> wow, the empty archive view is bad...
<benji> re. archive: not yet
<sinzui> I think there means something else
<benji> I was wondering if LP and MM aren't in sync
<sinzui> I tested an empty archive view a few weeks ago. I it pretty
<sinzui> benji: I think you are right. when we call create list, the archive is setup and we expect 2 index pages
<sinzui> date and thread
 * sinzui reads code to learn the order of events.
<sinzui> oh, we create the directory, but do not call mhonarc because there is nothing to do
<benji> sinzui: he just said that the first message showed up (https://lists.launchpad.net/syncany-team/)
<sinzui> benji: regardless of nhonarc, we know that mm knows about the list because it made the dir and it sent back the success message
<jcsackett> abentley: r=me, with a comment on the MP.
<abentley> jcsackett: thanks!
<jcsackett> benji: it's fine to hide a confused comment at the commentor's request.
<jcsackett> the control was changed to "Hide comment" for precisely that reason. spam is not the only reason we hide things.
<benji> jcsackett: great! thanks
<jcsackett> sinzui: have a few moments to chat?
<sinzui> benji, I was going to point you to a bug about the delay in the mailing list archive, but I do no see it.
<sinzui> jcsackett: I can talk
<sinzui> mumble or sip?
<jcsackett> i'm good with either. are you already on mumble?
<benji> sinzui: no problem, thanks for all your help!
<lifeless> mornink
 * benji suspects lifeless has been replaced by a Russian imposter.
<lifeless> in soviet russia, bugs close you
<jcsackett> that sounds ominous.
<benji> lol
<lifeless> flacoste: on bug 419531 - are you aware that the same thing makes the legacy input box unusable in those situations ?
<_mup_> Bug #419531: project name picker search / vocabulary is hard/impossible to use (too many results on exact searches) <disclosure> <project-picker> <vocabulary> <Launchpad itself:Triaged> < https://launchpad.net/bugs/419531 >
<bac> jcsackett: you have time for a review?
<flacoste> lifeless: it doesn't
<lifeless> flacoste: its a different cause?
<jcsackett> bac: sure.
<bac> jcsackett: https://code.launchpad.net/~bac/launchpad/bug-750984/+merge/62543
<sinzui> My team is work in that bug in the next two weeks, I have refrained from the niggling about its importance
<lifeless> sinzui: sure
<lifeless> and I'm ecstatic
<flacoste> lifeless: ok, if you mean that the Choose button suffers from the same bug, yes, that's true, but the previous popup suffered from the same problem there
 * sinzui think the search on summary and description leads to false matches
<flacoste> lifeless: but you can still enter 'launchpad' in the input box like you used to and it will accept your input
<lifeless> ah yes
<lifeless> mmm
<lifeless> this evaluation of regression is inconsistent with our bug triage docs & critical escalation rules. So I think we need to revisit them.
<flacoste> how is it inconsistent
<flacoste> ?
<flacoste> and fwiw, deryck is working on clarifications to the bugtriage rule concerning UI issues
<lifeless> thats great
<lifeless> https://dev.launchpad.net/PolicyAndProcess/ZeroOOPSPolicy
<lifeless> This mean that any user-visible error happening in production is a stop-the-line event and should be fixed ASAP
<deryck> yeah, I'm really not try to niggle, as sinzui called. ;)  Just want to be clear about when something is critical or not.
<lifeless> when we last spoke about, we decided js errors count as oopses
<sinzui> lifeless: I believe there is a regression. The current code does not scale to the volume of projects we have now. The vocabulary uses searching-like behaviour which did work for a small set of project (with poor summaries and descriptions).
<lifeless> and the motivation is solving problems users see promptly
<lifeless> sinzui: I do too
<lifeless> sinzui: but opinions differ
<flacoste> i probably does need clarifications
<sinzui> The useless number of results is because it matches "launchpad" or "ubuntu" in the project summary or description, but the user is thinking of the name or displayname
<flacoste> but for the record
<flacoste> i don't see any regression here
<flacoste> nor any JS OOPS
<lifeless> flacoste: I'm looking at the spirit not the legals :)
<flacoste> i do too
<flacoste> :-)
<flacoste> if this was new work
<flacoste> and would have been caught during acceptance testing, it would have warranted a High bug
<lifeless> really?
<lifeless> I would have marked something like this qa-bad and reverted.
<lifeless> I am sure others in the team would have too - in fact we have done so in the last few months
<flacoste> that doesn't match current practice
<flacoste> in some case we have reverted
<flacoste> in others we haven't
<lifeless> then current practice is inconsistent :)
<deryck> feature flags protect us, and prevent having to revert in some cases.
<flacoste> i think it is consistent
<flacoste> with applying common sense judgement
<flacoste> about when the error is found
<flacoste> and what's the impact of the error
<flacoste> remember that exploratory testing happens after deployment
<flacoste> so it's after the qa-bad step
<lifeless> flacoste: qa happens before deployment
<flacoste> i wouldn't thinkg the suitability for deployment check would have caught such a corner case
<flacoste> i might have confused you
<flacoste> by saying acceptance testing
<lifeless> flacoste: its not such a corner case though
<flacoste> when i really meant exploratory testing
<lifeless> no, I knew you meant at the stakeholder-check phase
<flacoste> yeah, that's after deployment
<lifeless> which is nicely decoupled from deploys now
<flacoste> again confusion with the terms
<flacoste> i mean live on production (maybe behind a feature flag)
<flacoste> but that's after qa-bad has any significance
<lifeless> yes, I know, there is no confusion
<lifeless> if its behind a feature flag it would not get qa-bad
<lifeless> if someone noticed during qa
<flacoste> of then we are saying the same thing then :-)
<lifeless> and this is such a glaring issue, I'm reaonably sure someone would have noticed
<lifeless> I usually test with launchpad itself on qastaging
<poolie> hi lifeless
<poolie> sorry about https://bugs.launchpad.net/bugs/786804
<_mup_> Bug #786804: branch scanner rlimit failures cause the next branch to be incorrectly scanned and fail <oops> <regression> <Launchpad itself:Triaged> < https://launchpad.net/bugs/786804 >
<flacoste> anyway, this is speculative at this stage
<lifeless> flacoste: the regression angle i'm arguing is that the user story we had got broken
<lifeless> flacoste: I know there are workarounds but the old story - including doing a choose based search - *no longer works*
<flacoste> yeah, from my point of view, there is no additional breakage
<lifeless> flacoste: and that is the *classic* definition of a regression.
<flacoste> i'm arguing it didn't work before
<flacoste> from the user point of view
<flacoste> in both case, the work-around was the same
<flacoste> enter the project name textually in the input box
<lifeless> flacoste: what happened before?
<flacoste> that hasn't regressed
<flacoste> you just had too many results to sift through
<flacoste> because we never ranked the result sanely
<flacoste> launchpad wasn't in the first page of results
<flacoste> might be hidden on the second or third page
<flacoste> which is just as bad
<flacoste> as saying: 'sorry dude, try again'
<lifeless> flacoste: but its not equivalent - annoying vs impossible
<flacoste> i bet there is a bug about that
<flacoste> semantic
<lifeless> poolie: thats fine; we traded 'the system dies in a fire' for 'bad jobs die quickly and oh we have missing code around handling such deaths'
<lifeless> poolie: its a move forward overall
<lifeless> deryck: feature flags are intensely useful
<lifeless> flacoste: so, it returns 464 items
<lifeless> flacoste: which is pretty useless
<lifeless> flacoste: for launchpad itself
<lifeless> searching for bazaar returns 320
<lifeless> and AIUI the vocab wasn't changed
<flacoste> yep
<lifeless> so - I buy your analysis on this one
<flacoste> lifeless: bug 255282
<_mup_> Bug #255282: The string "launchpad" is hard to find in the Projects search <lp-foundations> <Launchpad itself:Triaged> < https://launchpad.net/bugs/255282 >
<lifeless> hahahaha
<lifeless> sinzui: would it make your life hard if I dupe these ?
<flacoste> it would appear on page 5!
<deryck> we can all agree: if there's a dupe, it's not a regression! Consensus, yay! :-) :-)
<flacoste> 2008-08-06: "I'd go so far as to say that the project search pop-up is rendered so useless by this almost-but-not-quite findable presentation that it perhaps ought to be removed until it can be fixed. I"
<lifeless> deryck: heh. orthogonal I think :P
<sinzui> lifeless: they are not the same :(
<lifeless> sinzui: in that you can fix one without the other ? ok.
<deryck> dang you sinzui  ;)
<sinzui> lifeless: ProductSet find methods are not vocabulary lookup methods
<lifeless> sinzui: huh, thats the vocab in use with the old UI, isn't it ?
<sinzui> I noted to wgrant and jcsackett a few days ago that I was concern that PersonSet.findPerson() and the person vocabularies were both inconsistent and do not appear to know what use case they serve
<flacoste> sinzui: bug 255282 is about the Choose widget which uses a vocabulary
<_mup_> Bug #255282: The string "launchpad" is hard to find in the Projects search <lp-foundations> <Launchpad itself:Triaged> < https://launchpad.net/bugs/255282 >
<sinzui> This is true for several set vs vocabs and I would like dry code
<flacoste> sinzui: i agree that it would be sane to use the model's find() method from the vocabulary
<sinzui> flacoste: sorry I thought it was about /projects
<lifeless> sinzui: I would like that too; I don't see that this is a case of that though. They describe the same widget as it used to be.
<lifeless> I would like to dupe these on the old one and update its metadata.
<lifeless> sinzui: will that cause you any difficulty?
<flacoste> there is a slight variation
<sinzui> They It is a dupe. I did fix the tags a moment ago though
<flacoste> in that the new widget should allow exact match selection
<flacoste> even if there are too many results for the search
<lifeless> flacoste: thats about how we fix it
<flacoste> yeah, agreed
<lifeless> I think bugs should be separate if we're liable/reasonably going to fix one without the other
<lifeless> but I don't think that applies here
<sinzui> flacoste: the project vocan does not attempt to rank the quality of matches. people does, but I am given to understand the rank is dropped in a late step in the process :(
<lifeless> ok
<poolie> lifeless: glad to hear you think it's a step forward; on the whole so do i
<lifeless> I've made the old one the master, moved the metadata across.
<lifeless> EOF, next topic :)
<lifeless> sinzui: please let me know if/when you'd like me to chat with you guys.
<poolie> lifeless: should i feel the ulimit fallout is my obligation to address?
<lifeless> poolie: no
<lifeless> poolie: but I'd be delighted if you were to do so
<poolie> i proposed some ideas
<poolie> let me know what you think
<poolie> i won't treat it as personally critical then, but i may try something
<bigjools> can someone please look at this zope proxy problem I have and tell me where I am being completely stupid   http://pastebin.ubuntu.com/613493/
<lifeless> do you have access to the attribute?
<lifeless> if verifyObject runs as you, it might fail if you can't access the attribute
<bigjools> lifeless: branch is here: https://code.launchpad.net/~julian-edwards/launchpad/packageupload-with-pcj
<bigjools> ah cock
<bigjools> it's an old-style zcml declaration with explicit properties instead of interfaces
<lifeless> yup
<bigjools> I hate soyuz
<lifeless> only cause it hates you
<bigjools> in soviet russia ...
<bigjools> this is a good time to get the last ditch battery warning
<bigjools> good night!
<lifeless> nn
<bigjools> lifeless: can I push without a repack happening?
<thumper> email processing is fucked
<thumper> I just checked the logs
<thumper> not sure how long it has been fucked for though
<thumper> at least an hour
<lifeless> details!
<thumper> 2011-05-26 22:18:23 ERROR   An exception was raised inside the handler:
<thumper> http://launchpadlibrarian.net/72491135/1052aa70-87e6-11e0-a1bd-001e0bc3957e.txt
<thumper>  -> http://launchpadlibrarian.net/72491136/9DXUl51Txcr83QLvOBe5YOGwyvJ.txt (Msg <1306439950.10927.2.camel@nbjplevy> size 12930186 exceeds limit 10485760)
<thumper> every 30 seconds
<thumper> failing on the same incoming email message
<mwhudson> whoops
<wgrant> sinzui: The pickers and vocabs are pretty bad, yeah :(
* jcsackett changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: - | https://code.launchpad.net/launchpad-project/+activereviews | Critical bugs:209 - 0:[######=_]:256
#launchpad-dev 2011-05-27
<sinzui> wallyworld_: mumble?
<wallyworld_> yep
* wallyworld_ changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: - wallyworld (*jtv) | https://code.launchpad.net/launchpad-project/+activereviews | Critical bugs:209 - 0:[######=_]:256
<wgrant> sinzui: Did you see the fallout from the flag-expired-memberships.py fix?
<LPCIBot> Project windmill-devel build #143: STILL FAILING in 1 hr 10 min: https://lpci.wedontsleep.org/job/windmill-devel/143/
<timrc> lifeless, sorry, I'm a novice to lp... not sure I understand what you're saying on bug #724740.  I thought that adding the mutator was how to run business logic when the attribute is set...
<_mup_> Bug #724740: setting a ppa private cannot be done over the API <api> <oem-services> <ppa> <Launchpad itself:In Progress by timrchavez> < https://launchpad.net/bugs/724740 >
<timrc> lifeless, here's my code so far: http://bazaar.launchpad.net/~timrchavez/launchpad/set_ppa_private_from_api_724740/revision/13052
<timrc> (sorry there are still some inconsistencies in the comments and no tests yet :/)
 * timrc ducks
<StevenK> wgrant: Most of the failures for archiveuploder look like behaviour changes in notify_spr_less()
<StevenK> wgrant: Which I'm fixing
<StevenK> wgrant: Running -t archiveuploader to see what the fall-out is
<wgrant> StevenK: Hmm.
<LPCIBot> Project windmill-devel build #144: STILL FAILING in 1 hr 9 min: https://lpci.wedontsleep.org/job/windmill-devel/144/
<StevenK> wgrant: Hm?
<wgrant> Indeed.
<wgrant> !
<StevenK> wgrant: Down to one failure
<wgrant> StevenK: But who knows how many you broke in Soyuz :)
<wgrant> Anyway, let me review your diff.
<StevenK> I have local changes
<lifeless> timrc: hi
<StevenK> Including putting back sanitize_string :-(
<wgrant> StevenK: Why did the abomination need to return?
<lifeless> timrc: mutators can be used yes, but if we need it always then just using a property is better
<StevenK> wgrant: Since I can't call guess_encoding on a unicode string, and the mail text seems to change
<StevenK> For some tests it's unicode, and others it's ascii
<wgrant> StevenK: It'll automatically become unicode when a unicode is interpolated into it.
<wgrant> Because fuckyeah Python 2.
<wgrant> StevenK: This indicates that there are unicode-safety bugs.
<wgrant> StevenK: It should either always be a unicode or never be.
<StevenK> wgrant: Then there likely already were
<wgrant> Sure.
<StevenK> r13103 pushed
<wgrant> Hm.
<wgrant> The branch is shorter now?
<wgrant> Interesting.
<wgrant> StevenK: notify_spr_less seems to assume there's always an archive.
<wgrant> But that's not the case.
<wgrant> Or at least it shouldn't be.
<wgrant> StevenK: Perhaps s/notify_spr_less/reject_changes_file/?
<wgrant> I'd also continue to encourage you to remove get_status.
<StevenK> Oh, and just replace it with a dictionary?
<wgrant> The dictionary, yeah.
<wgrant> I'm doing a proper rereview now.
<StevenK> I've done those two suggestions locally
<wgrant> ACTION_DESCRIPTIONS[action] is not much worse than get_status(action) :)
<wgrant> Thanks.
<StevenK> The only test that still fails is testSourcePackageRecipeBuild_success_mail. Didn't poolie break that on purpose?
<StevenK> Ah, no. I've broken it so it sends mail
<StevenK> Ahh. I can't check PU.getSourceBuild()
<StevenK> :-(
<timrc> lifeless, this may sound dumb but when you say property are you referring to Python's @property? just making sure I'm understanding you correctly?
<StevenK> wgrant: I have those two changes to push, along with the fix for SPRB_success_mail.
<jtv-afk> hi wallyworld_
<wgrant> StevenK: I'm reworking _sendNotification
<StevenK> wgrant: Oh, you're going to give me a patch?
<wgrant> Yes.
<wgrant> 'cause I was about to shred it in the review.
<wgrant> Decided I would be a bit kinder.
<StevenK> Who are you, and what have you done with the real wgrant?
<wallyworld_> jtv-afk: hello
<jtv-afk> Oh, old nick
<jtv> wallyworld_: any reviews you need me to mentor?
* StevenK changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: wallyworld (*jtv) | https://code.launchpad.net/launchpad-project/+activereviews | Critical bugs:209 - 0:[######=_]:256
<wallyworld_> jtv: thanks for asking. i assigned 2 to you. did you not get the email?
<StevenK> That '- ' was annoying me
<jtv> wallyworld_: Probably.  But email is one of those "it's got to be in there somewhere" things.
<wallyworld_> :-)
<jtv> Ah, there they are.
<wgrant> StevenK: Slow tests :(
<wgrant> But they seem happy with my +11/-53 so far.
<StevenK> Crumbs!
<StevenK> wgrant: My local branch has all of the archiveuploader tests passing
<wgrant> StevenK: Have some comments.
<StevenK> wgrant: Your commit messages makes me sad
<wgrant> StevenK: It sucked years before you got to it.
<StevenK> True
<lifeless> timrc: yes, I am
<wgrant> StevenK: send_mail/_sendMail/_sendNotification/send_message still seem like they're designed to confuse me. Let's see if we can merge/rename some of them.
<wgrant> And they're not to be confused with sendmail(), which does the actual work!
<StevenK> wgrant: I'm thinking about solving the two calls to _sendNotification in notify()
<StevenK> The only difference is the summary
<wgrant> send_mail can also easily be merged into _sendNotification.
<StevenK> wgrant: You say the logging of the mail is still duplicated -- however the level is different?
<wgrant> ... huh.
<wgrant> Make it all debug.
<wgrant> If you're doing a dryrun you should be using -v anyway, surely.
<wgrant> StevenK: What's the difference between the two _sendNotification calls? rejected doesn't send the file summary with OK/NEW/etc.?
<wgrant> That seems sort of trivial to sort out.
<StevenK> wgrant: summarystring for !rejected, summary_text for rejected
<StevenK> Only
<wgrant> StevenK: Do you like my slight _sendNotification simplification?
<StevenK> I do
<timrc> lifeless, gotcha... so when is it favorable to use a mutator vs. @property out of curiosity?
<StevenK> wgrant: Re: your "What did it do before?" question, exactly that.
<wgrant> Huh.
<wgrant> I hadn't seen that before.
<wgrant> OK.
<lifeless> timrc: when clients through the API should do different things than clients inside the appserver.
<StevenK> wgrant: It was inside notify()
<StevenK> wgrant: I'd love to drop the hasattr() hack, too
<StevenK> wgrant: But tests insist on passing in StringIO as the file_object
<wgrant> StevenK: We should find some other way.
<wgrant> StevenK: How was it done before?
<wgrant> Did it use packageupload.changesfile.filename or something?
<timrc> lifeless, okay. so if a mutator is used then clients within the appserver would basically need to call setPrivate explicitly rather then .private = boolean
<timrc> lifeless, @property would give a common interface for both
<StevenK> wgrant: It used packageupload.changesfile.filename
<wgrant> StevenK: :( OK
<wgrant> StevenK: We may want to rationalise this eventually.
<wgrant> So we take a filename and either changesfile string or object... not all three.
<StevenK> Mmmmm
<StevenK> wgrant: Does http://pastebin.ubuntu.com/613578/ satisfy your "Can haz map?" comment?
<wgrant> StevenK: It does indeed, but the location could be better.
 * StevenK ponders lunch.
<wgrant> StevenK: So, do you have any more ideas for merging the Confusing Four?
<wgrant> send_mail can be pretty trivially folded into _sendNotification.
<StevenK> I agree about that
<StevenK> send_message() is also used by notify_spr_less's new name
<wgrant> _sendNotification can be folded into notify()
<wgrant> Right.
<wgrant> StevenK: Hm, notify_spr_less doesn't want to use _sendMail?
<StevenK> No, it's first argument is an apr
<StevenK> *SPR
<wgrant> It looks like it can be optional.
<wgrant> It says 'if spr:'
<wgrant> Indeed, it is optional.
<wgrant> Use it if you can.
<sinzui> wgrant: I did not see the fallout.
<wgrant> Then send_message can be folded into there.
<wgrant> sinzui: It hadn't been sending warnings.
<wgrant> sinzui: So about 300 memberships expired without warning.
<lifeless> timrc: right
<wgrant> sinzui: But only 20ish were in restricted teams that they could have renewed themselves, and I think the owners have fixed those.
<lifeless> mutators are basically fugly
<sinzui> wgrant: :(
<timrc> lifeless, aye
<sinzui> I had answered a users question today saying all was fixed. He had noticed his membership was expiring and had not gotten an email
<wgrant> lifeless: Let's rewrite Launchpad in Java.
<lifeless> wgrant: indeed, why have one problem when we can have two!
<timrc> wgrant, I think we should rewrite it in something more intuitive, like COW (http://www.bigzaphod.org/cow/)
<wgrant> timrc: But one of the instructions is actually intuitive :(
<StevenK> wgrant: The log level change is going to cause widespread changes to doctests
<wgrant> StevenK: Indeed... We'll see how widespread it is.
<wgrant> StevenK: If it's too hard, use a different log method for each, but still dedupe.
<StevenK> wgrant: Changes made, and your branch merged.
<StevenK> +16/-14 for nascent-announcements.txt
<StevenK> Which is much less than I feared.
<StevenK> wgrant: _sendNotification folded into notify()
<LPCIBot> Project windmill-db-devel build #328: STILL FAILING in 1 hr 9 min: https://lpci.wedontsleep.org/job/windmill-db-devel/328/
<wgrant> StevenK: Hexcellent.
<wgrant> Hmm, we haven't harrassed the LOSAs at all today :(
<wgrant> We are not up to our usual standard.
 * StevenK looks at merging send_mail() into notify()
<wgrant> StevenK: Yay
<wgrant> Er.
<wgrant> No.
<StevenK> Indeed, we call it twice
<StevenK> [13:30] < wgrant> send_mail can be pretty trivially folded into _sendNotification.
<StevenK> :-)
<wgrant> StevenK: Oh, right, send_mail, not send_message.
<wgrant> send_mail can be folded in.
<wgrant> send_message should be merged into _sendMail, but that first requires that reject_changes_file use the latter instead.
<StevenK> wgrant: However, we call send_mail twice -- once for annoucements and once for the message itself.
<wgrant> StevenK: The args are the same except for action, recipients, from_addr and bcc_addr, AFAICT.
<wgrant> Also, the subject can be shared, but the body cannot.
<wgrant> Possibly reintroduce the do_send_mail closure.
<wgrant> But at the end this time.
<wgrant> Will save like 13 arguments and a confusing function name.
<lifeless> poolie: you wanted to catch up more generally ?
<StevenK> wgrant: notification.py |   18 ++----------------
<StevenK> wgrant: From killing send_message
<jtv> StevenK: I notice that DSDJob.run finds its DSD by {distroseries, sourcepackagename}.  Doesn't that become ambiguous with multi-parent?
<StevenK> jtv: Yes, that should change.
<StevenK> wgrant: Make that +2/-19
<jtv> StevenK: do we have a bug/card for that?
<StevenK> I can't recall off the top of my head
<jtv> OK, I'll look.
<wgrant> StevenK: :)
<jtv> StevenK: bug 758906 may be a bit general
<_mup_> Bug #758906: Allow a derived series to have multiple parents <derivation> <Launchpad itself:Triaged> < https://launchpad.net/bugs/758906 >
<StevenK> AssertionError: Invalid recipient: '' in ['D', 'a', ...
 * StevenK digs
<wgrant> StevenK: Does that mean that send_mail and send_message are both dead?
<StevenK> send_mail is not, send_message is
<wgrant> I am disappoint.
<StevenK> Of course you are.
<wgrant> I'm hoping that notification.py drops below 500 lines.
<poolie> hi lifeless
<lifeless> o/ poolie
<StevenK> wgrant: 509
<poolie> sure
<poolie> now if you like
<wgrant> StevenK: Victory is near.
<lifeless> poolie: that would be great. shall we try skype?
<StevenK> wgrant: What was your plan to kill send_mail?
<wgrant> StevenK: Give me a sec.
<StevenK> wgrant: My branch has also fallen behind Full HD
<wgrant> StevenK: Hm, I have an odd failure in distroseriesqueue-notify.txt.
<wgrant> Have you fixed that one yet?
<StevenK> I've not been running the soyuz tests
<wgrant> r13104 broke it.
<StevenK> :-(
<StevenK> Ah, that could be the DEBUG vs INFO
<wgrant> Hm, possibly.
<wgrant> Doesn't look like it, though.
<StevenK> I'll dig when archiveuploader stops running
<wgrant> Also, just found a bug in r13104
<wgrant> -    for bpr in bprs:
<wgrant> -        names.add(bpr.build.source_package_release.name)
<wgrant> -        version = bpr.build.source_package_release.version
<wgrant> +    elif bprs:
<wgrant> +        names.add(bpr[0].build.source_package_release.name)
<wgrant> +        version = bpr[0].build.source_package_release.version
<wgrant> You need to add the names even if there are sprs.
<wgrant> Just not the version.
<wgrant> Oh.
<wgrant> Heh.
<wgrant> It was the DEBUG/INFO change: the unification added some new fields.
<StevenK> Ah
<StevenK> Sorry, I fixed that locally
<StevenK> In r13105
<wgrant> Great.
<StevenK> Do you have a diff for the -notify change?
<wgrant> Just pushing.
<StevenK> Or have you not fixed it?
<wgrant> Oh.
<wgrant> No, that one I haven't fixed.
<wgrant> There will be many more like it.
<wgrant> I'm pushing the send_mail merge.
<wgrant> Done.
<StevenK> I've just pushed r13107
<wgrant> Hm.
<wgrant> Have you?
<StevenK> Pushing
<StevenK> When that returns, I'll re-merge your branch
<StevenK> Hm. No progress either
<StevenK> connect(4, {sa_family=AF_INET, sin_port=htons(443), sin_addr=inet_addr("91.189.89.225")}, 16
<StevenK> Odd
<StevenK> ssh: connect to host bazaar.launchpad.net port 22: Connection timed out
<StevenK> Hmmm
<StevenK> But ssh -vv connects
<wgrant> WFM :(
<StevenK> So I'm confused
<StevenK> Ah, there it goes
<StevenK> wgrant: Sorry, pushed.
 * StevenK attempts to merge in wgrant's latest branch, dies from conflicts
<wgrant> :(
<StevenK> wgrant: I can't make sense of them either
<wgrant> Oh?
<wgrant> Let me see.
<wgrant> zomg conflicts
<wgrant> Oh.
<wgrant> You demolished _sendNotification.
<wgrant> I see.
<StevenK> You said to? :-)(
<StevenK> s/(//
<wgrant> StevenK: Pushed. You should just be able to pull.
<wgrant> 497 lines.
<StevenK> \o/
 * StevenK looks at distroseriesqueue
<wgrant> StevenK: Can you also fix bcc_addr calculation?
<wgrant> StevenK: It should use the spr name and then fall back to the first bpr name.
<wgrant> StevenK: And not duplicate the template.
<StevenK> wgrant: http://pastebin.ubuntu.com/613612/
<wgrant> StevenK: elif bprs
<wgrant> StevenK: I'm still not quite sure how notification.py has ended up 20% shorter, despite some stuff being moved into it from queue.py...
<StevenK> Haha
<StevenK> Iz bonus
<StevenK> wgrant: Fixed distroseriesqueue*
<wgrant> StevenK: I would avoid further elision like the nascentupload-announcements.txt fix.
<wgrant> It may conceal unexpected Bcc and From additions.
<StevenK> :-(
<StevenK> How did you know that's how I fixed them?
<wgrant> Heh
<wgrant> I was actually typing that before you said you'd fixed them :(
<StevenK> Oh, so since I've done it, I can leave it? :_)
<wgrant> No.
<StevenK> Should I fix nascentupload-announcements.txt as well?
<wgrant> StevenK: Ideally.
<lifeless> cody-somerville: your bug is fixed; will be deployed monday
<StevenK> % psql -l | egrep -c 'ftest_[0-9]'
<StevenK> 8
<StevenK> With no test suite running :-/
<StevenK> wgrant: distroseriesqueue* fixed properly
<wgrant> Great.
<wgrant> I think this is just about done.
<StevenK> The only thing I can think of is the hasattr() hack
<wgrant> Yes.
<wgrant> That's only used by notify(), right?
<wgrant> An extra arg at that single level won't hurt too much.
<StevenK> No, reject_changes_file
<wgrant> Well, yes, but that's called immediately by notify()
<wgrant> So it doesn't affect the majority of the code.
<StevenK> Bah, I called it reject_changes_file*s* ?
<wgrant> :( fixit
<StevenK> Fixed
<StevenK> wgrant: r13111 pushing
<StevenK> Hmm, it's back over Full HD again.
<StevenK> wgrant: Diff in the MP has updated, once more with feeling?
<wgrant> StevenK: Soyuz tests are passing?
<StevenK> Still running archivepublisher
<StevenK> But I can let you know in 30 minutes :-(
<wgrant> So, I'll rereview in a sec.
<StevenK> wgrant: Running soyuz now, no failures in archivepublisher
<wgrant> What are your next steps?
<wgrant> After this branch?
<StevenK> I was going to say hook this into the package copier, but that is dangerous
<wgrant> It might be worth seeing how hard it is to extract reject_changes_file() from notify(), making archiveuploader call it directly.
<wgrant> But if you want to go directly to the copier goal, you'll need to wean notify() and co. off the changesfile.
<StevenK> I tried that, archiveuploader deals with both forms of rejection in one method
<wgrant> Stop using it for Changed-By and Maintainer, for example.
<wgrant> Bah.
<wgrant> That's stupid.
<wgrant> But OK.
<StevenK> My concern with the package copier calling it is that notify() is fairly heavyweight, and the package copier needs to be lean and fast
<wgrant> Does it?
<StevenK> Otherwise copying times out and people yell at me. :-(
<wgrant> These copies won't be done inline.
<StevenK> Ahh
<wgrant> Well, for security copies they probably will for the short-term.
<StevenK> So call notify() in PCJ, *not* do_copy
<wgrant> But this is pretty light-weight...
<wgrant> No, do_copy.
<wgrant> But only security copies and PCJ will use it for now.
<StevenK> That's fairly easy guarded
<persia> On a vaguely related note, did the job to pro[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[Ccess all the old copyright files ever run?
<wgrant> No need.
<wgrant> The other types of copies are unsupported.
<wgrant> persia: Old changelog files? Yes.
<persia> Oh, excellent
<StevenK> persia: It took one month to run
<StevenK> persia: And processed roughly 580,000 SPRs.
<wgrant> Only 'cause you threw away my good one :P
<persia> StevenK: It took about one year to schedule, so I'm happy with the run time :)
<StevenK> So we thanked it by deleting it from the codebase.
<StevenK> wgrant: I got lifeless'd
<StevenK> wgrant: And I didn't throw it away, it mutated ...
<StevenK> wgrant: Uh? What about ppa-to-ppa copies?
<wgrant> StevenK: They shouldn't announce.
<wgrant> We will handle that in the integration.
<StevenK> Clearly
<wgrant> But it should be handled with the rest of the do-not-announce logic.
<wgrant> Not by not calling notify().
<StevenK> Right
<StevenK> wgrant: Are you reviewing, or waiting for soyuz test results?
<StevenK> Or doing something else entirely :-)
<wgrant> Will look shortly.
<StevenK> wgrant: 2 failures in soyzu
<StevenK> *soyuz
<StevenK> They're both doctests, so I guess it's more INFO vs DEBUG fallout
<wgrant> StevenK: Have you disposed of _filename yet?
<StevenK> wgrant: Yes
<StevenK> Trying to work out the xx-queue-pages.txt failure
<StevenK> soyuz-set-of-uploads failed due to INFO vs DEBUG
<wgrant> The rejection message seems to be None.
<StevenK> Yes, I think I've found the bug.
<StevenK> And found a stray place _filename hasn't been erricated yet
<StevenK> wgrant: r13112 pushed.
<StevenK> wgrant: Did you notice my drive-by to drop pocketsuffix from queue?
<wgrant> StevenK: I WTF'd at it for a minute and then worked out it was a drive-by, yes.
<wgrant> *Not* really the right sort of branch for drive-bys, but I won't hurt you too much.
<StevenK> Well, true, it's large and hurty enought
<StevenK> s/t$//
<StevenK> wgrant: Diff updated.
<wgrant> StevenK: The summarystring stuff could do with a shuffle.
<StevenK> wgrant: Er, which bit?
<wgrant> I'd move the buildSummary bit (line 144) into an else block after the "if action == 'rejected'"
<wgrant> Since the rejected bit clobbers it.
<StevenK> wgrant: Done
<wgrant> StevenK: Could you also de-underscore and de-CamelCase the remaining functions?
<wgrant> Apart from that I think it is pretty much landable.
<wgrant> We will see Julian scream in a few minutes, I guess?
<mrevell> How do?
<StevenK> wgrant: Does mean you're willing to Approve it?
<StevenK> s/s m/s that m/
<jml> huwshimi: hi
<huwshimi> jml: Hello!
<StevenK> wgrant: r13113 pushed.
<jml> huwshimi: let's talk :)
<huwshimi> jml: Skype?
<StevenK> wallyworld_: Are you still the OCR?
<wallyworld_> StevenK: perhaps not anymore
<wallyworld_> i should change the topic
 * wallyworld_ is about to eat dinner
* wallyworld_ changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer:  | Critical bugs:209 - 0:[######=_]:256
* StevenK changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: - | Critical bugs:209 - 0:[######=_]:256
<StevenK> I wonder if that 209 count is still right
<wgrant> It's 210 now, I think.
<wgrant> Possibly more.
<wallyworld_> it seems to be stuck around 200 :-(
<wgrant> It's decreasing.
<wgrant> But then stuff gets escalated.
<wallyworld_> right
<wgrant> http://webnumbr.com/launchpad-critical-bugs
<wallyworld_> nice graph. i hadn't seen that before
<wgrant> There are 9 escalated bugs.
<StevenK> wgrant: The diff has updated, can haz comment/approve?
<wgrant> bigjools must be subjected to it first.
<StevenK> bigjools: https://code.launchpad.net/~stevenk/launchpad/refactor-notification/+merge/61919
<bigjools> bzr repacking a commit when you just want to push before your battery runs out is sub optimal
<bigjools> hey wallyworld_ you're up late
<adeuring> goodmorning
<wgrant> The notification.py diff is pretty much useless, but the new file is short and pretty readable.
<wgrant> And self-contained.
<lifeless>  http://webnumbr.com/.join(launchpad-oops-bugs.all,launchpad-timeout-bugs.all,launchpad-critical-bugs.all)
* lifeless changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: - | Critical bugs:206 - 0:[######=_]:256
<jtv> Who's up for a review?  https://code.launchpad.net/~jtv/launchpad/db-bug-788526/+merge/62623
<adeuring> jtv: I'll look
* adeuring changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: adeuring | Critical bugs:209 - 0:[######=_]:256
<jtv> thanks adeuring
<LPCIBot> Project windmill-db-devel build #329: STILL FAILING in 1 hr 8 min: https://lpci.wedontsleep.org/job/windmill-db-devel/329/
<StevenK> Dear Unity, why are you making my laptop screen flicker?
<danilos> StevenK: it's not Unity, it's me!
<StevenK> danilos: How are you doing that, then? :-)
<danilos> StevenK: just saying I might drop by and then it shudders (manifests as flicker ;))
<StevenK> wgrant: bigjools has approved, your turn
<adeuring> jtv: r=me
<jtv> thanks!
<jml> Food time!!#@@!#
<jml> (I'm pretty excited about it)
<wgrant> StevenK: It is done.
<bigjools> wgrant: I'm sad that you're not on maintenance rotation any more, I was going to press-gang you into fixing the FatalUploadError-generating-an-oops-for-bad-gpg-signatures annoyance
<bigjools> since every single soyuz oops for the last few days has been those
<wgrant> bigjools: As I said, I was looking at that last week, but got distracted by the whole epic failure thing.
<wgrant> Yeah.
<wgrant> The missing mandatory field errors are now replaced by those.
<wgrant> Since it does signature verification first.
<StevenK> wgrant: bigjools wished for docstrings, so 527
<wgrant> bigjools is a terrible person.
<wgrant> But probably correct.
<StevenK> Haha
 * bigjools is merely enforcing the existing rules ;)
<wgrant> At least we removed a few functions that would have also needed 15-20 line docstrings.
<bigjools> yarp
<wgrant> I'm sure I can refactor some more out of it, but this is workable, landable, somewhat reviewable, and makes me cry less.
<lifeless> gpghandler-service ftw
<wgrant> Yes
<wgrant> Hopefully first thing.
<lifeless> wgrant: have you seen https://dev.launchpad.net/ArchitectureGuide/ServicesRequirements ?
<wgrant> I have.
<lifeless> good ;)
<wgrant> But it's 10pm on your Friday night, so you are not here.
<lifeless> I am writing up a mail to the list even now
<lifeless> I have a cat nestled in the crook of my legs, wife on the couch beside me and tv on
<wgrant> Seeing KPI used in a technical document makes me cringe, but I will survive.
<LPCIBot> Project windmill-devel build #145: STILL FAILING in 1 hr 9 min: https://lpci.wedontsleep.org/job/windmill-devel/145/
<jml> actually, still no food, and I have to run an errand.
<jml> back soon.
<jml> back
<adeuring> is anybody aroubd with Twisted-fu? I'm trying to let the test-keyserver return a 500 error when a non-existing key is looked up. (That's what our real-life keyserver does, really)
<wgrant> A 500?
<wgrant> That would be an SKS bug.
<adeuring> Using ErroPage() seems to work, but how can I test this?
<adeuring> wgrant: yes, it is -- but should we fix this too?
<adeuring> wgrant: just try http://keyserver.ubuntu.com:11371/pks/lookup?search=0x6F0DAED1&op=vindex
<adeuring> wgrant: but anyway, even if it would return a 404: My question is: How can I test that our test keyserver returns a 404 or 500 or whatever other error?
<wgrant> Nobody has done anything significant to zeca in quite a while... I'm not sure if it's tested.
 * wgrant looks around.
<wgrant> Oh, the tests aren't too bad.
<wgrant> adeuring: So you're not sure how to assert an HTTP error in test_web?
<adeuring> wgrant: right
<adeuring> wgrant: Returning ErrorPage(...) in web.py _seems_ to work, bt I'd like to test that properly...
<wgrant> adeuring: I presume twisted.web.client.getPage calls the errback if an error occurs.
<wgrant> Yeah: "Download a page. Return a deferred, which will callback with a page (as a string) or errback with a description of the error."
<adeuring> wgrant: I have no real clue about twisted... So I would have to add an error callback?
<wgrant> adeuring: getPage returns a Deferred, to which you can addCallback and addErrback.
<wgrant> adeuring: You'll see it currently uses addCallback to add an assertion.
<wgrant> On the success case.
<adeuring> wgrant: ah, right! thanks!
<spiv> adeuring: twisted.trial.unittest.TestCase has an assertFailure method that may help: http://twistedmatrix.com/documents/current/api/twisted.trial.unittest._Assertions.html#failUnlessFailure
<adeuring> spiv: thanks!
<spiv> adeuring: (despite the API doc link, the preferred name for that method is assertFailure not failUnlessFailure)
<spiv> adeuring: the basic trick to know is that trial's TestCase allow you to return a Deferred from your test method (and setUp etc), and the test will wait for the result/failure of that before completing.
<adeuring> spiv: ok
<jml> spiv: actually, most of our Twisted tests don't use trial TestCase
<spiv> jml: *shudder*
<jml> spiv: no, it's actually awesome
<spiv> jml: awesome can make a person shudder too!
<jml> spiv: 'run_tests_with = AsynchronousDeferredRunTest'
<spiv> jml: the interface for test writers is approximately the same?
<jml> spiv: yeah. things like the assert you mentioned are free-floating functions
<jml> spiv: and it's more strict with reactor cleanup
<jml> (and there's more helpers for debugging)
<jml> (and you need to do less multiple inheritance)
<spiv> jml: \o/
<jml> spiv: http://readthedocs.org/docs/testtools/en/latest/for-test-authors.html#twisted-support
 * henninge lunches
* bac changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: adeuring, bac | Critical bugs:209 - 0:[######=_]:256
<bac> hello adeuring.  busy today?
<adeuring> bac: not really
<henninge> Hi! I have re-installed my laptop and used rocketfuel-setup to get my LP setup going again. I also had saved my homedir and copied it back to the new installation, so I may have mixed stuff.
<henninge> Now make schema won't work, it breaks off like this: http://paste.ubuntu.com/613727/
<henninge> The last line is about "ident authentification failing for user henning"
<henninge> What's missing?
<henninge> Would rocketfuel-setup not create a user for my account?
<StevenK> rf-setup does not run utilities/launchpad-database-setup
<henninge> oh
<StevenK> henninge: "utilities/launchpad-database-setup $(whoami)"
 * henninge should have read that again
<henninge> StevenK: thanks!
<StevenK> I may be wrong, the rf-setup code makes me sob.
<henninge> no, it's coming back to me that we split that back when we open sourced because people did not like rf to mess with their database.
<StevenK> Well, truth be told, rf-setup is not something to be run often
<henninge> exactly
<henninge> make schema is doing what it should now. ;-)
<henninge> now henninge can run tests ...
 * StevenK high fives henninge 
<henninge> ;-)
<bac> hi sinzui, gary says you brought up bug 186660 to him recently as something we might look at soon.  it would possibly close several bugs but none are marked critical.
<_mup_> Bug #186660: Launchpad shouldn't store wiki names <lp-registry> <users> <Launchpad itself:Triaged> < https://launchpad.net/bugs/186660 >
<sinzui> bac: That is right. We have ignored the issue for many years. I was just noting that with the SSO change last year. We might be able to close all wikiname bugs except the one saying it should be removed
<jtv> allenap: you've got a test here called test_requestUpgrade_is_efficient.  It seems to assert that the query count does not increase with the number of jobs it needs to create.  I'm not so sure Storm will let us do that.
<jtv> bac: I've got an oversized and tedious MP here, and I have to leave.  But _if_ you have no problem with that, then I'd be grateful if you could review!  https://code.launchpad.net/~jtv/launchpad/db-bug-788956/+merge/62669
<bac> jtv: you're a natural-born salesman
<jtv> Yuck.
<jtv> I've done sales, but I'm only good at it if I believe in what I sell.
<bac> who could pass oversized, tedious, and unattended?
<jtv> You could if you wanted to!
<bac> jtv: i'll have look in a bit
<jtv> Great, thanks
<jtv> Have a good weekend, everyone!
<allenap> gmb: Do I get extra karma for getting two [r=allenap]s into a commit message? ;)
<gmb> allenap: Sadly not :).
<gmb> Hahah. "Strongly suggest you use lp-land." I did and it added the extra tags. Unhelpful.
<sinzui> jcsackett: so you have time to talk?
<jcsackett> sinzui: sure. mumble or sip?
<sinzui> sip?
<sinzui> We miss connected
<jcsackett> we did. :-P
<LPCIBot> Project windmill-db-devel build #330: STILL FAILING in 1 hr 12 min: https://lpci.wedontsleep.org/job/windmill-db-devel/330/
<abentley> adeuring or bac: could you please review https://code.launchpad.net/~abentley/launchpad/duplicate-msgid/+merge/62697 ?
<adeuring> abentley: sure
<abentley> adeuring: thanks.
<adeuring> abentley: r=me
<abentley> adeuring: thanks.
<LPCIBot> Project windmill-devel build #146: STILL FAILING in 1 hr 13 min: https://lpci.wedontsleep.org/job/windmill-devel/146/
<LPCIBot> Project windmill-db-devel build #331: STILL FAILING in 1 hr 7 min: https://lpci.wedontsleep.org/job/windmill-db-devel/331/
<abentley> bac: does OOPS-1963SMS24 make any sense to you?  It's apparently the cause of bug #784008.
<_mup_> Bug #784008: problem when pushing fix for a fix-committed bug <branch-scanner> <oops> <Launchpad itself:Triaged> < https://launchpad.net/bugs/784008 >
 * bac looks
<gmb> adeuring, bac : Could one of you take a look at https://code.launchpad.net/~gmb/launchpad/choose-affected-product-oops-bug-370117/+merge/62704 for me?
<bac> gmb: i will
<gmb> bac: Thankee
<bac> abentley: the OOPS does not make sense to me b/c the code it references no longer exists.
<abentley> bac: Thanks.  I guess the bug probably is fixed, too, but I'll poke at that later.
<bac> abentley: i hope so...
<bac> gmb: short, sweet, r=bac
<gmb> bac: Ta.
<abentley> bac: I have trouble seeing why the bug and the oops would be connected, so it's possible another issue was at play for the oops, and the bug still exists.
* adeuring changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: bac | Critical bugs:209 - 0:[######=_]:256
<nigelb> Hi, I just filed bug 789222, which I introduced with my "fixes"
<_mup_> Bug #789222: Sprints and specification subscribers are really not alphabetically sorted <Launchpad itself:New> < https://launchpad.net/bugs/789222 >
<nigelb> should I be separating the sprints and specification into separate branches and bugs or together is fine?
<nigelb> I know the fix for it and I'm working on it.
<jml> nigelb: if the branch gets too complex to think about or review, then you'll know you should have split it.
<jml> nigelb: but probably it's ok to do them together.
<nigelb> jml: Its basically just a small change of one lines each, unless you insist on writing new test cases to catch that.
<jml> nigelb: well, if you introduced a bug with your fixes, it shows that we don't have enough tests for this area
<jml> nigelb: so, yes I will insist :)
<nigelb> jml: heh :D
<nigelb> I knew you'd insist :D
<nigelb> jml: I don't know if I want to touch the doctests for the specifications.
<jml> nigelb: give it a try at least.
<nigelb> I'm tempted to just write new unit tests.
<nigelb> Wait, that's a trap. I'll end up writing a whole bunch of unit tests.  I take that back :p
<jml> nigelb: you'll need some kind of test. do which ever is easier for you.
<nigelb> jml: :)
<nigelb> Sorry for this sort of mess I caused :)
<jml> np.
<nigelb> jml: When is the sprint in Dublin?
<nigelb> I guess the pie on the face is less likely now :(
<jml> nigelb: June 27th: 1 month
<jml> nigelb: yes, quite unlikely.
<jml> nigelb: but still possible.
<nigelb> heh
 * jml is off
<jml> g'night all
<jml> see you in a bit over a week's time.
<nigelb> ouch, I messed up postgres :(
<nigelb> apparently, at some point I accidentally uninstalled it
<nigelb> I ran rocketful-setup again, but that didn't get the schema on there
<nigelb> how do I fix that?
<nigelb> nvm, I think I fixed it.
<nigelb> No I didn't, sigh.
<LPCIBot> Project windmill-db-devel build #332: STILL FAILING in 1 hr 8 min: https://lpci.wedontsleep.org/job/windmill-db-devel/332/
<flacoste> anyone knows how to make a source package branch the official one?
<LPCIBot> Project windmill-devel build #147: STILL FAILING in 1 hr 9 min: https://lpci.wedontsleep.org/job/windmill-devel/147/
<nigelb> bac: I'm picking that task up. Just that I broke my local dev set up and trying to figure out if I have to reinstall the whole thing.
<flacoste> bug 347770
<_mup_> Bug #347770: UI for making a branch an official package branch <lp-code> <package-branches> <ui> <Launchpad itself:Triaged> < https://launchpad.net/bugs/347770 >
<bac> nigelb: great to hear, except for the borked machine
<nigelb> Note to self: reinstalling rabbitmq was a bad idea.
<flacoste> james_w: is there an existing tool that uses the API to set official package branches? (is it done as part of package-importer?) or should I write my own (i want to set official package branches on another distribution)
<james_w> flacoste, there's set_official.py in lp:udd I think
<flacoste> james_w: i'll have a look
* bac changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: - | Critical bugs:209 - 0:[######=_]:256
<LPCIBot> Project windmill-db-devel build #333: STILL FAILING in 1 hr 7 min: https://lpci.wedontsleep.org/job/windmill-db-devel/333/
<poolie> has anyone seen this error from ec2:
<poolie> AttributeError: 'Entry' object has no attribute 'queue_status'
<poolie> what a silly error too
<abentley> poolie: it sounds like ec2 was expecting a merge proposal but got something else.
<poolie> yes
<poolie> i gave it a branch
<poolie> perhaps i'll shave that restfulclient yak
<LPCIBot> Project windmill-devel build #148: STILL FAILING in 46 min: https://lpci.wedontsleep.org/job/windmill-devel/148/
<nigelb> anyone around to review a branch?
<nigelb> poolie: You around?
<nigelb> I'm headed to bed. If someone can review https://code.launchpad.net/~nigelbabu/launchpad/789222-sort-fix/+merge/62739 that'd be awesome :)
<poolie> hi nigelb
<poolie> replied
<LPCIBot> Project windmill-db-devel build #334: STILL FAILING in 43 min: https://lpci.wedontsleep.org/job/windmill-db-devel/334/
<lifeless> *stretch*
<poolie> hi there
<lifeless> flacoste: there is an API call I believe. No web UI atm.
<flacoste> lifeless: yep
<lifeless> I see the futher discussion now ;)
<poolie> lifeless: if you're in lp mode, can you tell me if there's a better way to write https://code.launchpad.net/~nigelbabu/launchpad/789222-sort-fix/+merge/62739
<poolie> tx
<lifeless> no probs
#launchpad-dev 2011-05-28
<LPCIBot> Project windmill-devel build #149: STILL FAILING in 45 min: https://lpci.wedontsleep.org/job/windmill-devel/149/
<wallyworld> sinzui:
<wallyworld> sinzui: is there a reason why css_id doesn't like '_'
<wallyworld> ie it replaces '_' with '-'
<lifeless> wallyworld: css2 back -ages- ago forbade underscores
<lifeless> wallyworld: since permitted, but EOldBrowsers
<cjohnston> Can someone take a look at bug #787650.. It appears to be in lp:launchpad already.. Should it be either fix comitted or fix released?
<_mup_> Bug #787650: Add an IRC nick formatter <disclosure> <irc> <person-picker> <Launchpad itself:In Progress by wallyworld> < https://launchpad.net/bugs/787650 >
<cjohnston> or am I missing something
<lifeless> cjohnston: fix committed happens when it gets through CI
<lifeless> so in lp:launchpad/stable
<cjohnston> 'ok
<lifeless> aka lp:~launchpad-pqm/launchpad/stable
<wallyworld> lifeless: hmm. even though css_id() strips them, most all of our form fields still have them
<lifeless> fix released happens when its live and the bug corrected
<wallyworld> ie css_id is not used iniversally
<lifeless> this may explain some old browser issue - but it would need to be really old browsers :)
<wallyworld> lifeless: so since fields like target_branch etc are rendered in the current codebase, do we want to retain the behaviour of css_id() to strip the _
<lifeless> wallyworld: I'm not enough of a browser-changelog-guru to know
<wallyworld> np. i'm asking because it would allow me to eliminate some duplicate code
<lifeless> I suggest some research - find which version of IE started supporting them, for instance
<wallyworld> i'll have a look. but given we render nodes with ids containing _ right now, it's kinda moot, no?
<lifeless> well
<lifeless> we know we have ie6 issues
<lifeless> this might be it, for instance.
<lifeless> given we know we have some browser issues, its probably a good idea to exclude this change from those issues before making it ;)
<wallyworld> could be. i doubt we're going to address any ie6 compatability issue though?
<lifeless> we would like to
 * wallyworld hates ie6 :-(
<lifeless> yes
<lifeless> http://noscope.com/journal/2005/01/showing-css-to-ie-only-the-underscore-hack
<lifeless> ah, thats different
<lifeless> http://www.w3.org/TR/CSS21/syndata.html#characters is what permits _. http://www.w3.org/TR/2008/REC-CSS2-20080411/syndata.html#q4 is the older spec.
<lifeless> 6.0 was release in 2001
<lifeless> http://www.w3.org/TR/2002/WD-CSS21-20020802/ is the earliest 2.1 draft I've found so far
<wallyworld> lifeless: from what i can tell, ns4 has an issue with _. ie6 is ok
<wallyworld> although with ie6, _ as the first character has special meaning
<lifeless> yeah
<lifeless> wth they thought that was a good idea, we'll never know
<wallyworld> as you point out above, _ as the first char is for ie6 only properties
<wallyworld> so i think it should be ok to allow _, but not as the first char?
<lifeless> probably
<lifeless> I suspect you'll want to run it past e.g. sinzui
<wallyworld> yeah
<wallyworld> it would be nice to be able to do it so that we can have a cleaner codebase and consistent id generation for popups vs other fields
<LPCIBot> Project db-devel build #588: FAILURE in 5 hr 44 min: https://lpci.wedontsleep.org/job/db-devel/588/
<LPCIBot> Project windmill-db-devel build #335: STILL FAILING in 44 min: https://lpci.wedontsleep.org/job/windmill-db-devel/335/
<LPCIBot> Project windmill-devel build #150: STILL FAILING in 46 min: https://lpci.wedontsleep.org/job/windmill-devel/150/
<LPCIBot> Project windmill-db-devel build #336: STILL FAILING in 44 min: https://lpci.wedontsleep.org/job/windmill-db-devel/336/
<nigelb> poolie, lifeless: Thanks!
<LPCIBot> Yippie, build fixed!
<LPCIBot> Project db-devel build #589: FIXED in 5 hr 21 min: https://lpci.wedontsleep.org/job/db-devel/589/
<LPCIBot> Project windmill-db-devel build #337: STILL FAILING in 1 hr 7 min: https://lpci.wedontsleep.org/job/windmill-db-devel/337/
<LPCIBot> Project windmill-db-devel build #338: STILL FAILING in 1 hr 5 min: https://lpci.wedontsleep.org/job/windmill-db-devel/338/
<LPCIBot> Project db-devel build #591: FAILURE in 5 hr 21 min: https://lpci.wedontsleep.org/job/db-devel/591/
<LPCIBot> Project db-devel build #592: STILL FAILING in 5 hr 21 min: https://lpci.wedontsleep.org/job/db-devel/592/
#launchpad-dev 2011-05-29
<LPCIBot> Project windmill-devel build #151: STILL FAILING in 1 hr 8 min: https://lpci.wedontsleep.org/job/windmill-devel/151/
<magcius> how did you guys hack apart zope so that you wouldn't have to use ZODB^W pickle?
<wgrant> magcius: It basically just needs a custom publication class.
<wgrant> I tried a similar thing in 2007 or so, and basically ended up copying the base publication and deleting ZODBish bits, and it worked fine.
<wgrant> It wasn't practical to inherit much of it.
<wgrant> Because ZODB is so ingrained.
<wgrant> But it didn't end up that big, and that was the only thing to which I had to make significant changes, IIRC.
<wgrant> I
<wgrant> I may still have a copy of it around somewhere.
<magcius> zope hacker I'm talking to says that must "wreak havoc on security"
<wgrant> ... why?
<wgrant> The security machinery doesn't depend on ZODB :/
<wgrant> That would be very Zope, but it's not true :)
<magcius> "because the ZODB allows per object access control"
<wgrant> That's got very little to do with ZODB.
<wgrant> Security proxies work on Python objects.
<wgrant> Not in the ZODB.
<magcius> wgrant, can you point me to this code somewhere in LP?
<wgrant> It'll be in lib/canonical/launchpad/webapp somewhere.
<wgrant> canonical.launchpad.webapp.publication has our very custom publication.
<wgrant> May also need custom RequestPublicationFactories.
<wgrant> I forget where they live.
 * wgrant greps.
<wgrant> Possibly not.
<wgrant> Anyway, the critical bit is getApplication on the publication.
<wgrant> You can see here it gets the ILaunchpadRoot utility.
<wgrant> Rather than taking the root from a ZODB.
<wgrant> There's also lovely.zetup which does ZODB and ZConfig removal, IIRC>
<wgrant> But it's been a while :)
<lifeless> wgrant: magcius: the problem is that zopes entire programming model is based around aozdb
<lifeless> wgrant: magcius: so we're *still* paying for this hack in terms of performance
<magcius> heh
<magcius> if you could "do it again" today, what would you do
<lifeless> well
<lifeless> https://dev.launchpad.net/ArchitectureGuide/Services
<lifeless> we are doing it again :)
<lifeless> magcius: your zope hacker is right, it wreaks havoc
<lifeless> magcius: it works, but its terribly terribly slow unless we're careful to preload things
<lifeless> magcius: in terms of choosing an application stack, if we were setting out to do LP from scratch, I couldn't really say.
<magcius> OK.
<magcius> I'm getting into the scary world of web development here, and all the existing frameworks suck :(
<wgrant> I don't like Django for this sort of thing.
<wgrant> Zope sucks, but it is semi-appropriate for what LP is.
<wgrant> Django is less appropriate IMO.
<lifeless> I'd like to do something with http://liftweb.net/
<magcius> Here's what I like about Zope: the security/auth system is a bit neato, and I do like the interface and adapter system
<lifeless> mmm
<magcius> everything else? ZODB? meh.
<wgrant> The security stuff is awesome, but is really slow with SQL.
<lifeless> so I don't value those at all
<magcius> lifeless, I like that you have to define users and roles and what can access what.
<magcius> so that we don't get another diaspora :D
<lifeless> they don't really fit in a hugely dynamic language like python
<magcius> do you mean the auth system or z.i?
<lifeless> z.i.
<magcius> I assume you've read glyph's infamous post?
<lifeless> ignoring some of the fugly globals based implementation details
<lifeless> magcius: I think I have
<magcius> I just treat z.i as a better isinstance right now
<magcius> but not __subclasshook__. Because __subclasshook__ is all about trickery and deceit.
<magcius> lifeless, http://glyph.twistedmatrix.com/2009/02/explaining-why-interfaces-are-great.html
<lifeless> magcius: yes, that one
<lifeless> magcius: basically, I think in python you want to embrace the duck
<magcius> of course
<magcius> z.i doesn't disallow or try to prevent that
<magcius> but there are sometimes when you have an A, and you want a B
<lifeless> I argue that using z.i. shows a failure to embrace the duck.
<magcius> adaptation is perfect for that
<lifeless> its *a* solution.
<magcius> other solutions are isinstance
<magcius> but isinstance is a bit closed... you have to modify the method if you want to add new types...
<lifeless> isinstance, delegation, multimethods, type factories
<magcius> etc etc.
<lifeless> offhand
<lifeless> there are probably other approaches too.
<magcius> never heard of any of the latter
<magcius> to make isinstance extensible, they added ABCs and __subclasshook__
<lifeless> yes
<lifeless> this was a bit myopic.
<magcius> which allows you to lie: you can say you're a B, even though you just look almost like a B
<magcius> to me, that's the wrong way of doing it
<lifeless> having a problem with isinstance, they made it worse :)
<magcius> with z.i, you define what things you need in an interface, and then adaptation allows you to turn anything into what you need
<magcius> instead of lying
<lifeless> sure, if you don't mind the costs
<lifeless> and the inflexability
<lifeless> I dislike the costs and find the inflexability more a burden than a benefit, most of the time.
<magcius> what costs do you see?
<lifeless> its a global registry
<lifeless> you run more code when importing your app
<lifeless> You have to double-define everything
<magcius> double-define?
<lifeless> class IFoo(Interface): def foo():pass
<magcius> you don't have to do that, although you should for documentation.
<lifeless> class Foo(object): implements(IFoo); def foo(self): do stuff.
<magcius> z.i won't check to make sure you're implementing everything
<magcius> you can just do class IFoo(Interface): pass if you want
<lifeless> magcius: you asked what the costs are.
<lifeless> if you are writing in this style, this is something you do: its a cost.
<magcius> OK, sure.
<lifeless> magcius: we use interfaces very extensively in LP; I'm quite familiar with the system.
<magcius> Right.
<lifeless> I'm not arguing against it from a position of ignorance :)
<magcius> I wish there was something that provided adaptation as a language feature.
<lifeless> Oh, and I should add confusing-code to the list.
<magcius> z.i code is extremely messy, z.c more so
<lifeless> adaption is a poor way to write code, IMNSHO.
<lifeless> i don't mean the implementation of z.i. is confusing : I mean that adaption centric code is confusing for readers.
<magcius> if it was a Python language feature instead of ABCs, I would cheer.
<magcius> but yeah
<magcius> additionally, having to define roles and explicitly say "this is public" I like
<lifeless> If IFloat and IString existed, when would you use "%3.4f" % foo and when would you use IString(foo), if foo is a float ?
<magcius> those are dumb, though
<magcius> IString isn't an interface.
<magcius> what does it allow?
<magcius> it would be ISequence or IMapping
<lifeless> well, it would extend ISequence,
<magcius> what does IString add on top of ISequence?
<lifeless> actually IImmutableSequence
<lifeless> magcius: replace, format, for starters
<lifeless> magcius: join
<magcius> I would expect IString(float(foo)) to be implemented in terms of a simple adapter:
<magcius> def float_to_str(value): return str(value)
<lifeless> this demonstrates the point.
<lifeless> Thats approximately never the right way to convert a float to a string
<magcius> well sure
<lifeless> so having an adapter is bogus
<magcius> str(float(value)) is as useless, though
<lifeless> of course it is
 * magcius kicks XChat
<magcius> ^W in XChat is "close tab"
<magcius> I have no idea why...
<magcius> If you're saying IString(foo) is stupid because str(foo) is stupid...
<magcius> I don't see how this is an argument *against* interfaces or adaptation.
<magcius> we already do the same thing with duck typing -- we don't care what it is, we just want a string out of it
<magcius> str(foo) is the easy way to do this
<lifeless> cast-based adaption is only useful when there is one and only one right way to get from A to B
<magcius> of course.
<magcius> for most types, there usually is only one way
<lifeless> I disagree there.
<lifeless> Perhaps 50%. Perhaps.
<magcius> if there's a difference, you need to define a new interface IMHO
<lifeless> I need to sleep
<lifeless> I'll think more about how to articulate this.
<magcius> if you have an IUser and you want an IAuthToken, going through an intermediate IPotato shouldn't effect the result.
<lifeless> uhm
<lifeless> I think you're talking about something different.
<magcius> you go to bed. I will too.
<lifeless> Noddy example.
<lifeless> say you have ICar and IPassenger
<lifeless> and a Car instance with 3 Passengers.
<lifeless> Whats the behaviour of IPassenger(car)
<magcius> it throws an error
<magcius> or raises
<lifeless> if there is an adapter present it won't.
<lifeless> that doesn't make it useful.
<magcius> the adapter throws an error
<magcius> if there's more than one passenger :)
<lifeless> if you do that, then you have code that will sometimes work and sometimes not.
<magcius> realistically, this is the wrong use of adaptation
<lifeless> See - when I see a right use of it, I will be happy.
<magcius> in my code, all my use of interfaces is bascially 'isinstance' on steroids.
<lifeless> I have *honestly* not seen a problem in python made simpler by using interfaces.
<magcius> It's extremely, uh, method-based.
<lifeless> magcius: using isinstance is a design problem in the first place.
<magcius> lifeless, there are certain extreme cases where it is necessary.
<lifeless> !cite
<lifeless>  :)
<lifeless> I'm going to crash. Feel free to pick this up another time if you like.
<magcius> OK
<magcius> lifeless, http://paste.pound-python.org/show/TrBZVgWsMKEHo5EtQv8c/
<magcius> lifeless, you may say that's extreme edgecase... but this is what 90% of what my usage of z.i is
<magcius> I would have to subclass if I wanted to add something else, and it quickly became messy.
<magcius> so, I left a "hook".
<lifeless> http://paste.pound-python.org/show/7357/
<lifeless> very quick-n-dirty
<magcius> ... and it's failing to load
<magcius> great
 * magcius shoots _habnabit in the face.
<lifeless> I wouldn't normally do that, but am working on the assumption you can't fix your types ;)
<lifeless> do you want me to paste it somewhere else?
<lifeless> say so now cause *yawn*
 * lifeless waves gnight
<LPCIBot> Project windmill-db-devel build #339: STILL FAILING in 1 hr 8 min: https://lpci.wedontsleep.org/job/windmill-db-devel/339/
<LPCIBot> Yippie, build fixed!
<LPCIBot> Project db-devel build #593: FIXED in 4 hr 55 min: https://lpci.wedontsleep.org/job/db-devel/593/
<LPCIBot> Project windmill-db-devel build #340: STILL FAILING in 1 hr 7 min: https://lpci.wedontsleep.org/job/windmill-db-devel/340/
#launchpad-dev 2012-05-21
<StevenK> lifeless: ^ Did you get distracted?
<lifeless> yes
<lifeless> StevenK: uhm
<lifeless> auditorfixture appears to be auditor ?
<StevenK> How the heck did I screw *that* up
<lifeless> NFI
<lifeless> but
<lifeless> you did :)
<StevenK> Yeah, I'm just about to push --overwrite
<StevenK> lifeless: It should look better now?
<lifeless> p[-;;;;;;;;;;;;;;;;;;;;;xccccccccccccccccccbvvvvvvvvvvvvvvvvvvvvv
<lifeless> uhm, cat.
<StevenK> Hah
<lifeless> does runserver support dynamic port allocation ?
<StevenK> lifeless: From what I saw, yes.
<lifeless> StevenK: you did something similar to txlongpollfixture, I presume; that did what I did for rabbit, which doesn't support dynamic port allocation
<lifeless> a better thing, if its doable is to just run runserver and watch what port it chooses
<lifeless> StevenK: that said, if this works, great.
<StevenK> Which it doesn't.
<StevenK> But I've hit the limit of Django-ness -- I dislike the idea of forking off python to run it
<lifeless> StevenK: my main concern looking at is is that we'll have the same bugs that rabbitfixture has had w.r.t. race conditions on startup (such as the timeout logic); we should perhaps consolidate the common code into fixtures
<lifeless> StevenK: running django in a subprocess is entirely appropriate
<lifeless> StevenK: we /do not want/ to try and import django into our python environment - for multiple reasons, including django having a honking great set of global variables that will collide
<lifeless> https://docs.djangoproject.com/en/dev/ref/django-admin/#runserver-port-or-address-port suggests it does not do dynamic ports.
<lifeless> StevenK: ok, so it doesn't work. What happens, specifically ?
 * StevenK rips out the devnull, so he can see what is going on
<lifeless> StevenK: oh, I see the devnull. Don't do that :)
<StevenK> Hm, smoking gun
<lifeless> StevenK: you can capture the stdout to e.g. the logfile
<lifeless> StevenK: and you won't lose information that way
<StevenK> devnull is for stdin
<StevenK> However, for this I wanted the output to the terminal
<StevenK> python manage.py is all well and good -- when you're in auditor's directory
<lifeless> bleep, I misread.
<lifeless> yes.
<StevenK> So I wonder if auditor is hiding somewhere I can call it ...
<lifeless> why did you want terminal? To see what was up ?
<lifeless> The failure message should have given you the log file ocontents.
<StevenK> lifeless: Nope, I was just getting the raise Exception("Timeout ....
<lifeless> StevenK: so you have a fixture object
<lifeless> that.getDetails() should give you a path in, for future reference.
<lifeless> e.g. f.getDetails().items()[0].itertext()
<StevenK> lifeless: In _start(), that's going to be self, isn't it?
<lifeless> StevenK: yes, but _start doesn't need to change.
<lifeless> StevenK: you already attach the logfile; the caller can choose to look
<lifeless> StevenK: TestCase.useFixture attaches the details before it sets up, IIRC, so it should be doing this.
<StevenK> lifeless: I think _start() does need to change, since the error is "python: can't open file 'manage.py': [Errno 2] No such file or directory" :-)
<lifeless> StevenK: doesn't need to change w.r.t. reporting :P
<lifeless> StevenK: you need to introspect to get the path to the auditor egg, txlongpoll should have example code.
<StevenK> txlongpoll or txlongpollfixture?
<lifeless> the latter
<StevenK> lifeless: Right, I see how txlongpollfixture does it. But auditorfixture doesn't have parts/auditor :-(
<StevenK> lifeless: And I have a [auditor] section in buildout.cfg
 * wgrant cries
<wgrant>             for team in administers_and_in:
<wgrant>                 if (bug_supervisor is not None and
<wgrant>                     not team.inTeam(bug_supervisor)):
<wgrant>                     continue
<wgrant> inTeam is not preloaded
<lifeless> StevenK: uhm, your auditor section doesn't define an entry point
<StevenK> lifeless: It needs to? My buildout-foo is very weak
<lifeless> StevenK: I *think* so, it generates the right interpreter etc
<lifeless> I just realised, running 'python manage.py' is wrong because of that, it should be a buildout created python executable
<lifeless> which the entry points are used to do
<StevenK> lifeless: I still need a path to manage.py, I don't think it matters which interpreter I use.
<StevenK> txlongpollfixture somehow ends up with parts/txlongpoll
<lifeless> uhm
<lifeless> not sure where to pull on this one
<lifeless> you don't run the manage.py from auditor directly
<lifeless> python version mismatch etc stuff
<StevenK> Not even the unpacked egg?
<lifeless> indeed
<lifeless> txlongpollfixture doesn't either.
<StevenK> Ah. So what should I do?
<lifeless> so the basic idea
<lifeless> is you have a function which can be an entry point
<lifeless> e.g. def foo(argv=None):
<lifeless>     if argv is None: argv = sys.argv
<lifeless>     ...
<lifeless> that is probably inside manage.py already
<lifeless> you point at the *python path* to that function, which will be inside auditd (and you'll probably know far more about django glue than you want to by the time you're done)
<lifeless> buildout generates a local executable for you in bin/ when you set scripts=
<lifeless> see entry-points = twistd-for-txlongpoll=twisted.scripts.twistd:run
<lifeless> scripts = twistd-for-txlongpoll
<lifeless> for instance, in LP's buildout.cfg
<lifeless> so you'll have something like
<lifeless> entry-points = auditord-manager=auditord.manager:run
<lifeless> scripts = auditord-manager
<lifeless> then, the fixture runs bin/auditord-manager
<lifeless> probably adjusted to be os.path.join(os.getcwd(), 'bin/auditord-manager')
<lifeless> because of the cwd param to Popen
<lifeless> StevenK: does that make sense?
<StevenK> lifeless: Barely
<StevenK> lifeless: So I need a auditord module now?
<lifeless> you have one
<lifeless> the django app
<lifeless> whats the auditord project name ?
<StevenK> auditor
<lifeless> auditlog is your module name
<lifeless> you need to move / symlink or something manage.py within auditlog from the looks of things
<lifeless> and ugh yeah the one thing I really don't like about django - settings
<wgrant> Um
<wgrant> That's the one thing?
<StevenK> Haha
<wgrant> It's one of the worst things, but lots of the rest is also hideous :)
<StevenK> lifeless: And apparently a fixture would be *easy*
<StevenK> ... and other lies I've been told. :-)
<StevenK> lifeless: I'm not sure what module you want me to jump into?
<StevenK> auditor.manage seems pointless
<wgrant> lifeless: Shouldn't manage.py be replaced by a buildout-generated bin/manage?
<lifeless> wgrant: yes, thats what I'm describing
<lifeless> StevenK: you have a package 'auditlog'
<lifeless> StevenK: We have a project 'auditord', with a public name of 'auditor' and package 'auditlog' - I think you should choose one and go with it
<StevenK> lifeless: I can't!
<lifeless> StevenK: why not ?
<StevenK> You can't call the app 'auditor' if the project is.
<lifeless> .expand()
<wgrant> I'm pretty sure you can.
<StevenK> lifeless: I never picked the name auditord, that was you
<wgrant> It just gets a bit odd, due to implicit relative imports.
<lifeless> ok, so, auditor, fine.
<lifeless> What happens if you use the name auditor for the package ?
<StevenK> steven@undermined:~% django-admin startproject foobar
<StevenK> steven@undermined:~% cd foobar
<StevenK> steven@undermined:~/foobar% python manage.py startapp foobar
<StevenK> Error: You cannot create an app with the same name ('foobar') as your project.
<wgrant> Try it anyway
<wgrant> It'll probably work
<StevenK> I just did?
<wgrant> You tried using Django's stupid tool to do it.
<StevenK> And?
<StevenK> I'd been avoiding Django just fine until you two railroaded me into auditor.
<lifeless> https://code.djangoproject.com/ticket/1908
<wgrant> Well
<wgrant> I never suggested Django is a good fit.
<wgrant> It's really not.
<lifeless> 5 months ago
<lifeless>     Triage Stage changed from Accepted to Fixed on a branch
<lifeless> The bug has been fixed in trunk, apps with same name as project name can be created now
<StevenK> wgrant: I think that was lifeless
<lifeless> StevenK: what was me ?
<StevenK> Suggesting Django
<lifeless> its a preferred tech
<lifeless> I've said explicitly that I think its sufficient for the small targeted things we need
<lifeless> that it has some very good values around lean-ness, clean& understandable code, and performance
<wgrant> Um
<lifeless> and that it may be too heavyweight for some things (like gpg-httpd)
<wgrant> The problem is not that it's insufficient.
<StevenK> And then you try and write a fixture and go insane
<lifeless> StevenK: you're breaking new ground
<wgrant> It's that all of the awkward stuff it provides is entirely useless for this sort of service.
<wgrant> For auditordloger, the useful stuff it provides over raw WSGI is an ROM
<wgrant> ORM
 * lifeless notes that this is a distraction
<StevenK> Clearly you want the app to be called auditor too, which means I need to look at packaging Django 1.4
<lifeless> StevenK: auditlog is a package name, that is the package you need to make a manager.py exist in, it's probably easiest to have it be a new file with contents derived from the django made one.
<lifeless> StevenK: I do, but even that is a distraction
<lifeless> StevenK: and, its one that can be solved later. Because nothing in LP will know about *either* auditor or auditlog.
<StevenK> lifeless: Why does it need to exist under auditlog?
<lifeless> StevenK: so that buildout can import it.
<StevenK> But buildout can import from auditor too?
<StevenK> auditor/__init__.py exists, so it import auditor should work
<lifeless> wait, what
<lifeless> OHHH
<lifeless> crazy f*d up layout.
<lifeless> ok, so yes, auditor.manage:run
<lifeless> and edit manage.py to have a run() method
<lifeless> (and the usual if __name__ == '__main__': boilerplate
<StevenK> manage.py already has __main__ boilerplate
<lifeless> it does, it needs altering
<lifeless> e.g. http://paste.ubuntu.com/998398/
<lifeless> StevenK: btw, we probably want to change auditor from being a django project to being a django app (and then have a project which is for testing created on the fly, and one for prod deployment)
<lifeless> StevenK: but again, Future Work.
<lifeless> StevenK: I will make one note on the naming of things; the reason I propose auditord is so that auditor is available for a client library, if you were to decide you want one.
<lifeless> StevenK: as it is, you'll have to use auditorclient or something instead.
<StevenK> Client library? Err?
<StevenK> It returns JSON, how can that possibly require a client library?
<lifeless> you never know your own future :)
<lifeless> I'm not asking you to rename it, I'm explaining why I'm sticky on the d's - its an easy default and makes it easier for me to think about names.
<StevenK> I can see a gun and my temple in my future
<lifeless> I don't think quake3 supports that.
<StevenK> Who said anything about Quake 3?
<StevenK> lifeless: bin/auditor-manage can not import auditor.manage
<lifeless> StevenK: I did, because why not?.
<lifeless> StevenK: so, thats python path debugging time
<StevenK> sys.path = ['/home/steven/auditorfixture/parts/auditor', '/home/steven/auditorfixture/bin', '/usr/lib/python2.7', '/usr/lib/python2.7/plat-linux2', '/usr/lib/python2.7/lib-tk', '/usr/lib/python2.7/lib-old', '/usr/lib/python2.7/lib-dynload', '/home/steven/auditorfixture/eggs/auditor-0.0.1-py2.7.egg', '/home/steven/auditorfixture/eggs/Django-1.4-py2.7.egg']
<lifeless> and is manage.py present within /home/steven/auditorfixture/eggs/auditor-0.0.1-py2.7.egg ?
<StevenK> Sigh, no.
<StevenK> I guess that's due to find_packages() in auditor's setup.py
<lifeless> StevenK: yes, I suspect so
<lifeless> StevenK: note that its not finding the top level __init__.py at all
<lifeless> StevenK: so your egg only contains auditlog.
<StevenK> Yes
<lifeless> http://paste.ubuntu.com/998418/
<wgrant> lifeless: Oh yeah
<wgrant> lifeless: I discovered on Friday evening that my optimisation of bugsummary was bad.
<wgrant> lifeless: I'd accidentally typoed so it flushed the journal into the journal, so it was writing to a temp table, not a real one.
<wgrant> So it's only about 2.5x faster than before.
<wgrant> s/flushed the jounral into the journal/flushed the temp journal into the temp journal/
<wgrant> And that's with journal FKs removed.
<lifeless> wgrant: still, 2.5x is fast enough to run two in parallel
<wgrant> lifeless: I have a cunning plan to avoid having to do that.
<wgrant> A couple of unresolved issues, but it should work.
<lifeless> StevenK: that pastebin was to you, should work.
<wgrant> StevenK, wallyworld_: https://code.launchpad.net/~wgrant/launchpad/bugsummary-v2-app-no-fixed-upstream/+merge/106557
 * wallyworld_ looks
<wallyworld_> wgrant: looks ok to me. lots of code deletion :-)
<wgrant> wallyworld_: Yup. Thanks.
<mwhudson> https://bugs.launchpad.net/lava-deployment-tool/+bug/1000819/@@+huge-vocabulary timeouts are known?
<_mup_> Bug #1000819: uwsgi 1.0.3 no longer available <LAVA Deployment Tool:New> < https://launchpad.net/bugs/1000819 >
<mwhudson> https://lp-oops.canonical.com/oops.py/?oopsid=OOPS-f6d562060ee52035ead5254827fee6ab\
<wgrant> mwhudson: Yeah, branch search is criminally bad
<mwhudson> :(
<mwhudson> is it just select from branch where unique_name like '%${TERM}%' or something?
<wgrant> The query is awful awful awful
<wgrant> But it shouldn't be as slow as it is.
 * mwhudson looks at the code
<wgrant> It's in the OOPS
<wgrant> Top statement.
<mwhudson> i think my bad version is a lot better than this
<wgrant> Oh
<wgrant> wut
<wgrant> SELECT Branch.id
<wgrant> FROM Branch,
<wgrant>      Product
<wgrant> WHERE Branch.product = Product.id
<wgrant>   AND Product.name LIKE u$STRING)
<wgrant> I'm pretty sure that comes under "who could possibly think that was useful or a good idea"
<wgrant> I didn't realise it did that too
<mwhudson> OR Branch.url = u$STRING) is my favourite useless bit i think
<wgrant> Why?
<mwhudson> branch.url is NULL for !mirror branches
<wgrant> Oh, so it is.
<mwhudson> and noone case about mirror branches any more
<wgrant> Not unique_name, right
<wgrant> Yeah
<mwhudson> Product.name LIKE u$STRING is pretty special yes
<wgrant> The theory behind it seems to be "find ALL the things"
<wgrant> And then leave the user to page through the 9s batches.
<mwhudson> itym "try to find all the things, but timeout instead'
<StevenK> Apparently, find_packages() looks for __init__.py in directories. I think it lies. :-(
<wgrant> It works sometimes!
<wgrant> mwhudson: Part of the problem is bugtasks.
<wgrant> mwhudson: Because there's no logical context to search in.
<mwhudson> oh right
<mwhudson> well
<wgrant> So you have to search everywhere, probably.
<mwhudson> you could look in the context for all the extant bugtasks i guess?
<wgrant> Yeah
<mwhudson> but that gets a bit complicated
<wgrant> We introduced that concept with disclosure.
<wgrant> It's possible.
<mwhudson> cool
<wgrant> Then I'd do name LIKE '%foo%' or unique_name = 'foo', I think...
<mwhudson> seriously though unique_name like $string would be as good as this
<wgrant> Not sure it needs much more than that.
<lifeless> StevenK: it does, but it starts with listdir()
<lifeless> StevenK: did you apply my patch ?
<wgrant> unique_name LIKE '%foo%' is a bit bad, as it matches the product name.
<wgrant> But maybe.
<mwhudson> (of course, unique_name is only 4 years old or whatever, so much newer than this code)
<StevenK> lifeless: I was out for lunch with a friend, so if it was a patch to setup.py, I'm sorry but I missed it
<mwhudson> wgrant: not saying mine is good, just better than existing
<lifeless> 15:18 < lifeless> http://paste.ubuntu.com/998418/
<wgrant> mwhudson: With 9.1 we can even use trigram indices to make LIKE '%foo%' very fast.
<lifeless> (its now 17:00 and EOD)
<mwhudson> bug 372591
<_mup_> Bug #372591: Branch vocabulary and search needs thought and cleaning up <lp-code> <tech-debt> <Launchpad itself:Triaged> < https://launchpad.net/bugs/372591 >
<wgrant> mwhudson: That title has a few too many euphemisms.
<mwhudson> it's a dup of https://bugs.launchpad.net/launchpad/+bug/793830 anyway
<_mup_> Bug #793830: Branch:+register-merge time out due to substring matching many tables <critical-analysis> <timeout> <Launchpad itself:Triaged> < https://launchpad.net/bugs/793830 >
<wgrant> Yeah
<wgrant> That's the one that wallyworld was hitting a lot last week
<StevenK> lifeless: Right, I did miss it -- I scanned backlog until I hit wgrant begging for a review and stopped.
<wgrant> Oh, the prereq or target branch you specified doesn't match a unique_name exactly? Let me just do a 10s search for you.
<mwhudson> heh
<mwhudson> that makes the pain less avoidable i guess
<mwhudson> (my oops was from the suggestions in the 'link a branch' overlay)
<StevenK> lifeless: error: can't copy 'uditor.egg-info/PKG-INFO': doesn't exist or not a regular file
<StevenK> EWTF
<wgrant> mwhudson: I don't know about you, but my distribution of searches is roughly this: 95% of the time it's one of the last 5 branches I touched; 4% I want to search by substring of name in the current project; 1% I want to search by full unique_name (optionally including lp:)
<mwhudson> wgrant: certainly "last branch i touched" is a good guess a lot of the time
<mwhudson> and in this case it was a mixture of the first two points
<wgrant> Yeah, but ranking like that is hard.
<wgrant> So we tend to try to avoid combining them like that.
<mwhudson> solr-as-a-service!
 * mwhudson goes back to real work
<StevenK> mwhudson: We know where you live!
 * StevenK stabs PlainPackageCopyJobTests.createCopyJob() for failing
<lifeless> mwhudson: patches solicited
<lifeless> mwhudson: if you get tired of doing easy things
<StevenK> wgrant: Should I need to do anything after running acceptFromQueue() on the PU and then re-running the PCJ?
<StevenK> wgrant: Since IArchive.getPublishedSources(name=) is returning [] :-(
<wgrant> StevenK: Sounds like the PCJ is not running.
<StevenK> Oh, it does run, it just raises CannotCopy
<StevenK> Which it doesn't tell me
<StevenK> unique-from-test-packagecopyjob-py-line361-100000 1.0-1 in breezy-autotest (The signer of this package has no upload rights to this distribution's primary archive.  Did you mean to upload to a PPA?
<StevenK> Right
<lifeless> StevenK: how are you going ?
<StevenK> lifeless: I gave up in disgust
<StevenK> Trying to get my SPPH linked to PUs working with PCJs
<bigjools> why are you doing that?
<StevenK> bigjools: Because we'd like to know who accepted something, and that information is directly realted to the PackageUpload, and PCJs create PackageUploads if needed.
<bigjools> sounds like a hack
<StevenK> How?
<bigjools> did you consider any other solutions?
<StevenK> Like adding yet another column onto SPPH that shows who accepted it? Now that is a hack.
<bigjools> like an audit trail table
<StevenK> lifeless nacked it
<StevenK> Which is what auditor is for
<bigjools> linking to PU is horrible
<StevenK> But the acceptance is for the *PackageUpload* there is no publication at that point
<bigjools> not all SPPHes result from a PU
<StevenK> Of course
<bigjools> so, yet another nullable FK
<StevenK> Like sponsor? :-P
<bigjools> yes :/
<wgrant> Nullable FKs are only a problem in theoretical relational algebra.
<bigjools> I'm just interested in how you arrived at this solution
<bigjools> if it's the least horrible then fair enough
<StevenK> bigjools: It was the solution that made the most sense -- with auditor being used and keeping track of a PU as it moves through the queue, we needed a way to easily look up the PU for an SPPH
<wgrant> I think it's least bad for now.
<bigjools> the whole model is a mess :/
<wgrant> Eventually we should replace PU with something that tracks each operation on the archive.
<wgrant> Copies and uploads.
<wgrant> And overrides.
<wgrant> And deletions.
<bigjools> like an audit table ...
<wgrant> Audit tables have been vetoed.
<wgrant> Audit services have not.
<wgrant> So StevenK is writing an audit service.
<bigjools> what SPPH should have been
<wgrant> SPPH only tracks sources.
<bigjools> not audit audit ... just a history
<wgrant> That'll work OK if we can convince Ubuntu to turn into Gentoo.
<wgrant> But I think cjwatson might not approve.
<bigjools> lol
<bigjools> the lack of any recording on who did what action was a terrible omission from the start
<bigjools> but what is done is done
<wgrant> As long as we learn from our mistakes.
<wgrant> Which we haven't classically done, but we're hopefully getting better.
<bigjools> we mostly do
<wgrant> The audit service will hopefully mean we can do it easily.
<wgrant> For everything.
<wgrant> For every action.
<bigjools> well I see auditing as two things
<bigjools> hysterical data for later analysis
<bigjools> and stuff that affects immediate decisions
<bigjools> I can't see how a service helps with the latter
<lifeless> wgrant: bigjools: is there a link from bug(task) -> packageupload for fixed bugs?
<lifeless> bigjools: the service lets you query the actions on an object, so it should handle the latter nicely.
<StevenK> wgrant: https://code.launchpad.net/~stevenk/launchpad/populate-spph-pu/+merge/105601 now knows about PCJs. You can review it now if you wish, or tomorrow when you're OCR. :-P
<StevenK> lifeless: Fixing bugs does not use the PU, but the changesfile
<wgrant> lifeless: There isn't. You'd have to look for the comments
<lifeless> StevenK: so if ev wants to get from bug -> BPPH, he goes bug -> changesfile -> SPPH -> BPPH  or some insanity ?
<wgrant> StevenK: I'll do it tomorrow
<bigjools> lifeless: the upload code scans the changelog
<lifeless> bigjools: yah, is the other way round we want
<bigjools> I know :)  just telling you how shit it is right now
<lifeless> bigjools: so, schema work needed? Thats ok, just want to be sure.
<bigjools> it doesn't store the refs
<bigjools> just pokes the bugs
<StevenK> lifeless: Absolutely schema work needed
<StevenK> lifeless: Anyway, you want the SPPH, not the BPPH
<lifeless> StevenK: no, we want the BPPH
<bigjools> lifeless: bear in mind that ultimately it'd be really nice to serviceify soyuz
<lifeless> users don't download sources to fix bugs
<StevenK> lifeless: The bug is closed when the *source* is uploaded, not the binaries
<lifeless> bigjools: orly :P :P :P
<lifeless> StevenK: I know
<bigjools> lifeless: bugs are not filed on binaries
<lifeless> I know that too
<lifeless> expand your minds
<StevenK> I've been dealing with buildout and setuptools today. Maybe my mind will uncurl from the fetal position tomorrow morning.
<bigjools> linking at schema level adds more hurdles to serviceifying
<lifeless> bigjools: it changes the work needed, thats true.
<bigjools> lifeless: what is it that you're trying to achieve?
<bigjools> coz you're breaking your own rule here :)
<lifeless> otp right now, happy to expand later
<lifeless> what rule am I breaking ?
<bigjools> lifeless: report the problem not a proposed solution
<bigjools> no prob anyway, I am about to scoot off for dinner
<lifeless> The problem is how to map from bugs (apport ones specifically) to the binary packages that fix problems for users
<lifeless> ev and I are discussing how to do that, and trying to identify big stroke work needed to do that
<wgrant> Putting that in LP seems like entirely the wrong thing to do.
<wgrant> Isn't that a daisy thing?
<bigjools> lifeless: This info is in the bug comments
<lifeless> yah, we established that
<lifeless> sorry, really otp
<bigjools> so where's the problem?
<wgrant> Like
<lifeless> parsing bug comments in a service
<bigjools> ah "parsing", you didn't say that ;)
<wgrant> Why not have an external service do the same thing.
<lifeless> we're discussing that as an option
<wgrant> Keep a list of (bug#, package, version)
<adeuring> good morning
<wgrant> Done
<wgrant> Problem solved
<wgrant> No LP crap :)
<lifeless> wgrant: needs to cross reference tasks
* adeuring changed the topic of #launchpad-dev to:  http://dev.launchpad.net/ | On call reviewer: - | Firefighting: - | Critical bugs: 3.47*10^2
<lifeless> wgrant: so no, its not that simple.
<wgrant> lifeless: package is there
<wgrant> package + bug => task
<wgrant> done
<wgrant> no LP crap :)
<lifeless> wgrant: + series.
<wgrant> (bug#, series, package, version)
<wgrant> => task
<wgrant> done
<jam> wasn't there a simple script (that isn't setuplx
<jam> setuplxc) for setting up a dev environment? Just rocketfuel?
<jam> For some reason I remember a different command (like lpsetup) from the mailing list
<lifeless> ev: http://en.wikipedia.org/wiki/Tarjan%27s_strongly_connected_components_algorithm
<lifeless> ev: you have frozen
<lifeless> ev: http://www.apple.com/xsan/
<lifeless> ev: or rather, my desktop has
<ev> I'm back at "connecting..."
<ev> heh
<lifeless> power button pushed.
<lifeless> doo be doo
<wgrant> jam: lpsetup is being written.
<ev> haha
<wgrant> jam: But I don't think it's ready yet.
<lifeless> frankban is working on it in slack time
<wgrant> jam: https://dev.launchpad.net/Running/LXC is the recommended thing to do these days
<lifeless> vonderbar, hung on BIOS on reboot
<lifeless> OTOH it may have been probing RAM or something; silly quasi-server that it is.
<rick_h_> morning
<wgrant> stub: Does avahi work for you?
<wgrant> stub: It normally fails because the initscript sets a ulimit for the user process limit
<wgrant> stub: Which prevents more than ~1.5 avahi instances from running on a single kernel
<wgrant> Unless you were lucky and got different UIDs over your two installations.
* benji changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: benji | Firefighting: - | Critical bugs: 3.47*10^2
<rick_h_> benji: morning, time for a quick review when you get a sec please? https://code.launchpad.net/~rharding/maza-server/sso_cleanup/+merge/106615
<benji> rick_h_: sure, it will be a few minutes (on a call)
<rick_h_> benji: no rush, thanks
<mgz> so, my after-lunch question, when running launchpad and it gives a 503 on https://launchpad.dev/ where do I go to find out more?
<mgz> nothing in logs is very helpful
<wgrant> mgz: Sounds like it's not running.
<wgrant> Have you started it?
<wgrant> (make run or make start)
<mgz> I did make run
<mgz> and get the :( page
<wgrant> Oh
<wgrant> That sort of 503
<wgrant> That means there's no DB .
<wgrant> Possibly postgres isn't running
<mgz> that's something neither rocketfuel nor make run do?
 * mgz unskips the building step
<wgrant> mgz: launchpad-database-setup
<mgz> seems not, looks like new log to me
<wgrant> Hm?
<mgz> I'd wrongly assumed that would be done as part of the bootstrap
<wgrant> It wipes out your postgres installation.
<wgrant> So doing it implicitly would be a little impolite.
<mgz> the output lives in the launchpad tree under database?
<mgz> or in a system dir?
<wgrant> It rebuilds and reconfigures and populates your default system postgres instance.
<wgrant> Usually /var/lib/postgresql/9.1/main
<mgz> ace, thanks, so will need this as part of my instance setup
<wgrant> That's why it's in the instructions :)
<mgz> I blame the red rectangle for making me want to ignore that section
<mgz> okay, looks good, thanks wgrant
<mgz> really lunchtime
<wgrant> Heh
<jam> jtv: there seems to be a problem with the clean-up-the-database script that we were running trying to get Q translations working again.
<jam> I've heard that you're the guy who knows whats up, if you can give me a pointer, I can try to take it from there.
<jamestunnicliffe> benji: Hi, could I get a review started? https://code.launchpad.net/~dooferlad/launchpad/upcomingwork_show_incomplete_bp
<benji> jamestunnicliffe: you have good timing, I was just looking at it
<jamestunnicliffe> benji :-)
<benji> jamestunnicliffe: I approved your branch with a couple of small suggestions.
<jamestunnicliffe> benji: Thanks. I have another branch waiting you have a moment, but I don't want to hog all your reviewing time!
<benji> jamestunnicliffe: I just started on it too
<jamestunnicliffe> benji: Ah, thanks for the tal:conditional tip. I could not find a code example showing that. Clearly google was broken on Friday...
<benji> my pleasure
<benji> jamestunnicliffe: review done; I saw that you could remove a couple of (pre-existing) calls to float that aren't needed, if you feel like it.  Other than that the branch looks good.
<jamestunnicliffe> benji: I am up for changing to <tal:upcoming_work_class_name define="global upcoming_work_class_name string:expanded"/> but I have a question. The tal:<string> seems to be required but <string> doesn't seem to matter. Is it just convention that we match it (namespace?) with the name of the thing we are defining?
<benji> jamestunnicliffe: right, the element name can be whatever you want, people often use "<tal:block>" for those, so that's an option too
<jamestunnicliffe> benji: Thanks.
<benji> np
<jamestunnicliffe> benji: I have uploaded a new branch. I don't have merge privs though. Can you kick that into action?
<benji> jamestunnicliffe: sure
<jamestunnicliffe> benji: Will get rid of those float calls and ping you to merge that in a moment.
<benji> jamestunnicliffe: cool
<jamestunnicliffe> benji: lp:~dooferlad/launchpad/postponed-is-done is ready as well now. Thanks!
<benji> k
<lifeless> morning
<jelmer> hey lifeless
<lifeless> hey jelmer
<lifeless> any luck on subunit ?
<cjwatson> benji: Thanks for your review of https://code.launchpad.net/~cjwatson/launchpad/germinate-stale-files/+merge/106626; I've made adjustments.  Can you set Status to Approved if you're happy with that, and I'll land it?
<lifeless> jelmer: ^?
<benji> cjwatson: I've set it to approved.  You should feel free to do so too.
<cjwatson> Thanks
<benji> oh, and cjwatson: yes, you're weird ;)
<cjwatson> heh
<cjwatson> I guess I find it easier to be consistent about using context managers everywhere possible rather than having to think about (even trivial) exceptions
<cjwatson> or something.  hard to cognitively unpack
<cjwatson> Or possibly that it means that the safest ways to create an empty file and create a non-empty file are spelled more similarly
* benji changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: - | Firefighting: - | Critical bugs: 3.47*10^2
<wgrant> WebOps: how many OpenIDs are on https://launchpad.net/~dachary/+reviewaccount?
<spm> wgrant: just the 1
<wgrant> Oh hm
<wgrant> spm: SELECT * FROM openidrpsummary WHERE account = (SELECT id FROM account WHERE openid_identifier='dnTeEXH');
<wgrant> On SSO
<wgrant> Or equivalent web UI if there is one.
<spm> itt, we learn that wgrant is single handedly responsible for wvg task queuing
<spm> >:)
<wgrant> No, that's users who somehow create more than one SSO account without noticing.
<czajkowski> special
<spm> czajkowski: yes, yes I am. thanks for noticing!
 * czajkowski peer at spm have you been taking lessons from mr. StevenK 
 * StevenK raises an eyebrow.
<spm> pfft. try other way around. the grasshopper learns from the master
<StevenK> wgrant: Is bug heat dead enough to nail bug 705713 shut?
<_mup_> Bug #705713: Bug:EntryResource Timeout trying to set bug private - death by sql / late evalation <timeout> <Launchpad itself:Triaged> < https://launchpad.net/bugs/705713 >
<StevenK> Oh my god, I know what I need to fix
<wgrant> StevenK: Indeed, thanks.
<StevenK> RARGH
<StevenK> Any OOPS I've tried to see from following links in bug reports gives no such oops
<StevenK> And then Firefox crashed
#launchpad-dev 2012-05-22
<StevenK> wgrant: O hai, Mr. OCR. https://code.launchpad.net/~stevenk/launchpad/builder-description-mandatory/+merge/106729
<wgrant> StevenK: Does code rely on the description being set?
<wgrant> StevenK: The description shouldn't really exist at all; I'd prefer to fix the database to make it nullable.
<StevenK> wgrant: DB patches are making me very cranky at the moment.
<StevenK> wgrant: But if you think a DROP CONSTRAINT is the best way forward, I can do that.
<wgrant> StevenK: Check what uses Builder.description
<wgrant> I'm not sure infinity or lamont really want to keep coming up with awesome descriptions for builders.
<StevenK> steven@undermined:~/launchpad/lp-branches/devel% bzr grep -i Builder.description | wc -l
<StevenK> 0
<wgrant> It's never going to be used like that.
<wgrant> That'd only be if we were querying on it.
<StevenK> <- Literal troll
<bigjools> are there many bridges to hide under in western Sydney?
<wgrant> He lives next to a river.
<wgrant> So he should have one nearby.
<wgrant> I'm not sure about goats, though
<StevenK> There is one just outside, in fact.
<wgrant> Bogans might have to do.
 * StevenK kicks wgrant into Coburg.
<wgrant> We don't talk about Coburg.
<StevenK> I don't think Builder.description is actually used by anything at all.
<wgrant> I think it's in the UI
<bigjools> it should be nullable
<wgrant> Most definitely.
<wgrant> It's just a question of how easy it is
<wallyworld_> wgrant: it appears the legacy access mirroring trigger for bugsubscription creates an artifact grant even if the bug is already visible via a policy grant
<wgrant> I suspect the answer is "trivial"
<cjwatson> RFC by Soyuz people: https://bugs.launchpad.net/launchpad/+bug/1002589
<_mup_> Bug #1002589: Deploy new Ubuntu archive signing key <Launchpad itself:New> < https://launchpad.net/bugs/1002589 >
<wgrant> wallyworld_: That's correct and very deliberate.
<wallyworld_> why? :-(
<wgrant> cjwatson: :( why didn't we do that before precise
 * bigjools looks
<cjwatson> Because we sucked at getting the necessary people together.
<cjwatson> We finally managed it at UDS after lots of false starts.
<StevenK> cjwatson: Why is my MP linked in that bug?
<cjwatson> wuh
<cjwatson> paste damage, will edit, sorry
<bigjools> heh
<wgrant> wallyworld_: Because the rules around when an APG should prevent creation of an AAG etc. weren't defined when they were written.
<StevenK> I hate to say it, but maybe a new column on distroseries
<cjwatson> better
<wgrant> wallyworld_: And they would make it more complicated.
<cjwatson> StevenK: troublesome with dogfood though?
<bigjools> cjwatson: all the signing is done in ubuntu-specific shell scripts
<wgrant> cjwatson: We can change the DB on DF
<cjwatson> bigjools: I know, but it still needs to work on both prod and df
<wgrant> wallyworld_: Is this problematic?
<wgrant> wallyworld_: The triggers will be turned off before we use APGs
<cjwatson> wgrant: on every restore?
<wgrant> cjwatson: Yes
<StevenK> I didn't think DF had the archive signing key?
<bigjools> cjwatson: this is a great time to get the scripts out of the tree then
<wgrant> cjwatson: They only happen every 6 months
<cjwatson> Is there an automatic way to do that?
<bigjools> StevenK: it doesn't
<StevenK> bigjools: Good.
<cjwatson> wgrant: Which is just long enough for people to forget manual procedures
<wallyworld_> wgrant: so now i have a job which remove subscriptions for bugs which become invisible when info type changes. and my test was failing since the aag was providing visibility
<wallyworld_> so i will have to disable triggers in my test
<wgrant> cjwatson: People will remember them when the signing fails after they forget :)
<cjwatson> bigjools: Well, uh, sure, but we still need to solve the problem :-)
<bigjools> I am strongly of the opinion that there should be no LP changes here
<wgrant> wallyworld_: Ah, that's awkward.
<wallyworld_> wgrant: the aag was created by the trigger, not me
<bigjools> cjwatson: can move the ubuntu run-parts stuff out of the LP tree and hard code what you need in your own scripts?
<wgrant> cjwatson: The idea is that cronscripts/publishing/distro-parts/ubuntu shouldn't be part of the LP tree
<cjwatson> It is kind of bizarre that these Ubuntu-specific scripts were created without any provision for Ubuntu-specific editing outside LP, I'll grant you
<bigjools> huh?
<wgrant> cjwatson: The Ubuntu primary archive is signed by those scripts that are meant to be managed by !LP, not the native signing stuff.
<bigjools> there is provision
<cjwatson> Uh, not right now
<cjwatson> Except by us landing LP changes
<wgrant> You tell us you want to edit them.
<wallyworld_> wgrant:  i was creating an apg on info type 1 and then changing bug to info type 2 and expecting it would be invisible, but i had done a subscribe and that created the aag which kept the bug visible
<wgrant> We change the config to point at your copy on cocoplum.
<bigjools> it's always been waiting for you guys to move them out of tree
<wgrant> Done
<bigjools> we have done all we can
<cjwatson> If you're waiting for us, you need to tell us :-)
<cjwatson> So if it's config-able, fine, I can deal with that (maybe tomorrow)
<wgrant> wallyworld_: Right, you either need to not subscribe, or disable the trigger if this is a key part of your test.
<wgrant> cjwatson: wgrant@lplucid:~/launchpad/lp-production-configs/trunk$ bzr grep distro-parts
<wgrant> ftpmaster-publish/launchpad-lazr.conf:run_parts_location: cronscripts/publishing/distro-parts
<wgrant> cjwatson: We just need to change that to point at a location you can write to
<wallyworld_> wgrant: but i need to subscribe because the whole poiint of the test is to see that the unsubscribe happens :=)
<wallyworld_> so will disable triggers
<wgrant> cjwatson: Then we can delete these horrors from our otherwise perfect, flawless, and high-quality tree :)
<cjwatson> OK (and we'll need to set things up so that that's an ubuntu-archive owned branch that ops can pull, rather than direct editing on cocoplum, of course)
<wgrant> Right :)
<StevenK> *cough* cdimage
<wgrant> wallyworld_: Ah, right.
<wgrant> wallyworld_: Indeed
<cjwatson> StevenK: ?
<StevenK> cjwatson: Just thinking of cdimage-private, but that's special in it's own right, and not really relevant to this situation now that I think about it
<StevenK> wgrant: I can't see description on https://launchpad.net/builders/palmer
<cjwatson> Also (while it's not really relevant to this channel anyway) slangasek managed to get hold of sabdfl and get agreement that we don't need the private branch for its former reason any more, so that is cleanupable at some point
<StevenK> wgrant: Or /builders for that reason
<wgrant> StevenK: Maybe you can drop the column and nobody will notice.
<cjwatson> bigjools,wgrant: OK, thanks, I know what to do then; I'd like to keep the bug open until we have stuff moved over to our control, and probably then reassign it to some relevant component owned by us
<wgrant> cjwatson: sure.
<bigjools> cjwatson: perfect
<StevenK> wgrant: Two DB patches, I guess. DROP CONSTRAINT ; drop the model code; DROP COLUMN
<wgrant> StevenK: DROP NOT NULL, but yes.
<bigjools> cjwatson: out of interest, why are you guys wanting to use a new key?
<StevenK> bigjools: 1024D versus 4096E
<StevenK> 4096R
<bigjools> fairy muff
<cjwatson> Also that key was generated eight years ago
<cjwatson> Also https://bugs.launchpad.net/launchpad/+bug/804252 unfixable until we change keys
<_mup_> Bug #804252: Please support InRelease files <Launchpad itself:Triaged by cjwatson> < https://launchpad.net/bugs/804252 >
<bigjools> cool
<cjwatson> I think we should probably be cycling keys roughly once per LTS cycle
<bigjools> seems eminently sensible
<bigjools> isn't it kinda late there Colin? :)
<StevenK> wgrant: :-( I was going to delete the branch/MP
<cjwatson> bigjools: Yes
<wgrant> StevenK: That works too
<cjwatson> Haven't you learned that I have odd sleeping habits yet? :-)
<bigjools> cjwatson: I have noticed lately, yes!  EBABY?
<cjwatson> Sometimes that, sometimes EINSOMNIA
<StevenK> Yeah, Soyuz keeps me awake at night too
<bigjools> I get to say EBABIES soon...  /o\
<StevenK> :-P
<wallyworld_> bigjools: hahahahahahahahaha
<StevenK> bigjools: It's next month?
<bigjools> end of next month
<bigjools> cometh the apocalypse
<StevenK> Haha
<StevenK> I was about to ask if you're looking forward to it ...
<bigjools> I am
<bigjools> despite the stress I know is coming :)
<StevenK> But Uncle Ian will be a big help, won't he?
<bigjools> he *sure* will :D
<wgrant> lifeless: http://people.canonical.com/~wgrant/launchpad/soyuz-arch.png is roughly what I've envisaged for a few years. The Soyuz bit is largely non-LP-specific, can be used by LS/OEM/everyone/etc. And Launchpad the gigantic monolithic application, while it can display information using API calls, becomes largely unaware of the details of archives and packages -- they're all done through the separate (probably Launchpad-branded) package ...
<wgrant> ... service frontend.
<lifeless> so other than 'web frontend' there, that all makes a lot of sense to me
<lifeless> if you want to talk more, I'm off my prior call, and have had foodish
<wgrant> lifeless: What doesn't make sense about that?
<lifeless> why would LP talk to a web frontend to the package management service?
<lifeless> why wouldn't it talk to the service itself?
<wgrant> Um, yes, that.
<wgrant> Never mind my arrows.
<lifeless> wgrant: I'd expect users to still come through the LP frontend facade too, per what I was saying before
<lifeless> But I'd expect that to be very thin - just universal
<lifeless> There are about a billion ways to construct something functional
<lifeless> its hard to say what works best today
<lifeless> I'm not sure how to turn gut feelings in this space into cheap experiments
<wgrant> That interface would need to be really minimal for things to work well
<lifeless> I'm not sure why you say that
<lifeless> Are you worried about duplication in functionality?
<lifeless> Or development friction?
<wgrant> Development friction mostly.
<lifeless> So, I think there would be some friction, but conversely, I think having all the presentation and UI away from the service will make changes to the service rarer, and keep its API focused on functional changes.
<lifeless> So a bit of swings and roundabouts; I'd expect most changes that don't need schema evolution to be presentation only.
<lifeless> one way to think of this is that you're splitting out one piece at a time
<lifeless> and UI is a separate piece
<lifeless> it can always be split out itself later.
<wgrant> The presentation and UI is separate from the service in that diagram, and indeed would probably be the last bit to go.
<lifeless> so, one of the things I think we may disagree on is whether it *should* go
<lifeless> I don't think we know enough yet to know whether it should or shouldn't. I am arging that UI and presentation should split off from the data storing services (including lp:launchpad).
<lifeless> But I'm also arguing that all the UI should stay together.
<lifeless> I think it would be a nice environment to have a bunch of API only services (with no admin UI or anything), and a single UI+presentation layer
<lifeless> stateless UI+Presentation
<wgrant> For some bits, sure. I don't think it's particularly valuable to have Soyuz as one of those bits.
<lifeless> What makes it special?
<lifeless> also, in unrelated news. \o/. My belkin blocks multicast between LAN and Wifi AFAICT. Wiiiin. Not.
<wgrant> Oh, that's very handy of it
<lifeless> (Q: why does avahi not work in my network: A: Belkin)
<wgrant> This is why we don't use crappy home routers :)
<bigjools> Linksys FTW
<lifeless> Yeah, I had one, but it don't do N.
<bigjools> N schmen
<bigjools> I don't see the attraction of it, I rarely need that kind of speed on mobile devices and when I do I plug in
<bigjools> which is like once a year
<lifeless> my ethernet hub is hard to get at
<bigjools> when we get broadband speeds that regularly exceed G then I'll get N :)
<wgrant> I think my WAP is from like 2003
<wgrant> a/b/g
<wgrant> Works fine
<lifeless> wgrant: I don't want unnecessary friction in dev, definitely don't want.
<lifeless> wgrant: what I'm worried about is adding friction by partitioning things we should be integrating.
<wgrant> lifeless: Sure, that is a very real concern.
<lifeless> wgrant: I view services as a great way to stop square-peg round hole *and* to stop thousand-implementations-of-same-thing syndromes
<wgrant> But it is my feeling that there isn't a huge amount of beneficial integration that this will make awkward.
<lifeless> keeping the UI and presentation together is a relatively cheap way to mitigate the risk, because it will help folk think about the user experience specifically, when they are working on any UI or presentation issue.
<StevenK> bigjools: I just bought a Linksys WAP, it's been really good.
<wgrant> WAPs are easy to get right
<wgrant> Cheap routers aren't.
<lifeless> Leenuchs
<wgrant> Yeah
<wgrant> I hope Marvell will release another useful SoC soon.
<wgrant> ARMv5 Kirkwood is getting a bit old.
 * wgrant testfixes
<wgrant> Bah
<wgrant> I'm still 10 deletions away from being LOC-debt free.
<wgrant> (although I have lots of credit once DB patches are rolled into the main schema dump)
<StevenK> wgrant: I'm at +142 or so, but I should hit -2000 once I can delete show_information_type_in_ui
<wgrant> Yeah
<StevenK> stub: Thanks for the db review
<StevenK> wgrant: Hm, just thinking about it, I should drop into negative from dropping builder.description too
<wgrant> StevenK: The two sets of DB patch boilerplate will probably prevent that
<bigjools> StevenK: yeah I have the blue one that everyone has
<wgrant> WRT54GL?
<StevenK> I bought a E3200
<StevenK> It's a router, but I'm running it in bridge mode
<StevenK> wgrant: Oh, all +16 lines? :-)
<bigjools> wgrant: y6es
<StevenK> wgrant: I've grabbed a lock on LPS, I'll change your FDT to tonight
<lifeless> wgrant: You have mail; hopefully I haven't misrepesented you :)
<bigjools> though I am not using it any more since the modem has wifi which successfully covers the house
<bigjools> and while it's tempting to blanket the neighbourhood in my wifi signals, I won't
<StevenK> It's Queensland, like there are any other signals around.
<wgrant> StevenK: Ah, thanks
<bigjools> StevenK: I could say the same about Boganville where you live
<StevenK> bigjools: Pft, I moved out of Blacktown.
<wgrant> lifeless: I don't see mail.
<wgrant> Hopefully it's not going via forster, or I'll never see it.
<lifeless> that or I messed up
<lifeless> argh
<lifeless> will -> robbie.
<lifeless> FAIL
<wgrant> Heh
<wgrant> That's more like me.
<StevenK> 4287 AttributeError: 'NoneType' object has no attribute 'email'
<StevenK>     Bug: https://launchpad.net/bugs/580461
<_mup_> Bug #580461: Login with the email once associated to a merge account OOPS <lp-foundations> <oops> <openid> <qa-ok> <Canonical SSO provider:Invalid> <Launchpad itself:Fix Released by stub> < https://launchpad.net/bugs/580461 >
<StevenK> Hnady
<StevenK> *Handy
<wgrant> That might be from the user I suspended
<wgrant> Looking
<wgrant> Yeah
<wgrant> It is
<wgrant> lifeless: So codehosting doesn't handle it, but the auth API crashes...
<wgrant> 4285 * 20s == around a day
<wgrant> So it fits
<lifeless> win :>
<lifeless> and \y/ for complex internal models
<StevenK> wgrant: +18/-37 for the death of builder.description. I was hoping there was more.
<wgrant> lifeless: The email starts a little incoherent, but gets better.
<wgrant> I think you overstate the complexity in running a separate service, though :)
<StevenK> steven@undermined:~/launchpad/lp-branches/drop-builder-description% bzr di | tail -n 7 | head -n 3
<StevenK> -    ...     title="The Prosciuto Builder", description="Uhmmm", owner=cprov,
<StevenK> -    ...     virtualized=False)
<StevenK> +    ...     title="The Prosciuto Builder", owner=cprov, virtualized=False)
<StevenK> wgrant: ^ Oh look, there's all the proof you need that the column is utterly bong.
<wgrant> Oh
<wgrant> I didn't think it would actually have tests
<wgrant> That explains the diff.
 * StevenK destroys the use of a_builder
<StevenK> +27/-56 and the doctest sucks less
<StevenK> wgrant: https://bugs.launchpad.net/launchpad/+bug/973001 WTF?
<_mup_> Bug #973001: NotImplementedError BugTarget <security proxied lp.registry.model.distributionsourcepackage.DistributionSourcePackage instance at 0x12ef41d0> is not supported by . on +expirable-bugs page <oops> <Launchpad itself:Triaged> < https://launchpad.net/bugs/973001 >
<wgrant> StevenK: Seems pretty self-explanatory to me.
<StevenK> Hmmmm. Sure, but how the frack do you fix it
<wgrant> Make it work or deregister the view.
<wgrant> Making it work should be pretty trivial.
<wgrant> I'd expect at most two lines of code.
<wgrant> StevenK: ^^
<StevenK> wgrant: deregister the view is hard, given it's registered against IBugTarget
<wgrant> Alternatively delete the page
<wgrant> Replace it with a bug search
<wgrant> That's probably besty
<StevenK> wgrant: So just redirect?
<wgrant> The only difficulty is that the page currently has a block of text at the top
<jelmer> g'morning
<spm> o/ jelmer
<jam> morning jelmer, mgz, vila, czajkowski
<vila> morning all
<czajkowski> morning
<lifeless> jelmer: hi; subunit ?
<jelmer> lifeless: hi
<jelmer> lifeless: sorry, haven't had time to work on it yet
<czajkowski> hmm trying to mark a bug duplicate of another one have the bug open but it keeps saying invalid
<czajkowski> :/
<jelmer> lifeless: my spare time is limited at the moment; if you or somebody else wants to take over that's okay, I don't want this to block on me.
<czajkowski> nm engaged noggin
<lifeless> jelmer: if it helps, you can count it as work, as gary's squad is impacted in their work by it
<jelmer> czajkowski: :)
<jelmer> lifeless: that does help, thanks
<lifeless> czajkowski: how is bug 996537 a dupe of bug 68997 ?
<_mup_> Bug #996537: Apport-Collect post many times and spams many times <apport-bug> <i386> <precise> <Launchpad itself:Confirmed> <apport (Ubuntu):Invalid> < https://launchpad.net/bugs/996537 >
<_mup_> Bug #68997: kwalletmanager icon not in system tray, unstartable from menu <kdeutils (Ubuntu):Fix Released> < https://launchpad.net/bugs/68997 >
<lifeless> czajkowski: I'm guessing you dropped a digit ..
<adeuring> good morning
<czajkowski> lifeless: just running into the tower bbiab
<lifeless> czajkowski: don't hurt yourself :)
<czajkowski> lifeless: I didn't cant say the same about the two cyclists outside and a caby
<lifeless> czajkowski: ?!
<czajkowski> I didnt hurt myself, there was a cycle crash outside millbank on the way in
<lifeless> ah
<StevenK> wgrant: Heh, and I would have expected you to revert.
<wgrant> StevenK: The fix was trivial and restricted to the one test, so there was no reason to.
<wgrant> I don't revert pointlessly :)
<jamestunnicliffe> Hi, I have a buildbot fail (https://lpbuildbot.canonical.com/builders/lucid_lp/builds/2077) but I don't have permission to view the error. Can someone paste it somewhere for me?
<wgrant> jamestunnicliffe: There was a trivial bug in a test, which I landed a fix for.
<wgrant> -        item = self.MockWorkItem(False)
<wgrant> +        item = self.MockWorkItem(False, False)
<wgrant> Should be through buildbot in about 90 minutes
<jamestunnicliffe> wgrant: Ah, cool. Thanks.
* gmb changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: gmb | Firefighting: - | Critical bugs: 3.47*10^2
<czajkowski> gmb: top of the morning to you
<gmb> Morning czajkowski :)
* rick_h_ changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: gmb, rick_h | Firefighting: - | Critical bugs: 3.47*10^2
<cjwatson> How long should I expect it to take to mirror branches onto http://bazaar.launchpad.net?
<czajkowski> jelmer: ^^
<mgz> it should be instant, but it's lagging.
<cjwatson> I've got one that I pushed some minutes ago that's available on bzr+ssh but not http yet.  Wondering if this is normal and I should just be more patient, or if it's a problem.
<cjwatson> Right.  Anything to do with the replication lag?
<jelmer> cjwatson, czajkowski: they should start pretty much instantly
<cjwatson> It's lp:~ubuntu-archive/ubuntu-archive-publishing/trunk if that matters.
<jelmer> https://code.launchpad.net/+code-imports/+machines
<cjwatson> Oh, I didn't realise that went through the code import machinery.
<jelmer> cjwatson: ah, that's just the scanning you're waiting for
<jelmer> cjwatson: so it's indeed unrelated; there is no mirrorring involved for hosted branches
<mgz> last count db replication was three hours behind
<mgz> and a branch I pushed at 9:40 is still not visible over http
 * cjwatson upgrades dogfood for some QA
<cjwatson> OK, I've caused /srv/launchpad.net/publisher-parts/ubuntu/ to exist on both cocoplum and mawson.  Can somebody change run_parts_location to /srv/launchpad.net/publisher-parts in ftpmaster-publish/launchpad-lazr.conf and dogfood-publish/launchpad-lazr.conf?
<cjwatson> I chose the path to match what's used in derived-distro-publish/launchpad-lazr.conf.
<cjwatson> (Though, hmm, my import seems a bit buggy.  Fixing up by hand.)
<wgrant> cjwatson: Did you bzr split it?
<cjwatson> No, I used fastexport/import so I wouldn't have to carry LP's entire history around
<wgrant> Ah
<cjwatson> But it seemed to get confused and miss a few revisions at the end.  I recommitted them by hand.
<jamestunnicliffe> Hi, so I have got two changes that have landed on qastaging. While they are both functionally correct, there is a bug when they run together such that something correct, but slightly unexpected happens.
<jamestunnicliffe> The easiest thing to do would be land the changes and then submit a new bug fix (it is trivial)
<jamestunnicliffe> The alternative is back out both changes and create 1 change for both.
<jamestunnicliffe> Opinions?
<wgrant> Does it prevent them from being released to production?
<jamestunnicliffe> wgrant: I don't think so. https://qastaging.launchpad.net/~linaro-infrastructure/+upcomingwork shows the page that changed.
<wgrant> Then it doesn't matter -- do whatever is easiest.
<wgrant> All that matters is that everyone else is not blocked :)
<jamestunnicliffe> wgrant: Cool. Will tag the changes as OK then get a new fix in ASAP.
<wgrant> Great.
<czajkowski> oh great fire alarm on
<rick_h_> sinzui: great blog post!
 * rick_h_ does a happy dance with the idea of all that cruft removed/cleaned up
<sinzui> thank you rick_h_, the only developer to have added a sensible browser guard to the code base
<sinzui> rick_h_, I am really glad it is over. I packed away the sd card with the vm image. I now have full use of my computer. I can read mail and hack
<rick_h_> hah, yea. I was dreading that the job was heading my way and I'd have to reinstall the windows VM. Last time I had to create a usb install disk from the cd etc ugh!
<rick_h_> my goal in life is that the next laptop I get will support esata booting, and I'll just toss the drive it comes with into an external sata enclosure for IE testing
<sinzui> I really wish winetricks had created a true IE8 or IE9 browser. That was painless to install, but cookies and https rules are broken.
<rick_h_> yuck, but yea guess I can see the https part at least. So much backend stuff to make that work
<sinzui> The whole security model was broken. You could not log into Ubuntu SSO and state with cookies was out of the question.
<rick_h_> gmb: just want to check if you're peeking at either of the rewviews to avoid duping?
<gmb> rick_h_: I peeked at wallyworld's, saw the line count, thought better of it, and I don't have enough domain knowledge to safely review cjwatson's.
<rick_h_> ok
<nigelb> sinzui: Great glob post.
<nigelb> err
<nigelb> *blog post
<mgz> glob is a much better word.
<sinzui> glob is correct
<nigelb> glob is also a person I talk to :P
<nigelb> (on another network)
<gmb> nigelb: Never fear; this is Launchpad. Speling is optional.
<nigelb> gmb: Heh :)
<allenap> Anyone fancy doing a review for postgresfixture? https://code.launchpad.net/~allenap/postgresfixture/run-with-command/+merge/106830
<rick_h_> allenap: on the list. Getting backed up atm
<allenap> rick_h_: Cool, thanks.
<gmb> allenap, rick_h_ I'll take it and holler if I've any domain-specific questions.
<allenap> gmb: Thank you :)
<gmb> allenap: Although there should be a check on Merge Proposals: assert commit_text != description. ;)
<allenap> gmb: Sorry, not a very good description I know. It's a fairly simple change though, but I'll add more if you want?
<gmb> allenap: No, it's fine. I'll ask questions as I go if necessary :)
<jamestunnicliffe> gmb / rick_h_: Very quick review request (5 lines) if you want a not very involved one to tick off: https://code.launchpad.net/~dooferlad/launchpad/postponed-is-done-wi-auto-open-fix/+merge/106836
<gmb> jamestunnicliffe: I'll take a look when I've finished with allenap's.
<jamestunnicliffe> gmb: Thanks.
<allenap> gmb: Ah, to try it out: make && bin/python -m postgresfixture --help
<gmb> allenap: Ta.
<gmb> allenap: Scarily, it makes perfect sense to me. r=me.
<allenap> gmb: \o/ thanks
<gmb> jamestunnicliffe: Are you offsetting your +6 lines against danilos's megabranch (we should track this somewhere)?
<danilos> gmb, it's all tracked already :)
<gmb> Ah, cool.
<gmb> Where?
<danilos> gmb, https://docs.google.com/a/linaro.org/spreadsheet/ccc?key=0AvOsYPy8e7yUdGkyRmx2WGFwT3NnSjdHVW04Q1pvSmc
<gmb> danilos: Ah, great. Now, what we really need is to have that tracked in dev.launchpad.net - or at least linked to therefrom.
<danilos> gmb, I thought I added it to https://dev.launchpad.net/Projects/WorkItems/, but I guess not
<danilos> gmb, it's there now
<gmb> Thanks.
<gmb> jamestunnicliffe: r=me
<jamestunnicliffe> gmb: I don't have merge privs, so that is another job for you :-|
<rick_h_> thanks for the catch up gmb
<rick_h_> jamestunnicliffe: which branch is this? I'll submit to ec2 if needed
<jamestunnicliffe> rick_h_: lp:~dooferlad/launchpad/postponed-is-done-wi-auto-open-fix
<rick_h_> jamestunnicliffe: ok, pulling down to submit
<jamestunnicliffe> rick_h_: Thanks!
<sinzui> abentley, ping
<rick_h_> jamestunnicliffe: can you set a commit msg in there please?
<jamestunnicliffe> rick_h_: The descriptive merge message was at rev 15283. Am I missing something?
<rick_h_> jamestunnicliffe: in your MP there's a field for setting a commit message which gets used when things are merged together
<sinzui> jamestunnicliffe,  visit https://code.launchpad.net/~dooferlad/launchpad/postponed-is-done-wi-auto-open-fix/+merge/106836
<gmb> rick_h_: Thanks for taking care of that merge.
<rick_h_> gmb: np
* gmb changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: -, rick_h | Firefighting: - | Critical bugs: 3.47*10^2
<jamestunnicliffe> rick_h_, sinzui: Got it.
<gmb> Why did I put that hyphen in there?
* gmb changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: rick_h | Firefighting: - | Critical bugs: 3.47*10^2
<rick_h_> jamestunnicliffe: ty much
<jamestunnicliffe> rick_h_: Is this supposed to include [r=??]
<rick_h_> jamestunnicliffe: no. It's just the commit message about the change that will show in the bzr log of trunk
<jamestunnicliffe> rick_h_: OK
<rick_h_> and should be more of what changed vs the bug report
<rick_h_> So something like "auto expand work items list considers postponed as incomplete work"
<jamestunnicliffe> rick_h_: Done.
<rick_h_> awesome
<lifeless> cjwatson: the post-restore fixup code is in losa scripts, AFAIK
<rick_h_> jcsackett sinzui either of you avail for sanity check of https://code.launchpad.net/~cjwatson/launchpad/archive-build-score-api/+merge/106804
<rick_h_> it seems ok, but mentions some security sanity checking and I start to get nervous
 * sinzui looks
<rick_h_> jamestunnicliffe: so your branch is off to the land of ec2. Should get an email in 6hrs-ish if things failed at all/passed
<jamestunnicliffe> rick_h_: thanks
<cjwatson> lifeless: anything that's mortal-visible?
<czajkowski> sinzui: great blog post
<sinzui> rick_h_, the branch is good.
<rick_h_> sinzui: ok, thanks for the confirmation
<sinzui> I am going to comment about another bug that I think Collin to tell us if it is fixed, if if he knows how to fix it
<rick_h_> sounds like a plan to me
<cjwatson> done
<cjwatson> sinzui: think I'm OK to go ahead and land?  I don't think I can do better at that description myself (I'm fine at explaining to people who know the field, less so to people who don't)
<cjwatson> and it basically looks like wgrant sorted it out a while back but didn't see that bug
<sinzui> cjwatson, I agreed. thank you
<lifeless> cjwatson: possibly; nag someone in -ops perhaps
<abentley> sinzui: pong
<sinzui> hi abentley. I just sent an email to the launchpad-developers list asking if celery is ready in production. maybe you want to reply to it since I had a few questions
<abentley> sinzui: I don't see it yet, but I'll reply when I do.
<sinzui> abentley, thank you. I curse our slow mailman
* rick_h_ changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: - | Firefighting: - | Critical bugs: 3.47*10^2
<sinzui> wallyworld_, lib/lp/bugs/tests/test_doc.py
<sinzui> wallyworld_, wgrant, StevenK: this is one of my favorite bugs https://bugs.launchpad.net/launchpad/+bug/568768
<_mup_> Bug #568768: Bug UI Inconsistancy <confusing-ui> <javascript> <lp-bugs> <ui> <Launchpad itself:Triaged> < https://launchpad.net/bugs/568768 >
<rick_h_> sinzui: that's a good one
#launchpad-dev 2012-05-23
<rick_h_> that would be a good use case for the icon font, being able to make the icon significant (edit) but make the color signify action like link vs popup
<rick_h_> and then keep the color significance with the text to be link vs popup
<mwhudson> hah
<mwhudson> if you change the product that a bugtask is for, the other ajax controls break
<mwhudson> is this a known bug?
<mwhudson> i'll search but usually asking here is faster..
<rick_h_> probably
<rick_h_> but yea, don't have a number off the top of my head at all
<mwhudson> https://bugs.launchpad.net/launchpad/+bug/919369
<_mup_> Bug #919369: acting on a bug task which has been retargeted oopses <404> <bugs> <javascript> <oops> <Launchpad itself:Triaged> < https://launchpad.net/bugs/919369 >
<wgrant> mwhudson: IIRC it's usually not a problem except on multitask bugs, since changing the current context causes a refresh
<mwhudson> wgrant: this was a multitask bug, indeed
<mwhudson> but hm, it was the task i was accessing the bug through that i changed
<mwhudson> should that not still refresh?
<wgrant> Hmm
<wgrant> I'd expect that to work
<StevenK> wgrant: Given your mail, I hope I can avoid a DB trigger.
<StevenK> But I suspect I have to write some manual SQL to fix up the mistakes.
<wgrant> StevenK: Correct.
<wgrant> StevenK: The idea of this is to avoid a DB trigger.
<cjwatson> The fix for bug 1000570 has been deployed, but it hasn't been marked Fix Released AFAICS.  Did somebody miss a bug closure pass after an NDT?
<_mup_> Bug #1000570: "Packageset.score" is badly named <qa-untestable> <tech-debt> <trivial> <Launchpad itself:Fix Committed by cjwatson> < https://launchpad.net/bugs/1000570 >
<wgrant> cjwatson: Ah, I left that one because it hasn't been renamed in the DB, and doing so is awkward. Not sure what you want to do.
<cjwatson> Ah, well either way it shouldn't stay as Fix Committed.  I'm not planning on doing anything more to it for the time being, largely because I have no idea how to do so smoothly.
<cjwatson> Maybe -> Triaged/Low/unassigned.
<wgrant> That would be my suggestion.
<cjwatson> I'll do that then.
<wgrant> We haven't done a straight column rename since fastdowntime.
<wgrant> But I'd probably do it as a two-day trigger
<cjwatson> Oh, I can't set Triaged.
<wgrant> Doing
<cjwatson> ta
<StevenK> Bah. Dealing with both private and information_type *and* the creation rules in IBranchNamespace.createBranch() is giving me a headache.
<StevenK> (In LaunchpadObjectFactory.makeBranch())
<wgrant> StevenK: Both default to None. At most one can be not None.
<StevenK> Oh, bah, it doesn't help that I overwrite the values during the course of the method.
<StevenK> Right, that should fix it.
<StevenK> wgrant: Can you glance over https://code.launchpad.net/~stevenk/launchpad/branch-use-information_type/+merge/106109 again?
<wgrant> StevenK: Looking
<wgrant> StevenK: Can't the factory just use transitionToInformationType?
<StevenK> wgrant: Nope
<wgrant> It can
<wgrant> Why do you say it can't?
<wgrant> 120	+ if (self.stacked_on and self.stacked_on.information_type !=
<wgrant> 121	+ InformationType.PUBLIC and information_type !=
<wgrant> 122	+ self.stacked_on.information_type):
<wgrant> 123	+ raise BranchCannotChangeInformationType()
<wgrant> That's wrong
<wgrant> The indentation is still criminal
<wgrant> And it doesn't implement the rules we defined this morning
<wgrant> s/indentation/line-breaking/
<wgrant> 156	+ # If the branch we are stacking on is not public, and we are,
<wgrant> 157	+ # set our information_type to the stacked on's.
<wgrant> 158	+ if (self.stacked_on and self.stacked_on.information_type !=
<wgrant> 159	+ InformationType.PUBLIC and self.information_type ==
<wgrant> 160	+ InformationType.PUBLIC):
<wgrant> Should use PUBLIC_INFORMATION_TYPES
<StevenK> wgrant: In terms of lines 120-123, the information_type can change, as long as it remains in PRIVATE_INFORMATION_TYPES ?
<wgrant> StevenK: I think so.
<wgrant> The only constraint we have now is that a newly created branch probably shouldn't be public if it's stacked on something private.
<wgrant> Implementing the same rule on stacking changes is roughly free, so we should do that too.
<StevenK> wgrant: That's lines 156-160 in branchChanged
<wgrant> Right, just explaining that that's the full ruleset :)
<StevenK> wgrant: Are those your two issues?
<wgrant> StevenK: Mostly. Will give it another deep look-over when they're fixed.
<wgrant> lifeless: https://lpstats.canonical.com/graphs/PPR/20110123/20120524/ <- can I delete rosetta?
<StevenK> wgrant: I have them fixed locally, I'm just wondering if I have tests for the stacked on behaviour in transitionToInformationType
<lifeless> wgrant: I undesrstand the temptation
<bigjools> the quality of commit messages these days is awful :(
<wgrant> I did like the "use a private fork of bzr" this morning
<bigjools> no kidding
 * bigjools composes email to list
<lifeless> bigjools: got time for a brief call about your favourite launchpad component ?
<bigjools> lifeless: can do 5 mins now or more later
<StevenK> wgrant: And again?
<StevenK> Hmmm, that's an annoying bug.
<lifeless> bigjools: now is good
<lifeless> bigjools: skype? hangout ?
<StevenK> If you leave the media menu open and go to lunch, you can interact with the menu, but as soon as you close it, the screen locks.
<wgrant> StevenK: The factory is still insane.
<wgrant> StevenK: And it might break lots of text by already having a change event
<StevenK> wgrant: How so?
<wgrant> tests
<wgrant> not text
<wgrant> StevenK: Why doesn't it use transitionToInforationType?
<wgrant> With proper spelling.
<StevenK> wgrant: Because transitionToInformationType has a bunch of rules about when privacy/information_type can change, and some branches want to be created that break those rules.
<wgrant> StevenK: No they don't.
<wgrant> StevenK: If they really need to (eg. for testing that it's possible to fix that situation), I'd set the information_type manually in the test.
<wgrant> I don't think it's valuable to let the factory knowingly create invalid data.
<StevenK> wgrant: http://pastebin.ubuntu.com/1002332/
<wgrant> (because that means tests might unknowingly do it)
<wgrant> StevenK: Ah, I see. So there's no privacy policy?
<wgrant> I wouldn't expect there to be too much fallout from that, and it's one line to fix each test.
<StevenK> wgrant: It is?
<wgrant> Create an appropriate BranchVisibilityPolicy
<StevenK> wgrant: And the project doesn't exist until inside makeBranch()
<wgrant> StevenK: Ah, that's less than convenient.
<StevenK> wgrant: So I need to create a project, a team and a BranchVisibilityPolicy and then I can create a private branch just so makeBranch() can use transitionToInformationType?
<wgrant> StevenK: Technically, yes, you should.
<wgrant> StevenK: But perhaps that's not ideal here yet.
<StevenK> wgrant: There are 246 occurances of private=True in the tree -- some of those are going to be for makeArchive(), but ... ouch?
<StevenK> lib/lp/soyuz/tests/test_binarypackagebuildbehavior.py:            UPDATE archive SET private=True,buildd_secret='foo'
 * StevenK facepalms
<wallyworld_> wgrant: is there a trigger which updates btf.access_policies and btf.access_grants when a bug info type or bugtask target changes? if there is, i have missed finding it
<wgrant>     z_bug_maintain_bugtaskflat_trigger AFTER UPDATE ON bug FOR EACH ROW EXECUTE PROCEDURE bug_maintain_bugtaskflat_trig()
<wgrant>     z_bugtask_maintain_bugtaskflat_trigger AFTER INSERT OR DELETE OR UPDATE ON bugtask FOR EACH ROW EXECUTE PROCEDURE bugtask_maintain_bugtaskflat_trig()
<wgrant> wallyworld_: ^^
<wgrant> (z because they have to happen after the FTI updates etc :/)
<StevenK> wgrant: So, shall I leave the information_type = call in the factory?
<wallyworld_> wgrant: ok, thanks. something else is going wrong for me then.
<wgrant> StevenK: I suppose.
<StevenK> wgrant: Do you have any other issues with the MP?
<stub> Well thats amusing. Package builds fine under i386, not amd64
<wgrant> stub: Possibly because i386 builds arch-indep, amd64 doesn't.
<wgrant> If it's not an obvious arch difference.
<stub> https://launchpadlibrarian.net/105848284/buildlog_ubuntu-lucid-amd64.postgresql-9.1_9.1.3-2%2Blp4_FAILEDTOBUILD.txt.gz ?
<wgrant> Yeah
<wgrant> arch-indep difference
<wgrant> i386 will be building doc packages, amd64 won't
<wgrant> But both copy them into debian/tmp
<wgrant> StevenK: Your linebreaking is still a criminal offence.
<wgrant> 121	+ if (self.stacked_on and self.stacked_on.information_type in
<wgrant> 122	+ PRIVATE_INFORMATION_TYPES and information_type in
<wgrant> 123	+ PUBLIC_INFORMATION_TYPES):
<StevenK> wgrant: When can I expect you over to arrest me?
<StevenK> wgrant: What would you prefer, nested ifs? :-)
<wgrant> StevenK: Replied
<StevenK> wgrant: http://pastebin.ubuntu.com/1002355/
<wgrant> StevenK: I accept your plea bargain.
<StevenK> Haha
<StevenK> wgrant: And the linebreak for name='branch_%s' % x, private=x > 2) is still required.
<wgrant> Really? it didn't look that long.
<wgrant> How far over is it?
<StevenK> wgrant: 81
<wgrant> StevenK: s/branch_/b/g
<wgrant> Done
<StevenK> branch = self.factory.makeBranch(name='branch_' + x, private=x>2)
<wgrant> I mean, naming a branch "branch_foo" doesn't really make sense anyway.
<StevenK> The branches do not require names, we don't test for them.
<wgrant> Even better
<wgrant> Remove stupid piece of test, save a line.
<wgrant> Everybody wins.
<wgrant> One less crime.
<StevenK> wgrant: -4/+2
<StevenK> steven@undermined:~/launchpad/lp-branches/branch-use-information_type% bzr log | head -n 7 | tail -n 1
<StevenK>   Drop some useless names from a test, and fix my crimes against humanity.
<StevenK> wgrant: ^
<wgrant> They were merely capital offences, not crimes against humanity.
<lifeless> which capital?
<StevenK> lifeless: So, looks like I need some manual SQL. :-(
<wgrant> LATER
<wgrant> Not until this is deployed everywhere
<StevenK> But won't that leak some branches?
<wgrant> Don't you still respect transitively_private?
<StevenK> Only information_type in the New World Order.
<wgrant> Oh
<wgrant> Indeed you don't
<StevenK> In fact, I don't think Branch._private is even needed
<wgrant> There's only like a dozen branches that are affected, right?
<StevenK> 29
<wgrant> Any created recently?
<wgrant> Normally you'd do a 3-stage migration here, but they may be rare enough that you can get away with 2-stage with manual SQL.
<StevenK> wgrant: I'm going to kill Branch._private, it isn't needed because the two columns are named differently.
<wgrant> StevenK: Sounds reasonable.
 * wgrant fixes cjwatson's test failurse
<lifeless> 9.1 right ?
<lifeless>      559 /  222  Distribution:+ppas
<lifeless>      348 /   37  Builder:+history
<wgrant> lifeless: Yes
<StevenK> Builder:+history has a bug filed against it, at least.
<wgrant> I believe Distribution:+ppas is similar misbehaviour on a different query, but I haven't looked in depth.
<wgrant> Both rarely used pages, anyway.
<wgrant> Particularly since Distribution:+ppas timed out half the time anyway
<wallyworld_> wgrant: i have a question i'd like to ask if you have a moment. a bit of code which calls bug.subscribe() appears to mess with bug visibility checks. i have a bit of code in https://pastebin.canonical.com/66554/ to try and illustrate the issue
<wgrant> wallyworld_: Looking
<wallyworld_> thanks. hope it makes sense
<wgrant> wallyworld_: What's policy_team_grantee?
<wallyworld_> wgrant: typo, sorry
<wallyworld_> should be policy_gantee
<wallyworld_> argh, you know what i mean
<StevenK> Haha
<wallyworld_> can't type
<wallyworld_> i cut and pasted a bit of code from a test and stripped out bits
<StevenK> wallyworld_ is obviously so excited by the prospect of a NSW win tonight that his hands are shaking.
<wallyworld_> StevenK: you can go and f%^@@!%^%^@ yourself
<wallyworld_> blasphemy
 * wallyworld_ just hope to make it home from soccer training on time not to miss any
<wgrant> wallyworld_: You haven't disabled the triggers?
<wallyworld_> wgrant: no. i had. but there were other issues
<StevenK> wallyworld_: We have this thing, called a PVR. It can pause live TV.
<wgrant> wallyworld_: Toggling the information_type on line 40 without disabling the triggers will cause the grant to be recreated due to the subscription.
<wallyworld_> StevenK: but others will be watching
<bigjools> it's rugby *league*, who TF watches that crap
<wgrant> What's the difference?
<StevenK> Bwaha
<wallyworld_> wgrant: ah, bollocks. thanks for spotting that. i'll disable the triggers again and see if i can reproduce the issue that caused me to disable them
<StevenK> bigjools: wallyworld_ will be *glued* to the TV.
<wallyworld_> bigjools: it's state of origin. better than any super 16 or 6 nations stuff
<bigjools> you can tell yourself that, yeah :)
<StevenK> Who cares? Both rugby league and union are played by and watched by bogans.
<wallyworld_> bigjools: it's a lot tougher. i mean what sort of rugger pussy disallows a shoulder charge
<bigjools> wallyworld_: league is the one where after they get tackled they hump the ball for a bit right?
<wallyworld_> StevenK: i don't consider myself a bogan
<wgrant> wallyworld_: The thing to remember about the triggers is that they were designed to always keep the mirror in a sane state, regardless of whether it was sane to start with. Every time they're fired, they force any AAGs and APAs to match the legacy disclosure data.
<wallyworld_> bigjools: only in your dreams
<wgrant> wallyworld_: I couldn't just apply a delta, since the table started empty.
<wallyworld_> wgrant: ok. i'll try and get this particular test case to work with the legacy triggers disabled from the start. the sharing jobs in question are not really meant to be run with the triggers on
<wallyworld_> i ran into issues but can't recall what they are right now
<wgrant> I'm happy to help if you hit trouble.
<wallyworld_> no problem, thanks
<stub> Is there some magic to dput to get packages build for multiple releases since I can't use the web UI to rebuild packages for a different release in the same PPA?
<wallyworld_> wgrant: i'll send out an email maybe tonight to summarise where we are at with being able to turn off the legacy triggers. i think we will be close after i get this branch sorted. we'll have code to grant access on subscribe and also unsubscribe if access becomes verboten
<bigjools> stub: you need >1 upload
<bigjools> unless you want to copy the same build
<stub> Recipes let me do it I guess.
<bigjools> yep
<wgrant> wallyworld_: Marvellous.
<wgrant> stub: Right, recipes are handy for that. Or you can often build in lucid and copy up, as long as a soname bump hasn't happened.
<wallyworld_> wgrant: main outstanding bit will be some form of garbo / celerybeats / cron job to check for inconsistencies if we feel that's necessary
<wallyworld_> wgrant: one issue - without the legacy triggers, self.factory.makeAccessPolicyGrant does not allow the bug to be visible
<wgrant> wallyworld_: You probably want to disable bug_mirror_legacy_access_t, bugtask_mirror_legacy_access_t, bugsubscription_mirror_legacy_access_t
<wgrant> wallyworld_: You still want things mirrored to bugtaskflat, you just don't want the legacy sharing schema mirrored to the new one.
<wallyworld_> wgrant: those ones are the only 3 i am disabling
<wallyworld_> using a new fixture i wrote
<wgrant> wallyworld_: Are you creating the APA manually?
<wgrant> Disabling those triggers will prevent APAs from happening automatically
<wallyworld_> no
<wgrant> So the APG will not be effective for the bug.
<wgrant> Because it's not linked to any policies.
<wallyworld_> wgrant: i would have hoped that APA will be done by triggers still
<wgrant> wallyworld_: It's all done by the one trigger right now, unfortunately.
<wallyworld_> :-( so we need to do a new trigger
<wallyworld_> and strip out the APA stuff
<wallyworld_> from the legacy trggers
<wgrant> AAG stuff
<wgrant> Well
<wgrant> Split them up
<wgrant> Or we can find all the stuff that creates tasks, all the stuff that changes targets, and all the stuff that changes information types
<wallyworld_> ok, i'll do it bt hand for now and include an xxx
<wgrant> And make them create the APAs in the application.
<wgrant> That's the ideal situation, and it might be doable now
<wgrant> Since those things are reasonably well factored nowadays.
<wallyworld_> hmm. ok. this branch is already waaay to big. might do it in the test for now and do a new branch to do it in the model
<wgrant> Target setting is guarded, task creation is mostly factored into a single method, and information type transitioning is done by a single method.
<wgrant> It'll certainly need to be a separate branch or three.
 * wallyworld_ off to soccer and then straight home to watch the might maroons thrash nsw
<wgrant> Heh
<wgrant> Is that where bigjools heard the odd pronunciation?
<wallyworld_> odd? it's pronounced "marown"
<wgrant> Odd to him.
<bigjools> you're all wrong
<adeuring> good mmorning
<cjwatson> wgrant: Doh.  Thanks
<cjwatson> hate doctests
<wgrant> That's the point :)
<cjwatson> Confused at how that passed EC2.
* frankban changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: frankban* (rvba) | Firefighting: - | Critical bugs: 3.47*10^2
<cjwatson> Could somebody set https://code.launchpad.net/~cjwatson/launchpad/remove-distro-parts/+merge/106920 to Status: Approved (since it's had an Approve review) so I can land it?
<czajkowski> cjwatson: done
<cjwatson> Thanks
<wallyworld_> wgrant: around?
<wgrant> wallyworld_: Sure
<wallyworld_> wgrant: 1/2 watching footy so may no be seeing it, but see https://pastebin.canonical.com/66571/ - #4 is not as i expect
<wallyworld_> wgrant: triggers disabled
<wgrant> wallyworld_: Recall that the triggers are disabled, so changing bug.information_type isn't going to change the APA
<wallyworld_> wgrant: only the legacy triggers are disabked
<wgrant> Right, and they maintain APA
<wallyworld_> wgrant:  i thought there were separate trigers for info tpy changes
<wgrant> That's why you had to create it manually
<wgrant> No. Bug* -> Access* is all the legacy triggers
<wallyworld_> hmmm.
 * wallyworld_ sad
<wallyworld_> i think i will need to create a db-devel branch to fix that
<wgrant> I can reimplement all that in the app tomorrow if you want
<wallyworld_> before i land this branch
<wgrant> The APA stuff
<wgrant> Why do you need a db-devel branch?
<wallyworld_> yes, we really do need that now
<wallyworld_> db-devel  to add the triggers
<wgrant> We can't remove the triggers until after the appserver code is deployed.
<wallyworld_> can't we split out the APA stuff
<wallyworld_> still have the legacy triggers but have the APA stuff on its own
<wallyworld_> since we will need it even when legacy triggers are gone
<wallyworld_> oh, you want to do it in the app
<wgrant> Yeah
<wgrant> It's become a little less of a free-for-all in the last few months
<wgrant> So it's easy enough to do there now
<wallyworld_> whatever works :-)
<wallyworld_> i really don't want to "pollute" the tests with the APA stuff
<wgrant> Since we're going to be managing grants from the app, we might as well manage the APAs too
<wallyworld_> yeah. the calling code should be easy to write. at the moment it isn't :-)
<wallyworld_> well, not intuitive or transparent perhaps
<wgrant> The task creation code is all now nicely centralised, same with information_type, and I refactored target setting, so there's basically just 3 methods that need hacking
<wallyworld_> wgrant: we would need to use the legacy triggers removed feature flag
<wallyworld_> so the triggers and app code don't double up
<wallyworld_> oh, i haven't landed that flag yet
<wallyworld_> sigh
<wgrant> wallyworld_: They can't double up.
<wgrant> They're idempotent.
<wallyworld_> wgrant: i've been getting pk errors if i don't diable the triggers and have app code to do the same thing
<wgrant> wallyworld_: The triggers won't recreate stuff that already exists.
<wgrant> So the appserver code will just check what's there, and fix it if it's wrong.
<wgrant> Same as the triggers.
<wallyworld_> wgrant: yes but they fire first
<wallyworld_> and my app code didn;t check, just assumed the stuff needed creating
<wallyworld_> anyways, footy is back on :-)
<wgrant> Right. My centralised app code will render that no longer necessary, and will do the check.
<wgrant> Heh, k
<wgrant> Thanks for sorting all this stuff out!
<wallyworld_> np. i'll code my branch and it tests will fail and i'll merge devel when your stuff is in and tests will work
<wgrant> Yeah
<wgrant> I'll also grep for any makeAccessPolicyArtifact calls that are lying around
<wgrant> And delete them.
<wallyworld_> awesome
<wallyworld_> it's all quite verbose atm
<wgrant> Yeah
<wgrant> The transition phase wasn't meant to last quite as long as it has :)
<wgrant> Still, its end is nigh
<wallyworld_> yep, can't wait
<wallyworld_> and we nweed to do branches
<wgrant> Branches are easy :)
<wgrant> As of 12 hours ago...
<wallyworld_> i have a few XXXs in tests etc
<wallyworld_> waiting for it to be sorted
<wgrant> Great
<wallyworld_> so we need the same app code as we just discussd for bugs
<wgrant> Branch code will be entirely app. In fact, we'll lost the transitively_private trigger shortly.
<wgrant> But yeah
<wgrant> StevenK is basically cleaning up the points we'll need to hook into
<wallyworld_> cool
<jelmer> hmm, quantal and lxc don't seem to be an ideal combination here at the moment
<wgrant> jelmer: Oh?
<wgrant> What's breaking?
<jelmer> kernel panic :)
<wgrant> Oh
<wgrant> That's quite impressive.
<wgrant> 3.4?
<jelmer> yep
<jelmer> this is the second time it's happened to me, next time I'll take a photo
<mgz> ...you mean you don't have a serial cable hooked up?
<mgz> I'm disappointed in you jelmer.
<jelmer> hehe
<wgrant> mgz: Did you get your LP instance working?
<mgz> wgrant: yup, and forwarded via socks over chinstrap to the browser here
<mgz> one oddness was the new bugs page wasn't in, so maybe I didn't have it setup the same way the current live version is?
<mgz> are there more things I managed to ignore on the wiki about setting flags or similar?
<jelmer> mgz: you probably have to be in one of the special groups for that?
<mgz> I thought new bugs were on for everyone now?
<StevenK> I thought it was still behind a feature flag
<mgz> otherwise, how do you make the stub test data gain the feature flags you care about?
<StevenK> mgz: You add it? http://launchpad.dev/+feature-rules
<mgz> StevenK: thanks :)
<wgrant> It's still behind a feature flag, despite being enabled everywhere for nearly 6 months
<wgrant> The old code was never removed.
<mgz> are there other features that are enabled live but still not on by default for the test setup?
<StevenK> mgz: Sounds like a good learning exercise. ^ :-)
<mgz> or is the bug listing stuff unusual...
<StevenK> mgz: Some feature flags are supposed to be short lived
<StevenK> I'm hoping that disclosure.show_information_type_in_ui gets ripped out next week, for instance.
<mgz> is there somewhere that tells me what launchpad beta testers have turned on?
<StevenK> You can have a look at the feature rules on prod
<mgz> I can, or I can tell some poor l-osa to?
<deryck> Morning, all.
<StevenK> mgz: You can see them, you just can't edit.
<mgz> StevenK: is there some documentation you can point me at? things involving 'prod' I'm nervous prodding without knowing what I'm doing.
<mgz> am not even aware of which boxes are involved with the live launchpad setup for example.
<StevenK> mgz: You can't change anything there, so there is no harm in looking.
<StevenK> mgz: There is some doco around of the LP producution setup including a diagram. I can't recall where off the top of my head
<cjwatson> mgz: are you looking for the URL?  https://launchpad.net/+feature-rules
<mgz> ah, doh, I assumed that'd be a dev setup only thing
<mgz> and the changelog is very informative.
<cjwatson> It's restricted to ~admins and ~registry
<cjwatson> Mortals like myself just get 403 :)
<mgz> :)
<czajkowski> cjwatson: ah thought you were a demi god myself
<mgz> just in all other regards czajkowski :)
<cjwatson> Only for stuff under /ubuntu :)
<czajkowski> cjwatson: ahhh :)
<stub> I can't find my recipe
<stub> ahh... wrong branch
<jam> mgz: how would you travel from the UK to NL, I'm trying to decide if Eindhoven or Utrecht is a better meeting point. Ein has a direct Stansted <=> Ein flight, but Utrecht is a more central train station.
<mgz> jam: norwich 'international' airport has a hop to schiphol
<mgz> stanstead isn't too hard to get to either
<mgz> there used to be a ferry from these parts too, but I think no longer
<jam> mgz: could you do a quick check on pricing, including train/taxi/whatever for the two routes?
<mgz> sure
<cjwatson> Stansted is way easier to get to than Norwich from just about anywhere except Norwich
<cjwatson> Though I know the latter's probably more important to mgz :)
<jam> cjwatson: yeah
<jam> I was surprised how many people in directory.canonical.com are listed as in Norwich, though. (4)
<mgz> and amigadave is also norfolk
<mgz> we're doing pretty purple round here.
<mgz> so, NOR-AMS return is Â£120-150 range, then need train on the dutch side which you might have a better idea on
<mgz> heh, first web search hit is the dutch train operator, second in bbc news article about crash in april
<mgz> looks like about 8 euros one way to utrecht from schiphol, so not expensive
<mgz> from stansted I'm failing to find flights to eindhoven at all, but to schiphol it's Â£110 upwards
<jelmer> mgz: aren't you near Harwich? How long does the boat take?
<mgz> to get the train to eindhoven goes through utrecht, costs about another 10 euros
<mgz> jelmer: however long it takes me to hide in a container ship without being noticed? :)
<jelmer> mgz: hehe
<jelmer> mgz: I thought Stena Line operated from Harwich to Hoek van Holland?
<mgz> they do seem to, takes 6 hours, failing to find prices
<jelmer> ouch, 6 hours is quite long :-/
<mgz> not if you're sleeping :)
<mgz> Â£36 return, plus train at...
<mgz> less than Â£27 this end, plus...
<mgz> about 14 euros one way going via rotterdam to utrecht
<mgz> I'll send you an email later jam.
<cjwatson> webops: I have some QA to do on https://code.launchpad.net/~cjwatson/launchpad/archive-build-score-api/+merge/106804, and need admin help.  Could somebody in ~admins run http://paste.ubuntu.com/1003094/ at an 'lp-shell qastaging devel' prompt and tell me what it says?
<jam> mgz: http://www.ryanair.com/en
<gnuoy> cjwatson, I'll take a look
<jam> mgz: ams => ein is about 40 round trip
<cjwatson> Hopefully it should give no errors and the last line should return 100.
<jam> (40 Euros, not GBP)
<jam> mgz: note that right now, I'm looking at the week of June 25th, for availability.
<wallyworld_> abentley: hi. celery question. the wiki page says to add jobs to run_missing_ready(). Should that method use the feature flag to get the celery jobs to pass to find_missing_ready()? It's current just hard coded to look for branch scan jobs only.
<jam> mgz: also note that ryanair tends to add a lot after the initial decision.
<jam> like it is only 10.99 GBP for the return EIN => STE, but that doesn't include checking a bag, etc.
<abentley> wallyworld_: It can't use the feature flag to get the jobs, because that does not contain enough information.
<jam> and if you leave at 9am, vs 5pm, etc.
<wallyworld_> abentley: the feature flag contains the job class names. is that not enough?
<abentley> wallyworld_: Also, there's not a one-to-one relationship between job sources and job types.
<czajkowski> jam: mgz jelmer http://www.speedcommunications.com/blogs/speed/2012/05/22/ryanair-taking-the-biscuit/
<jam> mgz: and I swear they charge you a lot more for checking a bag. I thought it was ~20 Euro for round trip, but I'm seeing 50GBP for the first bag.
<abentley> wallyworld_: Indeed, that is not enough.
<cjwatson> jam: I just refuse to fly RyanAir, saves aggravation all round
<jam> cjwatson: I've had quite good success with them, but mostly because they fly out of my city
<jam> not a lot flies to Ein, and otherwise I have a 2hr train to AMS
<gnuoy> cjwatson, I got this: http://paste.ubuntu.com/1003098/
<wallyworld_> abentley: ok. so we should just create a list of jobs in the run_missing_ready() method and iterate over those? we really should set up a register method or something that jobs use to register themselves and it's all taken are of from there
<abentley> wallyworld_: Maybe.  It didn't seem like there were any great answers, so I did the simplest thing.
<cjwatson> gnuoy: That looks like a stale WADL cache, I think.  Could you run "find ~/.launchpadlib -name '*qastaging*devel*wadl*' | xargs rm" and try again?
<wallyworld_> abentley: sure :-) i'm jsut mking sure i understand what i need to do for my new jobs
<abentley> wallyworld_: So ideally, the registration is done in the same module as run_missing_ready.  If it's done in a Job-specific module, then there's the risk that the module won't be loaded.
<abentley> wallyworld_: So a list in run_missing_ready seems as good a place as any to "register" the job sources.
<wallyworld_> abentley: yes, same module. and this allows for any other required processing in addition to run_missing_ready to be hooked in
<abentley> wallyworld_: YAGNI
<gnuoy> cjwatson, still getting the same error: http://paste.ubuntu.com/1003102/
<wallyworld_> abentley: are we expecting all celery jobs need to be handled by run_missing_ready()
<cjwatson> gnuoy: Urr.
<cjwatson> Any launchpadlib/lazr.restfulclient/wadllib experts want to help me out understanding the above?
<abentley> wallyworld_: All jobs that run via celery should have sources handled by run_missing_ready.
<wallyworld_> abentley: so in that case i think registration should be automatic so it's not missed out
<wallyworld_> via whatever mechanism makes sense
<wallyworld_> otherwise it's a pita to have to define a new job and remember to add it to a list somewhere else
<wallyworld_> that would be quite fragile
<abentley> wallyworld_: I don't think registration can be automatic, because it needs coordination between two modules, the one where the class is defined and the one where the registration needs to happen.
<wallyworld_> why can't it be done? because thre are 2 modules inolved?
<wallyworld_> i'm not sure i see the problem
<abentley> wallyworld_: Yes, and because the module where the class is defined may not be loaded.
<abentley> wallyworld_: So you can't make declaring the class simply have a side effect of registering the class.
<wallyworld_> it would be the class which registeres itself
<abentley> wallyworld_: See above.
<wallyworld_> enum types seem to do that
<abentley> wallyworld_: Code that is not executed cannot have side effects.  A module that is not loaded cannot register a class.
<wallyworld_> right, i see what you are saying. perhaps there can be a config file where the known jobs are listed
<wallyworld_> i guess that's what the run_missing_ready is
<wallyworld_> or could be
<abentley> wallyworld_: Right.
<wallyworld_> so it seems there's a bunch of celery jobs missing from that list then
<cjwatson> gnuoy: Perhaps I could have a look at whatever file(s) are shown by "find ~/.launchpadlib -name '*qastaging*devel*wadl*'"?
<abentley> wallyworld_: Sure, but we also knew that we were only planning on doing BranchScanJobs for now, so I didn't add the others, because it would be a wasted of CPU.
<wallyworld_> i thought a number of other jobs had been ported to celery?
<abentley> wallyworld_: Right, but we're off maintenance, so we had to tie off work here.  We settled on doing BranchScanJobs as our spike.
<wallyworld_> ok. so those other jobs will just run as normal not with celery?
<gnuoy> cjwatson, I tried moving  ~/.launchpadlib out of the way and rerunning, seemed to get the same error. Let me get a colleague to give it a go as well to confirm its not just me
<abentley> wallyworld_: Right.
<wallyworld_> abentley: ok. so i have 2 new jobs written only for celery. so i'll add those to the list
<abentley> wallyworld_: Cool.  If you're in the area, it would make sense to change find_missing_ready to take a list of job sources, so that we only have to run it once.
<cjwatson> gnuoy: Weird.  I was hoping to look at the wadl to see if Archive.relative_build_score is there at all.
<abentley> wallyworld_: (which means we only have to query the queue once.)
<gnuoy> cjwatson, sure, let me get it
<wallyworld_> abentley: yes. i'll do a branch for that and get you to review it. can i land the branches with the jobs etc before i add the jobs to run_missing_ready?
<abentley> wallyworld_: Certainly.
<wallyworld_> abentley: cool. will do that. i have a 8 branch pipe i need to get landed before i go too much further
<wallyworld_> abentley: thanks for the clarifications on this stuff
<gnuoy> cjwatson, chinstrap.canonical.com:/home/liam/api.qastaging.launchpad.net,devel,-application,vnd.sun.wadl+xml,66834a8cfcc85749a969518f3c248856
<abentley> wallyworld_: np.
<cjwatson> gnuoy: Weird, Archive.relative_build_score does appear to be there.
<cjwatson> Maybe I should try myself on DF or something
<cjwatson> gnuoy: Oh, maybe this is bug 662740
<_mup_> Bug #662740: Setting an attribute on a shim object without first reading an attribute causes a crash <lazr.restfulclient:Triaged> < https://launchpad.net/bugs/662740 >
<cjwatson> gnuoy: Could you try just 'archive.relative_build_score' (i.e. read the attribute) before trying to set it?
<gnuoy> cjwatson, sure
<gnuoy> cjwatson, I seem to be able to read it fine: >>> archive.relative_build_score
<gnuoy> 0
<cjwatson> gnuoy: OK, then try the rest immediately after that
<cjwatson> Starting from 'archive.relative_build_score = 100'
<gnuoy> cjwatson, that works: http://paste.ubuntu.com/1003122/
<cjwatson> gnuoy: Great, that's step one then (that lazr.restfulclient bug is unrelated to my QA).  Would it be possible to add me briefly to commercial-admins so that I can do the final step?
<cjwatson> On qas, obviously
<gnuoy> cjwatson, sure
<gnuoy> cjwatson, done
<cjwatson> gnuoy: Great, thanks, that passed (i.e. commercial-admins can still write that attribute) - please remove me from that team again
<sinzui> jcsackett, This is the bug about the privacy banner. I think the issue is really about the switch to radio buttons: https://bugs.launchpad.net/launchpad/+bug/1003482
<_mup_> Bug #1003482: privacy banner is not shown by default when reporting a bug <disclosure> <regression> <ui> <Launchpad itself:Triaged> < https://launchpad.net/bugs/1003482 >
<jcsackett> sinzui: dig. i think i can whack that in line with the other banner stuff i'm doing.
<sinzui> jcsackett, ian and wallyworld_ believe the radio buttons are inconsistent. We are discussing changing the widget. Your fix is appreciated, we want to be sure what ever widget we choose, the banner appears
<jcsackett> sinzui: yeah, there's no reason the banner's appearance on a private-by-default project should be dependent on the info_type selector.
<jcsackett> or rather, on the *type* of selector.
<jam> have a good evening everyone, I'm EOD
<cjwatson> gnuoy: Oh, of course, I can leave a team myself.  Done
<gnuoy> cjwatson, thanks
<danilos> frankban, hi, can you please take a look at https://code.launchpad.net/~danilo/launchpad/bug-1000787/+merge/107041
<frankban> sure danilos, give me a minute and I'm on it.
<danilos> frankban, cool, thanks
* frankban changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: - | Firefighting: - | Critical bugs: 3.47*10^2
<jcsackett> rick_h_: is there way when using the combo loader and jsbuild_watch to not have minified js?
<jcsackett> it makes hunting a bug a bit harder.
<rick_h_> jcsackett: yea, add a filter='raw' to the config
<rick_h_> otp atm, but can get you a paste in a sec
<jcsackett> rick_h_: ok.
<danilos> frankban, rvba: hi, can you perhaps get the branch landed for me if you are fine with the latest changes as per your review?
<rvba> Hey danilos. Can you take care of it frankban?
<danilos> mrevell, hi, any news from ubuntu people on acceptance/headline and removal of the migration script?
<frankban> danilos: I will get your branch landed
<danilos> frankban, cool, thanks again
<frankban> you are welcome
<sinzui> czajkowski, do you have the mad skillz to make a video appear in a Wordpress post?
<czajkowski> um..........
<czajkowski> sure pp it in and I'll have a look in a wee bit if you like
<czajkowski> just heading out to grab some food
<czajkowski> *pop
<czajkowski> sinzui: hmm i dont have access to the YT account to upload the vide
<czajkowski> and clicking on it in chrome makes it open in another tab and not play :/
<sinzui> yes. I think chromium want webm
 * sinzui does not want to render the video again
<sinzui> czajkowski, We can wait for mrevell
<czajkowski> sinzui: see -ops found the link on wiki :)
<czajkowski> I think..
<gary_poster> StevenK, hey.  Is there a concrete reason why you think the new bug you reported has to do with us?  Ooh!  I have an idea.  The only way I can think of that we might be responsible is the new zope.testing.  I already reverted it locally because it is broken for us.  Maybe it is broken in whatever way you need too.
<StevenK> gary_poster: Yeah, you seemed like the most likely team implicated. :-)
<gary_poster> StevenK, fair enough.  I'm already well past my EoD and dealing with other gigantic new baffling issues of our own, so help with this would be appreciated.  I suggest reverting r15283.  I'm 80% sure that's the cause, and I know it's broken in other ways
<StevenK> gary_poster: r15283 has been reverted in r15293.
<gary_poster> StevenK, and so would that correlate with your failure?
<StevenK> gary_poster: But thank you for looking at it. :-)
<gary_poster> 15254
<StevenK> Oh, sorry
<lifeless> gary_poster: hi; I've commented on the LEP itself
<lifeless> gary_poster: rather than on the thread
<StevenK> I reverted my branch
<StevenK> [testfix][r=stevenk][rollback=15289] Revert r15289,
<StevenK>  it passed ec2 and failed buildbot.
<gary_poster> lifeless, oh ok thanks for heads up
<lifeless> gary_poster: I'd be delight to talk via voice, e.g. tomorrow, if you want to
<StevenK> gary_poster: Sorry, I'm a muppet. I can do a rollback of r15283 if you wish.
<StevenK> gary_poster: What breakage have you seen, so I can write a good commit message?
<gary_poster> lifeless, cool, thanks.  I see your gist now, and will look in more detail later.
<gary_poster> StevenK, the subunit output is broken.
<gary_poster> um, in particular...
<gary_poster> I believe that errors are inserted within other errors within the stream
<gary_poster> but I didn't look too closely beyond "broken! broken!"
<gary_poster> StevenK, ^^
<StevenK> gary_poster: Right, if the subunit stream is broken that would explain why ec2 didn't die horribly.
<gary_poster> yeah
<gary_poster> Must run.  bye all
#launchpad-dev 2012-05-24
<StevenK> wgrant: https://code.launchpad.net/~stevenk/launchpad/branch-use-information_type-redux/+merge/107150
<wgrant>     Resurrect branch-use-information_type and fix all of the test failures.
<wgrant> StevenK: Do you have a diff of just the last part?
<StevenK> Hm, I could possibly construct one
<wgrant> StevenK: Also, in branchChanged and the child bit of transitionToInformationType you're setting information_type directly.
<StevenK> wgrant: http://pastebin.ubuntu.com/1004087/
<wgrant> Version control systems are very good at this if you commit regularly :)
<StevenK> Shhh
<StevenK> wgrant: I don't like the idea of recursing into transitionToInformationType from transitionToInformationType.
<wgrant> StevenK: In fact, I don't like the idea of recursion at all
<wgrant> Drop it
<wgrant> The use cases are very few.
<wgrant> information_type is now only inherited to the branch that's doing the restacking.
<wgrant> Changing information_type no longer propagates.
<wgrant> The number of potential writes is unbounded, and I can't see why anybody would really want it.
<wgrant> We saw this morning that all the transitively_private cases are cases where people have project misconfigurations that will soon no longer be possible
<wgrant> And +sharing will soon say "You have 10 private branches, fool"
<wgrant> s/private/public
<wgrant> In the very rare case that it might happen.
<wgrant> I don't see how it will happen, though.
<wgrant> StevenK: So, drop the child code, and any tests that require it.
<wgrant> StevenK: Still, the branchChanged information_type setting thing needs to be transitionToInformationType
<wgrant> StevenK: This probably means adding a flag to transitionToInformationType that says "I actually know what I'm doing and really want this to happen; don't bother checking if the policies allow this branch to be private"
<StevenK> wgrant: Right -- I don't mind that idea, a strict_checking=True argument to transitionToInformationType.
<wgrant> I was thinking i_promise_i_know_what_i_am_doing, but that works too
<StevenK> wgrant: For a further win, I think that means the factory can use transitionToInformationType as well
<wgrant> StevenK: That would indeed be the case.
<StevenK> But I will worry about that after lunch.
<wgrant> Sounds like a plan.
<bigjools> r15291 \o/
<wgrant> Indeed
<wallyworld_> wgrant: StevenK: it seems to me that FileBugInlineFormView is defined and used in some tests but is not hooked up anywhere and so is never used. am i right?
<wgrant> wallyworld_: That seems to be correct.
<wallyworld_> wgrant: well, i'm going to delete it then i think
<wgrant> Probably back from the days when it was more AJAXy
<wgrant> Indeed
<wgrant> Kill
<wgrant> Note that some of the tests may be the only tests of the base class' functionality.
 * wallyworld_ gets out the BFG 9000
<wgrant> That may be why it was kept.
<wallyworld_> hmmm. perhaps. will look at that
<StevenK> wgrant: http://pastebin.ubuntu.com/1004143/
<wgrant> StevenK: Maybe s/strict_checking/verify_policy/, since that seems to be the only check that needs skipping
<wgrant> You can then reword the factory like this:
<wgrant> if private is not None:
<wgrant>     information_type = blah
<wgrant> if information_type is not None:
<wgrant>     tTIT
<wgrant> Rather than the nesting you have now
<StevenK> wgrant: http://pastebin.ubuntu.com/1004148/
<wgrant> StevenK: "Check if the new information type complies with the `IBranchNamespacePolicy`."
<wgrant> parens on line 42 are redundant.
<StevenK> wgrant: If True, check if ... or just what you have?
<wgrant> I think just what I have, but it's subjective.
<wgrant> On line 64 the flag should be named explicitly to avoid people copying the code badly.
<wgrant> Same with 127
<wgrant> Otherwise fine.
<wgrant> As long as information_type isn't set manually anywhere else.
<StevenK> wgrant: A bunch of tests do it
<wgrant> They are evil, but don't really count.
<wgrant> They can't break anything
<StevenK> wgrant: http://pastebin.ubuntu.com/1004155/
<wgrant> Hm
<wgrant> Buildbot passed less than 5:40 after you reverted your branch
<wgrant> Suspicious
<wgrant> Elapsed5 hrs, 37 mins, 26 secs
<StevenK> It was marked testfix, buildbot won't wait for those
<wgrant> Even so, builds have been taking more than 6 hours lately
<wgrant> I guess we have had some optimisations in the last couple of days.
<wgrant> test 16776 tests 16776 passed
<wgrant> So it's correct.
<wgrant> (well, there's potentially 300 missing, but I don't know quite how buildbot counts)
<StevenK> That buildbot pass should fix up the db-stable deployment report too
<wgrant> Not until it's on prod, IIRC
<wgrant> But I don't quite remember.
<StevenK> wgrant: Anyway, you seem fairly happy with this branch, shall I commit and push?
<wgrant> StevenK: Indeed, and I shall go over it with my Fine-Toothed Comb of Pedantry again.
<wgrant> Total LoC delta for William Grant: -740. Churn: 25890
<wgrant> Yay
<lifeless> wgrant: ohlo ?
<wgrant> cjwatson's thing
<lifeless> oh, I haven't seen that
<lifeless> linky ?
<wgrant> It's in lp-dev-utils now
<wgrant> Apparently
<lifeless> I should pull that :P
<lifeless> did he send mail about it ?
<wgrant> It counts from the introduction of the new policy by default
<wgrant> He didn't
<wgrant> It was discussed only on IRC and in the MP AFAIK
<lifeless> ah
<lifeless> perhaps I don't get lp-dev-utils MPs
<StevenK> wgrant: How do you get the churn?
<wgrant> StevenK: I used the vim option
<StevenK> Hah
<StevenK> wgrant: MP diff updated
<StevenK> wgrant: How goes the combing?
<wgrant> StevenK: Sorry, got distracted
<wgrant> Looking
<wgrant> wallyworld_: In your sharing jobs MP, I think you left the old sharing_jobs.error_dir config key there
<wgrant> You deleted the section header, but not the error_dir key
<StevenK> wallyworld_: It's raining in Sydney today -- you may think of it as the tears of NSW supporters if you wish.
<StevenK> wgrant: Stop getting distracted!
<wgrant> StevenK: Looks reasonable.
<StevenK> wgrant: Toss it at ec2?
<wgrant> StevenK: I'd test rather than land, given what happened before, but yes.
<StevenK> Haha
<StevenK> Trust our test suite much?
<wgrant> Test suite, sure
<wgrant> Testrunner... nope
<wallyworld_> StevenK: did you even watch the game?
<StevenK> wallyworld_: Nope
<wallyworld_> heathn
<wallyworld_> wgrant: you sure i left that error dir key in there? the mp diff says it's gone
<StevenK> wallyworld_: Why would I want to watch thugby?
<wgrant> wallyworld_: You removed the setting, not the schema key.
<wallyworld_> why not? it's a great game
<wgrant> 154	-dbuser: sharing-jobs
<wgrant> 155	-
<wgrant> 156	# See [error_reports].
<wgrant> 157	error_dir: none
<wallyworld_> ah ok. tests still run. but will remove, thanks
<wgrant> wallyworld_: How're you intending to land this series?
<wgrant> wallyworld_: It seems like the early bits could be landed now.
<wgrant> Rather than doing it all in a 10000 line diff
<wallyworld_> i rather do the whole lot
<wgrant> That's a really good way to ensure that bugs slip through.
<wgrant> But we'll see :)
<wallyworld_> the functionality is all very narrowly focused
<wgrant> Well
<wgrant> There are things there like magically making the (API-visible?) sharing service writable when the moment the triggers are disabled.
<wallyworld_> that can happen after the landing and we have seen that everything is ok for a bit
<wgrant> I mean it's not a sensible thing to happen ever.
<wgrant> How do those things follow?
<wallyworld_> hmm?
<wgrant> I don't understand the logic around "triggers are disabled, so it's OK for everyone to write to grants whenever they want"
<wallyworld_> i don't think those are connected wither
<wgrant> https://code.launchpad.net/~wallyworld/launchpad/subscribe-grants-access-1000045/+merge/106278 line 182
<wallyworld_> triggers disabled = turn on ff to manually keep subscriptions in line with access
<wallyworld_> sorry, what's your point?
<wallyworld_> we are replacing trigger code with model code
<wgrant> The sharing service magically becomes writable when the triggers are disabled
<wgrant> Rather than when writing is enabled.
<wgrant> This sharing service is callable over the API
<wallyworld_> only indirectly
<wallyworld_> the sharing service needs to be able to write to do the job the triggers used to do
<wgrant> The diff's ~6000 lines, and has a number of changes that I don't really understand, and dozens of conflicts that I can see. It's going to be very difficult to land it as a whole in any sensible fashion.
<wallyworld_> each individual piece has tests which need to pass
<wallyworld_> it's not really that complicated - the end result is two jobs which clean up subscriptions for inaccessible bugs, and optional code to grant access when someone is subscribed
<wgrant> And turning that on implicitly turns on APGs, because it makes the sharing service writable.
<wgrant> And other side-effects that I haven't noticed yet.
<wallyworld_> so what's wrong with turning on APGs?
<StevenK> branch-use-information_type-redux => devel  [FAILED]   (up for 1:53:05) i-37557451
<StevenK> :-(
<wgrant> They're a great way to spy.
<wgrant> Because they're not shown on the bug page yet.
<StevenK> At least I know its marked as failed correctly ...
<wallyworld_> they are shown on the +sharing page
<wallyworld_> and it's not a side effect if the behaviour is intended
<wgrant> The behaviour is not intended./
<wallyworld_> to me it is
<wgrant> And +sharing is not accessible to many people.
<wallyworld_> we are not allowed to change the bug ui
<wallyworld_> that was agreed to afaiui
<wallyworld_> so we cannot show apg on the bug page
<wgrant> Anybody who agreed to not showing APGs on the bug UI was silly and shall be overridden
<wallyworld_> who made you god?
<wgrant> We *cannot not show them*
<wgrant> That would make bug access entirely opaque.
<wallyworld_> it was agreed to by the product team and curtis i think but i could be wrong
<wgrant> It may have been agreed that we wouldn't need to show AAGs, because they require a subscription.
<wgrant> That is perhaps a reasonable position to take, but there is some dissent.
<wgrant> Not showing APGs is entirely unreasonable.
<wallyworld_> do they have to be shown on the bug page? what about the pillar page?
<wgrant> They should probably be on some publicly visible page on the pillar somewhere.
<wgrant> But it's essential that the bug page not make it impossible to see who has access.
<wallyworld_> that's what i thought we were going to do
<adeuring> good morning
<lifeless> stub: hi
<lifeless> stub: catchup ?
<stub> lifeless: sure.
<lifeless> stub: skype ?
<lifeless> stub: why you no answer :P
<stub> sorry - downstairs stealing my mike back
<lifeless> ah :)
<stub> better?
<stub> adjusting
<lifeless> stub: epic issues?
<stub> I was hearing you fine
<stub> I can hear you
<stub> dropped out
<jelmer> moin
<danilos> frankban, hi, it seems my branch hasn't landed even though it passed all the tests; perhaps pqm acting weird or something?
<frankban> yes danilos :-( it should be fixed now, restarted the full land for your branch
<danilos> frankban, ack, thanks (though, you can also do a direct pqm-submit when you don't want to wait for the full cycle: not a hurry with this one :)
<frankban> yes I know danilos, but since it was a testrunner bug, i just wanted to try the whole process again
<danilos> frankban, ok, sure thing, sorry for being annoying then :)
<frankban> danilos: you are not annoying, I always appreciate suggestions on processes :-)
<wgrant> frankban: You can just search for tracebacks in the subunit output to confirm a good run
<wgrant> frankban: It still ran all the tests.
<frankban> wgrant: you are right, thanks
<wgrant> If it was a good run you should see the full 17000ish count in the ec2 mail.
<wgrant> Without even going into the subunit stream.
<wgrant> AFAIK it's only on error that the stream gets corrupted.
<czajkowski> jam: jelmer vila could someone have a look at https://bugs.launchpad.net/launchpad/+bug/1003576
<_mup_> Bug #1003576: Automatic translation exports not committing PO files <regression> <translations> <Launchpad itself:Triaged> < https://launchpad.net/bugs/1003576 >
<jam> looking
<czajkowski> jam: thanks
<czajkowski> dpm ^^^^
<jam> I'm guessing this is the "translations for Q are fairly stalled right now".
<wgrant> It's probably unrelated.
<wgrant> But jtv would know
<jtv> ?
<dpm> hi jam, this is about translations for projects not being committed, rather than translations for the distro
<jam> dpm, czajkowski, so at this point, I don't really know how to dig into this further without just agreeing that it is a critical issue (user impacting), and it should go into the queue of ThingsThatShouldBeFixed
<czajkowski> jtv: hola there seems to be a possible regression bug https://bugs.launchpad.net/launchpad/+bug/1003576
<_mup_> Bug #1003576: Automatic translation exports not committing PO files <regression> <translations> <Launchpad itself:Triaged> < https://launchpad.net/bugs/1003576 >
<jtv> czajkowski: I'm looking into it â my guess is it's probably just transient problems or old locks sticking around.
<wgrant> Ah
<czajkowski> jtv: cheers
<wgrant> You know what
<wgrant> I haven't had export cronspam for days
<wgrant> I wonder if it was never reenabled after the attempted initialisation earlier in the week
<wgrant> I bet they reenabled stuff on ackee/loganberry, but not crowberry
 * wgrant checks.
<wgrant> Yeah'
<wgrant> gnuoy: Hi
<gnuoy> wgrant, hi
<wgrant> gnuoy: translations-export-to-branch.py is still disabled on codehost@crowberry due to RT#52815. The other disabled jobs were reenabled a couple of days ago, but that one was apparently forgotten.
<_mup_> Bug #52815: logitech usb headset sound output dies after 1 minute <gst-plugins (Ubuntu):Invalid> < https://launchpad.net/bugs/52815 >
<wgrant> Could you uncomment it, please?
<jtv> czajkowski: mystery solved.  :)
<gnuoy> ok course
<gnuoy> s/ok/of/
<wgrant> I shall also prepare an appropriate script-monitor invocation
<wgrant> As it appears to lack one.
<gnuoy> wgrant, done
<czajkowski> jtv: magic :)
<wgrant> gnuoy: Thanks.
<czajkowski> jtv: cheers
<wgrant> No point running it manually now -- it may well take more than an hour and get killed.
<czajkowski> dpm: yer sorted :)
<wgrant> But we could do a manual run in an hour if dpm thinks it's urgent.
<jtv> wgrant: if this script is not run for too long, it may lose updates.
<wgrant> It will automatically run in a little under 20 hours
<wgrant> It has not run in 4 days
<jtv> Enough to lose a day's worth of data.
<jtv> If it hasn't run in 4 days, then it may have lost 2 or 3 days' worth of data.  Adding a day is not good.
<dpm> wgrant, I agree with jtv, if possible, it'd be good to have a one-off run asap in addition to the next run in 20 hours
<wgrant> Surely it uses a last update time, not hardcoding 24 hours which will never work?
<jtv> Not 24 hours, but way back when it did hard-code some fixed time period.
 * wgrant checks.
<wgrant>         """Get date of last translations commit to `branch`, if any."""
<wgrant> That looks encouraging
<wgrant> Yeah, should be fine
<wgrant> it uses the last translations commit - a fudge factior
<wgrant> Otherwise I would have had to append the exporter to my list of things that really aren't very good ideas :)
<wgrant> gnuoy: I propose https://pastebin.canonical.com/66679/
<gnuoy> wgrant, seconded
<gnuoy> wgrant, done
<wgrant> gnuoy: Marvellous, thanks.
<gnuoy> np
<wgrant> dpm: The export script somehow missed out on our usual monitoring; that has now been rectified.
<dpm> thanks a lot for your help wgrant, great to hear it's sorted now
<dpm> and thanks czajkowski
<dpm> and gnuoy :)
<wgrant> We'll do a manual run right after fastdowntime.
<dpm> when's fastdowntime?
<wgrant> 47 minutes from now
<czajkowski> 10am UTC every day
<dpm> cool
<wgrant> It normally takes about 25 minutes, I think, but it's got 4 days of stuff to do
<wgrant> "It" being translations-export-to-branch
<wgrant> fastdowntime will hopefully take about 67 seconds.
<dpm> jtv, did your changes to fix translator credits and other stuff in Translations you were telling me the other day land?
<dpm> hi could someone give some guidance to the guy willing to help fixing bug 869824? I think just a few comments in the bug report would already be helpful. Thanks!
<_mup_> Bug #869824: Doing a search in the ddtp-ubuntu project's translations templates times out <ddtp> <nightmonkey> <search> <timeout> <Launchpad itself:Triaged> <Ubuntu Translations:Triaged> < https://launchpad.net/bugs/869824 >
<jtv> dpm: I added a note.
<wgrant> I'm trying with a trigram index
<wgrant> Should help at least a bit.
<wgrant> Will see in a few minutes
<jtv> You're trying it right now?
<wgrant> On dogfood, but yes.
<wgrant> Indexes building
<jtv> Ah, that explains.  :)
<wgrant> Although the potranslation index is going to be maaaaassive
<dpm> thanks jtv, glad to hear that the upgrade might help. Did you see the question above about the recent translations changes branch you mentioned a few days ago? ^
<jtv> dpm: oh, sorry â phone.
<jtv> Missed that.
<jtv> Well the branch I was waiting for has finally rolled out.
<jtv> The follow-up branches are now going through testing prior to landing.
<nigelb> wgrant, StevenK - are you both purple mafia too?
<wgrant> Yes
<jtv> If only they could have asked Al Capone such a simple question.
<nigelb> heh
<nigelb> wgrant: Wasn't there some work to change the diff generation from cron to job queue? rabbit? Did that land?
<nigelb> (or was that not purple)
<dpm> jtv, cool. So once they've all landed, if I recall correctly, some of the things that we'll see are: translator-credits cleanup, translations done with message sharing enabled will be correctly committed, regardless of whether they were done in the distro or in the upstream project. Is that correct? Anything else?
<wgrant> nigelb: The backend job stuff is currently being tested on staging.
<wgrant> It's not purple.
<nigelb> Ah. Deryck's team?
<wgrant> Yes
<nigelb> Aha.
<czajkowski> Orange :)
<jtv> dpm: I'm not sure what you mean with committing of translations, but the *credits* will start fixing themselves.
<wgrant> dpm: It won't affect the comitting
<wgrant> dpm: That requires additional work
<jtv> The credits for Ubuntu and upstream will be more or less identical.
<dpm> ok, understood, thanks wgrant and jtv
<StevenK> nigelb: Purple *Assassins*
<nigelb> StevenK: Right. Sorry. Hey, I suggested the name. I think.
<bigjools> Purple Headed Monsters
<StevenK> nigelb: Maybe a variation of it.
<wgrant> jtv: Bah
<wgrant> jtv: My plan is foiled
<nigelb> bigjools++
<wgrant> # CREATE INDEX temp_pomsgid ON pomsgid USING gist (msgid gist_trgm_ops);
<wgrant> ERROR:  index row requires 8436 bytes, maximum size is 8191
<nigelb> Any of you other than rob headed to kiwipycon?
<jtv> wgrant: no toast?
<wgrant> jtv: Not for an index, I guess.
<jtv> :(
<wgrant> jtv: We probably have to use full FTI
<wgrant> Rather than an ILIKE
<nigelb> StevenK: I think I gave it a different colour. Someone else picked the colour.
<bigjools> we should become the Rampant Reds
<jtv> Oh please no
<nigelb> Rowdie Reds
<jtv> wgrant: annoyingly, what we have works just great for everything else.  We've been hoping for ddtp-ubuntu to be split up a bit more, butâ¦
<StevenK> nigelb: We were Teal, until wgrant, sinzui and I revolted.
<czajkowski> wgrant: there is downtime today at 20:00 UTC
<bigjools> I now have the Monty Python scene in my head ... "Welease Woger!"
<wgrant> czajkowski: There is
<wgrant> jtv: I'm surprised it works
<wgrant> jtv: Given the whole seq scan thing
<czajkowski> wgrant: there is
<nigelb> StevenK: Haha
<bigjools> StevenK: you were already revolting
<wgrant> jtv: I guess every other template has few enough messages that it can just check them all...
<StevenK> bigjools: You must have taught me well, then.
<jtv> wgrant: Exactly.
<nigelb> bigjools: Where in Australia are you settled?
<StevenK> nigelb: In wallyworld_'s lap.
<mgz> the great barrier reef
<nigelb> lololol
<nigelb> I didn't realize this question would involve *hilarious* answers :)
<wallyworld_> i otlerate him for now
<wgrant> jtv: Also, that query is terrifying, btw
<bigjools> wgrant: your children will be next
<bigjools> err
<bigjools> wallyworld_: : your children will be next
<StevenK> Hahaha
<wgrant> That could be a while
<nigelb> best. tabfail. ever.
<mgz> ehehe
<jtv> wgrant: I don't recall it off the top of my head, mercifully
<bigjools> I hate irc
<wallyworld_> bigjools: next for what?
<jtv> Ice cream!
<wgrant> jtv: It is 94 lines in length.
<bigjools> wallyworld_: you've heard the Manic Street Preachers?
 * wallyworld_ shakes his head
<bigjools> you probably have but don't remember
<bigjools> I'll play some next time you're round
<wallyworld_> ok
<bigjools> one of their tracks is called Australia :)
<jtv> Why wait?  Turn up the volume a bit more.
<bigjools> ha
<wallyworld_> so long as it's not more van halen
<bigjools> HEATHEN
<nigelb> bigjools: Anyway, so you're in Sydney? Melbourne?
<bigjools> Bris
<nigelb> Ah.
<wgrant> ie. nowhere
<bigjools> I went to Melbournce once
<bigjools> that was enough
<jtv> Grrr why can't I connect to anything?
<bigjools> and Melbourne too
<nigelb> I should be touching one of Sydney, Melbourne, or Brisbane on my way.
<danilos> lifeless, hi, is there something I need to do to get db-stable r11610 deployed?
<czajkowski> danilos: wrong timezone for him
<danilos> czajkowski, he's lifeless, there's no wrong timezone for him ;)
<cjwatson> danilos: Wait.  See https://wiki.canonical.com/InformationInfrastructure/OSA/LaunchpadProductionStatus
<nigelb> danilos: valid point there :)
<danilos> cjwatson, thanks, but I've out of the process for a while, do you know if DB r11611 will also include r11610? or should I get a separate request up there
<danilos> cjwatson, btw, are you rotating in LP, or just hacking on it (not so) randomly? :)
<cjwatson> danilos: It means "everything up to r11611" (the rest are just merges and such)
<danilos> cjwatson, cool, thanks, so it's not cherrypicking, great
<wgrant> We haven't cherrypicked in a long time.
<cjwatson> danilos: Not rotating as such, but probably the majority of my scheduled work for this cycle involves improvements that Ubuntu Engineering needs in Launchpad
<danilos> cjwatson, right, cool
<cjwatson> danilos: So same kind of deal as the Linaro work items work AIUI
<danilos> cjwatson, yeah
<danilos> cjwatson, btw, while we are on the topic, do you know if ubuntu is planning to do any migration of their old workitems? I'd like to get rid of the migration script to help our LOC cause :)
<cjwatson> danilos: I was under the general impression we weren't (and I'm not sure I see the point myself), but I'm not authoritative for that
<cjwatson> AFAIK we're using the new work items field for all quantal work and no longer care about pre-quantal work items
<cjwatson> I thought Kate was the contact for verifying that
<danilos> cjwatson, she was, I haven't seen her on IRC for a few days though
<cjwatson> she's on IRC right now ...
<czajkowski> cjwatson: thoguht it was going to be raised to stakeholders mrevell may know more about that
<cjwatson> well, logged in I mean
<cjwatson> czajkowski: isn't Kate our stakeholders contact, or one of them?
<czajkowski> cjwatson: aye she is but in case there was further discussion needed maybe danilos should mail and disucss it there?
<danilos> czajkowski, that's in process (mrevell is handling that), so I was just trying to see if I can get a speedier response :)
<czajkowski> danilos: not something I'm sure that can/will rushed  :)
<danilos> czajkowski, what, a simple confirmation of the previous agreement/decision? I think it can :P
<cjwatson> Seems like it should be a no-brainer from our side, TBH, but I don't know if there was any contention or if this is just being careful
* jcsackett changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: jcsackett | Firefighting: - | Critical bugs: 3.47*10^2
<jamestunnicliffe> Hi sinzui, thanks for the review (https://code.launchpad.net/~dooferlad/launchpad/upcomingwork-expand-all/+merge/107083). Should I put the JS in lp/app/javascript? It just feels a bit specific to 1 page to put in a general directory.
<sinzui> jamestunnicliffe,  lp/blueprints/javascript/workitems.js maybe.
<jamestunnicliffe> sinzui: OK, will do.
<jam> mgz, vila, jelmer, gmb: Just to confirm, June 25th - June 29th works for everyone, right? I'm starting to fill in all the forms about where and when
<jam> I wouldn't book tickets just yet, but I would block out your calendars
<mgz> should be, will double-double check now
<mgz> jam: might be best to do these kind of arrangements in #launchpad-kitchen on the internal irc server
<jam> I didn't feel that a sprint was particularly private... but we could
<jam> I would have arranged in it #bzr in the past
<vila> jam: should be good here too
<vila> meh, I meant the dates, not the channel, I agree -kitchen sounds like the best place ;)
<jam> though I would have to actually join #launchpad-kitchen to discuss anything there :)
<mgz> it's less privateness and more relevence :)
<mgz> those dates are indeed good for me.
<gmb> jam, Yes, 25th-29th is fine.
<jam> gmb: good to hear
<barry> lifeless: any chance you might be around?
<jelmer> jam: hi
<czajkowski> sinzui: jpds raised https://bugs.launchpad.net/launchpad/+bug/1003972
<mgz> jelmer: he's just gone out to mini-swimming, presumably will be back a little later
<jelmer> mgz: aha
<czajkowski> sinzui: not sure that's a regression
<czajkowski> or planned
<mgz> jelmer: free for me to mumble at you briefly?
<wgrant> czajkowski: dupe
<jelmer> mgz: ish, 
<jelmer> mgz: I'm on leave today, at a conference at the momwnt
<mgz> ah... hadn't gathered. can wait then. you back tomorrow or off then too?
<mgz> no, off both by the staffing email.
<mgz> will bother you next week then.
<jelmer> sure :)
<jpds> wgrant: Shouldn't you be in bed? ;)
<sinzui> czajkowski, as you can see I discovered the bug yesterday. It is a regression
<wgrant> Probably
<mgz> can't a man hack from bed?
<wgrant> Have to revert bad changes to the help wiki first
<wgrant> And now I should sleep, before I discover again that someone is wrong on the Internet.
<czajkowski> what bad changes on the help wiki
<czajkowski> sinzui: okie dokie
<sinzui> jcsackett, Do you have time to review https://code.launchpad.net/~sinzui/launchpad/licenses-modified/+merge/107229
<jcsackett> sinzui: certainly, i'll take a look in just a moment.
<sinzui> thank you
<jcsackett> sinzui: r=me.
<rick_h_> hah, love jcsackett's branch name "simplify-everything"
<rick_h_> was expecting a bit more though :P
<lifeless> barry: hi
<lifeless> danilos: hi, we don't cherry pick db patches, ever :) - we deploy one and only one; and the policyandprocess wiki page should cover it all
<barry> lifeless: hi.  wanted to talk to you about py3 support for lazr.restfulclient and lazr.authentication
<lifeless> danilos: I'm thinking to make the FDT process the second checkmarkable test process
<lifeless> barry: go on :P
<barry> lifeless: you're probably seeing the internal discussion over on #c but, other than py3 compatible oauth, i need to get rid of wsgi_intercept, which afaict is only used in the package's test cases.  i am not going to port w_i because several of *its* dependencies need to be ported.  for l.rc and l.a, i plan to just steal the few bits of w_i it needs for the test suite
<barry> lifeless: e.g. lazr/restfulclient/testing/intercept.py
<barry> lifeless: tho it sucks that i'll have to cargo cult that into l.a too
<lifeless> barry: why not do the thing right?
<lifeless> barry: it sounds like you're adding technical debt for us to pay the next cycle over
<lifeless> barry: its hard to get excited about that :(
<barry> lifeless: the problem is wsgi_intercept has *a lot* of stuff we don't use, plus several dependencies which themselves would have to get ported.  i want to avoid a multiweek detour right now
<barry> lifeless: i could always disable the test suites <wink>
<lifeless> barry: I understand it has that; but you did an rdepends on the build graph before planning to push for 3-only this cycle
<lifeless> so I don't understand how it can be surprising or concerning
<barry> lifeless: not so surprising.  the debian package doesn't build-dep on it because its tests don't run during package build :(
<lifeless> so from the debian package perspective, there is no issue
<barry> lifeless: well, i can't just do the upstream port and cross my fingers
<lifeless> right? All you need to do is make changes to the package which a) don't break python2 and b) let it work in python3 on the CD
<barry> lifeless: how will i know it works on py3?  i can test that it installs, but can't run the upstream test suite.  doesn't give me lots of confidence
<lifeless> barry: you could, but thats orthogonal; how can I help you ?
<barry> lifeless: i want to know whether you'd accept a branch that removes the w_i dependency, and pulls in just the small bits needed for the l.rc and l.a test suites
<lifeless> barry: It makes me very uncomfortable
<lifeless> barry: our track record with undoing such short-term actions is deplorable
<lifeless> barry: Unless there is a committed project that guarantees the time to come back and set it right, it would just be adding maintenance costs
<barry> lifeless: i can appreciate that.  so which is the bigger evil, given that porting w_i's stack is simply not feasible right now
<lifeless> barry: have you tried porting it ?
<lifeless> (what makes it not feasible)
<barry> lifeless: i have looked at it, yes.  to run *its* test suite, means porting mechanize, mechanoid, zope.testbrowser, and maybe one or two others
<barry> webunit
<barry> Paste
<barry> and that's not even chasing *their* dependencies
<cjwatson> Would it be possible to build a py3 wsgi_intercept that doesn't support intercepting so many web testing frameworks?
<cjwatson> I mean, mechanize, mechanoid, zope.testbrowser are needed because wsgi_intercept has explicit support for intercepting them; but we wouldn't need that for lazr.restfulclient etc. if they aren't relying on those frameworks themselves
<cjwatson> So it could be a reduced-capability port
<barry> cjwatson: yes, that might be possible.  just the parts that lazr.* care about are about 400 lines
<barry> that code is honestly not that interesting.
<barry> cjwatson, lifeless: lazr.httplib2_intercept? :)
<cjwatson> Well, what I meant wasn't to fork wsgi_intercept into lazr, but to keep it being wsgi_intercept, just less featureful - but irrelevantly so for lazr
<cjwatson> I don't think lazr.httplib2_intercept would meet lifeless' concerns, as I read them above :)
<barry> cjwatson: could be feasible if i don't bother the w_i test suite under py3
<lifeless> barry: hi, sorry
<lifeless> I blame iwlwifi
<lifeless> again
<lifeless> + NM getting horrifically confused
<lifeless> I have a call to prep for, so I can't chat more now
<lifeless> 06:24 < cjwatson> I don't think lazr.httplib2_intercept would meet lifeless' concerns, as I read them above :)
<lifeless> was the last I saw before network fail
<barry> lifeless: no worries.  i'll look at a partial port of w_i but won't bother with its test suite, which would be infeasible right right now
<lifeless> my key criteria are:
<lifeless>  - do not want increased maint burden for LP team
<lifeless>  - do not want trouble maintaining the python 2 versions in existing stable releases
<lifeless>    (such as they need it)
<lifeless> Beyond that, go to town.
<jcsackett> sinzui: re bug 1003482, which project did you notice it on?
<_mup_> Bug #1003482: privacy banner is not shown by default when reporting a bug <disclosure> <regression> <ui> <Launchpad itself:Triaged by jcsackett> < https://launchpad.net/bugs/1003482 >
<abentley> jcsackett: Could you please review https://code.launchpad.net/~abentley/launchpad/short-celrerybeat-transactions/+merge/107277 ?
<jcsackett> abentley: looking now.
<abentley> jcsackett: thanks.
<jcsackett> abentley: that's quite an LoC credit.
<abentley> jcsackett: Thanks!  It was mostly deleting the create-merge-proposal job.
<jcsackett> abentley: i can see where that would benefit your credit a lot, yes. :-P
<abentley> jcsackett: There was a certain sacrifice involved; I originally wrote create-merge-proposal-job.
<jcsackett> abentley: they say for good writing, you have to murder your darlings. i suppose coding is much the same. :-P
<jcsackett> abentley: also, r=me. looks fine.
<abentley> jcsackett: thanks.
<jcsackett> sinzui: can i get you to look at two MPs for me? https://code.launchpad.net/~jcsackett/launchpad/simplify-everything/+merge/107240 and https://code.launchpad.net/~jcsackett/launchpad/privacy-banner-with-better-info/+merge/107244
<sinzui> okay
<jcsackett> the first of those fixes the private-by-default banner issue too.
<jcsackett> apparently my simplifications fixed the regression.
<sinzui> \o/
<jcsackett> yeah, it's always nice when that happens. :-)
<cjwatson> Maybe I should write a variant of (or option for) loc-contributions that ranks all contributors by how much credit they have.
<cjwatson> If you want to make progress on something, turn it into a competition :-)
<sinzui> jcsackett, simplify-everything is r=me when you fix the open span tag
<jcsackett> sinzui: ah! thanks. my eyes kept slipping right over that.
<sinzui> jcsackett, I have a gedit plugin that auto completed markup. When I pretended to create a near the end, it suggested that I close a span
<jcsackett> hm. may need to ping some vim experts and see if i can get the same sort of thing setup in my editor.
<sinzui> jcsackett, r=me for the other branch.
<jcsackett> sinzui: thanks!
<sinzui> jcsackett, my plugin is python, the rules for choosing what can be completed might be adaptable to vim
<rick_h_> jcsackett: xmledit is a great autocompleting plugin for tags. won't check for valid though
<jcsackett> rick_h_: you can bolt html linting into vim though, can't you?
<rick_h_> I'm looking, I've never lint'd my html. I so rarely have issues with it. Usually it comes up in browser or page testing I guess
<rick_h_> the trouble is that in any templating, code can so make linting go boom!
<sinzui> jcsackett, read markup generator: http://bazaar.launchpad.net/~sinzui/gdp/trunk/view/head:/plugins/gdp/complete.py
<rick_h_> but I do use xmledit for auto closing opened tags as I go along which is nice
<jcsackett> yeah, with vim-python and some elbow grease that can probably be made into a vim plugin.
<jcsackett> rick_h_: that would be good. i'll take a look at it.
<sinzui> jcsackett, Edwin wrote instructions to integrate pocket-lint to vim
<jcsackett> rick_h_: though i may take a stab at sinzui's plugin. i've been wanting an excuse to play with the black magic of vim plugin creation.
<rick_h_> jcsackett: yea, there's a book  in github that's "learn vimscript the hard way"
<jcsackett> sinzui: hrm. i remember that; i even had that working at one point. i wonder why it stopped.
<jcsackett> or rather, when i removed it, given i have no plugin for it now.
<sinzui> jcsackett, https://dev.launchpad.net/UltimateVimPythonSetup
<rick_h_> jcsackett: http://perlbuzz.com/2012/03/new-htmllint-beta-validates-html-entities.html
 * jcsackett bookmarks both links
<barry> lifeless: how's this for awesomeness.  wsgi_intercept can't even work with the python3 version of httplib2
<lifeless> gary_poster: bzr switch -b myname
<lifeless> flacoste: do you have a couple of minutes; I have a couple of backlogged things to chat about ?
* jcsackett changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: - | Firefighting: - | Critical bugs: 3.47*10^2
 * StevenK stabs the branch security adapter
<StevenK> wgrant: I'm seeing something very strange. test_branchlookup.TestGetByLPPath.test_private_branch is creating a branch that is USERDATA, but if I print the information_type in IBranch.visibleByUser it's PUBLIC
<wgrant> StevenK: Check DB logs to see what's happening, perhaps.
<wgrant> It's often the easiest way
<wgrant> Or show me the test :)
<StevenK>         branch = self.factory.makeAnyBranch(private=True)
<StevenK>         path = removeSecurityProxy(branch).unique_name
<StevenK>         self.assertRaises(
<StevenK>             NoSuchBranch, self.branch_lookup.getByLPPath, path)
<wgrant> StevenK: Is it possibly calling it early?
<wgrant> And caching?
<wgrant> Before transitionToInformationType is called
#launchpad-dev 2012-05-25
<StevenK> wgrant: I'm not sure -- if I print the information_type in the second line of the test it's USERDATA. So it should be that for everything after, right ...
<wgrant> StevenK: Not if the factory caused the caching to occur
<mwhudson> wgrant: i think you told me a while ago that ajax changing a bugtask's target for a single-bugtask bug should cause a page refresh
<mwhudson> this appears to not be the case currently
<wgrant> Hmm
<wgrant> It was, but perhaps it broke.
* wallyworld_ changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: wallyworld | Firefighting: - | Critical bugs: 3.47*10^2
 * mwhudson grrs at trying to search for a bug containing the string "403"
<mwhudson> wgrant: i should file a bug then?
<wgrant> mwhudson: Probably.
<mwhudson>     /* ChoiceSource doesn't seem to function in IE at present, breaking
<mwhudson>      * all of this AJAX except the assignee and product pickers.
<mwhudson>      */
<mwhudson> bet that comment makes sinzui happy
<wgrant> Nah
<wgrant> I added that just a couple of months back
<wgrant> Because we were about to fix ChoiceSource, and it was entirely broken
<mwhudson> wgrant: it looks like there is code to do a refresh around bug task _deletion_
<wgrant> So it would have been a major regression to enable it there
<mwhudson> nothing around reassignment
<wgrant> sinzui: Can that be removed now?
<wgrant> mwhudson: There's meant to be a hook on the context URL change
<mwhudson> ah
<mwhudson> i'm only looking in bugtask_index.js
<mwhudson> would it be somewhere else?
<wgrant> It came for free when that was implemented in recipes.
<mwhudson> ah
<mwhudson>             Y.on('lp:context:web_link:changed', function(e) {
<mwhudson>                   window.location = e.new_value;
<mwhudson>             });
<wgrant> That's the one.
<wgrant> Perhaps it's not updating the cache somehow.
<mwhudson> let's see if it works for recipes
<wgrant> Indeed.
<mwhudson> my survey says no
<mwhudson> yeah
<mwhudson> editing the name isn't updating
<mwhudson> LP.cache.context.web_link
<wgrant> File a bug, I'll criticalify it
<mwhudson> i think it probably is
<mwhudson> because e.g. the name is not being updated in the cache either
<wgrant> :(
<mwhudson> wgrant: do you know where the "update the cache on lp_save" stuff is wired up?
<mwhudson> this function called update_cached_object looks promising
<mwhudson> uhhh
<mwhudson>       if (cached_object.self_link === full_uri) {
<wgrant> It matches the cache entry based on URI?
<wgrant> That's going to work well :)
<mwhudson> yeah
<mwhudson> from the "did this ever work!?" department...
<wgrant> It looks like it might have been broken for a year
<wgrant> But surely not
<mwhudson> do you have a suspect change?
<mwhudson>   [r=jcsackett][bug=490826, 721064,
<mwhudson>         724004] Keep LP.cache up to date with launchpadlib PATCH changes,
<mwhudson>         and raise update events.
<mwhudson> is what i found that touched that area
<mwhudson> but surely that's the basic functionaliy
<wgrant> 14634 is interesting
<mwhudson> ah yes
<mwhudson> it broke the notification because entry.lp_original_uri is now more correct?
<wgrant> I assume so
<wgrant> It's not longer updating the cache
<wgrant> Because it uses the post-redirect URI
<wgrant> I suspect
<mwhudson> rings true
<mwhudson> not sure how to fix it though...
<StevenK> wgrant: It seems visibleByUser is called before the print from the test --how is that possible?
<wgrant> StevenK: Security proxy in the test, more than likely
<StevenK> And then that's cached? I'm somewhat confused by this mess.
<wgrant> StevenK: You access a proxied attribute in the factory
<wgrant> It calls the security adapter
<wgrant> Which calls visibleByUser
<wgrant> The security cache caches the adapter's response
<StevenK> Bleh, is this failure caused by transitionToInformationType?
<StevenK> Hm, seems not.
<wgrant> StevenK: branch.owner
<wgrant> +            removeSecurityProxy(branch).transitionToInformationType(
<wgrant> +                information_type, branch.owner, verify_policy=False)
<wgrant> That branch.owner call will use the security proxy
<StevenK> wgrant: I've replaced it with registrant (which I should have done anyway), and it still fails
<wgrant> StevenK: Huh?
<wgrant> That doesn't change anything
<wgrant> It's still using the object's proxy
<wgrant> rSP
<StevenK> wgrant: registrant is a Person
<StevenK> Not a Branch
<StevenK> -                information_type, branch.owner, verify_policy=False)
<StevenK> +                information_type, registrant, verify_policy=False)
<wgrant> Oh, not branch.registrant
<wgrant> Possibly branchChanged is doing it
<StevenK> This case has no stacking
<StevenK> So no branchChanged
<wgrant> StevenK: Break in visibleByUser
<wgrant> So you can see the call stack
<wgrant> StevenK: s/branch.owner/registrant/ fixes it for me
<StevenK> Indeed
<StevenK> Perhaps I hadn't saved
<StevenK> wgrant: Looks like most of the failures were caused by branch.owner
<sinzui> wgrant, mwhudson. We want to remove the refresh rule. The We have not worked out how to refresh the state of other things on the page. There is currently a bug where if you retarget a task, the state of the page is broken in subsequent ajax calls
<mwhudson> sinzui: yes
<mwhudson> sinzui: however it should be removed by removing it, not having it accidentally break
<sinzui> I agree
<StevenK> sinzui: But I remember fixing that with Tim at the first squad sprint
<StevenK> Perhaps it was only branches
<sinzui> StevenK, I remember it fixed as well.
<sinzui> wgrant, StevenK, mwhudson, once we are certain the JS always works, we can remove the inline task edit form. Non-js can go to task/+edit
<StevenK> That sounds good
 * sinzui thinks StevenK would like to kill that code and tests
<StevenK> sinzui: Are we certain the JS always works?
<mwhudson> https://bugs.launchpad.net/launchpad/+bug/1004248/comments/2 fwiw
<_mup_> Bug #1004248: hook to force page refresh on context url changing is not working <javascript> <regression> <ui> <Launchpad itself:Triaged> < https://launchpad.net/bugs/1004248 >
<wgrant> sinzui: There's still no other way to retarget -- the AJAX only works between projects.
<sinzui> I am not surprised. That bug is probably the root cause of...
<sinzui> bug 919369
<_mup_> Bug #919369: acting on a bug task which has been retargeted oopses <404> <bugs> <javascript> <oops> <Launchpad itself:Triaged> < https://launchpad.net/bugs/919369 >
<mwhudson> well
<mwhudson> i couldn't tell if _that_ bug was complaining about what happens when someone else has retargeted the bug
<mwhudson> task
<sinzui> Excellent. I think the bug is a dupe, but this time we got better data
<wgrant> Not quite
<wgrant> The refresh will only happen if you change the current context task
<wgrant> Retargetting a non-current task will still hit bug #919369
<_mup_> Bug #919369: acting on a bug task which has been retargeted oopses <404> <bugs> <javascript> <oops> <Launchpad itself:Triaged> < https://launchpad.net/bugs/919369 >
<lifeless> sinzui: shouldn't you be asleep :P
<sinzui> no. I should be talking to stevenk or wallyworld_
<StevenK> sinzui: Oh? Pick your victim
<StevenK> wgrant: http://pastebin.ubuntu.com/1005775/
<sinzui> StevenK,  so I made mwhudson's bug the master since it is the root cause. When  we fix that bug,  we really can remove bug task inline edit forms
<StevenK> Sadly, the root cause isn't really disclosure.
<wgrant> StevenK: Looks reasonable.
<StevenK> Let me finally land this thing then
<StevenK> lifeless: If I testr load from a subunit stream, shouldn't testr failing --list show me the broken tests?
<sinzui> StevenK, shall we mumble
<lifeless> StevenK: absolutely
<StevenK> steven@undermined:~/launchpad/lp-branches/branch-use-information_type-redux% zgrep -c '^Traceback' ~/Desktop/branch-use-information_type-redux-r15297.subunit.gz
<StevenK> 14
<StevenK> lifeless: ^
<StevenK> steven@undermined:~/launchpad/lp-branches/branch-use-information_type-redux% testr failing --list | wc -l
<StevenK> 0
<lifeless> StevenK: they may have not failed
<lifeless> StevenK: passing tests can have attachments too
<lifeless> StevenK: oopses have Traceback in them, and don't trigger an implicit failure (perhaps they should)
<lifeless> StevenK: forward me the .gz ?
<StevenK> lifeless: Is in my homedir on carob.
<lifeless> whats your username on carob
<StevenK> ~stevenk
<sinzui> wgrant, do you have time to mumble?
<wgrant> sinzui: Sure
<bigjools> bloody hell, did someone flick the turbo switch on bug searches?
<wgrant> Yes
<wgrant> I rewrote them from scratch a few weeks ago
<StevenK> bigjools: \d BugTaskFlat
<wgrant> That reminds me
<wgrant> I need to coerce stub into GINing
<bigjools> nice work
<lifeless> it is pretty cool stuff
<wgrant> Not fantastic yet.
<wgrant> GIN should help in-context searches
<wgrant> But contextless (eg. person) searches still suck
<wgrant> Just a bit less than they did before.
<sinzui> wallyworld, are you available to mumble?
<wallyworld> sinzui: sure, let me get my microphone
<wgrant> StevenK: Do you have plans to test searches with information_type instead of transitively_private early next week?
<StevenK> wgrant: Branch searches?
<wgrant> That one
<StevenK> I hadn't considered that, TBH
<wgrant> It needs to be done pretty soon
<StevenK> wgrant: I can look at doing it after the UI work
<StevenK> wgrant: Or I can drop the UI work and look at it now if you think it's urgent enough
<wgrant> Hopefully the UI won't take more than a couple of days.
<wgrant> The backend stuff isn't critical to look at early, as performance is already terrible, but we should get it out of the way soon.
<StevenK> wgrant: Can you add a card to Next then?
<wgrant> done
<wallyworld> StevenK: wgrant: i already have a card for branch searches using info type, and a 1/2 done branch
<StevenK> Hah
<wallyworld> i was waiting on steve's info type work to land before being able to finish it
<wgrant> Oh, so you do.
<wgrant> That's cheating.
<wallyworld> we discussed this last week :-)
<wallyworld> or was it this week
<wgrant> Hm
<wgrant> It was probably on last Friday's call, which I slept through.
<wgrant> That would explain it.
<wallyworld> mayben :-)
<wgrant> Given the timing of the bug
<StevenK> What? Lies. You don't sleep.
 * wallyworld off to do the school run. traffic will be awesome in the rain, not :-(
<wgrant> Oh, it's horrid up there too?
<wgrant> It's been about 8pm all day here
<wallyworld> yep
<wgrant> Only minor flooding.
<StevenK> It's bright and sunny here today
<wgrant> Ah
<wgrant> The cloud pattern is interesting
<wgrant> http://www.bom.gov.au/australia/satellite/
<StevenK> Haha
<wgrant> It like just avoids the southern half of NSW's coast.
<bigjools> pissing it down here
<wgrant> It's actually almost stopped here
<wgrant> But been raining and dark all day until now
<StevenK> wgrant: http://www.bom.gov.au/products/IDR023.loop.shtml is impressive
<wgrant> Myeees.
<wgrant> Better than it was.
<wgrant> And clearing.
<bigjools> actually only 2.2mm of rain here, although it feels like more
<StevenK> I think we got ~ 8mm yesterday
<wgrant> 27.2mm here!
<StevenK> Since 9am?
<wgrant> Yeah
<StevenK> Bloody heck, that would cause a little bit of flooding
<wgrant> It's the most we've had in a while.
<czajkowski> morning
<jtv> Hi there czajkowski
<adeuring> good morning
* wallyworld changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: - | Firefighting: - | Critical bugs: 3.47*10^2
* adeuring changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: adeuring | Firefighting: - | Critical bugs: 3.47*10^2
<lifeless> wgrant: btw, have you seen slow SSO evidence recently?
<wgrant> lifeless: Not really, no
<wgrant> lifeless: Bit sad
<lifeless> wgrant: so it might be fixed?
<wgrant> lifeless: Ah, I think there are still some timeouts
<wgrant> Although it may be the pgbouncer thing
<lifeless> stuartm is following up
<lifeless> I'd like to let him know if we sitll think its happening
* bac changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: adeuring, bac | Firefighting: - | Critical bugs: 3.47*10^2
<bac> hi adeuring
<adeuring> hi bac
<bac> adeuring, nothing in the queue.  yay.
 * cjwatson shall try to rectify that :-)
<cjwatson> I'm nuking some doctests from orbit
<bac> cjwatson, and i'll gladly review that!
<jam> mgz
<mgz> hey
<rick_h_> rvba: did you guys have any sass/less discussion on maas? In the wiki it links to a sass test but thought I recall some less emails in -dev before
<rvba> Hey rick_h_.  Not really, I talked with Huw about using it a while ago but tbh we didn't push the idea further.
<rick_h_> rvba: ok, I noticed you guys didn't use either so curious if there was a talk in there/not.
<rvba> rick_h_: I guess the only reason not to use it right now is lack of time to set it up and refactor the existing css.  But IIRC Huw was very much in favor of using it.
<rick_h_> ok, cool. I'll bring it up and see about getting it in place from our start. Thanks
<cjwatson> bac: https://code.launchpad.net/~cjwatson/launchpad/refactor-customupload-tests/+merge/107381
<bac> cjwatson, looking
<james_w> how do I run a specific doctest file?
<rick_h_> which one james_w ?
<james_w> rick_h_, lib/lp/soyuz/stories/webservice/xx-archive.txt
<rick_h_> james_w: think you can do ./bin/test -x -cvvt "xx-archive"
<james_w> thanks rick_h_
<rick_h_> np
<james_w> it runs a few more, but it's better than running them all :-)
<rick_h_> yea, you can adjust that, it's a regex to narrow it down
<rick_h_> and stuff like ./bin/test -x -m lp.bugs.model.tests -cvvt "TestBugTaskCreationPackageComponent"
<rick_h_> but that's not for doctests
<rick_h_> I'm still working on my tricks for speeding up test runs
<cjwatson> james_w: you can also pass the full relative path from the top of the branch to test -t, since in the doctest case the regex is matched against the full path
<james_w> and if it's a regex I guess I can use "$" to prevent substring matches from running other doctests in the same dir?
<james_w> or I guess the .txt in the path will do that
<james_w> if I want to reject certain values for a property based on values for other values on the object, do I just raise exceptions in the setter?
<james_w> or maybe I should just let the two things vary independently in this case
<james_w> we have "suppress_subscription_notifications" on a PPA, and subscription notifications are only generated if the PPA is private
<james_w> so it doesn't make sense to have it set to true when the PPA isn't private
<james_w> but it's far more code to enforce that than to just let people set a value which has no effect
<james_w> what do people think the approach should be?
<james_w> and that's not even getting in to having a good UI for setting these things on the web, with dynamically enabled checkboxes and the like
#launchpad-dev 2013-05-20
<wgrant> StevenK: That's going to issue several queries for each filter
<StevenK> wgrant: I'm not sure I can do it in one query given the number of tables involved
<StevenK> wgrant: But suggestions welcome
<StevenK> wgrant: DB patch with ON DELETE CASCADE?
<wgrant> StevenK: I'd add a batch filter deletion method
<StevenK> wgrant: So you're okay with multiple queries, say 4, but not 4 * number of filters ?
<wgrant> StevenK: Right, a constant number of queries is necessary
<wgrant> You can't sensibly do less than one per table.
<wgrant> But one per table per filter is excessive and banned.
<StevenK> wgrant: http://pastebin.ubuntu.com/5682621/
<wgrant> StevenK: I'd replace the list comp and helper method with a for loop
<wgrant> And you can hopefully kill off BSF.delete() now
<wgrant> Or at least reimplement it in terms of the deleteMultiple
<wgrant> If you can't remove it entirely
<StevenK> wgrant: http://pastebin.ubuntu.com/5682645/
<wgrant> StevenK: Right
<StevenK> wgrant: The MP has updated
<wgrant> StevenK: What about the structsub deletion behaviour in BSF.delete()? That was probably a misfeature, but it may need to be replaced.
<StevenK> wgrant: No tests fail with it removed
<StevenK> And I agree about it being a misfeature
<StevenK> Well, bin/test -vvt structuralsubscription doesn't show any failures
<wgrant> StevenK: No tests fail, but does the UI work?
<wgrant> The code appears to implement an invariant that there's always a filter for every structsub
<wgrant> It implements it racily, but it's still likely that something depends on it
<StevenK> wgrant: I can create a structsub
<StevenK> And change the filter
<StevenK> And delete both filters and create a new one
<StevenK> (Using Person:+structural-subscriptions)
<wgrant> StevenK: Yeah, but we'd have the odd behaviour that you could have a sub without a filter, which probably does nothing
<StevenK> wgrant: The UI actually copes with that.
<StevenK> Launchpad
<StevenK> Bug mail is not filtered; mail for Foo Bar about Launchpad will always be sent.
<StevenK> Create a new filter
<StevenK> wgrant: http://people.canonical.com/~stevenk/structsub.jpg
<wgrant> StevenK: But does the UI lie?
<wgrant> that's probably from the migration; I'm not sure if the backend will actually cope
<wgrant> AIUI it just looks for matching BSFs
<StevenK> wgrant: The Launchpad structsub was created by me
<StevenK> The other two are from sampledata
<wgrant> StevenK: How did you create it?
<StevenK> Product:+subscribe
<wgrant> And you didn't manually delete the BSF?
<StevenK> I did, but via Person:+structural-subscriptions
<wgrant> Right, so have you tested how the backend handles it?
<wgrant> I suspect that that UI code is just a leftover from the migration when not everything had a BSF yet
<StevenK> wgrant: Person:+structural-subscriptions was written for structsubs only, +subscriptions is the legacy stuff
<wgrant> +structural-subscriptions is legacy
<wgrant> +subscriptions is not legacy
<wgrant> +s-s predates BSF
<wgrant> So it is not a valid argument for anything
<wgrant> (structsubs date back to '07 or so; it's just BugSubscriptionFilter and co that are new)
<StevenK> wgrant: If I resurrect _has_other_filters, to restore the old behaviour, I can't move delete to using deleteMultiple, since deleteMultiple makes a choice at the end if it removes itself, or its parent structsub
<wgrant> StevenK: We first need to establish what the model invariants must be
<wgrant> Then we can work out implementation of deletion
<StevenK> wgrant: http://pastebin.ubuntu.com/5682717/
<wgrant> StevenK: Possibly, but I'd prefer to discover what the model constraints actually are
<StevenK> wgrant: I've been trying to work out which code determines if a notification will be sent based on filtering to see if it will gracefully cope with no filters, but I'm having trouble finding it
<wgrant> StevenK: Let me see
<wgrant> I hadn't noticed before, but it's actually more careful than any other LP code to be non-racy
<wgrant> So it seems to be pretty serious about maintaining that invariant.
<wgrant>     source = IStore(StructuralSubscription).using(
<wgrant>         StructuralSubscription,
<wgrant>         Join(BugSubscriptionFilter,
<wgrant>              BugSubscriptionFilter.structural_subscription_id ==
<wgrant>              StructuralSubscription.id),
<wgrant> QED
<StevenK> Ah, then there is UI bug too
<StevenK> The filter page allows you to delete the last filter
<StevenK> Maybe that's okay since it will destroy the structsub too
<wgrant> StevenK: Right, that's fine, because that's handled at the model level
<wgrant> That's the whole point of the code I'm questioning the removal of
<StevenK> wgrant: Then my resurrection that I pasted should be fine
<wgrant> StevenK: Sort of.
<wgrant> StevenK: It's an extremely confusing interface
<wgrant> And very likely to go wrong
<wgrant> Oh, also it doesn't actually do anything
<StevenK> wgrant: Oh?
<StevenK> Well, delete doesn't have to call into deleteMultiple
<wgrant> StevenK: deleteMultiple deletes all BSFs, then if delete_self then it deletes all BSFs.
<StevenK> wgrant: Not exactly. The filters aren't deleted, the internals are
<wgrant> StevenK: Oh, I see, I misunderstood
<wgrant> That's still O(len(filters)) queries
<wgrant> => fail
<StevenK> wgrant: How?
<StevenK> We loop over the kinds, not the filters
<wgrant> BSF.delete
<wgrant> Calls deleteMultiple for each BSF
<wgrant> Then checks for each BSF that there are other BSFs
<wgrant> Then if not it deletes the structsub
<StevenK> wgrant: Right. But deleting the structsub calls deleteMultiple directly
<wgrant> StevenK: This seems extremely convoluted. I'd probably have a method which deletes BSFs, then deletes any corresponding SSs if they have 0 BSFs.
<wgrant> Simple
<wgrant> Foolproof
<wgrant> Fast
<wgrant> Short
<wgrant> So the same as BSF.delete is today, except operating over more than one BSF
<StevenK> wgrant: Right, and then delete calls that. Do I need to call _has_other_filters for locking?
<wgrant> You'll want to do the same sort of locking, yes
<wgrant> And Storm doesn't have support for SELECT FOR UPDATE, so you'll probably have to use SQL or define a Select derivative.
<StevenK> wgrant: Bah, now I'm having trouble crafting a query to find the structsubs with no filters
<wgrant> StevenK: SELECT * FROM structuralsubscription WHERE NOT EXISTS (SELECT 1 FROM bugsubscriptionfilter WHERE bugsubscriptionfilter.structuralsubscription = structuralsubscription.id) FOR UPDATE;
<wgrant> With an added ID constraint to ask for just the structsubs that you've changed
<wgrant> No, not quite
<wgrant> Won't take enough of a lock
<wgrant> Invert the NOT EXISTS
<StevenK> I'm currently locking BSF only, currently
<wgrant> Then do set difference in Python
<wgrant> That's useless
<wgrant> BSF isn't the table that we want to lock
<wgrant> Also, this SELECT FOR UPDATE stuff will probably fail in pg9.3
<wgrant> Ah, no, it will work
<wgrant> Because of the backward compat changes that they made late.
<StevenK> wgrant: http://pastebin.ubuntu.com/5682802/ I'm on the right track?
<wgrant> StevenK: That sort of thing
<wgrant> However
<wgrant> I've just realised that the original code isn't as safe as it thinks it is.
<StevenK> wgrant: If can change that SELECT FOR UPDATE to return id rather than *?
<wgrant> StevenK: That won't affect the locking.
<wgrant> It still won't do what the author expected.
<StevenK> wgrant: Even the query you pasted above?
<wgrant> StevenK: Even that query.
<wgrant> Do you understand what it tries to do?
<StevenK> It tries to lock all structsub rows given a bunch of filter ids
<wgrant> Yes, but to what end?
<StevenK> So they don't change under us?
<wgrant> Not quite
<StevenK> wgrant: What then?
<wgrant> StevenK: It is trying to block multiple concurrent BSF.delete()s for the same SS
<wgrant> Not to prevent an FK violation
<wgrant> But to prevent two concurrent deletions from deleting the last two BSFs for an SS, with neither transaction noticing that it's the last.
<wgrant> Except that, because the SS row isn't actually being updated, the transactions can run concurrently as long as one commits before the other does the SELECT FOR UPDATE.
<wgrant> So it fails to prevent the race
<StevenK> wgrant: So what can we do instead?
<wgrant> StevenK: I think the only completely safe method is to do an actual UPDATE, even if it changes nothing
<wgrant> Or we could update the UI to say that something with no filters does nothing
<wgrant> And then get garbo to prune them daily
<wgrant> I don't think there's another way to obtain a lock before the snapshot is frozen.
<wgrant> Yeah, that shouldn't be possible, because you'd need a snapshot to find the tuples that you need to lock
<wgrant> So you can't obtain the lock before snapshotting
<wgrant> Oh
<wgrant> Or we just select the BSFs for SHARE
<wgrant> And of course I totally misread the code because it uses UPDATE when it actually wants SHARE.
<wgrant> So it is safe.
<wgrant> StevenK: SELECT id FROM structuralsubscription WHERE id IN (4) AND EXISTS (SELECT 1 FROM bugsubscriptionfilter WHERE bugsubscriptionfilter.structuralsubscription = structuralsubscription.id FOR SHARE);
<wgrant> The FOR SHARE in the subquery will prevent a concurrent transaction from updating or deleting the BSFs that we use as evidence for keeping the SS
<StevenK> wgrant: Ah, but you're missing one thing -- we have the BSF ids, not the SS ids.
<wgrant> StevenK: It is not a huge issue to obtain the SS IDs
<StevenK> wgrant: http://pastebin.ubuntu.com/5685381/ Branch from yesterday, against -r submit: for readability
<wgrant> StevenK: I'd grab the IDs from the FOR SHARE query, and use them to exclude SSs from deletion, rather than reselecting them potentially differently. Also, you shouldn't be using sqlvalues any more.
<wgrant> Otherwise I think that's good
<wgrant> I've got to run out for a few hours now; will be back lunchish.
#launchpad-dev 2013-05-21
<wgrant> StevenK: I return
<lifeless> and therefor you are?
<StevenK> wgrant: Just as I'm about to go to lunch, nice work. :-P
<StevenK> wgrant: I thought ? and params=(list,) were allergic?
<wgrant> StevenK: Ah, indeed. Though you can easily Stormify the outer bit of that and just use SQL for the inner bit, or even add FOR UPDATE/SHARE support to Select in about 4 lines of code.
<StevenK> Or I could ignore it and make you twitch
<StevenK> wgrant: http://pastebin.ubuntu.com/5685858/ btw
<wgrant> StevenK: Does that work?
<wgrant> Oh, I guess it does.
<wgrant> Because all the SSs that we touch will have BSFs at the start.
<StevenK> Exactly. And possibly none of them will at the end.
<wgrant> StevenK: However, does that query at the start perform adequately?
<wgrant> It would be more reliable to filter on SS ID in the outer query
<wgrant> Rather than relying on the planner to guess that the inner query results in extremely good selectivity.
<StevenK> Requires a JOIN in the outer query, ids == BSF not SS
<wgrant> Or obtain the IDs first, since you need them anyway
<StevenK> I had that, you said I should get them from the FOR SHARE query ... :-)
<StevenK> Bleh, do I need execute_zcml_for_scripts to do IStore(BranchMergeProposal) ?
<wgrant> You need the component architecture, yes.
 * StevenK stabs Zope
<wgrant> StevenK: http://pastebin.ubuntu.com/5685908/
<StevenK> Haha
<StevenK> I see you took the easy way of using FOR SHARE too
<StevenK> wgrant: Your filter_expr is too late
<StevenK> We've already deleted some BSFs by then
<wgrant> StevenK: Why is that a problem?
<wgrant> It's a NOT EXISTS at that point
<StevenK> Ah
<wgrant> It only locks the first row that it finds, since a NOT EXISTS obviously fails as soon as it finds a row, but that's fine for our needs.
<StevenK> wgrant: My MP script of doom is almost done too
<StevenK> Then we'll see PreviewDiffPruner likes a large dataset
<wgrant> :)
<StevenK> 2 PDs for MP 863
<StevenK> 4 PDs for MP 861
<StevenK> Come on!
<StevenK> 290
<StevenK> Oh, I think it's commint
<StevenK> *commiting
<wgrant> StevenK: Going OK?
<StevenK> I think it's still commiting
<wgrant> That's very unlikely
<wgrant> Committing is cheap
<wgrant> SELECT * FROM pg_stat_activity;
<StevenK> <IDLE> in transaction
<wgrant> What's the query start time?
<StevenK> 2013-05-21 04:09:56.441673+00 | 2013-05-21 04:09:56.448991+00 | 2013-05-21 05:21:26.137095+00
<wgrant> That's session | transaction | query
<wgrant> ?
<StevenK> backend_start         |          xact_start           |          query_start
<wgrant> So yes
<wgrant> You're not running $script with LP_DEBUG_SQL?
<StevenK> Nope
<wgrant> If this is local, enable statement logging in postgres
<StevenK> DF
<wgrant> If it's on DF, god help you.
<wgrant> Ah
<StevenK> It's adding an average of 3 PDs for every MP
<StevenK> (But only one actual diff, so the rest are just rows in PD)
<wgrant> I just caught it doing an INSERT
<wgrant> MP 128897
<wgrant> It's very slow
<wgrant> Where's the code?
<StevenK> codebase-current/foo, it's incredible naive
<StevenK> incredibly
<StevenK> Awwww
<StevenK> It just threw MemoryError
<wgrant> Yeah
<wgrant> It used all RAM
<wgrant> I saw it at 75%
<wgrant> Commit in batches
<StevenK> I would if you'd close it
<wgrant> Done
<StevenK> wgrant: Look sensible?
<wgrant> StevenK: I'd do more like 2000, but doesn't really matter, so go ahead :)
<wgrant> StevenK: It might still fail due to StupidCache, but it's worth a try
<StevenK> wgrant: I'd bump it to 2000, but you like leaving vim running
<wgrant> StevenK: E!
<StevenK> Hah
<StevenK> 3 PDs for MP 160882
<StevenK> 4 PDs for MP 160881
<StevenK> 5 PDs for MP 160880
<StevenK> Commit
<StevenK> Ah, look at that
<wgrant> Hm?
<StevenK> It finished
<StevenK> And I think we have OMGLOTS of previewdiffs to kill
<StevenK> SELECT count(*) FROM previewdiff WHERE EXTRACT(year FROM date_created) =  2001; => 570852
<wgrant> :)
<StevenK> 2013-05-21 06:12:14 DEBUG2  [PreviewDiffPruner] Iteration 14 (size 10000.0): 1.226 seconds
<StevenK> 2013-05-21 06:13:06 DEBUG2  [PreviewDiffPruner] Iteration 64 (size 10000.0): 0.002 seconds
<StevenK> 2013-05-21 06:13:06 DEBUG2  [PreviewDiffPruner] Done. 584116 items in 65 iterations, 68.688814 seconds, average size 8986.402205 (8503.80299128/s)
<StevenK> wgrant: So, I will put up a branch tomorrow with 44-4 that adds the two indices on pd and id.
<wgrant> StevenK: Sounds good
<StevenK> And PDP is already hellishly fast, so that's handy
<StevenK> It took 30 minutes to add them and 60 seconds to delete them
<wgrant> The deletion is optimised SQL
<wgrant> The insertion was awful Python
<StevenK> s/awful/naive/ :-)
<wgrant> No, the script wasn't bad, but the LP infrastructure is
<StevenK> wgrant: News at 11 :-P
<StevenK> Haha
<StevenK> DiffPruner punted the lone diff I added too
<wgrant> :)
<StevenK> 2013-05-21 06:14:10 DEBUG2  [DiffPruner] Iteration 0 (size 1.0): 132.183 seconds
<StevenK> I'm guessing DF wasn't so happy
<StevenK> Oh, it might have been transaction blocked
<StevenK> Given that time is extremly close to the entire runtime
<wgrant> More likely that it includes the time to calculate the condemned IDs
<wgrant> StevenK: Are these in -daily?
<StevenK> They both are, yes
<wgrant> Good
<StevenK> I wasn't sure how heavyweight they would be, and we don't buy anything with them in hourly
<StevenK> wgrant: If you're happy to accept r=sinzui on that MP, the changes you suggested have been pushed for a while.
<wgrant> I was hoping to relayer lp.bugs.m.bsf and .ss, but they're too intertangled, so I guess that will have to do
<wgrant> StevenK: r=me
<wgrant> StevenK: For QA, I'd recomment running a harness with LP_DEBUG_SQL and trying a deletion
<wgrant> The queries are designed to be very fast, but we should check that I didn't miss something
<czajkowski> jtv: you alive....
<jtv> czajkowski: barely... this particular jet lag always gets me in a weird way, where for a while I think I'm off the hook and then it just comes back.
<czajkowski> jtv: tomorrow could you look at https://answers.launchpad.net/launchpad/+question/229157 please....
<jtv> OK
<czajkowski> thanks
<jtv> StevenK: any chance you can get at any oopses generated from this error? https://answers.launchpad.net/launchpad/+question/229157
<czajkowski> wgrant: ^
<wgrant> Ah
<wgrant> I have tracebacks about that
<jtv> I thought it'd be the plurals, but I see nothing wrong with them.
<jtv> Nor does the empty Last-Translator cause it.
<wgrant> No, it's something to do with the query logging, IIRC
<wgrant> Yeah
<wgrant> The statement issued by _getPOTMsgSetBy causes the statement logger to choke when trying to expand parameters
<wgrant> It first started happening yesterday
<wgrant> Only on empathy and webtrees
<wgrant> Haven't had a chance to investigate yet
<jtv> wgrant: sorry for the silence â pre-imp call.  Do we have a bug for that problem?
<wgrant> jtv: Not yet
<jtv> wgrant: it sounds as if there isn't much I can do for this question...  would it be OK if I handed it off to you?
<wgrant> jtv: I'd love to say no, but sadly I cannot.
<wgrant> :)
<jtv> "To the first part or the second part?"  :)
<jtv> If there's a way I can usefully take a portion of this problem off your hands, I'd be happy to.
 * jtv has to relocate
<wgrant> jtv: You can't take it off my hands, so I can't say it's not OK to hand it off to me, as much as I'd love to have it not be my problem :P
<jtv> wgrant: then we both feel entirely the same.  That is one bright spot in the situation.  :)
<wgrant> Heh
 * jtv runs
<Laney> hmm
<czajkowski> Laney: what have you broken...
#launchpad-dev 2013-05-22
<StevenK> wgrant: https://code.launchpad.net/~stevenk/launchpad/stop-lying-please/+merge/165000
<wgrant> StevenK: I hope that breaks some tests
<StevenK> I doubt it, based on my grepping
<lifeless> wgrant: / StevenK: hey, I would be very interested in knowning a perf comparison between ec2 and hp's test perf for lp parallel single-node test runs
<wgrant> lifeless: At what sort of node sizes?
<lifeless> wgrant: well, the ones you do ec2 test runs on .. xlarge isn't it?
<wgrant> lifeless: ec2 isn't parallel today
<lifeless> wgrant: oh!
<lifeless> wgrant: why not :)
<wgrant> lifeless: Because if we care about timeliness then we just throw it straight at buildbot
<wgrant> ec2 test still uses a single test process on an m2.xlarge
<lifeless> I would be curious if that ran any better on the same size in HPCloud, but less interested than in the actually-uses-the-cpus-case :)
<wgrant> StevenK: I've shuffled gina around to be a bit more sensibly testable (eg. you now don't have to run scripts/gina.py to test anything at all...), so https://code.launchpad.net/~wgrant/launchpad/gina-skippage/+merge/165048 is now pretty trivial.
<StevenK> wgrant: Your indentation for the test looks a little suspect
<wgrant> StevenK: Howso?
<StevenK> wgrant: You open a listcomp and don't indent another level
<wgrant> StevenK: Ah, I indented one by a space, and one not
<wgrant> Must have miscopied
<wgrant> Or do you object to the second too?
<StevenK> I'd prefer 4, but I can deal with one
<wgrant> It's unambiguous and I think more readable than 4
<wgrant> Though it is questionable
<StevenK> wgrant: So fix the first one and land it, or refactor it to a function :-P
<wgrant> StevenK: Oh yeah, that's done
<StevenK> wgrant: It's been approved for a while
<wgrant> StevenK: Thanks
#launchpad-dev 2013-05-23
<StevenK> wgrant: Maybe my addition in model/archive would be better in the guts of canModifySuite rather than verifyUpload
<wgrant> StevenK: Probably not
<StevenK> Oh?
<wgrant> StevenK: That's meant to prevent deletions as well
<wgrant> I think
<StevenK> It can stay in verifyUpload
<StevenK> wgrant: Hmm, I think the deployment report broke
<wgrant> Quite likely, given what happened
<StevenK> ERROR:qa-tagger:Something went wrong when marking bugs: Unterminated string starting at: line 1 column 3068 (char 3068)
<wgrant> Oh
<wgrant> That's just a broken launchpadlib cache
<wgrant> nuke it
<StevenK> There's like 9000 files
<wgrant> Yes
<wgrant> Nuke it
<wgrant> The launchpadlib cache is almost entirely pointless
<StevenK> wgrant: I don't think this BPJ method is tested at all
<wgrant> StevenK: Quite likely
<wgrant> plsfix
<StevenK> Hmm
<StevenK> I can't just make the job, I need to queue it as well
<wgrant> StevenK: .queueBuild()
<wgrant> On the BPB
<StevenK> And that tells me it's already queued?
<StevenK> WTF?
<wgrant> What suggests it's not queued?
<StevenK> getUtility(IBinaryPackageBuildSet).getByQueueEntry(job)
<StevenK> That returns None
<StevenK> And if I call build.queueBuild() I get an IntegrityError
#launchpad-dev 2013-05-24
<wgrant> StevenK: getByQueueEntry takes a BQ
<wgrant> Not a BPJ or J
<wgrant> I think
<wgrant> Or it takes a BPJ
<wgrant> Something like taht
<StevenK> wgrant: That is from the method I'm testing
<wgrant> StevenK: What's your test?
<StevenK> wgrant: http://pastebin.ubuntu.com/5695481/
<wgrant> StevenK: makeJob is not meant to be used directly
<wgrant> It's mostly a private method to be used by queueBuild.
<wgrant> StevenK: https://code.launchpad.net/~wgrant/launchpad/clean-up-publication-methods/+merge/165512
 * StevenK waits for the branch scanner to let the source branch go, so the page will actually render.
<wgrant> StevenK: That's impossible
<wgrant> The page does not write
<wgrant> So it can't contend scanner locks
<StevenK> LaunchpadTimeoutError: Statement: 'SELECT BranchRevision.branch, BranchRevision.revision, BranchRevision.sequence ...
<wgrant> That's just a cold cache
<StevenK> 212# First up, publish everything in this build into that dar.
<StevenK> wgrant: While you're in the area, could you un-2005 that comment?
<wgrant> Done
<StevenK> wgrant: What did you replace it with?
<wgrant> # Publish all of the build's binaries.
<wgrant> Since the comment was a lie anyway; it may publish to all DASes if it's not arch-specific
<StevenK> Heh
<StevenK> wgrant: r=me
<wgrant> StevenK: Thanks
<wgrant> It'll probably break some more tests, but let's see what happens...
<StevenK> wgrant: I need to file a bug for denying uploads to OBSOLETE series?
<wgrant> StevenK: I would
<StevenK> wgrant: 902836 is close
<wgrant> StevenK: Not close -- exact.
<StevenK> Then I'll make use of it
<wgrant> Please do
<StevenK> wgrant: https://code.launchpad.net/~stevenk/launchpad/db-add-archive-permit-obsolete/+merge/165514 and https://code.launchpad.net/~stevenk/launchpad/deny-obsolete-series-uploads/+merge/165515
<StevenK> Damn it, wrong branch for the db patch :-(
<wgrant> StevenK: You'll also want a small garbo job
<wgrant> You can just steal the one I used two weeks ago
<wgrant> StevenK: Are you going to retarget that MP?
<StevenK> wgrant: Small garbo job why?
<StevenK> wgrant: https://code.launchpad.net/~stevenk/launchpad/db-add-archive-permit-obsolete/+merge/165517 is the new one
<wgrant> StevenK: To set the column's values to false
<StevenK> wgrant: ...
<StevenK> 11	+ALTER TABLE archive ALTER COLUMN permit_obsolete_series_uploads
<StevenK> 12	+ SET DEFAULT false;
<StevenK> Oh
<wgrant> Yes
<wgrant> That sets the default.
<StevenK> That won't rewrite the table
<StevenK> wgrant: You lose
<wgrant> StevenK: I knew I'd lose
<wgrant> Since my ec2 run only had mBPPH setting the first one to be published
<wgrant> I lost a lot less than I expected, though
<wgrant> Particularly since two of the three tests that failed were ones I added two weeks ago...
<Thetawaves> hows it going fellas
<StevenK> wgrant: Do we want another DB patch to ALTER COLUMN permit_obsolete_series_uploads SET NOT NULL ?
<wgrant> StevenK: I have an unlanded one to do the same for publish_debug_symbols. We would do well to combine them
<wgrant> So don't bother; I'll add it to mine.
<StevenK> wgrant: Difficult
<StevenK> Since permit_obsolete_series_uploads was added in 46-0
<wgrant> StevenK: That's why I'll renumber mine to 46-1 :)
<StevenK> Right
<StevenK> wgrant: I now have four MPs up
<theneb> Hi all, is there a version of launchpad that can be used on private servers?
<jelmer> theneb: the launchpad itself can be run on a private server
<theneb> Run launchpad on my private server for private projects
<samba_jelmer> theneb: yes, that's possible - the source is available and AFAIK there are a couple of private instances out there
<theneb> samba_jelmer: Thanks, except for the development source guide on the wiki, are there any other articles around it?
<samba_jelmer> theneb: not that I'm aware of
<theneb> Okay, I'll start getting stuck into configuring up the Apache side.
<samba_jelmer> it's not exactly trivial to setup from what I remember
#launchpad-dev 2014-05-19
<cjwatson> cprov: Should bug 1319124 be qa-ok again (since it was previously, but you set that before qastaging caught up) so that we can deploy?
<_mup_> Bug #1319124: UnicodeDecodeError mailing an inline code review comment <code-review> <fallout> <oops> <qa-needstesting> <Launchpad itself:Fix Committed by wgrant> <https://launchpad.net/bugs/1319124>
<wgrant> It's not currently possible to QA due to the incomplete staging move, so we might just have to skip it.
<cjwatson> wgrant: cprov QAed it some other way before it hit qastaging, I think, maybe on DF
<wgrant> DF doesn't have codehosting
<cjwatson> Hm, don't know then
<cprov> cjwatson: I was cheating the system in search for sense on the QA page. Haven't really QAed it.
<cprov> wgrant: uhm, you say our only chance to deploy today is to skip QA ? What about staging ?
<wgrant> cprov: staging might work, I forget how broken it all is.
<wgrant> I just restarted all the firewall rules pretty much from scratch, but they won't land until at least tomorrow.
<cprov> wgrant: can we wait or shall we deploy it, and since the audience is limited, do QA in production (if it makes any sense) ?
<wgrant> cprov: How about we deploy up to the rev before that, and I'll try to get the firewalls properly fixed in the morning?
<cprov> wgrant: 17009 to be qaed in prod. It works for me.
<cprov> if there is something broken, we will be blocked again on qastaging
<cprov> but that's fine.
<wgrant> Yeah
<wgrant> I've vaguely tested it locally, at leas.t
<wgrant> And it seems pretty safe.
<wgrant> qa-untestable and deploy away, I guess.
<wgrant> cprov: Can you arrange that?
<cprov> sure, will update the RT
<wgrant__> cprov, cjohnston: i have no Internet now, apparently, so i guess I'll see if we can talk in my morning.
<cjohnston> wgrant__: sounds good..
<cprov> wgrant__: np
<wgrant__> Could one of you possibly have a look at the remaining CSS issues in my latest branch? I had other things to do today
<cjohnston> ack
<wgrant__> Apart from being generally bad, i think the main issue is the width problem when the inline editor is open
<wgrant__> If not, I'll investigate tomorrow
<wgrant__> Otherwise, if you could just take some of the remaining inline comments tasks and try to sort them out
<wgrant__> cprov: did you request a deployment?
<cprov> wgrant__: updated the ticket, https://rt.admin.canonical.com/Ticket/Display.html?id=70321
<cprov> wgrant__: let me point a vanguard on it
<wgrant__> Thanks
<wgrant__> I'll prefer not to deploy tomorrow, as we're upgrading to postgres 9.3 at 06:00, and i want to be able to isolate any fallout from that
<wgrant__> I'm heading off now, Hangout or SMS me if something comes up.
<wgrant> cprov, cjohnston: I apparently have Internet now. Can we have a call at some point in the next few hours? If not, normal time tomorrow?
<cjohnston> I'm good whenever
<cprov> wgrant: now(ish) would be a great time, after dinner I have a babysitting shift :-/
<wgrant> cjohnston: I'm in a meeting atm
<wgrant> But in half an hour, I guess?
<cjohnston> works for me.. cprov ^
<cprov> I can try, will grab a quick dinner meanwhile
<cprov> any minute counts here ;-)
 * cjohnston did his first MP review today ;-) heh
<cjohnston> wgrant: need some wordsmithing...
<cjohnston> The link below the diff.. what should the text read
<wgrant> cprov, cjohnston: I'm in the normal hangout, when you're ready
<cjohnston> wgrant: https://code.launchpad.net/~cjohnston/launchpad/publish-ic-link/+merge/220103
#launchpad-dev 2014-05-20
<cjohnston> wgrant: can you look at what I have and explain to me what your wanting things to look like please?
<cjohnston> with all your changes, I'm not sure that I know
<wgrant> cjohnston: My branch doesn't impact publish-ic-link at all
<wgrant> I only changes stuff inside the diff table.
<cjohnston> correct...
<cjohnston> I'm talking about your branch
<wgrant> Oops, missed the fact that the MP you linked was more than two hours ago, not relevant to this convo.
<wgrant> Looking
<wgrant> cjohnston: What do you have?
<wgrant> I don't see a branch.
<cjohnston> wgrant: I have it up (plus one small change) at 162.213.35.40
<cjohnston> https://code.launchpad.dev/~name16/myproject/team/+merge/1
<wgrant> cjohnston: The main issue I see is the width.
<wgrant> cprov's comment blows things up, and opening a draft comment (at least in Firefox) has ridiculous padding on the right.
<cprov> what ? who ? me ?
<wgrant> Oh
<cjohnston> yes, you
<wgrant> cprov's comment is actually fine, until the draft super-padding appears.
<cjohnston> wgrant: so, the IC's, should they be 100% of the table.diff width?
<wgrant> cjohnston: I deliberately added some horizontal margin to make the comments not just look like another line of the diff.
<wgrant> My intent (feel free to do something less terrible) was to have them always fit inside the normal diff width
<wgrant> That all works well until you invoke a draft editor
<wgrant> The editor isn't meant to change the width at all.
<cjohnston> ok.. so you like the width as it is..
<cjohnston> (except the edit)
<wgrant> It's marginally the least bad so far, IMO.
<wgrant> But I'm very much open to suggestions.
<wgrant> And it's easy to change now we're not tied to the table-based layout.
<cjohnston> personally, I liked it the way it was.. but as we aren't designers, I don't care to play for hours on this
<wgrant> It was really hard to see the comments in the old one.
<cjohnston> I want a new icon for adding a comment
<cjohnston> I don't like that one
<cjohnston> (not your fault, just my opinion)
<wgrant> Oh yeah, that's by no means even vaguely final, I just needed one to throw in the CSS for experimentation.
<wgrant> And I remembered the name of that one :)
<cjohnston> lol
<cprov> wgrant: can you check qastaging mbox ? there should be a MP email with non-ascii diff content.
<wgrant> cprov: There are errors, rather
<wgrant> See #launchpad-ops; DB perms not there yet.
<cprov> putz
<wgrant> cprov: Fixed now, can you try again?
<wgrant> I just ran an ASCII one through, and the email worked fine.
<wgrant> brb
<cprov> done
<wgrant> > +Now let's mess with encoding Ã¡Ã©Ã­Ã³ÃºÃ§Ã£
<wgrant> >
<wgrant> zoing!
<wgrant> >
<wgrant> cprov: Looks good
<cprov> wgrant: great!
<cjohnston> wgrant: any chance you can look at adding a draft comment on my instance?
<wgrant> cjohnston: !
<wgrant> It looks like not totally terrible
<wgrant> Well done
<cjohnston> I just s/edit/add for the button, if you refresh you should be able to see that
<cjohnston> I think that's slightly better than edit
<wgrant> Indeed
<cjohnston> what other changes are needed for this branch?
<wgrant> cjohnston: We should monospace and fix the word breaking in the comments if possible.
<wgrant> cjohnston: word-wrap: break-word; works if the div is width-constrained. Somehow convincing it to not extend outside the table would work.
<daker> yo
<cjohnston> hey daker
<daker> wgrant: hi
<daker> technically you can't set the width to px since launchpad is using a fluid layout (% & auto so no fixed width)
<daker> i those cases, i think we can use "calc"
<daker> in*
<daker> calc(100% - Xpx)
<daker> width : calc(100% - Xpx);
<wgrant> daker, cjohnston: Or what about setting table-layout: fixed on table.diff, and then a fixed px or % width on table.diff .line-no?
<wgrant> That'll prevent the comment divs from causing the table to expand.
<daker> cjohnston: can you test that ?
<wgrant> Throw in a font-family: monospace and a word-wrap: break-word on the comment bodies, and it starts to look at bit more sane.
<wgrant> The only real problem with that approach is that the line-no column is no longer dynamically sized, but that's a bit weird anyway.
<cjohnston> wgrant: how's that look?
<wgrant> cjohnston: Not bad! The line-no needs to be able to cope with up to four digits plus space for the icon, but that's a trivial tweak.
<cjohnston> do you know how big that would need to be?
<cjohnston> and what happens when people give you 10000 + line MPs? ;-)
<wgrant> I think we trim it after like 10000 lines
<cjohnston> wgrant: would the problems from last night effect branch scanning at all?
<wgrant> cjohnston: At the moment? No.
<cjohnston> just LP being slow
<wgrant> cjohnston: !! it finally scanned
<cjohnston> yay!
<cjohnston> wgrant: just pushed a new rev.. now the big question... will it scan. lol
<wgrant> The button text doesn't show up in other places. Can you find out why?
<wgrant> We shouldn't be hiding it with an outrageous text-indent :)
<tasdomas> lbox propose gives me this error: error: Failed to update merge proposal log: Server returned 401 and body: (<BranchMergeProposal at 0x2af0e8f83850>, 'description', 'launchpad.Edit') - no idea, what's it about, though..
<wgrant> tasdomas: You don't have permission to edit that merge proposal.
<tasdomas> wgrant, you mean the merge proposal on launchpad?
<wgrant> tasdomas: Correct.
<tasdomas> wgrant, how can that be possible? (or, in other words, what do I do to fix this?)
<daker> wgrant: and their is a nasty bug in caused by the help modal
<wgrant> tasdomas: Should you have permission to edit that merge proposal? Which is it?
<tasdomas> wgrant, https://code.launchpad.net/~tasdomas/juju-core/ensure-availability-more-output/+merge/220306
<wgrant>         return (user.inTeam(self.obj.registrant) or
<wgrant>                 user.inTeam(self.obj.source_branch.owner) or
<wgrant>                 self.forwardCheckAuthenticated(
<wgrant>                     user, self.obj.target_branch) or
<wgrant>                 user.inTeam(self.obj.target_branch.reviewer))
<wgrant> Are you sure your client was authenticated as the correct user?
<tasdomas> wgrant, sorry to have bothered you - it was my own mistake
<tasdomas> but your replies pointed me in the right direction
<wgrant> tasdomas: What was the issue?
<wgrant> Wrong MP URL, or wrong creds?
<tasdomas> wrong creds
<wgrant> Ah, great :)
<wgrant> cjohnston, cprov: I'll be back in a few hours, please try to poke holes in inline comments.
<wgrant> Night
<cjwatson> Is there some documentation somewhere of how to upgrade local development Launchpad instances to PostgreSQL 9.3, so that we can stay in sync with production?
<stub> cjwatson: The production docs would walk you through it, but I suspect the pg_upgradecluster provided by the Debian package would work a treat.
<stub> cjwatson: https://pastebin.canonical.com/110495/ is the current draft for the production upgrades
<cjwatson> Thanks.  Will have a try tomorrow, maybe ...
<cjohnston> cprov: r17011 qa-ok
<wgrant> cjwatson: You can also just pg_dropcluster 9.1 main, install launchpad-database-dependencies-9.3, purge postgresql-9.1, and then rerun utilities/launchpad-database-setup and make schema
#launchpad-dev 2014-05-21
<wgrant> cprov, cjohnston: people.canonical.com/~wgrant/launchpad/almost-finalâ½.png
<cprov> wgrant: hey hey
<cprov> it is coming up very nicely and since the new layout is more flexible (div) it will be easier to increment.
<wgrant> I hope so.
<cprov> wgrant: I've just opened an asana task about finding a way to materialise the IC threads, specially in emails
<wgrant> I've also dropped the "draft" and "publish" terminology. It nows says "Unsaved comment", and then "Include X inline comments" next to the "Save Comment" button.
<wgrant> Yeah, I've been wondering about how to do that.
<wgrant> It's a bit awkward, as they're all at different depths potentially.
<cprov> wgrant: it's annoying to to read the reviews right now
<cjohnston> looks good wgrant
<wgrant> Someone want to review? https://code.launchpad.net/~wgrant/launchpad/ic-js-cleanup/+merge/219531
<cprov> wgrant: I will
<cprov> wgrant: can you review https://code.launchpad.net/~cprov/launchpad/delete-branches-with-ics ?
<wgrant> Hm
<wgrant> What do you guys think about renaming "inline" to "diff"?
<wgrant> That pretty much just affects the "Include X inline comments" checkbox
<wgrant> Makes it clearer what it refers to.
<cprov> wgrant: I wonder if I can just add few other 'ouch' in your MP and let you figure them out ;-)
<wgrant> Heh
<cjohnston> +1, more ouch
<cprov> wgrant: +1 on "diff comments"
<cjohnston> if you do that you should probably do the same in the email
<cjohnston> what about 'View inline comments'
<wgrant> Oh yeah, email and link too, forgot those.
<cjohnston> I would argue that 'regular' comments are diff comments too..
<cjohnston> IC is actually inline
<cprov> wgrant: table-layout: fixed, not only predictable but also faster/easier for browsers. Nice one!
<cprov> had to look it up ...
<wgrant> It's pretty useful here, since divs expand to fill their container, and tds expand to fit their contents.
<cprov> wgrant: what about the TextWidget for review_type ? what does it bring ?
<wgrant> cprov: I narrowed it from 20 characters to 15, to ensure that the widgets all fit on one line.
<cprov> ah, cool
<wgrant> cprov: The rename is pushed.
<cprov> wgrant: what does the CSS added in the BMP template do ?
<wgrant> cprov: That's a variation on cjohnston's. It adds a bit of space before the publish checkbox.
<wgrant> Also, I've just pushed a regression fix for something that seems to have broken a few days ago: the publish checkbox was being added unconditionally, so inline comments crashed and failed to load for anonymous users.
<cprov> I think I had fixed that in 17011, on top of cjohnston fixes ...
<wgrant> Oh, so you did
<wgrant> I'll uncommit and merge
<wgrant> cprov: Have you seen any more issues on (qa)staging since those firewall fixes a few hours ago?
<cprov> wgrant: wait, there were 2 problems, you have to manually merge the conflicting hunks. I've only fixed the checkbox being rendered multiple times (in tests, essentially) and you have fixed the not-logged-in issue.
<wgrant> Also, you added a tab!
<wgrant> Die
<cprov> I must, I don't know why the emacs in my vm is allowing this shit ... I will review my config
<cprov> jslint should have exploded in my face when I try to commit, that's how we should setup project for dummies (like me)
<wgrant> Heh
<wgrant> cprov: The line-no col is wide enough to contain five digits plus the add sprite.
<cprov> up to 99999 lines diffs then, not at problem
<wgrant> Well, if you go back to your old days... :P
<cprov> wgrant: so, you really want new methods on CRICSet ...
<wgrant> cprov: All the other manipulation code (apart from person merge) is in codereviewinlinecomment.py
<wgrant> I really don't want it to get like various other LP model classes, where there are ten different modules that alter them.
<wgrant> Makes it very difficult to change anything.
<cprov> right, you think I should implement also the removal on CRICSet methods, or should I return ResultSet to be removed on the callsite ?
<wgrant> I think you should just implement a remove method. We don't expose any ResultSets from CRICS at the moment, and I can't really see any valid reason for someone to want one.
<cprov> wgrant: agreed, also the removal() should really consider both CRIC and CRICD as it is implemented in BMP, right ? they are likely to be removed together every time.
<wgrant> cprov: Right, that might well be a good thing to do.
<wgrant> I can't see much of a reason to split them.
<cprov> wgrant: code moved and pushed
<wgrant> cprov: I'd rename the method to removeFromDiffs or similar, but that looks great otherwise. Thanks.
<cprov> wgrant: no problemo, renamed version pushed
<wgrant> cprov: How much longer are you likely to be around? I think I might implement a quick fix to only include commented files (not just hunks, at least not yet) in emails.
<cprov> longer enough.
<cprov> wgrant: http://lpbuildbot.canonical.com/builders/lucid_lp_lxc/builds/1172/steps/shell_9/logs/summary is that old spurious failure, right ?
<wgrant> cprov: Yup, already forced.
<cprov> wgrant: thanks
<cprov> wgrant: how is the ic-email improvement ?
<cjwatson> wgrant: Any luck with the livefs reviews?  I know there's been the whole pg9.3 thing, and lots of work on IC ...
<cjwatson> I've put the remaining things people are asking for on the shelf for now, since I reached the point where I could do a touch image build with an added PPA
<wgrant> cjwatson: Sorry, busy with staging and postgres and inline comments and PPA microservice and ephemeral PPAs and Malta... Three of those are coming to a close tomorrow, so I might actually have some time.
<cjwatson> OK, thanks
<wgrant> cjwatson: Is Laney's cron.germinate change sane? I saw some debate over what the name should be.
<cjwatson> Oh, that bit of the naming is fine.  I was meaning to review and land that for him today.
<cjwatson> It will cause tasks to appear for sdk-libs or something like that, but they were actually wondering why they weren't there anyway
<cjwatson> And it will cause cron.germinate to take that bit longer, of course, but I've been looking at parallelising it ...
<cjwatson> Laney: Could you please set a commit message for https://code.launchpad.net/~laney/launchpad/germinate-ubuntu-touch/+merge/220233 ?
<Laney> cjwatson: Oh right, yes
<Laney> try that
<cjwatson> Laney: Thanks, landing
<Laney> cheers
<cjohnston> wgrant: have the new IC changes been released?
<cjohnston> I guess maybe it's DC changes now? ;-)
<wgrant> cjohnston: In progress.
<cjohnston> cool
<cjohnston> How'd the DB stuff go?
<wgrant> Not quite uneventful, but eventually successful.
<cjohnston> cool
<wgrant> The master is now 9.3, and the slaves are rebuilding on 9.3.
<cjohnston> oh nice.. so the upgrade was done..
<wgrant> Yep
<wgrant> I poked the comments stuff pretty thoroughly this evening, and fixed a few bugs
<wgrant> Including a hilarious off-by-one error in the email formatter, where comments were showing up a line later, and the first line of the diff was always missing.
<jelmer> wgrant: is launchpad really going to do inline comments on code reviews?
<wgrant> jelmer: On Monday, with a bit of luck.
<jelmer> wgrant: good stuff
<cjohnston> by "with a bit of luck" he means pencil and paper
<cjohnston> wgrant: take a look at https://code.launchpad.net/~ursinha/uci-engine/ts-add-mp-support/+merge/220275  I think we may need to rethink the design of my "comment" a little... cjohnston a moment ago: (and then nothing)
<cjohnston> the way it is it looks really odd without any sort of review or comments or anything
<wgrant> cjohnston: Yeah, I think we probably want to move the "Show diff comments" link into the body somewhere. We really need space on the bottom of each comment to have links
<wgrant> Show diff comments, Reply, and Save/Cancel in the case of the inline editor.
<wgrant> There's no way to have actions today.
<wgrant> Also the Delete button, and possibly a future Edit feature
<cjohnston> edit!
<wgrant> It's all really awful atm, no consistency. And the inconsistent bits are dreadful anyway.
<wgrant> cjohnston: prod's all up to date now, if you want to prepare that email.
<cjohnston> ack. thanks
<cjohnston> wgrant: cprov, what do you think about moving the 'Add comment' stuff to below the diff...
<cjohnston> I think it would improve workflow
<cjohnston> If you are replying to a comment you may not need to see the diff, but otherwise, your probably going to be adding a comment after reading the diff
<cjohnston> so instead of having to click a link to go back up, why not just have the box below
<wgrant> cjohnston: The problem is that the diff is enormous.
<wgrant> This is why other sites have it on a separate tab.
<cjohnston> it apparently isn't completely discoverable That you have to click save comment
<wgrant> Yeah, that's one of the big remaining issues.
<wgrant> But moving the comment form below the diff would make it very difficult to find.
<wgrant> Even for MP-wide comments.
<cjohnston> personally i would expect it to be at the bottom and not Where it is
<cjohnston> it seems odd to have to scroll back up after scrolling to the bottom
<wgrant> Sure, but then you have to scroll through 5000 lines of diff.
<wgrant> Just to find the comment form
<wgrant> + it separates the comment form from the comment thread
<wgrant> Which is weird
<cjohnston> or scroll 10k lines to read the diff and then go back up
<cjohnston> what do you think about changing the new bottom link to be either 'return to add comment' or 'return to save comments' depending on if drafts are present or not
<cjohnston> that may help discoverability slightly
#launchpad-dev 2014-05-22
<cjohnston> wgrant ^
<cjwatson> wgrant,stub: Hm, it would be quite helpful if the necessary packages for PostgreSQL 9.3 were in ppa:launchpad/ppa.
<cjwatson> (I can grab the binaries from CAT, I guess, but that's not the point really.)
<wgrant> cjwatson: Oh, we normally copy them in from pitti's PPA. But that seems to be deprecated.
<timrc> wgrant, Is there a way for you to unsubscribe me from all bugs on the somerville project?  I'm apparently directly subscribed to a bunch of bugs and have no interest in being spammed (and it's probably a violation of company policy for me to see these sorts of things anyway)
<stub> cjwatson: I'd be jumping to trusty myself...
<wgrant> timrc-afk: Hum, that sounds Joey's script isn't being very thorough.
<stub> cjwatson: pitti's PPA has been replaced by apt.postgresql.org
<stub> wgrant: ^
<cjwatson> I don't really want to have my main test setup running a full LTS newer than production, in case I introduce dependencies by accident.
<timrc> elmo, wgrant Thanks
#launchpad-dev 2014-05-23
<jcw4> is karma down? and is this the place to ask about it?
#launchpad-dev 2015-05-18
<blr_> wgrant: using zope interfaces, is it possible to express that a concerete class should implement one _or_ another method but not both?
<wgrant> blr_: No.
<wgrant> blr_: What are you trying to do?
<blr_> wgrant: oh, not trying to do that, was discussing zope interfaces with someone and that came up.
<blr> wgrant: have added a new doctest for product +setbranch, however I'm matching elements against browser.contents
<blr> assuming it would be better to have a helper?
<wgrant> blr: There are a few helpers around already. None did what you need?
<blr> well, I want it to match a specific element for vcs
<blr> e.g. I currently have ">>> '<dl id="product-vcs">' in browser.contents"
<wgrant> blr: A number of tests use BeautifulSoup for that.
<blr> I'll dig around and look at more of the doctests.
<blr> hmm, lots of gpg-coc stories blowing up, would be surprised if they're regressions
<blr> wgrant: does bin/retest work reliably for you? it seems to hang with no output.
<wgrant> blr: You need to give it the list of failures from bin/test in one of two ways: a file specified at the commandline, or the raw text given to stdin.
<wgrant> If you don't specify a filename it will await input.
<blr> hah ok, not as magical as I had hoped :)
<wgrant> If you want properly magical, consider testrepository.
<cjwatson> blr: Does it have to be a doctest, notwithstanding the existing stuff that does that?  You can often do this pretty well in unit tests.
<cjwatson> wgrant: webhooks proxy> good idea.  I'd be inclined to just use squid as then we don't have to write a new charm at all.
<cjwatson> I'm sure we can control requests' proxy settings accurately enough.
<wgrant> Yep, as long as we're careful that should be fine.
<cjwatson> I'll get an RT in for an initial sanity check and a stagingstack tenant.
<wgrant> Need to check the PS4 egress firewalls thoroughly, but it should be fine.
<wgrant> block 10.0.0.0/8 and all our IP ranges except archive and blah blah blah
<cjwatson> Yeah
<wgrant> (except when people want to webhook internal hosts. but then we can open those up.)
<wgrant> I'll raise it on the IS call in 9 hours.
<cjwatson> Thanks.
<cjwatson> wgrant: https://rt.admin.canonical.com/Ticket/Display.html?id=81258
<cjwatson> (I forgot to specifically mention the needed access from ackee, but we can sort that out as we go.)
<wgrant> cjwatson: The instances probably want floating IPs if only so they have sane reverse DNS, but otherwise that looks good. Thanks.
<cjwatson> wgrant: Indeed, just didn't mention that because that can't be done until we have a spec deployed.
<cjwatson> bigjools: William made the change you asked for to default new private archives to 20GiB rather than 2GiB; that's live now.
<blr> cjwatson: I added doctests to mirror the existing ones for series setbranch. Do we want to consider them generally deprecated at his point?
<wgrant> blr: Yes, doctests are quite deprecated.
<blr> I'm not that fond of them tbh
<wgrant> It's okay to add to them when that's substantially lower friction than adding a unit test, but it is discouraged.
<blr> I'll get the unit tests in place this morning, and perhaps you can judge if we should just drop the new doctests entirely when you review the branch
<blr> wgrant: oh, regarding push instructions, was thinking the nicest thing to do perhaps is display them based on the toggle state of default vcs on project setbranch
<blr> wgrant, cjwatson: one other wee thing, in the case where we infer vcs default from BranchCollection, git will be set as the default in the case where there are both bzr and git branches - is that acceptable? I figured that was the right approach given we want to generally consider bzr deprecated.
<wgrant> That sounds reasonable to me.
<wgrant> blr: Which push instructions?
<cjwatson> Agreed, also we can mostly assume that people will be migrating from bzr to git rather than the other way round.
<blr> push intructions at the top of +setbranch
<blr> currently only for bzr
<cjwatson> Similar for Product:+branches and probably various other places
<wgrant> Product:+branches isn't really affected; it'll show whatever the current view is.
<cjwatson> True, +setbranch has a more awkward choice to make
<blr> I just thought that might be nice, to toggle between git and bzr intructions on the right hand side based on the vcs default selected
<blr> possibly it isn't even useful there, the code index view is really where you expect to see it.
<wgrant> +setbranch should have it, as that's probably the place you go when you create a new project.
<bigjools> cjwatson: wgrant: thanks a million, that's awesome
<wgrant> bigjools: np
<blr> wgrant: added a view test for default vcs to TestProductView - is there an equivalent set of tests for setbranch somewhere, or only doctests?
<wgrant> blr: I can't see any non-doctests.
<blr> ok, thanks for confirming, that's what I thought.
<blr> wgrant: `bin/test -cvvt xx-` should be catching all doctests yes?
<wgrant> blr: That should catch most of the browser doctests.
<wgrant> But that's not usually an interesting pattern.
<wgrant> -t registry.*product or something might be more interesting.
<wgrant> Or -t code.*product
<blr> wgrant: I found some regressions in translations, which I probably would not have noticed. I should probably just run the entire suite.
<wgrant> blr: That's buildbot's job.
<wgrant> Running the entire test suite takes more than three hours on my overclocked Haswell desktop.
<wgrant> But you could grep for tests that reference setbranch, for example.
<blr> okay, thanks
#launchpad-dev 2015-05-19
<blr> wgrant: think that's ready for review - sorry for the cjwatson sized diff.
<wgrant> blr: Do you want to perhaps split the reindenting into another branch?
<wgrant> Just create a fresh branch of devel, fix the indentation, and land that so the diff in this branch is reviewable.
<blr> sorry, reindenting?
<wgrant> productseries-setbranch.js at least
<wgrant> Half the file is correctly reindented from three to four spaces.
<blr> ah, sure.
<blr> I'm going to blame emacs.
<wgrant> Heh
<blr> might as well lint it while I'm at it, this file would make douglas crockford sad.
<wgrant> It certainly would.
<blr> wgrant: ~blr/launchpad/trivial-fix-productseries-js-indentation
<wgrant> blr: Too late :)
<blr> refreshing is hard.
<blr> wgrant: shall I resub that mp with a dependant branch, or wait for the merge.
<wgrant> blr: Land the reindentation, then merge it into your main branch, resolve the conflicts, commit and push.
<wgrant> (there isn't really a "wait for the merge" step, since buildbot runs *after* the merge)
<wgrant> cjwatson: Ah, hm. Git MPs live under their target ref, but their target ref may no longer exist.
<wgrant> eg. the launchpad:test MP on https://code.launchpad.net/launchpad/+activereviews
<cjwatson> wgrant: Mm, it might make sense to move those directly under GitRepository, indeed.
<cjwatson> And it would make the URLs a little less obnoxious ...
<cjwatson> wgrant: Will be much easier after git-repository-delete lands, since that incidentally needed to add some MP-related properties to GitRepository.
<wgrant> cjwatson: Ah good.
<wgrant> cjwatson: I'd not really be concerned about leaving redirects.
<cjwatson> wgrant: Perhaps not.
<blr> wgrant: thank you for the thorough review!
<blr> wgrant: cjwatson: this looks potentially useful https://github.com/DesertBus/sockjs-twisted/
<blr> sockjs seems to have a sensible approach to degrading transports - haven't used it in earnest however.
<cjwatson> entirely new to me :)
<blr> cjwatson: anecdotally better maintained and more robust than socket.io
<wgrant> blr: Do my issues/suggestions on the MP make sense?
<cjwatson> https://code.launchpad.net/~cjwatson/canonical-is-puppet/webhooks-proxy-tweaks/+merge/259558
<cjwatson> And now to bed.
<wgrant> cjwatson: Thanks, and night.
#launchpad-dev 2015-05-20
<blr> wgrant: ah didn't notice your question, sorry - yes, I think so. Not certain how to best manage the YUI css - you mentioned yuitab.css is in the tarball?
<wgrant> blr: find -name tabview.css in the tree finds a few unpacked copies in the YUI3 build directory.
<blr> wgrant: right, that's where it was copied from (without modification iirc), the build seemed to want it in components, however I haven't looked to closely at how the css concatenation process works in the build.
<wgrant> blr: Right, it'd need some (hopefully minor) build system changes, but might be worth looking at.
<wgrant> We currently concat lots of files and then minify. We'd just need to change one of those file paths, I suspect.
<blr> yep, will have a look
<blr> wgrant: ended up tweaking the tabview css, so we won't be able to use the bundled yui style. (once I corrected the sprite path, the focused tab is rendered with a dark blue sprite which was a bit gross)
<blr> there's also a gradient on the tabs now which I'm not 100% on. Have a look in your browser when you look at the branch again tomorrow.
<blr> hmm maybe it is okay
<cjwatson> Date:   Sat May 16 14:38:28 2015 +0000
<cjwatson>     Merge xmlrpc-py3-7795-6: Port twisted.web.xmlrpc to Python 3
<cjwatson> ooh
<cjwatson> so that's in 15.2.0, released yesterday
<wgrant> cjwatson: Ooh.
<blr> cjwatson: wgrant: that's good news, having turnip on py3 would be nice.
<wgrant> blr, cjwatson: conch still won't work, but at least most of it will.
<blr> wgrant: how painful would it be to run the ssh service as a python 2 process, and the rest python 3?
<wgrant> blr: Very doable, and I suspect that might be what we do.
<wgrant> They're already separate processes, and soon separate VMs.
<wgrant> The code running in the frontends is very small.
<wgrant> testtools probably needs a minor fix, but I think it's easy.
#launchpad-dev 2015-05-21
<blr> wgrant: have you looked at benchmarking pack.compression?
<wgrant> blr: No, but smart people have probably already set it sensibly.
<wgrant> There's nothing in our situation that should cause the defaults to be wrong, I don't think.
<blr> wgrant: can you think of any additional per repo defaults we need other than logallrefupdates and writeBitmaps?
<blr> wgrant: ensure_config is now pulling those values from git.config.yaml in turnip's root rather than defining them inline in the function
<blr> afaict there are no analagous per repo config variables for git repack's arguments
<wgrant> blr: some of them exist, but stuff like -l doesn't
<blr> http://git-scm.com/docs/git-config.html does warn that it isn't comprehensive, but I only see 'useDeltaBaseOffset', 'packKeptObjects' and 'writeBitmaps'
<wgrant> blr: git help config shows a number of pack.* options.
<wgrant> repack.* is just stuff that makes sense only for repack
<blr> wgrant: ah, I see --max-pack-size will honour pack.packSizeLimit if set
<wgrant> blr: Yeah. I don't think we care about pack size at the moment, though.
<blr> depth defaults to 50
<wgrant> We probably want to set window and depth up a way, but also limit pack.windowMemory. I'd set windowMemory to like 2g for now, and we can tweak as we go.
<blr> hmm the repack docs don't mention if --window-memory takes a value from pack.windowMemory
<wgrant>         if (!strcmp(k, "pack.windowmemory")) {
<wgrant>                 window_memory_limit = git_config_ulong(k, v);
<wgrant>                 return 0;
<blr> here I was thinking I could avoid looking at git source
<wgrant>         }
<wgrant> --window-memory overrides that.
<wgrant> Heh, no.
<wgrant> It's pretty bad, but you get used to it.
<blr> wgrant: {depth: 100, window: 20}? (doubling defaults)
<wgrant> blr: Let's make both 250. That's semi-aggressive, and windowMemory will save us if it goes awry.
<blr> cool
<wgrant> An object to pass -f would also be useful, but it shouldn't be the default.
<wgrant> Actually, maybe we should make the default depth and window less aggressive, but allow it to be overridden.
<wgrant> Hmm.
<wgrant> 100/100 with options, perhaps.
<wgrant> 250/250 is slower than I'd remembered.
<blr> sure
<blr> wgrant: as we're now calling ensure_config on repo init, presumably it is no longer needed in PackBackendProtocol.requestReceived()?
<blr> however, existing repositories would be in an undefined state
<blr> given we don't have any means of changing configuration, ensure_config _for now_ can be treated idempotently, but something we may need to consider later
<blr> so I would be inclined to leave it there...
<blr> sorry, thinking out loud really
<wgrant> blr: The idea is that it's cheap and safe enough to run before every write operation.
<wgrant> So when we change it we don't have to do any bulk operation.
<blr> yep, that makes sense.
<blr> wgrant: repack-api ready for review.
<wgrant> blr: Thanks, looking.
<blr> wgrant: r158 has the git config yaml file, whoops.
<blr> wgrant: 3x gives you a stronger, more flavourful json extraction...
<wgrant> Heh
<sil2100> Hey!
<sil2100> I have a question about launchpad authorization tokens
<sil2100> I would like to authorize a script that would be running outside of my local computer with my credentials
<sil2100> Someone mentioned that when an app is authorized, there should be some credentials file created in ~/.launchpadlib which I should be able to re-use elsewhere
<sil2100> But I couldn't find it, but I didn't authorize for 'forever', so maybe that's why
<cjwatson> sil2100: It lives in your desktop keyring, in fact.  But maybe you actually want to re-login with Launchpad.login_with(credentials_file=...).
<cjwatson> sil2100: Is the other computer entirely under your control?
<sil2100> cjwatson: not entirely, I was considering options on how to best set-up something like that on lillypilly or chinstrap
<sil2100> cjwatson: probably not safe enough
<sil2100> hmm
<cjwatson> sil2100: You'd be better off with a separate bot user, or a little dedicated openstack instance or something
<sil2100> Indeed!
<sil2100> Anyway, thanks :)
#launchpad-dev 2015-05-22
<blr> wgrant: did you forsee anything tricky setting up a celery queue for turnip? As you said, would be nice to have callback to lp for repack
<wgrant> blr: No, nothing terribly difficult, but it's not something we should do now.
<blr> yep, fair enough.
<blr> wgrant: how would you about deliberately creating two packs?
<blr> ^go about
<wgrant> blr: I think two commits in pygit2 might do it.
<wgrant> IIRC I generate two commits in my alternate tests.
<wgrant> s/commits/packs/
#launchpad-dev 2015-05-23
<cjwatson> wgrant: Is it intentional that AccessArtifact has FKs for bug and specification, but not branch?
<cjwatson> wgrant: Just tracking down the IntegrityError I found when QAing https://bugs.launchpad.net/launchpad/+bug/1456583, and wondering why Branch.destroySelf apparently doesn't need to clean up its AAs
<mup> Bug #1456583: Can't delete Git repositories <feature> <git> <qa-needstesting> <Launchpad itself:Fix Committed by cjwatson> <https://launchpad.net/bugs/1456583>
<cjwatson> I wonder if there are leftover AA rows for now-nonexistent branches as a result.
<wgrant>     bug integer REFERENCES bug,
<wgrant>     branch integer, -- FK to be added later.
<wgrant> Something must have been checking for branch FKs.
<wgrant> APGF.grantee is the only other one from that patch that wasn't fixed later.
<wgrant> Oh, no
<wgrant> APGF.grantee is deliberately not FKed, I think, because it shouldn't be touched by person merge.
<wgrant> It's purely maintained by triggers.
<cjwatson> So do I need something like getUtility(IAccessArtifactSource).delete([self]) ?
<cjwatson> One of these days I will properly understand the access model :)
#launchpad-dev 2016-05-23
<frecel> hello
#launchpad-dev 2016-05-25
<mwhudson> omg i actually have to wait for a ppa build to start
<mwhudson> haven't had that happen for months
<wgrant> mwhudson: Hm, that's a bit weird.
<wgrant> Oh someone is building about a billion mysqls and mariadbs.
<wgrant> And lcy01 fell over.
<mwhudson> i was wondering if that many buildds in cleaning was normal
<wgrant> Well.
<wgrant> It's normal, but not normal, if you know what I mean.
<wgrant> Not meant to happen, but, well, lcy01 is thpecial.
<mwhudson> i thought the cloud fixes everything
<mwhudson> did i read that memo wrong?
<wgrant> Clouds sometimes mean rain.
<wgrant> mwhudson: Stabbed a few things, and the queue should be clear in a minute or two.
<mwhudson> wgrant: awesome thanks
<mwhudson> wgrant: now can you make the publisher a million times faster
<wgrant> mwhudson: I'm not a wizard.
<mwhudson> shame
<cjwatson> Hm, not convinced that twisted.web.client.HTTPConnectionPool actually works quite the way I was thinking it did.
<cjwatson> maxPersistentPerHost limits the number of persistent connections it retains, but not the number it's willing to open in parallel.
<cjwatson> I think I'll need a subclass that overrides _{get,remove,put}Connection with some extra Deferreds.
<wgrant> That is unfortunate.
<wgrant> But Deferreds fix everything.
<cjwatson> Including "my program is too easy to debug".
<wgrant> That's most of the point.
<cjwatson> I think DeferredQueue is probably my friend.
<cjwatson> Ah, no, DeferredSemaphore.
#launchpad-dev 2017-05-23
<Guest36524> Hi guys, how do I add a apt sources list into my Launchpad PPA?
<wgrant> Guest36524: You can add other PPAs as build dependencies for your PPA using the "Edit PPA dependencies" link.
<Guest36524> Is it not possible if the repository is not on Launchpad?
<Guest36524> For example, deb http://packages.ros.org/ros/ubuntu trusty main
<Guest36524> If so, what would be the solution for this situation?
<wgrant> Guest36524: PPAs cannot depend on repositories outside Launchpad. You'd need to find (or create) a PPA that contains the dependencies that you need.
<Guest36524> Oh OK, that's a shame. Thanks.
#launchpad-dev 2020-05-18
<tomwardill> Total: 56 tests, 1 failures, 49 errors, 0 skipped in 1 minutes 48.597 seconds.
<tomwardill> that went... well
<SpecialK|Canon> but 6 succeeded right?
<SpecialK|Canon> gotta celebrate the wins
<SpecialK|Canon> `def test_foo(): pass`
<SpecialK|Canon> I had a colleague who used to do that actually - said he liked knowing that however much he'd broken stuff, at least something went right
<tomwardill> I've been known to break tests like that
<tomwardill> if you sufficiently mangle the test setup, there's no guarantee even that will work
<SpecialK|Canon> Oh absolutely
<SpecialK|Canon> It's not invulnerable, but it's a nice baseline
<tomwardill> lp.services.scripts.base.SilentLaunchpadScriptFailure: 1
<tomwardill> now, how can that just ... not be silent?
 * tomwardill breaks out pdb
<cjwatson> Yeah, garbo likes to do that
<cjwatson> Good opportunity to make your test add logger output as a test detail
<cjwatson> (Though TestGarbo.setUp is supposed to do that ...)
* SpecialK|Canon changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | Firefighting: - | Critical bugs: <98 | Support and spam reporting: https://answers.launchpad.net/launchpad
<tomwardill> cjwatson: does using <allow> in a configure.zcml allow write?
<tomwardill> getting a ForbiddenAttribute trying to set an attribute on OCIFile
<pappacena> maybe you need a <require> with set_schema?
<tomwardill> yeah,t hat's what I was wondering
<tomwardill> just slightly surprising that an explicit <allow> doesn't... allow
<tomwardill> which makes me think I've got something else wrong somewhere
<cjwatson> tomwardill: <allow> is only for gets, not sets
<cjwatson> As pappacena says, you need <require set_attribute=> or <require set_schema=> for sets
<cjwatson> Honestly I think removeSecurityProxy makes more sense than completely open set permissions
<tomwardill> hmm, yeah, that would work
 * tomwardill does that
<cjwatson> OK, I think my buildmaster tests all pass again.  Next step is to make them make sense
 * tomwardill knows that feeling
<tomwardill> thought i was done,t hen ran some more tests
<tomwardill> SURPRISE NOPE!
<tomwardill> mock_oci_datetime.noew = lambda: now
<tomwardill> well, found a bug
<pappacena> mock_oci_datetime.meow = lambda: cat
<tomwardill> hah
<tomwardill> there's a matching DB patch for this, but could do with someone checking I've not missed anything/done something silly in: https://code.launchpad.net/~twom/launchpad/+git/launchpad/+merge/384121
<tomwardill> garbage collection for OCIFile
<SpecialK|Canon> silly q - why did you move IOCIFS?
<tomwardill> so it was in a more appropriate place in the file (next to IOCIFile)
<tomwardill> as I had a 'I know I wrote this,w here the heck is the code' moment
<tomwardill> (blame past tom for putting it in a weird place to start with)
<cjwatson> I don't see anything obviously wrong at a first skim-read, but my head is mostly full of buildd-manager at the moment so I'll give it a proper look later.  Other people should feel free to review too
<tomwardill> "delivery area not accessible, follow up instructions on the way" no piano today then :(
<SpecialK|Canon> tomwardill: do these tests share state?
<SpecialK|Canon> ...I guess yes, shared testdb across tests
<SpecialK|Canon> no, I just can't count, ignore me
<tomwardill> SpecialK|Canon: no, the db is rolled back
<tomwardill> jaj
<tomwardill> *hah
<tomwardill> matching DB patch: https://code.launchpad.net/~twom/launchpad/+git/launchpad/+merge/384129
<cjwatson> Right, we roll back the db between every build, either by transaction.abort() if there've been no commits, or by restoring a fresh DB from a template
<cjwatson> *between every test
<cjwatson> And attack any case of state sharing between tests with fire
<SpecialK|Canon> nod, cheers, I've now regained my ability to count/read and noticed the second makeOCIFile call...
<cjwatson> SpecialK|Canon: https://code.launchpad.net/~cjwatson/launchpad-buildd/+git/launchpad-buildd/+merge/384140
<cjwatson> Very rough, but gets it out of my head
<pappacena> {"errors":[{"code":"MANIFEST_INVALID","message":"manifest invalid","detail":{}}]}
<pappacena> Thanks for such detailed message, DockerHub. That helped a lot... :unamused:
<pappacena> https://hub.docker.com/repository/docker/pappacena/test-image ð
<pappacena> It kind of worked! :-)
<pappacena> https://usercontent.irccloud-cdn.com/file/OBSogPP9/its-something.png
#launchpad-dev 2020-05-19
<tomwardill> pappacena: \o/ Did that need code changes?
<SpecialK|Canon> cjwatson: <3
<tomwardill> cjwatson: if I was to change my local DB set up to use a username/password, where would I put the password for LP to use it?
<cjwatson> tomwardill: launchpad-lazr.conf in [database]
<tomwardill> aha, of course
<cjwatson> tomwardill: I mean, for a hacky local setup
<cjwatson> tomwardill: For something more realistic, ~/.pgpass would probably be better
<tomwardill> hacky will do for now :)
<cjwatson> .pgpass is what we use on prod
<cjwatson> wgrant: question for you in https://code.launchpad.net/~cjwatson/launchpad/+git/launchpad/+merge/383880
<wgrant> cjwatson: Oh, doesn't phase A query per suite?
<wgrant> I forget.
<cjwatson> It does not - it gets everything and then groups.
<wgrant> Ah, so it does
<wgrant> Just found the code
<wgrant> Fair enough then.
<wgrant> A bit weird that it groups just for logging purposes!
<maozhou> Missing builds can't be created after running add-missing-builds.py, how to fix it? Can anyone help?
<wgrant> maozhou: Can you provide some more detail? What happens when you try?
<maozhou> The distribution is kylin, series is v101 . At first  arch i386 amd64 arm64 has been builded and published, now I want to add missing builds of mips64el , so I run " ./add-missing-builds.py  -d kylin -s v101 -a mips64el" , but the missing builds of mips64el can't be created
<wgrant> maozhou: Have you uploaded a mips64el chroot?
<maozhou> Yes I do
<maozhou> ./add-missing-build.py output like this "2020-05-19 08:10:30 INFO    Considering zzuf 0.15-1 in v101
<maozhou> 2020-05-19 08:10:30 INFO    Considering zzz-to-char 0.1.3-2 in v101
<maozhou> 2020-05-19 08:10:30 INFO    Considering zzzeeksphinx 1.0.20-3kylin2 in v101
<maozhou> 2020-05-19 08:10:36 INFO    Finished adding builds.
<maozhou> "
<wgrant> maozhou: Have you enabled mips64el on /kylin/+archive/primary/+edit?
<maozhou> Oh,I understand, the mips64el in distribution is not enabled, thank you very much.
<wgrant> Yeah, needs to be enabled on the archive (a new row in the ArchiveProcessor table).
<wgrant> ArchiveArch?
<wgrant> I forget.
<maozhou> Yeah, I can enabled it in url "https://launchpad.dev/kylin/+edit"
<wgrant> Ah yes, it's there too, of course.
<pappacena> tomwardill: I poked around because of that strange error, but in the end I think no significant code change was needed. Only the authentication part.
<tomwardill> what was needed to make the authentication work?
<pappacena> Basically, it uses bearer token and you only have the info on where to get the token if you do a request and it fails with 401. So, you need to fail, get the informatino where to do a request to fetch token, and use the token from that moment on...
<tomwardill> cjwatson: can you update dogfood so I can QA a build?
<cjwatson> tomwardill: it's already sufficiently up to date for that
<tomwardill> ah, excellent
<tomwardill> cjwatson: can you poke https://dogfood.paddev.net/~twom/ubuntu/+oci/test-twom-2020-04-22-2/+recipe/test-twom-apt-cacher-ng/+build/18 please :)
<tomwardill> (will need to do it twice to check the reuses doesn't explode things)
<pappacena> I need to edit an OCI Project to QA the "official recipe picker", but only the driver of the distribution has edit permission on that (is that actually correct?). Can you help me on that too, cjwatson?
<tomwardill> well, it built, but I think it needs the registry upload job running
<cjwatson> tomwardill: done, https://pastebin.canonical.com/p/fv3HkwbBxp/
<cjwatson> pappacena: hm, on dogfood you should already be in the OCI project admins for /ubuntu
 * tomwardill starts another build
<tomwardill> https://dogfood.paddev.net/~twom/ubuntu/+oci/test-twom-2020-04-22-2/+recipe/test-twom-apt-cacher-ng/+build/19
<cjwatson> pappacena: ah, looks like the OCI project admin role isn't complete
<cjwatson> pappacena: I think this would be a good time to fix the XXX in EditOCIProject (and probably similar in EditOCIProjectSeries)
<cjwatson> tomwardill: ^- do you agree?  it's your comment
<tomwardill> yeah, seems reasonable :)
<tomwardill> I can take that
<pappacena> Thanks, tomwardill !
<cjwatson> pappacena: for the short term, I've added you to ~ubuntu-drivers on dogfood so that you can QA that
<pappacena> Thanks! QAing now
<cjwatson> tomwardill: and https://pastebin.canonical.com/p/63HpfZmMsV/
<tomwardill> cool, that looks working to me :)
 * tomwardill pushes the green button
<tomwardill> thanks
<tomwardill> cjwatson: for this security.py change, do we want to keep isDriver, or should the only access be `admin` or `oci_project_admin`?
<cjwatson> tomwardill: I'm slightly inclined to keep isDriver
<cjwatson> On some kind of cascading permissions principle
<tomwardill> yeah, that was my thoughts for that
<tomwardill> also makes the tests a lot simpler
<tomwardill> add oci_project_admins to EditOCIProject et al. https://code.launchpad.net/~twom/launchpad/+git/launchpad/+merge/384191
<tomwardill> (aha! this IRC channel still works)
 * pappacena reviewing
<pappacena> tomwardill: I guess super(DelegatedAuthorization, self).__init__(self.obj, self.obj.oci_project) in this case is like "check permissions for `self.obj` the same way you would check for `self.obj.oci_project`".
<pappacena> Anyway, it's not that important change. IMHO, the MP is good to go either way.
<pappacena> I think the 2 MPs for dockerhub integration are ready for review. The first MP is just a refactoring, reorganizing some things to encapsulate registry-specific logic in another class (https://code.launchpad.net/~pappacena/launchpad/+git/launchpad/+merge/384052), and the other MP is actually implementing dockerhub-specific logic (https://code.launchpad.net/~pappacena/launchpad/+git/launchpad/+merge/384192).
<pappacena> The second one depends on the first one, so they need to be reviewed in order...
#launchpad-dev 2020-05-20
<tomwardill> hah, me and cjwatson collide on a pappacena MP again :)
<wgrant> It's always interesting to compare colliding reviews.
<tomwardill> in this case, I think it's the result of spending too much time reading the registry api docs
<cjwatson> wgrant: Could you have another look over https://code.launchpad.net/~cjwatson/launchpad/+git/launchpad/+merge/384035 ?  I think I've dealt with your suggestions (aside from the bit I split out to a Trello card)
<wgrant> Looking
<wgrant> The bulk helper I suppose?
<cjwatson> Yeah
<cjwatson> I agree it would be a good idea - the current necessary invocation is pretty messy
<wgrant> Yeah
<wgrant> Was hoping it'd gross you out enough that you'd fix it as a driveby, but it is non-trivial indeed.
<wgrant> Thanks for fixing my misguided NewBuildersScanner class.
<wgrant> cjwatson: Just a couple of trivial comments. Thanks.
<wgrant> Did my review of the followup make sense?
<cjwatson> I've only skimmed it so far, but it looked clear enough
<cjwatson> Need to QA the DMARC thing and maybe catch up on more reviews, but it's next on the list after that
<cjwatson> teward: Could you have a look over http://blog.launchpad.net/?p=4318&preview=true and see if it looks right to you?
<cjwatson> teward: (The change isn't on production yet - that post is in draft until it is, hopefully in a few hours)
<pappacena> tomwardill, cjwatson thanks for the great review. :-)
<tomwardill> heh
<teward> cjwatson: 404
<teward> Cant see it
<teward> Either i need rights on the site, or you will just need to pastebin or email me the contents for review :)
<cjwatson> teward: Ugh, sorry.  Plain-text part of it is https://paste.ubuntu.com/p/5RPrCrVKDf/ (some links, but nothing critical for understanding)
<teward> looks good to me.  be aware automatic address books and autocomplete in email clients will explode with thisnchange and thats beyond our ability to fix.  But otherwise the post contents look good to me.
<cjwatson> Right, although that's already the case for people with hide_email_addresses set
<cjwatson> And presumably mangling the real-name part of the address would just result in a vast pile of "Colin Watson via bug #1000", "Colin Watson via bug #1001", etc. junk in people's address books / autocomplete, which doesn't seem like an improvement
<mup> Bug #1000: There are too many bug reports in Malone <lp-foundations> <NULL Project:Invalid> <https://launchpad.net/bugs/1000>
<mup> Bug #1001: Distribution shouldn't have "Change Members" etc for those who can't <lp-foundations> <Launchpad itself:Fix Released by mpt> <https://launchpad.net/bugs/1001>
<teward> True cjwatson
<cjwatson> teward: Do you think I should add any explicit text about this, or just leave the "any strange behaviour in your email client" bit to do the work?
<teward> Just leave it as is
<cjwatson> OK
<cjwatson> Thanks
<teward> My statement was if people complain about autocomplete in their clients we Invalid and explain thats a problem we cant fix
<teward> More for us as a set of triage guidelines for those issues
<teward> Because they will crop up so
<cjwatson> Right, though it may be worth looking around in case there are some magic client-dependent strategies for avoiding it in certain cases
<cjwatson> We'll see
<cjwatson> Reminded of http://blog.launchpad.net/notifications/improved-filtering-options-for-gmail-users although the solution to that ended up being almost comically low-tech
<cjwatson> teward: https://blog.launchpad.net/notifications/bug-emails-now-use-the-bugs-address-in-the-from-header up now
<tomwardill> bug closed and commented: https://bugs.launchpad.net/launchpad/+bug/1589693/comments/27
<mup> Bug #1589693: Make Launchpad DMARC Compliant to avoid Launchpad mail considered spam <Launchpad itself:Fix Released by teward> <https://launchpad.net/bugs/1589693>
<teward> cjwatson: tomwardill: re 1589693 i edited the title/summary to reference bug mail specifically
<teward> i also opened 1879740 for the code notification issues, in the same vein as the other bug.  But with a fairly terse comment since it doesn't need reexplained DMARC importance now.
<teward> thank you though, all of you, for allowing me to work with you guys to get the bug notification bits pushed in (and working with me to get a dev environment up and helping me through the initial hurdles of LP dev heh)
<teward> and i've already started work on the code notification changes.  but something else broke in the test suite so i'm still digging there.
<SpecialK|Canon> teward: Thanks for contributing :)
<teward> happy to :)  and happy to continue to contribute at least a little :)
<cjwatson> Indeed :)
<cjwatson> Hm, I suppose I'd better not leave TestBinaryBuildPackageBehaviourBuildCollection broken overnight, bah
<cjwatson> Could I have a quick review of https://code.launchpad.net/~cjwatson/launchpad/+git/launchpad/+merge/384280 please?
<tomwardill> ð
<tomwardill> +1
<cjwatson> Ta
<teward> cjwatson: i can confirm the mail delivers properly with the new LP from addresses.  My syntax checker is doing a 'Hmmmmm..." because Reply-To and From are the same base address, but that's a tiny minor annoyance and not really breaking email at all
<teward> so thanks again for accepting the contribution in :)
<pappacena> tomwardill: you probably EODed already, but the MP with bearer token auth is ready to be reviewed. If you have some minutes tomorrow: https://code.launchpad.net/~pappacena/launchpad/+git/launchpad/+merge/384192
#launchpad-dev 2020-05-21
<maozhou> How to use manual mode of builders?
<wgrant> maozhou: A builder's manual flag just prevents dispatch of new builds.
<maozhou> wgrant: Cannot assign job manually by manual mode?
<maozhou> How to assign jobs to designated builders?
<wgrant> maozhou: That's not something that is supported at the moment.
<maozhou> wgrant: ok ,thanks
 * tomwardill returns
<tomwardill> "SyntaxError: Unexpected token {"
<tomwardill> and so it begins
<roadmr> Many thanks for the "create a merge proposal" remote git messages! they're a real time saver
#launchpad-dev 2020-05-22
<roadmr> every time the "propose an mp here" remote message saves me time, I'll use it to come here and sing the Launchpad team's praises ð  (or until you tell me to shut up and go away :)
<roadmr> oh also a question: has it ever been possible to log in using username instead of email on Launchpad or SSO? if so, any idea why that was changed to using email only?
<cjwatson> I saw a few upset SSO questions about that
<cjwatson> If it ever was in LP I think it was before my time
<cjwatson> But honestly I'm not sure it ever occurred to me to look
<roadmr> cjwatson: yes, people caring about that minute detail tend to be the upsetty kind :/
<roadmr> "please don't assume I know my email address" say WHAT
<cjwatson> I can sort of understand that from the "generate unique email address for each different thing" crowd
<roadmr> well yes but if you're in that crowd I'd expect you to have proper tracking of all the addresses you're spitting out I guess
<roadmr> cjwatson: I was thinking it might have happened in the SSO split - when Launchpad was in charge of logins, a username was mandatory and could be used to log in. SSO didn't require a username for a long time, so the email was the only consistent login option.
<roadmr> These days all new accounts do have a username
<cjwatson> It is possible, but all this was before my time and I've never done the archaeology, I'm afraid
<cjwatson> I can't immediately think of a reason why you shouldn't be able to log in by username.  It seems an odd omission.
<cjwatson> (Assuming you have one)
<roadmr> cjwatson: maybe we can ask the chief archaeologist in a couple of hours
<cjwatson> The theory that it was something to do with SSO not having usernames for a long time seems a likely one.
<tomwardill> cjwatson: if I wanted to add the nodejs from u1hackers to LP, is there anything special I'd have to do to test it? (I've run a LP make cycle locally and it seems happy)
<cjwatson> tomwardill: Check that CSS and JS works in a browser after make; run tests for --layer=YUITestLayer --layer=YUIAppServerLayer
<cjwatson> tomwardill: And we'd need to not land anything that depends on it until relevant machines (I guess carob, banana, nutmeg, atemoya, labbu) have been upgraded (which you can check in Landscape)
<cjwatson> Indeed upgrading would require a ticket to copy it into the relevant IS-owned staging and production PPAs
<cjwatson> https://launchpad.net/~canonical-is-sa/+archive/ubuntu/launchpad-staging and https://launchpad.net/~canonical-is-sa/+archive/ubuntu/launchpad
<tomwardill> right
<tomwardill> verbose 3.205 Performing "GET" request to "https://registry.yarnpkg.com/node-sass/-/node-sass-4.14.1.tgz".
<tomwardill> error request@2.88.2: The engine "node" is incompatible with this module. Expected version ">= 6".
<tomwardill> boo, okay, so we definitely need the SCA node backport
 * cjwatson nods
<tomwardill> okay, I'll start the upgrade dance on Monday I guess
<tomwardill> no wait
<tomwardill> Tuesday
<cjwatson> pappacena: thanks for that MP, dinnertime here now but I'll try to have a quick look at it this evening
 * pappacena was about to send the link here :)
<pappacena> Thanks, cjwatson. Whenever you have time. If you find something while reviewing, I'll probably still be around to make adjustments.
 * pappacena going back to OCI project / project stuff for now
<SpecialK|Canon> I just realised the views are live!
<SpecialK|Canon> Nice work
<SpecialK|Canon> Let's get that in next week's weekly
<pappacena> :-) and soon we will have those views for project based oci project project project too! hehe
<tomwardill> Sure you missed a âprojectâ in there.
<tomwardill> we can find room for more.
 * pappacena ð
<cjwatson> pappacena: looks fine, just one trivial nit
<cjwatson> pappacena: (though I was curious why the store.invalidate() stuff in the second test, rather than just testing the object you get back from .inject()?)
<pappacena> let me check
<pappacena> Ah, the idea was to flush, but I guess it doesn't make any difference since it's fetching from the database after saving. I'll remove it.
<pappacena> Fixed the typo. I'll land that MP
<cjwatson> Cool, thanks
<cjwatson> Should be able to deploy and rerun on Tuesday or so I guess
<pappacena> Cool. Hope everything will work then. Enjoy your weekend and thanks for the review!
<cjwatson> You too!
