[11:02] <gmb> danilos: Can I just check something about the team subscription opt-out story that I'm working on (I'm assuming you'll be able to answer this question based on my vague memory of what Gary said yesterday; if you can't, no worries)
[11:02] <gmb> danilos: Is the opt-out supposed to be for team *structural* subscriptions or for direct team subscriptions or both?
[11:51] <gmb> E_BRAIN_NOT_FIRING. Lunch
[12:08] <danilos> gmb, I think this was mostly about structural subscriptions
[12:17] <danilos> gmb, but, you are making a good point about direct team subscriptions
[12:47] <gmb> danilos: Damn, I was hoping you'd say "no, we don't care about direct subscriptions."
[12:47] <gmb> But thanks anyway :)
[12:47] <danilos> gmb, :)
[13:09] <gmb> gary_poster: Just the man! To repeat my question to danilo earlier: Is the opt-out from team subs supposed to be for team *structural* subscriptions or for direct team subscriptions or both?
[13:09] <gary_poster> gmb, only structural
[13:10] <gmb> HURRAH
[13:10] <gmb> I mean, cool.
[13:10] <gmb> :)
[13:10] <gary_poster> if it is direct, people can just mute the bug
[13:10] <gary_poster> :-)
[13:10] <gmb> Right.
[13:10]  * gmb goes to finish the DB patch.
[13:18] <gary_poster> gmb...spanner in the works, but thinking about how we are presenting this, either we need to be able to mute *filters* or we need to think about a good UI for muting the entire structural subscriptions
[13:19] <gary_poster> filters are the natural way to approach this from a UI perspective
[13:19] <gary_poster> from the DB perspective, that's no biggie...
[13:19] <gmb> gary_poster: Hmm. At the moment I've got a StructuralSubscriptionMute table; it doesn't take much to tie it to the filter rather than the subscription.
[13:19] <gmb> And I agree, filters are what we present to the user as "subscriptions"
[13:20] <gary_poster> from the perspective of how the muting would work, and efficiency, that's a bit more concerning...
[13:20] <gmb> You mean in terms of things-to-check-for-mutes-when-sending-emails?
[13:20] <gary_poster> yeah
[13:20] <gmb> Hmm.
[13:21] <gary_poster> lemme look at the code I'm trying to get reviewed...
[13:21] <gmb> ok
[13:22] <gary_poster> ok, so here's the good news...
[13:22] <gary_poster> we already have code that gets the pertinent filters for each person
[13:22] <gary_poster> and hopefully it is fairly efficient
[13:22] <gary_poster> with my branch that is to be reviewed
[13:23] <gmb> Ah, cool.
[13:23] <gary_poster> it will need slight tweaks but no big deal
[13:23] <gmb> So we could adapt that code so that "pertinent" doesn't include "muted"?
[13:24] <bac> hi gary_poster, my branch didn't get reviewed.  do you want to do it or shall i hit up gavin who is OCR?
[13:24] <gary_poster> well, sort of, yes.  It would always start out by giving all of the filters per person; then we would need to look for any mutes
[13:24] <gary_poster> bac, try gavin first; if he has a backlog, I can get to it within an hour I'd guess
[13:25] <bac> will do
[13:25] <gary_poster> thanks
[13:25] <gmb> Okay, cool.
[13:25] <gary_poster> so gmb, I *think* that we will be adding a single query per person
[13:26] <gmb> gary_poster: Right, okay. First things first, I'll update my DB patch, then I can come back to the code surrounding it. I think that updating your code will come late on in the process, so we've time to thrash out a solution to any performance issues.
[13:26] <gary_poster> something like "given this person and all of the possible filters that might be involved, [give me the ones that are muted and I'll subtract them and if they are all muted then nevermind OR give me the ones that are not muted]
[13:26] <gary_poster> "
[13:26] <gary_poster> cool
[13:27] <gmb> Right. That makes sense.
[13:28] <gary_poster> bac benji danilos gmb, mumble/kanban in 2
[13:28] <bac> yep
[13:29] <gmb> ack
[13:44] <bac> this https://wiki.canonical.com/2011Calendar shows week of june 27
[13:45] <gary_poster> Question for the Committee:
[13:45] <bac> based on danilos' problem with the renaming of the feature flag i think we should move to defining a symbol and never use the actual string in code
[13:45] <gary_poster> I have a test failing thusly:
[13:45] <gary_poster>     + ERROR   Unhandled exception
[13:45] <gary_poster>     +  -> http://localhost:56152/96/oG1i6vRy0DfzXWgDOpRsv7iAq4e.txt (permission denied for relation account
[13:45] <gary_poster>     + )
[13:46] <gary_poster> bac, good idea, +1
[13:46] <bac> hmm, that might be hard in the template...
[13:46] <gary_poster> could be request/feature_flag_names/THE_NAME maybe
[13:46] <gary_poster> but that would take a bit of doing
[13:46] <gary_poster> not much
[13:46] <gary_poster> but some
[13:47] <gary_poster> alternatively we could say that views should expose the constant
[13:47] <gary_poster> so it could be view/THE_NAME
[13:47] <gary_poster> that would be a bit lighter weight but also require a bit more coding for every template
[13:48] <gary_poster> so anyway, about my failing test...
[13:48] <gary_poster> I've looked for the file that is mentioned
[13:48] <gary_poster> in . and in tmp
[13:48] <gary_poster> not there
[13:48] <gary_poster> I assume that it is about SQL security
[13:48] <gary_poster> but it would be nice to actually see the traceback
[13:49] <bac> gary_poster: what is the failing test?
[13:50] <gary_poster> lp/registry/doc/distribution-mirror.txt line 650
[14:05] <bac> gary_poster: we won't be breaking the assurances you made to wgrant about xss if we turn on the feature flag on qastaging, will we?
[14:06] <gary_poster> bac, I thought about it and I don't see how.  qastaging does not use the prod db.  Please share your concerns!  Thinking about it some more...
[14:06] <gary_poster> turning on the feature flag for ourselves (~yellow, say) means that we might be able to be evil and hack data from the db
[14:07] <gary_poster> but there would presumable be much easier ways for us to di that than this XSS thing
[14:07] <bac> gary_poster: my concerns are ill-formed and i need to re-read the thread.  i just recall part of your argument for leaving the code in was that it wasn't turned on
[14:07] <gary_poster> true.  In this context, I would say this:
[14:07] <bac> gary_poster: well, i had intended to do malone-alpha
[14:07] <gary_poster> 1) It is not turned on in prod
[14:07] <bac> but i can request ~yellow instead -- a much better lpan
[14:07] <bac> plan
[14:08] <gary_poster> 2) qastaging for ~yellow sounds ok
[14:08] <gary_poster> ok cool bac
[14:11] <bac> gary_poster: while i wait on my review to be done i'm going to work on peer reviews.  i've got a bunch.
[14:11] <gary_poster> cool bac
[14:12] <gmb> Hah, Marianna has just emailed the list about Dublin. So now we know.
[14:14] <danilos> gmb, she seems to have forgotten to mention the month :)
[14:14] <gmb> danilos: It's in the subject.
[14:14] <gmb> 27th June.
[14:14] <gmb> (Week beginning)
[14:14] <danilos> gmb, ah, true :)
[14:14] <gmb> But yeah, that threw me for a moment :)
[14:25] <gary_poster> danilos or someone else, if I change security.cfg that means I have to land on db-devel (not good for this branch) right?
[14:28] <danilos> gary_poster, I don't think you do, but I might be wrong; maybe LOSAs know if security.py is run on "stable" rollouts if stub is not around
[14:28] <gary_poster> ack danilos, I'll check, thanks
[14:29] <danilos> gary_poster, (also, there's the pqm check which stops landing certain changes on devel, not sure if that includes security.cfg changes)
[14:36] <gary_poster> ack danilos.  I thought that code was actually in the tree somewhere but can't seem to find it
[14:36] <gary_poster> gmb, mumble?  now, or in a few?
[14:40] <gary_poster> danilos, security.py is in fact run for no-downtime deploys, so good.  If I want to just run the security changes such that tests will see them, will running "./database/schema/security.py" do the trick?
[14:42] <danilos> gary_poster, you'll have to do it for all the appropriate databases (i.e. launchpad_ftest_template I think), check "--help" :)
[14:42] <gary_poster> danilos, ack, I just did it "./database/schema/security.py -d  launchpad_ftest_template" and "./database/schema/security.py -d  launchpad_ftest_playground" ....we'll see
[14:43] <danilos> gary_poster, good luck :)
[14:43] <gary_poster> thanks :-)
[14:46] <gary_poster> worked, yay (both the security.py call, and the fix for at least one of the two failures)
[14:46] <gary_poster> boo, not the other one, but should be same idea...
[14:50] <gmb> gary_poster: Sorry, dropped offline for a bit... you were talking to my proxy.
[14:51] <gmb> I'm free now if you are.
[14:51] <gary_poster> :-) ok, 1 min the I'll be ready gmb.  thanks
[14:56] <danilos> crap, I'll need Irish visa (or, alternatively, I can land in Belfast and then travel to Dublin by bus/train :)
[15:19] <benji> gary_poster: no need for me to email about Windmill tests; Ian has a branch to fix them (although I do worry a little about our process: why didn't I know they were broken and how could I have found out (if I'd thought to go looking))
[15:22] <danilos> gary_poster, I've noticed that in JS setup_bug_subscriptions() wraps everything inside Y.on('domready') and it's further wrapped in Y.on('domready') in all callsites in templates; if I am to remove one, which one would you prefer? :)
[15:29] <gary_poster> benji: good point and not the first time. :-/
[15:29] <gary_poster> benji, I guess I'll bring it up to flacoste unless you want to take the initiative?  I think it bears raising
[15:30] <benji> have at it
[15:30] <gary_poster> k
[15:30] <gary_poster> danilos, I thought I vaguely remebered something like that.  We can talk about it on call :-)
[15:30] <gary_poster> (which I'm ready for anytime)
[15:31] <danilos> gary_poster, not sure what's wrong, give me a minute
[15:32] <danilos> I can
[15:33] <danilos> gary_poster, I don't get to see any input from my microphone
[15:33] <gary_poster> danilos, hm :-(
[15:33] <gary_poster> so Skype would not work either, I assume
[15:33] <danilos> gary_poster, let me try the test call
[15:33] <gary_poster> k
[15:34] <danilos> I hope my headset didn't die in the middle of the day
[15:34] <gary_poster> or just try me, whatever is convenient, danilos
[15:34] <gary_poster> I just logged on
[15:35] <danilos> gary_poster, nothing in the test call :(
[15:35] <gary_poster> :-(
[15:35] <gary_poster> ok...
[15:35] <gary_poster> maybe the computer mic?
[15:35] <danilos> gary_poster, I am looking if anyone of my friends have it here
[15:35] <gary_poster> sometimes headset output plus computer mic input is OK
[15:35] <gary_poster> ok cool
[15:36] <danilos> a desktop here, no mic on it :(
[15:36] <danilos> I'll try a windows technique now, rebooting
[15:36] <gary_poster> heh
[15:42] <danilos> gary_poster, didn't seem to have helped, I guess I am microphoneless right now :(
[15:43] <gary_poster> danilos, :-(
[15:43] <gary_poster> ok
[15:44] <danilos> gary_poster, just tried the headset on my friends computer, microphone is apparently dead
[15:44] <gary_poster> danilos, as to your question above, I'd prefer we do things similarly, and the .setup function uses domready on the template.  I don't really have a preference as to which approach we use
[15:44] <gary_poster> it would be easier to just use the one on the template
[15:45] <danilos> gary_poster, right, I'll make sure it's consistent across setup/setup_bug_subscriptions
[15:45] <gary_poster> because then you only change one thing
[15:45] <gary_poster> but do whichever you want
[15:45] <danilos> gary_poster, ack, was just wondering if you have any preferences
[15:45] <gary_poster> danilos, I'll make a card for us to have a replacement call tomorrow, assuming you have the mic sorted out by then
[15:45] <danilos> gary_poster, yeah, I'll make sure I do
[15:46] <danilos> gary_poster, sorry about this (I hope the fact that I muted on the headset and never silenced or muted mumble itself didn't kill it, because that would be very weird)
[15:46] <gary_poster> danilos, preference: naah, make a choice and I'll be happy with it.  I have reasons to prefer both approaches.
[15:46] <danilos> gary_poster, heh, ok, thanks for the input
[15:46] <gary_poster> danilos, np.  comuters suck ;-)
[15:47] <danilos> :)
[15:51] <gary_poster> ok gmb, I actually can start work on the mute thing now.  So, let's see: we need code that handles the muting, and we need to expose the table in the webservice, and we need JS changes.  I'm probably being crazy, but I think we can get this done pretty quickly.
[15:51] <gary_poster> The webservice part is the only thing I'm not sure how to do, amusingly enough given my Foundations background.
[15:51] <gary_poster> So...what should I do?  I think one of us should work on the webservice exposure, and one of us should work on the script.  I'm interested in the webservice change, but I also have a fair amount of knowledge of the email-sending script, so either would work for me.
[15:51] <gary_poster> I'd say my default choice would be the email, because I think I know exactly where the bodies are buried for that one.  What do you think?
[15:52] <gmb> gary_poster: I'm happy to work on the webservice side of things since it's close to the front of my brain right now.
[15:52] <gary_poster> ok cool
[15:52] <gmb> I'm just making final tweaks to my DB patch (so that it, yanno, works)
[15:52] <gary_poster> what's your branch, gmb?
[15:52] <gary_poster> oh ok
[15:52] <gary_poster> ping me when it's ready for me to run with it
[15:52] <gmb> SQL doesn't take kindly to missing keywords.
[15:52] <gary_poster> :-)
[15:53] <gmb> gary_poster: Sure. Should be ready in < 5 mins.
[15:53] <gary_poster> awesome
[16:05] <danilos> bac, are you able to QA any stuff on qastaging? (I can see the flag should be on for malone-alpha but the links don't turn green)
[16:06] <bac> danilos: you see a similar but different feature flag.  ours is not yet turned on.
[16:06] <bac> tom said he'd do it but was very busy ATM
[16:06] <bac> apparently some ISD issues are on fire
[16:06] <danilos> bac, ah, ok
[16:08] <gmb> ZCML, I hate you.
[16:10] <gmb> gary_poster: Branch is up here: lp:~gmb/launchpad/team-subscription-opt-out. It contains the DB patch and model / interface definitions.
[16:11] <gary_poster> awesome gmb, thanks.  I'll merge it with my optimization branches under review and work from there for the script.
[16:11] <gmb> Cool.
[16:11] <gmb> I'll ping you as I update things that might have an impact on what you're doing.
[16:16] <gary_poster> cool
[16:23] <bac> danilos: here is the FF request i made: http://paste.ubuntu.com/587806/
[16:25] <danilos> bac, looks good, shall we try with other LOSAs or are they all busy with the same emergency?
[16:25] <gary_poster> Weird.  Getting Graham's branch is taking the most work of any LP branch I've had in a long time
[16:25] <gary_poster> Fetching revisions:Inserting stream:repacking chk:chk node 422334/559643
[16:25] <gary_poster> most work in bzr I should say
[16:25] <gary_poster> bizarre
[16:25] <gary_poster> heh.  heh.
[16:25] <danilos> bac, btw, what does 16 indicate here? (I know I have 10 in my local set-up)
[16:26] <bac> it is the priority.  TBH i just stole the line from the other related FF.  it only comes into play if there are other flags with the same name in other scopes...maybe?
[16:27] <danilos> bac, ah, ok, thanks for the clarification, I was just wondering what it was supposed to be in the first place
[16:28] <gary_poster> wow... it is *still* going...
[16:30] <gmb> gary_poster: Might be something to do with it being a db-devel branch
[16:30] <gmb> I noticed the same thing when I pushed.
[16:37] <gary_poster> huh
[16:37] <bac> danilos: as to the LOSAs, i peeked into lp-ops a bit ago and they still seemed quite busy.  ping them later if you want.
[16:37] <gary_poster> well, I got it now
[16:41] <danilos> bac, yeah, I will thanks
[16:57] <gary_poster> biab
[18:18] <gmb> gary_poster: I'm about to EoD. I haven't added anything of note to my branch. If you need to add any APIs or what have you to IBugSubscriptionFilter or *Mute just add them and send me an email so I know to merge the changes.
[18:19] <gary_poster> gmb, sweet.
[18:49] <gary_poster> danilo, you around?  I have database things to ask you, if you are :-0
[18:49] <gary_poster> :-)
[18:50] <gary_poster> Storm, to be precise
[18:50] <danilos> gary_poster, I am
[18:50] <gary_poster> cool
[18:50] <gary_poster> so, if I want to get only fields as results from storm
[18:51] <danilos> gary_poster, you get tuples of actual values :)
[18:51] <danilos> gary_poster, but I suppose that's not the question
[18:54] <gary_poster>  :-) yeah sorry someone came to door
[18:54] <gary_poster> so yesterday I had a bit of annoyance
[18:54] <gary_poster> Storm was trying to cast values
[18:54] <gary_poster> and the values were sometimes None
[18:55] <gary_poster> which the casting didn't like so much
[18:55] <gary_poster> They were None for expected reasons
[18:55] <gary_poster> but the fields couldn't handle it
[18:55] <danilos> gary_poster, ah, interesting, did it happen with foreign references?
[18:55] <gary_poster> yeah
[18:55] <gary_poster> maybe
[18:55] <gary_poster> :-)
[18:55] <danilos> gary_poster, you should probably use "columnID" then
[18:56] <danilos> gary_poster, and probably file a bug against storm if there isn't one already
[18:56] <gary_poster> That worked well, actually, but...oh, here's my concrete example...
[18:56] <danilos> gary_poster, I am guessing it's a left join then :)
[18:57] <gary_poster> right
[18:57] <gary_poster> I had a left join
[18:57] <gary_poster> and I wanted to get a status
[18:57] <gary_poster> EmailAddress.status to be precise
[18:58] <gary_poster> and sometimes it would be None
[18:58] <gary_poster> because of left join
[18:58] <gary_poster> and things would fall over
[18:58] <gary_poster> and I wondered if there were a way to get the value without processing
[18:58] <gary_poster> like you can do columnID
[18:59] <gary_poster> Is there a trick like that?
[19:00] <danilos> gary_poster, how about coalesce?
[19:00] <gary_poster> ooh, sounds magic :-)
[19:01] <danilos> gary_poster, basically Coalesce(Something, -1) will return -1 if Something is NULL
[19:01] <gary_poster> I'll try googling storm coalesce
[19:01] <danilos> gary_poster, it's regular SQL COALESCE, just wrapped in Storm as Coalesce I believe
[19:02] <gary_poster> I'll play with it.  Thanks danilos!
[19:02] <danilos> gary_poster, it's probably no surprise that most of the occurrences are in lp/translations :)
[19:02] <gary_poster> :-)
[19:03] <gary_poster> danilos, btw, I had another idea on how to reduce SQL in sending email.  I think it might be important, since we are going to be adding yet more for the team subscription mute
[19:03] <danilos> gary_poster, ah, sounds nice :) what is it?
[19:05] <gary_poster> I'm trying to get all of the data for each notification batch at once (roughly), only retrieving the columns I want, and then divide things up in Python.  I'm not sure if its a win in the big picture yet, but it should certainly reduce our SQL.
[19:05] <gary_poster> I keep thinking I should make a harness to produce something like what we have in production...I'm not even entirely sure what that would look like.
[19:06] <gary_poster> Anyway, I'm experimenting.  I'll show you if I think it looks OK to see if you have any thoughts
[19:07] <danilos> gary_poster, yeah, probably having a script which creates a number of targets with a bunch of bugs, adds a few structural subscriptions and then does a few changes on existing bugs and files new ones would be nice for testing all this
[19:08] <gary_poster> The test factory seems like the easiest way to do that
[19:09] <danilos> gary_poster, yeah, definitely
[19:09] <gary_poster> I did a little bit of that for my last branch.  Maybe I'll do some more this time around
[19:09] <danilos> gary_poster, and, on the topic of getting everything once, what do you mean with "notification batch"? for a single bug notification?
[19:10] <gary_poster> the notification batch that construct_email_notifications gets
[19:10] <danilos> gary_poster, fwiw, that's also why we have split-up sample data between test and playground, you can probably commit some of it if you use nice product names and such :)
[19:11] <gary_poster> danilos, I was just going to write a "test" and use that, rather than anything permanent.  that way I can change things more easilt
[19:11] <gary_poster> y
[19:11] <gary_poster> do you suggest something diff?
[19:12] <danilos> gary_poster, I like the idea behind notification batches with a small finite number of queries, so big +++1 on that
[19:12] <gary_poster> cool
[19:12] <danilos> (that was stupid, +1000 is more like it :)
[19:12] <gary_poster> :-)
[19:12] <gary_poster> only stupid to a mathematician like you :-)
[19:13] <gary_poster> it won't be finite everywhere, but this last main loop can be finite, yeah
[19:13] <danilos> heh, yeah, to a developer like me, it evaluates to +(++1)==+2 :)
[19:13] <gary_poster> :-)
[19:14] <danilos> anyway, regarding testing, a test is good, but I always miss the actual data to easily test stuff, though it's probably better to have scripts that produce the data at will instead of including it in sample data
[19:15] <gary_poster> ok cool
[19:18] <danilos> gary_poster, the biggest win, though, will be from getting stuff for multiple persons at the same time, but as we all know, that's a "bit" more work
[19:19] <gary_poster> :-) well, that's what I'm trying to do.  Like I said, I'll ping you for your thoughts later.
[19:21] <danilos> gary_poster, but isn't a batch construct_email_notifications gets restricted to a single person?
[19:21] <danilos> gary_poster, or are you changing that to take batches with different recipients as well?
[19:21] <gary_poster> danilos, it's restricted to a single "owner" or "actor".  There are many recipients
[19:21] <danilos> gary_poster, aaaahhhh
[19:22] <danilos> gary_poster, sorry then, yeah, that'd be very good then :)
[19:22] <gary_poster> cool :-)
[19:32] <gary_poster> bac, no rush, but ready for call when you are
[20:00] <gary_poster> bac, Skype test call did not have static when I called them. :-/
[20:00] <gary_poster> just now I mean
[20:00] <bac> i'm all for using skype again
[20:00] <gary_poster> Yeah ok
[20:59] <gary_poster> benji, can you give me 10?
[20:59] <benji> sure
[20:59] <gary_poster> thx
[21:05] <benji> if anyone needs a task I have some html generation refactoring I can farm out
[21:18] <gary_poster> benji, sorry, took longer than expected but ready now.
[21:20] <bac> benji: i'm up for share cropping
[21:20] <benji> bac: can you take on setup_overlay
[21:20] <bac> sure.  branch?
[21:21] <benji> bac: lp:~benji/launchpad/better-HTML-generation (pushing now)
[21:21] <bac> benji: ok.  i'm in the middle of something ATM but can look soon
[21:25] <bac> gary_poster: doesn't this seem like an easier way to go that should be more efficient? http://pastebin.ubuntu.com/587978/
[21:27] <gary_poster> bac, if that is transitive, then yes, looks great, thank you
[22:03] <gary_poster> bac, please don't forget to claim "potential performance" card and move it into "Coding"
[22:04] <bac> gary_poster: ok
[22:04] <gary_poster> thanks
[22:05] <bac> done