=== Ursinha is now known as Ursinha-afk [06:08] lifeless: I see you were playing with the idea of sacrificing bug-count accuracy for private bugs. I sometimes wonder if it makes sense to treat them as the same thing in the first place. [06:09] UI-wise, that is. [06:11] If we had, say, private-bug portlets then we'd get more caching, less low-level decision-making ("which bugs should I count here exactly?") and maybe even less of the narrow-deep/wide-shallow query antipattern as a matter of course. [06:13] In that example (private-bug portlets), I imagine bug counts etc. including only public items, and logged-in users with access to private items getting something extra on the side to show those items. [06:13] In some ways that'd be worse UI, but in some ways it might be better: less "oh you can't access this it's private" and more confidence that private things really are private. [06:14] jtv: its an interesting discussion to have for sure [06:14] jtv: doesn't really impact the coding though - its easy enough to union in on search generation to avoid shallow-wide issues [06:16] jtv: caching of public stuff doesn't really interest me [06:16] shallow is rarely an issue when it comes to db query performance. :) [06:16] jtv: users seem to notice stale data [06:16] so I'm very keen on use ripping out caches that aren't keep fresh [06:17] Of course. [06:17] But how often do we need e.g. the Ubuntu bug count? [06:17] dunno :) [06:17] In fact... [06:18] I can tell you of course [06:18] this is way too generic, but [06:18] just a thought: [06:19] imagine we had support for short-lived caches with optional aggressive pre-seeding. [06:19] So basically on-demand caching with short lifetimes, but separately from that imagine we could pick the most popular items and refresh them before they expire. [06:20] there is a common antipattern that turns up there [06:20] That'd be a way to head off the thundering herd. [06:20] which is that the generation time is high [06:20] said generation time is then the lowest latency for updates to the cache [06:21] Why would a high generation time be the lowest latency for updating the cache? [06:22] either the data is incrementally updatable [06:23] in which case a cache is a bit of a misleading description (its more a memo that can be edited - like my fact table) [06:23] or the data has to be regenerated after a change occurs [06:24] in which case the shortest latency between change and inserting the updated cached item is the time it takes to generate the the item from scratch [06:27] So you're really saying the problem there is we lose sight of the time it takes for a change to become visible to the user? [06:27] yes [06:28] the key is to build data structures that let us do /all/ the work we need to do efficiently [06:28] e.g. [06:28] 'update the bug' [06:28] 'update the aggregates the bug appeared/now appears in' [06:50] lifeless: ISWYM [06:51] lifeless: got a moment to discuss something else? Pre-imp, basically. [06:52] sure [06:52] Thanks. It starts at this page: https://wiki.ubuntu.com/NewReleaseCycleProcess [06:53] Lots of manual steps on there, many of which could be automated, when a new Ubuntu distroseries is created. [06:53] Some of those steps apply to other distros as well. [06:54] I've been asked to automate steps 12 & 14. [06:55] Basically, two script runs for a new distroseries. [06:56] AFAICT those script runs can be done right after initialise-from-parent (step 10). [06:57] For Debian series, it basically doesn't matter whether we do that script run or not. [06:58] And I think for the foreseeable future, every series that matters is going to have a parent to initialise from. [06:58] The way I think I'd like to handle that is: [06:58] man, that whole thing should be automated [06:58] Exactly. [06:58] We don't follow that process for Debian at all, since we don't publish it [06:59] And I don't want my solution to get in the way of that, but also not go overboard on the design. [06:59] StevenK: no, I'm just using that page as a starting illustration here. [07:00] StevenK: as I said earlier, many of those steps are not Ubuntu-specific on the one hand, and on the other we can take or leave steps 12 & 14 for Debian. [07:01] What I have in mind for a solution when it comes to just steps 12 & 14, is to have a DistributionJob type that is handled by a runner on cocoplum. [07:01] If it fails, I'm thinking it should notify the distro owners who can then sort it out with us LP people. [07:02] jtv: so, what about the cron job disable/enable? [07:02] seems you need to fit in with that / make it unneeded [07:02] jtv: Debian's archive won't have its publish flag set, so you won't try to publish it. [07:02] What cron job disable/enable is that? [07:03] step 5 and 15 [07:03] that bracket the step syou're automating [07:03] wgrant: AIUI it doesn't matter whether we run this or not for Debian series. [07:03] jtv: Debian has nowhere to publish to, so it will fail if you try. [07:04] wgrant: you mean it doesn't have a publisher config? [07:04] jtv: Indeed, that is the case now. [07:06] lifeless: sorry if I left that bit out -- I was thinking to create this job in initialise-from-parent, so the run won't happen before the cron jobs are disabled. We'll want a notification to the distro owners in the case of success as well then, as one of the preconditions for re-enabling them. [07:06] jtv: We need to automate the disablement. [07:06] wgrant: you mean "yes it doesn't have a publisher config" or "yes it does have a publisher config"? (Asking because where I live, people tend to answer negative questions with "yes") [07:07] jtv: That is the case: it doesn't have a publisher config. [07:07] A month ago that wasn't the case. [07:07] Because there were no publisher configs. [07:07] wgrant: we ought to automate the whole bally lot of this. But that's for another day. I'm trying to solve a specific problem in such a way that it doesn't stand in the way of a wider solution. [07:07] So it would have happily published into completely the wrong place. [07:08] That's fine too, as long as it's not a place used by someone else. [07:08] I do wonder though whether a new non-Ubuntu, non-Debian distroseries will have a publisher config at this point. [07:08] If not, there's not much point to any of this. [07:09] Not right now, no. [07:09] But as soon as DDs are in use, yes. [07:09] as a DD, I object to that [07:09] we don't get 'used' [07:09] Ha ha. [07:09] Before I make the joke none of us wants to see me make... what's a DD? [07:09] debian developer [07:10] I don't think that's what wgrant meant... [07:10] I believe I meant Derivative Distros, but now I am not so sure. [07:11] Well this is _for_ derived distros (if you don't mind changing your words a bit) so will a new distroseries for a new derived distro have a publisher config right from the outset? [07:11] mind *me* changing your words, that is [07:12] jtv: A new distro had better have a publisher config before its series is initialised, yes. [07:12] If that job is not on the board yet, perhaps it should be. [07:13] Mm, not sure that's the case. [07:14] It's one of those things that can/will be done at the end once everything else is worked out. [07:14] It doesn't affect much else. [07:14] Unlike the other stuff that people seem to be pushing through excessively quickly :P [07:15] By the way, I suppose that the disabling/enabling the cron jobs can in future be done for just the distro. [07:15] Can't it already? [07:16] Don't think so, no. All the locking is still per-script. [07:17] Ah, assuming you're publishing multiple distros on a single machine, yeah. [07:17] Well otherwise there's not much difference, is there? :) [07:18] What do you mean? [07:18] There is one instance of each job for each distribution. [07:19] Yes, so disabling the cron jobs for all distributions on a machine would be pretty much the same thing as disabling the cron jobs for the one distribution on the machine. [07:20] By the way, we do have a ticket for making the scheduling of publisher jobs db-driven. [07:20] That'd be helpful in managing this. [07:20] Have you actually discussed how the scheduling is going to work? [07:20] And what the requirements for setting up a new distro are? [07:20] No, the scheduling thing is a lower-priority task. [07:21] The requirements for setting up a distro is effectively what we're discussing right now. [07:21] I imagine the Ubuntu procedure is about the best template we have for that. [07:22] The best template for what is necessary for creating a new series in an existing distribution. [07:22] Not for what we want creating a new series to be like, let alone creating a new distribution.' [07:25] grag [07:25] whats the bug # for users being confused about login.launchpad.net [07:29] jtv: I just remembered a great example about cache update latency [07:29] google used to run their search index in readonly mode [07:29] they would process newly scanned pages into a new index [07:29] and then run around their datacentres pivoting the new index into place [07:29] this was a fortnightly or something even [07:29] *event* [07:30] so a new page wasn't findable for weeks - as an expected thing [07:31] then, they made it transactionally updatable (they did a whitepaper on this, adding transaction abstraction to bigtable); and now scan pages directly into the live index, with sub-minute latency on event driven changes (like pingthesemanticweb) and 1/2-refresh-frequency for polled content [07:31] all about data structures [07:32] wgrant: yes, that is a big question right there. I have a feeling we'll either have to do another round of design (which will probably still miss things, I guess) or create a manual list similar to this one as we learn, and automate as a separate process. [07:36] lifeless: that's very much down to investing effort into the specific case though. There's often _also_ a lot to be gained from more generic partial solutions: partly as a stopgap while you direct the effort elsewhere, and partly as an actual solution for the more mundane problems. [07:37] jtv: perhaps. [07:38] For instance, caching can be bad in that it hides worst-case latency problems, but it can be good for reducing load-driven latency and a stopgap for average-case latency. [07:39] so, the latter two cases do make sense for judicious use of caching [07:40] however [07:40] identifying them is harder than you might think [07:40] wgrant: have you had a chance to look at the pythonic publish-ftpmaster again by the way? I'm really quite dependent on your judgment when it comes to Q/A. [07:41] jtv: at a guess, not till wed [07:41] jtv: its easter [07:41] + anzac day [07:41] jtv: lifeless speaks the truth. [07:42] When are we going to switch over to it? [07:42] I don't really have much idea of the timeframe of this whole thing. [07:42] anzac? aus&nz asomething csomething? [07:42] Other than suspicions that everyone's being very optimistic about timing. [07:42] wgrant: totally concur there, but I suspect the answer depends largely on "when wgrant is satisfied that it works properly." [07:43] jtv: australia new zealand armed core [07:43] jtv: gallipoli remembrance basically [07:43] Some kind of independence movement? [07:43] Joking, joking. [07:43] Perhaps not a good idea to joke about Gallipolli with the antipodeans... [07:43] I thought it was Army Core? [07:44] StevenK: probably [07:44] YUou mean Corps? [07:44] wgrant: I do [07:44] That makes slightly more sense. [07:44] <- tired [07:44] At 6:45pm? [07:44] StevenK: late night + early kitty-wake-up [07:44] A full corps would be a sensible unit for those population sizes, I guess. [07:49] lifeless, StevenK, wgrant: thanks for your help with this. The salient points have been recorded in notes. Which after an urgently approaching meal I shall start to turn into code. [07:50] jtv: I wish we had Julian this week :( [07:50] Having Julian is always good. === jtv is now known as jtv-eat === stub1 is now known as stub === jtv-eat is now known as jtv === almaisan-away is now known as al-maisan [09:03] wgrant: do you think anyone would mind terribly if I added an --all-suites option to publish-distro? [09:06] jtv: As opposed to omitting the '-s' option..? [09:06] wgrant: well I would have _expected_ omission to mean "all," but I don't see that in the code. [09:06] It does. [09:06] It leaves allowed_suites empty. [09:06] Is it hidden in getPublisher somehow? [09:07] Which means everything is allowed. [09:07] *somewhere [09:07] def isAllowed(self, distroseries, pocket): [09:07] return (not self.allowed_suites or [09:07] (distroseries.name, pocket) in self.allowed_suites) [09:07] if (self.allowed_suites and not (distroseries.name, pocket) in [09:07] self.allowed_suites): [09:07] self.log.debug( [09:07] "* Skipping %s/%s" % (distroseries.name, pocket.name)) [09:07] That's not in publishdistro.py, is it? [09:07] etc. [09:09] wgrant: intriguingly, I see different code for isAllowed. [09:09] Oh wait, you've got stuff after the return... what is that? [09:10] And more importantly, don't the publish-distro invocations in the Ubuntu release procedure manually specify all Ubuntu suites? [09:11] That plus the publishdistro.py code made me think that omitting -s couldn't mean "all suites" [09:12] jtv: The stuff after the return is from another method, doing a similar thing. [09:12] jtv: NRCP runs publish-distro for all suites in the new series. [09:12] Not all suites. [09:13] It passes "-s DSN -s DSN-updates -s DSN-security -s DSN-proposed -s DSN-backports" -- what's missing? [09:13] DSN-1, DNS-1-updates... [09:13] Ah OK. [09:13] Please don't imbue it with the knowledge of series. [09:14] It should operate on suites opaquely as far as is practical. [09:14] * jtv scribbles note [09:14] By the way, the distsroot that's passed for the partner archive looks to me like it's exactly what we would now get (thanks to archive configs, I take it) without the -R option. [09:15] We don't use .new? [09:15] Not there, no. [11:03] How to I get from a project page to its PPA? Do we have a link yet? [11:05] nm. projects can't specify an official ppa yet [11:25] and due to w----aacky UI choices its hard to it now [11:59] Morning, all. [12:03] hi deryck === al-maisan is now known as almaisan-away [12:36] morning deryck [12:49] mrevell, hi, we should have all the bits ready on production for feature review to start (a bit late because qa took longer than expected) [12:50] danilos, Superb. I'm working my way through the feature review notes at the mo. Ursinha should be around soon, too. [12:51] mrevell, cool === benji changed the topic of #launchpad-dev to: We have a 9s timeout!!! (2 months early!) | https://dev.launchpad.net/ | On call reviewer: benji | https://code.launchpad.net/launchpad-project/+activereviews [13:30] gary_poster: two will enter, one will leave [13:30] heh, but I will enter the wrong room [13:31] lol [13:35] where did jam go? :( [13:36] magcius: it's a holiday here in .nl [13:37] not in US [13:37] gary_poster: jam moved to Eindhoven in March [13:37] oh, cool! [13:37] * gary_poster mildly jealous :-) [13:38] gary_poster: there's no second easter day in the US? Do you at least get judgement day off? [13:39] heh, there's no first easter day off in the US. judgement day is still up in the air. we'll probably have off for the heat death of the universe. [13:40] In .au we have Tuesday off this year as well :D [13:40] heh :-P [13:41] wgrant: a third easter day? Don't they only have that in very catholic countries? [13:41] jelmer: ANZAC Day falls on Easter Monday this year, the so its public holiday is shifted to Tuesday. [13:41] So two Easter days plus ANZAC Day. [13:41] ah [13:46] gary_poster: I think I want a lib/lp/archivepublisher/configure.zcml, where there currently is none. Could you help with this? [13:48] jtv, sure, 1 sec, trying to tie something up [13:49] Thanks. I find one shoe at a time is easier. [13:50] :-) [13:51] (In this vague and poorly thought-out pun, I am the other shoe) [13:56] heh [13:56] jtv, ok, what's up [13:56] that is to say, what do you need help with? [13:57] Hi. I'm adding a job class in archivepublisher, and it has its own utility. [13:57] ok [13:57] So I reckon there ought to be a configure.zcml. [13:57] There is not. [13:57] lol, end of thought? [13:57] Get your dirty XML away from that innocent code! [13:58] gary_poster: I just discovered the zcml directory though. [13:59] I guess it goes in there now. [13:59] I don't think so [13:59] but maybe I'm wrong [13:59] I know the machinery, but I don't know our policies [13:59] Ah [14:00] Well there's a lib/lp/archivepublisher/zcml/configure.zcml that already has a utility... [14:00] oh *that* zcml directory [14:00] So at a minimum I suppose I wouldn't be making things substantially worse. [14:00] yeah that sounds fine [14:00] I thought you meant the top-level zcml directory [14:01] Whoooah no [14:01] :-) [14:01] Funny how these things always get rolling when you're explaining the problem. [14:01] :-) [14:01] ok, so you have found a happy home for your ZCML snippet, and all is well in this small corner of the world? [14:01] Yup. So thanks for, er, listening. :) [14:02] heh, yeah, np [14:38] benji, very small MP when you have a chance, please: https://code.launchpad.net/~gary/launchpad/bug770217/+merge/58958 [14:38] gary_poster: will do it now as I'm in the middle of a deep review that will take a while [14:39] ok thanks benji [14:47] gary_poster: if I only apply the patch to the test (and leave out the change to lib/lp/bugs/javascript/subscription.js, the new test doesn't fail [14:47] benji, huh, did for me [14:47] that's how I developed it anyway :-) [14:47] Lemme see... [14:47] I wonder if I did something wrong, double-checking. [14:51] gary_poster: my fault: I was running lib/lp/bugs/javascript/tests/test_subscriber.html instead of lib/lp/bugs/javascript/tests/test_subscription.html [14:51] benji, great. Thank you for checking [14:52] gary_poster: review done [14:53] thanks benji [14:53] my pleasure === salgado is now known as salgado-lunch [16:09] hello hello [16:10] Launchpad seems to be doing funny things in code hosting: http://paste.ubuntu.com/598761/ === deryck is now known as deryck[lunch] [16:18] beuno: I just confirmed I can push a new branch. Can you try pushing you branch with a letter at the start of the branch name [16:19] sinzui, I can, but branches i nPQM are failing for us now === Ursinha-afk is now known as Ursinha [16:20] I wonder if this related to thumper's change to branch ids to permit stable renames [16:20] beuno: I do not know how to test this kind of issue :( === salgado-lunch is now known as salgado === beuno is now known as beuno-lunch === beuno-lunch is now known as beuno [18:21] jcsackett: are available to mumble about question activity and email. My next step is pretty uncertain [18:43] sinzui: sorry, was brewing up coffee, missed your ping. i can mumble. [18:43] fab [18:45] jcsackett: can you hear me? [18:45] sinzui: i can hear you, i believe that you cannot hear me. i am rebooting mumble. [18:45] * sinzui too [19:15] has anyone online run windmill tests on natty? [19:16] I know wallyworld has, but anyone else? [19:18] benji, have a moment for https://code.launchpad.net/~gary/launchpad/bug770287/+merge/58979 ? Should be another short one. [19:18] (no) [19:18] gary_poster: sure [19:18] thanks benji [19:18] sorry, the "no" was to deryck [19:19] ("not me" would be more accurate :-) ) [19:19] heh, thanks for answering gary_poster :-) [19:19] :-) [19:19] I'm guessing no one has. [19:19] I'm guessing they are badly broken? === Ursinha is now known as Ursinha-lunch [19:22] gary_poster, a couple reporting on jenkins. I just want to verify a fix locally, but can't get it to run. [19:22] gotcha :-/ [19:23] No default or local profile error for windmill running FF 4. FF 4 doesn't have a profile template anymore. [19:24] gary_poster: done [19:24] benji, yippee! :-) thanks [20:05] jcsackett: ping [20:06] sinzui: pong. [20:09] morning [20:11] sinzui: hi, what's up? [20:12] jcsackett: did you update hide-comment.py to work with questions? [20:13] sinzui: i did locally, but it's periodically failing for reasons i haven't diagnosed; thanks for the reminder. i will prioritize that for the rest of today. [20:14] if you are encountering question spam, just assign to me. [20:14] I am updating my copy of the script now to close two questions. You may have already done this since I do not see spam comment on https://answers.launchpad.net/ubuntu/+question/144090 [20:16] sinzui: yes, i whacked spam from that earlier. [20:16] How do you know which comment to deactivate? [20:16] hide [20:16] i believe the question that points out the spam for that is already assigned to me, isn't it? i can update. [20:17] sinzui: right now? you have to count the messages. which is terrible. the ui work needs to add numbers to the message (a la bug comments). [20:17] okay. I will run my script [20:45] jcsackett: This is my updated script. Question comment silently fails: http://pastebin.ubuntu.com/598898/ [20:47] sinzui: fails as in you go to the page and the comment is still there? [20:48] if that's the case, wait a bit. i've found i was seeing delays between firing off setCommentVisibility and seeing it removed. [20:49] jcsackett: comment hiding is queued? [20:49] sinzui: i don't believe so. i haven't determined the cause of the delay. [20:50] jcsackett: were you anonymous...if so you get the cached page from the proxy [20:50] sinzui: i was not anonymous. [20:52] I was viewing the page logged in, then using an anon browser with a query string appended to ensure I do not get the cached [20:52] page [20:52] sinzui: ah, i see what you mean. [20:53] bug 691352 [20:56] sinzui: so, you are not observing anything happening after a delay, i take it. [20:59] jcsackett: that is correct [20:59] * jcsackett resists the urge to swear. [21:00] jcsackett: I can also state the comment is number 10000 [21:01] * jcsackett ceases resisting the urge. [21:01] I am pretty sure there are no questions with 10,000 comments [21:01] I already swore. [21:01] the latter is especially vexing, as you should get an index error on trying to do anything to comment 10000. [21:01] I have decided that I need to make questions faster so I will make that my top priority today [21:02] I have confirmed the script gets the correct question too [21:03] sinzui: i'll start a branch in a sec to see if the setCommentVisibility stuff became unbolted by something. [21:07] i will probably do my analysis far away from people, perhaps in a cave somewhere. [21:20] sinzui: hey [21:20] hi lifeless [21:21] I'm trying to find a bug [21:21] it was about how users get confused by login.launchpad.net [21:21] I think bug 770107 is a dup of it [21:21] canonical-identity-provider === lifeless changed the topic of #launchpad-dev to: Performance Tuesday | https://dev.launchpad.net/ | On call reviewer: benji | https://code.launchpad.net/launchpad-project/+activereviews [21:21] sinzui: thats the project [21:21] I thought we had a task on LP [21:21] but I can't find the darn thing [21:21] I was wondering if you remembered such a bug? [21:22] I am not seeing it either [21:22] But I think I can find a question that links to it === Ursinha-lunch is now known as Ursinha [21:28] lifeless: I am not hunting my email. This is infuriating [21:28] sinzui: s/not/now/ ? [21:28] yes [21:29] :) makes more sense [21:39] lifeless: I give up. I cannot find a login.launchpad.net bug in c-i-p or a login.ubuntu.com bug in launchpad. [21:54] thanks for looking === benji changed the topic of #launchpad-dev to: Performance Tuesday | https://dev.launchpad.net/ | On call reviewer: - | https://code.launchpad.net/launchpad-project/+activereviews [22:18] flacoste: ping [22:21] sinzui: my change wouldn't have effected beuno's branch like that [22:22] fab [22:57] I really wish we had full bug federation [23:01] sinzui: did you mean to /part #launchpad ? [23:02] No, emapathy is being a complete and utter prick [23:02] :( [23:02] its not being empathetic :) [23:09] Well I cannot workout where to report bugs about that launchpad-bugzilla, so I unconfigured the bugtracker. But since we are the maintainers, shouldn't we be tracking bugs in Lp? [23:22] Looks like empathy want me to be in one freenode channel at a time [23:23] lifeless: http://pastebin.ubuntu.com/598960/ [23:23] ^ My thoughts are really scattered [23:26] Seems reasonable [23:26] the snapshot things seems overly generic to me [23:26] its one of those attractive until you use it patterns [23:29] lifeless: I prefer putting the snaptshot/copy of the current question into the event. Let a subscribed function queue a job with the question and the event as metadata. [23:29] sinzui: sure; I think that the json should be quite targeted though. [23:30] lifeless: I am unsure now if QuestionNotification needs to change if I build something that Implements IQuestion, but is acutally using values instead of references. [23:31] sinzui: I'm confused about the implementing of IQuestion [23:32] lifeless: I was looking a PersonTransferJob that requires json-compatible values. Instead of keep a reference to the reviewer that is optional, the job has a property that instanties the user from the id in the metadata [23:33] lifeless: I do not want to build a schema for the job that create lots of references to other objects. I think the question plus a dict of values will work most of the time. Some of those values need to turned back into users, bugs, faqs and messages. [23:34] lifeless: I cannot put the snapshot into the db, so I need a light way to store the changes [23:35] sinzui: AIUI other deferred mailers [23:35] I think create a ro-implementation of question that uses values instead of objects to make the email may work [23:35] they generate the message [23:35] or a template for th message [23:35] and a list of subscribers [23:35] and they shove that into the queue to be processed on the backend [23:35] Well I want to put an end to timeouts and thoses tasks you mentioned are part of the time out [23:35] right [23:36] I'm just saying that /most/ or even /all/ of the objects you're mentioning are not needed [23:36] personMemebershipJob an PersonMergeJob do all the email building, themselves [23:36] but perhaps I misunderstand the problem [23:36] I think you do [23:36] As I said, I cannot get my thoughts in order [23:37] AIUI the work required is: [23:37] - update the db to show the event [23:37] - build a message to send to each of [long list of subscribers], and its customised per subscriber [23:38] - spool and send the message [23:38] How do I queue the event [23:38] QuestionNotification does th last two points perfectly [23:38] I was thinking you wouldn't queue the event at all [23:39] but generate the common message text /once/ [23:39] and then queue a job to do a mail-merge of the subscribers & the mail template [23:39] but I haven't looked at this code [23:39] if what I describe sounds harder or worse, you should ignore me :) [23:45] lifeless: That is doable by extracting the QuestionNotification.getBody() implementation to the subscriber function I envision. I think there are a few nuances though. headers and footers assume the current question state is correct, but infact, the state of things like assignee or target may have several changes in the same jobqueue [23:45] Thus I started to envision a very elaborate way of serialising the existing state. [23:46] wouldn't the state of things like assignee or target be captured when building the template to use for all subscribers ? [23:46] not as explicit fields, but as things injected to said template [23:48] The build op needs to create the common body, common header and common footer. We merge the to addresses, and the header/footer reasons later> [23:48] ? [23:52] lifeless: I think the X-Launchpad-Question and the body must be queued with the question since those describe the current state. Other headers, and the footer can created using the current question and its current subscribers [23:52] sinzui: sounds reasonable to me [23:53] So I think the schema is (question, body, header) header can be a string at this moment. It could be a serialised dict for extension.