[01:15] <StevenK> wgrant: http://pastebin.ubuntu.com/5688104/
[01:33] <wgrant> StevenK: Why does LP_JS_BUILD depend on YUI_DEFAULT_SYMLINK, and why does YUI_DEFAULT_SYMLINK depend on YUI2_BUILD?
[01:33] <StevenK> wgrant: So that we have one rule that will do all population via dependency resolution
[01:34] <wgrant> StevenK: Isn't that jsbuild?
[01:35] <StevenK> wgrant: I can expand it out into jsbuild if you wish
[01:35] <wgrant> StevenK: That seems better than having random deps like you have now
[01:38] <StevenK> wgrant: http://pastebin.ubuntu.com/5688136/
[01:40] <wgrant> StevenK: That seems more sensible, though it looks like you can probably now merge JS_OUT, JS_ALL and JS_BUILD
[01:41] <wgrant> Er
[01:41] <wgrant> s/JS_BUILD/jsbuild/
[01:41] <StevenK> JS_ALL and JS_OUT can not be merged
[01:42] <StevenK> The JS_ALL rule can be flattened
[01:42] <wgrant> They can be merged, because we never used launchpad.js now
[01:43] <wgrant> I don't think support YUI <3.5, and we only support 3.5 when it's comboloaded
[01:44] <StevenK> wgrant: You want me to rip out generation of launchpad.js?
[01:44] <StevenK> And yes, launchpad.js only supports YUI 3.3.0
[01:45] <wgrant> StevenK: Given you're doing this rework now, and it will be purely deleting code...
[01:45] <wgrant> 3.3 support was only kept so we could do a gradual migration to 3.5
[01:45] <wgrant> That migration is done
[01:45] <wgrant> Multi-version support should probably be preserved for future upgrades, but we don't need to preserve the non-comboloader approach
[01:46] <StevenK> js.combo_loader.enabled can die too
[01:46] <wgrant> Yes
[01:46] <wgrant> And the code that it disables
[01:46] <wgrant> Which is probably just two lines in the root template
[01:46] <StevenK> Yeah
[02:04] <StevenK> wgrant: It's a bit more than two lines
[02:04] <StevenK> % bzr di lib/lp/app/templates/base-layout-macros.pt | diffstat -s
[02:04] <StevenK>  1 file changed, 1 insertion(+), 20 deletions(-)
[02:06] <wgrant> :)
[02:08] <StevenK> Grrrr
[02:08] <StevenK> combine-css depends on jsbuild implicitly, since it wants all of the css files in icing
[02:14] <wgrant> StevenK: Can you do the CSS copying outside jsbuild?
[02:31] <StevenK> wgrant: Sorry, where jsbuild means bin/jsbuild
[02:44] <StevenK> wgrant: http://pastebin.ubuntu.com/5688275/
[03:03] <wgrant> StevenK: Doesn't js.yui-version want to stay?
[03:03] <wgrant> You can also drop YUI_VERSIONS down to just 3.5.1
[03:07] <StevenK> wgrant: js.yui-version isn't referenced anywhere else
[03:07] <StevenK> wgrant: And I can not.
[03:08] <StevenK> YUI 3.3.0 is linked into icing, and combo.css uses one of the files
[03:08] <wgrant>     @property
[03:08] <wgrant>     def yui_version(self):
[03:08] <wgrant>         """The version of YUI we are using."""
[03:08] <wgrant>         value = getFeatureFlag('js.yui_version')
[03:08] <StevenK> Haha
[03:08] <wgrant> StevenK: There's no corresponding file in 3.5.1?
[03:08] <StevenK> Note the disparity
[03:08] <wgrant> Indeed
[03:08] <wgrant> The name in the docs was wrong
[03:12] <StevenK> wgrant: Fixed the docs
[03:12] <StevenK> wgrant: cssreset/reset.css 3.3.0 and 3.5.1 has a slight difference
[03:40] <StevenK> wgrant: http://pastebin.ubuntu.com/5688372/
[04:13] <wgrant> StevenK: YUI_DEFAULT_SYMLINK probably wants a -n, and won't combo.css now get the 3.5.1 reset/grid CSS because of the default symlink change?
[04:14] <wgrant> Actually, we may still be using a copy of the 3.0.0 grid CSS...
[04:14] <wgrant> Yeah
[04:14] <wgrant> So it might just be reset that we care about
[04:15] <wgrant> Baaaah
[04:18] <StevenK> wgrant: Hmm?
[04:19] <wgrant> StevenK: Tried to reproduce the process-upload logging unicode issue this morning
[04:19] <wgrant> Finally realised what I was doing wrong now
[04:19] <wgrant> It's failing to write to the file stream, not the stderr steam
[04:19] <wgrant> stream
[04:19] <StevenK> wgrant: So the 3.5.1 link is okay in terms of reset.css?
[04:19] <wgrant> StevenK: I don't know; we'll want to test
[04:19] <wgrant> But it's probably fine.
[04:20] <StevenK> wgrant: I've fixed the -n for DEFAULT_SYMLINK. Shall I push this mess up?
[04:21] <wgrant> StevenK: Sounds like a plan
[04:21] <wgrant> I've wanted to fix this for a while :)
[04:22] <StevenK> wgrant: Which, the process-upload, or the YUI building thing that I got annoyed enough to take an axe too?
[04:22] <wgrant> StevenK: Both, but I was talking about the latter.
[04:32] <StevenK> wgrant: https://code.launchpad.net/~stevenk/launchpad/silence-yui-build/+merge/157272
[04:46] <wgrant> StevenK: Hmm
[04:46] <wgrant> StevenK: I think we need to always run lpjsmin, don't we?
[04:46] <StevenK> wgrant: Why?
[04:46] <wgrant> The launchpad.js call was guarded before, but the combo-rootdir one was not
[04:46] <StevenK> Yeah
[04:46] <wgrant> Because launchpad.js was either minified or not
[04:47] <wgrant> Whereas for the comboloader it produces separate .min versions
[04:47] <StevenK> It does, yes
[04:47] <wgrant> So we probably just want to always run it
[04:47] <wgrant> Unless there's a reason not to?
[04:47] <StevenK> Then the variable just dies
[04:47] <wgrant> Exactly
[04:47] <wgrant> And we remove more dev/prod crap from the makefile :)
[04:52] <StevenK> wgrant: Do we want to sprinkle in YUI 3.9.1 after this branch?
[04:52] <wgrant> StevenK: I imagine there'll be some porting work, but we should at least see how many tests break
[04:52] <wgrant> The 3.5 upgrade wasn't that bad, apart from the comboloader
[04:52] <StevenK> wgrant: 3.9.1 in YUI_VERSIONS will blow up tests?
[04:52] <wgrant> I've been meaning to modernise our Zope stack for a while
[04:53] <wgrant> StevenK: No, but once it's there we'll want to run the tests over it
[04:53] <wgrant> I guess that just means changing the default, jsbuild, then run the tests?
[04:53] <StevenK> Yeah
[04:53] <StevenK> I can add it to download-cache and prep a throwaway branch to toss at ec2 to do so
[04:54] <wgrant> I wouldn't bother ec2ing it. The tests don't take long to run locally.
[04:54] <wgrant> And easier to debug
[04:55] <wgrant> Then we can hopefully migrate to the new calendar and kill off yui3.3
[04:55] <StevenK> And 2
[04:55] <wgrant> Er
[04:55] <wgrant> That's what I meant, yeah
[04:55] <wgrant> 3.3's already dead :)
[04:55] <StevenK> :-)
[04:56] <StevenK> Not until you approve my branch it isn't
[04:56] <wgrant> Ah, didn't realise JS_BUILD had perished.
[04:57] <StevenK> I got distracted by URL-hacking to find a 3.9.1 zip that I didn't tell you I'd pushed
[04:57] <wgrant> I got distracted by failing to find a YUI3 changelog
[04:57] <StevenK> http://yuilibrary.com/download/yui3/ doesn't mention 3.9.1, but the zip file does exist
[04:59] <wgrant> The 3.6.0, 3.7.0, 3.8.0 and 3.9.0 announcements look safe enough
[04:59] <wgrant> So there shouldn't be much fallout, AFAICT
[05:00] <StevenK> wgrant: http://pastebin.ubuntu.com/5688475/
[05:01] <wgrant> StevenK: Leave 3.3.0 so current devel is buildable..
[05:02] <wgrant> But 3.3 and 3.4.1 can go
[05:02] <StevenK> wgrant: That's 3.3.0.zip
[05:02] <wgrant> Orright
[05:02] <StevenK> -rw-rw-r-- 1 steven steven 12M Jul 17  2012 yui_3.3.0.zip
[05:02] <StevenK> -rw-rw-r-- 1 steven steven 33M Jul 17  2012 yui_3.5.1.zip
[05:02] <StevenK> -rw-rw-r-- 1 steven steven 30M Apr  8 14:57 yui_3.9.1.zip
[05:02] <wgrant> :)
[05:03]  * StevenK watches bzr upload 30MiB
[05:04] <StevenK> wgrant: Can haz +1?
[05:05] <wgrant> StevenK: Is icing/yui's sole remaining purpose to feed the CSS combiner?
[05:05] <StevenK> Yes
[05:05] <wgrant> Yay
[05:05] <wgrant> r=me
[05:06] <StevenK> We could grab reset.css and drop that requirement
[05:06] <wgrant> Let's not do that right now
[05:06] <wgrant> This branch avoids evil so far
[05:07] <StevenK> Avoids? It outright removes it
[05:24] <StevenK> wgrant: -vvm javascript, or which tests were you thinking?
[05:24] <wgrant> StevenK: -vv --layer YUI
[05:28] <StevenK> wgrant: They're all failing with AssertionError: JS timed out.  The test may never have started.
[05:29] <StevenK> Oh, not all of them
[05:29] <wgrant> StevenK: Some of them may reference files that don't exist any more. Try running the failures in a browser
[05:29] <wgrant> Also make sure you've actually jsbuilt.
[05:30] <StevenK> Failure in /home/steven/launchpad/lp-branches/test-391/lib/lp/registry/javascript/sharing/tests/test_sharingdetails.html.Sharing Details.test_display_error_bug: Unexpected error: 'undefined' is not a function
[05:30] <StevenK> Passed tests:     93
[05:30] <StevenK> Failed tests:      7
[05:30] <wgrant> That's not bad, considering how much that page does
[05:31] <wgrant> How many test files fail vs pass?
[05:31] <StevenK> 91.1% pass, 6.9% fail
[05:31] <StevenK> 102 total
[05:32] <wgrant> Great.
[05:32] <StevenK> And 2 skipped, for some reason
[05:32] <wgrant> That's from testr?
[05:32] <StevenK> bin/test -vv --layer YUI --subunit, and then I ran subunit-stats over the stream
[05:32] <wgrant> Right
[05:33] <wgrant> So, sounds like the upgrade will be easy.
[05:33] <wgrant> Feel like finishing it off now?
[05:34] <StevenK> I don't think we'll jump to it by default, but I can fix the tests
[05:35] <wgrant> yeah, we'll want to trial it ourselves for at least a few days, but I think we can probably get the tests fixed and landed today
[05:35] <wgrant> Unless something terrible happens
[05:35] <StevenK> The YUIXHR tests all fail
[05:35] <StevenK> lib/lp/app/javascript/choiceedit/tests/test_choiceedit.html passes in my browser, but failed under bin/test
[05:35] <wgrant> THey're fragile at the best of times
[05:36] <wgrant> Do they even pass with 3.3?
[05:36] <wgrant> Or 3.5?
[05:36] <wgrant> Hmm
[05:36] <StevenK> I thought buildbot ran them?
[05:36] <wgrant> Yeah
[05:36] <wgrant> But I mean on your system
[05:39] <StevenK> devel gives the same failures against 3.5.1
[05:39] <StevenK> (WRT yuixhr)
[05:40] <wgrant> Right
[05:40] <wgrant> What about 3.3?
[05:44] <StevenK> wgrant: 3.3.0 (IE devel r-1) works
[05:47] <StevenK> Ah ha, they die on buildbot too
[05:48] <wgrant> Indeed. Do you want to look at that, or do you want to look at the eg. choiceedit failures while I poke at YUIXHR?
[05:49] <StevenK> wgrant: I'll see if I can work out the other 4 if you grab YUIXHR
[05:49] <wgrant> kk
[08:15] <czajkowski> jtv: does this sound right to tyou ??? https://bugs.launchpad.net/launchpad/+bug/1162192
[08:15] <_mup_> Bug #1162192: Partial pot-files destroy translations <lp-translations> <Launchpad itself:New> < https://launchpad.net/bugs/1162192 >
[08:26] <jtv> czajkowski: hi.  No, that does not sound right to me.
[08:27] <jtv> Maybe somebody got clever and decided we can delete TTIs instead of setting their sequence number to zero.
[08:27] <czajkowski> StevenK: wgrant has anything been done to translations in the last week or so ?
[08:28] <wgrant> jtv, czajkowski: I discussed that with danilos last week
[08:28] <jtv> TTI stands for TranslationTemplateItem.  It's basically an entry that says "this particular translatable message, or POTMsgSet, occurs in this particular template, and here's its message number in that template."
[08:28] <wgrant> It regressed in mid-2011
[08:28] <wgrant> When a POTMsgSet pruner was added
[08:28] <wgrant> It also prunes TTIs with sequence == 0
[08:28] <jtv> !
[08:28] <wgrant> Which I believe is insane
[08:28] <wgrant> But I was hoping to get input from you or danilos on that :)
[08:28] <czajkowski> jtv: morning btw :)
[08:29] <jtv> When a message was once in a template but no longer is, we keep it and set its sequence number to zero (not null unfortunately, for historical reasons) exactly so that it can be restored easily, with all its translations.
[08:30] <jtv> So wgrant, here's my input: yes that's insane.  :)
[08:30] <wgrant> jtv: That's what I thought, yeah. http://bazaar.launchpad.net/~launchpad-pqm/launchpad/devel/view/head:/lib/lp/scripts/garbo.py#L1230 is the code
[08:31] <jtv> There should be at least a grace period.
[08:31] <wgrant> Until last week I assumed that it just pruned POTMsgSets
[08:31] <wgrant> But it prunes obsolete TTIs too
[08:31] <wgrant> Yes
[08:31] <wgrant> That would be more reasonable :)
[08:31] <wgrant> But I'm more inclined to just not prune obsolete TTIs at all
[08:31] <wgrant> There shouldn't be that many of them
[08:31] <jtv> Well you can't prune POTMsgSets without getting rid of the TTIs.
[08:32] <wgrant> Well, POTMsgSets from deleted templates will get pruned
[08:32] <wgrant> But that's about it
[08:32] <wgrant> Though can you even delete templates...
[08:32] <jtv> You can mark them as deactivated, not delete them.
[08:32] <jtv> Or you can merge them into other templates.
[08:32] <wgrant> So perhaps we want to set a date when we obsolete a TTI
[08:33] <wgrant> Then kill them after 6 months or something
[08:34] <jtv> And then only prune POTMsgSets that don't have any TTIs.
[08:34] <wgrant> Well, FKs should ensure that.
[08:34] <wgrant> But yes
[08:35] <jtv> I mean: the two can be entirely separate processes.
[08:35] <jtv> Where one _only_ prunes long-unused TTIs, and the other prunes unreferenced POTMsgSets.
[08:35] <wgrant> Oh, I thought that's what it did already, but you're right.
[08:35] <wgrant> It is entirely TTI-driven
[08:36] <wgrant> I hadn't read the code beyond seeing the sequence == 0 bit and crying.
[08:36] <jtv> I'll bet the join is complicated by having to deal with both sequence == 0 *and* there being no TTI.
[08:36] <jtv> I wanted null for the unused sequence number.  Would have made it easy to sort them first or last as desired, for one.
[08:36] <wgrant> Though it's certainly faster to be TTI-driven, but there are few enough POTMsgSets that it might not matter
[08:37] <jtv> I would also expect obsolete POTMsgSets etc. to pile up slowly enough that it's more a matter of a one-off cleanup followed by reruns at glacial intervals.
[08:39] <wgrant> I think it currently runs daily mostly because there is no less frequent schedule
[08:39] <wgrant> But I was going to introduce a garbo-weekly or garbo-monthly for this
[08:40] <jtv> Nice.
[08:41] <jtv> For those jobs where the cost of checking for cruft vastly outweighs the cost of cleaning it up.
[08:41] <wgrant> Yep
[08:42] <wgrant> And the cost of the cruft is so minimal, except over huge intervals.
[08:45] <wgrant> StevenK: buildbot's just run the first yuixhr test, and it succeeded. the milestone one needed another YUI 3.5 fix, but I think they're all good in devel now. Hopefully that'll fix your issue with 3.9
[08:49] <wgrant> Huh
[08:49] <wgrant> The other 3 tests failed.
[08:50] <wgrant> Still pass locally :/
[08:55] <wgrant> Even in a clean branch