[16:56] <cmagina> a commit that was pulled into the recent trusty sru via lp1419838 broke booting on arm64. the fix is already upstream and in the ubuntu/linux tree as commit: d6ad36913083d683aad4e02e53580c995f1a6ede what is the process for getting that in as soon as possible?
[16:57] <apw> bjf, henrix ^
[16:57] <henrix> and kamal :)
[16:57]  * henrix goes look at the bug
[16:59] <henrix> ah, it's a stable update
[16:59] <cmagina> yeah, sorry, guess i should have left off the lp part so mup grabbed it
[16:59] <henrix> cmagina: a new SRU cycle will start next week
[17:00] <apw> bug #1419838
[17:00] <ubot5> bug 1419838 in linux (Ubuntu) "Trusty update to v3.13.11-ckt15 stable release" [Undecided,Confirmed] https://launchpad.net/bugs/1419838
[17:00] <cmagina> henrix: today is the last day for commits, correct?
[17:00] <henrix> cmagina: if we have that fix in, it will take ~3 weeks to be released.  is that soon enough?
[17:00] <cmagina> henrix: well, it currently breaks all arm64 boards that install the updates kernel
[17:00] <cmagina> they won't boot
[17:01] <henrix> yeah, a nasty one :-/
[17:01] <henrix> bjf: ?
[17:02] <apw> cmagina, do these board have a button to boot a previous krenel or are they bricks after this
[17:02] <cmagina> apw: not easily, they all use uboot. so requires a rescue environment
[17:03] <cmagina> so, pxeboot a working kernel and fix it there type deal
[17:04] <apw> cmagina, and that id in updates or proposed
[17:04] <apw> is
[17:04] <henrix> apw: updates
[17:04] <henrix> i was just checking that :)
[17:04] <henrix> so, it has been already released
[17:04] <cmagina> yeah, we missed it in our testing
[17:06] <cmagina> our infrastructure has been getting moved around
[17:06] <apw> as in we didn't do any i assume ?
[17:07] <cmagina> yeah, the tests that verified the proposed kernels was disabled
[17:07] <cmagina> were
[17:07] <cmagina> fun of moving labs
[17:09] <henrix> cmagina: do we have a bug # for that already?  let's wait for bjf and see what he thinks
[17:09] <cmagina> henrix: i will submit one
[17:09] <henrix> cmagina: thanks
[17:10] <apw> cmagina, and only trusty yes ?
[17:10]  * bjf reading
[17:10] <kamal> fyi, the offending commit was ported to 3.16 also
[17:11] <cmagina> apw: i have not tried utopic yet, its on my list
[17:11] <apw> did we get the fix above as well ?
[17:12] <kamal> ah, I see the fix got applied to 3.16
[17:13] <kamal> ... but hasn't yet appeared in utopic.
[17:14] <henrix> kamal: yeah, both commits are in 3.16 stable, but none has hit utopic yet
[17:14] <bjf> henrix, apw, cmagina are we only talking about Trusty at this point?
[17:15] <kamal> henrix, lucky!!
[17:15] <cmagina> bjf: thus far, i need to test utopic
[17:15] <apw> bjf, i think kamal is just saying that utopic has been spared
[17:15] <henrix> kamal: :)
[17:15] <kamal> bjf, yes, this looks like it should only affect trusty at this time
[17:15] <kamal> though cmagina is welcome to test it anyway
[17:17] <kamal> bjf, the fix commit is trivial.  I'll apply it to 3.13-stable immediately:
[17:17] <kamal> d6ad369 clocksource: arch_timer: Only use the virtual counter (CNTVCT) on arm64
[17:18] <bjf> kamal, henrix, cmagina, apw, i think we need to get this fixed and get that fix out asap 
[17:19] <bjf> kamal, henrix, cmagina, apw, put simply, we spin trusty with just this fix
[17:19] <kamal> bjf, I'm okay with that
[17:19] <kamal> cmagina is filing a bug (right now?) ... we'll need that bug number.
[17:20] <cmagina> kamal: yes, i'll get that now
[17:20] <bjf> kamal, henrix, cmagina, apw, however, we need to get the testing of these sorted out so we don't end up here again
[17:21] <henrix> would this also affect lts-backport-trusty?  since it's arm64, i guess not...
[17:21] <henrix> cmagina: ^
[17:21] <henrix> bjf: ^
[17:22] <apw> there is arm64 arch in trusty 
[17:23] <apw> but not in precise, so 
[17:23] <apw> i assume that means spinnin ghte lts-backport is pointless for this
[17:23] <henrix> cool
[17:24] <cmagina> kamal: bug 1426043
[17:24] <ubot5> bug 1426043 in linux (Ubuntu) "kernel 3.13.0-46.75 fails to boot on arm64" [Undecided,New] https://launchpad.net/bugs/1426043
[17:24] <kamal> cmagina, thanks much
[17:26] <bjf> apw, cmagina do we know if we use any of this HW for buildds ?
[17:28] <cmagina> bjf: i do not believe so. we did donate a board as a porter
[17:29] <cmagina> dannf: did we donate any of our arm64 hardware as buildd's?
[17:29] <dannf> bjf: yes, but they don't run ubuntu's kernel
[17:33] <bjf> dannf, we are not running our own kernels on our buildds ?
[17:35] <dannf> bjf: nope - and no technical reason ttbofmk. i send gentle pokes about that every month or so
[17:36] <bjf> dannf, do you know where the kernels come from that they are using?
[17:36] <dannf> bjf: yes - prebuilt binaries from apm
[17:37] <dannf> from their bsp (which worked before ours did)
[17:39] <dannf> to switch, they'll require a firmware update and initializing flash-kernel - i'm guessing that's just a man-effort issue
[17:40] <kamal> bjf, apw: I just sent a [Trusty SRU PATCH] request for this ^^ to kernel-team@  ...  awaiting some ack's.
[17:40] <kamal> Subject:	[Trusty SRU PATCH] clocksource: arch_timer: Only use the virtual counter (CNTVCT) on arm64
[17:49] <cmagina> kamal: what would the fixed kernel version be if thats possible to know at this point? that way i can get the word out to not reboot unless running that kernel
[17:50] <kamal> cmagina, it will be 3.13.0-46.76
[17:51] <cmagina> kamal: thanks
[19:39] <mssbrg> is virgin/linux-stable an exact mirror of mainline? what is the diff b/t virgin/linux-stable and virgin/linux?
[20:07] <kamal> mssbrg, virgin/linux.git is a mirror of mainline.  virgin/linux-stable.git is a mirror of the kernel.org stable repo (it includes all their stable branches, e.g. linux-3.19.y)
[23:06] <kamal> cmagina, still around?  could we get you to verify that our freshly baked kernel fixes the ARM64 boot problem?
[23:13] <bjf> dannf, ^ you available?
[23:40] <dannf> bjf: yeah - working on it. making sure i can reproduce w/ the old before trying the new
[23:40] <dannf> kamal: ^
[23:43] <bjf> dannf, thanks!