=== erkules_ is now known as erkules === bigcat is now known as bigcat[otp] === bigcat[otp] is now known as bigcat === chihchun_afk is now known as chihchun [06:55] pitti: ping. I think I have a test that is taking long to reboot. But there's no way to increase the reboot timeout in adt-run, right? [06:58] elopio: no, not right now; mind filing a bug for this? [06:58] pitti: sure. [07:02] pitti: https://bugs.launchpad.net/ubuntu/+source/autopkgtest/+bug/1488358 [07:03] Launchpad bug 1488358 in autopkgtest (Ubuntu) "Can't increase the reboot timeout" [Undecided,New] [07:04] mvo: could you look into https://bugs.launchpad.net/snappy/+bug/1488186 ? [07:04] Launchpad bug 1488186 in Snappy "Failed to update: gnutls_handshake() failed" [Undecided,New] [07:04] it's causing like 50% of the failures in jenkins. === chihchun is now known as chihchun_afk [07:07] * elopio goes to bed. See you tomorrow. [07:16] hey elopio, sure, I have a look [07:16] elopio: sleep well [07:17] ppisati: hey, for later (no rush) I would love to hear your opinion on https://bugs.launchpad.net/snappy/+bug/1480248 and if it needs to be re-assigned to the kernel or something else [07:17] Launchpad bug 1480248 in Snappy "automatic reboot fails with zero size initrd in bbb" [Critical,Triaged] [07:21] mvo: so the actual bug is that it hangs there instead of panicing [07:21] mvo: weird, let me see [07:34] elopio: asked some questions in the bug [07:40] mvo: why the watchdog doesn't kick in and reboot the board? [07:42] ppisati: AFAIK we don't use the watchdog, how can we enable it, that would be a nice fix indeed [07:43] mvo: that should be enabled in uboot [07:43] ppisati: I have the board in this state sitting on my desk if you want me to test anything [07:43] mvo: so if we don't reach the userspace in 60secs, it reboots the board [07:43] mvo: so, i did a couple of tests [07:43] ppisati: nice, is that a uboot config option or code? [07:43] mvo: yep [07:43] mvo: let me do a couple more checks and i'll comment in the lp bug [07:44] great, thanks [07:44] * ppisati needs more coffee first [07:55] elopio: I debug the issue a bit but so far nothing conclusive, maybe worth trying to run a curl test on the jenkins server (see bugreport) [08:59] mvo: I commented on #1480404, i think both front and backend need work [09:02] ta [09:03] ppisati, hmm, why in u-boot ? i thought thats a kernel feature [09:07] ogra_: i guess it needs enabling from kernel commandline to be useful? [09:07] * Chipaca guesses [09:07] well, we have panic=-1 by default there [09:08] panic= [KNL] Kernel behaviour on panic: delay [09:08] ... [09:08] timeout < 0: reboot immediately [09:08] ... is what the kernel docs say [09:14] ogra_: i didn't look at the option for the watchdog on the cmdline [09:14] well, we dont set nmi_watchdog=1 [09:14] ogra_: but AFAIK a userspace sw open /dev/watchdog [09:15] but we do set panic=-1 [09:15] ogra_: and have to reopen it before TIMEOUT expires [09:15] which shoud make the kernel reboot i think [09:15] etc [09:15] ogra_: moreover, what happens if thekernel hangs before the wdt is enabled? [09:15] the correct init should be: [09:16] uboot sets the wdt to X and start booting the board [09:16] i thought the nmi_watchdog is kernel only [09:16] kernel takes over and, either reach usrspace (and wdt is reset) [09:16] or uboot settings reset the board [09:17] BOOTPARAM_SOFTLOCKUP_PANIC and BOOTPARAM_HARDLOCKUP_PANIC shoudl provide that if i can belive the docs [09:17] anyhow, i spent the morning looking at uboot [09:17] and it seems the wdt is not supported there [09:19] ppisati, but why does the normal panic option not kick in here ? [09:19] ogra_: nmi_watchdog? [09:19] no, we dont have that on currently [09:19] just the normal panic= stuff [09:20] and it's x86 only [09:20] nmi_watchdog= [KNL,BUGS=X86] [09:20] he sayd he doesn't get any panic [09:20] oh, right [09:20] damn [09:20] just hangs there [09:20] oh, i see why :P [09:20] Waiting for root device /dev/disk/by-label/system-b... [09:21] no udev ... no /dev/disk/by-label/system-b ... [09:21] the kernel itself doesnt set these links [09:21] yeah, thats the symptom, but why does it not complain if initrd.img is invalid? [09:22] do we pass rootwait? [09:22] depends on the board [09:22] mvo: it doesn't get the initrd at all [09:22] if it is in the default env we dont wipe it [09:22] we dont add it explicitly iirc [09:22] mvo: if you pass a malformed initrd, like 1 byte, it says "bad format" or something [09:22] mvo: but it doesn't panic there either [09:23] i guess we could add a check to uboot (and switch system_ab based on it) [09:23] what really bugs me is that uboot says "watchdog enabled" but i can't fucking get that thing to work [09:23] and IMO it doesn't [09:24] i.e. look up the size and make sure it isnt zero after loading the initrd [09:24] but technically having the kernel fall over would be cleaner [09:25] is there any kernel option for "dont boot without initrd" ? [09:26] not that i am aware [09:26] mvo: do you have the rootwait option? [09:26] mvo: on the cmdline i mean [09:27] ppisati: yes, "Kernel command line: console=ttyO0,115200n8 root=/dev/disk/by-label/system-b init=/lib/systemd/systemd ro panic=-1 fixrtc rootfstype=ext4 rootwait" [09:27] ok [09:27] quick test [09:27] bah, why do we set rootfstype [09:27] /* wait for any asynchronous scanning to complete */ [09:27] if ((ROOT_DEV == 0) && root_wait) { [09:27] printk(KERN_INFO "Waiting for root device %s...\n", [09:27] saved_root_name); [09:27] ... [09:27] if loops there waiting for tghe rootdevice if root_wait = true [09:27] yeah [09:28] can you try to take that option out? [09:28] neither rootfstype nor rootwait should be set [09:28] thats comeing from the default env for the BBB [10:15] mvo, hmm, can it be that our bbb changes never actually landed in snappy-hub/snappy-systems ? [10:16] ogra_: I certainly send a MP [10:16] ogra_: but I haven't tracked (due to sprints/confs) if it actually landed :/ [10:16] https://code.launchpad.net/~mvo/snappy-hub/bbb-env/+merge/264975 [10:16] * ogra_ approves [10:16] (hopin thats the actual code we used for the last image :P ) [10:17] line 90 is the issue for the bog above [10:18] *bug [10:21] elopio: question: is there a reason _integration-tests/bin/ and _integration-tests/data/output aren't in bzrignore? [10:21] ogra_: thanks, branch updated [10:22] ogra_: cool, so if I remove the rootwait it will work? [10:22] ogra_: as a quick test? [10:22] mvo, you should test it, but it is very likely, yes [10:22] ogra_: does that also fixes the "1" byte file initrd.img ? or is that a separate bug ? [10:23] * mvo tests once that bbb finishes booting [10:23] just intercept the boot in uboot and set mmcrootfstype to nothing or just to ext4 or so [10:23] the 1byte initrd will also try to fall back to mount root= directly ... (and fail) the outcome should be the same [10:24] ogra_: cool, I set this now via fw_setenv and see what happens [10:24] +1 [10:24] nice panic [10:24] and reboot too ? [10:25] yes [10:25] yay ! [10:25] awesome [10:25] I try the one byte file now just for good measure [10:26] yeah [10:26] ogra_: I assume you are on the updated snappy-systems branch? [10:26] i would be surpised if it wouldnt do the same [10:26] i dont have my BBB powered/wired up atm ... [10:26] the Rpi is (on a local copy, since we dont have the Pi upstream) [10:27] ogra_: ok, I can work on the branch if you want, just don't want to dupliicate work :) [10:27] ah, k [10:27] same for 1 byte [10:27] yay [10:27] ppisati, ^^^ all fine then [10:31] ogra_: cool, but i still want to understand why the f*ck watchdog didn't reboot from uboot [10:32] 15.04 is go 1.3, right? [10:32] yes [10:33] ogra_: https://code.launchpad.net/~mvo/snappy-hub/lp1480248-norootwait/+merge/269032 [10:34] mvo, approved [10:34] ogra_: I'm doing a final test with a good initrd.img (takes forever because of the sync mount option) [10:34] ogra_: thanks [10:35] the sync mount option ? [10:35] ogra_: I copied the good initrd back and that cp takes a long time [10:35] ogra_: because we mount /boot sync [10:35] ah, right [10:35] ogra_: but all good, booted fine from b [10:35] i thought the boot takes forever :) [10:38] rsalveti: you around? [10:44] * ogra_ is afk for a bit (need to get a new laptop for susie ... her lenovo blew up the DC board last night ... out of the blue ... a month after the warranty is gone ... crap HW...) [10:49] * mvo tests bbb update snap [11:04] someone will need to approve my updated bbb snap [11:04] * mvo gets lunch first [11:58] hi all, snappy-remote keeps timing out on me [11:59] any ideas? I can easily ssh right into the BBB [12:00] I am unfamiliar with snappy-remote but if it has something to do with GitHub they are mitigating DDoS attack and restoring service which causes git pulls etc. to fail for me unrelated to snappy. [12:01] Mikaela, thanks for your answer, that is interesting [12:01] but, snappy-remote is just a wrapper around ssh, I think, to push and install your snap package [12:02] it looks like it is trying to do some kind of key exchange that is failing [12:02] rickspencer3, using an IP or webdm.local ? [12:02] ogra_, webdm.local [12:02] and ssh to webdm.local works from the same machine ? [12:05] * rickspencer3 tries with straight ip [12:05] ogra_, ssh works fine, yeah (using webdm.local) [12:06] and using the ip address with snappy remote worked fined as well [12:06] aha [12:06] I assume that I need to grant permissions for the snap to use the gpio pins? [12:07] you most likely need to grant access to teh respectiver /dev node with hw-assign [12:07] ogra_, is there a place where I can look up how to do that? [12:08] the webcam tutorial ... [12:08] https://developer.ubuntu.com/en/snappy/guides/appliance-builder-guide-webcam/ [12:08] you should actually see complaints in syslog about device access [12:29] ogra_, does this tell me how to use hw-assign? [12:32] nm, I figured it out [12:32] :) [12:53] ogra_: is lp #1429749 sill a issue? i.e. that the rpi2 is not updating ? aiui this was only a problem with early rpi2 images, right? [12:53] Launchpad bug 1429749 in Snappy trunk "Ubuntu Core updated but not switch to new version after reboot on Raspberry Pi 2" [Undecided,New] https://launchpad.net/bugs/1429749 [12:53] is there a Go library for gpio that is known to work with BBB and snappy? [13:06] hello hello [13:07] * sergiusens is in back to normal mode [13:09] hey sergiusens, good monring! [13:09] sergiusens: do you happen to know if we have a branch for snappy remote 0.4 somewhere? [13:10] sergiusens: I was looking into a issue that rickspencer3 reported earlier about slow webdm.local resolving [13:10] mvo: how is it related to snappy remote? [13:11] mvo: snappy-remote just uses avahi indirectly when using ssh [13:11] mvo: you probably want to up the broadcast of IPs in webdm itself, I think [13:11] sergiusens: I wanted to check what snappy-remote is doing and if we could cache the ip there for example [13:11] mvo: oh, it's just using 'ssh' [13:11] sergiusens: aha, that sounds like a better suggestions, thanks [13:12] sergiusens: yeah, I found the source for 0.3 but was wondering if 0.4 has changed anything, looks like bzr might need a bzr push with the latest changes :) [13:12] (or I looked at the wrong branch :/) [13:12] mvo: yeah, I might have forgotten to push and I'll need to power up my older laptop to get it :-/ [13:13] will do in a bit; the 0.4 thing only does some key magic [13:13] no worries, I can manually import from the deb if its trouble to find the old laptop [13:48] ogra_, ppisati: bug #1458866 is another one of the failover robustness that would be nice to get input on. this time its about a correpted dtb. could we do something smart if snappy_mode is try and it fails to read the dtb? [13:48] Bug #1458866: hangs in uboot boot prompt if dtbs dir is missing [13:48] bug 1458866 in Snappy "hangs in uboot boot prompt if dtbs dir is missing" [Undecided,New] https://launchpad.net/bugs/1458866 [13:58] lool: re bug #1459755 - could we simply unconditionally enable serail grub in our generic-amd64 snap? or is there a downside to this? [13:58] Bug #1459755: Need a way to drive GRUB over serial port [13:58] bug 1459755 in Snappy "Need a way to drive GRUB over serial port" [Undecided,New] https://launchpad.net/bugs/1459755 [14:39] elopio: hi, can we get click-reviewers-tools in the testbed? this triggered a issue in the intergration tests that caused the test failure for the snappy-review-tools-reneable branch [14:52] ppisati, ogra_: I subscribed you to some more of the failover bugs, would be nice to get your input (not urgent though) [14:52] sergiusens, any idea why azure needs rootwait ? that shouldnt be necessary (and is usually unused, since the initrd handled waiting itself) [14:52] *handles [14:53] do the azure images have a special initrd or some such ? [14:53] ogra_: iirc, rsalveti added it [14:54] right, but why ? [14:54] rootwait is only helpful when booting without initrd ... and in our case it is actually harmful :) [14:54] ogra_: azure needs it [14:54] because of azure :-) [14:54] or better, because of reasons [14:54] ogra_: ah, it's in there docs [14:54] lol [14:54] as a requirement [14:55] i dont get that [14:55] it should be a total no-op if you use an initrd [14:55] (apart from breaking our auto-panic ) [14:56] do we have any azure devs we could ask ? [14:56] for $reasons [14:56] ogra_: check our priv channel [14:56] ah, i need to re-loacate [14:56] * ogra_ goes to the office [14:58] mvo: we could, but it's always hard to install things in snappy. We could copy the python script, or we could make a snap. But did you see my comment here? [14:58] https://code.launchpad.net/~mvo/snappy/snappy-review-tools-reenable/+merge/264301/comments/676317 [14:59] elopio: aha, ok, sorry, I didn't - brain was in quick triage mode apparently :/ [15:01] mvo: wow, you triaged the world. Thanks! [15:06] elopio: don't make it sound like such a chore, it's easy, "The World" is broken, global warming, war, deforestation, over heating, yellowstone get more active, the list goes on ;) [15:45] davmor2: snappying the world it's such a chore. [15:45] barry: we started getting this error: https://bugs.launchpad.net/snappy/+bug/1488186 [15:45] Launchpad bug 1488186 in Snappy "Failed to update: gnutls_handshake() failed" [High,Triaged] [15:45] barry: any idea what could cause it? [15:48] elopio: i don't. i'll subscribe to the bug, but it seems like it's a lower level tls problem. is the server perhaps load balanced and maybe there's a problem there? [15:48] barry: who manages the server? I'm not sure whom to ask about that. [15:50] elopio: probably is, but slangasek has access === chihchun_afk is now known as chihchun === chihchun is now known as chihchun_afk [16:02] elopio, barry: I don't have any access to the web frontends; that's IS-managed [16:09] elopio: I guess a rt ticket is the best way forward at this point [16:11] mvo: ok, on it. [16:12] Chipaca: we wanted to move _integration-tests/bin/ and _integration-tests/data/output out of the tree, but didn't get to it yet. [16:12] a .bzrignore in the mean time seems appropriate. [16:25] ricmm_, how do you resize the partition on the kvm image ... seems gparted doesnt like it [16:25] ogra_: gparted offers to fix the partition table header for you [16:25] then it shows fine as unallocated space [16:26] hmm, doesnt here [16:26] then sfdisk actually does the job correctly when doing 2.26 [16:26] didnt offer you to fix? [16:26] I appended zeros to the image file first [16:26] ah, i was trying to shrink writable [16:27] so I'm trying right now with flashing the image as-is (4G) onto flash and see if that works [16:27] or if theres the same error as on kvm [16:28] i'd expect the same error with the old sfdisk [16:32] ricmm_, hmm, no errors on rolling ... but no resize either [16:33] http://paste.ubuntu.com/12193628/ [16:34] * ogra_ appends moar zeros [16:36] hmm, it sees that the disk grew bigger [16:37] ogra_: it gret 1.3 MB for you [16:37] yet you had added 500M of zeros [16:37] mmm [16:37] i added another GB [16:38] and it properly sees the device is 5GB now [16:38] but it doesnt resize [16:38] (doesnt error either) [16:44] Chipaca: https://code.launchpad.net/~elopio/snappy/bzr_ignore_integration_output/+merge/269095 [16:53] ogra_: hmmm it worked for me [16:53] did you open it with gparted at some point? [16:53] no [16:53] open it with gparted, lets try to understand what kind of fix that is doing [16:53] i added 1.5G [16:53] it will offer to "fix" the partitioning [16:53] yea but added it how [16:53] just appending zeros? [16:54] dd [16:54] yeah [16:54] dd oflag=append conv=notrunc if=/dev/zero of=kvm-rolling.img bs=1MB count=1000 [16:54] and before the same with 500 [16:54] now i'm in an initrd shell ... [16:54] lol [16:54] trying sfdisk there manually say the GPT is damaged and will be fixed on next write ... [16:55] yea wily didnt boot for me on real hw [16:55] right, thats the error that gparted fixes [16:55] (on purpose, i added break=premount to the cmdlijne) [16:55] well, but sfdisk tells me it will fix it on next write ... i tried to write it but that didnt help it seems [16:55] hmm [16:55] perhaps i need a reboot or re-read the partition table at the kernel level [16:56] hmm, nope [16:56] blockdev --rereadpt doesnt change a thing [16:59] sigh [17:00] i dont think there is a way around using parted [17:00] slangasek, do you happen to have any idea about teaching sfdisk about GPT ? (it works fine for all MBR based images just not for the amd64 one now) [17:01] well, not teaching it about it but making it repair the GPT to make it possible to resize the writable partition [17:05] ogra_: sfdisk knows GPT [17:05] thats not the problem [17:05] yes [17:05] thats why i wrote the second line :) [17:05] the problem is that when cloning disk images like this the secondary GPT header (footer) is misaligned [17:05] its not at the physical end of disk [17:05] kyrofa: nice video. Did you publish the source for your snap somewhere? [17:05] right [17:05] ogra_: parted can fix this I think but there should be an easier way to do so for GPT [17:05] maybe we can teach sfdisk to do so [17:05] :) [17:06] elopio, thanks! Yeah, it's here: https://github.com/kyrofa/piglowtop [17:06] elopio, for snappy-specific stuff, see the snappy readme (meta/readme) [17:06] well, sfdisk even *tells* me it will fix it "on next w(rite)" [17:06] but it seemingly doesnt [17:06] kyrofa: thanks! [17:06] ogra_: but did it do an actual write? how about read the partition as-is then write it again [17:06] it might realign it to end of disk [17:08] oh damn ! [17:08] i'm blind [17:08] so after printing a lot of info when i do the write ... it then says "use gparted to correct the GPT errors" [17:08] i missed that :P [17:09] sigh [17:10] so that silly GPT stuff will make me re-write the whole thing then :((( to use parted and bloat the initrd [17:10] well not necesarily [17:10] we can try to understand what gparted is doing to fix the stuff [17:10] and just teach sfdisk about it [17:10] if we want to avoid the bloat... [17:10] how big is this bloat anyways? why does it concern you so much [17:10] i guess thats a lot more effort [17:11] dunno, likely a few MB [17:11] it pulls in all of ncurses at least ... probably even more stuff (i never checked, ncurses was already enough to make me run away) [17:12] the code change alone is probably small [17:12] ogra_: ricmm_ parted should be compilable without curses [17:12] sergiusens, it is linked against it [17:12] ogra_: you need to do multibuild in the deb package [17:12] so copy_exec from initramfs-tools pulls in the libs [17:13] (copy_exec reversely runs ldd against all binaries and copies all deps in) [17:14] and i surely dont want to re-package parted :) [17:18] ogra_: if (q == PED_EXCEPTION_FIX) [17:18] { [17:18] last_usable = last_usable_if_grown; [17:18] /* clear the old backup gpt header */ [17:18] ptt_clear_sectors (disk->dev, [17:18] gpt_disk_data->AlternateLBA, 1); [17:18] gpt_disk_data->AlternateLBA = disk->dev->length - 1; [17:18] *update_needed = 1; [17:18] } [17:18] doesnt look that bad to teach sfdisk about it perhaps ;D [17:19] hmm [17:19] i think upstream sfdisk is even at 2.28 ... perhaps someone cared already :) [17:20] perhaps [17:20] ah, no, 2.27 ... [17:20] 2.28 is in development [17:20] yea 227 [17:22] hah ... they added json dumps :P [17:23] of course they did [17:23] going to rest my disk now [17:29] ogra_: https://github.com/karelzak/util-linux/commit/9d9a1b876094fe38c5539f19a57323437b8b8a0d [17:29] looks relevant [17:30] well, thats only checks [17:31] i was hoping for relocation code :) [17:45] so seemingly we could use gdisk ... but that has a similar dependency chain as parted [17:54] ricmm ogra_: then maybe the idea of using cloud-init is back up for discussion? [17:54] sergiusens, cloud-init ???? [17:54] you want to pull cloud-init into the initrd ? [17:55] * ogra_ wants cloud-init gone completely, its a PITA [17:55] and i surely dont want a python interpreter in the initrd :) [17:55] ogra_: cloud-init does growfs outside of init [17:55] initrd that is [17:55] sergiusens, with the whole bind mount farm active on top of the writable part ?? [17:56] thats like gambling :) [17:56] the only sane way to resize is from the initrd before we have the bind mounts active [17:57] (even before anything gets mounted at all for extra safety) [17:58] sergiusens, i'll pull parted in before this discussion even comes to speed [17:58] I'd like not have a bunch of python scripts meant for dispoable cloud instances running on unreachable hardware that is meant to stand the test of time (10 yrs) [17:58] doing partitioning that is [17:58] :) === ricmm_ is now known as ricmm [17:58] :) [18:01] Chipaca or mvo and elopio I am ready for slaughter https://code.launchpad.net/~sergiusens/snapcraft/meta-all-yaml/+merge/269100 :-) [18:26] sergiusens: I'll start looking at it after lunch. [18:26] sergiusens: could you look at https://code.launchpad.net/~elopio/snappy/remove_ssh_copy/+merge/268947 ? [18:27] elopio: sure [18:30] * mvo also needs review for some branches [18:47] ricmm, sfdisk -d /dev/sda|sfdisk /dev/sda ... that makes the error go away but doesnt allow me to resize anymore ... http://paste.ubuntu.com/12194329/ [18:47] ogra_: sfdisk: libfdisk/src/table.c:420: new_freespace: Assertion `end > start' failed. [18:47] looks like an error to me ! [18:48] well, it doesnt complain about the GPT anymore [18:48] right but it fails at "creating" the freespace definition [18:48] in other words I'll bet the GPT footer is still wrongly positioned [18:48] because it didnt get the padding from the unallocated space [18:49] mvo: bug #1459755: perhaps we can do this for now, but it's not a long-term solution as the serial port might be used for e.g. an UPS or whatever [18:49] Bug #1459755: Need a way to drive GRUB over serial port [18:49] bug 1459755 in Snappy "Need a way to drive GRUB over serial port" [Medium,Triaged] https://launchpad.net/bugs/1459755 [18:50] mvo: I guess in theory it ties into our hardware assignment story [18:50] but this is probably too remote for now [18:50] mvo: seems like we are in bargaining mode :-P [18:50] mvo: in any case, we need to set the right linux console [18:51] mvo: a config flag would be fine though; something like /writable/etc/grub.d/zzzSetserial.conf could be generated perhaps [18:52] ricmm, yeah, i'll switch to parted ... that just means a lot more math i was hoping to avoid [18:52] lool: we don't parse and generate grub.cfg anymore ;-) It goes straight into the snap [18:52] so it can be whatever you want [18:53] sergiusens: right; it doesn't really relate to what we used to do, but just to a config-time place to set bootloader options [18:54] that's why I thought of the single-instance writable partition which has some etc stuff rather than dealing with system-a/system-b stuff [18:55] lool: if it is a fixed function device problem it can certainly have its own gadget/oem snap defining the console [18:55] sergiusens: yes, that's exactly what I have in mind [18:55] if not, I prefer a snappy config for ubuntu-core (or the gadget) [18:55] although the gadget snap is not configured, it handles passing down config :-) [18:55] lool: great [18:55] sergiusens: I mean, we need to provide the config hooks to turn on serial console in a couple of places, and they need to set it [18:56] sergiusens: so typically one would set the config from gadget snap, and it would set config options provided by the kernel snap I guess [18:57] of course, there's a risk that walking down the path of adding config options makes us reinvent every single of GRUB's options [18:57] and here we can already see the difference between the GRUB notion of a serial console and the linux one [18:57] are we going to abstract it away for folks to consume [18:57] e.g. ubuntu-core/serial-console=ttyS0 and then map that to the right GRUB and Linux serial devices [18:58] or will we provide kernel/console=(or extra-cmdline=)/dev/ttyS0 and bootloader/serial-console=ttyS0 [18:58] etc. [18:59] ogra_: [18:59] /* check that the backup header is properly placed */ [18:59]