[09:55] <BigBangUDR> Hello all need help regarding kernel panic in 14.04 issue - http://imgur.com/a/3oKAQ
[10:10] <apw> BigBangUDR, that looks to be 4.4.0-72.93 ... the latest updates kernel is 4.4.0-87.110 have you tried that ?
[10:10] <BigBangUDR> I am new and not able to boot or get terminal
[10:11] <apw> do you have any other kernels installed, i assume there was another before that one broke you
[10:11] <apw> so as it crashed and did not boot, you should get a menu next time you boot
[10:11] <apw> which lets you select other older kernels
[10:11] <apw> i would recommend picking the previous one to this one if it is the latest installed
[10:12] <BigBangUDR> Try all kernal options available in grub menu none worked all went to panic 
[10:12] <BigBangUDR> even recovery mode
[10:13] <apw> oh your init is broken
[10:13] <apw> so yes all kernels will blow up, as it is simply sying "init exited"
[10:15] <apw> so you could edit the command line and try adding 'init=/bin/sh' to the end and see if you get a prompt then
[10:15] <apw> of course it will have root mounted read only etc
[10:19] <BigBangUDR> apw: i have another ubuntu with multi hard disk support, can i mount and do something
[10:20] <apw> i would try the init thing first
[10:20] <apw> as you can remount it r/w and ask apt what the heck is up with init
[10:21] <apw> as it looks like someone has removed libcap2 from teh system
[10:21] <apw> from the error you have, which is a hard dependency of systemd
[10:21] <apw> which is init in this context
[10:21] <BigBangUDR> apw: i am real new to this kernal things dont have any idea what needs to be excuted there
[10:22] <apw> i am saying select the latest kernel in grub, hit the e key to edit the entry
[10:22] <apw> and then cursor down to the line with linux at the front, and add init=/bin/sh to the end of the line
[10:22] <apw> then hit ctrl-x to execute it
[10:23] <apw> https://askubuntu.com/questions/19486/how-do-i-add-a-kernel-boot-parameter
[10:23] <apw> if that doesn't panic we have some hope of being able to recover from this
[10:23] <apw> is this is a server or desktop install ?
[10:23] <BigBangUDR> apw: Okay this is server install
[10:24] <apw> what was occuring when this started, as losing that file without systemd being removed
[10:24] <apw> implies the actual file was removed without removing the package
[10:24] <apw> so either someone with admin or a harward/software glitch that happened to remove/corrupt that file
[10:25] <BigBangUDR> This is our node development in-house server, i monitor it
[10:26] <BigBangUDR> we usually add and remove lots of programmes 
[10:26] <BigBangUDR> am trying your init solutions
[10:27] <apw> it isn't a solution, it is an attempt to get you to a point where you can repair this thing
[10:27] <BigBangUDR> Yes 
[10:27] <apw> it will not be up in any sense someone can use it from anywhere from the control
[10:27] <apw> console
[10:33] <BigBangUDR> apw: got the terminal
[10:33] <apw> then you have some hope
[10:33] <apw> then you need to make / writable
[10:34] <BigBangUDR> how to do it?
[10:34] <apw> note once you have done this you need to sync it manually at the end otherwise it might not be on disk
[10:34] <apw> this is very important
[10:34] <apw> mount -o remount,rw /
[10:34] <apw> but remember you can make a huge poo mess of this if you do not sync it at the end
[10:34] <apw> which you do in teh traditional way
[10:34] <apw> sync;sync;sync
[10:36] <apw> next we want to find out if you have libcap2 installed
[10:36] <apw> dpkg -l | grep libpcap2
[10:37] <apw> dpkg -l | grep libcap2
[10:37] <BigBangUDR> mount not working
[10:37] <apw> what does it say
[10:37] <BigBangUDR> error while loading shared libraries : libblkid.so.1
[10:37] <apw> does ls work ?
[10:38] <BigBangUDR> lls
[10:38] <BigBangUDR> yes
[10:38] <apw> ls -l /lib/*/libcap.so.2
[10:39] <apw> does that list any files ?
[10:39] <apw> ls -l /lib/*/libcap.so.2*
[10:39] <apw> and indeed does that
[10:40] <BigBangUDR> no such file or directory
[10:40] <apw> for both ?
[10:41] <apw> ls /lib 
[10:41] <apw> ?
[10:41] <BigBangUDR> ls it display all libraries
[10:42] <apw> dpkg -l | grep libcap2 
[10:42] <apw> does that work ?
[10:43] <BigBangUDR> yes it display some result like libcap2-bin and more
[10:44] <apw> dpkg --verify libcap2
[10:44] <apw> what does that say
[10:44] <BigBangUDR> its saying ??5????? and a path
[10:45] <apw> so you have lost that file, what path is it
[10:45] <apw> approximatly is close enough
[10:46] <apw> and what is the letter in the second column
[10:47] <BigBangUDR> This is the path  /lib/x86_64linux-gnu/libcap.so.2.24
[10:47] <apw> what does
[10:47] <apw> and ls of that directory short
[10:47] <apw> show
[10:47] <apw> ie is there anything else in there, or are all of your arch specific libraries gone
[10:48] <apw> as we know you are missing blkid already
[10:48] <apw> ...
[10:48] <apw> though regardless of that because you have lost your blkid library you are not going to be able
[10:48] <BigBangUDR> yes right
[10:48] <apw> to recover this from here easily as your disk is not converable to rw to fix
[10:49] <apw> so normally in this situation i would make and boot from a usb stick
[10:49] <apw> then mount up the root disk manually and chroot into it
[10:49] <apw> and see if i could reinstall the packages which are clearly broken
[10:50] <BigBangUDR> It throw the same error while boot from USB
[10:50] <apw> that cannot be, as the usb stick has its own libraries
[10:50] <apw> that sounds more like you are failing to boot the stick
[10:50] <BigBangUDR> Okay I'll try one more time
[10:51] <BigBangUDR> Hope it will work this time.
[10:51] <apw> once you have mounted root, and chrooted into it
[10:51] <apw> (remember to bind mount /sys and /proc into it)
[10:52] <BigBangUDR> Okay
[10:52] <apw> then you can try apt-get install --reinstall libcap2 libblkid1
[10:52] <apw> and see if that works
[10:52] <apw> of course you have little idea if anytyhing else got dammaged by whatever broke you
[10:53] <BigBangUDR> Sure I go through it and try to recover it. 
[10:53] <ogra_> are you sure thats not a failed disk in the raid ? 
[10:54] <ogra_> (looks like there is a md raid in use)
[10:54] <BigBangUDR> Yes
[10:55] <apw> if there is any risk that you have lost a disk in your raid you ought to take a backup of course
[10:55] <apw> well probabally you want to do that anyhow
[10:56] <BigBangUDR> Right
[10:56] <apw> ogra_, though as the disk has mounted clean i assume a disk loss would have had to have occured when the machine was not booted
[10:56] <apw> not impossible of course
[10:57] <BigBangUDR> Try to mount it with the help of another disk but it failed and dont show all data which is available earlier 
[10:58]  * ogra_ would actually boot with break=bottom and inspect the raid from there (also gives you accurate logs, i guess any interesting errors are scrolled off screen anyway)
[10:59] <BigBangUDR> Still throw the same error kernal error while boot from USB
[11:00] <apw> what usb stick image are you booting
[11:00] <apw> and is the output identicle
[11:00] <BigBangUDR> Might be some problem with its version.  
[11:01] <apw> whos version?
[11:01] <BigBangUDR> version 16.04
[11:01] <apw> those images are commonly used, so i doubt it
[11:01] <apw> what kernel version is shown in that panic
[11:01] <apw> when booting from usb
[11:01] <ogra_> which image are you using(whats the download URL yu used)
[11:04] <BigBangUDR> version 14
[11:04] <ogra_> ??
[11:04] <ogra_> thats not a url 
[11:04] <ogra_> what exactly did you download for your usb stick
[11:05] <ogra_> surely any imgaes you can download from releases.ubuntu.com will bot standalone
[11:05] <ogra_> *boot
[11:05] <apw> i asusme that is mean to be a kernel version, but its not one of those
[11:05] <apw> if the panic looks similar, then is says "Kernel Panic -...."
[11:06] <apw> and then two lines below on the right end of the CPU: 0 line it has a full kernel version/flavour/build number
[11:06] <apw> it is that i am after
[11:06] <BigBangUDR> https://www.ubuntu.com/download/desktop/thank-you?country=IN&version=16.04.2&architecture=amd64
[11:06] <BigBangUDR> This is the url
[11:07] <ogra_> how did you write it to the USB ?
[11:07] <ogra_> and what is rthe exact error you see when booting from it then
[11:09]  * apw boots that CD to find out what kernel is on it
[11:09] <apw> BigBangUDR, what was that kernel version when booting USB, or upload a picture of that too
[11:11] <BigBangUDR> kernal version is 4.053442
[11:11] <apw> nope
[11:12] <apw> that isn't a kernel version, they don't look like that
[11:12] <apw> 4.11.0-11-generic #16-Ubuntu
[11:12] <apw> they are shaped like that (from my machine here)
[11:12] <apw> shaped like that in a panic output
[11:14] <ogra_> 16.04 should have something like 4.4.0-*
[11:15] <ogra_> (though that is 16.04.2 ... that probably already has the hwe kernel )
[11:18] <BigBangUDR> Okay trying to chroot and bind 
[11:19] <apw> managed to get the usb image booted ?
[11:19] <BigBangUDR> yes
[11:20] <BigBangUDR> chroot / right?
[11:20] <apw> nope
[11:20] <apw> you need to bind before you chroot
[11:20] <apw> and then you chroot into wherever you mounted your real /
[11:20] <apw> chroot /mnt /bin/bash
[11:20] <apw> sort of thing
[11:23] <BigBangUDR> have mounted the disk at /mnt/kw
[11:23] <apw> then there
[11:23] <apw> mount -o bind /sys /mnt/kw/sys
[11:23] <apw> mount -o bind /proc /mnt/kw/proc
[11:23] <apw> chroot /mnt/kw /bin/bash
[11:23] <apw> and see if you can reinstall that libcap2 package
[11:23] <apw> if you can, you may have some hope
[11:24] <BigBangUDR> chroot: failed to run command ‘/bin/bash’: Exec format error
[11:24] <apw> try /bin/sh
[11:25] <apw> is the installed image also amd64 ?
[11:25] <BigBangUDR> chroot: failed to run command ‘/bin/sh’: Exec format error
[11:26] <apw> what does "file /mnt/kw/bin/sh" say
[11:27] <BigBangUDR>  /mnt/kw/bin/sh: symbolic link to `dash'
[11:28] <apw> what does "file /mnt/kw/bin/dsh" say
[11:28] <apw> what does "file /mnt/kw/bin/dash" say even
[11:28] <BigBangUDR>  /mnt/kw/bin/dash: ELF 64-bit LSB  shared object, x86-64, version 1 (SYSV), dynamically linked (uses shared libs), for GNU/Linux 2.6.32, BuildID[sha1]=504637666875a5d526ef51acfe601c99efc99114, stripped
[11:31] <ogra_> and you definitely downloaded ubuntu-16.04.2-desktop-amd64.iso ? are you sure it is not i386.iso ?
[11:31] <ogra_> what does: uname -m
[11:32] <ogra_> say ? 
[11:32] <BigBangUDR> Ohh sorry my bad it says i686
[11:33] <ogra_> i686 ? 
[11:33] <BigBangUDR> yes
[11:33] <ogra_> where does it say that 
[11:33] <BigBangUDR> uname -m
[11:34] <ogra_> apw, is that what we say on i386 installs ? 
[11:34]  * ogra_ wou8ld have expected x86 ... like amd64 says x86_64
[11:34] <apw> yes, that is what we say on i386, this amd64 box says x86_64
[11:34] <ogra_> yeah, wrong iso then
[11:35] <ogra_> http://releases.ubuntu.com/16.04.2/ubuntu-16.04.2-desktop-amd64.iso
[11:35] <ogra_> try this image
[11:36] <BigBangUDR> Okay let me download
[11:38] <BigBangUDR> ogra_: corrupted one is server edition
[11:39] <ogra_> there is no difference (apart from one shipping a desktop, they are the same underneath)
[11:39] <ogra_> but feel free to grab the server iso from the above url instead
[11:40] <BigBangUDR> Okay
[12:12]  * apw assumes BigBangUDR succeeded
[12:20] <BigBangUDR> apw not yet but in very good direction thanks to you
[12:22] <BigBangUDR> repeating all i have learn from you.
[15:29] <michael-vb> apw: Michael from the VirtualBox team here.  I was talking to LocutusOfBorg on #vbox-dev about how to deal with the VirtualBox kernel drivers (the host ones) and secure boot.
[15:29] <michael-vb> He said I should talk to you and thought you might be have plan to include them in the Ubuntu kernel package.
[15:42] <sforshee> michael-vb: we already have been including them in Ubuntu kernel packages for quite some time now
[15:42] <michael-vb> The host ones?
[15:42] <sforshee> oops, missed that part. No, not the host ones
[15:43] <michael-vb> See for example LP 1574300.
[15:44] <michael-vb> And it applies to our upstream builds as well of course.
[15:45] <sforshee> yeah, currently the only solutions are to disable secure boot or use install a mok which is used to sign dkms modules. I think cyphermox was planning to work on a user-friendly way to do the latter.
[15:45] <cyphermox> working on it, I'm testing code at the moment
[15:46] <michael-vb> Sounds interesting, any pointers?
[15:46] <cyphermox> when the module gets built though, you should currently be getting a prompt to disable Secure Boot
[15:49] <cyphermox> in other words, when you see the DKMS logs at package install (or upgrade), provided that you're in UEFI mode you should also be seeing a prompt come up to ask if you want to disable Secure Boot, and the pick a password -- you do want to do that
[15:51] <michael-vb> We no longer use dkms upstream.  Not all distributions supported it, and using it for some but not all made no sense.
[16:00] <LocutusOfBorg> I'm back with my flaky connection
[16:03] <michael-vb> Just looked at mokutil.
[16:04] <LocutusOfBorg> I didn't probably get the answer from kernel team :)
[16:05] <michael-vb> mokutil seems to be the answer in short form.
[16:06] <michael-vb> The long form might be, well, longer.
[16:07] <michael-vb> sforshee, cyphermox: thanks.
[16:07] <michael-vb> (I might be back some time...)
[19:21] <cyphermox> michael-vb: AFAIK what dkms fails is at least on Fedora, but that's because AIUI they decided to not care
[19:21] <cyphermox> fwiw, you can use update-secureboot-policy --disable to do the same, but a little more user-friendly maybe
[19:22] <cyphermox> and soon this might be rendered moot, since you'll be able to sign the modules a bit more easily
[19:22] <cyphermox> currently, you need to use mokutil & openssl to generate a key and enroll it, reboot, finish the enrollment, then use kmodsign to sign, if you manage to guess what parameters it wants :)
[19:23] <socratis> cyphermox: michael-vb has left the building a couple of hours ago, but I could "deliver" your message... 
[19:23] <cyphermox> socratis: thanks..
[19:24] <cyphermox> sorry, I knew something was up when my tab-completion didn't work, but I never got a part/quit message here
[19:25] <socratis> :D
[19:25] <socratis> I just happened to follow michael from the #vbox-dev channel, just to see where the conversation was going, and I ... stayed ;)
[19:27] <socratis> Anyway, if I see him tomorrow, I'll paas it along. Just out of curiosity, you mentioned that the self-signing will be rendered easier soon-enough, or did I miss something?
[19:31] <cyphermox> yes, that's what I'm working on atm
[19:32] <cyphermox> I don't know if you work on virtualbox or if you're just interested because; but the idea is that dkms should work, so we need to make sure some kind of signature happens, and that it remains secure so as not to diminish the value of secureboot
[19:34] <cyphermox> for vbox, it's not quite as critical (ie your system still works) but some wireless/ethernet drivers or even graphics make use of dkms, so it's nice when your computer still works :)
[19:35] <socratis> No, I'm not a developer (yet), I'm seriously interested. As far as I know, dkms was dropped because it wasn't working across platforms/distros.
[19:35] <socratis> But I don't want to possibly drive you to the wrong path.
[19:36] <socratis> As I said, my knowledge is surface only (if that), so I'll let the developers deal with it. Thank you so much in any event!
[20:13]  * LocutusOfBorg forwards things to michael-vb