=== StarHeart is now known as Edgan
[01:35] <pycurious> Is anyone using kubernetes on ubuntu on multiple clouds here? 
[01:36] <pycurious> Or perhaps the canonical offering?
=== genii is now known as genii-core
=== Droid is now known as Mekaneck
[06:50] <PMunch> Hi everyone, I recently updated my server through a standard apt upgrade and now it won't boot any longer. It seems to hang on "Scanning for BTRFS file system", then it starts kernel panicking because various tasks blocked for more than 120 seconds. Anyone know what might be up? Or what I could try to get the server back up?
[06:54] <TJ-> PMunch: boot to an older kernel, or in recovery mode
[06:56] <PMunch> Tried recovery mode without luck, I'll see if I can get an older kernel working
[07:15] <PMunch> Okay, booting from an older kernel worked
[07:16] <PMunch> Now just to make that change stick..
[07:42] <PMunch> Okay, 5.4.0-80 works, 5.4.0-81 fails, in case anyone was curious
[09:28] <lotuspsychje> PMunch: i dont see a recent bug on your errors right away, could you file a bug on that?
[12:22] <PMunch> Hmm, I'm not entirely sure what the actual bug is though
[12:22] <PMunch> @lotuspsychje, 
=== THEGODSQUIRREL is now known as thegodsquirrel
=== jinky is now known as jink
[12:55] <tomreyn> PMunch: Maybe there are logs of the failed boot?     journalctl -b-1     would give you the logs from last but one boot (i.e. the one before the current)
[12:55] <tomreyn> --list-boots lists boots which were recorded to logs (i.e. where the file system storing the logs was already writable)
[12:56] <PMunch> That's the problem, I don't think it found the file system at all
[12:56] <PMunch> Let me check
[13:00] <PMunch> Doesn't seem like it
[13:00] <tomreyn> right, if it failed to mount the same file system the logs would be written to then there won't be logs. you can then just reproduce the situation and gather logs from the screen or through a serial console. or netconsole
[13:00] <tomreyn> !bootlog
[13:00] <ubottu> To get a more verbose log of the boot process, remove "quiet" and "splash" from the kernel boot parameters and add "debug systemd.log_level=info". For info on editing kernel boot parameters, see https://wiki.ubuntu.com/Kernel/KernelBootParameters
[13:58] <lotuspsychje> PMunch tomreyn i found an (older) bug #1460447 check if related?
[13:58] <ubottu> Bug 1460447 in linux (Ubuntu) "Boot slow, 'scanning for btrfs filesystems' takes 100 seconds" [High, Confirmed] https://launchpad.net/bugs/1460447
[14:00] <PMunch> Hmm, that does seem similar
[14:00] <PMunch> Just that mine took so long that it timed out
[14:01] <lotuspsychje> maybe a new kernel version triggers this bug somehow
[14:03] <PMunch> Could be
[14:04] <PMunch> I have to go now though, might investigate further later
[14:04] <lotuspsychje> allrighty PMunch good luck
=== genii-core is now known as genii
[16:29] <patstoms> morning
[16:30] <patstoms> is there any way to fix apt when it says "You might want to run 'apt --fix-broken install' to correct these." "The following packages have unmet dependencies:"
[16:30] <patstoms> because when i run apt --fix-broken install
[16:37] <pycurious> anyone using kubernetes on aws/gcp here through ubuntu?
[16:37] <patdk-lap> not pure
[16:37] <patdk-lap> aws els and also kops
[18:01] <tomreyn> patstoms: "<patstoms> because when i run apt --fix-broken install" seemed like an incomplete sentence?
[18:02] <patstoms> oh, sorry
[18:02] <patstoms> i get an error that mysql can't be stopped
[18:02] <patstoms> i know that i have allready been in this situation
[18:02] <patstoms> and been asking this question here
[18:02] <patstoms> i just had a dejavu
[18:02] <patstoms> so i installed mariadb, then realised that i need mysql not mariadb because of compability with innodb
[18:03] <tomreyn> post the full command and its full output to a pastebin, and post the link here. that's usually the best way to get help in such situations.
[18:03] <patstoms> so i did apt purge mariadb-server and when i install mysql-server i am stuck with this 
[18:03] <patstoms> http://vpaste.net/axe1r
[18:05] <tomreyn> "There is a MySQL server running, but we failed in our attempts to stop it. Stop it yourself and try again!" seems like it should be a useful hint
[18:08] <tomreyn> from what i remember (hope that's still correct), just like "mariadb-server", "mysql-server" is also just a tiny package which actually depends on a versioned package, such as "mariadb-server-10.1" or "mysql-server-8" etc, which then depend on a lot more packages.
[18:09] <tomreyn> and while the unversioned package depends on the versioned package and is useful for installations and upgrades, for *removing* you'll need to also explicitly list the versioned package
[18:09] <tomreyn> patstoms: ^
[18:10] <tomreyn> also note the innodb compatibility situation of mariadb is documented at https://mariadb.com/kb/en/innodb/
[18:11] <tomreyn> most of the time i would assume that the mariadb implementation should still be backwards compatible to the oracle community edition
[18:12] <tomreyn> (in fact you may want to install the "default-mysql-server" package rather than selecting one explicitly, to benefit from migration/upgrade paths to different software in later ubuntu releases)
[18:14] <tomreyn> hmm, this package is in universe, not main, i guess this means this recommendation is actually invalid. sorry.
[18:17] <tomreyn> so long story short: you probably need to also purge package mariadb-server-10.3 (if my assumption, that you are running Ubuntu 20.04 LTS, is correct). be careful not to purge the databases, though, if you want to retain those.
[18:19] <tomreyn> taking https://mariadb.com/kb/en/innodb/ into account, re-using existing mariadb-server 10.3.31 databases (innodb engine) with the oracle community edition (mysql-server) may not work, though.
[18:29] <patstoms> so i deleted broken mysql packages using sudo dpkg --force depends --purge ...
[18:29] <patstoms> when i tried to install mysql-server again, it was still not working, but when i installed mariadb again, purged it without errors and install mysql-server again
[18:29] <patstoms> it worked
[18:30] <patstoms> i just had to remove mariadb with apt purge not apt remove 
[18:38] <tomreyn> you should not need to use "--force depends" there
[18:39] <tomreyn> yes, sometimes you need to use purge to remove incompatible configurations or data.
[18:39] <tomreyn> i actually prefer it, do not like the idea of old configurations lingering around.
[18:42] <patstoms> i had no other way because every apt command said that i need to use apt --fix-broken install
[19:34] <tomreyn> --fix-brokne isn't the same as --foce though. but maybe you're saying that --fix-broken had no effect, and that's why you did --force
[19:34] <tomreyn> --fix-broken isn't the same as --force though. but maybe you're saying that --fix-broken had no effect, and that's why you did --force.
[19:34] <tomreyn> ^ the same with less typos
=== nhaines_ is now known as nhaines