[08:29] <yurtesen> utkarsh2102: so this is the day we resolve it? :)
[09:58] <KeeperoftheKeys> hey
[09:58] <KeeperoftheKeys> unattended upgrades seems to not respect debconf settings - https://pastebin.com/ZNLA9xQK
[09:58] <KeeperoftheKeys> is this a known issue?
[10:00] <KeeperoftheKeys> Tested on ubuntu-server 20.04 and on ubuntu-desktop 22.04
[10:03] <KeeperoftheKeys> (to be clear what should have happened in that paste is that the debconf answer remained unchanged and /etc/apt/apt.conf.d/20auto-upgrades was updated with the disabled unattended upgrades file, instead debconf was updated and nothing was changed in 20auto-upgrades)
[12:48] <utkarsh2102> yurtesen: hey, yes, I started the draft, I'll finish it by my evening and send it over, CC'ing you. 
[12:49] <utkarsh2102> it'll basically be just a data dump with asking what next steps we can take. 
[12:49] <utkarsh2102> if you feel like adding something on top of it, please do :)
[13:41] <KeeperoftheKeys> I ran a further test it seems that unattended-upgrades only ignores debconf after that value has been true at some point
[14:13] <samy1028b> Hello all, I have a new Ubuntu 20.04 FIPS VM with a new MySQL 8.0.29 installation that is not starting with SSL.  It is throwing this in the error.log:  "The SSL library function CRYPTO_set_mem_functions failed"
[14:13] <samy1028b> I've found some references on Redhat with FIPS enabled.  Has anyone seen something similar with FIPS or have any ideas where to look further?
[14:14] <samy1028b> Any ideas if this might be something with Ubuntu 20.04 FIPS implementation?
[14:22] <ahasenack> maybe, you would have to reboot without fips to see if it then starts
[14:22] <ahasenack> and tune up debugging on mysql's side to see if it spits out which crypto algorithm it's trying to use that was denied by openssl fips
[14:25] <samy1028b> This particular VM is a prepackaged Azure Ubuntu 20.04 FIPS server VM.  How would we reboot without FIPS?
[14:26] <samy1028b> I'll give the idea on the debug tracking for the crypto algo to my tech who's working on this also.
[14:29] <ahasenack> you would have to reboot either with a command line disabling fips (something along the lines of fips_enabled=0, or fips=0, I don't remember), or boot into a non-fips kernel
[14:29] <ahasenack> I would first try debugging mysql to see what it's trying to do
[14:30] <ahasenack> algorithms that are non fips compliant include md5, and I think also sha1
[14:32] <samy1028b> ahasenack, that sounds like a good first step and thoughts of what to check for.  My tech is going to look into that shortly.  Thank you for the ideas!
[14:34] <ahasenack> samy1028b: did you build that mysql 8.0.29 yourself for ubuntu 20.04?
[14:35] <ahasenack> oh, no, I see it's available for focal, n/m
[14:39] <ahasenack> samy1028b: have you tried this command-line option? https://dev.mysql.com/doc/refman/8.0/en/server-system-variables.html#sysvar_ssl_fips_mode
[14:42] <samy1028b> hmm.  Currently it's just running "/usr/sbin/mysqld" and when I search for SSL variables via MySQL it shows "ssl_fips_mode = off" at the moment.
[14:43] <samy1028b> I would think that it should have tried to auto-start with ssl_fips_mode=on since this is pre-built FIPS VM?
[14:44] <samy1028b> and no, we haven't tried forcing it via the command line.
[14:46] <ahasenack> it's the normal mysql package from the distribution, it wouldn't default to on, and I don't think the cloud image can be customized that way for a package that isn't even installed yet (at boot time)
[14:47] <jchittum> I'm guessing that the installation of mysql doesn't detect that it is installed on a FIPS system, and doesn't turn on the variable
[14:47] <ahasenack> samy1028b: just for me to understand, you selected an image that is fips enabled from the start, right
[14:48] <ahasenack> you did not enable fips manually after booting the vm?
[14:53] <samy1028b> no, I assumed it would have loaded it as part of the deployed VM?
[14:54] <samy1028b> Yes, we deployed on Azure by selecting the Ubuntu 20.04 FIPS vm
[14:54] <samy1028b> https://azuremarketplace.microsoft.com/en-us/marketplace/apps/canonical.0001-com-ubuntu-pro-focal-fips
[14:54] <samy1028b> hmm... that could be part of the issue.
[14:55] <samy1028b> I just did a "ua status" and it shows ESM and FIPS as entitled/enabled.
[14:55] <samy1028b> but then it shows this: "NOTICES
[14:55] <samy1028b> Reboot to FIPS kernel required
[14:55] <samy1028b> "
[14:56] <samy1028b> uname -a        
[14:56] <samy1028b> Linux temp-test-01 5.4.0-1022-azure-fips #22+fips1-Ubuntu SMP Mon Dec 13 01:12:55 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
[14:57] <samy1028b> Any idea why "ua status" says a reboot to FIPS kernel required but uname -a shows I'm running a FIPS kernel?
[15:04] <ahasenack> unsure, but you seem to be on a fips kernel, so that's fine
[15:04] <ahasenack> another way to check is in /proc or /sys, let met get that path
[15:05] <ahasenack> samy1028b: cat /proc/sys/crypto/fips_enabled
[15:05] <ahasenack> iirc
[15:06] <samy1028b> cat /proc/sys/crypto/fips_enabled
[15:06] <samy1028b>     1
[15:07] <ahasenack> looks good
[15:07] <ahasenack> I would try that mysql command line option next then
[15:07] <samy1028b> ok
[15:07] <ahasenack> or config, doesn't have to be a command line option, it can be added to the config somewhere
[15:13] <orndorffgrant> is /var/run/reboot-required set? "ua" might be seeing that and assuming (wrongly) that the reboot is required for the fips kernel
[15:16] <samy1028b> No idea,  I'll get my tech to look at that and see what he can tell me.
[15:18] <ahasenack> samy1028b: I'll try to reproduce it and file a bug to see what we can do to make the experience better
[15:18] <ahasenack> I'll let you know the number here when I have it
[15:18] <ahasenack> having lunch now ;)
[15:20] <samy1028b> I have to jump off for a conference call myself.  Thank you for the help so far! :)
[15:49] <goddard> any easy to way to just copy apache/mysql files and data to another server?
[15:50] <goddard> running same versions of everything just different host
[15:50] <goddard> i imagien i could just rsync 
[15:50] <goddard> /etc/apache /var/www ?
[15:53] <sdeziel> goddard: yeah, rsync is the tool of choice I think
[15:53] <goddard> cool
[15:55] <goddard> any issues just mashing /var/lib/mysql into the new server ?
[15:55] <goddard> obviously would stop it on the old before doing it
[16:08] <patdk-lap> permissions, version
[16:22] <brad> Hello, I need help setting up wifi on my Ubuntu server, I am running it on a raspberry Pi 3b
[16:25] <ahasenack> brlin: https://netplan.io/examples/#connecting-to-a-wpa-personal-wireless-network is one way
[17:34] <goddard> patdk-lap: gotta preserve permissions then and just make sure they are the same versions
[17:45] <samy1028b> ahasenack: some news on the MySQL / 20.04 / FIPS issue - my tech has found that it appears MySQL FIPS requires OpenSSL 1.0.2 but Ubuntu 20.04 FIPS has everything with OpenSSL 1.1.1f.  And according to https://dev.mysql.com/doc/refman/8.0/en/fips-mode.html#fips-system-requirements  it appears this may be the root cause?
[17:45] <ahasenack> it still doesn't start, or are you just looking at the requirements?
[17:46] <ahasenack> I'm trying to reproduce it, but currently having some problem with my vm setup
[17:53] <samy1028b> it still doesn't start.
[17:54] <samy1028b> at the moment it looks like we may need to migrate (again) to an Ubuntu 18.04 FIPS with MySQL 5.7 FIPS enabled.
[17:56] <ahasenack> samy1028b: were there any support options given when selecting this image type?
[17:57] <samy1028b> I don't think so?  Let me check
[18:01] <samy1028b> it doesn't look like it.  It's just the basic install with Ubuntu Advantage, no support included.
[18:03] <ahasenack> samy1028b: can you show in a pastebin the output of apt-cache policy mysql-server-8.0
[18:06] <ahasenack> samy1028b: I added ssl-fips-mode=ON to /etc/mysql/mysql.conf.d/mysqld.cnf restarted
[18:06] <ahasenack> and now I get
[18:06] <ahasenack> 2022-05-05T18:05:44.070810Z 0 [System] [MY-013602] [Server] Channel mysql_main configured to support TLS. Encrypted connections are now supported for this channel.
[18:06] <ahasenack> so it seems to have worked
[18:06] <ahasenack> before I was getting the same error as you (but mysql was running, probably without tls though)
[18:06] <samy1028b> https://pastebin.com/Zpi5BDgB
[18:06] <ahasenack> let me just stop it to be sure the logs are fresh
[18:07] <ahasenack> stopped mysql, cleared the log, started: https://pastebin.ubuntu.com/p/nqXfK2bRXq/
[18:07] <ahasenack> still the error, but it also seems to have started
[18:12] <ahasenack> without setting the fips conf, the error is actually "Failed to set up SSL because of the following SSL library error: SSL_CTX_new failed
[18:12] <ahasenack> "
[18:16] <ahasenack> show variables like '%ssl%';
[18:16] <ahasenack> that command also now shows, when I set fips=on in the config:
[18:16] <ahasenack> | have_openssl                        | YES             |
[18:16] <ahasenack> | have_ssl                            | YES             |
[18:16] <ahasenack> and before it was saying DISABLED to both
[18:17] <ahasenack> samy1028b: I'd say ssl is working after adding the fips option the config
[18:18] <samy1028b> We're giving it a try also.
[18:19] <samy1028b> if it works, perhaps this should be raised as a usability item - you'd expect a FIPS deployed system should have FIPS enabled by default in a deployed package if it is available.  Otherwise we have to expend even more effort resolve issues like this.
[18:20] <ahasenack> sure
[18:30] <ahasenack> samy1028b: I'm using the client on the same host, and I'm finding that I also have to pass --ssl-fips-mode=on in the client command line
[18:30] <ahasenack> but it worked, see line 22: https://pastebin.ubuntu.com/p/2wBG8FT2tb/
[18:31] <ahasenack> without the --ssl-fips-mode on the client as well, I get
[18:31] <ahasenack> # mysql -uubuntu -p -h 127.0.0.1 
[18:31] <ahasenack> Enter password: 
[18:31] <ahasenack> ERROR 2026 (HY000): SSL connection error: SSL_CTX_new failed
[18:51] <samy1028b> ahasenack: yup, all of that worked.  And I can now connect properly from a remote connection also via TLS.
[18:52] <samy1028b> so, with the item, do you think this is something that upstream would be interested in?
[19:24] <ahasenack> samy1028b: upstream  you mean mysql upstream, oracle? 
[19:24] <ahasenack> I think in ubuntu we should at least document it, or find a way to make that experience better
[19:32] <KeeperoftheKeys> anyone familiar with debconf/unattended-upgrades?
[19:36] <samy1028b> ahasenack: not certain - I guess in Ubuntu for a documented use case in FIPS deployments would probably be best.
[19:39] <yurtesen> utkarsh2102: thanks, but I did not receive anything in CC I think
[19:54] <utkarsh2102> yurtesen: hey, you can't see the headers?
[19:57] <utkarsh2102> yurtesen: at least, I see a CC to eyurtese@abo.fi 
[20:06] <yurtesen> utkarsh2102L I got it now
[20:07] <ahasenack> samy1028b: I filed this bug: https://bugs.launchpad.net/ubuntu/+source/mysql-8.0/+bug/1971788
[20:08] <ahasenack> heh, sso
[20:08]  * ahasenack fixes the title
[20:09] <sarnold> heh
[20:10] <yurtesen> utkarsh2102: thanks
[20:10] <utkarsh2102> \o/
[20:10] <sarnold> hrm. I'm surprised they don't automatically switch to fips-compliant settings when booted on a fips machine, and use that setting to *disable* fips if necessary for some reason
[20:29] <samy1028b> Thank you ahasenack!