[06:48] <jchittum> samy1028b : on the `ua status` bit. The Ubuntu FIPS vm comes pre-installed with everything. However, `ua` still attempts to install FIPS packages, and presents the message. This can be considered a UX bug
[06:48] <jchittum> and I agree that packages should get "smarter" about FIPS within the Ubuntu ecosystem. that will be a large community effort
[06:56] <samy1028b> jchittum : yes, though things like this which become broken by default should probably be addressed first.
[06:57] <samy1028b> Unfortunately, many of the help items on Stackoverflow are rather poor, as are several other help sites out there.  The common recommendation is to disable TLS and disable FIPS and then bingo, it magically starts working, with zero security.
[06:57] <samy1028b> but at least people should find this bug now for better information
[07:00] <jchittum> agreed on all above. We (the Canonical Public Cloud team, the team that makes the FIPS image) need to help create better documentation around using the FIPS images, along with common tutorials (like installing mysql on top)
[07:12] <samy1028b> jchittum : I also have an Ubuntu 18.04 FIPS with MySQL that loads just fine with TLS.  Just an additional data point for you.
[07:23] <jchittum> samy1028b : that's really helpful. installing `mysql` from the Ubuntu archive on both? In which case, there may be a packaging difference between `mysql 8.0` and `mysql 5.7`
[07:25] <samy1028b> jchittum : yes, installing MySQL from the Ubuntu repository.
[07:26] <samy1028b> Also, I have an ubuntu 18.04 FIPS in both Azure and AWS with MySQL working just fine with TLS.
[07:28] <samy1028b> In our Ansible deployment script we have it do an "apt-get update; apt-get dist-upgrade" in a couple spots to ensure we have the latest version of packages before installing.
[07:28] <samy1028b> and then it simply does an "apt-get install mysql-server mysql-client" at that point
[07:29] <samy1028b> actually, come to think about it, I think it breaks down the LAMP package in Ansible and deploys each part of the LAMP stack, which obviously includes MySQL.
[07:29] <jchittum> great. I'm going to add that to the bug report. It'll help the maintainers dig in more for diffs
[07:34] <samy1028b> jchittum, this is the relevant part of our Ansible script:  https://pastebin.com/dejxMdmd
[07:44] <samy1028b> hmm, here's an odd thing.  With MySQL 8.0 "SHOW VARIABLES LIKE '%fips%';", I get ssl_fips_mode=off.  on MySQL 5.7 on 18.04 I get no results at all.  Any idea if it's not even a FIPS type package in MySQL 5.7 or does it just report differently?
[07:44] <samy1028b> btw, I just grabbed the following information for it also:  https://pastebin.com/6X2mWQSV
[07:44] <samy1028b> "apt-cache policy mysql-server"
[07:51] <jchittum> judging from the mysql 5.7 docs, it does not appear to have FIPS support
[07:54] <jchittum> which isn't to say that it cannot be FIPS compliant. The 8.0 documentation states that mysql itself doesn't have any FIPS specific code
[07:54] <jchittum> so it's passing off everything to OpenSSL. so it'd fall to configuration of OpenSSL (which with the FIPS versions is set properly) and configuration of the mysql server to use TLS, etc.
[07:55] <jchittum> a lot of settings can be done manually, but it looks like mysql 5.7 doesn't have a handy wrapper for FIPS to set everything. 
[07:57] <samy1028b> ahh, that makes sense
[07:58] <samy1028b> Yeah, trying to figure out things with FIPS takes a whole other level sometimes. :)
[07:59] <samy1028b> Well, I'm off to sleep now. it's 4am. :)
[12:10] <ahasenack> morning
[12:26] <ahasenack> samy1028b: hi, I did another fresh azure deploy to check on your other report, that `ua status` was asking you to reboot, but couldn't confirm that
[12:27] <ahasenack> I saw that right after deploy `ua` was still installing packages, and I had a `NOTICE` in `ua status` output, but it eventually went away, and there was no ask for a reboot
[12:27] <ahasenack> I used:
[12:27] <ahasenack> $ cat /etc/cloud/build.info 
[12:27] <ahasenack> build_name: pro-fips-server
[12:27] <ahasenack> serial: 20220429
[12:27] <ahasenack> maybe you enabled fips-updates at some point? That might require a reboot
[12:27] <ahasenack> because it changes the kernel iirc
[12:49] <ahasenack> when I log in on a jammy vm, the sysinfo load information is, erm, precise
[12:49] <ahasenack>   System load:  0.44482421875     Processes:               120
[13:16] <patdk-lap> needs to be these days of 64core cpu's :)
[13:40] <ahasenack> I'm accepting donations of such hardware to test this
[13:42] <patdk-lap> :)
[14:54] <samy1028b> Good morning ahasenack,  cat /etc/cloud/build.info  
[14:54] <samy1028b> build_name: pro-fips-server
[14:54] <samy1028b>  serial: 20220215.1
[14:55] <samy1028b> I haven't rebooted it, but I can see if what happens then.
[14:55] <ahasenack> interesting, it's an older image
[14:55] <ahasenack> all that's left to investigate is that warning you got that you should reboot, which I couldn't reproduce with mine
[14:59] <samy1028b> yeah, it's still saying this:
[14:59] <samy1028b> NOTICES
[14:59] <samy1028b> Reboot to FIPS kernel required
[15:00] <samy1028b> I'll reboot now and see what happens.
[15:04] <ahasenack> it should work, but I would like to understand why you got that
[15:04] <ahasenack> samy1028b: did you enable fips-updates by any chance?
[15:05] <samy1028b> https://pastebin.com/1YQp7tUC
[15:05] <ahasenack> and compare the running kernel, with the one you would reboot into (if you haven't rebooted yet)
[15:05] <samy1028b> I think my tech tried at one point
[15:05] <samy1028b> I just rebooted and then ran these commands
[15:06] <ahasenack> samy1028b: and /run/reboot.<tab><tab>? 
[15:06] <ahasenack> contents
[15:06] <ahasenack> I think it's reboot-required.pkgs
[15:08] <samy1028b> root@temp-test-01:~# ls -alhtr /run/reboot*
[15:08] <samy1028b>    ls: cannot access '/run/reboot*': No such file or directory
[15:08] <ahasenack> ah, you rebooted already
[15:08] <ahasenack> then it's gone
[15:13] <samy1028b> I checked on my production VM and it's showing the same information
[15:14] <samy1028b> looks like I'd rebooted that one at some point also.
[15:16] <ahasenack> after the reboot, ua status is still asking for a reboot?
[15:48] <patdk-lap> is there something special needed to use initramfs modules in arm64?
[15:48] <patdk-lap> just adding to /etc/initramfs/modules and rebuilding didn't include them, and the modules file didn't exist like on other archs
[15:58] <sdeziel> patdk-lap: here (on 20.04, amd64), the directory is /etc/initramfs-tools
[16:00] <samy1028b> ahasenack, yes, after reboot "ua status" still shows NOTICES
[16:00] <samy1028b> Reboot to FIPS kernel required
[16:22] <patdk-lap> odd, I wonder why I have two, initramfs and initramfs-tools
[16:22] <patdk-lap> never seen initramfs before and didn't notice
[17:35] <lucasmoura> hi samy1028b, can you inform us the output of ua version and ua status please ? So I can test an idea here
[18:10] <ahasenack> lucasmoura: some of that is here: https://pastebin.com/1YQp7tUC
[18:13] <samy1028b> root@temp-test-01:~# ua version
[18:13] <samy1028b> 27.7~20.04.1
[18:54] <lucasmoura_> samy1028b, it seems that this is indeed a bug. Can you please report all the steps you used to land on this issue on our bugs page: https://bugs.launchpad.net/ubuntu/+source/ubuntu-advantage-tools/+bugs
[19:07] <samy1028b> lucasmoura : ahasenack created a bug already which lists the equivalent steps for the initial problem:  https://bugs.launchpad.net/ubuntu/+source/mysql-8.0/+bug/1971788   Are you asking to create a new ticket on the "ua status" showing reboot required?
[19:07] <ahasenack> yes, these are different bugs
[19:09] <samy1028b> ahh, ok.  I'll have one of the team here document the steps and create an account there to report it.
[20:29] <samy1028b> ahasenack, lucasmoura,  https://bugs.launchpad.net/ubuntu/+source/ubuntu-advantage-tools/+bug/1972026
[20:29] <samy1028b> how does this look?
[20:30]  * ahasenack checks
[20:31] <ahasenack> samy1028b: looks good, thanks
[20:31] <ahasenack> samy1028b: lucas may ask for some data or files, but this is excellent, thanks for taking the time to fill it out
[20:34] <samy1028b> our pleasure.  We do a lot of this internally for our products and services.  We don't often submit something to a project like this though.
[20:35] <samy1028b> And thank you very much for helping us figure out the MySQL FIPS issue yesterday.  This weekend we'll be doing some updates to our scripts to ensure they connect properly once we enable FIPS mode MySQL on the 20.04 production VM.
[20:50] <lucasmoura> samy1028b, thanks for opening that bug. I will try to reproduce the error and come up with solution for it