| lordievader | Good morning | 06:37 |
|---|---|---|
| === Guest99098 is now known as lordievader | ||
| coreycb | icey: there's a mysql8 + alembic issue with barbican we'll need to look into: https://bugs.launchpad.net/ubuntu/+source/barbican/+bug/1899104 | 13:36 |
| ubottu | Launchpad bug 1899104 in barbican (Ubuntu) "barbican-manage db upgrade fails with MySQL8" [Undecided,New] | 13:36 |
| icey | I saw that come in this morning | 13:37 |
| icey | coreycb: | 13:37 |
| * RoyK wonders why people use mysql these days | 13:39 | |
| coreycb | icey: it looks like that may be fixed upstream https://storyboard.openstack.org/#!/story/2007732 | 13:46 |
| coreycb | icey: since you worked on that for groovy do you want to make sure it's fixed in groovy and handle the SRU to focal? | 13:46 |
| icey | sure | 13:48 |
| nopnotme | I have an issue when running any *apt* commands. I was originally trying to install nodejs on a Ubuntu 16.04 server that I only use via ssh, however I am getting an error using apt now regarding grub-common versions: https://pastebin.com/UwFi9fdF | 16:44 |
| teward | nopnotme: is this on an Amazon EC2 instance? | 17:09 |
| nopnotme | negative | 17:10 |
| teward | i only asked because the package that's installed (grub-legacy-ec2) is for EC2 instances | 17:11 |
| nopnotme | what it be in my interest to purge it? | 17:13 |
| nopnotme | even after purging i get the same error | 17:20 |
| mgedmin | by "it" do you mean grub-legacy-ec2 or grub2-common? | 17:26 |
| nopnotme | grub-legacy-ex2 | 17:27 |
| pgnd | Running regular maintenance today on an otherwise stable Ubu bionic server, 'apt update' FAILs with: "UnicodeDecodeError: 'utf-8' codec can't decode byte 0xc9 in position 4: invalid continuation byteError in sys.excepthook:" | 17:27 |
| pgnd | detail -> https://pastebin.com/raw/PVegTn92 | 17:27 |
| pgnd | What do we to workaround/fix that^ in the context of apt updates? | 17:27 |
| pgnd | s/we to/we do in order to/ | 17:29 |
| mgedmin | nopnotme: how can you have a conflict with grub-legacy-ec2 if grub-legacy-ec2 is no longer installed? pastebin the full error please | 17:41 |
| mgedmin | pgnd: I would first report a bug (ubuntu-bug command-not-found) | 17:46 |
| mgedmin | pgnd: hmm, what happens if you run `python3 -c 'import apt_pkg'` on that server? | 17:47 |
| mgedmin | maybe this needs to be investigated a bit to rule out local damange before assuming it's a bug | 17:47 |
| pgnd | mgedmin: hi. -> https://pastebin.com/raw/pddKjvbc | 17:49 |
| pgnd | something's clearly not happy ... | 17:50 |
| mgedmin | pgnd: very interesting, I've never seen it before | 17:50 |
| mgedmin | pgnd: what's the output of `locale charmap`? | 17:51 |
| pgnd | mgedmin: "UTF-8" | 17:51 |
| mgedmin | I'd next try `LC_ALL=C python3 -c 'import apt_pkg'` and `LC_ALL=C.UTF-8 python3 -c 'import apk_pkg'`, to see if the locale has anything to do with it | 17:52 |
| mgedmin | next idea: do you have debsums installed? check with `debsums python3-apt` if the package got corrupted somehow? | 17:53 |
| pgnd | mgedmin: -> https://pastebin.com/raw/zF7rFkKm | 17:55 |
| mgedmin | well now! | 17:56 |
| mgedmin | /usr/lib/python3/dist-packages/apt_pkg.cpython-36m-x86_64-linux-gnu.so FAILED | 17:56 |
| pgnd | yeah | 17:56 |
| mgedmin | make a copy of that file, then apt install --reinstall python3-apt | 17:56 |
| mgedmin | when apt is done you may want to cmp the file with the bad copy | 17:56 |
| mgedmin | see if maybe it differs in one bit only? could indicate hardware problems | 17:56 |
| * mgedmin can stop browsing https://bugs.launchpad.net/ubuntu/+source/python-apt for similar problems | 17:57 | |
| mgedmin | I would probably run memtest86 and/or hard disk SMART self-tests with smartctl from smartmontools | 17:58 |
| pgnd | mgedmin: the reinstall did the trick. thx! | 17:58 |
| pgnd | cmp returns: | 17:58 |
| pgnd | smart tests run nightly. all's good ... will rerun here shortly, just in case | 17:59 |
| mgedmin | cmp returns no output? | 17:59 |
| pgnd | huh? oh ... ry leading "/" :-( | 18:00 |
| pgnd | -> /usr/lib/python3/dist-packages/apt_pkg.cpython-36m-x86_64-linux-gnu.so /usr/lib/python3/dist-packages/apt_pkg.cpython-36m-x86_64-linux-gnu.so.ORIG differ: byte 258055, line 823 | 18:00 |
| mgedmin | ah, cmp -l will show the values of the differing bytes, and won't stop at the first difference | 18:01 |
| pgnd | mgedmin: https://pastebin.com/jZCM7dUX (somebody, aka 'me', needs to read up on cmp!) | 18:03 |
| pgnd | mgedmin: in any case, mny thx! | 18:06 |
| mgedmin | well that's a lot | 18:09 |
| mgedmin | cmp -l -p will also show ascii values, not sure if that's going to be helpful | 18:09 |
| mgedmin | converting file offsets from decimal to hex shows the differences are in the range of 0x3f007 to 0x3ffff | 18:10 |
| mgedmin | that upper value is very precisely aligned | 18:10 |
| mgedmin | it's one corrupted 4K block | 18:11 |
| mgedmin | exactly one memory page | 18:11 |
| mgedmin | oh! pgnd: in your place I'd run `debsums -c`, to check _all_ installed packages for corruption | 18:12 |
| pgnd | mgedmin: good point! doing that now | 18:13 |
| pgnd | mgedmin: only one, "debsums: Error: symlink loop detected in path 'usr/share/doc/module-init-tools/changelog.Debian.gz'. Please file a bug again module-init-tools." | 18:17 |
| mgedmin | heh | 18:18 |
| mgedmin | huh, on my bionic server /usr/share/doc/module-init-tools/changelog.Debian.gz is a regular file and not a symlink | 18:18 |
| mgedmin | and no component in that path is a symlink | 18:19 |
| pgnd | re-run of smart tests ... all good here | 18:19 |
| mgedmin | memtest time | 18:19 |
| mgedmin | oh, one more trick | 18:19 |
| mgedmin | you can ask the kernel to drop the page caches, then compare the two files again | 18:19 |
| mgedmin | they'll be read from disk afresh | 18:19 |
| mgedmin | if they're identical, then you definitely have a memory problem | 18:20 |
| pgnd | mgedmin: this server has been upgraded since forever ... not a clean install. i've seen similar links vs files issues, elsewhere, in the past | 18:20 |
| mgedmin | if they're still different, the wrong block got written out | 18:20 |
| mgedmin | you drop caches by doing echo 3 |sudo tee /proc/sys/vm/drop_caches | 18:20 |
| mgedmin | the server will be slow for a few minutes after that, while it reads everything back from disk | 18:20 |
| pgnd | mgedmin: re-cmp'd after the drop cache. similarly long output ... | 18:21 |
| pgnd | good news is, currently no more apt* problems that I can see. | 18:23 |
| pgnd | need to find Memtest86+ equivalent for UEFI: "Memtest86+ only works on systems with BIOS firmware. Support for UEFI systems is currently unavailable." | 18:34 |
| mgedmin | there's also apt install memtester | 18:36 |
| mgedmin | it runs inside linux, so it can't test _all_ memory, only the parts that are not in use | 18:36 |
| mgedmin | but it's better than nothing | 18:36 |
| pgnd | thx | 18:38 |
| ueberall | pgnd: What about https://www.memtest86.com/ ? | 18:40 |
| pgnd | ueberall: Didn't realize there was a different creature ... had just checked the pkg repos, which offer memtest86+ & memtester. thx, will take a look. | 18:42 |
| === genii is now known as genii-o-lantern | ||
| === genii-o-lantern is now known as genii | ||
| === arooni_team_b is now known as arooni | ||
Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!