/srv/irclogs.ubuntu.com/2020/12/23/#ubuntu-server.txt

FuZi0NI have a folder with around 4500 subdirectories. How can I delete all subdirectories that are smaller than 100 MB?00:15
JanCI'm pretty sure all those subdirectories are smaller than 100MB  :)01:01
JanCbut I guess you mean the files inside them01:02
JanCyou could script it in bash using a tool like 'du'01:03
zetherooI need help with getting an mdadm raid to boot. power was lost on the server and when restored the OS would not boot. After POST all I see is a blinking cursor on a black screen.08:25
zetherooI did 'grub-install' on both disks from an Ubuntu live session but now get errors when booting and end up in grub rescue08:28
RoyKzetheroo: first up, which versions of things are you running? can you see both drives if booting up on a rescue usb thing? do you see any issues in smart data on the drives?08:30
RoyKalso, do you have a backup?08:30
zetheroo RoyK: I have both disks (SSDs) attached to my PC via USB dock at the moment08:31
zetheroohttps://pastebin.com/K62C604U08:31
zetheroothe disks are both less than a month old08:31
zetherooone of the errors when booting is 'unknown filesystem'08:32
zetheroono, we don't have a current backup08:36
RoyKwhat does lsblk have to say?08:38
RoyKthe age of the drives is generally irrelvant - drives can fail at any time08:38
zetheroohttps://pastebin.com/ePn3BQ6t08:39
RoyKplease pastebin output of "smartctl -a /dev/sdX"08:39
RoyKerm - nothing on the raid? is lvm installed on the machine?08:40
RoyKalso, please don't use pastebin.com - it's spammy - https://paste.ubuntu.com/ or similar is quieter and works well ;)08:40
zetheroohttps://paste.ubuntu.com/p/vff6PvF8bW/08:41
zetheroonot sure about the LVM question - but very likely.08:42
RoyKtype 'vgs'08:42
zetheroono output08:43
RoyKlooks like the drives are new-ish - only been running for a week or so08:43
zetheroolvm2 was not installed on my PC so just installed it08:43
zetherooyep08:43
RoyKpvscan08:44
zetheroo  No matching physical volumes found08:44
RoyKjust to check if there's some lvm stuff there08:44
RoyKoh08:44
RoyKcould it be zfs?08:44
zetheroohmm ...08:44
RoyKnot likely08:44
RoyKsince there's md008:44
RoyKthere really should be something visible there08:45
RoyKwas this setup with crypto or something?08:45
zetheroono crypto08:45
zetheroois there a way to mount the disk?08:51
zetheroomaybe it's better to get all the critical data out and just do a fresh install08:52
RoyKthere should be, but before that, you should see partitions/filesystems08:52
RoyKand it's very strange that lsblk shows none08:52
RoyKerm08:53
RoyKosprober-linux-sda108:53
RoyKwhat is that?08:53
zetheroono idea ... looking it up08:54
zetheroohttps://github.com/openzfs/zfs/discussions/980108:54
zetherooZOL ...?08:54
RoyKI hope not08:55
RoyKzfs on top of mdraid sounds like a bad dream08:55
RoyKor a badtrip08:55
RoyKwhat does 'file -s /dev/md0' have to say?08:56
zetheroo/dev/md0: Linux rev 1.0 ext4 filesystem data, UUID=99f7fdf9-6f9a-4069-856e-4db4c529741e (needs journal recovery) (extents) (64bit) (large files) (huge files)08:57
RoyKhm - try to mount it somewhere08:57
zetherooshould I mount md0 or the individual disks?08:58
RoyKmd008:58
RoyKthe mirror seems to be in sync after all08:58
zetheroomount: /mnt/raidarray: cannot mount /dev/md0 read-only.09:01
zetheroomounting via fstab with:09:01
zetherooUUID=99f7fdf9-6f9a-4069-856e-4db4c529741e  /mnt/raidarray ext4 defaults 0 009:01
oerheksread only, sounds like a dirty filesystem09:02
RoyKoerheks: it is - it said above - power failure09:04
zetheroohttps://paste.ubuntu.com/p/J7BwF4RMm6/09:04
RoyKzetheroo: any valuable data on it?09:05
zetherooyep  -  it was our monitoring server :(09:05
RoyKzetheroo: not to rub it in, but try to use a backup next time ;)09:05
zetherooand it had no redundant PSU 😲09:05
RoyKzetheroo: keep a backup anyway09:05
RoyKtwo backups, preferably, at least09:06
zetherooyes, I know ... very bad practice with that :(09:06
RoyKanyway09:06
RoyKwhat does /proc/mdstat have to say now?09:06
zetheroohttps://paste.ubuntu.com/p/gp4KnDYSSs/09:07
RoyKyou might need to issue a raid sync/repair09:07
RoyKecho repair > /sys/block/md0/md/sync_action09:07
zetheroobash: /sys/block/md0/md/sync_action: Permission denied09:08
RoyK'sudo -i' first09:08
zetheroook, no output09:08
RoyKcheck mdstat09:08
RoyKit shouldn't take too long on a small ssd mirror09:09
zetheroohttps://paste.ubuntu.com/p/sNhRTr4PNh/09:09
zetheroohttps://paste.ubuntu.com/p/7WcdJM8xPT/09:10
RoyKtry to stop it and reassemble09:12
RoyKmdadm --stop /dev/md0 ; mdadm --asseble --scan09:12
zetheroomdadm: /dev/md/0 has been started with 2 drives.09:12
RoyKcheckk mdstat09:13
zetheroomd0 : active (read-only)  ...09:13
RoyKanything interesting in dmesg?09:13
RoyKdmesg -T to get proper timestamps09:13
zetheroohttps://paste.ubuntu.com/p/ZGPRcFTDvC/09:14
RoyKstop it again and try 'mdadm --assemble /dev/md0 --run'09:17
RoyKnever seen it go into read-only mode like that before09:17
zetheroooh ... I got it to mount09:17
zetherooat least I think so ... it's mounted read-only though09:17
zetherooUUID=99f7fdf9-6f9a-4069-856e-4db4c529741e/mnt/raidarrayext4noload0009:18
zetherooused the noload option09:18
zetheroois that ok, or should I unmount it?09:18
RoyKgood, get the data off the system first09:18
zetherooafter mount - a I see 'mount: /mnt/raidarray: WARNING: device write-protected, mounted read-only.'09:18
RoyKdid you try with --run?09:19
zetheroono, should I unmount it and do that?09:19
RoyKtry to umount, stop and reassemble with --run to see if it gets up without readonly09:20
zetheroook09:20
zetheroomd0 : active (read-only)09:21
RoyKesc09:21
RoyKsec09:21
zetherook09:21
RoyKhm - you might want to try mdadm -w /dev/md009:23
zetheroomdadm: failed to set writable for /dev/md0: Read-only file system09:23
RoyKdo you have another disk around to test if drives plugged into that thing become writable?09:24
zetheroooh, you mean the USB dock might be the problem?09:24
RoyKI have no idea, but I've never seen disks struggle with getting read-write like this09:25
zetheroook, will try another disk in the same USB dock ...09:25
zetherooplugged in another disk and it seems to also be read-only09:27
zetherooalthough root can write to it ...09:28
RoyKweird09:29
RoyKwhat's in dmesg?09:30
zetheroohttps://paste.ubuntu.com/p/bJhhhSQ9K6/09:31
zetherooI plugged in the two md0 SSDs again09:31
RoyK[Mi Dez 23 10:30:06 2020] sd 1:0:0:1: [sdd] Write Protect is off09:31
RoyKthat's good09:32
RoyKwhat happens if you reassemble the raid?09:32
zetheroonot sure how to do that ... looking it up ...09:33
RoyKmdadm --assemble /dev/md009:34
RoyKor mdadm --assemble --scan09:34
RoyKif there's an md0 already, --stop it09:35
zetheroohey09:36
zetheroomd0 : active raid1 sdd1[2] sdc1[3]09:36
RoyK:)09:36
RoyKnow try to mount it, get the data off and fsck it09:36
RoyKnormally just mounting it should take care of those things missing09:37
RoyKit's jouranlled after all09:37
zetheroohttps://paste.ubuntu.com/p/khs8B4FQfD/09:37
zetheroook09:37
RoyKdid it mount?09:37
oerheksclean, active ync, sounds good09:38
zetherooyes, it mounted with 'defaults' in fstab09:39
RoyKyou may want to umount it and fsck it just for kicks09:40
zetheroobut first get the data off, right? or is fsck safe ...?09:40
RoyKshould be safe09:40
RoyKcan you pastebin lsblk output again?09:41
zetheroohttps://paste.ubuntu.com/p/5CCxVbRx89/09:43
zetherooI'm grabbing the data just to be safe ...09:43
zetherooand then 'fsck /dev/md0' ?09:44
RoyKlooks good09:44
RoyKjust umount before fsck09:44
zetheroook09:44
RoyKit won't let you fsck a mounted filesystem unless you --force it, which is generally an extremely bad idea ;)09:45
zetheroook .. unmounted and going to run fsck on it09:58
zetheroohttps://paste.ubuntu.com/p/bTTwJFcCsq/09:59
zetheroowhere to now? :)10:19
zetherooRoyK: are you still there?10:25
=== falcojr5 is now known as falcojr
zetherooBtw ... I ended up trying Boot Repair on it from an Ubuntu Live Session .. and it's fixed :)11:59
tieinv /part13:37
jinkIs it safe to do-release-upgrade to 20.04.1 LTS yet? :D14:18
lotuspsychje!20.0414:29
ubot3Ubuntu 20.04 (Focal Fossa) is the 32nd release of Ubuntu and is the current !LTS release. Download at https://ubuntu.com/download - Release notes at https://wiki.ubuntu.com/FocalFossa/ReleaseNotes14:29
andolSomeone should maybe update ubot3 to not refer to Ubuntu 20.04 as an !LTS release?15:01
jink!LTS15:01
juliank!20.1016:49
ubot3Ubuntu 20.10 (Groovy Gorilla) is the 33rd release of Ubuntu and is the current regular release. Download at https://ubuntu.com/download - Release notes at https://wiki.ubuntu.com/GroovyGorilla/ReleaseNotes16:49
juliankandol: But ... 20.04 is an lts release16:49
mybalzitchdang, 33 releases of ubuntu already17:02
andoljuliank: Yes, and hence not a !LTS release.17:13
juliankandol: ! is like a hashtag17:13
juliankit's like saying it's the current #LTS release17:13
mybalzitchis systemd-resolve supposed to completely ignore my secondary nameserver entry?17:13
juliank!LTS17:13
juliankarguably ubot3 does not implement the LTS command right now17:14
genii!lts17:16
geniiHm17:16
andoljuliank: Yeah, but given the meaning a ! prefix generally has I'd say the result just ends up being somewhat confusing.17:17
mybalzitchlol, now netplan isn't respecting the routes I've defined. so glad we left ifupdown that acutally worked for this.17:33
mybalzitchgoddamn bugs in 18.0417:37
tewardmybalzitch: if you don't have a question then please don't rant.18:53
tewardas for netplan not respecting the routes you've defined that USUALLY means your routes are wrong18:53
mybalzitchteward: its a confirmed bug in 18.04 of netplan not applying changes properly, and I'll rant all I like thanks very much18:54
tewardno, you won't, not here you won't anyways.  This is a support channel, not a rant channel.18:54
tewardyou can take your rants elsewhere.18:54
mybalzitchokay18:54
DeeJayTwohi22:35
DeeJayTwoI need to restart a service each 5 minutes22:35
DeeJayTwoI made a script with service service_name restart22:35
DeeJayTwoand set to restart each 5 minutes22:35
DeeJayTwoI know it runs fine cause another line in the script is running22:35
DeeJayTwobut the service command doesn't seem to work22:35
DeeJayTwoany idea?22:36
tdsDeeJayTwo: are you running this script from cron or something similar?22:39
oerheksuse a systemd unit, like this? https://unix.stackexchange.com/a/265619 where RestartSec=300s ofcourse22:50

Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!