| plus2equalsme | Got an odd problem with a fresh install of 18.04.1 server. It's seeing an ntfs drive as vfat. | 01:37 |
|---|---|---|
| plus2equalsme | (ntfs format confirmed in windows and debian desktop) | 01:38 |
| ChmEarl | plus2equalsme, ntfs-3g is installed? | 01:39 |
| plus2equalsme | ChmEarl yes, result of install attempt is "ntfs-3g is already the newest version (1:2017.3.23-2)." | 01:41 |
| ChmEarl | sudo fdisk -l /dev/sdX <-- see if there is an ESP and an NTFS | 01:42 |
| plus2equalsme | Standby, need a couple minutes (drive currently disconnected, stupid of me) | 01:43 |
| plus2equalsme | hmmmmm, odd response, stand by while I check a few more things | 01:48 |
| plus2equalsme | ChmEarl sudo fdisk -l shows "/dev/sdb1 63 3906963395 3906963333 1.8T 7 HPFS/NTFS/exFAT" | 01:52 |
| plus2equalsme | blkid though shows it as type vfat, but, I also got a number of errors when I connected the drive (usb) | 01:52 |
| plus2equalsme | I can get the errors into a pastebin of some sort if you need to see them | 01:53 |
| ChmEarl | plus2equalsme, id=7 is ntfs | 01:54 |
| plus2equalsme | That's why I got confused. Let me get the errors on device attach into a paste, I'm starting to suspect that's part of the cause. | 01:55 |
| plus2equalsme | ChmEarl from dmesg as I plugged the device in https://paste.ubuntu.com/p/JdRzGdgrsH/ | 02:02 |
| ChmEarl | mount /dev/sdb1 /windows/D -t ntfs-3g | 02:12 |
| ChmEarl | prep /windows/D first | 02:12 |
| ChmEarl | mkdir -p /windows/D | 02:13 |
| ChmEarl | plus2equalsme, if it mounts as ^^ then you can dial in some options for user perms | 02:15 |
| plus2equalsme | Getting permission denied on mkdir | 02:16 |
| ChmEarl | plus2equalsme, you can use /mnt, I'm used to OpenSuSe conventions | 02:17 |
| plus2equalsme | no worries. I did get it to mount using sudo on the commands | 02:18 |
| ChmEarl | plus2equalsme, sdb1 is a data drive already populated? or its new and empty? | 02:21 |
| plus2equalsme | Already populated | 02:21 |
| plus2equalsme | ChmEarl I do have enough disk space elsewhere to temporarily move the data if a format is necessary, but I'd like to keep it as NTFS if possible | 02:34 |
| plus2equalsme | Should I be able to use fstab to mount it even with these errors? | 02:35 |
| ChmEarl | plus2equalsme, once you mounted it as above did the filesystem look/behave as you expected? | 02:40 |
| ChmEarl | plus2equalsme, what you showed in the pastebin is normal when there is no automount | 02:41 |
| plus2equalsme | ChmEarl There were two directories that I didn't expect, one a windows trashcan (understandable, it was recently on a windows system) and the other is called 'System Volume Information' | 02:42 |
| plus2equalsme | Good to know the pastebin is normal | 02:42 |
| ChmEarl | plus2equalsme, try using fstab with noauto so `mount windows/D` does the job | 02:42 |
| plus2equalsme | ChmEarl just to make sure I understand, if I use noauto in an fstab I would have to manually remount any time I have to reboot the machine? | 02:44 |
| ChmEarl | I use this with a data drive:/dev/sdb1 /windows/D ntfs-3g rw,users,gid=users,umask=0002,nosuid,noauto 0 0 | 02:44 |
| ChmEarl | note the `noauto` | 02:45 |
| ChmEarl | and I put target user in group users | 02:45 |
| plus2equalsme | understand | 02:46 |
| plus2equalsme | I'm going to have to finish working on this tomorrow though. Thank you for the help. | 02:48 |
| ChmEarl | plus2equalsme, manually remount: yes.. a data drive is always needed | 02:48 |
| ChmEarl | isn't always needed | 02:49 |
| plus2equalsme | ChmEarl, for my usecase, it is. But, it is easy enough to ssh and remount, just wanted to make sure I knew that it'd be necessary | 02:50 |
| === elsheepo is now known as beatzz | ||
| === Helenah is now known as MikeyTheEngineer | ||
| === MikeyTheEngineer is now known as MikeyTheGreasyEn | ||
| === MikeyTheGreasyEn is now known as TomTheTrooper | ||
| === TomTheTrooper is now known as Helenah | ||
| chiiiiiz | Hello | 15:23 |
| chiiiiiz | I need help with lvm backup | 15:23 |
| chiiiiiz | We have a server with 1 hd, 931 Gb. 1 Partition for swap, 1 partition for /boot and the rest is a vg. | 15:24 |
| chiiiiiz | No, sorry, swap is part of the vg | 15:25 |
| chiiiiiz | 2 lv : swap and /. / ist 903 Gb!!! | 15:25 |
| chiiiiiz | We have important data to backup before everything will be virtualized. We can't reboot, since we have jenkins running on the server, and the last time we had an update and a reboot, everything went down, jenkins did not work. | 15:27 |
| chiiiiiz | My question is: how can I backup the data. Given that 33% of the/ is used. I read about snapshots, but I do not understand were the snapshots are stored. | 15:27 |
| chiiiiiz | I have no choice, I did not setup the server, and I have to cope with that. | 15:28 |
| chiiiiiz | can I store the snapshot on a USB HDD? And then mount it and backup the data as an image and as a tar file? | 15:29 |
| chiiiiiz | Thanks for your insights | 15:29 |
| chiiiiiz | I have thought about shrinking the / lv, but since there is (according to the doc on the internet) a risk af data loss, I 'd rather not try this. | 15:30 |
| blackflow | chiiiiiz: first on google for "LVM snapshots" http://tldp.org/HOWTO/LVM-HOWTO/snapshots_backup.html | 15:42 |
| blackflow | meanwhile, I'd make a rsync backup too. | 15:43 |
| blackflow | btw... LVM snapshots are to make an atomic copy of the data, not for moving somewhere else by itself | 15:43 |
| blackflow | so with rsync, you make a LVM snapshot and then rsync it elsewhere | 15:43 |
| chiiiiiz | @blackflow: THanks, I already read this. What is not clear to me, is if the snapshot is stored somewhere on the volume you are "snapshooting", or if you do have to store it somewhere else. | 16:35 |
| chiiiiiz | what is the advantage of rsync over other methods? | 16:36 |
| chiiiiiz | I will have a USB HD available tomorrow with 1Tb of space. | 16:37 |
| TJ- | chiiiiiz: a snapshot is a list of all the extents used by the LV at a point in time. From then on, if the original volume writes anything then it uses Copy-On-Write which allocates a new block for the new data in the original LV - the blocks pointed to by the snapshot remain the same | 16:37 |
| TJ- | chiiiiiz: so space will only be consumed if the original LV is writing data because copies of the blocks need to be made | 16:38 |
| TJ- | chiiiiiz: so, you could snapshot, mount the snapshot, and use rsync to copy files out of it to your USB mass storage | 16:38 |
| chiiiiiz | so I do it directly from the / lv. No need to do the snapshot from another mount or partition. I will only need another mount or partition when backing up the data after mounting the snapshot | 16:38 |
| chiiiiiz | TJ-: thanks, it is clearer now. But a rsync of the / does not allow me to 'reuse' the backup for another /, if need be. It is not like a dd. | 16:39 |
| TJ- | chiiiiiz: correct. the snapshot is another LV as far as the system is concerned, so you can mount that somewhere and do what you need with it | 16:39 |
| chiiiiiz | I fear a dd of 300 Gb lasts days... | 16:40 |
| chiiiiiz | dd make an image only as big as the data, or as big as the partition it backs up? | 16:41 |
| TJ- | chiiiiiz: what you could do, if the VG has the free space, is shrink the file-system inside the snapshot, and the snapshot LV itself, then copy the result over | 16:41 |
| TJ- | chiiiiiz: there's another, sneaky, way to do this you know | 16:41 |
| chiiiiiz | The VG is 903 Gb, the lv is 903 Gb, and only used up to 33% | 16:42 |
| TJ- | chiiiiiz: instead of a snapshot, wait until you've got the USB drive connected, make it a PV (physical volume), add it to the existing VG, then change the rootfs LV to be a mirror with the USB PV used as the other 1/2 of the mirror. Give it time to sync across fully to the USB, then temporarily freeze the rootfs LV, break the mirror, and unfreeze. You'd get an exact copy of the LV on the USB | 16:43 |
| chiiiiiz | So it has the space. As said, I do not want to restart. So if shrinking the LV/FS does not need a restart, I am go for it. | 16:43 |
| TJ- | chiiiiiz: I'd suggest experimenting on another PC for either snapshot or mirror sceanrios first, using smaller data sizes, to practice the commands and checking the results, though | 16:43 |
| TJ- | chiiiiiz: what file-system does the rootfs use? Generally, you can extend, but not shrink, a live file-system online | 16:44 |
| chiiiiiz | OK I see the logic, but I need to have iot done until tomorrow, no time to experiment... | 16:44 |
| chiiiiiz | ext4. It is a ubuntu-server, quite standard, albeit the lvm thing. | 16:44 |
| TJ- | chiiiiiz: take the snapshot then, use it as the basis for whatever method you decide on | 16:44 |
| chiiiiiz | can a dd run on a live system? | 16:46 |
| TJ- | chiiiiiz: right, so you could snapshot, "resize2fs -M /dev/mapper/VG-snapshot", "lvreduce --size XXX /dev/mapper/VG-snapshot" .. read "man lvreduce" for the details. You would need to ensure you do not shrink it smaller than the ext4 file-system | 16:47 |
| chiiiiiz | I do not think so | 16:47 |
| TJ- | chiiiiiz: it can run yes but you'll get an inconsistent image. dd from a read-only snapshot would be OK though | 16:47 |
| chiiiiiz | I think I'll go for a dd of a snapshot, to be able to reuse the dd image to recreate a partition if need be. | 16:48 |
| chiiiiiz | TJ-:again my question. If a dd my /, will my image be 903Gb or 350 Gb (33% of it). | 16:52 |
| TJ- | chiiiiiz: word of warning - transferring that amount of data in one go over USB could cause corruption - depends on the USB<>SATA bridge, but I've seen LOTS of those devices overheat and fail when doing large continuous transfers | 16:53 |
| TJ- | chiiiiiz: the image size will be whatever the size of the containing block device is - in your case, the size of the LV containting the rootfs | 16:54 |
| chiiiiiz | TJ-, I feared this would be an issue too... | 16:54 |
| TJ- | chiiiiiz: this is why shrinking first is so useful when you've a large file-system that is only partially used | 16:54 |
| chiiiiiz | TJ-: so rsync is in this optic a much better solution. Although in case of a crash we would have to reinstall the server. | 16:57 |
| chiiiiiz | and maybe rsync is also faster?? | 16:57 |
| chiiiiiz | if I format my USB HDD in ext4, does rsync copy the permissions? | 16:58 |
| chiiiiiz | or will I have to use a ACL tool to copy the permissions in order to recreate them if need be? | 16:58 |
| TJ- | chiiiiiz: yes, rsync can/will copy onwership and modes | 16:59 |
| chiiiiiz | OK, rsync it is!!! | 17:01 |
| chiiiiiz | Thanks a lot. | 17:01 |
| TJ- | chiiiiiz: make sure to read the man-page and test on a small samlpe first :) | 17:06 |
| chiiiiiz | I want to format a encrypted partition ( I know the passphrase), so that the partition is not encrypted any longer. Shall i simply mount it, decrypt with the passphrase, start gparted and reformat my partition? I have read about it but I am not sure this change the encrypting... only that it changes the FS. | 17:21 |
| mybalzitch | you should be able to just newfs the partition. no need to mount it/decrypt it. Just blow it away | 17:35 |
| mybalzitch | if you want to retain the data, you'll have to mount it, decrypt it, copy the data to a place holder, then blow away the partition and copy the data back | 17:36 |
| chiiiiiz | mybalzitch: no need to retain any data, only need to remove the cryptage. | 17:50 |
| === JanC_ is now known as JanC | ||
| TJ- | chiiiiiz: is it LUKS/dmcrypt? you can use "cryptsetup luksErase/erase ..." respectively | 18:07 |
| TJ- | chiiiiiz: you'd also likely need to remove the associated entry from /etc/crypttab and /etc/fstab | 18:08 |
| chiiiiiz | TJ-:no entries in /etc/fstab or /etc/crypttab... it was a USB HDD | 18:10 |
| chiiiiiz | does ntfs FS store permissions? Still not I guess... | 18:12 |
| TJ- | chiiiiiz: ntfs uses ACLs | 18:13 |
| chiiiiiz | so if I make my backup on a ntfs formated USB HDD, I can still use the backup with the permissions to restore some data... I have been asked to store the backup on a Windows server... meaning formatting my USB disk in ntfs... | 18:14 |
| chiiiiiz | or I create 2 partitions on my USB disk. The first in ntfs, and a second in ext4.. just in case... | 18:15 |
| TJ- | charlton: it won't map Linux ownership/modes, no. I thought you were asking if NTFS supports such things | 18:30 |
| TJ- | grrr | 18:30 |
| TJ- | chiiiiz: it won't map Linux ownership/modes, no. I thought you were asking if NTFS supports such things | 18:30 |
| === elsheepo is now known as beatzz | ||
Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!