/srv/irclogs.ubuntu.com/2015/09/11/#ubuntu-kernel.txt

alkisgHi, I think I've found a bug in overlayfs, could someone please check that I'm not doing something wrong, before I file it upstream?07:32
alkisgAs root: cd $(mktemp -d); mkdir lower upper work root; truncate -s 4G lower/sparse; mount -t overlay -o workdir=work,upperdir=upper,lowerdir=lower overlay work; truncate -s 0 work/sparse; umount work07:32
alkisgThe error: truncate: cannot open ‘work/sparse’ for writing: Value too large for defined data type07:32
alkisgI.e. the truncate function call, fails on big files, when the underlying file system is overlayfs07:34
apwalkisg, what you are doing looks valid to my eye07:36
alkisgThank you apw, should I send a mail to the unionfs kernel ML?07:36
apwalkisg, to the maintainers of overlayfs in the first instance, as in whats in MAINTAINERS for fs/overlay07:36
alkisgThank you very much07:37
mjg59alkisg: https://github.com/docker/docker/issues/11700 ?07:38
mjg59We ran into that07:38
apwalkisg, seems like a copy up issue, as it is failing in open, opening it for write (if i am reading the trace right)07:41
alkisgmjg59: looking into that...07:41
alkisgapw, it fails for 2200M, it succeeds for 2000M, but it takes a very long time to truncate it, like it's trying to copy the file contents which imho it shouldn't07:42
apwalkisg, it is absolutly copying it up07:42
apwas truncate is "open for write, then ftruncate(N) on the fd"07:42
mjg59alkisg: Presumably failing at 2G07:43
alkisgRight07:43
alkisgHmm so when I want to empty some files, `truncate -s 0` isn't really an efficient way to do it... :)07:44
alkisgI thought the truncate function would special-case 0 there07:45
apwalkisg, the problem is not truncate, but the fact to use it you have to open the file07:46
alkisgHmm and also overlayfs doesn't use blocks, lists etc, so it wouldn't be able to actually "truncate" a file in half without copying it first07:47
alkisgI.e. mark the rest of the blocks as unused etc07:48
apwalkisg, right, though at the time we open it all we are saying is "open this so i can scribble on the current contents"07:50
apwand in that case all it can do is make sure it is on upper07:50
alkisgGotcha07:50
apwthat it fails on 2G+ is broken, but it will always be _slooow_07:51
alkisgRight, so I'll change the ltsp code so that it doesn't use truncate, but the bug still needs to be reported upstream07:52
alkisgI couldn't an official source for the maintainer though07:52
alkisgI think it's miklos, but I don't see his name in overlay.txt...07:52
alkisgs/an/find/...07:53
apwit is miklos yes, the main MAINTAINERS file in the top level lists them all07:54
apwalkisg, to: miklos cc:linux-unionfs07:56
alkisgty!07:56
alkisgReported to http://permalink.gmane.org/gmane.linux.file-systems.union/408 and fixed in LTSP, https://bugs.launchpad.net/ltsp/+bug/1494660. Thank you guys.09:18
ubot5Ubuntu bug 1494660 in LTSP "Replace truncate with tee to avoid overlayfs issues with big files" [Low,Fix committed]09:18
apwalkisg, i also see someone has worked up a fix for the fundamental issue, though if you can avoid the copy up ... all the better09:18
alkisgWe avoided it, but e.g. `useradd` fails on live booted PCs because it's using truncate (lastlog == sparse file of 3 GB on a system that uses LDAP)09:19
alkisgSo we'll keep an eye on that in case we find other side effects until overlayfs is fixed upstream09:20
apwalkisg, right, i am saying someone on linux-unionfs looks to have a simple fix09:20
alkisgI understood. Ah, could you give me the link?09:20
alkisgAh, a reply to mine, ok09:23
alkisg:)09:23
=== alkisg is now known as work_alkisg
=== ghostcube_ is now known as ghostcube
psivaacking: hello, about that failing health-check test, (which passed in today's run btw) do you think it would be beneficial to retain the old host  for any debugging.14:46
psivaaWe're having to handover the machine to the IS, but can retain for another week14:46
ckingpsivaa, i don't think we're going to gain much by keeping hold of the old host14:48
ckingso let it go14:48
psivaacking: ack, thank you14:48
TJ-apw: ping re: cryptsetup15:35
apwTJ-, i am blank, remind me15:40
TJ-apw: You signed off a couple of recent package uploads; I was wondering if you're semi-maintaining it ?15:45
apwi might :)15:45
TJ-apw: I work with LUKS/dm-crypt/cryptsetup extensively, and there are a couple of related issues showing with Wily which lead to non-boot regresssions15:46
apwTJ-, sounds bad15:46
apwi think i just merged it, so if you want to change it thats all good15:47
TJ-apw: do you have 5 minutes to discuss/bounce ideas?15:47
apwsure go ahead15:50
TJ-apw: I've used LUKS with keyscript= in crypttab for several years. The basic script simply takes the KEYFILE passed to it and reads it. That's worked fine with Upstart. Now, with systemd-cryptsetup, that is broken because it doesn't support keyscript.15:53
apwTJ-, "you don't want to do it like that"15:53
apwgood old systemd and its "i do everything you should use and nothing you do" support15:53
TJ-apw: systemd-cryptsetup does support finding/loading the keyfile directly, however, so if it doesn't require any special steps that will still work15:54
TJ-apw: systemd-cryptsetup complains about unknown option keyscript= though, regardless. If the user takes notice of that and removes the keyscript= option from cryptsetup it breaks the initramfs cryptroot script causing failure to unlock te container with the rootfs15:55
TJ-apw: Currently the cryptsetup initramfs hook script knows its a problem and will report "cryptsetup: WARNING: target LUKS_OS uses a key file, skipped"15:56
TJ-apw: That doesn't help the user though. I'm wondering if we shouldn't have the script simply assume the crypttab 'initramfs' option and do the cryptsetup install to the intrd.img regardless, rather than having no cryptdisk support at all15:57
TJ-apw: The hook script doesn't know if the LUKS slots may also contain a passphrase which the user can type instead of the key-file, but it does know cryptsetup is required to unlock the container with the rootfs in15:58
apwTJ-, you mean we are keying off that command line option to work out what file to put in the initrd16:00
apwTJ-, but systemd tells us to get rid of it?16:00
TJ-apw: we're keying off the crypttab entry for the container, and not installing cryptsetup at all if there is a key-file defined but no kescript16:01
TJ-apw: systemd-cryptsetup complains each time it sees 'keyscript=' which could lead to users (like me!) trying to tidy things up16:02
apwTJ-, so i tend to agree if we know root needs crypt to unlock we should always install that16:02
apwTJ-, even if it won't work, it definatly won't work without16:03
TJ-apw: It can be over-ridden in crypttab using the 'initramfs' option, but its not obvious16:03
TJ-apw: OK, I'll work up a patch and a bug report and let you know when its tested16:03
apwthanks.16:03
apwanything that gets them a step nearer to working is good, if root is in a container that needs it, it should be in the initrd even if we don't have enough info to actually open it16:04
TJ-Yeah... with keyscript= it's not a problem, so for users upgrading they should be OK, but for new users it could be an issue16:05
TJ-Getting the required support into systemd-cryptsetup is going to be a big job; upstream wants to use the kernel key-ring and the  PasswordAgents API16:08
apwTJ-, whatever is hard for everyone i am sure16:09
TJ-apw: makes sense, but a lot of work :)16:09
TJ-PA means easy support for SmartCards and other devices withou hackish scripts16:10
TJ-apw: got a fix in, cryptroot::get_device_opts() expands the existing  warning, recommends a pass-phrase, and doesn't return an error, so the device is included. "cryptsetup: WARNING: target LUKS_OS uses a key file, but no keyscript is set. Please ensure there is also a typed pass-phrase set"16:43
jdstrandcking: thanks for the super-quick turnaround for thermald! :)17:00
ckingjdstrand, no problem, I had the fix already,so it was a no-brainder17:01
cking*brainer17:01
* jdstrand nods17:01
jdstrandstill though-- very nice :)17:01
jdstrandcking: hope you enjoy your weekend :)17:01
ckingcheers, thanks, you too17:01
ckingjdstrand, and thanks for testing it :-)17:02
jdstrandI think it was literally the least I could do considering the rapid turnaround :)17:02
jdstrandbut, you're welcome :)17:03
TJ-apw: There are other similar bugs (e.g. bug 1487127) I'll squash whilst I'm at it, so they can be rolled up into one update. This issue is bug 149485117:16
ubot5bug 1487127 in cryptsetup (Ubuntu) "cryptsetup preventing proper boot swap AND root on crypt device together" [High,Triaged] https://launchpad.net/bugs/148712717:16
ubot5bug 1494851 in cryptsetup (Ubuntu) "initramfs cryptroot hook script doesn't install cryptsetup if keyfile but no keyscript" [Undecided,In progress] https://launchpad.net/bugs/149485117:16
=== work_alkisg is now known as alkisg

Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!