| === smb` is now known as smb | ||
| rbasak | apw: morning! Are you aware of any races in overlayfs that might be observable from userspace? I have a tar complaining "Directory renamed before its status could be extracted" but it seems to affect overlayfs only. | 07:00 |
|---|---|---|
| rbasak | 3.8.0-19-generic | 07:00 |
| rbasak | (raring) | 07:00 |
| apw | rbasak, there are any number of semantic issues with overlayfs, though it could be a bad assumption by the creating app as likely as anything | 07:33 |
| rbasak | apw: AFAICT, it's tar. Just unpacking a fairly big tarball (from a pipe, being delivered from a different filesystem) | 07:35 |
| apw | rbasak, is there a bug filed? if you get me the number i can have a look when i get to the office, in transit right now | 07:35 |
| rbasak | apw: unfortunately it's really hard to reproduce. I can do it reliably, but it requires an autopkgtest run of about three hours. I haven't filed a bug because I'm not sure it is a bug in the first place (or if so where it is), but I'd appreciate if you could take a look with me when you get in. | 07:36 |
| rbasak | apw: oh, you're sprinting this week, aren't you? I don't want to take your time away from that, actually. | 07:37 |
| apw | rbasak, heh, if we have time we have time :) still good to chat about it | 07:42 |
| rbasak | So I'll start explaining the problem, and you'll see it when you're ready :) | 07:45 |
| rbasak | I'm writing an adt-virt-lxc. It seems to work, and AFAICT, without any bugs, at least not that are relevant to this problem. I can get it to run in clone mode, or ephemeral mode. With clone mode it uses lxc-clone and overlayfs isn't involved. With ephemeral mode, it uses lxc-start-ephemeral which sets up an overlayfs against the template container's rootfs. In this case I'm using --keep-data, so the upper backing store is on a normal fs. | 07:46 |
| rbasak | After the container is started, the system uses lxc-attach to run commands inside the container (which has its root fs as the overlayfs in the failure case). | 07:47 |
| rbasak | adt-run transfers files in and out of the container using tar (so it's generic and can work with other virt mechanisms) | 07:48 |
| rbasak | When running in clone mode, this seems to work fine. But when in ephemeral mode (which uses the overlayfs), I get tar bombing out with many "Directory renamed before its status could be extracted" errors. | 07:48 |
| rbasak | tar's stdin is connected to another tar's stdout. The latter tar is running outside the container. The former is extracting inside the container. So this is transferring files into the container. | 07:49 |
| rbasak | I have a suspicion (but am not sure) that the problem occurs when the tar running outside the container has finished all I/O, so the only thing going on is that the pipe is being flushed out. | 07:50 |
| rbasak | This is inside a VM that I'm evidently sharing with I/O hungry users. Disk I/O seems to be extremely slow on this VM. So perhaps tar is extracting entirely into buffers in this case? | 07:51 |
| rbasak | The code path inside adt-virt-lxc is exactly the same in both modes. The only difference is the way the containers were created. Once they are up, both situations do exactly the same thing. The only difference is the underlying configuration of the container (native fs vs. overlayfs). | 07:52 |
| rbasak | So AFAICT, it's a problem affecting tar extractions on overlayfs only. | 07:52 |
| apw | rbasak, sounds compelling, and overlayfs does have some awkward semantics when copy up occurs | 07:54 |
| rbasak | I've just looked up the tar source to see exactly what it's complaining about | 07:55 |
| rbasak | Looks like it doesn't like it if st_ino or st_dev have changed. | 07:57 |
| rbasak | Against the known path/filename and an fstatat against the previously open fd, perhaps? SOmething like that. | 07:59 |
| zequence | apw: Are you going to DebConf by any chance? | 08:04 |
| apw | zequence, sadly no | 08:04 |
| smb | apw, morning | 08:04 |
| apw | smb, moin ... | 08:04 |
| apw | smb, having fun yet? overloaded and broken the coffee machine? | 08:05 |
| smb | apw, not much yet... reminds me | 08:05 |
| smb | coooffeee machiiine | 08:05 |
| zequence | apw: Maybe some other time then :). You want to do another upload of the saucy kernel before handing it over? | 08:07 |
| apw | zequence, yeah indeed, then we can sort out the sponsorship process for that, and we should work towards getting you upload for it | 08:09 |
| zequence | apw: That would be great. I've been meaning to get some more documented packaging experience beyond the kernel, but it's a bit slim so far. | 08:11 |
| apw | zequence, well getting a package upload is a good start on the path | 08:12 |
| rbasak | apw: over to you then I guess? | 08:17 |
| rbasak | apw: if the conclusion is "yeah, it does that" with no hope for a fix, that's fair enough. adt-virt-lxc defaults to clone mode - I was just hoping to make it faster for development purposes by adding --ephemeral. We can still use that with the caveat that if it breaks then the developer should switch to clone mode :) | 08:18 |
| rbasak | I don't think we'll use this in production, but if we do, then we'll use clone mode. | 08:18 |
| apw | rbasak, ack | 08:20 |
| === fmasi_afk is now known as fmasi | ||
| === fmasi is now known as fmasi_lunch | ||
| === fmasi_lunch is now known as fmasi_afk | ||
| === rtg is now known as Guest10102 | ||
| === fmasi_afk is now known as fmasi | ||
| === kentb-out is now known as kentb | ||
| === bdmurray_ is now known as bdmurray | ||
| === fmasi is now known as fmasi_afk | ||
| === kentb is now known as kentb-out | ||
Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!