[00:47] Can anyone point me to the status of aufs and it's replacement? [00:48] ubuntu status and/or upstream status? [00:48] as far as ubuntu is concerned, it's gone from karmic [00:48] it was turned down from upstream recently, too [00:49] that is sad. :( can you point me to the replacement? [00:56] nick_schembri: whatever Fedora/Red Hat is using [00:59] peter graner talked about dm-snapshot. I'm just trying to make sure I get the correct project. [01:00] that's correct [01:00] dtchen: thanks for the help. [01:02] nick_schembri: yw [04:19] question: the mainline daily ppa hasn't had an i386 build show up in nearly a week [04:19] is that known? [04:20] cwillu: what happened with your kernel lockups? [04:21] johanbr, nothing yet really [04:22] johanbr, although kms screws up suspend even quicker :p [04:22] :) [04:22] did you try the older kernel with newer userspace? [04:23] ya, that seemed to work fine, although I didn't run it for more than an hour or two, and I've had the newer kernel work that long once in a while [04:24] I'll try that again tomorrow for work [04:25] alright [04:33] latest daily that has an i86 panics on boot with a allocation of page_cgroup was failed [04:36] oh [04:37] * cwillu boots a mainline-ppa 2.6.29 [04:39] oh right, I remember why I hadn't tried 2.6.28 during a full work day [04:39] ext4 delete crasher :) [04:39] although that didn't exist in the mainline 2.6.28, so I'll grab one of those [04:45] 2.6.29-020629 looks good so far [04:53] So I have in my hands at least partial docs for programming a programmable HD DSP/Amp from AMD's HTPC platform. Where should I start looking for coding up a driver for it? [04:54] is there any sort of ALSA interface for programming a programmable dsp? [04:57] looks like it's controlled over the HD Audio bus; where do I find docs on using it? [05:00] johanbr, 2.6.29 seems to be working, I'm going to try that one tomorrow unless it gives me some trouble tonight [05:00] alright [05:00] good luck [05:32] johanbr, quick question (I'm on my way out, but I'll read when I'm back), if this 2.6.29 works, am I in bisect-land, and if so, should I be bisecting between 2.6.29 and an early 2.6.30, or is there a better starting point? (I remember you saying something about big suspend changes landing early on) [07:26] hey guys when trying to write a driver for an undocumented piece of hardware where should i begin in the reversing process [10:06] zeroprog: what kind of driver? and what's its interface? [10:12] * apw wonders how Keybuk's machine is ... [10:13] there must be communities who specialise in the reverse engineering process. i think nouveau was whole reverse engineered from the hardware. they may be a good group to approach [10:21] apw: still trying to work out why these renames fail [10:26] Keybuk, which renames, i thought all should fail pretty much [10:26] since it should be a rename of something on cow to something else on the cow [10:26] well you should be able to create a new file, then rename it elsewhere, no [10:27] yeah that is true, and those don't work either? [10:30] that's what I'm investigating [10:30] it seems not [10:31] t [10:31] that would be limiting indeed [10:49] apw: of course, this could be an APT bug, arguably [10:50] you would think that apt would have hit this issue before [10:51] dunno, I can't work out what it's doing at the moment ;) [10:51] the syscalls don't make much sense [10:51] heheh .... if you can't figure it out we are in a lot of trouble :) [10:52] Invalid cross-device link (/var/lib/apt/lists/security.ubuntu.com_ubuntu_dists_karmic-security_Release -> /var/lib/apt/lists/security.ubuntu.com_ubuntu_dists_karmic-security_Release). [10:52] now [10:52] correct me if I'm wrong [10:52] but aren't those the same path? [10:52] Their md5sum is identical, at least. [10:52] to my eye yes [10:53] interessting behaviour [10:54] * Keybuk clears out partial and tries again [10:54] ok, I get a different error now [10:54] apt clearly has a bug there, it's just one you can only hit if you first experience the failure mode of a union mount [10:56] I clearly need a smaller test case than apt [11:04] haha [11:04] strace strace ... [11:04] gives unexpected output ;) [11:05] iiiiiinteresting [11:06] I think they just didn't bother to implement rename at all ;) [11:06] root@union:/# echo "foo" > a_file_in_the_cow [11:06] root@union:/# ./overwrite a_file_in_the_cow [11:06] rename: Invalid cross-device link === smb is now known as smb_afk [11:12] apw: see my mail ;) [11:14] it shouldn't be a problem for rename ("some_path", "some_path.bak") to fail [11:14] because apps doing that should [11:14] a) check the return value and cope [11:14] b) not do that [11:15] but the other three rename()s I highlighted there should work, because they're not doing much cross-device [11:16] yeah i think the first one they lilley don't support, but it sounds like a sane description in tha tyou can do it all at the top level only ever looking at the toplevel [11:17] the other two sound like bugs, but i think there is rename code in there [11:17] so it must be broke [11:18] yeah [11:18] it should be easy enough to do [11:19] if source is on the top layer, the rename() should always be permitted [11:19] yeah that i think makes sense [11:19] if source is on a lower layer, the rename() should return EXDEV [11:19] if the source is on the top layer, and there's something of the same name on a lower layer, the rename() should be permitted and leave a whiteout in the old location [11:20] (otherwise you could overwrite a file on the top layer, rename() it, and suddenly the old file comes back) [11:20] its likely you could _always_ leave a whiteout, so you don't have to check the lower level [11:20] true [11:20] though i think once the names are copied up the first time, they directory becomes opaque anyhow [11:20] there is some mind bending optimisation in there [11:21] rename(src-on-lower, dest-on-upper) can't work because you need to copy up [11:21] but that's ok [11:21] sounds like we might be able to live with that [11:21] that the critical erm atomics are top to top only [11:22] right [11:22] I'd be interested if anything is actually trying the other form of rename() [11:22] since it shouldn't [11:22] right other than mv [11:22] you shouldn't rename() files to ".bak" because you end up with the permissions round the wrong way [11:22] and it really should cope [11:22] you should create .bak and copy into it before trunc()ing the original [11:23] and if you have a genuine reason to rename() a file, you should at least cope with EXDEV ;) [11:24] Keybuk, I'm curious: how does that interact with the ext4 delayed allocation semantic workaround? [11:24] i.e., does it still trigger an implicit fsync, or does that end up looking like something else? [11:25] cwillu: ? [11:25] -ECONTEXT [11:26] + /* renaming on unions is done by the user-space */ [11:26] + error = -EXDEV; [11:26] + if (is_unionized(oldnd.path.dentry, oldnd.path.mnt)) [11:26] + goto exit5; [11:26] + if (is_unionized(newnd.path.dentry, newnd.path.mnt)) [11:26] goto exit5; [11:26] [11:26] Keybuk, dispite the commentry in the patches saying that this is supported it appears that rename is simply not supported at all, fail. [11:26] Keybuk, currently with ext4, if you rename over top an existing file, you get an implied fsync of the new contents. This 'fixes' the empty-file-after-a-crash bug that people complained about [11:27] ext4 does not support union [11:27] cwillu: this is all done at the VFS level [11:27] its ext2 and tmpfs only right [11:27] sorry, I didn't read enough scrollback. I thought you were discussing application behaviour, not unions [11:27] cwillu: if you were using an ext4 top layer, and both the source and destination were on that, its semantics would apply [11:28] apw: I didn't think the patches were ext2 specific [11:28] the upper level fs has to have the ability to support whiteout entries [11:28] and i only see patches for tmpfs and ext2 for that [11:28] ext4 is based off that ext2 code though, right? [11:29] i don't think it is in this context [11:29] but we should ask. for our use cases i don't think we care [11:29] as any volatile and any non-volatile upper layer is fine [11:30] right [11:31] yeah looks like it's ext2 only - but relatively trivial to patch to ext3 and ext4 later [11:31] yeah and for the two uses we absolutly need for this work for karmic i think tmpfs and ext2 would be fine right? [11:32] yes [11:33] though I think the existing COW is ext3 [11:33] which briefly surprised me [11:33] * Keybuk tries to remember whether USB and journalling filesystems are friends or enemies [11:34] its all very perplexing [11:35] Eurasia! [11:36] Subject: Re: [PATCH 15/32] union-mount: Documentation [11:36] Message-ID: <20090618190558.GB22206@shell> [11:36] apw: val posted the old rename() patch to lkml [11:37] might be worth looking at [11:37] see if it's possible to keep rename() on top layers without the copy-up [11:37] yeah i think your minor exceptions would be implementable as they occur in one layer only [11:46] Keybuk, well it still seems to apply to the current stack, i'll apply it and get a kernel building and we can see what happens [11:47] apw: I was curious if it has been accepted upstream [11:47] vfs union-mounts ? [11:47] not yet, they are still being developed, i think they were given the nod as the right approach [11:47] apw: Yup [11:47] which is more than half the battle [11:48] they seem much less invasive than the alternatives to my eye [11:48] plus aside from a few minor issues, they work [11:48] Well, so did aufs :-P [11:48] yeah that they are this close to working so soon is always a good sign [11:49] cirtainly worth us spending a few man days playing with anyhow [11:59] Keybuk, i do wish .deb's were rsyncable [11:59] you can make them rsyncable [12:00] i can? how? [12:00] * apw slobbers at the thought [12:03] there's a patch to add it directly to dpkg [12:03] oh hrm. so not just a simple option then [12:03] you can manually recompress a .deb after making it with gzip --rsyncable as well [12:04] oh now that i would be happy to do [12:04] is that documented anywhere/so obvious i should know [12:06] apw: ar x .deb [12:06] gunzip *.tar.gz [12:06] ar p THINGY.deb | gunzip -c | gzip --rsyncable > data.tar.gz [12:06] gzip --rsyncable ... [12:06] ar r THINGY.deb data.tar.gz [12:06] err [12:06] ar p THINGY.deb data.tar.gz | gunzip -c | gzip --rsyncable > data.tar.gz [12:06] obv [12:07] heh thanks both, will give it a go [12:07] these 28 min uploads are killing me === smb_afk is now known as smb [15:19] anyone around? [15:19] i'm looking at linux/connector.h in karmic [15:20] shouldn't connector unique ids be uniq? [15:21] i remember having problems with drbd when it had the same cn_idx as v86d [15:21] both modules couldn't be loaded at the same time [15:22] actually, they could, but only the first one loaded worked, and the other didn't [15:23] by bad, they are unique :) [16:04] Keybuk, ok finally got that rename patch to build at least, so i've uploaded it to the same PPA. should be there in an hour ish [16:05] apw: tvm [16:06] its taken me an hour to get the PPA to accept it, what a waste of time [16:08] Keybuk, actually waht arch are you testing [16:09] i386 [16:10] i'll get you a dirty binary from my build farm ... much quicker [16:10] getting overly into PPAs and actually they are slowing me down [16:21] apw: just sent you a note about GEM/PAE. Have you any test results from your patch set? [16:21] not seen anything will poke them [16:32] is the kernel packaging (debian/ dir) kept in git as well? [16:32] mdz: yes [16:33] rtg: thanks. so if I have a packaging patch to submit, I should do it the same way as one would a kernel code patch? [16:33] mdz: exactly [16:33] Hi all. Digging with a collegue of mine today on bug #243393, we found that a one line fix (removing a warning) could fix it. Do you think this low risk fix could be integrated in the next hardy SRU kernel upload? [16:33] Malone bug 243393 in linux "dmesg is flooded with warnings in kvm/mmu.c" [Low,Fix released] https://launchpad.net/bugs/243393 [16:34] mdz: as an example, see how Luke has done it for the ports config changes: 'git://kernel.ubuntu.com/themuso/ubuntu-karmic.git master' [16:39] lionel, In general it could. But as this is a WARN_ON it needs to be made clear this is not just shutting the eyes in the face of some evil... [16:39] smb: agreed, but considering that it is now what's is done on newer kernel [16:40] smb: the problem is that is fill partitions (10GB of logs here for this :() [16:41] for me the crashkernel functionatlity is not work (bug #389449 and possible #364414) - is that a known issue? [16:41] Malone bug 389449 in linux "linux-crashdump not working: "crashkernel reservation failed - memory is in use" " [Undecided,New] https://launchpad.net/bugs/389449 [16:41] lionel, Well in that case, if you can provide that patch from upstream with an explanation on why and send it to the kernel-team mailing list. [16:41] smb: Ok, will do, thanks ! [16:46] Wondering where my RAM has got to: have 3GB installed, but Linux (and Memtest) only recognises 2.5G! [16:47] /proc/iomem tells a curious tale, not really sure how to interpret it... [16:53] Keybuk, ok ... http://people.ubuntu.com/~apw/vfs_union10-karmic/ [16:55] apw: if my computer burns, you owe me a new one :p [16:55] which reminds me, wonder what happened to the one pgraner was supposed to have sent me [16:56] Araneidae, look at the e820 section of the dmesg output, tells you where your ram is [16:56] Keybuk: I haven't sent it yet. My wife packed it my mistake (we are moving houses) and I will get it next week. Once that happens I'll drop it in FedEx. [16:56] Keybuk: Sorry I forgot to mention it to you earlier [16:56] apw, it's a bit all over the place -- and it's not all there! [16:56] ahh, no worries [16:56] not in any rush, just wondered whether the postman had stolen it ;P [16:56] or customs [16:56] Araneidae, paste the e820 in a pastebin [16:56] http://pastebin.com/d24698a24 [16:57] That's the first page of dmesg [16:58] I was kind of hoping 00000000 - bfffffff, but I guess that was too simple minded of me [16:59] smb|ogasawara: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/330824/comments/191 should probably get on your radar (as a separate bug) [16:59] Malone bug 330824 in linux "Soft lockups (freezes) when deleting files from ext4 partitions on 2.6.28" [Medium,In progress] [16:59] sbeattie: thanks [17:00] apw, am I just at the mercy of my BIOS here? [17:00] Araneidae, uless some of the ram has been shifted over 4GB cause of the other stuff [17:00] you might try booting with a pae kernel (like the server one) and see if its there [17:01] sbeattie, Thanks. Yeah should be there [17:02] Well, there's something funning going on -- when I tried Memtest86+ it also showed 2.5G; if I tried reconfiguring memory it crashed! [17:03] Probe => black screen; All BIOS => many memory error messages then hang [17:03] define reconfiguring memory [17:04] On the Memtest86+ screen it offers a couple of menu options, one is "c" => "Configure Memory" [17:05] This then offers a menu with a number of options, including "All BIOS" and "Probe" as well as the subset it had selected. [17:05] Unfortunately I'm wholly unfamiliar with this tool! [17:05] and with the BIOS interface it's using [17:06] is linux-meta in git as well? or just in the archive? [17:06] mdz, in git as well, in the ubuntu/ubuntu-series-meta.git [17:06] mdz, it is in git [17:07] I'm guessing my motherboard is messing me about; don't know if the kernel can do any better with what it's being told. [17:07] apw: can you translate that to a URL for me? I tried git://kernel.ubuntu.com/ubuntu/ubuntu-series-meta.git but that fails [17:07] if the memory is mapped high, and that happens often when you get abover 3GB then the kernle won't see it unless its PAE enabled [17:08] apw: oh, series=karmic [17:08] mdz s/series/karmic/ etc [17:08] i should have said -- [17:08] Is the Ubuntu stock kernel not PAE enabled by default? [17:08] only the server kernel [17:08] * Araneidae is running Ubuntu 9.04 "out of the box" [17:09] no, we will have it avilable in karmic [17:09] Ahhh [17:09] Is there a simple package to switch to PAE? Not that keen on maintaining a separate kernel build on this machine... [17:09] sbeattie: just fyi I created - https://bugs.edge.launchpad.net/ubuntu/jaunty/+source/linux/+bug/389555. also posted a comment to 330824 [17:09] Malone bug 389555 in linux "[PATCH] ext4 filesystem corruption" [Undecided,Fix released] [17:09] you can install the server kernel [17:11] Package linux-server ? [17:11] linux-image-server [17:12] apw: so I have an ubuntu-karmic tree and an ubuntu-karmic-meta tree containing local changes which I would like to submit to the kernel team for review. what's the best way to do it? [17:12] commit and then format-patch? [17:12] Guess linux-server is a meta-package, it depends on linux-image-server and linux-restricted-modules-server [17:12] yeah thats the simplest [17:13] mdz: the simplest is 'git format-patch -1 ', then email it [17:13] Araneidae, that will do the trick then [17:14] Ach: "There can be issues with third party drivers and the server kernel (specifically, nvidia drivers)". I'll have to suck it and see, I guess... [17:16] Can anybody here run `cat /proc/uptime` on a 2.6.30 kernel and tell me what it says? [17:16] apw@dm$ cat /proc/uptime [17:16] 270380.70 254.39 [17:17] Is that second, very small number, a likely measure of idle time on that machine? [17:17] If not, /proc/uptime is *still* broken [17:17] My patch didn't make it in... [17:17] i'd say its a load average [17:17] No, it's meant to be cumulative idle time. [17:17] but i don't know [17:17] Always was up to .28 [17:18] then its broken [17:18] Well, damn. I'll have to try prodding everyone who was cc'd before on this. Dammit, I thought it went in. [17:18] Araneidae, what was the title of the bug [17:18] patch eve [17:18] even [17:19] hang on, I'll try and hunt out the thread [17:19] smb, dunno if you watching the release meeting, they are on hardy .03 [17:20] apw, thanks missed it [17:20] apw, Thought after kernel team it is done... :-P [17:20] heh me too [17:21] just luck its in another window so i saw it [17:22] This was my repost of the patch: http://marc.info/?l=linux-kernel&m=124265309620490&w=2 [17:24] There was quite a lot of traffic with titles: "linux-next: cputime tree build warning" and "[GIT PULL] cputime patch for 2.6.30-rc6" [17:25] apw: I've attached two nearly-trivial patches to bug 382115 which fix it (and bug 247517) [17:25] Malone bug 382115 in linux "Consider building unversioned linux-doc package" [Wishlist,Triaged] https://launchpad.net/bugs/382115 [17:25] Malone bug 247517 in linux "linux-doc-2.6.26 unnecessarily conflicts with other linux-doc packages" [Medium,Triaged] https://launchpad.net/bugs/247517 [17:25] The build warning was an annoying accident, which probably didn't help it. [17:26] mdz, thanks for those [17:26] apw: is it sufficient to attach them to the bug, or would it help if I sent them to the mailing list? [17:28] if you have time to send them to the mainling list then they'll get sorted sooner [17:30] apw: done [17:31] thanks [17:32] apw: one of the things mdz's patch reminded me of is that we were considering the removal of generated files from the git repo, e.g., debian/control, debian/control.stub, debian/d-i/kernel, etc [17:33] that would make some of these patches a bit simpler [17:33] rtg: yeah, I was unsure about whether I should commit the autogenerated stuff or not [17:33] yeah i tend to send the patches without the generated stuff and let them come in automatically for that reason [17:33] different packages handle this differently [17:33] mdz: I usually strip that stuff out when I'm integrating the patch anyways [17:33] if you want me to strip it out, that's easy enough [17:33] mdz: thats ok, we can deal [17:33] ok, thanks [17:34] apw: so, lets try it for awhile. I'll go ahead and delete those generated files. OK ? [17:35] that would also help a common packaging mistake where debian/control doesn't get regenerated correctly [17:35] yeah i am down with that. [17:38] as long as the packaging will clean things before making the source package we will be ok i think [17:38] there was something about control needing to be there before building the src pckg which would need testing [17:38] apw: it does 'cause everything is dependent on debian/control [17:39] yep, so as long as dpkg-thingy doesn't need a control to exist before it hits clean we are good, and clean is the first thing in src generation [17:40] apw: it always does a clean first [17:41] I'll practice it to make sure [17:41] dpkg-buildpackage: source changed by Andy Whitcroft [17:41] dpkg-checkbuilddeps: failure: cannot read debian/control: No such file or directory [17:41] dpkg-buildpackage: warning: Build dependencies/conflicts unsatisfied; aborting. [17:41] dpkg-buildpackage: warning: (Use -d flag to override.) [17:41] dpkg-buildpackage: warning: This is currently a non-fatal warning with -S, but [17:41] dpkg-buildpackage: warning: will probably become fatal in the future. [17:41] fakeroot debian/rules clean [17:41] so control may need to be there still, [17:41] I'll work on it. === thegodfather is now known as fabbione [17:57] apw: so, I think I like the side effects of removing the auto-generated files. It pretty much forces you to generate them before you can package or build. [17:58] i can live with that, the instructions need to be 'fdr clean; debuild -s .... -sa' sort of thing i guess [17:58] If we go forward with this patch, then we'd better mention it on the maintenance wiki [17:58] apw: err, that one panic'd on boot [17:59] hrm i did what? [17:59] it even? [17:59] apw: panic, on boot [17:59] somewhere in populate_rootfs [17:59] before you did anything with unions ? [17:59] right, before it even got as far as the initramfs ;) [18:00] well heck what did i do wrong there [18:00] no idea [18:00] even "ls" produces wacky output [18:00] d??????? ? ?? ?? grub [18:00] there was a lot of porting forward for that patch, so i must have bust it [18:00] ls: cannot access grub: File exists [18:01] it looks like you screwed the VFS [18:01] gurgle [18:02] clearly it's that special time of the week that we call "Friday" [18:02] yeah ... clearly [18:04] so I can't tell you whether the rename patch works [18:04] because I can't access my filesystem [18:09] Keybuk, any stack info for me from it? [18:11] apw: only that it's somewhere in populate_rootfs() [18:11] annoyingly someone thought the top of the stacktrace should be off the top of the screen without the ability to scroll up ;) [18:12] yeah i know ... bloody thing [18:12] its inevitably rename === thegodfather is now known as fabbione [20:35] if something has a usb-ttl converter(pl2303) does that mean i can use serial i/o instead of having to write a driver? [20:59] anyone happen to have a moment to help me understand overcommitting memory? I heard that it's possible to allocate gigabytes of memory, more than what seems to fit in the void* address space. is that right? [21:03] cr3: No [21:04] Overcommit relates to whether allocated pages are backed by real memory or swap, not whether you can have more than 3GB of address space on a 32-bit CPU [21:04] mjg59: ok, I understand the first part but not the latter part [21:05] mjg59: to have more than 3GB, is the address still allocated on the heap somehow? [21:06] You can't allocate more than 3GB on a 32-bit CPU [21:07] mjg59: my bad, I misread "not whether" as "or whether". heh, maybe it was wishful thinking :) [21:08] PAE means you can have more than 4GB of RAM for the platform [21:08] But each process can only have 3GB in its address space [21:08] What you /can/ do is use shared memory segments (backed by RAM) as if they're files [21:08] Oracle does that [21:09] mjg59: gotcha, all makes sense again. I was given a false impression of what overcommitting really is. thanks! [22:08] johanbr, 2.6.29's suspend was working fine for me today under karmic's userspace [22:09] alright :) [22:09] so it's a problem with 2.6.30... [22:16] surprise surprise :p [22:18] you'd filed a bug, right? [22:18] did you add that information to the bug?