=== smb` is now known as smb
=== Guest14562 is now known as ogra
=== _LibertyZero is now known as LibertyZero
=== yofel_ is now known as yofel
=== sforshee is now known as saf-afk
=== saf-afk is now known as sforshee
[14:55] <ntemis> hi
[14:57] <tgardner> smb, did you run out of interest before reviewing Dapper CVE-2010-3880 ?
[14:57] <ubot2> tgardner: net/ipv4/inet_diag.c in the Linux kernel before 2.6.37-rc2 does not properly audit INET_DIAG bytecode, which allows local users to cause a denial of service (kernel infinite loop) via crafted INET_DIAG_REQ_BYTECODE instructions in a netlink message that contains multiple attribute elements, as demonstrated by INET_DIAG_BC_JMP instructions. (http://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2010-3880)
[14:58] <ntemis> my ethernet adapte supports jumbo frames but the driver dodnt let me go beyond 7k
[14:58] <smb> No, I stumbled over the openvz fixup (which is attributed wrongly it seems) and then forgot about it.
[14:58] <ntemis> it can do 9k
=== shadeslayer_ is now known as shadeslayer
[14:58] <ntemis> rlt 8111E
[14:58] <smb> tgardner, No, I stumbled over the openvz fixup (which is attributed wrongly it seems) and then forgot about it.
[14:58] <ntemis> in latest 2.6.35-25-server on x86_64
[14:59] <ntemis> the kernel loaded 8169 driver
[14:59] <smb> tgardner, Let me go back and do the Dapper one, but I think you should untangle the fix for openvz. (I'd probably do a separate patch anyway)
[15:00] <ntemis> it should be 8168
[15:00] <smb> tgardner, btw, when apw looked at it he noted that it seems to not only move it but the file modified had changed too. 
[15:00] <ntemis> i searched realtek site and they have a proper driver there
[15:01] <ntemis> r8168-8.021.00.tar
[15:01] <tgardner> smb, hmm, I can't see anything wrong with it. maybe I'm just dyslexic this morning.
[15:01] <ntemis> can you please include it in a natty kernel?
[15:02] <apw> ntemis, we are unlikely to pull in one of their tarball drivers if we have good support from an in kernel driver
[15:03] <apw> already ... those drivers are often very nasty to debug and harder to maintain
[15:03] <smb> tgardner, Ah, no. I guess he might have been misled by kernel/net.. and net/...
[15:03] <ntemis> http://218.210.127.131/products/productsView.aspx?Langid=1&PNid=13&PFid=5&Level=5&Conn=4&ProdID=239
[15:03] <ntemis> this is the ifo for the card
[15:03] <ntemis> LINUX driver for kernel 2.6.x and 2.4.x (Support x86 and x64)
[15:03] <ntemis> 	8.021.00	2011/1/27	56k
=== herton is now known as herton_lunch
[15:04] <smb> tgardner, The main issue is that the fixup is not required by your change/cve but was required by his. So the fixup would now be attributed to the wrong cve
[15:04] <ntemis> apw: why is using the 8169 driver? the manufacturer produce a 8168 driver for it
[15:05] <apw> because the upstream driver claims that device, thus are claiming to support it
[15:05] <apw> and indeed your own report here is that it does at least for frames up to a cirtain size right?
[15:06] <ntemis> yes
[15:06] <tgardner> smb, you're referring to the openvz patch?
[15:06] <ntemis> Supports jumbo frame to 9K bytes
[15:06] <ntemis> see features
[15:07] <ntemis> current driver only 7k
[15:07] <smb> tgardner, Yes, that additional line (setting that variable to zero) was part of a previous cve. (apparently someone did not do a complete test-build ;-)). You just stumbled over it when trying your build.
[15:07] <ntemis> and also i loose the connection every now and then on the server
[15:07] <ntemis> random disconnection
[15:07] <ntemis> no ssh at all
[15:07] <ntemis> i have to restart the server
[15:08] <apw> ntemis, the former i don't expect is very important, the second is an issue indeed
[15:08] <apw> realtek have not tried to make our lives easy by just dumpiing out tarballs for every driver
[15:08] <apw> they are not easy to intregrate, maintain, or debug ... we are therefore very resistant to pull them in
[15:09] <ntemis> :)
[15:09] <ntemis> so what now?
[15:09] <apw> i would recommend trying the latest natty kernel to see if the random dropouts are there too
[15:09] <ntemis> i have to binutils and linux tree and install the realtek ones?
[15:10] <ntemis> can i use the generic 2.3.37 stable kernel for natty on my server?
[15:10] <apw> ntemis, have you any reason to think they are working any better
[15:11] <ntemis> they are the manuf drivers :) ;)
[15:11] <apw> no comment
[15:11] <ntemis> dont know untill tested
[15:11] <ntemis> lol
[15:11] <ntemis> o.k
[15:11] <ntemis> can i use the 2.6.37 generic kernel in this server?
[15:12] <ntemis> the one in ppa
[15:12] <apw> there is no support for the mainline kernels, so they are probabally not recommended for long term use; they are intended for debugging
[15:12] <ntemis> http://kernel.ubuntu.com/~kernel-ppa/mainline/v2.6.37-natty/
[15:13] <ntemis> this will fuction ok on a 10.10 release?
[15:13] <sconklin> smb: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/579276
[15:13] <ubot2> Launchpad bug 579276 in linux "Lost network in KVM VM / virtio_net page allocation failure" [Medium,Triaged]
[15:13] <apw> ntemis, it may, it has no ubuntu sauce, so it may have trouble
[15:14] <ntemis> i just want to be able to use my server , now i have to force shutdown from power every now and then
[15:15] <ntemis> olny way to comunicate is from that eth0
[15:17] <ntemis> one final question apw
[15:17] <ntemis> apw: is the v2.6.37-natty kernel includes drivers for the nic's or i have to load them manually
[15:18] <apw> ntemis, as i say i would recommended testing with a latest natty kernel to see if that is fixed, helps us find fixed
[15:18] <apw> ntemis, i would say it has the same drivers as previous versions so should support the same cards
[15:18] <ntemis> you mean go for 2.6.38 rc?
[15:19] <apw> ntemis, i would myself, i would pick the natty kerenl as it has the ubuntu sauce and is more likely to work with your userspace
[15:20] <ntemis> am seeing this: v2.6.37-rc2-maverick/
[15:20] <ntemis> what do you think?
[15:20] <ntemis> or this:	v2.6.38-rc3-natty
[15:20] <apw> i personally wouldn't run any of the kernel-ppa ones in a production env for very long, i would go for a full 2.6.37 without -rc if wanting 27, but again there are 27 based natty kerenls, i would use those
[15:21] <ntemis> thanks
=== Quintasan_ is now known as Quintasan
[15:39] <tgardner> smb, so back to the Hardy review for Hardy CVE-2010-3880. Are you thinking the extra cruft should be folded back into the patch for CVE-2010-3876 (which is where it appears to have come from) ?
[15:39] <ubot2> tgardner: net/ipv4/inet_diag.c in the Linux kernel before 2.6.37-rc2 does not properly audit INET_DIAG bytecode, which allows local users to cause a denial of service (kernel infinite loop) via crafted INET_DIAG_REQ_BYTECODE instructions in a netlink message that contains multiple attribute elements, as demonstrated by INET_DIAG_BC_JMP instructions. (http://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2010-3880)
[15:39] <ubot2> tgardner: net/packet/af_packet.c in the Linux kernel before 2.6.37-rc2 does not properly initialize certain structure members, which allows local users to obtain potentially sensitive information from kernel stack memory by leveraging the CAP_NET_RAW capability to read copies of the applicable structures. (http://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2010-3876)
[15:40] <smb> tgardner, I probably would suggest the lazy mode. Which is split it off as separate patch and add references to 3876.
[15:41] <tgardner> smb, OK, I'll get to it in a bit.
[15:47] <apw> sforshee, can hear but not reply at the moment
[15:48] <apw> running strace on X to ensure you got the command right for someone, is a _really_ stupid idea
=== tgardner is now known as tgardner-afk
[15:55] <ntemis> apw: you here?
[15:56] <apw> ntemis, s'up
[15:56] <ntemis> i installed 2.6.37 natty
[15:56] <ntemis> 64bit
[15:56] <ntemis> all ok
[15:56] <apw> ok so upstream has fixed something then
[15:57] <ntemis> dont know yet
[15:57] <ntemis> i will leave it on for a few days
[15:57] <apw> ok
[15:57] <ntemis> if it looses connection is a negative
[15:57] <ntemis> am gonna try jumbo frames on this one to see any dif
=== herton_lunch is now known as herton
[16:00] <ntemis> nope
[16:00] <ntemis> same output
[16:00] <ntemis> SIOCSIFMTU: Invalid argument
[16:01] <ntemis> am stuck at 1500 mtu
[16:01] <ntemis> for now :)
[16:01] <ntemis> i hope 11.04 is the sweet spot
[16:04] <ntemis> all survive the update nfs server samba etc
[16:07] <ntemis> hmmm....
[16:07] <ntemis> server is faster on this kernel!
[16:07] <ntemis> is it a placebo?
[16:08] <apw> ntemis, depends what you are using it for, there are changes in there to make it more responsive
[16:08] <ntemis> it is more responsive!
[16:09] <ntemis> CPU usage 	27% user, 4% kernel, 52% IO, 18% idle
[16:09] <ntemis> and responsive as hell
[16:09] <ntemis> a job welldone!
[16:10] <ntemis> Real memory 	3.87 GB total, 498.02 MB used
[16:10] <ntemis> my god!
[16:10] <ntemis> thanks guys
[16:10] <ntemis> 2.6.37 rocks
[17:04] <ogra> tgardner-afk, the plan for OMAP4 was that we have an installable .deb of .38 in the archive so people can install and test it, we wont build images with that 
[17:04] <ogra> tgardner-afk, for *images* we need display, which is why i insisted to have it ready asap, a .deb and branch would be fine earlier
[17:06] <bjf> ogra, so the OMAP4 flavour (non-display) that is built from main, will that be an officially supported flavour ? 
[17:06] <ogra> bjf, that will become the natty kernel, once the missing features are in
[17:07] <ogra> and yes, omap4 is an officially supported platform 
[17:07] <ogra> (since maverick)
[17:08] <bjf> ogra, i'm trying to ask (poorly i know) will there be two supported Natty OMAP4 kernels, one built from the master branch (2.6.38) and one built from the ti-omap4 topic branch ?
[17:08] <ogra> no
[17:08] <ogra> the actual plan is that we get a working .38 kernel before kernel freeze 
[17:09] <bjf> ogra, and if that doesn't happen ?
[17:09] <ogra> the prob is that .38 is not yet feature complete so we need .35 for images until thats done
[17:09] <ogra> bjf, if that doesnt happen people will be killed i guess :)
[17:09] <ogra> or something similar will happen 
[17:10] <bjf> ogra, sounds like a plan ! :-)
=== skaet is now known as skaet_afk
=== sconklin is now known as sconklin-lunch
[17:51] <JFo> <-need more coffee, brb
=== tgardner-afk is now known as tgardner
[18:08] <rigved> hi everyone...i created a initramfs for 2.6.32-28, which i had built eariler (following Linux Kernel in a Nutshell guide). when i try to boot into this in qemu, i get the error - unable to find udev. can anyone tell me what i'm doing wrong here
[18:09] <apw> rigved, how did you make it?
[18:09] <rigved> apw: you mean the initramfs?
[18:09] <apw> rigved, yes
[18:10] <rigved> apw: using mkintramfs
[18:10] <rigved> apw: wait, i'll tell you the exact command
[18:13] <rigved> apw: sudo mkinitramfs -f -o /boot/initrd.img-2.6.32.28 -v 2.6.32.28
[18:14] <jjohansen> rigved: try sudo update-initramfs -c -k 2.6.32.28
[18:14] <rigved> jjohansen: ok. wait, i'll give it a try
[18:15] <jjohansen> rigved: you will also want to run update-grub after that so that grub knows about the new kernel
[18:16] <psurbhi> smb, ping
[18:16] <rigved> jjohansen: ya i had done that...i booted into the new kernel also...it showed me the Ubuntu 10.04 booting screen...with the dots...then it gave me this error and dropped me to a initramfs shell
[18:17] <tgardner> ogra, whats your call on the minimal ti-omap4 kernel for Natty? Is there any reason folks couldn't use the Linaro kernel until TI can get graphics in shape?
[18:18] <smb> psurbhi, hey, though I currently got only one eye and hand free
[18:19] <psurbhi> smb, what happened?
[18:19] <smb> psurbhi, I am on the phone :)
[18:19] <psurbhi> heh
[18:19] <psurbhi> i will wait for u
[18:21] <rigved> jjohansen: so when i run this command it generated the image, but it couldn't find any files under /lib/modules/2.6.32.28/ in the guide it said to put the built kernel in a test folder.
[18:22] <jjohansen> rigved: how did you build your kernel and how did you install it
=== sconklin-lunch is now known as sconklin
[18:24] <rigved> jjohansen: make O=~/linux/linux-2.6.32.28/ menuconfig.
[18:24] <ogra> tgardner, see above :)
[18:24] <rigved> jjohansen: then make modules_install.
[18:24] <ogra> <ogra> tgardner-afk, the plan for OMAP4 was that we have an installable .deb of .38 in the archive so people can install and test it, we wont build images with that 
[18:24] <ogra> <ogra> tgardner-afk, for *images* we need display, which is why i insisted to have it ready asap, a .deb and branch would be fine earlier
[18:24] <rigved> jjohansen: then make O=~/linux/linux-2.6.32.28/ ARCH=i386.
[18:25] <rigved> jjohansen: sorry that's make O=~/linux/linux-2.6.32.28/ ARCH=i386 install
[18:25] <ogra> tgardner, we should have a kernel in the archive asap so we can see whats broken in TIs tree and track the progress, while it is currently starting off from mainline i expect many patches that are pre-promoted in that tree before they hit upstream 
[18:26] <tgardner> ogra, so, you're willing to suffer the regression of no GUI for awhile? I'm not really interested in managing 2 ti-omap4 packages.
[18:26] <ogra> well, no
[18:26] <ogra> we need .35 for finishing off the GUI work on unity-2d
[18:26] <ogra> i dont expect any changes to .35 
[18:26] <ogra> none at all 
[18:27] <ogra> just leave it rot and lets care for .38
[18:27] <rigved> jjohansen: sorry again. i had typed sudo for the last two commands
[18:27] <tgardner> ogra, so, you want me to carry yet another kernel _and_ meta package for Natty? Or can folks just use the Maverick kernel after I pull 2.6.38 into the Natty ti-omap4 branch?
[18:28] <ogra> hmm, no, it needs a different name indeed
[18:28] <ogra> we dont need a meta
[18:28] <ogra> just a binary
[18:28] <jjohansen> rigved: so the make install, make modules install should have put a kernel image in /boot/ and installed the modules in /lib/modules/
=== sforshee is now known as sforshee-lunch
[18:29] <jjohansen> rigved: so what do you have under /lib/modules/
[18:29] <rsalveti> problem is that we want a new kernel but still need to use the older one to generate images
[18:29] <ogra> tgardner, we still need meta and .35 for images 
[18:29] <ogra> but threre wont be changes to it
[18:30] <tgardner> ogra, could we get by with building the development kernel in a PPA ?
[18:30] <apw> why can't the new kernel hang out in a PPA until its ready
[18:30] <rigved> jjohansen: there are quite some files in /lib/modules/2.6.32.28/
[18:30] <ogra> worst case that would suffice, yep
[18:30] <rigved> jjohansen: i'm going to try to boot into qemu using the new initrd-img
[18:30] <jjohansen> rigved: err I thought you said that lib /lib/modules/2.6.32.28/ was missing
[18:31] <rigved> jjohansen: that's what update-initramfs said. but there are files under it
[18:32] <apw> its not even clear we need it in a PPA, we do most testing with hand rolled .debs anyhow
[18:33] <jjohansen> rigved: oh, okay I didn't realize it was only update-initramfs that was having the problem with that
[18:34] <apw> ogra, its not even clear we need it in a PPA, we do most testing with hand rolled .debs anyhow
[18:34] <jjohansen> rigved: maybe try update-initramfs -u -k all
[18:35] <JFo> <- grabbing a bite to eat, back son
[18:35] <JFo> soon*
[18:36] <rigved> jjohansen: so in virtualbox i get a different error: ALERT! /dev/disk/by-uuid/blah blah blah does not exist. Dropping to shell. then i have a initramfs shell (busybox)
[18:36] <rigved> jjohansen: now i'll try qemu
[18:36] <rigved> jjohansen: then i'll try the other command which you gave
[18:36] <ogra> apw, we want a .deb available asap, i dont care at all where we pull it from (it would be cooler to have it in the archive but indeed i dont want to add extra work for you guys)
[18:37] <apw> ogra, so do we have a tree yet?
[18:37] <tgardner> apw, ogra: we could do this in the pre-proposed PPA perhaps.
[18:37] <apw> tgardner, if its arm does it need to be a de-virt one yet ?
[18:37] <apw> still ?
[18:37] <ogra> yes
[18:37] <tgardner> apw, uh, you might be right
[18:38] <ogra> until we get the pandas that are ordered since Nov.
[18:38] <tgardner> apw, we only have one de-virt PPA, right?
[18:38] <apw> yeah, and i'd say for this use case a cross-build is as good
[18:38] <apw> so the main blocker is having a tree to make it out of
[18:38] <tgardner> apw, well, we could carry a dev branch for awhile.
[18:39] <rigved> jjohansen: same error in qemu.
[18:39] <rigved> jjohansen: now i'll try the other command
[18:39] <apw> tgardner, yeah i recon keep it in a separate branch but internally thinking its ti-omap4 until its ready, in the main repo
[18:39] <ogra> apw, cross isnt an option 
[18:39] <apw> then we can just shove it over the top
[18:39] <apw> ogra, why not?
[18:39] <tgardner> apw, my thoughts exactly
[18:39] <ogra> we need the same thing we will get from the builders else testing is moot
[18:40] <bryceh> [ 4635.189079] exe (9084): /proc/9084/oom_adj is deprecated, please use /proc/9084/oom_score_adj instead.
[18:40] <apw> ogra, define the same ?
[18:40] <bryceh> ^^ does such an error message indicate the oom handler got called?
[18:40] <apw> its built witht he same compilers, just the cross versions
[18:40] <ogra> note that we also want to be sure the toolchain dtrt etc
[18:40] <apw> you normally test cross built ones
[18:40] <psurbhi> smb, i am pushng off for dinner ... its very late in here.. wanted to discuss bug 709392
[18:40] <ubot2> Launchpad bug 709392 in nfs-utils "NFS client does not submit "nfs_file_sync" write requests when the file open call includes O_SYNC." [High,Triaged] https://launchpad.net/bugs/709392
[18:40] <psurbhi> maybe  tomorrow
[18:40] <ogra> hmpf
[18:40] <psurbhi> see you around!
[18:41] <tgardner> apw, ogra: we can use the c-k-t PPA for Natty. It won't collide with anything going on right now. This should all resolve itslef by mid April
[18:41] <apw> tgardner, yep fine with me
[18:41] <ogra> tgardner, that sounds good
[18:41] <tgardner> ok, resolved.
[18:41] <apw> well other than the lack of a tree to package right ?
[18:41] <ogra> i'm really more confident in natively built packages, sorry for being so stuck in that 
[18:42] <tgardner> ogra, I'll respond to Sebastien and Nicholas
[18:42] <ogra> thanks, i seem to have a mail prob i just tried to send an answer
[18:42] <apw> ogra, you are right at the end of the process you can't say its deffo good unless its native
[18:43] <apw> ogra, for the majority of the builds before that i recon you are fine, and you save 6 hours a cycle
[18:43] <ogra> i would just like to exclude all possible issues so we can see the real probs with the code
[18:43] <apw> then you pay 7 hours a spin, i'd be suicidal at that
[18:43] <ogra> and i'm sure there will be enough
[18:43] <apw> but its your life
[18:44] <ogra> we dont user that kernel on any images so 7h are fine imho
[18:44] <ogra> it doesnt block anything 
=== sforshee-lunch is now known as sforshee
[18:51] <herton> bryceh: nope, just that the process is trying to set oom_adj, it's a deprecated interface now
[18:51] <herton> for oom parameter, process should use now oom_score_adj
[18:52] <herton> Documentation/feature-removal-schedule.txt explains it better in linux source (/proc/<pid>/oom_adj section)
=== skaet_afk is now known as skaet
[18:59] <ogra> whoever is ML admin for the kernel list, please reject the duplicated mail from ogra@grawert.net that will end up in your queue
[19:00] <ogra> (it was also sent from ogra@ubuntu.com and got through the queue already)
[19:01] <tgardner> ogra, thats me
[19:01] <ogra> just throw it away please (if yuo do your admin work anyway, no need for extra action)
[19:03] <tgardner> ogra, done
[19:03] <ogra> bah, so you did extra action :P
[19:03] <ogra> thansk :)
[19:05] <bryceh> herton, thanks
[19:12]  * tgardner --> lunch
[19:23] <niemeyer> Greetings
[19:23] <niemeyer> Is there a way to *reserve* a contiguous area of virtual address space within a process, without having it interpreted as an allocation (e.g. without hitting ulimit -v problems)?
=== poelzi_ is now known as poelzi
[19:30] <rigved> jjohansen: hi. i did some googling and found out that the problem is caused because the kernel is unable to mount a SATA HDD. So, in virtualbox, i changed the SATA HDD to a IDE HDD, and it worked. any idea what is happening here?
[19:33] <zul> has anyone seen this on 2.6.38 with running kpartx? http://pastebin.ubuntu.com/564033/
[19:33] <rigved> jjohansen: this is the bug report on launchpad - bug 576071
[19:33] <ubot2> Launchpad bug 576071 in ubuntu "After upgrade to Lucid /dev/disk/by-uuid missing when booting with initramfs" [Undecided,Incomplete] https://launchpad.net/bugs/576071
[19:35] <jjohansen> rigved: okay hrmm, I don't know anything atm but will take a look
[19:37] <herton> zul: looks you hit the same bug as LP#700165
[19:37] <herton> (https://bugs.launchpad.net/ubuntu/+source/linux/+bug/700165)
[19:37] <ubot2> Launchpad bug 700165 in linux "qemu-nbd kthread becomes defunct on disconnect" [High,Fix released]
[19:38] <rigved> jjohansen: i don't know what's happening exactly, i don't even understand teh bug report...but atleast now i can move forward in the kernel guide. thanks for your help. it's very late here. i'm off to sleep. good day
[19:56] <kees> smb: oh! http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commitdiff;h=f12d3d04e8f6223276abb068c5d72852174b8c31 should fix the Xen issues with RO/NX. can you re-enable that for the Xen builds?
[20:03] <niemeyer> kees: Any idea about the question above, or at least a direction I could look at?
[20:03] <niemeyer> herton: Bem vindo, btw :)
[20:03]  * kees reads backscroll
[20:03] <niemeyer> Is there a way to *reserve* a contiguous area of virtual address space within a process, without having it interpreted as an allocation (e.g. without hitting ulimit -v problems)?
[20:03] <niemeyer> kees: ^
[20:04] <kees> I'm not sure what the difference between "reserve" and "allocate" really would be.
[20:04] <kees> are you just trying to stop something from using a static range of memory?
[20:04] <herton> niemeyer: obrigado :)
[20:04] <niemeyer> kees: That's it
[20:05] <niemeyer> kees: So that in the future the page can be mapped continuously with the previous page, if necessary
[20:06] <niemeyer> kees: The trick being used now is to mmap() a big range with prot=0, and then changing the prot when it's actually needed
[20:06] <niemeyer> kees: It all works well, except that the dumb backend of ulimit -v restricts address space for no reason
[20:06] <niemeyer> (why the heck people use that)
[20:07] <kees> out of curiosity, why do you care about contiguous future memory allocation?
[20:07] <bguthro> kees / smb: I just noticed the kernel.org changeset above. Are there normally xen builds against the ubuntu kernel? If so - where could I find them?
[20:08] <kees> bguthro: the "linux-ec2" (Xen guest) is part of the normal natty kernel builds of the "linux" source package.
[20:09] <kees> niemeyer: anyway, you're only ever fighting against glibc, so you probably just want to use your own heap allocator.
[20:09] <bguthro> ah. Its a domU kernel. that makes more sense...I was hoping for the dom0 white whale
[20:09] <niemeyer> kees: It is "my own" heap allocator, actually :-) (Go's, to be more precise)
[20:10] <niemeyer> kees: The algorithm gets simpler and faster with it
[20:10] <kees> niemeyer: if you control the allocator, why do you need the reserve memory regions? you just need to not use them. :P
[20:11] <niemeyer> kees: Because other libraries have their own allocators, which will get in the way and explode the application if the area is not reserved
[20:12] <kees> niemeyer: right, I mean _replace_ the glibc allocator.
[20:13] <kees> niemeyer: so that your allocator is being used for all calls.
[20:13] <niemeyer> kees: I don't get what you mean.  Any library doing mmap() can take a page in a non-reserved space.
[20:13] <niemeyer> kees: It doesn't matter what glibc is doing
[20:13] <kees> niemeyer: ah, okay, sure. most just use malloc, though.
[20:14] <niemeyer> kees: mmap() is used for other reasons besides building the arena for malloc
[20:14] <kees> niemeyer: but anyway, I get what you're saying. I'm not aware of a way to mark mmap regions offlimits without actually allocating them.
[20:14] <niemeyer> kees: Ok, cool
[20:14] <niemeyer> kees: Do you know if there's a viable alternative to ulimit -v which doesn't hit the issue?
[20:15] <niemeyer> kees: It looks like most people doing ulimit -v are really trying to restrict swapping
[20:15] <niemeyer> kees: But the limit of virtual space is much more encompassing than this
[20:15] <kees> how much are you reserving? most sane users of ulimit -v shouldn't be setting it too low
[20:16] <kees> e.g. I use ulimit -v for local builds to make sure things like gcc and glibc testsuites don't wreck my system.
[20:16] <niemeyer> kees: A lot.. or nothing, depending on the POV :-)
[20:16] <niemeyer> kees: 16GB, for instance
[20:16] <niemeyer> It's quite a bit, but nothing taking a 64bit space in account
[20:16] <kees> O_o
[20:17] <kees> are you really growing continugous areas so frequently that you can't just use realloc?
[20:18] <bryceh> is   video=vesafb:off   the right way to disable vesafb?
[20:18] <niemeyer> kees: That doesn't work for a memory arena.. we're talking about the thing that backs malloc()
[20:18] <niemeyer> kees: realloc would mean all pointers inside the region shifting
[20:18] <niemeyer> kees: This is possible in the future, but it's a much larger problem than the one we're talking about
[20:19] <kees> niemeyer: but if it's for an arena why do you need it contiguous? can't you just have multiple arenas?
[20:20] <niemeyer> kees: Yes, that was the old implementation.. a contiguous region makes things a lot simpler and faster.
[20:21] <niemeyer> kees: E.g. 16GB is mapped with 22 bits (* page size)
[20:23]  * kees wonders if the kernel anonymous mmap routine uses greedy or "after last" allocations.
[20:23] <kees> i.e. alloc something, then alloc something 16GB later.
[20:23] <kees> will the next mmap end up in the middle, or after?
[20:26] <niemeyer> kees: Good question
[20:32] <kees> niemeyer: it works...
[20:32] <kees> wait, no
[20:32] <kees> yeah, seems to work.
[20:32] <kees> $ ./mmap 
[20:32] <kees> 0x7f3b94f0c000 -> 0x7f3f94f0c000: 16384M
[20:32] <kees> 0x7f3b94f0a000: ok
[20:32] <kees> 7f3b94f11000-7f3b94f12000 rw-p 00000000 00:00 0 
[20:32] <kees> 7f3f94f0c000-7f3f94f0d000 ---p 00000000 00:00 0 
[20:33]  * kees attempts a larger secondary allocation....
[20:34] <niemeyer> kees: Hmm.. I don't think I get it
[20:34] <niemeyer> kees: This shows you're actually allocating 16GB, right?
[20:34] <kees> nope.
[20:34] <kees>     start = mmap(NULL, 4096, PROT_READ | PROT_WRITE, MAP_PRIVATE | MAP_ANONYMOUS, -1, 0);
[20:34] <kees>     end = mmap(start + 16L*1024L*1024L*1024L, 4096, PROT_NONE, MAP_PRIVATE | MAP_ANONYMOUS, -1, 0);
[20:34] <kees>     more = mmap(NULL, 4096*1024, PROT_READ | PROT_WRITE, MAP_PRIVATE | MAP_ANONYMOUS, -1, 0);
[20:34] <kees> and more ends up below start, not above it
[20:34] <niemeyer> kees: Ahh, ok
[20:35] <niemeyer> kees: You mean below end?
[20:35] <kees> so... maybe you can use that, but I have no idea how stable it might be.
[20:35] <kees> no, 0x7f3b94f0a000 (more) is less than 0x7f3b94f0c000 (start)
[20:35] <kees> looks like it's growing the heap down.
[20:36] <kees> but this is probably all academic; there's nothing that says it won't change behavior at any moment. :P
[20:37] <niemeyer> kees: Well, hmm.. I don't think I understand why the experiment proves something useful
[20:37] <niemeyer> kees: You're allocating two pages consecutively, and they are 8k apart from each other?
[20:37] <niemeyer> >>> 0x7f3b94f0a000 - 0x7f3b94f0c000
[20:37] <niemeyer> -8192
[20:38] <kees> niemeyer: because the kernel didn't allocate anything between 0x7f3b94f0c000 and 0x7f3f94f0c000, leaving a 16GB gap
[20:39] <kees> and, ta-da, I broke it.
[20:40] <kees> niemeyer: it looks like it's just finding "best match" for anonymous mappings
[20:40] <kees> I can get it to alloc "start" below libc, and "end" after libc...
[20:40] <kees> so... nevermind! :P
[20:40] <niemeyer> I see
[20:40] <niemeyer> kees: Was a valid experiment, though
[20:40]  * kees nods
[20:41] <bryceh> JFo, bug #711568 turned out to be a kernel regression.  Upstream has committed a kernel patch, mind adding this to the kernel team's list to look at?
[20:41] <ubot2> Launchpad bug 711568 in xserver-xorg-video-intel "Dithering regression on Intel graphics with .38 Natty kernel" [Undecided,New] https://launchpad.net/bugs/711568
[20:41] <JFo> I don't mind at all bryceh :)
[20:43] <JFo> looks like apw beat me to the punch, but I went ahead and added the patch tag
[20:43] <JFo> thanks for the heads up bryceh :)
[20:44] <bryceh> sure
[20:45] <bryceh> hopefully I'll have more to come.  There's been a spate of gpu lockups lately, something must be futzed in the .38 drm still.
[20:53] <niemeyer> kees: How did you figure it wasn't deterministic?
[20:57] <kees> niemeyer: I switched start/end to use PROT_NONE, and looked at the maps file.
[20:57] <kees> niemeyer: there was all kinds of stuff between the two :(
[20:57] <niemeyer> Hmmm..
[20:57] <niemeyer> Ok
[20:58] <kees> niemeyer: http://paste.ubuntu.com/564073/
[20:58] <niemeyer> kees: Thanks
[20:59] <kees> np
[21:07] <niemeyer> kees: Hmmm
[21:07] <niemeyer> kees: That's not a definitive answer I think
[21:07] <kees> niemeyer: no, it shows it's pretty unstable.
[21:08] <niemeyer> kees: What I mean is that the compiler can choose to allocate certain libraries in fixed addresses
[21:08] <niemeyer> kees: This would affect such placement
[21:08] <kees> niemeyer: right, but that's against packaging policy (libraries must be -fPIC)
[21:09] <niemeyer> kees: Not really.. -fPIC just says the code should be relocatable
[21:09] <niemeyer> kees: It doesn't limit the freedom to put such code in a given address
[21:09] <niemeyer> kees: linker vs. compiler in this case
[21:11] <kees> niemeyer: okay, yes, that's true.
[21:12] <kees> but any pinning of objection locations would defeat the security purpose of ASLR
[21:42] <bjf> JFo, can you accept nominations on bug #714732 and bug #714846 ? thanks.
[21:42] <ubot2> Launchpad bug 714732 in linux "Maverick update to 2.6.35.11 stable release" [Undecided,New] https://launchpad.net/bugs/714732
[21:42] <ubot2> Launchpad bug 714846 in linux "CVE-2010-4242" [Undecided,New] https://launchpad.net/bugs/714846
[21:43] <JFo> bjf, I can indeed
[21:44] <JFo> bjf, done
[21:44] <JFo> :)
[21:44] <bjf> thanks
[21:45] <JFo> my pleasure
=== bjf is now known as bjf[afk]
=== sconklin is now known as sconklin-gone